V-Sido x Songle: A Real-time Control System for Robot Dancing

Powered by Asratec’s “V-Sido OS” and AIST’s “Songle”

Asratec Corp.
National Institute of Advanced Industrial Science and Technology (AIST)

  • Our system can make humanoid robots dance to music using dynamically-assigned dance motions
  • Over 800,000 songs analyzed by AIST’s Songle service are ready to be used for robot dancing
  • While a robot is dancing, a user can select a motion pattern or command the robot to walk
  • Asratec’s V-Sido OS stabilizes robot dancing even if the floor is tilted or a walking motion is started

Asratec Corp. (President: Masato Sakatani, hereafter “Asratec”) and the Information Technology Research Institute (Director: Tomohiro Kudoh) of the National Institute of Advanced Industrial Science and Technology (President: Ryoji Chubachi, hereafter “AIST”) announced the development of “V-Sido x Songle” (pronunciation: /bʊʃɪdoʊ sɔŋəl/), a system that can make humanoid robots of various shapes and sizes dance to music using motions dynamically assigned along the music structure. The system was jointly developed by the teams of Asratec Chief Robot Creator Wataru Yoshizaki and AIST Prime Senior Researcher Masataka Goto. This newly developed system uses Asratec’s “V-Sido OS” robot control system and AIST’s “Songle” active music listening service (http://songle.jp).

V-Sido x Songle is a music-synchronized robot control system that selects and uses predefined motion patterns one after another according to the beat structure (beats and bar lines) and music structure (chorus sections and repeated sections) automatically analyzed by Songle. It uses V-Sido OS’s robot control technologies to enable a variety of robots to dance in real time. The main features of V-Sido x Songle are as follows:

  • When a user selects a song on the Internet that has been automatically analyzed by Songle, a robot can instantly dance to that song.
  • Since differences in shapes and sizes of robots can be automatically compensated, different types of multiple robots can all dance in unison by simply giving the same choreographed motion patterns.
  • A user interface that makes it possible to assign different dancing motions for each section of the music structure enables motion patterns to be changed even while the robot is dancing.
  • When a user commands the robot to walk, the lower body of the robot starts walking in a specified direction while its upper body keeps dancing according to motion patterns.
  • Since robot-control information is dynamically generated, the robot can automatically stabilize its body to keep dancing even if the floor is tilted.

V-Sido x Songle generates the robot dancing by managing V-Sido OS and Songle via the Internet as shown in Figure 1. V-Sido x Songle leverages the “Songle Widget” player (http://widget.songle.jp) that enables external use of Songle’s analyzed results for over 800,000 songs. Although Songle Widget receives the analyzed results from Songle, a selected song is streamed directly from the original web site without passing through Songle. V-Sido x Songle controls a robot by switching motion patterns according to the current playback position of the song. For example, movements of hands or feet are synchronized with beats or bar lines and larger dancing motions can be automatically used during chorus sections. To control the motors for joints, V-Sido OS’s robot control technologies are used. V-Sido OS can compensate differences in shapes and sizes of various robots and ensure stable control robust against environmental disturbances. Furthermore, the V-Sido x Songle user interface enables a user to select a motion pattern and assign it to each section of the music structure obtained from Songle, and to command a robot to walk in a certain direction while the robot keeps dancing. Previously it was difficult to interactively change dance motions in this way while controlling robots in an automatic fashion.

figure1_eng

Figure 1: System overview of V-Sido x Songle.

V-Sido x Songle can control various robots that have already been supported by V-Sido, though some robots with slow actuators, such as those including hydraulic cylinders, could not keep up with the speeds required for dance motions. V-Sido x Songle has succeeded in controlling several different robots such as GR-001 (manufactured by HPI JAPAN), ASRA C1 (manufactured by Asratec Corp.), SE-01 (manufactured by Sagawa Electronics, inc.) and DARWIN-MINI (manufactured by ROBOTIS).

図2:「V-Sido × Songle」での動作実績のあるロボットの例(左から「ASRA C1」、「SE-01」、「GR-001」)

Figure 2: Examples of robots that are compatible with V-Sido x Songle (from left: ASRA C1, SE-01, and GR-001).

The newly developed V-Sido x Songle system is the result of joint research conducted by Asratec and AIST. Both entities will continue to conduct joint development to add functions and enhance performance, while cooperating with other companies for practical use, including usage at events and commercialization.

Demonstration video:

About V-Sido OS

V-Sido OS is real-time robot control software that can generate robot motions and ensure stability that can withstand sudden shocks and not trip over even on tilted surfaces. V-Sido OS has high versatility for various shapes and sizes of robots. No matter the application, V-Sido can control a variety of robots. V-Sido was developed by Asratec Chief Robot Creator Wataru Yoshizaki.

About Songle

Songle is a web service that analyzes music by using music-understanding technologies and visualizes musical elements (chorus sections, beats, melody line, and chords) so that a user can enjoy music listening in more active ways. It has already analyzed more than 800,000 songs in video format on video-sharing services (Niconico and YouTube) or in MP3 format on various web sites. The user can correct errors in the analyzed results when necessary. Songle features an interface that enables the user to actively browse music by jumping to different playback positions while seeing the musical elements. Moreover, Songle Widget enables external web applications to use the musical elements for music-synchronized display and control, which was hitherto difficult to achieve without music-understanding technologies. Songle has been developed by AIST under the research project “Building a Similarity-aware Information Environment for a Content-Symbiotic Society” (Research Director: Masataka Goto) that was accepted in the research area “Creation of Human-Harmonized Information Technology for Convivial Society” of the CREST (Core Research for Evolutional Science & Technology) funding program provided by JST (Japan Science and Technology Agency). It has been open to the public for research purposes.

Web site: http://songle.jp

Press contacts

Asratec Corp.
Roppongi 2-4-5, Minato-ku, Tokyo 106-0032, Japan
TEL: +81-3-5549-1400
E-mail:info01@asratec.co.jp

National Institute of Advanced Industrial Science and Technology (AIST)
Public Relations Department
8th Floor, Tsukuba Headquarters Bldg.
AIST Tsukuba Central 2, 1-1-1 Umezono, Tsukuba, Ibaraki 305-8568, Japan
TEL: +81-29-862-6216
FAX: +81-29-862-6212
E-mail:press-ml@aist.go.jp

Press contacts

Asratec Corp.
Roppongi 2-4-5, Minato-ku, Tokyo 106-0032, Japan
TEL: +81-3-5549-1400
E-mail:お問い合わせアドレス

National Institute of Advanced Industrial Science and Technology (AIST)
Public Relations Department

8th Floor, Tsukuba Headquarters Bldg.

AIST Tsukuba Central 2, 1-1-1 Umezono, Tsukuba, Ibaraki 305-8568, Japan

TEL: +81-29-862-6216

FAX: +81-29-862-6212

E-mail:press-ml@aist.go.jp