me

<< BACK <<
Dancing Miuro Project
ZMP Inc. / University of Tokyo, 2007
We collaborated with Japanese robot manufacturer ZMP to design a dance algorithm for commercial ipod/robot Miuro. The algorithm uses automatic beat analysis from music in real-time to command a chaotic controller, based on biological neuronal models. We demonstrated the system in a high-profile public operation, in Apple Store GInza, Tokyo (Japan).
Joint work with: Takashi Ikegami, Yuta Ogai, Eric Silverman (University of Tokyo), Masaki Segawa (ZMP Inc.), Keiichiro Shibuya (ATAK)

Implementation:

Our robotic platform, the MIURO manufactured by ZMP Inc, is a two-wheeled musical player equipped with an IPod mp3 player interface and a set of loudspeakers. It has built-in autonomous behaviour, which can be overridden by sending duplets of instantaneous speed values (left and right wheel), through wireless communication with a computer. In our implementation, musical playback is initiated on the robot using the manufacturer's remote control. Music is played by the embedded mp3 player on the robot, and rendered on the robot's loudspeakers. Simultaneously, the robot sends a notification, including an identifier of the song being played, to the client PC via wireless connection. The iPod music database is duplicated on the PC. Upon reception of the robot playback notification, real-time beat-tracking of the corresponding audio item is started (on the PC), and beats are sent to the network (also running on PC) as they are found. After propagation through the network, output pulse trains are converted to wheel velocities, and sent back via wireless to the robot at rate RTS (see paper). When playback is finished, no more beats are found by the audio analysis module, and the network quickly converges to zero output, putting the robot to halt.


Related publications:

  • Aucouturier, J.-J., Ogai, Y. and Ikegami, T. Using Chaos to Trade Synchronization and Autonomy in a Dancing Robot. IEEE Intelligent Systems, vol. 23, No. 2, 2008.
  • Aucouturier, J.-J., Ogai, Y. and Ikegami, T. Making a robot dance to music using chaotic itinerancy in a network of Fitzhugh-Nagumo neurons in Proceedings 14th International Conference on Neural Information Processing (ICONIP2007), Kitakyushu, Japan, Nov. 2007


    Public demonstration:

    Held at Apple Store Ginza, Tokyo, Japan on May 31st, 2007. (Music: Keiichiro Shibuya/Atak. Video: Yoshiro Osaka)




    Media reports:



    Simulations:

    Orbit #1: Network time scale: 1ms; Motor time scale: 1ms; Robot time scale: 1ms;
    Chaotic behaviour with CI, too fast for implementation in real robot.



    Orbit #2: Network time scale: 2ms; Motor time scale: 40ms; Robot time scale: 100ms;
    Chaotic behaviour with CI, now with realistic robot time scale.



    Orbit #3: Network time scale: 2ms; Motor time scale: 40ms; Robot time scale: 100ms;
    Same as above, but different song creates different attractors.



    Orbit #4: Network time scale: 5ms; Motor time scale: 50ms; Robot time scale: 50ms;
    Similar as above, creates a fixed point, too fast for real robot.



    Orbit #5: Network time scale: 50ms; Motor time scale: 100ms; Robot time scale: 100ms;
    Network update rate is slow: long attractors, only slightly perturbated. very clear synchronization with music, but no so exploratory.

    Orbit #6: Network time scale: 100ms; Motor time scale: 100ms; Robot time scale: 100ms;
    Network update rate too slow: no chaos, stable attractor, no synchronization to music, because neurons always in refractory mode