Sony Computer Science Laboratories, 2002-2006
As part of the music research team of Sony CSL (led by Francois Pachet), I designed and developped several music interactive systems based on real-time audio processing. This aimed to fill the gap between "playing" and "listening" to music: these new instruments use musical materials automatically extracted from arbitrary mp3 files. If you like Paul McCartney, why not play his voice like you can play a piano?
Joint work with: Francois Pachet, Anthony Beurive, Aymeric Zils, Peter Hanappe.
Our "SongSampler" system automatically samples audio events (e.g. notes) from real music signals, analyses their pitch and other musical properties (timbre...) with pattern recognition techniques, and maps them to a Midi keyboard. The samples are automatically pitch-shifted and time-stretched to match Midi input. With one click, the user can play with an instrument such as Paul McCartney's voice, or Jimi Hendrix's guitar.
Related publications:
Aucouturier, J.-J., Pachet, F. and Hanappe, P. From sound sampling to song sampling. Proceedings of the 5th International Conference on Music Information Retrieval (ISMIR), Barcelona, Spain, Oct. 2004.
All demos below were produced by interaction in real-time, on a commercial midi keyboard. They do not result from sequencing or
editing, contrary to traditional sampling.
First version
Proof of concept, with quick and dirty algorithms.
French nursery rhyme "J'ai du bon tabac", played with instruments sampled from:
State-of-art algorithms for time-stretching, etc. + real-time interface using Korg Microkontrol (see video)
This uses automatically sampled sounds from songs by Ben Harper (extract),
Gorillaz (extract) and Beethoven (extract)
This is a Beatles bossa nova (or whatever), using guitar sounds automatically selected and sampled from Beatles - Yesterday (extract) and voice from Beatles - Blackbird (extract)
