Webcam Piano 2.0 (2010)

Interactive sound and video installation
1ch HD LCD, 2ch sound, infrared camera, custom software
Duration: N/A
Dimensions: variable (42″ – 90″ screen)

Webcam Piano is an ongoing project researching interpretation of movement into sound, primarily classical influenced (though not necessarily western), traditional sounding music as opposed to abstract electronic audio or soundscapes. My main aim is to create an instrument that sounds conventional – and is governed by conventional musical rules established over centuries of musical and social evolution – but is performed via a highly un-conventional – yet hopefully very intuitive – approach, allowing users to get deeply involved within minutes, if not seconds of starting to play with it.

The first incarnation of Webcam Piano came about as a little experiment in 2008, as an opensource Quartz Composer patch. This was shortly followed by an opensource Java / Processing implementation and again quickly followed by an opensource C++ /openFrameworks implementation which also evolved into a large scale multi-instrument installation at the Glastonbury festival.

Webcam Piano 2.0, uses much smarter algorithms (it was developed before affordable depth cameras like the kinect were available) to try and understand motion and interpret how the person (or people) are moving and what kind of sounds they might be wanting and expecting to create with their movements. It also offers a simple gestural interface to switch between 3 (potentially more) different musical modes and color schemes, each representing a different emotion and feel.


made with openFrameworks

Related keywords

c++, computer vision, fluid simulation, generative music, generative visuals, glsl, infrared, installation, interactive, midi, motion tracking, opencv, openframeworks, optical flow, particles