The AudioCubes are a collection of wireless intelligent light emitting objects, capable of detecting each other's location and orientation, and user gestures, and were created by Bert Schiettecatte. They are an electronic musical instrument used by electronic musicians for live performance, sound design, music composition, and creating interactive applications in max/msp, pd and C++. The AudioCubes are an example of a Tangible User Interface as well as an ambient device. The concept of the AudioCubes was first presented by Bert Schiettecatte in April 2004 at the CHI2004 conference in Vienna. A first prototype of the AudioCubes was shown at the Museum for Contemporary Art, MUHKA in Antwerp in December 2004. They were used in an art installation created in collaboration with Peter Swinnen during the Champ D’Action Time Canvas festival. In January 2007, the AudioCubes were launched on the market and offered online on the website of Percussa, the company which Bert Schiettecatte founded in October 2004 to further develop and commercialize the AudioCubes.
Hardware architecture
Each AudioCube is identical, and has a small built-in computer which is able to measure distances as well as detect the position and location of the other cubes in a network. The AudioCubes work without drivers, are plug and play and communicate using high speed HID. An AudioCube has four onboard infrared sensors to communicate and measure distances to objects nearby, digital signal processors, a USB-rechargeable battery, and a translucent housing.
The AudioCubes work with any software and hardware which is MIDI compatible. To hook up the AudioCubes to MIDI compatible software/hardware a middleware application, called MIDIBridge, has to be used. Besides MIDI compatibility, the AudioCubes also come with an OSC server to send and receive OSC data.
Software Applications
Several applications have been created for the AudioCubes, each focusing on a different use: sound design, music composition, live performance, as well as creating applications in max/msp, pd and C++.
MIDIBridge: sends/receives MIDI from/to other MIDI compatible software/hardware
DeckaBridge: specifically created to let AudioCubes work with the DJ software Deckadance
PluginWrapper: application to use VST-plugins together with the AudioCubes.
Loopshaper: to make sounds and loops with one AudioCube
Modulor: detects and communicates to a network of AudioCubes wirelessly, and reassembles data from a MIDI device
Improvisor: a generative MIDI step sequencer to create generative music using AudioCubes
Evolvor: generate complex LFO waveforms using AudioCubes
Flext external for Max/Msp: to create your own max patches for the AudioCubes
OSC server: sends information about the location, orientation, and sensor data of the AudioCubes to other OSC applications
SDK C++ library: to create your own applications for the AudioCubes
FM synthesizer for AudioCubes: creating sounds with AudioCubes without needing any additional hardware/software.
In addition, a number of Max/MSP patches were created to work with AudioCubes
Software Uses
An overview of how AudioCubes can be used for different uses
AudioCubes for Live Performance
The AudioCubes can be used to send MIDI notes to MIDI compatible software/hardware using MIDIBridge. When two AudioCubes are put next to each other, they detect each other, and triggers are sent as MIDI notes. These triggers can then be used to control on/off type of signals, such as start and stop audio clips in a Digital Audio Workstation, such as Ableton Live. At each face of the cube, a different audio clip can be assigned in a DAW. The AudioCubes can also measure distances to nearby objects or your hands when configured as a sensor cube in MIDIBridge. In the same way, this sensor data is sent to the computer as a continuous controller which can be used to control parameters in the DAW. Since each cube has 4 sensors, up to 4 parameters can be controlled per AudioCube. In addition, you can also control the RGB colors of the AudioCubes and use this information as feedback during a live performance.
AudioCubes for Sound Design
The sensors of the AudioCubes can also be used to shape sounds. By moving hands and fingers closer or further away from the 4 sensors, it generates 4 different MIDI CCs which can be sent to MIDI compatible instruments. When using an AudioCube in this way, it can be compared to a 4D optical theremin. The AudioCubes can also be linked to LFOs by using the software application Evolvor. The LFO waveforms are designed in the graphical editors of evolvor. Each AudioCube is then automatically linked to an LFO, because of the topology detection. LFO signals can be added and removed, by adding and removing AudioCubes. The signals can also be mixed and matched, by mixing and matching AudioCubes.
When using the Improvisor application, velocity as well as semitone patterns are automatically linked to every AudioCube. Every AudioCube plays the melody created by both patterns. When cubes are placed next to each other they can follow the melody of each other. In this way, you can easily compose music by mixing and rearranging AudioCubes.
AudioCubes for Creating Interactive applications
Several tools have been created to make your own applications for the AudioCubes in max/msp, pure data, C++
Artists
The AudioCubes are used by many live performers all over the world such as Mark Mosher, Pearls for Swines, Richard Devine, Steve Baltes, Bostich from Nortec, Ilan Kriger, Arecio Smith, Julien Pauty, the European Bridges Ensemble
Example of Tangible User Interface
The AudioCubes are an example of a Tangible User Interface. The past few years a lot of research has been done in the field of Tangible User Interface. The Reactable is another example of such an interface. It is an installation on which people can move around objects which are followed by a camera and projector on a surface.
Awards
For the creation of the AudioCubes, Bert Schiettecatte received in 2009 the prestigious Qwartz Electronic Music Awards in Paris. He was also invited to give a talk at TEDx Mediterranean in Cannes, September 2010.
