You are here.

• Concept
• Technology
• Final Product
• Audio/Video
• Paper

all content and design by Yann-Loïc Seznec
2007

The technical creation of the Interactive Sound Room centers around the Arduino USB I/O controller board. I was familiar with the Arduino, having used it to interface sensors and switches with Max/MSP in my Tea + Jam project. It is a very simple board with 13 digital and 6 analog pins, which can be powered by USB. A free open-source Max/MSP patch provides easy access to the data from the sensors and switches. The only downside had been the very small and awkward pinholes into which everything had to be plugged in. Therefore my first step in the building of the sensor system was to construct a much more solid modular switching system.

I decided to use 1/4" TRS jack plugs, because each sensor or switch needs three pins (power, ground, signal). In addition, they are cheaply and easily available at virtually any music or electronics store, and are space efficient and relatively solid. Once the holes had been drilled, the female jacks mounted, and the wiring soldered inside, I had a fully functioning and very sturdy Arduino Box.

The only problem I had not foreseen had to do with the physical nature of TRS jacks; when a male jack is plugged into a female jack the ring and sleeve are in contact for a split second. As I had wired those as power and ground, the board would short out and restart each time I plugged anything in, forcing me to unplug the USB cable and restart Max/MSP. This problem was easily surmounted, however, by placing a "normally open" button on the power circuit, which bypasses the power when pressed. This allows me to plug or unplug any sensor into any slot without unplugging the board or closing Max/MSP.

When thinking about sensors for building a motion tracking sound installation, the best type of sensor seemed to be ultrasonic. Their main advantage is the ability to give a continuous precise reading, showing exactly how far away an object is at every given point (usually reading 20 times a second or more). The MaxSonar EZ-1 was the most cost and space effective solution, being only several centimeters square and costing only £15 each. I ordered four of these and attached them to 1/4" jack plugs using ethernet cable.

My initial plans for using the ultrasonic sensors all revolved around covering as much surface area as possible with the four sensors. I soon discovered, however, that using more than one ultrasonic sensor pointing in even remotely the same direction is more or less impossible. The sensors, which all operate on the same frequency, appear to get their signals confused, and every five or six seconds the normally stable output will jitter widely for one or two seconds before returning to the normal measurements. By only using two sensors, separating them apart about three meters, and smoothing their output in Max/MSP (see the programming section), I was able to get usable data for two "corridors" in the space. This was not nearly enough to carry out my initial vision, however, so I had to resort to other means.

I had originally spurned the use the infrared sensors in my project, dismissing them because they can only detect motion, rather than distance. Since I wanted to create a system that responded as smoothly to a user's movements as possible the use of PIR's did not seem logical. The failure of my ultrasonic concept, however, forced me to think of different solutions to the motion-tracking problem. The infrared sensors I found, when plugged into the Arduino, send a "1" when they sense something moving (more precisely, when they sense a difference between readings), and a "0" when they sense nothing. While these were not particularly useful for me as separate values, these could be combined to give an approximate location of the user along one axis. Together with the ultrasonic sensors I had three axis's that could be used to control the sound environment.

My goal, in terms of the Max/MSP programming for this project, was to create a set of tools for using the sensor system I had built to generate sound environments based on the location and movements of the user. In order to do that I needed to make the data as smooth and accurate as possible, which required different techniques for the ultrasonic and infrared sensors.

For the ultrasonic sensors, the best way to clean up the data was to use the limit the speed of the polling to 10 hz, then use the "slide" object. This smoothed the data out a little bit, but every 5 or 6 seconds the sensors would still get confused. Instead of taking this data, then, I do a running average of the last twenty values, as well as using the "look before you leap" object by Matt Wright, which ignores large jumps in a stream of numbers. This provides a reasonably accurate reading from the sensors, however the averaging does slow down the response time. Shrinking the size of the averaging buffer helps this response time, but renders the readings more susceptible to inaccuracies.

The three infrared sensors, I decided, could be used together to generate an approximation of the x-axis. If they were spaced equally, pointed in the same direction, I reasoned that a single person would only be standing in front of one at a time. Therefore I made a system whereby when one sensor was sending a "1", signaling that it sensed movement, it triggered a "metro" object, which sent a corresponding value (0, 50, or 100 on a scale of 100) 10 times a second to a "slide" object. This means that when someone is in the middle, a value of 50 is continuously sent. However, when the user moves towards the right, that sensor begins sending values of 100, so the "slide" object then begins smoothly moving the data upwards with each bang from the "metro" object. This proved to be a remarkably accurate and useful way to generate an x-axis, with a very low rate of accidental readings. The only disadvantage is that it can only work with one user at a time, but since that was my original plan anyway it was not really an issue.

At first I did all of my audio playback and manipulation in Max/MSP alone, however this grew to be needlessly complicated as I began triggering and mixing 16 or more stereo tracks at once. I thus moved all of the audio playback to Ableton Live, which I controlled via MIDI from Max/MSP. In some ways this felt like cheating, after all of the headaches I had trying to get the audio to work well in Max, but it freed me up to create and explore the sound design at a much faster rate - my final version had nearly 50 stereo tracks running at once.

Each axis in the room, then, is mapped to six breakpoint envelopes in Max. This is an arbitrary number, and many more could be added if needed, but I found it largely sufficient. Each breakpoint interface can have a different envelope, so one axis can control many different parameters in vastly different ways. Each of these envelopes is being sent as a MIDI control out, which can then be assigned to any parameter is Ableton Live.

In addition, a counter object is keeping track of how often the user reaches the sides of the installation, and therefore measuring how much they are exploring the space. If enough exploration is done, then different "scenes" (banks of audio samples) are triggered in Ableton Live. If not much motion is detected then the system plays through only one composition, entitled "BeepBeepBeepBeep".

The entire system is turned on and off by a switch hanging from the ceiling of the installation.

Click here to download the Max/MSP MIDI control patch used in 'You are here'.

The patch uses two third party externals (from the CNMAT Max objects); lbyl ("Look Before You Leap") Version 0.2.1 by Matt Wright and running-average.js Version 1.1 by Daniel Cullen and John MacCallum. They are both included in the zip file, but can also be found here.