What do you get when you combine a box of sand with a Microsoft Kinect, an overhead projector, and a series of illuminated pucks? The Sand Noise Device (SND), the musical brainchild of a talented team of multimedia graduate students at California State University East Bay. Consisting of Jay Van Dyke, Devin Dominguez, Matt Roads, and James Saxon, the team describes SND as:
The SND is both a complex generative music system as well as a novel and intuitive interface for influencing and interacting with this system. The interface consists of a table height box filled with sand, a Microsoft Kinect (which provides an RGB camera and depth sensing capabilities), an overhead projector, and several internally lit tangible objects. The Kinect is used to track object position, object color, and sand topography. The projector is used to provide visual feedback. Sound is provided by a multi-channel loudspeaker system arranged around the box.
The various parameters that determine the functionality of the SND’s generative music system are influenced by the user’s interactions with the sandscape and provided tangible objects. By manipulating the sand and objects the user is able to influence various aspects of the generative music system.
The SND will be on interactive display at the upcoming Maker Faire Bay Area, taking place May 17 and 18 at the San Mateo Fairgrounds. MAKE videographer Emmanuel Mota recently visited the team to shoot some footage of the SND in action.
We also followed up with the team, fresh back from displaying the SND at Moogfest, to uncover their backstory and learn more. Read on for more details and come play with the SND at Maker Faire Bay Area.
1. How did the idea for the SND and what was your R+D process like?
Our initial plan was to develop a musical interface that was inviting and intuitive for users of all abilities and age levels. We also wanted to give the user the fine-grained level of control, such as one would find in a modular analog synthesizer. Very early on in our development, we decided that sand would be a great medium for control due to people’s familiarity with it as well as its tactile nature.
The original plan for user control was to track gestures and shapes in the sand as well as measure its topography. Our initial implementations and user testing lead us to the conclusion that directly linking the physical configuration of the sand to the instrument’s various sonic parameters results in an underwhelming experience. In order for users to get a satisfying sense of control, we had to exaggerate the sonic changes to a level we all felt was unmusical. By giving too much control to the user we ended up with a system that was neither inviting nor intuitive.
Our next iteration included tracking the location of colored objects in the sand. This gave the users an easy to understand the one-to-one relationship between their physical actions and the sound in a manner similar to a common X/Y pad. While this approach allowed for more intuitive interactions, it wasn’t an engaging enough experience to capture most of our testers’ interest.
Our current design is an attempt to replace direct control with a more indirect system. Instead of providing a sand-based musical controller, we instead decided to design a generative system and provide the user with a way to control its functionality. We used the same color-tracking and topographical mapping approaches as before but instead created a much more autonomous system. We also added a visual feedback component as well as revamping the style of the various physical components. At this stage we feel that we are providing the user with just the correct amount of control for them to find the experience engaging but not so much that they will get frustrated.
2. Who are the team members and what was each person responsible for?
Our team has no defined roles. We meet regularly to decide what needs to be done next and then each of us grabs a piece of it and gets to work. Because no one ever says “no,” leadership changes from task to task and often several times in between. Having said that, we all have our strengths and weaknesses. Jay Van Dyke brings a strong sense of focus and organization, not to mention an encyclopedic knowledge of all things Macintosh. Devin Dominguez is the first person to ask in all matters relating to sound design and generation. Matt Roads is usually the one to catch the details and handle logistics, and James Saxon often took point in fabrication.
3. You recently displayed the SND at Moogfest. How was it received?
We had an amazing experience at Moogfest. Being that it was on the other side of the country, it came with some interesting logistical and financial challenges. Once we got there the response was overwhelming. We were asked by Moogfest to set up our project at the opening night VIP party, and it was an immediate hit. When we eventually set it up at our permanent installation location for the weekend, we had a nonstop flow of visitors. Everybody from local Asheville [N.C.] residents just walking down the street to talented electronic musicians, such as Dan Deacon and members of Kraftwerk, were drawn in and seemed to keep coming back. By the end of the festival, we had people telling us that they heard about it on Twitter or Instagram and they just had to come check it out.
4. What inspired you to show the project at Maker Faire Bay Area?
As multimedia graduate students at CSU East Bay, part of our education included attending the 2013 Maker Faire Bay Area. This experience was among the most valuable things we did during our first year in the program. It was really the first opportunity most of us had to see the sort of things we were studying up close. It was a very inspiring experience that helped us and our fellow classmates bridge the mental gap between the skills we had been acquiring and actual real world applications. In addition, one of CSU East Bay’s thesis groups (“Beyond the Water Curtain“) had the opportunity to present their creation at last years Maker Faire, and it provided them with tons of valuable feedback.
5. Are there any developments in the works for the SND? What do you hope to do with it in the future?
The SND has a few potential directions for the future. We’ve been asked by many people about whether the setup is for sale, and we’ve been approached already about buying the entire idea and installation. There are several minor changes that we would like to make based on our experience at Moogfest, but primarily we would like to streamline the setup and calibration process so that it doesn’t require as much time and attention as it currently does. We have contemplated the idea of getting it permanently installed somewhere as part of an interactive exhibit or we could continue to tour it at festivals like Moogfest and Maker Faire. Outside of the SND we all enjoy working together as a unit and would love an opportunity to continue developing interactive art.