Nintendo controllers as musical instruments

Jeff Hoefs is doing some really amazing hacks and mods with Nintendo controllers. He’s made PowerPad midi keyboards, PowerGlove music controllers and Nintendo Uforce controllers to create music. All at once too! He sent this overview in of what’s what he’s up to, along with some great photos of the mods in action!

Nintendo controllers as musical instruments – Jeff Hoefs
I spotted a Nintendo PowerPad at a thrift store a couple of years ago and thought it would make an interesting midi keyboard. Hacking this controller was fairly simple. Inside are two 4021 8-channel parallel to serial converters. 8 pads are wired to one of the ICs and the remaining 4 pads are wired to the other. I used a microcontroller to convert the status of the shift register into midi data. My midi converter also has 2 pots – one to change octaves (including a position that will keymap to my drum machine) and another to set the midi velocity (volume). With this interface, the pad can be connected to a synthesizer, drum machine, or to a computer.

After getting the glove to work I thought it would be cool to use a PowerGlove as a musical controller as well. I purchased a used glove on eBay and found a timing diagram and some C code on the Internet that I used to write an assembly routine to access the glove’s analog data (hires mode) and send it via serial to a computer. After I got the glove working I tried using it for everything from controlling filter parameters, to punching out beats, a virtual keyboard, and even a ‘battle’ with a DJ. Unfortunately due to the amount of time it takes to output all 12 of the glove’s data packets, it is not the best controller for timing critical functions like playing notes or manipulating filter parameters.

I was using the glove and pad simultaneously in a band. The best function of the glove was the buttons on the wrist. I created a program that allowed me to use the glove’s button data to select sample banks, filter parameters, and switch between various uses for the xyz and finger data. For example, in one mode I could make a fist and dance out a simple beat on the pad, then when I opened my hand, the beat would continue as a loop, I could then add additional layers to the loop by repeating the process. By pressing another button or combination of buttons on the wrist I could then select another mode such as the virtual record-scratching mode to battle with the band’s DJ.

I also found a couple of Nintendo Uforce controllers (horizontal and vertical planes of infrared sensors) so I made another midi interface that could be used to input data from both Uforces, a PowerPad, and a standard NES controller simultaneously. I even used the actual sockets from old NES consoles so I didn’t have to add my own plugs to the controllers. As is, this system cannot directly used with a synthesizer, it has to work with a program that allows custom mapping of midi data (Max, PD, Supercollider, etc.). However, the code could be rewritten the code to allow direct connection – that’s the beauty of the DIY world.