Nintendo controllers as musical instruments

glove_interface_guts.jpgJeff Hoefs is doing some really amazing hacks and mods with Nintendo controllers. He’s made PowerPad midi keyboards, PowerGlove music controllers and Nintendo Uforce controllers to create music. All at once too! He sent this overview in of what’s what he’s up to, along with some great photos of the mods in action!

Nintendo controllers as musical instruments – Jeff Hoefs
I spotted a Nintendo PowerPad at a thrift store a couple of years ago and thought it would make an interesting midi keyboard. Hacking this controller was fairly simple. Inside are two 4021 8-channel parallel to serial converters. 8 pads are wired to one of the ICs and the remaining 4 pads are wired to the other. I used a microcontroller to convert the status of the shift register into midi data. My midi converter also has 2 pots – one to change octaves (including a position that will keymap to my drum machine) and another to set the midi velocity (volume). With this interface, the pad can be connected to a synthesizer, drum machine, or to a computer.

padnglove_action1.jpg

padnglove_action2.jpg

After getting the glove to work I thought it would be cool to use a PowerGlove as a musical controller as well. I purchased a used glove on eBay and found a timing diagram and some C code on the Internet that I used to write an assembly routine to access the glove’s analog data (hires mode) and send it via serial to a computer. After I got the glove working I tried using it for everything from controlling filter parameters, to punching out beats, a virtual keyboard, and even a ‘battle’ with a DJ. Unfortunately due to the amount of time it takes to output all 12 of the glove’s data packets, it is not the best controller for timing critical functions like playing notes or manipulating filter parameters.

glove_interface_guts.jpg

glove_interfaceb.jpg

nintendo_midi_interface.jpg

I was using the glove and pad simultaneously in a band. The best function of the glove was the buttons on the wrist. I created a program that allowed me to use the glove’s button data to select sample banks, filter parameters, and switch between various uses for the xyz and finger data. For example, in one mode I could make a fist and dance out a simple beat on the pad, then when I opened my hand, the beat would continue as a loop, I could then add additional layers to the loop by repeating the process. By pressing another button or combination of buttons on the wrist I could then select another mode such as the virtual record-scratching mode to battle with the band’s DJ.

nintendo_midi.jpg

I also found a couple of Nintendo Uforce controllers (horizontal and vertical planes of infrared sensors) so I made another midi interface that could be used to input data from both Uforces, a PowerPad, and a standard NES controller simultaneously. I even used the actual sockets from old NES consoles so I didn’t have to add my own plugs to the controllers. As is, this system cannot directly used with a synthesizer, it has to work with a program that allows custom mapping of midi data (Max, PD, Supercollider, etc.). However, the code could be rewritten the code to allow direct connection – that’s the beauty of the DIY world.

uforces.jpg

uforce_use.jpg

12 thoughts on “Nintendo controllers as musical instruments

  1. i think this is really fantastic! i find it very strange that not more than two days ago i was imagining something exactly like this… AND HERE IT IS!

    i’ve got to get to work on some of my own

  2. very cool yes, but at what cost? i promise you this guy isn’t getting any poonanny for the next 13 years. not so cool after all!

  3. OMG. I LOVE Ninetendo’s Powerpad. I played with it sooooo much when I was younger. I remember playing the this skateboarding game and World Class Track Meet. Ahhh, memories.

  4. you said that you can make a fist, record a loop on the pad, and then release your fist to keep the loop going.

    What software do you use to make it loop in realtime? I saw Kid Beyond doing that one time and I’ve always wanted to myself.

  5. any diagrams to do this? is it based off of an open midi converter design?
    i would love to be able to use a standard nintendo controller to control a midi device but i dare not try it on my own.

    jeff hoef give us some details

Comments are closed.

current: @adafruit - previous: MAKE, popular science, hackaday, engadget, fallon, braincraft ... howtoons, 2600...

View more articles by Phillip Torrone