Computers & Mobile
The Future of Interfaces Is Multi-Touch

109397667 A875C72D91
Jeff Han gave an amazing talk and demo at Etech – pictured here, LCARS multi-tounch interface!

While touch sensing is commonplace for single points of contact, multi-touch systems enables a user to interact with a system with more than one finger at a time, allowing for the use of both hands along with chording gestures. These kinds of interactions hold tremendous potential for advances in efficiency, usability, and intuitiveness. Multi-touch systems are inherently also able to accommodate multiple users simultaneously, which is especially useful for collaborative scenarios such as interactive walls and tabletops.

  • Video of multi-touch – Link.
  • Photo gallery of the multi-touch gear – Link.
  • Transcript of Jeff Han’s talk – Link.
  • I shot some video, was far away – but here it is – Link.

Bonus: I talked to Jeff after his session and I think we’ll have a DIY version on MAKE soon!

50 thoughts on “The Future of Interfaces Is Multi-Touch

  1. This is something many Mac laptop users are already quite fond of, albeit in a simpler form. New Mac laptop trackpads allow “two finger scrolling”. When using one finger on the trackpad you control the mouse. If you chord on the trackpad with two fingers you can scroll horizontally and vertically.

    I too look forward to the Make version, the interfaces seen in Minority Report are getting much closer to a reality!

  2. If this is rear projection, the way it works is simple:

    Coloured light (IR?) is shone into the edges of the display, and when you put your finger on the surface, it changes the refractive index at the surface and the image of your fingerprint will appear on the back of the panel. They then have a camera behind the panel to pick up where is being touched.

    However, what would be really interesting is how they make it thin, like the Jazzmutant.

    Bring on the guide!

    ch424

  3. Jeff Han’s display isn’t the first or the only way to do multi-touch – the interesting thing is that it’s so simple, scalable, and cheap to build with readily-available materials. The downside is it needs the extra space at the back with a projector and camera. The FTIR technique is essentially what you describe (it’s in the already linked FTIRtouch/FTIRsense pages and the ACM paper) with a few improvements.

    Jazzmutant’s Lemur is based on very different tech – a standard LCD screen with a commercially produced touch surface overlaid onto it – since it doesn’t use the FTIR technique it can be as thin as the screen is.

  4. Ok, now the only question i have is once you build one, what software do you use? I’m sure there isn’t a windows, linux, insertyourfavoriteoshere driver out there for this. Thats the one thing stopping me from getting on building this this weekend.

    Any one out there feel like taking a stab at writing driver software for this? At least modifying some code to take advantage of this kick ass interface.

  5. I’ve been planning software for a week or two now; first I have to get a simple DirectShow framegrabber routine working (my current camera is an IR-hacked webcam :) then I’m off and racing :)

    …who knows, some of the FTIRtouch software might be made available for the Make howto? (hint, hint :)

  6. Pelrun, thanks for the ACM link. (As the AmEx commercial says, “membership has its privileges”. For anyone thinking of building one that actually functions, the paper will save you a lot of time.) I’m curious how you coupled the LEDs to the glass/plexi? I was going to file and polish the ends to be smooth and then fix them with index-matching U/V glue. Also, did you source a band-pass IR filter?

    Someone above asks about software to “drive” it. I’d opine that that’s what this research project is all about. The author makes the point that the screen is dead simple. *All* of the work is in the software. I will be using DirectShow for the image capture, OpenCV for the computer vision/tracking, and Ogre3D for graphics. While I’ll be working in Visual Studio on XP, Mac wouldn’t be a problem: OpenCV builds on any POSIX system.

  7. Pelrun, thanks for the ACM link. (As the AmEx commercial says, “membership has its privileges”. For anyone thinking of building one of these multi-touch screens, the paper will save you a lot of time.) I’m curious how you coupled the LEDs to the glass/plexi? I was going to file and polish the ends to be smooth and then fix them with index-matching U/V glue. Also, did you source a band-pass IR filter?

    Someone above asks about software to “drive” it. I’d opine that that’s what this research project is all about. The author makes the point that the screen is dead simple. *All* of the work is in the software. I will be using DirectShow for the image capture, OpenCV for the computer vision/tracking, and Ogre3D for graphics. While I’ll be working in Visual Studio on XP, Mac wouldn’t be a problem: OpenCV builds on any POSIX system.

  8. Pelrun, thanks for the ACM link. (As the AmEx commercial says, “membership has its privileges”. For anyone thinking of building one of these multi-touch screens, the paper will save you a lot of time.) I’m curious how you coupled the LEDs to the glass/plexi? I was going to file and polish the ends to be smooth and then fix them with index-matching U/V glue. Also, did you source a band-pass IR filter?

    Someone above asks about software to “drive” it. I’d opine that that’s what this research project is all about. The author makes the point that the screen is dead simple. *All* of the work is in the software. I will be using DirectShow for the image capture, OpenCV for the computer vision/tracking, and Ogre3D for graphics. While I’ll be working in Visual Studio on XP, Mac wouldn’t be a problem: OpenCV builds on any POSIX system.

  9. I’d be happy to help with software if someone builds me one ;) I’m also experienced with Ogre3d. Is the paper available anywhere that doesn’t require registration? I’m too cheap to spend the 10$. What kind of material do they project the screen onto? I assume it’s something that absorbs regular light but lets IR pass thru..

  10. Pelrun, any luck with your screen? I just got a rough prototype up and “running” and the sensitivity is extremely low. Here’s what I’ve done.

    • Bought a 16″ x 20″ piece of clear plexi (acrylic?). Sanded and buffed the edges to transparent.
    • Purchased 6 IR LEDs (1.5V/100wA, 100mW typ (presumably this is optical power?), 880nm, 40 degree beam angle) from DigiKey.
    • Ground the normally rounded LED casing flat and sanded/buffed them transparent. Edge mounted the LEDs to the plexi with Loctite U/V cured glue. I’ve currently mounted 3 along one edge, 3 along a neighbouring edge.
    • Wired the LEDs in parallel with 35 ohm current limiting resistors, powered by a 5V supply. As expected, it pulls about 600mA.
    • I’m still waiting on a 12mm dia 880nm bandpass filter from Newport Optical, should be here Tuesday.

    For the moment I’m using my Logitech Quickcam with the lights off ;-) With the gain of my camera jacked up ridiculously high, I’m seeing only very faint glow spots, and even then only within about 2″ (straight out) from a LED. Perhaps I am not dumping enough light near the critical angle? And I wonder if elliptical LEDs might be a good idea to get more coverage in the plane?

    Thoughts anyone?

  11. Probably you just need more LED’s. Also, maybe the angle is a bit much. I’ve decided to give a go at making a screen too. So far I’ve hacked my webcam and I’m gonna pick up some acrylic tomorrow.

  12. It struck me tonight that instead (or as well as) boosting the IR LED power, I should really look at the detection end – the webcam. A quick search revealed that webcams typically (universally?) have an IR filter in place! I’m looking forward to removing this tomorrow in the hope that I can more clearly observe the FTIR. dwallin33, is this what you meant by “hacking” your webcam?

  13. Yeah absolutely.. I don’t think you’d be able to see any IR with the filter in there. It’s fairly easy to remove once you get the lense part open (I have a create webcam I’m using right now). If you haven’t, check out this page: http://www.hoagieshouse.com/IR/

  14. Why don’t you email me at ruiner33 [_at_] hotmail . com maybe we can collaborate on the software or share some tips at least.

  15. dwallin33, yes, I discovered that page last night. Removing the filter, I’m now getting beautiful FTIR-induced tracking spots. I also noticed what Han reports in his paper – the effect is greatly diminished if the user has dry skin.

  16. Cool, glad to hear it! I just got the projector I ordered off ebay and my LED’s should be arriving any day now. I’ll be posting about my progress on my blog too: http://www.whitenoiseaudio.com/blog/

    BTw, I have some ideas about how the software detection could work if you are interested.

  17. For all those excited about this enough to look into creating their own table (which I am also doing). The best place to go is http://www.nuigroup.com/forums . They call themselves the natural user interface group and they have alot of information about building and coding for multitouch interfaces. There are libraries out that have some basic functionality.

Comments are closed.

Tagged

current: @adafruit - previous: MAKE, popular science, hackaday, engadget, fallon, braincraft ... howtoons, 2600...

View more articles by Phillip Torrone