In 2003, my friend Tony, aka graffiti artist Tempt1, was diagnosed with ALS, a progressive disease that left him almost completely paralyzed except for his eyes. In order to help him continue to make his art, I collaborated with a group of software developers and hardware hackers, including members of Free Art and Technology (FAT), OpenFrameworks, and the Graffiti Research Lab, to make the EyeWriter (eyewriter.org), a low-cost, open source, eye-tracking system that would allow Tempt1 and other ALS patients to draw and control a computer using just their eyes.
Our goal was a super-cheap system that could be made by almost anyone, almost anywhere. There are commercial and research eye-tracking systems, but they’re complex and expensive, ranging upwards of $10,000. We limited the EyeWriter’s design to emphasize low cost and ease of construction over other aspects of performance.
Design
The EyeWriter works by illuminating the user’s eyes in a controlled way and analyzing a video image of his eye movements. For the camera, we chose a PlayStation Eye (aka PS3 Eye), a small $25 webcam sold as a game system accessory.
The PS3 Eye captures 640×480 NTSC video and can be modified for high sensitivity to infrared, which makes the camera a favorite of the multi-touch hacking community. The PS3 Eye then feeds its video via USB to a computer, where it’s captured by the EyeWriter application built in C++ using openFrameworks (openframeworks.cc).
For our first EyeWriter, completed in 2009, we put the PS3 Eye camera in front of one eye, mounting it to an extension attached to the front of a pair of eyeglass frames. The software cropped the video image, boosted its contrast, and thresholded it to show just a black pupil dot against a white background. To increase this dark-pupil effect, we illuminated the eyeball with 2 near-infrared LEDs mounted near the camera.
The software located the pupil’s coordinates in the webcam image, and used a map to associate this location with the spot where the drawing brush needs to be on the user’s computer screen. To build the map, the user ran through a calibration routine where he moved his eyes to visually follow a sequence of dots appearing around the screen.
This 1.0 version was simple and it worked, but only as long as the user’s eyes stayed stationary relative to the screen. Move the head, and it threw the calibration off. So we decided to build a 2.0 version that’s a bit more complex but allows for more normal head movement.
EyeWriter 2.0
In EyeWriter 2.0, the camera sits fixed just below the screen, rather than wearable on eyeglass frames. There are 3 infrared illuminators: a ring of 16 IR LEDs surrounding the camera’s lens, and two 8-LED illuminators on either side of the screen. We hacked the PS3 Eye to tap its VSYNC signal and feed the electrical signal into an Arduino, which uses it to strobe the illuminators with each video frame, alternating between the center and both sides. With the center illuminator on, the IR bounces off the back of the eye and creates the same glowing “red eye” effect that you see in flash photography, but with the side illuminators on, the pupils appear dark. This lets the software locate pupil position as the part of the image that alternates between light and dark.
Each side illuminator creates a “glint” where its light bounces straight off the eyeball and into the camera. By tracking the glint locations on both sides, as well as the center of the pupil, the system can calculate eye orientations no matter where the head is or which way the face is turned.
My students at Parsons School of Design have modified the EyeWriter software so that it can control the cursor at the operating system level, not just in the eye-drawing application. Along with any button or other click input, this turns EyeWriter into a general-purpose mouse/trackpad replacement. EyeWriter 2.0 costs less than $150 to make, and the system holds its own against systems more than 50 times its price.
EyeWriter has won awards and has been well received, but it isn’t reaching nearly the number of people that it could benefit. I’ve heard from many who want an EyeWriter for someone they love, but for whom making one is still a high hurdle. So we have been collaborating with MakerBot Industries on a kit that would include 3D printed pieces and everything else you need to build an EyeWriter 2.0. We expect to make this kit available sometime in 2012.
The biggest challenge with the kit right now is sourcing the camera. We want one that’s higher grade than the PS3 Eye, has the same kind of VSYNC output pin, isn’t expensive, and doesn’t require cracking open a plastic case and throwing it away along with 4 unused microphones. Suggestions are welcome.
See blog.makezine.com/errata/make-volume-29-web-extras-and-downloads/ for how to build an EyeWriter 2.0.
ADVERTISEMENT
