inForm: An Ephemeral Construction Machine

Disruptive or revolutionary technology are terms that get tossed around rather liberally. But one look at the MIT Media Lab’s inForm project and it’s hard not think you are looking at the future. Combining elements of Kinect, 3D printing, and telepresence, inForm heralds something very new and, I dare say, revolutionary. inForm was a collaboration between research assistant Sean Follmer and Daniel Leithinger at Heroshi Ishii’s Tangible Media Group in the MIT Media Lab. What exactly is it and how might it be used? Jay Silver interviewed Sean Follmer to learn more.–Stett Holbrook, MAKE senior editor

“It’s so open, you can just turn the camera to anything — it is just a blank canvas.” —Sean Follmer

Let’s start with what matters: Examples, examples, examples. In the world of *Tech (that is: technologies that interact with anything in the world), you had better show some broad examples of how a technology can be used in one way, then two ways, then three ways. Hopefully the three-plus ways are disparate enough that it starts to feel like a point, then a line, then a volume, then a near infinite space of possibilities. With inForm, moving a ball and using props shows us we’ve gone beyond the “control a robot hand” dead end of telepresence. When I see the examples in the video, I do get a sense that I could really improvise: give high fives, play a board game, perhaps apply acupressure.

Implementation is everything. Arguably inForm is an old idea, implemented correctly, which is to say it’s a new idea: new’ish Kinect hardware, combined with the zeitgeist of 3D printing/scanning, and correct techniques borrowed directly from roboticists. The hardware is ready. The world is ready. There’s an alchemy in any good implementation where the designer can’t quite predict the outcome until the implementation has begun, and the implementation gives new direction to the designer: iteration. Don’t let anyone tell you “That’s been done.” Nothing’s been done. Implement it a new way with a new design and it’s a new thing.

Now consider the device as a coupled 3D printer/scanner. As far as resolution and number of axes goes it’s terrible. It has 30×30 = 900 pixels and 1 freely continuous dimension (technically it is 7 bits, but it feels continuous). What about time scale though? inForm can “3D print” a temporary object 60 times per second. If a fast 3D printer can print a similar sized object in 5 hours, then inForm is about a million times faster than fast 3D printing!

I called Sean on the phone (yes the not Skype, the phone) and interviewed him. Here are some of his comments:

Jay: Comment on comparison with a 3D printer

Sean: In the limit, as 3D printers get faster there’s really no difference between a 3D printer and what people call programmable matter or shape displays.

Jay: How big is this puppy?

Sean: 15” x 15” x 4” formable space. 60 fps network speed.*

Jay: What’s the most important thing here?

Sean: This idea of being able to appropriate any object you want and to start to interact with that digitally, to interact with digital information in a very flexible way, to use anything to interact with digital information when we interact with touch screens, you use like one part of your finger. Trades all have special tools to do stuff. A sculptor has all of these tools that are of different sizes. You can use any part of your hand or your full body to interact with inForm.

Jay: Video demos can lie, tell the truth, or sometimes in the middle. For example, one of the most famous Media Lab demos ever, does some misleading in how it was presented. Is your video sped up at any point?

Sean: Nothing is sped up — it’s all shot directly. It all moves that fast.

Jay: Does the system work the way it’s represented in the video? What were the hardest and easiest shots to show working correctly? This will give us a sense of what works really solidly vs. what works better only in video.

Sean: Yes, everything really does work the way it is shown without doing several takes. The hardest take to get was with the finger tracking which was a little finicky, so the extrusion was picking up your arm sometimes. The easiest take was moving the objects.

Jay: That’s good news since moving the objects is the most powerful demonstration. How did you come up with the examples to show in the video?

Sean: We thought a lot about the examples for a while, and trying to find things that we thought would really communicate the abilities of the system, and also thinking about a future way of interacting with information. You have to use different examples for different audiences. For example, we had a paper at the ACM UIST conference. We used an example of a marble answering machine, based on Dural Bishop’s interface from the early 90’s for receiving phone messages with marbles representing the messages. Dural only made a video, but we made a demo. But that wasn’t a good example for showing the power of inForm to non-conference-goers. Moving a ball was.

Jay: Since you’re not the first to build something like this, why does this capture the imagination of so many people?

Sean: There is this thing called pin tooling used by mechanical engineers to do prototyping very quickly with vacuum forming. Then there are other people who have built shape displays before, Ivan Poupyrev has a project called Lumen when he was working at Sony, and other related work such as Wavehandling, so we’re not the first to do that, maybe not even the best. We hit a nice sweet spot in terms of the density of “pixels” and how fast each pixel can move. In other uses of shape displays, people haven’t considered as much how we interact with shape displays. They thought it was a way to render 3D models physically, but more important is how people interact with shape changing user interfaces now and in the future.

Jay: Last thoughts… I like to think of inForm as a type of ephemeral construction machine.

Sean: Hmmm. Construction that comes from your own body and objects you appropriate… For me inForm is about collaboration and communication more than anything else.

*Editor’s note: Sean says there are limitations of how fast the motors can move up and down because it can’t move the pins as fast as it can update where they should be, i.e. it takes time for the pins to get to the right space, and that time is dependent on how far they are from the target position.

Jay Silver is founder of Joylabz.com which produces MaKey MaKey. He also attended the MIT Media Lab, where he met fellow student Sean Follmer. For more *Tech by Sean Follmer see KidCad.