Hacked iPhone API takes AR to the next level

Computers & Mobile

From ReadWriteWeb:

An international team of computer scientists has created software that lets anyone perform on-the-fly analysis of live streaming video on the iPhone. Used alongside existing methods of displaying data on top of the camera’s view, this new functionality signals a fundamental change in the kinds of Augmented Reality (AR) that iPhone developers can create. Existing AR apps, like Yelp, Layar, Wikitude and others display data on top of a camera’s view but don’t actually analyze what the camera sees. This new development changes that.

The video is pretty amazing. I can’t wait to see where this tech goes from here…

Devs Hack iPhone API for True Augmented Reality

12 thoughts on “Hacked iPhone API takes AR to the next level

  1. Toaste says:

    Very impressive demo in the video, it certainly raises the bar for what desktop AR can achieve.

    But I don’t see an iPhone anywhere in there. Video capture is through a webcam and processing is done on a laptop. Does the article mean they stream video to a desktop and apply AR there?

    The reason I’m skeptical is that the demo shows interaction that is impressive even for AR running on a laptop, but the article claims it’s for iPhone. While portable AR has awesome applications, chaining it to a desk is significantly less useful.

    Any modern laptop ought to be able to push 2 GFLOPS (2 billion floating-point operations per second) and effective use of SIMD instructions can push that a bit higher. Offload some computation to the GPU with CUDA can get you tens of GFLOPS even on a laptop GPU. The ARM processor in a 3GS may be impressively power-efficient, but you just aren’t going to get the same level of performance on computationally intense applications.

    Like physics simulation and video processing.

    1. Chris says:

      I’m with you toast, I don’t buy it. Someone’s a wannabe, unless you can let me get my hands on it

  2. Gareth Branwyn says:

    If you click through the links, you’ll see that they are offering the source code to the public, so feel free to get your hands on it.

  3. Anonymous says:

    I kept seeing that Ah-ha video…

  4. Tim says:

    … has anyone thought of a useful use of AR yet? Layar is just a gimmicky map, this is a gimmicky physics simulation (the sketch analysis doesn’t really have anything to do with AR), etc.

    The only thing I’ve seen where AR vaguely adds to things was that tabletop zombie game.

  5. Free iPhone 3GS says:

    Wow, that’s impressive, but I don’t see how that’s done on an iPhone?

Comments are closed.

Discuss this article with the rest of the community on our Discord server!

Gareth Branwyn is a freelance writer and the former Editorial Director of Maker Media. He is the author or editor of over a dozen books on technology, DIY, and geek culture. He is currently a contributor to Boing Boing, Wink Books, and Wink Fun. His free weekly-ish maker tips newsletter can be found at garstipsandtools.com.

View more articles by Gareth Branwyn
Maker Faire Bay Area 2023 - Mare Island, CA

Escape to an island of imagination + innovation as Maker Faire Bay Area returns for its 15th iteration!

Buy Tickets today! SAVE 15% and lock-in your preferred date(s).