Control Robots to Solve Puzzles in This Twitch-Style Robot Livestream

Robotics is a booming field that gets everyone excited, but when the talk comes to real life applications the response is a bit more tepid. Pundits tend to point to the “3 D’s”—dangerous, dirty and dull—but they often overlook a market with short term payoff: Entertainment.

We are beginning to see some development in this area. Jibo and Pepper are paving the way in social robotics. Robot Wars and BattleBots are coming back to our TVs and the MegaBots project has manage to capture people’s imagination, but it’s still a nascent industry. I was happy to stumble upon Let’s Robot, a project that is creating real life robots which can be controlled online. The robots have on-board cameras that stream video in real time to Twitch.

Using a chat system, anyone from anywhere can collaboratively control these robots. The robots can roam about different “sets” or levels. Users can try to find clues to puzzles or attempt to complete certain tasks. They are basically using robots as a gaming avatar.

I contacted Jillian Ogle, founder and creator of Let’s Robot, and she kindly agreed to chat with me. Jillian seems to be the type of renaissance person that navigates effortlessly between arts and technology. Her background is in graphic design and illustration, but later she transitioned to game design and development. She picked up coding along the way, because “if you can’t find an engineer, you have to do it yourself.” For this project she is knee deep in hardware: Designing, building, and programming robots.

This is quite an ambitious endeavor for an indie game shop. They are dealing with mechanics, electronics, video streaming, wifi networks, real time multi-player environments,and community management. Each one of those areas, a potential minefield by itself. But as Jillian says, “If it was easy, everybody will be doing it.”

Let’s Robot hosts streaming events a couple of times every week. Each event is embraced as an opportunity to test a new feature or improve their setup.

Their robot roster currently consists of a number of homebrew robots, from simple “Frankenbots” made of Lego bricks to custom 3D printed all terrain rovers. The robots have sensors and actuators controlled by a Parallax activity board and Arduino micro-controllers. A Rasberry Pi parses chat messages and forwards the on-board video to a desktop machine via WiFi. This machine runs Unity and combines the video feed with HUD graphics (and soon AR) to create a final stream that is captured by Open Broadcaster  and sent off to Twitch.

One of the venues they might pursue in the future, is to develop a “turn-key” system for people who want to run their own robot at home. Near term priorities though are to focus on the user experience (UI, latency, immersion, and collaboration), but hardware is hard and robots are needy! A lot of time goes into tending to their whims.

Let’s Robot is breaking new ground in gaming. There is a lot of potential in this concept and a lot of ideas left to explore. If you want to tag along this journey, follow Jillian on Twitter. She generously shares her experiences as she face her share of hurdles, all while having fun on Let’s Robot!