Take me to the Steps

LegoWeek_620x70_01jbr

Using Lego Mindstorms motors and blocks, the Raspberry Pi, and a BrickPi, we built our own digital book reader out of Legos that can digitize real paper books.

We wanted to develop a book digitizer that could read books aloud. We were fascinated by Google’s Google Books project, and thought “Why couldn’t we do this at home?”

In our first attempt, we tried a proof of concept where we built the BrickPi Bookreader to read a Kindle aloud.

We redesigned the Bookreader to read real paper books. In our example, we digitize a paper copy of Horowitz and Hill’s The Art of Electronics.

We read a book, with a robot, made of Lego.
We read a book, with a robot, made of Lego.

How It Works

The Bookreader flips through the pages of a book, taking pictures of each page, and then turns each picture into a text document.

First, the bookreader prepares a page to turn by rotating a Lego motor. Gravity keeps just enough friction on the book page to inch the page forward. Next, a Lego arm beam swings around, forcing the page over.

After a new page is turned, the Raspberry Pi camera takes an image of the new page in JPEG format, and saves it to file. Using open source Optical Character Recognition software on the Raspberry Pi, the BrickPi turns the page image into text.

Finally, the Pi saves the text. Now we’ve digitized the page, and you have the start of your book. Just for fun, in our example we use some free text-to-speech software, and the Raspberry Pi reads the book out loud over some speakers we attached. Once the page is read and stored, the Raspberry Pi, through the BrickPi, turns to the next page of the book.

Project Steps

Below, we have some step-by-step directions for building your own Bookreader with the Raspberry Pi and Lego Mindstorms.  All of the code and designs are in the Dexter Industries Github Repository.

Project Steps

Gather the Parts

In this project we use the EV3 system. However, the project can be done with the NXT system as well. You can also order the part set by using our Lego Digital Design Files: a full Bill of Materials can be produced with the software.

Build the Bookreader

Since we wanted to use the Raspberry Pi camera, the BrickPi and Raspberry Pi setup had to be placed above the book to get a good picture of the page. The structure that we ended up with looks a bit like a lean AT-AT walker from Star Wars.

The Camera is Placed Squarely Above the book. The Camera is Placed Squarely Above the book. 

In our example, we used the Lego Mindstorms EV3 kit.

LEGO Digital Design Files for the BrickPi Bookreader with Lego and Raspberry Pi

The trickiest part of setup lies with the motor connected to the wheel. The motor is not fixed to the platform, but can rotate freely from the point that it is connected to the platform. This is important because it keeps the pages pressed down so that the wheels can get a grip on the page. Another important thing to do is to rotate the wheel only for the time it takes for one page to come out of the wheel, then rotate the wheel back so that any extra pages that might have been pulled up are brought down.

A closer look at the page turning mechanism. A closer look at the page turning mechanism.

In the Dexter Industries Github Code Repository, there is a test code called arm⎽test.py to help you calibrate the arm for perfect movement.  Connect the motor to Port A of BrickPi and change the values of  ‘speed⎽roller’, ‘speed_arm, ‘t1’ and ’t2’, until you get flawless movement of your book pages.

Set Up the Camera

Raspberry Pi Camera mounted with Lego Raspberry Pi Camera mounted with Lego.

The Raspberry Pi camera packs a lot of punch: there are a lot of options; it’s easy to set up; and the image quality is acceptable for our project. We held it in place by pinching it between two Lego Technic beams in the previous section.

After connecting the camera, there is one more thing to do: change the focus of the Raspberry Pi camera. The camera comes with its focus fixed at infinity, and since it is a fixed focus camera, you have to manually change it.  Here are two helpful links to do focus the camera:

After setting up the camera, take a test image to see that it is properly focused.  In the rig we built, we have the camera about 10 inches above the book. (You may need to adjust the height depending on your book size.)

To take a test picture, simply run “raspistill -o image.jpg”

Take a few test images to check if the images are clear and the whole page is captured. If the camera is initialized properly you’ll see a new file image.jpg in your present folder. Open it to see the image.

You may need to readjust the focus of the camera and angle at this point.

Set Up the Text To Speech Software

First we’ll setup the Text-To-Speech engine (TTS). TTS takes text data and reads it aloud. In our project, we use eSpeak.  The voice is robotic, but using eSpeak, we can select a few accents and choose the gender of our reader.

First, test if the audio is working on the Raspberry Pi. Plug a headphone or speakers in the audio jack and run the following command:

aplay /usr/share/sounds/alsa/*

If you are able to hear the sounds, move to the next step. If not, this tutorial  may help you set up the audio.

Now, install eSpeak.  Run the following in terminal:

sudo apt-get install espeak

After it is successfully installed, run the following command (disregard any error messages on the terminal if you can hear sound):

espeak “hello”

If you are able to hear “hello” from the headphones or the speakers, move to the next step.

Setup the Optical Character Recognition (OCR) Engine

Optical Character Recognition is really cool; it can read a picture of text, and turn it into text on the computer. We chose Tesseract OCR Engine. It runs smoothly on the Pi, it does not require an online connection, and it reliably converts images to text.

First, install tesseract:

sudo apt-get install tesseract-ocr

Next, test the OCR engine. Take a good image of a piece of text and run tesseract:

tesseract image.jpg o

Image.jpg is the image which was taken by the raspberry pi camera and o is the file in which the text will be saved (tesseract will make it o.txt, so no need to add the extension).

Now, wait. The OCR takes a lot of processing power. When the Pi is done processing, open o.txt. If the OCR did not detect anything at all, you can try rotating the image and running the tesseract again.

Put it All Together: Digitize a Book

With the camera, software, and platform working it is now time to bring the Bookreader to life.

Digitize a book with the Raspberry Pi.

Download the program bookreader.py from the BrickPi GitHub Python Repo and put it in the directory where you have the BrickPi.py library. Run the code to bring it to life:

sudo bookreader.py

And start digitizing the worlds millions of books!

Like the Bookreader project? The BrickPi is an opensource hardware and software project. You can contribute to the GitHub repository for the BrickPi and make it better!