10,000 Bricks And An AI Comprise This Ultimate Lego Sorting Machine

Maker News
YouTube player


Ever since I saw Akiyuki’s Lego-sorting machine way back in 2011, I’d had the idea of building one as well. About 3.5 years ago I was in university for computer science, and started playing around with computer vision. About a year later, I started reading some articles on AI, and realized it was perfectly suited for the task.

My machine first separates the bricks into a stream, using belts and a vibrating table powered by a Lego motor spinning an offset weight. This part of the machine went through more iterations than any other component, and even then I’m not totally happy with the solution — sometimes parts will fall through more than one at a time.

The separated parts then move through a “lightbox” that records video for AI classification. Mechanically this is the simplest part of the machine — all the hard work is left to the AI.

The parts are then distributed into 18 output buckets, using a series of gates controlled by servo motors. I’ve never seen anything like this done before, and it means that the machine can be used for self-contained Lego storage.

The most difficult and complex part of the project is the software. A Raspberry Pi has to interpret individual frames of the scanner’s video to extract cropped images of the parts. Those images are sent to the AI convolutional neural network, which is trained to detect the part number out of almost 3,000 possible options. The neural network is trained using 25 million “synthetic” images of digitally rendered Lego bricks. To improve accuracy, the network is also trained on about 200,000 “real” images, which had to be manually labeled with their correct part numbers.

The video I’ve released has inspired lots of others to start working on their own machines, and I’m very excited to see how people will improve and refine the technology!

Discuss this article with the rest of the community on our Discord server!
Daniel West

is an AI and computer vision software engineer based in Wollongong, Australia.

View more articles by Daniel West


Maker Faire Bay Area 2023 - Mare Island, CA

Escape to an island of imagination + innovation as Maker Faire Bay Area returns for its 15th iteration!

Buy Tickets today! SAVE 15% and lock-in your preferred date(s).