LEGO Sorting Machines - an Overview
Probably the best-known LEGO sorter is Daniel West's "WORLD'S FIRST Universal LEGO Sorting Machine" (Dec, 2019). According to Daniel, his work is inspired by Akiyuki's "LEGO Mindstorms NXT Vision Guided Brick Sorter" (March, 2011) and Jacques Mattheij's Sorting two metric tons of lego (April, 2017). Focusing on hardware, all three machines show a similar four stage design:
Bulk storage → Part separation → Scanner → Sorter
The following overview, will look at a larger field of machines and show the creative variety in designing each stage.
Stage 1 - Bulk Storage
In bulk storage you have a large amount of unsorted, entangled bricks and the goal is to feed it to the machine progressively. There are two typical designs used for bulk brick storage. Nr. 1 is what I call the "belt slowly moving under pile of bricks" type. Two nice examples can be seen with Daniel's and Jacques Mattheij's machines.
While in Daniel's Design, we have a narrow bulk storage where the full width of the ground is transporting bricks, Jacques's machine features small 1x4 tiles being glued to the belt. As a result only the small middle part is feeding a small amount of bricks to the machine. If you want to learn more about these two machines, both Daniel (here and here) and Jacques published additional background information (Part 1, Part 2 and Part 3).
The bulk storage design Nr. 2 is based on a "step feeder", where a step is moving vertically through the storage pile and feeds a small portion of bricks into the machine. Akiyuki's, Francisco Garcia's and Johann Rocholl's machines employ this design idea.
From Akiyuki's LEGO Mindstorms NXT Vision Guided Brick Sorter ver1.
From Francisco Garcia's Lego Sorter with TensorFlow on Raspberry Pi (First Run).
From Johann Rocholl's Brick feeder prototype.
Stage 2 - Part separation
The task of part separation is to take the entangled stream of bricks and separate it into individual parts which can be passed on to the scanner strictly one by one at a time. Separation is especially challenging with LEGOs, since the parts have a huge variety in size (e.g. 1x1 plates stuck inside a window) and also friction (e.g. bricks vs. rubber wheels). On the machines I saw so far, part separation is built up from one of the following tools.
Chicaning Conveyors are belts with diagonal barriers which force the parts on the conveyor to align sequentially. The delicious video below shows the chicanes being used in the food industry.
From Dorner Conveyors: Chocolate Chicaning Conveyor.
Here are nice examples of chicaning conveyors in Lego sorting machines.
From Francisco Garcia's Lego Sorter with TensorFlow on Raspberry Pi (First Run).
Two-speed belt steps are another possible component to achieve part separation. They consist of a slow moving conveyor belt, which drops parts onto a second, much faster moving belt. This feature takes parts which are already sequentially aligned and spreads them out longitudinally.
From Akiyuki's LEGO Mindstorms NXT Vision Guided Brick Sorter ver1.
From Francisco Garcia's Lego Sorter with TensorFlow on Raspberry Pi (First Run).
From Jacques Mattheij's Sorting two metric tons of lego.
From Johann Rocholl's Brick feeder prototype.
From Peter V's Lego Sorting machine close-up.
Also combinations of these designs are possible. The LegoLAS system by Jörn Schlingensiepen runs two vibration feeders at different slopes and thus effectively different transport speeds. This combines the sequential alignment of the vibration feeder with the longitudinal spreading of the belt step.
After part separation, the bricks are transported one by one into the scanner. The scanner's task is to recognize/classify parts or part categories. Most scanners rely on visual recognition via camera images. There are different approaches in how camera based scanners are designed. Driving motivation is to capture parts from multiple perspectives because sometimes parts are hard to distinguish from certain angles. Also strong, non-flickering lighting, which is decoupled from the environment is important. It allows cameras to run on very short exposure times to minimize motion blur.
Multiple Poses - Daniel's machine achieves capturing of multiple perspectives by letting the parts run towards the camera. The angle of observation and the scale of the part thus changes slightly while the parts are approaching. A specialty of this setup is, that the classifier needs to become scale invariant. It thus can't use absolute image dimensions as a classification cue.
Multiple Cameras - Gijs van Haeff's sorting machine instead uses multiple cameras to capture multiple perspectives simultaneously.
From Gijs van Haeff's Universal LEGO Sorting Machine.
Mirror - Johann Rocholl's sorter features a single camera and a mirror to capture two perspectives at once. This design reduces complexity, ensures synchronization between the captured images and greatly reduces bus/CPU load compared to grabbing multiple cameras.
Smart Phone - Spencer Hunber's Nexus sorting machine is using a smartphone for high quality camera and potential CNN accelerator hardware.
From Spencer Hubert's Nexus
Scale - Part weights are a great complementary feature to standard visual input for classification. Akiyuki fitted a digital scale to his Lego sorting machine. This can be achieved using either a USB scale or via a camera looking at the display of the scale and doing basic character recognition. However, the classifier needs to be adapted to accept multimodal input. A mechanism is also needed for pushing parts onto and off the scale. The weight feature could also be useful for detecting multiple parts erroneously presented to the camera at the same time. By combining visual input with part weights, sorting machines potentially can achieve higher classification accuracy.
Stage 4 - Sorter
After the correct part type has been identified inside the scanner. The part can be directed to its correct storage location. This can be achieved using one of the following approaches.
Horizontal Gates - Usually a conveyor belt with mechanical arms which direct the part into the intended bucket. The two examples below don't just guide the part but even actively push the parts into the bucket while closing. This allows parts to follow each other more closely on the belt.
Frim Gijs van Haeff's Universal LEGO Sorting Machine
From NealAnthoons's Lego Sorting Machine
Vertical Gates turn the principle around and replace the conveyor with a vertical channel through which the parts are falling. A gate directly above the target bin then directs them to the right place. The speed of the falling part is high thus allowing parts to be sorted quickly.
Pneumatic Sorters use a short, directed blast of air to push the parts off the belt and into the storage bin. They feature very little moving parts and typically use a solenoid valve which can be triggered from GPIO ports. Timing is crucial in order for the parts not to fly sideways into the wrong bin. A belt position encoder (e.g. optically via the wiggly line in Jacques's design) helps measuring the exact belt and thus part position. A belt with lateral cleats (see Eppos Vision System below) helps to keep the parts on course when they fly with the air stream. Jacques Mattheij's machine apparently even adjusts the dose of air based on the respective part size. Finally it is very helpful to use a silent air compressor, since normal ones are quite loud for long term usage. Apart from the compressor noise, personally I think pneumatic sorters at work are super relaxing to watch 😉
From Peter V's The Shape Sifter - First working demonstration
From Eppos Vision System's Automatic LEGO sorting machine
Rotary Slides again use mechanical servos but instead of just binarily opening or closing a gate they choose one out of many storage locations by pointing a slide towards the target. This design is very efficient in the number of servos required. In theory multiple such slides could even be cascaded. The slides are also quicker in handling the part, thus allowing the parts to be processed in rapid succession.
From Francisco Garcia's Lego Sorter with TensorFlow on Raspberry Pi (First Run).
From Claus Christiansen's Automated AI LEGO Sorting machine part 2
From Michael Talarczyk's LEGO Sortierer (Color Sorter ) mit raspberry pi
Picking Robots are the final variant of sorting systems. A robotic arm is grabbing the parts, moving over a 2D set of storage locations and dropping it into the respective bin. While such part handling takes more time, the two-dimensional storage matrix allows for an extremely large amount of bins.
From Tampere University Robotics' LEGO sorting robotwith machine learning
This concludes my overview of the machines I founds so far. If I missed one, please leave a comment - always happy to discover new machines.
Comments
Post a Comment