Fully Autonomous AI Powered Sorting Machine | Using Artificial Intelligence to Complete Real-World Tasks

Hello, my name is Caleb, and this project was created for the Butwin Elias Science and Technology Award (BEST Award). Over the past few months, I designed and built a fully autonomous AI-powered sorting machine capable of identifying and sorting real-world objects using computer vision, artificial intelligence, and custom robotics.

Coming up with an idea for this project was tricky. I had a couple of neat project ideas come to my mind.

However, I soon realized that, besides being cool for a couple of weeks, building an RC vehicle or making a video game was nothing special. These things already exist, and you can find guides to build them everywhere. I wanted to make something original.

And then it hit me: AI.

Artificial Intelligence is part of the future, like it or not, and there is no denying it. But instead of getting intimidated, what if I could harness the capabilities of AI and use it to actually solve a real-world issue, or automate a task?

This caught my attention. I had a good amount of knowledge of code already, so I told myself I should be able to do it. However, I still needed an actual project idea, and after some brainstorming, I decided. I was going to build an AI-powered sorting machine that would be fully automated and operate on its own.

The goal of this project was to combine multiple areas of engineering and technology into one complete working system. The machine uses a Raspberry Pi paired with the Raspberry Pi AI HAT+ to run real-time object detection through YOLOv8. Once an object is identified through the camera feed, a fully custom robotic arm reacts automatically using a geared stepper motor and servo-driven claw system to sort the object into the correct category.

Almost every part of this build was custom-made. I designed the robotic arm and housing in CAD, 3D printed the structural components, built the electronic control system, configured the AI acceleration hardware, and programmed the full detection and movement code in Python. The project required a combination of mechanical design, electronics, computer programming, AI integration, and a large amount of troubleshooting and testing.

One of the biggest challenges of this project was that there were very few existing guides or tutorials for building something exactly like this using the hardware I chose. A large amount of the development process involved experimentation, debugging, and figuring out solutions through trial and error. Getting the AI acceleration hardware and object detection pipeline working reliably on the Raspberry Pi took several weeks of testing and optimization, and the entire project took me around 5 months to complete.

This project taught me a huge amount about robotics, artificial intelligence, Linux systems, CAD design, hardware integration, and problem-solving. More importantly, it showed me how powerful modern AI tools can become when combined with physical hardware and automation. Because of this project, I have now grown passionate about AI, software engineering, and robotics.

In this Instructable, I will walk through the entire design and build process, including:

1. Materials
2. Circuit design
3. AI setup
4. CAD modeling
5. 3D printing
6. Wiring
7. Programming
8. Testing and calibration

By the end of this Instructable, you will see how all of these different systems come together to create a fully autonomous AI sorting machine.

I hope you enjoy this project as much as I do.

Let's begin.

While the total cost is significant, much of the hardware (Pi, tools, and development equipment) is reusable across future projects. Some components can also be substituted for different brands or different models.

Core Computing Parts

1. Raspberry Pi 5 4 GB - This is the brain of the entire setup. It runs the operating system, handles object detection, controls the motors, and manages communication for remote access.
2. Price ~$85 - $110 You can buy one here. (Please note, as of early 2026, Ram prices have absolutely skyrocketed. Fortunately, I purchased this Raspberry Pi right before the bad part of the leap. That said, there are cheaper alternatives available. Ram is probably not going down anytime soon, so devices like the Raspberry Pi 5 2 GB or even earlier Pi models may be a good alternative.)
3. Micro SD Card 64 GB - Stores the Raspberry Pi OS, dependencies, and all project files. Acts as the system’s primary storage.
4. Note: If you do not already have one, make sure to get a micro SD-to-USB adapter. This will be used to "flash" the micro SD on a separate computer before putting it into the Raspberry Pi.
5. Price $16 Buy here.
6. Raspberry Pi AI HAT+ - A dedicated AI acceleration module that takes the heavy workload of running neural networks off the Pi’s main CPU. This made a huge difference in keeping detection fast enough for real-time use.
7. Price $70 I purchased mine from here.
8. Cooling Fan - Keeps the Pi from throttling under sustained load, especially when the AI pipeline is running continuously.
9. Price $12 Buy here.
10. 5V Power Cord - This cord will be used for powering the Raspberry Pi.
11. Price $12 Buy here.
12. Brass Standoffs - Used to physically separate and mount the Pi and AI HAT so everything stays aligned and has airflow.
13. Price $10 Buy here.
14. 40 Pin GPIO Pin Extenders - Extends the GPIO pins so they remain accessible even with the AI HAT installed.
15. Price $14 Buy here.
16. Micro HDMI to HDMI Cable - Used to connect the Raspberry Pi to a screen.
17. Price $9 Buy here.
18. Keyboard & Mouse - You will need a keyboard and mouse to connect to the Raspberry Pi.
19. Monitor - You do need a monitor or some sort of screen to connect to the Raspberry Pi. For this project, I did most of my coding on a 10-inch portable screen that I got for free. If you do not have access to any computer monitors, you can also connect the Pi to a TV via HDMI.

Vision

1. Raspberry Pi Camera Module 3 - The main sensor for object detection. This feeds live video into the AI model.
2. Price $40 Buy here
3. Ribbon Cable - The camera for the Raspberry Pi connects via a thin ribbon cable. Since the machine was quite big, I bought a 500mm long one.
4. Price $8 Buy here

Motors

1. Nema 17 Geared 5:18:1 Stepper Motor - Drives the base rotation of the arm. The gearbox was a necessary upgrade from a standard stepper — the added torque makes the system reliable under load.
2. Price $40 Buy here.
3. MG996R Metal Gear Servo Motor - Controls the claw. I chose this specifically because plastic-geared servos tend to wear out or slip under constant gripping force.
4. Price $14 Buy here.
5. DRV8825 Stepper Motor Driver - Converts control signals from the Pi into the higher-power signals needed to drive the stepper motor.
6. Price $14 (5 Pack, best value) Buy here.
7. DRV8825 Stepper Motor Driver Expansion Board - This is basically just the harness for the DRV8825 driver to sit in. It keeps things clean and helps remove a lot of unnecessary wiring.
8. Price $10 (2 Pack) Buy here
9. 12V Power Supply Cord with Adaptable Heads - Dedicated supply for the stepper, so it isn’t pulling power from the Pi.
10. Price $15 Buy here.
11. 5V Power Supply Cord With DC Head - Keeps the servo isolated from the Pi’s power rail to avoid brownouts and jitter.
12. Price $7 Buy here.

Fabrication

1. Access to 3D Printer - The majority of the sorting machine is 3D printed using a Bambu Lab P1S. The base and entire arm were printed using that printer.
2. Filaments Used
3. PLA Basic - Used for lightweight non-load-bearing parts like covers and camera mounts.
4. PLA Metal - Used for the claw components where strength and wear resistance matter.
5. PETG HF - Used for the main structural frame due to its strength and better heat resistance.
6. Foam Tape - I used a cheap roll of foam tape from Amazon to give the claw some extra grip and tolerance to objects that may vary in size.
7. Price $15 Buy here

Electrical

1. 100µF 35V Capacitor - In this project, I used a capacitor to help prevent voltage spikes in the servo motor and to assist in adjusting the voltage of the motor driver. I recommend getting a 100 μF 35V capacitor to keep things running smoothly.
2. Price $6 Buy here.
3. Jumper Wires - Used throughout the system for power, control signals, and GPIO connections. If you choose to buy from a different link than the one listed below, make sure to buy an assortment of male-male, female-female, and male-female.
4. Price $7 Buy here.
5. Multimeter - Critical for setting the DRV8825 voltage correctly. I ended up relying on this a lot after burning out drivers early on.
6. Price $18 I borrowed one to use for this project, but you can find a cheap one here.
7. Breadboard - I decided to use a breadboard to not only initially test the circuits, but also to adjust the voltage on the DRV8825 stepper motor driver.
8. Price $7 Buy here.

Tools

1. Screwdrivers - I only had to use a single small Philips head screwdriver for the entire project.
2. Electrical tape - Thin strips of electrical tape, or really any tape in general, are helpful for making sure that connected jumper wires stay connected.

Optional -

1. Soldering Iron - Along with a roll of solder, I borrowed a soldering iron for this project for connecting wires and soldering some things together. However, soldering is not 100% necessary for this project to work.

In total, I ended up spending roughly $440 on this project, though that includes many trial-and-error purchases and replacements.

With better sourcing and fewer mistakes, it could realistically be built closer to $250–$300, especially if you already have basic tools or a Raspberry Pi setup.

Before any of the AI or control systems come into play, the Raspberry Pi needs a stable thermal setup. The AI workloads in this project run in a continuous loop, so the Pi can generate quite a bit of heat.

Fans for the Raspberry Pi are all pretty similar, so the directions are transferable even if you decide to get a different one than I purchased.

The cooling fan mounts on top of the Raspberry Pi using the included adhesive pads and screws. The adhesive pads help transfer heat into the heatsink, and the fan pulls that heat away from the system once it builds up.

To install it, first peel and place the thermal pads onto the heatsink side of the fan assembly. Then line everything up with the mounting holes on the Raspberry Pi. It’s worth making sure the alignment is right before tightening anything, because once the screws are in, it should sit flat and not shift at all.

After it’s physically secured, plug the fan into the dedicated fan header on the board (see third image). This header lets the Pi control fan speed automatically based on temperature. The fan will typically stay turned off under normal usage, like writing code or installing packages, but will kick in when the CPU gets around 50-60°C and begin cooling.

It’s a simple part, but once everything is running at once (camera, AI, motors), it ends up being pretty important.

Now that the cooling fan is on, we want to add the standoffs and pin extenders to prepare for installing the AI HAT.

The AI HAT does come with nylon standoffs as well as a pin extender, but as I found out the hard way, the standoffs do not give the fan enough room to breathe, and the pin extenders are not long enough to connect wires to.

Fortunately, this step is quick and simple.

To attach the GPIO Pin extenders, just slide them over the GPIO pins on the Raspberry Pi. I have attached a photo of the two extenders I used from my kit.

Also, make sure you are using the correct extenders. Test the height beforehand to make sure it will poke through the AI HAT. When I was doing this, I put an extender on that wasn't long enough - twice. They are an absolute pain to remove, as you have to do it slowly or the pins will bend and be rendered useless.

With the pin extenders on, the standoffs can now be attached.

Refer to the image above showing which standoffs I used.

Unlike the pins, these screw on and off very easily, so you can always switch them around if you would like. I decided to put four short standoffs under the Pi to lock it into the machine later and allow more airflow. I then decided on the longer standoffs for the top as the AI HAT can sit on them without having to hang from the GPIO pins.

Admittedly, this combination of the right size of pin extenders and standoffs took a lot of trial and error. It was a lot of guessing and checking, but I think this is the most optimal setup for height.

Now that the setup is ready, we can attach the AI HAT.

Installation is simple as the HAT just slides on over the GPIO pins. Once each corner of the hat is lined up with the brass standoffs, you can use four screws from the kit to secure the hat on. Refer to the above images for reference.

Now that the hat is screwed on, you may have noticed the small orange ribbon cable on the end of it. This is called the PCIe port. We will plug this cable right into the port right under it on the Raspberry Pi. The connector must be unlocked by lifting the retaining latch, after which the ribbon cable is inserted fully, and the latch is pressed back down to secure the connection.

First, grab each end of the connector - there should be a small flap that you can grip with the tip of your finger. Lift it gently, and you should feel it open. This part of the clasp should now be slightly loose. Now insert the orange cable into the slot, making sure it is all the way down. Once seated, push the clasp back down. Look at the images above if you are confused about this step.

Now that the HAT is installed, it helps to explain what it actually does in the system.

So what is the Raspberry Pi AI HAT+?

To understand it, it helps to think of the Raspberry Pi as the main controller of the system. It runs code, communicates with hardware, handles motor control, and manages everything happening at once. This is a lot for the Pi to handle at once.

Running real-time object detection on top of that doesn't help anything either. The Raspberry Pi is still able to run the AI object detection, but it can only get around 1-3 frames per second.

During early development, I spent a significant amount of time just getting basic object detection working on the Pi. There isn’t much documentation at all for this exact kind of setup, so most of it came down to trial and error, rather than following a guide.

The first "functional" result I reached was with YOLOv4-tiny, a lightweight object detection model. It worked, and the system could correctly identify simple objects like apples and oranges through the camera. However, it made mistakes very often and was unreliable.

I then moved to YOLOv8, which improved detection quality significantly. It took some setup work, but the AI was running just fine with no accuracy issues.

The remaining issue was performance. Despite my attempts at optimizing it, inference speed stayed at 1-3 FPS.

The Raspberry Pi was simultaneously handling object detection, motor control, and background system tasks. With only 4GB of RAM, the AI was of little priority and was not getting the processing time that it needed to run smoothly.

After some research, I discovered the Raspberry Pi AI HAT+ 13 Tops.

For the AI object recognition to work, there has to be a lot of math.

The AI HAT+ acts as a dedicated accelerator for neural network inference. Instead of the Pi handling all computations directly, the heavy math operations are offloaded to the accelerator to handle, leaving the Raspberry Pi free to do other stuff (like control motors or run code).

If the Raspberry Pi is the main controller, the AI HAT+ is like a dedicated compute unit for AI tasks.

The module is rated at 13 TOPS, which stands for trillions of operations per second. This means the extremely repetitive mathematical work required for object detection can be handled independently, freeing the CPU.

Attaching the camera to the Raspberry Pi is similar to attaching the PCIe cable on the AI HAT.

To start, locate the small CSI camera port on the Raspberry Pi, usually near the HDMI ports.

Gently lift the black locking tab on the connector.

Take the ribbon cable and insert it into the port with the metal contacts facing the Ethernet port. Make sure the cable is fully seated and straight, then press the locking tab back down to secure it.

Finally, connect the other end of the ribbon cable to the camera module the same way. Make sure that the side of the ribbon cable with the metal contacts is facing toward the front of the camera. Refer to any of the above pictures for reference.

Note: If you are concerned about the camera not having a proper stand, all the 3D printing will take place later in this Instructable. We do not necessarily need to use the camera that much now, so the stand is not needed yet.

By this point, the Pi should be fully assembled (except for the micro SD). This means we are ready to start.

Despite its similarity to a computer, the Raspberry Pi isn't like a plug-and-play Windows laptop. It does not come with built-in storage, which is why we need to use a micro SD card.

We will be installing Raspberry Pi OS, an operating system for the Raspberry Pi. It's pretty much Linux with some Pi-specific features, which makes it great for coding and developing.

I found the Pi OS to be a lot more rewarding and engaging than typical operating systems or other microcontroller interfaces, not because of clean features, but rather the opposite. Coding on the Raspberry Pi is so much better simply because of everything it doesn't have. It's not some flashy desktop interface pumped with clean features and widgets, but rather just the essentials. It is extremely customizable, you have complete control of your device, and everything is done in the terminal, meaning you get to write "code" for everything. After about three months of use, I have become pretty fluent in terminal commands and have gotten a lot better at fixing issues that arise.

All you need for this is your micro SD with the USB adapter (if needed), and any computer.

Insert your micro SD card into the adapter and then plug it into your computer.

Head over to the Raspberry Pi official website.

Go to the top of the website and click on "Software."

Now you can download Raspberry Pi Imager for Windows, macOS, or Linux. Depending on what computer you are using, click on one of those options. I currently use Windows on my desktop, so I selected Windows.

Once the file is downloaded, open it.

From here, you can refer to the above images.

A window will open asking you to "select your Raspberry Pi device." We are using a Raspberry Pi 5, so click on that. Hit next.

You will then be asked to "choose an operating system." Select Raspberry Pi OS (64-bit). This one is the fastest and best for our use case.

After this, you will be asked to "select your storage device." Your micro SD card should already be detected, but if you don't see it, you can open your files.

Once all this is finished, you will be asked to fill out some personal information, such as creating a username and password, connecting to your Wi-Fi, and, if you want, enabling Raspberry Pi Connect, which lets you screen share your Pi with another computer over Wi-Fi. We won't be using it for this project, but if you think that it is useful, feel free to set it up.

After this, you've made it to the end. You will be asked to "write image." Click Write. Keep in mind that doing this will wipe the micro SD card to install the operating system. Make sure you don't have anything important on it.

After selecting Write, you will have to wait a couple of minutes for everything to install and verify. Once everything is done, you can click on "Finish."

Now that the Pi is fully set up and we have the operating system downloaded, you can go ahead and do the following:

1. Insert the micro SD card into the slot underneath the Raspberry Pi.
2. Connect a keyboard and mouse to the Raspberry Pi via the USB ports.
3. Connect the Raspberry Pi to a monitor.
4. Plug in the Raspberry Pi using the dedicated 5V power supply.

Once the power turns on, you should begin to see the boot screen on your screen. It took a little longer to boot up for the first time for me, but once it is done, you should be greeted with what more or less looks like a standard desktop home screen.

If not already done during the OS installation, make sure you are connected to Wi-Fi. You will see the Wi-Fi logo at the top right of the screen.

Now that we are in, let's navigate to the terminal, where we will be spending the entire software portion of this project. The terminal will be located on the top left of your screen. It is a small black box with a >_ in it. Click on the icon, and the terminal will open up.

At this point, you should see a black window open in the middle of your screen with your username + device name + ~S. This is the command line. This is where we will be updating systems, installing packages, and performing countless other tasks.

However, our first order of business is to update everything. In the terminal type (or paste) the following line.

Sudo apt update will check which updates are available, and sudo apt upgrade -y installs them. Depending on the last time the Pi was updated, this could take anywhere from 30 seconds to 15 minutes, so go grab a snack.

Once the Pi finishes updating, type:

It is good practice to reboot after updates to make sure everything is installed and being used.

Once the Pi turns back on, we can begin our next step.

This was one of the more difficult parts of the project, both in terms of setup and troubleshooting. Getting a working AI pipeline on the Raspberry Pi alone took about two weeks, followed by another few weeks of configuring the AI HAT software stack.

This wasn’t a straightforward process with clear documentation. There are very few complete references for this exact combination of hardware and real-time object detection, so it pretty much came down to testing what I know, seeing what I could learn, and just trying to fix issues as they come.

Before the AI HAT was installed, I initially got a lightweight model running on the Pi that could perform basic detection, but accuracy was inconsistent and unreliable in practice. From there, I worked through progressively better models, eventually reaching YOLOv8.

When I installed the Hat, I assumed it would just accelerate my code that I already had. I couldn't have been more wrong. There is a lot of setup that needs to be done with this thing.

Eventually, I was able to get a stable setup working, which is what the next steps cover.

Step 1 - Enable PCIe Gen 3 Speed

Once your Raspberry Pi is booted back up from the last step, reopen the terminal and type:

This will bring you to the configuration menu. Using the arrow keys on your keyboard and enter to select, navigate to:

Advanced Options → PCIe Speed → Yes

Hit Enter to confirm that you would like to switch to Gen 3 speed and reboot when prompted. Similar to earlier, rebooting just ensures that the changes made can actually happen.

Note: Do not attempt to add any third-party Hailo repositories manually. Everything needed is available through the Raspberry Pi apt repository. Any instructions telling you to add a custom GPG key or repository for Hailo are outdated or incorrect and will waste your time. Trust me.

Step 2 - Install the Official Hailo Packages

Once the Pi boots back up, go back to the terminal and type:

This installs everything, including the driver, firmware, runtime libraries, and Python bindings. It will take 10-20 minutes.

Step 3 - Install DKMS and kernel headers

DKMS stands for Dynamic Kernel Module Support. It is a framework that automatically recompiles kernel modules when the OS is updated. This allows the hardware to continue functioning without needing to be fixed. Installing these ensures that the driver gets compiled for your specific kernel. Paste the following lines:

Once those are done installing, you can - surprise, reboot!

After the reboot, you can verify if Raspberry Pi is now detecting the Hat. Type:

If everything worked, you should see a line come back containing:

If you do not see this, it is because the PCIe ribbon cable is not fully seated. Power down and reseat it before continuing.

Step 4 - Load the Driver

Now that we have the driver installed, let's activate it. Paste:

When you press Enter, it should jump to the next command without doing anything. This means the driver was loaded with no errors.

Step 5 - Verify the Firmware

Just as an extra check, paste:

You should see an output like:

If there is no output and it jumps to the next line, it means the driver was not compiled for your kernel, and you will have to redo Step 3. This happened to me multiple times, but not because this code is flawed; it was because I was trying different methods and figuring things out. Safe to say, if you followed the steps correctly up to here, you should not have an error.

Step 6 - Set Up the Python Environment

Paste the following lines into the terminal:

This will clone the Hailo examples repository and make a folder. It could take anywhere from 5 to 15 minutes, depending on your internet speed.

Once that completes, run the installer.

This will take 10 to 20 minutes. It creates a virtual environment*, installs all required Python packages including GStreamer bindings, and downloads the model files.

*A virtual environment is an isolated folder that contains its own installations and Python libraries. It allows you to install project-specific dependencies without interfering with the entire Raspberry Pi system. We must use a virtual environment in the project, as some installations and code we will be running can interfere with outside systems. It is a good habit to use virtual environments for different projects, not just because of the safety, but also for organization.

Possible Issues That I Faced.

1. If at any point you see some red lines of code appear during installation, don't worry. It does not matter, and the script will continue attempting to reinstall.
2. If the script appears to freeze during a step that mentions cloning from GitHub, leave it running. It can take a while on a slow connection.
3. If the script fails before finishing, simply run bash install.sh again. It is safe to re-run.

Once this finishes, we will want to activate the virtual environment that we created. Type:

Once you type that command, your terminal prompt should now have (venv_hailo_rpi_examples) at the end. The first time I did this, I received an error saying the virtual environment was not found. If this happens, it means there was an error in the install, and you have to run bash install.sh again.

Step 7 - Fix the Post-Processing Library Path

This was for sure an interesting obstacle for me.

Long story short, the detection pipeline we installed is told to look for a specific library file in a certain location. Turns out, the file does not install in the location that the pipeline expects.

To fix this, you set up a new file directory (ridiculously long command) to fix the library path and then download the example resources now that you are in the right place.

Anyways, let's redirect this file.

Paste the following:

Once you have the new file directory set up, you can download the example resources.

Step 8 - Test the AI Detection

Before we test anything, reboot the Pi one last time.

Once it powers back on, open the terminal and paste the following two lines.

The first line opens the folder we will be doing this project in, and the second line opens the virtual environment.

Now we can test the object detection. Paste:

Running this will open a camera window with the object detection running. I have to say, when I saw this for the first time, it was impressive.

In the early stage of this project, when I downloaded YOLOv4, I played around with it for probably upwards of an hour, despite its 3 frames per second, terrible accuracy, and inverted colors. One of the above pictures in this step shows the different AI phases I went through in this project, each one better than the last. The difference between the beginning and now is crazy to me, and I'm glad I didn't settle for "good enough" the first time I got things working.

When this project is fully built, you probably don't want to plug a keyboard and mouse into the Raspberry Pi every time you want to run the code. It is messy, inconvenient, and defeats the purpose of the machine being an autonomous tool. Instead, we will be using Termius, a free app that lets you connect to devices via SSH. Eventually, we will be able to plug our machine in, open our phone, and type "sortingmachine," and the code will run.

Step 1 - Install Termius

The first step is actually installing Termius on your phone (or tablet). I have an iPhone, so I just searched it up in the App Store and downloaded it.

Step 2 - Enable SSH on Raspberry Pi

Termius connects to device terminals via SSH or secure shell. You can think of SSH as a very secure, encrypted tunnel that connects your phone to the Pi. For this tunnel to be active, we need to open the door on one side.

Turn on the Raspberry Pi and open the configuration menu like we did earlier.

Once in the menu, use your arrow keys to scroll down to Interface Options and press Enter.

Select SSH and press Enter.

It will ask, "Would you like the SSH server to be enabled?" Select Yes.

Select Ok, then Finish to exit.

Now we need to install Tmux, a terminal multiplexer that lets you manage multiple command-line sessions from a single window. Without this installed, Termius would not work.

Type:

Step 3 - Create Host

Now that SSH is enabled on the Pi, we need to find its IP address. In the terminal, paste the following line:

This command will print your IP. Write this down on a sticky note and head over to the Termius app on your phone. When you first open the app, you will be asked to sign in, create a username, etc. Once that is complete, go to the top right of the screen, tap on the plus ( + ) icon, and select New Host.

A menu will open with settings for the new connection. All you need to do is make a name for the connection, enter the IP address that the Raspberry Pi gave, and make sure that you have Use SSH selected.

Tap save, and you now have a way to access the Pi from your phone.

Before building the circuit, the DRV8825 stepper motor driver needs to be configured with the correct voltage limit. This prevents the driver and motor from overheating and ensures the motor runs reliably.

In my original testing with the circuit, I fried 4 different drivers. Adjusting the current is crucial.

You may be wondering why a stepper motor driver is needed in the first place, when we will be controlling the servo straight from the Raspberry Pi.

A Raspberry Pi cannot directly power or control a stepper motor on its own. The DRV8825 handles the higher power requirements of the motor and interprets control signals from the Pi. It is basically the brain for the stepper motor.

For this step, you are going to need the following materials listed earlier in this Instructable.

1. Raspberry Pi
2. Jumper wires
3. Breadboard
4. Capacitor
5. DRV8825 stepper motor driver with heatsink
6. 12V power supply with DC barrel jack screw terminal adapter (the green tip with two screws)
7. 5V power supply for Raspberry Pi
8. Multimeter (voltage reader)
9. Small screwdriver

Start by attaching the heatsink to the DRV8825. It has adhesive on the back and should be placed on top of the large black chip. The motor we will be using can use a lot of energy, which in turn can heat the chip. A heatsink will go a long way.

Next, assemble the circuit according to the wiring diagram that I made above.

1. Insert the DRV8825 into the center gap of the breadboard.
2. Connect two jumper wires to the screw terminal adapter for the 12V power supply.
3. Use jumper wires to connect the Raspberry Pi GPIO pins to the appropriate rows on the breadboard.
4. Insert the capacitor in the correct orientation.

A few important details before powering anything on:

1. The RESET and SLEEP pins on the DRV8825 must be connected. In the wiring diagram, there is a small jumper wire between those two pins. Forgetting this connection is one of the most common reasons the driver will not function properly.
2. Make sure the capacitor is installed correctly. The striped side of the capacitor is the negative side and must connect to GND. *IF YOU ORIENT THE CAPACITOR WRONG, IT CAN EXPLODE.*

Once the circuit is assembled, leave the power disconnected for now while calculating the driver’s voltage limit.

The correct voltage depends on the current rating of your stepper motor. Most motors include a specification card listing the rated current in amps.

For the DRV8825, the reference voltage is typically set to half of the motor’s rated current. The equation is:

Vref = Imax/2

A motor rated for 1.0A would therefore use a reference voltage of roughly 0.5V.

My motor is rated for 1.7A, which would technically allow a 0.85V limit. However, running the driver at the maximum value tends to generate unnecessary heat, so I lowered mine to 0.6V to play it safe.

To adjust the voltage, connect power to the circuit:

1. Plug the 5V power supply into the Raspberry Pi.
2. Connect the 12V power supply to the screw terminal adapter connected to the breadboard.

Turn on the multimeter and place the negative probe on the GND pin of the driver. Touch the positive probe to the metal shaft of the screwdriver while the screwdriver touches the small adjustment potentiometer on the DRV8825.

Slowly rotate the potentiometer until the multimeter reads your target voltage.

Once the desired voltage is set, the driver is ready to use.

If at any point you are lost, look to the above images. I have included pictures of the driver, circuit, where to use the multimeter, and a picture of my motor specifications.

Before we take the next step in this project and begin building the circuit, I would like to give some insight into why I chose the motors that I chose to use for this project.

Nema 17 Geared Stepper Motor 5:18:1 Gear Ratio

The base rotation of the robotic arm is driven by a NEMA 17 geared stepper motor. In my early testing, I purchased a standard (non-geared) NEMA 17 stepper. This motor helped clarify the difference between geared and non-geared stepper motors to me, as it was not strong enough to move the arm.

Stepper motors operate using electromagnetic stepping. Inside the motor are multiple coils that generate magnetic fields when energized. By activating these coils in a controlled sequence, the rotor moves in precise angular steps. This is what allows stepper motors to achieve accurate, repeatable positioning.

In testing, however, the standard version of the motor did not provide enough torque for this application. While it could rotate freely under no load, it lacked the holding strength needed for a full robotic arm assembly. The shaft could still be turned by hand with relatively little resistance.

After some research into CNC and robotics builds, a geared version of the same motor came up as a common solution for increasing torque. The motor comes with an integrated gearbox that greatly increases torque and optimizes power over speed.

The final geared NEMA 17 was selected because it combines electromagnetic stepping with mechanical gears, resulting in much higher usable torque for the base joint.

MG996R Metal Gear Servo Motor

The claw mechanism is driven by an MG996R metal gear servo motor. This component acts as the “gripping force” of the system and is responsible for holding and releasing objects.

When I was still in the design phase, I initially thought that a servo motor would be best suited for the job of the claw, as it is lightweight but also strong enough to clasp an object. However, not all servo motors are the same.

Many hobby-grade servos use plastic gears, which can wear down or slip under continuous load. For a claw mechanism, this becomes an issue because the motor often needs to maintain force while gripping an object, not just move it.

The MG996R was selected specifically because of its metal gear train and higher torque rating. This allows it to maintain a stable hold without the internal gears degrading or slipping under load, which is critical for reliable object handling.

Images of both motors and what they look like inside are included above, along with a picture showing the original non-geared stepper that was ultimately replaced due to insufficient torque.

The DRV8825 stepper motor driver is now set, so we can assemble the complete electrical circuit.

This part can be a little tedious, and you have to pay attention to the tiny details to make sure that everything works.

I tried my best to design a diagram of the circuit to make this step as easy to understand as possible.

Materials Needed:

1. Raspberry Pi
2. DRV8825 stepper motor driver
3. DRV8825 stepper motor driver expansion board
4. Nema 17 geared stepper motor
5. Jumper wires (assortment of male-male, female- female, and male-female)
6. 12V power cord for stepper motor
7. 5V power supply cord for the servo motor
8. 5V power supply for Raspberry Pi

Step 1: Wire the Servo Motor Circuit

We'll start by hooking up the servo motor and its dedicated 5V power supply. Powering the servo motor directly from the Raspberry Pi 5V pin is a bad idea. If the servo stalls and draws too much current, it could destroy the Raspberry Pi's pins.

1. Connect the Power Jack: Take your first barrel jack connector (for the 5V supply). Connect the Red (VCC) wire from your servo to the positive (+) terminal, and the Brown/Black (Ground) wire from the servo to the negative (-) terminal.
2. Add the Capacitor: This part is optional. I decided to solder a capacitor to the power and ground wire of the servo motor to protect against any voltage spikes. This is not necessary, but it is nice to have a little extra protection just in case. I don't think I am qualified to give soldering instructions, but here is a YouTube video I found helpful if you are interested.
3. Establish a Common Ground: Run another jumper wire (shown in brown) from the negative (-) terminal of the barrel jack to a Ground pin on the Raspberry Pi. In the diagram, this is connected to the ground pin underneath GPIO 12
4. Connect the Signal Wire: Take the Yellow/Orange (Signal) wire from the servo and connect it to GPIO 12, right above the last ground pin.

Step 2: Set Up DRV8825 Expansion Board

The expansion board for the DRV8825 driver is basically just a little PCB that the chip can sit in. It helps get rid of a lot of wiring by using built-in traces. All you have to do is push the driver into the red holes making sure each pin lines up. You can look at the above images to see which direction the chip should be facing. The side with the potentiometer should be closest to the power jack.

1. Connect the 12V Power Jack: Take your second barrel jack connector (the one from the stepper motor power cord). Run a red wire from the positive (+) terminal to the positive input on the driver board's green screw terminal.
2. Connect the Ground: Run a jumper wire from the negative (-) terminal on the barrel jack to the negative input on the driver board's green screw terminal.
3. Plug in the Stepper Motor: Plug the 4 wires from the Nema 17 stepper motor directly into the white receptacle on the stepper driver expansion board. Keep in mind you will probably have to connect each wire from the motor to a jumper wire first, and then to the board. Ensure the wires are seated securely. The stepper motor has 4 wires, but really has two coils. Make sure that each coil is paired on the expansion board. For this motor, the two coils were green and black, and red and blue. You can see this in the diagram.

Step 3: Connect the Stepper Driver to the Raspberry Pi

Finally, we need to connect the logic pins from the DRV8825 expansion board to the Raspberry Pi so the Pi can tell the motor when and how to move.

Using female-to-female jumper wires, make the following connections from the driver board to the Pi:

1. Ground (Black Wire): Connect the GND pin on the driver board to a Ground pin on the Raspberry Pi. I used pin 39 on the very bottom left.
2. Direction (Yellow Wire): Connect the DIR (Direction) pin on the driver board to Pin 38 (GPIO 20) on the Pi.
3. Step (Purple Wire): Connect the STEP pin on the driver board to Pin 40 (GPIO 21) on the Pi.
4. 3.3V (Green Wire): Connect the pin labeled V on the board to the first pin on the Pi (3.3V).
5. Enable (Green Wire): Connect the EN (Enable) pin on the driver board to a ground pin on the board. This just wakes up the driver.
6. This wasn't needed before when we were using the breadboard, but for some reason, the driver needs to be "awakened" when it is in the expansion board. This may have to do with the connections that the board makes that would be made differently on a breadboard. When testing the motors with this circuit originally, I could not figure out why the motor was not working. It took me a while to realize it was not a software issue, but just something as simple as connecting two pins together.

At this point, the circuit is fully assembled, and we are ready to get to the 3D printing and designing of the robotic arm and parts.

Now comes a fun part: 3D designing the pieces for this build.

While I have used CAD before, this project required a much more practical approach as most of the dimensions came directly from physical components. The entire computer design phase took around 2.5 weeks. To me, CAD is one of those things where starting is the hardest part. Once you get rolling, it's honestly fun. I will break this section down into 3 main parts and describe the design process of each.

Software

I used a combination of different CAD tools for this project.

Fusion 360 was mainly used at the beginning for laying out parts and figuring out sizes. It was helpful for quickly importing reference models and checking how everything would physically fit together, especially for the claw.

After that, I switched to Onshape for most of the actual design work. It’s cloud-based, which made it easier to work on the project from different devices while iterating on parts over time.

Section 1: Servo Claw

The first part of the CAD was the claw. I wanted to make something simple yet effective that the servo motor could work well inside of and not be constantly fighting against.

I decided to use a two-finger claw, with each jaw being slightly angled to maximize contact points with the object it was going to be picking up.

I started by creating a spur gear and then branching off of that.

The right claw locks in place on the servo via a screw, and the left claw locks in place on the pole via a 3D printed nut I designed.

The hardest part of this section was actually the base around the claw. It needed to hold the servo, connect cleanly to the rest of the arm, and also support a mount for the camera. On top of that, it had to allow space for wiring without everything feeling cramped.

Most of this came down to measuring the servo carefully with a caliper and adjusting the CAD until everything fit correctly.

I also added a mounting slot for the camera stand, which slides into place on top like a cover.

Section 2: Pi Housing

The rear section of the arm holds the Raspberry Pi and the rest of the electronics. I placed it behind the main arm shaft so it also helps act as a counterweight, instead of just being extra weight on one side.

I started the design by laying out simplified shapes for the Raspberry Pi, AI HAT stack, motor shaft, and cable paths. Once those were positioned, I built the housing around them.

Some of the main features include:

1. Ventilation holes for airflow around the Pi
2. Four mounting points for standoffs to lock the Pi in place
3. A lid with a cutout for the camera ribbon cable
4. A small internal wall for the DRV8825 driver board to sit in
5. A d-lock hole for the stepper motor shaft to lock into
6. Openings for wiring access between sections

This part was mostly about fitting everything together cleanly in a tight space while still keeping access to the wiring and components.

Section 3: Base

The base is where everything is anchored. It holds the stepper motor, supports the arm, and also stores most of the wiring.

This was the simplest part to design since it mostly came down to the dimensions of the stepper motor.

Key features include:

1. Hollow interior for cable management
2. Central support column for structural strength
3. Recessed mount for the NEMA 17 geared stepper motor

Overall, I am really proud of how these models all came out. I put a lot of work into making everything as accurate as can be, and they worked even better than I expected. Now lets get to 3D printing.

For the 3D printing, I used a Bambu P1S with the filaments PLA Basic, PETG HF, and PLA Metal.

PLA Basic

1. Silver color, used for the Pi housing lid as well as the lid/base for the camera stand.

PLA Metal

1. Shiny blue color, used for the two jaws of the claw, as well as the nut.

PETG HF

1. Used for the rest of the parts of the arm, including the entire base.

These prints did not just come out great on the first try. If you look at the above image, you can see all of the prototypes I went through. It was a lot of testing different infills, wall loops, and making sure everything fit.

Eventually, I went with the following settings:

Infill - 6% Gyroid

Wall Loops - 3

Speed - Default

1. I also added a modifier on the connecting pins coming from the Pi housing, and made them solid with 100% infill to make sure they did not break off inside the other attachment.

Also, the camera case was based on an STL file from Printables.com and then used in the build. I had to modify the file to fit the new Raspberry Pi Camera. The original file can be found here.

With all printed parts complete, the sorting machine can now be assembled. Refer to the images above and/or the images in step 12 for overall placement and alignment of components.

Step 1: Connect the Arm Sections

Begin by connecting the Raspberry Pi housing to the servo/claw section of the arm. The alignment pins are designed to slide in with a snug fit. Apply even pressure to ensure both sections are fully seated and aligned.

Step 2: Install the Servo Motor and Camera Mount

Mount the MG996R servo motor into its designated slot on the claw assembly. Align the mounting holes with the printed structure and secure it using the four included screws.

Next, assemble and attach the camera stand to the lid section. This stand should fit right into the rectangular slot in the stand/lid.

Step 3: Add Foam Grip to Claw

Cut strips of the foam tape and apply them to the inner surfaces of both claw fingers. This will improve the grip and also add a tolerance for objects that may vary in size. Additional strips may be required to fully cover the contact area.

Step 4: Install Claw Mechanism

Attach the right claw to the servo horn and secure it using the black screw that comes in the bag with the servo motor.

The left claw mounts onto the opposing shaft and is locked in place using the printed nut. Once the nut slides down, it should be tight enough that glue is not needed.

Step 5: Install Electronics and Wiring

Carefully transfer the circuit into the housing section of the arm. Try to contain the wires inside the arm and keep the circuit flowing in as straight a line as possible; the more tangled things are, the more difficult it becomes to detect and fix issues.

Feed the Raspberry Pi camera ribbon cable through the designated slot in the lid and connect it to the camera module. Mount the camera into the camera stand once connected.

Place the Raspberry Pi into its area, making sure that the standoffs lock in place in the slots.

Step 6: Secure Wiring

You will have to connect multiple jumper wires to extend a connection in some cases in the arm. Use electrical tape to bundle and secure connection points on the jumper wires where necessary. This helps prevent wires from loosening or disconnecting during movement of the arm.

Step 7: Install Stepper Motor

Mount the stepper motor into the recessed slot in the base. It should sit flush within the hole.

Step 8: Route Wiring Through Base

Run all required wiring through the internal channels of the base and up into the arm assembly. Keep cables organized to avoid interference with motor movement or rotation. This part can take a bit of time, but it is just organizing cables.

In this step, we will be making a file and writing code to test different values for the claw. For example, we will see how far the claw needs to go before it has hold of something like a water bottle or box. We will then transfer these values to our final script in the next step.

Step 1: Power on the Raspberry Pi

Even though the Raspberry Pi is now mounted inside the machine, it can still be accessed normally.

Connect the HDMI cable and 5V power supply through the side access openings in the housing. Then plug in a keyboard and mouse through the USB ports.

Once everything is connected, power on the system and wait for the Raspberry Pi OS to boot.

Once the Raspberry Pi boots, open the terminal.

Step 2 — Enable hardware PWM for servo control

This allows us to manually tell the Raspberry Pi that we are using GPIO 12 for the servo motor. In the terminal, paste the following line:

A file will open. Scroll down to the very bottom and add this line:

Save with Ctrl+O, Enter, then Ctrl+X to exit.

Reboot:

After rebooting, re-enter the folder and reactivate the virtual environment we were using earlier for the AI.

Step 3: Create New File

Now we can create the file we will run the servo calibration code in. Paste:

This will open a blank file in the terminal editor.

At this point, the script is ready for the code to be added.

In the file, paste this code:

Save and exit with Ctrl+X and then Ctrl+Y.

Step 4: Run the Code

Now that the code is pasted and saved into our file, we can run it using this command:

Enter numbers between 500 and 2500, and the claw will move to those values. Once you find the values that fit your object, you can save them and exit by typing "quit".

How does this code work?

This code is basically a manual calibration tool for the servo motor controlling the claw. Instead of guessing positions in the main code, it lets you directly test and store exact pulse widths that correspond to real physical positions.

Servo motors don’t take angle values directly. They respond to pulse width signals (in microseconds), so this script is basically a way to map “numbers” to real movement.

How it works

GPIO setup

The script starts by using the lgpio library to control the Raspberry Pi’s GPIO pins at a low level.

This opens the GPIO chip and sets pin 12 as an output pin for the servo signal. If this fails, the script exits immediately to avoid running without hardware control.

Servo signal control

The servo is controlled using two key functions.

set_servo_idle()

This function shuts off the PWM signal after each movement:

Instead of continuously powering the servo, it cuts the signal after movement. This helps reduce jitter and avoids the “bad PWM micros” error that can happen if the signal is left running.

move_and_detach(us)

This is the main movement function

1. us is the pulse width in microseconds
2. This directly controls the servo position
3. After sending the signal, the script waits briefly so the servo has time to physically move
4. Then it shuts the signal off again using set_servo_idle()

You can think of it like this:

send signal → wait for movement → cut signal

Interactive calibration loop

The main() function is where the user can put in numbers.

It repeatedly asks for input:

You can do three things here:

1. Enter a number that moves the servo to that position
2. Type the save name, which stores the current position
3. Type quit, which exits the program

The script keeps track of the last valid position in last_us, so when you save a name, it knows what value to store.

Saving calibration data

When you exit the program, any saved positions are written to a JSON file:

This creates a simple lookup table like:

This is what allows the main sorting program to use consistent claw positions without recalibrating every time.

Cleanup process

At the end (even if the program crashes or is interrupted), the script:

1. Stops the servo signal
2. Releases the GPIO chip

This prevents the GPIO pins from being left in an active or unstable state after shutdown.

This script is basically a calibration layer between raw servo control and the main sorting system. It lets you test real motion values, assign them names, and store them so the final project can open and close the claw without guesswork.

Now, there may be a better way to test the values for a servo, but overall, I am really happy with how this code turned out. It took roughly a weekend of tinkering, and I'm glad I got it figured out and usable. It is easy to use, error-free, and gets the job done perfectly.

Finally, we get to my favorite part of the project; the main software powering the machine. This script combines the AI detection pipeline, servo motor, stepper motor, and sorting logic into one system.

The Raspberry Pi camera continuously looks for objects, the AI identifies what it sees, and the motors respond automatically based on the detected object.

Step 1: Open the Development Environment

If you are not already inside the AI environment, reopen the terminal and run:

This enters the Hailo project folder and activates the virtual environment containing all of the required AI libraries and dependencies.

Step 2: Create the Sorting Script

Inside the environment, create a new Python file for the sorting system:

This opens a blank file inside the Nano text editor.

In the file, paste this code:

Once the code is pasted in, save and exit Nano using:

1. Ctrl + X
2. Y
3. Enter

Step 3: Running the Program Remotely

Normally, you could run the program using:

However, since the Raspberry Pi camera preview needs to open on a physical display connected to the Pi, this command alone will not properly launch the camera window when executed remotely through SSH.

Instead, run:

This tells the program to open the camera window on the Raspberry Pi’s connected display instead of trying to display it through the SSH session.

Step 4: Create a Bash Alias Shortcut

Typing the full startup sequence every time becomes tedious very quickly:

To simplify this, we can create a Bash alias.

First, open a new terminal window so you exit the virtual environment.

Then open the .bashrc file:

Scroll to the bottom of the file and paste:

Save and exit using:

1. Ctrl + X
2. Y
3. Enter

Now reload the file:

Nothing visible will happen, but the shortcut is now registered.

From this point forward, the entire sorting system can be started simply by typing:

This command automatically:

1. Opens the correct project folder
2. Activates the virtual environment
3. Launches the AI pipeline
4. Opens the camera preview window
5. Starts the sorting system

It is a massive time saver.

Now, I did want to clutter this step with explanation of the main code, so I will explain it in the next step.

Now that the full script is written, I would like to explain how the code actually works. This project is much more than just “AI object detection.” We are handling live video processing, object detection, GPIO communication, motor timing, threading, and hardware protection logic all at once, which makes for a complex chunk of code (at least for me). I put a lot of effort into writing this code to be as optimal as possible, and I would like to explain the process.

I split the code into four main sections for easy understanding:

1. Initial setup and hardware configuration
2. Motion control functions
3. AI detection pipeline
4. Main runtime and shutdown handling

Section 1: Initial Setup and Imports

The first part of the script imports all of the libraries needed for the project.

These are mostly built-in Python libraries used for:

1. Managing the operating system
2. Exiting the program safely
3. Running multiple tasks at once using threads
4. Timing motor movement
5. Suppressing unnecessary warning messages

Next are the hardware libraries:

These are what allow the Raspberry Pi to physically communicate with the motors and AI pipeline.

1. gpiozero controls GPIO pins more cleanly
2. lgpio gives lower-level hardware access for accurate servo timing
3. gi is used for the GStreamer AI pipeline

GPIO Factory Setup

This line forces GPIOZero to use the lgpio backend instead of the default Raspberry Pi GPIO library.

This ended up being extremely important. When I was developing this code, I had issues with unstable servo movement and timing issues. Using lgpio gave me much more reliable hardware timing, especially while the AI pipeline was running at the same time.

Importing the Hailo AI Pipeline

These imports bring in the Hailo AI libraries and the GStreamer framework.

GStreamer is what handles the live video pipeline from the Raspberry Pi camera. Frames from the camera continuously move through the pipeline, where the AI model processes them using the complex math we talked about and returns object detections.

Section 2: Hardware Setup

This section defines all of the physical hardware settings for the robotic arm.

Servo Motor Configuration

The servo motor is connected to GPIO pin 12 and runs at 50Hz PWM frequency, which is the standard frequency used by most hobby servo motors.

Initializing GPIO

This opens the Raspberry Pi GPIO chip and claims the servo pin as an output.

If this fails for any reason, the script exits immediately:

This prevents the program from running without hardware control properly initialized.

Servo Calibration Values

By using the claw calibration script earlier, I found that these values work best for my claw. For the sake of simplicity right now, I will be using the AI to sort between a bottle and a cup.

Servo motors work using pulse widths measured in microseconds instead of direct angle values. Different pulse widths correspond to different physical positions of the claw.

These values determine:

1. Open claw position
2. Bottle gripping position
3. Cup gripping position

Stepper Motor Setup

The stepper motor driver uses two GPIO signals:

1. STEP pin → tells the motor to move
2. DIR pin → tells the motor which direction to rotate

Gear Ratio and Microstepping

This is one of the most important parts of the motion system.

The motor itself has:

1. 200 physical steps per revolution
2. 32x microstepping enabled on the DRV8825 driver

This means the motor is divided into:

Then the gearbox multiplies the effective torque and precision by 5.18.

This allows the arm to rotate smoothly and accurately instead of moving in large visible jumps.

It takes a bit of math to write this degree code correctly. It is just a bunch of conversion, multiplication, and division.

Section 3: Motion Functions

This section contains the functions responsible for physically moving the motors.

Servo Movement Function

This function moves the claw to a target pulse width.

The movement process happens in three stages:

1. Send PWM Signal

This generates the PWM signal needed to move the servo to the desired position.

2. Wait for Mechanical Movement

This gives the physical gears inside the servo time to actually reach the target position.

Without this delay, the code could continue running before the servo finished moving.

In development, I was having issues where the arm would detect objects and start a new code cycle, before the task was complete. This fixes that.

3. Disable the Signal

This intentionally shuts off the PWM signal afterward.

Originally, I left the signal active continuously, but it caused the servo to jitter and lose its position. I was convinced there was just an error in the code, but as I soon realized, it is just because the servo is pulling excessive power that the Pi can't supply through the GPIO pin. Disabling the signal after movement made the claw much more stable.

Stepper Motor Ramping

This function rotates the stepper motor smoothly using acceleration and deceleration ramps.

Without ramping, the motor would attempt to instantly jump to full speed, which can cause it to miss steps or stall. The idea here is to get the motor to slowly break inertia before it takes off.

Acceleration Logic

The delay between pulses starts large and gradually decreases.

Large delay = slow movement

Small delay = faster movement

This creates acceleration.

Deceleration Logic

Toward the end of the movement, the delays increase again.

This smoothly slows the arm down before stopping instead of abruptly halting rotation.

Converting Degrees Into Steps

This equation converts the motor rotation degrees into motor microsteps.

I learned that you can't just tell the motor to rotate and expect it to be fully accurate. You need to calculate the exact number of pulses the motor needs to reach an exact angle.

I came to learn pretty soon that the hardware portion of this code was essentially making equations for a giant math problem. How fun.

Tracking Position

This part was fairly simple. I essentially just added/subtracted from a variable to keep a log of where the motor has been.

When I call on the variable later, it allows the arm to return to its exact starting point using:

Section 4: Sorting Sequence

This is the actual robotic behavior.

When an object is detected, this function executes the full sorting process.

This code took quite a while if I am being honest.

Hardware Locking

This prevents multiple sorting operations from happening simultaneously.

Without this lock, multiple AI detections could attempt to move the arm at the same time, which would completely break the system.

Sequence Flow

The arm performs the following actions:

1. Close claw
2. Rotate arm
3. Release object
4. Return to home position

Claw Control

The servo closes differently depending on the detected object.

For example:

1. Bottle → tighter grip
2. Phone → different grip width

This improved sorting reliability significantly.

Rotation

The stepper rotates the arm 90 degrees either clockwise or counterclockwise depending on the object type.

Section 5: AI Detection Pipeline

This is where the live AI object detection happens.

GStreamer Application Class

This inherits from the official Hailo detection pipeline and modifies it slightly for this project.

AI Callback Function

This function runs continuously while the camera feed is active.

Every frame from the camera passes through here.

Reading AI Detections

The Hailo AI accelerator analyzes the frame and returns detected objects.

Each detection includes a box tracking the object's location and labels.

Confidence Threshold

This prevents weak or uncertain detections from triggering the sorter.

Earlier versions used lower thresholds, but the AI occasionally made incorrect predictions, especially with oddly shaped objects. This confidence requirement makes the AI extremely confident.

Object-Based Sorting Logic

The system checks the detected object label and launches a sorting thread depending on what was found.

You can change this part of the code for different objects.

Why Threading Is Used

The sorting sequence runs in a separate thread so the AI pipeline can continue processing video simultaneously.

Without threading, the camera feed would freeze every time the arm moved. This isn't really needed, but in testing, I found that having the video frozen most of the time makes displaying the video of the AI's viewpoint useless.

I have used threading before in other software projects, but for some reason, I had some trouble with this project. It could have been that I was unfamiliar with Linux-based coding, but I did a little more debugging than I would have liked to.

Section 6: Main Runtime and Shutdown Logic

Finally, the program starts the full system.

Initial Servo Position

The claw starts in the open position before detection begins.

Launching the AI Pipeline

This starts the live detection system and continuously processes camera frames.

At this point, the sorting machine is fully autonomous.

Safe Shutdown Handling

Executing a safe shutdown in projects like these is extremely important.

An issue I faced in this project was unexpected shutdowns, caused by accidentally closing something or unplugging a component. An unexpected shutdown leaves motors in the middle of what they were doing and makes the next setup a lot harder than it needs to be.

If motors are left powered or GPIO pins are left active, strange behavior can happen the next time the system starts.

That is why I designed a cleanup function inside the code to shut everything down safely.:

This guarantees that cleanup happens even if the program crashes.

Shutdown Process

During shutdown, the system:

1. Stops new detections
2. Returns the arm to home position
3. Disables the servo PWM signal
4. Releases the GPIO chip

This ensures the Raspberry Pi exits cleanly and leaves the hardware in a stable state.

Overall, this script became much more than just “AI object detection.” By the end of development, the Raspberry Pi was handling the following things all at once.

1. Live camera input
2. AI inference through the Hailo accelerator
3. Real-time GPIO communication
4. Stepper acceleration control
5. Servo pulse timing
6. Multithreaded execution
7. Hardware safety logic
8. Autonomous decision making

Despite my knowledge of code going into this, getting all of these systems to work together reliably took far more iteration and debugging than I originally expected. Now, there are most definitely software engineers out there or people better at coding than I who think this is not hard, but I am very proud of this code. Not only does it function flawlessly, but I made something original that other people don't have. I wasn't able to follow some YouTube video or another Instructable; I actually figured something out on my own.

And that is a good feeling.

Up to this point, a lot has gone into this build. The AI is installed, the code is written, the electronics are wired, and the robotic arm is fully assembled. Now it is finally time to run the autonomous AI sorting machine.

Before starting, make sure the Raspberry Pi, servo motor, and stepper motor are all connected and powered properly. At this stage, you can unplug the mouse and keyboard from the Raspberry Pi since they are no longer needed. I chose to connect a monitor to the Raspberry Pi to see a live feed of the camera footage.

Open the mobile app Termius on your phone and connect to your Raspberry Pi. Once connected, a remote terminal window will open. This terminal is essentially a live copy of the Raspberry Pi’s terminal, just being accessed wirelessly from your phone.

To start the project, simply type:

Because of the shortcut we created earlier in the Bashrc file, this single command automatically:

1. Opens the correct project folder
2. Activates the AI virtual environment
3. Launches the camera pipeline
4. Starts the object detection and sorting program

After a second, the camera feed should appear on the monitor connected to the Raspberry Pi. At this point, the machine is fully active.

Hold an object in front of the camera, and the AI model will identify it in real time. In my case, I used the machine to sort bottles and cups. Once an object is detected with a high enough confidence level, the robotic arm grabs the object, rotates to the desired drop-off point, and drops the item.

When you eventually want to close the program, just type Ctrl+C on Termius.

I can't really explain how it felt to see this thing work for the first time. Five months of continuous effort had built up to this point, and it finally worked with no issues.

One of the best parts about this project is that the AI model can be changed and expanded almost endlessly.

For this project, I used YOLOv8 trained on the COCO Dataset.

YOLO stands for “You Only Look Once.” It is an object detection model designed to identify and locate objects extremely quickly in real time. Instead of slowly analyzing different parts of an image one at a time, YOLO processes the entire frame at once and predicts what objects are visible along with where they are located.

The COCO dataset is one of the largest and most commonly used object detection datasets. It contains hundreds of thousands of labeled images across 80 different object categories. Some examples include:

1. Bottles
2. Cell phones
3. Cups
4. Chairs
5. People
6. Cars
7. Dogs
8. Books

In my case, I programmed the sorting machine to react specifically to bottles and cups. When the camera detects one of those objects with high enough confidence, the robotic arm performs a sorting sequence based on the label returned by the AI model.

Because the model was already trained on the aforementioned categories, I did not need to manually train my own AI model from scratch. The AI already knew what common objects looked like before I even started the project. Even if I wanted to, I would likely not be able to train a good model, as AI training requires a lot of VRAM. For any PC people out there, I would need at least an RTX 5090 in my computer to effectively train the model, and around 64GB of DDR5. Unfortunately, I am not exactly able to drop $3,000 on a new GPU and $200 on RAM right now.

One important thing to understand is that the AI is not actually “seeing” objects the way humans do. The model is really just doing an enormous amount of pattern recognition and math extremely quickly. During training, the model analyzed massive numbers of labeled images and learned what visual patterns are commonly associated with certain objects. When the camera feed comes in, the AI compares what it sees against those learned patterns and makes a prediction.

The interesting part is that this project could easily be expanded far beyond what I used it for. Since the AI model already supports many object categories, the machine could be modified to:

1. Sort recyclables
2. Separate tools or hardware parts
3. Organize products
4. Count inventory
5. Detect defective items
6. Trigger other automated systems

You could also train a completely custom model using your own dataset. For example, instead of detecting bottles or cups, you could train the AI to recognize specific brands, custom parts, or even unique objects that are not included in the COCO dataset.

Overall, the AI side of this project was easily the most difficult part, but also the most interesting. Watching the robotic arm physically react to objects detected by a real-time neural network made the entire project feel far more advanced than a normal robotic arm build.

When I first started this project, I honestly underestimated how difficult it was going to be. I originally thought I would just connect a few motors, run an AI model, and have a working robotic sorting machine within a couple of weeks. That idea disappeared pretty quickly.

Almost every stage of this build introduced a completely new problem to solve. Some parts were mechanical, like designing a robotic arm that was actually strong enough to move reliably. Other problems were electrical, such as stabilizing motors and managing wiring in a compact space. The most difficult challenges were definitely software and AI related. Getting real-time object detection running smoothly on the Raspberry Pi, integrating the AI HAT+, calibrating the motors, building the sorting logic, and debugging hardware communication took far longer than I originally expected.

What made this project especially challenging was the lack of existing documentation for exactly what I was trying to do. There was no complete tutorial showing how to build an AI-powered robotic sorting machine using this hardware combination. A large amount of the process came down to research, experimentation, failure, and trying again.

At multiple points during development, I had systems partially working but failing somewhere else. I would fix the AI speed, then discover latency problems. I would solve the motor control, then run into mechanical limitations. I would redesign parts, rewrite code, recalibrate hardware, and repeat the cycle. While frustrating at times, that process ended up being the most valuable part of the project.

What I built is more than just a robotic arm moving objects around. It is a system where mechanical engineering, electronics, artificial intelligence, programming, and CAD design all depend on each other simultaneously. The project only works because all of those different pieces work together.

The part that still stands out the most to me is watching the machine recognize an object through the camera, process that information, physically react to it, and then return home ready for the next object completely autonomously. There is something genuinely fascinating about seeing software interact with the physical world in real time.

What excites me most is that this project is really just a starting point. The same concepts used here scale far beyond a desktop sorting machine. I believe that this project shows that AI can be used in manufacturing, recycling, warehouse automation, agriculture, quality control, and just robotics in general. As AI hardware becomes smaller, faster, and more accessible, things like this will only become more capable.

This project pushed me far outside of my comfort zone technically and mentally, but it also showed me how much can be learned simply by continuing to iterate on a problem long enough. Every failure, redesign, and rewrite directly contributed to the final system.

Six months ago, this machine was just an idea. Now it is a fully autonomous AI-powered machine capable of detecting and reacting to the world around it in real time.

I am excited to see where AI can take us in the future and how it can help society. I hope you enjoyed this project as much as I did.

Thank you for your time,

-Caleb