Potholes & Animals Detection

This project describes the development of a real-time road pothole and animal detection and classification system, designed as a comprehensive solution for improving road safety. The system leverages modern computer vision techniques to automatically detect and classify different types of road potholes (dangerous and safe) as well as animals, providing timely alerts to the driver.

The primary motivation for this project is the increasing need to ensure driver and passenger safety, reduce vehicle damage, and minimize accidents caused by undetected road hazards. The solution is intended for integration into vehicles, city transportation, or for use by road maintenance organizations.

This document provides a step-by-step guide for assembling, configuring, and operating the system, making it suitable for both educational purposes and real-world prototyping.

Laptop (with GPU preferred): Used for model training and initial data processing.

Web Camera: Standard USB webcam for live video capture.

Raspberry Pi 5: Acts as a remote notification unit, processes commands, and drives display/alerts.

Ethernet Cable: For direct and stable connection between the laptop and the Raspberry Pi.

16x2 I2C LCD Display: Used to show driver alerts (all clear, warning, safe pothole, etc.).

Buzzer (Passive Audio Output): Provides audible alerts in case of detected danger (dangerous pothole or animal).

Basic electronic components and wiring: For connecting the LCD and buzzer to the Raspberry Pi.

Additional: You can use Freenova Projects Kit for Raspberry Pi (different versions).

Software & Files Needed

Python 3.9 or later (installed on both laptop and Raspberry Pi)

Ultralytics YOLOv8 library (for object detection and classification)

OpenCV (for video frame handling and visualization)

Flask (for communication between laptop and Raspberry Pi)

RPi.GPIO (for controlling GPIO pins on Raspberry Pi)

LCD library for Raspberry Pi (e.g., LCD_config.py with class LCD)

Custom-trained YOLOv8 model weights for pothole and animal detection (e.g., best.pt)

Supporting scripts:

1. Model inference script (runs on laptop, captures and processes webcam/video/image, sends results to Raspberry Pi)
2. Flask server script (runs on Raspberry Pi, receives results, manages LCD and buzzer)

Install Python and Pip

1. Make sure Python 3.9+ is installed on both your laptop and Raspberry Pi 5.
2. On Ubuntu/macOS, use:

sudo apt update

sudo apt install python3 python3-pip

Set up Virtual Environment (Recommended)

1. To avoid conflicts, create and activate a virtual environment:

python3 -m venv yolovenv

source yolovenv/bin/activate

Install Required Python Libraries

1. On your laptop (for training/running inference):

pip install ultralytics opencv-python requests

1. On your Raspberry Pi (for LCD/buzzer control, Flask server):

pip install flask RPi.GPIO smbus

Install and Set Up YOLOv8

1. The system uses YOLOv8, a state-of-the-art, real-time object detection model from Ultralytics.
2. Install using:

pip install ultralytics

1. [Optional] Verify installation:

from ultralytics import YOLO

model = YOLO("yolov8n.pt")

Prepare and Place Trained Model Weights

1. Place your trained YOLOv8 model (e.g., best.pt) in a known path (as referenced in your script).
2. The model must be trained to detect at least three classes:
3. Dangerous pothole
4. Safe pothole
5. Animal

On the Vehicle (Prototype Setting):

1. Webcam Placement:
2. For this project, a standard USB webcam is used. Ideally, it should be mounted in the location where a real-world automotive radar/camera would be installed — for example, on the vehicle's front bumper, aimed at the road ahead.
3. In the prototype, the webcam is simply connected to the laptop via USB.
4. Laptop:
5. Used for real-time video capture and running the YOLOv8 detection model.
6. Handles all frame processing and detection, then sends results to the Raspberry Pi via Ethernet.

On the Raspberry Pi:

1. I2C LCD Display:
2. Connect the 16x2 LCD display via I2C to the Pi.
3. Buzzer:
4. Connect the passive buzzer to a GPIO pin (e.g., BCM 4), for audible alerts.
5. Ethernet Connection:
6. Connect the laptop and Raspberry Pi using an Ethernet cable (static IP recommended for easier communication).

Configure IP Addresses:

1. Assign static IPs to both the laptop and Raspberry Pi for reliable communication.

Edit Python Scripts:

1. Ensure the correct paths for model weights, video source, and Raspberry Pi IP address are set in the detection script.

At this stage, a dataset of images containing road potholes and animals is prepared and annotated for supervised learning. The YOLOv8 neural network is trained on this dataset using a laptop. The model is optimized to distinguish between different types of potholes (safe and dangerous) and to recognize animals. After training, the model is evaluated on a separate validation set to ensure high accuracy and reliability in detection.

Once the model is trained, the inference pipeline is deployed on the laptop, which is connected to a standard webcam. The webcam captures real-time road video. Each frame is processed by the trained YOLOv8 model, which detects and classifies objects of interest (potholes and animals) in real-time. The results—including object type and bounding box—are displayed directly on the screen with colored overlays. The software also forms a status message based on the detected classes.

The processed detection results are transmitted from the laptop to the Raspberry Pi 5 using an Ethernet cable for fast and reliable data transfer. The Raspberry Pi receives information about detected potholes and animals via HTTP requests. Based on the received data, the Pi displays relevant messages on the LCD screen. If a dangerous pothole or animal is detected, an audible warning is triggered using a buzzer to immediately alert the driver. Safe potholes or clear road status are also communicated through the LCD.

The entire system operates in real time: the webcam constantly streams video, the laptop performs object detection and classification, and the Raspberry Pi provides timely notifications to the user. All alerts are displayed on the LCD, and critical events (such as dangerous potholes or animals) are accompanied by a buzzer signal. The terminal on the laptop also provides textual information about detections for operator reference. The system is designed for ease of use and rapid response, ensuring that drivers receive immediate feedback on hazardous road conditions.

Thank you for your attention!

Feel free to contact me if you have any questions about it.

Contact information:

Email: anton.tykhonenko@student.howest.be

Phone number: +380(50)-658-43-32