Hand Wrist Rotation Mimc Using Esp32 and Python

This project is a stepper motor control system designed and constructed by me based on the computer vision of real-time hand movements. My project was aimed to investigate the possibilities of software and hardware to collaborate to develop interactive and responsive systems. I followed my hand movements with the help of a webcam and a Python program on my laptop and used control commands to an ESP32 microcontroller via a serial connection. These commands were then converted to electrical signals by the ESP32 which then drove a stepper motor using a motor driver, the DRV8825.

One of the major aspects that I considered during my design was control, safety, and reliability. To do this, I added a logic gate that will serve as a hardware verification of permission whereby the motor will not move unless certain conditions are met.

Supplies Used

-Esp32 Mirco controller

1. Type C To usb wires
2. Jumper cables
3. DRV8825 Stepper motor Driver
4. NIMA Stepper Motor
5. 100uf polarized type capacitor
6. 12 Volt Power Supply
7. Glue gun
8. Glue sticks
9. Cardboard

1. Place the driver onto the bredboard
2. Plug in your GPIO pins In step and DIR according to the code, in this case DIR = 5 and Step = Pin 18
3. Wire the grounds and 5V
4. Wire the stepper motor according to your NIMA stepper motor color combo, in order of the schmatic
5. Connect the 12V power supply, with a 100uf capac between gnd and 5v on the motor supply pins

The Python program uses the camera to watch my hand and figure out where it is on the screen. It turns that movement into a simple number and sends it to the ESP32 through a USB cable. The ESP32 then uses that number to control how the stepper motor moves.

Turns on your laptop’s camera.

1. Looks at each camera frame to see if a hand is visible.
2. Finds points on your hand (like your finger or wrist).
3. Uses one point’s position to tell where your hand is on the screen.
4. Smooths the movement so small shakes don’t cause sudden changes.
5. Ignores tiny movements near the center so the motor doesn’t move for no reason.
6. Changes your hand position into a number that represents how the motor should move.
7. Sends that number as text through the USB cable to the ESP32.
8. Keeps doing this many times per second while the camera is open.
9. Stops when you press q or close the program.

HOW TO USE

1. Install Python and the needed libraries on your computer.
2. Set the correct USB port in the code so Python knows where the ESP32 is.
3. Make sure nothing else is using the USB port.
4. Run the Python file from the terminal or command prompt.
5. Move your hand in front of the camera and watch the motor respond.
6. Press q to close the camera window and stop the program.

Python code:

import cv2

import mediapipe as mp

import serial, time, glob, math

#zaidmadethiscode

# ===== SETTINGS =====

RANGE_STEPS = 3200 # must match ESP32 code PLEASASEASE

SEND_HZ = 30

SMOOTH_ALPHA = 0.25 # 0.15 smoother, 0.35 more responsive

DEAD_DEG = 2.0 # ignore tiny jitter in degrees

# Map wrist roll range (degrees) to stepper range:

MIN_DEG = -60 # wrist rotated left limit

MAX_DEG = 60 # wrist rotated right limit

# ====================

ports = glob.glob("/dev/cu.*")

SERIAL_PORT = "please replace your port here "

for p in ports:

if ("usbserial" in p) or ("SLAB_USBtoUART" in p) or ("wchusbserial" in p):

SERIAL_PORT = p

break

if SERIAL_PORT is None:

print("No ESP32 port found. Run: ls /dev/cu.*")

raise SystemExit

print("Using serial port:", SERIAL_PORT)

ser = serial.Serial(SERIAL_PORT, 115200, timeout=0.1)

time.sleep(2)

mp_hands = mp.solutions.hands

hands = mp_hands.Hands(max_num_hands=1, min_detection_confidence=0.6, min_tracking_confidence=0.6)

cap = cv2.VideoCapture(0)

last_send = 0.0

send_interval = 1.0 / SEND_HZ

angle_smooth = 0.0

last_angle = 0.0

target_steps_sent = RANGE_STEPS // 2

def clamp(v, lo, hi):

return lo if v < lo else hi if v > hi else v

def send_position(steps):

ser.write(f"P {int(steps)}\n".encode())

while True:

ret, frame = cap.read()

if not ret:

break

frame = cv2.flip(frame, 1)

h, w, _ = frame.shape

rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)

result = hands.process(rgb)

angle_deg = None

if result.multi_hand_landmarks:

lm = result.multi_hand_landmarks[0].landmark

# landmark 5 = index MCP, 17 = pinky MCP

x5, y5 = lm[5].x, lm[5].y

x17, y17 = lm[17].x, lm[17].y

# angle of the knuckle line in image

ang = math.degrees(math.atan2((y17 - y5), (x17 - x5)))

angle_deg = ang

# Smooth

angle_smooth = (1 - SMOOTH_ALPHA) * angle_smooth + SMOOTH_ALPHA * angle_deg

# Deadband

if abs(angle_smooth - last_angle) < DEAD_DEG:

angle_smooth = last_angle

else:

last_angle = angle_smooth

# Map angle to 0..RANGE_STEPS

a = clamp(angle_smooth, MIN_DEG, MAX_DEG)

t = (a - MIN_DEG) / (MAX_DEG - MIN_DEG) # 0..1

target_steps = int(t * RANGE_STEPS)

# send at rate

now = time.time()

if now - last_send >= send_interval:

send_position(target_steps)

target_steps_sent = target_steps

last_send = now

# Draw landmarks line

p5 = (int(x5*w), int(y5*h))

p17 = (int(x17*w), int(y17*h))

cv2.line(frame, p5, p17, (0, 255, 0), 3)

cv2.putText(frame, f"Wrist roll(deg): {angle_smooth:.1f}", (20, 40),

cv2.FONT_HERSHEY_SIMPLEX, 1, (255,255,255), 2)

cv2.putText(frame, f"Target steps: {target_steps_sent}/{RANGE_STEPS}", (20, 80),

cv2.FONT_HERSHEY_SIMPLEX, 1, (255,255,255), 2)

cv2.imshow("Wrist Mimic (Roll) -> Stepper", frame)

if cv2.waitKey(1) & 0xFF == ord('q'):

break

# hold

send_position(target_steps_sent)

cap.release()

cv2.destroyAllWindows()

ser.close()

This project uses a 74HC08 as a safety/permission gate for the stepper motor. The gate is therefore a hardware lock. In the case of ALLOW being LOW, the step pulses are inhibited and the motor will not move, even when the software malfunctions. This proves handy as it provides additional control and safety on the hardware level.

Power pins (must be connected)

1. 74HC08 pin 14 (VCC) → ESP32 3.3V
2. 74HC08 pin 7 (GND) → ESP32 GND
3. Add a 0.1µF (100nF) capacitor between pin 14 and pin 7 (this helps keep the chip stable)

One AND gate (first gate on the chip)

Using the first AND gate inside the chip:

1. 74HC08 pin 1 (1A) → ESP32 STEP pin (GPIO18)
2. 74HC08 pin 2 (1B) → ESP32 ALLOW pin (GPIO19)
3. 74HC08 pin 3 (1Y output) → DRV8825 STEP input

// - If ALLOW is LOW, motor will not move even if STEP toggles.

// - This sketch sets ALLOW HIGH only when target speed

//codebyzaideiaz

const int DIR_PIN = 5;

const int STEP_PIN = 18;

const int ALLOW_PIN = 19;

volatile int targetSPS = 0;

unsigned long lastStepMicros = 0;

const int MAX_SPS = 1200;

//codebyzaidriaz

void stepPulse() {

digitalWrite(STEP_PIN, HIGH);

delayMicroseconds(5);

digitalWrite(STEP_PIN, LOW);

}

void setup() {

Serial.begin(115200);

pinMode(DIR_PIN, OUTPUT);

pinMode(STEP_PIN, OUTPUT);

pinMode(ALLOW_PIN, OUTPUT);

digitalWrite(STEP_PIN, LOW);

digitalWrite(ALLOW_PIN, LOW);

}

void loop() {

// ---- Read serial command ----

if (Serial.available()) {

String line = Serial.readStringUntil('\n');

line.trim();

if (line.length() > 0 && line.charAt(0) == 'S') {

int sp = line.substring(1).toInt();

// clamp speed

if (sp > MAX_SPS) sp = MAX_SPS;

if (sp < -MAX_SPS) sp = -MAX_SPS;

targetSPS = sp;

}

}

// this is for the gate controll

if (targetSPS == 0) {

digitalWrite(ALLOW_PIN, LOW); // block step pulses

return;

} else {

digitalWrite(ALLOW_PIN, HIGH); // allow step pulses

}

// ---- Direction of the stpper and stufuf

digitalWrite(DIR_PIN, (targetSPS > 0) ? HIGH : LOW);

// timing of the step

unsigned long interval = 1000000UL / (unsigned long)abs(targetSPS);

unsigned long now = micros();

if (now - lastStepMicros >= interval) {

lastStepMicros = now;

stepPulse();

}

}