AirMIDI - Beyond Buttons & Knobs

As a long-time music lover, I’ve always been fascinated by how performers connect with their instruments. Whether it’s a guitarist bending strings, a pianist flowing across keys, or a DJ shaping sound with nothing but hand movements over a mixer, there’s something powerful about expressing music through motion.

I kept thinking about DJs and live performers who move their hands in the air, controlling sound almost instinctively. What if those gestures could actually become music?What if motion itself could trigger beats, samples, and effects?

That question became the spark for AirMIDI.

AirMIDI is a touchless, gesture-controlled musical instrument that lets you play music in mid-air. By waving your hands over distance sensors, you can trigger sounds, control parameters, and perform expressive gestures — all without touching a single button. A vibrant LED strip responds in real time, turning invisible motion into a visual performance as well.

This project isn’t about replacing traditional instruments. It’s about exploring a new way to interact with sound — one that feels intuitive and expressive.

1. ESP32 Development Board - used for BLE MIDI, sensor processing, and LED control
2. VL53L0X Time-of-Flight Distance Sensors ×3 - Used to detect hand presence, taps, holds, swipes, and push/pull gestures.
3. WS2812B Addressable RGB LED Strip (48 LEDs) - Provides visual feedback for idle state and gestures (I have taken 144LEDS/m strip for more dense light effects)
4. External 5V Power Supply - Required for stable operation of the LED strip
5. Jumper Wires
6. A cardboard box to fit all the components. A vinyl frosted sheet is used on LED strip to disperse the light.

For this Project I am using three VL53L0X ToF sensors, but we can further increase the sensor count to add more and precise gestures.

You can make the housing from:

1. Cardboard (quick prototype)
2. Laser-cut acrylic
3. 3D-printed enclosure (best long-term option)

Design considerations:

1. Rectangular box
2. Open top cavity for the LED strip
3. Three sensor slots aligned side-by-side
4. Space inside for the ESP32 and wiring

Use three VL53L0X ToF sensors spaced equally across the top front edge.

1. Sensors should align horizontally
2. Maintain uniform spacing between sensors
3. Keep sensors flush with the surface to avoid reflections
4. Leave a small slit for each sensor to “see” out of the enclosure

Mark sensor positions before mounting for better symmetry.

Use a WS2812 / Neopixel LED strip

1. Cut a narrow cavity/slot along the front edge of the enclosure
2. Place the LED strip 2–3 cm below the surface
3. Add a frosted / opal acrylic diffuser on top to smooth LED visibility

This diffuser significantly improves visual clarity and reduces LED pixel harshness.

Inside the enclosure, place:

1. ESP32-S3 board
2. 3x VL53L0X sensors
3. LED strip wiring

Fix components in place using hot - glue.

Ensure:

1. No wires obstruct sensor view
2. LED strip faces outward
3. ESP32 antenna remains unobstructed

Connections include:

1. I2C (SCL / SDA) shared across all sensors
2. Individual XSHUT pins for address configuration
3. 5V + GND for sensors and LED strip
4. LED data to ESP32 pin

Fix the wires using hot-glue.

Working Principle

Sensor Layout

The three VL53L0X sensors are placed horizontally with equal spacing, creating three interaction zones:

1. Left
2. Center
3. Right

Each sensor continuously measures the distance to the user’s hand over the I2C.

Gesture Detection Logic

🔹 Tap

1. A quick hand movement in and out of a sensor’s active range
2. Used to trigger MIDI notes (pads)

🔹 Hold

1. Keeping the hand within range for a defined duration
2. Used to arm special modes (like swipe)

🔹 Swipe

1. A sequential activation of sensors (left → center → right or vice versa)
2. Used for navigation or control changes

🔹 Push / Pull

1. Moving the hand closer or farther from the sensors
2. Used for continuous MIDI control (CC)

All gestures are time-based and debounced to avoid false triggering.

Wireless MIDI Communication

Once a gesture is detected and converted into a MIDI message by the ESP32, the data is transmitted wirelessly using Bluetooth Low Energy (BLE MIDI). BLE MIDI is a modern MIDI transport method that allows low-latency, cable-free communication between controllers and music software.

In this project, the ESP32 acts as a BLE MIDI peripheral, advertising itself as a MIDI device. On the receiving end, the Koala Sampler app on iOS recognizes AirMIDI as a standard MIDI controller. After pairing, MIDI notes and control messages generated from hand gestures are directly mapped to sampler pads, effects, or parameters inside the app using Koala’s MIDI Map feature.

Modes of Operation

The device provides two main performance modes:

1. Tap Mode (Note Mode) - Used to trigger percussion or one-shot samples.
2. FX Mode - Used to manipulate real-time effects such as filters, reverb, or pitch using continuous control (CC).

Tap Mode – Note Trigger Logic

1. Gesture Input: Hand distance from each VL53L0X sensor is analyzed.
2. Triggering: Short “tap” gestures are detected based on sensor entry + fast release.

Notes:

1. There are three banks.
2. Each bank contains three MIDI notes (one per sensor).
3. Users can map these notes to samples inside a MIDI app.

LED Feedback: Only the LED segment corresponding to the tapped sensor lights up, showing which note was triggered.

Gesture Detection: Tap detection is based on:

1. distance window
2. time window
3. release speed

FX Mode – Swipe Activation

Activation Gesture: Swipe across the three sensors to toggle FX mode on/off.

FX Sensors: Each sensor controls one effect (example):

1. Left → Filter
2. Center → Reverb
3. Right → Pitch

Push/Pull Control: Distance is continuously measured and mapped to MIDI CC values.

Mapping Logic: Sensor distance (in mm) → Continuous CC (0-127)

LED Feedback: LED strip displays FX intensity and active sensor color.

1. Install ESP32 Board Files by adding following link to File -> Preferences https://espressif.github.io/arduino-esp32/package_esp32_dev_index.json

2. Install following libraries to start with code:

1. Adafruit VL53L0X (Time-of-Flight sensors)
2. Adafruit NeoPixel (LED strip control)
3. BLEMIDI_Transport + BLEMIDI_ESP32 (Bluetooth MIDI / Koala)

You can check full Code on Github.

AirMIDI started as a simple idea inspired by motion, music, and curiosity, and evolved into a fully functional, touchless MIDI controller built entirely from components found in a junk drawer. Through this project, it became clear that meaningful and expressive musical instruments do not always require complex hardware or expensive parts—sometimes, creativity and thoughtful design are enough.

One of the most important lessons from this project was the importance of timing, debouncing, and feedback. Gesture-based interfaces are highly sensitive to noise and unintended movements, and achieving stability required careful tuning of thresholds and delays.