Lumisense - Adaptive Lighting Driven by AI

Developed a working AI prototype that uses a camera and detects the behaviour of people in a room and decides how much light is needed, with the objective to use less electricity as possible and spare energy. Can be used in places where sensors are not efficient. |

Target group: | Large buildings where different human actions and activities need different types of light intensities, or where sensors are ineffective due to the size of the space. |

Core problem: | Current lights in buildings are manual or rely on basic motion sensors that only detect presence. They can't distinguish between different actions of people. In big spaces like large warehouses they can't be used because of their low range, resulting in lights at full intensity even when nobody is present or when the task being performed requires low intensity light. |

Key question: | How can we reduce electricity consumption of lights in large buildings occupied by people?

Impact: Less electricity consumed, resulting in lower energy bills for businesses and institutions, and a more sustainable use of resources. |

No keyboard, mouse or SSH is needed during operation. The Pi starts everything on power-up, the ring LED and OLED report progress, and the OLED shows the address to open. All configuration happens on the Gradio website. |

Visual feedback: OLED display (128×64, I2C at 0x3C, mounted upside down and corrected with `rotate=2`). During start-up it reports progress and then the address to open. While running it flips every 3 seconds through people in view, room light, lamp level, what people are doing, and energy saved today — one big number at a time, because the screen is too small to read four lines at once. |

Hardware feedback: Ring of 8 addressable LEDs (WS2812) driven over SPI. Amber chase while starting, three green blinks when everything is up, red blinks when the database or the website fails to answer. |

Extra sensor or actuator:PIR motion sensor on GPIO 4. It gates the AI: the model only runs when the PIR has seen movement in the last 60 seconds, or while it still sees somebody, so an empty room costs one GPIO read per second instead of a neural network. Webcam over USB, also used as a light meter by locking its exposure. |

Before anything can be trained, the model needs to see what a room full of people actually looks like from where the camera will hang.

I mounted a USB webcam on a tripod at 2.2 m, angled slightly downward. That height is not random: at desk height you get faces and shoulders, and the model has no way to tell someone leaning back in a chair from someone standing up. From 2.2 m you get whole bodies and the posture is readable, which is exactly what the four classes depend on.

I shot in several places and in several lighting conditions — daylight, blinds closed, ceiling lights only, evening. A model trained on one room in one light learns that room, not the activity.

Roughly 1000 images went into the set.

Every person in every image gets a bounding box, and the box carries the activity as its label:

- active — working at a desk

- standing — on their feet, looking around, passing through

- passive — sitting still, resting, leaning back

- using_phone — attention on a handheld screenµ

Annotation was done in Roboflow and exported in YOLO format. The set went through several versions as I added images; version 8 is the one that trained the final model.

Training happens once, in the cloud — a Colab notebook with a GPU. The Raspberry Pi only ever runs the finished model; training on the Pi itself would take days and produce exactly the same file.

I picked the s size on purpose. The nano model is faster but loses the fine posture differences the four classes depend on; the medium and large models score better on paper but were too slow on the Pi even after conversion. The s model was the one that stayed accurate enough while still fitting in the frame budget.

Results:

| Metric | Value |

| mAP50 | 0.835 |

| mAP50-95 | 0.577 |

| Precision | 0.829 |

| Recall | 0.791 |

Trained for 100 epochs in about 31 minutes on a Colab GPU, starting from the pretrained `yolo26s.pt` weights. The best result landed at epoch 85.

The confusion matrix shows the honest weak spot: 【which two classes get mixed up — passive vs using_phone is the usual one】. That makes sense, because the difference between them is a phone in the hand, which at 2.2 m across a room is only a few pixels wide.

Flash Raspberry Pi OS 64-bit with the Raspberry Pi Imager. Before writing, open the settings gear and set the hostname, user, and Wi-Fi — the device is meant to run with no keyboard or screen attached, so it has to come up on the network by itself the first time.

The Shelly lamps do not join the building's Wi-Fi — the Pi runs its own hotspot and the bulbs connect to that. It keeps the lighting network self-contained, so the device works in a room whose Wi-Fi you do not control, and no lamp is reachable from the outside.

Four parts hang off the Pi. Wire everything with the Pi powered off.

| Component | Connection | Address / device |

| PIR motion sensor | GPIO 4 (+ 5 V, GND) | — |

| Ring LED, 8× WS2812 | SPI MOSI, GPIO 10 (+ 5 V, GND) | `/dev/spidev0.0` |

| OLED display 128×64 | I²C SDA/SCL, GPIO 2 / 3 (+ 3.3 V, GND) | bus 1, `0x3C` |

| USB webcam | any USB port | `/dev/video0` |

Two things that cost me time and are worth knowing before you start:

The ring LED is driven over SPI, not over a bit-banged GPIO pin. On the Pi 5 the old WS2812 libraries do not work; the `pi5neo` library pushes the data out of the SPI port instead. That is why SPI has to be enabled even though there is no obvious SPI device in the build.

The OLED is mounted upside down. It was the only way to route its cable cleanly inside the enclosure. Rather than fight the wiring, it is corrected in software with `rotate=2` when the display is opened — a zero-cost fix for a mechanical problem.

Gradio runs on port 7860. Four pages, each with one job.

About — the problem, what the device does, the four activities it recognises, the hardware, and what the energy numbers mean. Written for someone who has just walked up to it.

Data — people over time, activity breakdown, energy used against two baselines (lamps always on, and a plain motion switch), a per-zone and per-lamp breakdown, a heat map of where each activity happens in the room, and a calendar of average occupancy per day. All of it filterable to a single recording session.

Operating — the controls. Lighting mode is Off, Energy saving, or Continuous, and the page only shows the controls that do something in the chosen mode: a brightness slider per lamp in Off and Continuous, session and zone setup in Energy saving. Zones are drawn as polygons with any number of points, directly on a blurred still from the camera — enough to see where the desks are, not enough to identify anyone. Shelly lamps are discovered on the hotspot, named, assigned to a zone, given a brightness per activity, and can be made to blink so you can tell which physical lamp you are configuring.

Debugging — a drawing of the front panel that lights up in sync with the real hardware, plus a test button for the PIR, webcam, ring, OLED and light meter. A wiring check reports the GPIO pin, SPI device and I²C address the Pi can actually see, so a loose cable is obvious. Live detection with boxes and confidence scores is drawn on a black background by default — the boxes and labels without the camera image — so the model can be verified in a room full of people without putting anyone on screen.

The front panel is 220 × 60 mm, with a capsule-shaped opening for the webcam, a round opening for the PIR dome, and openings for the ring LED and the OLED.

Print settings:

| Setting | Value |

| Material | 【PLA】 |

| Layer height | 【0.2 mm】 |

| Infill | 【20%】 |

| Supports | 【yes 】 |

| Print time | 【20 h】 |

The device has to work with no keyboard, no screen and no SSH. Plug it in, and it comes up.

Three containers are defined in `compose.yaml` — the app, PostgreSQL, and Adminer — all with `restart: unless-stopped`. The app container gets direct access to the hardware:

```yaml

devices:

- /dev/video0 # webcam

- /dev/i2c-1 # OLED

- /dev/spidev0.0 # ring LED

- /dev/gpiochip0

- /dev/gpiochip4 # PIR

```

A systemd unit brings the stack up at boot, after Docker is ready:

```bash

sudo cp lumisense.service /etc/systemd/system/

sudo systemctl daemon-reload

sudo systemctl enable --now lumisense.service

```

What the user sees, with nothing plugged in but power:

1. Amber chase around the ring while the containers start and the database comes up.

2. Three green blinks when the API answers and everything is live.

3. The OLED shows the address to open — the Pi finds its own IP and displays it, so there is nothing to look up.

4. Red blinks if the database or the web app fails to answer, so a failure is visible from across the room instead of hidden in a log.

From then on the OLED cycles through the live numbers every 3 seconds — one big number at a time, because 128×64 pixels cannot show four readable lines at once.