HearSight AI - AI-Based Emotion Recognition System for the Visually Impaired

Facial expressions are an important part of human communication. However, visually impaired individuals often face challenges in understanding the emotions of people during conversations and social interactions.

HearSight AI is an assistive technology project designed to recognize facial expressions in real time using Artificial Intelligence and provide audio feedback to the user. The system helps visually impaired individuals understand the emotional state of the person standing in front of them, improving communication and social awareness.

Tools required to create this project are:

1. Pictoblox Software: Download PictoBlox | Windows, MacOS, Linux, Chromebook, Android & iOS
2. Laptop/PC
3. Inbuilt or External Camera Setup

Launch PictoBlox software on your computer, Choose Block coding

Add the following extension( Left Bottom)

1. Face Detection

Record or add audio files for:

1. Happy
2. Neutral
3. Sad
4. Angry
5. Surprised

Use the following blocks:

1. When Green Flag Clicked
2. Turn on video on stage
3. Set video transparency to 0%
4. Show bounding box
5. Set detection threshold to 0.5

Purpose

This initializes the camera and activates the face detection system.

These conditional blocks help the AI system identify different facial expressions detected through the camera. When a specific emotion such as happy, sad, angry, neutral, or surprised is recognized, the program plays the corresponding audio message. This allows visually impaired users to understand the emotional state of the person in front of them through sound feedback.

The project successfully performs:

1. Real-time face detection
2. Emotion recognition
3. Instant audio feedback

Detected Emotions

1. Happy
2. Neutral
3. Sad
4. Angry
5. Surprised

The system improves emotional awareness and communication support for visually impaired users.

The project successfully performs:

1. Real-time face detection
2. Emotion recognition
3. Instant audio feedback

Detected Emotions

1. Happy
2. Neutral
3. Sad
4. Angry
5. Surprised

The system improves emotional awareness and communication support for visually impaired users.

Assistive technology for visually impaired individuals

Smart accessibility devices

Emotion-aware AI systems

Human–computer interaction projects

Educational AI applications

The project can be enhanced by:

1. Adding more emotion categories
2. Supporting multiple languages
3. Using wearable smart glasses
4. Improving detection accuracy with advanced AI models
5. Adding offline processing support

HearSight AI demonstrates how Artificial Intelligence can improve accessibility and social communication for visually impaired individuals. By converting facial expressions into audio feedback, the system helps users better understand emotions during conversations.

This project highlights the power of AI in building inclusive and assistive technologies for real-world applications.