AI Social Anxiety Smartglasses

by jfunky1111 in Circuits > Assistive Tech

43 Views, 1 Favorites, 0 Comments

AI Social Anxiety Smartglasses

20260123_192741.jpg
20260123_192754.jpg

Sometimes when people get a little bit overwhelmed, they need to take a break and talk to someone. This is the solution: Smartglasses tailored to help you get through whatever your problem is! All for the insane cost of less than $60.

Supplies

51hgdPB7atL._AC_UY218_.jpg
61CwvnM+KZL._AC_AA152_.jpg
617qjlsnE1L._AC_AA152_.jpg
618PNglGeML._AC_AA152_.jpg

---PARTS---


ESP32 S3 Sense

Wires, I used these

I2S Audio driver, I used this MAX98357A

Bone Conduction Speaker

Any generic 1S LiPo battery

Any generic switch

Any generic pair of glasses/sunglasses



---TOOLS---

Soldering Iron

Wire cutters/strippers

Hot glue gun


---SOFTWARE---

Arduino IDE

Amazon.com

Assembly

Screenshot 2026-02-01 105502.png

Time to get the most tedious part out of the way...

Find S3 pinout at https://files.seeedstudio.com/wiki/SeeedStudio-XIAO-ESP32S3/img/2.jpg

Attachments

Hot glue the:

  1. ESP32 Sense to the right front outside of the glasses.
  2. Bone conduction speaker to the area of the glasses right by your right temple.
  3. Speaker driver to the opposite side of the arm as the ESP32.


Power

  1. Wire the RED side of the battery to one side of the switch
  2. Wire the other side of the switch to the ESP32 BAT+ pad
  3. Wire the GND side of the battery to the ESP32 BAT - pad
  4. MAX VCC or VIN to ESP32 3v3
  5. MAX GND to ESP32 GND

Sound

  1. MAX DIN to S3 D3
  2. MAX BCLK to S3 D4
  3. MAX LRC to S3 D5
  4. MAX GAIN to MAX GND
  5. SPK RED to MAX +
  6. SPK BLACK to MAX -

Other

  1. Hot glue a wire from D0 to a point on the sunglasses you want to be able to touch to talk to the AI (make sure the wire end is exposed so you can touch it)
  2. Insert a FAT32 format SD card of less than or equal to 32 GB into the S3's sd card slot on the extension board
  3. Attach the IPEX antenna to the port on the S3 base board, and put the antenna on part of your smartglasses that is NOT near metal
  4. Attach the extension board to the main board

Code

Screenshot 2026-02-22 192423.png
  1. API Stuff
  2. Go to https://groq.com/
  3. Click Start Building
  4. Make an account
  5. Click API Keys in the top right
  6. Click create API key, give it any name, and an unlimited expiration date
  7. DON'T CLOSE THE POPUP! YOU WILL NEED THE API KEY!!
  8. Go to wit.ai
  9. Create account/login
  10. Click New App, name it anything
  11. Go to Management -> Settings
  12. DON'T CLOSE THIS EITHER! YOU WILL NEED THE KEY!!
  13. Actual Code Stuff
  14. Install Arduino IDE
  15. Copy this code
#include <Arduino.h>
#include <WiFi.h>
#include <WiFiClientSecure.h>
#include <HTTPClient.h>
#include <SPI.h>
#include <SD.h>
#include <ArduinoJson.h>
#include <ESP_I2S.h>
#include "WitAITTS.h"
//CHANGE THESE \/\/\/\/\/
#define WIFI_SSID_PLACEHOLDER "SSID_PLACEHOLDER"
#define WIFI_PASSWORD_PLACEHOLDER "PASSWORD_PLACEHOLDER"
#define GROQ_API_KEY_PLACEHOLDER "GROQ_API_KEY_PLACEHOLDER"
#define WIT_TOKEN_PLACEHOLDER "WIT_TOKEN_PLACEHOLDER"

#define SD_CS_PIN 21
#define PDM_CLOCK_PIN 42
#define PDM_DATA_PIN 41
#define I2S_DOUT_PIN 4
#define I2S_BCLK_PIN 5
#define I2S_WS_PIN 6

#define SAMPLE_RATE_HZ 16000

const char* WHISPER_HOST = "api.groq.com";
const char* WHISPER_PATH = "/openai/v1/audio/transcriptions";
const char* GROQ_CHAT_URL = "https://api.groq.com/openai/v1/chat/completions"

;

I2SClass pdmRecorder;

WitAITTS ttsEngine(I2S_BCLK_PIN, I2S_WS_PIN, I2S_DOUT_PIN);

volatile bool recordingRequested = false;
volatile bool recordingStopRequested = false;

const char* wifiSsid = WIFI_SSID_PLACEHOLDER;
const char* wifiPassword = WIFI_PASSWORD_PLACEHOLDER;
const char* groqApiKey = GROQ_API_KEY_PLACEHOLDER;
const char* witToken = WIT_TOKEN_PLACEHOLDER;

TaskHandle_t recorderTaskHandle = nullptr;
TaskHandle_t networkTaskHandle = nullptr;

const char* WAV_FILE_PATH = "/r.wav";

inline unsigned long nowMs() { return millis(); }

void dbgPrint(const String &s) { Serial.println(s); }

void writeWavHeader(File &fileHandle, uint32_t sampleRate, uint32_t dataBytes) {
fileHandle.seek(0);
fileHandle.write((const uint8_t*)"RIFF", 4);
uint32_t fileSizeMinus8 = 36 + dataBytes;
fileHandle.write((const uint8_t*)&fileSizeMinus8, 4);
fileHandle.write((const uint8_t*)"WAVE", 4);
fileHandle.write((const uint8_t*)"fmt ", 4);
uint32_t subChunk1Size = 16;
fileHandle.write((const uint8_t*)&subChunk1Size, 4);
uint16_t audioFormat = 1;
uint16_t numChannels = 1;
uint16_t bitsPerSample = 16;
uint32_t byteRate = sampleRate * numChannels * (bitsPerSample / 8);
uint16_t blockAlign = numChannels * (bitsPerSample / 8);
fileHandle.write((const uint8_t*)&audioFormat, 2);
fileHandle.write((const uint8_t*)&numChannels, 2);
fileHandle.write((const uint8_t*)&sampleRate, 4);
fileHandle.write((const uint8_t*)&byteRate, 4);
fileHandle.write((const uint8_t*)&blockAlign, 2);
fileHandle.write((const uint8_t*)&bitsPerSample, 2);
fileHandle.write((const uint8_t*)"data", 4);
fileHandle.write((const uint8_t*)&dataBytes, 4);
}

void recorderTask(void* pv) {
const size_t READ_BUF_SZ = 256;
uint8_t readBuffer[READ_BUF_SZ];

for (;;) {
if (!recordingRequested) {
vTaskDelay(pdMS_TO_TICKS(20));
continue;
}

recordingRequested = false;
recordingStopRequested = false;

File wavFile = SD.open(WAV_FILE_PATH, FILE_WRITE);
if (!wavFile) {
dbgPrint("[REC] WAV open failed");
vTaskDelay(pdMS_TO_TICKS(500));
continue;
}

uint8_t zero44[44] = {0};
wavFile.write(zero44, 44);

pdmRecorder.setPinsPdmRx(PDM_CLOCK_PIN, PDM_DATA_PIN);
pdmRecorder.begin(I2S_MODE_PDM_RX, SAMPLE_RATE_HZ, I2S_DATA_BIT_WIDTH_16BIT, I2S_SLOT_MODE_MONO);

size_t totalBytes = 0;
unsigned long startMs = nowMs();

while (!recordingStopRequested) {
int r = pdmRecorder.readBytes((char*)readBuffer, READ_BUF_SZ);
if (r > 0) {
wavFile.write(readBuffer, r);
totalBytes += (size_t)r;
}
vTaskDelay(pdMS_TO_TICKS(2));
}

pdmRecorder.end();

writeWavHeader(wavFile, SAMPLE_RATE_HZ, (uint32_t)totalBytes);
wavFile.close();

dbgPrint("[REC] Finished bytes=" + String(totalBytes) + " ms=" + String(nowMs() - startMs));

if (networkTaskHandle) xTaskNotifyGive(networkTaskHandle);
}
}

bool streamFileToWhisper(String &outTranscript) {
File wavFile = SD.open(WAV_FILE_PATH);
if (!wavFile) {
dbgPrint("[WHISPER] WAV open failed");
return false;
}

WiFiClientSecure client;
client.setInsecure();

if (!client.connect(WHISPER_HOST, 443)) {
dbgPrint("[WHISPER] TLS connect failed");
wavFile.close();
return false;
}

String boundary = "----ESP32BOUNDARY";

String head =
"--" + boundary + "\r\n"
"Content-Disposition: form-data; name=\"model\"\r\n\r\n"
"whisper-large-v3-turbo\r\n"
"--" + boundary + "\r\n"
"Content-Disposition: form-data; name=\"file\"; filename=\"audio.wav\"\r\n"
"Content-Type: audio/wav\r\n\r\n";

String tail = "\r\n--" + boundary + "--\r\n";

uint32_t contentLength = head.length() + (uint32_t)wavFile.size() + tail.length();

client.printf("POST %s HTTP/1.1\r\n", WHISPER_PATH);
client.printf("Host: %s\r\n", WHISPER_HOST);
client.printf("Authorization: Bearer %s\r\n", groqApiKey);
client.printf("Content-Type: multipart/form-data; boundary=%s\r\n", boundary.c_str());
client.printf("Content-Length: %u\r\n", contentLength);
client.print("\r\n");

client.print(head);

uint8_t buffer[512];
while (wavFile.available()) {
int r = wavFile.read(buffer, sizeof(buffer));
if (r <= 0) break;
client.write(buffer, r);
}

wavFile.close();

client.print(tail);

String response;
unsigned long lastRead = nowMs();

while (client.connected() || client.available()) {
if (client.available()) {
response += client.readStringUntil('\n');
lastRead = nowMs();
if (response.length() > 128 * 1024) break;
} else {
if (nowMs() - lastRead > 5000) break;
delay(5);
}
}

client.stop();

int jsonStart = response.indexOf('{');
if (jsonStart < 0) {
dbgPrint("[WHISPER] No JSON in response");
return false;
}

String jsonPart = response.substring(jsonStart);

StaticJsonDocument<8192> doc;
DeserializationError err = deserializeJson(doc, jsonPart);
if (err) {
dbgPrint("[WHISPER] JSON parse error");
return false;
}

if (!doc.containsKey("text")) {
dbgPrint("[WHISPER] No text field");
return false;
}

outTranscript = String((const char*)doc["text"]);
dbgPrint("[WHISPER] Transcript: " + outTranscript);

return true;
}

bool queryGroqChat(const String &promptText, String &outReply) {
String historyData = "";

if (SD.cardType() != CARD_NONE && SD.exists("/chat_hist.txt")) {
File histFile = SD.open("/chat_hist.txt", FILE_READ);
while (histFile.available() && historyData.length() < 1000) {
historyData += histFile.readStringUntil('\n') + "\n";
}
histFile.close();
}

WiFiClientSecure client;
client.setInsecure();

HTTPClient http;

if (!http.begin(client, GROQ_CHAT_URL)) {
dbgPrint("[CHAT] HTTP begin failed");
return false;
}

http.addHeader("Content-Type", "application/json");
http.addHeader("Authorization", String("Bearer ") + groqApiKey);

DynamicJsonDocument body(12288);

body["model"] = "openai/gpt-oss-20b";

JsonArray messages = body.createNestedArray("messages");
JsonObject m = messages.createNestedObject();
m["role"] = "user";
// Feel free to customise this prompt as much or as little as you want. Make sure to include the bit about keeping it inder 120 characters, as WitAITTS cuts it off there.
String therapyPrompt =
"Act as a supportive therapy bot focused on social anxiety. Provide short, practical steps, breathing cues, grounding exercises, and empathetic reflections. Keep replies concise and calm and under 120 characters. Avoid medical diagnosis. If a crisis is indicated, advise seeking immediate professional help.";

String fullPrompt = (historyData.length() > 0) ? ("HISTORY:\n" + historyData + "\n") : "";
fullPrompt += therapyPrompt + "\n" + promptText;

if (fullPrompt.length() > 2000) fullPrompt = fullPrompt.substring(fullPrompt.length() - 2000);

m["content"] = fullPrompt;

String payload;
serializeJson(body, payload);

int code = http.POST(payload);

if (code != 200) {
dbgPrint("[CHAT] HTTP Error " + String(code) + ": " + http.getString());
http.end();
return false;
}

String resp = http.getString();
http.end();

DynamicJsonDocument res(12288);
if (deserializeJson(res, resp)) {
dbgPrint("[CHAT] JSON parse error");
return false;
}

if (res["choices"][0]["message"].containsKey("content")) {
outReply = res["choices"][0]["message"]["content"].as<String>();

if (SD.cardType() != CARD_NONE) {
File f = SD.open("/chat_hist.txt", FILE_WRITE);
if (f) {
f.println("U: " + promptText);
f.println("A: " + outReply);
f.close();
}
}

return true;
}

return false;
}

void handleAiReply(const String &replyText) {
dbgPrint("[AI] Reply: " + replyText);
ttsEngine.speak(replyText);
}

void networkTask(void* pv) {
for (;;) {
ulTaskNotifyTake(pdTRUE, portMAX_DELAY);

String transcript;
if (!streamFileToWhisper(transcript)) {
dbgPrint("[NET] Transcription failed");
continue;
}

String aiReply;
if (!queryGroqChat(transcript, aiReply)) {
dbgPrint("[NET] Chat failed");
continue;
}

handleAiReply(aiReply);
}
}

void connectToWiFi() {
WiFi.disconnect(true);
WiFi.mode(WIFI_STA);
WiFi.begin(wifiSsid, wifiPassword);

unsigned long start = millis();

while (WiFi.status() != WL_CONNECTED && millis() - start < 30000) {
delay(500);
yield();
}

if (WiFi.status() == WL_CONNECTED) {
dbgPrint("[WIFI] Connected");
} else {
dbgPrint("[WIFI] Connection failed");
}
}

void setup() {
Serial.begin(115200);
delay(1000);

SPI.begin();

if (!SD.begin(SD_CS_PIN)) {
dbgPrint("[SETUP] SD init failed");
}

connectToWiFi();

ttsEngine.begin(wifiSsid, wifiPassword, witToken);
ttsEngine.setVoice("wit$British Butler");
ttsEngine.setStyle("default");
ttsEngine.setSpeed(100);
ttsEngine.setPitch(100);
ttsEngine.setGain(1.0);
ttsEngine.printConfig();
ttsEngine.setDebugLevel(DEBUG_VERBOSE);

xTaskCreatePinnedToCore(recorderTask, "RecorderTask", 4096, nullptr, 2, &recorderTaskHandle, 1);
xTaskCreatePinnedToCore(networkTask, "NetworkTask", 8192, nullptr, 2, &networkTaskHandle, 1);

ttsEngine.speak("System online");
}

void loop() {
ttsEngine.loop();
}
  1. In Arduino IDE, click File -> New Sketch
  2. Paste the code
  3. Replace the 4 placeholders with your wifi and your API keys
  4. Click File -> Preferences -> Additional board manager URLs, and enter this https://espressif.github.io/arduino-esp32/package_esp32_index.json
  5. Go to board manager (the second button down on the left), search for ESP32 by Espressif Systems, and click install
  6. For each of the .zip files in this .zip folder, click Sketch -> Include Library -> Add .zip library -> Select the zip
  7. Plug in your ESP32 S3 with a data-capable cable, click the little box to the right of the 3 round buttons on the top left, search for XIAO_ESP32S3, and click select while having your S3's COM port selected
  8. Exit the board selector
  9. Hit UPLOAD!!!! (the little blue sideways arrow in the top left)
  10. Open Serial Monitor (the little magnifying glass with the dots behind it in the top right corner)
  11. Watch it connect to your designated WiFi network
  12. YOUR NETWORK MUST BE A 2.4 GHZ BAND, with LESS THAN WAP3 security! IT WILL NOT CONNECT IF EITHER OF THESE IS TRUE!
  13. If it succeeds, continue. If not, ask me in the comments for help. I should respond in less than 24 hours.
  14. Press the wire on the side, say hello, and then let go! If everything is correct, then it should say a reply in under 30 seconds, depending on your wifi speed and RSSI.
  15. Turn the switch on, disconnect it from your device, and then try again. If it still works, good news!

You are now done!


Customization

As this is based on ESP32, it is fully customizable, and I put no copyright whatsoever on this. Do whatever you want with it, please, because if you've read this far, there has no doubt been "blood, sweat, toil, and tears," as the great Winston Churchill said, invested in this project. For me, it took months to make and was fun (mostly) all the way.

Backstory: My friend has autism and was having problems concentrating. I decided I wanted to do something about it, so I made these, and now he can take them in the bathroom and calm down in seconds. It has worked, and he thanked me many times for my effort! You can make this for a friend, or just make it for yourself and change it however you want!

Good luck!

- James F, teaching and technology obsessive