Accessible Bluetooth Computer Switch Access

by WSUAssistiveTech in Circuits > Assistive Tech

83 Views, 0 Favorites, 0 Comments

Accessible Bluetooth Computer Switch Access

Bluetooth_Computer_Switch_Access.jpg

This project was designed to satisfy the task of providing our client, a 4 year old child with Cerebral Palsy, with a controller that allows him to control keyboard/mouse inputs with accessibility switches. We met with our client's family and physical therapist to establish the goals of this project, which were:

  1. Connect via Bluetooth
  2. Connect more than two switch inputs
  3. Allow remapping buttons for future use
  4. Visually accessible
  5. Remain within physical reach of our client

Supplies

Charge_Port.jpg
Battery.jpg
Arduino_NanoESP32.jpg
Stereo_Jack_Port.jpg
Velcro.jpg
AbleNet_Button.jpg
Voltage_Boost_Converter.jpg

Materials

Arduino Nano ESP32 or Adafruit Feather nRF52832 Microcontroller

AbleNet Jelly Bean or your own printed Buttons

Navona Velcro Strip (optional, to keep the case secured to a surface)

Micro-Lipo Charger for LiPoly Battery (If you use the Nano ESP32)

Voltage Boost Converter (Same as above)

Rechargeable Battery

Headphone Jack Sockets (These are stereo, but we recommend buying mono jacks instead)

Breadboard (For Testing)

Tools

Soldering Iron & Lead (or any alternative materials)

Hot Glue and Plaster

Scissors

3D Printer

Hardware Setup (testing)

Set_Up_Before_Battery.jpg

We set up a simple circuit with the microcontroller and headphone jack sockets to test if our switches were properly connected. In the Arduino IDE, you can check to see if your switch is connected to the pin by using Serial.println(digitalRead(*PIN*)); in the loop function. It should print 0 in the serial monitor whenever the button is held down.

For our final design, we decided to use the Adafruit Feather nRF52832 microcontroller since it has a built-in battery port and charger. If you know how to properly connect the Micro-Lipo charger and voltage boost converter to the ESP32 Nano, then you can continue with that.

If you would like to 3D print the buttons, we've attached the .stl files needed to create your own.

Downloads

Standard Switch 3 .3mf
Standard Switch 3 Base.stl
Standard Switch 3 Clip.stl
Standard Switch 3 Spring.stl
Standard Switch 3 Top.stl

Software Setup

Arduino IDE Download

The code allows the microcontroller to advertise itself as a Bluetooth keyboard. This will only need a Bluetooth connection to function, though do note that our design only accounts for Bluetooth and not any other type of connection. Each button is assigned to a GPIO pin on the microcontroller, and when the button is pressed, it sends an keyboard/mouse input to the computer. The seventh headphone jack was designed to put the microcontroller into deep sleep to save battery life.

Code for Arduino Nano ESP32

Disclaimer: When we tested this code, it functioned as intended. However, we were unable to develop this code further as our prototype that utilized the Arduino Nano ESP32 microcontroller experienced irreparable damage, so we pivoted to the Adafruit Feather nRF52832 microcontroller.

Bluetooth Keyboard Library: https://github.com/T-vK/ESP32-BLE-Keyboard/releases/tag/0.3.0

*We were using this specific version of this library due to issues with Bluetooth connection with the latest release.

#include <BleKeyboard.h>
#include <BLEDevice.h>
#include <BLEUtils.h>
#include <BLEScan.h>
#include <BLEAdvertisedDevice.h>
#include "driver/rtc_io.h"
#include "esp_sleep.h"

#define BUTTON1 D2
#define BUTTON2 D3
#define BUTTON3 D5
#define BUTTON4 D6
#define BUTTON5 D8
#define BUTTON6 D9
#define BUTTON7 D11
#define KEY_SPACE 0x20

BleKeyboard bleKeyboard(
"Nano ESP32 Keyboard", // Device Name
"Arduino" // Manufacturer
);

int buttonState1 = 1;
int buttonState2 = 1;
int buttonState3 = 1;
int buttonState4 = 1;
int buttonState5 = 1;
int buttonState6 = 1;
int buttonState7 = 1;

void setup() {
Serial.begin(115200);
bleKeyboard.begin();
Serial.println("BLE Keyboard initializing..");
pinMode(BUTTON1, INPUT_PULLUP);
pinMode(BUTTON2, INPUT_PULLUP);
pinMode(BUTTON3, INPUT_PULLUP);
pinMode(BUTTON4, INPUT_PULLUP);
pinMode(BUTTON5, INPUT_PULLUP);
pinMode(BUTTON6, INPUT_PULLUP);
pinMode(BUTTON7, INPUT_PULLUP);
esp_sleep_enable_ext0_wakeup(GPIO_NUM_5, 0);
rtc_gpio_pulldown_dis(GPIO_NUM_5);
rtc_gpio_pullup_en(GPIO_NUM_5);
}

void loop() {
buttonState1 = digitalRead(BUTTON1);
buttonState2 = digitalRead(BUTTON2);
buttonState3 = digitalRead(BUTTON3);
buttonState4 = digitalRead(BUTTON4);
buttonState5 = digitalRead(BUTTON5);
buttonState6 = digitalRead(BUTTON6);
buttonState7 = digitalRead(BUTTON7);

if (bleKeyboard.isConnected()) {
if (buttonState1 == LOW) {
bleKeyboard.write(KEY_SPACE);
Serial.println("Pressed Space");
}
if (buttonState2 == LOW) {
bleKeyboard.write(KEY_TAB);
Serial.println("Pressed Tab");
}
if (buttonState3 == LOW) {
bleKeyboard.write(KEY_RETURN);
Serial.println("Pressed Enter");
}
if (buttonState4 == LOW) {
bleKeyboard.write('a');
Serial.println("Pressed a");
}
if (buttonState5 == LOW) {
bleKeyboard.write('b');
Serial.println("Pressed b");
}
if (buttonState6 == LOW) {
bleKeyboard.write('c');
Serial.println("Pressed c");
}
delay(100);
}
else {
Serial.println("Not connected");
}

if (buttonState7 == LOW) {
Serial.println("Button pressed");
esp_deep_sleep_start();
Serial.println("This line will never be printed.");
}

delay(100);
}


Code for Adafruit Feather nRF52832

#include <bluefruit.h>

BLEHidAdafruit blehid;

#define BUTTON1 A0
#define BUTTON2 A1
#define BUTTON3 16
#define BUTTON4 15
#define BUTTON5 7
#define BUTTON6 A2
#define BUTTON7 A3

// Used the keycodes from the link to define the keys
// https://docs.circuitpython.org/projects/hid/en/latest/api.html#adafruit_hid.keycode.Keycode
#define KEY_SPACE 44
#define KEY_TAB 43
#define KEY_ENTER 40
#define KEY_LEFT_ARROW 80
#define KEY_RIGHT_ARROW 79

// For arrow keys
int arrow_delay = 0;
// For all other buttons and mouse
int normal_delay = 500;

void setup() {
Serial.begin(115200);

pinMode(BUTTON1, INPUT_PULLUP);
pinMode(BUTTON2, INPUT_PULLUP);
pinMode(BUTTON3, INPUT_PULLUP);
pinMode(BUTTON4, INPUT_PULLUP);
pinMode(BUTTON5, INPUT_PULLUP);
pinMode(BUTTON6, INPUT_PULLUP);
pinMode(BUTTON7, INPUT_PULLUP);

// Sets wake-up pin
nrf_gpio_cfg_sense_input(A3, NRF_GPIO_PIN_PULLUP, NRF_GPIO_PIN_SENSE_LOW);

Bluefruit.begin();
Bluefruit.setTxPower(4);
Bluefruit.setName("Bluetooth Switch Controller");

blehid.begin();

startAdv();

Serial.println("BLE Keyboard Started");
}

// Advertises the bluetooth connection
void startAdv() {
Bluefruit.Advertising.addFlags(BLE_GAP_ADV_FLAGS_LE_ONLY_GENERAL_DISC_MODE);
Bluefruit.Advertising.addTxPower();
Bluefruit.Advertising.addAppearance(BLE_APPEARANCE_HID_KEYBOARD);

Bluefruit.Advertising.addService(blehid);

Bluefruit.ScanResponse.addName();

Bluefruit.Advertising.restartOnDisconnect(true);
Bluefruit.Advertising.setInterval(32, 244);
Bluefruit.Advertising.setFastTimeout(30);
Bluefruit.Advertising.start(0);
}

// Sends a keystroke
void sendKey(uint8_t keycode, const char* message) {
uint8_t keycodes[1] = {keycode};
// Presses Key
blehid.keyboardReport(0, keycodes);
// Releases Key
blehid.keyRelease();

Serial.println(message);
}

void loop() {
if (digitalRead(BUTTON1) == LOW) {
sendKey(KEY_SPACE, "Pressed Space");
delay(normal_delay);
}
if (digitalRead(BUTTON2) == LOW) {
sendKey(KEY_TAB, "Pressed Tab");
delay(normal_delay);
}
if (digitalRead(BUTTON3) == LOW) {
sendKey(KEY_ENTER, "Pressed Enter");
delay(normal_delay);
}
if (digitalRead(BUTTON4) == LOW) {
Serial.println("Mouse Clicked");
blehid.mouseButtonPress(MOUSE_BUTTON_LEFT);
delay(normal_delay);
blehid.mouseButtonRelease();
}
if (digitalRead(BUTTON5) == LOW) {
sendKey(KEY_LEFT_ARROW, "Pressed Left Arrow");
delay(arrow_delay);
}
if (digitalRead(BUTTON6) == LOW) {
sendKey(KEY_RIGHT_ARROW, "Pressed Right Arrow");
delay(arrow_delay);
}
if (digitalRead(BUTTON7) == LOW) {
delay(2000);
Serial.println("Entering deep sleep");
blehid.keySequence("Entering deep sleep", 5);
blehid.keyRelease();
// Puts microcontroller into deep sleep
sd_power_system_off();
Serial.println("Shouldn't print");
}
}

Enclosure

Enclosure.jpg

We designed a box with the appropriate openings to allow the charging port and the headphone jack ports to be accessible from the outside while keeping the internal components protected. Our team members possessed little to no experience with CAD or 3D modeling so we could only design a basic box with some support blocks. However, if you have experience with such, you can go ahead and add other details such as standoffs for the microcontroller, supports to keep the headphone jacks stable, or even a more appealing enclosure. Attached here are the .stl files for our enclosure if you would rather not make your own.

Downloads

Box.stl
lid.stl
SupportBlocks.stl

Solder

Soulder_FirstStep.jpg
Soulder_JackPorts.jpg
Soulder_JackPorts_Together.jpg
Complete_Solder.jpg
Arduino_Nano_ESP32.png
Adafruit_Feather_nRF52832_Circuit-Diagram.png

First, we soldered wires into the GPIO pins we decided to use. The pins we used are:

  1. ESP32: D2, D3, D5, D6, D8, D9, D11
  2. nRF52832: A0, A1, 16, 15, 7, A2, A3

We then soldered the other end of the wires to the headphone jacks. The area to be soldered can be seen in the second picture, with an example of the applied solder in the third. (Note: We used stereo jacks. If we could go back in time, we would've used mono jacks as that would have been much more simple.)

Once all of the headphone jacks are soldered, now we need to solder all of the jacks to ground. In the third picture, ground is the silver area on the jack the two wires are touching. We used six different small wires to connect all of the jacks together, i.e., the ground on jack 1 is connected to the ground on jack 2, the ground on jack 2 is connected to the ground on jack 3, and so on. We wouldn't recommend this as it makes the connection unstable. It would be a good idea to connect the ground on all of the jacks to something like a strip of metal.

In the fourth image, you can see an example of our finished product using the Adafruit Feather nRF52832, with all jacks aligned in a row.

In the fifth and sixth images, you can see the schematic diagram for wiring the ESP32 and nRF52832 respectively. As you can see, we connected the sleeve of each port together and connected it to ground. As for the Adafruit Feather nRF42832, pins 16, 15, and 7 correlate to GPIO11, GPIO12, and GPIO13 respectively.

Assembly

End_Result.jpg
Open_Box.jpg

Once everything was soldered, we secured the microcontroller into the enclosure using our supports and glue. As for the headphone jack sockets, we used sealant to secure them in place so they wouldn't push back whenever we plugged in the switch. Along with that, we placed labels above the headphone jack socket holes to show what key each switch is assigned to. As these are label stickers, they can be replaced should any of the buttons be remapped to a different key. For the lid of the container, we used tape to keep it in place as we had not included a way to attach and detach the lid. It would be wise not to glue the lid in case of hardware failure.

Delivery

Delivery_Completion.jpg

On our delivery date, we brought it to our client and showcased it to his PT and teacher. They tested it out with games that our client likes to play, along with the academic curriculum program his school is using, both of which were accessible using the project. As such, they were happy with the project and plan to make use of it during our client's development.