Automated Touch on Display

I built a device that automatically presses the buttons in the game Don’t Press the White Tiles. It uses photoresistor sensors to detect the tiles and transistors to control solenoids that physically press the screen.

Here is the list of required materials: 4 photoresistors, 4 transistors, and 4 solenoids. Additionally, you will need some screws and access to a 3D printer to create a custom mount for your phone (basic CAD skills are recommended).

The first step is to create a basic CAD design based on the specific solenoid model you purchased, so you can properly mount and align them with your phone.

The second step is to understand how a transistor works, as it is essential for this project. The Arduino does not provide enough power to drive the solenoids directly, so transistors are needed to act as switches and control them safely. The provided circuit diagram is very helpful for understanding how to connect the transistors to the Arduino and to the external power source.

The third step is to understand how a photoresistor works. It detects changes in light intensity—distinguishing between brighter and darker areas—which makes it ideal for this application, since the game features white and black tiles that can be easily identified using photoresistors. I also recommend running a calibration (tuning) program beforehand to achieve high accuracy for each individual sensor.

The fourth step is to prepare the solenoids so they have a larger contact surface that is also conductive, allowing them to mimic a human finger and properly interact with the touchscreen.

The fifth step is to connect all the components together and ensure good wire management to prevent interference or tangling. Once everything is properly assembled and organized, you can move on to the coding stage.

The sixth and final step is to develop a functional code that triggers the solenoids whenever the photoresistors detect a change in light intensity. The image includes sample code snippets, as well as the full program I wrote to make the system actually play the game.

In the given pictures you can also find explanations for each piece of code.