A Raspberry Pi and Python powered robot that plays Stick Hero, a mobile game. It was created by me and 4 of my classmates for one of our courses. We used the NumPy, Pillow, GPIOZero, and PiCamera libraries.
A demo of the robot playing the game.
The robot consists of 4 main components, two hardware and two software. The hardware components are the motor and the camera. The software components are the image processing and the game logic calculation scripts. The camera, controlled by a Python script using the PiCamera library, takes a picture of the game running on the device below it at regular intervals. This raw image is sent to the image processing script as a Pillow Image object. The image processor then crops the image down to only include the phone screen. The cropped image is turned into a NumPy array of RGB tuples which the processor uses to find the distance between the current and target platforms.
Before (left) and after (right) image processing:
The distance in platforms is returned by the image processor and then fed to the game logic calculation script. The game logic script returns the duration (in seconds) that the robot should press down on the screen for in order to successfully cross the gap. It does this by dividing the distance by an experimentally determined constant. Finally this time is sent to the motor control script, which uses the GPIOZero library. The motor control script holds the arm down for the duration calculated, and the result is the robot successfully crossing the gap.
NumPy: https://numpy.org/doc/stable/reference/index.html
Pillow (PIL Fork): https://pillow.readthedocs.io/en/stable/
GPIOZero: https://gpiozero.readthedocs.io/en/stable/
PiCamera: https://picamera.readthedocs.io/en/release-1.13/