touch-gamepad

Touch Gamepad is an ESP-IDF 6.x firmware for ESP32-S3 touch displays that turns touchscreen gestures into gamepad input for a PC or console host. It renders a gesture surface with LVGL, reads a capacitive touch panel, and reports the resulting buttons and joystick axes over BLE HID or (on capable boards) USB HID.

The firmware presents itself to the host as a Microsoft Xbox Wireless Controller (VID 0x045E, PID 0x0B13), so it is recognised and mapped like a standard Xbox gamepad. The input report carries the genuine Xbox fields plus a trailing Rx/Ry axis pair, which hosts that read the HID report map can use as the right stick (see the mapping sub-menu below); the leading bytes stay byte identical to the genuine controller report. The BLE transport advertises as "Xbox Wireless Controller XXXX", where XXXX is the last two octets of the Bluetooth MAC address as a 4-digit hex number, so several gamepads in radio range can be told apart.

Supported boards

Board	Display	Touch	Transports
Guition ESP32-S3-4848S040	480x480 ST7701 RGB	GT911	BLE only
Waveshare ESP32-S3-Touch-LCD-3.5B (untested)	320x480 AXS15231B QSPI (portrait)	AXS15231B	BLE or USB

Warning: the Waveshare ESP32-S3-Touch-LCD-3.5B support is untested on real hardware. Its pin map, panel and touch bring-up (AXS15231B display and touch, AXP2101 PMIC, TCA9554 I/O expander) and transports come from the vendor board support package and have not been confirmed on a device. Treat this board as experimental. The Guition ESP32-S3-4848S040 is the supported, working target.

The Guition ESP32-S3-4848S040 does not expose a USB device port, so it acts only as a BLE gamepad. The Waveshare ESP32-S3-Touch-LCD-3.5B is intended to act as either a BLE or a USB gamepad, with the active transport chosen from the configuration menu and persisted across reboots, but see the warning above about its untested state.

Reference material for the two boards:

• Guition ESP32-S3-4848S040
  • https://www.guition.com/esp32-display-module/4-inch-esp32s3-display-module
  • https://devices.esphome.io/devices/guition-esp32-s3-4848s040/
  • https://github.com/agillis/esphome-modular-lvgl-buttons
• Waveshare ESP32-S3-Touch-LCD-3.5B
  • https://www.waveshare.com/esp32-s3-touch-lcd-3.5b.htm
  • https://www.waveshare.com/wiki/ESP32-S3-Touch-LCD-3.5B

Gesture model

The screen is split into four tap zones and a joystick area. Their positions depend on the selected screen layout (see the configuration menu):

• 4 buttons on top (default): the four tap zones fill the top third in a 2x2 grid and the joystick fills the bottom two thirds.

• 4 buttons on the sides: two tap zones stack along the left border and two along the right border, and the joystick occupies the whole-height middle column, which takes 70% of the screen width (the two side columns share the remaining 30%). On the Waveshare 3.5B this layout is drawn in landscape (the panel's long edge becomes horizontal); the display and touch are rotated 90 degrees to match. The square Guition panel keeps its native orientation. The 4 buttons on top layout is always drawn in portrait.

• Tap zones

  • Touching a zone presses the gamepad button bound to that zone and lifting the finger releases it, so the button is held for exactly as long as the finger rests on the zone. The zone lights up while it is held.
  • Each zone shows its current assignments: the one-finger button on the left and the two-finger button on the right.

• Joystick area

  • Acts as a one-finger analog joystick with a fixed central point. The joystick tracks a single finger (any additional fingers on the joystick area are ignored). Sliding the finger away from the center drives a joystick axis pair whose values are proportional to the distance from the center, and the axes re-center when the finger lifts.
  • The screen draws the central point and a vector pointing from it to the current touch point.

• Configuration menu unlock gesture

  • Tap the corners in the order lower-left, upper-left, upper-right, lower-right. Repeating the sequence closes the menu (discarding unsaved edits, like Cancel).

Configuration menu

The menu is a full-screen, scrollable list driven entirely by one-finger taps. Each row is a large touch target. Tap a row to select it:

• On a multiple-choice row, tap the left half of the row to rotate to the previous value or the right half to rotate to the next value. The current value is shown between < and > arrows as a reminder.
• On an action row (Buttons and axes, BLE pairing reset, Save, Cancel), a tap runs the action.

If the list is taller than the screen, a one-finger slide up or down scrolls it so every row (including Save and Cancel at the bottom) can be reached. Edits are made on a working copy and only written to NVS when you tap Save; Cancel (or the unlock sequence) leaves the menu without saving.

Main screen rows:

1. Transport (BLE/USB)
   • The Waveshare 3.5B rotates between BLE and USB; the transport is restarted when the edits are saved.
   • Guition stays locked to BLE because the board has no USB device port.
2. Color theme
   • Rotates between blue-on-black and green-on-black. The change is previewed live and persisted on Save.
3. Screen layout
   • Rotates between top (4 buttons on top) and sides (4 buttons on the sides). The tap zones and joystick are repositioned when the edits are saved.
4. Screen updates
   • Rotates between on and off. When set to off, the live gameplay screen (tap-zone highlights, joystick vector and status line) stops being redrawn so the picture stays static; gameplay and HID output are unaffected. The configuration menu still renders so the setting can be turned back on.
5. Joystick sensitivity
   • Rotates through levels 1 to 5. Tapping the right half of the row raises the level and the left half lowers it (clamped, no wrap). Higher values make the analog joystick react more strongly to the same finger movement; level 3 is the neutral default.
6. Buttons and axes
   • Opens the mapping sub-menu (see below).
7. BLE pairing reset
   • Removes the current BLE bond and restarts advertising so a new host can pair. Only meaningful in BLE mode; the action runs immediately.
8. Save / Cancel
   • Save writes every edit to NVS and closes the menu. Cancel discards the edits and closes the menu.

Mapping sub-menu rows:

• Eight Zone N M-finger choice rows, each rotating through the gamepad button bound to that tap (one- and two-finger taps edit separate bindings).
• A Joystick axes choice row that rotates the joystick's axis pair through X/Y, Z/Rz, Rx/Ry, X/Z and Y/Rz. X/Y is the Xbox left stick and Z/Rz the Xbox-native right stick. Hosts that map the pad from the HID report map instead of the fixed Xbox report layout usually expect the right stick on Rx/Ry; pick that pair when Z/Rz is not recognised as the right stick.
• Save returns to the main menu keeping the mapping edits; Cancel returns to the main menu discarding the mapping edits made in the sub-menu.

Firmware architecture

• main/main.c runs the boot sequence and the touch event loop: it polls the touch controller, drives the buttons and the joystick from the live touch state every poll (press, hold and release in real time), and still tracks corner taps to detect the menu unlock sequence and to drive the menu.
• main/touch_gamepad.c / main/touch_gamepad.h hold the board preset table, the gesture-detection state machine, the menu state machine, and the NVS-backed configuration.
• main/boards.h centralizes the per-board display and touch pin assignments.
• main/display.c brings up the panel and the LVGL port: an ST7701 RGB panel on the Guition, or an AXS15231B QSPI panel (portrait) on the Waveshare 3.5B.
• main/touchpad.c wraps the touch controller (GT911 on the Guition, AXS15231B on the Waveshare 3.5B) and returns up to two touch points.
• main/ui.c builds the LVGL screen: the tap zones that light up while pressed and show their button assignments, the joystick surface with a central point and a live vector to the touch point, a status line, and the menu / mapping overlays.
• main/gamepad_hid.h defines the shared Xbox Wireless Controller HID report descriptor and input report struct used by both transports.
• main/gamepad_backend.c caches the logical gamepad state and translates it into the Xbox input report for the active transport.
• main/ble_gamepad.c implements the BLE HID gamepad (Bluedroid + esp_hid).
• main/usb_gamepad.c implements the USB HID gamepad (TinyUSB); it compiles to a stub on boards without USB.

Building

Use ESP-IDF 6.x. The managed component dependencies (LVGL, esp_lvgl_port, the ST7701, GT911 and AXS15231B drivers, and esp_tinyusb) are listed in main/idf_component.yml and fetched automatically.

Guition preset (BLE only)

idf.py -D SDKCONFIG_DEFAULTS="sdkconfig.defaults;sdkconfig.defaults.guition" set-target esp32s3 build

Waveshare 3.5B preset (BLE or USB)

idf.py -D SDKCONFIG_DEFAULTS="sdkconfig.defaults;sdkconfig.defaults.waveshare35b" set-target esp32s3 build

CMake presets are also provided (guition and waveshare35b) that layer the shared sdkconfig.defaults with the matching board overlay.

Hardware pin mapping

All display and touch pins live in main/boards.h. The Guition values come from the community and ESPHome reference designs listed above. The Waveshare ESP32-S3- Touch-LCD-3.5B pins, timing and AXS15231B init sequence come from the official Waveshare board support package. That board carries an AXP2101 PMIC and a TCA9554 I/O expander on the touch I2C bus (SDA/SCL shared with the AXS15231B touch controller); waveshare_board_bringup() powers the display rails through the AXP2101 and releases the panel reset through the TCA9554 before the display and touch controllers are initialized. The panel is driven over QSPI and used in its native 320x480 portrait orientation.

Persistence

Runtime configuration is stored in NVS under the touchgp namespace and survives a reboot:

• selected transport mode
• selected color theme
• selected screen layout
• screen updates enabled or disabled
• eight tap-to-button bindings
• joystick slide axis pair

ASCII-only rule

Every source and configuration file in this repository uses ASCII characters only. This applies to C, header, CMake, Kconfig, YAML, JSON, and Markdown files.

Validation status

This firmware targets physical ESP32-S3 hardware and the ESP-IDF 6.x build system with the managed components above; it is intended to be built and flashed with the commands in the Building section. Because the panel and touch pin maps (especially for the Waveshare board) come from vendor references, confirm them on the target hardware before relying on a build.

The Guition ESP32-S3-4848S040 preset is the validated target. The Waveshare ESP32-S3-Touch-LCD-3.5B preset is untested on hardware: its panel and touch bring-up, PMIC and I/O expander sequence, touch orientation and transports come from vendor references and have not been confirmed on a device, so they should be considered experimental.