Seeing with the Hands

This repository contains the source code used in the research publication "Seeing with the Hands" from the Human-Computer Integration Lab at the University of Chicago.

Programs (in this repository)

• Arduino_ESP32: Arduino code for the ESP32 that controls the stimulator and switching board.
• Processing_Electrotactile_Calibration_GUI: Processing graphic user interface for calibration of stimulation intensity.
• Python_Main_Webcam: the main Python program that runs the camera (OpenCV) and write Serial commands to ESP32 for outputting electrotactile stimulation (need to change ser.port accordingly).

Hardware

• Electrotactile toolkit
• Switching board & Flexible electrodes
• USB webcam

Citing

When using or building upon this device in an academic publication, please consider citing as follows:

Shan-Yuan Teng, Gene S-H Kim, Xuanyou Liu, and Pedro Lopes. 2025. Seeing with the Hands: A Sensory Substitution That Supports Manual Interactions. In CHI Conference on Human Factors in Computing Systems (CHI ’25), April 26–May 01, 2025, Yokohama, Japan. ACM, New York, NY, USA, 14 pages. https://doi.org/10.1145/3706598.3713419

Contact

For any questions about this repository, please contact tengshanyuan@uchicago.edu