Made by: Stanley Wijaya, Liu Ting Yu, Chua Qing Wei Ryan
Deaf individuals face significant barriers in daily life, i.e. from accessing education and services without an interpreter to experiencing social isolation due to everyday communication gaps. Additionally, learning sign language remains costly, difficult, and lacking in real-time feedback. Our product addresses this with a one-handed ASL learning platform featuring a sensor-equipped glove, real-time AI recognition powered by Ultra96 V2 FPGA board, and an intuitive web application that makes learning sign language interactive, affordable, and accessible to everyone.
Contains the full ML pipeline, such as data preprocessing and calibration, model training, model testing for both software and hardware implementation, hyperparameter tuning, and HLS C++ export for deployment on the Ultra96 V2 FPGA via PYNQ.
Handles all communication between the Ultra96 V2 board, gloves and web application.
Handles raw flex sensor and IMU data readings which are packaged and transmitted to the rest of the system.
Contains the web application built for displaying real-time gesture recognition feedback, tracking user progress, and providing an interactive interface.