This repository contains a variety of tools to build up a experimental ecosystem for recognizing signs of the german sign language (DGS). Our claim is an experimental attempt at live subtitling of gestures. For this we train a deep learning model (RNN) for predicting the actualsigns made by a person filmed. Therefore we use MediaPipe, a framework for building ML pipelines, to extract face and hand positions, including multiple coordinates for each finger. The project is to help the society of deprived people who are facing difficulties to speaking and hearing. Python (3.7.4), cv2 (openCV) (version 3.4.2), numpy, cvzone, Jupyter notebook.
The code is written using Anaconda Python >= 3.6 and Pytorch 1.7 with OpenCV. Detailed enviroment requirment can be found in requirement.txt in each code folder. For convenience, we provide a Nvidia docker image to run our code.