How to Build a Neural Network to Translate Sign Language into English

Real-Time Sign Language Translation using Computer Vision

This repository includes all source code for the (soon-to-be) tutorial on DigitalOcean with the same title, including:

• A real-time sign language translator based on a live feed.
• Utilities used for portions of the tutorial, such as dataloaders.
• Simple convolutional neural network written in PyTorch, with pretrained model.

created by Alvin Wan, November 2019

Getting Started

For complete step-by-step instructions, see the (soon-to-be) tutorial on DigitalOcean. This codebase was developed and tested using Python 3.6. If you're familiar with Python, then see the below to skip the tutorial and get started quickly:

(Optional) Setup a Python virtual environment with Python 3.6.

1. Install all Python dependencies.

pip install -r requirements.txt

2. Navigate into src.

cd src

3. Launch the script for a sign language translator:

python step_5_camera.py

How it Works

See the below resources for explanations of related concepts:

• "Understanding Least Squares"
• "Understanding Neural Networks"

Acknowledgements

These models are trained on a Sign Language MNIST dataset curated by tecperson, as published on Kaggle.