ASL-Classifier

Real-Time Hand Sign Recognition System

Overview

This project is a computer vision-based hand sign recognition system that uses OpenCV, CVZone, and a TensorFlow/Keras classification model to recognize hand gestures in real time.

The system consists of two main stages:

1. Data Collection – Capturing and organizing custom hand sign images for training.
2. Real-Time Classification – Using a trained machine learning model to detect and classify hand signs from a webcam feed.

The current model is trained to recognize the hand signs A, B, and C, but additional gestures can be added by collecting more data and retraining the model.

---

Features

• Real-time hand detection using CVZone.
• Webcam-based hand sign classification.
• Custom dataset collection.
• Image preprocessing and normalization.
• TensorFlow/Keras model integration.
• Live prediction display with bounding boxes.

---

Technologies Used

• Python 3.11
• OpenCV
• TensorFlow / Keras
• CVZone Hand Tracking Module
• NumPy

---

Project Structure

Hand-Sign-Recognition/
│
├── Data/
│   ├── A/
│   ├── B/
│   └── C/
│
├── Model/
│   ├── keras_model.h5
│   └── labels.txt
│
├── dataCollection.py
├── test.py
└── README.md

---

File Descriptions

dataCollection.py

This script is used to collect the custom dataset required for training the hand sign classification model.

The program uses the webcam to capture hand images and saves them into a selected class folder. The user can collect images for different hand signs by changing the folder path in the script.

The workflow:

1. Opens the webcam feed.
2. Detects the user's hand using CVZone's Hand Tracking module.
3. Extracts and crops the detected hand region.
4. Presses s to save the current hand image.
5. Stores the image inside the selected class folder.

Example:

Data/
│
├── A/
│   ├── Image_1.jpg
│   ├── Image_2.jpg
│
├── B/
│   ├── Image_1.jpg
│
└── C/
    ├── Image_1.jpg

To collect images for a specific hand sign, update the folder path:

folder = 'Data/A'

For example:

folder = 'Data/B'

will save captured images as training data for the B hand sign.

The collected dataset is then used to train the TensorFlow/Keras classification model.

---

test.py

This script performs real-time hand sign classification using the trained model.

The workflow:

1. Opens the webcam feed.
2. Detects a single hand using CVZone.
3. Crops the hand region from the frame.
4. Resizes and normalizes the image into a 300×300 format.
5. Passes the processed image into the TensorFlow/Keras model.
6. Displays the predicted hand sign in real time.

---

Model Directory

The Model folder contains the trained classification model and label mappings.

Model/
│
├── keras_model.h5
└── labels.txt

keras_model.h5

Contains the trained neural network used for classifying hand signs.

labels.txt

Maps the model output indexes to their corresponding labels.

Example:

0 A
1 B
2 C

---

Environment Setup

Create a Virtual Environment

python -m venv venv

Activate the environment:

Windows

venv\Scripts\activate

macOS/Linux

source venv/bin/activate

---

Install Dependencies

Install the required packages:

pip install opencv-python
pip install tensorflow
pip install cvzone
pip install numpy

---

Running the Project

1. Collect Dataset Images

Run:

python dataCollection.py

Change the folder path inside the script depending on the hand sign being collected:

folder = 'Data/A'

Press s while showing the desired hand sign to save images.

---

2. Train the Model

Train a TensorFlow/Keras classification model using the collected images.

After training, place the generated files inside:

Model/
│
├── keras_model.h5
└── labels.txt

---

3. Run Real-Time Classification

Start the recognition system:

python test.py

The webcam will open and display the predicted hand sign.

Press q to exit.

---

System Pipeline

              Webcam
                 |
                 v
       Hand Detection (CVZone)
                 |
                 v
       Dataset Collection (s key)
                 |
                 v
        Model Training
                 |
                 v
     TensorFlow/Keras Classification
                 |
                 v
       Real-Time Sign Prediction

---

Future Improvements

• Expand recognition to the full sign language alphabet.
• Add dynamic gesture recognition.
• Improve accuracy with larger datasets.
• Implement word and sentence prediction.
• Deploy as a web or mobile application.

---

License

This project is intended for educational and research purposes.