This repository contains code for a real-time American Sign Language (ASL) recognition application built using Streamlit for the frontend and FastAPI for the backend. The application utilizes MediaPipe for pose estimation and a pre-trained TensorFlow Lite model for ASL classification.
-
main.py: This file contains the Streamlit application code. It allows users to use their webcam or upload a video file for ASL recognition. The application sends video frames to the FastAPI backend for processing and displays the predicted ASL action.
-
app.py: This file contains the FastAPI backend code. It defines routes for processing video frames uploaded by the Streamlit application. The backend performs pose estimation using MediaPipe and predicts ASL actions using a TensorFlow Lite model.
-
mediapipe50.tflite: This file contains the pre-trained TensorFlow Lite model for ASL recognition. It is used by the FastAPI backend to predict ASL actions based on the detected keypoints.
- Python 3.x
- OpenCV
- MediaPipe
- NumPy
- FastAPI
- TensorFlow Lite
- Streamlit
- Clone the repository:
git clone <repository_url>
cd <repository_directory>- Install the dependencies:
pip install -r requirements.txt- Run the FastAPI backend:
uvicorn app:app --reload- Run the Streamlit application:
streamlit run main.py- Access the application in your web browser using the provided URL.
- Upon running the Streamlit application, users can choose between using their webcam or uploading a video file for ASL recognition.
- The application sends video frames to the FastAPI backend, which performs pose estimation and predicts ASL actions based on the detected keypoints.
- Predicted ASL actions are displayed in real-time on the Streamlit interface.
- This project utilizes MediaPipe for pose estimation and TensorFlow Lite for ASL classification.
- Streamlit and FastAPI are used for building the web application and backend API, respectively.