CSC Summer Impactathon Submission
Bridging the communication gap for Deaf and hard-of-hearing students in inclusive classrooms.
In traditional classrooms, Deaf and hard-of-hearing (DHH) students often face significant communication barriers. Interpreters are not always available for every class, and speech-to-text systems rarely offer real-time two-way interaction. As a result, DHH students struggle to ask questions quickly, signal urgency, or stay fully engaged with spoken teacher lectures.
SignBridge is a lightweight, accessible web application designed to create a truly inclusive classroom environment through computer vision and speech AI:
- Real-Time Gesture Translation: Translates American Sign Language (ASL) & standard classroom hand gestures into instant teacher alerts via webcam.
- Instant Teacher Captions: Converts live speech from teachers into real-time on-screen subtitles for students.
- AI Lesson Summaries: Generates automated summaries and transcripts of classroom sessions using Google Gemini AI.
- Interactive Gesture Directory: Displays 15 supported classroom gestures (e.g., Raise Hand, Need Help, Repeat Please, Understand) to help both students and teachers learn essential signs.
- 🖐️ Gesture Recognition — Detects predefined classroom gestures such as Raise Hand, Need Help, Repeat Please, and Understand.
- 🎙️ Live Captions — Converts teacher speech into real-time captions using the Web Speech API.
- 🤖 AI Lesson Summaries — Uses Groq Llama 3.3 70B to turn classroom transcripts into structured summaries.
- 👨🏫 Teacher Dashboard — Shows connected students and their classroom interactions in real time.
- 🧑🎓 Student Workspace — Provides gesture recognition and access to live classroom captions.
- 🔄 Realtime Classroom — Supabase Realtime Presence and Broadcast connect students and teachers across different devices.
- 🌙 Accessible UI — Responsive interface with readable typography, high contrast, and dark/light mode.
- Frontend: HTML5, Vanilla JavaScript, Tailwind CSS, Lucide Icons
- Computer Vision: Google MediaPipe Hands
- Speech: Web Speech API
- Generative AI: Groq — Llama 3.3 70B Versatile
- Realtime: Supabase Realtime — Presence & Broadcast
- Development: Vite, Node.js, Express
AI tools were used during development for brainstorming, implementation assistance, debugging, refactoring, and API integration.
The application itself uses:
- MediaPipe Hands for real-time hand landmark detection.
- Groq Llama 3.3 70B for generating structured lesson summaries.
The final application logic, classroom workflow, gesture recognition behavior, and accessibility-focused UI were customized for the SignBridge project.
SignBridge combines speech recognition, computer vision, realtime communication, and generative AI into one classroom-focused accessibility platform.
It aims to make communication more accessible in inclusive classrooms by creating a two-way bridge:
Teacher Speech → Live Captions → Student
Student Gesture → Recognition → Teacher Alert
- Node.js (v18 or higher)
- A web browser with webcam and microphone permissions enabled (Chrome / Edge recommended)
# 1. Clone the repository
git clone https://github.com/your-username/signbridge.git
# 2. Navigate into directory
cd signbridge
# 3. Install dependencies
npm install
# 4. Start the development server
npm run devOpen your browser at http://localhost:3000 to run the application.
This project is open-source and available under the MIT License.
Submitted for the CSC Summer Impactathon 2026.