🌸 Vaani — Voice Made Visible
Vaani is an accessibility-focused Android application that combines real-time speech transcription, Indian Sign Language (ISL) interaction, and AI-powered accessibility workflows to create a more inclusive communication experience.
The project leverages Generative AI concepts, speech recognition, and MediaPipe-based computer vision technologies to help users interact more effectively during conversations and accessibility-driven scenarios.
⸻
✨ Features
🎤 Real-Time Speech Transcription
- Converts live speech into real-time text captions
- Continuous speech recognition support
- Responsive and lightweight Android implementation
⸻
💾 Offline Transcript Management
- Save transcripts locally on the device
- View latest conversations
- Clear transcript history
- Offline accessibility support
⸻
🔔 Notification-Based Accessibility Controls
Manage transcription sessions directly through Android notifications.
Supported controls include:
- Stop transcription
- Mute transcription
- Quick accessibility access
This improves usability during multitasking and live interactions.
⸻
🤟 Indian Sign Language (ISL) Interaction
Vaani includes MediaPipe-powered hand tracking and gesture interaction support for Indian Sign Language accessibility workflows.
The application uses real-time camera processing to support gesture-based interaction and accessibility-focused communication.
⸻
✨ Teach Sign Feature
The application also includes a Teach Sign feature for interactive sign visualization and learning support.
⸻
🛠️ Technologies Used
Android Development
- Android Studio
- Java
- XML
AI / Generative AI Technologies
- Speech Recognition APIs
- Real-Time Transcription Workflows
- Accessibility-Focused AI Interaction
Computer Vision
- MediaPipe
- Hand Tracking
- Gesture Processing
⸻
🚀 How It Works
- Launch the application
- Grant microphone and camera permissions
- Start live transcription
- View real-time captions
- Save and manage transcripts
- Use ISL interaction mode
- Manage sessions through notification controls
⸻
📱 Screenshots
(Add screenshots here)
Suggested screenshots:
- Home Screen
- Live Transcription
- Notification Controls
- ISL Interaction Mode
- Teach Sign Feature
⸻
🎯 Problem Statement
Communication accessibility remains a challenge for many individuals during everyday interactions, especially for users with hearing or speech-related difficulties.
Vaani aims to improve accessibility by combining:
- speech recognition,
- computer vision,
- Indian Sign Language interaction,
- and Generative AI-powered workflows
into a practical Android application.
⸻
🧠 Challenges Faced
Some major challenges during development included:
- handling continuous speech recognition,
- optimizing Android background services,
- reducing transcription latency,
- integrating real-time gesture tracking,
- and balancing accessibility with mobile performance.
⸻
🔥 Key Highlights
- Accessibility-focused Android application
- Real-time speech transcription
- Offline transcript support
- MediaPipe-powered gesture interaction
- Indian Sign Language accessibility features
- Notification-level accessibility controls
- Generative AI-oriented interaction workflows
⸻
🌱 Future Improvements
Planned future enhancements include:
- multilingual transcription,
- improved ISL recognition accuracy,
- AI-powered conversation summarization,
- cloud synchronization,
- and enhanced accessibility customization.
🎥 Demo Video
(https://youtu.be/0nIeDy_3_RM?si=FMGj31U9KHPo8j16)
⸻