Repository navigation
v0.1.15
Release Notes - v0.1.15
This release introduces real-time voice mode subtitles, dynamic word-by-word fade-in animations, text formatting cleanup, server-side caching, and persistent question/answer display.
What's New
🎤 Real-Time Subtitles & Dynamic Word-by-Word Timings
-
Dynamic Timing Calculation: Subtitle words are dynamically highlighted in perfect sync with the audio. Rather than relying on hardcoded speed averages or failing on streaming
Infinitydurations, timings are calculated on the backend using the exact audio sample counts from the Kokoro pipeline:
$$\text{Duration} = \frac{\text{sample count}}{24000}$$
The duration is linearly distributed across the words in the corresponding text segment to produce exact start/end timestamps. -
Micro-Animations: Renders words as individual spans, transitioning spoken words to
opacity-100 translate-y-0 scale-100 font-mediumand keeping future words hidden atopacity-0 translate-y-2 scale-95. This creates a smooth slide-up and fade-in typing effect as the assistant speaks. - Transparent Layout: Subtitles are displayed in a borderless, shadowless, and fully transparent container for a cleaner floating aesthetic.
🧠 Dedicated Voice Chat & Prompt Formatting
- Strict Formatting Prompts: Created a dedicated
POST /voice-chatendpoint that uses a customized, voice-mode system prompt instructing the AI not to generate markdown bolding (*), hashes, dashes, separators, or emojis. - Text Cleaning Pipeline: Added a Python regex cleaner on the backend to filter out any stray formatting symbols or emojis, ensuring completely clean spoken output.
- Persistent Text Rendering: Spoken prompts (questions) and AI answers remain visible on the screen after the AI finishes speaking (when the avatar returns to
idle), instead of disappearing instantly. - New Session Reset: Old questions and answers are cleared strictly when the user holds the halo to start a new speaking query.
⚡ Server-Side Audio Caching
- Cached WAV Playback: The backend caches the generated WAV audio file locally during the
/voice-chatcall. When the browser makes the subsequent GET/ttsrequest, the server immediately streams the cached file, resulting in zero generation latency at the moment of playback. - Git Ignore: Ignored the
backend/cache/directory in.gitignoreto prevent caching artifacts from cluttering commits.