Skip to content

v0.1.15

Choose a tag to compare

@AXEUS5346 AXEUS5346 released this 26 Jun 09:59
· 45 commits to main since this release

Release Notes - v0.1.15

This release introduces real-time voice mode subtitles, dynamic word-by-word fade-in animations, text formatting cleanup, server-side caching, and persistent question/answer display.

What's New

🎤 Real-Time Subtitles & Dynamic Word-by-Word Timings

  • Dynamic Timing Calculation: Subtitle words are dynamically highlighted in perfect sync with the audio. Rather than relying on hardcoded speed averages or failing on streaming Infinity durations, timings are calculated on the backend using the exact audio sample counts from the Kokoro pipeline:
    $$\text{Duration} = \frac{\text{sample count}}{24000}$$
    The duration is linearly distributed across the words in the corresponding text segment to produce exact start/end timestamps.
  • Micro-Animations: Renders words as individual spans, transitioning spoken words to opacity-100 translate-y-0 scale-100 font-medium and keeping future words hidden at opacity-0 translate-y-2 scale-95. This creates a smooth slide-up and fade-in typing effect as the assistant speaks.
  • Transparent Layout: Subtitles are displayed in a borderless, shadowless, and fully transparent container for a cleaner floating aesthetic.

🧠 Dedicated Voice Chat & Prompt Formatting

  • Strict Formatting Prompts: Created a dedicated POST /voice-chat endpoint that uses a customized, voice-mode system prompt instructing the AI not to generate markdown bolding (*), hashes, dashes, separators, or emojis.
  • Text Cleaning Pipeline: Added a Python regex cleaner on the backend to filter out any stray formatting symbols or emojis, ensuring completely clean spoken output.
  • Persistent Text Rendering: Spoken prompts (questions) and AI answers remain visible on the screen after the AI finishes speaking (when the avatar returns to idle), instead of disappearing instantly.
  • New Session Reset: Old questions and answers are cleared strictly when the user holds the halo to start a new speaking query.

⚡ Server-Side Audio Caching

  • Cached WAV Playback: The backend caches the generated WAV audio file locally during the /voice-chat call. When the browser makes the subsequent GET /tts request, the server immediately streams the cached file, resulting in zero generation latency at the moment of playback.
  • Git Ignore: Ignored the backend/cache/ directory in .gitignore to prevent caching artifacts from cluttering commits.