v0.8.16

💃 Model

Extracted speech handling into SpeechController and face expression handling into FaceController from ModelController to improve maintainability.

• Improve Blink and FaceExpression handling for various 3D avatars by @lavender-snow in #471
• Refactor Blink setup and validation logic #473
• Refactor ModelController: split face and speech logic #475
• Fix null reference error in FaceController initialization #476

Thank you so much for your contribution, @lavender-snow ! ✨🙏✨

🦜 Dialog

Improved the conversation experience with time-aware dialogue that references real-world timestamps, and barge-in support that stops AI speech when the user interrupts.

• Add periodic timestamp insertion to prompts #489
• Add barge-in support to interrupt AI speech by user voice #484

🎓 LLM

• Allow enabling or disabling each tool via the Inspector #477
• Add thinkingConfig field to Gemini requests #485

🎙️ Speech Listener

Added WebSocket-based streaming speech recognition. Since both VAD and speech recognition are performed server-side, VAD processing load is offloaded from the client. Additionally, by completing speech recognition during the turn-end detection wait time, overall response latency is reduced by several hundred milliseconds.

• Add speaker matching and post-processing to AIAvatarSpeechListener #478
• Support switching multiple SpeechListeners in the inspector #474
• Add AIAvatarKitStreamSpeechListener for real-time streaming speech recognition #483
• Support API key auth for WebSocket STT #486
• Use subprotocols to send headers in browser WebSocket #490

🗣️ Speech Synthesizer

• Add support for Kotodama API as TTS service #480

🍩 Others

• Update architecture overview image in README #470
• Add null checks for Authorization header in API requests #481
• Make volume slider feel natural and intuitive #487
• AIAvatar prefab: add barge-in, audio mixer, fix flags #488

Full Changelog: v0.8.15...v0.8.16