💡 Multi-Language Voice Interface: Automatic Language Detection for Global Non-Technical Users

[github-actions[bot]]

Summary

Support 20+ languages for voice input and avatar speech with automatic language detection — a Spanish-speaking PM just starts talking and the avatar responds in Spanish. Leverage the existing STT/TTS abstraction layer to route to multilingual providers without architecture changes, dramatically expanding TalkTerm's addressable market beyond English-speaking users.

Market Signal

Every major STT/TTS provider now supports 30-70+ languages as of mid-2026: Deepgram Flux Multilingual (36+ languages, built-in end-of-turn detection under 300ms), ElevenLabs Flash v2.5 (70+ languages, sub-500ms latency), and HeyGen Avatar IV (175+ languages). Gemini Live added regional dialect options in Q2 2026. The voice AI market consensus is multilingual-by-default — English-only is now a limitation, not a starting point. Cartesia has achieved sub-150ms TTS latency, making real-time multilingual conversation technically feasible at conversational speed.

User Signal

TalkTerm's PRD targets non-technical knowledge workers (PMs, designers, analysts) but implicitly assumes English. The product vision — 'zero learning curve delegation' — is fundamentally undermined if it only works in one language. The BMAD community is English-speaking, but the broader non-technical user market is global. A PM in São Paulo, a designer in Tokyo, or an analyst in Berlin should be able to speak naturally and have the avatar respond in kind. Language exclusion is the single largest addressable-market barrier for a voice-first product.

Technical Opportunity

The STT/TTS abstraction layer (SpeechToText, TextToSpeech interfaces) was designed from the start for swappable providers — this is the exact extensibility seam. Adding language detection and routing is a provider configuration change, not an architecture change. Deepgram Flux Multilingual includes built-in end-of-turn detection that could simplify TalkTerm's barge-in state machine (Story 3.6), replacing custom silence detection. ElevenLabs Eleven v3 supports 70+ languages with emotion and delivery control. The Claude API itself supports multilingual conversation natively — no translation layer needed between STT output and agent input.

Assessment

Dimension	Score	Rationale
Feasibility	high	Existing abstraction layer designed for this. Provider APIs handle language detection. No architecture changes needed.
Impact	high	Expands addressable market from English speakers (~1.5B) to multilingual knowledge workers globally. Critical for enterprise adoption in non-English markets.
Urgency	med	Phase 2 feature — English-first MVP is fine, but the STT/TTS abstraction layer should be designed with multilingual routing in mind from Epic 3 onward.

Adversarial Review

Strongest objection: Translation quality varies dramatically by language. STT accuracy for low-resource languages may be poor, creating a frustrating experience. Avatar personality and cultural norms differ across languages — a 'friendly colleague' tone in English may feel inappropriate in Japanese business culture.

Rebuttal: Start with the top 10 languages by target market size (English, Spanish, Portuguese, French, German, Japanese, Korean, Mandarin, Hindi, Arabic). Use provider-reported accuracy benchmarks to gate language availability — only enable languages meeting a minimum WER threshold (e.g., <15% word error rate). Cultural adaptation of avatar personality (formality levels, greeting styles, honorifics) is a Phase 3 concern; Phase 2 focuses on language comprehension and response accuracy. The phased approach manages risk while unlocking the majority of the global market.

Suggested Next Step

Benchmark Deepgram Flux Multilingual and ElevenLabs Flash v2.5 accuracy and latency for the top 10 target languages. Design the language detection and routing flow in the STT/TTS abstraction layer interfaces (ensure language is a first-class parameter in SpeechToText.start() and TextToSpeech.speak()). Create a language support matrix with accuracy thresholds and provider recommendations per language.