upgrade deepgram to the latest models - nova-3 20251210; enable custom vocabulary

[beastoin]

upgrade deepgram to the latest models - nova-3 20251210, including 33 languages, keyword/keyterm support, and enhanced transcription/diarization quality

nova-2 now handles 4 languages that are missing from nova-3, defaulting to nova-3

enable custom vocabulary on stt

demo:

omi     | connect_to_deepgram_with_backoff
omi     | {                                                                                               
omi     |     "channels": 1,
omi     |     "diarize": true,                                                                            
omi     |     "encoding": "linear16",
omi     |     "endpointing": 300,
omi     |     "filler_words": false,
omi     |     "interim_results": false,
omi     |     "keyterm": [
omi     |         "omi",
omi     |         "openai",
omi     |         "Omi"
omi     |     ],
omi     |     "language": "en",
omi     |     "model": "nova-3",                                                                          
omi     |     "multichannel": false,
omi     |     "no_delay": true,
omi     |     "punctuate": true,
omi     |     "profanity_filter": false,
omi     |     "sample_rate": 16000,                                                                       
omi     |     "smart_format": true
omi     | }

omi     | connect_to_deepgram_with_backoff
omi     | {
omi     |     "channels": 1,
omi     |     "diarize": true,
omi     |     "encoding": "linear16",
omi     |     "endpointing": 300,
omi     |     "filler_words": false,
omi     |     "interim_results": false,
omi     |     "keyterm": [
omi     |         "omi",
omi     |         "openai",
omi     |         "Omi"
omi     |     ],
omi     |     "language": "multi",
omi     |     "model": "nova-3",
omi     |     "multichannel": false,
omi     |     "no_delay": true,
omi     |     "punctuate": true,
omi     |     "profanity_filter": false,
omi     |     "sample_rate": 16000,
omi     |     "smart_format": true
omi     | }

deploy:

• deploy deepgram self-hosted 20251210 Update Deepgram images, helm chart and Nova-3 models #3753
• deploy backend

#3720

[gemini-code-assist]

Code Review

This pull request updates the Deepgram STT integration by limiting the vocabulary passed as keywords to the first 100 items in backend/routers/transcribe.py. In backend/utils/stt/streaming.py, it refactors Deepgram language support by removing a generic deepgram_languages list, introducing deepgram_nova2_languages for specific non-multi-language support, expanding deepgram_nova2_multi_languages and deepgram_nova3_multi_languages to include more English variants, and populating the deepgram_nova3_languages list with a comprehensive set of supported languages. The changes also include updating the language check for nova-2 models, changing the fallback STT service from Deepgram Nova-2 to Nova-3, and re-enabling the passing of keywords to Deepgram connections. Review comments suggest converting deepgram_nova2_multi_languages and deepgram_nova3_languages from lists to sets for improved O(1) lookup performance in the get_stt_service_for_language function, which is on a hot path, and also recommend importing Set from typing and considering similar changes for other language lists.

[beastoin]

--

nova-2 + nova-3 support:

Count Description
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
37 Unique languages
49 Unique language codes

Nova-3 only: 33 languages Nova-2 adds: 4 additional languages (Chinese Simplified, Chinese Traditional,
Cantonese, Thai)

Here are the 4 languages unique to nova-2:

• Chinese (Mandarin, Simplified)
• Chinese (Mandarin, Traditional)
• Chinese (Cantonese, Traditional)
• Thai