fix(plugins/sarvam): thread language_probability into SpeechData.confidence

[hashirventhodi]

Summary

Fixes #5829. The livekit-plugins-sarvam STT silently dropped Sarvam's language_probability and reported confidence=1.0 everywhere. This wires the real value through both the REST and WS paths.

Changes

• livekit-plugins/livekit-plugins-sarvam/livekit/plugins/sarvam/stt.py:

  • REST recognize(): parse language_probability from response JSON; thread into SpeechData(confidence=...).
  • WS _handle_transcript_data(): parse language_probability from transcript_data; thread into SpeechData(confidence=...).
  • Both sites: isinstance(_, (int, float)) guard with 1.0 fallback so callers don't crash on null/missing/string values. logger.debug on unexpected types so contract drift is visible.

• livekit-plugins/livekit-plugins-sarvam/tests/test_language_probability.py (new):

  • Asserts SpeechData.confidence == 0.87 for a mocked WS payload containing language_probability: 0.87.
  • Asserts fallback to 1.0 when the field is absent, null, or has a wrong type.
  • REST test stubbed (mocks aiohttp session — same isinstance-guard logic as WS).

Validation

Tested in a downstream voice assistant via a vendored fork carrying these exact patches: worker logs show confidence varying across utterances rather than the previous flat 1.0. The defensive fallback handles missing/null cases without crashing.

Notes / Open question for review

• language_probability is documented for Sarvam's REST batch endpoint. Open question (see plugins/sarvam: STT hardcodes confidence=1.0; should thread Sarvam's language_probability into SpeechData #5829): is it guaranteed present on Saaras v3 WS chunks, or best-effort? The defensive isinstance + 1.0 fallback makes the PR safe under either answer, but if it's guaranteed, the fallback path becomes dead code we could tighten later.
• @dhruvladia-sarvam — would appreciate confirmation on field stability so we know whether to keep or remove the fallback in a follow-up.
• Defensive fallback to 1.0 preserves backward behaviour for callers that read confidence defensively.
• No public API changes.
• CLA: will be signed before merge (first-time contributor).

[CLAassistant]

All committers have signed the CLA.

[hashirventhodi]

Thanks @devin-ai-integration — addressed all four findings in eb7a39af:

🔴 P1 (tests): Rewrote test_language_probability.py to:

• Target SpeechStream._handle_transcript_data (not STT — that was just wrong).
• Use async def so pytest-asyncio (mode=auto, already in pyproject.toml) awaits the call.
• Drop the bogus request_id="test" kwarg — method signature is (self, data: dict).
• Use the real WS payload shape: outer {"type": "data", "data": {...}} with nested transcript / language_code / language_probability / speech_start / speech_end / metrics / request_id.

14 tests pass locally:

• 5 happy-path values (0.0, 0.123, 0.5, 0.87, 1.0) thread through to SpeechData.confidence.
• missing / None / wrong-type (str, list, dict, object) fall back to 1.0.
• 3 out-of-range values pass through verbatim (not clamping at this layer).

🟡 P2 (REST logger): Switched logger.debug → self._logger.debug to match every other log in _recognize_impl and the equivalent WS site at stt.py:1505.

Apologies for the busted first cut — the original tests were written before I had access to read the real method signature and silently never executed. Fresh CI run should reflect the green local result.

[davidzhao]

let's refactor this code instead of duplicating it in both places.

[hashirventhodi]

@davidzhao thanks — refactored in 3493a50b (pushed). Both call sites now read confidence=_extract_confidence(payload, self._logger); the parse + isinstance-guard + 1.0 fallback + drift-log live in one module-level helper. 14 tests still pass.

[hashirventhodi]

@devin-ai-integration good catch — fixed in 0b1fe91b. Guard now reads isinstance(value, (int, float)) and not isinstance(value, bool), matching the livekit-plugins-slng pattern you cited. Added True/False to the parametrize'd bad-value test (16 cases pass).

[dhruvladia-sarvam]

Thank you @davidzhao @hashirventhodi
For your context, this was upcoming this week wrt to recent updates to our STT model, as language confidence is only populated with the relevant custom value when language code is unknown.