STT time-to-first-partial metric (speech onset → first interim), the STT analog of ttft/ttfb

[mahimairaja]

Feature Type

Would make my life easier

Feature Description

STT has no head-latency metric. LLMMetrics has ttft and TTSMetrics has ttfb, but STTMetrics exposes only duration (0.0 for streaming), audio_duration, and streamed. There is no equivalent for "how fast did the first word come back."

Two different STT latencies exist, and only the tail is measurable today:

Interval	Meaning	Status
end-of-speech → FINAL transcript	EOUMetrics.transcription_delay	exists (includes endpointing)
audio-end → FINAL transcript	proposed in #5063 / PR #4966	in review
speech onset → first INTERIM	first-partial responsiveness	missing

The head metric is the one that predicts barge-in and perceived snappiness, and it is endpointing-free. transcription_delay cannot stand in for it: it is dominated by the endpointing/EOU wait, so it moves when you tune silence thresholds even if the STT's first-word speed never changed.

Both operands are already present in AudioRecognition._on_stt_event: the VAD-backdated onset self._speech_start_time, and the INTERIM_TRANSCRIPT branch. The metric is a wall-clock subtraction there, no provider timestamps involved:

# AudioRecognition._on_stt_event, INTERIM_TRANSCRIPT branch
elif ev.type == stt.SpeechEventType.INTERIM_TRANSCRIPT:
    if self._first_interim_at is None and self._speech_start_time is not None:
        first_interim_delay = time.time() - self._speech_start_time

Proposed carrier (leaning EOUMetrics, since that is where it is computed and it stays symmetric with transcription_delay):

first_interim_delay: float
"""Seconds from speech onset (VAD start) to the first interim transcript.
The STT-responsiveness analog of LLMMetrics.ttft / TTSMetrics.ttfb.
Unset / 0.0 when the STT emits no interims (non-streaming or interim_results=False)."""

Alternative placement is STTMetrics.first_interim_latency, but the plugin never sees VAD onset, so it would need framework enrichment anyway. Happy to take direction on which surface you'd prefer.

Why the #5063 end_time-reliability concern does not apply here: this metric never reads a provider end_time. Onset comes from VAD; the first interim's timestamp is just its wall-clock arrival. It is unaffected by the endpointing-accuracy objection that blocks the tail metric.

Workarounds / Alternatives

• EOUMetrics.transcription_delay: measurable today, but it is the tail (end-of-speech → FINAL) and folds in endpointing, so it conflates STT speed with EOU tuning.
• Timing user_input_transcribed at the AgentSession layer: works downstream but adds the session emit hop on top of the real STT latency, so it overstates it and drifts with unrelated scheduling.
• Both are proxies. The value already exists one layer down in _on_stt_event; surfacing it removes the guesswork.

Additional Context

• Not a duplicate of Feature Request: Add utterance_end_latency to STTMetrics for streaming STT #5063 / feat: Add STT utterance_end_latency metric for streaming STT #4966 / Add metric for first transcript after end-of-speech #4795. Those are all the tail (audio/speech end → FINAL). This is the head (speech onset → first interim), and the two are independent numbers.
• The _on_stt_event mechanism above was confirmed by a community member on the LiveKit forum.
• I have this working as a downstream proxy in my own tooling and would be glad to bring a PR (latch + field + test) if the direction and carrier are agreeable.

This thread was first started at Livekit Community

[spyrux]

I'd like to work on this. I plan to add the metric at the first interim STT event, reset it per utterance, expose it through the existing metrics path, and add hermetic unit tests. Does that scope match the intended API?

[IgorGanapolsky]

For STT time-to-first-partial, the reliability trap is treating “metric missing” the same as “metric zero”.

A durable pattern:

1. Emit a per-utterance id at speech onset (before first interim)
2. On first interim: attach that id + wall-clock delta; if STT never produces interim, record an explicit timeout/absent outcome (not silent green)
3. Separate counters: partial_received vs partial_timeout vs stt_error — so dashboards don’t collapse failure into “0ms”

If your path can hang without first partial, fail closed to a terminal status with the utterance id so retries don’t double-count. Happy to map expected vs actual fields if you paste one sanitized event.