turn-detector-v1: high eot variance on short utterances — guidance on tuning for short affirmations?

[Propconnect-ai]

Bug Description

Context

I run a production voice agent for lead qualification (currently on another platform) and was evaluating LiveKit. While reading the debug logs from two console sessions I noticed something I'd like to understand better before building on it.

This is a question, not a bug report — I may well be misreading the design intent.

What I observed

turn-detector-v1, English, unlikely_threshold: 0.56, no overrides.

The same word, spoken twice, in two sessions — same microphone, same room:

Utterance	eot probability	threshold	endpointing_delay
"Hello?" (session 1)	0.477	0.56	2.5 s
"Hello?" (session 2)	0.650	0.56	0.3 s

Identical input, opposite sides of the threshold, ~8× difference in wait before the agent responded.

Two semantically identical affirmations, same session:

Utterance	eot probability	endpointing_delay
"Yeah."	0.127	2.5 s
"Yes."	0.649	0.3 s

"Yeah." scored 0.127 — the model was ~87% confident the turn had not ended, when it plainly had. It waited 2.5 seconds.

And one that cleared by a rounding error:

Utterance	eot probability	threshold	margin
"I'm not sure."	0.5613	0.560	+0.0013

Why I'm asking

A qualification call is largely short affirmations — yeah, okay, sure, right, correct, no. If the score distribution for those straddles the threshold, a meaningful fraction of turns will fall below it and trigger the full 2.5 s endpointing delay.

On a phone call with a stranger, 2.5 s of dead air after they say "yeah" doesn't read as thoughtfulness — it reads as the line dropping. Callers start talking over the agent or hang up.

I'm not claiming the model is wrong. Short utterances genuinely carry less prosodic and lexical signal, and I don't know what the training distribution looked like. But the variance appears highest on the shortest and most frequent utterances in a real call, which is where it matters most.

Questions

1. Is this variance expected on very short utterances, or does it suggest something wrong with my setup?
2. The config exposes "overrides": null alongside the per-language thresholds. Is per-language (or per-use-case) override the intended lever here? I couldn't find guidance on when or why to reach for it.
3. Is there a recommended pattern for turn-taking in short-answer-heavy flows (qualification, IVR-style, yes/no gating) — a lower threshold, a shorter endpointing_delay, or something else entirely?

Happy to gather more data across languages if that's useful — I run Hindi/English bilingual agents, and I noticed hi: 0.575 is tuned separately, which suggests someone has already thought hard about this.

Expected Behavior

Identical or near-identical utterances should produce comparable eot scores.

"Hello?" scoring 0.477 in one session and 0.650 in the next — same word, same
microphone, same room, same config — puts the same input on opposite sides of the
0.56 threshold and produces an 8x difference in response delay (2.5s vs 0.3s).

I'd expect either:
(a) more stable scoring on short utterances, or
(b) documented guidance on tuning for short-answer-heavy flows, since the variance
appears highest on the utterances most common in real qualification calls.

Reproduction Steps

1. lk app create --template agent-starter-python <name>
2. uv sync
3. uv run src/agent.py console   (no config changes, stock template)
4. Say "Hello?" — note eot probability and endpointing_delay in the debug log
5. End session, restart, say "Hello?" again — compare
6. Within one session, say "Yeah." and then "Yes." — compare

Operating System

Windows 11 (console mode, local)

Models Used

Stock agent-starter-python defaults, all via LiveKit Inference: - STT: deepgram/nova-3 (language: multi) - LLM: google/gemma-4-31b-it - TTS: cartesia/sonic-3 - Turn detection: turn-detector-v1 (thresholds: en 0.56, hi 0.575; overrides: null)

Package Versions

livekit-agents 1.6.5
livekit-rtc 1.1.13
livekit-plugins-ai-coustics 0.3.0

Session/Room/Call IDs

Console mode (local) — mock-job-e675b127a1ef, mock-job-054d4e0446a2
No cloud room IDs. Happy to reproduce in a cloud session if that's more useful.

Proposed Solution

Additional Context

No response

Screenshots and Recordings

No response

[IgorGanapolsky]

This looks less like a single global-threshold problem and more like a calibration problem across language, dialogue state, and the active EOT model.

I would run a paired-repeat test before changing production defaults:

1. Capture a redacted sample of 100–200 utterances, or at least 30 affected calls. Log audio duration, final transcript, detected language, active turn model, EOT probability, active threshold, endpoint delay, last-speech-to-agent-audio time, and whether the caller barged in or hung up.
2. Stratify short acknowledgements separately from full answers, and split English/Hindi plus the qualification state. A global average will hide the exact failure you described.
3. Compare: (A) current baseline; (B) two language-specific threshold candidates around the current value; and (C) a lower maximum endpoint delay only in short-answer qualification states, restoring the normal policy elsewhere. Current LiveKit exposes runtime turn-detector threshold updates and session endpointing options, so this can be state-scoped instead of global: https://docs.livekit.io/agents/logic/turns/turn-detector/ and https://docs.livekit.io/agents/logic/turns/tuning/
4. Set acceptance criteria before rollout: p95 last-speech-to-agent-audio under about 900 ms for short acknowledgements, premature endpoints below 1%, and no regression in barge-ins or hangups.

Also record whether the request used the intended EOT model or a fallback. If identical audio really produces the same transcript but different probabilities under the same pinned model and state, that gives maintainers a much stronger reproducible case than the two-token examples alone.

If you post a redacted 30-call export with those fields, I can help separate model variance from endpoint policy. If this is already causing real production hangups, I also run fixed-scope voice-agent reliability pilots that take this from reproduction through acceptance-tested containment.

[IgorGanapolsky]

Policy before action on voice tools:

1. Capability allowlist at session start (tools/languages/models)
2. Refuse unsupported combos before first media frame
3. Durable session config hash so mid-run swaps do not silently change behavior
4. Expected vs actual: advertised capabilities vs accepted session config

Architecture only.

[Propconnect-ai]

Reporter here. Correcting my own framing, with a larger sample.

livekit-agents 1.6.5, turn-detector-v1, console mode, two sessions, no executor stalls. Twenty short utterances:

Above threshold (commit in 0.3s):

• "Bye." — 0.990
• "Bye bye." — 0.985
• "You." — 0.908
• "Excellent." — 0.791

Below threshold (commit on the 2.5s timeout):

• "One way" — 0.522
• "Yes." — 0.519
• "Fastest." — 0.484
• "1B." — 0.465
• "Yes." — 0.462
• "Morning flights." — 0.400
• "Yes." — 0.382
• "1b." — 0.346
• "Hello?" — 0.283
• "Yes." — 0.245
• "Hi." — 0.194
• "1b" — 0.112
• "Yeah." — 0.111
• "Okay." — 0.028
• "Hello?" — 0.020

Full sentences behave correctly throughout - 0.687, 0.782, 0.799, 0.984, all committing in 0.3s. So neither "variance" nor "short utterances" was the right framing, and my original title is wrong.

What the data does show is a bimodal split with nothing between 0.522 and 0.791. The threshold sits at 0.56, inside that gap. Above it are farewells and closings; below it are answers to the agent's task questions. Every one of those costs the full 2.5s endpointing delay, so a user answering "yes" to "shall I book it?" waits 2.5 seconds.

Repeated tokens are separate recordings, so the spread within them ("Yes." at 0.245–0.519) is presumably intonation, not instability.

Is this the intended separation, with backchannel_threshold as the lever? Or should task answers be scoring higher? Two sessions on one machine, so treat as directional.

[chenghao-mou]

Thanks for reporting the issue. To help us identify the cause, could you share some cloud sessions links (make sure you check the box to allow LiveKit support to view the content)? As the model is audio-based, so audio recording is the best way for us to reproduce and locate the issue.

[Propconnect-ai]

Thanks Chenghao. Reproduced on LiveKit Cloud (1.6.5, turn-detector-v1), support access enabled:

https://cloud.livekit.io/projects/p_19mwyin9l4l/sessions/RM_DBfSGxQqRj3t/observability?mode=transcript

The near-threshold behavior shows across the session — with the English threshold at 0.56, several turns commit right around it (0.451, 0.478, 0.578, 0.581). The audio and synced transcript are on that session.

One thing I saw on Cloud that wasn't in my local runs: a "turn detection transport latency is too high: 583ms" warning (region India West). Not sure if related, flagging in case it's useful.

Also happened to catch the #6504 null-probability case in the same session (late STT final after a turn flush) — I'll add detail on that PR. Happy to share raw EOT logs with timestamps if helpful.