Skip to content

agent simulation: enrich Turn metrics#1673

Merged
Bobronium merged 5 commits into
mainfrom
arseny/sim-enrich-turn-metrics
Jul 21, 2026
Merged

agent simulation: enrich Turn metrics#1673
Bobronium merged 5 commits into
mainfrom
arseny/sim-enrich-turn-metrics

Conversation

@Bobronium

@Bobronium Bobronium commented Jul 20, 2026

Copy link
Copy Markdown
Member

Homogenizes Turn/Job/Run LLM/TTS/STT shapes

Turn: flat fields -> stt/llm/tts/conversation groups
      transcription_delay_ms -> stt.transcription_latency_ms

New:
  turns[].conversation.agent_yield_latency_ms      per barge-in; presence = barge-in
  turns[].conversation.turn_taking_score           0-1 per event
  turns[].conversation.{eot_misprediction,
    awkward_silence, unanswered}_count             0/1 per turn; replaces flags
  turns[].llm.tokens_per_second
  Conversation.response_latency_ms                 per-turn slot; unset on job/run
  TTS.heard_words, TTS.speech_ms                   speech_rate pooling stats

optional uint32 agent_reported_e2e_latency_ms = 6;
// persona rows: agent audio continuing past this turn's start; set iff the turn barged in
optional uint32 agent_yield_latency_ms = 7;
repeated string flags = 8; // judge failure tags ("verbosity_or_filler", ...)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's unclear what the flags could be, is it on purpose?

@Bobronium Bobronium Jul 21, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The field was intended to carry information about the turn such as "barge_in", "unanswered". "eot_misprediction", "awkward_silence". Comment was stale by the time of the PR.

Now with conversation message included (it contains dedicated fields for this), it's redundant and removed.

@changeset-bot

changeset-bot Bot commented Jul 20, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 2abd1c8

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages
Name Type
github.com/livekit/protocol Patch
@livekit/protocol Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@Bobronium
Bobronium marked this pull request as ready for review July 21, 2026 08:06
@Bobronium
Bobronium merged commit e91b5cd into main Jul 21, 2026
8 checks passed
@Bobronium
Bobronium deleted the arseny/sim-enrich-turn-metrics branch July 21, 2026 08:28
@github-actions github-actions Bot mentioned this pull request Jul 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants