Skip to content

Fix crash in STT/TTS latency calc on TurnMetricsData entries#72

Merged
gabegma merged 1 commit intomainfrom
ggm/fix-latency-issue
Apr 23, 2026
Merged

Fix crash in STT/TTS latency calc on TurnMetricsData entries#72
gabegma merged 1 commit intomainfrom
ggm/fix-latency-issue

Conversation

@gabegma
Copy link
Copy Markdown
Collaborator

@gabegma gabegma commented Apr 23, 2026

Summary

  • Latency calculators in ConversationWorker crashed the entire pass (returning None for STT/TTS latency) whenever pipecat_metrics.jsonl contained a TurnMetricsData entry, because Pipecat's smart-turn detector writes a dict-shaped value ({is_complete, probability, e2e_processing_time_ms}) and the numeric range check 0 < value_sec < N ran before the type filter.
  • Reordered both _calculate_stt_latency and _calculate_tts_latency to filter by type / processor / stage first, then validate value is numeric via isinstance before multiplying.
  • Wrapped per-line parsing in try/except so a single malformed entry no longer aborts the whole file. On skip we log the line number, the value and its Python type, the metric.type, and the truncated raw JSON — so future shape surprises surface as warnings instead of silently losing all latency data.
  • Upgraded the outer catch to logger.exception for a traceback if something unexpected does slip through.

Symptom that triggered this

Failed to calculate TTS latency: '<' not supported between instances of 'int' and 'dict'

Raw offending entry:

{"type": "TurnMetricsData", "processor": "BaseSmartTurn", "value": {"is_complete": false, "probability": 0.028, "e2e_processing_time_ms": 51.2}}

Test plan

  • Run a benchmark whose pipecat_metrics.jsonl contains TurnMetricsData entries and confirm latency.stt / latency.tts are populated in the result instead of null.
  • Confirm no regression on runs that only contain ProcessingMetricsData / TTFBMetricsData / LatencyMetric entries — latency numbers should match prior runs.
  • If a malformed entry is introduced, verify the warning log now prints line number, value type, and raw JSON and processing continues.

🤖 Generated with Claude Code

Pipecat's smart-turn detector writes TurnMetricsData entries to
pipecat_metrics.jsonl whose `value` field is a dict, not a number. The
latency calculators ran the numeric range check `0 < value_sec < N`
before filtering by metric type, so the dict comparison raised
TypeError and aborted the whole file — returning no latencies at all.

Reorder the checks so type/processor/stage filtering runs first, then
validate `value` is numeric via isinstance. Also wrap per-line parsing
in try/except so one malformed entry no longer discards the rest, and
log the offending entry (line number, value type, raw JSON) for
future diagnosis.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@gabegma gabegma self-assigned this Apr 23, 2026
@gabegma gabegma requested a review from katstankiewicz April 23, 2026 00:23
@gabegma gabegma added this pull request to the merge queue Apr 23, 2026
Merged via the queue into main with commit bd0c350 Apr 23, 2026
1 check passed
@gabegma gabegma deleted the ggm/fix-latency-issue branch April 23, 2026 18:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants