Automate matched real-call RTCP experiments

[engkimo]

Summary

• attach the read-only Asterisk AMI RTCP collector automatically to each provider-backed AudioSocket run
• record a safe experiment condition in run metadata for matched comparisons
• stop collector tasks before completing the associated run and report collector failures without failing the voice call
• document a repeatable no-interruption versus intentional-barge-in workflow, including caller-side reference recording and single-call attribution limits

Why

The first real Gemini call exposed barge-in evidence and an AGC transient, but it contained no RTP/RTCP observations. That made packet loss neither provable nor excludable. The previous manual collector workflow also required copying a run ID during an active call, which made the evidence gap easy to reproduce accidentally.

This change makes the transport evidence opt-in with one flag and binds it to the correct run lifecycle automatically.

Validation

• ruff check .
• pytest -q — 323 passed, 4 skipped
• PostgreSQL integration — 4 passed
• npm --prefix web run build
• npm --prefix web audit --audit-level=high — 0 vulnerabilities
• local Asterisk health and read-only AMI authentication probe
• git diff --check

Operator note

Automatic AMI collection supports one active test call at a time. The privacy-safe adapter deliberately discards Channel, address, and SSRC fields, so concurrent aggregate RTCP events cannot be attributed safely. Exact RTP sequence-gap proof still requires the library packet-observation adapter.

Summary by CodeRabbit

• New Features
  • Added optional RTCP collection for realtime calls, with automatic attachment and cleanup during bridge runs.
  • Added experiment-condition labels for matched no-interruption and intentional-barge-in calls.
  • Added configurable AMI connection settings and direct command-line support.
• Documentation
  • Expanded RTCP setup, attribution guidance, troubleshooting, and recording recommendations.
  • Clarified that standalone RTCP collection remains available.
• Bug Fixes
  • Improved validation and reporting for invalid experiment conditions and collection failures.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: b3f873cb-27a5-4dfc-9dee-2725a5c42627

📥 Commits

Reviewing files that changed from the base of the PR and between c2c5abd and 09ba8b0.

📒 Files selected for processing (8)

• README.md
• docs/demo-live-softphone.md
• scripts/asterisk-local
• src/voxbench/cli/main.py
• src/voxbench/live_demo/observed_run.py
• src/voxbench/telephony/audiosocket.py
• tests/test_asterisk_local_example.py
• tests/test_audiosocket.py

---

📝 Walkthrough

Walkthrough

The realtime AudioSocket CLI now supports experiment-condition labels and optional AMI RTCP collection. Collector tasks attach to call sessions and stop during shutdown. Documentation describes matched call experiments, attribution limits, configuration, and troubleshooting.

Changes

RTCP collection and experiment attribution

Layer / File(s)	Summary
Experiment condition metadata src/voxbench/live_demo/observed_run.py, src/voxbench/cli/main.py, tests/test_audiosocket.py	Experiment-condition aliases are normalized and validated. Valid conditions are added to observed-run notes and tags.
Bridge-managed RTCP lifecycle scripts/asterisk-local, src/voxbench/cli/main.py, src/voxbench/telephony/audiosocket.py, tests/*	The launcher forwards --collect-rtcp and AMI defaults. The CLI starts collectors and passes their tasks to sessions. Session shutdown cancels and awaits those tasks.
RTCP workflow documentation README.md, docs/demo-live-softphone.md	The documentation describes automatic collection, direct configuration, standalone collection, matched experiments, attribution limits, and troubleshooting.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Operator
  participant audiosocket_realtime
  participant RealtimeCallSession
  participant AMI_RTCP_collector
  participant observed_run
  Operator->>audiosocket_realtime: enable --collect-rtcp
  audiosocket_realtime->>AMI_RTCP_collector: start and bind after provider connection
  audiosocket_realtime->>RealtimeCallSession: pass background_tasks
  AMI_RTCP_collector->>observed_run: record and flush RTCP observations
  RealtimeCallSession->>AMI_RTCP_collector: cancel and await on close
  RealtimeCallSession->>observed_run: complete or fail run

Loading

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch agent/reproducible-real-call-experiment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}