Skip to content

docs(examples), test: fix first-run docs and timing flake#5593

Open
shizhigu wants to merge 5 commits intolivekit:mainfrom
shizhigu:codex/examples-first-run-audit
Open

docs(examples), test: fix first-run docs and timing flake#5593
shizhigu wants to merge 5 commits intolivekit:mainfrom
shizhigu:codex/examples-first-run-audit

Conversation

@shizhigu
Copy link
Copy Markdown

@shizhigu shizhigu commented Apr 29, 2026

Summary

This keeps the examples first-run docs accurate and folds in the small AgentSession timing-test stabilization that was already opened as #5597.

The docs part fixes stale example paths and expands examples/.env.example so a new user can see the common LiveKit/provider variables without hunting through every example. The test part fixes the repeated CI failure this PR hit twice in tests/test_agent_session.py::test_tool_call: wall-clock event emission can be a little late under CI scheduler load, but duration/playback-position checks stay strict.

I folded #5597 into this PR so reviewers do not have to reason about a docs PR blocked by a separate test-only PR.

What changed

  • Expanded examples/.env.example with commented, opt-in variables for common example categories.
  • Fixed stale README paths in avatar and telephony examples.
  • Removed voice-agent README entries for files that are no longer present.
  • Updated the examples README Python version from 3.9 to 3.10.
  • Split deterministic timestamp checks from wall-clock event timestamp checks in tests/test_agent_session.py.
  • Kept early wall-clock assertions explicit while allowing a small extra late margin for event emission under CI load.

Testing

  • make check
  • PYTHONPATH="$PWD" uv run ruff check tests/test_agent_session.py
  • PYTHONPATH="$PWD" uv run ruff format --check tests/test_agent_session.py
  • PYTHONPATH="$PWD" uv run pytest tests/test_agent_session.py::test_generate_reply
  • Earlier targeted run on this branch: PYTHONPATH="$PWD" uv run pytest tests/test_agent_session.py::test_tool_call tests/test_agent_session.py::test_interruption_before_speaking tests/test_agent_session.py::test_generate_reply tests/test_agent_session.py::test_preemptive_generation -q
    • 7 passed
  • Earlier full file run on this branch: PYTHONPATH="$PWD" uv run pytest tests/test_agent_session.py -q
    • 20 passed
  • git diff --check

Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

@shizhigu shizhigu changed the title docs(examples): fix stale README paths docs(examples): fix first-run example docs Apr 29, 2026
@shizhigu
Copy link
Copy Markdown
Author

The current unit-tests failure is unrelated to this docs-only diff. It is the existing tests/test_agent_session.py::test_tool_call timing flake that also showed up on prior runs. I opened #5597 with a focused test fix so this PR does not need to carry unrelated test changes.

@shizhigu shizhigu force-pushed the codex/examples-first-run-audit branch from e848a4c to 41bc785 Compare April 29, 2026 20:53
@shizhigu shizhigu changed the title docs(examples): fix first-run example docs docs(examples), test: fix first-run docs and timing flake Apr 29, 2026
@shizhigu shizhigu force-pushed the codex/examples-first-run-audit branch from 50a910e to 60a84ca Compare May 1, 2026 17:12
Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

View 4 additional findings in Devin Review.

Open in Devin Review

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Incomplete transformation: wall-clock timestamp check not converted in test_start_boundary_does_not_block_vad_interruption

The PR systematically converts all check_timestamp(*.created_at - t_origin, ...) calls to check_wallclock_event_timestamp(...) to use asymmetric tolerance (tighter early, looser late) and stabilize CI flakiness. However, line 757 in test_start_boundary_does_not_block_vad_interruption still uses the old check_timestamp with the symmetric max_abs_diff=0.75 for a wall-clock created_at - t_origin comparison. The new function's docstring at tests/test_agent_session.py:1094 explicitly states: "Use this for created_at event timestamps; keep check_timestamp for deterministic pipeline durations, playback positions, and same-turn deltas." This test remains susceptible to the same CI flakiness the PR aims to fix.

(Refers to line 757)

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant