test(voice): repair and re-enable turn-detection tests in CI - #6715
Open
brandonbodily wants to merge 1 commit into
Open
test(voice): repair and re-enable turn-detection tests in CI#6715brandonbodily wants to merge 1 commit into
brandonbodily wants to merge 1 commit into
Conversation
|
|
Four test modules are marked `pytest.mark.audio_eot`, covering 83 tests of turn-detection behaviour: tests/test_audio_recognition_turn_detection.py 29 tests/test_audio_turn_detector_fallback.py 40 tests/test_turn_detection_fsm.py 11 tests/test_turn_detection_cloud_stream.py 3 CI runs `make unit-tests` -> `pytest --unit`, which deselects all of them, and no workflow passes `--audio_eot`. None of these 83 tests has been running. test_audio_recognition_turn_detection.py drifted out of sync with the source as a result; 8 of its 29 fail on a clean checkout: - `AudioRecognition.update_turn_detector` -> `_update_turn_detector` - `AudioRecognition.clear_user_turn` -> `_clear_user_turn` - the `_make_full_recognition_for_eou` stub was missing `_turn_backchannel_over_agent` and `_stt_consumer_atask` - the "already flushed" log assertion no longer matched the warning text, which now reads "transcript arrives after turn has been committed" This repairs all four drift points and adds `--audio_eot` to the `unit-tests` target. The other three modules were already green -- they simply never ran. Verified: `pytest --audio_eot` goes from 8 failed/75 passed to 83 passed. `pytest --unit` and `pytest --unit --audio_eot` are identical on every other test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
brandonbodily
force-pushed
the
fix/reenable-turn-detection-tests
branch
from
August 5, 2026 18:55
d0ef2b2 to
3a6664f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Four test modules are marked
pytest.mark.audio_eot, covering 83 tests of turn-detection behaviour:tests/test_audio_recognition_turn_detection.pytests/test_audio_turn_detector_fallback.pytests/test_turn_detection_fsm.pytests/test_turn_detection_cloud_stream.pyCI runs
make unit-tests→pytest --unit, which deselects all of them, and no workflow passes--audio_eot:None of these 83 tests has been running in CI.
Consequence
test_audio_recognition_turn_detection.pydrifted out of sync with the source. On a clean checkout ofmain, 8 of its 29 fail:AudioRecognition.update_turn_detector_update_turn_detector(renamed private)AudioRecognition.clear_user_turn_clear_user_turn(renamed private)_make_full_recognition_for_eoustub_turn_backchannel_over_agent,_stt_consumer_atask"already flushed""transcript arrives after turn has been committed"Each is a one-line repair. The other three modules were already green — they simply never ran.
Changes
--audio_eotto theunit-teststarget so these run alongside--unit.No library code is touched — only the test module and the make target.
Verification
pytest --unitandpytest --unit --audio_eotproduce identical results across every other test in the suite.Why this matters beyond the repair
This is the coverage for turn-detection commit policy, which is where several open correctness issues live (#6504, #3515). With the suite green, adding a single assertion to the existing
test_late_stt_final_after_flush_short_circuitsreproduces #6504 deterministically:That test already drives the exact #6504 scenario — a late STT final after
_turn_detector_flushedis set — and asserts onpredict, onon_eot_prediction, and on the warning, but never on whether a turn is committed. The sibling test two functions below (test_predict_timeout_signals_fallback_and_drops_future) does asserton_end_of_turn.assert_called_once(), so the pattern is established in this file.I've deliberately left that assertion out of this PR — it would fail until #6504 is fixed, and PR #6516 is already open against it. This PR is only the CI repair, so it lands independently and green.
Two other markers look dark, but I didn't touch them
While tracing this I noticed
plugin(26 modules) anddocs(1) are also never selected by any workflow — they're only reachable throughtests/Makefile:unit-tests(--unit --plugin --evals --docs), and CI'smake unit-testsstep runs from the repo root, so it hits the rootmakefiletarget instead.To be clear about what still does work: the plugin matrix isn't idle —
tests/Makefile:testrunspytest --tts, andtest_tts.pyreads thePLUGINenv var (lines 286, 551) to filter its parametrized provider list, so each matrix job genuinely exercises that provider's TTS. What it doesn't run are the 26 dedicatedtest_plugin_*modules, including the*_stt.pyones.I've left those out of this PR on purpose — re-enabling 26 provider-integration modules is a credentials-and-CI-cost decision for maintainers, not something to slip into a test repair. Happy to open a separate issue if that's useful.