Skip to content

test(desktop): make PTT HAL liveness deterministic - #10725

Closed
tianmind-studio wants to merge 3 commits into
BasedHardware:mainfrom
tianmind-studio:codex/ptt-input-probe-liveness
Closed

test(desktop): make PTT HAL liveness deterministic#10725
tianmind-studio wants to merge 3 commits into
BasedHardware:mainfrom
tianmind-studio:codex/ptt-input-probe-liveness

Conversation

@tianmind-studio

@tianmind-studio tianmind-studio commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Fixes #10724.

What changed and why

  • wait until the injected CoreAudio probe has actually entered its wedged semaphore before checking responsiveness
  • replace the fixed > 10 timer/read throughput requirements in a 300 ms window with three explicit main-actor heartbeat completions and three explicit currentSnapshot completions
  • keep snapshot reads off-main so a blocking regression times out without preventing the test from releasing and joining the parked HAL probe
  • exempt the real Swift package test path, desktop/macos/Desktop/Tests/, from the user-facing changelog gate and cover that path classification with regression cases

testMainRunLoopKeepsPumpingWhileTheHALIsWedged was testing a binary liveness contract with host-throughput thresholds. Under macOS runner contention, the main actor and snapshot path still made progress, but the isolated suite completed too few iterations. The production contract is that the main actor and turn-start snapshot reads both keep completing while the HAL probe remains blocked off-main, so the new test observes those two completions independently instead of measuring scheduler throughput.

The changelog gate already exempts internal test paths because its post-merge push run cannot see a PR-only no-changelog-needed label. It covered the legacy lowercase desktop/macos/tests/ directory but not the Swift package tests under Desktop/Tests/, so this test-only patch exposed the same open failure class on the actual Swift test path.

Failure evidence

All sibling PTTInputDeviceProbeTests cases passed in those failures.

Product invariants affected

none

How it was verified

  • python3 .github/scripts/test_desktop_changelog.py passed: 4 tests
  • python3 .github/scripts/check-desktop-changelog.py --base origin/main --head HEAD passed: no desktop changes require a changelog entry
  • python3 desktop/macos/scripts/check_desktop_test_quality.py passed
  • git diff --check origin/main...HEAD passed
  • reviewed the semaphore ordering: started is fulfilled immediately before release.wait(), main-actor heartbeat and snapshot-read expectations settle before release.signal(), and finished is awaited afterward

This contribution was prepared on Windows, so the local commit hook could not run the Xcode-only swift-format command (xcrun is unavailable). Desktop Swift CI is the authoritative compile, formatter, and XCTest verification for the Swift test change.

Tests

  • ChangelogRequirementTests now covers both a top-level and nested Swift package test path.
  • The rewritten PTT test is the regression proof for the three linked flaky runs; it fails in bounded time if either the main actor stalls or snapshot reads block while the injected HAL probe is parked.

Failure class (fixes)

Failure-Class: FC-push-gate-internal-path-scope

Scoped cleanups

The changelog path exemption is a separate commit because the initial test-only head demonstrated that relying on no-changelog-needed would pass only the PR lane and could still redden the post-merge push lane.

Fixes BasedHardware#10724.

Wait until the injected HAL probe is actually wedged, then require three main-actor snapshot reads to complete. This preserves the liveness contract without treating macOS runner throughput in a fixed 300 ms window as correctness.

Tests: desktop/macos/scripts/check_desktop_test_quality.py (macOS Swift CI required; this Windows host has no Xcode toolchain).

Failure-Class: none

Copy link
Copy Markdown
Contributor Author

Authoritative macOS verification is green on head 9c008a289e: Desktop Swift CI run 30286448998 passed the macOS manifest/formatter checks, the full Desktop Swift test suite, and the final Desktop Swift Build & Tests verdict. The rewritten PTT liveness regression completed on the real M1 runner without the scheduler-throughput assertions that failed in the three reproduced runs.

@Git-on-my-level Git-on-my-level added candidate-clean-pr Small, safe, well-scoped, well-tested contribution; model clean PR workflow-review Needs maintainer review for workflow, automation, hooks, or CI behavior labels Jul 27, 2026

@Git-on-my-level Git-on-my-level left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the focused fix here. I reviewed the diff and the current head CI evidence.

Positive signal from my side: the test rewrite is a good direction. Waiting for the injected HAL probe to actually enter the wedged path removes the old race, and replacing scheduler-throughput thresholds with explicit main-actor and off-main snapshot-read completions better matches the production contract this regression is supposed to pin. I also like that the parked probe is still released and joined before teardown.

The changelog-gate update also looks narrowly scoped: Desktop/Tests/ is the real SwiftPM test location and the added unit coverage keeps that exemption from leaking into product source paths.

I’m not formally approving from automation because this touches CI/changelog policy code and the approval gate requires human maintainer sign-off for workflow-sensitive paths, even though this looks clean and the relevant Desktop Swift CI run is green.


by AI on behalf of David — if you need David’s attention urgently, please @Git-on-my-level and escalate with need human response.

@Git-on-my-level

Copy link
Copy Markdown
Collaborator

Superseded by canonical-repo PR #10742 (same fix rebased onto current main).

@github-actions

Copy link
Copy Markdown
Contributor

Hey @tianmind-studio 👋

Thank you so much for taking the time to contribute to Omi! We truly appreciate you putting in the effort to submit this pull request.

After careful review, we've decided not to merge this particular PR. Please don't take this personally — we genuinely try to merge as many contributions as possible, but sometimes we have to make tough calls based on:

  • Project standards — Ensuring consistency across the codebase
  • User needs — Making sure changes align with what our users need
  • Code best practices — Maintaining code quality and maintainability
  • Project direction — Keeping aligned with our product principles and locked invariants

Before your next PR, please skim:

  • PRODUCT.md — product north star
  • Product invariants — locked rules (shared chat, memory tiers, agent control plane, integrations, brand)

If this was declined for direction or taste, maintainers should cite an invariant ID or open a proposed one — ask if that citation is missing.

Your contribution is still valuable to us, and we'd love to see you contribute again in the future! If you'd like feedback on how to improve this PR or want to discuss alternative approaches, please don't hesitate to reach out.

Thank you for being part of the Omi community!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

candidate-clean-pr Small, safe, well-scoped, well-tested contribution; model clean PR workflow-review Needs maintainer review for workflow, automation, hooks, or CI behavior

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky: PTTInputDeviceProbeTests uses CI throughput as a liveness assertion

2 participants