CI: run curated tests_v2 socket subset on PRs (#145) - #161
Merged
Conversation
CI never executed the python socket suites; tests_v2/** was only a path trigger. New tests-v2-subset job (macos-15, 40m cap) builds the full app, boots it with a tagged socket, and runs the stable pure-socket files listed in tests_v2/ci_subset.txt via the new scripts/run-tests-v2-ci.sh (extracted from run-tests-v2.sh minus the VM-only guard; direct binary launch, both socket env vars exported, unique DerivedData cache key). Closes #145
surface.focus rejected bare indices in #149; both the VM script and the extracted CI script still passed 0, so the workspace bootstrap died before any test ran. list_surfaces rows are (index, id, focused) — pass the id.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Socket-level regressions can now fail a PR before merge. Until now the python socket suites never ran in CI at all — a fix like #144's could ship with its regression test never executed. This adds a required job running a curated stable subset (12 pure-socket files) against a freshly built, booted app on every app-affecting PR.
Closes #145. This PR's own CI run is the live proof of the job.
Summary
tests-v2-subsetjob in ci.yml: macos-15 (not the timeout-prone macos-26), 40-minute cap, gated by change-detection like the other app jobsscripts/run-tests-v2-ci.sh: launch/retry logic extracted fromrun-tests-v2.sh, minus the VM-only user guard; direct binary launch (noopen), exports bothPROGRAMA_SOCKET_PATHandPROGRAMA_SOCKETtests_v2/ci_subset.txt: the gating list — grow it over time; interactive/drag/visual suites stay VM-onlyderiveddata-tests-v2-macos15-*,spm-tests-v2-*) so a failed run can't poison sibling jobsTest Plan
bash -non the new script; YAML parses; all 12 subset files existtests-v2-subsetjob passes (the real verification)