feat(testing): Add headless launch mode#435
Merged
Merged
Conversation
Add an opt-in headless launch policy that prevents macOS and simulator launches from stealing focus during automated runs. Update UI automation guidance, snapshot normalization, benchmark suite execution, and fixtures so snapshot and unit tests remain stable across current simulator and Xcode environments.
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Pin Claude UI benchmark suites to the expected model and record model, Claude Code, and timing metadata in benchmark output. Tighten suite prompts and tool guidance so benchmark runs follow the intended baseline sequences. Increase post-action snapshot settle headroom to reduce timing-sensitive UI automation recovery after action tools, and refresh simulator list fixtures after removing stale Claude UI benchmark simulators. Co-Authored-By: OpenAI Codex <noreply@openai.com>
Allow snapshot normalizer tests to inject the temp directory used for path normalization. This keeps artifact path expectations stable across macOS and Linux CI hosts. Co-Authored-By: OpenAI Codex <codex@openai.com>
commit: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add an opt-in headless launch policy for automated runs that should not steal macOS focus.
When
XCODEBUILDMCP_HEADLESS_LAUNCHis enabled, macOS app launches use backgroundopen -g, Simulator.app GUI launches are skipped while still booting the runtime throughsimctl, and keyboard shortcuts fail with a clear foreground-focus requirement. Snapshot tests enable this automatically so local and CI-style runs are less disruptive.This also tightens UI automation next-step targeting, moves snapshot helper coverage into non-snapshot tests, normalizes additional Xcode/simulator variability in fixtures, and adds a directory runner for Claude UI benchmark suites.
Validated locally with
npm run lint,npm run format:check,npm run typecheck,npm run build, andnpm run test.