Skip to content

fix: prevent runner XCTest attachment bloat#520

Merged
thymikee merged 1 commit into
callstackincubator:mainfrom
spilist:fix/xctest-attachment-bloat
May 14, 2026
Merged

fix: prevent runner XCTest attachment bloat#520
thymikee merged 1 commit into
callstackincubator:mainfrom
spilist:fix/xctest-attachment-bloat

Conversation

@spilist
Copy link
Copy Markdown
Contributor

@spilist spilist commented May 14, 2026

Summary

Prevent long-running AgentDeviceRunnerUITests sessions from leaving large XCTest automatic attachments behind.

  • Pins an explicit AgentDeviceRunnerUITests.xctestplan in the scheme with screenshot capture and keepNever system/user attachment lifetimes.
  • Normalizes the generated per-session .xctestrun immediately before test-without-building, because Xcode 26.2 can emit attachment lifetime values that differ from the plan.
  • Keeps this policy scoped to XCTest's automatic attachments; agent-device screenshot and screen-recording command paths are unchanged.

Validation

  • zsh -lic 'source ~/.nvm/nvm.sh && nvm use 24.13.0 >/dev/null && pnpm install'
  • zsh -lic 'source ~/.nvm/nvm.sh && nvm use 24.13.0 >/dev/null && pnpm check:fallow --base origin/main'
  • zsh -lic 'source ~/.nvm/nvm.sh && nvm use 24.13.0 >/dev/null && pnpm check:unit'
  • AGENT_DEVICE_XCUITEST_PLATFORM=ios AGENT_DEVICE_IOS_RUNNER_DERIVED_PATH=/tmp/agent-device-pr520-derived-ios-cleanup AGENT_DEVICE_IOS_CLEAN_DERIVED=1 sh ./scripts/build-xcuitest-apple.sh
  • xcodebuild -project ios-runner/AgentDeviceRunner/AgentDeviceRunner.xcodeproj -scheme AgentDeviceRunner -showTestPlans
  • xcodebuild test-without-building against a patched runner .xctestrun on iOS Simulator, including runner shutdown
  • Manual runner screenshot and screen-recording probes
  • node --check src/platforms/ios/runner-xctestrun.ts
  • node --check src/platforms/ios/__tests__/runner-xctestrun.test.ts
  • git diff --check

@thymikee thymikee force-pushed the fix/xctest-attachment-bloat branch from cbd13c3 to 0be9034 Compare May 14, 2026 09:21
@thymikee thymikee changed the title fix: prevent testmanagerd attachment bloat by pinning an explicit xctestplan fix: prevent runner XCTest attachment bloat May 14, 2026
@thymikee thymikee force-pushed the fix/xctest-attachment-bloat branch from 0be9034 to c13fbcd Compare May 14, 2026 09:24
Copy link
Copy Markdown
Member

@thymikee thymikee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! much appreciated and good find

@thymikee thymikee force-pushed the fix/xctest-attachment-bloat branch 2 times, most recently from 9c374f6 to 055568b Compare May 14, 2026 11:30
Long-running AgentDeviceRunnerUITests sessions can inherit Xcode 26 screenRecording/deleteOnSuccess defaults and leave hidden testmanagerd attachments behind when sessions terminate outside a clean XCTest success path.

Pin an explicit AgentDeviceRunnerUITests.xctestplan with screenshots and keepNever attachment lifetimes, then normalize the generated per-session .xctestrun copy before test-without-building. The .xctestrun normalization protects stale cached artifacts and Xcode versions that ignore some test-plan attachment lifetime keys during build-for-testing.

This only affects XCTest's automatic attachments; user-requested agent-device recordings still use the existing record command paths.
@thymikee thymikee force-pushed the fix/xctest-attachment-bloat branch from 055568b to 82b80d9 Compare May 14, 2026 11:44
@thymikee thymikee merged commit 2e04edd into callstackincubator:main May 14, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants