Skip to content

Fix iOS simulator recording overlays so gesture commands can run during recording #574

@thymikee

Description

@thymikee

What to build

Make iOS simulator recordings with touch overlays resilient while gesture commands are running. Today, record start without --hide-touches couples overlay telemetry to a stable iOS runner session. Gesture commands such as pan, fling, pinch, and rotate-gesture also execute through the runner and can restart or replace the runner session. When that happens, recording health marks the recording invalid with iOS runner session restarted during recording, even though the raw simulator video is still recordable.

The clean scalable direction is to decouple overlay telemetry from a single long-lived runner session identity. Recording should treat touch telemetry as command-level evidence that can be appended across runner lifecycle changes, while record stop should always finalize the raw video and only degrade overlay output if telemetry is genuinely unusable.

Acceptance criteria

  • iOS simulator record start without --hide-touches can stay active while pan, fling, pinch, and rotate-gesture commands execute through the runner.
  • Runner session restarts during an active iOS simulator recording do not automatically invalidate the recording when command-level gesture telemetry can still be collected.
  • record stop always attempts to finalize and return the raw video path, even if touch overlay telemetry is incomplete or invalid.
  • If overlays cannot be burned in, the result reports a clear warning such as overlay unavailable instead of failing the whole recording.
  • Touch overlay timing remains correct when runner telemetry is available, including across more than one gesture command in the same recording.
  • Coverage includes a daemon/request-level regression for recording health during runner restarts and an iOS simulator recording flow with at least one gesture command.

Context

This surfaced while verifying the gesture test app on iPhone 17. Raw simctl io recordVideo worked, and agent-device record start --hide-touches is expected to avoid the runner-health dependency. The bug is specific to overlay-enabled recordings, where recording health currently treats runner session identity changes as fatal.

Blocked by

None - can start immediately.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions