Skip to content

Sadiq/ios simulator#6

Open
MdSadiqMd wants to merge 7 commits intohyperpush-org:mainfrom
MdSadiqMd:sadiq/ios-simulator
Open

Sadiq/ios simulator#6
MdSadiqMd wants to merge 7 commits intohyperpush-org:mainfrom
MdSadiqMd:sadiq/ios-simulator

Conversation

@MdSadiqMd
Copy link
Copy Markdown
Contributor

Summary

  • Replace iOS Simulator's screenshot polling (~1.7 FPS) with a Swift helper daemon (xero-ios-helper) using ScreenCaptureKit for ~30 FPS frame capture and IndigoHID mach IPC for reliable touch/swipe input
  • Add Metro Inspector bridge (Phase 2) for React Native / Expo element-at-point inspection with hover-to-highlight and source file mapping via Metro's CDP WebSocket
  • Add native Swift AX inspection (Phase 3) via AXUIElement for element inspection on native iOS apps with "Search project" affordance
  • Add Inspect button in toolbar with overlay that shows element type, label, bounds, and source location
  • Add Screen Recording TCC permission detection + banner matching existing Accessibility permission flow
  • Make ios-grpc a default feature so touch/swipe/HID works out of the box
  • All existing paths preserved as fallback: helper → idb_companion → screenshot poll → AppleScript click

Related Issues

Closes #4

Change Type

  • Bug fix
  • Feature
  • Refactor
  • Documentation
  • Test/build tooling

Testing

  • pnpm --dir client test
  • pnpm --dir client lint
  • cargo check --manifest-path client/src-tauri/Cargo.toml
  • Manual Tauri desktop verification
  • Not applicable, with reason:

UI Checklist

  • Uses ShadCN/Radix UI patterns where possible
  • Adds only user-facing UI, with no temporary debug or test UI
  • Includes screenshots or recordings for visible UI changes

Notes

  • Swift helper (xero-ios-helper) compiled via swiftc in build.rs, bundled as sidecar. Set XERO_SKIP_IOS_HELPER=1 to skip. Binary auto-copied by tauri-dev-runner.sh into Xero Dev.app bundle.
  • UDS socket path uses /tmp/xero-ih-{8chars}.sock to stay under macOS's 104-byte sockaddr_un.sun_path limit.
  • Permissions required: Screen Recording (ScreenCaptureKit), Accessibility (touch input + AX inspection), Automation/System Events (AppleScript fallback). App identity is dev.sn0w.xero ("Xero" / "Xero Dev" in System Settings).
  • iOS 26 boot: Simulator.app must remain visible during cold boot for the display pipeline to initialize. Window auto-minimizes after frame pump starts. Screenshot fallback tolerates up to 60 consecutive failures during boot warmup.
  • IndigoHID: Uses mach IPC (same protocol as FBSimulatorControl). Private API isolated in the helper binary. Port names are Xcode-version-stable.
  • Metro inspector: Auto-connects on iOS streaming start, probes ports 8081 + 19000-19006. Falls back to AXUIElement for native apps. Inspect button appears in toolbar between Rotate and Stop.
  • Status sync: Frontend force-sets status to "streaming"/"idle" after emulator_start/emulator_stop returns to work around async event listener race condition.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: rewrite iOS Simulator integration with Swift helper + RN/native inspector bridges

1 participant