Multi-touch gestures, RN debugger, profiling, network inspection, crash reports + 7 more debugging tools#11
Merged
Merged
Conversation
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
iOS: new GesturePathRequest model and GesturePathRouteHandler that build an EventRecord(.multiFinger) with one XCPointerEventPath per finger. Registered in the Route enum, RouteHandlerFactory, and the xcodeproj iOS + tvOS Sources build phases. Android: new gesturePath gRPC + GesturePathRequest/GestureFingerPath/ GestureStep/GesturePathResponse messages. Handler uses UiAutomation.injectInputEvent with multi-pointer MotionEvent (dispatchGesture is on AccessibilityService, not UiAutomation). Each path is a separate pointer; positions are resampled to a global 16ms grid and dispatched as ACTION_DOWN / ACTION_POINTER_DOWN / ACTION_MOVE / ACTION_POINTER_UP / ACTION_UP. The CLI's bundled proto copy is synced for grpc-js code generation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Module: packages/cli/src/commands/gestures.ts. Builds synchronized multi-finger paths and dispatches them through the new driver gesturePath routes added in the previous commit. - pinch: two fingers along an axis (--angle), span scaled by --scale - rotate: two fingers traced along an arc of --degrees - gesture: arbitrary multi-finger JSON path IOSDriver and AndroidDriver gain gesturePath() methods. Dispatcher wiring lands in a later commit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- clipboard read/write via xcrun simctl pbcopy/pbpaste on the target UDID. - paste types the clipboard contents into the focused field (iOS has no universal Cmd+V; the read+input-text pattern matches Argent's approach for their custom simulator-server paste command). Android is explicitly unsupported — `cmd clipboard set-primary-clip` is API 31+ and brittle. Callers get a clear "use input-text instead" message matching Argent's iOS-only paste tool surface. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extend `inspect` with `--at x,y` and `--tappable` flags. Adds findIOSViewAtPoint / findAndroidViewAtPoint / findWebViewAtPoint to element-resolver — depth-first walks returning the deepest (smallest-area) node whose rect contains the point, optionally filtered to interactive elements (iOS XCUIElementType, Android clickable, Web aria roles). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- packages/cli/src/drivers/metro-cdp.ts: one-shot cdpCall() and stateful
MetroCdpClient. Reuses fetchTargets / selectTargetForDevice from the
existing MetroLogSource discovery code instead of duplicating it.
Includes Runtime.evaluate, Runtime.addBinding for async callbacks, and
per-domain enable tracking — used here by metro reload, later by the
experimental debug/network/profile commands.
- packages/cli/src/commands/metro.ts:
metro stop — lsof -ti tcp:<port> + SIGTERM (escalate to SIGKILL).
metro reload — Page.reload over CDP, falls back to POST /reload.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- run-sequence: batched serial dispatch of conductor commands against one
session. Reads {"steps":[{"cmd","args","flags"}]} from --file or stdin;
stops on first non-zero exit. Saves agent roundtrips when a multi-step
flow doesn't need real flow-runner semantics.
- workspace info: single read-only report of project type (RN / Expo / iOS
/ Android / Web / mixed), bundle ids, configured devices, current Metro
port. Lets agents skip the package.json + list-devices + bundle-id
derivation dance.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Command-level recorder — Conductor drivers don't expose an input event channel, so we record the commands the agent issues rather than user gestures. This is the same trade-off Argent makes with their flow-add-step meta-tool; we go a step further with an auto-append hook in the dispatcher (wired in a later commit) so the agent doesn't need to wrap each step explicitly. - flow-recorder.ts: session-scoped active recording path, append helpers, commandToYamlStep mapping covering the action-command subset that faithfully replays through flow-runner.ts. - flow-record.ts: the start/echo/status/finish subcommand surface. Output flow YAML round-trips through the existing run-flow command. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Net-new capability — Argent's README claims it but ships no dedicated tool.
We normalise iOS host-side .ips/.crash files (from
~/Library/Logs/DiagnosticReports/) and Android logcat -b crash blocks
into a shared { id, timestamp, app, type, signal, threadName, topFrames,
sourceFile, platform } shape so agents don't have to parse platform-specific
text. tail() streams new reports via fs.watch + adb logcat.
Parser is heuristic — fields are best-effort across iOS versions. dSYM
symbolication and a stable Android schema are follow-ups.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three command groups sharing the MetroCdpClient and an Argent-style script
library. Marked experimental because they depend on React Native runtime
internals (fiber shape, UIManager / nativeFabricUIManager,
renderer.rendererConfig) that may drift across RN versions.
- metro-scripts.ts: makeComponentTreeScript and makeInspectElementScript.
component-tree: detects Fabric vs Paper, finds UIManager via __r,
batch-measures rects, filters a curated SKIP set
(View, RNSScreen, NavigationContent, Provider
wrappers, etc.). Returns via Runtime.addBinding.
inspect-element: uses React's own renderer.rendererConfig.
getInspectorDataForViewAtPoint, walks up via
.return, resolves source from _debugStack /
_debugSource.
- debug.ts: status / evaluate / component-tree / inspect-element /
log-registry / reload subcommands over Metro CDP.
- network.ts: idempotent fetch + XHR shim injected via Runtime.evaluate,
read back through a ring buffer. `network request` issues a fetch from
the app's network context.
- profile.ts: cpu (xctrace / simpleperf), memory polling, react commit
profiler via __REACT_DEVTOOLS_GLOBAL_HOOK__.onCommitFiberRoot.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds imports, COMMAND_HELP entries, minimist flag declarations, NO_DEVICE list updates, and case branches for every command introduced in the preceding commits: pinch, rotate-gesture, gesture, clipboard, paste, inspect --at / --tappable, metro, run-sequence, workspace, flow record, crashes, debug, network, profile. Also installs the flow-recording auto-append hook — when a recording is active for the current session, successful action commands are appended to the flow YAML automatically via commandToYamlStep(). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- commands.md: new entries under Interaction (pinch, rotate-gesture, gesture, clipboard, paste), Inspection (inspect --at), Flows (run-sequence, flow record). New top-level Workspace, Metro, and Crashes sections. - experimental.md (new): RN debugger, network inspection, profiling. Each group ships with explicit caveats — Hermes-only, Fabric/Paper drift, fetch/XHR shim limits, etc. Stays in this section until a command survives a full RN minor-version cycle without script changes. - marketing-manifest.json: register the new Experimental page under Reference. The houwert.dev sync script picks it up automatically on the next site build. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drops the unused `fs` import in profile.ts that failed the no-unused-vars lint rule and broke CI on the improvements branch. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
These four files had lines that violated the prettier printWidth rule, failing `prettier --check` in CI once the preceding eslint error was resolved. Reformatted with `prettier --write`; no logic changes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the debugging-tool gaps identified by walking
software-mansion/argent's tool surface against Conductor. Eleven new command groups split across stable and experimental tiers, plus a multi-finger driver route that's net-new infrastructure.What's new
Stable
pinch,rotate-gesture,gesture <json>gesturePathroute — iOSXCSynthesizedEventRecord(.multiFinger), AndroidinjectInputEventwith multi-pointerMotionEvent.clipboard read/write,pastexcrun simctl pbcopy/pbpaste. Android gets a clear "useinput-text" error, matching Argent's iOS-only paste tool.inspect --at x,y [--tappable]metro stop,metro reloadlsof+SIGTERM;Page.reloadover CDP with HTTP/reloadfallback.run-sequenceworkspace infoflow record start/echo/status/finishrun-flow.crashes list/show/tail.ipsfiles + Androidlogcat -b crash, normalised JSON. Net-new — Argent's README claims it but ships no dedicated tool.Experimental
These depend on RN runtime internals and may drift with RN major versions:
debug status/evaluate/component-tree/inspect-element/log-registry/reloadUIManager.measureInWindowornativeFabricUIManager.measure.inspect-elementuses React's authoritativerenderer.rendererConfig.getInspectorDataForViewAtPoint.network logs/requestfetch/XHRshim injected viaRuntime.evaluate. Captures method/URL/status/timing in a 200-entry ring buffer.network requestruns inside the app's runtime — honours cookies, auth, TLS pinning.profile cpu/memory/reactxctrace, Androidsimpleperf, polled memory sampling, React commit profiler viaonCommitFiberRoot.Infrastructure
packages/cli/src/drivers/metro-cdp.ts— reusable CDP client withRuntime.addBindingfor async callbacks. Used bymetro reload,debug,network,profile.packages/cli/src/drivers/metro-scripts.ts— Argent-style fiber walker scripts (makeComponentTreeScript,makeInspectElementScript) with curated SKIP set, source resolution from_debugStack/_debugSource.GesturePathRouteHandlerand Kotlin handler usingUiAutomation.injectInputEventwith multi-pointerMotionEvents (dispatchGestureis onAccessibilityService, notUiAutomation— discovered the hard way during the first build).Driver rebuild
make build && make package-cliwas run against this branch — bundled zips inpackages/cli/drivers/{ios,tvos,android}/are fresh (uncommitted; they're regenerated by CI from source at release time and not part of the npm package).End users automatically get the new drivers on
npm update—bootstrap.tsdownloadsdrivers.tar.gzfrom the matching GitHub release, keyed on CLI package version. No separate driver-upgrade step.Caveats
Documented in
docs/experimental.md:__REACT_DEVTOOLS_GLOBAL_HOOK__, fiber shape, andrenderer.rendererConfig. Hermes-only.network logsshim doesn't see native networking modules (OkHttp turbo-modules, nativeURLSessioncalls) or WebSockets..dSYMand isn't yet wired in.profile cpurequiresxctrace(Xcode) orsimpleperf(Android NDK) onPATH.Test plan
pnpm exec tsc -p tsconfig.json— cleanpnpm exec tsc -p tests/tsconfig.json && node dist-tests/tests/all-tests.js— 169/169 passingmake build-ios-driver(** TEST BUILD SUCCEEDED **)make build-tvos-driver(** TEST BUILD SUCCEEDED **)make build-android-driver(BUILD SUCCESSFUL)make package-cliwrites fresh zips/APKs topackages/cli/drivers/debug component-treeagainst a real RN app on Fabricdebug component-treeagainst a real RN app on Papernetwork logscaptures afetchcall aftermetro reloadcrashes listparses a real iOS.ipsflow recordproduces a YAML thatrun-flowreplaysCommit structure
Eleven feature-by-feature commits ending with one dispatcher-wiring commit (
feat(cli): wire new commands into the dispatcher) and one docs commit. Each intermediate commit type-checks; the dispatcher-wiring commit is what exposes the new commands at the CLI.🤖 Generated with Claude Code