Skip to content

feat(core): iOS element regions for Maestro (WDA-direct + maestro-hierarchy paths)#2202

Open
Sriram567 wants to merge 21 commits intomasterfrom
feat/ios-element-regions-maestro-hierarchy
Open

feat(core): iOS element regions for Maestro (WDA-direct + maestro-hierarchy paths)#2202
Sriram567 wants to merge 21 commits intomasterfrom
feat/ios-element-regions-maestro-hierarchy

Conversation

@Sriram567
Copy link
Copy Markdown

@Sriram567 Sriram567 commented Apr 29, 2026

Summary

iOS element-region resolver for PERCY_REGIONS element selectors during a live maestro test flow. Two implementation paths land here:

  • Plan A — WDA-direct (default): wda-hierarchy.js resolves regions via direct WebDriverAgent integration with the Maestro-spawned WDA session. Stable and proven end-to-end on BS iOS host 52 in the 2026-04-22 demo. Blocked at runtime by realmobile-side PER-7281 until that ships.
  • Plan B — maestro-hierarchy (behind PERCY_IOS_RESOLVER env switch): Cross-platform maestro --udid <udid> hierarchy resolver. Same Maestro JSON walker that Android already uses, behind a switch so default behavior is unchanged. Customers see no production change until a future PR flips the default and deletes Plan A.

This PR combines what was previously split as #2201 (Plan A) + #2202 (Plan B Phase 1). Single review surface for the iOS element-regions feature; reviewers no longer need to walk a 3-PR stack. Phase 2.2 Android perf work stays as #2210 on top of this PR (it's an independent Android-only optimization).

What lands

Plan A: WDA-direct iOS resolver (B1 → B4 + 3 fixes)

  • d097f077 B1png-dimensions.js: PNG IHDR parser for the iOS width-ratio scale-factor.
  • 1792e376 B2wda-session-resolver.js: reader for the realmobile ↔ Percy CLI wda-meta.json contract (TOCTOU-safe via O_NOFOLLOW + fstat, per SEI CERT POS35-C).
  • d0cae9c3 B3wda-hierarchy.js: iOS element-region resolver. Source-dump path (GET /session/:sid/source); XCUI_ALLOWLIST for class-name normalization; in-bounds + min-area bbox validation; scrubbed reason-tag log surface.
  • d2eb348f B4 — wires the iOS resolver into the /percy/maestro-screenshot relay handler.
  • e7b9938b 3 fixes from 2026-04-23 live validation:
    • Fix C — feature-detect AbortController (BS hosts run Node 14.17.3 where it's not a global; previously threw ReferenceError masked as 'wda-error').
    • Fix B — extract active sid from WDA error envelope and retry once on stale-session.
    • Fix A — surface optional wdaSessionId from contract v1.1.0+ wda-meta.json payloads.
      See percy-maestro/docs/solutions/integration-issues/ios-wda-session-id-and-node14-abortcontroller-2026-04-23.md.

Plan B: maestro-hierarchy iOS resolver (Phase 1, behind env switch)

  • 9e2f3815 Rename adb-hierarchy.jsmaestro-hierarchy.js + 5-line compat shim. Pure rename; no behavior change. (This rename is also load-bearing for feat(core): direct gRPC client for Maestro view-hierarchy resolver (Phase 2.2) #2210 Phase 2.2 gRPC.)
  • 403d89fc Platform-dispatch scaffolding in dump({ platform }). iOS branch reads PERCY_IOS_DEVICE_UDID + PERCY_IOS_DRIVER_HOST_PORT env vars (realmobile-injected) and currently returns { kind: 'unavailable', reason: 'not-implemented' } as a stub — full iOS attribute mapping lands in a Phase 0.5 follow-up. Also lands R1 vocabulary parity: resource-id is now also surfaced as id on Android, so {element: {id: "X"}} works on both platforms.
  • 1b98ece6 api.js dispatch behind PERCY_IOS_RESOLVER env switch. Default off → existing WDA-direct iOS path. Switch on → cross-platform Android-style lazy dump + per-region firstMatch.
  • 616cdd56 Cross-platform parity test + XML-path id alias fix.

Why both paths land together

The 2026-04-27 spike (run on BS iOS host 52) showed that the brainstorm's "session-exclusive" claim against maestro hierarchy for iOS was wrong — it conflated maestro hierarchy with maestro studio. Plan B is the long-term direction (one resolver module, same yaml shape, same docs) but Plan A is what actually works in production today. Shipping both as a single feature behind an env switch means we can flip the default in a follow-up PR without re-shipping any iOS code.

A future Phase 4 PR will:

  1. Empirically validate Plan B end-to-end on a BS iOS host (Phase 0.5 probe).
  2. Flip PERCY_IOS_RESOLVER default to maestro-hierarchy.
  3. Delete wda-hierarchy.js, wda-session-resolver.js, png-dimensions.js, and the realmobile wda-meta.json contract on the realmobile side.

Testing

  • All wda-hierarchy.test.js, maestro-hierarchy.test.js, and Plan B parity test specs pass locally.
  • Plan A pipeline proven end-to-end on BS iOS host 52 in 2026-04-22 demo (Percy build ⬆️ Bump regenerator-runtime from 0.13.5 to 0.13.7 #16, build URL in project_e2e_validation_state.md memory).
  • Plan B is gated behind PERCY_IOS_RESOLVER=maestro-hierarchy; Phase 0.5 empirical probe is the next validation step.

Post-Deploy Monitoring & Validation

  • What to monitor / search
    • Logs:
      • [percy:core:wda-hierarchy] resolved iOS element region (...) — Plan A healthy.
      • [percy:core:maestro-hierarchy] dump took Nms via maestro (N nodes) — Plan B healthy (only when env switch is set).
  • Expected healthy behavior
    • No regression in Android element-region snapshot pass rate.
    • iOS element-region requests routed to Plan A by default; no wda-error or new dump-error reason tags above the 2026-04-22 baseline.
  • Failure signal / rollback trigger
    • Increase in wda-error rate above the validated baseline → investigate; the realmobile-side blocker (PER-7281) is the most likely cause.
  • Validation window & owner
    • Window: 1 week post-rollout. Owner: @arumullasriram.

Compound Engineering v2.54.0
🤖 Generated with Claude Opus 4.7 (1M context, extended thinking) via Claude Code + Compound Engineering v2.54.0

Sriram567 and others added 20 commits March 26, 2026 21:01
- Add empty body guard (400 instead of TypeError)
- Add Busboy fileSize limit to reject oversized uploads during parsing
- Use Object.create(null) and field allowlist to prevent prototype pollution
- Add stream error handler on Readable source
- Use HTTP 413 for oversized files
Reject immediately on Busboy 'limit' event with 413 instead of
setting fileBuffer to null which produced 'Missing required file part'.
Accepts {name, sessionId} as JSON, finds the screenshot file on disk
at /tmp/<sessionId>_test_suite/logs/*/screenshots/<name>.png,
base64-encodes it, and processes as a standard comparison.

This enables real-time Percy uploads from Maestro flows where the
JS sandbox cannot access screenshot files directly.
… metadata

Accept statusBarHeight, navBarHeight, fullscreen from request instead of
hardcoding 0/false. Transform coordinate-based regions to CLI boundingBox
format. Add sync mode support via percy.syncMode() + handleSyncJob().
Forward thTestCaseExecutionId to comparison pipeline.

Element-based regions log a warning and are skipped — ADB uiautomator
resolution will be added as a follow-up.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Platform-aware screenshot discovery:
- Accept platform field with strict whitelist (ios/android); 400 on unknown
- iOS glob: /tmp/{sessionId}/*_maestro_debug_*/{name}.png
- Android glob unchanged; backward compat with SDK v0.2.0 (no platform → Android)

Path-safety hardening:
- Tighten name/sessionId from blocklist to strict character-class allowlist
- fs.realpath canonicalization + session-root prefix check defeats symlink swap
- Handles macOS /tmp → /private/tmp symlink transparently

Pick most recently modified file when multiple match (iOS same-name-across-flows).
Introduces packages/core/src/adb-hierarchy.js with two plain exports:
dump() and firstMatch(nodes, selector). The resolver:

- Reads process.env.ANDROID_SERIAL; falls back to one adb devices probe
  (requires exactly one attached device to avoid wrong-device dumps under
  multi-session CLI concurrency). Never accepts serial from request input.
- Shells out via cross-spawn with a 2s hard timeout (mirrors the
  browser.js:256-297 spawn+cleanup pattern).
- Classifies results into one of three shapes — unavailable, dump-error,
  hierarchy — so the relay can distinguish environmental failures from
  transient dump failures.
- Streams primary via adb exec-out uiautomator dump /dev/tty; falls back
  to file-based dump + cat only on wrong-mechanism signals (exit≠0 or
  missing <?xml prefix). Terminal signals (oversize / parse-error) do not
  retry — prevents attack amplification on adversarial payloads.
- Slices the XML envelope to the first </hierarchy> (strips uiautomator's
  trailer line and defends against embedded adversarial XML blocks).
- Enforces a 5MB stdout cap before parse.
- Parses with fast-xml-parser configured for defense-in-depth
  (processEntities: false, allowBooleanAttributes: false).
- Exposes firstMatch with pre-order DFS + strictly-anchored bounds regex;
  zero-area nodes are non-matches, negative coordinates (clipped views)
  are allowed.

Adds fast-xml-parser ^4.4.1 as a new dependency of @percy/core.

27 unit tests cover the parser + selector logic and all classification
branches via a parameter-injected execAdb seam. No real ADB calls; no
filesystem or network access.
Wires the /percy/maestro-screenshot relay to the new adb-hierarchy
resolver. Replaces the existing element-region warn-and-skip stub with
actual resolution via ADB + uiautomator dump on Android.

Handler changes:
- Early 400 validation on region shape before file I/O or ADB work:
  whitelist selector keys (resource-id/text/content-desc/class), require
  exactly one selector key per region, string-typed value, length ≤512,
  total regions per request ≤50.
- Android element regions: lazy dump on first element region, memoize
  the result (including error classes) for the whole request. Pre-scan
  element-region count so the skip warning reports N regions accurately.
  Both unavailable and dump-error poison the rest of the request with
  one warning — bounds worst-case per-request ADB time to one 2s timeout
  regardless of element-region count (closes the timeout-accumulation
  DoS vector).
- iOS element regions: preserve existing warn-and-skip semantics. Not
  a 400. Avoids a breaking change for any iOS caller today.
- Coordinate regions: unchanged; still transform {top,bottom,left,right}
  to elementSelector.boundingBox.
- Miss on element resolution: per-element warning, region skipped,
  request still uploads.

First-ever /percy/maestro-screenshot handler tests cover input
validation (9 × 400 paths), coordinate-only flow regression, iOS
warn-and-skip behavior, end-to-end forwarding of testCase/labels/
thTestCaseExecutionId/tile-metadata/sync, and the missing-screenshot
404 path. ADB-integration paths (element resolution against a real
device) are covered by the adb-hierarchy unit tests and Unit 7 E2E
validation on BrowserStack Maestro.
E2E validation on BrowserStack Maestro against host 31.6.63.33 / Pixel 7 Pro
showed the primary exec-out path intermittently returning no-xml-envelope
and the file-dump fallback exiting 137 (SIGKILL of uiautomator on the
device). The kill is triggered by concurrent uiautomator/automation
activity on the device during a live Maestro session — not a device-wide
or permissions issue (manual dumps from the shell return 44KB XML fine).

A single 300ms-delayed retry of the fallback dump command recovers the
common case without masking genuine device unavailability. If the second
attempt also fails, we still fall through to the existing dump-error
classification.

Test: the adb-hierarchy spec adds a retry test where the first fallback
exec returns 137 and the second returns the fixture XML; resolver returns
hierarchy and fileDumpCalls == 2.
Strengthens the SIGKILL retry from a single 300ms attempt to three retries
at 500ms/1s/2s (3.5s total window). Exits early as soon as a dump succeeds.

Rationale: single short retry wasn't enough against persistent device
contention observed during BrowserStack Maestro sessions. The wider budget
catches transient uiautomator kills on less-contended devices while still
failing fast on genuinely unavailable devices. Captured limitation: when
Maestro holds uiautomator throughout a flow (its observed behavior on real
devices), no reasonable retry count recovers — the mechanism itself needs
to change (e.g., Maestro API integration or an accessibility-service
sidecar). That's a Phase 2 follow-up, not part of this patch.

Tests cover both the "succeeds on Nth retry" case and the "all retries
exhausted" case.
E2E on BrowserStack Maestro showed `adb exec-out uiautomator dump` is
fundamentally incompatible with live Maestro flows — Maestro holds the
uiautomator lock throughout a flow and competing dumps get SIGKILLed.
The `maestro --udid <serial> hierarchy` CLI command reuses Maestro's
existing gRPC connection to dev.mobile.maestro on the device and works
reliably during live sessions (verified by probing twice mid-flow —
both probes returned valid JSON while the flow was running).

Changes in packages/core/src/adb-hierarchy.js:

- Primary dump mechanism is now `maestro --udid <serial> hierarchy`.
- Parse the resulting JSON (slice from the first `{` to tolerate banner
  lines), flatten the tree into the existing node shape.
- Map `accessibilityText` → `content-desc` at flatten time so `firstMatch`
  still uses the SDK's selector vocabulary unchanged.
- Maestro CLI timeout: 15s (JVM cold start ~9s + headroom).
- Honor `MAESTRO_BIN` env var for alternate paths; default `maestro`
  on PATH.
- New `spawnWithTimeout` helper shared between maestro and adb code paths.
- Classification extended with maestro-specific reasons (`maestro-not-found`,
  `maestro-timeout`, `maestro-no-device`, `maestro-no-json`,
  `maestro-parse-error:*`, `maestro-spawn-error:*`, `maestro-exit-*`,
  `maestro-oversize`).

Fallback: when maestro returns anything other than `hierarchy`, fall
through to the existing `adb exec-out uiautomator dump` flow (including
SIGKILL retry/backoff and file-dump fallback). Useful when the maestro
binary isn't installed on the CLI host.

Cost: 9s JVM cold start per screenshot that uses element regions.
Acceptable today because the alternative is 100% skip. Phase 2.2 follow-up:
replace the CLI invocation with a direct gRPC client against device port
6790 (typical latency <100ms) — infrastructure already in place (adb
forwards tcp:8206 → 6790 per device on BrowserStack hosts).

Tests: 36 specs total. New `dump (maestro hierarchy primary)` describe
block adds 7 scenarios (happy path, content-desc mapping, ENOENT→adb
fallback, unavailable propagation when both fail, timeout → adb recovery,
banner prefix tolerance, no-json). Existing 29 tests now inject an
execMaestro stub that reports ENOENT so they exercise the adb fallback
path exactly as before.
New module png-dimensions.js serves both:
- existing /percy/comparison/upload signature check (api.js import)
- upcoming /percy/maestro-screenshot iOS path (scale factor via
  pngWidth / wda_window_logical_width + aspect-ratio landscape fallback)

Exports:
- PNG_MAGIC_BYTES (moved from api.js route-local scope)
- parsePngDimensions(buf) → {width, height} via IHDR hand-parse
  (24-byte prefix read, no new dependency)
- isPortrait / isLandscape with default threshold 1.25
  (iPad portrait ratio 1.334; margin empirically confirmable via A1 Probe 6)
- DEFAULT_ORIENTATION_THRESHOLD exported for override in tests / A1 Probe 6

Test-first: 17 specs covering happy path iPhone/iPad portrait+landscape,
dimensions > 65535, truncated buffer, bad signature, zero width/height,
non-Buffer input, threshold override, near-square ambiguity. All pass.

api.js: removes inline PNG_MAGIC_BYTES declaration from the upload route
handler; imports the shared constant. Upload signature-check behavior
unchanged.

Unit B1 of the iOS Maestro element-regions plan (v1.0); serves as the
Phase 1 CI coverage preflight per plan.
…-meta.json

Reader side of the realmobile ↔ Percy CLI contract v1.0.0 for iOS
element-region resolution on shared BS iOS hosts. Given a Maestro sessionId,
resolves /tmp/<sid>/wda-meta.json → { ok: true, port } or { ok: false, reason }
with TOCTOU-safe validation per contract §8:

File-level (SEI CERT POS35-C ordering, no lstat prefix):
- openSync(path, O_RDONLY | O_NOFOLLOW | O_NONBLOCK) — atomic symlink refusal;
  ELOOP → 'symlink', ENOENT → 'missing', else → 'read-error'
- fstatSync on the opened fd — authoritative mode/uid/nlink check:
  - st.mode mismatch 0o100600 → 'wrong-mode'
  - st.uid mismatch getuid() → 'wrong-owner'
  - st.nlink != 1 → 'multi-link' (hardlink attack per Apple Secure Coding
    Guide, CVE-2005-2519 class)
  - !st.isFile() → 'not-regular-file'

Content validation (contract §2):
- JSON.parse → 'malformed-json'
- schema_version semver-major != 1 → 'schema-version-unsupported'
  (accepts 1.x minor bumps; unknown fields ignored for forward-compat)
- wdaPort out of 8400-8410 integer range → 'out-of-range-port'
- sessionId mismatch vs request → 'session-mismatch'
- flowStartTimestamp < getStartupTimestamp() - 5min → 'stale-timestamp'

Input guardrails:
- sessionId regex [A-Za-z0-9_-]{16,64} + null-byte/slash rejection →
  'invalid-session-id' (path-traversal defense before any fs touch)

Log scrubbing (contract §5):
- All failure paths emit only the reason tag via logger.debug()
- No selector values, sessionIds, port numbers, paths, or uids in logs
- Verified by a cross-scenario scrub-assertion test

DI: { getuid, getStartupTimestamp } injected for deterministic tests.

22 specs pass. Tests use real fs tmpdirs (bypass memfs) because the module
relies on POSIX O_NOFOLLOW / hardlink semantics memfs doesn't implement.
… (B3)

Core iOS element-region resolver for /percy/maestro-screenshot. Single
GET /session/:sid/source per screenshot, parsed locally via fast-xml-parser,
mirrors the Android adb-hierarchy.js architecture.

Exports:
- resolveIosRegions({regions, sessionId, pngWidth, pngHeight, isPortrait, deps})
  → {resolvedRegions: [{elementSelector, boundingBox, algorithm}], warnings: []}
- shutdown() — aborts all in-flight WDA HTTP AbortControllers (wired to
  percy.stop() by B4)
- XCUI_ALLOWLIST — exported Set of ~80 XCUIElement.ElementType values from
  the Xcode 16 SDK (Apple XCUIElement.ElementType docs); serves as DoS
  guardrail per WDA issue #292

Resolution path (A1-chosen):
1. Landscape gate (isPortrait arg)
2. Kill-switch gate (process.env.PERCY_DISABLE_IOS_ELEMENT_REGIONS from
   startup env only; NOT tenant-forwarded via appPercy.env)
3. readWdaMeta dep returns port from realmobile-written wda-meta.json; port
   validated in 8400-8410 range
4. GET /wda/screen (loopback-only) → scale from integer `scale` field;
   fallback to width-ratio (pngWidth / logical_w) snapped to {2, 3};
   fail-closed on raw ratio outside [1.9, 3.1]; LRU cache cap 64 per-session
5. GET /session/:sid/source (loopback-only):
   - 20 MB response cap enforced BEFORE parse
   - Pre-parse DOCTYPE/ENTITY regex rejection (primary XXE defense)
   - fast-xml-parser with processEntities:false (defense-in-depth)
   - Cached per screenshot; all regions reuse single fetch
6. Per region:
   - Only `id` and `class` accepted in V1; `text`/`xpath` → selector-key-not-in-v1
   - class short-form (Button) normalized to long-form (XCUIElementTypeButton);
     rejected if normalized form not in allowlist → class-not-allowlisted
   - selector > 256 chars → selector-too-long
   - tree pre-order first match (zero-match on no-match)
   - scale points → pixels, validate in-bounds + non-trivial area (≥4×4) →
     bbox-out-of-bounds / bbox-too-small
   - outbound elementSelector.class uses normalized long-form (canonical form
     on Percy dashboard regardless of customer input style)

HTTP: @percy/client/utils#request via injectable httpClient dep; 500 ms
AbortController timeout per call; retries: 0 to keep timeout honest.
inflight Set tracks active controllers; shutdown() aborts all.

Log scrubbing (contract §5): reason tags only. Verified across all paths —
no selector values, sessionIds, ports, or coords in logs.

23 specs pass. Tests use an injectable fake httpClient + in-memory
handlers; no real network required.
…lay (B4)

Wires B1/B2/B3 into api.js's /percy/maestro-screenshot handler. For iOS
requests with element regions:

1. Parse IHDR from the already-read fileContent (one buffer read total —
   no extra fs hit). Failure → warn-skip all iOS element regions with
   png-unparseable; coord regions + screenshot upload continue.
2. Call resolveIosRegions() once per request with a real @percy/client/utils
   #request httpClient and a resolveWdaSession-wrapped readWdaMeta dep.
3. Surface each warning to percy.log.warn so support runbook tags are
   visible in Maestro stdout.
4. Walk the original regions array in input order; positional index into
   the sparse resolvedRegions produced by the resolver keeps coord and
   element regions interleaved correctly in the outbound Percy payload.

wda-hierarchy now returns a SPARSE array (one entry per input element
region; null = skipped) instead of a dense array. Preserves input ordering
when element and coord regions are interleaved. All B3 unit tests updated
accordingly (22 still pass).

percy.js stop() invokes wdaHierarchyShutdown() before server.close() to
abort in-flight WDA HTTP calls — http.request has no SIGKILL analog, so
a slow /source fetch could otherwise keep the event loop alive past
graceful-shutdown timeout.

api.test.js: replaced the pre-V1 iOS stub test (which asserted
"Element-based region selectors are not yet supported on iOS") with a V1
behavioral test that exercises the full iOS element-region pipeline with
a real PNG IHDR header fixture (1170×2532 iPhone 14) and asserts V1
warn-skip semantics for an Android-style `resource-id` selector on iOS
(not-in-V1 key).

Test suite baseline: 28 pre-existing failures (chromium/doctor download
tests unrelated to this change). After B4: 27 failures — same chromium/
doctor failures, plus the iOS stub test now passes with its updated V1
assertions. Zero iOS/wda-hierarchy/maestro-screenshot regressions.

Kill-switch (PERCY_DISABLE_IOS_ELEMENT_REGIONS=1) read from Percy CLI
process startup env inside wda-hierarchy.js per plan — host-level only,
NOT forwarded from tenant appPercy.env (A0.3 property: pending staging
verification).
…retries

Implements the three layered fixes documented in
percy-maestro/docs/solutions/integration-issues/ios-wda-session-id-and-node14-abortcontroller-2026-04-23.md.
Each addresses a distinct iOS-region failure mode that surfaced during
2026-04-23 BrowserStack live validation on host 52:

Fix C — Node 14 AbortController feature-detect (callWda):
  BS iOS hosts pin to Node 14.17.3 (Nix). AbortController became a global
  in Node 15. Without feature detection, the timeout path threw
  ReferenceError caught by generic error handling and surfaced as the
  same 'wda-error' tag as legitimate WDA failures, masking the other two
  fixes during diagnosis. Now: typeof globalThis.AbortController guard +
  Promise.race fallback. Adds diagnostic logging on /wda/screen failures
  showing err.name/message/code/status/aborted/body.

Fix B — Stale WDA sessionId retry via error-envelope extraction:
  WDA's session-scoped routes (/session/:sid/source) reject any sid that
  isn't the currently-active session. Maestro spawns its own WDA session
  per xctest run, so realmobile's write-time sid capture goes stale
  during the test. Refactored fetchAndParseSource into tryFetchSource +
  retry coordinator. On staleSession (`{ value: { error: 'invalid session
  id' } }`), extracts the top-level `sessionId` from the error envelope
  (authoritative for "currently active") and retries once. Falls back to
  /status probe if the error body lacks a usable sid.

Fix A (reader side) — wdaSessionId surfacing per contract v1.1.0:
  realmobile contract v1.1.0+ probes /status at write_wda_meta time and
  surfaces the WDA UUID under wda-meta.json's optional `wdaSessionId`
  field. wda-session-resolver now validates the field against
  /^[A-Fa-f0-9-]{16,64}$/ (generous bounds for cross-version tolerance)
  and surfaces it on the {ok, port, wdaSessionId?} return shape. v1.0.0
  writers that omit the field cause callers to fall back to SDK
  sessionId (the fast path 404s, then Fix B's retry recovers).

Tests cover all three paths: feature-detected timeout, staleSession
retry from error envelope, /status fallback when error body lacks sid,
v1.1.0 wdaSessionId pass-through, v1.0.0 absence handling, malformed
wdaSessionId rejection.

Note for downstream: this WDA-direct path is gated for deletion by the
2026-04-27 plan (percy-maestro/docs/plans/2026-04-27-001-feat-ios-element-regions-maestro-hierarchy-plan.md)
once Phase 0.5 empirical probe passes. Until then, this is the
production iOS resolver path.
The Android view-hierarchy resolver is becoming the cross-platform Maestro
resolver (per percy-maestro/docs/plans/2026-04-27-001-feat-ios-element-regions-maestro-hierarchy-plan.md
Unit 1). Rename + shim is purely additive — no behavior change.

- Move src/adb-hierarchy.js → src/maestro-hierarchy.js (git mv preserves history).
- Move test/unit/adb-hierarchy.test.js → test/unit/maestro-hierarchy.test.js.
- Move test/fixtures/adb-hierarchy/ → test/fixtures/maestro-hierarchy/.
- Replace src/adb-hierarchy.js with a 5-line re-export shim. Removed in V1.1
  per the plan's deprecation guidance.
- Update api.js import to ./maestro-hierarchy.js.
- Update logger namespace from core:adb-hierarchy → core:maestro-hierarchy.
- Update file header to reflect cross-platform intent (the file body has been
  maestro-first for some time; the previous file name was always misleading).
- Update test describe block + import + fixture path.

Behavior unchanged. Subsequent units in Phase 1 will add the iOS branch and
api.js dispatch logic; this commit is just the rename so the diffs in those
units stay focused.
…parity

Phase 1 Unit 2a per percy-maestro/docs/plans/2026-04-27-001-feat-ios-element-regions-maestro-hierarchy-plan.md.
Lands the platform-dispatch scaffolding and the cross-platform selector
vocabulary alias. Real iOS resolver implementation deferred to Unit 2b
post Phase 0.5 fixture capture (FIXME-PHASE-0.5 in code).

Platform dispatch:
- dump({ platform }) accepts 'android' (default — backwards compatible) or
  'ios'. iOS branch reads PERCY_IOS_DEVICE_UDID + PERCY_IOS_DRIVER_HOST_PORT
  from env (realmobile-injected per Unit 10a; the wda_port + 2700 formula
  is realmobile-owned per maestro_session.rb:831). Warn-skip with
  reason='env-missing' if either var is unset. Otherwise calls
  runMaestroIosDump which currently returns
  { kind: 'unavailable', reason: 'not-implemented' } as the FIXME-PHASE-0.5
  stub.
- iOS path never invokes adb (verified by test).

R1 vocabulary parity (Android `id` alias):
- flattenMaestroNodes (Android branch) now surfaces resource-id under both
  `resource-id` AND `id` canonical keys on each node. Customer selectors
  `{element: {id: "submit-btn"}}` and `{element: {resource-id: "submit-btn"}}`
  resolve the same node. iOS users writing `{id: ...}` and Android users
  writing the same yaml hit the same code path. Full unified-key migration
  (deprecating `resource-id`) deferred to V1.1.
- SELECTOR_KEYS_UNION = [resource-id, text, content-desc, class, id]
  drives firstMatch validation. ANDROID_SELECTOR_KEYS_WHITELIST and
  IOS_SELECTOR_KEYS_WHITELIST exported separately for callers that want
  per-platform validation.

Tests added:
- Android `id` alias resolves same bbox as `resource-id` (3 tests).
- iOS env-missing path (3 tests covering each env-var combination).
- iOS env-set returns 'not-implemented' (FIXME stub).
- iOS dispatch never invokes adb.
- Default (no platform arg) preserves Android behavior.

Smoke-tested via direct node import; full @percy/core test suite has 27
pre-existing failures in Unit / Install in executable Chromium (unrelated
infrastructure issue), but no regressions in the resolver tests.
Phase 1 Unit 3 per percy-maestro/docs/plans/2026-04-27-001-feat-ios-element-regions-maestro-hierarchy-plan.md.

Wires the maestro-hierarchy resolver into the /percy/maestro-screenshot
relay's iOS element-region dispatch, gated by an env switch so default
(unset) behavior is unchanged. Phase 0.5 empirical probe gates the
default flip to the new path; Phase 4 deletes the legacy iOS branch.

- New: read PERCY_IOS_RESOLVER from process.env. When equal to
  'maestro-hierarchy', iOS element regions flow through the same
  lazy maestroDump({ platform: 'ios' }) + per-region firstMatch
  pattern Android already uses. When unset (or any other value),
  legacy WDA-direct path remains active — no behavior change for
  customers in production today.
- Refactor: the up-front PNG-parse + resolveIosRegions block now only
  fires when the env switch is OFF. With the switch on, that work is
  unnecessary (the resolver is engineered to be lazy + per-region).
- The cross-platform branch in the per-region loop now also covers iOS
  when the switch is on. Same shape as Android: cachedDump lazy memo,
  warn-skip on hierarchy-unavailable, firstMatch + bbox forward on
  success.

Today (env switch unset): only the cross-platform Android path is
exercised. The iOS branch with the switch on is exercised by the
maestro-hierarchy unit tests landed in Unit 2a (which covers the
'env-missing' and 'not-implemented' stub paths). Unit 4 adds the
parity test that exercises both platforms via the same handler.

A real production rollout flips the default to 'maestro-hierarchy'
in Phase 4 (Unit 9) after Phase 0.5 PASSes; until then, keep the
default off.
Phase 1 Unit 4 per percy-maestro/docs/plans/2026-04-27-001-feat-ios-element-regions-maestro-hierarchy-plan.md.

New test file: test/unit/maestro-hierarchy.parity.test.js. Locks in the
contract that both platform branches return the same { kind, ... } envelope,
that the public API surface (SELECTOR_KEYS_WHITELIST + per-platform
whitelists) is consistent, and that platform dispatch isolates the env-var
reads (Android never reads PERCY_IOS_*; iOS never reads ANDROID_SERIAL).

Bug fix discovered during smoke test:
- flattenNodes (the XML/uiautomator code path) was missing the R1 `id`
  alias surface that flattenMaestroNodes (the maestro CLI JSON path)
  already had. So `firstMatch(nodes, { id: 'X' })` worked when nodes came
  from the maestro path but returned null when nodes came from the adb
  fallback path. Now both code paths surface resource-id under both
  `resource-id` and `id` keys consistently.

iOS-side parity assertions in this test are scoped to what Unit 2a's stub
can actually cover — envelope shape, whitelist exports, dispatch isolation.
The Phase 4 follow-up (post Phase 0.5 + Unit 2b) extends this file with
real iOS attribute-mapping assertions backed by a captured iOS hierarchy
fixture.

Smoke-tested via direct node import. The full @percy/core test suite has
27 pre-existing Chromium-installer failures unrelated to this work.
@Sriram567 Sriram567 changed the title feat(core): Phase 1 — iOS element regions via maestro hierarchy (cross-platform parity, behind env switch) feat(core): iOS element regions for Maestro (WDA-direct + maestro-hierarchy paths) May 4, 2026
@Sriram567 Sriram567 changed the base branch from feat/maestro-multipart-upload to master May 4, 2026 08:13
@Sriram567 Sriram567 marked this pull request as ready for review May 5, 2026 05:34
@Sriram567 Sriram567 requested a review from a team as a code owner May 5, 2026 05:34
Fixes the 128 lint errors that broke CI on this branch:
- object-property-newline: split inline { foo, bar } literals onto separate lines
  in api.js, api.test.js, wda-hierarchy.test.js
- no-multi-spaces: collapse alignment whitespace before trailing comments in
  wda-hierarchy.test.js and png-dimensions.test.js, and inside slice(0, 200) in
  wda-hierarchy.js
- import/no-duplicates: merge the two fs imports in wda-session-resolver.js
- no-unused-vars: drop unused `crypto` import in wda-hierarchy.js (sessionId
  hashing was scoped out) and unused `logger` named import in
  maestro-hierarchy.test.js

No behavior change. yarn lint now exits clean.
Comment thread packages/core/src/api.js
let entries;
try { entries = await fs.promises.readdir(dir, { withFileTypes: true }); } catch { return; }
for (let entry of entries) {
let full = path.join(dir, entry.name);
Comment thread packages/core/src/api.js
let entries;
try { entries = await fs.promises.readdir(dir, { withFileTypes: true }); } catch { return; }
for (let entry of entries) {
let full = path.join(dir, entry.name);
Comment thread packages/core/src/api.js
let baseDir = `/tmp/${sessionId}_test_suite/logs`;
let logDirs = await fs.promises.readdir(baseDir);
for (let dir of logDirs) {
let screenshotPath = path.join(baseDir, dir, 'screenshots', `${name}.png`);
Comment thread packages/core/src/api.js
let baseDir = `/tmp/${sessionId}_test_suite/logs`;
let logDirs = await fs.promises.readdir(baseDir);
for (let dir of logDirs) {
let screenshotPath = path.join(baseDir, dir, 'screenshots', `${name}.png`);
Comment thread packages/core/src/api.js
let baseDir = `/tmp/${sessionId}_test_suite/logs`;
let logDirs = await fs.promises.readdir(baseDir);
for (let dir of logDirs) {
let screenshotPath = path.join(baseDir, dir, 'screenshots', `${name}.png`);
import { setupTest } from '../helpers/index.js';

const fixtureDir = path.resolve(url.fileURLToPath(import.meta.url), '../../fixtures/maestro-hierarchy');
const loadFixture = name => fs.readFileSync(path.join(fixtureDir, name), 'utf8');
import { setupTest } from '../helpers/index.js';

const fixtureDir = path.resolve(url.fileURLToPath(import.meta.url), '../../fixtures/maestro-hierarchy');
const loadFixture = name => fs.readFileSync(path.join(fixtureDir, name), 'utf8');
}

function writeMeta(baseDir, sid, content, { mode = 0o600, dirMode = 0o700 } = {}) {
const sidDir = path.join(baseDir, sid);
}

function writeMeta(baseDir, sid, content, { mode = 0o600, dirMode = 0o700 } = {}) {
const sidDir = path.join(baseDir, sid);
const sidDir = path.join(baseDir, sid);
fs.mkdirSync(sidDir, { mode: dirMode });
fs.chmodSync(sidDir, dirMode); // mkdir mode is umask-masked
const file = path.join(sidDir, 'wda-meta.json');
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants