Skip to content

refactor: make touch target resolution policy explicit #509

@thymikee

Description

@thymikee

Context

PR #508 fixed #507 by preserving the resolved node center for semantically touchable iOS TabView buttons even when XCTest reports them as hittable:false.

That fix exposed a maintainability issue: touch coordinate resolution currently mixes several subtly different ideas in one place:

  • hittable:true ancestor promotion for labels/text inside rows
  • non-hittable but semantically touchable controls with valid rects
  • same-rect descendant promotion
  • overly broad ancestor avoidance
  • role/type string heuristics that overlap with screenshot overlay logic

The current behavior is working, but the policy would be easier to maintain if those rules were named, shared where appropriate, and tested directly.

Goal

Refactor touch target resolution so future wrong-tap bugs are easier to reason about, without changing behavior.

Suggested Scope

Keep this scoped to the interaction target resolver and directly related tests/helpers.

Likely files:

  • src/commands/interaction-targeting.ts
  • src/__tests__/runtime-interactions.test.ts
  • Possibly src/daemon/screenshot-overlay.ts only if extracting shared semantics clearly reduces duplication
  • Possibly a new helper under src/utils/ if the extracted policy is genuinely shared

Implementation Guidance

  1. Extract a clearly named helper for the semantic-touchability predicate introduced in fix: preserve iOS tab button tap centers #508.

    • Prefer a narrow name such as isSemanticTouchTarget or isTouchCoordinateSource.
    • Avoid generic names like isInteractive, because snapshot filtering, overlay refs, selector reads, and tap resolution do not all mean the same thing.
  2. Consider returning a resolver decision object from resolveActionableTouchNode, for example:

type ActionableTouchResolution = {
  node: SnapshotNode;
  reason:
    | 'same-rect-descendant'
    | 'semantic-target'
    | 'hittable-ancestor'
    | 'overly-broad-ancestor'
    | 'original';
};

If changing the return shape creates too much churn, keep the public helper returning a node and add an internal helper that returns the reason for tests.

  1. Keep the resolver priority explicit:

    • Prefer same-rect hittable descendants.
    • Keep the original node if it is a semantic touch target with valid bounds.
    • Promote to the nearest hittable ancestor when appropriate.
    • Do not promote to overly broad viewport/window-like ancestors.
    • Fall back to the original node.
  2. If sharing with screenshot overlay logic, preserve the different contracts.

    • Overlay actionability is for visual/reference presentation.
    • Touch target resolution is for choosing a coordinate to send to the platform.
    • Do not force both call sites to use a helper unless the helper name and behavior match both domains.

Tests

Add or update focused tests around the policy decisions, not just final coordinates:

  • non-hittable iOS tab/button with valid rect keeps its own center
  • static text inside a hittable row still promotes to the row
  • same-rect hittable descendant still wins where currently expected
  • full-screen/window-like ancestor does not steal the tap

If resolver reasons are exposed internally, assert the reason in tests.

Non-goals

  • Do not change CLI behavior.
  • Do not change iOS runner snapshot generation unless a separate bug is found.
  • Do not broaden this into selector parsing, replay healing, or platform dispatch.
  • Do not add new user-facing docs unless behavior changes.

Validation

Run the checks required for a TS interaction resolver change:

  • pnpm format
  • pnpm exec vitest run src/__tests__/runtime-interactions.test.ts
  • pnpm check:quick
  • pnpm check:unit if the daemon/shared interaction path changes beyond the resolver tests

Manual device verification is not required if behavior is unchanged, but it is useful to re-run the #507 repro if any coordinate policy changes are made.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions