Skip to content

v1.8.0

Choose a tag to compare

View all tags
@igorzheludkov igorzheludkov released this 16 May 22:33
· 628 commits to main since this release
v1.8.0
d44dab2

First release since 1.6.16. Combines the queued 1.7.0 tap-improvements branch with the
input-handling primitives, OCR phrase reconstruction, Android coord accuracy work, and
several bug fixes from live testing against gifted.co and Boardwise.

New MCP tools

  • ios_input_text(text, udid?) — type text into the focused iOS field via the active UI driver (AXe/IDB). Mirrors android_input_text.
  • clear_focused_input — clear the currently focused TextInput via React onChangeText, keeping controlled-form state consistent. Bridgeless/Fabric only.
  • dismiss_keyboard — blur the currently focused TextInput.

New tool options

  • replace: true on both android_input_text and ios_input_text — clear the focused field before typing in a single round trip.

Tap engine

  • iOS labelContains / identifierContains now reject RN accessibilityLabel-rollup wrapper matches with an actionable suggestion (use component= / coordinates) instead of silently tapping the wrong centroid.
  • C1 ambiguity guards (fiber / iOS a11y / Android a11y) collapse geometrically-identical wrapper matches (e.g. ThemedButton → TouchableOpacity → TouchableOpacity at identical coords) before refusing.
  • screenshot and verify are now independent knobs. verify=true runs the diff even when screenshot=false; the response always carries verification — either a real diff or { skipped: true, skippedReason }.
  • OCR strategy reconstructs phrases from same-baseline word detections — recovers cases like Continue with Google when iOS Vision fragments the phrase due to a leading icon.
  • formatTapSuccess adds a TextInput-specific warning when meaningful=false for a focused input, pointing at the hardware-keyboard suppression case.

Screenshot enrichment

  • iOS screenshots detect native system overlays (auth sheets, permission dialogs, MSAL sign-in) and append a warning pointing the agent at tap(x, y, native=true). RN accessibility rollups are guarded out as false positives.
  • Android pressables list coords are reconciled against uiautomator so coord taps land on the actual element. Dynamic status-bar offset applied to icon-only fallbacks. Previously inflated coords (out of viewport on high-density emulators) are fixed.

Bug fixes

  • getPressableElements E1 fallback fires per-device, not on global connectedApps emptiness — a stale connection to an unrelated app no longer suppresses the a11y fallback.
  • fix(focused-input): match AndroidTextInput host fiber correctly for clear_focused_input / dismiss_keyboard.
  • fix(input-text-replace): thread device parameter through to the underlying clear call so multi-device sessions disambiguate correctly.
Assets 2
Loading