Skip to content

v1.1.25

Choose a tag to compare

View all tags
@omnarayan omnarayan released this 25 Aug 05:14
· 35 commits to main since this release
v1.1.25
c766416

Release 1.1.25

This release is about the first five minutes and the oldest complaints: a doctor that says exactly what your machine is missing, a devices listing, a screenshot command, runShell for the one adb call every suite eventually needs, and an inputText that checks what actually landed in the field. It is also about gestures that behave like a user's finger: a real dragAndDrop on every driver — press, hold until the item lifts, move slowly, settle, release — and a scrollUntilVisible that stops when the element is actually visible instead of one pixel in. Flutter apps go from barely drivable to ahead of WDA on the DeviceLab iOS driver, --step-delay slows any flow down for demos and animation-heavy apps, and JUnit reports carry test-tracking properties and failure artifacts into CI.

Added

  • doctor — checks the toolchain and says what to do about each gap, with --json for CI. A missing platform is a warning rather than an error, so it gates a Linux runner without failing it for having no Xcode; only a broken install or a team ID Xcode does not have exits non-zero. Three checks earn their place: Command-Line-Tools-instead-of-full-Xcode, which otherwise surfaces far from its cause; an AVD naming a system image that is not installed, which nothing reports at run time and which shows up only as a driver polling for a device that will never boot; and --team-id checked against the signing certificates in the login keychain, which is the entire content of error: No Account for Team "..." at WDA build time. 
    ✗ Signing identity for team abcd123456 — no certificate belongs to team abcd123456;
  this Mac has: A3RCAA2YAX
    Pass one of the team IDs listed above as --team-id, or add that team's account in Xcode.
  • devices — Android devices and emulators, iOS simulators and connected phones in one listing, with --json and --all for the shut-down simulators. A phone that answers usbmux but refuses the lockdown handshake is listed as not-ready with the reason, instead of looking identical to a working one.
  • screenshot — capture the current screen with the same device and driver flags a run takes. Writes a file, or - for stdout.
  • runShell — run a host command from a flow, for the adb, simctl or xcrun call every suite eventually needs. 
    - runShell:
    command: adb -s $MAESTRO_DEVICE_ID shell getprop ro.build.version.sdk
    output: SDK
    output: binds the command's trimmed output to a flow variable for later steps. The environment carries the flow's variables, the step's own env:, and MAESTRO_DEVICE_ID / MAESTRO_PLATFORM / MAESTRO_APP_ID — the device id being the one that matters, since a bare adb shell fails outright when --parallel has two devices attached. Bounded at 30s by default (timeout: to change it), output capped at 64KB keeping the tail, and a non-zero exit fails the step unless it is optional: true.
  • scrollUntilVisible can scroll inside a container — from: names the scrollable element, for a screen whose scrolling part is an inner list or a horizontal carousel rather than the screen itself. The gesture is centred in the container and inset at each end, so a swipe starting on its edge is not claimed by the parent list instead. UIAutomator2 for now; the other drivers refuse from: by name rather than silently scrolling the screen. 
    - scrollUntilVisible:
    element:
      id: "product-item-Playwright"
    from:
      id: "products-list"
  • --video never|always|on-failure — keep screen recordings only for the runs worth watching. --record still means always. A recording cannot be started retroactively, so on-failure records every flow and discards the passing ones at the end. The vocabulary matches --artifacts rather than inventing a second spelling for the same idea.
  • lint --json — {checked, failed, results[]} on stdout, for editors, CI and anything generating flows that wants to check its own output before it costs a device.
  • dragAndDrop — long-press an element (or point) and drag it to another, the way reorder UIs expect. from:/to: each take a selector or a point:; holdDuration (ms, default 1000) is the press before movement, duration (ms, default 1000) the movement itself. Works on every driver: UIAutomator2 (W3C actions with precise hold and paced moves), DeviceLab Android (input draganddrop, Android 12+ — hold length comes from the system long-press timeout), WDA (press(forDuration:thenDragTo:) — XCUITest paces the move itself), DeviceLab iOS (runner drag with a new backward-compatible moveDurationMs field), web (paced CDP mouse sequence), and Appium (W3C actions). 
    - dragAndDrop:
    from:
      id: "item-3"
    to:
      point: "50%, 20%"
    holdDuration: 800
    Heads-up for web: pages using native HTML5 draggable ignore synthetic mouse events by design — mouse/touch/pointer-event drag implementations work.
  • --step-delay <ms> — pace a pause between top-level steps, for demos and for apps whose animations outrun the assertions. Also MAESTRO_STEP_DELAY in the environment, and per-flow override with stepDelay: in the flow config.
  • Flow properties: in JUnit reports — a flow's properties: map (Maestro-compatible syntax) now lands as <property> entries on its JUnit testcase, next to the standard file and device properties — so flows can carry test-tracking ids into CI (#84). 
    properties:
  testID: Test-1234
  • Faster parallel runs — the work queue is filled longest-flow-first using durations from the previous run, instead of file order. A run cannot finish before its longest remaining flow does, so one slow flow late in the alphabet used to set the wall clock for every worker. Flows with no recorded time are weighted as the median. Best-effort: no previous run, or an unreadable report, falls back to file order.
  • A mid-flow app death is explained rather than blamed on the selector (Android) — when the app under test dies, the runner now asks the platform why instead of reporting a bare "element not found". That covers the two cases logcat cannot see: a low-memory kill writes nothing at all, and a system resource kill writes nothing useful. The DeviceLab driver had no crash detection whatsoever and is now level with UIAutomator2.
  • Failure artifacts in JUnit reports — failed testcases attach the failing step's screenshot, and any --record video, via the [[ATTACHMENT|path]] convention Jenkins-style tooling reads; paths are relative to the report directory. A green run attaches only the video.
  • Negation globs in config.yaml — a flows: pattern prefixed with ! subtracts the files it would otherwise have selected, so a workspace can take everything and carve out the fixtures. Exclusions reuse the inclusion matcher, so !a/** excludes exactly what a/** would have included. 
    flows:
  - "**"
  - "!fixtures/**"
  • DeviceLab iOS targets the app that's actually on screen — a command with no appId used to activate the runner's own placeholder host app, hijacking whatever you were looking at with a blank screen, because XCUITest's public API can't address "the app in front". It now resolves the frontmost application through the active-application PIDs and targets it with no activation at all, falling back to the host app (and logging which guard missed) when any step is unavailable.
  • The DeviceLab iOS runner source ships inside the Go module — library consumers previously built the runner from whatever was in ~/.maestro-runner/drivers/ios, a hidden runtime dependency and a protocol-skew hazard where a pinned Go client could drive runner sources from a different version. The runner a consumer runs is now version-locked to the module it compiled against, with no maestro-runner installation needed. The CLI keeps using the installed directory — identical content, shared build cache.

Fixed

  • inputText reported success without checking that the text arrived — nothing read the field back, so a character lost to a janky frame was indistinguishable from a clean run, and the flow failed several steps later on an assertion that had nothing to do with the cause. Every Android path now reads the field after typing and, when characters are missing, clears and types once more; a dropped keystroke is a timing accident, and the second attempt almost always succeeds. Three rules keep it from causing more harm than it prevents: the check is a suffix rather than an equality, because mobile text entry appends to whatever the field held; a reading that did not change is treated as telling us nothing, because some drivers report an empty field's hint, so "Username" comes back whether or not a name was typed; and a value at least as long as what was typed has been reformatted by the app — a phone mask, an autocomplete, a secure field reporting bullets — so those are recognised and left alone. A field that still disagrees after the retry is reported in the step result rather than failed. Measured on a Pixel 4a at roughly 75ms per field. Not yet wired on WDA or DeviceLab iOS.

  • Two device errors described something other than what had happened — on iOS, with nothing booted and no device named, an empty device list was reported as a code-signing error demanding --team-id, because the "is this a simulator?" check correctly answers no when there is no simulator. An empty device list now says so, before signing is considered at all. On Android, the parallel path reported "no available Android devices found" whether nothing was attached or every attached device was already being driven by another maestro-runner; those have completely different fixes, and offline or unauthorized devices are now counted separately again. Both point at maestro-runner devices for what the machine can actually see.

  • scrollUntilVisible stopped on elements it shouldn't have — every driver accepted any 1px viewport overlap (or mere presence) as "visible", and the documented visibilityPercentage knob was parsed but wired to nothing. The stop criterion is now a real visibility check on every driver: fully visible by default, honoring visibilityPercentage when set. Heads-up: flows that relied on a sliver of the element counting as visible may scroll one step further now. On DeviceLab iOS, frames that arrive pre-clipped to the viewport (Flutter semantics especially) additionally need to hold still across one extra scroll before being accepted, so a 12pt sliver of an 80pt row can't masquerade as fully visible.

  • Flutter apps were undrivable on the DeviceLab iOS driver — four stacked causes, each fixed: the Flutter VM fallback was wired only into the WDA path; the driver implemented neither the coordinate-tap step the fallback delivers taps through nor tapOn: point:; the fallback's short find windows leaked into the runner's HTTP timeout and cut XCUITest calls off mid-flight; and scroll gestures released mid-motion, so iOS spent the next tap cancelling residual deceleration instead of activating anything — a silent no-op. Scrolls now hold still 250ms before lifting, the same dead-stop lesson the Android agent swipes learned. On the Flutter issue-repro suite the driver goes from 9/19 to 17/19 — ahead of WDA-with-fallback at 14/19; the two remaining failures reproduce open Flutter framework bugs and fail on every driver.

  • assertVisible on the Appium driver asserted presence, not visibility — it returned success as soon as the element was findable, and assertNotVisible required it to be unfindable, so a present-but-hidden element passed the first and failed the second. Both now check the displayed state, which was already being fetched and discarded. Heads-up: a flow that passed on a hidden element will now fail — that is the intent, but it is a behaviour change. Android only: XCUITest reports elements as not displayed that are plainly on screen, so the check is not applied there.

  • The Appium driver granted 32 permissions the app never asked for — launchApp walked a hardcoded list and issued one pm grant shell call per entry, ignoring the failures that most of them produced, since granting an undeclared permission raises a SecurityException. It now reads what the manifest declares and grants that list in a single call. On a Pixel 4a that is 33 requests and 2.0s down to 3 requests and 0.3s per launch, and it works on hosts that disable the adb_shell insecure feature, which the old approach silently did not.

  • The Appium driver ran every selector strategy against elements that were not there — finding by text tried six UiAutomator strategies in order, and while polling for an element that had not appeared yet all six missed, every cycle. Measured on a device, 22 of 30 finds in one flow were misses costing more than the successful finds. The case-insensitive forms are supersets of the rest, so two probes now decide whether anything matches before the specific strategies run.

  • The Appium driver fetched element properties nothing read — describing an element cost five round trips, of which enabled was never consumed and the accessibility description was wanted by one command out of nineteen. Now three, with the description fetched on demand.

  • point: was silently ignored on doubleTapOn and longPressOn by the Appium and web drivers, which tapped the element centre instead. Every driver now honours it, so a knob that worked on four drivers and was quietly dropped by two behaves the same everywhere. Fixing the web path surfaced a second dead option: web long press hardcoded a one-second hold and dropped duration: entirely.

  • assertScreenshot compared whatever frame arrived — capturing mid-animation seeds a baseline nothing will match again, and the failure that follows looks exactly like a real visual regression. It now re-captures until two consecutive frames agree, the same settle the drivers already used for waitForAnimationToEnd. A screen that never settles (a spinner, a video, a caret) falls through to the last frame rather than hanging.

  • DeviceLab iOS rejected clipped frames by scrolling again — iOS clamps a descendant's frame to its scroll container's visible bounds, so a sliver of a row far past the fold reads as fully visible and the tap that follows does nothing. Rather than spending an extra scroll to see whether a suspicious rect moves, the driver now checks the parent chain: a descendant claiming to be on screen beneath an ancestor that is wholly off it is a contradiction, found in one pass.

  • Web flow-header variables reached the browser unexpanded — a flow declaring url: ${BASE_URL} sent the literal ${BASE_URL} to Chromium, which rejected it as an invalid URL, so --platform web could not be driven from the environment at all. The header is now expanded through the same script engine, in the same precedence order, that expands steps — so -e, --env-file and workspace config all reach the initial navigation (#145). The header is now expanded once centrally, so Android and iOS stop silently dropping the app version from reports when a flow uses appId: ${VAR}, and an unset variable is named at startup instead of surfacing later as a bare "no URL specified".

  • A dead DeviceLab iOS runner left no trail — mid-session runner deaths surfaced only as connection refused: runner.log was truncated on every start, the runner exited on the first listener failure, and nothing distinguished a requested shutdown from a spontaneous one. The listener now rebinds on failure (5 retries, 1s backoff) rather than exiting, since simulator network daemons crash-loop on some Xcode/runtime combinations; every deliberate exit logs its reason; runner.log rotates through three generations so the log explaining a death survives the burst of failed restarts that follows it; and an xcodebuild that exits mid-session without a Stop being requested now says so on stderr.

Contributors

@humuhimi

  1. Reported and fixed web flow-header variables not being expanded before browser launch (#145, #146)

@eyalcohen

  1. Asked why assertVisible needed five element requests, which led to the Appium driver making roughly 38% fewer calls per flow

@zcsteele

  1. Requested custom properties in JUnit report files (#84)

Installation

Quick Install

curl -fsSL https://open.devicelab.dev/install/maestro-runner | bash

Install a specific version

curl -fsSL https://open.devicelab.dev/install/maestro-runner | bash -s -- --version 1.1.25

Manual install (from the assets below)

# 1. Download the tarball for your platform.
# 2. Extract:
tar -xzf maestro-runner-1.1.25-darwin-arm64.tar.gz

# 3. Run setup (installs into ~/.maestro-runner/bin/):
cd maestro-runner && ./setup.sh

Verify Installation

maestro-runner --version

Platform Support

  • macOS Intel (amd64) - Signed & Notarized
  • macOS Apple Silicon (arm64) - Signed & Notarized
  • Linux amd64
  • Linux arm64

Built by DeviceLab.dev

Assets 10
Loading