Skip to content

0.80.0

Choose a tag to compare

@takahirom takahirom released this 01 Sep 02:11
· 43 commits to main since this release
be2a55c

Everything here comes from measuring replayWithFallback against a real CI suite. The mode shipped
in 0.79.0 saved far less than it should have, and the numbers pointed at three separate causes.

A Fallback Now Costs Only the Task That Failed (#423)

A replayed scenario that failed anywhere restarted every task under the AI, including the tasks
that had just replayed perfectly — the whole saving the mode exists for, thrown away by one
mismatched screen.

Now only the failing task is re-run by the AI. The tasks before it keep what they replayed, and the
tasks after it go back to replaying:

Replay fallback for scenario checkout, task 3: an image assertion failed.
Re-running this task in normal mode and keeping the tasks before it.
-> resumed Replay at task(s) [4, 5]

The trace recorded on success is stitched back together so the next run replays the task from its
real starting point. A task that begins by resetting the device — launchApp, cleanupData,
openLink, or a Maestro flow — records its re-run on its own instead, since it is back at the same
place either way.

A Replay Target Is Identified by the Text in Its Descendants (#420)

Replay re-locates each recorded element before acting on it, so an index that has shifted since the
recording is either rebound or reported as a divergence. That only works if the element could be
identified in the first place.

Identity was read from the node's own attributes only. On Android TV that is exactly the wrong
place to look: the focusable container carries no text, resource-id or accessibility text at all,
and the label lives in a child TextView — the same descendant the AI is shown. So no identity was
recorded, and a replayed index action silently operated on whatever happened to sit at that index.
Identity is now searched over descendants the same way.

Image Assertions See the Screenshot Before the One They Judge (#421, #422)

imageAssertionHistoryCount: 2 was handing the assertion the current screenshot and then skipping
a step backwards, so a two-image assertion had nothing to compare against until the third step of a
task and then compared non-adjacent screens. An assertion about what changed between images cannot
pass while it is handed only one.

The history now starts at the immediately preceding screenshot. When a task is too early to have
that many, the missing frames are taken then and there rather than letting the assertion fail for
lack of images, and the whole list is ordered newest first however it was filled.

What's Changed

  • Identify a replay target by text found in its descendants by @takahirom in #420
  • Start image assertion history at the previous screenshot by @takahirom in #421
  • Take the missing frames when assertion history is short by @takahirom in #422
  • Fall back to the AI for the task that failed, not the whole scenario by @takahirom in #423

Full Changelog: 0.79.1...0.80.0