Skip to content

v0.7.0

Choose a tag to compare

@kblok kblok released this 24 Oct 21:01
· 390 commits to main since this release
1265579

What's Changed

  • Audit COMPATIBILITY.md against upstream main (ee901b3) by @kblok in #463
  • Accept zero for EAS minute limits and name the bad option by @kblok in #464
  • Match Android test ids without their package prefix in DeviceLocator by @kblok in #465
  • Redact explore finding text and cap reportedFindings by @kblok in #466
  • Observe responses before the first navigation by @kblok in #467
  • Add OnAction hook to agent.act by @kblok in #469
  • Resolve locators in the engine, so an unreadable Android snapshot keeps polling by @kblok in #474
  • Show what each step did, the hook it ran in, and what the app logged on the trace page by @kblok in #460
  • Add Browser.UnrouteAllAsync by @kblok in #476
  • Keep artifact screenshots after a secret fill by @kblok in #477
  • Fail a model GitHub Copilot does not serve with MODEL_UNAVAILABLE, naming the model by @kblok in #480
  • Record the agent's turns of a step and show them on the trace page by @kblok in #481
  • Pass the engine's measured keyboard state to executors by @kblok in #482
  • Add Screen.TreeAsync and Screen.DebugAsync by @kblok in #478
  • Explore: accept severity 3.0 and record a cancelled step by @kblok in #485
  • Name an attempt's directory with resultSegment by @kblok in #486
  • Tell what became of each step's recording and which model recorded it by @kblok in #488
  • Test the stored model attribution of a cache entry and document it in the debugging guide by @kblok in #489
  • Explore: surface a cancel in the finding tool as a cancel by @kblok in #490
  • Failure evidence: look at the app before the test's own teardown by @kblok in #491
  • Show the prompt cache split and the cost of each model call in the agent step table by @kblok in #492
  • Read an empty iOS text field as empty, not as its placeholder by @kblok in #494
  • Port the OAuth errors and token-endpoint tests by @kblok in #495
  • Trace page: tell every attempt of a test on one page by @kblok in #497
  • ToHaveScreenshotAsync fails with POLICY_DENIED when masking is unproven by @kblok in #498
  • Make the missing-session DevFlow test independent of the clock by @kblok in #502
  • Read the OAuth token response through readJsonBody by @kblok in #500
  • Attempt directories: tell two tests with one title apart by their class by @kblok in #504
  • Fix the flaky timed-out body test in ExpectSoftInARunTests by @kblok in #505
  • MediaEmulationTests: wait for a popup's media instead of reading it at load by @kblok in #506
  • Report: add the report-1 schema and the atomic report writer by @kblok in #507
  • Test AgentDiscovery.WaitForStatusAsync deadline and never-answers paths by @kblok in #508
  • Record StreamRedactor as not ported by @kblok in #509
  • Port the secret-never-reaches-the-model test from upstream 5b6a731e by @kblok in #510
  • Follow upstream 5b6a731e in the explore tests by @kblok in #511
  • Build the report's run-level parts: targets, summary, usage, limits, status, VCS by @kblok in #514
  • Port the merged unit tests from upstream fe2c3450 by @kblok in #516
  • Port the agent and cache test changes from upstream adcfc2ba and c9445ce6 by @kblok in #517
  • Mask a password inside a closed shadow root in artifact screenshots by @kblok in #519
  • EngineLocateTests: give the retry test a deadline that fits its poll sleeps by @kblok in #520
  • COMPATIBILITY: list the upstream test-only tests the port does not run by @kblok in #522
  • Give the step start observation Truncated and Path by @kblok in #525
  • Port the engine cancellation guards and merged web tests from upstream a13e119c by @kblok in #521
  • Remove port tests whose upstream it was deleted or merged by @kblok in #526
  • Extract StaleOr and port the upstream staleOr tests by @kblok in #530
  • Stop the test host crash from a connection reset before accept by @kblok in #529
  • Write report.json after every attempt by @kblok in #531
  • Bind the OAuth callback server to the IPv4 loopback address by @kblok in #535
  • Remove DecideTraceReplayThroughTheActTests, which have no upstream it by @kblok in #536
  • Wait through an iOS app not running yet after a launch by @kblok in #537
  • Port the tests that need a report now that report.json exists by @kblok in #541
  • Expose the closed session report as E2ESession.Report() by @kblok in #538
  • Classify a content-filter refusal as MODEL_REFUSED by @kblok in #539
  • Report: a blocked run reads exit code 2 by @kblok in #545
  • Reopen the app inside the app.restart and app.clearState step by @kblok in #546
  • Prefix a failed or blocked verdict message with the api and status by @kblok in #547
  • Return a typed Report from E2ESession.Report() by @kblok in #550
  • Make the cleanup budget test independent of machine speed by @kblok in #548
  • Skills: start each test near its screen and fix a failed test from its trace page by @kblok in #555
  • TestMu: take upstream's Edge flag, idleTimeout and project defaults, and capability validation by @kblok in #556
  • Mobile: start the iOS keyboard-dismissing drag clear of controls by @kblok in #557
  • Add a CodeRabbit upstream parity check (warning) by @kblok in #559
  • Pin agent-device 0.21.29 (upstream 07eb481) by @kblok in #561
  • Pick the engine of a config target in e2e explore and e2e mcp by @kblok in #560
  • Make the CodeRabbit Upstream parity check an error by @kblok in #563
  • Launch a device's pinned app before the first observation in explore and mcp by @kblok in #562
  • Make CodeRabbit request changes while findings are open by @kblok in #564
  • Bump Microsoft.Playwright to 1.64.0 by @kblok in #565
  • Support hosted device providers in e2e explore and e2e mcp by @kblok in #566
  • Keep a timeout a timeout in StaleOr and NavigationStaleOr by @kblok in #572
  • Note truncated agent observations in the report (upstream #1301) by @kblok in #575
  • Show the test line for each action in an e2e mcp session by @kblok in #579
  • Keep the request prefix a Claude thinking block is bound to by @kblok in #578
  • Resolve test lines through the engine's LocateAsync by @kblok in #580
  • Extract openStandaloneAttempt and port upstream's host lifecycle tests by @kblok in #582
  • Make a zero git budget deterministic by @kblok in #583
  • docs: record why upstream's off-site iframe text is not ported (#1333) by @kblok in #594
  • Run the drafted test lines of an e2e mcp session in a test run by @kblok in #584
  • docs(skills): rerun one test with the replay cache on while iterating by @kblok in #595
  • test(cli): bound the drafted run's output reads and escape its project paths by @kblok in #603
  • Keep the MCP session open between test runs (upstream #1313) by @kblok in #597
  • test(cli): make the CLI tests pass on Windows by @kblok in #604
  • fix(mobile): keep the known viewport when the app reads smaller than the screen by @kblok in #596
  • Ignore presentational role on focusable controls and named images (upstream #871) by @kblok in #598
  • Put the test title in agent context (upstream #1323) by @kblok in #601
  • Document ReportStep.ObservationTruncated in the skill and API docs (upstream #1301) by @kblok in #620
  • Record the mobile known-viewport rule from upstream #1291 in COMPATIBILITY.md by @kblok in #619
  • Document presentational-role conflicts and named empty-alt images in the skill (upstream #871) by @kblok in #616
  • Join snapshot lines with LF, as upstream (#618) by @kblok in #628
  • Report cache eviction only after store deletion succeeds by @kblok in #599
  • Combine locator alternatives with Or (upstream #1049) by @kblok in #600
  • Exclude hidden select choices before observation limits (upstream #900) by @kblok in #602
  • Document the engine prepare/finish hooks and the host lifecycle (upstream parity for #582) by @kblok in #622
  • List an iframe with a presentational role as a frame boundary by @kblok in #630
  • fix(tests): make the E2E.Tests Windows-only failures pass by @kblok in #605
  • Match upstream's shape of the thinking-prefix handling (#578 follow-up) by @kblok in #624
  • Add the Windows app engine and a windows target for explore and mcp by @kblok in #621
  • Write cache, screenshot, and AI trace files through AtomicWrite by @kblok in #627
  • Report a failed engine finish as a cleanup error of the exploration by @kblok in #629
  • Keep the test title out of the system prompt so prompt caching spans tests (upstream #1344) by @kblok in #631

Full Changelog: v0.6.0...v0.7.0