Skip to content

v0.9.0

Choose a tag to compare

@kblok kblok released this 08 Nov 20:34
· 192 commits to main since this release
da5bb69

What's Changed

  • Add a Releases section to the docs, starting with v0.8.0 by @kblok in #931
  • Restructure the docs sidebar and remove Samples by @kblok in #932
  • Fix docs accuracy from CodeRabbit review on #932 by @kblok in #933
  • Drop the last stale #648 line from COMPATIBILITY.md by @kblok in #934
  • Make the API reference stable: drop line numbers and regenerate by @kblok in #935
  • Add MCP start_recording and stop_recording (upstream e2e-inc/e2e#612) by @kblok in #936
  • Accept several targets in the config (TARGET_REQUIRED) by @kblok in #937
  • Fail a locator waitFor timeout with LOCATOR_NOT_FOUND and verify the earlier act by @kblok in #938
  • Give e2e mcp node ids and diff results (upstream e2e-inc/e2e#231) by @kblok in #939
  • Record the container key of a target and require it on replay by @kblok in #940
  • Accept credentials in the config by @kblok in #941
  • Accept the artifacts key and code-only cache and artifact stores by @kblok in #942
  • Port saved sessions to the xUnit v3 adapter by @kblok in #944
  • Print the failure details of a test run in a session and refuse an ambiguous one (upstream #1315, in part) by @kblok in #943
  • Read configured credentials in Credentials.User and e2e explore by @kblok in #945
  • Port the bug bash guide, docs page, and verification-skill fan-out by @kblok in #947
  • Fix the ToHaveURLAsync name in the docs by @kblok in #949
  • Port vision on act: the screenshot tool and pixel mode by @kblok in #951
  • Accept the output key and refuse a directory the run cannot own by @kblok in #946
  • Port the migration guides and the Limrun app-flow test by @kblok in #948
  • Port the pointer path, tap position, and selectOption forms by @kblok in #950
  • Offer configured credentials through type_secret in e2e mcp by @kblok in #952
  • Test the per-turn elision of stale screens against the thinking prefix by @kblok in #953
  • Fix more flaky timing tests by @kblok in #956
  • Port the markdown report renderer to the public Report tree by @kblok in #957
  • Port the agent verbs long_press, right_click, hover, and drag by @kblok in #960
  • Print Skipped After Failure, Warnings, and Repeats in the list reporter by @kblok in #961
  • Add the point tools (tap_at, hover_at, drag_at, type_at, press_at, select_at) by @kblok in #959
  • Wire test filters into xUnit v3 and into report.json by @kblok in #964
  • Make ReportExploreFinding.ArtifactId an artifact id, not a file path by @kblok in #966
  • Port the BrowserStack device provider and the lease helper by @kblok in #967
  • Choose reporters from e2e.config.json by @kblok in #969
  • Print the code frame and video rows under a failure in the list reporter by @kblok in #963
  • Add E2E.GitHub, the pull request comment reporter by @kblok in #962
  • Resolve reporter types when the run ends, not when the config loads by @kblok in #979
  • Bump agent-device to 0.21.36 by @kblok in #981
  • Rename EasSimulators to ExpoSimulators and use Expo's service names by @kblok in #982
  • Count steps whose recording was not saved in the Cache summary row by @kblok in #983
  • Name the agent-device session of a device-cloud lease by @kblok in #984
  • Read a date with an ordinal suffix as a date in cache anchors by @kblok in #980
  • Expose point coordinates in onAction for tapAt, hoverAt, dragAt by @kblok in #985
  • Rename the E2E.Eas package to E2E.Expo by @kblok in #987
  • Add e2e doctor to check the setup before a run (Refs #971) by @kblok in #988
  • Validate the shape of Expo Simulators answers in the sessions client by @kblok in #989
  • Print agent step lines and recording mismatch notices in the list reporter by @kblok in #991
  • Apply test filters to the mobile and Windows adapters by @kblok in #990
  • Port the engine keyboard: typeText, pressKey, and keyboard point tools by @kblok in #992
  • Fix stale lines in COMPATIBILITY.md, add serial groups and engine protocol entries by @kblok in #993
  • Carry who acts in an OperationContext so a test's step acts on a device without settling by @kblok in #995
  • Check mobile targets and models in e2e doctor (Closes #971) by @kblok in #996
  • Emit run events and give reporters traces and the AI trace path by @kblok in #997
  • Export the built-in agent's grammar tools, verdict tool, and rules by @kblok in #998
  • Rerun what failed with E2E_LAST_FAILED and fold it in the GitHub comment by @kblok in #999
  • Extract LocatorEngine so its upstream tests port as written by @kblok in #1002
  • Add E2E.Acp: run steps on an ACP coding agent by @kblok in #1000
  • Add Claude Code and Codex presets to E2E.Acp (Refs #743) by @kblok in #1004
  • Give a --last-failed rerun its own attempt directory by @kblok in #1010
  • Retry test-site listeners on a new port when the probed one is taken by @kblok in #1011
  • Finish the agent's point tools: dragAt join, press times, failure streak (Refs #632) by @kblok in #1013
  • List the candidates each filter step left on a failed filtered locator by @kblok in #1015
  • Open about:blank in the operation-budget CDP test by @kblok in #1017
  • Stop a paced drag on abort even when it is behind schedule by @kblok in #1016
  • Give the drawn keypad step a longer timeout by @kblok in #1019
  • Report a repainted screenshot under an unchanged listing in the built-in agent by @kblok in #1023
  • Port the list reporter's live window and console output (Refs #843) by @kblok in #1024
  • Scaffold a test project in e2e init, with --framework, --install and --example by @kblok in #1027
  • Run the slow last read tests on a fake clock by @kblok in #1029
  • Port the agent's dismiss_keyboard tool and the dismissKeyboard executor verb by @kblok in #1026
  • Port the list reporter goldens, Left out and Cleanup rows, and closing Failed rows by @kblok in #1030
  • List the elements a CSS selector names, so Browser.Locator("body") matches by @kblok in #1033
  • Render explore events and the live rows of an exploration in the list reporter by @kblok in #1031
  • Give the built-in agent upstream's #id listing and diff results by @kblok in #1034
  • Keep marked selector elements listed past the node cap and inside shadow roots by @kblok in #1040
  • Label replayed steps as recorded in an earlier run in the ledger by @kblok in #1041
  • Explore: runs up to 100 steps and 2 hours, edge-case lenses by @kblok in #1043
  • Let the built-in agent resize the viewport (resize_viewport) by @kblok in #1045
  • Follow up resize_viewport: stable enum order, validated replay (#1035) by @kblok in #1047
  • Tell the explorer it starts signed in when a session is restored by @kblok in #1048
  • Print e2e explore through the list reporter by @kblok in #1042
  • Run two more real-clock deadline tests on a fake clock (Refs #1006) by @kblok in #1049
  • Note that test.fail is not ported by @kblok in #1056
  • Test the explorer's signed-in context and fix a COMPATIBILITY.md typo by @kblok in #1057
  • Match the Playwright migration mapping to what e2e does by @kblok in #1061
  • Port Playwright's check, wait-for-url, locator-list and selectors-text tests by @kblok in #1064
  • Web engine: observe in turn, hidden iframe content reads hidden, screenshot past a held frame by @kblok in #1063
  • Read attributes of a text match inside a line by @kblok in #1065
  • Port Playwright's selectors and locator page tests (80 tests, Refs #1060) by @kblok in #1077
  • Port Playwright's fill, select-option, set-input-files and filechooser tests by @kblok in #1069
  • Quote a glob in a pattern step label, as upstream's patternLabel (Refs #1060) by @kblok in #1082
  • Read an empty box as hidden and match mixed and absent role states by @kblok in #1088
  • Clear file inputs with an empty list; binary bodies, header drops, set-cookie in routes by @kblok in #1058
  • Port Playwright's frame page tests (Refs #1060) by @kblok in #1070
  • Port Playwright's click, click-scroll and elementhandle-scroll-into-view tests by @kblok in #1084
  • Stop a frame-scoped query at a nested frame by @kblok in #1086
  • Fix window.Node, zero-sized label controls and scrollIntoView on hidden nodes by @kblok in #1090
  • Drop the known-difference markers of five Playwright tests by @kblok in #1091
  • Port Playwright's route, request-fallback and request-fulfill tests (Refs #1060) by @kblok in #1071
  • Port Playwright's keyboard tests by @kblok in #1085
  • Match text and labels on the DOM, as Playwright does by @kblok in #1089
  • Port Playwright's dialog and download tests (Refs #1060) by @kblok in #1078
  • Port Playwright's request-continue, wait-for and network-idle tests (Refs #1060) by @kblok in #1083
  • Point the report footer at e2e.com by @kblok in #1104
  • Share a pending screen-recording stop with cleanup by @kblok in #1106
  • Port timeout 0 as no step deadline by @kblok in #1108
  • Measure tap position from inside the border, port the remaining click tests by @kblok in #1112
  • Classify aborted navigations and capture screenshots across navigations by @kblok in #1113
  • Port Playwright's page screenshot tests by @kblok in #1111
  • Fix screenshot masks and subjects of Browser selector and frame locators by @kblok in #1115
  • Throw on inputValue and isChecked of the wrong element; uncheck clicks ARIA radios by @kblok in #1114
  • Document NativeScript test ids by @kblok in #1105
  • Make browser.evaluate survive clobbered page globals and return plain JSON only by @kblok in #1116
  • Run the screenshot tests that waited on #1109 and #1095 by @kblok in #1117
  • Add the full digit image set to the grid.html testbed page by @kblok in #1118
  • Extract the surface guard and port the three remaining surface-guard its by @kblok in #1121
  • Use SHA-256 for AgentObservation.PixelDigest, as upstream's pixelDigest by @kblok in #1123
  • Name the operation in a browser step's timeout message by @kblok in #1119
  • Stabilize the video and shared-app process tests like upstream by @kblok in #1128
  • Run the mobile input-budget tests on a clock the test moves by @kblok in #1129
  • Give custom executors upstream's #id listing as ExecutorObservation.Text by @kblok in #1133
  • Fix two web-engine races behind flaky tests (frame gone after count, chooser vs deadline) by @kblok in #1134
  • Bind the Codex login tests' callback server to an OS-picked port by @kblok in #1136
  • Apply upstream testbed edits: stale markers and buffer body test by @kblok in #1138
  • Hold the port in the never-ready ManagedProcess test by @kblok in #1139
  • Translate the engine error of every fixture call, not only Browser steps by @kblok in #1140
  • Raise the restore-build-test timeout to 30 minutes by @kblok in #1177
  • Port upstream artifact-segment tests (resultSegment) by @kblok in #1178
  • Port errors.test.ts its for exit codes, classify, provisioning, serialize, withHint, truncateUtf8 by @kblok in #1183
  • Upstream test: ids.test.ts its for test id join, result id bytes, and title validation by @kblok in #1209
  • Port the DialogRouter and native dialog kind tests (#1167) by @kblok in #1210
  • Port locators.test.ts frameSelectors its (Refs #1171) by @kblok in #1211
  • Port keyboard-shadow-focus.test.ts (#1168) by @kblok in #1212
  • Upstream test: nested-frames chain its and frame-screen its by @kblok in #1216
  • Upstream API: loop-guard thresholds option (checkLoopGuards / checkFailureStreak) by @kblok in #1217
  • Upstream test: protected-app-options.test.ts option validation its by @kblok in #1218
  • Upstream test: expect-missing-frame (whole-set matchers on a missing frame) by @kblok in #1214
  • Upstream test: single missing its in closed-shadow, line-text, media, clear-cookies and others by @kblok in #1219
  • Upstream test: testbed pointer.e2e.ts, five tests marked not ported by @kblok in #1234
  • Upstream test: observation-capture, semantic-fallback and stalled-frame its by @kblok in #1232
  • Port secret-ledger fragment whitespace and stream edge-case tests (#1160) by @kblok in #1227
  • Port web engine lifecycle.test.ts integration its (#1169) by @kblok in #1225
  • Upstream test: testbed todos.e2e.ts (7 tests on the playground /todos page) by @kblok in #1233
  • Port ElementList and RefRegistry, and upstream's element-list and refs-disposal tests (#1172) by @kblok in #1228
  • Upstream test: testbed dashboard.e2e.ts saved-session tests and auth.setup by @kblok in #1242
  • Upstream test: testbed controls.e2e.ts and network.e2e.ts remaining tests by @kblok in #1243
  • Upstream test: engine-secrets (page echo, base64 header redaction) by @kblok in #1245
  • Upstream test: testbed forms, wizard, smoke and board e2e tests by @kblok in #1247
  • Run the mobile shared-budget observe test on a clock the test moves by @kblok in #1141
  • Upstream API: MCP run_tests tool (#1520) documented as unported by @kblok in #1251
  • Upstream API: BROWSER_INSTALL_FAILED error code for failed browser provisioning by @kblok in #1252
  • Upstream test: plain-string-secrets (redacted title/id, typing the real value, output scan) by @kblok in #1249
  • Upstream test: config 'filters' validation (name rules, 256 limit, duplicates, holes) by @kblok in #1256
  • Upstream docs: environment variable reference (NO_COLOR, PLAYWRIGHT_BROWSERS_PATH, E2E_AGENT_DEVICE_POOL_*) by @kblok in #1255
  • Upstream API: derivedReason, the rule that keeps a run-derived typed value out of the replay cache by @kblok in #1257
  • Upstream test: FileCacheStore byte cap on write and non-writable store by @kblok in #1260
  • Map Credentials.User fallback names with ConfigSecrets.EnvName (#1261) by @kblok in #1263
  • Make the NUnit tests pass on Windows (temp-repo cleanup, contextmenu order) by @kblok in #1264
  • Upstream test: agent-policy 'tool calls outside the vocabulary' (unrecognized fields, batched turn, refusal streak) by @kblok in #1254
  • Upstream API: core 'press' key grammar check (internal/keys.ts requireKey) by @kblok in #1265
  • Upstream docs: when to use agentic or deterministic tests (#1583) by @kblok in #1266
  • Upstream API: cache relocation by position, placeholder and anonymous twins (relocate.ts), PR 1 of 2: extraction and recording by @kblok in #1267
  • Upstream test: tool-loop forced and downgraded tool choice (tool-loop-tool-choice), PR 1 of 3: forced-conclusion loop policy by @kblok in #1269
  • Upstream: PNG decode accepts 1/2/4-bit gray and compares a 16-bit transparent color at 16 bits (#1587) by @kblok in #1270
  • Upstream docs: Android switches and CheckBox report a checked state (#1585) by @kblok in #1271
  • e2e mcp redacts what user code prints (#1201) by @kblok in #1274
  • Upstream test: agent-config named-agent selection and 'did you mean' for an unknown agent by @kblok in #1276
  • Upstream test: web-platform 'clicks hold the modifiers they name' by @kblok in #1277
  • Upstream docs: writing an engine guide and engine reference (#1205) by @kblok in #1280
  • Upstream test: trace-recorder bare-point tap and drag recording its by @kblok in #1282
  • Upstream test: step-cache semantic-evidence its and recordedVerdictOf (#1186) by @kblok in #1284
  • Tool choice: PR 2/3 ToolChoice plumbing by @kblok in #1285
  • Upstream test: agent-trace-cache modes describe (--no-cache, CI read-only demotion) by @kblok in #1288
  • Relocation: PR 2/2 wire Relocate into replay (#1156) by @kblok in #1289
  • Upstream docs: error reference codes (#1203) by @kblok in #1291
  • Register the MCP server for Codex in .codex/config.toml (#1237) by @kblok in #1275
  • Upstream test: engine-contract its portable at session level (OperationContext, error mapping, requires, fixtures) by @kblok in #1294
  • Upstream test: trace-cache-replay record-then-replay flows (11 its) by @kblok in #1295
  • Upstream test: trace-cache-replay two targets and --repeat-each describes by @kblok in #1297
  • Upstream API: staleAdvice for REPLAY_STALE (cache dir, knob, custom store) by @kblok in #1298
  • Upstream docs: App.RestartAsync and ClearStateAsync name APP_UNREACHABLE (follow-up to #1294) by @kblok in #1299
  • Upstream test: agent-trace-cache end-state anchors and typed-value replay describes (#1182) by @kblok in #1302
  • Upstream API: resolved config digest (configDigest) and the digest its of config/agent-config (#1192) by @kblok in #1305
  • Tool choice: PR 3/3 recognizers, retries, forced initial mode by @kblok in #1301
  • Upstream test: agent-observation prepareObservation redaction, evidence and node-index its by @kblok in #1304
  • Upstream API: secret and credential provider functions, origin narrowing, engine-declared secrets by @kblok in #1317
  • Config digest follow-up: cite tracked issues, drop stale no-digest docs (Refs #1192) by @kblok in #1318
  • Upstream API: model reasoning carried onto step model events (act turn and judgment) by @kblok in #1319
  • Align the expect failure message wording with upstream (observed: ...) by @kblok in #1320
  • Tool loop: align guard and streak notices with upstream's [SYSTEM] style by @kblok in #1316
  • Upstream API: truncated traces (recorder poison, action/input caps) and truncated-entry miss by @kblok in #1321
  • Upstream API: wire the config digest into the report project (project.configDigest) by @kblok in #1326
  • Upstream test: tool-call provider metadata across a forced-conclusion retry by @kblok in #1328
  • Upstream test: lastError retry chain in isForcedToolChoiceRejected by @kblok in #1332
  • Upstream Change: replay of typeSecret runs the secret fill authorization by @kblok in #1333
  • Upstream Change: prepareObservation, node index, role and attribute-name redaction, settle without a shape by @kblok in #1334
  • Upstream API: --no-cache run option overriding the cache config by @kblok in #1335
  • Upstream API: failOnSkippedFailure and maxDuration config options by @kblok in #1338
  • Upstream API: Secrets.Get names E2E_SECRET_ and the matching credential (Refs #1314) by @kblok in #1340
  • Upstream API: createAgentCacheContext so the REPLAY_STALE advice its port whole by @kblok in #1341
  • Upstream API: code-built config with engines per target (internal TargetConfig.Engine) so resolve runs checkEngineSecrets by @kblok in #1345
  • Upstream Change: engine-to-session adapter normalizes a plain engine throw to ENGINE_FAILURE by @kblok in #1342
  • Upstream Change: recorded action summary (describeAction) on trace actions (Refs #1283) by @kblok in #1343
  • Upstream change: trace provider metadata of tool calls and steps in AiTraceModel by @kblok in #1346
  • Upstream change: step-cache entry records its target (recordedFor.targetId) by @kblok in #1349
  • Upstream: cache date patterns move to internal/dates and drop false-positive matches (#1511) by @kblok in #1358
  • Upstream: toHaveURL prints the expected URL resolved against the app URL (#1592) by @kblok in #1359
  • Upstream: device.setPermission validates name and state before the device call (#1605) by @kblok in #1360
  • Upstream Change: 16-bit PNG samples scale to 8 bits with rounding (image.ts narrow) by @kblok in #1361
  • Docs: errors reference lists BROWSER_INSTALL_FAILED by @kblok in #1365
  • Upstream: scroll_to reads each page once when it landed (#1570) by @kblok in #1364
  • Upstream Change: extract resolveAgents (cli agents, agentNames) by @kblok in #1367
  • Upstream Change: OperationContext carries runId and attemptId by @kblok in #1370
  • Upstream Change: replay of an undeclared secret fails POLICY_DENIED like the dispatcher by @kblok in #1371
  • Upstream: value expectations on Expect (values.ts) by @kblok in #1373
  • Upstream change: e2e cache ls, stats, clear (cli/cache.ts) (#1347) by @kblok in #1374
  • Upstream change: e2e cache follow-ups (catch any error, directory entries, NU1701) (#1375) by @kblok in #1376
  • Upstream test: legacy-entry setup for the CI-demoted read-only replay (#1287) by @kblok in #1377
  • Upstream Change: authorizeSecretFill records secret.* policy events by @kblok in #1378
  • Upstream Change: --cache-dir run option by @kblok in #1379
  • Upstream Change: observation secureNodeCount from the engine for pixel-only observations by @kblok in #1380
  • Upstream docs: cache Playwright browsers in CI; name the authentication guide (#1591, #1594) by @kblok in #1385
  • Accept tz database link names without the host's tzdata-legacy by @kblok in #1396
  • MessageWithCauses: follow AggregateException members (#1208) by @kblok in #1384
  • Upstream: line text reads no aria-hidden inline text (line-text) (#1223) by @kblok in #1386
  • Document what timeout 0 and a negative timeout do (#1387) by @kblok in #1392
  • Apply the canonical engine-error mapping inside ErrorSerializer.Classify by @kblok in #1393

Full Changelog: v0.8.0...v0.9.0