Skip to content

e2e@0.23.0

Choose a tag to compare

@github-actions github-actions released this 05 Nov 14:33
· 109 commits to main since this release
5f25692

Minor Changes

  • #1552 d2556bc Thanks @okwasniewski! - Breaking: inputValue() on a node with no value, such as a heading or a button, and isChecked() on a node with no checked state throw ACTION_FAILED, as in Playwright, instead of returning "" and false. A field with no value still reads "", and a checkable role with no state reported still reads false. uncheck() on an ARIA role="radio" now clicks it, as in Playwright; a native radio still refuses.

Patch Changes

  • #1524 cba5e0e Thanks @okwasniewski! - The built-in agent can resize the viewport with the new resize_viewport tool, so agent.act and e2e explore can check layouts at desktop, tablet, and phone widths. The tool is offered when the target's engine implements the new optional session.resize hook. Custom executors call ctx.actions.resize({ width, height }), and the replay cache records and replays it.

  • #1479 d15a98e Thanks @okwasniewski! - app.baseUrl no longer ends in a slash, so ${app.baseUrl}/api/users requests /api/users instead of //api/users. With test from @e2e-dev/web, app.baseUrl is typed string, since a web() target always declares app.url. A test that relied on the trailing slash, such as ${app.baseUrl}api/users or new URL('users', app.baseUrl) on a URL with a path, adds the slash itself.

  • #1491 108bc60 Thanks @okwasniewski! - Bump @agentclientprotocol/sdk to 2.2.0, @ai-sdk/provider to 4.0.35, and oxc to 0.156.0.

  • #1516 0410fc9 Thanks @okwasniewski! - Bump agent-device to 0.21.36, @agentclientprotocol/sdk to 2.4.0, @vitest/expect to 5.1.0, and oxc to 0.157.0.

  • #1573 3d5f4ea Thanks @okwasniewski! - Bump @ai-sdk/provider to 4.0.36.

  • #1451 b4dd4bd Thanks @amoscicki! - The run summary's Cache line counts the steps of passing and flaky tests whose recording was not saved (step.cache.write of unconfirmed, no-change, or not-written in the attempt that passed), such as 4 missed · 1 not saved.

  • #1493 91bd614 Thanks @okwasniewski! - Add e2e doctor: checks that e2e run can start without starting anything. It covers the Node.js runtime, the config and its targets, each engine's own checks, and the model each agent resolved to. --json prints the doctor-1 report for a host, --check-model calls each model once, and the command exits 2 when a check failed. Engines can report their checks through the new optional doctor hook.

  • #1575 7d7d0e1 Thanks @kacperkapusciak! - e2e init and e2e --help show the new e2e logo, its six rows of bars drawn in full blocks, one row per bar. In e2e init the logo shades in from ░ to █, top rows first, in about half a second. The logo needs a terminal at least 52 columns wide, and prints without motion on one shorter than 7 rows.

  • #1478 2208056 Thanks @okwasniewski! - CONFIG_NOT_FOUND for an explicit config path (--config <path>, the MCP open_session config input, or openHost({ config: '<path>' })) now names the absolute path it checked and, for a relative path, the working directory it resolved against.

  • #1572 0c04696 Thanks @okwasniewski! - Documentation links in CLI help, e2e init output, and the MCP server listing now point to https://e2e.com/docs.

  • #1232 3f43edd Thanks @okwasniewski! - Add the engine protocol: connectEngine, serveEngine, and transports in e2e/engine run an engine in another process, on another machine, or in another language over LSP-style JSON-RPC. Every engine built with defineEngine now runs through the protocol in process, so the harness drives every engine the same way.

    Engine.dispose now runs only once init was called, even when init failed. A worker that never initialized no longer disposes its engine.

  • #1522 73734c6 Thanks @okwasniewski! - e2e explore digs deeper once the main flows are covered: it returns to them with edge-case inputs, combined filters and sorting, reloads, and other accounts, finishes early only when nothing new is reachable, and checks other screens that offer a control it found broken.

  • #1523 f4be1c4 Thanks @okwasniewski! - e2e explore runs longer: --max-steps goes up to 100 and --timeout up to 7200000 (2 hours).

  • #1480 723c1af Thanks @okwasniewski! - A failed test's block in the terminal and in summary.md quotes the first 3 errors the app logged (console errors, uncaught exceptions, failed requests, 5xx responses), so an app error behind a locator failure shows without opening trace.md.

  • #1499 1d671f6 Thanks @okwasniewski! - The markdown report footer (summary.md, the @e2e-dev/github pull request comment and job summary) now reads Tested by e2e <version> · ..., with e2e linking to https://e2e.com.

  • #1464 57f9ed9 Thanks @okwasniewski! - locator.waitFor, expect on a locator, and toHaveScreenshot give their last read extra time past the timeout, up to their slowest read so far and within the test timeout, so a screen that shows up during a slow last read still passes.

  • #1576 33a1541 Thanks @kdy1! - Start faster by loading heavy dependencies when a command or session needs them. The CLI no longer loads the MCP server, the prompt library, or the schema library until mcp, init, login, logout, models, explore, upgrade, doctor, or --watch runs, so e2e --version and --help take about 25% less time. @e2e-dev/web imports Playwright when a browser is first launched or looked up, so a process that only reads the config skips it: e2e list on a web project takes about 44% less time, and importing the package half as long. A worker loads Playwright for its first browser instead of before its first test, so that test's reported duration grows by about 0.14 s while the run takes no longer. e2e/runner no longer loads the explorer when imported. @e2e-dev/acp loads the ACP and MCP SDKs when its first session starts, and @e2e-dev/limrun loads @limrun/api when its first device is leased.

  • #1434 50d8063 Thanks @DeryFerd! - Later agent steps now see a fully replayed step's summary labeled recorded in an earlier run instead of observed.

  • #1482 d94593b Thanks @okwasniewski! - The list reporter prints each test as it finishes when stdout is not a terminal (piped, redirected, most CI), instead of only once its whole file finishes, and ends every run with a Failed row per failed test (file:line, the error's first line, the trace page when one was written), so e2e run | tail still names what broke.

  • #1416 79e2e13 Thanks @DeryFerd! - Filtered-locator failures can list how many candidates each filter step left, when the post-failure diagnostic pass can read the chain, so a step that narrowed too much or too little names itself. The message says the counts were read then.

  • #1520 50fa7af Thanks @okwasniewski! - e2e mcp has a run_tests tool: it runs a test file, one test (tests/checkout.e2e.ts:12), or the whole suite as e2e run does and returns what the run printed, so a coding agent can run tests without a shell.

    run_tests {files: ["tests/checkout.e2e.ts:12"]}
  • #1498 4b56473 Thanks @21prnv! - A replay no longer fails its end check on a date written with an ordinal suffix, such as October 26th or 3rd October, when the day differs from the recording.

  • #1571 4c44e44 Thanks @kdy1! - The agent compares screenshots with a native SHA-256 instead of a byte-by-byte loop in script, so a settle poll or screen update that carries pixels no longer spends about 8 ms of the event loop per megabyte of image.

  • #1493 91bd614 Thanks @okwasniewski! - A model call without a response schema that the provider ends on an error finish now fails as MODEL_PROVIDER_FAILED instead of MODEL_OUTPUT_INVALID.

  • #1500 7b1b8b2 Thanks @bheemreddy-samsara! - Add coordinates to agent.act's onAction event: narrow on event.action to read point for tapAt and hoverAt, or from and to for dragAt, replay included. Points are in the same space as screen.tapAt and boundingBox(), rounded and clamped to the viewport.

    ActionEvent is now a union keyed by action, typed ActionName or tool:<name> instead of string. New ActionName and ActionDetails types.

  • #1474 d10655f Thanks @samanyudas! - e2e run without the list reporter prints each run error on stderr as CODE: message. Before, --reporter junit or markdown with a test file that failed to collect, a config that failed to load, or no model for the agent exited 2 and printed nothing.

  • #1481 f078364 Thanks @okwasniewski! - The list summary names the first 10 app.screenshot files in a Screenshots row, then counts the rest. The MCP screenshot tool takes an optional path (a .png, relative to the project root): it saves the masked full-resolution screenshot there and returns the absolute path.

  • #1470 b8485ba Thanks @samanyudas! - run.summary.executed now counts a serial member whose body ran, the same way it counts an ordinary test. Before, it left out a member that called test.skip() in its body, and a member that ran in one group attempt before a retry skipped it behind a failed predecessor. The run's telemetry cache tiers count them the same way.

  • #1483 0b84d35 Thanks @okwasniewski! - open_session now names the app command it started: its executable, the URL it answers on, the port a free-port url was assigned, and the env and inheritEnv names it got. It prints no args and no env values. It also says the config reads the MCP server's environment, not the agent's shell. e2e run prints the URL on the app command's setup line, and the app setup event carries it as url.

  • #1546 a8ed7f7 Thanks @okwasniewski! - locator.setInputFiles([]) clears a file input and fires input and change, as in Playwright. It used to throw INVALID_ARGUMENT. A blank path is still refused.

  • #1567 65ebc7d Thanks @NathanWalker! - The bundled e2e skill now names NativeScript's testID among the ids getByTestId reads on a device.

  • #1538 c7cbcef Thanks @kacperkapusciak! - The bundled skills declare what they need to run (node and npx, and the optional AI_GATEWAY_API_KEY for the e2e skill) in metadata.openclaw, the frontmatter ClawHub reads. All three are now on ClawHub under @e2e.

  • #1547 4dce5a7 Thanks @okwasniewski! - tap({ position }) measures the offset from inside the node's border, as Playwright's click({ position }) does. Before, a bordered element shifted the tap by its border width. Engines can report that point as the new optional SemanticNode.origin.

  • #1539 732ea8a Thanks @okwasniewski! - Add test.fail to mark a test that is expected to fail: test.fail(title, fn), test.fail(condition, reason) inside a body, or the fail test option. A marked test that fails with a test failure passes, and the report keeps the reason and the original error under expectedFailure. A marked test that passes fails with UNEXPECTED_PASS, so a fixed gap gets noticed. Infrastructure errors such as ENGINE_FAILURE and blocked agent verdicts are not covered.

  • #1505 bdf769b Thanks @okwasniewski! - A fixture call that times out names what it waited for: browser.waitForResponse('foo.css') fails with operation timed out in browser.waitForResponse "foo.css". Steps for waitForURL, waitForResponse, route, and unroute show a string pattern in quotes and a RegExp as its literal.

  • #1554 d2953bb Thanks @okwasniewski! - A call's timeout: 0 now means no deadline of its own, as in Playwright: locator actions, waitFor, matchers, toHaveScreenshot, expect.poll, contributed fixture calls, and browser.goto, waitForResponse, and waitForDownload wait until the test timeout (cleanupTimeout in afterEach) instead of failing at once or being refused. A negative, NaN, or infinite timeout is INVALID_ARGUMENT on every call, locator actions included (INVALID_CONFIG on expect.poll, as before). expect.poll with timeout: 0 outside a test is INVALID_CONFIG. Engine authors get stepTimeout from e2e/engine to give a contributed call's timeout option the same rules.

  • #1471 80c2fda Thanks @samanyudas! - A string a test or the config writes to process.stdout or process.stderr with an encoding, such as hex, base64, or latin1, now prints decoded from a worker, as Node prints it. The runner process decoded only hex, base64, and base64url, and now decodes every encoding Node accepts too. Bytes that end inside a UTF-8 character no longer print after the string written next.