Repository navigation
e2e@0.24.0
Minor Changes
-
#1891
847a09eThanks @KrzysztofMoch! - Breaking:test.extend()fixtures are set up on demand, as in Playwright and Vitest. A test sets up the fixtures its body and itsbeforeEachandafterEachhooks name in their first parameter, and the fixtures those fixtures name; a suite hook sets up the worker fixtures it names. A fixture nothing names no longer runs.To migrate:
- A fixture that exists for its side effect (it seeds data, installs a build, starts a watcher) and that no test names needs
auto: true:[fn, { auto: true }]runs for every test,[fn, { scope: 'worker', auto: true }]once per worker. - A fixture that relies on another one running first names it:
task: async ({ project, agent }, use) => {}. - A test body, a hook, and a fixture function destructure their first parameter.
async (fixtures) => {},async (_, use) => {}, a rest property ({ app, ...rest }), and a bound function (fn.bind()) are now aCOLLECTION_ERROR; a fixture that needs nothing writesasync ({}, use) => {}.
A fixture can now name the fixtures declared beside it in the same
test.extend()call, in any order; a cycle is aCOLLECTION_ERRORnaming it.These mistakes fail at collection, before any engine starts, with the file and line of the code: a fixture name the target does not have (it was
UNSUPPORTED_CAPABILITYin the test), and abeforeAll,afterAll, or worker fixture that names a test fixture (it wasHOOK_FAILED). A test that does not run on a target (platforms,requires,test.skipIf) is not checked there.An
afterEachhook that names a fixture whose setup failed is skipped. On an engine withisolation: 'worker', a suite hook or a worker fixture teardown that names no engine fixture,app, orscreenopens no engine attempt. - A fixture that exists for its side effect (it seeds data, installs a build, starts a watcher) and that no test names needs
-
#1967
0e9d58eThanks @okwasniewski! - Breaking:e2e mcpno longer serves therun_teststool. Run tests withnpx e2e run, which takes the same files and flags. The server keepsopen_session,tools,call, andclose_session, andopen_session {test}still runs one test in a session.
Patch Changes
-
#1785
ceaea64Thanks @okwasniewski! - Bump playwright-core to 1.65.1 and@ai-sdk/providerto 4.0.38. Smol sandboxes use the Playwright 1.65.1 image. -
#1871
8af43f5Thanks @okwasniewski! - Bump@agentclientprotocol/sdkto 2.7.0, and oxc-parser, oxc-transform,@oxc-project/runtimeto 0.159.0. -
#1958
ad641beThanks @okwasniewski! - Bump@ai-sdk/providerto 4.0.39,@vitest/expectto 5.1.1, and@agentclientprotocol/sdkto 2.8.0. -
#1861
726dd76Thanks @kdy1! - The CLI starts without loading the YAML and TOML parsers, which onlye2e inituses to write a CI pipeline and a Codex MCP config. Every command starts a little faster. -
#1892
d16ad2cThanks @GaspardIV! - Adde2e reportto open the HTML report written by@e2e-dev/html-reporterin the default browser. -
#1633
0b44859Thanks @leszko11! - Read a listed node's computed style from a read-only project tool with
getToolContext(options).read({ id }, { style: ['color', 'font-weight'] }).
The web engine returns allow-listed CSS values, secret-redacted, under the
step's deadline. Engines can implement the optionalreadStylemethod over the
engine protocol for thecomputed-stylecapability; a read on an engine
without it fails withPOLICY_DENIED. -
#1965
60d29cfThanks @szymonrybczak! - List the maintainers'@e2e.comaddresses incontributors. -
#1911
902f044Thanks @FaZeDrug! - Usereporters: ['dot']or--reporter dotto print one character for each selected test result. -
#1737
0a5ee16Thanks @kdy1! - New package@e2e-dev/core: the engine contract, locators, agent, replay cache, and engine sessions thate2eis built on.e2edepends on it at the same version, ande2e/engineande2e/agentre-export it unchanged.@e2e-dev/weband@e2e-dev/mobilepeer on@e2e-dev/corebesidee2e, and import the contract from@e2e-dev/core/engine, which a third-party engine can now import too. npm, pnpm and Bun install the new peer themselves. A Yarn project lists@e2e-dev/corebeside the engine;e2e initande2e upgradeadd it. -
#1734
dbe3dc5Thanks @kdy1! - New package@e2e-dev/core: drive a browser or a native app from plain code, with no test runner, config file, or report.launch(web())prepares an engine and returns a host;host.newSession()opens an isolated session holdingscreen,app,platform, and the engine's fixture.session.saveState()captures state in memory, andSessionState.export({ key })returns it only AES-256-GCM encrypted, assession-state-1bytes. Secrets go tolaunch({ secrets })and fill throughhost.secret(name).createAgent(session, { model, cache })from@e2e-dev/core/agentadds the agent: withcache, everyactnames its recording withcacheKey.@e2e-dev/weband@e2e-dev/mobileneede2eonly for thetesttheir main entry exports, so theire2epeer is optional. Withoute2e, the main entry still loads and its engine factory works;test,describe, and the hooks throwRUNNER_NOT_INSTALLEDwhen called. Its types still namee2e, so a TypeScript program without it imports from the/engineentry or setsskipLibCheck.@e2e-dev/core/agentalso exports whate2e/agentdoes,defineToolamong it.Device targets take
app.linkBase:app.open('/orders/42')then opens the path as a link in the pinned app, through the new optional engine hooksession.openLink(protocol methodsession/openLink).@e2e-dev/mobileimplements it.@e2e-dev/web/engineand@e2e-dev/mobile/engineexport the engine factories and their types withouttest, so a program that only drives an engine never loads the test runner.defineEngine<Fixtures>()andEngineHandle<Fixtures>now carry the fixtures an engine contributes as a type, anddefineEnginerequires a factory for each oneFixturesnames:web()is anEngineHandle<WebFixtures>,mobile()anEngineHandle<MobileFixtures>.defineEnginerefuses a fixture namedclose,saveState, orsetDefaultTimeout, the methods of a session.@e2e-dev/core/engine(ande2e/engine) exportsisRefusedLinkScheme, the scheme list app links are refused by, which@e2e-dev/mobilenow uses fordevice.openLink. -
#1891
847a09eThanks @KrzysztofMoch! -test.extend()can extend a fixture thetestalready has, a built-in one included:base.extend({ device: async ({ device }, use) => { await device.setPermission('camera', 'deny'); await use(device); } }). Inside the function the fixture's own name is the fixture it extends (the built-in, or an earlier fixture of that name), and tests, hooks, and other fixtures that name it get the value passed touse. It keeps the type, the scope, and theautoof the fixture it extends on the target, so adeviceoverride on a mobile target runs once per worker and reachesbeforeAll, and an override of anautofixture is set up for every test; ascopeor anautothat differs is aCOLLECTION_ERRORnaming both. On a target without that built-in, nothing fails until code that runs there names it.A fixture named like a core fixture (
agent,app,screen,platform,session), like one the target's engine contributes, or like one an earlierextenddefined was refused before (COLLECTION_ERROR, orTEST_SETUP_FAILEDin the test). -
#1874
6c4edddThanks @okwasniewski! -e2e init, the update notice, ande2e upgradenow ask for a GitHub star. -
#1647
d593863Thanks @kacperkapusciak! -e2e initcan write a CI pipeline: a GitHub Actions workflow or a GitLab CI/CD file with one job for each platform the config's targets run on. Web targets run in a web job; iOS and Android targets get their own jobs, which build the app, boot a simulator or an emulator, and install the app. A config with both gets all of its jobs. The wizard asks for it in a project with the web or mobile engine and preselects the service the repository points to;--ci github-actionsor--ci gitlabanswers the question, and--yesalone writes none. The mobile jobs build Expo and bare React Native apps; any other app gets a build step to replace. GitLab's mobile jobs need self-managed runners. The jobs use the project's package manager and Node version and keep the report; for a project in a subfolder the file goes to the repository root.initnames the secret or CI/CD variable to add and warns about what would fail on the first push. It never overwrites a pipeline file: beside an existing.gitlab-ci.ymlit writes.gitlab/e2e.gitlab-ci.ymland prints theinclude:to add. -
#1868
b68bc47Thanks @JaynouOliver! -e2e initno longer says to addtestto the stages of a.gitlab-ci.ymlthat already lists it with unindented items, or with a comment or blank line between them. -
#1799
cfa2aa0Thanks @samanyudas! - Workers no longer die about 1 ms after a stop request when a grace budget built fromtimeoutandcleanupTimeoutis longer than 2^31-1 milliseconds. Ctrl-C now runsafterEachand disposes the engine withtimeoutnear that limit, and engine disposal at the end of a run or on a second Ctrl-C finishes withcleanupTimeoutnear it. -
#1860
aadf955Thanks @kdy1! - The JUnit report keeps the\r\nline breaks of failure messages and skip reasons instead of writing U+FFFD:junit.xmlreads&[#13](https://github.com/e2e-inc/e2e/issues/13);&[#10](https://github.com/e2e-inc/e2e/issues/10);, andreport.jsonand reporter events keep the\r. A\rwith no\nafter it still becomes U+FFFD, and the terminal output is unchanged. -
#1935
9d240acThanks @okwasniewski! - A secret passed only tolaunch()inside ane2e runtest is now redacted from the test's report, trace, downloads, and model input until the test ends, and the host's sessions redact the config's secrets. Every session in a process redacts the secrets of the hosts and sessions open beside it, on a browser and on a device alike, without carrying them into a saved state. An error a session throws no longer carries a secret in itsdetails. -
#1935
9d240acThanks @okwasniewski! - A session a test opens withlaunch()is now in the test's report: its steps underlaunch-1,launch-2, in the order they ran, its model usage counted with the test's, and its video kept beside the test's when the test records one. Steps and video artifacts carry anactor, and so doesprogress.identityfor reporters. The test owns what it opened: a host or session it leaves open is closed when the test ends, a step it leaves running fails the test withSTEP_NOT_AWAITED, and a timeout cuts a session's step as it cuts the test's own. The trace page, the HTML report, and the Markdown and JUnit failure text name the session, and the HTML report plays each step in its own session's video. While such a session records for the test,app.startRecording()on it fails withUNSUPPORTED_CAPABILITY, as it does on the test's own session. -
#1865
fabb0c0Thanks @imranbarbhuiya! -expect(locator, message),expect(screen, message), and theirexpect.softforms take an optional message, as value expectations already do. It opens every assertion failure,.notandtoHaveScreenshotincluded, and a locator'sLOCATOR_AMBIGUOUSerror:the cart is empty: expect.toBeVisible failed. -
#1814
5c7e18fThanks @jxnxts! - The Markdown summary renders faster for files with many results. Output is unchanged. -
#1749
6136421Thanks @bardrop! - With several workers,--max-failuresno longer lets the worker whose failure reaches the limit start its next test. That test was cancelled a moment later and reportedinterrupted; it is now skipped with causefailure-limit, as the docs say. -
#1762
5874e47Thanks @BloggerBust! - When an engine package provides separateimportandrequireentries, the MCP server now uses theimportentry when opening a session without a config file. -
#1835
95c574aThanks @mvanhorn! - A built-in agent tool that takes no input drops keys the model sends and runs.observe,back,dismiss_keyboard, andscreenshot(when it has nopath) used to reject a padded call such asscreenshot({"description": "check the home screen"}), and some models resent that refusal until the step budget ended. A tool that declares fields still rejects a key it does not declare. -
#1808
66d07caThanks @samanyudas! - Parallel workers no longer fail and ask you to sign in again when a saved subscription login expires. -
#1926
6fbcb66Thanks @KrzysztofMoch! - The bundled e2e skill now tells an OpenCode agent to registere2e mcpin the project'sopencode.json, and not withopencode mcp add, which writes the global config. -
#1736
9949c8aThanks @kdy1! -e2eexportsProjectConfig, the part ofE2EConfigwithoutreportersandfilters. -
#1750
2359860Thanks @KrzysztofMoch! -e2e run --stepworks without a terminal, as in a coding agent's shell. The run goes on in the background and the command returns at the first held step, with the screen that step waits on.e2e run --step nextruns the held step, prints how it ended, and returns at the next one;e2e run --step restruns the rest, ande2e run --step stopends the run. The last answer says the run has ended. -
#1904
4893d1dThanks @satya164! - Addscreen.pause()to pause a test at a line, with the app open, until you
resume it: press Enter in the terminal, or rune2e run --step nextfrom
another shell in the same directory. WithCIset, the test fails with
PAUSE_UNAVAILABLE. -
#1943
5e9b82eThanks @Jaaneek! - The SuperGrok subscription provider is nowspacexsi:e2e login spacexsi,e2e logout spacexsi,e2e models spacexsi, and its messages say SpaceXSI.spacexaiandgrokstay as aliases on all three commands. A login stored underspacexai, in the credentials file or inE2E_OAUTH_CREDENTIALS, is still read and stays under that key, so an older e2e on the same machine keeps finding it.grok()frome2e/oauth/grokande2e init --gateway grokare unchanged.e2e doctor --jsonreports the login of agrok()model asspacexsi. -
#1891
847a09eThanks @KrzysztofMoch! -STEP_NOT_AWAITEDnames the code that left the step running. A step call withoutawaitin abeforeEachhook, a fixture's setup, or abeforeAllorafterAllhook used to readthe test body returned before .... It now readsthe beforeEach hook returned before app.open "/" finished,the fixture "account" setup returned before ..., orthe beforeAll hook returned before .... A session opened with@e2e-dev/coresaysthe hosted session. -
#1889
4519ee7Thanks @satya164! -e2e run --stepnow removes its socket file when a third Ctrl-C forces the run to exit. -
#1810
3761bb9Thanks @samanyudas! ---strict-cacheno longer fails withREPLAY_STALEa never-recorded step whose test id, target, or agent name is over 300 characters and starts like a recorded one. -
#1882
a213247Thanks @bardrop! - Addtest.each(rows)andtest.for(rows), which register one test per row with a title formatted from the row ($user,%s,%i,%#).test.eachspreads an array row into the body's arguments andtest.forpasses it whole. The fixtures come last, and the body names the ones it uses by destructuring that parameter:async ({ user }, { app }) => {}. -
#1870
1d0271bThanks @samanyudas! - Addtest.skipIf(condition)andtest.runIf(condition), which skip or run a test based on a condition, without wrappingtest()in anif. -
#1857
d4fb5ccThanks @Asheesh7298! - Addexpect(locator).toHaveRole(role), which waits for the locator's match to have that role. -
#1851
9034956Thanks @Wali05! - Token counts just under a million now read1.0Minstead of1000.0k, and the HTML report shows millions as2.5Minstead of2500.0k. -
#1819
3422abfThanks @kdy1! -agent.waitForjudges a screen change as soon as it sees it instead of first waiting outintervalafter the previous judgment, so a condition that comes true mid-wait resolves within about 500 ms plus one model call rather than up to 3 s later. While the screen keeps changing (a progress counter, a clock), it still judges at most once perinterval. -
#1891
847a09eThanks @KrzysztofMoch! -test.extend()takes worker fixtures:[fn, { scope: 'worker' }], typed with the second type argument (base.extend<{}, { database: Database }>(...)). A worker fixture is set up once per worker and target, the first time a test or suite hook registered through itstestruns, and torn down when the worker ends. It sees worker fixtures only.beforeAllandafterAllnow receive worker fixtures:platformand the worker fixtures of thetestthey were registered through. Reading a test fixture there fails the hook withHOOK_FAILEDnaming the fixture.SuiteFixturesandSuiteHookFnare deprecated aliases ofWorkerFixturesandTestHookFn<WorkerFixtures>.defineEnginetakesisolation: 'worker', which the engine protocol's manifest carries too. It makes the engine's fixtures,app, andscreenworker fixtures.@e2e-dev/mobiledeclares it, so a mobilebeforeAllcan set a permission or open the app. -
#1941
34b52baThanks @Jaaneek! -e2e loginand the token refresh of a subscription login now sendUser-Agent: e2e/<version> (<platform>; <arch>)to the provider's sign-in server, as model requests already do. They used to send the runtime's default,node. -
Updated dependencies [
6a1cb9a,336c428,f3cd032,60d29cf,51d304e,0a5ee16,dbe3dc5,847a09e,1b7c3a9,aadf955,9d240ac,9d240ac,fabb0c0,95c574a,847a09e,0cd14a0,4893d1d,d960d2c,847a09e,3761bb9,a213247,1d0271b,3422abf,847a09e]:- @e2e-dev/core@0.24.0