Repository navigation
e2e@0.19.0
Minor Changes
-
#947
f3aa4c8Thanks @kblok! -locator.dblclick()is an alias ofdoubleTap(), asclick()is oftap(), so a test ported from Playwright or Cypress keeps its double clicks. -
#869
1b92a6dThanks @okwasniewski! -tracekeeps its modes but now writes e2e's own trace instead of a Playwright zip: a markdown page per test, on every engine, and the terminal names it under each failed test (❯ trace .e2e/results/<test>/trace.md). The default isretain-on-failure, locally and in CI. Each step on the page shows what it did: the node an action landed on, the values an assertion orwaitForread while it waited, the replay cache's decision (where a replay stopped and why, what became of the recording, the entry file), what the app logged meanwhile (console output, uncaught exceptions, failed requests, and 4xx and 5xx responses, also gathered in one App log section with each line's step), where the page went (a navigation, a new tab, a frame), and how the screen changed since the step before. The page also names the browser and user agent, or the device, the test ran on. Themarkdownreporter'ssummary.mdnames each failure's trace page when one was kept.report.jsonis still written for scripts, but thelistsummary no longer prints its path (--debugdoes), and printsReport (not written)only when it is missing. Everything a test left now sits in one directory,<output>/results/<test>/(.e2eby default): itstrace.md(which shows the failure screenshot inline and links each file) and anattempt-<n>/per attempt, numbered from 1, with the screen at failure asscreen-at-failure.txt. It replaces.e2e/artifacts/<target>/<test id>/<agent>/attempt-<n>/; a--last-failedrerun writes under each test'srerun-<n>/. Point CI uploads at.e2e/results, and add.e2e/results/to.gitignore(or rerune2e init). A stale recording made from this version on names what changed in its key under--strict-cache, such as the agent's context. For engine authors:startTraceandstopTraceare no longer called,traceleavesartifactCapabilities,EngineAttemptContext.appLogtakes what the app logs,navigationwhere it went on its own,screentakes a screen for the trace (one alocatematched against, or one read right after an action, bounded), andenvironmenttakes what the attempt runs on. report-1 drops thetraceartifact kind and thenot-requiredredaction (StoredArtifact.kindandredactionnarrow the same way), dropsTRACE_WITHHELD, and addsattempt.appLog(each line with its step),navigationevents,failure.viewportandfailure.nodes,step.phase,step.screen,attempt.environment, andstep.cache.entry,detail, andwrite(report only;result.cachefromagent.act()gainsentryanddetail).FinishedRun.tracesmaps results to their pages. -
#965
b92658bThanks @okwasniewski! - A customStepExecutorthat declaresvision: truereceivesagent.assert(..., { vision })asctx.step.visioninstead ofUNSUPPORTED_CAPABILITY. -
#978
2f60b54Thanks @mvanhorn! -listfrome2e/runnerreports the tests a config, file selection, and title grep would select, in process, without starting an app, an engine, or a worker. The result includes pairs a filter removed, positional arguments that matched no file, and the selected target names.e2e list --reporter jsonrenamesskipReasontoreasonand adds each pair's identity:id,source,session,sessions,serialId, andagent, beside the fields it already printed. -
#924
d3d9081Thanks @okwasniewski! - Breaking: a failedexpect.softbeforetest.skip(...)now fails the test, as in Playwright. Before, the test was reportedskippedand the run exited 0 unlessfailOnSkippedFailurewas set. The attempt isfailedwith the soft failures as its error, keeps the skip reason in itsskipfield, and retries like any failure. Inreport.json, an attempt or serial member may now carryskipwith a failed status. The CLI prints the reason under the failure, and the trace page (trace.md) names it.failOnSkippedFailurenow covers only a retry that skips after an earlier failed attempt. -
#977
5137778Thanks @okwasniewski! - AddtoHaveScreenshotfor visual comparison on every engine that captures pixels:await expect(screen).toHaveScreenshot('home.png')compares the screen andexpect(locator).toHaveScreenshot()one element against a PNG stored beside the test file, one per target and operating system. A missing one fails the test and is written beside it, or in CI attached to the results instead;--update-snapshots(-u) writes missing and different ones and passes. Options:threshold,maxDiffPixels,maxDiffPixelRatio,mask,maskColor,timeout. A mismatch attaches the expected, actual, and diff images to the step. On iOS and Android the status bar is held in a fixed state for these screenshots.
Patch Changes
-
#893
7eb03caThanks @rudycelekli! - Stop a fixture setup that finishes after its attempt was abandoned from starting later fixtures, hooks, or the test body. The late fixture still releases its own teardown. -
#921
f2cf2adThanks @okwasniewski! - Agent steps on Meta's Muse Spark models, through AI Gateway or OpenCode Console, no longer fail withMODEL_PROVIDER_FAILED: only auto is supported for tool_choice. The refusal now reads as one, so the step retries withautoand the tool-calls-only rule, as it does for Anthropic and DeepSeek. -
#929
e77c17aThanks @okwasniewski! - The replay cache key no longer carries the engine's version, so upgrading@e2e-dev/webor@e2e-dev/mobilekeeps every committed recording. A recording re-finds its nodes at replay, which is where a node an engine release resolves differently is caught. Entries recorded before this release sit under the old keys: aread-writerun records them again, and--strict-cachenames them asREPLAY_STALEuntil it does. -
#935
3c37d98Thanks @rudycelekli! - Cancel the ChatGPT authorization-code token exchange when sign-in is interrupted. -
#901
67b11b5Thanks @rudycelekli! - Skip MCP tool execution when the client cancels a queued call or cancels during argument validation. -
#936
5cdd3c6Thanks @rudycelekli! - Cancel in-flight app and service readiness HTTP probes when the run is interrupted. -
#899
d5a4eabThanks @rudycelekli! - Cancel the initial OAuth device-code request promptly when login is interrupted, without showing or saving a cancelled login. -
#918
9fb70b9Thanks @okwasniewski! - A credential whoseusernameis not a string, or with a key other thanusernameandpassword, fails the config load withINVALID_CONFIGnaming the field, instead of a raw "value of type function is not JSON-safe" error. Onlypasswordmay be a function. A credential with no username in the config orE2E_USER_<NAME>_USERNAMEisINVALID_CONFIGtoo. -
#907
fef1246Thanks @rudycelekli! - Keep direct locator reads within their original action timeout when a stale reference requires resolution again. -
#887
4132715Thanks @DeryFerd! -e2e initno longer writes an MCP config through a symlink: a.mcp.jsonor.cursor/mcp.jsonreached through a link, or under a linked directory, is skipped with a warning naming the link and its target, the way a linked skill directory already was. Before, the merge read and overwrote the file the link pointed at, which could be a dotfiles repo or a shared config outside the project. -
#919
b6efcf7Thanks @okwasniewski! -store: falseand a prompt cache key now go only to OpenAI and Azure OpenAI Responses models and toopenai/*models on the gateway. An OpenAI-compatible provider namedopenaino longer fails withUnknown parameter: promptCacheKey. -
#920
745b353Thanks @okwasniewski! - A polling assertion takes its last read at its timeout instead of up to 100 ms after it, so it no longer passes on a state that arrived after the timeout. -
#922
c10dc00Thanks @okwasniewski! - When the provider package can't read a provider's error body,MODEL_PROVIDER_FAILEDno longer shows an empty message. One example is the ChatGPT backend's HTTP 400{"detail": "Unsupported service_tier: fast"}. The message and the--ai-traceentry now show the HTTP status and the body, with secrets redacted and the body cut at 1 KB. Anagent.actstep that used up its transport retries now names the last attempt's failure, as judgments already did. -
#905
ea0681aThanks @rudycelekli! - Reject read-only environment credentials before starting a provider login that cannot be saved. -
#923
99d5ab5Thanks @okwasniewski! ---strict-cache(andcache.strict) never writes or deletes a cache entry, even inread-writemode, and retries replay like the first attempt. Before, a retry skipped replay, called the model, and overwrote the committed entry, and a failed attempt could evict it. -
#984
bc453bdThanks @ASVLCII! - Aligne2e cache lscolumns by displayed width for test file names containing wide characters, combining marks, or emoji. -
#881
cae1c65Thanks @fecolinhares! - A recorded step the cache hands to the agent no longer accumulates the replay wait or the agent's turn into the next replay's end wait. The wait is measured from the agent's verdict if it settled without acting, or from its last action if it acted. -
#927
0ddea76Thanks @rudycelekli! - Match a Unicode code point with the?test glob wildcard, so a test file named with one emoji is selected by a single?. -
#998
841ea3fThanks @okwasniewski! - A recorded tap on a row of a windowed list no longer depends on which row the list rendered first. The container key of a row was its list's first text, a scroll position on Android (Row 0512 in Row 0500), so the replay found the row and still handed off whenever the window started one row later.