Browser Tests

The browser tests run the built ALPS in Chromium, with Playwright, against a complete mail stack started for the run: sora for IMAP, ManageSieve and Sieve filtering, plus an object store, an SMTP submission server, a CalDAV/CardDAV server and a control API. This page covers what the harness needs, how to run the whole suite or part of it, the variables it reads, what it starts, the optional cluster, preview and DAV connection runs, how to write a test with the shared fixtures, and how to read a failed run. The reference is docs/E2E.md. For how this tier fits with the other two, see Testing Overview.

What the browser tests catch

The Go tests and the frontend unit tests never run the application in a browser against real servers, so they miss a class of faults: a component that reads a field the backend does not send, a request the browser refuses, a page that logs an error and renders nothing, a flow that breaks only when a real IMAP server, a real SMTP submission and a real browser all take part.

Every browser test fails on any console error or uncaught exception on its page, not only on its own assertions. Most of these faults show up only as a logged error, with nothing visibly broken.

The repository has no CI configuration, so the browser tests run only when someone runs them locally. The README asks for the browser suite when a change touches UI flows.

Requirements

• go, npm, psql, curl and lsof on PATH. The script checks them in that order and stops at the first one it cannot find. psql is only the client. With Postgres.app, add its bin directory to PATH.
• Go 1.25.0 or later (go.mod also names toolchain go1.26.5).
• Node.js 20 (20.19 or later), 22 (22.12 or later), or 24 and later. On Node.js 23, npm prints an engine warning for the locked Vitest.
• A PostgreSQL server that the sora you build supports. sora asks for PostgreSQL 14 or later with the pg_trgm and btree_gin contrib extensions installable. The PostgreSQL user must be able to connect to the postgres database and to create and drop databases.
• A sora checkout, by default at ../sora next to the ALPS checkout. The directory must contain cmd/sora/main.go.
• Playwright's Chromium, or an installed Chromium named with PW_CHROMIUM_PATH.

Set up once, from the repository root:

cd frontend
npm install
npx playwright install chromium
cd ..

Running all or part of the suite

Run from the repository root:

scripts/e2e.sh                            # the whole suite
scripts/e2e.sh e2e/reading.spec.ts        # one file
scripts/e2e.sh -g "Reply All"             # tests whose title matches
scripts/e2e.sh --last-failed              # only the tests that failed in the previous run
make test-e2e                             # the whole suite
cd frontend && npm run test:e2e -- e2e/reading.spec.ts   # the same script, arguments after --

• Every argument goes to npx playwright test, which the script runs in frontend/ with e2e/ as the test directory.
• Playwright reads a file argument as a regular expression over the spec's full path. A bare reading.spec.ts therefore also runs threading.spec.ts. Keep the e2e/ prefix.
• make test-e2e takes no arguments, but it passes the environment through, so E2E_PORT=8950 make test-e2e works.
• --last-failed runs only the tests that failed in the previous run that used the same results directory, E2E_RESULTS. With the defaults, that is the previous run on the same E2E_PORT. Playwright keeps the record of that run as .last-run.json in E2E_RESULTS, and the script puts it back after it deletes the logs directory. When there is no record, the script stops before it builds anything, with e2e: --last-failed needs a previous run record at <dir>/.last-run.json; run the suite once first.
• Unless E2E_SKIP_BUILD is set, every run first rebuilds the frontend with npm run build. That rewrites the checked-in frontend/dist in place and can leave it modified in git. Run git status before you commit. When you change only a spec and frontend/dist already matches the sources, E2E_SKIP_BUILD=1 scripts/e2e.sh e2e/your.spec.ts tests the frontend/dist on disk and saves the frontend build. ALPS, the test stack and sora are compiled on every run either way.
• npx playwright test run in frontend/ without the harness starts no servers. It targets the default port layout (ALPS 8900, IMAP 8901, ManageSieve 8902, sora admin API 8903, DAV 8906, control API 8907), so it only works against a stack that is already up there.

The specs run one at a time, in one worker, with no retries. The suite runs against one account, so tests cannot run in parallel.

Variables

Variable	Default	What it does
SORA_DIR	../sora	The sora checkout to build.
E2E_PORT	8900	ALPS's port. The other servers use the next eight ports, or the next eleven with E2E_CLUSTER=1.
E2E_PG_HOST, E2E_PG_PORT, E2E_PG_USER, E2E_PG_PASSWORD	localhost, 5432, postgres, empty	The PostgreSQL server and the user that creates the run's database.
E2E_SKIP_BUILD	unset	Test the frontend/dist already on disk instead of rebuilding it.
E2E_KEEP_STATE	unset	Keep the state directory and the database after the run.
E2E_CLUSTER	unset	Start a second ALPS clustered with the first. See Cluster mode.
E2E_PREVIEW	unset	Turn on message previews in sora and run the positive preview test. See Message previews.
E2E_DAV_PERSONAL	unset	Leave the default CalDAV and CardDAV servers out of the generated alps.toml, for e2e/dav-connections.spec.ts. See DAV connections.
E2E_STATE	${TMPDIR:-/tmp}/alps-e2e-<port>	The state directory: the built binaries, the generated alps.toml and sora.toml (and alps-secondary.toml with E2E_CLUSTER=1), sora's cache, upload and relay directories, and ALPS's temp_dir.
E2E_LOGS	${TMPDIR:-/tmp}/alps-e2e-logs-<port>	The server and build logs, and by default Playwright's results.
E2E_RESULTS	$E2E_LOGS/test-results	Playwright's output directory, for traces and screenshots of failed tests and for .last-run.json, the record --last-failed reads. Playwright reads that record, then empties the directory when a run starts. The script makes the default an absolute path, so with a relative E2E_LOGS it is under the directory you started the script in. A relative E2E_RESULTS that you set is read from frontend/.
PW_CHROMIUM_PATH	unset	Use this Chromium binary instead of Playwright's.

E2E_SKIP_BUILD and E2E_KEEP_STATE act on any non-empty value, even 0. E2E_CLUSTER, E2E_PREVIEW and E2E_DAV_PERSONAL act only on exactly 1.

Once its checks pass, a run deletes E2E_STATE and E2E_LOGS with rm -rf, the default results directory inside E2E_LOGS included, apart from the previous run's .last-run.json, which it puts back. It deletes E2E_STATE again at the end unless E2E_KEEP_STATE is set. Never point either one at a directory that holds anything else. State and a database kept with E2E_KEEP_STATE last only until the next run on the same port, which deletes the state directory and drops the database before creating them again.

Ports and runs side by side

Every port is an offset from E2E_PORT, and every server listens on 127.0.0.1:

Offset	Server
+0	ALPS
+1	sora IMAP
+2	sora ManageSieve
+3	sora admin API
+4	object store (S3)
+5	SMTP submission
+6	CalDAV/CardDAV
+7	control API
+8	relay for sora's own outgoing mail
+9	the second ALPS (E2E_CLUSTER=1 only)
+10, +11	the gossip ports of the first and second ALPS (E2E_CLUSTER=1 only)

Before it builds or starts anything, the script stops if any of these ports already has a listener. The ports, the database alps_e2e_<port> and the default state, log and results directories all follow E2E_PORT. Two runs can go side by side when:

• their E2E_PORT values are at least 9 apart, or 12 apart when either one uses E2E_CLUSTER=1;
• they do not share an E2E_STATE, E2E_LOGS or E2E_RESULTS. With the defaults they never do; only values you set yourself can clash. Playwright empties its output directory when a run starts, and would delete the other run's traces and screenshots.

The runs still share two things:

• The build step. frontend/dist is one directory that every ALPS binary embeds, so the frontend build and the go build steps run under a lock at ${TMPDIR:-/tmp}/alps-e2e-build.lock. The lock is taken even with E2E_SKIP_BUILD. A run that finds it held prints ==> waiting for another run's build (pid N) and waits up to about ten minutes. A lock whose holder has exited is removed. After the build, each run uses its own binaries.
• The PostgreSQL server's connection limit. That is why sora's read and write pools are capped at 5 connections each.

Runs side by side has the details.

What the harness starts

scripts/e2e.sh works through these steps:

1. Checks the tools, the sora checkout, the PostgreSQL connection and the ports, and, when given --last-failed, that the previous run's record exists.
2. Deletes and recreates the state and log directories, keeping the previous run's record.
3. Under the build lock, builds the frontend (unless E2E_SKIP_BUILD), then ALPS, the test stack (internal/e2estack) and sora.
4. Drops and creates the database alps_e2e_<port> and runs CREATE EXTENSION IF NOT EXISTS pg_trgm in it.
5. Writes sora.toml and alps.toml into the state directory.
6. Starts the test stack and sora, and waits for each one to answer.
7. Creates the run's account, alice@example.test, through sora's admin API.
8. With E2E_CLUSTER=1, writes alps-secondary.toml, a copy of alps.toml with the second ALPS's address and WebAuthn origin, and adds a [cluster] section to each. Starts ALPS, then the second ALPS when there is one, and waits for each one to answer.
9. Runs Playwright with the servers' addresses and E2E_RESULTS in the environment.
10. Checks that every server is still running, then stops them all, drops the database and deletes the state directory (unless E2E_KEEP_STATE).

PostgreSQL ─┐
e2estack ───┼─► sora (IMAP, ManageSieve, admin API) ─┐
(object store, SMTP, CalDAV/CardDAV, control API) ───┴─► alps ─► Chromium

sora

sora is a complete IMAP server with METADATA (where ALPS keeps its settings), ManageSieve, Sieve filtering on delivery, shared folders under Shared/ (IMAP ACL), and an admin API that the tests use to create accounts and deliver mail. The tests therefore exercise ALPS's IMAP provider, not a stand-in. Its generated sora.toml is tuned for a test run:

• bcrypt_cost = 10, the lowest cost sora accepts, so sign-ins hash quickly;
• the lookup cache is off on the IMAP and ManageSieve servers, so a deleted account or a changed password takes effect at once;
• the per-account IMAP search rate limit is raised to 1,000,000 a minute, because ALPS lists a folder with THREAD and SORT, which count against it;
• loopback is a trusted network;
• the relay queue is worked every second, with a retry backoff of 1 s, so a Sieve redirect or vacation reply reaches the outbox quickly;
• database pools of 5 connections.

The test stack

internal/e2estack runs the servers around sora, all in memory:

• An S3-compatible object store for sora's message bodies.
• The SMTP submission server that ALPS sends through. It checks credentials with an IMAP login to sora and records every message it accepts in one outbox. Mail for example.test is delivered back into sora, so a message you send to yourself arrives like real mail. Mail to any other domain is only recorded. A sender address ending in @not-yours.test is refused with 553 5.7.1, so a spec can provoke a refused send.
• A relay port without authentication that takes sora's own outgoing mail (Sieve redirects, vacation replies). That mail lands in the same outbox, with an empty username.
• A CalDAV/CardDAV server. Each account gets one calendar for events and tasks and one address book, both created on first use. The account can make, rename and delete further calendars. Its own calendar can be renamed but not deleted.
• A control API that the fixtures call to read and empty the outbox, to put an account's ALPS settings, Sieve scripts and DAV data back to their defaults, and to create, subscribe or unsubscribe a folder over IMAP as another client would. The control API lists its requests.

SMTP and DAV credentials are checked with an IMAP login to sora, and a successful login is cached for 5 seconds.

ALPS

ALPS is built from your checkout and runs with every plugin enabled: base, caldav, carddav, managesieve, gpg and password. Password changes go to sora's admin API. Its generated alps.toml differs from a deployment's defaults:

• condstore, push_new_mail and push_subscribed_folders are on, so every spec runs with CONDSTORE and with the event stream open;
• rate limiting is off ([server.rate_limit] enabled = false), because every sign-in comes from one address;
• WebAuthn is bound to rpid = "localhost" with the origin http://localhost:<E2E_PORT>;
• a fixed test login_key is set, because the DAV connection tests save separate passwords, which ALPS encrypts with it. Every sign-in therefore also gets the alps_login_token cookie, but no spec tests restoring a session from it, or linked accounts and account switching (see Sessions and Sign-in). e2e/session-expiry.spec.ts removes that cookie before it makes a session expire;
• dav_allow_private_servers = true, because the DAV servers the tests enter in Settings are the test stack's own, on loopback over plain HTTP;
• [plugin.caldav] and [plugin.carddav] name the test stack's DAV server, unless E2E_DAV_PERSONAL=1 leaves it out (see DAV connections).

The generated configurations connect over plain text on loopback, with rate limiting off and fixed test keys. They suit only a throwaway local run. Do not base a deployment on them; start from Configuration Overview and Running in Production instead.

What the specs are given

Playwright reads frontend/playwright.config.ts:

• specs are the *.spec.ts files under frontend/e2e/;
• one worker, not fully parallel;
• 30 s per test and 10 s per expect, no retries, the list reporter;
• baseURL from ALPS_URL, browser locale en-US unless a spec sets another;
• a trace recorded for every test and kept only when it fails, and a screenshot only on failure;
• the output directory from E2E_RESULTS, or frontend/test-results when that is unset;
• one project, chromium (Desktop Chrome).

The harness passes the servers' addresses in ALPS_URL, CONTROL_URL, SORA_ADMIN_URL, SORA_ADMIN_KEY, DAV_URL, SIEVE_ADDR, IMAP_ADDR, E2E_USER and E2E_PASSWORD, plus ALPS_SECONDARY_URL with E2E_CLUSTER=1, and always sets E2E_RESULTS. Where one of the addresses is unset, the config and the fixtures fall back to the E2E_PORT=8900 layout.

Cluster mode

E2E_CLUSTER=1 scripts/e2e.sh e2e/cluster-recovery.spec.ts

With E2E_CLUSTER=1, the harness also starts a second ALPS on E2E_PORT+9, against the same sora. Each ALPS gets a [cluster] section with its own node_id. The first gossips on E2E_PORT+10, and the second on +11 with the first as its peer. They are separate processes with separate sessions, and they share the gossip, the test login_key, sora and the rest of the test stack. Because both have the same login_key, either ALPS can restore a session from a login token the other issued. Clustering and docs/CLUSTER.md explain what the cluster carries.

• Specs get the second ALPS's address in ALPS_SECONDARY_URL. e2e/cluster-recovery.spec.ts needs it and skips itself otherwise. It creates a throwaway account, signs in to both processes with it, and checks that each two-factor recovery code works only once, whichever ALPS it reaches.
• The second ALPS logs to alps-secondary.log. If it does not come up, the message names secondary alps. If it dies during the run, the message names alps-secondary.
• The cluster ports +9 to +11 are checked for listeners too, so a cluster run needs 12 free ports.
• The second ALPS runs from a copy of the first's configuration with its own listen address and its own WebAuthn origin, http://localhost:<E2E_PORT+9>. Both use the relying party localhost, so a security key can be added or used through either ALPS.

Run the cluster mode when you change how ALPS stores and updates security keys and recovery codes, together with the security-key spec:

E2E_CLUSTER=1 scripts/e2e.sh e2e/cluster-recovery.spec.ts e2e/settings-security.spec.ts

WEBAUTHN_COORDINATION.md lists the other checks for that change.

Message previews

A default run leaves message previews off. e2e/message-preview.spec.ts then checks that a server without previews shows none and that ALPS makes no extra requests for them, and it skips the positive test. To test previews, point SORA_DIR at a sora checkout that implements FETCH PREVIEW and opt in:

E2E_PREVIEW=1 SORA_DIR=../sora-preview scripts/e2e.sh e2e/message-preview.spec.ts

E2E_PREVIEW=1 adds the PREVIEW keys to the generated sora.toml and runs the positive test, which fails if the server cannot supply previews. Without it, the generated sora.toml has no PREVIEW keys, so the default suite also runs against sora checkouts that do not know them.

DAV connections

e2e/dav-connections.spec.ts tests the CalDAV and CardDAV connections users set up under Settings > Calendar and Settings > Contacts. Run it both ways:

scripts/e2e.sh e2e/dav-connections.spec.ts
E2E_DAV_PERSONAL=1 scripts/e2e.sh e2e/dav-connections.spec.ts

Both runs sign in as USER and save separate credentials for a throwaway account, first with a password the DAV server refuses. They then make a calendar or a contact through the saved connection, reload it, and remove it. In the default run, ALPS has the test stack's DAV server as its default server, so the spec leaves the server URL blank, and removing the connection goes back to the default server with the mail credentials. With E2E_DAV_PERSONAL=1, the generated alps.toml loads both DAV plugins without a server, so the spec enters the test stack's server as the user's own: the Calendar and Contacts tabs appear only once the connection is saved, and go again when it is removed. Give E2E_DAV_PERSONAL=1 only with this spec: the other calendar, task and contact specs need the default servers and do not skip themselves without them. docs/CDAV.md describes the connections, and Plugin Configuration the server settings.

Writing a test

Specs live in frontend/e2e/*.spec.ts. Import test and expect from ./fixtures, never from @playwright/test. The fixture's page is what fails a test on console errors. Types such as Page still come from @playwright/test.

import { allowConsoleErrors, deliver, expect, login, test } from "./fixtures";

test("a delivered message is listed with its sender", async ({ page }) => {
  const subject = `Example ${Date.now()}`;
  await deliver({ subject, from: "Grace Hopper <grace@remote.test>" });

  await login(page);
  const row = page.locator("alps-message-list .message-item").filter({ hasText: subject });
  await expect(row).toBeVisible();
  await expect(row).toContainText("Grace Hopper");
});

test("a refused request is allowed in this one test", async ({ page }) => {
  await page.goto("/#/login");
  allowConsoleErrors(page, /status of 401/);
  const status = await page.evaluate(async () => (await fetch("/mailboxes/INBOX/status")).status);
  expect(status).toBe(401);
});

The console watcher

The fixture's page fails its test on any console message of type error and on any uncaught page error, including unhandled rejections. It checks after the test body, so a failed assertion is reported before the error it caused.

• A short built-in list is ignored: favicon, Lit is in dev mode, [vite], and Chromium's Blocked script execution in 'about:srcdoc', which the sandboxed message reader frame logs on every HTML message.
• allowConsoleErrors(page, ...patterns) permits more, for one test only. Call it before the action that provokes the error.
• Chromium logs Failed to load resource: the server responded with a status of 401 (Unauthorized) (or 403, 429 and so on) for every refused request, even one the test expects. Allow that exact line in that test. Do not widen the built-in list, because that would hide a real error everywhere else.
• Only the fixture's page is watched. A page the spec opens itself, in its own browser.newContext() or as another page of the same context, can log errors without failing the test. Such contexts take the config's baseURL and locale unless the spec overrides them.

Fixtures

Helper	What it does
USER	The run's account, { address, password }, from E2E_USER and E2E_PASSWORD.
login(page)	Puts the signed-in mailbox on screen. It reuses the run's last working session when that still works, and signs in through the form otherwise.
signIn(page), signInAs(page, address, password)	A real sign-in through the form, every time. Use them in tests about signing in.
deliver({ subject, from, to, body, headers, messageId, raw, recipient, mailbox })	Delivers a message through sora's delivery API, so Sieve filters run on it. mailbox files it straight into that folder and skips the filters. recipient delivers to another account. raw replaces the generated plain-text message. Each generated message gets a unique Message-ID unless messageId is given.
sentMessages()	Every message the SMTP server accepted, oldest first, as { from, recipients, username, data, at } (the SentMessage type), with the raw message in data.
waitForSent(match, timeout = 10_000)	Polls the outbox until a message satisfies match and returns it.
clearSent()	Empties the outbox.
resetSettings(account?), resetSieve(account?), resetDav(account?)	Put the account's ALPS settings, Sieve scripts, or calendars and contacts back to their defaults. resetSettings() deletes everything ALPS stores for the account in IMAP METADATA, its linked accounts, security keys, GPG keys and CalDAV and CardDAV connections included. For USER it also forgets the cached session, so the next login() signs in through the form.
mailboxElsewhere(verb, mailbox, account?)	"create", "subscribe" or "unsubscribe" a folder over IMAP, as another client would. A folder made with "create" is left unsubscribed. ALPS keeps the folder list in its session for minutes, so a test that wants the sidebar to show the change asks for a refreshed listing first.
seedEvent(…), seedCalendarObject(uid, ics, account?), readCalendarObject(uid, account?), removeCalendarObject(uid, account?)	Calendar objects written, read and deleted as a DAV client would.
seedContact(…), readContact(uid, account?), removeContact(uid)	Contacts written, read and deleted as a DAV client would.
createAccount({ address, password }), deleteAccount(address)	Throwaway accounts in sora.
allowConsoleErrors(page, ...patterns)	Permits matching console errors in one test.
toast(page, text)	The visible toast that carries text.
setDocked(page, docked)	Docks or collapses the desktop sidebar and waits for its width to settle.
hasFocus(locator)	Whether the element has the focus, looking into shadow roots.
tabTo(page, target, what, presses = 60, key = "Tab")	Presses key until target has the focus, and fails naming what if it never does.

account defaults to USER. Passing another account is how a spec cleans up after a throwaway one. seedEvent, seedContact and removeContact always act on USER.

frontend/e2e/sieve-server.ts talks ManageSieve to sora directly. storedScripts(), storeActiveScript(name, content) and storeSwitchedOffScript(name, content) read and write the account's Sieve scripts as another client would, so a test can check what the server holds rather than what ALPS shows. All three sign in as USER.

login() decides whether to reuse the cached session with one rule, kept in frontend/e2e/session-probe.ts and pinned by the Vitest test frontend/test/session-probe.test.ts. It probes GET /mailboxes/INBOX/status outside the page, and only a 401 counts as a dead session. Any other answer, including a 5xx, or a probe that never completes, keeps the cached cookies.

Tests that need a hostname

The suite reaches ALPS at http://127.0.0.1:<port>, but a security key is bound to a hostname and Chromium takes no IP address as one. A spec that registers a security key sets test.use({ baseURL }) to ALPS_URL with 127.0.0.1 replaced by localhost, the relying party the harness configures. e2e/settings-security.spec.ts does this.

Rules for a serial suite on one account

• Use unique subjects, for example with Date.now(). Never assume the inbox is empty.
• Put back anything global a test changes, with resetSettings() and the other reset helpers in afterEach.
• A test that deletes its account or changes its password uses an account from createAccount, never USER, and deletes it at the end.
• Wait for a result with expect(…) or expect.poll(…), not with a fixed sleep. The few page.waitForTimeout(…) calls in the suite either show that something does not happen within a window (no draft autosave, no second event stream) or outlast a known delay, such as the draft store's half-second wait. If you need one, say why in a comment.
• A spec that needs an optional part of the stack skips itself when the variable for it is missing, as e2e/cluster-recovery.spec.ts does with test.skip(!secondary, …).

Typechecking the specs

cd frontend && npm run typecheck:e2e

This typechecks frontend/e2e and playwright.config.ts with Node.js types, and it rejects unused imports and parameters. It cannot see a locator whose text or role the page does not render. Only running the suite shows that.

Reading a failed run

Before the tests start

Message	What to do
e2e: <tool> is required and was not found on PATH.	Put go, npm, psql, curl and lsof on PATH.
e2e: no sora checkout at <dir>; set SORA_DIR.	Point SORA_DIR at a directory that contains cmd/sora/main.go.
e2e: cannot reach PostgreSQL as <user>@<host>:<port>; set E2E_PG_*.	Set the E2E_PG_* variables. The user also needs permission to create and drop databases.
e2e: port <port> is in use; stop whatever holds it or set E2E_PORT.	An earlier run is still up, or something else holds the port. Stop it, or pick another E2E_PORT.
e2e: --last-failed needs a previous run record at <dir>/.last-run.json; run the suite once first.	No earlier run left a record in that results directory. Run the suite, or the specs you want, without --last-failed first, with the same E2E_PORT, E2E_LOGS and E2E_RESULTS.
e2e: gave up waiting for the build lock at <path>	Another run held the build lock for about ten minutes.
The last 30 lines of frontend-build.log	The frontend build failed.
e2e: <server> exited during startup; <log>: or e2e: <server> did not become ready; <log>:	<server> is the test stack, sora, alps or secondary alps. The last 20 lines of its log follow. A server has about 60 seconds to come up.

A run that stops at one of the first four messages has not deleted anything yet. One that stops at the --last-failed message keeps the logs directory, but, unless E2E_KEEP_STATE is set, drops a database and state directory that an earlier run on the same port kept. The port check sees only servers that are already listening. A second run started on an overlapping E2E_PORT while the first is still building passes the check, and one of the two then fails at startup.

During and after the tests

• ==> a server died during the run: <names> — the results above are void. Every test after that point failed for that reason alone. Read <name>.log in the logs directory the message prints before you look at the failures. The run exits 1 whatever Playwright reported.
• Error: the page logged errors. The application logged a console error or threw. The report lists each one as console.error: … or uncaught: …. That is a finding, not noise. If the error is a refusal the test expects, allow that line in that test.
• sign-in did not reach the mailbox. A sign-in through the form did not reach the mailbox within 15 s. When the form showed an error, the message goes on with — the login form says: and that error, for example Login failed. Please check your credentials. A refused sign-in also logs a 401 console line, so the same test usually reports the page logged errors as well.
• no matching message was sent. waitForSent found no matching message in the outbox before its timeout.

When tests fail and every server is still running, the script ends with two lines, ==> server logs: <dir> and ==> traces and screenshots: <dir>. After a server died during the run, the next line is Logs: <dir> instead. The logs directory holds e2estack.log, sora.log and alps.log, plus frontend-build.log when the frontend was built and alps-secondary.log with E2E_CLUSTER=1, and, unless E2E_RESULTS names another place, Playwright's test-results/. All of it is kept after the run and deleted when the next run on the same port starts, apart from Playwright's .last-run.json, which the next run keeps for --last-failed.

Each failed test gets its own directory under E2E_RESULTS, by default ${TMPDIR:-/tmp}/alps-e2e-logs-<port>/test-results/. It holds trace.zip, error-context.md and, when the test used a page, test-failed-1.png. Open the trace from frontend/, with the directory the script printed:

cd frontend && npx playwright show-trace <traces and screenshots directory>/<test directory>/trace.zip

To look at the database and state of a failed run, run it again with E2E_KEEP_STATE=1. The script then prints e2e: state kept in <dir> and database alps_e2e_<port>. The servers are stopped either way. To run only the tests that failed, run scripts/e2e.sh --last-failed with the same E2E_PORT, and the same E2E_LOGS or E2E_RESULTS if you set them (see Running all or part of the suite).

When a run fails in docs/E2E.md has the full list.