Testing

Testing

Unit and integration tests

cargo test --workspace --locked   # Rust: all crates, including src-tauri
npm test                          # UI: vitest run (Testing Library, axe-core accessibility checks)

Live engine tests (crates/engine/tests/live*.rs) are ignored by default. They need the local v15 server and use netsim on 127.0.0.1:64800/64801. They drive their scripted second users through the mumbot library (see below):

testenv/up.sh v15
MUMCORD_LIVE=1 cargo test -p mumcord-engine --features test-backend --test live -- --ignored --test-threads=1

See Development for the test servers.

Other checks: cargo fmt --all --check, cargo clippy --workspace --all-targets --locked -- -D warnings, npm run lint, npm run typecheck.

The test bot (mumbot)

The scripted second participant for tests is mumbot, which lives in its own repository (it used to be crates/bot; see its Test bot CLI page). Mumcord uses a pinned release:

• scripts/mumbot.env holds the version and the SHA256 of every release archive. scripts/fetch-mumbot.sh <linux-x86_64|linux-aarch64|windows-x86_64|macos-universal> <dest> downloads it, verifies the hash and unpacks the binary; archives are cached in ~/.cache/mumbot. Bump all lines together when updating.
• E2E: the build step fetches the linux-x86_64 binary next to the debug binaries, so it ships to the shard jobs in the existing tarball. The container exports MUMCORD_TESTENV_DIR and MUMCORD_BOT_DATA for the bot.
• Smoke test: each job fetches the binary for its own target and passes it to smoke.sh --bot.
• Live engine tests: they use the mumbot library at the same tag. A [patch.crates-io] section in the workspace Cargo.toml builds it against this workspace's protocol and codec, and .cargo/config.toml points it at testenv/ for the SuperUser password.

End-to-end tests

UI end-to-end tests drive the real app (real engine inside Tauri) through WebDriver, against the local test servers.

npm run e2e                                              # whole suite
npm run e2e -- --spec e2e/specs/smoke/app.e2e.ts         # one spec (args go to `wdio run`)
MUMCORD_E2E_SHARD=2 npm run e2e                          # one of the four CI shards (e2e/shards.json)

• Parallel local runs: build once, then start one run per shard, each with its own run id (MUMCORD_E2E_RUN, 1 to 9, needs MUMCORD_E2E_STEP=test):

  MUMCORD_E2E_STEP=build npm run e2e
  for n in 1 2 3; do
    MUMCORD_E2E_STEP=test MUMCORD_E2E_RUN=$n MUMCORD_E2E_SHARD=$n npm run e2e > e2e-run-$n.log 2>&1 &
  done
  wait
  

  Run n gets its own display, WebDriver ports, home directory, compose project and test server ports (MUMCORD_PORT_BASE, default 40000+1000n; keep it in the 4xxxx range). run.sh takes care of locking, so builds wait for running tests. 2 to 3 parallel runs work on a 16-core machine; more cause timing flakes.

• Requires only Docker (your user in the docker group) and npm ci on the host. Everything else runs in the mumcord-e2e image (WebKitWebDriver, tauri-driver, Xvfb, gnome-keyring).

• Cold run takes about 5 minutes (image build and first debug build), a warm run about 15 seconds without code changes. Build caches live in ~/.cache/mumcord-build/e2e/ (override with MUMCORD_E2E_CACHE).

• Extra docker run flags go into MUMCORD_E2E_DOCKER_ARGS, for example MUMCORD_E2E_DOCKER_ARGS="--cpus=2" npm run e2e to mimic a 2-core CI runner. MUMCORD_E2E_IMAGE changes the image name.

• Audio: the app is built with the test-backend feature, and MUMCORD_AUDIO_BACKEND selects null (silence in, discard out; default) or file:<in.wav>:<out.wav> (48 kHz mono 16-bit).

• Every spec file gets a fresh profile and an empty keyring; spec files run one at a time.

• Failures save a screenshot and page source to e2e/artifacts/ (git-ignored); axe reports go there too.

• Specs live in e2e/specs/<area>/<name>.e2e.ts (mocha + WebdriverIO) and use helpers from e2e/helpers/index.ts for starting servers, running the bot, using netsim and restarting the app. See e2e/README.md.

In CI the suite runs on pull requests to main: one job builds the test image and app once, five shard jobs run the spec files listed in e2e/shards.json in parallel (each with its own test servers), and the "Linux E2E" job sums them up. CI is now faster (cached apt packages and parallel shards); check the Actions page for current times. A new spec file must be added to e2e/shards.json, otherwise the run fails.

Packaging smoke test

scripts/smoke/smoke.sh starts an installed build and checks that it connects to a local server. The test-builds workflow runs it on Linux, Windows and macOS.

CI workflows

Workflow	File	Runs on	What it does
CI	.github/workflows/ci.yml	every push and pull request (build jobs only for code changes, see below)	On Ubuntu, Windows and macOS: cargo fmt, clippy, cargo test, UI lint, typecheck, UI tests, and a Tauri debug build (--debug --no-bundle)
E2E	.github/workflows/e2e.yml	pull requests to main, manual	Linux E2E suite via npm run e2e; uploads failure artifacts
Release	.github/workflows/release.yml	pushes of v* tags	Unsigned AppImage, Windows installer and universal DMG, installed and smoke-tested on fresh runners, then a draft GitHub Release with SHA256SUMS (see Development)

When the build and E2E jobs run

scripts/ci/code-changed.sh decides for each pull request whether it changes source code or a build input. It reads the pull request's file list and compares it with an allowlist: Rust sources and manifests (crates/, src-tauri/, Cargo.toml, Cargo.lock, proto/), the UI (ui/, TypeScript, JavaScript, CSS, HTML, package.json, package-lock.json, build and lint configs), and test and packaging inputs (e2e/, testenv/, scripts/, assets/, licenses/, THIRD_PARTY_NOTICES). Anything not on the list, such as docs, specs, screenshots, LICENSE, .github/ and scripts/ci/, skips the build and E2E jobs. So a change that only touches workflows or CI scripts is exercised fully only on the push to main or by a manual run. Every other trigger counts as a code change: a push to main and a manual run always run everything, so main is always fully tested.

Dependabot

.github/dependabot.yml checks Cargo (the workspace and the smoke-test seeder), npm and GitHub Actions every Monday and opens at most 5 pull requests per ecosystem. Minor and patch updates are grouped into one pull request per ecosystem. Some majors are grouped too (all GitHub Actions majors in one; the @wdio/* packages together; eslint with @eslint/js). A few majors are held back on purpose, each with a reason in the file (for example tauri-specta, specta, TypeScript 7 and @types/node).

apt mirror fallback

scripts/ci/apt-install.sh installs apt packages on the Ubuntu runners. The default mirror sometimes stalls for tens of minutes, so every connection has a short timeout, every attempt a hard deadline, and each failed attempt switches to the next mirror (a second mirror per architecture) before retrying. The workflows use it instead of plain apt-get install.

Manual checks

Things that cannot be judged automatically (how audio sounds, Windows and macOS behaviour) are checked by hand against a local test server, following the acceptance criteria in the spec.