Testing

Testing

Job Matcher has three layers of tests. Unit tests sit next to the modules they cover and run with bun test. API integration tests in apps/api/test/ boot the real Nest application against PostgreSQL and Redis and drive it over HTTP with supertest. Playwright end-to-end tests run the API, the worker and the web app on fake providers in their own database and click through the product in Chromium. Every test runs without network access to Firecrawl, OpenRouter, Discord or the ATS APIs. This page explains how each layer is set up, how the databases are provisioned, how to run one test, and the conventions for new tests. For how CI runs them, see CI/CD and Releases.

Summary

Layer	Location	Runner	Needs	Command
API unit	apps/api/src/**/*.test.ts (39 files)	bun test	The generated Prisma client types (bun install generates them); no database	bun run test
API integration	apps/api/test/*.test.ts (16 files)	bun test	Postgres and Redis from .env, migrations applied	bun run test
Web unit	apps/web/lib/*.test.ts (3 files)	bun test	Nothing	bun run test
End-to-end	e2e/*.spec.ts (3 files)	Playwright, Chromium	Postgres and Redis, ports 3000 and 4000 free for the API	bun run test:e2e

bun run test at the root runs bun --filter '*' test, which runs the test script in apps/api and apps/web (packages/shared has none). bun run check runs lint, typecheck, test and test:e2e in that order.

API tests

Configuration

The API's test script is bun --env-file=../../.env test --timeout 30000: it loads the root .env and allows 30 seconds per test. apps/api/bunfig.toml sets:

Setting	Value	Effect
preload	./test/setup.ts	Runs before any test file
coverage	true	Coverage is always collected
coverageThreshold	{ lines = 0.9, functions = 0.75 }	Applied per loaded file: every source module a test loads must reach 90% of lines and 75% of functions, or the run fails
coveragePathIgnorePatterns	src/generated/**, test/**	The Prisma client and the test helpers are not measured

The function threshold is 75% because Bun counts a function on each Data.TaggedError line in src/matching/errors.ts that no test can call; the comment in bunfig.toml notes every other file is at 80% or more. src/matching/modules.test.ts imports every matching module, so a module without its own test still counts against the threshold.

The preload: test/setup.ts

apps/api/test/setup.ts makes it impossible for a test to reach a real provider:

1. Deletes FIRECRAWL_API_KEY, OPENROUTER_API_KEY and FIRECRAWL_MODE from the environment, so no live layer can be built (this also covers the Firecrawl SDK, which uses axios rather than fetch).
2. Sets PROVIDERS=fake.
3. Replaces globalThis.fetch with a wrapper that rejects any request to firecrawl.dev, openrouter.ai, discord.com, discordapp.com, greenhouse.io, lever.co, ashbyhq.com, smartrecruiters.com or recruitee.com (and their subdomains) with "Tests must not call ".

Test database

The API tests do not create a separate database. They use DATABASE_URL and REDIS_URL from the root .env, which in development is the same job_matcher database the dev server uses. Isolation comes from the data, not the database:

• Each integration test signs up fresh users with random emails (test-<uuid>@example.com) through signUp(), or creates rows with test-<uuid> ids.
• afterAll hooks delete the rows the file created (deleteMany({ where: { userId } })) and remove queued BullMQ jobs.
• onboarding-backfill.test.ts runs the backfill SQL from a migration inside a transaction that is rolled back, so the shared database is left as it was.

The schema must be migrated before the tests run: bun run db:migrate in development, bun x --bun prisma migrate deploy in CI. Redis must be reachable because the API's health check, the queue and the schedulers use it.

Helpers

File	Exports	Use
helpers.ts	startApp(), signUp(server)	startApp() calls the same createApp() as production with the logger off and initialises it. signUp() creates an account through POST /api/auth/sign-up/email and returns a supertest agent that keeps the session cookie.
ats.ts	fixture(name), fakeHttp(replies) and related	Serves scripted replies per URL (a body string or { status, body }) through the HttpFetch service; any other URL fails like an offline network. Used by the ATS adapter tests.
pages.ts	pageFixture(name)	Careers page markdown as Firecrawl returns it.

Fixtures

apps/api/test/fixtures/ holds the data both the tests and demo mode use. The fake providers in providers/fake.ts read listing-page.md, listings.json, resume.md and jobs/*.md at runtime, so these files ship in the API image.

Folder or file	Contents
ats/	Recorded Greenhouse, Lever, Ashby, Recruitee and SmartRecruiters API responses, and careers pages that embed an Ashby or Greenhouse board
jobs/	Three job pages (backend-payments.md, full-stack.md, senior-platform.md), each with a Verdict: line the fake evaluator reads
pages/	Careers pages in markdown and HTML, including one with no links and one with only apply links
scrapes/	Scrape outcomes: a job page, a Cloudflare challenge, 403, 404 and a thin shell page
listing-page.md, listings.json	The fake careers page and its extracted listings
resume.md, job-ad.pdf	The fixture resume and a PDF job ad

Integration test files

File	Covers
cache.test.ts	DELETE /api/cache: auth, clears only the signed-in user's pages
fake-providers.test.ts	The fake pipeline end to end, scripted Firecrawl 429, search sources, local search rules, reported credits, blocked real hosts, fake Discord URL check
matches.test.ts	Listing, status filter, cursor paging, status updates, ownership
onboarding.test.ts	Step derivation, completion on first manual or scheduled run, complete, dismiss and restore, concurrency
onboarding-backfill.test.ts	The onboarding migration's backfill SQL
overview.test.ts	GET /api/overview: empty state, per-user counts, next scheduled tick
page-cache.test.ts	The PageCache store: misses, overwrites, per-user isolation
process-run.test.ts	The run pipeline: dedupe and filters, run size, context reuse and retries, OpenRouter 401, preflight and local Firecrawl stops, structured data, title filters, triage on and off, the credit guard (reserve, daily allowance shared across scheduled runs), search source skips, redelivery and recovery
recover-run.test.ts	Ending runs whose jobs were lost, stalled or failed
runs-processor.test.ts	Worker startup recovery and schedule restore, run jobs, schedule ticks
runs.test.ts	POST /api/runs validation, one active run per user, simultaneous starts, lost-job recovery, Redis outage; run listing and details; retrying a failed match save and its Discord alert; /api/health and /api/health/worker
schedule.test.ts	enqueueRun(): queue failure marks the run FAILED; job id equals run id
scheduled-run.test.ts	Scheduled run creation rules
scrape-routes.test.ts	The ScrapeRoute store: block counting, cloud preference, reset on success
settings.test.ts	Defaults, saving, trimming, validation errors, scheduler sync
sources.test.ts	Add, list, delete, invalid and duplicate URLs, kind detection, kind override clearing the cache, search sources, ownership

Unit tests in src/ cover each matching module and each source adapter, plus onboarding/onboarding.test.ts and overview/overview.service.test.ts.

Web tests

apps/web runs bun test with no extra configuration and no coverage threshold. The three test files cover pure logic in lib/:

File	Covers
safe-next.test.ts	The ?next= redirect validation (open-redirect cases, encoded slashes, auth pages)
serial.test.ts	The one-at-a-time task queue used by settings saves
welcome-steps.test.ts	Wizard step order, reachability and resume logic

Components and pages are covered by the Playwright suite rather than component tests.

End-to-end tests

Configuration

playwright.config.ts:

Setting	Value
testDir	e2e
timeout	60 s per test
retries	1 in CI, 0 locally
reporter	list; in CI also html (never opened)
use	baseURL: http://localhost:3000, trace: retain-on-failure
Projects	chromium (Desktop Chrome)

Isolated database and fake providers

The config loads .env when DATABASE_URL is not already set, then appends _e2e to the database name, for example job_matcher becomes job_matcher_e2e. The first web server command runs prisma migrate deploy against that URL, which creates the database if needed. The API and worker receive PROVIDERS=fake and the _e2e DATABASE_URL, so the suite never calls a provider and never writes into development data. They share the development Redis.

Web servers

Server	Command	Ready when	Reuse
API and worker	bun x --bun prisma migrate deploy && bun run --cwd apps/api --parallel start start:worker	http://localhost:4000/api/health/worker answers (a worker is connected, so a started run is picked up at once)	Never: a running dev API would be on live providers. Stop bun run dev first, or port 4000 is taken.
Web	Locally bun run --cwd apps/web dev; in CI bun run --cwd apps/web build && bun run --cwd apps/web start	http://localhost:3000 answers (timeout 180 s)	Locally a running web server on port 3000 is reused; in CI never

The ports are fixed at 3000 and 4000. To run the suite while a development server holds those ports, run it from a separate checkout after stopping the dev API, or change the ports in a local copy of the config.

Shared steps: e2e/support.ts

e2e/support.ts exports the values and steps the specs share:

Export	What it does
password, sourceUrl (https://board.example.com/jobs), matchTitle (Full Stack Developer)	Shared constants. With fake providers this source lists the fixture jobs, and with no experience set only the listing without an experience range passes the filters.
signUp(page)	Signs up with a random email and checks the wizard opens on "How it works"
signIn(page, email), signOut(page)	Account menu flows
addResume(page)	From the wizard's welcome, saves https://example.com/cv.pdf and checks it moves to "Matching basics"
completeSetup(page)	Walks the whole wizard: resume, Use defaults, adds the source, Start first run, and checks the run page opens with ?first=1 and the "What happens during a run" region

Specs

Spec	Tests
happy-path.spec.ts	Sign up, complete the wizard, see the run complete with a match, open See your matches, Mark applied, then find it under Applied with Move to pending
navigation.spec.ts	A signed-in visitor skips the landing and auth pages; a signed-out visitor returns to the requested page after sign-in; unsafe next values fall back to /home; sign-out protects the app again; a returning user who finished setup lands on /home, never the wizard; the phone navigation menu; a stale session cookie is cleared instead of looping; switching accounts in another tab resets open pages
onboarding.spec.ts	Leaving the wizard keeps progress and Continue setup resumes at the right step; the setup checklist can be hidden (persisted across reloads) and Setup guide brings it back

When a test fails, Playwright keeps its trace in test-results/. CI uploads playwright-report/ and test-results/ as the playwright artifact.

Running a single test

# One API test file (from the repository root)
bun run --cwd apps/api test test/runs.test.ts

# Tests whose name matches a pattern
bun run --cwd apps/api test -t "stops on an OpenRouter 401"

# One web test file
bun run --cwd apps/web test lib/safe-next.test.ts

# One Playwright spec, or one test by title
bun run test:e2e e2e/happy-path.spec.ts
bun run test:e2e -g "the checklist can be hidden"

# Watch the browser
bun run test:e2e --headed

Running a single API file still applies the coverage thresholds to every source file that file loads. A file that loads modules it only partly exercises can fail on coverage even when its tests pass; run the whole suite to judge coverage.

Writing tests

Convention	Detail
Placement	Unit tests next to the module as <module>.test.ts; tests that need the Nest app, Postgres or Redis in apps/api/test/; browser flows in e2e/
Imports	import { describe, expect, it } from "bun:test" (the API tests use describe/it)
Names	Plain sentences describing behaviour, for example it("rejects a second run while one is active")
Users and data	Create fresh users with signUp() or random ids; never depend on existing rows; delete what you created in afterAll
Providers	Never call real services. Use the Effect service fakes (HttpFetch.of, Firecrawl, MatchEvaluator.of, ...) or the fixture-backed layer in providers/fake.ts. Script failures with ?fake-error=<status> or ?fake-blocked URLs, or a job page's Verdict: line (match, auth, invalid).
Fixtures	Add files under apps/api/test/fixtures/. Recorded ATS responses go in ats/. Keep anything the demo mode reads backwards compatible.
Coverage	A new source file must reach 90% of lines and 75% of functions in whatever test loads it. A new matching module is loaded by modules.test.ts, so it needs tests from the start.
Every endpoint	Include a "rejects requests without a session" test and an ownership test ("hides another user's ..."), as the existing files do
e2e	Use roles and labels (getByRole, getByLabel), not CSS selectors. Next.js keeps earlier pages in the DOM hidden, so filter with { visible: true } when the same text can appear on a previous page. Reuse the steps in support.ts.