Skip to content

Backend API

Karthik P edited this page Oct 15, 2026 · 1 revision

The backend is a NestJS 12 application in apps/api, run on Bun. It serves a JSON HTTP API under the /api prefix, authenticates every route with better-auth unless a route opts out, validates request bodies and queries with Effect Schemas shared with the web app, and persists through Prisma. It never scrapes or calls an LLM itself: starting a run creates a database row and queues a BullMQ job for the run worker (see Architecture). This page lists the module layout, every endpoint, and the conventions the controllers follow.

Bootstrap

File Role
main.ts Creates the app, enables shutdown hooks, listens on API_PORT.
app.ts createApp(): NestFactory.create(AppModule, { bodyParser: false }) and setGlobalPrefix("api"). Tests build the app through the same function.
app.module.ts Root module of the HTTP API.
config.ts Decodes and validates the environment at import time.

Nest's own body parser is disabled because the better-auth integration (@thallesp/nestjs-better-auth) installs JSON and URL-encoded parsers that skip /api/auth/*, leaving those requests' bodies for better-auth to read. No global validation pipe, exception filter or CORS is configured.

Module map

flowchart TB
  subgraph AppModule["AppModule (main.ts)"]
    direction TB
    infra["PrismaModule (global)<br/>RedisModule (global)<br/>QueueModule (BullMQ 'runs')<br/>AuthModule (better-auth, global guard)"]
    health["HealthController"]
    sources["SourcesController + SourcesService"]
    settings["SettingsController + SettingsService"]
    runs["RunsController + RunsService + MatchAlerts"]
    matches["MatchesController + MatchesService"]
    cache["CacheController + CacheService"]
    overview["OverviewController + OverviewService"]
    onboarding["OnboardingController + OnboardingService"]
  end
  subgraph WorkerModule["WorkerModule (worker.ts)"]
    winfra["PrismaModule, RedisModule, QueueModule"]
    proc["RunsProcessor"]
    rt["EffectRuntime (ManagedRuntime)"]
  end
Loading
Module or provider Source Notes
PrismaModule / PrismaService prisma/ Global. PrismaService extends PrismaClient with the PrismaPg adapter on DATABASE_URL; disconnects on module destroy. The client is generated into apps/api/src/generated/prisma.
RedisModule / RedisService redis/ Global. RedisService extends Redis (ioredis) on REDIS_URL with maxRetriesPerRequest: null, so commands wait for a reconnect instead of failing. Quits on shutdown.
QueueModule queue/queue.module.ts BullModule.forRootAsync reusing RedisService as the connection, and registerQueue({ name: "runs" }). Exports BullModule, so services inject the queue with @InjectQueue(runsQueue).
AuthModule auth/auth.module.ts better-auth mounted at /api/auth, plus a global guard. See Authentication.
MatchAlerts runs/match-alerts.ts Sends the Discord alert for a match saved by the retry-save endpoint. Uses the fake notifier when PROVIDERS=fake. A failed alert is logged, not returned.

Queue constants live in runs/runs.queue.ts: queue runs, run jobs { runId } named run, schedule ticks { userId } named scheduled, scheduler id user:<userId>.

Authentication on routes

AuthModule registers a global guard (the library's default). Every controller route requires a valid better-auth session unless it is marked @AllowAnonymous(); only HealthController is. A request without a valid session receives 401 Unauthorized. Controllers read the user with the @Session() session: UserSession parameter decorator and scope every query by session.user.id, so a user can only see and change their own rows; a lookup of another user's id returns 404.

Endpoints

All paths include the /api prefix. "Session" means a valid better-auth session cookie is required. Response bodies are JSON; Prisma rows are returned as stored, with dates as ISO 8601 strings. Request schemas are in packages/shared/src/index.ts.

Summary

Method Path Auth Success Controller
GET /api/health none 200 health.controller.ts
GET /api/health/worker none 200 same
GET /api/sources session 200 sources.controller.ts
POST /api/sources session 201 same
POST /api/sources/search session 201 same
PATCH /api/sources/:id session 200 same
DELETE /api/sources/:id session 204 same
GET /api/settings session 200 settings.controller.ts
PUT /api/settings session 200 same
GET /api/runs session 200 runs.controller.ts
GET /api/runs/:id session 200 same
POST /api/runs session 201 same
POST /api/runs/:id/results/:resultId/save session 200 same
GET /api/matches session 200 matches.controller.ts
PATCH /api/matches/:id session 200 same
DELETE /api/cache session 200 cache.controller.ts
GET /api/overview session 200 overview.controller.ts
POST /api/onboarding/complete session 200 onboarding.controller.ts
POST /api/onboarding/dismiss session 200 same
POST /api/onboarding/restore session 200 same
any /api/auth/* varies better-auth handler (see Authentication)

Health

GET /api/health

Checks Postgres (SELECT 1) and Redis (PING, cut off after 2 seconds) in parallel.

  • 200 { "database": "up", "redis": "up" } when both answer.
  • 503 with the same object (for example { "database": "up", "redis": "down" }) when either fails.

The production Compose file uses this as the API container's health check, and the web container waits for it.

GET /api/health/worker

Asks BullMQ for the workers connected to the runs queue (cut off after 2 seconds).

  • 200 { "worker": "up" } when at least one is connected.
  • 503 { "worker": "down" } otherwise, including when Redis does not answer.

The Playwright setup waits on this endpoint before running tests.

Sources

Service: sources.service.ts. A source is a JobSource row; see Data Model and Job Sources and Scraping.

GET /api/sources

Returns the user's JobSource rows, newest first.

POST /api/sources

Body field Schema Rules
url HttpUrl Trimmed; must parse as a URL with http: or https: protocol

Creates a source. The kind and adapter key are detected from the URL by detectSource (matching/sources/detect.ts): known ATS hosts become GREENHOUSE, LEVER, ASHBY, SMARTRECRUITERS or RECRUITEE with their board token; anything else is a PAGE. Returns the created row.

  • 409 "This source is already added" when the user already has the same URL (unique userId, url).

POST /api/sources/search

Body field Schema Rules
query trimmed string Not empty; at most 200 characters

Creates a SEARCH source. Runs of whitespace in the query collapse to one space, and the row is stored with kind = SEARCH, the query, and the synthetic URL search:<query in lower case> (searchSourceUrl in matching/sources/search.ts), so the same query, ignoring case, can only be added once. Returns the created row.

  • 409 "This search is already added" for a duplicate.

PATCH /api/sources/:id

Body field Schema Rules
kind SourceKind Any kind except SEARCH
adapterKey string or null Trimmed; empty or matching ^[A-Za-z0-9][A-Za-z0-9_.-]*$. Required (non-empty) when kind is an ATS kind
useMap boolean, optional Page sources: read job links from Firecrawl's map of the site

Switches a source's kind, for example to accept an embedded-board suggestion. In one transaction it updates the row (setting adapterKey to null for PAGE, clearing suggestedKind, suggestedAdapterKey and listingHash, and setting useMap only when provided) and deletes the user's SOURCE_LISTING cache entry for that URL, since cached listings came from the previous way of reading the page. Returns the updated row.

  • 404 "Source not found" if the id is not the user's.
  • 400 "A search source cannot be switched to another type" for a SEARCH source.

DELETE /api/sources/:id

Deletes the source. 204 No Content, or 404 "Source not found". Past runs keep the id in their sourceIds array; there is no foreign key.

Settings

Service: settings.service.ts.

GET /api/settings

Returns the user's settings form fields (the Settings row without userId and the two onboarding timestamps). A user who has never saved gets defaultSettings from the shared package; no row is created.

PUT /api/settings

The body is the full UpdateSettings object (every field is required):

Field Type Rules Default
resumeUrl string or null HttpUrl null
discordWebhookUrl string or null https: on discord.com, discordapp.com, ptb.discord.com or canary.discord.com, path starting /api/webhooks/ null
experienceYears integer >= 0 0
experienceMonths integer 0 to 11 0
targetMinYears number or null >= 0 null
targetMaxYears number or null >= 0; at least targetMinYears when both are set null
locations string array []
includeUnknown boolean true
titleInclude string array At most 30 keywords, each trimmed, non-empty, at most 60 characters []
titleExclude string array Same as titleInclude []
maxJobsPerRun integer 1 to 20 5
triageEnabled boolean true
scheduleMinutes integer or null >= 15; null turns scheduled checks off null

Upserts the Settings row and then synchronises the user's BullMQ job scheduler (syncSchedule): a number upserts scheduler user:<userId> repeating every scheduleMinutes minutes; null removes it. Returns the saved form fields. The meaning of each setting is on Configuration Reference.

Runs

Service: runs.service.ts. See Run Pipeline for what happens after a run is queued.

GET /api/runs

The user's 20 most recent Run rows, newest first, without results.

GET /api/runs/:id

One Run with its results (JobResult rows, oldest first). 404 "Run not found" if it is not the user's. The run page polls this every 2 seconds while the run is QUEUED or RUNNING.

POST /api/runs

Body field Schema Rules
sourceIds non-empty array of non-empty strings Duplicates are removed; every id must belong to the user

Creates a MANUAL run and queues it. Steps:

  1. createRun (create-run.ts) runs one transaction: it takes pg_advisory_xact_lock(hashtext(userId)), checks the resume URL, source ownership and that no run is QUEUED or RUNNING, inserts the run with a provisional contextKey, and sets onboardingCompletedAt if it is still null.
  2. If a run is already active, the service tries to end the user's orphaned runs (bounded to 2 seconds) and, if any were ended, tries createRun once more.
  3. enqueueRun adds job run with jobId = runId and attempts: 1.

Returns the created Run (status QUEUED).

Status Message Cause
400 Add a resume URL in settings before starting a run No resumeUrl saved
400 Source not found Empty list after de-duplication, or an id that is not the user's
409 A run is already in progress Another run is QUEUED or RUNNING
503 Could not queue the run. Try again shortly. Adding the job failed; the run is marked FAILED

POST /api/runs/:id/results/:resultId/save

Retries saving a match whose first save failed, without scraping or evaluating the job again.

  • 404 "Result not found" unless the result belongs to that run and user.
  • 409 "Only a match can be saved" unless isMatch is true.
  • If matchSaved is already true, returns the result unchanged.

Otherwise it upserts the Match, then in a transaction flips matchSaved from false to true and, only if this request flipped it, decrements the run's failed counter. The request that flipped the flag also sends the Discord alert, if a webhook is set. Returns the updated JobResult. Responds 200.

Matches

Service: matches.service.ts.

GET /api/matches

Query parameter Schema Notes
status PENDING or APPLIED, optional Filter
cursor non-empty string, optional The nextCursor from the previous page

Returns { "items": Match[], "nextCursor": string | null }. Pages hold 20 matches, ordered by matchedAt descending and then id descending so the order is stable. nextCursor is the last item's id when another page exists.

PATCH /api/matches/:id

Body field Schema
status PENDING or APPLIED

Sets the match's application status and returns the updated Match. 404 "Match not found" if it is not the user's.

Cache

DELETE /api/cache

Deletes every PageCache row for the user (resume, job pages and source listings) and returns { "deleted": <count> }. The Settings page's "clear cache" control calls this. Scrape routes are shared across users and are not cleared. See Firecrawl Credits and Caching.

Overview

GET /api/overview

What the home page shows, typed by the shared Overview schema:

Field Type Meaning
setup.hasResume boolean A resume URL is saved
setup.sourceCount integer Number of sources
setup.hasPreferences boolean A setting other than the resume differs from the defaults
setup.hasRun boolean At least one run exists
latestRun object or null id, status, createdAt, finishedAt, matchCount (results with isMatch = true) of the newest run
activeRun object or null id and status (QUEUED or RUNNING) of the newest active run
pendingMatches integer Matches with status PENDING
nextScheduledRunAt string or null Next schedule tick; null when the schedule is off or setup is incomplete
onboarding Onboarding See below

nextScheduledRunAt comes from the BullMQ job scheduler's next time, read with a 1-second timeout. If Redis does not answer, the service estimates it from the user's last SCHEDULED run: it steps forward whole intervals from that run's start to the first tick after now (estimateNextRun in overview.service.ts). Before the first scheduled run there is nothing to estimate from, so it is null.

Onboarding

Service: onboarding.service.ts; derivation in onboarding.ts.

All three endpoints take no body, respond 200, are idempotent, and return the user's Onboarding object:

Field Meaning
steps.resume A resume URL is saved
steps.preferences A saved setting other than resumeUrl differs from defaultSettings (optional step)
steps.source At least one source
steps.firstRun At least one run, of either trigger
nextStep resume, preferences, source, firstRun or null: the first incomplete required step, except that preferences comes before source while neither is done
completedAt ISO time the wizard was finished or the first run started, else null
dismissedAt ISO time the setup checklist was hidden, else null
Endpoint Effect
POST /api/onboarding/complete Sets onboardingCompletedAt if it is null
POST /api/onboarding/dismiss Sets onboardingDismissedAt if it is null
POST /api/onboarding/restore Clears onboardingDismissedAt

Settings rows are created lazily. complete and dismiss run a transaction of createMany(..., skipDuplicates: true) (inserting a row with the default settings and the flag) followed by an updateMany guarded on the flag being null, so concurrent first calls cannot fail on the primary key and repeat calls keep the first time. The steps themselves are never stored; they are derived from the user's data on every request.

Request validation

common/schema.pipe.ts defines SchemaPipe, a Nest PipeTransform that decodes the input with an Effect Schema (Schema.decodeUnknownResult). On failure it throws BadRequestException with the schema error's message; on success it returns the decoded value (trimmed strings, for example). Controllers apply it per parameter:

@Body(new SchemaPipe(UpdateSettings)) body: UpdateSettings

The schemas live in packages/shared (@job-matcher/shared) and are imported by both the API and the web app, so the browser validates forms with the same rules and messages. Route parameters such as :id are plain strings and are not validated; an unknown id produces a 404 from the service.

Timeouts

Redis commands wait for a reconnect instead of failing (maxRetriesPerRequest: null), so any request path that must answer while Redis is down gives its Redis call its own deadline with withTimeout (common/timeout.ts) or an equivalent race:

Call Deadline On timeout
Health PING 2 s Redis reported down
Health worker lookup 2 s Worker reported down
Overview scheduler lookup 1 s Next run estimated from past runs
Orphaned-run recovery during POST /api/runs 2 s Treated as nothing recovered; the start answers 409

queue.add in POST /api/runs has no explicit deadline.

Configuration loading

config.ts decodes process.env with an Effect Schema at module load (Schema.decodeUnknownSync), so the API and the worker both refuse to start when a variable is missing or invalid:

Variable Schema Default
DATABASE_URL non-empty string required
API_PORT finite number from string required
BETTER_AUTH_SECRET non-empty string required
BETTER_AUTH_URL non-empty string required
REDIS_URL non-empty string required
OPENROUTER_MODEL non-empty string openai/gpt-6-luna:floor
PROVIDERS live or fake live

Provider settings are read elsewhere, and only by the worker: OPENROUTER_API_KEY in providers/live.ts (live only), FIRECRAWL_* in scrape-router.ts, and OPENROUTER_* routing options in routing.ts. In development the API scripts load the repository-root .env with bun --env-file=../../.env. Every variable is listed on Configuration Reference.

Error handling conventions

Services throw Nest's built-in HTTP exceptions with a short, user-facing sentence, and Nest's default exception filter turns them into:

{ "statusCode": 409, "message": "A run is already in progress", "error": "Conflict" }

The web client (apps/web/lib/api.ts) shows message when it is a string and otherwise "Something went wrong. Try again."

Status Used for
400 Schema validation failures (SchemaPipe), missing resume, unknown sources, switching a search source
401 No valid session (global auth guard)
404 Any id that does not exist or belongs to another user
409 Duplicate source or search (Prisma error P2002), an active run, saving a non-match
503 Health checks failing; a run that could not be queued

Conventions that hold across services:

  • Ownership in the query. Reads use findFirst({ where: { id, userId } }); writes use updateMany or deleteMany with { id, userId } and treat a count of 0 as not found. This avoids a separate ownership check.
  • Idempotent writes. Guarded updateMany calls (such as matchSaved: false or a null onboarding flag) make repeated requests safe and keep counters and timestamps correct under concurrency.
  • Side effects after commit. Discord alerts are sent after the database write, and a failed alert never fails the request.

Related pages

Clone this wiki locally