Skip to content

Data Model

Karthik P edited this page Oct 15, 2026 · 1 revision

Job Matcher stores all durable state in PostgreSQL 17, defined by one Prisma schema at prisma/schema.prisma and evolved by the migrations in prisma/migrations. There are seven application models (Settings, JobSource, Run, JobResult, Match, PageCache, ScrapeRoute) plus better-auth's four (User, Session, Account, Verification). Application rows are scoped by a plain userId string with no foreign key to user; the only foreign keys are JobResult to Run and better-auth's Session and Account to User. This page documents every model and enum, the relations and constraints, the mechanics that depend on them, and the migration history.

Prisma setup

File Purpose
prisma/schema.prisma Models and enums. The prisma-client generator writes an ESM client for the Bun runtime to apps/api/src/generated/prisma (not committed; generated on install). The datasource is postgresql with no URL in the schema.
prisma.config.ts Prisma CLI config: schema path, migrations path prisma/migrations, and the datasource URL from DATABASE_URL.
apps/api/src/prisma/prisma.service.ts Runtime client: PrismaClient with the @prisma/adapter-pg driver adapter on DATABASE_URL.

IDs for JobSource, Run, JobResult and Match default to uuid(7), which are time-ordered UUIDs. better-auth generates its own IDs for its tables.

Entity relationships

Solid lines are database foreign keys. Dotted lines are logical references through a userId (or runId array) that the database does not enforce.

erDiagram
  User ||--o{ Session : "userId (FK, cascade)"
  User ||--o{ Account : "userId (FK, cascade)"
  User ||..o| Settings : "userId (no FK)"
  User ||..o{ JobSource : "userId (no FK)"
  User ||..o{ Run : "userId (no FK)"
  User ||..o{ Match : "userId (no FK)"
  User ||..o{ PageCache : "userId (no FK)"
  Run ||--o{ JobResult : "runId (FK, cascade)"
  Run }o..o{ JobSource : "sourceIds array (no FK)"

  User {
    String id PK
    String name
    String email UK
    Boolean emailVerified
  }
  Session {
    String id PK
    String token UK
    String userId FK
    DateTime expiresAt
  }
  Account {
    String id PK
    String userId FK
    String providerId
    String password
  }
  Verification {
    String id PK
    String identifier
    String value
  }
  Settings {
    String userId PK
    String resumeUrl
    Int maxJobsPerRun
    Int scheduleMinutes
    DateTime onboardingCompletedAt
  }
  JobSource {
    String id PK
    String userId
    String url
    SourceKind kind
    String adapterKey
  }
  Run {
    String id PK
    String userId
    RunStatus status
    RunTrigger trigger
    String contextKey
  }
  JobResult {
    String id PK
    String runId FK
    String userId
    String contextKey
    String jobUrl
  }
  Match {
    String id PK
    String userId
    String jobUrl
    MatchStatus status
  }
  PageCache {
    String userId PK
    String url PK
    CacheKind kind PK
    Json content
  }
  ScrapeRoute {
    String domain PK
    Int localBlocks
    DateTime preferCloudUntil
  }
Loading

ScrapeRoute and Verification have no relation to users. ScrapeRoute is deliberately global: it describes websites, which every user shares.

Because application tables have no foreign key to user, deleting a user row does not delete that user's settings, sources, runs, results, matches or cache. The onboarding migration notes this explicitly for Run.

Application models

Type notation: ? marks a nullable column, [] a Postgres array.

Settings

One row per user, holding the settings form and two onboarding timestamps. Rows are created lazily: by the first PUT /api/settings, by POST /api/onboarding/complete or dismiss, or by the onboarding backfill migration. Until then the API returns defaultSettings from packages/shared.

Field Type Default Purpose
userId String Primary key; the better-auth user id.
resumeUrl String? URL of the resume (a page or a PDF). A run requires it.
discordWebhookUrl String? Discord webhook for match alerts.
experienceYears Int 0 The user's experience, whole years.
experienceMonths Int 0 Extra months, 0 to 11.
targetMinYears Float? Lowest experience requirement of jobs to consider.
targetMaxYears Float? Highest experience requirement of jobs to consider.
locations String[] Accepted job locations.
includeUnknown Boolean true Keep jobs whose location or experience is not stated.
titleInclude String[] [] A title must contain one of these words or phrases (empty means any title).
titleExclude String[] [] A title must contain none of these.
maxJobsPerRun Int 5 Jobs evaluated per run, 1 to 20.
triageEnabled Boolean true Rank titles against the resume before reading job pages.
scheduleMinutes Int? Interval of scheduled checks; null means none. At least 15 when set.
onboardingCompletedAt DateTime? When the setup wizard was finished, or the first run was created.
onboardingDismissedAt DateTime? When the setup checklist was hidden; cleared when restored.

Range rules (months 0 to 11, jobs 1 to 20, schedule at least 15) are enforced by the request schema, not the database. See Configuration Reference for how each setting affects a run.

JobSource

A careers page, ATS board or web search the user follows.

Field Type Default Purpose
id String uuid(7) Primary key.
userId String Owner.
url String The source URL. For SEARCH sources, the synthetic search:<query in lower case>.
kind SourceKind PAGE How listings are read. Detected from the URL on creation; can be switched.
adapterKey String? ATS board token or company slug for ATS kinds; null for PAGE.
query String? The search query, for SEARCH sources.
selector String? CSS selector for the part of a page that lists jobs, passed to Firecrawl as an include tag. No endpoint sets it.
maxPages Int 3 How many "next page" links a page source follows. No endpoint sets it.
useMap Boolean false Page sources: read job links from Firecrawl's map of the site first.
listingHash String? SHA-256 of the page source's whitespace-normalised listing Markdown (or of its mapped job URLs), used to tell whether the page changed. Cleared when the kind is switched.
suggestedKind SourceKind? An ATS board found embedded in a page source, offered as a switch.
suggestedAdapterKey String? The embedded board's token.
lastCheckedAt DateTime? When a run last read this source's listings.
createdAt DateTime now() Creation time; sources list newest first.

Constraint: @@unique([userId, url]), so a user cannot add the same URL (or the same search, ignoring case) twice. The API maps the violation (P2002) to 409.

Run

One analysis of a set of sources, manual or scheduled.

Field Type Default Purpose
id String uuid(7) Primary key; also the BullMQ job id.
userId String Owner.
trigger RunTrigger MANUAL (Analyze) or SCHEDULED (a schedule tick).
status RunStatus QUEUED Lifecycle state; see Run statuses.
contextKey String Hash of resume content, source ids, model, requirements and title filters; see contextKey.
sourceIds String[] The sources this run covers, de-duplicated. Not a foreign key.
found Int 0 Listings found across sources, after de-duplication.
eligible Int 0 Listings left after the requirement and title filters.
checked Int 0 Jobs the run has settled (any outcome).
failed Int 0 Jobs that errored, or whose match could not be saved. Decremented when a failed save is retried successfully.
stopReason String? Why the run stopped or failed, shown to the user.
cloudCredits Int 0 Firecrawl cloud credits the run used.
cloudCreditsAtStart Int? The cloud plan's remaining credits when the run started; null when there was no cloud plan to read.
createdAt DateTime now() Creation (queue) time.
finishedAt DateTime? When the run reached a final status.
results JobResult[] Back-relation.

Index: @@index([userId, createdAt]), serving the "recent runs" list and the newest-run lookups.

JobResult

The outcome for one job in one context. Rows are upserted, so a job evaluated again in the same context updates its row rather than adding one.

Field Type Default Purpose
id String uuid(7) Primary key.
runId String The run that last wrote this row. Foreign key to Run.id, ON DELETE CASCADE.
userId String Owner.
contextKey String The run's context key when the result was written.
jobUrl String The job page URL.
title String Job title.
company String Company name.
locations String[] Job locations, from the listing or the job page's structured data.
expMinYears Float? Minimum years of experience required, when known.
expMaxYears Float? Maximum years of experience, when known.
isMatch Boolean? The verdict; null when the job errored or was skipped.
reason String? The model's explanation, or why the job was ruled out or skipped.
error String? Error message when the job failed.
skipped SkipReason? Set when the job was settled without an evaluation.
scrapedVia ScrapeVia? How the job's text was read; null when the page was not read, or a PDF was downloaded directly.
matchSaved Boolean false Whether the corresponding Match was saved.
createdAt DateTime now() Creation time; results list oldest first.

Constraint: @@unique([userId, contextKey, jobUrl]). This is what makes reruns resumable (see contextKey).

Match

A job found to fit, kept across runs and tracked through applying.

Field Type Default Purpose
id String uuid(7) Primary key; also the pagination cursor.
userId String Owner.
jobUrl String Job page URL.
title String Job title.
company String Company.
locations String[] Locations.
expMinYears Float? Minimum experience.
expMaxYears Float? Maximum experience.
reason String Why it matched (latest evaluation).
status MatchStatus PENDING PENDING until the user marks it APPLIED.
matchedAt DateTime now() Last time a run matched it; matches list newest first by this.

Constraint: @@unique([userId, jobUrl]). Saving a match (save-match.ts) first looks for an existing match with the same URL, or the same canonical URL (matches saved before URLs were canonicalised may be stored under another spelling). An existing match only gets a new reason and matchedAt, so it keeps its APPLIED status.

PageCache

Per-user cache of scraped content, stored in Postgres so it survives restarts and needs no extra service.

Field Type Default Purpose
userId String Owner. Part of the primary key.
url String The page, source or resume URL. Part of the primary key.
kind CacheKind What is cached. Part of the primary key.
content Json Job page Markdown and structured posting data, extracted listings, or the resume text with its hash.
expiresAt DateTime? When the entry stops being fresh; null never expires.
createdAt DateTime now() Reset to the current time whenever the entry is rewritten.

Primary key: @@id([userId, url, kind]). See PageCache mechanics.

ScrapeRoute

What hybrid scraping mode remembers about sites that block the self-hosted Firecrawl. Shared by every user.

Field Type Default Purpose
domain String Primary key: the site's domain.
localBlocks Int 0 Consecutive blocks by the local Firecrawl; a local success resets it.
preferCloudUntil DateTime? Until when the site goes straight to the cloud.
updatedAt DateTime @updatedAt Last change.

See Scrape routes.

Authentication models

These four tables follow better-auth's schema and are mapped to lower-case table names with @@map. They are managed by better-auth through its Prisma adapter (apps/api/src/auth/auth.ts); application code does not write them. See Authentication.

User (user)

Field Type Purpose
id String Primary key.
name String Display name from sign-up.
email String Unique email address.
emailVerified Boolean Default false; verification is not used.
image String? Avatar URL (unused).
createdAt, updatedAt DateTime Timestamps (updatedAt is @updatedAt).
sessions, accounts relations Back-relations.

Session (session)

Field Type Purpose
id String Primary key.
expiresAt DateTime Session expiry.
token String Unique session token, carried by the session cookie.
createdAt, updatedAt DateTime Timestamps.
ipAddress, userAgent String? Client details recorded by better-auth.
userId String Foreign key to user.id, ON DELETE CASCADE. Indexed.

Account (account)

Field Type Purpose
id String Primary key.
accountId, providerId String The provider and the user's id at that provider. Email and password sign-up creates a credential account.
userId String Foreign key to user.id, ON DELETE CASCADE. Indexed.
accessToken, refreshToken, idToken String? OAuth tokens (unused: no social providers are configured).
accessTokenExpiresAt, refreshTokenExpiresAt DateTime? OAuth token expiry (unused).
scope String? OAuth scope (unused).
password String? Password hash for credential accounts.
createdAt, updatedAt DateTime Timestamps.

Verification (verification)

Field Type Purpose
id String Primary key.
identifier String What is being verified. Indexed.
value String The verification value.
expiresAt DateTime Expiry.
createdAt, updatedAt DateTime Timestamps.

Enums

Enum Values Used by Meaning
SourceKind GREENHOUSE, LEVER, ASHBY, SMARTRECRUITERS, RECRUITEE, SEARCH, PAGE JobSource.kind, JobSource.suggestedKind ATS kinds read the board's public API by adapterKey; SEARCH is a web search of ATS boards; PAGE is any other careers page.
RunStatus QUEUED, RUNNING, COMPLETED, STOPPED, FAILED Run.status See Run statuses.
RunTrigger MANUAL, SCHEDULED Run.trigger Who started the run.
SkipReason EXPIRED, TRIAGE JobResult.skipped EXPIRED: the job page's structured data says the posting has closed. TRIAGE: title triage ranked the job below the threshold.
ScrapeVia ATS, CACHE, LOCAL, CLOUD JobResult.scrapedVia ATS: with the listing or from the board's API. CACHE: the page cache. LOCAL: the self-hosted Firecrawl. CLOUD: the Firecrawl cloud, including a hybrid run's fallback.
MatchStatus PENDING, APPLIED Match.status Application tracking.
CacheKind RESUME, JOB_PAGE, SOURCE_LISTING PageCache.kind What the cache entry holds.

Constraints and indexes

Table Constraint or index Kind Why
Settings userId Primary key One settings row per user.
JobSource (userId, url) Unique No duplicate sources per user.
Run (userId, createdAt) Index Recent-runs list and newest-run queries.
JobResult (userId, contextKey, jobUrl) Unique One result per job per context; upsert target.
JobResult runId to Run.id Foreign key, cascade delete Results belong to a run.
Match (userId, jobUrl) Unique One match per job per user; upsert target.
PageCache (userId, url, kind) Primary key One entry per user, URL and kind.
ScrapeRoute domain Primary key One routing record per site.
user email Unique One account per email.
session token Unique Session lookup by cookie.
session, account userId Index, foreign key with cascade Per-user lookups; deleting a user removes sessions and accounts.
verification identifier Index Lookup by identifier.

Notable mechanics

Run statuses

stateDiagram-v2
  [*] --> QUEUED: createRun
  QUEUED --> RUNNING: worker claims (updateMany WHERE status = QUEUED)
  QUEUED --> FAILED: could not queue, or job lost (recovery)
  RUNNING --> COMPLETED: every selected job settled
  RUNNING --> STOPPED: no resume, or an error that stops the run
  RUNNING --> FAILED: unexpected error, interrupted worker, or failed job
  COMPLETED --> [*]
  STOPPED --> [*]
  FAILED --> [*]
Loading
Status Set by Meaning
QUEUED createRun (create-run.ts) Saved and waiting for the worker.
RUNNING The worker's claim in process-run.ts Being processed. Only a QUEUED run can be claimed, so a duplicate delivery does nothing.
COMPLETED The worker All selected jobs were settled (individual jobs may still have failed).
STOPPED The worker The run ended early on purpose: no resume, or an error that stops the run (for example a rejected OpenRouter key or Firecrawl out of credits). stopReason says why.
FAILED The worker, the API, or recovery (recover-run.ts) An unexpected error, a job that could not be queued, a job BullMQ lost or failed, or an interrupted worker. Results saved so far are kept.

QUEUED and RUNNING are the active statuses: a user can have at most one active run, which createRun enforces under an advisory lock. Every transition to a final status is a guarded updateMany (on status = RUNNING, or on status IN (QUEUED, RUNNING) for recovery), so a run that has already ended keeps its first reason.

contextKey

contextKey (matching/context-key.ts) is the SHA-256 of a canonical JSON array of:

  1. the SHA-256 of the resume content (so a new file at the same URL counts as a new resume),
  2. the sorted, de-duplicated source ids,
  3. the OpenRouter model,
  4. the requirements: experience years and months, target minimum and maximum years, sorted unique locations, and includeUnknown,
  5. the canonical title filters, appended only when either list is non-empty, so keys computed before title filters existed still match.

How it is used:

  • createRun stores a provisional key computed with the resume URL in place of the content hash, because the content is not known yet. The worker replaces it once it has resolved the resume.
  • Before evaluating, the worker loads the JobResult rows for this user and key, among the eligible job URLs, that have no error. Those jobs are already settled and are not evaluated again. With triage off, jobs that triage previously skipped are evaluated after all.
  • Results are upserted on (userId, contextKey, jobUrl), so a rerun after an interruption or with a larger maxJobsPerRun picks up where the last one stopped and never pays twice for the same job.
  • Changing any input (a new resume, different sources, another model, edited requirements or title filters) produces a new key and a fresh set of results.

Not part of the key: maxJobsPerRun, triageEnabled, the schedule and the Discord webhook.

Advisory locks

createRun runs in one interactive transaction that starts with:

SELECT pg_advisory_xact_lock(hashtext($userId))

This transaction-scoped lock serialises run creation per user: a second concurrent start (a double click, or a schedule tick racing a manual start) waits until the first transaction commits, then sees its QUEUED run and returns "already active". The lock is released automatically at commit or rollback. hashtext maps the user id to the lock's integer key, so in principle two users can share a key; that only serialises their starts and does not affect correctness.

PageCache mechanics

Implemented in matching/page-cache.ts and matching/resume-source.ts:

Kind Key URL Content Expiry
SOURCE_LISTING The source's url Extracted Listing array 15 minutes; 24 hours for SEARCH sources
JOB_PAGE The job URL Job page Markdown and parsed structured posting data Never (null)
RESUME The resume URL Resume text and its SHA-256 24 hours for a scraped page; null for a PDF, which is revalidated with a conditional GET on every run

Behaviour:

  • A missing, expired or undecodable row is a miss. An expired row is passed to the scrape, which may reuse it (for example, a page source whose page hash is unchanged keeps its listings).
  • Expiry is measured from when the scrape finished, because scrapes can wait on the rate limit.
  • Failed scrapes are never stored.
  • Writes upsert on the primary key and reset createdAt.
  • Switching a source's kind deletes its SOURCE_LISTING row (sources.service.ts).
  • DELETE /api/cache deletes all of the user's rows. Nothing else deletes expired rows; they are overwritten by the next fetch.

Credit estimates also consult the cache (isCached), since a cached page costs no credits. See Firecrawl Credits and Caching.

Cloud credits tracking

Two Run columns record Firecrawl cloud usage (process-run.ts):

  • cloudCreditsAtStart is written once, before any scraping, with the plan's remaining credits when the run had a cloud plan to read (cloud and hybrid modes). It stays null otherwise.
  • cloudCredits is updated with the run's running total each time a job result is recorded, and again when the run finishes.

For scheduled runs, the worker sums cloudCredits over the user's other SCHEDULED runs created since midnight UTC, and uses that to keep each day's scheduled runs within an even share of the remaining credits. JobResult.scrapedVia records which path (ATS, cache, local or cloud) read each job's text. The budget rules are on Firecrawl Credits and Caching.

Scrape routes

In hybrid mode the worker scrapes with the self-hosted Firecrawl first and falls back to the cloud for pages that block it. matching/scrape-routes.ts stores per-domain memory in ScrapeRoute:

  • Each local block upserts the row and increments localBlocks.
  • After 2 consecutive blocks, preferCloudUntil is set to 7 days ahead, and pages on that domain go straight to the cloud until then.
  • A local success resets localBlocks to 0 and clears preferCloudUntil, writing only when there is a count to reset.
  • The count is kept after the cloud period ends, so one more local block sends the site back to the cloud.
  • All access is best-effort: a database error is logged and the page is routed as if the site had no history.

Onboarding state

Only two moments are stored, on Settings: onboardingCompletedAt and onboardingDismissedAt. The four setup steps (resume, preferences, source, first run) are derived from the user's data on each request (onboarding/onboarding.ts). onboardingCompletedAt is set by POST /api/onboarding/complete or by the first run's createRun, whichever is first; both are guarded on the column being null, so the first time is kept. onboardingDismissedAt is set by dismiss and cleared by restore. See Backend API.

Migration history

One row per folder in prisma/migrations, in order.

Migration Changes
20261010084657_init Creates enums RunStatus, RunTrigger, MatchStatus, CacheKind. Creates Settings (resume, webhook, experience, target years, locations, includeUnknown, maxJobsPerRun, scheduleMinutes), JobSource (id, user, url, lastCheckedAt, createdAt), Run (with counters and stopReason), JobResult (without listing details), Match and PageCache. Adds unique indexes JobSource(userId, url), JobResult(userId, contextKey, jobUrl), Match(userId, jobUrl), index Run(userId, createdAt), and the JobResult.runId foreign key with cascade delete.
20261010120000_add_auth Creates better-auth's user, session, account and verification tables, unique user.email and session.token, indexes on session.userId, account.userId and verification.identifier, and cascading foreign keys from session and account to user.
20261011041913_job_result_listing Adds locations, expMinYears and expMaxYears to JobResult.
20261012080421_source_kinds Creates enum SourceKind. Adds kind (default PAGE), adapterKey, query, selector, maxPages (default 3) and listingHash to JobSource. Backfills existing PAGE sources on known ATS hosts (Greenhouse board, embed and API URLs; Lever; Ashby; SmartRecruiters; Recruitee) to their ATS kind and board token with regular expressions that mirror detectSource.
20261012083317_source_suggestions Adds suggestedKind and suggestedAdapterKey to JobSource.
20261012090331_source_map_option Adds useMap (default false) to JobSource.
20261012092937_job_result_skipped Creates enum SkipReason (EXPIRED, TRIAGE) and adds JobResult.skipped.
20261012093603_title_filters Adds titleInclude and titleExclude (default empty arrays) to Settings.
20261012094208_triage_setting Adds triageEnabled (default true) to Settings.
20261012102630_scrape_routes Creates the ScrapeRoute table.
20261012131814_run_cloud_credits Creates enum ScrapeVia, adds JobResult.scrapedVia, and adds cloudCredits (default 0) and cloudCreditsAtStart to Run.
20261013075002_onboarding_state Adds onboardingCompletedAt and onboardingDismissedAt to Settings. Backfills: every user who has a run (and still exists in user) gets onboardingCompletedAt set to the migration time, inserting a default Settings row if needed and leaving an existing completion time alone. apps/api/test/onboarding-backfill.test.ts runs this statement.

migration_lock.toml pins the provider to postgresql.

Changing the schema

  1. Edit prisma/schema.prisma.

  2. With the development database running (docker compose up -d, or bun run dev), run:

    bun run db:migrate

    This runs prisma migrate dev, which diffs the schema against the database, asks for a migration name, writes prisma/migrations/<timestamp>_<name>/migration.sql and applies it. The postdb:migrate script then runs prisma generate to refresh the client in apps/api/src/generated/prisma.

  3. Review the generated SQL. Add hand-written statements when existing rows need a backfill, as source_kinds and onboarding_state do, and test the backfill when it is not trivial.

  4. Commit the migration folder together with the schema change.

Deployed stacks apply migrations with prisma migrate deploy: the migrate service in deploy/compose.yml runs it before the API and worker start, and the Playwright setup runs it against its own _e2e database. bun install also runs prisma generate through the root postinstall script. See Development Setup and Deployment.

Related pages

Clone this wiki locally