AI Matching

AI Matching

Job Matcher uses an LLM, through OpenRouter, for three jobs: reading job listings from careers pages whose links carry no titles (listing extraction), ranking job titles against the resume before any job ad is read (title triage), and judging each job ad against the resume and the user's requirements (evaluation). All three share one client, jsonCompletion, that sends a single chat completion with a strict JSON schema, routes it to OpenAI's flex tier under a per-token price cap, and backs off when flex capacity runs out. Before each run, a read-only preflight checks the API key and the model's endpoints so that a bad configuration fails before any scrape or completion is paid for. The evaluation returns a yes or no verdict with a short reason; there is no numeric match score.

The code lives in apps/api/src/matching/. The run that calls it is described in Run-Pipeline.

The three LLM uses

Use	Source	When it runs	Schema name	max_completion_tokens	Prompt cache key
Listing extraction	listing-extractor.ts	A page source whose links show no titled jobs	job_listings	16,000	none
Title triage	triage.ts	Once per run (in batches of 150), when triage is on in Run settings	title_triage	12,000	none
Evaluation	evaluate.ts	Once per selected job whose page passed the requirement checks	job_match	2,000	sha256 of the resume text

All three use the same model (OPENROUTER_MODEL) and the same routing (see The OpenRouter client). Each is an Effect service (ListingExtractor, TitleTriage, MatchEvaluator) with a live layer built in providers/live.ts and a fake one in providers/fake.ts.

Listing extraction

ATS boards (Greenhouse, Lever, Ashby, SmartRecruiters, Recruitee) are read through their public APIs and never reach the model. For a web page source, sources/page.ts first reads job links from the page markdown with rules. The model is called only when those links yield no titled jobs.

• Input. The markdown of every scraped page of the source (following "next" pages up to maxPages), joined and cut to the first 60,000 characters (maxExtractionChars, about 15k tokens). A log line records when the cut happens.
• System prompt. Extract only individual listings explicitly on the page; copy titles and companies; return real job-detail URLs, resolving relative links against the page URL; exclude navigation, search pages and company profiles; record locations and minimum/maximum years only when explicitly stated (for "3+ years", minimum 3 and maximum null); never invent jobs or infer location or experience from titles.
• User prompt. Page URL: <url> followed by the page markdown.
• Output. { jobs: [{ title, url, company, locations, experience_min_years, experience_max_years }] }. All six fields are required; years are number | null and must be 0 or more when decoded.
• Guard against invented links. The extractor keeps only listings whose normalised URL appears among the links on the scraped pages, so a made-up or mangled URL is dropped. An empty company falls back to the page's site name or host.
• Caching. The extracted listings are stored in the page cache as a SOURCE_LISTING entry (15 minutes for page sources; see Firecrawl-Credits-and-Caching).
• Failure. Any extraction error becomes a ScrapeFailed with the message "Could not read the listings on : …". Listings are read before any job is evaluated, so this stops the run (only a failing search source is skipped).

Title triage

Triage ranks the remaining titles against the resume in a cheap request so that only likely fits are scraped and evaluated. It is on by default (Settings.triageEnabled, toggled in Run settings).

• Input. A resume summary (whitespace collapsed, first 3,000 characters) and a JSON array of { id, title, company, location }, where id is the listing's index in the run and location joins the listing's locations with "; ".
• System prompt. Score each job from its title, company and location alone: 3 strong fit, 2 plausible, 1 unlikely, 0 clearly unrelated. The prompt tells the model to judge field and role first, because experience is checked later against the job ad: a seniority word alone is no reason to score below 2 (a Senior title often asks for 4 to 6 years). It scores low for seniority only on a clear gap, such as Staff, Principal, Director, Head of or manager roles for a candidate with under 8 years, or intern roles for an experienced one. When unsure, score 2. A short reason is required for scores below 2, and every job id must get a score.
• Output. { scores: [{ id, score, reason }] }, with score an integer enum 0–3 and reason a string or null.

Batching and ordering, from triageListings:

Rule	Value
Jobs per request (triageBatchSize)	150; a typical run fits in one request
Threshold (triageThreshold)	2; jobs scoring below it are not read or evaluated
Job the model left out	Kept with the threshold score (2), so it is still checked
Unknown, out-of-batch or repeated ids	Ignored (the first score for an id wins)
Order of kept jobs	Highest score first; listing order within a score
No pending listings	No request is made

Rejected jobs are saved as JobResult rows with skipped = TRIAGE, isMatch = null and the reason "Skipped by title triage: ". They show on the run page as Skipped and are not triaged again in the same context. With triage turned off, those jobs become eligible again and are evaluated.

A triage failure that would stop the run (see Errors and their effect) stops it. Any other failure (an invalid output, a 5xx) is logged as "Title triage failed: …" and the run continues without triage, because triage only saves credits.

Run size

After triage, the run takes the first maxJobsPerRun jobs of the ranked list (default 5, allowed 1–20, set in Run settings). A scheduled run may then drop jobs that would not fit the day's Firecrawl allowance; jobs read for free (ATS API or cache) always fit. See Run-Pipeline and Firecrawl-Credits-and-Caching.

Evaluation

For each selected job, the worker reads the job page, then checks it before calling the model (runs/process-run.ts):

1. If the page's schema.org JobPosting says the posting closed (validThrough is in the past), the job is settled as Expired with no model call.
2. The listing is enriched with the page's structured data. If the enriched job fails the requirements, it is recorded as a non-match without a model call, with one of two fixed reasons: "Doesn't meet your requirements (from the job page's structured data)" or, when details are unknown and the user excluded unknown jobs, a reason saying so.
3. Otherwise the model evaluates it.

The job text sent to the model is the JobPosting description (formatted with the listing details) when the page has one, and otherwise the page markdown cut to 24,000 characters (maxJobPageChars in job-posting.ts).

System prompt. The model acts as an experienced job interviewer, is told to be critical, and accepts only candidates who meet at least 75% of the requirements.

User prompt, in this order:

1. Resume: and the full resume text.
2. When requirements are set: User's job search requirements:, a JSON object, and a guidance paragraph.
3. Job Posting: and the job text.
4. A closing instruction to decide whether the candidate is a good fit and explain why briefly.

The job text goes last so that every job in a run shares the same cached prompt prefix (resume plus requirements). The prompt cache key is the resume's sha256 hash.

The requirements JSON uses snake case keys, as the guidance refers to include_unknown:

Key	Settings field	Meaning
experience_years	experienceYears	The candidate's whole years of experience
experience_months	experienceMonths	Extra months, 0–11
job_min_years	targetMinYears	Lower bound of the experience range the user targets; null means no bound
job_max_years	targetMaxYears	Upper bound; null means no bound
locations	locations	Acceptable cities or aliases
include_unknown	includeUnknown	Whether to accept jobs whose location or experience cannot be verified

The guidance paragraph tells the model to use the supplied experience, to treat listed locations as acceptable city names or aliases, to require the job's experience range to overlap the requested range and the candidate to meet the job's stated minimum, to reject jobs with a conflicting location or experience requirement even when skills match, to say what could not be verified instead of assuming a match, and to reject unverifiable jobs when include_unknown is false.

The evaluation output

The model must answer with this schema (matchJsonSchema in evaluate.ts, sent with strict: true and additionalProperties: false):

Field	Type	Meaning
is_match	boolean	Whether the candidate is a good fit for the job
reason	string	A brief explanation of why the candidate is or is not a good fit

match-output.ts decodes the reply into { isMatch, reason }. Decoding fails with MatchOutputInvalid when:

• the model returned a refusal (even if content is also present): "Model refused: …";
• the content is empty: "Model returned empty match content.";
• the content is not JSON, misses a field, has a wrong type (for example "false" as a string), or has an extra property (onExcessProperty: "error").

Scoring

Evaluation produces no number. The only numeric scale in AI matching is the 0–3 triage score, which decides whether a job's ad is read at all and in which order; it is not stored or shown. The "How matching works" panel (scoring-help.tsx) tells users this directly: each job gets a yes or no verdict with a reason. It explains the three stages (settings filters, optional title ranking, the full job ad check) and the 75% rule, and points to the "Include listings with unknown experience or location" setting.

How results are stored and shown

Every settled job is a JobResult row keyed by (userId, contextKey, jobUrl). A match is also upserted into Match (see Data-Model) and, when a Discord webhook is set, posted as an alert. A failed save keeps the verdict without an error, so the job is not analysed again and the match can be saved later from the run page.

The web app maps each row to a verdict badge (verdictOf in run-card.tsx, badges in verdict-badge.tsx):

Verdict	Badge label	Condition (checked in this order)	Hint shown to the user
expired	Expired	skipped = EXPIRED	The posting had closed
skipped	Skipped	skipped = TRIAGE	Its title looked unlikely to fit, so the ad wasn't read
failed	Couldn't check	error set, or isMatch null	The ad couldn't be read or judged; it's tried again on the next run
match	Match	isMatch = true	The job ad fits your resume and preferences
no-match	No match	isMatch = false	The job ad doesn't fit closely enough; the reason says why
checking	Checking	Not produced by verdictOf; used only by the illustrative example run (example-run.tsx), with a pulse animation	This job is being checked now

The hint is a tooltip on the badge and screen-reader text beside it. Under each job, the run page shows the error when there is one, otherwise the reason (with the "Skipped by title triage" prefix removed for triaged jobs).

The OpenRouter client

completion.ts exports jsonCompletion, built on @effect/ai-openrouter's OpenRouterClient.

Request shape

Field	Value
model	OPENROUTER_MODEL
stream	false
messages	One system message (the use's prompt) and one user message
response_format	{ type: "json_schema", json_schema: { name, strict: true, schema } }
service_tier	OPENROUTER_SERVICE_TIER (default flex)
provider.only	OPENROUTER_PROVIDERS, split on commas (default ["openai"])
provider.allow_fallbacks	false
provider.require_parameters	true, so only endpoints that support every sent parameter (including structured output) are used
provider.max_price	{ prompt, completion } from the price caps, as strings, in USD per million tokens
reasoning.effort	OPENROUTER_REASONING_EFFORT (default low)
max_completion_tokens	Per use: 16,000, 12,000 or 2,000
prompt_cache_key	Evaluation only: the resume hash

The fetch timeout is 15 minutes (completionRequestInit), because Bun's fetch drops a connection idle for 5 minutes and flex completions can take longer.

Response handling

• Usage is logged on every reply: prompt tokens, cached prompt tokens and completion tokens.
• Tier mismatch. When the configured tier is flex and the reply's service_tier is set to something else, the worker logs a warning such as "OpenRouter served the evaluation on the default tier, not flex." The purpose field (evaluation, title triage, listing extraction) names the request. The reply is still used.
• A reply with no choices fails with "OpenRouter returned no completion choices."
• A choice whose finish_reason is not stop (for example length, content_filter, tool_calls) fails with "OpenRouter completion did not finish: ".
• An error body returned with HTTP 200 becomes OpenRouterRequestFailed with no status, which fails only the current job.

Backoff on 429 and 503

Flex capacity comes and goes. A 429, or a 503 that is not a price-cap rejection, is retried on an exponential schedule starting at one minute, up to 5 times:

Attempt	Wait before it	Time since the first attempt
1	none	0 min
2	1 min	1 min
3	2 min	3 min
4	4 min	7 min
5	8 min	15 min
6	16 min	31 min

The error body is not inspected, so a key-level rate limit and missing flex capacity are treated alike. If the last attempt still fails, the error becomes OpenRouterRequestFailed with "OpenRouter flex capacity unavailable after 5 retries: …", which fails only that request, not the run. Other statuses are not retried.

Price-cap rejections

OpenRouter answers a request that no endpoint can serve with a 404 or a 503 whose message mentions price, "no endpoints", "no allowed providers" or "routing requirements". The client maps these to OpenRouterUnavailable with the message "No flex endpoint under the price cap (prompt $X, completion $Y per million tokens): ". These are not retried, since waiting does not change the cap, and OpenRouterUnavailable stops the run.

Errors and their effect

Status mapping is in errors.ts:

Condition	Error	Effect
HTTP 401	OpenRouterAuthError	Stops the run
HTTP 402, 403, 404, or 429 outside the retry path (for example in preflight)	OpenRouterUnavailable	Stops the run
Price-cap 404 or 503	OpenRouterUnavailable	Stops the run
Management or provisioning key	ManagementKeyError	Stops the run (preflight)
Model without a suitable endpoint	ModelUnsupported	Stops the run (preflight)
429 or 503 after 5 retries	OpenRouterRequestFailed	Fails only this job
Other statuses (400, 5xx), network errors, error body on 200	OpenRouterRequestFailed	Fails only this job
Refusal, unfinished completion, invalid JSON	MatchOutputInvalid	Fails only this job

A job that fails is recorded with its error and shows as Couldn't check. Only results without an error count as analysed, so the job is tried again on the next run in the same context. For triage and listing extraction, the effect differs as described in their sections.

Preflight

preflight.ts runs at the start of every run, before the Firecrawl health check, the credit check and any scrape. It makes only GET requests:

1. GET /key. Fails on a non-2xx status (mapped as above, so a rejected key is an auth error and the model is not checked). If the key is a management or provisioning key, it fails with: "The OpenRouter API key is a management or provisioning key, which cannot run AI matching. Create a regular API key at https://openrouter.ai/settings/keys."
2. GET /models/<base model>/endpoints. The base model drops a variant suffix such as :floor or :free, since variants share the base model's endpoints. An unknown model (404) is OpenRouterUnavailable.
3. Endpoint checks, in order. The first that fails becomes a ModelUnsupported error, and its message becomes the run's stop reason:

Check	Failure message (abridged)
Some endpoint supports both response_format and structured_outputs	The model has no provider supporting JSON Schema structured output. Choose a compatible model.
One of those is from an allowed provider (matched case-insensitively on provider_name or the tag's prefix)	No endpoint from the allowed providers supporting JSON Schema structured output. Change OPENROUTER_PROVIDERS or the model.
When the tier is flex and the endpoint data marks any endpoint as flex (tag ending /flex), one allowed endpoint is flex	No flex endpoint from the allowed providers. Change OPENROUTER_PROVIDERS, OPENROUTER_SERVICE_TIER or the model.
One remaining endpoint is within both price caps (per-token prices times one million, with a 1e-9 tolerance; an endpoint without pricing passes)	No (flex) endpoint from the allowed providers under the price cap. Raise OPENROUTER_MAX_PRICE_PROMPT and OPENROUTER_MAX_PRICE_COMPLETION or choose another model.

A response that does not match the expected shape fails with "OpenRouter returned an unexpected response from ." A network failure is OpenRouterRequestFailed. Every preflight failure stops the run, because it happens inside the run's preparation step.

Requirements

requirements.ts defines the user's requirements (built from Settings by requirementsOf in runs/run-settings.ts) and two deterministic checks that run before the model:

• acceptsJob. With locations set, a job that states locations must mention one of the cities as a whole word (Unicode-aware, case-insensitive); a job with no locations passes only when includeUnknown is true. A job whose stated minimum exceeds the candidate's experience (years plus months/12) fails. With a target range set, a job with no stated experience passes only when includeUnknown is true, and a job whose range does not overlap the target fails.
• rulesOutJob. The same check with includeUnknown forced on, so missing details never rule a job out. It filters listings before triage, because a listing often omits what its job page states.

makeRequirements validates input: experience years a whole number of 0 or more, months 0–11, job years 0 or more, and the minimum not above the maximum.

The defaults are 0 years, 0 months, no target range, no locations and includeUnknown: true.

Context key and resumability

context-key.ts hashes (sha256) a canonical JSON array of the resume content hash, the sorted unique source ids, the model, the requirements (with sorted locations) and, only when used, the title filters. Each JobResult is stored under this key. A rerun with the same key skips every job that already has a result without an error, including triage skips while triage is on, so it never pays twice for the same job. Changing the resume content, sources, model, requirements or title filters produces a new key and a fresh evaluation. The run is created before the resume is read, so the worker replaces the run's key once it knows the resume hash. See Firecrawl-Credits-and-Caching for the page cache that sits alongside it.

One job, end to end

sequenceDiagram
    autonumber
    participant W as Run worker
    participant OR as OpenRouter
    participant FC as Firecrawl or ATS API
    participant DB as PostgreSQL
    participant D as Discord

    W->>OR: GET /key and /models/{model}/endpoints (preflight)
    OR-->>W: Regular key, flex endpoint under the price cap
    W->>FC: Read source listings
    FC-->>W: Listing markdown or ATS JSON
    opt Page links carry no titles
        W->>OR: Listing extraction (job_listings schema)
        OR-->>W: jobs[]
    end
    W->>W: Dedupe, rulesOutJob, title filters
    W->>DB: Load results already saved for this context key
    DB-->>W: Settled job URLs
    opt Triage enabled
        W->>OR: Title triage (title_triage schema, up to 150 jobs per request)
        OR-->>W: scores[] from 0 to 3
        W->>DB: Save jobs scoring below 2 as skipped (TRIAGE)
    end
    W->>W: Take the first maxJobsPerRun ranked jobs
    W->>FC: Read the job page (cache, ATS, local or cloud)
    FC-->>W: Markdown and JobPosting data
    alt Posting expired or page fails requirements
        W->>DB: Save JobResult (Expired or No match), no model call
    else
        W->>OR: Evaluation (job_match schema, flex, prompt_cache_key = resume hash)
        OR-->>W: is_match and reason
        W->>DB: Save JobResult under the context key
        opt is_match is true
            W->>DB: Upsert Match
            W->>D: Post match alert (failure is only logged)
        end
    end

Loading

Fake provider

With PROVIDERS=fake, the worker swaps the three LLM services for fixtures (providers/fake.ts). Demo mode and the end-to-end tests use them; no OpenRouter key is needed.

Service	Fake behaviour
Preflight	Always passes
Listing extraction	Returns the listings in apps/api/test/fixtures/listings.json; fails with MatchOutputInvalid if the file is missing or does not decode
Title triage	Scores every job 3 with no reason, so every job is kept in listing order
Evaluation	Reads a Verdict: line in the job page: match gives a match ("The resume covers the listed stack."); auth fails with a scripted 401 OpenRouterAuthError (stops the run); invalid fails with MatchOutputInvalid (fails the job); anything else gives a non-match ("The listing needs skills not on the resume.")

The fixture job pages in apps/api/test/fixtures/jobs/ script their outcomes this way. See Testing.

Cost model

The code does not compute or store a cost. It controls spend in these ways:

• Price caps. provider.max_price is sent on every request, and preflight refuses to start a run unless an allowed endpoint is within the caps. The defaults are $0.05 per million prompt tokens and $0.25 per million completion tokens.
• Flex tier. Requests ask for OpenAI's flex service tier by default, and a reply served on another tier is logged.
• Bounded output. max_completion_tokens caps each reply (2,000 for an evaluation), and reasoning effort defaults to low.
• Bounded input. Job text is at most 24,000 characters (or the structured description), the triage resume summary at most 3,000 characters, and extraction input at most 60,000 characters.
• Prompt caching. The evaluation prompt puts the resume and requirements first and the job last, with the resume hash as prompt_cache_key, so jobs in a run share a cached prefix. Cached token counts are logged.
• Fewer calls. Settings filters and rulesOutJob drop listings for free; triage scores up to 150 titles in one request; expired and structurally unsuitable jobs skip the model; the run size caps evaluations at 1–20 per run; and the context key prevents re-evaluating settled jobs.

As a ceiling at the default caps: an evaluation billed at 10,000 prompt tokens and the full 2,000 completion tokens costs at most 10,000 × $0.05/M + 2,000 × $0.25/M = $0.0005 + $0.0005 = $0.001. The actual figures for a run are in the worker's usage log lines.

Environment variables

OPENROUTER_MODEL is read in config.ts, the routing variables in routing.ts, and the key in providers/live.ts. None is needed with PROVIDERS=fake.

Variable	Default	Allowed values	Meaning
OPENROUTER_API_KEY	none (required with PROVIDERS=live)	Non-empty string	A regular OpenRouter API key, not a management or provisioning key
OPENROUTER_MODEL	openai/gpt-6-luna:floor	Any non-empty OpenRouter model id	The model for all three uses; also part of the context key
OPENROUTER_SERVICE_TIER	flex	auto, default, flex, priority, scale	The service_tier sent on each request
OPENROUTER_PROVIDERS	openai	Comma-separated provider names	provider.only; fallbacks to other providers are disabled
OPENROUTER_MAX_PRICE_PROMPT	0.05	Finite number	Price cap, USD per million prompt tokens
OPENROUTER_MAX_PRICE_COMPLETION	0.25	Finite number	Price cap, USD per million completion tokens
OPENROUTER_REASONING_EFFORT	low	max, xhigh, high, medium, low, minimal, none	reasoning.effort

For the routing variables, an empty value (as left in a copied .env) keeps the default. An invalid value makes the live provider layer fail to build when the worker starts. See Configuration-Reference for every other variable.

Related pages

• Run-Pipeline: the whole run, of which AI matching is one part
• Job-Sources-and-Scraping: how listings and job pages are read
• Firecrawl-Credits-and-Caching: credits, the page cache and resumable runs
• Data-Model: JobResult, Match and Settings
• Troubleshooting: preflight and price-cap failures