Job OS: Evidence-Backed Job Discovery

Job matching that shows its work — every claim of fit is traced back to a quoted line in your resume and a quoted line in the posting.

▶ Live demo · Architecture · Status & limitations

---

Demo

A real resume uploaded, parsed into a scored Evidence Bank, matched against live Greenhouse postings, and opened as a requirement-by-requirement Role Brief:

---

The problem

Job seekers drown in postings but can't answer the two questions that actually matter: "Which of these are worth my time?" and "How do I prove I'm qualified?"

Keyword search answers neither. A similarity score answers neither — a number with no argument behind it is just a vibe. Job OS is built around the opposite premise: a match is only worth showing if it can be defended line by line.

Four steps:

1. Parse — your resume becomes a structured Evidence Bank (experience, projects, skills, education), each item scored for confidence and embedded for semantic search.
2. Monitor — job boards are polled on a schedule; an LLM extracts each posting's requirements verbatim.
3. Match — every requirement is matched to your evidence through a two-stage retrieval-then-validation pipeline.
4. Rank — a Fresh Match Queue ordered by 0.7 × fit + 0.3 × freshness, showing exactly what you can prove and where your gaps are.

Every mapping stores the quoted source text from both sides. No black-box scores.

---

Architecture

flowchart TB
    subgraph UI["Interface — Next.js App Router"]
        EB["Evidence Bank"]
        MQ["Fresh Match Queue"]
        RB["Role Briefs"]
    end

    subgraph API["API layer"]
        AE["/api/evidence"]
        AJ["/api/jobs"]
        AM["/api/matching"]
        AC["/api/cron/*"]
    end

    subgraph WORK["Execution — serverless, no worker process"]
        RP["Resume parser<br/>(gpt-4o-mini)"]
        JP["Job poller<br/>(Greenhouse)"]
        RE["Requirement extractor<br/>(gpt-4o-mini)"]
        MP["Matching pipeline<br/>(vector → LLM)"]
        ND["Notification dispatcher"]
    end

    subgraph DB["PostgreSQL 16 + pgvector"]
        EI["evidence_item<br/>(embedding, HNSW)"]
        RQ["requirement<br/>(embedding, HNSW)"]
        EM["evidence_mapping<br/>(provenance + model versions)"]
        AU["audit tables"]
    end

    UI --> API
    API --> WORK
    RP --> EI
    JP --> RQ
    RE --> RQ
    EI -->|"cosine top-5"| MP
    RQ -->|"cosine top-5"| MP
    MP --> EM
    EM --> AU
    EM --> MQ
    EM --> RB

Loading

The matching pipeline

Pure vector similarity produces confident nonsense. Pure LLM comparison doesn't scale — every requirement × every evidence item is a quadratic bill. So the cheap stage does the fan-out and the expensive stage only ever sees a shortlist:

sequenceDiagram
    participant R as Requirement
    participant PG as pgvector HNSW index
    participant LLM as gpt-4o-mini
    participant DB as evidence_mapping

    R->>PG: cosine search over evidence embeddings
    PG-->>R: top-5 candidates, under 10ms, no API cost
    Note over R,LLM: all candidates sorted by similarity<br/>hard cap — top 25 pairs per run reach the LLM
    R->>LLM: requirement + candidate, structured output
    LLM-->>DB: match / weak_match / no_match<br/>+ high / medium / low confidence<br/>+ quoted excerpts from both sides
    Note over DB: cached forever — re-runs never re-pay<br/>for a pair that has already been judged

Loading

---

Engineering notes

The parts I'd actually want to talk through in an interview.

Cost engineering — bounded worst case, unbounded search

The product promise is "search everything" — all companies, all functions. The constraint is that an LLM call per requirement-evidence pair is a quadratic bill attached to a student's API key. Both are satisfiable at once:

Guardrail	Mechanism
Top-K evaluation cap	Candidate pairs are ranked by vector similarity first; only the top 25 unvalidated pairs per run reach the LLM. The rest wait for the next run.
Extraction cap per poll	New jobs land at parseStatus='pending'; each poll drains at most 25 extractions, freshest first — a self-healing backlog, no state machine.
Validation caching	Every judged requirement↔evidence pair is persisted. Re-runs skip them entirely — zero repeat LLM calls.
Right-sized models	gpt-4o-mini for all high-volume extraction and validation; no measurable quality loss on structured tasks at a fraction of GPT-4-class cost.
Token discipline	Condensed system prompts, hard input caps (15k chars resume, 16k chars job description), batched embedding calls.

Worst case is ~50 mini calls — roughly a cent — per sync cycle, no matter how wide the search criteria are. Both caps are env-tunable (MAX_EVALUATIONS_PER_RUN, MAX_EXTRACTIONS_PER_POLL).

Running long pipelines on serverless

There is no background worker process and no queue daemon — it deploys to Vercel, so it has to survive inside a request lifetime. The heavy phases run in after(), past the response, and each one is wrapped in an explicit wall-clock budget:

• A phase returns how much work is left rather than being killed at maxDuration with half its work done and nothing to say about it.
• When a slice ends with work remaining, it hands off to a fresh invocation — which gets a fresh budget instead of inheriting the leftovers of this one. Chain depth is bounded (SWEEP_MAX_CHAIN_DEPTH), so a stuck backlog degrades into "picked up by the next cron" rather than an infinite loop.
• Extraction is prioritised by real users' criteria and spread across companies, so one company with 500 postings can't starve everyone else's queue.

Trust and hallucination control

• Structured outputs — Zod schemas drive both runtime validation and the OpenAI response_format, so a prompt change can't silently break the data contract.
• Conservative prompting — verbatim extraction only, no inferred requirements; "when in doubt, weak_match".
• Provenance at write time — source_requirement_text and source_evidence_excerpt are stored on every mapping. The UI never shows a claim it can't quote.
• Version tracking — embedding model, LLM model, and prompt version are recorded per mapping, so results stay reproducible and A/B-able across prompt revisions.
• Human-in-the-loop — uncertain matches are flagged needsReview; manual overrides are never overwritten by re-runs.
• Full audit trail — parser outputs and user corrections are logged, which is deliberately the shape of a labeled dataset for a future eval harness.

---

Features

• Evidence Bank — resume parsing (PDF/DOCX → structured items with confidence scores), manual entry, automatic embedding of everything.
• Discover mode — leave criteria empty to search everything, or target up to 15 specific companies; filter by full-time / part-time / internship / contract.
• Fresh Match Queue — ranked by fit × freshness with human-readable reasons and status tracking (New / Save / Apply / Applied / Ignore).
• Role Briefs — requirement-by-requirement breakdown with mapped evidence, gap analysis, and a print-optimised proof summary.
• Alerts — email digests when new high-fit roles appear, rate-limited.

---

Tech stack

Layer	Technology
Frontend	Next.js 16 (App Router, RSC, Turbopack), React 19, TypeScript, Tailwind CSS v4, TanStack Query, nuqs
Backend	Next.js API routes, Drizzle ORM, Better Auth, Resend
Data	PostgreSQL 16 + pgvector — HNSW indexes, 1536-dim embeddings, expression indexes for case-insensitive filtering
AI	OpenAI gpt-4o-mini (structured outputs for parsing, extraction, validation), text-embedding-3-small
Infra	Docker Compose (local), Vercel (deploy + cron), GitHub Actions CI, Husky + lint-staged, Vitest

---

Screenshots

Evidence Bank	Fresh Match Queue	Role Brief

---

Getting started

Prerequisites: Node.js 18+, Docker Desktop, an OpenAI API key. A Resend key is optional — email features degrade gracefully without it.

git clone https://github.com/mekyle-s/Job-OS.git
cd Job-OS
npm install

cp .env.example .env.local
# Set DATABASE_URL (the default matches the Docker container),
# OPENAI_API_KEY, and BETTER_AUTH_SECRET / CRON_SECRET
# (generate with: openssl rand -base64 32)

npm run db:up        # PostgreSQL 16 + pgvector
npm run db:migrate   # apply migrations (incl. HNSW indexes)
npm run dev

Then open http://localhost:3000, sign up, and upload a resume.

npm run db:studio    # Drizzle Studio (DB GUI)
npm run db:down      # stop the container
npm test             # Vitest
npm run lint

Keep spend down while experimenting:

MAX_EVALUATIONS_PER_RUN=6 MAX_EXTRACTIONS_PER_POLL=2 npm run dev

---

Project structure

src/
├── app/
│   ├── (auth)/                 # sign-in / sign-up / password reset
│   ├── dashboard/              # evidence, jobs, queue, role briefs
│   └── api/                    # REST endpoints + cron routes
├── lib/
│   ├── db/                     # Drizzle schema + query layer
│   ├── jobs/
│   │   ├── sources/            # board adapters (Greenhouse)
│   │   ├── parsers/            # LLM requirement extraction
│   │   ├── workers/            # poller, parsers, dispatcher
│   │   ├── extraction-queue.ts # prioritisation + fair spread
│   │   └── sweep.ts            # budgeted phase slices + hand-off
│   ├── matching/
│   │   ├── embedder.ts         # batched embedding generation
│   │   ├── similarity.ts       # pgvector cosine search
│   │   ├── mapper.ts           # LLM validation + decision bands
│   │   ├── ranker.ts           # fit × freshness scoring
│   │   └── pipeline.ts         # orchestration + cost guardrails
│   ├── parsers/                # PDF/DOCX extraction
│   └── schemas/                # Zod contracts
migrations/                     # Drizzle migrations (incl. HNSW)
scripts/e2e/                    # standalone end-to-end checks

---

Status & limitations

Actively developed portfolio project. What's real and what isn't, plainly:

Working end to end: resume upload → LLM parsing → Evidence Bank → scheduled Greenhouse polling → verbatim requirement extraction → pgvector retrieval → LLM validation with stored provenance → ranked queue → role briefs with gap analysis → manual overrides and audit trail → email alerts. The demo above is a real run against live postings, not a mockup.

Known limitations:

• One job source. Only the Greenhouse adapter is implemented. The schema and adapter interface anticipate Lever, but that code isn't written.
• Matching quality is not formally evaluated. There's no labeled benchmark and no offline eval harness yet, so I can't quote precision/recall. The audit trail exists precisely to build that, and until it does, treat quality claims as unproven.
• Scheduled polling is daily, not continuous (two Vercel cron jobs). Saving criteria triggers an immediate poll, so you don't have to wait for cron to see results.
• Alerts need a Resend API key. Without one, the app runs fine and simply doesn't send.
• The live demo is a personal deployment with cost caps applied. It's for showing the flow, not for production job hunting.

---

Roadmap

• Additional sources (Lever, Ashby)
• Offline eval harness using audit-trail corrections as labels
• Prompt caching + OpenAI Batch API for the extraction backlog
• Outcome-based ranking (which mappings actually led to interviews)
• Chrome extension for save-to-queue

---

Related work

Same thread — retrieval and messy-data pipelines — in a different domain:

• TradeShow-Calendar-Cleaner — ETL for trade-show event data built during my GES internship, cutting manual data prep time by 90% and feeding a Power BI market-intelligence dashboard.

---

Contributing & license

Primarily a portfolio project, but feedback and PRs are welcome — see CONTRIBUTING.md. MIT licensed, see LICENSE.

Mekyle Siddiqi · GitHub · LinkedIn

Open to roles in AI Engineering, Solutions / Forward-Deployed Engineering, Analytics Engineering, and Software Engineering.