Part 2: Extend, then evaluate — the checkable job agent

The Observable Job Agent, Part 2: Extend, then evaluate.

Pick a job from Part 1's ranked list and the agent prepares the application: a personalized cover letter plus a tailored CV for that exact posting, grounded strictly in what your CV actually says. Then it checks its own work, claim by claim, and shows you a warning card for everything it cannot verify.

📖 Blog post: https://jamwithai.substack.com/p/build-your-own-job-agent-part-2

What's inside

• One graph, two doors: a conditional entry router sends a second invocation on the SAME thread into the tailor node — profile, ranked jobs and CV text come back from the checkpoint, nothing re-runs
• The candidate corpus: your CV (plus an optional official LinkedIn export) becomes typed, addressable items — the only permissible source for tailoring (corpus_ref on every bullet)
• A deterministic fabrication validator: zero LLM calls, env-tunable thresholds (SCOUT_FAB_*), every report records the thresholds it ran with
• PDF rendering with a degradation contract: Jinja2 → LaTeX → tectonic, falls back to .tex + Overleaf, never fails a run
• The human decides where to search: a location + remote chooser whose choices persist with your saved profile
• The evaluation stack in Opik: datasets built from your own traces, deterministic metrics vs G-Eval judges, trajectory metrics, an annotation queue, and judge-vs-human calibration (the same 30 outputs scored 0.44 by one judge and 0.84 by another — that split is the lesson)

Get started

git clone --branch part2.0 https://github.com/jamwithai/observable-job-agent
cd observable-job-agent
uv sync --all-groups
cp .env.example .env    # add one LLM key (OPENAI_API_KEY, or free via groq:/ollama:)
make test               # 129 tests, no network or keys needed
make app                # upload a CV, search, pick a job, tailor