Misconception Map

Misconception Map is a local-first, teacher-facing diagnostic workspace for middle-school algebra and fractions, with an access-code-protected hosted judge demo. It turns a structured exam and deidentified student work into four answers a teacher can read quickly:

1. Where am I?
2. What did the AI settle, and where was it uncertain?
3. How is the class doing, exercise by exercise?
4. What happened on this copy?

Grading follows one non-negotiable rule: AI proposes; the teacher decides. The model may suggest evidence-grounded points after correction, but no proposal enters the gradebook or any class/student statistic until the teacher reviews every question and explicitly validates the final values. The diagnostic layer still abstains when evidence is weak, and every Student Model remains a versioned, testable hypothesis rather than a fixed learner attribute.

What the demo proves

• A visible four-step assignment path: Exam source → Student copies → AI correction → Results.
• Four unambiguous top-level destinations: Classes → Assignments → Analytics → Prediction Lab. Assignments owns the complete correction workflow; Analytics owns every result view.
• Hierarchical extraction that preserves every printed exercise, shared stimulus, and question label. The real six-page French brevet fixture returns all six exercises; geometry, probability, and statistics questions remain visible as teacher-selected exam content and are marked “not yet diagnosed” when the current algebra/fractions engine cannot assess them.
• A read-only Results summary where ambiguous, unreadable, or unmatched work remains visibly flagged without creating a teacher task.
• Analytics that lead with the answer: headline counts, then each frequent difficulty with the affected students named inline (one click from a student chip to their exact work), then the class summary by exercise, then the compact per-student evidence grid.
• Corrected copies grouped by exercise, with shared context shown once and feedback written in the language of the exam.
• Assignment-owned point schemes, per-question AI proposals with grounded partial-credit explanations, explicit no-score abstentions, and an auditable teacher validation step before any grade becomes statistical data.
• Targeted five-question micro-practice, a teacher answer key, and a short “Teach This Tomorrow” intervention.
• A one-click follow-up evaluation drafted in the same structure and language as the source exam, where every question is pinned to an observed mistake — repeated misconceptions, one-off slips, and items the AI could not settle — with a printable student sheet and a teacher answer key that names what each question retests. Identical inputs reuse the saved draft instead of a new API call, and the draft never enters the gradebook.
• A Prediction Lab that locks flawed-rule, demonstrated-mastery, or abstain predictions before held-out work exists, then scores, revises, or invalidates them transparently.

Misconception Map is built for the Education category of OpenAI Build Week. Next.js and SQLite run on the teacher’s machine. Only live extraction, diagnosis, grading proposals, model synthesis, practice, briefs, follow-up evaluations, and predictions call the OpenAI API.

Quickstart

Prerequisites: Node.js 20.9 or newer (Node.js 24 recommended) and npm.

git clone <your-repository-url>
cd "Misconception Map"
npm install
cp .env.example .env.local
npm run seed
npm run dev

Open http://localhost:3000. The local clone remains the full, unlimited local-first experience.

For live AI features, put one key in .env.local before starting the app:

OPENAI_API_KEY=your_key_here

npm run dev applies checksummed SQLite migrations before starting Next.js. The app binds to 127.0.0.1; do not expose this single-teacher build through a LAN binding or public reverse proxy.

Hosted judge demo

Hosted URL: web-production-6c624.up.railway.app. The shared access code is intentionally not in this public README; put <JUDGE_ACCESS_CODE from the author> only in the Devpost testing-instructions field.

The hosted path is the same application with HOSTED_MODE=1: every product page and API route requires a seven-day, httpOnly, HMAC-signed access cookie; failed gate attempts and live-AI requests are rate-limited. A banner appears only when live AI is unavailable (for example when the daily demo budget is reached). Sample fixtures remain downloadable at /api/fixtures/full-page-followup.jpeg and /api/fixtures/negative-distribution.jpeg; upload only synthetic or de-identified work to the shared instance. Seeded Analytics views, grouped copies, and Prediction Lab remain available if live AI is unavailable.

Railway deployment:

1. Create a Railway service from this repository. Railway detects the root Dockerfile and uses /access as the healthcheck from railway.toml.
2. Attach one persistent volume at /data.
3. Set HOSTED_MODE=1, DATA_DIR=/data, JUDGE_ACCESS_CODE=<long shared secret>, OPENAI_API_KEY=<server key>, and OPENAI_DAILY_BUDGET_USD=5. Optionally adjust OPENAI_REQUESTS_PER_SESSION_HOUR from its default of 20.
4. Generate a Railway public domain. npm start binds to 0.0.0.0:$PORT, applies checksummed migrations, and idempotently restores the synthetic demo at every boot.

The daily guard estimates spend from saved input/output token counts at the configured GPT‑5.6 rates. At the cap it disables only AI actions with “Daily demo budget reached — resets at midnight UTC; clone the repo to run unlimited.” Seeded reads keep working. This in-app guard is defense in depth; also set a project budget/alert in the OpenAI dashboard.

Production verification on July 17, 2026 covered the access gate, persistent seeded classroom, downloadable fixture, one live full-page GPT‑5.6 correction, grouped French corrected copy, print/PDF action, and Prediction Lab. The synthetic page matched all five visible questions across three exercises and saved 6,915 input plus 3,008 output tokens; /status reported an estimated $0.125 for the run.

No API key? Use the judge path

Leave OPENAI_API_KEY empty, run npm run seed, and open the app. The deterministic synthetic classroom works fully without network access or API spend. It includes exactly:

• 20 synthetic learners and two completed assignments;
• the four-step assignment path, reopening at Results;
• a Results summary with 18 corrected copies, 1 item flagged as uncertain, and 1 outside misconception analysis;
• a three-exercise Analytics hub with success rates, dominant misconceptions, and flagged counts;
• exercise-grouped corrected copies with French prompts and French step feedback;
• a printable targeted worksheet and teacher answer key linked to Ex. 2 · Q2.2;
• a Teach This Tomorrow brief;
• one pending grading proposal with an abstained question requiring manual points, plus one teacher-validated proposal and audit trail;
• a 4-of-5 consistency-weighted Student Model, a locked mastery prediction, a teacher-reviewed revision suggestion, abstentions, matches, mismatches, and invalidation history.

Live-only controls remain visibly disabled and explain that OPENAI_API_KEY must be added to .env.local. There are no dead clicks; seeded views remain readable.

Five-minute product tour

1. Open Classes and inspect the 20-learner synthetic roster.
2. Open Assignments, then Unit 3 follow-up · Held-out check. The assignment resumes at step 4 of the Exam → Student copies → AI correction → Results path.
3. Open Results for the completion summary, then follow its numbered cards into Analytics and read a corrected copy. The uncertainty flag and its reason stay attached to the evidence; nothing needs to be cleared or marked complete.
4. Open Analytics and visit all three assignment tabs: Class by exercise, Corrected copies, and Practice & brief. Class by exercise leads with headline counts and the most frequent difficulties, each naming its affected students inline; a corrected copy’s summary chips jump directly to each exercise. Print/PDF hides the application chrome.
5. Open Prediction Lab. Inspect predictions that were timestamped and locked before the held-out responses, including visible abstentions and invalidated historical trials.

Navigation and legacy links

Teacher destination	Canonical route
Classes and rosters	/classes
Assignment list, creation, stepper, correction, and Results	/assignments and /assignments/:assignmentId/...
Analytics picker and assignment result tabs	/analytics and /analytics/:assignmentId/...
Prediction Lab	/prediction-lab

Bookmarks remain safe: /diagnose redirects permanently to /assignments, /dashboard redirects permanently to /analytics, and the former per-assignment dashboard, corrected-copy, and practice-sheet URLs redirect permanently to their equivalent /analytics/:assignmentId/... pages.

Run one live correction

All live calls use gpt-5.6, strict Structured Outputs, store: false, prompt/schema versions, input/output hashes, token counts, and latency provenance.

Synthetic image fixture

1. Configure the key, seed the demo, and start the app.

2. Open Assignments, choose New assignment, and select the synthetic class.

3. Create an Algebra assignment. Paste a short teacher source such as:

   Exercice 1 — Signe et parenthèses
   1.1 Développer puis réduire −3(x + 4). Réponse : −3x − 12.
   

4. Confirm the extracted Ex. 1 · Q1.1 question.

5. Upload sample-work/01-negative-distribution.jpeg for a demo learner, attest that it contains no identifying information, check the match, and run correction. For exercise/question segmentation against the seeded follow-up, use sample-work/09-full-page-followup.jpeg.

6. The permanent regression fixture fixtures/student-work/sign-error-equals-regression.jpeg is also available. It protects the case where a faint handwritten = was once read as a dash.

All files in sample-work/ and fixtures/student-work/ are synthetic and name-free. In hosted mode, the same single-answer and full-page fixtures are downloadable from the privacy banner.

Real French exams, included for judging

sample-exams/ contains the material I actually test with: five past papers from the 2019 French brevet (the national exam at the end of middle school, ~grade 8/9), each with a teacher subject PDF and three handwritten student booklets.

Two things to know before opening them:

• Everything in there is French. I am French, and the complete national exam sets I had access to are the French ones. The app itself is in English; only the exam content and the generated feedback follow the language of the exam, which is what a returned copy should do anyway.
• The students are invented. Cecilia, Julia and Thomas do not exist. I wrote every booklet by hand myself, giving each invented student a different profile of mistakes. No real student work is in this repository; npm run verify:privacy allows exactly this folder and still rejects databases, uploads, and everything under data/.

To run one exam end to end (with OPENAI_API_KEY set):

1. Assignments → New assignment, pick a class, domain Algebra, source Exam or worksheet, and upload for example sample-exams/France/2019_04_France_Serie_generale_SUJET.pdf. Review the extracted exercises, then Confirm worksheet and add student work. Exercises outside algebra/fractions (geometry, probability, Scratch) stay visible as exam content — they are never forced into an algebra misconception.
2. Under Student copies, upload the three booklets from the same folder, tag each one to a student, attest that no real names are visible, and hit Diagnose. A full six-page booklet takes about two to four minutes.
3. Open the assignment's Analytics for the class view, the returnable corrected copies, and the follow-up evaluation generator.

I ran all five exams through the pipeline on July 20–21, 2026: 15 booklets, 393 questions diagnosed, 59 items flagged as uncertain instead of guessed, and no forced verdict. Matching uses printed cues such as 1.1; ambiguous work stays visibly unmatched rather than being guessed into a slot. The France exam is loaded and fully diagnosed on the hosted judge demo, so the fastest way to see the result is to open its analytics there.

Teacher workflow and evidence model

1. Exam source

Teacher intake accepts typed text, JPEG, PNG, WebP, and PDF up to 15 MB. Printed sources use low-detail vision and low reasoning effort. The extraction schema is a strict root object containing exercises, optional shared context represented explicitly as null, original or synthesized question labels, self-contained statements, expected answers, printed points when present, answer kinds, domains, confidence, and review notes.

An identical extraction input hash reuses the stored run. The teacher edits labels, statements, context, domains, answers, and per-question points before confirmation. Missing printed points default to equal weighting. The confirmed barème is assignment-owned; confirmed grouping is immutable, and legacy assignments are migrated into a default exercise without renumbering already-diagnosed work.

2. Student copies

Student intake accepts the same formats up to 10 MB per file, 20 files and 80 MB per queue, or 20 typed responses of up to 8,000 characters. Work is saved locally before diagnosis. Roster names and original filenames are excluded from OpenAI payloads and hashes.

Single-question images are auto-oriented, line-aware cropped, contrast-normalized, and retained with a metadata-stripped full-frame fallback. Full pages and PDFs are not cropped. Handwritten diagnosis keeps high-detail vision and medium reasoning effort. The prompt explicitly warns that = can resemble a short dash, and an implausible variable-bearing final fragment caps confidence instead of allowing a guessed diagnosis.

3. AI correction

The model segments visible work against confirmed exercise and question labels, transcribes exact steps, and returns one strict diagnosis object per match. A deterministic policy then enforces evidence grounding, domain compatibility, transcription quality, and the 0.72 confidence threshold.

A wrong answer alone never becomes a misconception. A definitive label requires a grounded incorrect step, exact evidence quote, observed transformation, and enough confidence. OpenAI failures persist a sanitized retry state. Repeating the same submission replays the saved diagnosis before the API-key check, so refreshes and retries do not create duplicate spend.

4. AI grading proposal and teacher validation

After correction, the teacher may request a strict Structured Output containing one proposed score per safely corrected question. Full credit requires demonstrated correct reasoning. Partial credit requires a grounded correct prefix before the first flawed step, with a concise explanation in the exam’s language; zero also requires an evidence-based explanation. NEEDS_REVIEW, abstained, or uncorrectable questions receive no AI score and remain explicitly manual, so the proposal total is provisional and incomplete.

Every proposal begins as PROPOSED. The teacher sees the AI value, quoted student work, and justification beside the correction, may edit every per-question score, and must explicitly choose Validate grade & add to gradebook. Validation writes the final total to exam_grades and appends an audit row containing the AI-proposed value, teacher-final value, and timestamp for each question. Only VALIDATED totals are read by gradebook statistics.

5. Results

Results presents corrected-copy, uncertainty, and misconception-analysis counts without creating a task list. Teacher-selected exam content always remains part of the assignment. Analytics includes a complete error inventory ranked by pedagogical priority: taxonomy misconceptions by distinct students and occurrences, settled one-off slips by exercise, then items the AI could not settle. Every entry links to its evidence and corrected copy. An isolated slip never becomes a misconception or Student Model evidence; this follows Sleeman’s (1984) distinction between systematic, clerical, and random errors. The cross-assignment class profile rolls misconception evidence forward while keeping slips attached to their source assignment. Analytics then exposes the student-by-difficulty evidence grid. It uses identical legend semantics everywhere:

• mint: correct reasoning shown;
• amber: seen once;
• coral: seen repeatedly;
• gray: not assessed.

All surfaces use the same Ex. 1 · Q1.2 reference formatter: queue, Results, heatmap drawers, practice sheets, Prediction Lab, corrected copies, and print.

Student Model and Prediction Lab

A Student Model is a versioned, falsifiable learner hypothesis tied to exact work. One response can create only a provisional version. Support requires evidence from at least two distinct problem fingerprints without contradiction. Each new version separately records how often the flawed rule appeared when it could have applied and which related skills were demonstrated correctly; legacy versions keep those nullable fields as “consistency unknown.”

Prediction Lab applies one supported model version to unseen content without sending the student name. Each prediction is locked and timestamped before held-out work exists as one of three kinds: FLAWED_RULE_APPLIES, MASTERY, or ABSTAIN. Flawed-rule confidence snapshots the observed application rate; mastery requires matching demonstrated-correct skill evidence and predicts the expected correct answer. Later work appends the same deterministic match/mismatch outcome for every answer prediction. A miss creates a strict, null-based revision suggestion; only a teacher confirmation creates a provisional v+1. Updating a model invalidates its older locks without deleting history or silently moving trials to the new version.

Expected-versus-actual reporting treats a 3-of-4 result as consistent with a 0.8-application model rather than implying deterministic failure. This follows the within-student strategy variability documented by Siegler & Pyke (2013). The Prediction Lab is deliberately not weakened for demo convenience: abstentions remain visible, invalid trials remain visible, and syntactically nonidentical equivalent forms are not silently counted as matches.

Status, costs, and verification

Open /status to see database migration state, taxonomy synchronization, whether live AI is configured, the hosted daily aggregate/cap, and the most recent saved runs with input, output, and total tokens, latency, and cache-hit status. Per-request API cost is not exposed; only the daily aggregate is estimated from the ledger.

The July 18 task-tier benchmark compared the models exposed by the account (gpt-5.6-sol, gpt-5.6-terra, and gpt-5.6-luna) on the production preprocessing, strict schemas, two handwriting regressions, one South America booklet page, the six-page printed exam, and practice generation. The final 15-call matrix cost an estimated $0.828. Terra reduced the representative page run from 101.1 s / $0.211 to 30.0 s / $0.087, but failed the handwritten-equals fidelity gate; Luna also disagreed on page verdicts. Diagnosis therefore remains on Sol. Extraction and practice stay on Sol too because one passing case is not enough independent gold evidence to change a production default. See docs/model-benchmark.md for the cost-versus-quality table and abstention checks.

Useful commands:

Command	Purpose
npm run dev	Migrate and start development mode on loopback.
npm run seed	Idempotently load the 20-learner synthetic classroom.
npm run db:migrate	Apply checksummed SQL migrations.
npm run db:check	Verify SQLite integrity and bootstrap records.
npm run sample-work	Regenerate the nine synthetic JPEG fixtures with Sharp.
npm run bench:models -- --dry-run	Validate the capped 15-call Sol/Terra/Luna benchmark matrix without inference calls.
npm run verify:privacy	Refuse tracked databases, uploads, PDFs, named booklet files, or unreviewed image locations.
npm run verify:phase1	Domain model, evidence, versioning, prediction, and invalidation invariants.
npm run verify:hierarchy	Hierarchical extraction, legacy migration, label matching, and grouped demo shape.
npm run verify:phase2	Diagnosis schema, grounding, confidence, domain, and abstention policy.
npm run verify:images	Faint-ink, crop, and handwritten-equals regression fixtures.
npm run verify:pdf	PDF signatures, generated API filenames, and local persistence.
npm run verify:diagnosis-contracts	Compile-time-safe single/full-page persistence contracts.
npm run verify:full-page-timeout	85 s/300 s timeout separation, bounded page output, truthful timeout persistence, and quality-only fallback.
npm run verify:phase4	4-of-5 consistency, expected/actual fit, all three prediction kinds, revision suggestions, practice, and briefs.
npm run verify:navigation	Four-tab order, assignment-first root, Analytics sub-tabs, canonical cross-links, and permanent legacy redirects.
npm run verify:ai-grading	Grounded partial credit, no AI score on abstention, audit rows, and the one-way PROPOSED → VALIDATED grade boundary.
npm run verify:follow-up	Follow-up evaluation schema strictness, target coherence, append-only storage, mistake-coverage enforcement, and the guarded generation route.
npm run verify:readiness	Fresh/seeded DBs, no-key states, language, print, accessibility, cost, cache, and status ledger.
npm run check	Lint, typecheck, every verifier, and production build.

For an isolated database, prefix commands in the shell. Migration scripts intentionally do not read this override from .env.local:

MISCONCEPTION_MAP_DB_PATH=/tmp/misconception-map-smoke.db npm run seed
MISCONCEPTION_MAP_DB_PATH=/tmp/misconception-map-smoke.db npm run dev

Clean-install proof

Verified on July 16, 2026 from commit 5a9b614 with Node v24.5.0 and npm 11.13.0. The proof used a new local clone with no copied database or node_modules:

git clone --no-local "/path/to/Misconception Map" /tmp/misconception-map-clean-install
cd /tmp/misconception-map-clean-install
npm install
cp .env.example .env.local
OPENAI_API_KEY= npm run seed
OPENAI_API_KEY= npm run check
OPENAI_API_KEY= npm run dev -- --port 3200

The no-key rendered-route smoke covered Classes, the Assignments stepper and 18/1/1 Results summary, every Analytics sub-tab, a grouped corrected copy, Prediction Lab (4 of 5 observed applications, 3 actual versus 3.2 expected flawed-rule hits, one mastery prediction, and one revision suggestion), assignment setup, and /status. Live-only controls were disabled with the .env.local explanation.

For the live smoke, the local key was added to the clone’s ignored .env.local, the server was restarted, and sample-work/01-negative-distribution.jpeg was uploaded for a new synthetic learner against Ex. 1 · Q1.1. GPT-5.6 returned a grounded French NEEDS_REVIEW result instead of forcing a taxonomy match. The run saved 6,151 input and 1,393 output tokens; /status displayed 7,544 total. Repeating the identical diagnosis returned the persisted result while the database remained at exactly one diagnosis run.

Architecture

• Next.js App Router, React Server Components, TypeScript, and Tailwind CSS.
• Local SQLite through better-sqlite3, with checksummed migrations and integrity triggers.
• Small client islands for intake, heatmap drawers, Prediction Lab, and print actions.
• Node.js route handlers for local file processing and OpenAI Responses API calls.
• gpt-5.6 is the only live model.
• Strict structured output on worksheet extraction, page segmentation, diagnosis, grading proposals, Student Model synthesis, practice, teaching briefs, mistake-targeted follow-up evaluations, three-kind predictions, and outcome-driven model-revision suggestions.
• Append-oriented answer versions, diagnoses, Student Models, prediction locks/outcomes, and AI provenance.

The data graph covers classes, memberships, exercises, reusable problems, assignment items and point schemes, protected assets, upload batches, submissions, answer versions, diagnosis steps/candidates, grading proposals and validation audits, validated exam grades, Student Model evidence/opportunities/mastery, append-only revision decisions, worksheets, teaching briefs, frozen predictions, token provenance, and redacted audit events. Historical teacher-review columns remain migration-compatible but are no longer read or written. Composite foreign keys and triggers keep every record inside its class and assignment.

Accessibility and print

• Visible :focus-visible treatment across links, controls, fields, and disclosure widgets.
• Native buttons for heatmap cells with descriptive student, question, state, severity, frequency, and evidence labels.
• Results links expose corrected copies and evidence without a modal task flow.
• Evidence drawers close with Esc and return focus to their source cell.
• Icons and explicit text accompany every color state.
• Corrected copies and practice/answer keys print to A4 without the sidebar or application header.
• The corrected-copy summary remains on page 1; exercises start on clean page boundaries; long questions break only between readable feedback blocks.

Troubleshooting

Live button is disabled. Add OPENAI_API_KEY to .env.local, stop the running server, and restart npm run dev. /status must say gpt-5.6 is configured.

Hosted live AI is disabled. Read the hosted banner or /status. A missing server key needs an operator fix; a daily-budget block resets at midnight UTC; an hourly session limit reports its own retry window. Seeded views do not require a retry.

Hosted data disappeared after redeploy. Confirm the Railway volume is mounted at /data and that DATA_DIR=/data. The database is /data/misconception-map.db; protected uploads are under /data/uploads/.

Fresh clone shows no classes. Run npm run seed. The empty state intentionally gives that as its only next action.

A PDF is rejected. Confirm it has a valid %PDF- signature and is below 15 MB for a teacher source or 10 MB for student work. Password-protected or malformed PDFs are not supported.

An exercise appears as not yet diagnosed. It remains part of the teacher’s exam, but only algebra and fractions currently receive taxonomy diagnoses. Edit a wrongly inferred domain during review; do not relabel geometry/probability merely to force a diagnosis.

OpenAI failed mid-flow. The local source or submission remains saved with a sanitized error. Use the single retry action. Do not re-upload: identical stored work is reused.

Port 3000 is busy. Stop the other local Next.js process or run npm run dev -- --port 3001 and open the displayed loopback URL.

Native SQLite module fails after changing Node versions. Remove node_modules, run npm install again with Node 20.9+, then run npm run check.

Privacy

All committed and seeded student work is synthetic and name-free. Raw roster names remain in local SQLite and are never sent to OpenAI. Before any upload, the teacher must attest that visible names and identifying PDF properties were removed. Typed sources are also blocked when they contain an exact local roster name or a textual roster-name component of two or more characters; purely numeric roster suffixes do not block ordinary math answers. This is a narrow guard, not general personal-data detection.

Images have metadata removed. PDF API filenames are generated. Protected uploads live outside public/ and are served only through database-owned routes: loopback plus same-origin guards locally, and the signed access gate plus same-origin guards when hosted. Responses use private, no-store caching. The app does not encrypt or automatically purge files. Anything still visible inside an attested image or PDF is sent to OpenAI, so this hackathon build is not a substitute for institutional consent, retention, security, or child-safety review.

Before publication, npm run verify:privacy audits the Git index. Real exam/booklet PDFs, SQLite files, uploads, and known student-booklet names are ignored or rejected; only the synthetic fixture directories and product screenshots are allowlisted for tracked images.

OpenAI calls use store: false. The teacher source, assignment context, and deidentified work needed for the task are sent; local roster labels and original filenames are not.

Roadmap

• Native in-app PDF page rendering and page-aware feedback markers.
• Hierarchical cross-page matching v2, including page-numbered regions for multi-page booklets.
• A student mode for receiving corrected copies and completing discrepant-event practice.

How Codex and GPT-5.6 were used

I came to this with a teacher problem and a short list of things an AI grader must never do. Codex wrote most of the code. The split stayed the same for the whole build: I decided what the product refuses to do, the agent implemented it, and a growing pile of verifier scripts kept both of us honest.

The rules I set on day one and never traded away: the AI proposes, the teacher decides, so no AI score reaches the gradebook or any statistic before explicit validation. A wrong answer alone is never a misconception; a label needs a quoted incorrect step from the copy. And when evidence is weak, the engine abstains visibly instead of guessing. What I like about how these came out is that they are not UI copy, they are schema. Grade proposals live in append-only rows with a one-way PROPOSED → VALIDATED transition enforced by SQL triggers, Student Models are versioned hypotheses with recorded evidence, and predictions are timestamped and locked before the held-out work exists.

Two failures shaped the codebase more than any feature did. Early on, a live handwriting test read a faint handwritten = as a dash. Instead of prompt tweaking, that became engineering: the image pipeline fix and fallback are in 7b061d0, and the exact failing image is a permanent fixture (fixtures/student-work/sign-error-equals-regression.jpeg) that npm run verify:images replays forever. Later, Codex found a quieter bug I would have shipped: spreading a full GPT service result into a narrower repository shape let fields drift across a strict schema boundary. The fix in cfde617 added explicit field selectors and compile-time satisfies checks, plus a verifier so it cannot come back.

That is the pattern the whole test suite follows. It grew adversarially, around failure modes rather than happy paths: faint ink, implausible steps, PDF signature abuse, oversized intake, stale runs, ungrounded evidence, cross-domain labels, legacy migrations, ambiguous label matching, no-key routes, cache reuse, A4 page fragmentation, Prediction Lab invalidation. npm run check runs seventeen verify:* scripts plus lint, typecheck, and a production build, and it gates every deploy.

Some of my calls are visible in the history as me overruling the tooling. One session renamed "Assignments" to "Exams" across the app; I kept "Assignments" when merging (da2e48b). The first class heatmap was technically fine and pedagogically unreadable, so I had it rebuilt around the questions a teacher actually asks: headline counts first, then each frequent difficulty with the affected students named inline, then the compact per-student grid. And right now the Prediction Lab shows nothing for my real Grade 8 class, because nobody repeated the same flawed rule on two distinct problems. I left it that way. Lowering the evidence thresholds would have made a better-looking demo and a worse product.

GPT-5.6 powers every live call and no other model is used. Every call is a strict root-object Structured Output: teacher-source vision extraction, full-page segmentation, single and booklet diagnosis, per-question grading proposals, Student Model synthesis and revision suggestions, discrepant-event practice, Teach This Tomorrow briefs, mistake-targeted follow-up evaluations, and three-kind held-out predictions. Printed sources use low image detail and low reasoning effort; handwritten diagnosis uses medium. Identical inputs are reused by hash before any new call, and /status shows per-run tokens and cache hits. When I benchmarked the account's model tiers on July 18 (docs/model-benchmark.md), the faster tier was three times cheaper but failed the handwritten-equals gate, so diagnosis stayed on the accurate one. The final proof run was July 20–21: all five brevet exams, 15 handwritten booklets, 393 questions diagnosed, 59 honest abstentions, and one misconception found and named.

License

MIT