What is Northstar
An exam hall for AI-generated statistical code: submit an implementation, get a four-layer verdict (L1 calibration / L2 cross-reference / L3 boundary coverage / L4 degenerate inputs). No LLM self-evaluation anywhere — verdicts come from deterministic programmatic checks against independent references.
What is in this release
- Four-layer exams — pearson_chi2 and wilcoxon families, fully open (
exams/, recompile from spec and diff fingerprints) - Invariant exams — reference-free identity checks (direction complementarity), catching flip-style bugs at 500-sample resolution (REJECT 492/500 on flips, 0/500 false positives on correct implementations)
- MCP gateway — stdio server wired to the real spsl examiners (statistical / conclusion_anchor / demo_data / state_estimator / invariant)
- Conclusion anchors — coverage-anchored verdicts on real data (generator-backed, no external data)
- Embodied-data QC — dirty-sample detection (NaN / out-of-range / jumps): 60/60 detected, 0/30 false positives on 402 real robot trajectories
- Self-proof — calibrator re-derives the acceptance numbers from first principles (sensitivity 1.000, false-kill 0.020)
- Determinism — re-running in the same environment gives byte-identical verdicts; 5 exam pairs verified byte-identical
All measured numbers are from internal acceptance evaluations — the exams and the full exam-setting kit are open so you can re-derive everything yourself.
Environment
Python 3.14.6 / NumPy 2.4.4 / SciPy 1.18.0. License: BUSL 1.1 (free for personal/research use; commercial use requires a license — contact the maintainer).