Repository navigation
v0.2.0 — hardening: frozen reference, honest calibration, abstention
Hardening pass over the v0.1 authorship-verification CLI. All changes make the calibrated verdict harder to fool and its reported numbers honest. Source-only release.
Highlights
- Frozen reference model.
calibratefreezes the exact reference (vocab + mean/SD) into the calibration, so a calibrated profile'scosine_deltaandP(same author)stay bit-identical when other profiles are added or removed. Profile-set drift downgrades to a warning (the imposter pool changed) instead of silently shifting the verdict. - Honest threshold accuracy. The decision threshold is selected on a train split and its accuracy reported on a contiguous held-out tail it never saw (
holdout_accuracy), instead of on the data it was fit on. The shipped threshold/logistic then refit on all data. - CV-selected regularization. The logistic's L2 strength is picked by 3-fold cross-validated Brier score over a grid (weak lambdas gated behind a 30-sample minimum), replacing the hard-coded λ=0.5 that squashed all probabilities toward 0.5.
- PAN c@1 + abstention. P(same) in the 0.35–0.65 band returns an
inconclusiveverdict rather than a forced call;calibratereports c@1 (with abstention) and holdout Brier alongside AUC. - Guard-rail warnings. Length-mismatch warning when the query is <0.5× or >2× the calibration chunk length; minimum-imposter warning when calibrating against fewer than 3 other authors (probabilities would be overconfident).
- stdin input.
compare -or piped stdin, alongside a file path and--text. - Unicode tokenizer unification. Chunk sizing, the short-text gate, and length warnings all count Unicode words — the same segmentation the features use.
Packaging
- Package renamed to
stylometry-clion crates.io (the bare name was taken); the installed binary staysstylometry. - Install:
brew install paperfoot/tap/stylometry,cargo install stylometry-cli, or from source.
72 tests green; eval suite passes (AUC 1.0, LOWO 5/6).