Repository navigation
Releases: m-sanchez/calibrated
Releases · m-sanchez/calibrated
Release list
calibrated 2.0.1
Numerical corrections for temperature scaling. npm registry publication is pending; the corrected source is available in PR #1.
- Compute NLL without probability clipping and preserve raw-logit decisions when probabilities round to ties.
- Handle extreme finite logits, validate inputs and search settings, and report fit status and iteration count.
- Evaluate the seeded demo on separate calibration and test data; clarify the limits of ECE resampling and null simulations.
- Compatibility: empty NLL returns NaN; empty fitting and empty softmax throw. Invalid temperatures, dimensions and search settings throw. Code constructing
TemperatureFitobjects must supply the newstatusanditerationsfields. - Validation: 46 tests, typecheck, build and package-install checks passed on Node 22, 24 and 26 in CI. Independent local SciPy checks passed for 403 probability/loss fixtures and three fits.
v2.0.0
ECE was not a function of the data. Equal-mass binning - the strategy the README recommends for exactly the top-heavy regime where it broke - split runs of tied confidences across bin boundaries. 1,000 predictions at confidence 0.9 with 60% correct (a true gap of 0.30) read 0.4146, and MCE swung 0.1111 to 0.1570 across 50 shuffles of the same rows. Cuts now snap to value boundaries, and a permutation-invariance property test locks it.
effectiveBinsreports how many bins actually carried data, so a reliability diagram cannot be labelled with a bin count it does not have.- New
eceInterval()(seeded bootstrap CI) andnullEce(), the noise floor. ECE is positively biased: a perfectly calibrated n=100 at 15 bins reads a median of 0.0874. The README publishes that table, because a bare ECE of 0.08 means nothing without it. calibrationError([])andbrier([])now returnNaNrather than0- no data was quietly passing anece <= 0.1ship bar as perfect calibration.fitTemperaturereportsatBoundwhen the optimum is pinned to its own bracket, and validates labels and logits instead of returning NaN.softmaxno longer blows the stack on a real vocabulary (tested at 200,000 classes).
36 tests.
v1.0.1
Packaging release.
- Package renamed to
@m-sanchez/calibrated; install withnpm install @m-sanchez/calibratedonce published, or from this tag:github:m-sanchez/calibrated#v1.0.1. - CI packs the scoped tarball and proves it installs and imports on Node 22, 24 and 26.
- README states provenance: where the code came from and when it was first published.
No behaviour changes.