Repository navigation
v2.0.0
ECE was not a function of the data. Equal-mass binning - the strategy the README recommends for exactly the top-heavy regime where it broke - split runs of tied confidences across bin boundaries. 1,000 predictions at confidence 0.9 with 60% correct (a true gap of 0.30) read 0.4146, and MCE swung 0.1111 to 0.1570 across 50 shuffles of the same rows. Cuts now snap to value boundaries, and a permutation-invariance property test locks it.
effectiveBinsreports how many bins actually carried data, so a reliability diagram cannot be labelled with a bin count it does not have.- New
eceInterval()(seeded bootstrap CI) andnullEce(), the noise floor. ECE is positively biased: a perfectly calibrated n=100 at 15 bins reads a median of 0.0874. The README publishes that table, because a bare ECE of 0.08 means nothing without it. calibrationError([])andbrier([])now returnNaNrather than0- no data was quietly passing anece <= 0.1ship bar as perfect calibration.fitTemperaturereportsatBoundwhen the optimum is pinned to its own bracket, and validates labels and logits instead of returning NaN.softmaxno longer blows the stack on a real vocabulary (tested at 200,000 classes).
36 tests.