Repository navigation
v0.7.25
Changed — a story sentence says only what the run's numbers support
The phase stories were written as flavour and filled with real numbers. The
numbers were always from the log; the words around them were not. They claimed
outside standards ("near-expert performance", "competition-grade", "clinical
precision", "ready to ship", "indistinguishable from real"), invented specifics
("the model handles glasses, makeup, and partial occlusion", "distinguishes car
from bicycle, cat from dog" — and "the model sees faces but not people", told
about the bundled fingerprint demo), and stated things that could be false
for the run ("the gap between train and val narrows", "loss curves bend
downward", "only one direction from here").
- All 135 phase templates are rewritten, in English, French and Farsi. A
sentence now states the reading, names the phase, and asserts only what the
engine's own rules guarantee while that story is told: that the value is at
or within 1% of the run's best so far, and how far the phase's threshold
puts it from where it started — "three quarters of the way to a perfect
score, or more" for a score in the mastering phase, "down at least three
quarters from its first reading" for an error. A metric Epochix does not
recognise is only named.tests/unit/test_story_claims.pychecks those
quantities against the engine's thresholds over a grid of runs, so changing
a threshold fails the sentences that state it. - A change is told in the reading's own unit. "accuracy 70.7%, a change of
+0.0545" is now "a change of +5.5 points" ({delta_pct}, both engines). - The past-peak story no longer says the model "has passed its peak" — a later
epoch can beat it, and on the ResNet-18 run one did; it says the run is
below its best so far. - In the plain-English panel, "Started · knew little" and "Learned · found the
patterns" read the same for every run; they state the first reading and the
change since. "Gets about 10 in 10 right" appeared at 95.1%; the meter no
longer rounds up to ten below 99.5%. - The README's hero GIF was a July recording of a different run ("Fashion-MNIST
CNN") beside text describing the bundled Keras demo, and the social preview
showed "the model thinks like an expert". Both are re-recorded from the
bundled demo on this version's dashboard, by stepping its epoch scrubber.
Fixed — two panels gave one run opposite verdicts
- The same train/validation gap was "overfitting" on one card and "a small
gap, so it learned the real patterns" on the panel beside it. On the real
ResNet-18 run (99.2% training accuracy, 93.9% validation, validation loss at
its lowest on the last epoch) the diagnostics card read "Overfitting — it
memorises training data more than it learns" under a red alert, from the
loss gap; the plain-English panel read the opposite, from the accuracy gap.
Neither follows from the numbers: a gap alone shows neither memorising nor
its absence, and overfitting is validation getting worse while training
still improves. The gap is now read once (frontend/src/generalisation.js)
from its size and from where validation is heading, and both panels say the
same thing — here, "better on training data than on new data; validation is
still improving, so the gap is not costing it yet". "Overfitting" is said
when validation has passed its best, with the epoch it peaked at. - The student analogy says what it stands for: "the questions it practised
on (training data)" and "questions it had never seen (validation data)",
where it said "practice questions" and "the real test" — validation data is
not a test set. Its bold numbers no longer break the sentence into columns. - A ResNet-18 was captioned "12 layers". The count is of what is drawn — a
stem, eight residual blocks, a classifier — so it reads "12 modules"; only a
list of plain layers is counted as layers.