Skip to content

v0.7.14

Choose a tag to compare

@github-actions github-actions released this 25 Sep 12:50
· 100 commits to main since this release

Fixed — what the Python engine reads out of a log

Found by running every log in the repository through the pipeline and
comparing what came out with what the file says. Two fixtures had never been
read by any test.

  • A web server's log got a training story. "Response sent: 200 OK" and
    "Numbers but no keys: 0.234" were read as metrics, filed under one name,
    "custom", and narrated as a model "past its best, 200.0 → 0.234", graded F.
    A metric whose name we do not recognise now keeps that name — two unknown
    metrics are two series, not one — and a name printed only once is held until
    it recurs, since a single "word: number" is indistinguishable from prose.
  • The fingerprint demo told 50 epochs as one reading. Its Epoch 15/50
    lines win the Keras parser, which could not read the train_loss=… EER=…
    lines under them. A line the run's parser cannot read now gets a second look
    from the universal parser, keeping only metrics we recognise, on a scratch
    context so prose cannot move the step axis.
  • The Keras parser read any "word: number" anywhere — the dataset sizes
    ("Train: 4200 | Val: 800"), and a tqdm timestamp "[00:12<00:00" as a metric
    named 00. Keras prints metrics as " - name: value"; that is now required,
    and metric names must start with a letter in the universal parser too.
  • Every Keras and Lightning run's first frame was TRAINING accuracy, and the
    rest validation accuracy: one chart, two series, and the baseline taken from
    the wrong one. A one-epoch run was graded on training accuracy. The engine
    now sees every metric on a line before choosing which one tells the story.
  • Colour codes whose escape byte was lost (common in saved CI output) became
    part of metric names: 1mloss, 1maccuracy.
  • A Hugging Face story left out epoch 1's accuracy. The task was classified
    on the metrics processed so far, so an eval_loss, eval_accuracy line was
    judged on eval_loss alone. Classification now sees the whole line.
  • A line read by the fallback parser lost its epoch ({'epoch': 3.0}), so
    the lines after it were stamped with an older one.
  • A JSON true was charted as a reading of 1.0 — in Python a bool is an
    int — by the Hugging Face, Accelerate and universal parsers.

Fixed — the VS Code extension reads logs as the Python engine does

The extension's own engine, used without the Python package, disagreed with
Python on 25 of the 38 logs in the repository. It narrated metrics named
using (a Lightning banner), summary (a YOLO model summary), 00 (a tqdm
timestamp) and Train (a dataset size); lost every YOLO validation row behind
a preamble; had no fastai parser; recorded a YOLO epoch once per progress-bar
redraw; and graded Hugging Face runs on training loss. It now agrees on all 38.

  • Its parsers are ported from Python rather than approximated: the full
    universal parser (qualified splits, two-word names, cross-validation folds,
    constructor masking, progress-bar skipping, holding a once-printed name), the
    Keras, Lightning and Hugging Face fixes, and new fastai and Accelerate
    parsers. The never-metric names and sniff settings are generated from Python.
  • One parser per run, chosen on a 200-line sample as in Python, with the
    universal parser reading recognised metrics off lines it cannot — instead of
    "best parser plus universal on every line". A watched terminal settles on
    what it has after 1.5 s of quiet, so a live run still draws as it goes.
  • Lines are cleaned as in Python (colour codes, carriage-return redraws), and
    files are read in raw chunks: readline split a redraw into separate lines.
  • The task is re-classified while it is still unknown, and the story's metric
    is re-chosen line by line with its baseline restarted on a switch — the
    engine had locked both after four metrics.
  • Frames now carry the metric they measure, so the dashboard no longer formats
    a box_loss frame with mAP50's formatter.
  • epochix-vscode/src/test/suite/corpusParity.test.ts replays the same
    tests/fixtures/corpus_truth.json the Python pipeline is held to.

tests/integration/test_log_corpus_truth.py drives all 38 logs through the
pipeline against reviewed expectations — including the epoch each story's
metric starts at — and fails for a log without one.