Skip to content

v1.7.0 — charts in the viewer

Choose a tag to compare

@cattolatte cattolatte released this 27 Jul 12:31

The viewer was ninety-nine lines listing runs. Every manifest michi has written since v0.2 carried a confusion matrix, calibration bins, an ECE, per-subgroup scores, and ten metrics with intervals — and none of it was ever drawn.

A terminal can rank models and print a confusion table. It cannot draw a reliability diagram, and that's the whole reason the viewer exists.

pip install 'komichi[ui]'
michi ui

Four charts

Chart What it answers
Metrics with intervals Which numbers this dataset actually pinned down. A wide bar is a metric you shouldn't quote to three decimals.
Confusion matrix Which errors, not how many. Shaded by share of each true class.
Calibration Whether a probability means what it says, with the expected calibration error beside it.
Score by subgroup The group the average hides. Worst-first, only the worst one coloured.

Every chart renders from the manifest and computes nothing — a chart that recomputed a number could disagree with the terminal, and then two of michi's surfaces would describe the same run differently.

Inline SVG throughout: no plotting library at render time, no CDN, no JavaScript. The viewer still works air-gapped. A chart that can't be drawn honestly isn't drawn — past ten classes the grid is unreadable, so the table is shown instead.

Three faults found by looking at the page

  • The charts were invisible. They shipped with a hardcoded near-black for text and rules — exactly the viewer's background colour. Text and rules now inherit the page's own colour, so one chart is legible in both the dark viewer and a light printed report. A test asserts it for all four.
  • Confusion shading was per matrix. On an imbalanced problem that makes the majority class the only visible cell, hiding the failure the chart is read to find. Now per row.
  • Subgroup and metric labels collided with their own bars.

Also

michi --help gains a stage map naming each verb by the kanji for what it does:

道 the path — 見 inspect · 整 clean · 比 bench · 確 eval · 探 tune · 作 fit · 記 report · 出 export

And the CLI module docstring stopped claiming the console "will become" available in v0.5.

Install: pip install 'komichi[ui]'