Skip to content

v1.11.0 — text, time, out-of-fold, calibration

Latest

Choose a tag to compare

@cattolatte cattolatte released this 27 Jul 16:27

Five gaps, all of them places a practitioner had to leave michi to do something ordinary.

Text

A free-text column could previously only be dropped or one-hot encoded — useless for free text.

michi clean reviews.csv --target rating --text-length note --tfidf note
Op What it does
text-length Characters and words. Often most of the signal — how much someone wrote predicts more than any single word
tfidf Term frequencies. Fitted, so it runs inside the fold where the test rows can't vote on which words exist

Time

michi clean sales.csv --target churned \
  --order-by date --lag revenue=1 --rolling revenue=7:mean

Both require --order-by, because "earlier" needs a definition and michi won't guess a sort column. Both accept a group so one entity's history never leaks into another's.

Both are classified deterministic despite depending on other rows — which looks wrong until you see why: they read only earlier rows in a stated order, so no future value can reach a past one. That's exactly the property that makes them safe outside the fold.

bench --oof

Out-of-fold predictions, one column per model:

purchased,linear,rf,dummy
1,1.0,1.0,0.0

Every value was predicted by a fold that did not train on that row. That's what makes them safe to stack on — and what separates them from predictions of the training data, which would give a meta-model the base models' memory rather than their generalisation.

bench --balance

Weights classes by inverse frequency where the estimator accepts it. A model without class_weight isn't an error — the request simply doesn't apply, and saying so would be noise on a leaderboard where other models did honour it.

fit --calibrate

Closes a loop open since v0.2: eval could report that a model says 0.9 and is right 0.7 of the time, and offered no way to fix it.

michi fit data.csv --target churned --calibrate isotonic -o model.joblib

Fitted by internal cross-validation, so the score→probability mapping is learned on folds the base model didn't train on. isotonic is flexible and wants more data; sigmoid assumes a shape and survives small samples.

Fixed

  • bench --oof was declared and never threaded into run_benchmark — it silently wrote nothing. Found by running it, now five for five on how the real bugs here surface.
  • The roadmap listed its 1.x milestones in descending order after 1.2, and still named deep-learning training loops a deliberate non-goal six releases after torch-mlp shipped.

618 tests. Install: pip install komichi