Skip to content

v1.6.0 — ensemble

Choose a tag to compare

View all tags
@cattolatte cattolatte released this 27 Jul 12:14
v1.6.0
9de5cde

Every tool will report an ensemble's score. Almost none report it beside the models it combines — which is the only comparison that answers whether combining bought you anything.

michi ensemble data.csv --target churned --models linear,rf,hist-gbm
  model      balanced_accuracy      95% interval   vs leader                     fit
  ──────────────────────────────────────────────────────────────────────────────────
  linear                 0.884   0.8402 – 0.9285   leader                       0.0s
  rf                     0.884    0.8144 – 0.954   tied with leader (p=0.99)    0.5s
  ensemble               0.881   0.8368 – 0.9261   tied with leader (p=0.667)   7.1s
  hist-gbm               0.869    0.807 – 0.9309   tied with leader (p=0.478)   1.0s
  dummy                    0.5         0.5 – 0.5   worse (p=0.0071)             0.0s

That output is the feature. The stack tied its best member and cost 7.1 seconds against 0.0. Reported alone, it would have looked like a result.

Two ways to combine

--method What it does
stack (default) Trains a meta-learner on out-of-fold member predictions. --final chooses it.
vote Averages members. Soft voting where every member can express confidence, hard voting otherwise.

Stacking leaks unless the meta-learner is fed out-of-fold predictions. sklearn's stacking estimators cross-validate internally to build the meta-features, and michi wraps that in its own outer folds — so the reported number comes from data neither layer was fitted on.

Each member carries its own preparation: standardisation is right for a linear model and pointless for a tree, and sharing one pipeline would impose one model's needs on all of them.

What it will not do

michi picks no members, prunes none for scoring poorly, and weights none by validation score. Those are modelling judgements, and a tool that makes them silently is an AutoML system wearing a different hat.

An ensemble is assembled from what you named, so it joins the catalogue for the duration of one command and leaves cleanly — including when the run raises. That's why run_benchmark could treat it as an ordinary model, and why nothing downstream needed a branch asking whether a row is an ensemble.

Install: pip install komichi

Assets 2
Loading