v1.3.0 — target encoding, and a menu for choosing what to engineer

Target encoding

Replacing a category with the target's mean is the strongest encoding available for a high-cardinality column, and the easiest way to destroy a model. Encoded naively, a unique id maps one-to-one onto its own label.

Measured on 600 random labels against 600 unique ids — data containing no signal whatsoever:

Encoding	Cross-validated accuracy	Truth
Naive (fit on everything)	1.000	0.500
michi (out of fold)	0.517	0.500

The naive number isn't a bug in the model; it's the model faithfully reporting that it memorised the answer key. A test asserts both halves of that table, because a trap is only instructive if you can watch it spring.

michi clean data.csv --target churned --target-encode city,postcode

michi ships its own encoder rather than scikit-learn's. sklearn's TargetEncoder is not reproducible on any version michi supports — the parameter that made it deterministic is deprecated in 1.9 and removed in 1.11, and the replacement needs a newer floor than >=1.4. Bumping the floor to fix one operation would have dropped users. export writes the encoder class into the generated file rather than importing it, so exported pipelines still depend on pandas and scikit-learn alone, and you can read exactly what the encoding did.

The feature menu

The five operations added in 1.1 were reachable only by flag. The wizard walks findings, and nothing is wrong with a column that could be logged — so no finding ever raised it, and the interactive path couldn't reach feature engineering at all.

michi clean now follows the findings triage with a column picker:

  Feature engineering — optional, and michi has no opinion here.
  Nothing is preselected; space to pick, enter to move on.

  [2/4]  Compress a long tail
         These columns are skewed enough that a few large values dominate
         every distance and every coefficient.
  ? Which columns?
   ◯ fare   (skew +6.87, 467 distinct)
   ◯ salary (skew +0.31, 511 distinct)

An operation appears only where the shape fits: datepart with timestamps, log when something is actually skewed, target-encode when a categorical column is too wide for one-hot. michi lists shapes and stays quiet about worth — whether a product term helps your problem is domain knowledge michi doesn't have, and a preselected checkbox would be a recommendation wearing a checkbox.

Also

• examples/sweep.yaml was the last file in the repo calling itself an example without ever having been run — and it named a target the example dataset doesn't have. It's now the verified plan: 12 cells in 6.1s, with the ranked output in the examples README. The seeds make the point the whole project rests on: linear spans 0.8907–0.8988 across four seeds and rf spans 0.8779–0.8967, so the gap between the best of each is smaller than the spread within either.
• The recipe file header had listed seven operations since v1.0. It now lists all thirteen, and the leakage note names every fitted step.

---

The scikit-learn floor stays at >=1.4. Install: pip install komichi