v1.8.0 — bayesian search

tune sampled at random, halved, or exhausted a grid — all three treat every configuration as equally worth trying. A model-based optimiser does something different in kind: it fits a model of the objective and proposes the next configuration from its own beliefs. A thing that forms opinions and acts on them, inside a tool whose first principle is that it does not.

pip install 'komichi[bayes]'
michi tune data.csv --target y --model hist-gbm --strategy bayes --save-params best.yaml

Why it's allowed — ADR-0004

The obvious argument for ("finds better hyperparameters faster") is true and insufficient — AutoML also finds better everything faster.

The argument that decides it is narrower. Ask what the optimiser has an opinion about. Not the model. Not the space. Not the metric, when to stop, or what to ship. It chooses the order in which candidates from your own space are evaluated — the same category of decision as which rows land in which fold. A mechanic, not a judgement.

The hazard is the opposite of the obvious one

A model-based search is better at overfitting the inner folds. That's the mechanism working, not failing: it spends its budget near whatever looked good on the inner split, including where that split was lucky.

So the optimism gap grows with the optimiser's strength. Measured on the example dataset:

Strategy	Gain over defaults	Optimism gap
random	0.02019	0.00519
bayes	0.02303	0.00828

It found the better configuration and flattered itself more — exactly as the ADR reasoned. A tool reporting the inner score gets more wrong the better its search gets, which is the worst possible direction for an error to move. michi already printed both numbers adjacently; that decision is now load-bearing rather than merely correct.

Five constraints, each tested

1. The space stays printable (--list-space) and the optimiser may not widen it
2. Nested scoring is not configurable — there's no flag to report the flattering number
3. The optimism gap is always shown
4. Reproducible under a seed
5. A missing dependency names the extra and never falls back silently — asking for bayes and getting random would compare two runs that were never the same experiment

Fixed

Counting evaluated configurations assumed sklearn's cv_results_["params"]. Optuna's wrapper has no such key — it records trials_. The first Bayesian search to finish a fold raised KeyError. Found by running it, not by a test.

Install: pip install 'komichi[bayes]'