Problem
Studies that gate on a "regime" factor (e.g. data-size buckets: small / medium / large) currently optimize each regime independently — every regime gets its own grid, its own pareto front, and there is no information sharing across regimes. This has two costs:
- Wasted information. Settings that work well at
n=1k are usually informative priors for n=3k, but we can't borrow strength.
- Hard bucket boundaries. Real data sizes are continuous, but the user has to pick discrete buckets and re-run if a new size shows up between buckets.
A regime-conditional surrogate would let us interpolate recommended settings across regime features instead of relying on hard gates.
Proposed
Build on top of #82 (general GP / RF surrogate from a ResultsTable) with a regime-aware variant:
- Treat regime descriptors (e.g.
n_samples, noise_level, class_balance) as additional input dimensions of the surrogate, alongside the model factors.
fit_regime_surrogate(results, regime_features=[...], factors=[...], targets=[...]) -> RegimeSurrogateRegimeSurrogate.recommend(regime: dict, *, objective, mode="min") -> dict — optimize factors at a query regime by minimizing the surrogate's predicted objective (with optional acquisition for GP).RegimeSurrogate.predict(regime, factors) -> dict[str, float] for inspection / plotting.- Plot helpers: factor recommendation vs. regime feature, with uncertainty bands (GP).
Example
surrogate = fit_regime_surrogate(
results,
regime_features=["n_samples", "noise"],
factors=["lr", "patience", "active_criteria"],
targets=["val_loss", "wall_time"],
method="gp",
)
best = surrogate.recommend({"n_samples": 2200, "noise": 0.1}, objective="val_loss")Notes
Problem
Studies that gate on a "regime" factor (e.g. data-size buckets:
small / medium / large) currently optimize each regime independently — every regime gets its own grid, its own pareto front, and there is no information sharing across regimes. This has two costs:n=1kare usually informative priors forn=3k, but we can't borrow strength.A regime-conditional surrogate would let us interpolate recommended settings across regime features instead of relying on hard gates.
Proposed
Build on top of #82 (general GP / RF surrogate from a
ResultsTable) with a regime-aware variant:n_samples,noise_level,class_balance) as additional input dimensions of the surrogate, alongside the model factors.fit_regime_surrogate(results, regime_features=[...], factors=[...], targets=[...]) -> RegimeSurrogateRegimeSurrogate.recommend(regime: dict, *, objective, mode="min") -> dict— optimize factors at a query regime by minimizing the surrogate's predicted objective (with optional acquisition for GP).RegimeSurrogate.predict(regime, factors) -> dict[str, float]for inspection / plotting.Example
Notes
active_criteria) need one-hot or kernel handling — RF handles this cleanly, GP needs a mixed kernel.trade-study[surrogate]extra.