hyperparameter constraint

[better629]

Hi，
Usually, params in hyper space are independent, but there are also constraint between different params. Like, param_a = 'x' while param_b can't be 'y'. Does the turbo support this?

Thank you!

[mbway]

imposing constraints like this essentially is preventing samples from being taken in regions of the parameter space where the constraints are violated. Note that Bayesian optimisation cannot properly deal with discrete parameters, so I'm assuming that in your example you intended the constraints to apply to ranges of values rather than single discrete values.

turbo doesn't support such constraints. The closest thing I can think of is 'Safe Bayesian optimisation', where constraints are used to prevent parameters that could be dangerous in some way, but the technique could be used to apply arbitrary constraints as well. Safe Bayesian optimisation might be overly complex though as I think it is designed for situations where the constraints aren't known beforehand and have to be discovered.

There are probably several ways that you could modify turbo to introduce constraints if you wanted. Perhaps you could create a new surrogate model which is a mixture of a regular model like a GP and a function which is 1 everywhere except for in the bad regions of the parameter space and smoothly falls to 0 inside those regions. That way the optimisation will never want to sample in those regions.