How many hyperparams can open box can handle?

[yuzizbk]

Hi, if there are hundreds of hyperparams, can I use open-box to get a good result? And is there a empirical relationship between run times and number of hyperparams?

[thomas-young-2013]

Given a higher-dimensional search space (with more hyperparameter to optimize), hyperparameter tuning tools (e.g., openbox) need more trials to reach a good result.

[jhj0411jhj]

Please try to use probabilistic random forest as surrogate model (set surrogate_type='prf') to deal with high-dimensional problems (with hundreds of hps). Prf is also faster than other surrogates such as GP. In this paper[1], prf performs best in Auto-WEKA problem with 786 hps.

Other methods to tackle high-dimensional problems are under developing.

Ref:
[1] K. Eggensperger, M. Feurer, F. Hutter, J. Bergstra, J. Snoek, H. Hoos, and K. Leyton-Brown. Towards an empirical foundation for assessing bayesian optimization of hyperparameters. In NIPS workshop on Bayesian Optimization in Theory and Practice, 2013.

[yuzizbk]

Thanks for your suggesttions.