Feature: Remove narrow_space_from_best_guess function for HyperparameterSpace, and find alternative to nullify

[vincent-antaki]

HyperparameterSpace has two function which aren't used, or tested, anywhere (nullify and narrow_space_from_best_guess). I believe they may be remnant of old hyperparameter research strategy. In my humble opinion, they should be removed. or at the very least moved to a class which implement the hyperparam research strategy interface.

[guillaume-chevalier]

Nice catch. narrow_space_from_best_guess should be removed without further questionning. Note that it is explained in some documentation example notebooks on the sphinx site, so the according documentation shall be removed as well upon doing this.

For the nullify method, I think that despite this method may seem available, that it may not work properly already. If done correctly, nullifying could be an indicator for the TPE of which Optional(step...) variables to ignore (or consider) for certain trials into how it accounted, or not, in the score obtained. Other hyperparameter selection strategies we would further code could make use of nullified params as well.

If we remove the idea of nullifying some params, we need a replacement for indicating which parameters are nullified from optionals. Perhaps a new apply method like pipeline.get_nullified_hyperparams() which only optionals would implement, and which would be useable by (or sent to) the TPE ?

@vincent-antaki

[guillaume-chevalier]

Related: #254