Numeric and Log-Scale Choice

[wistuba]

There is no equivalent of choice for numeric values. E.g., in the FCNet blackbox the learning rate is defined as 'hp_init_lr': choice([0.0005, 0.001, 0.005, 0.01, 0.05, 0.1]). This will not allow model-based approaches to encode this hyperparameter correctly. Would be great to identify them as numeric and also indicate whether log transform is needed.

[geoalgo]

We have an equivalent that works for regular ranges (e.g. [0.0005, 0.001, 0.002, 0.004]), since finrange and logfinrange allows to encodes finite value ranges correctly.

What is not supported is non regular range (as the one you give as an example). That being said, it is also not clear to me how frequent will that use-case be and how impactful it would be in term of final performance.

[mseeger]

Hi guys, with the forthcoming PR on DEHB, I'll also introduce an "ordinal" type which provides what Martin is asking for. This will be choice, but with an integer encoding internally. I can also split this out if that is simpler.

[mseeger]

However, "ordinal" will not support a log transform. For that, please use logfinrange and use a regular stepsize in log domain.

[mseeger]

Actually, this is already in there: #277 . Martin, let me know if this is what you need.

[mseeger]

To be clear: ordinal([0.0005, 0.001, 0.005, 0.01, 0.05, 0.1]) uses an int encoding (0 to 5), but this is not aware of numerical values,

[wistuba]

I would still need the original values. how about choice where we can set int or log?

[mseeger]

Hmm. ordinal is really just mapping the list of values (say: ['A', 'B', 'C']) to int (say, [0, 1, 2]), the values do not have to be numbers.

Maybe Martin has something more intesting in mind, in which case maybe he wants to change 'ordinal' in the first place?

[wistuba]

I simply want to get the same hyperparameter representation for choice([0.0005, 0.001, 0.005, 0.01, 0.05, 0.1]) as for log_uniform(0.0005, 0.1).

[mseeger]

I think what you have in mind is something like [0, 1] is partitioned into 6 intervals of different sizes (even after log transform). Sampling is from U[0,1], and you map it to a value by checking in which interval you land.

This is not supported right now, at least not for arbitrary increasing values.

[wistuba]

this wouldn't work with surrogate benchmarks, would it?