int_to_unit and float_to_unit break if max == min

[JesseLivezey]

In spearmint/tasks/base_task.py, the functions int_to_unit and float_to_unit assume that vmax > vmin. When running experiments with trivial parameter ranges (vmax == vmin), this breaks when I think the reasonable behavior is to just always choose the one value.

Spearmint/spearmint/tasks/base_task.py

Line 372 in 92d6b5d

def int_to_unit(self, v, vmin, vmax):

[JasperSnoek]

I think it is silly to have the code break when it would be much better to return an error message. However, I can't think of a good reason why anyone would set vmax == vmin besides by mistake. Perhaps we should just add a helpful error message here?

[JesseLivezey]

I don't have a good reason, but I do have a bad reason! (laziness)

One example would be that I'm trying to optimize multiple hyperparameters at once. Later, I want to run the same optimization, but fix one of the parameters rather than optimizing over it. If I can set vmin == vmax and just get the trivial result for that parameter, then the rest of the code can stay the same. If I have to remove it from the list of hyperparameters, I then have to insert the parameter somewhere else in the code which is just more work for me.

I tried hacking it in by having a check here:

Spearmint/spearmint/tasks/base_task.py

Line 375 in 3b0f241

unit = (np.double(v) - vmin) / (vmax - vmin)

and then setting unit to zero if vmin == vmax. This runs, but I don't understand the code well enough to know if this will break any of the GP stuff down the line. Do you think something like this would work?

I can make a PR with at least a better warning.

[JasperSnoek]

Hi Jesse, yeah I understand that. It would require building in some logic
to have those variables bypass the GP, since otherwise there would be
unnecessary computation happening (in computing covariances between these
zeros). I think your trick would work though, so you can do it in your own
branch if you like.

I'd rather leave that kind of bookkeeping up to the user than try to build
it in to Spearmint (or perhaps a user made wrapper around Spearmint to do
neat user-friendly bookkeeping, etc.) @mgelbart, what do you think?

Definitely a better warning would be great :-) I added a more useful
assert. Thanks!

Jasper

On Mon, May 11, 2015 at 7:24 PM, Jesse Livezey notifications@github.com
wrote:

I don't have a good reason, but I do have a bad reason! (laziness)

One example would be that I'm trying to optimize multiple hyperparameters
at once. Later, I want to run the same optimization, but fix one of the
parameters rather than optimizing over it. If I can set vmin == vmax and
just get the trivial result for that parameter, then the rest of the code
can stay the same. If I have to remove it from the list of hyperparameters,
I then have to insert the parameter somewhere else in the code which is
just more work for me.

I tried hacking it in by having a check here:

Spearmint/spearmint/tasks/base_task.py

Line 375 in 3b0f241

unit = (np.double(v) - vmin) / (vmax - vmin)

and then setting unit to zero if vmin == vmax. This runs, but I don't
understand the code well enough to know if this will break any of the GP
stuff down the line. Do you think something like this would work?

I can make a PR with at least a better warning.

—
Reply to this email directly or view it on GitHub
#17 (comment).

[mgelbart]

I think if you remove the parameter from the config entirely then everything should be ok, but as you said your code would need to know that this parameter is not coming in. I agree with Jasper's assessment.

Mike

[JesseLivezey]

Thanks for the feedback!

I'm probably also going to write something that just filters out parameters with trivial ranges and adds them back in at the end. Once I get it working, I'll post it here in case it's useful for others.

[JasperSnoek]

That sounds great!

Cheers,

Jasper

On Wed, Jun 3, 2015 at 8:23 PM, Jesse Livezey notifications@github.com
wrote:

Thanks for the feedback!

I'm probably also going to write something that just filters out
parameters with trivial ranges and adds them back in at the end. Once I get
it working, I'll post it here in case it's useful for others.

—
Reply to this email directly or view it on GitHub
#17 (comment).