add base implementation of optimum in OptimizerInterface

[amueller]

This adds an implementation of "optimum" to the BayesianOptimizerProxy based on the public get_all_observations.

I'm not sure how @byte-sculptor feels about adding implementations to the interface class. This implementation could probably be shared by all derived classes so this felt like a natural place.

Another concern with the current trivial implementation is that right now it adds a copy of the data in BayesianOptimizer. That could be easily addressed by adding a allow_view=False argument to get_all_observations to not make the copy unless it's required.

Obviously still missing tests

[bpkroth]

This adds an implementation of "optimum" to the BayesianOptimizerProxy based on the public get_all_observations.

I'm not sure how @byte-sculptor feels about adding implementations to the interface class. This implementation could probably be shared by all derived classes so this felt like a natural place.

Depends on whether the language will accept it, but I'm generally okay with that.
Other option is to define a base class that by convention everything inherits from to provide your default implementations, but that seems like unnecessary extra classes to me, so avoid it if you can?

Another concern with the current trivial implementation is that right now it adds a copy of the data in BayesianOptimizer. That could be easily addressed by adding a allow_view=False argument to get_all_observations to not make the copy unless it's required.

👍

Only question I have is what the default should be. How often do you need a copy, and what are the potential damages by not always returning a copy by default?

Obviously still missing tests

[bpkroth]

For the record, I believe this is to address issue #71, which one of the students reported.

[byte-sculptor]

Looks, good. I think this class is the best place for the implementation, though it should never have been named an OptimizerInterface. So if it's not too much work to rename the class to OptimzierBase, I think that would be a nice improvement.

[amueller]

Only question I have is what the default should be. How often do you need a copy, and what are the potential damages by not always returning a copy by default?

I think the potential danger is having the copy overhead. I assume that to be negligible, based on no data at all. If you call "optimum" in a tight loop then the copy might become a bottleneck, but I would call the premature optimization.

[bpkroth]

Only question I have is what the default should be. How often do you need a copy, and what are the potential damages by not always returning a copy by default?

I think the potential danger is having the copy overhead. I assume that to be negligible, based on no data at all. If you call "optimum" in a tight loop then the copy might become a bottleneck, but I would call the premature optimization.

Oh, my fear was the other way. By returning the reference, not a copy, then you might risk unintended modification to the internals.

I think the risk of it being slow would be somehow more obvious and benign (it just takes longer, not we've accidentally altered the internal data which is harder to track down).

[amueller]

Oh yeah I totally agree, the overwriting is the much bigger risk. The current implementation basically requires get_all_observations to return a copy which should be super safe, though the responsibility of being safe is on get_all_observations (which should be reasonably easy to test).

[bpkroth]

[1] seems like the more common value to be asking about, maybe it should be returned first?
Also, named values/properties seems maybe better.

[amueller]

having configuration, optimum makes more sense to me, and I think actually the configuration is the more interesting one, but it's not used in the tests right now because it might not have been available before?

[byte-sculptor]

We can always log both :)

[byte-sculptor]

Also, are there any invariants we can assert about the optima in this test and in the tests below? Specifically:

1. That it is in fact the optimum
2. That the config belongs to the parameter space
3. That the optimum belongs to the objective space
4. That the optimum is monotonically decreasing like you did in one of the tests below.
5. Anything else you can think of

[amueller]

I'm happy to add some of these but I feel that's mostly unrelated to this PR, right? More testing is always good but I'd love to push this to the students.

[amueller]

had to resolve some merge conflicts so please check again.

[bpkroth]

LGTM. Thanks!