Next point suggestion with mixed-integer optimisation

[udicaprio]

Hi all,

I see that mixed-integer optimisation has been allowed in the library. This is a very good improvement. Looking at the code, I see that the suggestion of the next point still happens with L-BFGS-B, which is meant for continuous optimisation. Is that the case? Or is there a part of the code I missed?

If so, I propose to implement something able to an optimiser able to deal with mixed-integer. I propose applying the differential evolution with L-BFGS-B refinement on the continuous variables algorithm that is implemented in scipy. However, I am open to suggestions.

Is this a wanted feature for this project? If so, I can work on the implementation.

[till-m]

Hi @udicaprio,

good catch! However, the L-BFGSB optimization is indeed only over continuous parameters, see the relevant lines starting roughly here.
In short, it does L-BFGS-B optimization over the continuous dimensions, starting from the n best randomly sampled points. This might not be the ideal way of doing it, in the paper they don't really discuss the optimization of the acquisition function very much. In my (very limited) experiments, this worked reasonably well, but I did feel it could be a bit finicky. Your approach sounds like it could be significantly more stable.

In this sense: I think this could really improve the library!

[till-m]

If you do decide to work on this, I think it would make sense to hold off on the full 3.0 release for now, depending on how long you think it will take you (no pressure though).

[udicaprio]

Hi @till-m,

Thank you. I will be working on this in the coming weeks.
What is the deadline for the full release of 3.0?

[till-m]

Hey @udicaprio,

nothing fixed, really. A few weeks is great honestly!

[udicaprio]

The pull request implementing these functions has been merged