Bug for EI?

[glouppe]

As observed in https://github.com/MechCoder/scikit-optimize/pull/14, the approximated objective when using EI is really weird. What is the issue?

[MechCoder]

I'll have a detailed look across the weekend (among other things).

[MechCoder]

I looked at this research paper (http://arxiv.org/pdf/1206.2944.pdf) and cross-checked the implementation once more and it seems correct to me. Will you be able to have a quick look?

@jmetzen It would be great if you could let me know which formula the line here (https://github.com/jmetzen/bayesian_optimization/blob/master/bayesian_optimization/acquisition_functions.py#L109) follows, It seems to be different from the one mentioned in the paper.

[MechCoder]

In particular, for the case when the std is zero, I've set the exploration term to zero. Is that right?

[jmetzen]

You can ignore self.kappa in my implementation; it defaults to zero and I keep it at this value typically.
Moreover, the sign of gamma_x depends on whether you want to minimize or maximize the objective function. The Snoek paper uses the notation for minimization as far as I remember. If you now insert the identity Φ(z) = 1/2[1 + erf(z/ 2)] and do some arithmetic, you should end up with the formula used in my implementation (hopefully ;-))

[glouppe]

@MechCoder Yeah, indeed, I just had look myself and I did not realize that EI was mostly flat everywhere. Only UCB is giving these nice figures :) Closing.

[MechCoder]

Great, that saves me the effort of reading yet another paper :P