ConstantKernel
#2511
ConstantKernel
#2511
Conversation
SebastianAment
force-pushed
the
constant_kernel
branch
5 times, most recently
from
027fc30
to
2e65b14
Compare
| @@ -82,6 +82,7 @@ def find_version(*file_paths): | |||
| "nbclient<=0.7.3", | |||
| "nbformat<=5.8.0", | |||
| "nbsphinx<=0.9.1", | |||
| "lxml_html_clean", | |||
There was a problem hiding this comment.
Orthogonal dependency fix, see e.g. https://github.com/assafelovic/gpt-researcher/pull/424/files for a similar fix.
There was a problem hiding this comment.
Hmm that link doesn't really have much information on what exactly this fixes / why this dependency is needed. Is there more info on this?
There was a problem hiding this comment.
There was a change in some package required for the sphinx compilation recently, which outsourced part of the functionality into this new package (there's more info in the description tab of the linked PR).
The doc compilation will fail for any upcoming commit without this, so this is independent of the other changes made here.
There was a problem hiding this comment.
Copying over the error message from the link, which was raised on this commit as well before I added the line:
ImportError: lxml.html.clean module is now a separate project lxml_html_clean.
Install lxml[html_clean] or lxml_html_clean directly.
There was a problem hiding this comment.
lgtm, thanks!
FYI @gpleiss @jacobrgardner - IIRC we had a constant kernel in the past but removed it. @SebastianAment's results on the numerical issues in the MAP estimation due to the unregularized kernel are interesting and suggest that using such a kernel may be preferable to using a ConstantMean.
| @@ -82,6 +82,7 @@ def find_version(*file_paths): | |||
| "nbclient<=0.7.3", | |||
| "nbformat<=5.8.0", | |||
| "nbsphinx<=0.9.1", | |||
| "lxml_html_clean", | |||
There was a problem hiding this comment.
Hmm that link doesn't really have much information on what exactly this fixes / why this dependency is needed. Is there more info on this?
SebastianAment
force-pushed
the
constant_kernel
branch
from
2e65b14
to
eeb21c1
Compare
Head branch was pushed to by a user without write access
SebastianAment
force-pushed
the
constant_kernel
branch
from
eeb21c1
to
3e990b3
Compare
SebastianAment
force-pushed
the
constant_kernel
branch
from
3e990b3
to
89dfb46
Compare
This commit adds
ConstantKernel, moving the inference of additive constants into the kernel.Status quo: By default both GPyTorch and BoTorch are currently using a
ConstantMeanthat is inferred during the optimization of the marginal likelihood. Because the mean parameter does not show up in the log determinant, it is un-regularized, which can lead to some counter-intuitive values for the constant mean, especially when the other parameters are constrained or subject to a non-trivial prior.The inference of the posterior mean of the constant using the kernel approach is carried out with the usual linear algebraic formulation, which is more robust. See the following for an illustrative example of what can go wrong with the current
ConstantMeanapproach, and how theRBF + Constantkernel behaves as expected.