Choice of length_scale in GP kernel - wow!

[bwheelz36]

Hi all,
this isn't so much an issue as an observation that I hope will be helpful for other coming here.
After playing around with this (btw very nicely written and useful) code a lot, I finally decided to get my hands dirty and start fiddling with the choice of kernel using
optimizer.set_gp_params(kernel=...)

I basically cyled through all the Kernel options described here (see section 1.7.5 a bit down the page). not many of them made much of a difference to my problem. Also changing the default Matern Kernel nu value didn't make much difference at all.

I was ready to conclude that the default Kernel settings were indeed about as good as one can get, when I decided to experiment with the length_scale parameter. An explanation of that which even I sort of understood is here. So, I plugged in an array for length scale, one for each of my variables (based on my estimate of their various sensitivities):

optimizer.set_gp_params(normalize_y=True, kernel=sklearn.gaussian_process.kernels.Matern(length_scale=[1,50,0.2]),
                                n_restarts_optimizer=20) 

This really blew me away. After spending about 2 hours fiddling with Kernels, it took me about 10 seconds to estimate some length scales and it made a massive difference to the ability of the gaussian process model to predict the objective function.

So just a few questions/suggestions:

• At the moment, length_scales are not really mentioned anywhere in the documentation or examples. In fact, my impression from reading the documentation is that only really hard core users would ever want to touch the Kernel parameter. However, my experience suggests that these can make a massive difference. So should this be updated?
• At least in my case, even doing something as simple as setting the length_scales to 10% of the range (max-min) of each variable worked incredibly well. So should something like this be the default behavior, or at least an option?