[WIP] GBTRegressor does not provide uncertainty estimates

[betatim]

Started a simple wrapper for regressors like GBTRegressor to provide uncertainty estatimates as well
as central predictions.

Any ideas on how to test that this indeed gives the central and 68% confidence interval (right terminology??) predictions? Best idea so far is to use a very large number of samples on a simple problem and check that the predicted standard deviation is approximately equal to the std used when sampling.

Todo:

• documentation
• check that GP uses same definition of uncertainty
• smarter tests

[betatim]

should pass in rng/use check_random_state

[betatim]

use random_state pattern instead of relying on global rng

[glouppe]

I am not fond of the nomenclature, std != uncertainty != quantile.

[betatim]

I'm flexible, but out of ideas at the moment for a good (short) name.

GBTQuantileRegressor?

GBTQuantiles(quantiles=list_of_quantiles) with per_quantile_prediction = predict(). You set all the quantiles you want estimated and we return each single one? In this case GBTQuantiles([0.18, 0.5, 0.84])

[MechCoder]

I'm planning to review my GradientBoosting know-how, this weekend. Can have a look after that.

[betatim]

If you need this merge it, I'm away till Monday night without internet

[MechCoder]

return self

[MechCoder]

@glouppe @betatim
How do we compute the standard deviation of the predictions given the quantiles? There is no assumption about the conditional distribution of Y given X right?

[betatim]

The last item on the to do list: "check definition of uncertainty" is something I'd like to punt to #23, instead of waiting to merge this till we converge.

[MechCoder]

This looks ok to me.

[betatim]

📈

[glouppe]

Would be good to expose the base_estimator in this newly introduced class. Default params of GradientBoostingRegressor are very likely to give poor results.