Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
[MRG] FIX gradient boosting with sklearn estimator as init #12436
Gradient Boosting used to fail when init was a sklearn estimator, which is a bit ironic :)
Edit Also accept initial estimator which does not support sample weights as long as the gradient boosting is not fitted with sample weights
rth left a comment
I'm not too familiar with GB code, but this looks good to me except for a minor comment below.
You also need to repeat "close" or "fix" in the description for every issue you want to auto-close: i.e.
I can confirm the added test fails on master.
Maybe @glemaitre would be able to do another review..
changed the title
[MRG] FIX gradient boosting with sklearn estimator as init
Oct 23, 2018
By "this" I mean "a non-constant init estimator". So far in our implementation init estimators were always constant. After this patch they will not be.
Thinking about this again, I don't think it makes any sense to have an init predictor in a classification setting (It's OK for regression).
This is not a problem in regression because
I would propose to either:
Slightly related: lots of the docstrings actually state that
Hope it's clear... I can try to clarify if needed
That would be slightly less inappropriate than using
In binary classification:
My proposition is to replace the constant
It does not make sense to set
I propose the following:
Each loss function should have a
For the binary classification loss, this would look like
while for least squares this would be
The built-in init estimators like
It would be similar to what we do in pygbm, except that pygbm does not accept custom init estimators.
@jeremiedbb you probably didn't sign up for this so if you want, I can take it up. I was planning on refactoring the GB code anyway (putting losses in another file, renaming
I don't think so because