Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
[MRG+2] Faster Gradient Boosting Decision Trees with binned features #12807
This PR proposes a new implementation for Gradient Boosting Decision Trees. This isn't meant to be a replacement of the current sklearn implementation but rather an addition.
This addresses the second bullet point from #8231.
Algorithm details and refs
The main differences with the current sklearn implementation are:
Notes to reviewers
This is going to be a lot of work to review, so please feel free to tell me if there's anything I can do / add that could ease reviewing.
Here's a list of things that probably need to be discussed at some point or that are worth pointing out.
API differences with current implementation:
Happy to discuss these points of course. In general I tried to match the parameters names with those of the current GBDTs.
Changed parameters and attributes:
Unsupported parameters and attributes:
Future improvement, for later PRs (no specific order):
Done on my laptop, intel i5 7th Gen, 4 cores, 8GB Ram.
Comparison between proposed PR and current estimators:
on binary classification only, I don't think it's really needed to do more since the performance difference is striking. Note that for larger sample sizes the current estimators simply cannot run because of the sorting step that never terminates. I don't provide the benchmark code, it's exactly the same as that of
Comparison between proposed PR and LightGBM / XGBoost:
referenced this pull request
Jan 2, 2019
Thanks a lot @glemaitre ! (any idea why this is working?)
@ogrisel , if you are you OK with the changes made to
coverage run is using is looking up it's configuration file by default from
@NicolasHug I fixed the coverage thingy. Reading the coverage report, there is a bunch of things that we could improve with respect to test coverage. But this is experimental and I don't want to delay the merge further. Let's merge once my last commit is green on CI.