XGBoostLSSRegressor - Scikit Learn API

[joshdunnlime]

Is there an appetite to add a scikit learn API for this? If so, very happy to help contrib. Also for LightGBMLSS.

[StatMixedML]

@joshdunnlime Thanks for the interest in the project.

An extension to scikit-learn or sktime is on the to-do, but I won't find the time for that. Very happy if you volunteer for this!

[joshdunnlime]

Amazing. I'm very happy to start looking into and hopefully start a PR. It would be good to get some input on naming conventions and over all design when I get there.

[StatMixedML]

It would be good to get some input on naming conventions and over all design when I get there.

Sure let's align on this.

[kmedved]

I just want to add my support for this. This would dramatically increase the ease of use for this package.

[joshdunnlime]

I've made the first few commits and added the first PR #68.

Most of the code is taken from the XGBoost Sklearn package and adapted. I think it makes sense to align with that as closely as possible.

There are a few feature that I haven't preserved fully.

• Cupy support. I'm on a mac m1 so cannot test - nvidia have stopped supporting. I have no idea if this is something we wish to support in xgblss. This can easily be added in another version.
• Incremental/continuous training. Feels like much added complexity for a niche feature. Does this work with LSS training?
• validate_features. to be added.

[joshdunnlime]

Any comments or reviews on this? I plan to pick this up again this week

[StatMixedML]

Thanks for the PR. I am still on vacation and haven't had the time to look into it so far. Please continue.

[StatMixedML]

@joshdunnlime skpro is a good motivation for having a scikit learn api.

[joshdunnlime]

For sure! There are many other use cases as well. This will certainly make it easier for the wider python DS community to get their hands dirty with!

[joshdunnlime]

Funnily enough, I scheduled some time this evening to pick this back up. I've now push a new PR. I think it is close to being ready to review and merge.

[joshdunnlime]

Additionally, there is an ongoing discussion on sklearn about how probabilistic predictions are made in an sklearn context. An agreed API here would be an excellent thing to adopt but for now I have followed XGBLSS's predict methodology. scikit-learn/scikit-learn#23334

[fkiraly]

skpro proposes an API for probabilistic predictions and implements already an interface definition, with stringent tests.

It also integrates fully with sktime, the "scikit-learn-like" for time series forecasting.

Opened issue with suggestion on integration here: #69

[joshdunnlime]

+1 from me

[StatMixedML]

@joshdunnlime @fkiraly

I have added separate branches to intergrate the project to the respective frameworks:

• sklearn_integration
• sktime_integration
• skpro_integration

This allows us to have the different versions separate from the current master-branch. Once the integrations are done we can merge them to master.

@joshdunnlime Can you please move the PR to the new branch, thanks.

[joshdunnlime]

@StatMixedML my understanding is that simply adding the sklearn API will allow skpro to utilise xgblss (with some small changes on the skpro side). This in turn will allow sktime to use xgblss.

Most of the work is to add missing xgblss distributions in skpro/sktime side as detailed in #69. In the meantime, any not implemented could simply have a NotImplemented error. The other thing to consider here is pytorch vs tensorflow distributions and normalising flows in tensorflow.

In addition, the xgblss native predict api could be extended to accommodate the predict_probab etc methods.

I will shift my branch to the new branch and add a PR.