LightGBM Random Forest

[justinormont]

LightGBM added random forest support in July 2017. ML.NET should expose this functionality.

This is exposed as another booster type. Currently we support { Tree Booster, Dropout Tree Booster, and Gradient-based One-Size Sampling } (src) boosters of LightGBM.

For our defaults for the new booster type we could copy the test:
https://github.com/Microsoft/LightGBM/blob/fb28070e1daa500b087d3102145ae48988030195/tests/python_package_test/test_engine.py#L53-L62

I don't see that LightGBM provides a default for these parameters. So we need to provide reasonable defaults for the user; without ff/bagfrac set to <1.0, LightGBM throws an error. I think ff=0.5 & bagfrac=0.5 may be a bit low; perhaps 0.7? As always benchmarks on representative datasets speak louder than I.

This adds to our existing FastTree Random Forest implementation, and is another implementation that could be use for #1729.

[yaeldekel]

Hi @justinormont , thanks for the suggestion! I am not sure if we can add new features before 1.0, but we should definitely triage this.

[Ivanidzo4ka]

I notice some work by @singlis regarding LightGBM and bringing monotonic support. Do you have time to look on random forest support?

[luisquintanilla]

Closing for now, in the meantime, you might be able to import a RF model into ML.NET and inference that way.