Feature importance from random forest regression

[fipelle]

I am trying to extract the ranking implied by a random forest regression estimated with DecisionTree.jl. However, I cannot find an example on the documentation for that. Could you please clarify how to do it?

[rikhuijzer]

You mean that the predictions are converted to ranks? In other words, take the predictions (value between -Inf and Inf), sort the predictions by value, and replace the prediction values by their index in the the sorted table?

[fipelle]

No, I mean computing a measure of feature importance. See for instance this example in a classification setting. I would like to do something similar with a random forest regressor.

[rikhuijzer]

Shapley.jl should work for that.

Maybe you can also use LightGBM.jl, by the way. It generally has a better predictive performance. I wrote a tutorial about it at https://huijzer.xyz/posts/shapley/.

If you can model your problem as binary classification, I can highly recommend SIRUS.jl. SIRUS.jl has a slightly lower predictive performance than LightGBM and the Random Forest regression from DecisionTree.jl, but is way more explainable. Regression is technically possible with the SIRUS algorithm too, but unfortunately I haven't had the time to implement that yet (should take at most a few days).

[rikhuijzer]

Feature importances are also implemented inside DecisionTree.jl: https://github.com/JuliaAI/DecisionTree.jl/blob/dev/README.md#feature-importances.

I think we can close this issue?

[fipelle]

Thanks! Sorry I must have missed it