BUG: SHAP doesn't work with PySpark loaded models

[mriomoreno]

Issue Description

When you load a pyspark model and the you fit treeExplainer you get the following error:

AssertionError: The background dataset you provided does not cover all the leaves in the model, so TreeExplainer cannot run with the feature_perturbation=\"tree_path_dependent\" option! Try providing a larger background dataset, or using feature_perturbation=\"interventional\"

Minimal Reproducible Example

from pyspark.ml.classification import GBTClassificationModel
loaded_model = GBTClassificationModel.load("path_to_model")

import shap
explainer = shap.TreeExplainer(loaded_model )

explainer.shap_values(df)

Traceback

AssertionError: The background dataset you provided does not cover all the leaves in the model, so TreeExplainer cannot run with the feature_perturbation=\"tree_path_dependent\" option! Try providing a larger background dataset, or using feature_perturbation=\"interventional\"

Expected Behavior

#2700 This PR should fix the problem.

@CloseChoice, @thatlittleboy , @slundberg could anyone merge the PR?

Thanks!

Bug report checklist

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest release of shap.
• I have confirmed this bug exists on the master branch of shap.
• I'd be interested in making a PR to fix this bug

Installed Versions

latest