BUG: base_score attribute of the XGBTreeModelLoader is broken for all exponential losses (e.g. tweedie, poisson)

[bastienqb]

Issue Description

When trying to run a TreeExplainer for a XGBoost model setup with the tweedie loss, I get an error stating that some internal self.base_score of the TreeExplainer object cannot be found.

Minimal Reproducible Example

from xgboost import XGBRegressor
import shap
import numpy as np

X, y = np.random.randn(100, 5), np.random.exponential(size=100)
model = XGBRegressor(
    objective="reg:tweedie",
)
model.fit(X, y)
explainer = shap.TreeExplainer(model)

Traceback

File .../xgboost_error.py:7
      3 model = XGBRegressor(
      4     objective="reg:tweedie"
      5 )
      6 model.fit(X, y)
----> 7 explainer = shap.TreeExplainer(model)

File .../site-packages/shap/explainers/_tree.py:195, in TreeExplainer.__init__(self, model, data, model_output, feature_perturbation, feature_names, approximate, link, linearize_link)
    193 self.feature_perturbation = feature_perturbation
    194 self.expected_value = None
--> 195 self.model = TreeEnsemble(model, self.data, self.data_missing, model_output)
    196 self.model_output = model_output
    197 #self.model_output = self.model.model_output # this allows the TreeEnsemble to translate model outputs types by how it loads the model

File .../site-packages/shap/explainers/_tree.py:1090, in TreeEnsemble.__init__(self, model, data, data_missing, model_output)
   1088 elif safe_isinstance(model, "xgboost.sklearn.XGBRegressor"):
   1089     self.original_model = model.get_booster()
-> 1090     self._set_xgboost_model_attributes(
   1091         data,
   1092         data_missing,
   1093         objective_name_map,
   1094         tree_output_name_map,
   1095     )
   1096     # Some properties of the sklearn API are passed to a DMatrix object in
   1097     # xgboost We need to make sure we do the same here - GH #3313
   1098     self._xgb_dmatrix_props = get_xgboost_dmatrix_properties(model)

File .../site-packages/shap/explainers/_tree.py:1315, in TreeEnsemble._set_xgboost_model_attributes(self, data, data_missing, objective_name_map, tree_output_name_map)
   1308 def _set_xgboost_model_attributes(
   1309     self,
   1310     data,
   1311     data_missing, objective_name_map,
   1312     tree_output_name_map,
   1313 ):
   1314     self.model_type = "xgboost"
-> 1315     loader = XGBTreeModelLoader(self.original_model)
   1317     self.trees = loader.get_trees(data=data, data_missing=data_missing)
   1318     self.base_offset = loader.base_score

File .../site-packages/shap/explainers/_tree.py:1873, in XGBTreeModelLoader.__init__(self, xgb_model)
   1864     self.base_score = scipy.special.logit(base_score)
   1865 elif self.name_obj in (
   1866     "reg:gamma",
   1867     "reg:tweedie",
   (...)
   1871 ):
   1872     # exp family
-> 1873     self.base_score = np.log(self.base_score)
   1874 else:
   1875     self.base_score = base_score

Expected Behavior

The base_score attribute should be initialised properly with no error.

The line 1873 should be self.base_score = np.log(base_score) without referring to self.

Bug report checklist

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest release of shap.
• I have confirmed this bug exists on the master branch of shap.
• I'd be interested in making a PR to fix this bug

Installed Versions

shap==0.45.0

[chandubhujang]

This issue also exists for objective="count:poisson"

[Vijesh2]

using xgboost (2.0.3) and shap (0.45), I get the following error trace. It seems like it might be related to the same issue.

AttributeError Traceback (most recent call last)
Cell In[32], line 2
1 # Create the SHAP explainer with the model
----> 2 explainer = shap.TreeExplainer(fit_xgb)
4 # Compute SHAP values for the training set
5 shap_values = explainer.shap_values(preprocessed_data)

File c:\n\venv\Lib\site-packages\shap\explainers_tree.py:195, in TreeExplainer.init(self, model, data, model_output, feature_perturbation, feature_names, approximate, link, linearize_link)
193 self.feature_perturbation = feature_perturbation
194 self.expected_value = None
–> 195 self.model = TreeEnsemble(model, self.data, self.data_missing, model_output)
196 self.model_output = model_output
197 #self.model_output = self.model.model_output # this allows the TreeEnsemble to translate model outputs types by how it loads the model

File c:\n\venv\Lib\site-packages\shap\explainers_tree.py:1063, in TreeEnsemble.init(self, model, data, data_missing, model_output)
1061 elif safe_isinstance(model, “xgboost.core.Booster”):
1062 self.original_model = model
-> 1063 self._set_xgboost_model_attributes(
1064 data,
1065 data_missing,
1066 objective_name_map,
1067 tree_output_name_map,
1068 )
1069 elif safe_isinstance(model, “xgboost.sklearn.XGBClassifier”):
…
-> 1873 self.base_score = np.log(self.base_score)
1874 else:
1875 self.base_score = base_score

AttributeError: ‘XGBTreeModelLoader’ object has no attribute ‘base_score’

[couetilj]

I am having the same bug for v0.45.0 using the cox loss. Downgrading to v0.44.1 is a temporary solution.

[MelanWang]

Same issue here shap == 0.45.0, pls fix it ASAP! really need it to explain a survival:aft model