BUG: Strange base value of exact explainer

[mayer79]

Issue Description

I expect the baseline of the exact explainer to equal the average prediction on the background data (masker). However, there are cases where this is not true, see example.

Minimal Reproducible Example

import numpy as np
import pandas as pd
import shap

n = 101
x = np.arange(n)
X = pd.DataFrame(dict(x1=x, x2=np.flip(x)))
print(X.x1.mean())  # 50
X.head()

def true_model(X):
    return X.x1

ex = shap.Explainer(true_model, masker=X, algorithm="exact")

ps = ex(X[0:3])
ps

# Output
#.values =
#array([[-50.06,   0.  ],
#       [-49.06,   0.  ],
#       [-48.06,   0.  ]])
#
#.base_values =
#array([50.06, 50.06, 50.06])
#
#.data =
#array([[  0, 100],
#       [  1,  99],
#       [  2,  98]])

Expected Behavior

The average of 0, ..., 100 is 50, so I'd expect the baseline to equal 50.

Running above example with n = 100 gives a baseline of 49.5 (as expected).

Bug report checklist

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest release of shap.
• I have confirmed this bug exists on the master branch of shap.
• I'd be interested in making a PR to fix this bug

Installed Versions

0.42.1

[znacer]

Hi,
This seems to be caused by a hard coded limitation of the masker size.
The class handling Tabular maskers has a limitation (max_samples) set to 100. Above this limit, only a subsample of the masker dataset is considered.

A possible fix could be to add a property max_samples to Explainer make it possible for users to tune this limitation.

[mayer79]

You are right, it affects even linear explainers.

[CloseChoice]

@znacer I implemented your proposal. That sounds like a good approach to me.

[mayer79]

Awesome, thanks @znacer and @CloseChoice

[connortann]

Hi, This seems to be caused by a hard coded limitation of the masker size. The class handling Tabular maskers has a limitation (max_samples) set to 100. Above this limit, only a subsample of the masker dataset is considered.

A possible fix could be to add a property max_samples to Explainer make it possible for users to tune this limitation.

The explainer object already accepts a masker, so I think it would be preferable to pass in a masker with the desired number of samples rather than expose more masker params in the Explainer class.

What do you think, would this be acceptable?

ex = shap.Explainer(true_model, masker=maskers.Independent(X, max_samples=1000), algorithm="exact")

[mayer79]

Good idea, that would be quite elegant, indeed.

[connortann]

What do you think @CloseChoice , happy for me to close this issue for now with the approach above as a recommended way to set the masker params?

[CloseChoice]

@connortann, that's fine for me