[Bug] [Important] attributions of sklearn wrapped models

[AntoninPoche]

---

name: Bug report
about: attributions of sklearn wrapped models
title: "[BUG] - attributions of sklearn wrapped models are incoherent"
labels: ''wrapper", "bug", "attributions"
assignees: ''

---

Select the modules to which the bug refers:

• Attributions Methods
• Feature Visualization
• Concepts
• Metrics
• Documentation

Describe the bug
When a wrapper is used on regression model from sklearn, the obtained attributions are not coherent. (Problem can be larger).

Screenshots

Desktop:

• OS - windows
• Python version - 3.7.12
• Sklearn - 1.0.2
• Xplique - 0.2.6

To Reproduce

import numpy as np
from numpy.random import seed, normal
import sklearn
import xplique
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split

seed(0)

# dataset parameters
features_coef = np.array([1, -2, -3, 4, 0, 0, 0, 0])
nb_samples = 1000
nb_features = len(features_coef)


# create dataset
dataset = normal(0, 1, (nb_samples, nb_features))
noise = normal(0, 1, nb_samples)
target = dataset.dot(features_coef) + noise

# split dataset
X_train, X_test, y_train, y_test = train_test_split(dataset, target, test_size=0.05, shuffle=True)


# train model
sk_linear = LinearRegression().fit(X_train, y_train)

# Create the wrapper class
class Wrapper():
    # The init method is necessary for every class
    def __init__(self, model):
        self.model = model

    # The call method calls the predict method
    def __call__(self, inputs):
        return self.model.predict(inputs)

# wrap model
sk_model = Wrapper(sk_linear)

# adapt inputs/outputs
inputs_tf = tf.cast(X_test, tf.float32)
targets_tf = tf.ones((X_test.shape[0], 1))

# instanciate explainer
explainer = KernelShap(
    model,
    nb_samples=200,  # 2000
    ref_value=0.0
)

# compute explanations
explanations = abs(explainer(inputs_tf, targets_tf))

print(np.array(explanations).mean(axis=0))

# [4.328179, 4.357149, 4.6055717, 5.554367, 3.5661576, 4.1552, 3.5590754, 4.7494626]

To ease the debugging, here is a minimal example notebook (There are several lines for the visualization, but you can jump to the end of the notebook to see the different attributions) : https://colab.research.google.com/drive/1zvt4I9lVpvzg1oWPUNZoFs39w8MPSz_b?usp=sharing

Expected behavior
The 4 last attributions values should be close to 0, far inferior to the 4 first.

Additional context
_

[AntoninPoche]

With David we found the origin of the problem and a temporary solution :

The problem comes from the shape of the prediction over one batch. In this case, if the model have only one output, the shape of the prediction will be (n_sample, ), which only have one dimension, the dimension corresponding to the dimension of the output disappear.

The temporary solution is thus to expend the dimension of the prediction in the wrapper :

# Create the wrapper class
class Wrapper():
    # The init method is necessary for every class
    def __init__(self, model):
        self.model = model

    # The call method calls the predict method
    def __call__(self, inputs):
        pred = self.model.predict(inputs)
        pred = tf.expand_dims(pred, axis=1)
        return pred

However, for me, this is only temporary as the library should be as user friendly as possible. Thus I suggest to add a few lines in the function xplique.common.callable_operations.predictions_ont_hot_callable() to verify if we are in the case of a batch were the output dimension disappeared and expand the dimension in this case.

[fel-thomas]

+1 on this.
useful for users, and I think it shouldn't be too hard for us, quick win ! ;)