SHAP with TF Encoder Decoder timeseries | AssertionError

[AminDar]

Discussed in #3284

^{Originally posted by AminDar September 26, 2023}
I've been reading the issues on SHAP repository and it seems this question came up couple of times and never been solved completely.

For a LSTM seq2seq timeseries mostly a windowed dataset which is a tf tensor built to feed the data to the model with a shape of [samples, timesteps, features].

print(model.input_shape)
[(None, 168, 21), (None, 1, 21)]

print(model.output_shape)
(None, 1, 1)

As you already see the shape is not a 1D vector and therefore
e = shap.DeepExplainer(model,windowed_test)
will end up with this error:
AssertionError: The model output must be a vector or a single value!
and
shap.GradientExplainer(model,windowed_test)
will end up as:
TypeError: '_PrefetchDataset' object is not subscriptable

So given the model in my case, even when I just want to predict one output (and not a multi step output) I encountered this problem and haven't solved it yet.
The model can be defined as :

latent_dim = 16
past_inputs = tf.keras.Input(shape=(24, 5), name='past_inputs')
# Enoding the past
encoder = tf.keras.layers.LSTM(latent_dim, return_state=True)
encoder_outputs, state_h, state_c = encoder(past_inputs)
future_inputs = tf.keras.Input(shape=(1, 1), name='future_inputs')
# Combining future inputs with recurrent branch output
decoder_lstm = tf.keras.layers.LSTM(latent_dim, return_sequences=True)
x = decoder_lstm(future_inputs,
                 initial_state=[state_h, state_c])

x = tf.keras.layers.Dense(16, activation='relu')(x)
x = tf.keras.layers.Dense(16, activation='relu')(x)
output = tf.keras.layers.Dense(1, activation='relu')(x)

model = tf.keras.models.Model(inputs=[past_inputs, future_inputs], outputs=output)

and then

print(model.output_shape)
(None, 1, 1)

I've read 1, 2, 3
but none of them solved my issue or even the original problem.
Specially when you look at the SHAP own example 4 one can immediately see that
It's not a Seq2Seq LSTM for timeseries and

x_train.shape
Out[4]: (25000, 80)

which doesn't follow the encoder/decoder LSTM windowed dataset structure used in timeseries [samples, timesteps, features].

The question is now to see if SHAP can work with encoder decoder seq2seq at all or not?

[qq492947833]

I have same problem.The shap's explainer just can received np.array or pd.dataframe,they are must be regular。If your input have two，and they have not same shape，it can't be convert to np.array or pd.dataframe,and the model will give you a error:data can't be list！

[CloseChoice]

I am actively working on this, see #3419. Would be great if you could provide an example including the generation of the data so that I can include this as test in the PR and make sure that the issue is fixed.

[qq492947833]

I am actively working on this, see #3419. Would be great if you could provide an example including the generation of the data so that I can include this as test in the PR and make sure that the issue is fixed.

I provide a example in #3343 ,if this example can help you fix this issue, I will very proud for this.Thanks again for your work!

[AminDar]

I am actively working on this, see #3419. Would be great if you could provide an example including the generation of the data so that I can include this as test in the PR and make sure that the issue is fixed.

Sorry for the late reply.

As an example we can use an online bike dataset, I'll omit any unnecessary steps to keep it clean as much as possible.

wget https://archive.ics.uci.edu/ml/machine-learning-databases/00275/Bike-Sharing-Dataset.zip unzip Bike-Sharing-Dataset.zip -d bike_data

def select_columns(df):
    cols_to_keep = [
        'cnt',
        'temp',
        'hum',
        'windspeed',
        'yr',
        'mnth',
        'hr',
        'holiday',
        'weekday',
        'workingday'
    ]
    df_subset = df[cols_to_keep]
    return df_subset

dataset=select_columns(df)

The batched and windowed data set can be created using following lines, assuming
window_len = 24 * 7 * 2
forecast_len = 24 * 4
n_total_features = len(dataset.columns)
n_aleatoric_features = 4
n_deterministic_features = n_total_features - n_aleatoric_features

def create_dataset(df, n_deterministic_features,
                   window_size, forecast_size,
                   batch_size):
    shuffle_buffer_size = len(df)
    total_size = window_size + forecast_size
    data = tf.data.Dataset.from_tensor_slices(df.values)
    data = data.window(total_size, shift=1, drop_remainder=True)
    data = data.flat_map(lambda k: k.batch(total_size))
    data = data.shuffle(shuffle_buffer_size, seed=42)
    data = data.map(lambda k: ((k[:-forecast_size],
                                k[-forecast_size:, -n_deterministic_features:]),
                               k[-forecast_size:, 0]))
    return data.batch(batch_size).prefetch(tf.data.experimental.AUTOTUNE)

create the dataset:

training_windowed = create_dataset(dataset,
                                   n_deterministic_features,
                                   window_len,
                                   forecast_len,
                                   batch_size)

check the shapes:

for i in training_windowed.take(1):
    (enc, dec_in), dec_out = i

print('All shapes are: (batch, time, features)')
print(f'Encoder inputs shape: {enc.shape}')
print(f'Decoder inputs shape: {dec_in.shape}')
print(f'Decoder outputs shape: {dec_out.shape}')
input_shape = enc.shape[1:]  # Shape of input observations tensor
output_shape = dec_in.shape[1:]  # Shape of target labels tensor

All shapes are: (batch, time, features)
Encoder inputs shape: (32, 336, 10)
Decoder inputs shape: (32, 96, 6)
Decoder outputs shape: (32, 96)

Use and encode decoder LSTM model to train:

import tensorflow as tf

latent_dim = 16
past_inputs = tf.keras.Input(
    shape=(window_len, n_total_features), name='past_inputs')
encoder = tf.keras.layers.LSTM(latent_dim, return_state=True)
encoder_outputs, state_h, state_c = encoder(past_inputs)
future_inputs = tf.keras.Input(
    shape=(forecast_len, n_deterministic_features), name='future_inputs')
decoder_lstm = tf.keras.layers.LSTM(latent_dim, return_sequences=True)
x = decoder_lstm(future_inputs,
                 initial_state=[state_h, state_c])
x = tf.keras.layers.Dense(16, activation='relu')(x)
x = tf.keras.layers.Dense(16, activation='relu')(x)
output = tf.keras.layers.Dense(1, activation='relu')(x)
model = tf.keras.models.Model(
    inputs=[past_inputs, future_inputs], outputs=output)
optimizer = tf.keras.optimizers.Adam()
loss = tf.keras.losses.Huber()
model.compile(loss=loss, optimizer=optimizer, metrics=["mae"])
hist = model.fit(training_windowed, epochs=25)

Use SHAP

import shap
shap.__version__
Out[25]: '0.42.1'

explainer = shap.DeepExplainer(model, training_windowed)

AssertionError: The model output must be a vector or a single value!