NoneType object has no attribute '_keras_shape'

[alewarne]

The innvestigate analyzer is not able to analyze the following network (created with keras). I suppose that it is related to the Embedding Layer from keras.

from keras import Sequential
from keras.layers import Dense, Conv1D, Embedding, GlobalMaxPooling1D
import numpy as np
import innvestigate

model = Sequential()
model.add(Embedding(input_dim=219, output_dim=8))
model.add(Conv1D(filters=64, kernel_size=8, padding='valid', activation='relu'))
model.add(GlobalMaxPooling1D())
model.add(Dense(16, activation='relu'))
model.add(Dense(2, activation=None))

#test
model.predict(np.random.randint(1,219, (1,100)))  # [[0.04913538 0.04234646]]

analyzer = innvestigate.create_analyzer('lrp.epsilon', model, neuron_selection_mode=max_activation, **{'epsilon': 1})
analyzer.analyze(np.random.randint(1, 219, (1,100)))

I get the following error when trying to execute the above:

Traceback (most recent call last):
  File "test.py", line 357, in <module>
    analyzer.analyze(np.random.randint(1,218, (1,100)))
  File "/tensorflow/lib/python3.6/site-packages/innvestigate/analyzer/base.py", line 469, in analyze
    self.create_analyzer_model()
  File "/tensorflow/lib/python3.6/site-packages/innvestigate/analyzer/base.py", line 407, in create_analyzer_model
    model, stop_analysis_at_tensors=stop_analysis_at_tensors)
  File "/tensorflow/lib/python3.6/site-packages/innvestigate/analyzer/relevance_based/relevance_analyzer.py", line 457, in _create_analysis
    return super(LRP, self)._create_analysis(*args, **kwargs)
  File "/tensorflow/lib/python3.6/site-packages/innvestigate/analyzer/base.py", line 696, in _create_analysis
    return_all_reversed_tensors=return_all_reversed_tensors)
  File "/tensorflow/lib/python3.6/site-packages/innvestigate/utils/keras/graph.py", line 888, in reverse_model
    "stop_mapping_at_tensors": local_stop_mapping_at_tensors,
  File "/home/alex/virtualEnvs/tensorflow/lib/python3.6/site-packages/innvestigate/analyzer/relevance_based/relevance_analyzer.py", line 481, in _default_reverse_mapping
    Xs, Ys, reversed_Ys, reverse_state)
  File "/tensorflow/lib/python3.6/site-packages/innvestigate/analyzer/base.py", line 589, in _gradient_reverse_mapping
    return ilayers.GradientWRT(len(Xs), mask=mask)(Xs+Ys+reversed_Ys)
  File "/tensorflow/lib/python3.6/site-packages/keras/engine/base_layer.py", line 497, in __call__
    arguments=user_kwargs)
  File "/tensorflow/lib/python3.6/site-packages/keras/engine/base_layer.py", line 565, in _add_inbound_node
    output_tensors[i]._keras_shape = output_shapes[i]
AttributeError: 'NoneType' object has no attribute '_keras_shape'

I tested with the newest version 1.0.4 of the innvestigate package.

[sebastian-lapuschkin]

Hi @DonquixoteRosinante,

based on your minimal working example and https://stats.stackexchange.com/questions/270546/how-does-keras-embedding-layer-work I have implemented a LRP resolution for Embedding layers (pull request waiting for merge)

My understanding is that the EmbeddingLayer serves as a mapping from index-type inputs (numbers, strings) into a vector space accessible to other layers of the network.
Thus, there exists a 1:1 relation between input index and mapping vector.
In terms of the basic decomposition rules of LRP, we have for an indexed input index i vocabulary-size-many outputs j, with each of those j receiving forward messages z_{ij} from only one i and each i only feeding the j of exactly one mapped vector.

The only thing remaining to do here should be the aggregation of the relevance scores of each mapping vector and their attribution to their corresponding index-type inputs i.

That being said, I am not sure how analysis methods based on computing derivatives could be supported wrt the method canonically meaningfully.

Did I get anything wrong about the way the EmbeddingLayer operates, or is anything unclear?

[alewarne]

The understanding of the Embedding layer is correct and the proposed changes solve the problem.

[amarbit]

check keras version

[fwei-aim]

Has the fix of this issue been merged? I ran into the same problem with 1.0.8 of innvestigate. Thanks.

[alewarne]

Has the fix of this issue been merged? I ran into the same problem with 1.0.8 of innvestigate. Thanks.

It seems that the changes from the feature branch did not make it into the master branch finally.

[albermax]

Hi guys,
sorry (alewarne) that the feature has not made it yet.
We are reworking the library at the moment and will then try to add it.
Cheers,
Max

[vedhas]

Has it been fixed yet? I have the latest innvestigate version (1.0.8) but I get the same error as above.

It would be great if you could release the feature branch, if it is not fully ready to be merged yet! 😉 🙂

My CNN is similar to the one in the link above (summary below).
The only differences:

1. output is 2D (for every instance) = a 1 dimensional time series of shape (instance, time_steps, 1 output).
2. the last layer is time distributed.

Layer (type)	Output Shape	Param #
input_1 (InputLayer)	(None, 1768, 37)	0
conv1d_1 (Conv1D)	(None, 1768, 1)	2961
activation_1 (Activation)	(None, 1768, 1)	0
dropout_1 (Dropout)	(None, 1768, 1)	0
time_distributed_1 (TimeDist	(None, 1768, 1)	2
activation_2 (Activation)	(None, 1768, 1)	0

With your fix, I hope to be able to:

1. Analyse how the feature values contributed to the individual output neuron (through likely LRP or PatternAttribution) given an input matrix.
2. Compute input signal that maximises any one output neuron (GuidedBackprop or PatternNet)

Thank you for your awesome work! :)

[PunitShah1988]

Hi,

I understand this post is pretty old and I am new to this Innvestigate lib.
I want to know can I use this lib on NON-Conv neural net.

I have model structure somewhat like below.

input_text = Input(shape=(1,), dtype="string")
embedding = Lambda(ELMoEmbedding, output_shape=(1024,),)(input_text)
dense3 = Dense(256, activation='relu', kernel_regularizer=keras.regularizers.l2(0.001))(embedding)
dense4 = Dense(128, activation='relu', kernel_regularizer=keras.regularizers.l2(0.001))(dense3)
pred = Dense(1, activation='sigmoid')(dense4)
model = Model(inputs=[input_text], outputs=pred)

I am using this on NLP.
I want to decompose this model in order to make something like mentioned on INNvestigate github.

Any help is highly appreciated.

Regards,
PS

[alewarne]

As mentioned by Sebastian, the main problem with these embedding layers is that they map a token to a vector and this mapping has no backpropagation rule atm - the most intuitive thing is probably to sum the relevances of each dimension to obtain the relevance of the token.

A workaround I would suggest is the following: Build a new model, that is like your old model but starts at layer dense3 (and with input_shape(1024,)). Transform your data to the embedding space using the output of the Lambda-Layer. Then, you can use lrp with the transformed data and obtain relevances for each dimension, eventually you should sum them up like mentioned above.

[PunitShah1988]

Thanks for the quick one.

So if I understood it right, I should transform data with 1024 dimension of embedding & start the NN from there.
Basically we are just removing the embedding layer. Correct ?

[alewarne]

Exactly.
Alternatively, you can dive into the code and adjust the issue yourself. As a template, you can use the code which Sebastian implemented for fixing this issue for the standard keras embedding layer here.

[adrhill]

Closing this as it looks like it should have been automatically closed when merging PR #222.
Feel free to reopen this issue if the error still persists.