Multi-View CNN support

[danielbraun89]

I want to use this terrific tool on an MVCNN type of network, which require to feed the keras model a list of ndarrays rather than a single ndarray,
I have tried the naive approach and fed the analyzer a list of ndarrays but it cant get past the collection of standardize_user_data methods. Is there a way to use innvestigate on a MVCNN or is it simply not supported?

[albermax]

Hi Daniel,

thanks for your interest! The design of the library should already support this, but nobody tested it yet. Thus I'm sure we can get it to work!

Could you please post your error message here?

Cheers,
Max

[danielbraun89]

On a more thorough attempt, it appears the library indeed support multi inputs when no neuron is given but not when a neuron is given.

When compiling and feeding any analyzer with a multi view input in addition to a neuron number, an error is being thrown:

File "/home/username/.conda/envs/testing_env/lib/python3.5/site-packages/keras/engine/training.py", line 1266, in predict_on_batch 
x, _, _ = self._standardize_user_data(x)
  
File "/home/username/.conda/envs/testing_env/lib/python3.5/site-packages/keras/engine/training.py", line 749, in _standardize_user_data
    exception_prefix='input')
 
File "/home/username/.conda/envs/testing_env/lib/python3.5/site-packages/keras/engine/training_utils.py", line 80, in standardize_input_data
    data = [np.asarray(d) for d in data]
 
File "/home/username/.conda/envs/testing_env/lib/python3.5/site-packages/keras/engine/training_utils.py", line 80, in <listcomp>
    data = [np.asarray(d) for d in data]
 
File "/home/username/.conda/envs/testing_env/lib/python3.5/site-packages/numpy/core/numeric.py", line 492, in asarray
    return array(a, dtype, copy=False, order=order)
ValueError: could not broadcast input array from shape (100,100,4) into shape (1,100,100)```

[albermax]

Thank you! Good to know.

Is it possible that you also submit a minimal example? Then I can reproduce your problem.

[danielbraun89]

Thank you! Good to know.

Is it possible that you also submit a minimal example? Then I can reproduce your problem.

Yes of course,

The problem derived from the different form factor of the inputs (Keras multi-view implementation requires a list of ndarrays as inputs to the model instead of a single ndarray in case of classic model) and not the actual number of inputs the model accepts.
Therefore this problem is very easy to replicate based on your own Jupiter examples that I have simplified (Creating a model with no softmax to begin with, and running just on few simple non-compiled analyzers) in order to focus on the problem itself.

link to example

You would see the difference between the cases is merely the way input is being fed (analyzer.analyze([x]) vs analyzer.analyze(x) etc)
All of them works as expected instead of the last case which throws the exception I have posted above

[albermax]

Hi Daniel,

is there a specific reason why you pass a single tensor in a list?
This error is Keras specific and I guess it is because the list gets interpreted as first "dimension".

I just tried the following code snippet and it shows that using multiple inputs works (it's build on top of the mnist_compare_methods.ipynb):

# Create & train model                                                                                                                                                               
if keras.backend.image_data_format == "channels_first":
    input_shape = (1, 28, 28)
else:
    input_shape = (28, 28, 1)

input1 = keras.layers.Input(shape=input_shape)
input2 = keras.layers.Input(shape=input_shape)
x = keras.layers.Average()([input1, input2])

x = keras.layers.Conv2D(32, (3, 3), activation="relu")(x)
x = keras.layers.Conv2D(64, (3, 3), activation="relu")(x)
x = keras.layers.MaxPooling2D((2, 2))(x)
x = keras.layers.Flatten()(x)
x = keras.layers.Dense(512, activation="relu")(x)
x = keras.layers.Dense(10, activation="softmax")(x)
model = keras.models.Model(inputs=[input1, input2], outputs=[x])

x_train, y_train, x_test, y_test = data
# convert class vectors to binary class matrices                                                                                                                                     
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)
x_train, y_train = x_train[:100], y_train[:100]
x_small = x_test[:2]

model.compile(loss="categorical_crossentropy",
              optimizer="adam",
              metrics=["accuracy"])
history = model.fit([x_train, x_train], y_train,
                    batch_size=64,
                    epochs=5,
                    verbose=1)
score = model.evaluate([x_test, x_test], y_test, verbose=0)

# Create model without trailing softmax                                                                                                                                              
model_wo_softmax = iutils.keras.graph.model_wo_softmax(model)
analyzer = innvestigate.create_analyzer("gradient", model_wo_softmax)
ret = analyzer.analyze([x_small, x_small])
print(ret[0].shape, ret[1].shape)

The output is:

100/100 [==============================] - 1s 7ms/step - loss: 2.6047 - acc: 0.1000
Epoch 2/5
100/100 [==============================] - 0s 5ms/step - loss: 1.9415 - acc: 0.2900
Epoch 3/5
100/100 [==============================] - 0s 5ms/step - loss: 1.4558 - acc: 0.8300
Epoch 4/5
100/100 [==============================] - 0s 5ms/step - loss: 0.9749 - acc: 0.8900
Epoch 5/5
100/100 [==============================] - 0s 5ms/step - loss: 0.5507 - acc: 0.9300
(2, 28, 28, 1) (2, 28, 28, 1)

Hope this helps!

Cheers,
Max

[danielbraun89]

Thanks for the reply max!
I'm afraid your example does not replicate the problem, as it only occur when using a neuron,

Try the following code
(same as yours with neuron_selection_mode="index" and adding an arbitrary neuron to the analyze() function, will now throw an exception):

if keras.backend.image_data_format == "channels_first":
    input_shape = (1, 28, 28)
else:
    input_shape = (28, 28, 1)

input1 = keras.layers.Input(shape=input_shape)
input2 = keras.layers.Input(shape=input_shape)
x = keras.layers.Average()([input1, input2])

x = keras.layers.Conv2D(32, (3, 3), activation="relu")(x)
x = keras.layers.Conv2D(64, (3, 3), activation="relu")(x)
x = keras.layers.MaxPooling2D((2, 2))(x)
x = keras.layers.Flatten()(x)
x = keras.layers.Dense(512, activation="relu")(x)
x = keras.layers.Dense(10, activation="softmax")(x)
model = keras.models.Model(inputs=[input1, input2], outputs=[x])

x_train, y_train, x_test, y_test = data
# convert class vectors to binary class matrices                                                                                                                                     
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)
x_train, y_train = x_train[:100], y_train[:100]
x_small = x_test[:2]

model.compile(loss="categorical_crossentropy",
              optimizer="adam",
              metrics=["accuracy"])
history = model.fit([x_train, x_train], y_train,
                    batch_size=64,
                    epochs=5,
                    verbose=1)
score = model.evaluate([x_test, x_test], y_test, verbose=0)

# Create model without trailing softmax                                                                                                                                              
model_wo_softmax = iutils.keras.graph.model_wo_softmax(model)
analyzer = innvestigate.create_analyzer("gradient", model_wo_softmax, neuron_selection_mode="index")
ret = analyzer.analyze([x_small, x_small], 2)
print(ret[0].shape, ret[1].shape)

[albermax]

Oh I remember! Sorry and thank you for clarifying. I will have a look asap!

[albermax]

Hi Daniel,

the bug should be fixed in the develop branch! Could you please check if this solves your problem and in case close the issue?

Cheers,
Max

[danielbraun89]

The problem seems to persist but indeed has moved on to a different exception:

ValueError                                Traceback (most recent call last)
<ipython-input-4-4b8da4af3913> in <module>()
     63 model_wo_softmax = iutils.keras.graph.model_wo_softmax(model)
     64 analyzer = innvestigate.create_analyzer("gradient", model_wo_softmax, neuron_selection_mode="index")
---> 65 ret = analyzer.analyze([x_small, x_small], 2)
     66 print(ret[0].shape, ret[1].shape)

c:\users\username\appdata\local\continuum\miniconda3\envs\offline_viewer\lib\site-packages\innvestigate\analyzer\base.py in analyze(self, X, neuron_selection)
    487                     "do theoretically not support multi-neuron analysis. "
    488                     "Consider using a Sum layer.")
--> 489             ret = self._analyzer_model.predict_on_batch([X, neuron_selection])
    490         else:
    491             ret = self._analyzer_model.predict_on_batch(X)

c:\users\username\appdata\local\continuum\miniconda3\envs\offline_viewer\lib\site-packages\keras\engine\training.py in predict_on_batch(self, x)
   1264             Numpy array(s) of predictions.
   1265         """
-> 1266         x, _, _ = self._standardize_user_data(x)
   1267         if self._uses_dynamic_learning_phase():
   1268             ins = x + [0.]

c:\users\username\appdata\local\continuum\miniconda3\envs\offline_viewer\lib\site-packages\keras\engine\training.py in _standardize_user_data(self, x, y, sample_weight, class_weight, check_array_lengths, batch_size)
    747             feed_input_shapes,
    748             check_batch_axis=False,  # Don't enforce the batch size.
--> 749             exception_prefix='input')
    750 
    751         if y is not None:

c:\users\username\appdata\local\continuum\miniconda3\envs\offline_viewer\lib\site-packages\keras\engine\training_utils.py in standardize_input_data(data, names, shapes, check_batch_axis, exception_prefix)
     99                 'Expected to see ' + str(len(names)) + ' array(s), '
    100                 'but instead got the following list of ' +
--> 101                 str(len(data)) + ' arrays: ' + str(data)[:200] + '...')
    102         elif len(names) > 1:
    103             raise ValueError(

ValueError: Error when checking model input: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 3 array(s), but instead got the following list of 2 arrays

Full code to reproduce:

import imp, os, keras, keras.backend, keras.models, innvestigate
import numpy as np
import innvestigate.utils as iutils
eutils = imp.load_source("utils", "../utils.py")
mnistutils = imp.load_source("utils_mnist", "../utils_mnist.py")

# Load data and prepare model
data_not_preprocessed = mnistutils.fetch_data()
input_range = [-1, 1]
preprocess, revert_preprocessing = mnistutils.create_preprocessing_f(data_not_preprocessed[0], input_range)
data = (preprocess(data_not_preprocessed[0]), data_not_preprocessed[1], preprocess(data_not_preprocessed[2]), data_not_preprocessed[3] )
num_classes = len(np.unique(data[1]))
label_to_class_name = [str(i) for i in range(num_classes)]
input_shape = (1, 28, 28) if keras.backend.image_data_format == "channels_first" else (28, 28, 1)

input1 = keras.layers.Input(shape=input_shape)
input2 = keras.layers.Input(shape=input_shape)
x = keras.layers.Average()([input1, input2])
x = keras.layers.Conv2D(32, (3, 3), activation="relu")(x)
x = keras.layers.Conv2D(64, (3, 3), activation="relu")(x)
x = keras.layers.MaxPooling2D((2, 2))(x)
x = keras.layers.Flatten()(x)
x = keras.layers.Dense(512, activation="relu")(x)
x = keras.layers.Dense(10, activation="softmax")(x)
model = keras.models.Model(inputs=[input1, input2], outputs=[x])

x_train, y_train, x_test, y_test = data
# convert class vectors to binary class matrices                                                                                                                                     
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)
x_train, y_train = x_train[:100], y_train[:100]
x_small = x_test[:2]

model.compile(loss="categorical_crossentropy",
              optimizer="adam",
              metrics=["accuracy"])
history = model.fit([x_train, x_train], y_train,
                    batch_size=64,
                    epochs=5,
                    verbose=1)
score = model.evaluate([x_test, x_test], y_test, verbose=0)

# Create model without trailing softmax                                                                                                                                              
model_wo_softmax = iutils.keras.graph.model_wo_softmax(model)
analyzer = innvestigate.create_analyzer("gradient", model_wo_softmax, neuron_selection_mode="index")
ret = analyzer.analyze([x_small, x_small], 2)
print(ret[0].shape, ret[1].shape)

[albermax]

Hi Daniel,

I ran your code example and cannot reproduce your bug. Not on the master nor the develop branch.
Could please update iNNvestigate once more and try again?
Which version of Keras do you use?

Cheers,
Max

[MissNautilus]

Hello,

referring to the code example above I am wondering:
how could I display corresponding heatmaps for two inputs?

Thanks in advance :)
Vanessa

[albermax]

Hi Vanessa,

I guess that depends on the input format. :-)
Assuming you have images of different scales I would display different heatmaps (but you might want to check that all of the heatmaps are normalized into the same value range!)
Otherwise summing attributions pixel-wise is an option. In the end it depends on your problem setting!

Cheers,
Max

PS: I close the issue because the "issue" itself seems done. You can still answer!