Errors in models for API and ReLU

[cap76]

Hi,

I'm trying to got through an example using a simple API keras model. I've loaded the model in e.g.,
deeplift_model = kc.convert_model_from_saved_files('Model.h5',
nonlinear_mxts_mode=NonlinearMxtsMode.RevealCancel)

Now I'm attempting to get to the next step to compile the functions (compile_func) and check everything is the same vs the standard keras model. I can't seem to find the list of layers:

deeplift_model.get_name_to_blob().keys()
Traceback (most recent call last):
File "", line 1, in
AttributeError: 'GraphModel' object has no attribute 'get_name_to_blob'

I seem to be able to compile the function via:

deeplift_prediction_func = compile_func([deeplift_model.get_name_to_layer()["input_1_0"].get_activation_vars()], deeplift_model.get_name_to_layer()["preact_dense_1_0"].get_activation_vars())

where
deeplift_model.get_name_to_layer().keys()
odict_keys(['input_1_0', 'preact_conv1d_1_0', 'conv1d_1_0', 'max_pooling1d_1_0', 'dropout_1_0', 'flatten_1_0', 'preact_dense_1_0', 'dense_1_0', 'preact_dense_2_0', 'dense_2_0', 'preact_dense_3_0', 'dense_3_0'])

However, when I try to call get_target_contribs_func

revealcancel_func = deeplift_model.get_target_contribs_func(find_scores_layer_name="input_1_0", pre_activation_target_layer_name="preact_conv1d_1_0")

I get told off about the final activation function:

Traceback (most recent call last):
File "", line 1, in
File "/Users/christopher_penfold/Desktop/Code/deeplift/deeplift/models.py", line 113, in get_target_contribs_func
return self._get_func(*args, func_type=FuncType.contribs, **kwargs)
File "/Users/christopher_penfold/Desktop/Code/deeplift/deeplift/models.py", line 269, in _get_func
**kwargs)
File "/Users/christopher_penfold/Desktop/Code/deeplift/deeplift/models.py", line 47, in _get_func
self._set_scoring_mode_for_target_layer(target_layer)
File "/Users/christopher_penfold/Desktop/Code/deeplift/deeplift/models.py", line 182, in _set_scoring_mode_for_target_layer
+final_activation_type)
RuntimeError: Unsupported final_activation_type: ReLU

Sure I'm doing something obvious wrong somewhere ... any pointers in the right direction would be appreciated.

[AvantiShri]

Hi Christopher,

You’re fine, the issue is with deeplift.

(1) get_name_to_blob is indeed deprecated to get_name_to_layer. Where did you encounter get_name_to_blob? I thought I had updated the README, evidently i missed something somewhere.

(2) Some context about the second error you are getting: traditionally, people use sigmoid or softmax outputs when doing classification, and linear outputs when doing regression. In the case of the former, DeepLIFT computes the contribution wrt the linear activation preceeding the final nonlinearity to avoid the saturation effects of the sigmoid/softmax. In the case of a linear output, DeepLIFT computes contributions wrt to the provided linear layer. You are trying to compute contributions to a linear layer that is followed by a ReLU nonlinearity. I can add this case in (would need to clarify how you want to handle ReLU saturation), but I also note that you are evidently computing contributions to neurons in the first convolutional layer. Is that actually the layer you are interested in? And do you care about interpeting individual neurons in the conv layer, or the activation of the entire layer as a whole? If you care about interpreting every individual neuron in the conv layer, you might find it easier to compute the contributions using a series of numpy matrix multiplications (which is tractable to implement for just a single layer; I can describe how to do this if that is in fact what you are after). DeepLIFT is currently set up to compute contributions to one target output neuron at a time, so it would be cumbersome to perform DeepLIFT for every individual neuron in the conv layer.

[cap76]

Hi Avanti,

Thanks for the quick response. I think I came across get_name_to_blob scrolling through the issues thread.

2. You're right, the code I pasted was wrong (I'd been playing around with it a little), and I do not really want the contribution to layers in the first convolutional layer. I think the correct call would be:

revealcancel_func = deeplift_model.get_target_contribs_func(find_scores_layer_name="input_1_0", pre_activation_target_layer_name="preact_dense_3_0")

This seems to throw a different error:

File "", line 1, in
File "/Users/christopher_penfold/Desktop/Code/deeplift/deeplift/models.py", line 113, in get_target_contribs_func
return self._get_func(*args, func_type=FuncType.contribs, **kwargs)
File "/Users/christopher_penfold/Desktop/Code/deeplift/deeplift/models.py", line 269, in _get_func
**kwargs)
File "/Users/christopher_penfold/Desktop/Code/deeplift/deeplift/models.py", line 49, in _get_func
find_scores_layer.update_mxts()
File "/Users/christopher_penfold/Desktop/Code/deeplift/deeplift/layers/core.py", line 183, in update_mxts
output_layer.update_mxts()
File "/Users/christopher_penfold/Desktop/Code/deeplift/deeplift/layers/core.py", line 183, in update_mxts
output_layer.update_mxts()
File "/Users/christopher_penfold/Desktop/Code/deeplift/deeplift/layers/core.py", line 184, in update_mxts
output_layer._update_mxts_for_inputs()
File "/Users/christopher_penfold/Desktop/Code/deeplift/deeplift/layers/core.py", line 346, in _update_mxts_for_inputs
neg_mxts_increments) = self._get_mxts_increments_for_inputs()
File "/Users/christopher_penfold/Desktop/Code/deeplift/deeplift/layers/pooling.py", line 89, in _get_mxts_increments_for_inputs
pos_mxts_increments = self._grad_op(self.get_pos_mxts())
File "/Users/christopher_penfold/Desktop/Code/deeplift/deeplift/layers/pooling.py", line 79, in _grad_op
return tf.squeeze(nn_grad.gen_nn_ops.max_pool_grad(
AttributeError: module 'tensorflow.python.ops.gen_nn_ops' has no attribute 'max_pool_grad'

For my setup I'm doing classification (potentially multi class multi output classification) I was using the RelU layers just based on some code I had. Should be easy enough to switch to sigmoid or softmax if these would be more appropriate and don't expect the accuracy to be

[AvantiShri]

Yes, I think sigmoid/softmax outputs are def more appropriate for classification if trained with binary/categorical crossentropy losses.

I think the error you got is caused by using an old version of tensorflow. Can you upgrade to tensorflow version 1.7 or later?

[cap76]

Ah! Of course facepalm 😳Seems to work now, will continue going through, but I think we can call this issue closed