Relevance of hidden layers

[francescomalandrino]

Is there a simple (or simple-ish) way to obtain the relevance scores of hidden layers, i.e., what is visualized in the cell starting with "for i,l in enumerate([31,21,11,1]):" here?
https://git.tu-berlin.de/gmontavon/lrp-tutorial

The library sure computes them, but I could not find a way to recover those. Thanks!

[fhvilshoj]

I guess that this is actually a bit cumbersome. One way would be to break the network up into multiple lrp.Sequentials and then use the intermediate outputs to reference their output.grad variables.

It relates to this post about obtaining grads (explanations in our case) and this post about getting intermediate outputs of pretrained networks.

I would be happy to discuss if there is a better way to structure this code such that similar tasks become easier.

[francescomalandrino]

hey, thanks for the answer!

yes, "cumbersome" is the right word here!

pull request #3 gets the job done, but it's kind of hack-ish... not sure if there's a better/more elegant way to do it!

[fhvilshoj]

I have merged your pull request although I never liked global variables. In this case, however, it does make sense to some extend. Thanks for the contribution!
I will integrate your code a bit more and push an update soon.
In particular, the trace should be done in all backward computations and not only for the Epsilon rule.

[francescomalandrino]

The best way would be to pass a list to fill with the trace when calling backward -- that way, we would not need global variables:

tl=[]
y_hat.backward(trace_list=tl)

However, I think backward up there calls PyTrorch's backward first, which then calls your own backward. If this is the case, PyTorch's function will throw a TypeError exception upon seeing the extra parameter.

[fhvilshoj]

Yes, this is exactly the issue.