Visualize Attention

[MatCorr]

Does anyone have a code that allows one to visualize the result of the attention heads of the last layer? I'm thinking of something similar to what was done in DinoV1 through the visualize_attention.py script.

[ozkilim]

https://gitlab.com/ziegleto-machine-learning/dino/-/tree/main/

[MatCorr]

Thanks a lot!

[MatCorr]

Uh oh, I just noticed your documentation says the solution doesn't work for XFORMERS, and that's exactly what we're using.

So if anyone has a code for that, I'd appreciate it.

[ozkilim]

#90 (comment)

Change these lines and will work for XFORMERS