How to visualize the attention map of t2t-vit?

[Salen158]

The refered example file was using vit, which provided attention weights output. But the t2t-vit model only have logists output, so i can't reuse their code. I really wish a more detaild way to visualize the attention map of t2t-vit. It's really important to me.
thx a lot

[yuanli2333]

Hi, you can also run our t2t-vit by exacting the attention weights from our pretrained models in each layers. We have released the pretrained weights for each t2t-vit model.

[Salen158]

Hi, you can also run our t2t-vit by exacting the attention weights from our pretrained models in each layers. We have released the pretrained weights for each t2t-vit model.

thx a lot. I have done it successfully.

[bayesandblues]

@Salen158 I'm also very interested in this functionality - any chance you have a simple notebook example you're willing to share?

[Salen158]

@Salen158 I'm also very interested in this functionality - any chance you have a simple notebook example you're willing to share?

u can modify the trainsformer_block.py and t2t_vit.py in models folder to grab and returen the attention weights in forward propagation