-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
visualization #13
Comments
Answer to 1 and 2: Yes, but not perfect.
Answer to 3: Actually I save all attention weights of corresponding generated words and then generated the visualization image for each word, and finally I use Visio to manually compose the visualization images and words to fit the paper width. I have not scripted the full process into one stage. |
Hello, I would like to know which step of inference should be taken for attention weight, and which stage of attention weight should be taken when generating each word?Thanks |
@Lujinfu1999 hello, did u find the answer to this?
|
I tried get attention weights from the last decoder's cross-attention's last head,maybe you can try it. @not-hermione |
Was the attention maps visualization convincing? |
Hi, author ! Thanks for this great code!
I want to reproduce the visualization results, but I cannot find the corresponding code in this repo.
I read the paper but I think it's not easy for me to reproduce the visualization results correctly.
May I have the code to produce the visualization results, or is there anything I missed?
Thank you very much!
The text was updated successfully, but these errors were encountered: