New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

为什么预训练时，做attention的时候不需要mask #40

Closed

LLH1818 opened this issue Oct 13, 2023 · 0 comments

LLH1818 commented Oct 13, 2023

初学者请教一下。为什么flash版本attention没有mask，manual版本也是score + mask，不应该是score * mask吗

LLH1818 changed the title ~~为什么预训练attention的时候不需要mask~~ 为什么预训练时，做attention的时候不需要mask

LLH1818 closed this as completed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment