New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

mask中为什么使用了triu函数，而不是tril函数 #597

Open

Jia-hn opened this issue Jan 3, 2024 · 1 comment

Jia-hn commented Jan 3, 2024

mask中使用triu函数导致上三角为True，也就是每个query只考虑之后key，而不是之前的key

2421468125 commented Jan 21, 2024

没有错，FullAttention里把triu中为1的元素填充为-np.inf了

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment