add the attention mask for flash-attention.

[MayDomine]

No description provided.

[ZhongYingMatrix]

Could we use (-inf, 0) bias to mock mask feature?

[MayDomine]

Could we use (-inf, 0) bias to mock mask feature?

Yes, that would be another way to implement mask. But if the input mask has a whole row of false, it will be quite tricky to handle the nan value in the softmax(qk). I wrote this because I have this kind of need that bias can not satisfy.

[Zyriix]

it this work? why it was closed?

[nofreewill42]

"But if the input mask has a whole row of false, it will be quite tricky to handle the nan value in the softmax(qk)."

What about handling it with an assertion?