the code of multi head external attetion can't work

[DRJYYDS]

It's obvious that the code is wrong. You should check the reshape and transpose part in your code.
And what is the coef? It has not been mentioned in the paper.
And also you didn't import the torch.
And the qkv_bias, qk_scale is not used here.
Did you really run the multi-head version in your paper?

[MenghaoGuo]

I think the code is correct.
We import toch in this line.

You can read the code carefully and run it and you will find it is correct.

Some code comments may inaccurate and we will organize this repo after this article is accepted.

[DRJYYDS]

I think the code is correct. We import toch in this line.

You can read the code carefully and run it and you will find it is correct.

Some code comments may inaccurate and we will organize this repo after this article is accepted.

So the actual shape for X is [B,C,N]? or the code can't work at all.

Btw, the screenshot is from the file 'multi_head_attention_torch.py'. You did not import torch here.

[MenghaoGuo]

it is b n c and nn.Linear can be used in this situation.

We do not import this multi head file in the EAMLP，Please check it.