[Transformer Tutorial Error] Positional Encoding is not match the input tensor size in train() function

[ericzhou571]

Hi,

I carefully go through the network definition in Transformer Tutorial (beginner_source/transformer_tutorial.py) and I found there is an error in Positional Encoding.
Positional Encoding suppose the input of its forward function has size (sequence length, batch size, embedding size).
But in train() function the input tensor 'data' has size (batch size, sequence length) and after token embedding the result is still (batch size, sequence length, embedding size).
I hope someone with push authority can correct this error.
It is every easy to solve, just add a transpose operation on data tensor in train( ) function.

Best

[parmeet]

Hi @ericzhou571 , the size of input tensor is indeed (sequence length, batch size). Please refer here where the data is created using get_batch() function.