why Permute before attention dense layer in attention_3d_block?

[guiyuliu]

    a = Permute((2, 1))(inputs)
    a = Dense(TIME_STEPS, activation='softmax')(a)

this line ,why do you permute times_tep and input_dim
what if I don't permute , and followed by a dense layer with input_dim ? since dense layer is with the shape of "time_Step *time_step" ,what is the difference when I change it to "input_dim * input_dim"
Dense(input_Dim activation='softmax')(a)

[Dongzhixiao]

I think the attention mechanism is used in time step dimension. So the parameters to be trained are the weights to multiply by the time steps.

[philipperemy]

The attention block has been updated. It's a much better version than the old one. This issue is now deprecated.