Why there is two position embeded in your code ?

[machanic]

one is in https://github.com/soskek/attention_is_all_you_need/blob/master/net.py#L345,
and another in https://github.com/soskek/attention_is_all_you_need/blob/master/net.py#L346-L350,
by the way, is that the calculate of sin&cos position embeded in self.initialize_position_encoding ( return the "all position encoding array"), am I correct?

[soskek]

embed_pos is just an option feature, which is off as default. you can ignore it.

[machanic]

@soskek
One question about https://github.com/soskek/attention_is_all_you_need/blob/master/net.py#L345
you only add self.position_encoding_block[:, :, :length], there is no x_block or any data information pass into it? if there is two sentence with the same length, do you mean there position encoding is the same??

How to understand it?

[soskek]

Check what is length. It is derived from each data given through https://github.com/soskek/attention_is_all_you_need/blob/master/net.py#L418-L419