Training and inference issue

[qtli]

Dear authors, thanks for your great work.

I have a little question. The paper says that the spans are shuffled in the training time.
I was wondering whether the predicted spans are in order when inference, and how to make them orderly?

Look forward to hearing from you.

[qtli]

Does this is implemented due to the position representations?

[duzx16]

In most downstream tasks the model only needs to predict one span. In the blank infilling task, for simplicity we always predict the missing spans in a left-to-right order.
Note that you can specify which span to predict by specifying the position id of [SOP] token.

[qtli]

In most downstream tasks the model only needs to predict one span. In the blank infilling task, for simplicity we always predict the missing spans in a left-to-right order. Note that you can specify which span to predict by specifying the position id of [SOP] token.

Get it! Thanks for your prompt reply!! :)