How many embeds are connected?

[AliHaiderAhmad001]

My greetings,
Thank you for these ideas and this work.
I would like to know how many embeds are connected (two, three or four?), is it constant during training, and is there an optimal number according to your experience?

[wangxinyu0922]

Hi, the number of embeddings depends on the language and the task. It usually use more than 10 embedding in ACE training. During training, the embeddings are selected by the controller. You may read the appendix of the paper for more details about the embedding selections.

[AliHaiderAhmad001]

thank you for replying,
I know that, but the question is how many embeddings the controller will choose, is it fixed or task dependent? And if task-dependent, how many embeddings will there be on average (approximately)?

[wangxinyu0922]

It is task dependent. About half of the candidate embeddings are selected.

[AliHaiderAhmad001]

Thanks!