The size of embedding dim

[liu-zhenhua]

This work is very interested and fascinating. I have a question : the size of embedding size and how you decide it?
Look forward to the release of pretrained model.

[gupta-abhay]

embedding size is coming from the paper definition -- the hidden dim of the encoder and the embedding dim. are usually the same.

regarding the pre-trained models, I'll take a call about it at some point. I have it tested for about 100 iterations, but running for 300 epochs on ImageNet-1K, requires serious juice. And unless I can find a sustainable way to do it, I will not pursue this effort.