how to use this model to train and evaluate on Imagenet?

[Alexanzhuo]

No description provided.

[Alexanzhuo]

Thank you for providing this network!
I want to use this model "ViT" to classify Imagnet,but the accuracy is not good.I try it on mini-Imagenet first.
I use the network like this:
net = ViT(
image_size = 224,
patch_size = 16,
num_classes = 64,
dim = 1024,
depth = 6,
heads = 8,
mlp_dim = 2048,
dropout = 0.1,
emb_dropout = 0.1
).cuda(device)

[Alexanzhuo]

But the accuracy just stops increasing.

[lucidrains]

@Alexanzhuo Hi Alex, you won't see any positive results in the small data regime. What you can do, however, is to run self-supervised learning (BYOL) on a bunch of unlabelled images first, and then train on your tiny mini-Imagenet corpus.

Or you can just use Ross' version with the pretrained weights released by Google