Pretraining segformer on ImageNet-22K

[htzheng]

The Swin transformer release a large model pretrained on ImageNet-22K for semantic segmentation and achieved a good result. I wonder if you are interested in improving segformer in a similar way? Thanks!

[xieenze]

Hi,
Good question!
Yes, we are planning to pre-train on Imagenet-22k. But actually, it is not easy for us.
Firstly before we did not have enough GPU resources, but recently we have sufficient resources now.
Then we haven't master multi-node training on our GPU clusters yet, we are still trying to figure it out.
We believe that recently we can pre-train it on Imagenet-22k and release stronger models.

Thanks for the attention!

[htzheng]

That would be really great! Thanks for the great work and good luck to you!

[armored-guitar]

@xieenze how many epochs have you trained your model on imagenet? cannot find this info

[pablogiaccaglia]

Hi, any news on this?