When will release mae pretrained weights?

[shi-lt]

No description provided.

[UmaisZahid]

Any update on on the pretrained MAE release? Even just having the pretrained decoder weights would be quite useful.

[dbolya]

@UmaisZahid We are currently going back and forth about how to release these weights. May I ask, what do you plan to use the decoder for?

The issue is the models were trained with a normalized pixel loss (i.e., each patch that the decoder had to predict was normalized within that patch). Thus, using the pretrained decoder for reconstructing a masked image wouldn't look correct. Hence, we were planning on removing the decoder when releasing the weights (I believe MAE does the same thing).

Would the decoder still be useful in spite of that?

[UmaisZahid]

Thanks for the response @dbolya.

I believe so yes. One use case for the decoder would be to further pretrain on another dataset. The ImageNet pretraining would act as a warm start or inductive bias of sorts in this case - which I imagine is particularly useful in the cases where the second pretraining dataset is smaller.

You could release the decoder and perhaps remind people explicitly in the GitHub Readme what you just mentioned about the reconstructive loss - that would be very useful!

[dbolya]

Thanks for the context. We've decided to release all the pretrained weights with decoders and all!

The pretrained models are available in v0.1.2 (see the readme on how to use them). Closing the issue now, but feel free to reopen if you run into any trouble.

[abhishekaich27]

@dbolya Are these weights for MViT2 or modified MViT2 as proposed in the paper?

[dbolya]

@abhishekaich27 These weights are for Hiera, the final model that we end up with after making all modifications to MViTv2.

[abhishekaich27]

Thanks @dbolya! Do you plan to release or provide the MAE pre-trained MViT2 model?

[dbolya]

@abhishekaich27 No plans for that afaik.