awesome-MIM

Reading list for research topics in Masked Image Modeling(MIM).

We list the most popular methods for MIM, if I missed something, please submit a request. (Note: We show the date of the first version of Arxiv here. But the link of paper may be not the early version.)

Self-supervied Vision Transformers as backbone models.

Date	Method	Conference	Title	Code
2020-xx-xx(maybe 2019)	iGPT	ICML 2020	Generative Pretraining from Pixels	iGPT
2020-10-22	ViT	ICLR 2021(Oral)	An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale	ViT
2021-06-10	MST	NeurIPS 2021	MST: Masked Self-Supervised Transformer for Visual Representation	None
2021-06-14	BeiT	ICLR 2022(Oral)	BEiT: BERT Pre-Training of Image Transformers	BeiT
2021-11-11	MAE	Arxiv 2021	Masked Autoencoders Are Scalable Vision Learners	MAE
2021-11-15	iBoT	ICLR 2022	iBOT: Image BERT Pre-Training with Online Tokenizer	iBoT
2021-11-18	SimMIM	Arxiv 2021	SimMIM: A Simple Framework for Masked Image Modeling	SimMIM
2021-11-24	PeCo	Arxiv 2021	PeCo: Perceptual Codebook for BERT Pre-training of Vision Transformers	None
2021-12-16	MaskFeat	Arxiv 2021	Masked Feature Prediction for Self-Supervised Visual Pre-Training	None
2021-12-20	SplitMask	Arxiv 2021	Are Large-scale Datasets Necessary for Self-Supervised Pre-training?	None
2022-01-31	ADIOS	Arxiv 2022	Adversarial Masking for Self-Supervised Learning	None
2022-02-07	CAE	Arxiv 2022	Context Autoencoder for Self-Supervised Representation Learning	None
2022-02-07	CIM	Arxiv 2022	Corrupted Image Modeling for Self-Supervised Visual Pre-Training	None

Image generation.

Date	Method	Conference	Title	Code
2022-02-08	MaskGIT	Arxiv 2022	MaskGIT: Masked Generative Image Transformer	None