MIMIR: Masked Image Modeling for Mutual Information Based Adversarial Robustness

Environment settings

check requirements.txt

Install AutoAttack

pip install git+https://github.com/fra31/auto-attack

Evaluation and Training Commands

check ./script

Checkpoints

MIMIR + more advanced fine-tuning (2-step APGD adversarial training, 300 epochs) for SOTA performance on ImageNet-1K

eps 4

Model	Natural	AA	CheckPoint
ViT-S	71.00	46.10	Link
ViT-B	76.32	54.28	Link

CIFAR-10

eps 8

Model	Natural	AA	CheckPoint
ViT-T	84.82	52.96	Link
ViT-S	88.11	53.18	Link
ViT-B	89.30	54.55	Link
ConViT-T	80.74	45.04	Link
ConViT-S	87.49	52.54	Link
ConViT-B	89.30	55.64	Link

ImageNet-1K

eps 2

Model	Natural	PGD 20	CheckPoint
ViT-S	74.60	54.56	Link
ViT-B	75.88	55.42	Link
CaiT-XXS24	73.39	53.39	Link
CaiT-S36	76.05	56.78	Link

eps 4

Model	Natural	PGD 20	CheckPoint
ViT-S	71.29	40.98	Link
ViT-B	73.22	41.26	Link
CaiT-XXS24	69.90	40.53	Link
CaiT-S36	73.57	40.03	Link

MIMIR Pre-train checkpoints

Dataset	Model	CheckPoint
CIFAR-10	ViT-T	Link
CIFAR-10	ViT-S	Link
CIFAR-10	ViT-B	Link
CIFAR-10	ConViT-T	Link
CIFAR-10	ConViT-S	Link
CIFAR-10	ConViT-B	Link
ImageNet-1K	ViT-S	Link
ImageNet-1K	ViT-B	Link
ImageNet-1K	CaiT-XXS24	Link
ImageNet-1K	CaiT-S36	Link

Acknowlegements

This repository is built upon the following repositories:

https://github.com/facebookresearch/mae

https://github.com/wzekai99/DM-Improves-AT

https://github.com/yuxi120407/DIB

https://github.com/choasma/HSIC-bottleneck