Decomposing the Neurons: Activation Sparsity via Mixture of Experts for Continual Test Time Adaptation [Arxiv]

Rongyu Zhang, Aosong Cheng*, Yulin Luo*, Gaole Dai, Huanrui Yang, Jiaming Liu, Ran Xu, Li Du, Yuan Du, Yanbing Jiang, Shanghang Zhang

Installation

Please create and activate the following conda envrionment.

# It may take several minutes for conda to solve the environment
conda update conda
conda env create -f environment.yml
conda activate moase 

Classification Experiments

• ViT as the backbone

Our source model is from timm, you can directly donwload it from the code.

Cifar10-to-Cifar10C task

Please load the source model from here

bash run_cifar10.sh # MoASE

Cifar100-to-Cifar100C task

Please load the source model from here

cd cifar
bash run_cifar100.sh # MoASE

For segmentation code, you can refer to cotta and SVDP. As for the source model, you can directly use Segformer trained on Cityscapes.

Citation

Please cite our work if you find it useful.

@article{zhang2024decomposing,
  title={Decomposing the Neurons: Activation Sparsity via Mixture of Experts for Continual Test Time Adaptation},
  author={Zhang, Rongyu and Cheng, Aosong and Luo, Yulin and Dai, Gaole and Yang, Huanrui and Liu, Jiaming and Xu, Ran and Du, Li and Du, Yuan and Jiang, Yanbing and others},
  journal={arXiv preprint arXiv:2405.16486},
  year={2024}
}

Acknowledgement

• CoTTA code is heavily used. official
• KATANA code is used for augmentation. official
• Robustbench official

Data links

• ImageNet-C Download