SoC4SS-FGVC

This repo is the official Pytorch implementation of our paper:

Roll with the Punches: Expansion and Shrinkage of Soft Label Selection for Semi-supervised Fine-Grained Learning
Authors: Yue Duan, Zhen Zhao, Lei Qi, Luping Zhou, Lei Wang and Yinghuan Shi

• Quick links: [arXiv | Published paper | Poster/Slides | Video | Code download]

• Latest news:

  • Our paper is accepted by AAAI Conference on Artificial Intelligence (AAAI) 2024 🎉🎉. Thanks to users.

• Related works:

  • Interested in robust SSL in MNAR setting with mismatched distributions? 👉 Check out our ICCV'23 paper PRG [arXiv | Repo] or our ECCV'22 paper RDA [arXiv | Repo].
  • Interested in the conventional SSL or application of complementary label in SSL? 👉 Check out our TNNLS paper MutexMatch [arXiv | Repo].

Requirements

• numpy==1.20.3
• Pillow==9.3.0
• scikit_learn==1.1.3
• torch==1.8.0
• torchvision==0.9.0

How to Train

Important Args

• --net [resnet18/resnet50/resnet101/wrn/wrnvar/preresnet/cnn13]: By default, ResNet-50 is used for experiments. We provide alternatives as follows: ResNet-18/101, Wide ResNet, Wide ResNet-Var (WRN-37-2), PreAct ResNet and CNN-13.
• --dataset [semi-fungi/semi-aves] and --data_dir: Your dataset name and path. We support two datasets: Semi-Fungi and Semi-Aves (See "A Realistic Evaluation of Semi-Supervised Learning for Fine-Grained Classification" for details).
• --unlabel [in/inout]: The unlabeled data used for training. The unlabeled data of Semi-Fungi and Semi-Aves contains two subsets. One has in-distribution data only, the other one has both in-distribution and out-of-distribution data.
• --alpha: $\alpha$ for confidence-aware k selection.
• --num_tracked_batch: $N_{B}$ for class transition tracking (CTT).
• --resume, --load_path, --load_path_soc and --pretrained:
  • If you want to resume training without soc.pkl, i.e., saved data of centroids (for CTT-based k-means clustering) and label_matrix (for CTT), please set --resume --load_path @path to your checkpoint@.
  • If you want to resume training with soc.pkl, please set --resume --load_path @path to your checkpoint@ --load_path_soc @path to your soc.pkl@.
  • If you want to use the expert model or MoCo model, please set --resume --load_path @path to expert model/MoCo model@ --pretrained.

Training with Single GPU

python train_soc.py --rank 0 --gpu [0/1/...] @@@other args@@@

Training with Multi-GPUs

• Using DataParallel

python train_soc.py --world-size 1 --rank 0 @@@other args@@@

• Using DistributedDataParallel with single node

python train_soc.py --world-size 1 --rank 0 --multiprocessing-distributed @@@other args@@@

Examples of Running

To better reproduce our experimental results, it is recommended to use multi-GPUs with DataParallel for training.

Using In-distribuion Unlabeleda Data

Training from scratch

python train_soc.py --world-size 1 --rank 0 --seed 1 --num_eval_iter 2000 --overwrite --save_name aves_in_sc --dataset semi_aves --num_classes 200 --unlabel in 

Training from scratch with MoCo

python train_soc.py --world-size 1 --rank 0 --seed 1 --num_eval_iter 1000 --overwrite --save_name aves_in_sc_moco --dataset semi_aves --num_classes 200 --unlabel in --resume --load_path @path to MoCo pre-trained model@ --pretrained --num_train_iter 200000

Training from expert or expert with MoCo

python train_soc.py --world-size 1 --rank 0 --seed 1 --num_eval_iter 500 --overwrite --save_name aves_in_pre --dataset semi_aves --num_classes 200 --unlabel in --resume --load_path @path to pre-trained model@ --pretrained --lr 0.001 --num_train_iter 50000

The expert models and MoCo models can be obtained here (provided by https://github.com/cvl-umass/ssl-evaluation).

Using Out-of-Distribuion Unlabeleda Data

Training from scratch

python train_soc.py --world-size 1 --rank 0 --seed 1 --num_eval_iter 2000 --overwrite --save_name aves_inout_sc --dataset semi_aves --num_classes 200 --unlabel inout 

---

Evaluation

Each time you start training, the evaluation results of the current model will be displayed. If you want to evaluate a model, use its checkpoints to resume training, i.e., use --resume --load_path @path to your checkpoint@.

Results (e.g. seed=1)

Dateset	Unlabeled Data	Pre-training	Top1-Acc (%)	Top5-Acc (%)	Checkpoint
Semi-Aves	in-distribution	-	32.3	55.5	here
		MoCo	39.5	62.5	here
		ImageNet	56.8	79.1	here
		ImageNet + MoCo	57.1	79.1	here
		iNat	71.0	88.4	here
		iNat + MoCo	70.2	88.3	here
	out-of-distribution	-	27.5	50.7	here
		MoCo	40.4	65.9	here
Semi-Fungi	in-distribution	-	38.50	61.35	here
		MoCo	46.9	71.4	here
		ImageNet	61.1	83.2	here
		ImageNet + MoCo	61.8	85.9	here
		iNat	62.3	85.0	here
		iNat + MoCo	62.2	84.4	here
	out-of-distribution	-	35.6	60.6	here
		MoCo	50.0	74.8	here