Prerequisites

This repo is built on top of USB. USB is built on pytorch, with torchvision, torchaudio, and transformers.

To install the required packages, you can create a conda environment:

conda create --name usb python=3.8

then use pip to install required packages:

pip install -r requirements.txt

From now on, you can start use USB by typing

python train.py --c config/usb_cv/fixmatch/fixmatch_cifar100_200_0.yaml

Running the Calibration Method

You can modify the config files to add the two parameters, margin and the weight to the penalty term. For example,

margin_hyperparam: 10
p_margin: 0.1

Alternatively, you can run with a modified config file :

python train.py --c config/usb_cv/fixmatch/fixmatch_cifar100_200_0_penalty.yaml

To evaluate the model for ECE and Errors :

python eval.py --dataset cifar100 --num_classes 100 --load_path ./saved_model/best_model.pth

Cite Us

Please cite us if you find this project helpful for your project/paper:

@misc{mishra2024trust,
      title={Do not trust what you trust: Miscalibration in Semi-supervised Learning}, 
      author={Shambhavi Mishra and Balamurali Murugesan and Ismail Ben Ayed and Marco Pedersoli and Jose Dolz},
      year={2024},
      eprint={2403.15567},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}