Hierarchical Visual Categories Modeling: A Joint Representation Learning and Density Estimation Framework for Out-of-Distribution Detection
- python 3.8.13
- cuda 11.1
- numpy 1.22.4
- Pillow 9.4.0
- progress 1.6
- scikit_learn 1.2.2
- scipy 1.7.3
- torch 1.13.1
- torchvision 0.14.1
Please download ImageNet-1k and place the training data and validation data in ./hvcm_imagenet/data/train and ./hvcm_imagenet/data/val, respectively.
we follow MOS and use Texture, iNaturalist, Places365 and SUN, and de-duplicated concepts overlapped with ImageNet-1k. To further explore the limitation of our approach, we follow VIM and use ImageNet-O and OpenImage-O.
For iNaturalist, SUN, and Places, we have sampled 10,000 images from the selected concepts for each dataset, which can be download via the following links:
wget http://pages.cs.wisc.edu/~huangrui/imagenet_ood_dataset/iNaturalist.tar.gz
wget http://pages.cs.wisc.edu/~huangrui/imagenet_ood_dataset/SUN.tar.gz
wget http://pages.cs.wisc.edu/~huangrui/imagenet_ood_dataset/Places.tar.gz
For Textures, we use the entire dataset, which can be downloaded from their original website.
ImageNet-O and OpenImage-O can be Download from VIM.
Please put all downloaded OOD datasets into ./hvcm_imagenet/data/ood_data/.
Please download Pre-trained models and place in the ./hvcm_imagenet/results folder.
The downloading process will start immediately upon running.
we follow Energy and KNN and use Texture, SVHN, Place365, iSUN, LSUN-Crop, LSUN-Resize.
We provide links and instructions to download each dataset:
- SVHN: download it and place it in the folder of
./hvcm_cigar10/data/svhn. Then runpython select_svhn_data.pyto generate test subset. - Texture: download it and place it in the folder of
./hvcm_cigar10/data/dtd - Places365download it and place it in the folder of
./hvcm_cifar10/data/places/test_256. We randomly sample 10,000 images from the original test dataset by runningpython sample_places.py. - LSUN-C:
download it and place it in the folder of
./hvcm_cigar10/data/LSUN/. - LSUN-Rdownload it and place it in the folder of
./hvcm_cigar10/data/LSUN-R/. - iSUN:
download it and place it in the folder of
./hvcm_cigar10/data/iSUN/.
Please download Pre-trained models and place in the ./hvcm_cifar10/results/cifar10/ResNet18Gram/cifar_hvcm folder.
- Demo code for ImageNet Experiment
Run HVCM with ResNet-50 network on a single node with 4 GPUs for 300 epochs with the following command.
cd hvcm_imagenet
sh main.sh
Run sh ind_acc.sh for calculating in-distribution accuracy.
Run sh get_gau.sh for getting GMMs components.
Runsh ood_maha.sh for OOD detection.
- Demo code for CIFAR Experiment
cd hvcm_cifar10
sh train.sh
Run sh ind_acc.sh for calculating in-distribution accuracy.
Run sh get_gau.sh for getting GMMs components.
Run sh ood_maha.sh for OOD detection.
Emerging Properties in Self-Supervised Vision Transformers
Regularizing Class-wise Predictions via Self-knowledge Distillation
MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Space
Energy-based Out-of-distribution Detection
Out-of-distribution Detection with Deep Nearest Neighbors
@InProceedings{Li2023hvcm,
author = {Li, Jinglun and Zhou, Xinyu and Guo, Pinxue and Sun, Yixuan and Huang, Yiwen and Ge, Weifeng and Zhang, Wenqiang},
title = {Hierarchical Visual Categories Modeling: A Joint Representation Learning and Density Estimation Framework for Out-of-Distribution Detection},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2023},
pages = {23425-23435}
}