BNN-concepts

PyTorch implementation of the paper "Bayesian Neural Networks Avoid Encoding Complex and Perturbation-Sensitive Concepts" in ICML 2023 (pdf).

Requirements

• Python 3.9
• pytorch 1.7.1
• CUDA 11.0
• numpy 1.22.3
• torchvision 0.8.2

Models were trained and tested on the NVIDIA GeForce RTX 3080 GPU.

Usage

Training

Use the following script to train the models.

./scripts/bash_train.sh

You can change the --gpu_id argument in the script to specify the index of GPU on your machine. Model parameters will be saved to the checkpoints folder by default.

Verifying the sensitivity of high-order interactions to perturbations

Use the following script to run the experiment and reproduce the results in Figure 3 in the paper.

./scripts/bash_verify_stability.sh

Results will be saved to the results/verify_stability folder by default.

Demonstrating BNNs are less likely to encode high-order interactions

Use the following script to run the experiment and reproduce the results in Figure 4 in the paper.

./scripts/bash_compare_BNN_with_std_DNN.sh

Results will be saved to the results/compare_BNN_with_std_DNN folder by default.

Citation

@inproceedings{ren2023bayesian,
  title={Bayesian Neural Networks Avoid Encoding Complex and Perturbation-Sensitive Concepts},
  author={Ren, Qihan and Deng, Huiqi and Chen, Yunuo and Lou, Siyu and Zhang, Quanshi},
  booktitle={International Conference on Machine Learning},
  pages={28889--28913},
  year={2023},
  volume={202},
  series={Proceedings of Machine Learning Research},
  month={23--29 Jul},
  publisher={PMLR},
}