Deep Neural Networks Tend To Extrapolate Predictably

Code for reproducing the experiments in Deep Neural Networks Tend To Extrapolate Predictably.

Setup

To install the necessary packages for this codebase, run:

conda create -n cautious_extrapolation python=3.7
conda activate cautious_extrapolation
pip install -e .

To download the datasets needed for training, please follow the directions specified in the following links:

• CIFAR10, CIFAR10-C
• ImageNet, ImageNet-R(endition), ImageNet-Sketch
• MNIST
• OfficeHome
• SkinLesionPixels
• UTKFace
• Amazon

Update cautious_extrapolation/data_paths.py to include the directory paths in which the datasets were downloaded.

Usage

This codebase is organized such that each dataset is associated with a folder inside cautious_extrapolation\. To train a model on a particular dataset, navigate to the folder associated with the dataset, and run:

python train.py [args]

To evaluate the model on the holdout and OOD datasets, run:

python eval.py --run-name=[run name] [other args]

To reproduce the figures in our paper, please see plot.ipynb and analyze.ipynb.

Code for BREEDS living-17 and non-living-26 are coming soon!

Acknowledgements

The codebase is built on top of multiple publicly available repos:

• pytorch_resnet_cifar10
• wilds
• examples
• DomainBed
• UTK-Face
• regression_uncertainty