Skip to content

An automatic and efficient tool to describe functionalities of individual neurons in DNNs

Notifications You must be signed in to change notification settings

Trustworthy-ML-Lab/CLIP-dissect

Repository files navigation

CLIP-Dissect

An automatic and efficient tool to describe functionalities of individual neurons in DNNs.

This is the official repository for our paper: CLIP-Dissect: Automatic Description of Neuron Representations in Deep Vision Networks published at ICLR 2023.

Update 6/5/23: We have conducted a crowdsourced evaluation of our description quality, results are available on arxiv (Appendix B).

Overview

Installation

  1. Install Python (3.10)
  2. Install Pytorch (tested with 1.12.0, also works with 2.0) and Torchvision >= 0.13 following instructions from https://pytorch.org/get-started/previous-versions/
  3. Install remaining requirements using pip install -r requirements.txt
  4. Download the Broden dataset (images only) using bash dlbroden.sh
  5. (Optional) Download ResNet-18 pretrained on Places-365: bash dlzoo_example.sh

We do not provide download instructions for ImageNet data, to evaluate using your own copy of ImageNet validation set you must set the correct path in DATASET_ROOTS["imagenet_val"] variable in data_utils.py.

Quickstart:

This will dissect 5 layers of ResNet-50(ImageNet) using Broden as the probing dataset. Results will be saved in results/resnet50_{datetime}/descriptions.csv.

python describe_neurons.py

Recreating experiments

The results used for figures and tables of our paper can be recreated by running the corresponding notebook in the experiments folder, for example to reproduce Table 1 run experiments/table1.ipynb.

How to modify:

Dissecting your own model

  1. Implement the code to load your model(in eval mode) and a preprocess function to correctly load images for your model in get_target_model function of data_utils.py under an if statement for target_name of you choice.
  2. Dissect the model by running python describe_neurons.py --target_model {model_name}

Using your own probing dataset

  1. Implement code to load your dataset as a torchvision DataSet uin the get_data function of dataset_utils.py
  2. Add your dataset name into the choices of --d_probe argument in describe_neurons.py
  3. Dissect the model by running python describe_neurons.py --d_probe {dataset_name}

Using your own concept set

  1. Create/download a .txt file containing your concept set, which each concept on a separate line
  2. Dissect the model by running python describe_neurons.py --concept_set {path_to_conceptset}

Specifying device

You can specify which device is used with the --device argument, which defaults to cuda, i.e. python describe_neurons.py --device cpu

Sources:

Common errors

Incorrect activations cached:

The code automatically caches the saved activations of target model and CLIP in saved_activations, and if a file already exists with the same save name the code will load these activations instead of recalculating. However sometimes you may wish to modify the pipeline in a way that doesn't change the name of the saved activations and want to recalculate the activations. In this case you need to manually delete the relevant files from saved_activations before rerunning CLIP-Dissect, as using incorrect activations will give incorrect results.

Cite this work

T. Oikarinen and T.-W. Weng, CLIP-Dissect: Automatic Description of Neuron Representations in Deep Vision Networks, ICLR 2023.

@article{oikarinen2023clip,
  title={CLIP-Dissect: Automatic Description of Neuron Representations in Deep Vision Networks},
  author={Oikarinen, Tuomas and Weng, Tsui-Wei},
  journal={International Conference on Learning Representations},
  year={2023}
}

About

An automatic and efficient tool to describe functionalities of individual neurons in DNNs

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages