This repository contains code for the paper New Definitions and Evaluations for Saliency Methods : Staying Intrinsic, Complete, and Sound. NeuRIPS 2022.

To cite this paper, please use the following BibTex entry:

@inproceedings{2022soundness,
  title = {New Definitions and Evaluations for Saliency Methods : Staying Intrinsic, Complete, and Sound},
  author = {Arushi Gupta, Nikunj Saunshi, Dingli Yu, Kaifeng Lyu, Sanjeev Arora},
  booktitle = {NeuRIPS},
  year = {2022}
}

We now describe how to run our simple masking method and compute completeness and soundness on the resulting maps.

#First, please run save noise batch to save the batch of images that will be used to create Gamma for our method #You may not get the exact same results we did because we changed batch_size to 1000 in save_noise_batch.py so we had 1000 noise images. Here we have set it to 100 because of github file size limitations

python3 save_noise_batch.py

#Next, we generate some random sample inputs (of course please replace with your own inputs if need be) #We have generated these images so their labels lie in [526 527 664 673 782 851] python3 generate_random_inputs.py

#This should save in the folder input_data 2 files. sample_images.npy and sample_labels.npy

#Next, to run our saliency method please type

python saliency_certificate_Imagenet_desktop.py  --label_list 526  527 664 673 782 851  --image_input_path input_data/sample_images.npy --label_input_path input_data/sample_labels.npy  --steps 2000 --lr 0.05 \
--bs 10 --noise_mode rand_image --noise_bs 10 --K 1 \
--scale_list 4 --reg_l1_list 0.00002 --reg_tv_list 0.01 \
--fit_label correct  \
--start_end 0 300 --debug

#Where label_list is the list of relevant labels from ImageNet (to save the computational burden of computing on 1000 labels) and image_input_path is the path to the images file and label_input_path is the path to the relevant labels.

#Finally, to compute completeness and soundness, please run

python mask_like_certifier_imagenet_desktop.py --label_list 526  527 664 673 782 851   --image_input_path input_data/sample_images.npy --label_input_path input_data/sample_labels.npy  --steps 200 --lr 0.05 --bs 10 --noise_mode rand_image --noise_bs 10 --K 1 \
--scale_list 4 --reg_l1_list 0.00002 --reg_tv_list 0.01 --fit_label correct \
--start_end 0 50 --debug 

#where start_end 0 50 means indices 0 through 50 will be processed