Skip to content
master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
lib
 
 
 
 
 
 
 
 
 
 

Deep Model Transferbility from Attribution Maps

Getting Started

These instructions below will get you a copy of the project up and running on your local machine for development and testing purposes.

Prerequisites

Install the following:

- Python >= 3.6
- Tensorflow >= 1.10.0
- Matlab R2019a

Then, install python packages:

pip install -r requirements.txt

In order to generate attribution maps from Deep Models, you also need to download DeepExplain which this project utilizes to generate attribution maps, and copy it to your project directory $DIR.

cp -r DeepExplain-master/deepexplain $DIR/lib/

Probe datasets

Those datasets involved in this project are:

Make sure to download them and move to $DIR/dataset.

Those datasets need to be arranged in the following format:

|- dataset
|   |---taskonomy
|   |   |---collierville_rgb
|   |   |   |---point_0_view_0_domain_rgb.png
|   |   |   |---...
|   |   |---corozal_rgb
|   |   |---darden_rgb
|   |   |---markleeville_rgb
|   |   |---wiconisco_rgb
|   |---coco
|   |   |---COCO_val2014_000000000042.jpg
|   |   |---...
|   |---indoor
|   |   |---Images
|   |   |   |---airport_inside
|   |   |   |   |---airport_inside_0001.jpg
|   |   |   |---bowling
|   |   |   |---...

You can also check $DIR/explain_result/name_of_dataset/imlist.txt to find out how those images are arranged(Note that images in the imlist.txt are randomly selected from dataset, please feel free to test using more images).

Pre-trained Models

Download pre-trained models:

sh tools/download_model.sh

Running the tests

First, generate Attribution Maps and save corresponding Attribution maps to explain results directory:

cd tools
python deep_attribution.py --explain_result_root explain_result --dataset taskonomy --imlist-size 1000
python deep_attribution.py --explain_result_root explain_result --dataset coco --imlist-size 1000
python deep_attribution.py --explain_result_root explain_result --dataset indoor --imlist-size 1005

Calculate affinity matrix of those tasks according to the Attribution Maps:

python affinity.py --dataset taskonomy --imlist-size 1000
python affinity.py --dataset coco --imlist-size 1000
python affinity.py --dataset indoor --imlist-size 1005

Visualization

Plot P@K, R@K Curve, make sure you've already completed testing on three datasets:

python plot.py --fig-save fig

e.g.

图片名称

Plot Task Similarity Tree:

matlab -nosplash -nodesktop task_similarity_tree.m

e.g.

图片名称

Todo

Release code of experiment generating attribution maps based on other kinds of pre-trained models(different tasks, different architectures, etc.).

Apply more explain methods and see how they work.

Acknowledgement

This repo is built upon the code from Taskonomy.

And we thank deepexplain for providing the attribution tools.

Citation

If you find this code useful, please cite the following:

@inproceedings{ModelTransfer_NeurIPS2019,
  title={Deep Model Transferbility from Attribution Maps},
  author={Jie Song, Yixin Chen, Xinchao Wang, Chengchao Shen, Mingli Song},
  booktitle={NeurIPS},
  year={2019}
}

Contact

If you have any question, please feel free to contact

Jie Song, sjie@zju.edu.cn;

Yixin Chen, chenyix@zju.edu.cn.

Homepage of VIPA Group, Zhejiang University, China

icon

About

(NeurIPS 2019) Deep Model Transferbility from Attribution Maps

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages