Skip to content
(NeurIPS 2019) Deep Model Transferbility from Attribution Maps
Python Other
  1. Python 99.5%
  2. Other 0.5%
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
dataset
demo
experiments/final
explain_result
lib
tools
.DS_Store
.gitattributes
LICENSE
README.md
requirement.txt
vipa-logo.png

README.md

Deep Model Transferbility from Attribution Maps

Getting Started

These instructions below will get you a copy of the project up and running on your local machine for development and testing purposes.

Prerequisites

Install the following:

- Python >= 3.6
- Tensorflow >= 1.10.0
- Matlab R2019a

Then, install python packages:

pip install -r requirements.txt

In order to generate attribution maps from Deep Models, you also need to download DeepExplain which this project utilizes to generate attribution maps, and copy it to your project directory $DIR.

cp -r DeepExplain-master/deepexplain $DIR/lib/

Probe datasets

Those datasets involved in this project are:

Make sure to download them and move to $DIR/dataset.

Those datasets need to be arranged in the following format:

|- dataset
|   |---taskonomy
|   |   |---collierville_rgb
|   |   |   |---point_0_view_0_domain_rgb.png
|   |   |   |---...
|   |   |---corozal_rgb
|   |   |---darden_rgb
|   |   |---markleeville_rgb
|   |   |---wiconisco_rgb
|   |---coco
|   |   |---COCO_val2014_000000000042.jpg
|   |   |---...
|   |---indoor
|   |   |---Images
|   |   |   |---airport_inside
|   |   |   |   |---airport_inside_0001.jpg
|   |   |   |---bowling
|   |   |   |---...

You can also check $DIR/explain_result/name_of_dataset/imlist.txt to find out how those images are arranged(Note that images in the imlist.txt are randomly selected from dataset, please feel free to test using more images).

Pre-trained Models

Download pre-trained models:

sh tools/download_model.sh

Running the tests

First, generate Attribution Maps and save corresponding Attribution maps to explain results directory:

cd tools
python deep_attribution.py --explain_result_root explain_result --dataset taskonomy --imlist-size 1000
python deep_attribution.py --explain_result_root explain_result --dataset coco --imlist-size 1000
python deep_attribution.py --explain_result_root explain_result --dataset indoor --imlist-size 1005

Calculate affinity matrix of those tasks according to the Attribution Maps:

python affinity.py --dataset taskonomy --imlist-size 1000
python affinity.py --dataset coco --imlist-size 1000
python affinity.py --dataset indoor --imlist-size 1005

Visualization

Plot P@K, R@K Curve, make sure you've already completed testing on three datasets:

python plot.py --fig-save fig

e.g.

图片名称

Plot Task Similarity Tree:

matlab -nosplash -nodesktop task_similarity_tree.m

e.g.

图片名称

Todo

Release code of experiment generating attribution maps based on other kinds of pre-trained models(different tasks, different architectures, etc.).

Apply more explain methods and see how they work.

Acknowledgement

This repo is built upon the code from Taskonomy.

And we thank deepexplain for providing the attribution tools.

Citation

If you find this code useful, please cite the following:

@inproceedings{ModelTransfer_NeurIPS2019,
  title={Deep Model Transferbility from Attribution Maps},
  author={Jie Song, Yixin Chen, Xinchao Wang, Chengchao Shen, Mingli Song},
  booktitle={NeurIPS},
  year={2019}
}

Contact

If you have any question, please feel free to contact

Jie Song, sjie@zju.edu.cn;

Yixin Chen, chenyix@zju.edu.cn.

Homepage of VIPA Group, Zhejiang University, China

icon
You can’t perform that action at this time.