CartoonX (Cartoon Explanation) is a novel model-agnostic explanation method tailored towards image classifiers and based on the rate-distortion explanation (RDE) framework. Natural images are roughly piece-wise smooth signals—also called cartoon-like images and tend to be sparse in the wavelet domain. CartoonX is the first explanation method to exploit this by requiring its explanations to be sparse in the wavelet domain, thus extracting the relevant piece-wise smooth part of an image instead of relevant pixel-sparse regions.
Python 3.7.x and newer are supported:
# clone project
git clone https://github.com/skmda37/CartoonX.git
# Enter directocry
cd CartoonX
# Create and activate virtual environment (or conda environment)
python -m venv env
source env/bin/activate
# install pytorch wavelets package (see https://pytorch-wavelets.readthedocs.io/en/latest/readme.html for the docs)
git clone https://github.com/fbcotter/pytorch_wavelets
cd pytorch_wavelets
pip install .
pip install -r tests/requirements.txt
pytest tests/
cd ..
# install other project dependencies from requirements file
pip install -r requirements.txtThere are two folders:
First, do
cd code/. Then you can explain models with CartoonX in visualize_example_explanations.ipynb. The actual code for CartoonX is in ./code/cartoonx.py.
@inproceedings{kolek2023explaining,
title={Cartoon Explanations of Image Classifiers},
author={Kolek, Stefan and Nguyen, Duc Anh and Levie, Ron and Bruna, Joan and Kutyniok, Gitta},
booktitle={European Conference of Computer Vision (ECCV)},
year={2022}
}
