R. Hesse, S. Schaub-Meyer, and S. Roth. FunnyBirds: A Synthetic Vision Dataset for a Part-Based Analysis of Explainable AI Methods. ICCV, 2023, oral presentation.
⚠️ Disclaimer: This repository will contain the full code to reproduce our work, including the dataset rendering, the custom evaluations, and the framework code for all methods (code coming soon). As this code can be a bit unclear, we additionally provide a separat FunnyBirds framework repository that provides the minimal working code to run your own evaluations on the FunnyBirds framework. If you just want to run the framework evaluation, we thus recommend using the FunnyBirds framework repository.
News:
- The FunnyBirds framework repository is ready to evaluate your own methods (September 13, 2023)
- You can now render the dataset yourself (October 23, 2023)
- We have added the code for the custom evaluations (February 14, 2024)
The following section provides a very detailed description of how to use the FunnyBirds framework. Even if the instructions might seem a bit long and intimidating, most of the steps are finished very quickly. So don't lose hope and if you have recommendations on how to improve the framework or the instructions, we would be grateful for your feedback.
The dataset requires ~1.6GB free disk space.
cd /path/to/dataset/
wget download.visinf.tu-darmstadt.de/data/funnybirds/FunnyBirds.zip
unzip FunnyBirds.zip
rm FunnyBirds.zip
If you use conda you can create your environment as shown below:
conda create --name funnybirds-framework python=3.7
conda activate funnybirds-framework
conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia
conda install captum -c pytorch
conda install -c conda-forge tqdm
conda install -c anaconda scipy
conda install -c conda-forge nodejs
pip install opencv-contrib-python (conda install -c conda-forge opencv messes the environment up)
To render the FunnyBirds dataset, please refer to the render folder.
To run the custom evaluations, please refer to the custom_evaluation folder.