saliency-metrics

An Open-source Framework for Benchmarking Explanation Methods in Computer Vision.

---

⚠️ Note: The first version (0.1.0) of this package is still a work in progress.

Supported Benchmarks

• Sanity Check
• Sensitivity-N
• Insertion/Deletion
• ROAR

Contribution Guideline

Set up environment

1. Fork the repo and then clone it.
2. ⚠️ Install python >= 3.8.
3. Install pre-commit hooks: pip install pre-commit and then pre-commit install. The hooks will be automatically triggered when running git commit. One can also manually trigger the hooks with pre-commit run --all-files.
4. (Optional) Install torch.
5. Install the requirements for development: pip install -r requirements_dev.txt.
6. Install the package in editable mode pip install -e . .
7. (Optional) Add custom directories in .git/info/exclude to make them ignored by git. Note that some common directories like .idea, .vscode are already in the .gitignore file.

Code linter, formatter, and unit tests

• flake8
• isort
• black
• mypy
• pytest. Run unit tests with sh tests/run_tests.sh

Build docs locally

1. Go to docs/ and install the requirements: cd docs/ && pip install -r requirements_doc.txt.
2. Now the current directory should be under docs/. Build the html webpage: make html.
3. Go to docs/build/ and then host the webpage locally: cd build/ && python -m http.server <port> , where port is a number (e.g., 1234).
4. Open the webpage localhost:<port> in a browser.