Paper Archive Link	AAAI Conference Link	LIME_Stratified	LIME_Stratified Exampless

lime-stratified-examples

• This Repositry contains codes to reproduce results for our publication 'Using stratified sampling to improve LIME Image explanations' published at Proceedings of the AAAI Conference on Artificial Intelligence by Modifying Lime-Image and by introducing a novel sampling stragtegy into LIME-Image. The proposed strategy merged with Orignal LIME and named as LIME_STRATIFIED.

• The Effect of introducing this sampling stratgey is presented in Results Section and can be validated by running codes in next few sections.

Using Stratified Sampling to Improve LIME Image Explanations

PDF:

Published Paper

Authors ✍️:

Muhammad Rashid¹, Elvio G. Amparore¹, Enrico Ferrari², Damiano Verda²

1. University of Torino, Computer Science Department, C.so Svizzera 185, 10149 Torino, Italy, http://www.di.unito.it/do/home.pl
2. Rulex Innovation Labs, Rulex Inc., Via Felice Romani 9, 16122 Genova, Italy, https://www.rulex.ai/

Paper contribution 📃

In this paper we:

• investigate the distribution of the dependent variable in the sampled synthetic neighborhood of LIME Image, identifying in the undersampling a cause that results in inadequate explanations;
• delve into the causes of the synthetic neighborhood inad- equacy, recognizing a link with the Shapley theory;
• reformulate the synthetic neighborhood generation using an unbiased stratified sampling strategy;
• provide empirical proofs of the advantage of using strat- ified sampling for LIME Image on a popular dataset.

🔧 Dependencies and Installation

• Python
• Tensorflow
• Option: NVIDIA GPU + CUDA

Install all the required libraries by running following line:

git clone https://github.com/rashidrao-pk/lime-stratified-examples/
cd /lime-stratified-examples
git clone https://github.com/rashidrao-pk/lime_stratified
pip install -r requirements.txt 

Structure of the artifact 💻

This artifact is structured as follows:

• the data/ folder contains a subset of the ImageNet Object Localization Challenge (the first 150 images) used to generate the plots and benchmarks of the paper;

• the lime_stratified/ that contains a modified version of LIME with the proposed stratified sampling approach.

• the results/ folder, which contains the results after running the artifact.

• two notebooks:

  • Run_Benchmark.ipynb runs the (long) benchmark on a subset of the ImageNet Object Localization Challenge, generating the CSV file in the results/ folder. Inside the file there is an option to also run a "simplified" version of the benchmark (just one image) instead of the "full" version. The simplified version will run all the steps, but on a single image.

  • Paper_Figures.ipynb which reads the CSV file and generates the plots included in the paper.

The precomputed CSV file is already included in the results/ folder.

---

How to run the artifact 📊

The Run_Benchmark.ipynb will take a long tame (more than 1 day) to run all the experiments and generate the CSV file. All explanations are computed 10 times and then averaged, to better stabilize the results and reduce randomization.

The Paper_Figures.ipynb has several targets: for the single explanations, it will take a few minutes. For the plots generated starting from the CSV file, in will take a few seconds.

---

Run Examples

You can run Run_Benchmark.ipynb to regenerate all the results. Alternatively, the precomputed CSV file used for the paper is included.

Then run Paper_Figures.ipynb to generate the plots.

Performance Measures to Evaluate Explanations: 🎯

How can we evaluate an XAI method for images? to decide the quality of an explanation
we have utilised few evaluation metrices to quantify the base Lime-Image approach and lime_stratified approach including.

1. Visual Evaluation
   • Heatmaps for explanation
2. Statistical Evaluation
   • Coeffecient of Variation $(CV(\beta))$
   • Inter-Quantile Range Coverage $(RC_Y)$
   • $R_2$ Score from Explanations

Cite

If you use our proposed strategy, please cite us:

Rashid, M., Amparore, E. G., Ferrari, E., & Verda, D. (2024). Using Stratified Sampling to Improve LIME Image Explanations. Proceedings of the AAAI Conference on Artificial Intelligence, 38(13), 14785-14792. https://doi.org/10.1609/aaai.v38i13.29397 

Motivation behind lime-stratified 💪

How LIME Image supposed to work and how does it actually works

Workflow of LIME Image

We breifly overview and the Workflow of LIME Image is presented in the following figure.

Keywords 🔍

XAI · LIME · Stratified Sampling . ML: Transparent, Interpretable, Explainable ML, RU: Stochastic Optimization, SO: Sampling/Simulation-based Search