examples/integrated_gradients_imdb.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Integrated gradients for text classification on the IMDB dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example, we apply the integrated gradients method to a sentiment analysis model trained on the IMDB dataset. In text classification models, integrated gradients define an attribution value for each word in the input sentence. The attributions are calculated considering the integral of the model  gradients with respect to the word embedding layer along a straight path from a baseline instance $x^\\prime$ to the input instance $x.$ A description of the method can be found [here](https://docs.seldon.io/projects/alibi/en/latest/methods/IntegratedGradients.html). Integrated gradients was originally proposed in Sundararajan et al., [\"Axiomatic Attribution for Deep Networks\"](https://arxiv.org/abs/1703.01365)\n",
    "\n",
    "The IMDB data set contains 50K movie reviews labelled as positive or negative. \n",
    "We train a convolutional neural network classifier with a single 1-d convolutional layer followed by a fully connected layer. The reviews in the dataset are truncated at 100 words and each word is represented by 50-dimesional word embedding vector. We calculate attributions for the elements of the embedding layer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "TF version:  2.3.1\n",
      "Eager execution enabled:  True\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "import os\n",
    "import pandas as pd\n",
    "from tensorflow.keras.datasets import imdb\n",
    "from tensorflow.keras.preprocessing import sequence\n",
    "from tensorflow.keras.models import Model\n",
    "from tensorflow.keras.layers import Input, Dense, Embedding, Conv1D, GlobalMaxPooling1D, Dropout \n",
    "from tensorflow.keras.utils import to_categorical\n",
    "from alibi.explainers import IntegratedGradients\n",
    "import matplotlib.pyplot as plt\n",
    "print('TF version: ', tf.__version__)\n",
    "print('Eager execution enabled: ', tf.executing_eagerly()) # True"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Loading the imdb dataset. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "max_features = 10000\n",
    "maxlen = 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loading data...\n",
      "25000 train sequences\n",
      "25000 test sequences\n",
      "Pad sequences (samples x time)\n",
      "x_train shape: (25000, 100)\n",
      "x_test shape: (25000, 100)\n"
     ]
    }
   ],
   "source": [
    "print('Loading data...')\n",
    "(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=max_features)\n",
    "test_labels = y_test.copy()\n",
    "train_labels = y_train.copy()\n",
    "print(len(x_train), 'train sequences')\n",
    "print(len(x_test), 'test sequences')\n",
    "y_train, y_test = to_categorical(y_train), to_categorical(y_test)\n",
    "\n",
    "print('Pad sequences (samples x time)')\n",
    "x_train = sequence.pad_sequences(x_train, maxlen=maxlen)\n",
    "x_test = sequence.pad_sequences(x_test, maxlen=maxlen)\n",
    "print('x_train shape:', x_train.shape)\n",
    "print('x_test shape:', x_test.shape)\n",
    "\n",
    "index = imdb.get_word_index()\n",
    "reverse_index = {value: key for (key, value) in index.items()} "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A sample review from the test set. Note that unknown words are replaced with 'UNK'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def decode_sentence(x, reverse_index):\n",
    "    # the `-3` offset is due to the special tokens used by keras\n",
    "    # see https://stackoverflow.com/questions/42821330/restore-original-text-from-keras-s-imdb-dataset\n",
    "    return \" \".join([reverse_index.get(i - 3, 'UNK') for i in x])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "a powerful study of loneliness sexual UNK and desperation be patient UNK up the atmosphere and pay attention to the wonderfully written script br br i praise robert altman this is one of his many films that deals with unconventional fascinating subject matter this film is disturbing but it's sincere and it's sure to UNK a strong emotional response from the viewer if you want to see an unusual film some might even say bizarre this is worth the time br br unfortunately it's very difficult to find in video stores you may have to buy it off the internet\n"
     ]
    }
   ],
   "source": [
    "print(decode_sentence(x_test[1], reverse_index)) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The model includes one convolutional layer and reaches a test accuracy of 0.85. If `save_model = True`, a local folder `../model_imdb` will be created and the trained model will be saved in that folder. If the model was previously saved, it can be loaded by setting `load_model = True`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "batch_size = 32\n",
    "embedding_dims = 50\n",
    "filters = 250\n",
    "kernel_size = 3\n",
    "hidden_dims = 250"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "load_model = False\n",
    "save_model = True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "filepath = './model_imdb/'  # change to directory where model is downloaded\n",
    "if load_model:\n",
    "    model = tf.keras.models.load_model(os.path.join(filepath, 'model.h5'))\n",
    "else:\n",
    "    print('Build model...')\n",
    "    \n",
    "    inputs = Input(shape=(maxlen,), dtype='int32')\n",
    "    embedded_sequences = Embedding(max_features,\n",
    "                                   embedding_dims)(inputs)\n",
    "    out = Conv1D(filters, \n",
    "                 kernel_size, \n",
    "                 padding='valid', \n",
    "                 activation='relu', \n",
    "                 strides=1)(embedded_sequences)\n",
    "    out = Dropout(0.4)(out)\n",
    "    out = GlobalMaxPooling1D()(out)\n",
    "    out = Dense(hidden_dims, \n",
    "                activation='relu')(out)\n",
    "    out = Dropout(0.4)(out)\n",
    "    outputs = Dense(2, activation='softmax')(out)\n",
    "        \n",
    "    model = Model(inputs=inputs, outputs=outputs)\n",
    "    model.compile(loss='categorical_crossentropy',\n",
    "                  optimizer='adam',\n",
    "                  metrics=['accuracy'])\n",
    "\n",
    "    print('Train...')\n",
    "    model.fit(x_train, y_train,\n",
    "              batch_size=256,\n",
    "              epochs=3,\n",
    "              validation_data=(x_test, y_test))\n",
    "    if save_model:  \n",
    "        if not os.path.exists(filepath):\n",
    "            os.makedirs(filepath)\n",
    "        model.save(os.path.join(filepath, 'model.h5')) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Calculate integrated gradients"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The integrated gradients attributions are calculated with respect to the embedding layer for 10 samples from the test set. Since the model uses a word to vector embedding with vector dimensionality of 50 and sequence length of 100 words, the dimensionality of the attributions is (10, 100, 50). In order to obtain a single attribution value for each word, we sum all the attribution values for the 50 elements of each word's vector representation.\n",
    " \n",
    "The default baseline is used in this example which is internally defined as a sequence of zeros. In this case, this corresponds to a sequence of padding characters (**NB:** in general the numerical value corresponding to a \"non-informative\" baseline such as the PAD token will depend on the tokenizer used, make sure that the numerical value of the baseline used corresponds to your desired token value to avoid surprises). The path integral is defined as a straight line from the baseline to the input image. The path is approximated by choosing 50 discrete steps according to the Gauss-Legendre method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "n_steps = 50\n",
    "method = \"gausslegendre\"\n",
    "internal_batch_size = 100\n",
    "nb_samples = 10\n",
    "ig  = IntegratedGradients(model,\n",
    "                          layer=model.layers[1],\n",
    "                          n_steps=n_steps, \n",
    "                          method=method,\n",
    "                          internal_batch_size=internal_batch_size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_test_sample = x_test[:nb_samples]\n",
    "predictions = model(x_test_sample).numpy().argmax(axis=1)\n",
    "explanation = ig.explain(x_test_sample, \n",
    "                         baselines=None, \n",
    "                         target=predictions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'name': 'IntegratedGradients',\n",
       " 'type': ['whitebox'],\n",
       " 'explanations': ['local'],\n",
       " 'params': {'method': 'gausslegendre',\n",
       "  'n_steps': 50,\n",
       "  'internal_batch_size': 100,\n",
       "  'layer': [1]}}"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Metadata from the explanation object\n",
    "explanation.meta"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['attributions', 'X', 'baselines', 'predictions', 'deltas', 'target'])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Data fields from the explanation object\n",
    "explanation.data.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Attributions shape: (10, 100, 50)\n"
     ]
    }
   ],
   "source": [
    "# Get attributions values from the explanation object\n",
    "attrs = explanation.attributions[0]\n",
    "print('Attributions shape:', attrs.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sum attributions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Attributions shape: (10, 100)\n"
     ]
    }
   ],
   "source": [
    "attrs = attrs.sum(axis=2)\n",
    "print('Attributions shape:', attrs.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualize attributions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "i = 1\n",
    "x_i = x_test_sample[i]\n",
    "attrs_i = attrs[i]\n",
    "pred = predictions[i]\n",
    "pred_dict = {1: 'Positive review', 0: 'Negative review'}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicted label =  1: Positive review\n"
     ]
    }
   ],
   "source": [
    "print('Predicted label =  {}: {}'.format(pred, pred_dict[pred]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can visualize the attributions for the text instance by mapping the values of the attributions onto a matplotlib colormap. Below we define some utility functions for doing this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import HTML\n",
    "def  hlstr(string, color='white'):\n",
    "    \"\"\"\n",
    "    Return HTML markup highlighting text with the desired color.\n",
    "    \"\"\"\n",
    "    return f\"<mark style=background-color:{color}>{string} </mark>\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "def colorize(attrs, cmap='PiYG'):\n",
    "    \"\"\"\n",
    "    Compute hex colors based on the attributions for a single instance.\n",
    "    Uses a diverging colorscale by default and normalizes and scales\n",
    "    the colormap so that colors are consistent with the attributions.\n",
    "    \"\"\"\n",
    "    import matplotlib as mpl\n",
    "    cmap_bound = np.abs(attrs).max()\n",
    "    norm = mpl.colors.Normalize(vmin=-cmap_bound, vmax=cmap_bound)\n",
    "    cmap = mpl.cm.get_cmap(cmap)\n",
    "    \n",
    "    # now compute hex values of colors\n",
    "    colors = list(map(lambda x: mpl.colors.rgb2hex(cmap(norm(x))), attrs))\n",
    "    return colors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below we visualize the attribution values (highlighted in the text) having the highest positive attributions. Words with high positive attribution are highlighted in shades of green and words with negative attribution in shades of pink. Stronger shading corresponds to higher attribution values. Positive attributions can be interpreted as increase in probability of the predicted class (\"Positive sentiment\") while negative attributions correspond to decrease in probability of the predicted class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "words = decode_sentence(x_i, reverse_index).split()\n",
    "colors = colorize(attrs_i)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<mark style=background-color:#faebf3>a </mark><mark style=background-color:#60a22d>powerful </mark><mark style=background-color:#f9eff4>study </mark><mark style=background-color:#f3f6ed>of </mark><mark style=background-color:#c2e596>loneliness </mark><mark style=background-color:#fbe8f2>sexual </mark><mark style=background-color:#faedf3>UNK </mark><mark style=background-color:#f1f6e8>and </mark><mark style=background-color:#f9eef4>desperation </mark><mark style=background-color:#f4f7f0>be </mark><mark style=background-color:#bee490>patient </mark><mark style=background-color:#fbe6f1>UNK </mark><mark style=background-color:#f9eff4>up </mark><mark style=background-color:#f2f6ec>the </mark><mark style=background-color:#7fbc41>atmosphere </mark><mark style=background-color:#ecf6de>and </mark><mark style=background-color:#eba3cd>pay </mark><mark style=background-color:#f5f7f2>attention </mark><mark style=background-color:#f4f7f0>to </mark><mark style=background-color:#f0f6e7>the </mark><mark style=background-color:#286619>wonderfully </mark><mark style=background-color:#f8f5f6>written </mark><mark style=background-color:#f6c7e3>script </mark><mark style=background-color:#f5f7f2>br </mark><mark style=background-color:#f1f6e8>br </mark><mark style=background-color:#f3f6ed>i </mark><mark style=background-color:#eef6e2>praise </mark><mark style=background-color:#f6f7f5>robert </mark><mark style=background-color:#f2badc>altman </mark><mark style=background-color:#fce5f1>this </mark><mark style=background-color:#ecf6de>is </mark><mark style=background-color:#f5f7f3>one </mark><mark style=background-color:#f7f7f6>of </mark><mark style=background-color:#f1f6e8>his </mark><mark style=background-color:#ebf6db>many </mark><mark style=background-color:#f7f6f7>films </mark><mark style=background-color:#f4f7f0>that </mark><mark style=background-color:#529624>deals </mark><mark style=background-color:#f6f7f5>with </mark><mark style=background-color:#dbf0bf>unconventional </mark><mark style=background-color:#42841f>fascinating </mark><mark style=background-color:#f9eff4>subject </mark><mark style=background-color:#c6e79c>matter </mark><mark style=background-color:#fbe8f2>this </mark><mark style=background-color:#f2f6ec>film </mark><mark style=background-color:#f1f6ea>is </mark><mark style=background-color:#f9eff4>disturbing </mark><mark style=background-color:#f7f6f7>but </mark><mark style=background-color:#eff6e4>it's </mark><mark style=background-color:#f1f6e8>sincere </mark><mark style=background-color:#f6f7f5>and </mark><mark style=background-color:#ebf6dc>it's </mark><mark style=background-color:#eff6e4>sure </mark><mark style=background-color:#f3f7ef>to </mark><mark style=background-color:#fbe9f2>UNK </mark><mark style=background-color:#f2f6ec>a </mark><mark style=background-color:#2b691a>strong </mark><mark style=background-color:#b5df82>emotional </mark><mark style=background-color:#fce4f0>response </mark><mark style=background-color:#f6f7f5>from </mark><mark style=background-color:#ecf6de>the </mark><mark style=background-color:#f4f7f0>viewer </mark><mark style=background-color:#faeaf2>if </mark><mark style=background-color:#e4f4cd>you </mark><mark style=background-color:#f8f5f6>want </mark><mark style=background-color:#f4f7f0>to </mark><mark style=background-color:#e1f3c7>see </mark><mark style=background-color:#f9eff4>an </mark><mark style=background-color:#276419>unusual </mark><mark style=background-color:#f8f5f6>film </mark><mark style=background-color:#f5f7f3>some </mark><mark style=background-color:#fbe6f1>might </mark><mark style=background-color:#faedf3>even </mark><mark style=background-color:#f7f7f7>say </mark><mark style=background-color:#fad6ea>bizarre </mark><mark style=background-color:#fce3f0>this </mark><mark style=background-color:#ecf6de>is </mark><mark style=background-color:#b5df82>worth </mark><mark style=background-color:#eff6e5>the </mark><mark style=background-color:#edf6e1>time </mark><mark style=background-color:#f9f1f5>br </mark><mark style=background-color:#faebf3>br </mark><mark style=background-color:#fcdded>unfortunately </mark><mark style=background-color:#e9f5d8>it's </mark><mark style=background-color:#ecf6de>very </mark><mark style=background-color:#f8f3f6>difficult </mark><mark style=background-color:#f5f7f3>to </mark><mark style=background-color:#f9eff4>find </mark><mark style=background-color:#f5f7f2>in </mark><mark style=background-color:#eaf5d9>video </mark><mark style=background-color:#f9f0f5>stores </mark><mark style=background-color:#cdeaa7>you </mark><mark style=background-color:#dbf0bf>may </mark><mark style=background-color:#f8f2f5>have </mark><mark style=background-color:#edf6df>to </mark><mark style=background-color:#a3d36c>buy </mark><mark style=background-color:#f1f6e8>it </mark><mark style=background-color:#f9eff4>off </mark><mark style=background-color:#f2f6ec>the </mark><mark style=background-color:#ecf6de>internet </mark>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "HTML(\"\".join(list(map(hlstr, words, colors))))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}