examples/cfproto_mnist.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Counterfactuals guided by prototypes on MNIST"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This method can generate counterfactual instances guided by class prototypes. It means that for a certain instance X, the method builds a prototype for each prediction class using either an [auto-encoder](https://en.wikipedia.org/wiki/Autoencoder) or [k-d trees](https://en.wikipedia.org/wiki/K-d_tree). The nearest prototype class other than the originally predicted class is then used to guide the counterfactual search. For example, in MNIST the closest class to a 7 would be a 9. As a result, the prototype loss term will try to minimize the distance between the proposed counterfactual and the prototype of a 9. This speeds up the search towards a satisfactory counterfactual by steering it towards an interpretable solution from the start of the optimization. It also helps to avoid out-of-distribution counterfactuals with the perturbations driven to a prototype of another class.\n",
    "\n",
    "The loss function to be optimized is the following: \n",
    "\n",
    "$Loss$ = c$L_{pred}$ + $\\beta$$L_{1}$ + $L_{2}$ + $\\gamma$$L_{AE}$ + $\\theta$$L_{proto}$\n",
    "\n",
    "The first loss term relates to the model's prediction function, the following 2 terms define the elastic net regularization while the last 2 terms are optional. The aim of $\\gamma$$L_{AE}$ is to penalize out-of-distribution counterfactuals while $\\theta$$L_{proto}$ guides the counterfactual to a prototype. When we only have acces to the model's predict function and cannot fully enjoy the benefits of automatic differentiation, the prototypes allow us to drop the predict function loss term and still generate reasonable counterfactuals. This drastically reduces the number of predict calls made during the numerical gradient update step and again speeds up the search.\n",
    "\n",
    "Other options include generating counterfactuals for specific classes or including trust score constraints to ensure that the counterfactual is close enough to the newly predicted class compared to the original class. \n",
    "\n",
    "The different use cases highlighted above are illustrated throughout this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "import keras\n",
    "from keras import backend as K\n",
    "from keras.layers import Conv2D, Dense, Dropout, Flatten, MaxPooling2D, Input, UpSampling2D\n",
    "from keras.models import Model, load_model\n",
    "from keras.utils import to_categorical\n",
    "import matplotlib\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import os\n",
    "import tensorflow as tf\n",
    "from time import time\n",
    "from alibi.explainers import CounterFactualProto"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load and prepare MNIST data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x_train shape: (60000, 28, 28) y_train shape: (60000,)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7fd0ba0f40b8>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAADXZJREFUeJzt3X+IHPUZx/HPU5uAaFGT0uMwttGohSj+CKcUCaVFjVZiYkA0wT9SWnr9o0LF+ItUUChiKf1B/wpEDCba2jRcjFFL0zZUTSEJOSVGo1ETuWjCJdcQ0QSRmuTpHzvXXvXmu5uZ2Z29PO8XHLc7z+7Mw3Kfm5md3e/X3F0A4vlS3Q0AqAfhB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8Q1Jc7uTEz4+OEQJu5u7XyuFJ7fjO70czeNrPdZvZAmXUB6Cwr+tl+MztN0juSrpe0T9I2SYvc/c3Ec9jzA23WiT3/1ZJ2u/t77v5vSX+UNL/E+gB0UJnwnyvpgzH392XL/o+Z9ZvZoJkNltgWgIq1/Q0/d18uabnEYT/QTcrs+fdLOm/M/WnZMgATQJnwb5N0kZmdb2aTJS2UtL6atgC0W+HDfnc/ZmZ3Stog6TRJK9x9Z2WdAWirwpf6Cm2Mc36g7TryIR8AExfhB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBdXTobhRzzz33JOunn356bu2yyy5LPvfWW28t1NOoZcuWJeubN2/OrT355JOlto1y2PMDQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCM3tsFVq9enayXvRZfpz179uTWrrvuuuRz33///arbCYHRewEkEX4gKMIPBEX4gaAIPxAU4QeCIvxAUKW+z29mQ5KOSDou6Zi791XR1Kmmzuv4u3btStY3bNiQrF9wwQXJ+s0335ysz5gxI7d2xx13JJ/76KOPJusop4rBPL7r7ocqWA+ADuKwHwiqbPhd0l/N7BUz66+iIQCdUfawf7a77zezr0n6m5ntcveXxz4g+6fAPwagy5Ta87v7/uz3iKRnJF09zmOWu3sfbwYC3aVw+M3sDDP7yuhtSXMkvVFVYwDaq8xhf4+kZ8xsdD1/cPe/VNIVgLYrHH53f0/S5RX2MmH19aXPaBYsWFBq/Tt37kzW582bl1s7dCh9Ffbo0aPJ+uTJk5P1LVu2JOuXX57/JzJ16tTkc9FeXOoDgiL8QFCEHwiK8ANBEX4gKMIPBMUU3RXo7e1N1rPPQuRqdinvhhtuSNaHh4eT9TKWLFmSrM+cObPwul944YXCz0V57PmBoAg/EBThB4Ii/EBQhB8IivADQRF+ICiu81fgueeeS9YvvPDCZP3IkSPJ+uHDh0+6p6osXLgwWZ80aVKHOkHV2PMDQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFBc5++AvXv31t1CrnvvvTdZv/jii0utf+vWrYVqaD/2/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QlLl7+gFmKyTNlTTi7pdmy6ZIWi1puqQhSbe5+4dNN2aW3hgqN3fu3GR9zZo1yXqzKbpHRkaS9dR4AC+99FLyuSjG3dMTRWRa2fM/IenGzy17QNJGd79I0sbsPoAJpGn43f1lSZ8fSma+pJXZ7ZWSbqm4LwBtVvScv8fdR+eIOiCpp6J+AHRI6c/2u7unzuXNrF9Sf9ntAKhW0T3/QTPrlaTsd+67Pu6+3N373L2v4LYAtEHR8K+XtDi7vVjSs9W0A6BTmobfzJ6WtFnSN81sn5n9UNIvJF1vZu9Kui67D2ACaXrO7+6LckrXVtwL2qCvL3221ew6fjOrV69O1rmW3734hB8QFOEHgiL8QFCEHwiK8ANBEX4gKIbuPgWsW7cutzZnzpxS6161alWy/uCDD5ZaP+rDnh8IivADQRF+ICjCDwRF+IGgCD8QFOEHgmo6dHelG2Po7kJ6e3uT9ddeey23NnXq1ORzDx06lKxfc801yfqePXuSdXRelUN3AzgFEX4gKMIPBEX4gaAIPxAU4QeCIvxAUHyffwIYGBhI1ptdy0956qmnknWu45+62PMDQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFBNr/Ob2QpJcyWNuPul2bKHJf1I0r+yhy119z+3q8lT3bx585L1WbNmFV73iy++mKw/9NBDhdeNia2VPf8Tkm4cZ/lv3f2K7IfgAxNM0/C7+8uSDnegFwAdVOac/04z22FmK8zsnMo6AtARRcO/TNIMSVdIGpb067wHmlm/mQ2a2WDBbQFog0Lhd/eD7n7c3U9IekzS1YnHLnf3PnfvK9okgOoVCr+ZjR1OdoGkN6ppB0CntHKp72lJ35H0VTPbJ+khSd8xsyskuaQhST9uY48A2qBp+N190TiLH29DL6esZt+3X7p0abI+adKkwtvevn17sn706NHC68bExif8gKAIPxAU4QeCIvxAUIQfCIrwA0ExdHcHLFmyJFm/6qqrSq1/3bp1uTW+sos87PmBoAg/EBThB4Ii/EBQhB8IivADQRF+IChz985tzKxzG+sin376abJe5iu7kjRt2rTc2vDwcKl1Y+Jxd2vlcez5gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAovs9/CpgyZUpu7bPPPutgJ1/00Ucf5daa9dbs8w9nnXVWoZ4k6eyzz07W77777sLrbsXx48dza/fff3/yuZ988kklPbDnB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgml7nN7PzJK2S1CPJJS1399+Z2RRJqyVNlzQk6TZ3/7B9rSLPjh076m4h15o1a3JrzcYa6OnpSdZvv/32Qj11uwMHDiTrjzzySCXbaWXPf0zSEnefKelbkn5iZjMlPSBpo7tfJGljdh/ABNE0/O4+7O6vZrePSHpL0rmS5ktamT1spaRb2tUkgOqd1Dm/mU2XdKWkrZJ63H30uO2AGqcFACaIlj/bb2ZnShqQdJe7f2z2v2HC3N3zxuczs35J/WUbBVCtlvb8ZjZJjeD/3t3XZosPmllvVu+VNDLec919ubv3uXtfFQ0DqEbT8FtjF/+4pLfc/TdjSuslLc5uL5b0bPXtAWiXpkN3m9lsSZskvS7pRLZ4qRrn/X+S9HVJe9W41He4ybpCDt29du3aZH3+/Pkd6iSWY8eO5dZOnDiRW2vF+vXrk/XBwcHC6960aVOyvmXLlmS91aG7m57zu/s/JeWt7NpWNgKg+/AJPyAowg8ERfiBoAg/EBThB4Ii/EBQTNHdBe67775kvewU3imXXHJJst7Or82uWLEiWR8aGiq1/oGBgdzarl27Sq27mzFFN4Akwg8ERfiBoAg/EBThB4Ii/EBQhB8Iiuv8wCmG6/wAkgg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gqKbhN7PzzOwfZvamme00s59myx82s/1mtj37uan97QKoStPBPMysV1Kvu79qZl+R9IqkWyTdJumou/+q5Y0xmAfQdq0O5vHlFlY0LGk4u33EzN6SdG659gDU7aTO+c1suqQrJW3NFt1pZjvMbIWZnZPznH4zGzSzwVKdAqhUy2P4mdmZkl6S9Ii7rzWzHkmHJLmkn6txavCDJuvgsB9os1YP+1sKv5lNkvS8pA3u/ptx6tMlPe/ulzZZD+EH2qyyATzNzCQ9LumtscHP3ggctUDSGyfbJID6tPJu/2xJmyS9LulEtnippEWSrlDjsH9I0o+zNwdT62LPD7RZpYf9VSH8QPsxbj+AJMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQTQfwrNghSXvH3P9qtqwbdWtv3dqXRG9FVdnbN1p9YEe/z/+FjZsNuntfbQ0kdGtv3dqXRG9F1dUbh/1AUIQfCKru8C+vefsp3dpbt/Yl0VtRtfRW6zk/gPrUvecHUJNawm9mN5rZ22a228weqKOHPGY2ZGavZzMP1zrFWDYN2oiZvTFm2RQz+5uZvZv9HneatJp664qZmxMzS9f62nXbjNcdP+w3s9MkvSPpekn7JG2TtMjd3+xoIznMbEhSn7vXfk3YzL4t6aikVaOzIZnZLyUddvdfZP84z3H3+7ukt4d1kjM3t6m3vJmlv68aX7sqZ7yuQh17/qsl7Xb399z935L+KGl+DX10PXd/WdLhzy2eL2lldnulGn88HZfTW1dw92F3fzW7fUTS6MzStb52ib5qUUf4z5X0wZj7+9RdU367pL+a2Stm1l93M+PoGTMz0gFJPXU2M46mMzd30udmlu6a167IjNdV4w2/L5rt7rMkfU/ST7LD267kjXO2brpcs0zSDDWmcRuW9Os6m8lmlh6QdJe7fzy2VudrN05ftbxudYR/v6Tzxtyfli3rCu6+P/s9IukZNU5TusnB0UlSs98jNffzX+5+0N2Pu/sJSY+pxtcum1l6QNLv3X1ttrj21268vup63eoI/zZJF5nZ+WY2WdJCSetr6OMLzOyM7I0YmdkZkuao+2YfXi9pcXZ7saRna+zl/3TLzM15M0ur5teu62a8dveO/0i6SY13/PdI+lkdPeT0dYGk17KfnXX3JulpNQ4DP1PjvZEfSpoqaaOkdyX9XdKULurtSTVmc96hRtB6a+ptthqH9Dskbc9+bqr7tUv0Vcvrxif8gKB4ww8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFD/Abw9Wv8QfFP9AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()\n",
    "print('x_train shape:', x_train.shape, 'y_train shape:', y_train.shape)\n",
    "plt.gray()\n",
    "plt.imshow(x_test[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prepare data: scale, reshape and categorize"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x_train shape: (60000, 28, 28, 1) x_test shape: (10000, 28, 28, 1)\n",
      "y_train shape: (60000, 10) y_test shape: (10000, 10)\n"
     ]
    }
   ],
   "source": [
    "x_train = x_train.astype('float32') / 255\n",
    "x_test = x_test.astype('float32') / 255\n",
    "x_train = np.reshape(x_train, x_train.shape + (1,))\n",
    "x_test = np.reshape(x_test, x_test.shape + (1,))\n",
    "print('x_train shape:', x_train.shape, 'x_test shape:', x_test.shape)\n",
    "y_train = to_categorical(y_train)\n",
    "y_test = to_categorical(y_test)\n",
    "print('y_train shape:', y_train.shape, 'y_test shape:', y_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "xmin, xmax = -.5, .5\n",
    "x_train = ((x_train - x_train.min()) / (x_train.max() - x_train.min())) * (xmax - xmin) + xmin\n",
    "x_test = ((x_test - x_test.min()) / (x_test.max() - x_test.min())) * (xmax - xmin) + xmin"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define and train CNN model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def cnn_model():\n",
    "    x_in = Input(shape=(28, 28, 1))\n",
    "    x = Conv2D(filters=64, kernel_size=2, padding='same', activation='relu')(x_in)\n",
    "    x = MaxPooling2D(pool_size=2)(x)\n",
    "    x = Dropout(0.3)(x)\n",
    "    \n",
    "    x = Conv2D(filters=32, kernel_size=2, padding='same', activation='relu')(x)\n",
    "    x = MaxPooling2D(pool_size=2)(x)\n",
    "    x = Dropout(0.3)(x)\n",
    "    \n",
    "    x = Flatten()(x)\n",
    "    x = Dense(256, activation='relu')(x)\n",
    "    x = Dropout(0.5)(x)\n",
    "    x_out = Dense(10, activation='softmax')(x)\n",
    "    \n",
    "    cnn = Model(inputs=x_in, outputs=x_out)\n",
    "    cnn.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])\n",
    "    \n",
    "    return cnn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "input_1 (InputLayer)         (None, 28, 28, 1)         0         \n",
      "_________________________________________________________________\n",
      "conv2d_1 (Conv2D)            (None, 28, 28, 64)        320       \n",
      "_________________________________________________________________\n",
      "max_pooling2d_1 (MaxPooling2 (None, 14, 14, 64)        0         \n",
      "_________________________________________________________________\n",
      "dropout_1 (Dropout)          (None, 14, 14, 64)        0         \n",
      "_________________________________________________________________\n",
      "conv2d_2 (Conv2D)            (None, 14, 14, 32)        8224      \n",
      "_________________________________________________________________\n",
      "max_pooling2d_2 (MaxPooling2 (None, 7, 7, 32)          0         \n",
      "_________________________________________________________________\n",
      "dropout_2 (Dropout)          (None, 7, 7, 32)          0         \n",
      "_________________________________________________________________\n",
      "flatten_1 (Flatten)          (None, 1568)              0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 256)               401664    \n",
      "_________________________________________________________________\n",
      "dropout_3 (Dropout)          (None, 256)               0         \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 10)                2570      \n",
      "=================================================================\n",
      "Total params: 412,778\n",
      "Trainable params: 412,778\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "cnn = cnn_model()\n",
    "cnn.summary()\n",
    "cnn.fit(x_train, y_train, batch_size=64, epochs=3, verbose=1)\n",
    "cnn.save('mnist_cnn.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Evaluate the model on test set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test accuracy:  0.9862\n"
     ]
    }
   ],
   "source": [
    "score = cnn.evaluate(x_test, y_test, verbose=0)\n",
    "print('Test accuracy: ', score[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define and train auto-encoder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def ae_model():\n",
    "    # encoder\n",
    "    x_in = Input(shape=(28, 28, 1))\n",
    "    x = Conv2D(16, (3, 3), activation='relu', padding='same')(x_in)\n",
    "    x = Conv2D(16, (3, 3), activation='relu', padding='same')(x)\n",
    "    x = MaxPooling2D((2, 2), padding='same')(x)\n",
    "    encoded = Conv2D(1, (3, 3), activation=None, padding='same')(x)\n",
    "    encoder = Model(x_in, encoded)\n",
    "    \n",
    "    # decoder\n",
    "    dec_in = Input(shape=(14, 14, 1))\n",
    "    x = Conv2D(16, (3, 3), activation='relu', padding='same')(dec_in)\n",
    "    x = UpSampling2D((2, 2))(x)\n",
    "    x = Conv2D(16, (3, 3), activation='relu', padding='same')(x)\n",
    "    decoded = Conv2D(1, (3, 3), activation=None, padding='same')(x)\n",
    "    decoder = Model(dec_in, decoded)\n",
    "    \n",
    "    # autoencoder = encoder + decoder\n",
    "    x_out = decoder(encoder(x_in))\n",
    "    autoencoder = Model(x_in, x_out)\n",
    "    autoencoder.compile(optimizer='adam', loss='mse')\n",
    "    \n",
    "    return autoencoder, encoder, decoder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "input_1 (InputLayer)         (None, 28, 28, 1)         0         \n",
      "_________________________________________________________________\n",
      "model_1 (Model)              (None, 14, 14, 1)         2625      \n",
      "_________________________________________________________________\n",
      "model_2 (Model)              (None, 28, 28, 1)         2625      \n",
      "=================================================================\n",
      "Total params: 5,250\n",
      "Trainable params: 5,250\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/avl/anaconda3/envs/alibi/lib/python3.6/site-packages/keras/engine/saving.py:292: UserWarning: No training configuration found in save file: the model was *not* compiled. Compile it manually.\n",
      "  warnings.warn('No training configuration found in save file: '\n"
     ]
    }
   ],
   "source": [
    "ae, enc, dec = ae_model()\n",
    "ae.summary()\n",
    "ae.fit(x_train, x_train, batch_size=128, epochs=4, validation_data=(x_test, x_test), verbose=1)\n",
    "ae.save('mnist_ae.h5')\n",
    "enc.save('mnist_enc.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compare original with decoded images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABAwAAADnCAYAAACaL8QkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XmQVNX58PEz7KvssoiyuUQFBERU1CiGiLKIqKgB93JJKi4pEDVixTUxhSXGuECwyp1CIyilgAtJBDQadVQ0CJiAAiLIIgwybA7MvH+89Tt5znPnPnP72t3TM3w/fz23Tve9p6k+fe5cznOeooqKCgcAAAAAACDVqe4OAAAAAACAwsMDAwAAAAAAEMEDAwAAAAAAEMEDAwAAAAAAEMEDAwAAAAAAEFHPaiwqKqKEQuHYXFFR0a66O4HCwNgsKIxNeIzNgsLYhMfYLCiMTXiMzYJS6dhkhUHNsbq6OwCgUoxNoDAxNoHCxNgEClOlY5MHBgAAAAAAIIIHBgAAAAAAIIIHBgAAAAAAIIIHBgAAAAAAIIIHBgAAAAAAIIIHBgAAAAAAIIIHBgAAAAAAIIIHBgAAAAAAIIIHBgAAAAAAIKJedXcgqZtuuik4bty4sY979+4dtJ1//vmx55kyZUpw/N577/n42Wef/TFdBAAAAACg1mCFAQAAAAAAiOCBAQAAAAAAiCjolIQXXnjBx1aagVZeXh7bdu211wbHgwcP9vHChQuDtjVr1iS+JoDsOvzww328fPnyoO3GG2/08cMPP5y3PgG1RdOmTYPj+++/38d6nvzoo4+C49GjR/t49erVOegdAAAoFKwwAAAAAAAAETwwAAAAAAAAETwwAAAAAAAAEQW1h4Hcs8C55PsW6PzmN954w8fdu3cP2kaMGBEc9+jRw8djx44N2u67775E1weQfX379vWx3pdk7dq1+e4OUKt07NgxOL766qt9rMfbscceGxwPHz7cx48++mgOegfUbv369QuOX3rpJR937do159c/44wzguNly5b5+Ouvv8759YH9jfz785VXXgnarrvuOh9PnTo1aNu3b19uO5YQKwwAAAAAAEAEDwwAAAAAAEBEtack9O/f38ejRo2Kfd3nn38eHJ999tk+3rx5c9BWWlrq4wYNGgRt//rXv4LjY445xsdt2rRJ0GMA+dCnTx8f79ixI2h7+eWX890doMZr166dj59++ulq7AmwfxsyZEhw3LBhw7xeX6fnXnnllT6+6KKL8toXoDbSf1M+9thjsa995JFHfPzEE08Ebbt27cpux1JihQEAAAAAAIjggQEAAAAAAIjggQEAAAAAAIio9j0MZGmnoqKioE3uW6DzvdavX5/o/OPHjw+OjzrqqNjXzp07N9E5AWRfz549g2NZZubZZ5/Nd3eAGu+GG24Ijs855xwfDxgwIPV5f/rTn/q4Tp3w/x0+/fRTHy9atCj1NYDapl69/91yDx06tBp74txHH30UHI8bN87HTZs2Ddr0HkIAqibnSeec69y5c+xrZ8yY4ePdu3fnrE8/BisMAAAAAABABA8MAAAAAABARLWnJLz66qs+PvTQQ4O27du3+3jLli2pzq/Lw9SvXz/VeQDk1k9+8pPgWC6LfOGFF/LdHaDGe/DBB4Pj8vLyrJz33HPPrTR2zrnVq1f7+MILLwza9DJoYH8yaNAgH5944olB26RJk/Lal1atWgXHMl23SZMmQRspCUDVdGnUiRMnJn6vTLutqKjIWp+yiRUGAAAAAAAgggcGAAAAAAAgggcGAAAAAAAgotr3MJBk7uOPMWHCBB8ffvjh5mvff//9SmMA+XXzzTcHx/L3oLi4ON/dAWqkefPm+ViXPEzru+++C45LS0t93KVLl6CtW7duPv7ggw+Ctrp162alP0BNoEsFy9JpK1euDNr+8Ic/5KVP/2fkyJF5vR5Q2/Xq1Ss4PvbYY2Nfu3fv3uD4tddey0mfsokVBgAAAAAAIIIHBgAAAAAAIKKgUhLSGj58eHB89913+7hBgwZB28aNG4Pj3/72tz7euXNnDnoHoDJdu3YNjvv37x8c/+c///ExZZ2Ayp166qnB8RFHHOFjXUYxaVnFqVOnBsdvvvlmcLxt2zYfn3766UGbVUrqV7/6lY+nTJmSqC9ATXX77bcHx7JU8Jlnnhm0yTSfXGndurWP9e9GtkquAvur8847L/Fr9ZxaE7DCAAAAAAAARPDAAAAAAAAARPDAAAAAAAAARNSKPQx07rPet0B64YUXguOFCxfmpE8AbDqHUtu0aVOeegLULHL/j+effz5oa9u2baJz6DLGs2bN8vFdd90VtFn7++jzXHPNNT5u165d0DZp0iQfN2rUKGh75JFHfFxWVhZ7PaCQnX/++T4eOnRo0LZixQofV0epYLm/iN6zYMGCBT4uKSnJV5eAWuOnP/2p2f7DDz/42Nrrp1CxwgAAAAAAAETwwAAAAAAAAETU2JSE2bNn+/iMM86Ifd0zzzwTHOsyNwCqR69evcx2uXwZwP/Uq/e/qTtpCoJzYQreRRddFLRt3rw5VV90SsJ9993n48mTJwdtTZo08bEe36+88oqPV65cmaovQHUbPXq0j+X33TnnHnvssbz2RZcuHjt2rI/37dsXtN17770+JiUISGbgwIGVxpWR5cEXL16csz7lCisMAAAAAABABA8MAAAAAABABA8MAAAAAABARI3Zw6Bjx47BscwVadiwYdAmczFlXpZzzpWWluagdwCSOOGEE3x8xRVXBG2ffPJJcDx//vy89AmorXTptiuvvNLHafcsqIrci0DmTDvn3HHHHZeTawLVpUWLFsGxnOO0KVOm5Lo7AVni1Llwv5Nly5YFbW+99VZe+gTUJpnMafke/9nGCgMAAAAAABDBAwMAAAAAABBRY1ISZs2aFRy3adMm9rXPPfecjynPBBSOwYMH+7h169ZB2+uvvx4c7969Oy99AmqyOnXin/sff/zxeezJ/1dUVORj3Terr3feeaePL7nkkqz3C8gFnRJ70EEH+XjGjBn57k6gR48esW1LlizJY0+A2ql///6xbSUlJcExKQkAAAAAAKDW4YEBAAAAAACI4IEBAAAAAACIKOg9DM4++2wf9+vXL/Z1CxYsCI7vuOOOXHUJwI9wzDHH+LiioiJomzlzZr67A9RIv/zlL31cXl5ejT2JGjFihI/79u0btMm+6n7LPQyAmmL79u3B8eLFi33cu3fvoE3u27Nly5ac9OfAAw/08fnnnx/7unfeeScn1wdqs5NPPjk4HjNmTOxrt23bFhyvXbs2J33KF1YYAAAAAACACB4YAAAAAACACB4YAAAAAACAiILaw6BNmzbB8W233ebj+vXrx75P5ow551xpaWl2OwYglQ4dOgTHp5xyio+/+OKLoO3ll1/OS5+Amk7uE1Ad2rVr5+OjjjoqaJPztmXTpk3BcVlZ2Y/vGJBnu3btCo5Xrlzp4/POOy9omzt3ro8nT56c6no9e/YMjrt37x4cd+3a1cd6nyCp0PY+AWoC/XdqnTrx/+8+f/78XHcnr1hhAAAAAAAAInhgAAAAAAAAIgoqJWH8+PHB8XHHHRf72tmzZ/uYMopAYbr88suDY1ny6bXXXstzbwBkw8SJE33861//OvH7Vq1a5ePLLrssaFuzZs2P7hdQ3eT9aFFRUdA2bNgwH8+YMSPV+Tdv3hwc67SDtm3bJjrPU089ler6wP7MKlVaUlISHP/lL3/JdXfyihUGAAAAAAAgggcGAAAAAAAgggcGAAAAAAAgoqD2MBg3blzi11533XU+powiUJi6dOkS27Z169Y89gRAWvPmzQuOjzjiiFTnWbp0qY/feeedH9UnoBAtX77cxxdccEHQ1qdPHx8feuihqc4/c+ZMs/3pp5/28dixY2Nfp8tBAqhc586dfTxmzJjY161duzY4Li4uzlmfqgMrDAAAAAAAQAQPDAAAAAAAQERBpSRkonXr1j4uKytLfZ5t27bFnqd+/fo+btGiRew5WrZsGRwnTa3Yt29fcHzLLbf4eOfOnYnOARSy4cOHx7a9+uqreewJUHvIcm116sQ/9z/rrLNi26ZNmxYcd+rUKfa1+hrl5eVVdbFSI0aMSPU+oDZYvHhxpXE2ffnll4le17Nnz+B4yZIluegOUOMNHDjQx9Z8O3v27Hx0p9qwwgAAAAAAAETwwAAAAAAAAETwwAAAAAAAAETU2D0MPvvss6yc58UXX/Tx+vXrg7b27dv7+MILL8zK9Szffvutj3//+9/n/HpALpx88sk+7tChQzX2BKidpkyZ4uNJkybFvm7OnDnBsbX3QCb7EiR97dSpUxOfE8CPJ/c3kbHGngVAMm3atIlt27x5s48feuihfHSn2rDCAAAAAAAARPDAAAAAAAAARBRUSsK8efOC45EjR+b8mqNHj071vr179/rYWp75yiuvBMfFxcWxr3377bdT9QUoJKNGjfJx3bp1g7ZPPvnEx4sWLcpbn4Da5KWXXvLxhAkTgrZ27drl/PqbNm3y8bJly4K2a665xsc6zQ9AblVUVFQaA0hnyJAhsW1r1qzx8bZt2/LRnWrDCgMAAAAAABDBAwMAAAAAABDBAwMAAAAAABBRUHsYnHvuucHxzTff7OP69esnPs/RRx/t40zKIT7xxBPB8apVq2JfO2vWLB8vX7488TWA2qZJkybB8dChQ2NfO3PmTB/v27cvZ30CarPVq1f7+KKLLgrazjnnHB/feOONObm+LPv76KOP5uQaADLXqFGj2LZdu3blsSdAzaT/3uzRo0fsa3fv3u3jsrKynPWpELDCAAAAAAAARPDAAAAAAAAARBRUSoI2adKkH32OMWPGZKEnAOLoZVhbt271sS4r+tBDD+WlT8D+Qpcnlcdvvvlm0CZLHo4YMSJok2N12rRpQVtRUVFwvHTp0nSdBZBTV1xxhY9LSkqCtnvuuSff3QFqnPLy8uC4uLjYxz179gzaVqxYkZc+FQJWGAAAAAAAgAgeGAAAAAAAgAgeGAAAAAAAgIiC3sMAQOHTexgMHDiwmnoCQHr99dfNYwC1y4cffujjyZMnB21vvfVWvrsD1Di65PfEiRN9XFFREbR99NFHeelTIWCFAQAAAAAAiOCBAQAAAAAAiCjSyyuCxqKi+Ebk20cVFRX9q7sTKAyMzYLC2ITH2CwojE14jM2CwtiEx9gsKJWOTVYYAAAAAACACB4YAAAAAACACB4YAAAAAACACB4YAAAAAACACB4YAAAAAACACB4YAAAAAACACB4YAAAAAACACB4YAAAAAACACB4YAAAAAACACB4YAAAAAACAiHpVtG92zq3OR0dQpS7V3QEUFMZm4WBsQmJsFg7GJiTGZuFgbEJibBaOSsdmUUVFRb47AgAAAAAAChwpCQAAAAAAIIIHBgAAAAAAIIIHBgAAAAAAIIIHBgAAAAAAIIIHBgAAAAAAIIIHBgAAAAAAIIIHBgAAAAAAIIIHBgAAAAAAIIIHBgAAAAAAIIIHBgAAAAAAIIIHBgAAAAAAIIIHBgAAAAAAIIIHBgAAAAAAIIIHBgAAAAAAIIIHBgAAAAAAIIIHBgAAAAAAIIIHBgAAAAAAIIIHBgAAAAAAIKKe1VinTp2KunXr+uPy8vKcdyipioqKrJynqKgo6+dI2jf9OnmeOnXCZzl79+7dXFFR0S5lF1HL6LG5b9++2NfK71W2xk0uWOMhH5L+O8l/d+cYmwgV0ryZdrzne+xlgnkTadWpU6eiXr3/3fYW6ryZ9p6ykOl5s6ysjLEJrzbe01oymWNz/RmT3tOaDwzq1q3r2rZt649LS0t9rD+AnKgzufGXXwo92Uv6pkteY+/evYn7otWvXz+2n/JYX19eQ/c76Q2i7recyJo2bRq0ffvtt6sTnRT7hbp167pWrVr54+3bt/tYf+flj4Fuy8UPkTyn9Vugx4meIGS/9Q+afK/1W2T9pmiyb2VlZUGbHJvNmjUL2jZt2sTYhFe3bl3XunVrfyznTS3pXJV2nKYd73q86WN5HmtuzsWDB2ve1GNzw4YNjE149erVC+5p5bypJX3ol8nYtMaG9eDLGm/W/ac1/6b9gyXp/bxz9thct24dYxOedU+r5XreTKuq+VaOHT2nWg9Bkn7GTMamvH7Se1pSEgAAAAAAQAQPDAAAAAAAQISZklBeXu527txZaZu1RDgT1hIpK5/FWoYhl6tUkjeV+PrWNaxl15akS1JKSkoSnxP7n/Lycrdjx45K26zlw1quc5Wt88vlis7ZqRQNGzYM2uSyZGtJdNp9EfS/ofxN2bp1a6JzYP9kzZvWUmNL2nGq3yfHirWUWb9Pj1U9j8ZdM1v7kljzpvwczJuw6LGZNA1AS7u0P2kqnXU/XVWaXdLzWDL5TNa/oWxj3oTFuqfN97yZVlXp6dZ4TLpnQ9o23bc08yYrDAAAAAAAQAQPDAAAAAAAQISZkuBc/NKPbC01lMsirGWQermGfK1eLmkt+7J2mLXaspWSYL02acUIoKKiIvHYzDfrt8Da3VaP47j3VXYcd329o7rF+neT52Fsoipx6Wz5GJtJlwhrsmKQngv37NkTHFupDdacmov0KDlvFnI5SFS/TObNbJQkzaQ8b4MGDWLfJ7/j1r2wlkmVBOt+15pHk95v688LSHps1pTSiUnHlG6X861zduW/XKT8p5k3ufMFAAAAAAARPDAAAAAAAAARPDAAAAAAAAARGe1hYOUfx71Hv0/nSlj5lj/88IOPBwwYELQNGzbMxz179gzaOnfu7GOdF7106dLg+LnnnvPxggULgjaZR5J2n4JMcsjiXgdUJm5sWq+rblaf9Xde5nhZ+xvs2rUrOLbyq63rW2VUrX4CWtKxGfeeqiQtXWiVdbLyjRs1ahS0WeWIrXxLa4xb+wJZrPexvwiqkvSe1hpjSUtuW3OFNTb1niGZzO/ytdb+ItbeX5r8HHoPhaR9Y2wiE0nvs6r7/jbp74Rz9v4+8u9d631J+1JVW5qxyQgGAAAAAAARPDAAAAAAAAARVaYkSEmX2ltLHa2UBF1mYtKkST4ePHhw0NayZUsf7969O2jT55EGDRoUHB900EE+XrJkSdC2bt06H2dSyqamLKVB7ZFJilCuyeVNetzIvugyS7qf7du39/H06dODtqlTp/p45syZQZu1JDuTElRAWklLQqUtM2jNxXrMSday41tvvdXHV155ZdC2atWq4PiSSy7x8fr164M2q3ya9XmT3idoNaX8FgpPNsqVZZISJMeGXgYs71utdDl9TmuOteiUBNk3nYKUdNxqSVNwAS3p99gaf2mvZ80/ViqRTs/Tc2HSOU6nJDVu3LjSfmpW2kEmf7PHYYUBAAAAAACI4IEBAAAAAACI4IEBAAAAAACIyGgPAymTHH6Zt6H3F2jYsKGPH3jggaDtzDPP9HGTJk2Cto0bN/q4uLg4aPvqq698fPDBBwdtei+ELl26VHo955x76qmnKv0MzuWmPBv5l8iFpPmW2WKNDSu/Wud/XXbZZT4+4YQTgrann37axzrfS45V+fti9dM59jdAbuRi/Fk5lZLOoZRjrF+/fkHb5Zdf7uOOHTsGbXLPIOfCvYCeeeaZoE2Ox2bNmsX2O5PSUWlLLgKWpGMz7V4jVp5y9+7dgza5h8jEiRODtq1bt/pYz5OZlEuzShXLsbphwwYXxyqVat0nA5nIxvhLe720qtqX64YbbvDxVVddFbTNmDHDx/pvYVly0Zon05Z4TfrZWWEAAAAAAAAieGAAAAAAAAAiUqckZLLUSC5T0suHe/bs6eOf/exnQZss7fLZZ58FbbfddpuPP/jgg6Bt586dPj7kkEOCtr/+9a/B8aGHHupjvURLL/2S0pZOZMkkci3faQf6/HLJlPU7oZdWNW/ePDg+5phjfLxixYqgTZZS1Msg5XmttnwscwPSlodK+121yprKUqW/+c1vgjY5327bti32nM45d+CBB8Ze3yoPJVMLMykHmfTzM26RLWnHrZxz9LiRS5b1kmSZ5lNaWhq07dixw8f6PjVtiXNdYnzYsGE+vvbaa10cq8Q4KQjIltrwXdKpfXLMH3bYYUHbueee6+MHH3wwaCuUMcYKAwAAAAAAEMEDAwAAAAAAEMEDAwAAAAAAEJF6DwOLziPUpSakTp06+Xj79u1B2zfffOPj3/3ud0Hbv//9bx/rnK4GDRr4uFWrVkFb27ZtY/uir28hVxL7M+v7b5V5k+/Te4ToEmy9e/f28Zw5c4I2uU9J06ZNgzaZ46Wvwb4FKFSZlPW09hCR8+Fxxx0XtN17770+Pvzww4M2mTetc6h1OeSzzjrLxzqnWZaH+v7774M2uU+C/kzWfUJStSHvFYVPjjm9T4Gcc+T33blwHA0ZMiRoKykpiX1fo0aNYvuix58cA9Y+IX379g3ahg8f7mM9F8t7Y+u3yCoxDtQG1t+3svyhc9F9Qho3buxjPccuWbLEx7t37w7a5G+Knout8Z7te1pWGAAAAAAAgAgeGAAAAAAAgIjUKQmZLHWQy5T0kqV3333Xx2eccUbQJks7yTQDfX297LhFixY+1qkMbdq0CY7l0o/PP/+88g/gossl5dIulmEB/2OVOZNjQ6cuXHDBBcGxXBY5d+7c2PPo8W+VtbIUSukawLnk30G9DFLOcQ888EDQ1qNHDx/rpY1yrMiUH+ei47hbt24+njBhQtAmx7Gef//5z39Wej19rMe0nG+tMo5AJpKW/dVzlfw+6ntDeWzNcR06dAjaHn/8cR/r9FhZjtQqP+pcOB5032R/9DWszyTb9PWt+3srPRGoiay/9w444ICgbfTo0cGxTInXaQdPPvmkj/X8Z81x1u+NlbqYBisMAAAAAABABA8MAAAAAABABA8MAAAAAABAROo9DKzyDbpN5nzotu+++67SczgX7lug8zTlOXUu5vXXX+/jE0880ez3hx9+6OMVK1YEbdZnsnKxyYVGbZO2XItVKk2XbpKl2pxzbseOHT5+5513gjaZ42XlaWpWv8mFRnWy5k2rdKLOKb7zzjt93L1796BNzqP6fXIcNW/ePGjT+ZZyjwP92v79+/v47rvvDtpkmSn9meQ59ZxulbFjjkVaSe/xLNZ8c9BBBwXHN910k49XrlwZtN1+++0+1ve7cjzo6+nxIO+b9WeQ5zn66KNj+22VjtPXl8eZ/F0A1BTWXljyXrRz585BW69evWLP89VXXwVtcj8/uWeJpudta/xZ/U4zHllhAAAAAAAAInhgAAAAAAAAIlKnJGhJlzfo18mllVYpJb3seM+ePT7WS6uGDx/uY70kY8uWLcHxn/70Jx/rUlJWKoXVxtIr1AZWSRZrqaFVclTSyzX79u0bHMuxqse/XLKll2/KvsnfF+fC3xhSEFDdrHFkfT/l93jkyJFB25AhQ3xslVmyxsbChQuDtpdffjk4Xrx4sY9HjRoVtN14440+PuKII4K28ePH+/ihhx4K2phvkW9pyypaZIrOtGnTgraOHTv6+KqrrgraZNqPTvOx+mmNcT03yrJvJ598ctAm76llOqBz4fxrzff6N0v2jfkWhSxpGVU9b8rxMGbMmKBNllF0LhyP7733XmxfqiqdGscaY9mYJ1lhAAAAAAAAInhgAAAAAAAAInhgAAAAAAAAIrK2h4HFyv+SuVE6/8LKxWratKmP77rrrqCtU6dOPm7YsGHQNmXKlOD4k08+8bGVb51JDhulZFDbWd9r6/sv9wkZOHBg0NaqVavg+O233/axHn9lZWWVXk9fU+eCkQuNQpL2OyfLo11yySWxbXpOa9SokY91WSeZb/3iiy8GbSUlJcGxnLcnT54ctPXp08fHcj8F55y79NJLfaz3E3ruued8rPczkqzxDmQi6V481h4+eo7p16+fj/UcV1xc7OM33ngjaGvcuHHs9az7T6vMmn6fvFfW+dXz58+Pvb61n5HFuqcGCon8jlv7CenxJsftSSedFLRZ85jeF0iO1aRlHHXfco3RDAAAAAAAInhgAAAAAAAAIvKSkiCXV+jlHFb5CGuJxi233OJjXY5NnvPvf/970Pboo48Gx3J5h75G0tIWuVgSQgkaFDJrab9VgqZ169Y+Pv3004M2XcrpySef9LEuedqgQYPY6ydNcwKqW9Il0bqtZcuWPu7atWvQJsecLkc6Y8YMH+tUvtLS0tjrNWvWLDiWKUE67U8utTzttNOCti5duvhYl6CSKQl67pWfw1qezfhGWpncc1nL/q+77jofy3nKOefmzp3rYzmG9Hn0smP5/ddzqj6W5REPPvjgoE2WHN+1a1fQ9vDDD/tY/hY4F6Y5afL6VioFYxM1hXXfKMeXc851797dxz169Aja9N+US5cu9fHHH38ctFmlS+WxntOt1CkpG/fCrDAAAAAAAAARPDAAAAAAAAARPDAAAAAAAAAROdnDIJNcCSvnQuZmdOvWLWgbNWpU7PlXr17t49tuuy1o03lbScus6Tw1+b5M8kaSIt8L1c367srxoF8nx4bOxWzRooWPdckpXebtww8/9LHOobZKJ1p5k5Q8RSGxvoNyHOmywgcccICP9diQZZ50ecSJEyf6+Pvvv499nx7TehzLvu3evTtoe/755308bty4oO2QQw7xsZ7TBwwY4OOFCxcGbXJM6xxOib1/kC1W3rAcj3LcOOfc0Ucf7eMNGzYEbbKs9/bt24M2a58C+b3WedHWWP35z38etLVp08bHX375ZdAm51v9max74bjXVdY3oFBZpUOt7/HIkSN9rMet9vrrr/tYz+ly3OrzJL2n1bJ9j8sKAwAAAAAAEMEDAwAAAAAAEJG1lARrqW/S5VR6aUXz5s19fOeddwZtcmmVXpIsS9esX78+aNPXl/RSK6vfUibLnq3Pay2JAQqV9T3WY/MXv/iFj1u1ahW0vfTSS8GxLO1kLdGqqj9AobCWOlrfad0my5zJedK5MNXgz3/+c2ybLvmWtMSxc3ZaQNu2bX28bdu2oE2WpNKpFDIl4b333ou9vpUeSJoRssVaBiy/c7p0YYcOHXws70WdC8sDW8v+dalSOd50WTf9myJ/G3r37h20yTG+bNmyoE0ukbbGt8W6F2BsopCkTSWXpcGdc+6kk05rx5FWAAAOGUlEQVTycZMmTYI2XZ70qaee8rEuI96oUSMfW783Vuq8hbKKAAAAAAAgJ3hgAAAAAAAAInhgAAAAAAAAIrK2h0Ha/CQrN0OWXTvxxBODNpmP8c033wRtsqyTLgelc6Fzkf+RNG/LykWx9kwACon+rspjnac5bNgwH+sSp7NmzQqOrRzSuNdp7GeAQpY0N18bPHiwj2WpUuec+/bbb32sy7pZJU+tecsqK6dzqi+++GIf6/xunZstrV27NrZvVsk5q8QrkA1yXwDnwlKiGzduDNrkmJP7eejjkpKSoE2OjaTlxp2Ljk25T9CgQYOCNjmuFi9eHNtm3Sdbe33losQ4kA9ybtTfVTnm9NiQ5YH1nP3f//43OP7uu+98LEsjO2ffC1j3uEllYw8R/ioFAAAAAAARPDAAAAAAAAARPDAAAAAAAAARWdvDQOZ86FwJK8dJ5mboPLEbbrjBxy1btgzaZK7IG2+8EbRt3rzZx7rWtJV/pVm5kfL6+pzy81vv06hZi3yzvqtpyRyvLl26BG1HHnmkj9etWxe0FRcXB8dy7Fo5XXpMsacBaiJrbtS5/6eeemqlr9P0/CdzMa19eay+6NdefvnlQdv111/vY52nKfc0+fjjj4O2OXPmJLq+ziG17j2ApKzxoOcYa7+B2bNn+3j06NFBm9xfa/78+UHb0qVLfazvd+U40rXe9f5e/fv3jz2P/Exr1qwJ2uReCGVlZS5O0ntYoNAk3RtE/y0qx4PeF6Rjx44+lnubOOfcokWLgmO5T0hVc2xSSee/TPbhi8MKAwAAAAAAEMEDAwAAAAAAEJGTsopWSoK1tHjChAlBW79+/XzcvHnzoG369Ok+njZtWtC2Y8cOH+vlW3o5lVwiovttlXy0lrZY55TLvjJZggoUEuv7L8usnXDCCUFb06ZNffyPf/wjaNNjRY6jTNIOLCyZRD6kKZGrXyeXQTZq1Cho69Spk4/1En1Jl2CT5RD1skt5Hr0kWV9/3LhxPr766quDNnle/Zm2b9/u40mTJgVtejmnJOdNKz2C8Y20rPs/a77ZunVrcHzPPff4WN+3nnnmmT4eMGBA7Hn0eJNjU5dj3LZtW3As52Y9puT4//zzz2PfZ6XuZrLsGShUmZTqluNv6NChsefR75Npdrrdui/IJM09n3MeKwwAAAAAAEAEDwwAAAAAAEAEDwwAAAAAAEBE1vYwsMg8Sp2bIcvFnH322UFbs2bNfPzpp58GbTKHUuY6OxfmjcmcLeeiuZmy7FTSXFN9TX0NeR6dJyrzTXbu3Bm0yfxuvfeCzlMDcs3KW7TKs8kxdfzxx8eeX5dV0+PY2kNEtln7G2Sy9wiQb1ZuojzW+xRYeyTs2rUrtq1Vq1Y+tvYp6Nq1a9A2duzY2GOdby33MNFlFWfMmOHjt99+O2iz5l/5e2O9jvGNbJFjTI8VK29ZfldlaXDnnOvTp4+PZYlh5+zy43K/AV2OWM9x999/v4+HDx8etG3YsCH2PHKfED0Xx70OqC3kONbf//bt2/v4pJNOCtrk/LdkyZKg7YsvvgiO5Tyu9xeSsrVnV7axwgAAAAAAAETwwAAAAAAAAETkJCXBWhaol1bItIMDDzww9n2lpaXBsSy5aC1tbtmyZdCml3bKUjY6tcAqgSj73a1bt6Ctb9++Pj722GODtjZt2vh40aJFQduLL77o48aNGzugOlnlyqxlwTINR45T/b7Vq1cHbXqpo1wGai3JttIOKLOGmkJ/x+U8ppdIrl+/3sd6TpVzx6WXXhq0FRcX+3jEiBFB26BBg3ws0+Mqu4ak0+VkSuDatWuDtvvuu8/Hsvyxc9H0vTh6KWcmqYRAUvJ7pb9zcq7S92rytTqV4YMPPvDx+++/H3ttPRda6Qr6t0Eui9YpCdb75Hn1mJKfiTQ/1FTW/aA13k855RQfy7/9nAvnxldffTVos8qa6vmuJqTSMtMCAAAAAIAIHhgAAAAAAIAIHhgAAAAAAICI1HsY6JwKKzfEyn/8/vvvfbxp06agTeaGde7cOWi79957faxLN8kyFzq/TF9j+fLlPpb7GTgXlpY6+OCDgzaZx2LlUOpcUJmbdthhhwVt7777ro/1ng1AvskxrsewPNZ5Wh07dqw0di7MW166dGnQpvcQ0TndcX0DCk3S76ccR3qMWWVFv/zySx/LOdS5MDdy9OjRQZvc00CXTpR9lqUZnYuOcTmOdb6nzNuePn16bL+tfUk0OW/q1xVKySnUbPp7JY/1ngJybFrjNum+HM7Ze+9YY8P6rdHXl2NTz7dWqUjremn7BuRaJvvpWSWO5d9/+rdA/k355JNPBm16Hm3YsGGl13POnu+zsRdXNs7BCgMAAAAAABDBAwMAAAAAABCROiUhk2VIcgmHXs6xc+dOH8uST84516tXLx/rpf2yBKNc5uFcuAxEL6Vs3759cCzLLup+yyWTOu1AvlYvCZUlr/SyL/laXYJDlqBq0KCBA3LNWrJlLcmUSxubNGkStN1xxx0+1ilBn376qY+//vrroE0vn5TXtFIirM+Q9PNV9VogrbRLdvWYkx5//HEf69Klct7UqXzSxo0bY/upry3T/PRrP/vss6BNLsucN29e0GaNWysFKinGMLIlk7kjzfsyOYd8rU7lscaKLusoU2mt8oyZlC5N++8E5Js1x8nvv/77S86xerzJ+1id1m6VYNRjjLKKAAAAAACgRuKBAQAAAAAAiOCBAQAAAAAAiMhaWUXJyr+w3nfrrbcGx1988YWPdZ50SUmJj3VO1yGHHOLjAQMGBG26POKWLVt8vGrVqqBtxYoVlcbOhbnYuk3uW6D3MJCssh768wJJZZLvZJWSkd9PK4dRlzU97bTTYl+7YMECH+vSoTpvTOZ4WdfPd75XoeSToWZIuseGzKHUbXoekXmT48ePD9puuukmHw8ePDj2nFY5RL33yLp164JjuU/Bgw8+GLTJ3w09puVcbZWj0+QYt14HZEvSPTWsedPK4c+kdJo8p97rRx/Lfbr0HLtnzx4fW/NtJiVPLcyVyLVM7v+s+115rPfs6d69e+z15BiT48u56DyadIznYo7LpBxqHGZeAAAAAAAQwQMDAAAAAAAQkTolQbOWb1mlLOT79BKJxx57zMey/KI+j7V8Qy+t0ktE5BISXYJGlmvUfZNLRPVSD7lETF9f0v9OckmY/rxAvllLu+R3Vb9OpujIlB/nnJs8ebKPrVKJzoXj2lqunXbZY9pyjEBV4pb76e+V/I7rFAFJlw6Wr12yZEnQdvHFF/v4vPPOC9ouuOACH3fo0CFo27Bhg4//9re/BW1vvvlmcLx69Wof68+kSyBLchxbS7J1G2kIyJakYzNpmyWTc1pLhOX3X79Oj7fevXv7WN9/Tp8+3cf6HjNp+kQmY5OyisiElTIgWX83JpVJ6dLly5f7+Mgjjwza5N+CLVu2DNp0ikI2SgdXJ2ZhAAAAAAAQwQMDAAAAAAAQwQMDAAAAAAAQUeUeBnE5FzpvKWnuic7TlOfZtWtXbJtVZtDKBdFtOm9L9tUqJaM/k3yt/kxWLqqVeyP3ZbDOATiXfGxKaUvQaHKfDj2mZN60zuGS9H4i+jtvjT9yI1HI4n7nrRJsVpknaw8Pa66YNWtWcDxnzhwf63Js1r45usyUHLtWeUY9/8p5U/9Opf0NAzKRdGxKmZRHjLtWJtez3qd/C/Q4fvfdd308bdq0oG3GjBmx15RjTu81Zu2hkPQ3jHGLqsR9X6y/N7Wkbda8uXv37uD4j3/8Y6V9dM65uXPn+njHjh2J+2KN8WyNlWyPP1YYAAAAAACACB4YAAAAAACAiCpTEpIuoZIyWdolX2stO7HSDqzrZbKURUta9iLtsjPrenqZmV4iA8R9z6zvox4P1pIlaxligwYNfCzLymh6aWNcv3Rfqrp+LsqsJf3dosQbqpJ26XFcm56LZJsci/q1VoljXUZYpgtYqRP6PHppp3yvHv/Wssu0pVIZm0gr37/51nybdC7W400vg77rrrt8bJUKzySVV47jtGkH1r0AoKX9ezPp34P6uyrnP/331tdff+3jq6++OvZ6Vpqdlq172myUAE96DmZXAAAAAAAQwQMDAAAAAAAQwQMDAAAAAAAQYe5hUFRUFOQdWeVTrLzJpPnVafcpsF5b1Z4FSc+bSXmMpKx/C8oqwqLHZtLSbXqMybwp6336+69LO0nytXp/Ayv32sp3ziQ3zJKNfC+dFwpoacZmJvv7SHosWuXR4q6tr2eVTqvqtXGv0+fJVvlXyfpdArJ1T5uNPQ3S7vWlWeVQ9fiX40PfY8r9DbSk41az9kwBpKKiotg9BtLOm2nLcev3yXGjz2GdU48xOR6t/fXS7uGjJf2NSfr3JisMAAAAAABABA8MAAAAAABARJUpCbK8365du4K2OGnLp6Qt15LJObW057WukY1zNm7cODi2lp1h/6PHZtKym5mUOdTXi3utNW512oFs08ugMkllymQ5Z1JJ36fHplVWEvufbI3NpN/HTEqlWuM26Zh2zl7KHfc6/dpM5s2kyzUbNWoUHMt7FsC6p63stf8nk/SZpNIupf4x95vyN0aX7pbzsZ6bs/H59bypy0Fi/5aLeTOTsZG0BGIm40/f/yYtK5wtaefNPXv2VPo6VhgAAAAAAIAIHhgAAAAAAIAIHhgAAAAAAICIoiryljc551bnrzswdKmoqGhX3Z1AYWBsFhTGJjzGZkFhbMJjbBYUxiY8xmZBqXRsmg8MAAAAAADA/omUBAAAAAAAEMEDAwAAAAAAEMEDAwAAAAAAEMEDAwAAAAAAEMEDAwAAAAAAEPH/ADfDQm9liYQMAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1440x288 with 10 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "decoded_imgs = ae.predict(x_test)\n",
    "n = 5\n",
    "plt.figure(figsize=(20, 4))\n",
    "for i in range(1, n+1):\n",
    "    # display original\n",
    "    ax = plt.subplot(2, n, i)\n",
    "    plt.imshow(x_test[i].reshape(28, 28))\n",
    "    ax.get_xaxis().set_visible(False)\n",
    "    ax.get_yaxis().set_visible(False)\n",
    "    # display reconstruction\n",
    "    ax = plt.subplot(2, n, i + n)\n",
    "    plt.imshow(decoded_imgs[i].reshape(28, 28))\n",
    "    ax.get_xaxis().set_visible(False)\n",
    "    ax.get_yaxis().set_visible(False)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generate counterfactual guided by the nearest class prototype"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Original instance:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7fd04d57c6d8>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAADO5JREFUeJzt3V2IXfW5x/Hf76QpiOlFYjUMNpqeogerSKKjCMYS9VhyYiEWg9SLkkLJ9CJKCyVU7EVzWaQv1JvAlIbGkmMrpNUoYmNjMQ1qcSJqEmNiElIzMW9lhCaCtNGnF7Nsp3H2f+/st7XH5/uBYfZez3p52Mxv1lp77bX/jggByOe/6m4AQD0IP5AU4QeSIvxAUoQfSIrwA0kRfiApwg8kRfiBpD7Vz43Z5uOEQI9FhFuZr6M9v+1ltvfZPmD7gU7WBaC/3O5n+23PkrRf0h2SxiW9LOneiHijsAx7fqDH+rHnv1HSgYg4FBF/l/RrSSs6WB+APuok/JdKOjLl+Xg17T/YHrE9Znusg20B6LKev+EXEaOSRiUO+4FB0sme/6ikBVOef66aBmAG6CT8L0u6wvbnbX9a0tckbelOWwB6re3D/og4a/s+Sb+XNEvShojY07XOAPRU25f62toY5/xAz/XlQz4AZi7CDyRF+IGkCD+QFOEHkiL8QFKEH0iK8ANJEX4gKcIPJEX4gaQIP5AU4QeSIvxAUoQfSIrwA0kRfiApwg8kRfiBpAg/kBThB5Ii/EBShB9IivADSRF+ICnCDyRF+IGkCD+QFOEHkmp7iG5Jsn1Y0mlJH0g6GxHD3WgKQO91FP7KrRHx1y6sB0AfcdgPJNVp+EPSVts7bY90oyEA/dHpYf+SiDhq+xJJz9p+MyK2T52h+qfAPwZgwDgiurMie52kMxHxo8I83dkYgIYiwq3M1/Zhv+0LbX/mo8eSvixpd7vrA9BfnRz2z5f0O9sfref/I+KZrnQFoOe6dtjf0sY47Ad6rueH/QBmNsIPJEX4gaQIP5AU4QeSIvxAUt24qy+FlStXNqytXr26uOw777xTrL///vvF+qZNm4r148ePN6wdOHCguCzyYs8PJEX4gaQIP5AU4QeSIvxAUoQfSIrwA0lxS2+LDh061LC2cOHC/jUyjdOnTzes7dmzp4+dDJbx8fGGtYceeqi47NjYWLfb6Rtu6QVQRPiBpAg/kBThB5Ii/EBShB9IivADSXE/f4tK9+xfe+21xWX37t1brF911VXF+nXXXVesL126tGHtpptuKi575MiRYn3BggXFeifOnj1brJ86dapYHxoaanvbb7/9drE+k6/zt4o9P5AU4QeSIvxAUoQfSIrwA0kRfiApwg8k1fR+ftsbJH1F0smIuKaaNk/SbyQtlHRY0j0R8W7Tjc3g+/kH2dy5cxvWFi1aVFx2586dxfoNN9zQVk+taDZewf79+4v1Zp+fmDdvXsPamjVrisuuX7++WB9k3byf/5eSlp0z7QFJ2yLiCknbqucAZpCm4Y+I7ZImzpm8QtLG6vFGSXd1uS8APdbuOf/8iDhWPT4uaX6X+gHQJx1/tj8ionQub3tE0kin2wHQXe3u+U/YHpKk6vfJRjNGxGhEDEfEcJvbAtAD7YZ/i6RV1eNVkp7oTjsA+qVp+G0/KulFSf9je9z2NyX9UNIdtt+S9L/VcwAzCN/bj4F19913F+uPPfZYsb579+6GtVtvvbW47MTEuRe4Zg6+tx9AEeEHkiL8QFKEH0iK8ANJEX4gKS71oTaXXHJJsb5r166Oll+5cmXD2ubNm4vLzmRc6gNQRPiBpAg/kBThB5Ii/EBShB9IivADSTFEN2rT7OuzL7744mL93XfL3xa/b9++8+4pE/b8QFKEH0iK8ANJEX4gKcIPJEX4gaQIP5AU9/Ojp26++eaGteeee6647OzZs4v1pUuXFuvbt28v1j+puJ8fQBHhB5Ii/EBShB9IivADSRF+ICnCDyTV9H5+2xskfUXSyYi4ppq2TtJqSaeq2R6MiKd71SRmruXLlzesNbuOv23btmL9xRdfbKsnTGplz/9LScummf7TiFhU/RB8YIZpGv6I2C5pog+9AOijTs7577P9uu0Ntud2rSMAfdFu+NdL+oKkRZKOSfpxoxltj9gesz3W5rYA9EBb4Y+IExHxQUR8KOnnkm4szDsaEcMRMdxukwC6r63w2x6a8vSrknZ3px0A/dLKpb5HJS2V9Fnb45J+IGmp7UWSQtJhSd/qYY8AeoD7+dGRCy64oFjfsWNHw9rVV19dXPa2224r1l944YViPSvu5wdQRPiBpAg/kBThB5Ii/EBShB9IiiG60ZG1a9cW64sXL25Ye+aZZ4rLcimvt9jzA0kRfiApwg8kRfiBpAg/kBThB5Ii/EBS3NKLojvvvLNYf/zxx4v19957r2Ft2bLpvhT631566aViHdPjll4ARYQfSIrwA0kRfiApwg8kRfiBpAg/kBT38yd30UUXFesPP/xwsT5r1qxi/emnGw/gzHX8erHnB5Ii/EBShB9IivADSRF+ICnCDyRF+IGkmt7Pb3uBpEckzZcUkkYj4me250n6jaSFkg5Luici3m2yLu7n77Nm1+GbXWu//vrri/WDBw8W66V79psti/Z0837+s5K+GxFflHSTpDW2vyjpAUnbIuIKSduq5wBmiKbhj4hjEfFK9fi0pL2SLpW0QtLGaraNku7qVZMAuu+8zvltL5S0WNKfJc2PiGNV6bgmTwsAzBAtf7bf9hxJmyV9JyL+Zv/7tCIiotH5vO0RSSOdNgqgu1ra89uercngb4qI31aTT9gequpDkk5Ot2xEjEbEcEQMd6NhAN3RNPye3MX/QtLeiPjJlNIWSauqx6skPdH99gD0SiuX+pZI+pOkXZI+rCY/qMnz/sckXSbpL5q81DfRZF1c6uuzK6+8slh/8803O1r/ihUrivUnn3yyo/Xj/LV6qa/pOX9E7JDUaGW3n09TAAYHn/ADkiL8QFKEH0iK8ANJEX4gKcIPJMVXd38CXH755Q1rW7du7Wjda9euLdafeuqpjtaP+rDnB5Ii/EBShB9IivADSRF+ICnCDyRF+IGkuM7/CTAy0vhb0i677LKO1v38888X682+DwKDiz0/kBThB5Ii/EBShB9IivADSRF+ICnCDyTFdf4ZYMmSJcX6/fff36dO8EnCnh9IivADSRF+ICnCDyRF+IGkCD+QFOEHkmp6nd/2AkmPSJovKSSNRsTPbK+TtFrSqWrWByPi6V41mtktt9xSrM+ZM6ftdR88eLBYP3PmTNvrxmBr5UM+ZyV9NyJesf0ZSTttP1vVfhoRP+pdewB6pWn4I+KYpGPV49O290q6tNeNAeit8zrnt71Q0mJJf64m3Wf7ddsbbM9tsMyI7THbYx11CqCrWg6/7TmSNkv6TkT8TdJ6SV+QtEiTRwY/nm65iBiNiOGIGO5CvwC6pKXw256tyeBviojfSlJEnIiIDyLiQ0k/l3Rj79oE0G1Nw2/bkn4haW9E/GTK9KEps31V0u7utwegV1p5t/9mSV+XtMv2q9W0ByXda3uRJi//HZb0rZ50iI689tprxfrtt99erE9MTHSzHQyQVt7t3yHJ05S4pg/MYHzCD0iK8ANJEX4gKcIPJEX4gaQIP5CU+znEsm3GcwZ6LCKmuzT/Mez5gaQIP5AU4QeSIvxAUoQfSIrwA0kRfiCpfg/R/VdJf5ny/LPVtEE0qL0Nal8SvbWrm71d3uqMff2Qz8c2bo8N6nf7DWpvg9qXRG/tqqs3DvuBpAg/kFTd4R+tefslg9rboPYl0Vu7aumt1nN+APWpe88PoCa1hN/2Mtv7bB+w/UAdPTRi+7DtXbZfrXuIsWoYtJO2d0+ZNs/2s7bfqn5PO0xaTb2ts320eu1etb28pt4W2P6j7Tds77H97Wp6ra9doa9aXre+H/bbniVpv6Q7JI1LelnSvRHxRl8bacD2YUnDEVH7NWHbX5J0RtIjEXFNNe0hSRMR8cPqH+fciPjegPS2TtKZukdurgaUGZo6srSkuyR9QzW+doW+7lENr1sde/4bJR2IiEMR8XdJv5a0ooY+Bl5EbJd07qgZKyRtrB5v1OQfT9816G0gRMSxiHilenxa0kcjS9f62hX6qkUd4b9U0pEpz8c1WEN+h6SttnfaHqm7mWnMr4ZNl6TjkubX2cw0mo7c3E/njCw9MK9dOyNedxtv+H3ckoi4TtL/SVpTHd4OpJg8ZxukyzUtjdzcL9OMLP0vdb527Y543W11hP+opAVTnn+umjYQIuJo9fukpN9p8EYfPvHRIKnV75M19/MvgzRy83QjS2sAXrtBGvG6jvC/LOkK25+3/WlJX5O0pYY+Psb2hdUbMbJ9oaQva/BGH94iaVX1eJWkJ2rs5T8MysjNjUaWVs2v3cCNeB0Rff+RtFyT7/gflPT9Onpo0Nd/S3qt+tlTd2+SHtXkYeA/NPneyDclXSRpm6S3JP1B0rwB6u1XknZJel2TQRuqqbclmjykf13Sq9XP8rpfu0JftbxufMIPSIo3/ICkCD+QFOEHkiL8QFKEH0iK8ANJEX4gKcIPJPVP82g/p9/JjhUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "X = x_test[0].reshape((1,) + x_test[0].shape)\n",
    "plt.imshow(X.reshape(28, 28))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Counterfactual parameters:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "shape = (1,) + x_train.shape[1:]\n",
    "gamma = 100.\n",
    "theta = 100.\n",
    "c_init = 1.\n",
    "c_steps = 2\n",
    "max_iterations = 500\n",
    "feature_range = (x_train.min(),x_train.max())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run counterfactual:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time to find prototypes each class: 16.354 sec\n",
      "Explanation took 4.252 sec\n"
     ]
    }
   ],
   "source": [
    "# set random seed\n",
    "np.random.seed(1)\n",
    "tf.set_random_seed(1)\n",
    "\n",
    "# define models\n",
    "cnn = load_model('mnist_cnn.h5')\n",
    "ae = load_model('mnist_ae.h5')\n",
    "enc = load_model('mnist_enc.h5')\n",
    "\n",
    "sess = K.get_session()\n",
    "\n",
    "# initialize explainer, fit and generate counterfactual\n",
    "cf = CounterFactualProto(sess, cnn, shape, gamma=gamma, theta=theta,\n",
    "                         ae_model=ae, enc_model=enc, max_iterations=max_iterations,\n",
    "                         feature_range=feature_range, c_init=c_init, c_steps=c_steps)\n",
    "start_time = time()\n",
    "cf.fit(x_train)  # find class prototypes\n",
    "print('Time to find prototypes each class: {:.3f} sec'.format(time() - start_time))\n",
    "start_time = time()\n",
    "explanation = cf.explain(X)\n",
    "print('Explanation took {:.3f} sec'.format(time() - start_time))\n",
    "\n",
    "sess.close()\n",
    "K.clear_session()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Counterfactual prediction: 9\n",
      "Closest prototype class: 9\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7fd04bfb67b8>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAD75JREFUeJzt3WuMlGWaxvHrFkU8tAqiLWqveCDGCYYeJUZdXA8rRskITkjI+GFlzWSamCEycUw0+kFj/DCaGYx+MWLGiJtZxkk8gMm4q9sxgrpRDqIgchIYBRoahCB4oBXu/dAvsz3a7/22dXqr+/n/EkJ1XfVUPRRc1OGpeh9zdwFIz1FlTwBAOSg/kCjKDySK8gOJovxAoig/kCjKDySK8gOJovxAoo5u5I2ZGR8nBOrM3W0gl6vqkd/MbjSzdWa20czurea6ADSWVfrZfjMbJmm9pMmStkpaKulWd18TjOGRH6izRjzyXyZpo7tvcvceSX+WNK2K6wPQQNWU/yxJn/X5eWt23j8wsw4zW2Zmy6q4LQA1Vvc3/Nx9nqR5Ek/7gWZSzSP/NkltfX4+OzsPwCBQTfmXShpnZuea2XBJv5C0qDbTAlBvFT/td/fvzGy2pP+WNEzSM+7+Uc1mBqCuKl7qq+jGeM0P1F1DPuQDYPCi/ECiKD+QKMoPJIryA4mi/ECiKD+QKMoPJIryA4mi/ECiKD+QKMoPJIryA4mi/ECiKD+QKMoPJIryA4mi/ECiKD+QKMoPJIryA4mi/ECiKD+QKMoPJIryA4mi/ECiKD+QKMoPJIryA4mqeItuSTKzLZL2Szok6Tt3n1iLSQ02w4YNC/NDhw41aCZDy/Dhw8O8p6enQTMZmqoqf+Zad99dg+sB0EA87QcSVW35XdJrZrbczDpqMSEAjVHt0/5J7r7NzE6X9LqZrXX3xX0vkP2nwH8MQJMxd6/NFZk9KOmAu/8+uExtbqzJ8IZfffCGX2Xc3QZyuYqf9pvZCWbWcuS0pBskra70+gA0VjVP+1slvWRmR67nP939v2oyKwB1V7On/QO6sSH6tL9ara2tYb5z584GzeSHHnjggTBfu3ZtmG/bti03e+uttyqa00BlD0z9Ovfcc8OxmzZtqvV0GqbuT/sBDG6UH0gU5QcSRfmBRFF+IFGUH0gUS30D1NLSkpsde+yx4djdu+MvPba3t4f5iSeeWPH133///eHYzZs3h/npp58e5rNmzQrzhx9+ODcrWk477bTTwvzRRx+tanxk165dFY8tG0t9AEKUH0gU5QcSRfmBRFF+IFGUH0gU5QcSxTr/AEVH6yk6Us8555wT5u+8806YP/3002F+/vnnV5QNxLPPPhvm119/fZhfdNFFuVnRv70DBw6E+Zo1a8L8lFNOyc1mzJgRjj18+HCYNzPW+QGEKD+QKMoPJIryA4mi/ECiKD+QKMoPJIp1/gEaOXJkbrZ3795w7Pjx48N88uTJYX7JJZeE+TXXXJObLV26NBx76aWXhnlbW1uYv/zyy2Ee/dk++eSTcOwXX3wR5suWLQvzsWPH5mZFxxK4++67w7xI0TEYij7DUA3W+QGEKD+QKMoPJIryA4mi/ECiKD+QKMoPJOrooguY2TOSfiap293HZ+eNkvS8pLGStkia4e7xYvcgV7SWHxkxYkSYf/rpp2E+d+7cMF+1alVuVnRs+8svvzzM161bF+Y9PT1hPm7cuNxs9uzZ4dgrr7wyzB955JEwX7JkSW5WtN9Ateq5jl8rA3nkf1bSjd87715Jne4+TlJn9jOAQaSw/O6+WNKe7509TdL87PR8SbfUeF4A6qzS1/yt7t6Vnd4hqbVG8wHQIIWv+Yu4u0ef2TezDkkd1d4OgNqq9JF/p5mNkaTs9+68C7r7PHef6O4TK7wtAHVQafkXSZqZnZ4paWFtpgOgUQrLb2YLJP2vpAvNbKuZ/VLS7yRNNrMNkq7PfgYwiPB9/syYMWPCvKurK8wj1157bZgX7SMf7RkgSQsWLMjNrr766nDsTTfdFOadnZ1hfscdd4T5K6+8kptdddVV4djrrrsuzN9///0wf+GFF3Kz7u7cV6qSpNdeey3MixT9nRXt9VANvs8PIET5gURRfiBRlB9IFOUHEkX5gUSx1JdpaWkJ8/3799fttqdOnRrm27dvD/PzzjuvokySNmzYEOYrV64M86K5v/TSS7nZ5s2bw7FFDh48GObRMuSLL74Yjt23b19Fc2oGLPUBCFF+IFGUH0gU5QcSRfmBRFF+IFGUH0hU1YfxGirquY4ffbVUig9vLUmvvvpqmN9zzz252ejRo8Oxu3fvDvPbb789zN97770wj766+tBDD4Vji7bJLvqadfRnP/po/unzyA8kivIDiaL8QKIoP5Aoyg8kivIDiaL8QKJY7GyANWvWhPlxxx0X5m+//XaYjxw5MjebM2dOOPakk04K86eeeirMp02bFubR8QI2btwYji1a5y/6jMHy5cvDPHU88gOJovxAoig/kCjKDySK8gOJovxAoig/kKjC4/ab2TOSfiap293HZ+c9KOlXknZlF7vP3f9aeGNNfNz+ItF6eNGxAM4444wwf+yxx8L8ySefDPPo2PyPP/54OLZonb9aX3/9dW521FHxY8/ixYvDfMqUKWH+7bffhvlQVcvj9j8r6cZ+zn/M3duzX4XFB9BcCsvv7osl7WnAXAA0UDWv+Web2Ydm9oyZ5X++FEBTqrT8T0o6X1K7pC5Jf8i7oJl1mNkyM1tW4W0BqIOKyu/uO939kLsflvS0pMuCy85z94nuPrHSSQKovYrKb2Zj+vz4c0mrazMdAI1S+JVeM1sg6RpJo81sq6QHJF1jZu2SXNIWSbPqOEcAdVC4zl/TGxvE6/yR4cOHh/mZZ54Z5tOnTw/z7u7uMF+0aFFudsEFF4Rji/7+V6xYEebVHItg2LBh4dibb745zEeMGBHm69evD/Ohqpbr/ACGIMoPJIryA4mi/ECiKD+QKMoPJIqlvkGgra0tzKOvxn755Zfh2KItuousWrUqzFtaWnKzN954IxxbtD140f3y2WefhflQxVIfgBDlBxJF+YFEUX4gUZQfSBTlBxJF+YFEsUX3IFDmevXUqVPDvOhruW+++WZu9u6771Y0pyNSXcevFR75gURRfiBRlB9IFOUHEkX5gURRfiBRlB9IFOv8TWD06NFhfuDAgTD/5ptvKr7ts88+O8znzp0b5lu3bg3zY445Jjd7/vnnw7GoLx75gURRfiBRlB9IFOUHEkX5gURRfiBRlB9IVOFx+82sTdJzkloluaR57v64mY2S9LyksZK2SJrh7nsLrmtIHrd/1KhRYb5nz54wL/pO/IQJE8I8Onb+7Nmzw7G33XZbmLe3t4f5DTfcEOb79u3LzXp6esKxK1euDHP0r5bH7f9O0m/d/SeSLpf0azP7iaR7JXW6+zhJndnPAAaJwvK7e5e7r8hO75f0saSzJE2TND+72HxJt9RrkgBq70e95jezsZJ+KuldSa3u3pVFO9T7sgDAIDHgz/ab2YmSXpD0G3f/wuz/X1a4u+e9njezDkkd1U4UQG0N6JHfzI5Rb/H/5O4vZmfvNLMxWT5GUnd/Y919nrtPdPeJtZgwgNooLL/1PsT/UdLH7t73K16LJM3MTs+UtLD20wNQLwNZ6pskaYmkVZIOZ2ffp97X/X+R9E+S/qbepb5wTWuoLvXV25w5c8L8q6++ys2iQ2dL0tq1ayua0xFXXHFFmO/atSs3O/7448Oxq1evrmhOqRvoUl/ha353f0tS3pX964+ZFIDmwSf8gERRfiBRlB9IFOUHEkX5gURRfiBRhev8Nb0x1vkrcuqpp4b52LFjc7M777wzHNvV1VVV/sQTT4R5tJZ/4YUXhmNXrFgR5uhfLb/SC2AIovxAoig/kCjKDySK8gOJovxAoig/kCi26B4EPv/88zC/6667crOLL744HLtjx44w/+CDD8K86HMiUb59+/ZwLOqLR34gUZQfSBTlBxJF+YFEUX4gUZQfSBTlBxLF9/mHgM7Oztxsw4YN4diOjngntUmTJoX58uXLw/zgwYNhjtrj+/wAQpQfSBTlBxJF+YFEUX4gUZQfSBTlBxJV+H1+M2uT9JykVkkuaZ67P25mD0r6laQjG7Df5+5/rddEh7KTTz45zNvb28N84cKFudm4cePCsdOnTw/zPXv2hDnr+IPXQA7m8Z2k37r7CjNrkbTczF7Pssfc/ff1mx6Aeiksv7t3SerKTu83s48lnVXviQGorx/1mt/Mxkr6qaR3s7Nmm9mHZvaMmY3MGdNhZsvMbFlVMwVQUwMuv5mdKOkFSb9x9y8kPSnpfEnt6n1m8If+xrn7PHef6O4TazBfADUyoPKb2THqLf6f3P1FSXL3ne5+yN0PS3pa0mX1myaAWissv5mZpD9K+tjd5/Y5f0yfi/1c0uraTw9AvQzk3f5/lvRvklaZ2crsvPsk3Wpm7epd/tsiaVZdZjgEHHVU/H/s/v37w7zo0N0TJkzIzQ4dOhSOXbJkSZjv3bs3zDF4DeTd/rck9ff9YNb0gUGMT/gBiaL8QKIoP5Aoyg8kivIDiaL8QKI4dPcQN2rUqDAv+souBh8O3Q0gRPmBRFF+IFGUH0gU5QcSRfmBRFF+IFGNXuffJelvfc4aLWl3wybw4zTr3Jp1XhJzq1Qt53aOu582kAs2tPw/uHGzZc16bL9mnVuzzktibpUqa2487QcSRfmBRJVd/nkl336kWefWrPOSmFulSplbqa/5AZSn7Ed+ACUppfxmdqOZrTOzjWZ2bxlzyGNmW8xslZmtLHuLsWwbtG4zW93nvFFm9rqZbch+73ebtJLm9qCZbcvuu5VmNqWkubWZ2RtmtsbMPjKzOdn5pd53wbxKud8a/rTfzIZJWi9psqStkpZKutXd1zR0IjnMbIukie5e+pqwmf2LpAOSnnP38dl5j0ra4+6/y/7jHOnu9zTJ3B6UdKDsnZuzDWXG9N1ZWtItkv5dJd53wbxmqIT7rYxH/sskbXT3Te7eI+nPkqaVMI+m5+6LJX3/aBvTJM3PTs9X7z+ehsuZW1Nw9y53X5Gd3i/pyM7Spd53wbxKUUb5z5L0WZ+ft6q5tvx2Sa+Z2XIz6yh7Mv1ozbZNl6QdklrLnEw/CndubqTv7SzdNPddJTte1xpv+P3QJHe/RNJNkn6dPb1tSt77mq2ZlmsGtHNzo/Szs/TflXnfVbrjda2VUf5tktr6/Hx2dl5TcPdt2e/dkl5S8+0+vPPIJqnZ790lz+fvmmnn5v52llYT3HfNtON1GeVfKmmcmZ1rZsMl/ULSohLm8QNmdkL2RozM7ARJN6j5dh9eJGlmdnqmpIUlzuUfNMvOzXk7S6vk+67pdrx294b/kjRFve/4fyLp/jLmkDOv8yR9kP36qOy5SVqg3qeB36r3vZFfSjpVUqekDZL+R9KoJprbf0haJelD9RZtTElzm6Tep/QfSlqZ/ZpS9n0XzKuU+41P+AGJ4g0/IFGUH0gU5QcSRfmBRFF+IFGUH0gU5QcSRfmBRP0fBFhEZ2V9Y4gAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "print('Counterfactual prediction: {}'.format(explanation['cf']['class']))\n",
    "print('Closest prototype class: {}'.format(cf.id_proto))\n",
    "plt.imshow(explanation['cf']['X'].reshape(28, 28))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The counterfactual starting from a 7 moves towards its closest prototype class: a 9. The evolution of the counterfactual during the first iteration can be seen below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "iteration c 0\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA8cAAACWCAYAAADzNfqVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xd4FWXexvEJgZBgCS2hNxUBBY2IwmapIiu6CpYVV11YdV2QCxELXmLBjiK7FtaCqCyyqKwi2EAIiiBiQwORJqEHkE4SQJIASc77h6/v9c7cvyRjyslJ5vv577nznHMmyZM583iY26hQKOQAAAAAABBkNSr7AAAAAAAAqGxsjgEAAAAAgcfmGAAAAAAQeGyOAQAAAACBx+YYAAAAABB4bI4BAAAAAIFXs7gvRkVF8f95QrFCoVBUuF6L9YiShGs9shZREtYiIgVrERFkfygUSgjHC7EeUZKizo18cgwAAACgomVU9gEAJWFzDAAAAAAIPDbHAAAAAIDAK/aeYwBAsEVHR0tWUFBQ4uNiY2Mly8vLK5djAgAAqAh8cgwAAAAACDw2xwAAAACAwGNzDAAAAAAIPDbHAAAAAIDAo5ALAFAkq3zLW7ZlFW1RvgUAAKoaPjkGAAAAAAQem2MAAAAAQOCxOQYAAAAABB6bYwAAAABA4FHIBUSg6OhoyaxiJKAy+CnbqllT317y8/Mr4nAAIOLExcVJlpubK1m9evUky8rKqpBjAlAyPjkGAAAAAAQem2MAAAAAQOCxOQYAAAAABB6bYwAAAABA4FHIBVSydu3aSZaTkyPZ9u3bw3E4JaJoKbJZ5S7JycmSxcbGSjZr1izJJk6cKNnbb7/tGjds2FDmfPPNN5Lt3btXMsDi9zyTlJTkGqelpVXYMaF6sc6VVtmgVaIVHx8v2eHDh0t8nIXyLSCy8MkxAAAAACDw2BwDAAAAAAKPzTEAAAAAIPCqxD3HtWvXluzo0aOVcCQVj/s5K4Z1f6V1b1FlSE9Pl6xjx46Slec9xwkJCZLt27fP12NZj5HNWusxMTGSWfcX33XXXZJdcMEFkq1atco1Hjx4sMwZNmxYscf5K+5DRteuXSWrU6eOZD/++KNkfu4x5n0VFut98MiRI5K1adNGskOHDkm2cuXK8jkwVCvR0dGSFRQUVMKRlC/ruuLYsWOVcCTlj0+OAQAAAACBx+YYAAAAABB4bI4BAAAAAIHH5hgAAAAAEHhRoVCo6C9GRRX9xUpWo4bu6wsLCyUbOHCgZFbhwv79+yVbvXp1icfRqFEjyZKTkyVLTU2VbPPmzZINGjRIspSUFMkOHjxY4rGFQygUigrXa1X0erRKExo3bixZixYtJOvUqZNkr776aqmPpXXr1pJ51/LEiRNlTpMmTSTbtWuXr9e0SsD8/A1EknCtx0g+N1r69Okj2aJFi3w9tn379pJ53zesUjlL/fr1JcvMzPT12KomaGvRKtDKycmRrKILNq3zWLdu3SSbO3euZNa5slevXpJlZWVJZpUxnXDCCZJZ1x8VLWhrsby1bNlSst27d0vmXStWadeFF14o2QcffODrOE488UTJvvnmG8n8FtBV0vpMDYVCXSr6RRyn6q1Haz+xZ8+ecnv+hx56SLJ169ZJ9tNPP0m2dOnSUr9uVJSefqySO2tPVNGKOjfyyTEAAAAAIPDYHAMAAAAAAo/NMQAAAAAg8NgcAwAAAAACT+/aD6PTTz9dMquwxSoZGjBggGTvvvuuZAcOHJDMKt+ySmfWr1/vGls3rt93332STZ8+XbLDhw9L9sorr0hm3ZD/zjvvSGaVRxUUFEiGX9SqVUuy48ePu8bWz88qJrj++uslswq0/vnPf0o2evTo4g7z/1x00UWSXXbZZa6xVeKxZcsWye6++27JrFIxq5zj5ptvlsz6WZ588smSeX++juM4eXl5rvGxY8dkDhynbt26klk/T6vwyCqRswrj/vWvf0nWtGlTyaxzspdV1jF27FjJnnjiCcmsc+OcOXMkO//88yUbPHhwicfmOPb5Mj4+3jWursVgFaFevXqSWSVVFqt8q3v37pJdfvnlku3du1cy7/WBtZ6sv5OpU6dKZq2ndu3aSfbZZ59JduaZZ0q2Zs0ayVC+rPOdVZblV79+/SRbsGCBZFZ5W0JCQomPGzFihK/Mej9esWKFZFYRnHVtvW3bNsms9xSUzUknnSSZVUJo7UOsvU7btm19Pfb++++XzHs9mJiYKHMefvhhyR5//HFfx+Fd747jOBMmTJCsYcOGklnn6UjCJ8cAAAAAgMBjcwwAAAAACDw2xwAAAACAwGNzDAAAAAAIvKhQKFT0F6Oiiv5iKXhLe7p27SpzCgsLJfvkk08kW758uWRvvPGGZM8884xkR44ckcwqNTh48KBrfPHFF8ucr7/+WrJXX31Vsv79+0tm3VT/4IMPSmaVitWoof9dw/rZVbRQKBQVrtcq7/Xoh1VGZJW3jR8/XrITTzxRMqvwpWPHjpKNGjVKsri4ONc4IyND5rRq1UqypKQkyawyka1bt0pmFTndfvvtkv33v/+VLCYmRrKff/7ZNS7vQpBwrceyrEWrDO3QoUOusVVwZhVSJScnSzZx4kTJbrnlFsnuvffeYo+zrKziJeu8bRUTduvWTTKrHM9an9Y51PvzdRw9r1rHWxZVYS360atXL8k+//xzX4+dMmWKZBs2bJDsySef/O0H9r+s92A/rHIvq7DGsmzZMsmsghlrLRZ3zVVRqstatMTGxkpmlahZRZLff/+9ZNb6tNbF5s2bJfMWGlrnnX379klmadOmjWRW4ebChQslW7t2rWQzZszw9bphkBoKhbqE44UqYz36Lcu1rtW++uoryaz9xKmnnuor83r99dclu/DCCyXr0KGDZNZ5y3s95zj22rMKRgcNGiRZJO1h+OQYAAAAABB4bI4BAAAAAIHH5hgAAAAAEHhsjgEAAAAAgRfWQq569eq5xo0aNZI51vHMnj1bMusG97POOkuy6dOnS2YVZ1g3hzdo0EAyL+t4P/vsM8n69u0r2aJFiySzvgerFMq6md/Kli5dKll5qu6FXNY6e+KJJyTLzMyUzCqDe+655ySzihSsEpzFixe7xmvWrJE5zZo1k6xHjx6SXXLJJZJddNFFkn366aeSWa976623SvbCCy9IVtGqQvHMmWeeKdlJJ53kGq9YsULmTJ48WTKrKGbs2LGSWcUzlnPPPVey2267TbJvvvnGNba+p2nTpkn2/PPP+3rNlJQUyS699FLJrMKbRx99VLL8/HzJ1q9f7xrPmzdP5pRFVViLfljlRunp6aV+vp49e0p29dVXS7Zt2zbJ+vXrJ9ncuXNdY+t9cMSIEZJZfxNWwd3OnTsl8xYvOY5dNjdz5kzJrFIcb3njxo0bZU5ZVJe1eNppp0nm92dlFWlaJauWO++8UzLrvXb79u2usXV+sgo4rYI3a152drZk1lq0CvNuuOEGyXJyciRLSEhwja33+zKq1oVc3n2O4zhOVlaWZNbv1zq/de7cWbLevXtL9t1330nmfW+1ykDff/99X8exadMmyaz1bZ1XW7duLZlVaDd69GjJvKyiW6sYzC8KuQAAAAAAKAKbYwAAAABA4LE5BgAAAAAEHptjAAAAAEDg1Qzni3lvSu/fv7/MueyyyyTzlr84juN0797d12t6CxIcx3Heeustyb744gvJatWq5Rpb5RoPPPCAZFb5h1VqUqdOHckWLlwomXUDevPmzSWzyre8RT9WGVkQtGzZUjKr8MUrOjpaMmvtWT97q3Dhnnvukezaa6+VzCrUiImJKfI4f3XHHXdIZpWFFVfE9/+1adNGMutvaty4cb6eD6UvOLGK4EaNGiVZamqqZNddd51kVkGLtcaswhvvPKuEw8reeecdya644grJDh48KJlVHrNu3TrJnn76acmGDBkimVWSAuW3fMsqezl69KhkS5YskSwuLk4y65zqLVFzHC2Ms/5OrEIlq8DG0rVrV8nS0tIks0qVXn75ZcmsAsbY2FjXuLwLuaoLvz+XLl2078k6Hx05ckQy6/144MCBksXHx0vm/VuxSgRfeeUVyazC1qFDh0pmFRZZ1zFWoae13sePHy+ZVRgH//y+r3j/5h3H/l1axa6rVq2SbMKECZJ169bNNbbO5ceOHZOsbdu2klmlq1aB4VNPPSWZtb8aNmyYZH6UpXzrt+CTYwAAAABA4LE5BgAAAAAEHptjAAAAAEDgsTkGAAAAAAReWAu5mjZt6hrXrVtX5ljFMZbs7Gxf86wSAquQ66677pJs+fLlrnFubq7MsUq6vv32W8mGDx8u2U033SRZRkaGZGeddZZk9erVk2zLli2SBbGAy7vOHMdxOnfuLJmfQi5voYHj2MVAlry8PMkmT54s2Y033ujr+crTX//6V8kuvvhiycaMGSOZVZQzffp0yRITEyXbu3ev30OstqwiPm8xjLV2evToIZlVzGeVX1glFo0bN5bMKj+0Mu+52zoOy8knnyzZSy+9JNnzzz8vmVW+ZZV61K9fX7L58+dL1r59+yKPE7+dVV5o/b1b5ZR+fz+HDh2S7O2333aNrVJLi1WmaZU29e7dW7IGDRpIZr3XWtcp1nu89X2h9LzXbo5jX+PNnTtXMuu9rFGjRpJZpUXe87hVvmWxStqs8i3rb8xine+tItGoqCjJDhw44Os14DhNmjSRbNeuXb4e6y3LdRz792vtiWbMmCGZVcLmfY+3ykCtAmGrBMzar1xwwQWSvf/++5LNmjVLsnPOOUeyBQsWSOZl/YwKCgpKfNxvxSfHAAAAAIDAY3MMAAAAAAg8NscAAAAAgMBjcwwAAAAACLywFnLt3LnTNbZKE/yyyrymTZsm2cqVKyWzbhjfvn27ZH6KCaxirPXr10vWr18/yaxymk6dOvl6rHVD/vHjx4s8zl9ZxRJWeYO3WKIq8a4zx7F/535Y66xDhw6SWb+3KVOmSPbGG2/4el2rrKFnz56ucWxsrMzJz8+XzO/3bpXnWOVO1vdlrRdrPcbExLjGx44d83VskaZGjRpOXFycKzvhhBNknlXIYxW6eQtZfve738mcHTt2SPbll19KZhW8WVnr1q0lu+aaayQbOHCgZN6SkPfee0/m9OnTR7I333xTMuucZBWCWAVNVpHN2rVrJbPWGeVw5cvve0Z6erpkzz33nK95P/30k2TJycklvqa1ThISEiQbNGiQZNbfYigUkswq4rTs37/f1zz4c8UVV0jWpUsXyRYvXixZSkqKZH//+98lW7RokWRpaWmS+SnM6tu3r2RWsdP1118vWWFhYYnP7ziO88gjj0hmnWet93f4Z107+2WtqQEDBkhmXc9a56lTTjlFMm8h8ebNm2WOtQasY/v3v/8tmcUq+vzwww8l83u+9KqI8i0LnxwDAAAAAAKPzTEAAAAAIPDYHAMAAAAAAo/NMQAAAAAg8MJayOUt42nevLnMsW7SvuWWWySzbj4fMmSIZFFRUZJZN7OvW7dOslq1arnGVsGQdeP6rl27JLMKtKyb43v06CGZVTjip3zLsmfPnlI9riqxfudWeVVubm6Jz7V7927JrAIDq4wmKSlJMqvMy2IVe/zhD39wjW+77TaZ8/nnn0tm/TysQhmroOaqq66S7E9/+pNkFqtwzk/JXVVQWFgov/OylNh5zwXWucEyZswYyaySQKtoa9y4cZJ9/fXXklmFRH6O49lnn5XMWjtW0YdV6mGVNlnlRlW15K0q6d69u2RLly719dgzzjjDV9a5c2fJrAIc7zq2zvWbNm2SrHfv3pK99NJLklnnyh9++EEyq0AMFc8qXbUKVs877zzJXnjhBcmsIkHvtavjOM7IkSMl85Z5tWzZUuZYRbSJiYmS9e/fX7LBgwdLNn/+fF/Z6tWrJbN437fz8vJkDufYXxw+fLjUj501a5Zkbdu2lWzevHmS3XPPPZI1bNhQMu/7o1XMuWzZMsms0qtHH31UstGjR0tm7X+sY6tZM6zbz9+MT44BAAAAAIHH5hgAAAAAEHhsjgEAAAAAgRfWf/TtvU/B+rfplpdfftnXPOvf5o8dO1ayxx57TLKFCxdKdvToUdfY+vf1kyZNksy6X+7QoUOStWnTRjLrf/ht3fdq3VODX1j3iPm5v9hi3VNi3ReSlpYmmXV/sbVGP/roI8mse+BHjRpV7Lgov//9733Ne+SRRySz7tXzy7ofFDbvPTnW76xp06aS7dixQ7INGzZI9t5770lm3Q9nnVcaNWok2cGDB11ja62//vrrkuXn50tm3VsXHx8vmfU9WN9/gwYNJKsu97pHiiZNmpT6sWvXrpXM26fgOPY9ktZ9euvXr3eNzz77bJljdZlY59jMzExfj/3www99PdZv3wNKLy4uTrLWrVtLZl2r3XvvvZJZ/QkPPPCAZFZ/gvd6wbp3MysrSzLvGnYcx3nrrbckmzNnjmT/+Mc/SjwOx7F7eqxuC++1ap06dWQO9xyXnXUetNbyl19+KZnVK2JdD3rvH588ebLMGThwoGTWNcTGjRsls/ZE1j3MqampkkU6PjkGAAAAAAQem2MAAAAAQOCxOQYAAAAABB6bYwAAAABA4EUVVw4RFRVVrs0R3pvIrbIKq9TCYt1YPmzYMMms72/KlCmSWQUDw4cPd42t47VMnDhRMqvo5qKLLpKsX79+kiUlJUnWrFkzyT755BPJKro4IRQK+fuhlIPyXo+lZf3sf/rpJ8nuuOMOyZ599lnJVq5cKVmnTp0k85aMXHnllTLn2muvley8886TzNKxY0fJrOKdTz/91NfzVYZwrceyrEWrdKO0hXGWiy++WLIvvvhCsu7du0tWq1YtyaziQG8p3aOPPipz9u3bJ1liYqJkVgGO9bdjnX+t8i0rs4pnrHKw8lQV1mJFu+qqqySrUUP/m/zMmTMls967H3zwQcmaN2/uGg8dOlTmTJs2TbIWLVpI1qdPH8msss5BgwZJZhUtRYrqvBatokKr2NQq6bKuj8aNGyfZDTfcIJlVvOo9D1pr8YcffpBs7ty5kt13332SWWWyVnb8+HHJvH8njmMXGoZBaigU6hKOF6qM9egtwXIcuyDNWj/W9aFVJGeVq1n7DutY/LCuR6zz9pIlSyS75JJLJLPWY6Qo6tzIJ8cAAAAAgMBjcwwAAAAACDw2xwAAAACAwGNzDAAAAAAIvJrhfLHyLKzYsmWLZOPHj5fMKnFZsGCBZLVr15bMW2DQsmVLmZORkVHscRb3mg899JCvx1rlEunp6ZJVdPlWECUkJEjmLSNyHLuQ68UXX5QsJSVFMquEYerUqSUem7fgznEcZ+DAgZLNmTNHMqucJD4+XrKylG9Z5U4FBQWucWFhYamfv6oqz/Ity0033SRZ7969JbOKM6wiuNLq1q2bZNbfRIcOHSTr37+/r9fIy8uT7Oeff5YsiOustLzvhaeddprMWbNmja/n+vjjjyWzSrVWrFjh6/msIqwrrrjCNbaKaazyy169ekk2Y8YMyaw1axXsILJZ12ojR46U7M4775TMOn9kZ2dL9uqrr7rGsbGxMscqWL377rslW79+vWTz58+XzG/ZUSWVbwXOoUOHJIuJiZHM2nN89913kln7jtmzZ0tmFbt27tzZNV6+fLnMsQpCf/zxR8mio6Ml+9vf/iaZVeBpreVIxyfHAAAAAIDAY3MMAAAAAAg8NscAAAAAgMBjcwwAAAAACLyoUChU9Bejoor+Yhj17dtXsoULF/p67JgxYySzirss3hvQvWVCjuM4Bw8elMwqxnr88cclmzJlimRWmYxVKtaqVSvJtm7dKllFC4VCenAVpDLWo1Vc5ffn3KdPH8kSExMlGzZsmGSbN2+WzFuCk5mZKXOskofJkydLNm/ePMleeeUVyazvNS0tTTKLVUJR0aVx4VqPkXJutPTs2VMyq0xj0aJFklmFROPGjZNs9erVJR5HampqiXMcx15PVtGHX02bNpXMKjWsaKxFx7nxxhsl85bEOI79/nj++edL9tFHH5X4mlYRnFWYuHbtWsmsc+Bdd90lWc2a2mWan59f4rFVFtai4wwfPlyyZcuWSWadt6xrsOKunX9Vv359yaxyuOnTp0tmlbha1wo5OTmSHThwoMRjq0SpoVCoSzheKJLXo18tWrSQrEYN/VzzyJEjku3fv7/E51+1apVkJ510kmTW9YJ1freOd/v27SUeR2Up6tzIJ8cAAAAAgMBjcwwAAAAACDw2xwAAAACAwGNzDAAAAAAIPG2ViEB+y7cs11xzjWTLly+XLCMjQ7L09PQSnz8+Pt7XcaxYsUKy+fPn+3pNq/ihMsq3gsgqvfLLKjX49ttvJfvqq68ks4rZcnNzXWOreGbo0KG+ju2SSy6RrGPHjpK1a9dOMr+FXBVdvgXbkiVLSv3Ya6+91te8Xr16ucbJyckyxyq7sUqQrOKlsqiM8i3YrBIXqxxuz549klnlWzfffLNk3vNRvXr1ZM7XX39d7HH+atu2bb7mWUWNGzdu9PVY+OctHiosLCz1c82cOVMyP4VFjmNfg1lroE2bNq5xnTp1ZM5bb70l2dKlSyV76qmnJOvQoYNkVllYSkqKZKiayrPMasCAAZJZ5+PPP/9cMuva1RLJ5Vu/BZ8cAwAAAAACj80xAAAAACDw2BwDAAAAAAKPzTEAAAAAIPCqRCHX6aefLtn69eslswqQpk2bJtmCBQtKdRw33nijZFlZWZJZJUZWwZKfwi+Ej1VscejQoVI/X1xcnGRWkVrDhg0lS0pKkmzlypWu8eLFi2XOpEmT/B+gx3XXXSfZa6+9Jpn1c7IKS1A56tevL5l1nrLmHThwQLIuXbpItmPHDtfYKpmxyhDPOeccyQYPHiwZqgfrfGedU/yyirsSEhJcY+ucbZ2frOKugwcP+joO6+8E5a8sBVxefsu3LCeccIJkzZo1k8z7Hm2tk9WrV0vWo0ePUh9bzZpV4jIePljXgtbeIS8vz9fzNW/e3DV+5plnZI73vdxxHKdWrVqSvf32275es7rgk2MAAAAAQOCxOQYAAAAABB6bYwAAAABA4LE5BgAAAAAEXpW4k99vKdLhw4clq1u3rmRW2VFaWlqJzz9lyhTJ/vjHP0o2fvx4yW666aYSn78o1veQnZ1d6ueDrSylUq1bt5YsIyPD12OtopDExETJxo0b5xrfcsstMqd9+/aSPf7445I1adJEstTUVMmaNm0q2ebNmyVD5MjMzPQ1zyqLsc6N999/v2TLli1zjb1r03Hsc+OsWbN8HRuqh+7du0tmne/WrVvn6/lycnJKnGOV2hw5csTXPKv8pqCgQDKr4A5Vj7UGrPVprR/resFbTDh16lSZY611q0zWb0Fofn6+ZIgsVvml9T5tnVfOPvtsyVatWiXZrbfeKtmQIUNc41NPPVXmDB8+XDKrmLBVq1aSVefzIJ8cAwAAAAACj80xAAAAACDw2BwDAAAAAAKPzTEAAAAAIPCqRCHX7t27S/3Yd999V7LVq1dLFh0dLZm3iMMqImrcuLFkVpFCv379ij3O4lglDPHx8ZJZN9EjPHbs2CGZ32KkUaNGSTZx4kTJvOU2hYWFvp7/gQcekGzs2LGS7dq1S7INGzZIFhMTI9mxY8d8HQsqR5cuXSTbs2ePZLVr15Zs9OjRknlL3lJSUmSOtSasIrjjx49Ltm3bNslQ9SxdutTXvK5du0o2ZswYyS699FLJ5s+f7xpb79O33XabZC1btpRs06ZNxR4nqherfMvy7bffSnb++edL9sEHH7jG1vnuP//5j2QNGjSQbOvWrb6ODZHP77WgVf7Xo0cPyc4991zJ5s6dK9nTTz/tGlslctb+wvq7qFOnjmTVGZ8cAwAAAAACj80xAAAAACDw2BwDAAAAAAKPzTEAAAAAIPCqRCGXXy1atJDMKt9q1aqVZDVr6o/CW0a0Zs0amXPBBRdItnfvXsmys7Ml88sqXsrNzS3186H8WSVsVtGBxSrfsowcOdI1topnpk6dKpnftZeYmChZTk6OZBS/RQ4/RYKO4zjff/+9r+c744wzJLNKB6+55hrX2Do3pqenS5aUlCRZQkKCZBRyVQ9WadGyZct8zXvttdckq1u3rmTe0i+rQO7999+X7Oqrr5asXbt2kr388suSoXqwrvuSk5MlGzFihGQvvviiZJMmTXKNrfPpnDlzJLv//vslswq53njjDclQvVm/89atW0tmraEJEya4xlbpqnU+tsq3rHNjdcYnxwAAAACAwGNzDAAAAAAIPDbHAAAAAIDAY3MMAAAAAAi8alXItXv3bl/zLr/8csn8lCJZJVhnn322ZEOGDJHMKgEri2PHjpXr86FsGjZsKJlVfnDhhRdK9umnn0o2Y8YMyR577DHXeMuWLTLnlFNOkcz6u7BKtTIzMyWzihlYe5HDKt8qi549e0pmFWZ99NFHrrF3bTqO43Ts2FGyxYsXS9ayZcvfcISoSqyyl9q1a0v2/PPPS2YVElmlgb169XKNn3zySZljne9++OEHyXbu3CmZVbYYCoUkQ9VjlQ0uWbJEsnHjxkl23nnnSeYtfuvfv7/MsdbO7NmzJWvevLlkCJ4DBw5Iduedd0rWqVMnybzXftY5z1qPVmadG6szPjkGAAAAAAQem2MAAAAAQOCxOQYAAAAABF5UcffOREVFReyNNQ0aNJCsbdu2kln3ZX722WeSnXnmmZJ5fzZ//vOfZc7pp58umXWf5pQpUyTbuHGjZFVNKBTSG7IqSCSvxwEDBkiWl5cn2ZgxYyS78sorJcvKypLsvvvuc42feOIJX8dm3Ve1Z88eyVq0aCGZ9T3s27fP1+tWhnCtx0hei351795dso8//liyE088UTLvunj44YdlzoQJEySz7mm27vGrDliLjhMfHy+Zdf9v+/btJYuJiZFs5cqVkg0ePNg1/vHHH2VOdna2ZNb9y3v37pWM92n/Inkt+lWjhn5mdNVVV0kWHR0tWe/evV3j1157TeZ8//33klm9C7Vq1ZJs06ZNklVBqaFQqEs4Xqg6rEfLwoULJduwYYNkQ4cOdY2t9/zU1FTJjh49Woajq1qKOjfyyTEAAAAAIPDYHAMAAAAAAo/NMQAAAAAg8Nih+IbbAAADP0lEQVQcAwAAAAACr2ZlH4BX7dq1JbNuDrcyqzxo5syZkhUUFPg6Fm9RTK9evWTOiBEjJBs5cqRkOTk5vl4TVdPy5cslu/feeyWzyg9mzZolmVWUd/z4cde4Q4cOMmfIkCGSWeVblu3bt0tWs2bEnSJQTtLT0yWbOHGiZG+++aZkJ598smu8bNkyX6+5f/9+yVq1aiWZVYyUm5vr6zUQOazyrVNOOUWygQMHSnb77bdL5i0ldBzHmT17domvadm6datkVsEmqi/r/a1+/fqSWWtx3rx5knnft63yLcu2bdskO+2003w9FtWHVWCYlJQk2QcffCCZVUjsLZLLzMyUOUEq3/ot+OQYAAAAABB4bI4BAAAAAIHH5hgAAAAAEHhsjgEAAAAAgRdlFf/83xejoor+YhXhvSHdcezSjSeffFIybzGDdTO7JS4uTrI6depIduDAAV/PF8lCoVBUuF4rktdjkyZNJEtOTpYsNjZWsh49ekhmrbWUlBTXOC0tzdex+S2osdSoof/9rLCwsNTPV9HCtR4jeS36LTU899xzJbPKP8aMGSPZM8884xpPmjTJ17FZ68nK8vPzfT1fJGMtlr++fftKlpGR4Rpv3LgxXIdTZbAWy1/jxo0lO+uss1zjBQsWhOtwqpLUUCjUJRwvFMnr0Xrfs5xxxhmS3XHHHZJZRcPeAsOsrCxfjwuSos6NfHIMAAAAAAg8NscAAAAAgMBjcwwAAAAACDw2xwAAAACAwKv2hVx+y2nKk3WjvXUcubm5FXoc4UAh1y+aN28u2c8//yxZdna2ZKeeeqpkmzZtkqxp06au8c6dO2VOs2bNJNu1a5dkkVyqVRYUz5S/v/zlL5LNnTvXNbaKPoKOtVj+rHKavXv3usb79+8P1+FUGazF8PBe+1XX99kyopCrAngLhB3Hf4lwkFHIBQAAAABAEdgcAwAAAAACj80xAAAAACDw2BwDAAAAAAKvpEKufY7jZITvcFDFtAqFQgnhejHWI0oQtvXIWkQJWIuIFKxFRBLWIyJFkWux2M0xAAAAAABBwD+rBgAAAAAEHptjAAAAAEDgsTkGAAAAAAQem2MAAAAAQOCxOQYAAAAABN7/AMOENCbeYoE7AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1440x288 with 6 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "iter_cf = 0\n",
    "print('iteration c {}'.format(iter_cf))\n",
    "n = len(explanation['all'][iter_cf])\n",
    "plt.figure(figsize=(20, 4))\n",
    "for i in range(n):\n",
    "    ax = plt.subplot(1, n+1, i+1)\n",
    "    plt.imshow(explanation['all'][iter_cf][i].reshape(28, 28))\n",
    "    ax.get_xaxis().set_visible(False)\n",
    "    ax.get_yaxis().set_visible(False)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Typically, the first few iterations already steer the 7 towards a 9, while the later iterations make the counterfactual more sparse."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Remove the auto-encoder loss term"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the previous example, we used both an auto-encoder loss term to penalize a counterfactual which falls outside of the training data distribution as well as an encoder loss term to guide the counterfactual to the nearest prototype class. In the next example we get rid of the auto-encoder loss term to speed up the counterfactual search and still get decent results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "theta = 500."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Explanation took 2.877 sec\n"
     ]
    }
   ],
   "source": [
    "# set random seed\n",
    "np.random.seed(1)\n",
    "tf.set_random_seed(1)\n",
    "\n",
    "# define models\n",
    "cnn = load_model('mnist_cnn.h5')\n",
    "enc = load_model('mnist_enc.h5')\n",
    "\n",
    "sess = K.get_session()\n",
    "\n",
    "# initialize explainer, fit and generate counterfactual\n",
    "cf = CounterFactualProto(sess, cnn, shape, theta=theta,\n",
    "                         enc_model=enc, max_iterations=max_iterations,\n",
    "                         feature_range=feature_range, c_init=c_init, c_steps=c_steps)\n",
    "cf.fit(x_train)\n",
    "start_time = time()\n",
    "explanation = cf.explain(X)\n",
    "print('Explanation took {:.3f} sec'.format(time() - start_time))\n",
    "\n",
    "sess.close()\n",
    "K.clear_session()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Counterfactual prediction: 9\n",
      "Closest prototype class: 9\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7fd04a3d3fd0>"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAEFNJREFUeJzt3XusVeWZx/Hf44Fya0XxQhCPWhE1QgiMJyiRIGNtRYRoJdEaMmESAo2pZmoM8Z7hHxOjtqbRSHLqDby1qG00sTp1CMpoRhGREQQVrFRBLiLKRUXh8MwfZ9Ee8axnbff98H4/CWGf/ex374cNP9bae631vubuApCewxrdAIDGIPxAogg/kCjCDySK8AOJIvxAogg/kCjCDySK8AOJ6lXPFzMzTicEaszdrZTHVbTlN7NJZvauma0zs+sreS4A9WXlnttvZi2S3pP0U0kbJL0u6Qp3Xx2MYcsP1Fg9tvxjJa1z97+5+zeS/iDp4gqeD0AdVRL+oZI+6vLzhuy+bzGz2Wa2zMyWVfBaAKqs5l/4uXu7pHaJ3X6gmVSy5d8oqbXLz8dn9wHoASoJ/+uShpvZj83sB5J+IemZ6rQFoNbK3u13931mdpWk/5LUIukBd3+7ap0BqKmyD/WV9WJ85gdqri4n+QDouQg/kCjCDySK8AOJIvxAogg/kCjCDySK8AOJIvxAogg/kCjCDySK8AOJIvxAouo6dTfS09LSklvr6OioYyc4GFt+IFGEH0gU4QcSRfiBRBF+IFGEH0gU4QcSxXH+OujVK36b9+3bV6dO6o9j+c2LLT+QKMIPJIrwA4ki/ECiCD+QKMIPJIrwA4mq6Di/ma2XtEtSh6R97t5WjaZ6moEDB4b1HTt21KmTnqV3795hfe/evXXqJE3VOMnnX919WxWeB0AdsdsPJKrS8Lukv5rZG2Y2uxoNAaiPSnf7x7v7RjM7VtILZvaOuy/p+oDsPwX+YwCajLl7dZ7IbK6k3e5+Z/CY6rxYk+ELv/LwhV9tuLuV8riyd/vNbICZ/ejAbUk/k7Sq3OcDUF+V7PYPlvRnMzvwPI+5+/NV6QpAzVVtt7+kFztEd/sbbfz48bm1Y445Jhy7Zs2asD5ixIiwPn369LA+c+bM3Nppp50Wjh01alRYb29vD+upqvluP4CejfADiSL8QKIIP5Aowg8kivADiWLq7hIde+yxZY/ds2dPWI8Oh0nSc889F9YHDBiQW1u5cmU4du3atRXVTz311LB+5525J3zq2muvDccedlhl26bDDz88t7Zz585wbLS0uHRoTEnOlh9IFOEHEkX4gUQRfiBRhB9IFOEHEkX4gURxSW+JRo4cWVZNknbv3h3WTzzxxLA+duzYsD5mzJjc2saNG8OxRbMQ1dJLL70U1pcuXRrW58yZE9Yfe+yx3NqTTz4Zjt28eXNYb2Zc0gsgRPiBRBF+IFGEH0gU4QcSRfiBRBF+IFFcz1+iDRs25NaKzpVobW0N68OHDw/rJ5xwQljv1Sv/r3HYsGHh2D59+oT1hx56KKzfcsstYT3S1hav6L5qVbwGTPTnlqTFixd/755SwpYfSBThBxJF+IFEEX4gUYQfSBThBxJF+IFEFR7nN7MHJE2RtNXdR2b3DZL0R0knSVov6TJ3/6x2bTbe559/XlZNktatWxfWi84D6N+/f1i/9NJLc2ujR48Ox65evTqsF81VcN5554X1aP77oiW2L7jggrD+xBNPhPVLLrkkt3YoX89fqlK2/A9JmnTQfddLWuTuwyUtyn4G0IMUht/dl0jaftDdF0uan92eLyn/v1gATancz/yD3X1TdnuzpMFV6gdAnVR8br+7ezQ3n5nNljS70tcBUF3lbvm3mNkQScp+35r3QHdvd/c2d4+v4gBQV+WG/xlJM7LbMyQ9XZ12ANRLYfjN7HFJ/yvpNDPbYGYzJd0m6admtlbS+dnPAHqQws/87n5FTuknVe6loS688MKwvnz58tzali1bwrFFx5SnTp0a1mfNmhXWt28/+GDMP5nFU7gfffTRYX3RokVhvUjfvn1za++++244dsKECWF91KhRYT06h+GTTz4Jx6aAM/yARBF+IFGEH0gU4QcSRfiBRBF+IFFM3Z3ZtGlTWC86nBcpuvS06LLbSZMOvqjy26JDfUW2bdtW9thSPPXUU7m1N998Mxx77733hvWi5cejw5RF036ngC0/kCjCDySK8AOJIvxAogg/kCjCDySK8AOJ4mBnZsWKFWWPPeuss8L6lClTwvrtt98e1oumsI4um/3444/DsUWKzlG45557wvrxxx+fW1u2bFk4duHChWG9aMr06JLhokt6oynHJamjoyOs9wRs+YFEEX4gUYQfSBThBxJF+IFEEX4gUYQfSJS55660Vf0XC5b1anZDhgzJrU2bNi0c+/zzz4f1oiW8b7rpprD+7LPP5tYuuuiicOytt94a1ovcfPPNYT1awvucc84Jx77yyithfe7cuWF9yZIlYf1Q5e7xfO0ZtvxAogg/kCjCDySK8AOJIvxAogg/kCjCDySq8Hp+M3tA0hRJW919ZHbfXEmzJB24KPpGd/9LrZpsBvv27cutFV13XjSv/l133RXWX3vttbD+4IMP5tZaW1vDsdG8+lLxPAcDBw4M6/369cutTZw4MRxbNC//1q1bw3okOm9DKl7H4VBQypb/IUndrRpxl7uPzn4d0sEHDkWF4Xf3JZLKXxIGQFOq5DP/VWb2lpk9YGZHVq0jAHVRbvjnSRomabSkTZJ+k/dAM5ttZsvMLJ6wDUBdlRV+d9/i7h3uvl/S7yWNDR7b7u5t7t5WbpMAqq+s8JtZ169Kfy5pVXXaAVAvpRzqe1zSRElHm9kGSf8paaKZjZbkktZL+mUNewRQA4Xhd/crurn7/hr00tSK5nmPzJo1K6xfc801Yb3oWPyYMWO+d08H9O7dO6w/8sgjYT26Xl+SJk3q7ihxp2+++SYcO2/evLD+zjvvhPVICsfxi3CGH5Aowg8kivADiSL8QKIIP5Aowg8kiqm7q6BPnz5h/aijjgrrRcton3766WF9586dubXJkyeHY++7776wXvRnu+6668J6NO14NOW4JF1++eVhvWhp9JdffjmsH6qYuhtAiPADiSL8QKIIP5Aowg8kivADiSL8QKI4zl8HLS0tYb2jo6Oi54+m596xY0c4dv/+/WF96tSpYX3BggVlP/+5554bjn311VfDOrrHcX4AIcIPJIrwA4ki/ECiCD+QKMIPJIrwA4kqnLobxfr27RvW9+zZU9Hzn3/++WE9mg/go48+CscWzTVw9dVXh/U77rgjrE+bNi23Nm7cuHBspcf5oz9b0fkP0ZLshwq2/ECiCD+QKMIPJIrwA4ki/ECiCD+QKMIPJKrwOL+ZtUpaIGmwJJfU7u6/M7NBkv4o6SRJ6yVd5u6f1a7V5lV0vX6lPvjgg7B+xBFH5NaKepszZ05Yf/HFF8N6UW/RugH9+vULx1bq008/za0VLU2eglK2/PskXevuZ0g6W9KvzOwMSddLWuTuwyUtyn4G0EMUht/dN7n78uz2LklrJA2VdLGk+dnD5ku6pFZNAqi+7/WZ38xOkjRG0muSBrv7pqy0WZ0fCwD0ECWf229mP5T0lKRfu/tOs39OE+bunjc/n5nNljS70kYBVFdJW34z663O4D/q7n/K7t5iZkOy+hBJW7sb6+7t7t7m7m3VaBhAdRSG3zo38fdLWuPuv+1SekbSjOz2DElPV789ALVSym7/OZL+TdJKM1uR3XejpNskLTSzmZL+Lumy2rTY/L744ouaPv/7778f1qdPn55b27VrVzh28OD4q5rouaX4kl0pvuT3uOOOC8eeffbZYb0Se/furdlz9xSF4Xf3lyXlzQP+k+q2A6BeOMMPSBThBxJF+IFEEX4gUYQfSBThBxLFEt110PVU6O5U+ncwdOjQ3Nq8efPCsV999VVYX7NmTVi/++67w3p0WW0z69+/f1j/8ssv69TJ98cS3QBChB9IFOEHEkX4gUQRfiBRhB9IFOEHEsUS3XVQdBz/yiuvDOtFx+qnTp2aW1u8eHE4duHChWH9zDPPDOvRtOGS9Nln+bO5n3zyyeHYiRMnhvWHH344rEdLdEfLmkvNfRy/WtjyA4ki/ECiCD+QKMIPJIrwA4ki/ECiCD+QKK7n7wFGjBgR1ocNG5ZbGzduXDj2ww8/rKhedL3+0qVLc2tF5whs3749rKN7XM8PIET4gUQRfiBRhB9IFOEHEkX4gUQRfiBRhdfzm1mrpAWSBktySe3u/jszmytplqRPsofe6O5/qVWjh7IzzjgjrL/33nthfcKECbm1Xr3iv+Kia+qL5gPYv39/WJ8yZUpu7YUXXgjHtrS0hPUjjzwyrG/bti2sp66UyTz2SbrW3Zeb2Y8kvWFmB/7W7nL3O2vXHoBaKQy/u2+StCm7vcvM1kjKXyIGQI/wvT7zm9lJksZIei276yoze8vMHjCzbvfBzGy2mS0zs2UVdQqgqkoOv5n9UNJTkn7t7jslzZM0TNJode4Z/Ka7ce7e7u5t7t5WhX4BVElJ4Tez3uoM/qPu/idJcvct7t7h7vsl/V7S2Nq1CaDaCsNvnUvM3i9pjbv/tsv9Q7o87OeSVlW/PQC1UnhJr5mNl/Q/klZKOnBc50ZJV6hzl98lrZf0y+zLwei5uKS3DDfccENYjy67ffTRR8OxgwYNCutFl9WecsopYX3dunVhvRJ9+vQJ619//XXNXruZlXpJbynf9r8sqbsn45g+0INxhh+QKMIPJIrwA4ki/ECiCD+QKMIPJIqpu4FDDFN3AwgRfiBRhB9IFOEHEkX4gUQRfiBRhB9IVCmz91bTNkl/7/Lz0dl9zahZe2vWviR6K1c1ezux1AfW9SSf77y42bJmnduvWXtr1r4keitXo3pjtx9IFOEHEtXo8Lc3+PUjzdpbs/Yl0Vu5GtJbQz/zA2icRm/5ATRIQ8JvZpPM7F0zW2dm1zeihzxmtt7MVprZikYvMZYtg7bVzFZ1uW+Qmb1gZmuz3+Olauvb21wz25i9dyvMbHKDems1s8VmttrM3jaz/8jub+h7F/TVkPet7rv9ZtYi6T1JP5W0QdLrkq5w99V1bSSHma2X1ObuDT8mbGYTJO2WtMDdR2b33S5pu7vflv3HeaS7X9ckvc2VtLvRKzdnC8oM6bqytKRLJP27GvjeBX1dpga8b43Y8o+VtM7d/+bu30j6g6SLG9BH03P3JZIOXjXjYknzs9vz1fmPp+5yemsK7r7J3Zdnt3dJOrCydEPfu6CvhmhE+IdK+qjLzxvUXEt+u6S/mtkbZja70c10Y3CXlZE2SxrcyGa6Ubhycz0dtLJ007x35ax4XW184fdd4939XyRdKOlX2e5tU/LOz2zNdLimpJWb66WblaX/oZHvXbkrXldbI8K/UVJrl5+Pz+5rCu6+Mft9q6Q/q/lWH95yYJHU7PetDe7nH5pp5ebuVpZWE7x3zbTidSPC/7qk4Wb2YzP7gaRfSHqmAX18h5kNyL6IkZkNkPQzNd/qw89ImpHdniHp6Qb28i3NsnJz3srSavB713QrXrt73X9JmqzOb/zfl3RTI3rI6etkSf+X/Xq70b1Jelydu4F71fndyExJR0laJGmtpP+WNKiJentYnas5v6XOoA1pUG/j1blL/5akFdmvyY1+74K+GvK+cYYfkCi+8AMSRfiBRBF+IFGEH0gU4QcSRfiBRBF+IFGEH0jU/wMyrEfZB4RVAAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "print('Counterfactual prediction: {}'.format(explanation['cf']['class']))\n",
    "print('Closest prototype class: {}'.format(cf.id_proto))\n",
    "plt.imshow(explanation['cf']['X'].reshape(28, 28))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Add trust scores"
   ]
  },
  {
   "attachments": {
    "b01g0t0nb7.png": {
     "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAADctJREFUeJzt3W+MVPX1x/HP0UJihAdgdbNaKkj815hI64qNJUpjrYhEaEKUVRs0tauxJiXpg6JNrPFRrS21j0i2Sro0FVrTGjApWiW/hF+jVhakKiLsYoAurkADsRJjqnj6YC7tFvd+Z5h/9w7n/Uo2O3PP3LknN/vZO3e+d+Zr7i4A8ZxWdAMAikH4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8E9bl2bszMuJwQaDF3t1oe19CR38zmmdlOMxs2s+WNPBeA9rJ6r+03s9Ml7ZJ0vaQRSZsl9br7W4l1OPIDLdaOI/9sScPu/o67/0vSWkkLG3g+AG3USPjPk/T3MfdHsmX/w8z6zGzQzAYb2BaAJmv5G37u3i+pX+JlP1AmjRz590uaNub+F7JlADpAI+HfLOlCM5thZhMlLZG0vjltAWi1ul/2u/snZna/pOclnS5plbtvb1pnAFqq7qG+ujbGOT/Qcm25yAdA5yL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+IKi2TtHdybq6unJrBw4caGMnQHNw5AeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoBqapdfM9kj6QNIxSZ+4e0+VxzNLbx0uueSSZP3tt99uUyfoBLXO0tuMi3y+7u7/aMLzAGgjXvYDQTUafpf0ZzPbYmZ9zWgIQHs0+rJ/jrvvN7NzJL1gZm+7+6axD8j+KfCPASiZho787r4/+31Q0jOSZo/zmH5376n2ZiCA9qo7/GZ2pplNPn5b0jclvdmsxgC0ViMv+7skPWNmx5/nKXd/rildAWi5usPv7u9IuryJvZyy7r777mT9iSeeSNZnzpyZrDPOj3ow1AcERfiBoAg/EBThB4Ii/EBQhB8Iiq/urtHixYtza9m1DrnmzJmTrB85ciRZnz9/frK+c+fOZD1leHi47nXR2TjyA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQjPPX6LHHHsutvfjii8l1q309+sSJE5P1WbNmJetbt27NrW3fvj257qlsZGQkt/boo48m1x0cHGx2O6XDkR8IivADQRF+ICjCDwRF+IGgCD8QFOEHgmKcv0b33Xdfbq3aFNo7duxI1i+99NJk/aWXXkrWzznnnNzapEmTkuuuXbs2WV+yZEmyfv755yfrr776am7t2muvTa576NChZH3v3r3J+jXXXJNb27dvX3JdxvkBnLIIPxAU4QeCIvxAUIQfCIrwA0ERfiAoq/ZZczNbJWmBpIPuflm2bKqk30maLmmPpFvcPf3l85X10htDXaZMmZJbq/ZdAK+99lqyfsUVVyTr9957b7Kesnr16mT9lVdeSdavu+66ZH369Om5tffffz+57sqVK5P1MnP39EQSmVqO/L+WNO+EZcslbXT3CyVtzO4D6CBVw+/umyQdPmHxQkkD2e0BSYua3BeAFqv3nL/L3Uez2+9J6mpSPwDapOFr+93dU+fyZtYnqa/R7QBornqP/AfMrFuSst8H8x7o7v3u3uPuPXVuC0AL1Bv+9ZKWZreXSlrXnHYAtEvV8JvZGkkvS7rYzEbM7DuSfiLpejMbkvSN7D6ADlJ1nL+pG2OcHydh0aL0INKCBQuS9csvvzy3dsMNNyTXPXz4xAGuztHMcX4ApyDCDwRF+IGgCD8QFOEHgiL8QFB8dTcaMnfu3GT9nnvuya2ddlr62HPxxRcn69Wm2d6wYUNurZOH8pqFIz8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBMVHelGYRx55JFm//fbbk/WpU6cm68uWLcutDQwM5NY6HR/pBZBE+IGgCD8QFOEHgiL8QFCEHwiK8ANB8Xl+JKWm/5akCy64IFlfvjx/AueFCxcm13333XeT9Wrrb9q0KVmPjiM/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRVdZzfzFZJWiDpoLtfli17WNJ3JR3KHvagu/+pVU2iOEeOHEnWt2zZkqwPDQ3l1iZMmJBcd3h4OFl/+eWXk3Wk1XLk/7WkeeMs/4W7z8p+CD7QYaqG3903SWJ6E+AU08g5//1m9rqZrTKz9DWgAEqn3vCvlDRT0ixJo5J+nvdAM+szs0EzG6xzWwBaoK7wu/sBdz/m7p9K+pWk2YnH9rt7j7v31NskgOarK/xm1j3m7rckvdmcdgC0Sy1DfWskzZX0eTMbkfRjSXPNbJYkl7RHUv48zABKqWr43b13nMVPtqAXtMDkyZOT9Q8//DBZP3bsWLJ+xhlnJOtXX311bq23d7w/rf/at29fsv7xxx8n60jjCj8gKMIPBEX4gaAIPxAU4QeCIvxAUEzRjYY89NBDyXpqGu3u7u7cmiTdeuutdfUUHVN0A0gi/EBQhB8IivADQRF+ICjCDwRF+IGgmKIbSTfddFOynvrIriRdddVVubUbb7yxrp7QHBz5gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAoPs8f3FlnnZWsb968OVmfMWNGsr5u3brc2qJFi5Lroj58nh9AEuEHgiL8QFCEHwiK8ANBEX4gKMIPBFV1nN/MpklaLalLkkvqd/dfmtlUSb+TNF3SHkm3uPuRKs/FOH8d1qxZU/e6d9xxR7K+YcOGZH1oaChZP/vss5P1Bx54ILe2e/fu5LqoTzPH+T+R9AN3/5Kkr0r6npl9SdJySRvd/UJJG7P7ADpE1fC7+6i7b81ufyBph6TzJC2UNJA9bEASl2sBHeSkzvnNbLqkL0v6q6Qudx/NSu+pcloAoEPU/B1+ZjZJ0h8kLXP3f5r997TC3T3vfN7M+iT1NdoogOaq6chvZhNUCf5v3f2P2eIDZtad1bslHRxvXXfvd/ced+9pRsMAmqNq+K1yiH9S0g53XzGmtF7S0uz2Ukn5H98CUDq1DPXNkfT/kt6Q9Gm2+EFVzvt/L+mLkvaqMtR3uMpzMdTXZhdddFGyvnPnzoae/+abb07Wn3322YaeHyev1qG+quf87v4XSXlPdt3JNAWgPLjCDwiK8ANBEX4gKMIPBEX4gaAIPxAUX93dAbq7u5P1xx9/PLf23HPPJdf96KOPkvVzzz03WV+xYkWy3s6/L1Tw1d0Akgg/EBThB4Ii/EBQhB8IivADQRF+IKiav8YLxRkdHU3Wt23blltbtWpVQ9u+8sork3XG8TsXR34gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIpx/g7w9NNPJ+vz5s3Lrd12220Nbfuuu+5K1gcHBxt6fhSHIz8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBFV1nN/MpklaLalLkkvqd/dfmtnDkr4r6VD20Afd/U+tajSyXbt2JeuLFy/OrT311FPJdXfv3p2sP//888k6OlctF/l8IukH7r7VzCZL2mJmL2S1X7j7z1rXHoBWqRp+dx+VNJrd/sDMdkg6r9WNAWitkzrnN7Ppkr4s6a/ZovvN7HUzW2VmU3LW6TOzQTPjOlCgRGoOv5lNkvQHScvc/Z+SVkqaKWmWKq8Mfj7eeu7e7+497t7ThH4BNElN4TezCaoE/7fu/kdJcvcD7n7M3T+V9CtJs1vXJoBmqxp+MzNJT0ra4e4rxiwfO3XstyS92fz2ALRKLe/2f03StyW9YWbHvyP6QUm9ZjZLleG/PZLuaUmH0NGjR5P11Md277zzzuS6vb29yfrhw4eTdXSuWt7t/4uk8eb7Zkwf6GBc4QcERfiBoAg/EBThB4Ii/EBQhB8Iyto5xbKZMZ8z0GLuPt7Q/Gdw5AeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoNo9Rfc/JO0dc//z2bIyKmtvZe1Lord6NbO382t9YFsv8vnMxs0Gy/rdfmXtrax9SfRWr6J642U/EBThB4IqOvz9BW8/pay9lbUvid7qVUhvhZ7zAyhO0Ud+AAUpJPxmNs/MdprZsJktL6KHPGa2x8zeMLNtRU8xlk2DdtDM3hyzbKqZvWBmQ9nvcadJK6i3h81sf7bvtpnZ/IJ6m2Zm/2dmb5nZdjP7fra80H2X6KuQ/db2l/1mdrqkXZKulzQiabOkXnd/q62N5DCzPZJ63L3wMWEzu0bSUUmr3f2ybNlPJR12959k/zinuPsPS9Lbw5KOFj1zczahTPfYmaUlLZJ0pwrcd4m+blEB+62II/9sScPu/o67/0vSWkkLC+ij9Nx9k6QTZ81YKGkguz2gyh9P2+X0VgruPuruW7PbH0g6PrN0ofsu0Vchigj/eZL+Pub+iMo15bdL+rOZbTGzvqKbGUdXNm26JL0nqavIZsZRdebmdjphZunS7Lt6ZrxuNt7w+6w57v4VSTdK+l728raUvHLOVqbhmppmbm6XcWaW/o8i9129M143WxHh3y9p2pj7X8iWlYK7789+H5T0jMo3+/CB45OkZr8PFtzPf5Rp5ubxZpZWCfZdmWa8LiL8myVdaGYzzGyipCWS1hfQx2eY2ZnZGzEyszMlfVPlm314vaSl2e2lktYV2Mv/KMvMzXkzS6vgfVe6Ga/dve0/kuar8o7/bkk/KqKHnL4ukPS37Gd70b1JWqPKy8CPVXlv5DuSzpK0UdKQpBclTS1Rb7+R9Iak11UJWndBvc1R5SX965K2ZT/zi953ib4K2W9c4QcExRt+QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeC+jchAHb0cVndSAAAAABJRU5ErkJggg=="
    },
    "mnist7g100t100.png": {
     "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAADyRJREFUeJzt3W2sVdWdx/HfnydFIAIFrggotZEm+ICdIJFAho4P1VEUGxOtxoRJmsKLmky1LzRO4kCMsZn0wb6BhEYsTqqtWqq8MDM4qBGMNFwNAwiCiBfhcuHaIPaqKAL/eXE3nVu8e+3jedrn8v9+EsI5+3/W2Ss7/Nj7nLXPWubuAhDPoLI7AKAchB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFBDmrkzM+N2QqDB3N0qeV1NZ34zu8HMdprZbjN7oJb3AtBcVu29/WY2WNIuSddJ2i9pk6Q73X17og1nfqDBmnHmnyVpt7vvcfdjkn4vaUEN7wegiWoJ/yRJ+/o8359t+ztmtsjM2s2svYZ9Aaizhn/h5+4rJK2QuOwHWkktZ/5OSVP6PJ+cbQMwANQS/k2SLjazb5rZMEk/kLSmPt0C0GhVX/a7+3Ezu0fSf0saLGmlu79dt54BaKiqh/qq2hmf+YGGa8pNPgAGLsIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCqnqJbkkysw5JPZJOSDru7jPr0SkAjVdT+DP/5O5/qcP7AGgiLvuBoGoNv0taa2ZvmtmienQIQHPUetk/1907zWyCpJfM7B13f63vC7L/FPiPAWgx5u71eSOzJZI+cfefJ15Tn50ByOXuVsnrqr7sN7MRZjbq1GNJ35O0rdr3A9BctVz2t0n6k5mdep+n3P2/6tIrAA1Xt8v+inbGZX/LGTp0aLL+5ZdfJusTJ05M1gcNyr+47OrqSrY9efJkst5IQ4akz4vHjx9vUk++voZf9gMY2Ag/EBThB4Ii/EBQhB8IivADQTHUV6HUkNjYsWOTbceMGZOs7969O1mfPHlysp4alpoxY0ay7YQJE5L1W265JVkvGo7r7OzMrS1btizZdvPmzck6+sdQH4Akwg8ERfiBoAg/EBThB4Ii/EBQhB8IinH+Cl1yySW5taKx8ldeeSVZv+qqq5L1m266KVm/++67c2sHDx5Mth0+fHiyfvTo0WS9SOqnr0U/my3y2WefJevPPfdcbm358uXJtrX2rcyf/DLODyCJ8ANBEX4gKMIPBEX4gaAIPxAU4QeCqscqvSGMHj06t1Y0jj9+/Phkffbs2cn69OnTk/XUmHTRXANFv8fv6OhI1qdOnZqsX3DBBbm1s846K9n2448/TtZff/31ZP3mm2/Ore3YsSPZdsOGDcl6K0/dXSnO/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QVOHv+c1spaT5krrd/dJs21hJf5A0VVKHpNvd/aPCnQ3g3/PXouj3+kWK5v3ft29fbq1oHH/79u3J+jXXXJOsL1iwIFl/4403cmsffPBBsu0XX3yRrD/66KPJemosfunSpcm2GzduTNZbWT1/z/9bSTectu0BSevc/WJJ67LnAAaQwvC7+2uSDp+2eYGkVdnjVZJurXO/ADRYtZ/529y9K3t8UFJbnfoDoElqvrff3T31Wd7MFklaVOt+ANRXtWf+Q2Y2UZKyv7vzXujuK9x9prvPrHJfABqg2vCvkbQwe7xQ0gv16Q6AZikMv5k9LekNSd82s/1m9kNJP5N0nZm9K+na7DmAAYR5+1tA0dz5559/frK+f//+3FrRWHkru+2225L1hx9+OFl/6qmncmvLli1Ltj18+PQBroGDefsBJBF+ICjCDwRF+IGgCD8QFOEHgmLq7iYomrp7/vz5yfq5556brD/22GO5tdSU45J05MiRZL1ouG3WrFnJ+tq1a3Nrc+bMSba9+uqrk/Wi45qa2nsgD+XVC2d+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiKcf4KpabP/uij9KzlEyZMSNYHDx6crM+bNy9ZT02PXbRE965du5L1UaNGJeuXXXZZsn7XXXdVve+iexSeffbZZP3DDz/MrRUtDz6QfwpdKc78QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4/wVOnbsWG6t6Hfpqd+VS9LOnTuT9Z6enmT9wgsvzK0VLbE9cuTIZP3TTz9N1levXp2sp+4zmDZtWrLtoEHpc9MzzzyTrHd35y4kpbPPPjvZlnF+AGcswg8ERfiBoAg/EBThB4Ii/EBQhB8IqnCc38xWSpovqdvdL822LZH0I0mnfjD9oLu/2KhOtoLUeHfROH7REtxHjx5N1vfu3ZusX3vttbm1++67L9l2xowZyfr777+frHd1dVX9/kXrEXR2dibrr776arKeUjSHQgSVnPl/K+mGfrb/yt2vyP6c0cEHzkSF4Xf31ySxvAlwhqnlM/89ZrbFzFaaWf4cVwBaUrXhXy7pW5KukNQl6Rd5LzSzRWbWbmbtVe4LQANUFX53P+TuJ9z9pKTfSMpdrdHdV7j7THefWW0nAdRfVeE3s4l9nn5f0rb6dAdAs1Qy1Pe0pO9KGmdm+yX9u6TvmtkVklxSh6TFDewjgAYoDL+739nP5scb0JeWNmzYsNzauHHjkm0PHDhQ076L7gPYtGlTbq2trS3Zds+ePcl60dz57p6sp+Y6uOiii5Jtly5dmqzX4sSJEw1774GCO/yAoAg/EBThB4Ii/EBQhB8IivADQVnRUE1dd2bWvJ2hKR566KFkPTVc196evuP7yiuvrKpP0bm7VfI6zvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBTj/C3gnHPOSdaLftJ73nnn5daKptYuMn/+/GT9+eefT9ZTU55ff/31ybYbN25M1tE/xvkBJBF+ICjCDwRF+IGgCD8QFOEHgiL8QFCFU3ej8T7//PNkvehejFrG8qdNm5asP/LII8l60VLXL76Yv4Bz0bThaCzO/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QVOE4v5lNkfSkpDZJLmmFu//azMZK+oOkqZI6JN3u7h81rqtnrpMnTzbsvadPn56s33vvvcn65Zdfnqy/9957yXrqPoHu7u5kWzRWJWf+45J+6u7TJV0l6cdmNl3SA5LWufvFktZlzwEMEIXhd/cud38re9wjaYekSZIWSFqVvWyVpFsb1UkA9fe1PvOb2VRJ35H0Z0lt7n7qvtKD6v1YAGCAqPjefjMbKemPkn7i7n81+/9pwtzd8+bnM7NFkhbV2lEA9VXRmd/Mhqo3+L9z99XZ5kNmNjGrT5TU77c37r7C3We6+8x6dBhAfRSG33pP8Y9L2uHuv+xTWiNpYfZ4oaQX6t89AI1SOHW3mc2VtF7SVkmnxqQeVO/n/mckXSBpr3qH+g4XvBdTdzfAuHHjcmtjx45Ntt25c2dN+549e3ayfuDAgdxaT09Psu2RI0eS9WZOOz+QVDp1d+FnfnffICnvza75Op0C0Dq4ww8IivADQRF+ICjCDwRF+IGgCD8QFFN3nwFGjBiRW7vjjjuSbQ8ePJisr169Ollvb29P1o8fP56sozyc+YGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMb5zwCLFy/OrY0cOTLZdsuWLcn6E088kawXjeMPHTo0tzZp0qRk246OjmQdteHMDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBMc5/Bhg9enRubdOmTcm2W7durWnfw4cPT9aPHj2aW2Mcv1yc+YGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gqMJxfjObIulJSW2SXNIKd/+1mS2R9CNJH2YvfdDdX2xURyObM2dOsv7OO+/k1u6///5k223btiXr69evT9ZT9xhI6XF+lKuSm3yOS/qpu79lZqMkvWlmL2W1X7n7zxvXPQCNUhh+d++S1JU97jGzHZLSU7AAaHlf6zO/mU2V9B1Jf8423WNmW8xspZmNyWmzyMzazSy9rhOApqo4/GY2UtIfJf3E3f8qabmkb0m6Qr1XBr/or527r3D3me4+sw79BVAnFYXfzIaqN/i/c/fVkuTuh9z9hLuflPQbSbMa100A9VYYfjMzSY9L2uHuv+yzfWKfl31fUvprYwAtxdw9/QKzuZLWS9oq6WS2+UFJd6r3kt8ldUhanH05mHqv9M7Qr3nz5lXddvz48cn6yy+/nKwfPny46n2jHO5ulbyukm/7N0jq780Y0wcGMO7wA4Ii/EBQhB8IivADQRF+ICjCDwRVOM5f150xzt8QQ4bkj9gWLaGNM0+l4/yc+YGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gqGYv0f0XSXv7PB+XbWtFrdq3r/SrhcbyW/WYSXH6dmGlL2zqTT5f2blZe6vO7deqfWvVfkn0rVpl9Y3LfiAowg8EVXb4V5S8/5RW7Vur9kuib9UqpW+lfuYHUJ6yz/wASlJK+M3sBjPbaWa7zeyBMvqQx8w6zGyrmW0ue4mxbBm0bjPb1mfbWDN7yczezf7ud5m0kvq2xMw6s2O32cxuLKlvU8zsFTPbbmZvm9m/ZttLPXaJfpVy3Jp+2W9mgyXtknSdpP2SNkm60923N7UjOcysQ9JMdy99TNjM/lHSJ5KedPdLs23/Iemwu/8s+49zjLun1+FuXt+WSPqk7JWbswVlJvZdWVrSrZL+RSUeu0S/blcJx62MM/8sSbvdfY+7H5P0e0kLSuhHy3P31ySdvmrGAkmrsser1PuPp+ly+tYS3L3L3d/KHvdIOrWydKnHLtGvUpQR/kmS9vV5vl+tteS3S1prZm+a2aKyO9OPtj4rIx2U1FZmZ/pRuHJzM522snTLHLtqVryuN77w+6q57v4Pkv5Z0o+zy9uW5L2f2VppuKailZubpZ+Vpf+mzGNX7YrX9VZG+DslTenzfHK2rSW4e2f2d7ekP6n1Vh8+dGqR1Ozv7pL78zettHJzfytLqwWOXSuteF1G+DdJutjMvmlmwyT9QNKaEvrxFWY2IvsiRmY2QtL31HqrD6+RtDB7vFDSCyX25e+0ysrNeStLq+Rj13IrXrt70/9IulG93/i/J+nfyuhDTr8ukvS/2Z+3y+6bpKfVexn4pXq/G/mhpG9IWifpXUn/I2lsC/XtP9W7mvMW9QZtYkl9m6veS/otkjZnf24s+9gl+lXKceMOPyAovvADgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxDU/wFrVPMwp+XDjQAAAABJRU5ErkJggg=="
    },
    "proto3.png": {
     "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAEsNJREFUeJzt3W9slfd1B/DvuYS/poAdYscBAqyBhAgxd7HQJJKlU9eKkkbQF0mKooVlUemLRlqVvliUvVikaVI0ra3yYqpEFwRMWdpJJIIo1daMTMsiRRUEZSGUZDACMghsCAk2f43tsxd+Mjng55zr+7vPfa4534+EsO+5v3t/fnyPn/vc8/sjqgoiiqdSdgeIqBxMfqKgmPxEQTH5iYJi8hMFxeQnCorJTxQUk58oKCY/UVC3NPLJKpWKVir5f2+Gh4drfmwRqbktAHCkYzFSfy9l8V4PRb7evMeeMmVKbmx4eBgjIyNVdS4p+UVkLYAXAUwB8I+q+oJ1/0qlgnnz5uXGP//885r7Yh0QwP9lpiR/0X84vBdDygupaGU/f62KTv5r167lxm65xU7LuXPn5sYmkkM1v+0XkSkA/gHAtwHcC2CjiNxb6+MRUWOlXPOvBnBEVY+q6iCAXwJYX59uEVHRUpJ/AYCeMd+fyG77EhHZLCL7RGQfr6uJmkfhn/ar6hZV7VbV7sl6/Ud0M0pJ/pMAFo35fmF2GxFNAinJvxfAMhFZKiLTAHwPwO76dIuIilZzqU9Vh0TkaQD/htFS31ZVPWi1GRkZwcDAQG7cuyywxgikfp7gPbcVTy0LFd13mrjUY+q1t8p51uscAM6ePVtTn64njfwQrlKp6NSpU3PjIyMjXvvcWJm19rKTn5qP9zu3XuvemJWrV6+acVWt6i8Xh/cSBcXkJwqKyU8UFJOfKCgmP1FQTH6ioBo6nx8obupsmeWy1Of26roeqzSUMnaiGin1cK+06x1Xb/0H7/FT2qaWb6340NBQzc89kdciz/xEQTH5iYJi8hMFxeQnCorJTxQUk58oqIaW+lTVLEWklp2KZJVXvH6nTO8E0kpWXt+KPubWz+6V6ryyVZmvlyLLu95jW6Vdr0z4pT5UfU8iuqkw+YmCYvITBcXkJwqKyU8UFJOfKCgmP1FQDZ/SaylyJ90il2L2Vlv1TJ8+vebnBuxxAl7d19otthre41t9s1ZyBtLHT1jjIwYHB822qa/FybAiM8/8REEx+YmCYvITBcXkJwqKyU8UFJOfKCgmP1FQSXV+ETkGYADAMIAhVe127p9Ub09p69WMvbqsVZP26tVev+fOnWvGly9fbsYfeuih3FhXV5fZtrOz04x7YxjOnz9vxg8ezN+1/fXXXzfbvvPOO2bc+51dvnw5N5Y6hiD19WRJWb9hIuoxyOePVbU+G4YTUcPwbT9RUKnJrwB+IyLvicjmenSIiBoj9W3//ap6UkTaAbwpIh+p6ttj75D9UeAfBqImk3TmV9WT2f99AF4DsHqc+2xR1W5V7U6dXENE9VNz8otIi4h85YuvAXwLwIf16hgRFSvlbX8HgNeys/ktAP5ZVf+1Lr0iosLVnPyqehTA70+wjVn/LHIMQOra+Va9e9asWWbblpYWM/7YY4+Z8bVr15rx9vb23Fhra6vZdubMmWZ82rRpZnzRokVm/Pbbb8+NLVu2zGy7dOlSM75z504zbtXiBwYGzLZenT51HYSi1qaYSA6x1EcUFJOfKCgmP1FQTH6ioJj8REEx+YmCaujS3alTeouUspW1N6V3wYIFZtybVustM33mzJncWH9/v9nWmz46Y8YMM+5NR7ZYJUoA6O42Z4jjwIEDZvzIkSO5sYsXL5ptU0vHRbLKhBMpIfLMTxQUk58oKCY/UVBMfqKgmPxEQTH5iYJi8hMF1VRbdKdIrcumbLlsLRENAD09PWZ8x44dZvzKlStm3Jpe6k1V9n5ub9nwBx980Izfc889ubHZs2ebbYeHh814W1ubGU95bO+4eOMjihwHwCm9RJSEyU8UFJOfKCgmP1FQTH6ioJj8REEx+YmCamidv8iluz1eXdabzz80NJQb82rGly5dMuO9vb1m3Nsm21pe26vze0tzW3PiAWDVqlVm3Jo3762DcOHCBTPujZ+wxkeUufR2s+CZnygoJj9RUEx+oqCY/ERBMfmJgmLyEwXF5CcKyq3zi8hWAN8B0KeqK7Pb2gD8CsASAMcAPKqqnxXXTV9q3dWr1afMoU7ZE6Cax7fGAXjHZfr06Wbcm6/f1dVlxjs6OnJjXt/Onz9vxlO22fbGTnh7JXh9T10/ohGqOfNvA3D9BvHPAtijqssA7Mm+J6JJxE1+VX0bwLnrbl4PYHv29XYAG+rcLyIqWK3X/B2qeir7+jSA/Pd2RNSUksf2q6qKSO4FjIhsBrA59XmIqL5qPfP3ikgnAGT/9+XdUVW3qGq3qtq7LhJRQ9Wa/LsBbMq+3gRgV326Q0SN4ia/iLwC4F0Ad4vICRF5CsALAL4pIocB/En2PRFNIu41v6puzAl9o859KVTKuvye1D0DvJqzNw7AmrO/YsUKs+2TTz5pxtesWWPGW1tbzbg1n//jjz822x49etSMnz171oxbtfoiXw/1aN8IHOFHFBSTnygoJj9RUEx+oqCY/ERBMfmJgrpptuguujSTsuS499heqc9b4vqJJ57IjT3yyCNm2yVLlpjxuXPnmnHvZ7OWyPaWU+/v7zfjHu/xLanl28mAZ36ioJj8REEx+YmCYvITBcXkJwqKyU8UFJOfKKibps6fKqUW79V8vTp9S0uLGd+wwV4f9ZlnnsmNeVNura3HAf+4eLV0q87vbR/e1tZmxj3W46eOC0mt8zfDlF+e+YmCYvITBcXkJwqKyU8UFJOfKCgmP1FQTH6ioCZVnT9lTr2nyPnZXi3ce+4rV66Y8SNHjuTGFi9ebLb16s2p4wCuXr2aG5szZ47ZduXKlWb83XffNeMnT57MjXljDLzfiTV+AUir46du/10tnvmJgmLyEwXF5CcKislPFBSTnygoJj9RUEx+oqCkipriVgDfAdCnqiuz254H8H0AZ7K7Paeqv/aerFKpqFdfLYu3DbZVW/XW3ffqsrNmzTLjt912mxmfN29ebqy9vd1sOzw8bMa9vnd2dprxVatW5cbuu+8+s+3ly5fNuFfn37ZtW27s3LlzZttLly6ZcW/shndcLSljBIaGhjAyMlLVQIBqzvzbAKwd5/afqWpX9s9NfCJqLm7yq+rbAOw/k0Q06aRc8z8tIh+IyFYRsdeKIqKmU2vy/xzAVwF0ATgF4Cd5dxSRzSKyT0T2NcO6ZUQ0qqbkV9VeVR1W1REAvwCw2rjvFlXtVtXum2FzQ6KbRU3JLyJjP+L9LoAP69MdImoUt+4mIq8A+DqA+SJyAsBfA/i6iHQBUADHAPygwD4SUQHc5FfVjePc/FItT6aqhe1zX+YlRepnGd58/Z6eHjP+6aef5saOHz9utvX6Pn36dDP+0UcfmfFDhw7lxubPn2+2XbhwoRnv6Ogw47Nnz86NeXV+b+xG6pz7Zvj8iyP8iIJi8hMFxeQnCorJTxQUk58oKCY/UVANnV8rIkklOaut97hePKW0k1oWSt0Gu7+/34yn8LYX90qBFq/ctmDBAjNuTWUGgDvvvDM31tfXZ7b1lixPmbLbLHjmJwqKyU8UFJOfKCgmP1FQTH6ioJj8REEx+YmCamidP3VKb0qd31ua24tbtXyvFp4qdZxACu9nmzZtmhlvbc1f3tFbxt0b3+DV4gcHB3NjqWMrypySaz33RPrFMz9RUEx+oqCY/ERBMfmJgmLyEwXF5CcKislPFNSkms+fwqvjezVnq33qfH6vbyn17tTxDx7vZ1+zZk1u7O677zbbzpw504yfP3/ejPf29ubGUufjp9b5uXQ3EZWGyU8UFJOfKCgmP1FQTH6ioJj8REEx+YmCcuv8IrIIwA4AHQAUwBZVfVFE2gD8CsASAMcAPKqqn6V0JmU+v1dv9njz1q1xAN6c9hkzZpjxgYEBM57Cqyd79e5bb73VjD/++ONm/IEHHsiNeVtsnz592owfPnzYjFvH1fu5vbEVqeMErNdyo8YAVHPmHwLwY1W9F8AfAvihiNwL4FkAe1R1GYA92fdENEm4ya+qp1R1f/b1AIBDABYAWA9ge3a37QA2FNVJIqq/CV3zi8gSAF8D8FsAHap6KgudxuhlARFNElWP7ReR2QB2AviRqvaPvWZRVRWRcS9URGQzgM2pHSWi+qrqzC8iUzGa+C+r6qvZzb0i0pnFOwGMu/Ohqm5R1W5V7S5rUg8R3chNfhnN2JcAHFLVn44J7QawKft6E4Bd9e8eERWlmrf9awD8KYADIvJ+dttzAF4A8C8i8hSA4wAe9R4odelu77Et3tRVr721FfWKFSvMtuvWrTPjXpnSKyVaJS2vVLd06VIzftddd5nx+fPnm/ErV67kxnp6esy2+/fvN+NvvfWWGf/ss/zK86VLl8y2XinPi6dMlW5Uqc9NflV9B0BeVn6jvt0hokbhCD+ioJj8REEx+YmCYvITBcXkJwqKyU8U1E2zdHfq46ZM8fSmAy9fvtyML1682Ix7tXqrLuwtf+31PXVq6yeffJIbe+ONN8y2L7/8shnv7+8349euXcuNedt7ez93qpQpvSlb1Y/FMz9RUEx+oqCY/ERBMfmJgmLyEwXF5CcKislPFJQ0cqtgEVFr7nrK8tte29Stqq34HXfcYbb1tqJ++OGHzXh3d7cZb29vz41Z6xAAfq3cWx577969ZnzXrvw1Xrz5/N6ce69Wb7HWGQDSt0332he1dPfQ0BBGRkaqKvbzzE8UFJOfKCgmP1FQTH6ioJj8REEx+YmCYvITBdXQOn+lUlFrq2uvL1Yt36vjF1mX9dbVnzNnjhn3avFFunr1qhkvct67txaAV4v34tbvzPu5Ul4P1ShqXQvW+YnIxeQnCorJTxQUk58oKCY/UVBMfqKgmPxEQbl1fhFZBGAHgA4ACmCLqr4oIs8D+D6AM9ldn1PVXzuPZdb5vdpnynrlKWMIqmmf0rbImnHqYxfZ99QxJta6/IBdq/fGJ5RZx/eOi/VzDQ4OVl3nr2bTjiEAP1bV/SLyFQDvicibWexnqvr31TwRETUXN/lV9RSAU9nXAyJyCMCCojtGRMWa0DW/iCwB8DUAv81uelpEPhCRrSLSmtNms4jsE5F9ST0lorqqemy/iMwG8J8A/lZVXxWRDgBnMfo5wN8A6FTVP3ceg9f84+A1f214zX+jiVzzV3XmF5GpAHYCeFlVX8062Kuqw6o6AuAXAFZX81hE1Bzc5JfRP2EvATikqj8dc3vnmLt9F8CH9e8eERWlmlLf/QD+C8ABAF+8V3oOwEYAXRh9238MwA+yDwdzVSoVtaa/przd8aZopjy2p+xLjqKmh1bz3Clv3VOnWae8dU+95Ei5RPV4fbOmQo+MjEBVq3ryhs/nZ/LfiMk/Pib/+OqV/BzhRxQUk58oKCY/UVBMfqKgmPxEQTH5iYKqZlZf3YiIuUz1xYsXzfYpy0SnbP8NpJeGLN4S1ilSS1JFlhG9Ul5qibORZeyJPnfKcW1pacmNeTk0Fs/8REEx+YmCYvITBcXkJwqKyU8UFJOfKCgmP1FQDZ3SKyJnABwfc9N8jC4F1oyatW/N2i+AfatVPfu2WFVvq+aODU3+G55cZJ+qdpfWAUOz9q1Z+wWwb7Uqq298208UFJOfKKiyk39Lyc9vada+NWu/APatVqX0rdRrfiIqT9lnfiIqSSnJLyJrReRjETkiIs+W0Yc8InJMRA6IyPtlbzGWbYPWJyIfjrmtTUTeFJHD2f/jbpNWUt+eF5GT2bF7X0TWldS3RSLyHyLyOxE5KCJ/kd1e6rEz+lXKcWv4234RmQLgfwB8E8AJAHsBbFTV3zW0IzlE5BiAblUtvSYsIn8E4AKAHaq6Mrvt7wCcU9UXsj+crar6l03St+cBXCh75+ZsQ5nOsTtLA9gA4M9Q4rEz+vUoSjhuZZz5VwM4oqpHVXUQwC8BrC+hH01PVd8GcO66m9cD2J59vR2jL56Gy+lbU1DVU6q6P/t6AMAXO0uXeuyMfpWijORfAKBnzPcn0FxbfiuA34jIeyKyuezOjKNjzM5IpwF0lNmZcbg7NzfSdTtLN82xq2XH63rjB343ul9V/wDAtwH8MHt725R09Jqtmco1PwfwVYxu43YKwE/K7Ey2s/ROAD9S1f6xsTKP3Tj9KuW4lZH8JwEsGvP9wuy2pqCqJ7P/+wC8hubbfbj3i01Ss//7Su7P/2umnZvH21kaTXDsmmnH6zKSfy+AZSKyVESmAfgegN0l9OMGItKSfRADEWkB8C003+7DuwFsyr7eBGBXiX35kmbZuTlvZ2mUfOyabsdrVW34PwDrMPqJ//8C+Ksy+pDTr98D8N/Zv4Nl9w3AKxh9G3gNo5+NPAXgVgB7ABwG8O8A2pqob/+E0d2cP8BoonWW1Lf7MfqW/gMA72f/1pV97Ix+lXLcOMKPKCh+4EcUFJOfKCgmP1FQTH6ioJj8REEx+YmCYvITBcXkJwrq/wDSJVEUfS1SsAAAAABJRU5ErkJggg=="
    },
    "proto9.png": {
     "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAEd1JREFUeJzt3W9sXOWVBvDn2LEt5S9/QqwoDSFUBoEQIpGJIi1CXXVbkShSqJBQ8mHJClQXUaQt9MMi+qFIKyS02rbiQ1UphdCw6tKuFCLyoVrKhtVCxRISHCAQCqQlURKc2EkIsTHEdub0g28qN/ieM5537twZn+cnRbHn+J15PZ7HM+Nz3/uKqoKI4mkrewJEVA6Gnygohp8oKIafKCiGnygohp8oKIafKCiGnygohp8oqDmNvLG2tjZta8v/fXPhwoWar1tEah4LADzSkerJezxajzdvrJWhSqWCSqVSVRiSwi8idwB4EkA7gKdU9Qnr69va2rBo0aLc+meffVbzXNrb2826F+6U8LfyL47UX5qzlfcz9e43rz4+Pp5bmzPHjuXChQtza2fPnjXHTlXzy34RaQfwcwDrANwIYLOI3Fjr9RFRY6W8518D4JCq/llVxwD8BsDG+kyLiIqWEv5lAI5O+fxYdtnfEJE+EdknIvta+eUx0WxT+F/7VXWrqvaqai/fXxI1j5TwHwewfMrnX8suI6IWkBL+vQB6RGSliHQC2ARgV32mRURFq7nVp6oTIvIggBcx2erbpqrvWWMqlQpGRkZy6yn9zdRWXspbEr6dmX1Sf6beeKudZz3OAeD06dM1zelS0sg/wrW1tWlHR0duvVKpeONza2WGn+hS3uPJeqx7x6ycP3/erKtqVQ9mHt5LFBTDTxQUw08UFMNPFBTDTxQUw08UVEPX8wPFLZ1t5VZe6txT1obPZq26lmRiYsKsWz/TmXzPfOYnCorhJwqK4ScKiuEnCorhJwqK4ScKqqGtPlU1WxHeUsbU27aU2RIrevlokdddZBvSW+XpKfJ+SW0jpszNWvXntQmn4jM/UVAMP1FQDD9RUAw/UVAMP1FQDD9RUAw/UVANX9JrKXIn3dTbThlbdB/fO9trynUX2StPPa5jJj3tS3nbwac+1lphOTGf+YmCYviJgmL4iYJi+ImCYviJgmL4iYJi+ImCSurzi8hhAMMALgCYUNXeekzKuL2axxa5/rroPr7XD7fGW1tBA36/2zuGwFtzn7Ibrfcz8+4X63vzbtu7X4rUqGME6nGQz9+r6qk6XA8RNRBf9hMFlRp+BfB7EXlTRPrqMSEiaozUl/23qepxEVkC4CUR+aOqvjL1C7JfCvzFQNRkkp75VfV49v8ggJ0A1kzzNVtVtbfoPwYS0czUHH4RmSciCy5+DODbAN6t18SIqFgpL/u7AezM2kxzAPynqv53XWZFRIWTRq47FhG1+s4p54AvWpHr2ru6upJu2+pZL1iwwBzb2dlp1levXm3WlyxZYtbPnDmTW9u/f785dmhoyKynnPd/fHzcHDs2NlbzdadKOT/ExMQEKpVKVQ9WtvqIgmL4iYJi+ImCYviJgmL4iYJi+ImCaqpTdxepyGW3qafW7ujoMOve8tJVq1bl1h5++GFz7Nq1a836ZZddZta9JcPDw8O5tYGBAXPsjh07zPpTTz1l1r/44ovcmtfqS11mXeRp6K2xM7lePvMTBcXwEwXF8BMFxfATBcXwEwXF8BMFxfATBRWmz5+6jbZV9/r0Xk/YW1a7adMms37PPffk1pYvX26O9Xz66adm3evznz9/Prfmfd8rV6406ytWrDDrhw4dyq15P2/v+/KOEyhSvU4jz2d+oqAYfqKgGH6ioBh+oqAYfqKgGH6ioBh+oqBaqs9v9TBT+/heL95ak+/1hL3rvvfee836Aw88YNatU3+fPXvWHHvgwAGz/vrrr5v1kZERs97T05Nbu/XWW82x8+fPN+u33367WbfOFzA6OmqOtc4FAKSfZj7lsVwvfOYnCorhJwqK4ScKiuEnCorhJwqK4ScKiuEnCsrt84vINgAbAAyq6k3ZZVcA+C2AawAcBnC3qtoLv+vA6n+mrMcH/F681cv3+vzXX3+9Wd+wYYNZ9xw8eDC39swzz5hjX375ZbPu9cO97cWtPQWuvfZac6y3vfiVV15p1i3eXgip590vcrv5Rp63/1cA7rjkskcA7FbVHgC7s8+JqIW44VfVVwCcueTijQC2Zx9vB3BnnedFRAWr9T1/t6pePHbyBIDuOs2HiBok+dh+VVURyX2jISJ9APpSb4eI6qvWZ/6TIrIUALL/B/O+UFW3qmqvqvbWeFtEVIBaw78LwJbs4y0AXqjPdIioUdzwi8hzAP4fwPUickxE7gPwBIBvichHAP4h+5yIWoj7nl9VN+eUvlnnubhmck7ymfL6/FZ97ty55tj777/frHd3238v/fjjj836448/nlvr7+83x3755Zdm3btfvD0Lrr766tzakiVLzLHeev6FCxea9RSpj7VGrclPwSP8iIJi+ImCYviJgmL4iYJi+ImCYviJgmqpU3dbUpf0ptS9Vt91111n1oeHh83622+/bdaHhoZya167bN68eWbdW/rqfW+bN+d1iv0lvWNjY2bd2ybbGl+pVMyxqUt6WwGf+YmCYviJgmL4iYJi+ImCYviJgmL4iYJi+ImCmjV9fq/v6i1NTVnC6fXSveMAzp07l1S3luVOTEyYY70luatXrzbrDz30kFm3Tt3t8bYX95YrW9tse8cINPOpu+uFz/xEQTH8REEx/ERBMfxEQTH8REEx/ERBMfxEQc2aPn+Zp1r2euknT54064sWLTLr3rr3u+66K7c2MjJijl22bJlZv/nmm826NzdLZ2enWR8czN0ICgDwxhtvmPWUn6m33j+V9Xht1DEAfOYnCorhJwqK4ScKiuEnCorhJwqK4ScKiuEnCsrt84vINgAbAAyq6k3ZZY8B+C6AiyeMf1RVf1fUJBvB661afd/R0VFz7O7du836unXrzPoNN9xg1nt6enJrqT1j7zwI58+fN+vWMRBz5tgPvw8//NCsnz592qxbPzPv+/L2K2iG9fipqnnm/xWAO6a5/Geqekv2r6WDTxSRG35VfQXAmQbMhYgaKOU9/4Mi8o6IbBORy+s2IyJqiFrD/wsAXwdwC4ABAD/J+0IR6RORfSKyr8bbIqIC1BR+VT2pqhdUtQLglwDWGF+7VVV7VbW31kkSUf3VFH4RWTrl0+8AeLc+0yGiRqmm1fccgG8AWCwixwD8GMA3ROQWAArgMIDvFThHIiqAG35VnW6D9acLmEuhUvr4Xt07v/zOnTvN+okTJ8z62rVrzbrVs7bO6Q/43/dVV11l1r1jEBYvXpxbO3XqlDl2z549Zt07vsI6BsHr43tSzx/RDHiEH1FQDD9RUAw/UVAMP1FQDD9RUAw/UVAtdepuq13ntV68lpY33moNWVtBA/6pu1988UWz/tprr5l1a+7e9+1tL75+/XqzvmLFCrNuteP2799vjn311VfNunfK9NR2niV1i+5mwGd+oqAYfqKgGH6ioBh+oqAYfqKgGH6ioBh+oqBaqs9vST2Vsjc+pWfsjfV68d5xBFZP2evje/Wuri6z3t7ebtY/+eST3Nq+ffaZ3cbHx5Nuu8htsFuhj+/hMz9RUAw/UVAMP1FQDD9RUAw/UVAMP1FQDD9RULOmz+8p8jiA1OseGxsz614/2zpOwJtbZ2dnUt07bfnQ0FBube/eveZYb/tvr55yv0TAZ36ioBh+oqAYfqKgGH6ioBh+oqAYfqKgGH6ioNw+v4gsB/AsgG4ACmCrqj4pIlcA+C2AawAcBnC3qn5a3FSL5a2pt3jr9VPP8e7VreMAvLHeen5v/NGjR836Bx98kFuzjgGohje3lH0eIhwHUM0z/wSAH6rqjQDWAvi+iNwI4BEAu1W1B8Du7HMiahFu+FV1QFX7s4+HAbwPYBmAjQC2Z1+2HcCdRU2SiOpvRu/5ReQaAKsA7AHQraoDWekEJt8WEFGLqPrYfhGZD2AHgB+o6rmp75lUVUVk2jdJItIHoC91okRUX1U984tIByaD/2tVfT67+KSILM3qSwEMTjdWVbeqaq+q9tZjwkRUH274ZfIp/mkA76vqT6eUdgHYkn28BcAL9Z8eERWlmpf9fwfgHwEcEJG3ssseBfAEgP8SkfsAHAFwdzFTbH1eW6mtzf4d7NXnzMn/MVo1AFi4cKFZ//zzz826t/14f39/bu3cuXPm2JT2K1DsMuzZwA2/qv4BQN6j95v1nQ4RNQqP8CMKiuEnCorhJwqK4ScKiuEnCorhJwoqzKm7PV7f1+u1p4z16t6puzs6OnJrc+fONccuWLDArB85csSsHz9+3Kxbxwmk9vFTFL2kN2UL75Ql4DO5XT7zEwXF8BMFxfATBcXwEwXF8BMFxfATBcXwEwXV8D5/yumUU3qnzSzl1NwA0NXVlVvzTs3trfcfHR01697crG20i+7zl/l4KbJXX6/zFPCZnygohp8oKIafKCiGnygohp8oKIafKCiGnyiohvb5RcRcu+71fVPW1Kduk51yfILHG+9tAT4+Pp5bm5iYMMeeOnXKrHvjh4eHzbp1nMDY2Jg51vq+AH9u1v1W5J4ArYLP/ERBMfxEQTH8REEx/ERBMfxEQTH8REEx/ERBSRX97+UAngXQDUABbFXVJ0XkMQDfBTCUfemjqvo757rUWz9usfr8ZZ5n3Tv+wLtub7zXk+7s7Kz5tlN+HoB/v1tz945f8HjHCVi82049r3+R5+23Hi9jY2OoVCpV3Xg1P/kJAD9U1X4RWQDgTRF5Kav9TFX/vZobIqLm4oZfVQcADGQfD4vI+wCWFT0xIirWjN7zi8g1AFYB2JNd9KCIvCMi20Tk8pwxfSKyT0T2Jc2UiOrKfc//1y8UmQ/g/wA8rqrPi0g3gFOY/DvAvwJYqqr3OtfB9/w1jOd7/unxPf9XzeQ9f1XP/CLSAWAHgF+r6vPZBE+q6gVVrQD4JYA11VwXETUHN/wy+SvsaQDvq+pPp1y+dMqXfQfAu/WfHhEVpZpW320AXgVwAMDF13CPAtgM4BZMvuw/DOB72R8Hc7W1tan1EjXlpZS3vNOTctrwol/2p2zhnXo6dO9nkvKy33s749VT3jYUvSS3yJf93lJlVa1Pq09V/wBguisze/pE1Nx4hB9RUAw/UVAMP1FQDD9RUAw/UVAMP1FQVR/eWw/t7e06b9683Lq3HXRK77TM7ZrL3Hq87G3NU7aTng2nxy6CtSX76OgoLly4UL/De4lo9mH4iYJi+ImCYviJgmL4iYJi+ImCYviJgmpon19EhgAcmXLRYkyeCqwZNevcmnVeAOdWq3rObYWqXlXNFzY0/F+5cZF9qtpb2gQMzTq3Zp0XwLnVqqy58WU/UVAMP1FQZYd/a8m3b2nWuTXrvADOrValzK3U9/xEVJ6yn/mJqCSlhF9E7hCRD0TkkIg8UsYc8ojIYRE5ICJvlb3FWLYN2qCIvDvlsitE5CUR+Sj7f9pt0kqa22Micjy7794SkfUlzW25iPyviBwUkfdE5J+zy0u974x5lXK/Nfxlv4i0A/gQwLcAHAOwF8BmVT3Y0InkEJHDAHpVtfSesIjcDmAEwLOqelN22b8BOKOqT2S/OC9X1X9pkrk9BmCk7J2bsw1llk7dWRrAnQD+CSXed8a87kYJ91sZz/xrABxS1T+r6hiA3wDYWMI8mp6qvgLgzCUXbwSwPft4OyYfPA2XM7emoKoDqtqffTwM4OLO0qXed8a8SlFG+JcBODrl82Nori2/FcDvReRNEekrezLT6J6yM9IJAN1lTmYa7s7NjXTJztJNc9/VsuN1vfEPfl91m6quBrAOwPezl7dNSSffszVTu+YXAL6OyW3cBgD8pMzJZDtL7wDwA1U9N7VW5n03zbxKud/KCP9xAMunfP617LKmoKrHs/8HAexE8+0+fPLiJqnZ/4Mlz+evmmnn5ul2lkYT3HfNtON1GeHfC6BHRFaKSCeATQB2lTCPrxCRedkfYiAi8wB8G823+/AuAFuyj7cAeKHEufyNZtm5OW9naZR83zXdjteq2vB/ANZj8i/+fwLwozLmkDOvawG8nf17r+y5AXgOky8DxzH5t5H7AFwJYDeAjwD8D4Armmhu/4HJ3ZzfwWTQlpY0t9sw+ZL+HQBvZf/Wl33fGfMq5X7jEX5EQfEPfkRBMfxEQTH8REEx/ERBMfxEQTH8REEx/ERBMfxEQf0Fyii/rBTs8YIAAAAASUVORK5CYII="
    }
   },
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Explainability through counterfactuals can involve a trade off between sparsity and interpretability. The first counterfactual below is sparse and flips the original 7 into a 3. It is obtained by setting the auto-encoder and prototype loss terms to 0, moving the objective function towards an elastic net regularizer which generates sparse solutions. It is clear however that this is not a very interpretable 3 which lies very far from its class prototype or the training data distribution:\n",
    "\n",
    "![b01g0t0nb7.png](attachment:b01g0t0nb7.png)\n",
    "\n",
    "Class 3 prototype:\n",
    "\n",
    "![proto3.png](attachment:proto3.png)\n",
    "\n",
    "By adding in the nearest prototype and auto-encoder loss terms, the counterfactual becomes more interpretable but less sparse. The example below illustrates this by flipping the same 7 into a 9:\n",
    "\n",
    "![mnist7g100t100.png](attachment:mnist7g100t100.png)\n",
    "\n",
    "Class 9 prototype:\n",
    "\n",
    "![proto9.png](attachment:proto9.png)\n",
    "\n",
    "In order to help interpretability, we can add a trust score constraint on the proposed counterfactual. The trust score is defined as the ratio of the distance between the encoded counterfactual and the prototype of the class predicted on the original instance, and the distance between the encoded counterfactual and the prototype of the class predicted for the counterfactual instance. Intuitively, a high trust score implies that the counterfactual is far from the originally predicted class compared to the counterfactual class. For more info on trust scores, please check out the [documentation](../methods/TrustScores.ipynb). The example below shows the impact of such a trust score constraint:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "theta = 100.\n",
    "c_init = 1.\n",
    "c_steps = 5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "# set random seed\n",
    "np.random.seed(1)\n",
    "tf.set_random_seed(1)\n",
    "\n",
    "# define models\n",
    "cnn = load_model('mnist_cnn.h5')\n",
    "enc = load_model('mnist_enc.h5')\n",
    "\n",
    "sess = K.get_session()\n",
    "\n",
    "# initialize explainer, fit and generate counterfactual\n",
    "cf = CounterFactualProto(sess, cnn, shape, theta=theta,\n",
    "                         enc_model=enc, max_iterations=max_iterations,\n",
    "                         feature_range=feature_range, c_init=c_init, c_steps=c_steps)\n",
    "cf.fit(x_train)  # find class prototypes\n",
    "explanation1 = cf.explain(X, threshold=0.)\n",
    "explanation2 = cf.explain(X, threshold=.8)\n",
    "\n",
    "sess.close()\n",
    "K.clear_session()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Original instance:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7fd0431fca90>"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAADO5JREFUeJzt3V2IXfW5x/Hf76QpiOlFYjUMNpqeogerSKKjCMYS9VhyYiEWg9SLkkLJ9CJKCyVU7EVzWaQv1JvAlIbGkmMrpNUoYmNjMQ1qcSJqEmNiElIzMW9lhCaCtNGnF7Nsp3H2f+/st7XH5/uBYfZez3p52Mxv1lp77bX/jggByOe/6m4AQD0IP5AU4QeSIvxAUoQfSIrwA0kRfiApwg8kRfiBpD7Vz43Z5uOEQI9FhFuZr6M9v+1ltvfZPmD7gU7WBaC/3O5n+23PkrRf0h2SxiW9LOneiHijsAx7fqDH+rHnv1HSgYg4FBF/l/RrSSs6WB+APuok/JdKOjLl+Xg17T/YHrE9Znusg20B6LKev+EXEaOSRiUO+4FB0sme/6ikBVOef66aBmAG6CT8L0u6wvbnbX9a0tckbelOWwB6re3D/og4a/s+Sb+XNEvShojY07XOAPRU25f62toY5/xAz/XlQz4AZi7CDyRF+IGkCD+QFOEHkiL8QFKEH0iK8ANJEX4gKcIPJEX4gaQIP5AU4QeSIvxAUoQfSIrwA0kRfiApwg8kRfiBpAg/kBThB5Ii/EBShB9IivADSRF+ICnCDyRF+IGkCD+QFOEHkmp7iG5Jsn1Y0mlJH0g6GxHD3WgKQO91FP7KrRHx1y6sB0AfcdgPJNVp+EPSVts7bY90oyEA/dHpYf+SiDhq+xJJz9p+MyK2T52h+qfAPwZgwDgiurMie52kMxHxo8I83dkYgIYiwq3M1/Zhv+0LbX/mo8eSvixpd7vrA9BfnRz2z5f0O9sfref/I+KZrnQFoOe6dtjf0sY47Ad6rueH/QBmNsIPJEX4gaQIP5AU4QeSIvxAUt24qy+FlStXNqytXr26uOw777xTrL///vvF+qZNm4r148ePN6wdOHCguCzyYs8PJEX4gaQIP5AU4QeSIvxAUoQfSIrwA0lxS2+LDh061LC2cOHC/jUyjdOnTzes7dmzp4+dDJbx8fGGtYceeqi47NjYWLfb6Rtu6QVQRPiBpAg/kBThB5Ii/EBShB9IivADSXE/f4tK9+xfe+21xWX37t1brF911VXF+nXXXVesL126tGHtpptuKi575MiRYn3BggXFeifOnj1brJ86dapYHxoaanvbb7/9drE+k6/zt4o9P5AU4QeSIvxAUoQfSIrwA0kRfiApwg8k1fR+ftsbJH1F0smIuKaaNk/SbyQtlHRY0j0R8W7Tjc3g+/kH2dy5cxvWFi1aVFx2586dxfoNN9zQVk+taDZewf79+4v1Zp+fmDdvXsPamjVrisuuX7++WB9k3byf/5eSlp0z7QFJ2yLiCknbqucAZpCm4Y+I7ZImzpm8QtLG6vFGSXd1uS8APdbuOf/8iDhWPT4uaX6X+gHQJx1/tj8ionQub3tE0kin2wHQXe3u+U/YHpKk6vfJRjNGxGhEDEfEcJvbAtAD7YZ/i6RV1eNVkp7oTjsA+qVp+G0/KulFSf9je9z2NyX9UNIdtt+S9L/VcwAzCN/bj4F19913F+uPPfZYsb579+6GtVtvvbW47MTEuRe4Zg6+tx9AEeEHkiL8QFKEH0iK8ANJEX4gKS71oTaXXHJJsb5r166Oll+5cmXD2ubNm4vLzmRc6gNQRPiBpAg/kBThB5Ii/EBShB9IivADSTFEN2rT7OuzL7744mL93XfL3xa/b9++8+4pE/b8QFKEH0iK8ANJEX4gKcIPJEX4gaQIP5AU9/Ojp26++eaGteeee6647OzZs4v1pUuXFuvbt28v1j+puJ8fQBHhB5Ii/EBShB9IivADSRF+ICnCDyTV9H5+2xskfUXSyYi4ppq2TtJqSaeq2R6MiKd71SRmruXLlzesNbuOv23btmL9xRdfbKsnTGplz/9LScummf7TiFhU/RB8YIZpGv6I2C5pog+9AOijTs7577P9uu0Ntud2rSMAfdFu+NdL+oKkRZKOSfpxoxltj9gesz3W5rYA9EBb4Y+IExHxQUR8KOnnkm4szDsaEcMRMdxukwC6r63w2x6a8vSrknZ3px0A/dLKpb5HJS2V9Fnb45J+IGmp7UWSQtJhSd/qYY8AeoD7+dGRCy64oFjfsWNHw9rVV19dXPa2224r1l944YViPSvu5wdQRPiBpAg/kBThB5Ii/EBShB9IiiG60ZG1a9cW64sXL25Ye+aZZ4rLcimvt9jzA0kRfiApwg8kRfiBpAg/kBThB5Ii/EBS3NKLojvvvLNYf/zxx4v19957r2Ft2bLpvhT631566aViHdPjll4ARYQfSIrwA0kRfiApwg8kRfiBpAg/kBT38yd30UUXFesPP/xwsT5r1qxi/emnGw/gzHX8erHnB5Ii/EBShB9IivADSRF+ICnCDyRF+IGkmt7Pb3uBpEckzZcUkkYj4me250n6jaSFkg5Luici3m2yLu7n77Nm1+GbXWu//vrri/WDBw8W66V79psti/Z0837+s5K+GxFflHSTpDW2vyjpAUnbIuIKSduq5wBmiKbhj4hjEfFK9fi0pL2SLpW0QtLGaraNku7qVZMAuu+8zvltL5S0WNKfJc2PiGNV6bgmTwsAzBAtf7bf9hxJmyV9JyL+Zv/7tCIiotH5vO0RSSOdNgqgu1ra89uercngb4qI31aTT9gequpDkk5Ot2xEjEbEcEQMd6NhAN3RNPye3MX/QtLeiPjJlNIWSauqx6skPdH99gD0SiuX+pZI+pOkXZI+rCY/qMnz/sckXSbpL5q81DfRZF1c6uuzK6+8slh/8803O1r/ihUrivUnn3yyo/Xj/LV6qa/pOX9E7JDUaGW3n09TAAYHn/ADkiL8QFKEH0iK8ANJEX4gKcIPJMVXd38CXH755Q1rW7du7Wjda9euLdafeuqpjtaP+rDnB5Ii/EBShB9IivADSRF+ICnCDyRF+IGkuM7/CTAy0vhb0i677LKO1v38888X682+DwKDiz0/kBThB5Ii/EBShB9IivADSRF+ICnCDyTFdf4ZYMmSJcX6/fff36dO8EnCnh9IivADSRF+ICnCDyRF+IGkCD+QFOEHkmp6nd/2AkmPSJovKSSNRsTPbK+TtFrSqWrWByPi6V41mtktt9xSrM+ZM6ftdR88eLBYP3PmTNvrxmBr5UM+ZyV9NyJesf0ZSTttP1vVfhoRP+pdewB6pWn4I+KYpGPV49O290q6tNeNAeit8zrnt71Q0mJJf64m3Wf7ddsbbM9tsMyI7THbYx11CqCrWg6/7TmSNkv6TkT8TdJ6SV+QtEiTRwY/nm65iBiNiOGIGO5CvwC6pKXw256tyeBviojfSlJEnIiIDyLiQ0k/l3Rj79oE0G1Nw2/bkn4haW9E/GTK9KEps31V0u7utwegV1p5t/9mSV+XtMv2q9W0ByXda3uRJi//HZb0rZ50iI689tprxfrtt99erE9MTHSzHQyQVt7t3yHJ05S4pg/MYHzCD0iK8ANJEX4gKcIPJEX4gaQIP5CU+znEsm3GcwZ6LCKmuzT/Mez5gaQIP5AU4QeSIvxAUoQfSIrwA0kRfiCpfg/R/VdJf5ny/LPVtEE0qL0Nal8SvbWrm71d3uqMff2Qz8c2bo8N6nf7DWpvg9qXRG/tqqs3DvuBpAg/kFTd4R+tefslg9rboPYl0Vu7aumt1nN+APWpe88PoCa1hN/2Mtv7bB+w/UAdPTRi+7DtXbZfrXuIsWoYtJO2d0+ZNs/2s7bfqn5PO0xaTb2ts320eu1etb28pt4W2P6j7Tds77H97Wp6ra9doa9aXre+H/bbniVpv6Q7JI1LelnSvRHxRl8bacD2YUnDEVH7NWHbX5J0RtIjEXFNNe0hSRMR8cPqH+fciPjegPS2TtKZukdurgaUGZo6srSkuyR9QzW+doW+7lENr1sde/4bJR2IiEMR8XdJv5a0ooY+Bl5EbJd07qgZKyRtrB5v1OQfT9816G0gRMSxiHilenxa0kcjS9f62hX6qkUd4b9U0pEpz8c1WEN+h6SttnfaHqm7mWnMr4ZNl6TjkubX2cw0mo7c3E/njCw9MK9dOyNedxtv+H3ckoi4TtL/SVpTHd4OpJg8ZxukyzUtjdzcL9OMLP0vdb527Y543W11hP+opAVTnn+umjYQIuJo9fukpN9p8EYfPvHRIKnV75M19/MvgzRy83QjS2sAXrtBGvG6jvC/LOkK25+3/WlJX5O0pYY+Psb2hdUbMbJ9oaQva/BGH94iaVX1eJWkJ2rs5T8MysjNjUaWVs2v3cCNeB0Rff+RtFyT7/gflPT9Onpo0Nd/S3qt+tlTd2+SHtXkYeA/NPneyDclXSRpm6S3JP1B0rwB6u1XknZJel2TQRuqqbclmjykf13Sq9XP8rpfu0JftbxufMIPSIo3/ICkCD+QFOEHkiL8QFKEH0iK8ANJEX4gKcIPJPVP82g/p9/JjhUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(X.reshape(28, 28))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The counterfactual below without the trust score threshold predicts a 3 which actually starts to look like a 9 since that is the nearest prototype class. This counterfactual is slightly sparser than the second counterfactual below which has the trust score threshold at 0.8. The sparser counterfactual does not meet this trust score constraint and is therefore rejected in the second example. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Counterfactual prediction: 3\n",
      "Closest prototype class: 9\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7fd04315a710>"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAADeRJREFUeJzt3W+MVOUVx/HfKaVKaBMRhGwsSm20sTEIZAWTYkGthCoJVjYGTcw2ISwvamK1IVX7omh80ZCWpq+I25QUldqaUISYxmJJ4xb/rC5GENyCSCBdwr+6JkWjocDpi722W9z7zDBzZ+4s5/tJNjtzzzwzJwO/vXfmmTuPubsAxPOFshsAUA7CDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gqC8288HMjI8TAg3m7lbN7era85vZQjPba2b7zezheu4LQHNZrZ/tN7MxkvZJuk3SgKQ3Jd3j7u8mxrDnBxqsGXv+2ZL2u/sBdz8l6feSFtdxfwCaqJ7wXy7pH8OuD2Tb/o+ZdZlZn5n11fFYAArW8Df83L1bUrfEYT/QSurZ8x+WNHXY9a9m2wCMAvWE/01JV5vZ18zsS5KWStpSTFsAGq3mw353P21m90v6s6Qxkta5+57COgPQUDVP9dX0YLzmBxquKR/yATB6EX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxBUzUt0S5KZHZR0UtIZSafdvb2IpgA0Xl3hz9zs7v8s4H4ANBGH/UBQ9YbfJW01sx1m1lVEQwCao97D/rnuftjMJkt6ycz+7u49w2+Q/VHgDwPQYszdi7kjs1WSPnL3nyduU8yDAcjl7lbN7Wo+7Dez8Wb2lc8uS1ogaXet9wegueo57J8iaZOZfXY/v3P3FwvpCkDDFXbYX9WDcdjfdDt37kzWr7/++rruf86cOcl6b29vXfeP89fww34AoxvhB4Ii/EBQhB8IivADQRF+ICim+qrU0dGRW7v11luTYy+++OJk/dNPP03WN2zYkKwfPXo0t7Z///7k2ErWrFmTrD/00EM13/eKFSuS9SeffLLm+46MqT4ASYQfCIrwA0ERfiAowg8ERfiBoAg/EBTz/FU6cOBAbm337vR3mEyePDlZv+aaa5L1Sy65JFk/efJkbm3Pnj3JsReygYGB3Nrq1auTY/v6+opup2mY5weQRPiBoAg/EBThB4Ii/EBQhB8IivADQRWxSm8Iy5cvz61Nnz49Oba/vz9Zv/baa5P1WbNmJevz58/Prd14443JsYcOHUrWK31GYdy4ccl6yunTp5P1EydOJOuTJk1K1idMmJBbW7p0aXLsaJ7nrxZ7fiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IquL5/Ga2TtIiScfd/bps26WS/iBpmqSDku529w8rPtgoPp+/laXms2fMmJEcu2PHjmT9hhtuqKmnalRar2Dfvn3J+iuvvJKsjx8/Prf2xBNPJMeuXbs2WW9lRZ7P/1tJC8/Z9rCkbe5+taRt2XUAo0jF8Lt7j6TBczYvlrQ+u7xe0p0F9wWgwWp9zT/F3Y9kl49KmlJQPwCapO7P9ru7p17Lm1mXpK56HwdAsWrd8x8zszZJyn4fz7uhu3e7e7u7t9f4WAAaoNbwb5HUmV3ulLS5mHYANEvF8JvZs5Jek/QNMxsws2WSfibpNjN7T9J3susARhG+t78JHn/88WT9yiuvTNY7OzuT9QvVkiVLkvXnnnsuWU+tp3DzzTcnxw4OnjvBNXrwvf0Akgg/EBThB4Ii/EBQhB8IivADQTHVh7qMGTMmWe/t7c2tPfLII8mxzzzzTLJe6WvFOzo6cmsbN25Mjh3NmOoDkET4gaAIPxAU4QeCIvxAUIQfCIrwA0GxRPcF4L777sutPf300w197DNnziTr7e35X+D02GOPJcdedtllyfqHH6a/LX7v3r3JenTs+YGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKM7nb4LUXLck9fX1JesTJ05M1j/44IPz7qlaqc8QSJU/RzBv3rzc2tatW5Njx44dm6zPnz8/We/p6UnWL1Sczw8gifADQRF+ICjCDwRF+IGgCD8QFOEHgqp4Pr+ZrZO0SNJxd78u27ZK0nJJJ7KbPeruf2pUk6NdpXn8Su66665kvb+/P7e2ffv25NhTp04l65Xm2ivN8y9YsKDm+962bVuy/tprryXrSKtmz/9bSQtH2P5Ld5+R/RB8YJSpGH5375E02IReADRRPa/57zezXWa2zswmFNYRgKaoNfxrJX1d0gxJRyT9Iu+GZtZlZn1mVt8LXwCFqin87n7M3c+4+1lJv5Y0O3Hbbndvd/f02S0Amqqm8JtZ27Cr35O0u5h2ADRLNVN9z0qaL2mSmQ1I+qmk+WY2Q5JLOihpRQN7BNAAnM+PuowbNy5ZT33O4KqrrkqOveOOO5L1V199NVmPivP5ASQRfiAowg8ERfiBoAg/EBThB4JiiW7UZeXKlcn6zJkzc2svvvhicixTeY3Fnh8IivADQRF+ICjCDwRF+IGgCD8QFOEHguKUXiRVOq32+eefT9Y//vjj3NrChSN9KfT/vP7668k6RsYpvQCSCD8QFOEHgiL8QFCEHwiK8ANBEX4gKOb5g5s4cWKy/sYbbyTrq1evTtbnzZuXW7v33nuTY1Eb5vkBJBF+ICjCDwRF+IGgCD8QFOEHgiL8QFAV5/nNbKqkpyRNkeSSut39V2Z2qaQ/SJom6aCku939wwr3xTx/A3R0dOTWNm3alBzb29ubrM+aNStZf//995P11Dn7lcaiNkXO85+W9CN3/6akGyX9wMy+KelhSdvc/WpJ27LrAEaJiuF39yPu/lZ2+aSkfkmXS1osaX12s/WS7mxUkwCKd16v+c1smqSZknolTXH3I1npqIZeFgAYJapeq8/Mvixpo6Qfuvu/zP73ssLdPe/1vJl1Seqqt1EAxapqz29mYzUU/A3u/sds8zEza8vqbZKOjzTW3bvdvd3d24toGEAxKobfhnbxv5HU7+5rhpW2SOrMLndK2lx8ewAapZqpvrmS/ibpHUlns82Pauh1/3OSrpB0SENTfYMV7oupvia76KKLkvVbbrklWd+8Of03fcmSJcl6d3d3bq2trS05FrWpdqqv4mt+d98uKe/Obj2fpgC0Dj7hBwRF+IGgCD8QFOEHgiL8QFCEHwiq6o/3onU98MADubUHH3wwOfaKK65I1leuXJmsv/DCC8k6c/mtiz0/EBThB4Ii/EBQhB8IivADQRF+ICjCDwTFPP8FYPLkybm1adOmJceePXs2WX/55ZeT9WYu8Y5isecHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAqfm9/oQ/G9/bXZO7cucl6T09Pbm34smojqfTvP3v27GS9r68vWUfzFblEN4ALEOEHgiL8QFCEHwiK8ANBEX4gKMIPBFXxfH4zmyrpKUlTJLmkbnf/lZmtkrRc0onspo+6+58a1WhkN910U7I+ffr03NqyZcuSYxctWpSsf/LJJ8k6Rq9qvszjtKQfuftbZvYVSTvM7KWs9kt3/3nj2gPQKBXD7+5HJB3JLp80s35Jlze6MQCNdV6v+c1smqSZknqzTfeb2S4zW2dmE3LGdJlZn5nxOVCghVQdfjP7sqSNkn7o7v+StFbS1yXN0NCRwS9GGufu3e7e7u7tBfQLoCBVhd/Mxmoo+Bvc/Y+S5O7H3P2Mu5+V9GtJ6TNAALSUiuG3odPCfiOp393XDNs+fPnV70naXXx7ABqlmnf7vyXpPknvmNnb2bZHJd1jZjM0NP13UNKKhnSIinbt2pVb27lzZ3LsnDlzkvXBwcGaekLrq+bd/u2SRjo/mDl9YBTjE35AUIQfCIrwA0ERfiAowg8ERfiBoPjqbuACw1d3A0gi/EBQhB8IivADQRF+ICjCDwRF+IGgqjmfv0j/lHRo2PVJ2bZW1Kq9tWpfEr3Vqsjerqz2hk39kM/nHtysr1W/269Ve2vVviR6q1VZvXHYDwRF+IGgyg5/d8mPn9KqvbVqXxK91aqU3kp9zQ+gPGXv+QGUpJTwm9lCM9trZvvN7OEyeshjZgfN7B0ze7vsJcayZdCOm9nuYdsuNbOXzOy97PeIy6SV1NsqMzucPXdvm9ntJfU21cz+ambvmtkeM3sg217qc5foq5TnremH/WY2RtI+SbdJGpD0pqR73P3dpjaSw8wOSmp399LnhM3s25I+kvSUu1+XbVstadDdf5b94Zzg7j9ukd5WSfqo7JWbswVl2oavLC3pTknfV4nPXaKvu1XC81bGnn+2pP3ufsDdT0n6vaTFJfTR8ty9R9K5q2YslrQ+u7xeQ/95mi6nt5bg7kfc/a3s8klJn60sXepzl+irFGWE/3JJ/xh2fUCtteS3S9pqZjvMrKvsZkYwJVs2XZKOSppSZjMjqLhyczOds7J0yzx3tax4XTTe8Pu8ue4+S9J3Jf0gO7xtST70mq2VpmuqWrm5WUZYWfq/ynzual3xumhlhP+wpKnDrn8129YS3P1w9vu4pE1qvdWHj322SGr2+3jJ/fxXK63cPNLK0mqB566VVrwuI/xvSrrazL5mZl+StFTSlhL6+BwzG5+9ESMzGy9pgVpv9eEtkjqzy52SNpfYy/9plZWb81aWVsnPXcuteO3uTf+RdLuG3vF/X9JPyughp6+rJO3MfvaU3ZukZzV0GPhvDb03skzSREnbJL0n6S+SLm2h3p6W9I6kXRoKWltJvc3V0CH9LklvZz+3l/3cJfoq5XnjE35AULzhBwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gqP8ATLCQiRcyZ34AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "print('Counterfactual prediction: {}'.format(explanation1['cf']['class']))\n",
    "print('Closest prototype class: {}'.format(cf.id_proto))\n",
    "plt.imshow(explanation1['cf']['X'].reshape(28, 28))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Counterfactual prediction: 9\n",
      "Closest prototype class: 9\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7fd043130dd8>"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAADfxJREFUeJzt3W+MVPW9x/HPFy4kRvoAreJKUbEx1eKD5WbFmyxcufbScLEJNhICJg1NSLcPMGkTJNd4NcVnjfZP+ohkSUnxplcwAZQY49WCYcU/1V2jiCK4JTQsYVkqaIVEK/Ltgz3Yre75nXHmzJxZvu9XstmZ850z881kP3vOzDm/8zN3F4B4JlXdAIBqEH4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0H9SytfzMw4nRBoMne3Wh7X0JbfzBab2UEzGzSz+xp5LgCtZfWe229mkyUdkrRI0pCk1yStdPd3Euuw5QearBVb/nmSBt39sLv/TdIWSUsbeD4ALdRI+GdKOjrm/lC27J+YWY+Z9ZtZfwOvBaBkTf/Cz917JfVK7PYD7aSRLf8xSbPG3P9GtgzABNBI+F+TdIOZzTazqZJWSNpZTlsAmq3u3X53P2dm90j6f0mTJW1y97dL6wxAU9V9qK+uF+MzP9B0LTnJB8DERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQdU/RLUlmdkTSR5I+k3TO3bvKaApA8zUU/sx/uPtfSngeAC3Ebj8QVKPhd0nPmtmAmfWU0RCA1mh0t3++ux8zsyslPWdm77p739gHZP8U+McAtBlz93KeyGy9pDPu/ovEY8p5MQC53N1qeVzdu/1mdqmZfe3CbUnflbS/3ucD0FqN7PbPkLTDzC48z/+5+zOldAWg6Urb7a/pxdjtn3AmTUrvHJ4/f75FnaBWTd/tBzCxEX4gKMIPBEX4gaAIPxAU4QeCKmNUXwjLli3LrZ0+fTq5btHhsLNnzybrU6dOTdaHh4dza4ODg8l1t2zZkqxv2LAhWd+zZ0+yvnHjxtza7Nmzk+uuXLkyWT958mSyjjS2/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFEN6a7Ro0aLc2qeffppcd+7cucn6lClTkvUFCxYk61dffXVu7ZNPPkmuW+TFF19M1ru7uxt6/mYaGhrKrT388MPJdfv7+8tup2UY0gsgifADQRF+ICjCDwRF+IGgCD8QFOEHgmI8f41SY/KLjuMfOHAgWb/pppuS9Q8++CBZ7+zszK3NnDkzue7Ro0eT9bVr1ybrjTh37lyyXjRev6OjI1l/6qmncmsrVqxIrjuRj/PXii0/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRVOJ7fzDZJ+p6kEXe/OVt2maStkq6TdETScndPX7xeE3s8f5W2b9+erK9evTq3ljoHQJIGBgaS9VtuuSVZb8THH3+crB86dChZLzp/wix/WPsDDzyQXLdovoJ2VuZ4/t9JWvyFZfdJ2uXuN0jald0HMIEUht/d+ySd+sLipZI2Z7c3S7qz5L4ANFm9n/lnuPvx7PawpBkl9QOgRRo+t9/dPfVZ3sx6JPU0+joAylXvlv+EmXVIUvZ7JO+B7t7r7l3u3lXnawFognrDv1PSquz2KklPltMOgFYpDL+ZPSbpZUnfMrMhM1st6eeSFpnZe5L+M7sPYAIp/Mzv7nmTpH+n5F4uWu+++26yfuONNybrb775ZrJ++nT+KRbPP/98ct0iu3btamj9Rtx1113J+vTp05P1/fv359a2bt1aV08XE87wA4Ii/EBQhB8IivADQRF+ICjCDwTFFN0tMGlS+n9s6rLgE93w8HBurehQXtFQ5iuvvDJZX7ZsWW5t27ZtyXUnMqboBpBE+IGgCD8QFOEHgiL8QFCEHwiK8ANBMUV3C6xZsyZZ37FjR7Le19eXrA8ODubW9u3bl1z33nvvTdYbddVVV+XWHnrooeS6V1xxRbKeGsosSQcPHkzWo2PLDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBMZ4fSevWrUvWH3nkkWS9u7s7t7Z79+7kulOmTEnWFy5cmKwXnR9xsWI8P4Akwg8ERfiBoAg/EBThB4Ii/EBQhB8IqnA8v5ltkvQ9SSPufnO2bL2kH0k6mT3sfnd/ullNRjcwMJCsv/DCC7m1ojHzqWmspeI5B4qO8y9ZsiS3VnQcv2h68JdffjlZR1otW/7fSVo8zvJfu3tn9kPwgQmmMPzu3ifpVAt6AdBCjXzmv8fM9pnZJjObXlpHAFqi3vBvkPRNSZ2Sjkv6Zd4DzazHzPrNrL/O1wLQBHWF391PuPtn7n5e0kZJ8xKP7XX3LnfvqrdJAOWrK/xm1jHm7vclpb8yBtB2ajnU95ikhZK+bmZDkn4maaGZdUpySUck/biJPQJoAsbzoyGXXHJJsr53797c2pw5c5Lr3n777cn6Sy+9lKxHxXh+AEmEHwiK8ANBEX4gKMIPBEX4gaCYohsNKbq099y5c3NrzzzzTHJdhuw2F1t+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiKIb1IuuOOO5L1J554Ilk/e/Zsbm3x4vEuCv0Pr7zySrKO8TGkF0AS4QeCIvxAUIQfCIrwA0ERfiAowg8ExXH+4C6//PJk/dVXX03W33///WR9cHAwt3b33Xcn10V9OM4PIInwA0ERfiAowg8ERfiBoAg/EBThB4IqvG6/mc2S9KikGZJcUq+7/8bMLpO0VdJ1ko5IWu7up5vXKuoxefLkZL3o2vmzZ89O1s+fP5+sP/jgg8k6qlPLlv+cpLXu/m1J/yZpjZl9W9J9kna5+w2SdmX3AUwQheF39+Pu/np2+yNJByTNlLRU0ubsYZsl3dmsJgGU7yt95jez6yTNlfRHSTPc/XhWGtboxwIAE0TNc/WZ2TRJ2yT91N3/avaP04fd3fPO2zezHkk9jTYKoFw1bfnNbIpGg/97d9+eLT5hZh1ZvUPSyHjrunuvu3e5e1cZDQMoR2H4bXQT/1tJB9z9V2NKOyWtym6vkvRk+e0BaJbCIb1mNl/SC5LeknThuM79Gv3c/7ikayT9WaOH+k4VPBdDelvs+uuvT9bnzJmTrD/++OPJ+vLly5P1Dz/8MLfW19eXXBf1qXVIb+FnfnffKynvyb7zVZoC0D44ww8IivADQRF+ICjCDwRF+IGgCD8QVM2n96J9XXvttbm13bt3J9e95pprkvV169Yl6wcPHkzWDx06lKyjOmz5gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAojvNfBHp68q+S1tnZmVx3ZGTcCzB9bs+ePcl6I8fxi641cPjw4bqfG8XY8gNBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIXX7S/1xbhuf13mz5+frD/99NO5tWnTpjX02vPmzUvW+/v7G3p+lK/W6/az5QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoArH85vZLEmPSpohySX1uvtvzGy9pB9JOpk99H53zz/gjLotWLAgWd+0aVNdNUnatm1bsn7mzJlkHRNXLRfzOCdprbu/bmZfkzRgZs9ltV+7+y+a1x6AZikMv7sfl3Q8u/2RmR2QNLPZjQForq/0md/MrpM0V9Ifs0X3mNk+M9tkZtNz1ukxs34z4zxQoI3UHH4zmyZpm6SfuvtfJW2Q9E1JnRrdM/jleOu5e6+7d7l7Vwn9AihJTeE3sykaDf7v3X27JLn7CXf/zN3PS9ooKT0CBEBbKQy/mZmk30o64O6/GrO8Y8zDvi9pf/ntAWiWWr7t75b0A0lvmdkb2bL7Ja00s06NHv47IunHTekQhbq7u3Nrt912W3LdW2+9NVk/depUXT2h/dXybf9eSeOND+aYPjCBcYYfEBThB4Ii/EBQhB8IivADQRF+ICgu3Q1cZLh0N4Akwg8ERfiBoAg/EBThB4Ii/EBQhB8Iqpbx/GX6i6Q/j7n/9WxZO2rX3tq1L4ne6lVmb9fW+sCWnuTzpRc362/Xa/u1a2/t2pdEb/Wqqjd2+4GgCD8QVNXh76349VPatbd27Uuit3pV0luln/kBVKfqLT+AilQSfjNbbGYHzWzQzO6rooc8ZnbEzN4yszeqnmIsmwZtxMz2j1l2mZk9Z2bvZb/HnSatot7Wm9mx7L17w8yWVNTbLDN73szeMbO3zewn2fJK37tEX5W8by3f7TezyZIOSVokaUjSa5JWuvs7LW0kh5kdkdTl7pUfEzazf5d0RtKj7n5ztuxhSafc/efZP87p7v7fbdLbeklnqp65OZtQpmPszNKS7pT0Q1X43iX6Wq4K3rcqtvzzJA26+2F3/5ukLZKWVtBH23P3PklfnDVjqaTN2e3NGv3jabmc3tqCux9399ez2x9JujCzdKXvXaKvSlQR/pmSjo65P6T2mvLbJT1rZgNm1lN1M+OYkU2bLknDkmZU2cw4CmdubqUvzCzdNu9dPTNel40v/L5svrv/q6T/krQm271tSz76ma2dDtfUNHNzq4wzs/Tnqnzv6p3xumxVhP+YpFlj7n8jW9YW3P1Y9ntE0g613+zDJy5Mkpr9Hqm4n8+108zN480srTZ479ppxusqwv+apBvMbLaZTZW0QtLOCvr4EjO7NPsiRmZ2qaTvqv1mH94paVV2e5WkJyvs5Z+0y8zNeTNLq+L3ru1mvHb3lv9IWqLRb/z/JOl/qughp6/rJb2Z/bxddW+SHtPobuCnGv1uZLWkyyXtkvSepD9IuqyNevtfSW9J2qfRoHVU1Nt8je7S75P0RvazpOr3LtFXJe8bZ/gBQfGFHxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoP4ON9aY0qRSMr4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "print('Counterfactual prediction: {}'.format(explanation2['cf']['class']))\n",
    "print('Closest prototype class: {}'.format(cf.id_proto))\n",
    "plt.imshow(explanation2['cf']['X'].reshape(28, 28))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Specify prototype classes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For multi-class predictions, we might be interested to generate counterfactuals for certain classes while avoiding others. The following example illustrates how to do this. It does not guarantee that the avoided classes are excluded from the counterfactuals, but does not include those classes when looking for the nearest prototype class. So as a result, it will not drive the counterfactual to the prototype of the classes that should be avoided."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7fd04308e240>"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAADXVJREFUeJzt3WuoXfWZx/HfTycFsQEvoeHEBk8nyECTQBIOIjTGaMfihEIsAakvmgxKU7GBKfbFeCFMMG9k7MUiUkhobNRqK2lLItYY5zAohbEkxlQ9cRozJaUJxyTFSo0XMibPvDgrndN49n/v7NvaJ8/3A4fsvZ51eVzml7X2XuusvyNCAPK5oO4GANSD8ANJEX4gKcIPJEX4gaQIP5AU4QeSIvxAUoQfSOrv+rkx29xOCPRYRLiV+To68tu+yfbvbB+0fXcn6wLQX2733n7bF0o6IOlGSYcl7ZZ0a0TsLyzDkR/osX4c+a+WdDAifh8RJyX9VNLKDtYHoI86Cf8Vkv446f3hatrfsL3W9h7bezrYFoAu6/kXfhGxSdImidN+YJB0cuQ/ImnupPefraYBmAY6Cf9uSVfZ/pztT0n6qqQd3WkLQK+1fdofER/bXifpeUkXStoSEWNd6wxAT7V9qa+tjfGZH+i5vtzkA2D6IvxAUoQfSIrwA0kRfiApwg8kRfiBpAg/kBThB5Ii/EBShB9IivADSRF+ICnCDyRF+IGkCD+QFOEHkiL8QFKEH0iK8ANJEX4gKcIPJEX4gaQIP5AU4QeSIvxAUoQfSIrwA0kRfiCptofoliTbhyS9J+mUpI8jYqQbTQHovY7CX7k+Iv7UhfUA6CNO+4GkOg1/SNpl+xXba7vREID+6PS0f2lEHLH9GUkv2P7viHhp8gzVPwr8wwAMGEdEd1Zkb5B0IiK+U5inOxsD0FBEuJX52j7tt32x7ZlnXkv6kqQ32l0fgP7q5LR/tqRf2j6znicjYmdXugLQc1077W9pY5z2Az3X89N+ANMb4QeSIvxAUoQfSIrwA0kRfiCpbvxWHwbYokWLivWNGzcW6ytWrCjWL7igfPw4ffp0w9q2bduKy953333F+vj4eLF+/fXXN6yNjo4Wl/3www+L9fMBR34gKcIPJEX4gaQIP5AU4QeSIvxAUoQfSIrr/NPAjBkzivXrrruuYe3RRx8tLjs0NFSsN/uV79J1/GbLr1q1qrhss2vtc+fOLdaXL1/esLZmzZrisk888USxfj7gyA8kRfiBpAg/kBThB5Ii/EBShB9IivADSXGdfxpYsmRJsb5zZ/vDJTT7nfh169YV6x988EHb277yyiuL9ffff79Yf/jhh4v1kydPNqw1++/OgCM/kBThB5Ii/EBShB9IivADSRF+ICnCDyTV9Dq/7S2SvizpWEQsqKZdJulnkoYlHZJ0S0T8uXdtnt/mz59frO/YsaPtdTd7Pv0999xTrO/du7ftbTczZ86cYn379u3F+iWXXFKsP/jggw1rzfZLBq0c+X8s6aazpt0taTQirpI0Wr0HMI00DX9EvCTpnbMmr5S0tXq9VdLNXe4LQI+1+5l/dkScuT/ybUmzu9QPgD7p+N7+iAjbDR/UZnutpLWdbgdAd7V75D9qe0iSqj+PNZoxIjZFxEhEjLS5LQA90G74d0g68/jTNZLKX8sCGDhNw2/7KUn/JekfbB+2fbukByTdaPstSf9YvQcwjTT9zB8RtzYofbHLvaS1fv36Yn3WrFnF+rPPPtuwdtdddxWXPXjwYLHeSwsWLCjWFy9e3NH6O3nOQQbc4QckRfiBpAg/kBThB5Ii/EBShB9Iys2GYO7qxgq3AZ/PNm/eXKzfdtttxXqzR1hfc801DWv79+8vLttrpeHFd+3aVVx22bJlxfqLL75YrN9www3F+vkqItzKfBz5gaQIP5AU4QeSIvxAUoQfSIrwA0kRfiAphujug5GR8kOMmt1rceLEiWK9zmv5pev4krRx48aGtWuvvba4bLP9cv/99xfrKOPIDyRF+IGkCD+QFOEHkiL8QFKEH0iK8ANJcZ0fRcPDw8X6nXfeWaw3e3R4yfj4eLG+b9++ttcNjvxAWoQfSIrwA0kRfiApwg8kRfiBpAg/kFTT6/y2t0j6sqRjEbGgmrZB0tclHa9muzciftWrJqe7Zr9vv3DhwmL98ssvL9ZfffXVc+6pVc2GB58zZ06x3sm4EKOjo8X6u+++2/a60dqR/8eSbppi+vcjYlH1Q/CBaaZp+CPiJUnv9KEXAH3UyWf+dbZfs73F9qVd6whAX7Qb/h9KmidpkaRxSd9tNKPttbb32N7T5rYA9EBb4Y+IoxFxKiJOS9os6erCvJsiYiQiyk+xBNBXbYXf9tCkt1+R9EZ32gHQL61c6ntK0nJJs2wflvRvkpbbXiQpJB2S9I0e9gigB9zJddhz3pjdv40NkIsuuqhYf/rpp4v1FStWFOv9/H94tpUrVxbrq1evblhbtWpVcdmlS5cW6y+//HKxnlVEuJX5uMMPSIrwA0kRfiApwg8kRfiBpAg/kBSX+qaB5cuXF+vNhgAvGRsbK9afe+65Yv2RRx4p1u+4446GtQMHDhSXXbZsWbF+/PjxYj0rLvUBKCL8QFKEH0iK8ANJEX4gKcIPJEX4gaS4zo+OnDp1qlgv/f168skni8uWfh0YjXGdH0AR4QeSIvxAUoQfSIrwA0kRfiApwg8k1fS5/chteHi4o+VPnDjRsPbQQw91tG50hiM/kBThB5Ii/EBShB9IivADSRF+ICnCDyTV9Dq/7bmSHpM0W1JI2hQRP7B9maSfSRqWdEjSLRHx5961ijqsX7++o+WfeeaZhrW9e/d2tG50ppUj/8eSvh0Rn5d0jaRv2v68pLsljUbEVZJGq/cApomm4Y+I8YjYW71+T9Kbkq6QtFLS1mq2rZJu7lWTALrvnD7z2x6WtFjSbyTNjojxqvS2Jj4WAJgmWr633/anJf1c0rci4i/2/z8mLCKi0fP5bK+VtLbTRgF0V0tHftszNBH8n0TEL6rJR20PVfUhScemWjYiNkXESES0P5okgK5rGn5PHOJ/JOnNiPjepNIOSWuq12skbe9+ewB6pZXT/i9I+pqk123vq6bdK+kBSU/bvl3SHyTd0psW0Uvz588v1letWtXR+p9//vmOlkfvNA1/RPxaUqPngH+xu+0A6Bfu8AOSIvxAUoQfSIrwA0kRfiApwg8kxaO7k1uyZEmxPnPmzGK92RDvH3300Tn3hP7gyA8kRfiBpAg/kBThB5Ii/EBShB9IivADSXGdP7lZs2YV682u44+NjRXr27ZtO+ee0B8c+YGkCD+QFOEHkiL8QFKEH0iK8ANJEX4gKa7zJ7d69eqOln/88ce71An6jSM/kBThB5Ii/EBShB9IivADSRF+ICnCDyTV9Dq/7bmSHpM0W1JI2hQRP7C9QdLXJR2vZr03In7Vq0bRG/v37y/WFy5c2KdO0G+t3OTzsaRvR8Re2zMlvWL7har2/Yj4Tu/aA9ArTcMfEeOSxqvX79l+U9IVvW4MQG+d02d+28OSFkv6TTVpne3XbG+xfWmDZdba3mN7T0edAuiqlsNv+9OSfi7pWxHxF0k/lDRP0iJNnBl8d6rlImJTRIxExEgX+gXQJS2F3/YMTQT/JxHxC0mKiKMRcSoiTkvaLOnq3rUJoNuaht+2Jf1I0psR8b1J04cmzfYVSW90vz0AvdLKt/1fkPQ1Sa/b3ldNu1fSrbYXaeLy3yFJ3+hJh+ipnTt3Fuvz5s0r1nfv3t3NdtBHrXzb/2tJnqLENX1gGuMOPyApwg8kRfiBpAg/kBThB5Ii/EBSbjYEc1c3ZvdvY0BSETHVpflP4MgPJEX4gaQIP5AU4QeSIvxAUoQfSIrwA0n1e4juP0n6w6T3s6ppg2hQexvUviR6a1c3e7uy1Rn7epPPJzZu7xnUZ/sNam+D2pdEb+2qqzdO+4GkCD+QVN3h31Tz9ksGtbdB7Uuit3bV0lutn/kB1KfuIz+AmtQSfts32f6d7YO2766jh0ZsH7L9uu19dQ8xVg2Ddsz2G5OmXWb7BdtvVX9OOUxaTb1tsH2k2nf7bK+oqbe5tv/T9n7bY7b/pZpe674r9FXLfuv7ab/tCyUdkHSjpMOSdku6NSLKY0X3ie1DkkYiovZrwraXSToh6bGIWFBN+3dJ70TEA9U/nJdGxL8OSG8bJJ2oe+TmakCZockjS0u6WdI/q8Z9V+jrFtWw3+o48l8t6WBE/D4iTkr6qaSVNfQx8CLiJUnvnDV5paSt1eutmvjL03cNehsIETEeEXur1+9JOjOydK37rtBXLeoI/xWS/jjp/WEN1pDfIWmX7Vdsr627mSnMroZNl6S3Jc2us5kpNB25uZ/OGll6YPZdOyNedxtf+H3S0ohYIumfJH2zOr0dSDHxmW2QLte0NHJzv0wxsvRf1bnv2h3xutvqCP8RSXMnvf9sNW0gRMSR6s9jkn6pwRt9+OiZQVKrP4/V3M9fDdLIzVONLK0B2HeDNOJ1HeHfLekq25+z/SlJX5W0o4Y+PsH2xdUXMbJ9saQvafBGH94haU31eo2k7TX28jcGZeTmRiNLq+Z9N3AjXkdE338krdDEN/7/I+m+Onpo0NffS/pt9TNWd2+SntLEaeD/auK7kdslXS5pVNJbkv5D0mUD1Nvjkl6X9JomgjZUU29LNXFK/5qkfdXPirr3XaGvWvYbd/gBSfGFH5AU4QeSIvxAUoQfSIrwA0kRfiApwg8kRfiBpP4PIblH+EkgAM0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "X = x_test[12].reshape((1,) + x_test[1].shape)\n",
    "plt.imshow(X.reshape(28, 28))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "theta = 100.\n",
    "gamma = 100.\n",
    "c_init = 1.\n",
    "c_steps = 3\n",
    "max_iterations = 1000"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "# set random seed\n",
    "np.random.seed(1)\n",
    "tf.set_random_seed(1)\n",
    "\n",
    "# define models\n",
    "cnn = load_model('mnist_cnn.h5')\n",
    "ae = load_model('mnist_ae.h5')\n",
    "enc = load_model('mnist_enc.h5')\n",
    "\n",
    "sess = K.get_session()\n",
    "\n",
    "# initialize explainer, fit and generate counterfactual\n",
    "cf = CounterFactualProto(sess, cnn, shape, gamma=gamma, theta=theta,\n",
    "                         ae_model=ae, enc_model=enc, max_iterations=max_iterations,\n",
    "                         feature_range=feature_range, c_init=c_init, c_steps=c_steps)\n",
    "cf.fit(x_train)  # find class prototypes\n",
    "explanation1 = cf.explain(X)\n",
    "proto_class1 = cf.id_proto\n",
    "explanation2 = cf.explain(X, target_class=[7])\n",
    "proto_class2 = cf.id_proto\n",
    "\n",
    "sess.close()\n",
    "K.clear_session()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The closest class to a 9 is 4. This is evident by looking at the first counterfactual below. For the second counterfactual, we specified that the prototype class used in the search should be a 7. As a result, the 9 is not pushed towards a 4."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Counterfactual prediction: 4\n",
      "Closest prototype class: 4\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7fd048f87ef0>"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAADwtJREFUeJzt3V2MVHWax/HfQw8vkTcRtNNxiCC0o2giQ1pdlZjZzErUTAS8MOOFwcQMk4jJTpzEJe7FotyYzbwwVygTcHBBwWSGiHEEXLLKTrKZ2CKDCtIiQnhpgUEi3UZE6Gcv+jBpkfqfot5OdT/fT9LpqvPUqfNQ4dfnVP1Pnb+5uwDEM6zoBgAUg/ADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwjqe43cmJlxOiFQZ+5u5Tyuqj2/md1jZnvMbK+ZLa7muQA0llV6br+ZtUjqknS3pEOS3pH0kLvvSqzDnh+os0bs+W+VtNfd97n7GUnrJM2t4vkANFA14b9a0sEB9w9ly77FzBaaWaeZdVaxLQA1VvcP/Nx9haQVEof9QDOpZs9/WNLkAfe/ny0DMAhUE/53JLWb2VQzGyHpp5I21qYtAPVW8WG/u581s8clbZbUImmVu39Ys84A1FXFQ30VbYz3/EDdNeQkHwCDF+EHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQTV0im4MPsOGpfcPfX19Derku0aPHp2snz59umTt3LlztW5n0GHPDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBVTVLr5ntl9Qj6Zyks+7ekfN4ZultMLP0hK2NnKX5QhMnTkzWT5w4kaw387+tSOXO0luLk3z+2d3/XoPnAdBAHPYDQVUbfpe0xczeNbOFtWgIQGNUe9g/290Pm9lVkt40s4/cfdvAB2R/FPjDADSZqj7w+9YTmS2R1Ovuv0o8JuYnMAVq5g/F+MCvPsr9wK/iw34zG21mY8/fljRH0geVPh+AxqrmsL9V0obsr+/3JL3k7ptq0hWAuqvZYX9ZG+OwHzV0/fXXJ+sfffRRxc89YsSIZP3MmTMVP3e91f2wH8DgRviBoAg/EBThB4Ii/EBQhB8Iikt3D3EzZ85M1pcuXZqsv/fee8l63qW7U8NtY8aMSa67efPmqrZ91VVXlaz19PQk17322muT9QMHDiTrvb29yXozYM8PBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0Exzj8IjB8/Plm/5ZZbStZeeOGF5LotLS3J+u7du5P1I0eOJOupsfaHH344ue7s2bOT9V27diXr11xzTcnak08+mVy3u7s7Wc973QYD9vxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBTj/E1g2LD03+C8S1Q/99xzJWttbW3JddesWZOsv/LKK8n6hAkTkvXUv239+vXJdR944IFk/eabb07Wjx49WrLW3t6eXHfnzp3Jet61BAYD9vxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EFTuOL+ZrZL0E0nH3P2mbNkVktZLmiJpv6QH3f1k/doc3EaOHJmsT58+PVnfsGFDsp76bvmWLVuS6y5fvjxZ7+zsTNbz3HbbbSVre/bsSa6bd13/GTNmJOubNm0qWduxY0dy3QjK2fP/QdI9FyxbLGmru7dL2prdBzCI5Ibf3bdJ+vyCxXMlrc5ur5Y0r8Z9AaizSt/zt7r7+escfSaptUb9AGiQqs/td3c3My9VN7OFkhZWux0AtVXpnv+ombVJUvb7WKkHuvsKd+9w944KtwWgDioN/0ZJC7LbCyS9Wpt2ADRKbvjN7GVJ/yfpB2Z2yMwelfSspLvN7GNJ/5LdBzCI5L7nd/eHSpR+XONehqy877w/88wzyXpra/rz1Ndff71k7Yknnkiuu3fv3mS9WgcPHqx43bxx/K+++ipZT43zgzP8gLAIPxAU4QeCIvxAUIQfCIrwA0Fx6e4GWLp0abI+b176e1G9vb3J+htvvFGyljccNm7cuGT91KlTyXqe48ePl6ytW7cuua6ZJetbt25N1ru6upL16NjzA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQjPPXwPjx45P1jo70RYxSU0lL+WPtqa/0Hj58OLlutS6//PJkffHi0hd2vuOOO5Lrps4RkKRly5Yl693d3SVrY8eOTa7b09OTrA8F7PmBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjG+Wvg7NmzVa0/atSoqta/9957S9aef/75qp57ypQpyfpjjz2WrOddOjzlm2++SdbzrlWQcvr06YrXHSrY8wNBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUObu6QeYrZL0E0nH3P2mbNkSST+TdP4L10+5+59zN2aW3tgQtXbt2mT9/vvvT9ZHjBiRrKe+t37y5MnkunkmTZqUrOf1NnHixIq3vWbNmmT9kUceqfi586Y9z7vGQjNz9/SEB5ly9vx/kHTPRZb/1t1nZj+5wQfQXHLD7+7bJH3egF4ANFA17/kfN7OdZrbKzCbUrCMADVFp+JdLmiZppqRuSb8u9UAzW2hmnWbWWeG2ANRBReF396Pufs7d+yT9XtKticeucPcOd09fxRJAQ1UUfjNrG3B3vqQPatMOgEbJ/Uqvmb0s6UeSJpnZIUn/IelHZjZTkkvaL+nndewRQB3kjvPXdGNBx/lnzZqVrD/99NPJ+rRp05L16667rmStpaUlue65c+eS9Tx9fX3J+sGDB0vWrrzyyuS68+fPT9bfeuutZL3af9tgVctxfgBDEOEHgiL8QFCEHwiK8ANBEX4gKIb6BoE777wzWb/99ttL1vIuUZ36OrAkTZ06NVm/4YYbkvXU1267urqS6951113Jet4U3lEx1AcgifADQRF+ICjCDwRF+IGgCD8QFOEHgmKcvwmMHDkyWf/6668rfu7hw4cn6xMmpC+/OH369GR906ZNyfpll11WsvbSSy8l1120aFGynncOQ94U30MV4/wAkgg/EBThB4Ii/EBQhB8IivADQRF+IKjc6/aj/qoZx8+TN9add47BkSNHkvW8Kbp7e3tL1pYtW5Zct6enJ1k3K2s4GyWw5weCIvxAUIQfCIrwA0ERfiAowg8ERfiBoHLH+c1ssqQXJbVKckkr3P13ZnaFpPWSpkjaL+lBdz9Zv1ZRD6kptCVp5cqVVT3/a6+9VrK2ffv25LqjRo1K1vPOUfjiiy+S9ejK2fOflfRLd58h6Z8kLTKzGZIWS9rq7u2Stmb3AQwSueF39253357d7pG0W9LVkuZKWp09bLWkefVqEkDtXdJ7fjObIumHkv4qqdXdz8/19Jn63xYAGCTKPrffzMZI+qOkX7j7qYHnVbu7l7o+n5ktlLSw2kYB1FZZe34zG67+4K919z9li4+aWVtWb5N07GLruvsKd+9w945aNAygNnLDb/27+JWSdrv7bwaUNkpakN1eIOnV2rcHoF7KOey/U9LDkt43sx3ZsqckPSvpFTN7VNIBSQ/Wp0XU04033pisz5kzJ1n/9NNPk/XNmzdfck/n5V2au5GXnR+KcsPv7n+RVOqL0z+ubTsAGoUz/ICgCD8QFOEHgiL8QFCEHwiK8ANBcenuIS5viu5Zs2ZVtf6hQ4eS9dRYfd6lt/PG8et5yfMI2PMDQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCM8w9xeVN0T5o0KVlvaWlJ1vft25esv/322xVv+/jx48k6qsOeHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCYpw/uLa2tmR93Lhxyfonn3ySrPf19ZWs5Z2DgPpizw8ERfiBoAg/EBThB4Ii/EBQhB8IivADQeWO85vZZEkvSmqV5JJWuPvvzGyJpJ9JOv+l66fc/c/1ahT10dnZmawfOHAgWe/q6krWT5w4cck9oTHKOcnnrKRfuvt2Mxsr6V0zezOr/dbdf1W/9gDUS2743b1bUnd2u8fMdku6ut6NAaivS3rPb2ZTJP1Q0l+zRY+b2U4zW2VmE0qss9DMOs0sfXwJoKHKDr+ZjZH0R0m/cPdTkpZLmiZppvqPDH59sfXcfYW7d7h7Rw36BVAjZYXfzIarP/hr3f1PkuTuR939nLv3Sfq9pFvr1yaAWssNv/VPpbpS0m53/82A5QO/DjZf0ge1bw9AvVjeNMhmNlvS/0p6X9L572c+Jekh9R/yu6T9kn6efTiYeq70xtBw06dPT9bb29uT9S+//DJZ37Zt2yX3hOq4e3ru80w5n/b/RdLFnowxfWAQ4ww/ICjCDwRF+IGgCD8QFOEHgiL8QFC54/w13Rjj/INO/zlepTXy/w/KU+44P3t+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiq0VN0/13SwGtBT8qWNaNm7a2hfV3iOH6zvmZSnN6uKfeBDT3J5zsbN+ts1mv7NWtvzdqXRG+VKqo3DvuBoAg/EFTR4V9R8PZTmrW3Zu1LordKFdJboe/5ARSn6D0/gIIUEn4zu8fM9pjZXjNbXEQPpZjZfjN738x2FD3FWDYN2jEz+2DAsivM7E0z+zj7fdFp0grqbYmZHc5eux1mdl9BvU02s/8xs11m9qGZ/Wu2vNDXLtFXIa9bww/7zaxFUpekuyUdkvSOpIfcfVdDGynBzPZL6nD3wseEzewuSb2SXnT3m7Jl/ynpc3d/NvvDOcHd/61JelsiqbfomZuzCWXaBs4sLWmepEdU4GuX6OtBFfC6FbHnv1XSXnff5+5nJK2TNLeAPpqeu2+T9PkFi+dKWp3dXq3+/zwNV6K3puDu3e6+PbvdI+n8zNKFvnaJvgpRRPivlnRwwP1Daq4pv13SFjN718wWFt3MRbQOmBnpM0mtRTZzEbkzNzfSBTNLN81rV8mM17XGB37fNdvdZ0m6V9Ki7PC2KXn/e7ZmGq4pa+bmRrnIzNL/UORrV+mM17VWRPgPS5o84P73s2VNwd0PZ7+PSdqg5pt9+Oj5SVKz38cK7ucfmmnm5ovNLK0meO2aacbrIsL/jqR2M5tqZiMk/VTSxgL6+A4zG519ECMzGy1pjppv9uGNkhZktxdIerXAXr6lWWZuLjWztAp+7Zpuxmt3b/iPpPvU/4n/J5L+vYgeSvR1raS/ZT8fFt2bpJfVfxj4jfo/G3lU0kRJWyV9LOm/JV3RRL39l/pnc96p/qC1FdTbbPUf0u+UtCP7ua/o1y7RVyGvG2f4AUHxgR8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaD+H7Uc4XAuzjuNAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "print('Counterfactual prediction: {}'.format(explanation1['cf']['class']))\n",
    "print('Closest prototype class: {}'.format(proto_class1))\n",
    "plt.imshow(explanation1['cf']['X'].reshape(28, 28))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "While the counterfactual still looks like a 9 at first sight, it is clear by looking at the difference plot between the counterfactual and the original instance that the pixels generating the circle of the 9 are being removed, moving towards a 7."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Counterfactual prediction: 7\n",
      "Closest prototype class: 7\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7fd0485354e0>"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAADtVJREFUeJzt3W+MVfWdx/HPVxz8AwSEymQQ4kCjJo1/pisxJoVNTRfiGszIE1IeVKoGakTTmiYucR8sPjAxmwLpE5sMSsClS7umbcSk7NYFgyGsjUAQBhHEBsL/P0FkEAIz+t0HczBTnfM74/137vh9v5IJd87nnnu/3viZc++cM+eYuwtAPNeUPQCAclB+ICjKDwRF+YGgKD8QFOUHgqL8QFCUHwiK8gNBXdvIJzMzDicE6szdbSj3q2rLb2YPmtk+MztgZkuqeSwAjWWVHttvZiMk7Zc0S9IRSe9Jmu/uHyTWYcsP1Fkjtvz3STrg7n9z9yuSfieps4rHA9BA1ZT/FkmHB3x/JFv2d8xskZltM7NtVTwXgBqr+y/83L1LUpfE236gmVSz5T8qacqA7ydnywAMA9WU/z1Jt5nZVDMbKenHktbXZiwA9Vbx23537zOzpyX9j6QRkla5+56aTQagrire1VfRk/GZH6i7hhzkA2D4ovxAUJQfCIryA0FRfiAoyg8ERfmBoCg/EBTlB4Ki/EBQlB8IivIDQVF+ICjKDwRF+YGgKD8QFOUHgqL8QFCUHwiK8gNBUX4gKMoPBEX5gaAoPxAU5QeCovxAUJQfCIryA0FRfiCoii/RLUlmdlBSj6TPJfW5+/RaDAWg/qoqf+YBdz9Tg8cB0EC87QeCqrb8LukvZrbdzBbVYiAAjVHt2/4Z7n7UzCZKesvMPnT3dwbeIfuhwA8GoMmYu9fmgcyWSrrg7r9K3Kc2TwYgl7vbUO5X8dt+MxtlZmOu3pY0W1J3pY8HoLGqedvfKulPZnb1cf7T3f+7JlMBqLuave0f0pPxtj+ca6/N375kG45cU6dOTeYXLlxI5seOHUvm31Z1f9sPYHij/EBQlB8IivIDQVF+ICjKDwTFrr4GaGlpSea9vb11e+6Ojo5k/uKLLybzCRMmJPMpU6Yk80mTJuVma9euTa77wgsvJPPTp08n8/b29txs//79yXUvXbqUzJsZu/oAJFF+ICjKDwRF+YGgKD8QFOUHgqL8QFDs56+Btra2ZH7zzTcn88uXLyfzUaNGJfPOzs7c7PHHH0+uW3QMwvvvv5/ML168mMznzJmTm23fvj257r59+5J50TEGY8eOzc1WrFiRXLfoGIRmxn5+AEmUHwiK8gNBUX4gKMoPBEX5gaAoPxBULa7SG15PT08yv+GGG5L57bffnsyfeuqpZJ7al15k7969yfzll19O5kX7+bu6unKzomMQvvjii2RedK6CQ4cO5WbHjx9PrhsBW34gKMoPBEX5gaAoPxAU5QeCovxAUJQfCKpwP7+ZrZI0R9Ipd78zWzZe0u8ltUs6KGmeu39SvzHLd801+T8niy4VfccddyTz+++/P5nPnDkzmZ84cSI36+7uTq67ZMmSZL5jx45kXo2iYwSWL1+ezMeNG5fMU8cYfPjhh8l1IxjKln+1pAe/smyJpI3ufpukjdn3AIaRwvK7+zuSzn5lcaekNdntNZIeqfFcAOqs0s/8re5+9fjIE5JaazQPgAap+th+d/fUufnMbJGkRdU+D4DaqnTLf9LM2iQp+/dU3h3dvcvdp7v79AqfC0AdVFr+9ZIWZLcXSHqjNuMAaJTC8pvZOkn/J+kOMztiZk9IeknSLDP7SNI/Zd8DGEYKP/O7+/yc6Ec1nqWpFf1teUrR+emXLVuWzIv2Z2/ZsiU3e/bZZ5PrHjhwIJnX09SpU5P5xIkTk3nR3+Rv2LAhNzt9+nRy3Qg4wg8IivIDQVF+ICjKDwRF+YGgKD8QFKfuboBXXnklmRf9yW/qFNSS9Nhjj+VmZ86cSa574403JvOiP7stMmPGjNxs/vy8vchDU3Ta8c2bN1f1+N92bPmBoCg/EBTlB4Ki/EBQlB8IivIDQVF+ICj289dAe3t7Mi+6BHfRn9W+/vrrybyvry+Zp1S7H7+lpSWZP/zww7nZrFmzqnru5557rqr1o2PLDwRF+YGgKD8QFOUHgqL8QFCUHwiK8gNBsZ+/BlKX75aksWPHJvN169Yl8+uvvz6Zjxw5MjcbNWpUct3PPvssmRcdw7B48eJk/uSTT+ZmZpZct+j4h9SpuVGMLT8QFOUHgqL8QFCUHwiK8gNBUX4gKMoPBFW4n9/MVkmaI+mUu9+ZLVsqaaGkq9c5ft7d/1yvIZvd2bNnk/mePXuSeVtbWzJfuXJlMr/nnntys6Lz8hedO/+uu+5K5kXHCaQube7uyXW3bt1a8WOj2FC2/KslPTjI8hXu3pF9hS0+MFwVlt/d35GU3rQBGHaq+cz/tJntMrNVZnZTzSYC0BCVlv83kr4rqUPScUnL8u5oZovMbJuZbavwuQDUQUXld/eT7v65u38haaWk+xL37XL36e4+vdIhAdReReU3s4G/np4rqbs24wBolKHs6lsn6YeSvmNmRyT9m6QfmlmHJJd0UNLP6jgjgDooLL+7D7Yj+NU6zDJsnTt3LpkvXLgwmc+dOzeZHz16NJmnzifwwAMPJNc9duxYMu/t7U3mRf/t1113XW42efLk5LqrV69O5ufPn0/mSOMIPyAoyg8ERfmBoCg/EBTlB4Ki/EBQnLq7AYr+7HXt2rVVPf4nn3ySm02bNi257v79+5P5pk2bkvmKFSuS+d13352bbd68ObludzfHjtUTW34gKMoPBEX5gaAoPxAU5QeCovxAUJQfCIr9/N9yGzduTOaXL19O5uPHj0/m8+bNS+ap03MfPny44nVRPbb8QFCUHwiK8gNBUX4gKMoPBEX5gaAoPxAU+/m/5VpbW5N50b72jo6OZD5hwoRk3tPTk5sVnQvgzJkzyRzVYcsPBEX5gaAoPxAU5QeCovxAUJQfCIryA0EV7uc3symSXpPUKskldbn7r81svKTfS2qXdFDSPHfPP4E8SlF0ee+iv9d/5plnqnr+N998MzfbsWNHVY+N6gxly98n6Zfu/j1J90tabGbfk7RE0kZ3v03Sxux7AMNEYfnd/bi778hu90jaK+kWSZ2S1mR3WyPpkXoNCaD2vtFnfjNrl/R9SX+V1Orux7PohPo/FgAYJoZ8bL+ZjZb0B0m/cPfzZvZl5u5uZoOecM3MFklaVO2gAGprSFt+M2tRf/F/6+5/zBafNLO2LG+TdGqwdd29y92nu/v0WgwMoDYKy2/9m/hXJe119+UDovWSFmS3F0h6o/bjAaiXobzt/4Gkn0jabWY7s2XPS3pJ0n+Z2ROSDklKn8MZuVpaWpJ5b29vxY89e/bsZF50euyZM2cm8yNHjiTzDRs2JHOUp7D87r5FkuXEP6rtOAAahSP8gKAoPxAU5QeCovxAUJQfCIryA0Fx6u4mUM1+/CJvv/12Mu/s7EzmY8aMSeabNm1K5rt3707mKA9bfiAoyg8ERfmBoCg/EBTlB4Ki/EBQlB8Iiv38TWDkyJHJfNy4ccn81KlBT6IkSTp//nxy3YkTJybzvr6+ZH7ixIlkvmvXrtxs7NixyXU//fTTZI7qsOUHgqL8QFCUHwiK8gNBUX4gKMoPBEX5gaDYz98Erly5kswvXbqUzFPn/Z82bVpy3UcffTSZjxgxIpl//PHHyXzSpEm52blz55Lror7Y8gNBUX4gKMoPBEX5gaAoPxAU5QeCovxAUIX7+c1siqTXJLVKckld7v5rM1sqaaGk09ldn3f3P9dr0MhuvfXWZD569Ojc7N13302uu3Xr1opmuurixYvJPHU+gaJjCFBfQznIp0/SL919h5mNkbTdzN7KshXu/qv6jQegXgrL7+7HJR3PbveY2V5Jt9R7MAD19Y0+85tZu6TvS/prtuhpM9tlZqvM7KacdRaZ2TYz21bVpABqasjlN7PRkv4g6Rfufl7SbyR9V1KH+t8ZLBtsPXfvcvfp7j69BvMCqJEhld/MWtRf/N+6+x8lyd1Puvvn7v6FpJWS7qvfmABqrbD8ZmaSXpW0192XD1jeNuBucyV11348APUylN/2/0DSTyTtNrOd2bLnJc03sw717/47KOlndZkQ6u6u38/Vol199957bzIvmq2np+cbz4TGGMpv+7dIskEi9ukDwxhH+AFBUX4gKMoPBEX5gaAoPxAU5QeCMndv3JOZNe7JUBPt7e3J/ODBgw2ZA0Pn7oPtmv8atvxAUJQfCIryA0FRfiAoyg8ERfmBoCg/EFSj9/OflnRowKLvSDrTsAG+mWadrVnnkpitUrWc7VZ3v3kod2xo+b/25GbbmvXcfs06W7POJTFbpcqajbf9QFCUHwiq7PJ3lfz8Kc06W7POJTFbpUqZrdTP/ADKU/aWH0BJSim/mT1oZvvM7ICZLSljhjxmdtDMdpvZzrIvMZZdBu2UmXUPWDbezN4ys4+yfwe9TFpJsy01s6PZa7fTzB4qabYpZva2mX1gZnvM7OfZ8lJfu8RcpbxuDX/bb2YjJO2XNEvSEUnvSZrv7h80dJAcZnZQ0nR3L32fsJn9o6QLkl5z9zuzZf8u6ay7v5T94LzJ3f+lSWZbKulC2Vduzi4o0zbwytKSHpH0U5X42iXmmqcSXrcytvz3STrg7n9z9yuSfieps4Q5mp67vyPp7FcWd0pak91eo/7/eRouZ7am4O7H3X1HdrtH0tUrS5f62iXmKkUZ5b9F0uEB3x9Rc13y2yX9xcy2m9misocZRGt22XRJOiGptcxhBlF45eZG+sqVpZvmtavkite1xi/8vm6Gu/+DpH+WtDh7e9uUvP8zWzPtrhnSlZsbZZArS3+pzNeu0ite11oZ5T8qacqA7ydny5qCux/N/j0l6U9qvqsPn7x6kdTs31Mlz/OlZrpy82BXllYTvHbNdMXrMsr/nqTbzGyqmY2U9GNJ60uY42vMbFT2ixiZ2ShJs9V8Vx9eL2lBdnuBpDdKnOXvNMuVm/OuLK2SX7umu+K1uzf8S9JD6v+N/8eS/rWMGXLmmibp/exrT9mzSVqn/reBver/3cgTkiZI2ijpI0n/K2l8E832H5J2S9ql/qK1lTTbDPW/pd8laWf29VDZr11irlJeN47wA4LiF35AUJQfCIryA0FRfiAoyg8ERfmBoCg/EBTlB4L6f5TdwP1cfugZAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "print('Counterfactual prediction: {}'.format(explanation2['cf']['class']))\n",
    "print('Closest prototype class: {}'.format(proto_class2))\n",
    "plt.imshow(explanation2['cf']['X'].reshape(28, 28))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7fd0493b3c88>"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAADrBJREFUeJzt3W+MVfWdx/HPFxjEAUZgGMcRiHaJVo1aaia4pmbtptvGmibYJ6Y8aGg0pQ9q0iY1WcM+qI+M2Wzb+GDThK6kuOlaNrZGHpjdUrKJabKpjsAOWEEoDuHPwAygDEWBGea7D+bYjDj3d673nHvPHb7vVzKZe8/3nnu+XPhw7r2/e38/c3cBiGdO1Q0AqAbhB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8Q1LxWHqyrq8t7enpaeUgglNHRUY2NjVk9ty0UfjN7WNLzkuZK+jd3fy51+56eHj377LNFDgkgYdOmTXXftuGn/WY2V9K/Svq6pLskrTezuxq9PwCtVeQ1/1pJh9z9sLtflvRrSevKaQtAsxUJ/wpJR6ddP5Zt+wQz22hmA2Y2MDY2VuBwAMrU9Hf73X2zu/e7e39XV1ezDwegTkXCf1zSqmnXV2bbAMwCRcL/pqTbzOxzZjZf0rckbS+nLQDN1vBQn7tPmNmTkv5bU0N9W9z97dI6A9BUhcb53f01Sa+V1AuAFuLjvUBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRVaJVeMxuSdF7SFUkT7t5fRlMAmq9Q+DN/7+6nS7gfAC3E034gqKLhd0m/M7O3zGxjGQ0BaI2iT/sfdPfjZnajpB1mtt/dX59+g+w/hY2StHz58oKHA1CWQmd+dz+e/R6R9IqktTPcZrO797t7f1dXV5HDAShRw+E3s4Vmtvjjy5K+JmlfWY0BaK4iT/t7Jb1iZh/fz3+4+3+V0hWApms4/O5+WNIXSuwF16C5c+fWrGUnjpp6e3uT9YsXLybrZ86cSdajY6gPCIrwA0ERfiAowg8ERfiBoAg/EFQZ3+pDjo6OjmR9fHy8RZ18dgcOHEjW8z6y3d3dXbOW9+c+ceJEsp4aRpSk0dHRmrWenp7kvhFw5geCIvxAUIQfCIrwA0ERfiAowg8ERfiBoBjnL8GyZcuS9RtuuCFZv3z5crI+f/78ZL2zs7Nmbd++9PwqY2NjyfpHH32UrL/77rvJ+v3331+z9t577yX3vf3225P1iYmJZD31Z2OcnzM/EBbhB4Ii/EBQhB8IivADQRF+ICjCDwTFOH8J8sbCr7vuumR9xYoVyfrq1auT9W3bttWs5U2PPW9e+p/AmjVrkvU8qftfv359ct/Jyclkfffu3cl63ucjouPMDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANB5Y7zm9kWSd+QNOLud2fblknaJulWSUOSHnP395vXZvXmzKn9/2TeOP/KlSuT9euvvz5Zf/nll5P199+v/dAvXbo0ue/ChQuT9aJS37k/efJkct/jx48XOvaqVatq1vLWGzh9+nShY88G9Zz5fynp4au2PS1pp7vfJmlndh3ALJIbfnd/XdLZqzavk7Q1u7xV0qMl9wWgyRp9zd/r7sPZ5ZOSekvqB0CLFH7Dz91dkteqm9lGMxsws4G8+eIAtE6j4T9lZn2SlP0eqXVDd9/s7v3u3t/V1dXg4QCUrdHwb5e0Ibu8QdKr5bQDoFVyw29mL0n6X0mfN7NjZvaEpOckfdXMDkr6h+w6gFkkd5zf3Wt96forJffS1vK+W55y8ODBZD1vTPnKlSvJet5Yfru6ePFisn7mzJlkPW+ugtR6CufOnUvuGwGf8AOCIvxAUIQfCIrwA0ERfiAowg8ExdTdbeDw4cPJ+ty5c5P1qU9YzyxvefAFCxYk63nDcXnuuOOOmrW9e/cWuu8lS5Yk66mhwPHx8ULHvhZw5geCIvxAUIQfCIrwA0ERfiAowg8ERfiBoBjnr1NqzLi3Nz2FYd44ft4U1oODg8n6448/XrN24cKF5L5Fx/Hzvla7f//+mrWOjo7kvt3d3cl66iu7knT27NXzzmI6zvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBTj/HVKfWc+b6y7s7MzWX/jjTeS9bzv5KfGy/OW/85bXjxP6nGRpEuXLtWs5c0lkNpXYhy/KM78QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxBU7ji/mW2R9A1JI+5+d7btGUnflTSa3WyTu7/WrCbb3djYWKH9876XvmPHjmQ9Nd49f/785L4PPfRQsp7X24kTJ5L1lStX1qzlfUYgr/eFCxcm63lzGURXz5n/l5IenmH7z9x9TfYTNvjAbJUbfnd/XRIfpQKuMUVe8z9pZoNmtsXMlpbWEYCWaDT8P5e0WtIaScOSflLrhma20cwGzGyg6GtjAOVpKPzufsrdr7j7pKRfSFqbuO1md+939/6urq5G+wRQsobCb2Z9065+U9K+ctoB0Cr1DPW9JOnLkpab2TFJP5b0ZTNbI8klDUn6XhN7BNAEueF39/UzbH6hCb3MWnnjyXPmpJ9g5b0cGhkZSdZ3795ds3bvvfcm9921a1eynjeXwFNPPZWsDw8P16wdOHAguW/evP2M4xfDJ/yAoAg/EBThB4Ii/EBQhB8IivADQTF1dwtMTk4m60W/Vnvw4MGatQceeCC5b19fX7Ke97XagYGBZD1lfHw8Wc+buhvFcOYHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAY558F7rnnnobrecuD5421L1q0KFk/cuRIsr548eKataVL01M/5n2dOG9auLypwaPjzA8ERfiBoAg/EBThB4Ii/EBQhB8IivADQTHOf41bsmRJsn769OlkPW8Z7NQ4flHnzp1r2n2DMz8QFuEHgiL8QFCEHwiK8ANBEX4gKMIPBJU7zm9mqyS9KKlXkkva7O7Pm9kySdsk3SppSNJj7v5+81pFI86cOZOs531f//z582W2gzZSz5l/QtKP3P0uSX8r6ftmdpekpyXtdPfbJO3MrgOYJXLD7+7D7r4ru3xe0juSVkhaJ2lrdrOtkh5tVpMAyveZXvOb2a2Svijpj5J63X04K53U1MsCALNE3eE3s0WSfiPph+7+icnTfGqytBknTDOzjWY2YGYDeXOuAWidusJvZh2aCv6v3P232eZTZtaX1fskjcy0r7tvdvd+d+/v6uoqo2cAJcgNv5mZpBckvePuP51W2i5pQ3Z5g6RXy28PQLPU85XeL0n6tqS9ZrYn27ZJ0nOS/tPMnpB0RNJjzWnx2jdvXvqvYWJiouH7vu+++5L1oaGhZH14eDhZzxtK7O7uTtZRndzwu/sfJFmN8lfKbQdAq/AJPyAowg8ERfiBoAg/EBThB4Ii/EBQTN3dBoqM4+cZHBxM1i9cuFDo/g8dOpSsX7p0qWbt5ptvLnRsFMOZHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCYpy/DXR0dCTrectkf/DBBzVrRcfxJycnk/W82ZlWrFhRs9bZ2Znct2jvSOPMDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBMc7fBsbHx5P1y5cvJ+upef/7+vqS+x49ejRZnzMnfX648847k/XUvP2M41eLMz8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBJU7zm9mqyS9KKlXkkva7O7Pm9kzkr4raTS76SZ3f61ZjUZ24403JusLFiyoWdu/f39y3w8//DBZHx0dTdbzPkeQuv+8zxCguer5kM+EpB+5+y4zWyzpLTPbkdV+5u7/0rz2ADRLbvjdfVjScHb5vJm9I6n29CwAZoXP9LzLzG6V9EVJf8w2PWlmg2a2xcyW1thno5kNmNnA2NhYoWYBlKfu8JvZIkm/kfRDdx+T9HNJqyWt0dQzg5/MtJ+7b3b3fnfvz5vvDUDr1BV+M+vQVPB/5e6/lSR3P+XuV9x9UtIvJK1tXpsAypYbfjMzSS9Iesfdfzpt+/S3eb8paV/57QFolnre7f+SpG9L2mtme7JtmyStN7M1mhr+G5L0vaZ0CA0NDTXtvvOmz77lllsK3X/eUCKqU8+7/X+QZDOUGNMHZjE+ZQEERfiBoAg/EBThB4Ii/EBQhB8Iiqm7kXTTTTcl6ydPnmxRJygbZ34gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCMrcvXUHMxuVdGTapuWSTresgc+mXXtr174kemtUmb3d4u499dywpeH/1MHNBty9v7IGEtq1t3btS6K3RlXVG0/7gaAIPxBU1eHfXPHxU9q1t3btS6K3RlXSW6Wv+QFUp+ozP4CKVBJ+M3vYzA6Y2SEze7qKHmoxsyEz22tme8xsoOJetpjZiJntm7ZtmZntMLOD2e8Zl0mrqLdnzOx49tjtMbNHKuptlZn9j5n9yczeNrMfZNsrfewSfVXyuLX8ab+ZzZX0rqSvSjom6U1J6939Ty1tpAYzG5LU7+6Vjwmb2d9J+oukF9397mzbP0s66+7PZf9xLnX3f2yT3p6R9JeqV27OFpTpm76ytKRHJX1HFT52ib4eUwWPWxVn/rWSDrn7YXe/LOnXktZV0Efbc/fXJZ29avM6SVuzy1s19Y+n5Wr01hbcfdjdd2WXz0v6eGXpSh+7RF+VqCL8KyQdnXb9mNpryW+X9Dsze8vMNlbdzAx6s2XTJemkpN4qm5lB7srNrXTVytJt89g1suJ12XjD79MedPf7JH1d0vezp7dtyades7XTcE1dKze3ygwrS/9VlY9doytel62K8B+XtGra9ZXZtrbg7sez3yOSXlH7rT586uNFUrPfIxX381fttHLzTCtLqw0eu3Za8bqK8L8p6TYz+5yZzZf0LUnbK+jjU8xsYfZGjMxsoaSvqf1WH94uaUN2eYOkVyvs5RPaZeXmWitLq+LHru1WvHb3lv9IekRT7/j/WdI/VdFDjb7+RtL/ZT9vV92bpJc09TRwXFPvjTwhqVvSTkkHJf1e0rI26u3fJe2VNKipoPVV1NuDmnpKPyhpT/bzSNWPXaKvSh43PuEHBMUbfkBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgvp/WSGfmQ69p0UAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow((explanation2['cf']['X'] - X).reshape(28, 28))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Speed up the counterfactual search by removing the predict function loss term"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also remove the prediction loss term and still obtain an interpretable counterfactual. This is especially relevant for fully black box models. When we provide the counterfactual search method with a Keras or TensorFlow model, it is incorporated in the TensorFlow graph and evaluated using automatic differentiation. However, if we only have access to the model's predict function, the gradient updates are numerical and typically require a large number of prediction calls because of the predict loss term. These prediction calls can slow the search down significantly and become a bottleneck. We can represent the gradient of the loss term as follows:\n",
    "\n",
    "\\begin{equation*} \\frac{\\partial L_{pred}}{\\partial x} = \\frac{\\partial L_{pred}}{\\partial p} \\frac{\\partial p}{\\partial x} \\end{equation*}\n",
    "\n",
    "where $L_{pred}$ is the prediction loss term, $p$ the predict function and $x$ the input features to optimize. For a 28 by 28 MNIST image, the $^{\\delta p}/_{\\delta x}$ term alone would require a prediction call with batch size 28x28x2 = 1568. By using the prototypes to guide the search however, we can remove the prediction loss term and only make a single prediction at the end of each gradient update to check whether the predicted class on the proposed counterfactual is different from the original class. We do not necessarily need a Keras or TensorFlow auto-encoder either and can use k-d trees to find the nearest class prototypes. Please check out [this notebook](./cfproto_housing.ipynb) for a practical example.\n",
    "\n",
    "We'll first illustrate this method by using a Keras model and then try a black box model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7fd04d26f128>"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAADO5JREFUeJzt3V2IXfW5x/Hf76QpiOlFYjUMNpqeogerSKKjCMYS9VhyYiEWg9SLkkLJ9CJKCyVU7EVzWaQv1JvAlIbGkmMrpNUoYmNjMQ1qcSJqEmNiElIzMW9lhCaCtNGnF7Nsp3H2f+/st7XH5/uBYfZez3p52Mxv1lp77bX/jggByOe/6m4AQD0IP5AU4QeSIvxAUoQfSIrwA0kRfiApwg8kRfiBpD7Vz43Z5uOEQI9FhFuZr6M9v+1ltvfZPmD7gU7WBaC/3O5n+23PkrRf0h2SxiW9LOneiHijsAx7fqDH+rHnv1HSgYg4FBF/l/RrSSs6WB+APuok/JdKOjLl+Xg17T/YHrE9Znusg20B6LKev+EXEaOSRiUO+4FB0sme/6ikBVOef66aBmAG6CT8L0u6wvbnbX9a0tckbelOWwB6re3D/og4a/s+Sb+XNEvShojY07XOAPRU25f62toY5/xAz/XlQz4AZi7CDyRF+IGkCD+QFOEHkiL8QFKEH0iK8ANJEX4gKcIPJEX4gaQIP5AU4QeSIvxAUoQfSIrwA0kRfiApwg8kRfiBpAg/kBThB5Ii/EBShB9IivADSRF+ICnCDyRF+IGkCD+QFOEHkmp7iG5Jsn1Y0mlJH0g6GxHD3WgKQO91FP7KrRHx1y6sB0AfcdgPJNVp+EPSVts7bY90oyEA/dHpYf+SiDhq+xJJz9p+MyK2T52h+qfAPwZgwDgiurMie52kMxHxo8I83dkYgIYiwq3M1/Zhv+0LbX/mo8eSvixpd7vrA9BfnRz2z5f0O9sfref/I+KZrnQFoOe6dtjf0sY47Ad6rueH/QBmNsIPJEX4gaQIP5AU4QeSIvxAUt24qy+FlStXNqytXr26uOw777xTrL///vvF+qZNm4r148ePN6wdOHCguCzyYs8PJEX4gaQIP5AU4QeSIvxAUoQfSIrwA0lxS2+LDh061LC2cOHC/jUyjdOnTzes7dmzp4+dDJbx8fGGtYceeqi47NjYWLfb6Rtu6QVQRPiBpAg/kBThB5Ii/EBShB9IivADSXE/f4tK9+xfe+21xWX37t1brF911VXF+nXXXVesL126tGHtpptuKi575MiRYn3BggXFeifOnj1brJ86dapYHxoaanvbb7/9drE+k6/zt4o9P5AU4QeSIvxAUoQfSIrwA0kRfiApwg8k1fR+ftsbJH1F0smIuKaaNk/SbyQtlHRY0j0R8W7Tjc3g+/kH2dy5cxvWFi1aVFx2586dxfoNN9zQVk+taDZewf79+4v1Zp+fmDdvXsPamjVrisuuX7++WB9k3byf/5eSlp0z7QFJ2yLiCknbqucAZpCm4Y+I7ZImzpm8QtLG6vFGSXd1uS8APdbuOf/8iDhWPT4uaX6X+gHQJx1/tj8ionQub3tE0kin2wHQXe3u+U/YHpKk6vfJRjNGxGhEDEfEcJvbAtAD7YZ/i6RV1eNVkp7oTjsA+qVp+G0/KulFSf9je9z2NyX9UNIdtt+S9L/VcwAzCN/bj4F19913F+uPPfZYsb579+6GtVtvvbW47MTEuRe4Zg6+tx9AEeEHkiL8QFKEH0iK8ANJEX4gKS71oTaXXHJJsb5r166Oll+5cmXD2ubNm4vLzmRc6gNQRPiBpAg/kBThB5Ii/EBShB9IivADSTFEN2rT7OuzL7744mL93XfL3xa/b9++8+4pE/b8QFKEH0iK8ANJEX4gKcIPJEX4gaQIP5AU9/Ojp26++eaGteeee6647OzZs4v1pUuXFuvbt28v1j+puJ8fQBHhB5Ii/EBShB9IivADSRF+ICnCDyTV9H5+2xskfUXSyYi4ppq2TtJqSaeq2R6MiKd71SRmruXLlzesNbuOv23btmL9xRdfbKsnTGplz/9LScummf7TiFhU/RB8YIZpGv6I2C5pog+9AOijTs7577P9uu0Ntud2rSMAfdFu+NdL+oKkRZKOSfpxoxltj9gesz3W5rYA9EBb4Y+IExHxQUR8KOnnkm4szDsaEcMRMdxukwC6r63w2x6a8vSrknZ3px0A/dLKpb5HJS2V9Fnb45J+IGmp7UWSQtJhSd/qYY8AeoD7+dGRCy64oFjfsWNHw9rVV19dXPa2224r1l944YViPSvu5wdQRPiBpAg/kBThB5Ii/EBShB9IiiG60ZG1a9cW64sXL25Ye+aZZ4rLcimvt9jzA0kRfiApwg8kRfiBpAg/kBThB5Ii/EBS3NKLojvvvLNYf/zxx4v19957r2Ft2bLpvhT631566aViHdPjll4ARYQfSIrwA0kRfiApwg8kRfiBpAg/kBT38yd30UUXFesPP/xwsT5r1qxi/emnGw/gzHX8erHnB5Ii/EBShB9IivADSRF+ICnCDyRF+IGkmt7Pb3uBpEckzZcUkkYj4me250n6jaSFkg5Luici3m2yLu7n77Nm1+GbXWu//vrri/WDBw8W66V79psti/Z0837+s5K+GxFflHSTpDW2vyjpAUnbIuIKSduq5wBmiKbhj4hjEfFK9fi0pL2SLpW0QtLGaraNku7qVZMAuu+8zvltL5S0WNKfJc2PiGNV6bgmTwsAzBAtf7bf9hxJmyV9JyL+Zv/7tCIiotH5vO0RSSOdNgqgu1ra89uercngb4qI31aTT9gequpDkk5Ot2xEjEbEcEQMd6NhAN3RNPye3MX/QtLeiPjJlNIWSauqx6skPdH99gD0SiuX+pZI+pOkXZI+rCY/qMnz/sckXSbpL5q81DfRZF1c6uuzK6+8slh/8803O1r/ihUrivUnn3yyo/Xj/LV6qa/pOX9E7JDUaGW3n09TAAYHn/ADkiL8QFKEH0iK8ANJEX4gKcIPJMVXd38CXH755Q1rW7du7Wjda9euLdafeuqpjtaP+rDnB5Ii/EBShB9IivADSRF+ICnCDyRF+IGkuM7/CTAy0vhb0i677LKO1v38888X682+DwKDiz0/kBThB5Ii/EBShB9IivADSRF+ICnCDyTFdf4ZYMmSJcX6/fff36dO8EnCnh9IivADSRF+ICnCDyRF+IGkCD+QFOEHkmp6nd/2AkmPSJovKSSNRsTPbK+TtFrSqWrWByPi6V41mtktt9xSrM+ZM6ftdR88eLBYP3PmTNvrxmBr5UM+ZyV9NyJesf0ZSTttP1vVfhoRP+pdewB6pWn4I+KYpGPV49O290q6tNeNAeit8zrnt71Q0mJJf64m3Wf7ddsbbM9tsMyI7THbYx11CqCrWg6/7TmSNkv6TkT8TdJ6SV+QtEiTRwY/nm65iBiNiOGIGO5CvwC6pKXw256tyeBviojfSlJEnIiIDyLiQ0k/l3Rj79oE0G1Nw2/bkn4haW9E/GTK9KEps31V0u7utwegV1p5t/9mSV+XtMv2q9W0ByXda3uRJi//HZb0rZ50iI689tprxfrtt99erE9MTHSzHQyQVt7t3yHJ05S4pg/MYHzCD0iK8ANJEX4gKcIPJEX4gaQIP5CU+znEsm3GcwZ6LCKmuzT/Mez5gaQIP5AU4QeSIvxAUoQfSIrwA0kRfiCpfg/R/VdJf5ny/LPVtEE0qL0Nal8SvbWrm71d3uqMff2Qz8c2bo8N6nf7DWpvg9qXRG/tqqs3DvuBpAg/kFTd4R+tefslg9rboPYl0Vu7aumt1nN+APWpe88PoCa1hN/2Mtv7bB+w/UAdPTRi+7DtXbZfrXuIsWoYtJO2d0+ZNs/2s7bfqn5PO0xaTb2ts320eu1etb28pt4W2P6j7Tds77H97Wp6ra9doa9aXre+H/bbniVpv6Q7JI1LelnSvRHxRl8bacD2YUnDEVH7NWHbX5J0RtIjEXFNNe0hSRMR8cPqH+fciPjegPS2TtKZukdurgaUGZo6srSkuyR9QzW+doW+7lENr1sde/4bJR2IiEMR8XdJv5a0ooY+Bl5EbJd07qgZKyRtrB5v1OQfT9816G0gRMSxiHilenxa0kcjS9f62hX6qkUd4b9U0pEpz8c1WEN+h6SttnfaHqm7mWnMr4ZNl6TjkubX2cw0mo7c3E/njCw9MK9dOyNedxtv+H3ckoi4TtL/SVpTHd4OpJg8ZxukyzUtjdzcL9OMLP0vdb527Y543W11hP+opAVTnn+umjYQIuJo9fukpN9p8EYfPvHRIKnV75M19/MvgzRy83QjS2sAXrtBGvG6jvC/LOkK25+3/WlJX5O0pYY+Psb2hdUbMbJ9oaQva/BGH94iaVX1eJWkJ2rs5T8MysjNjUaWVs2v3cCNeB0Rff+RtFyT7/gflPT9Onpo0Nd/S3qt+tlTd2+SHtXkYeA/NPneyDclXSRpm6S3JP1B0rwB6u1XknZJel2TQRuqqbclmjykf13Sq9XP8rpfu0JftbxufMIPSIo3/ICkCD+QFOEHkiL8QFKEH0iK8ANJEX4gKcIPJPVP82g/p9/JjhUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "X = x_test[0].reshape((1,) + x_test[0].shape)\n",
    "plt.imshow(X.reshape(28, 28))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "c_init = 0.  # set weight predict loss term to 0\n",
    "c_steps = 1  # do not update c further\n",
    "max_iterations = 2000\n",
    "theta = 100.\n",
    "gamma = 100."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "# set random seed\n",
    "np.random.seed(1)\n",
    "tf.set_random_seed(1)\n",
    "\n",
    "# define models\n",
    "cnn = load_model('mnist_cnn.h5')\n",
    "ae = load_model('mnist_ae.h5')\n",
    "enc = load_model('mnist_enc.h5')\n",
    "\n",
    "sess = K.get_session()\n",
    "\n",
    "# initialize explainer, fit and generate counterfactual\n",
    "cf = CounterFactualProto(sess, cnn, shape, gamma=gamma, theta=theta,\n",
    "                         ae_model=ae, enc_model=enc, max_iterations=max_iterations,\n",
    "                         feature_range=feature_range, c_init=c_init, c_steps=c_steps)\n",
    "cf.fit(x_train)  # find class prototypes\n",
    "explanation = cf.explain(X)\n",
    "\n",
    "sess.close()\n",
    "K.clear_session()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Still get a reasonable 9:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Counterfactual prediction: 9\n",
      "Closest prototype class: 9\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7fd04d689438>"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAEDFJREFUeJzt3W+MVfWdx/HP19GCjkZAEZGO4qpZUaKwTMA/ZKMWG1ZrlCekPmho0kj9g7Hqg0VWWNQnzbqlSjQmNDXCpqslsURNRFTEsA1iRIOIuCqSaYDADAooJWKd4bsP5tgdZc73jPffuTO/9yshc+d+7pn7y4UP59455/x+5u4CkJ7jyh4AgHJQfiBRlB9IFOUHEkX5gURRfiBRlB9IFOUHEkX5gUQd38gnMzNOJwTqzN1tII+ras9vZjPN7EMz225m86v5WQAayyo9t9/MWiR9JOlaSbskvSXpZnffFmzDnh+os0bs+adK2u7uO9z9b5KekXRjFT8PQANVU/5xknb2+X5Xdt+3mNlcM9tkZpuqeC4ANVb3X/i5+zJJyyTe9gPNpJo9/25JbX2+/2F2H4BBoJryvyXpAjM718x+IOmnkp6vzbAA1FvFb/vdvdvM5klaI6lF0pPu/n7NRgagrio+1FfRk/GZH6i7hpzkA2DwovxAoig/kCjKDySK8gOJovxAohp6PT+GnpaWljDv6elp0EjwfbHnBxJF+YFEUX4gUZQfSBTlBxJF+YFEcahvCLj66qtzs66urnDb99+Pr8IeNWpUmB8+fDjMOdTXvNjzA4mi/ECiKD+QKMoPJIryA4mi/ECiKD+QKGbvRWnM4klmG/lvcyhh9l4AIcoPJIryA4mi/ECiKD+QKMoPJIryA4mq6np+M+uQdEhSj6Rud2+vxaDw/YwfPz43u/POO8NtFy1aFObd3d1hft5554X5tm3bcjOO45erFpN5XO3un9bg5wBoIN72A4mqtvwu6WUze9vM5tZiQAAao9q3/dPdfbeZnSHpFTP7X3df3/cB2X8K/McANJmq9vzuvjv72iVplaSp/Txmmbu388tAoLlUXH4zazWzU765LenHkrbWamAA6quat/1jJK3KLss8XtJ/u/tLNRkVgLqruPzuvkPSpTUcy5DV2toa5uecc06Yn3TSSWE+b9683Ozkk08Oty0a28UXXxzmn3/+eZiXafjw4bnZkSNHGjiS5sShPiBRlB9IFOUHEkX5gURRfiBRlB9IFEt018C0adPC/M033wzzs846K8xfffXVMH/33Xdzs6IluIuW8L777rvD/PTTTw/zhQsX5mZTpkwJt12/fn2YX3ppfKT5ww8/zM041MeeH0gW5QcSRfmBRFF+IFGUH0gU5QcSRfmBRLFE9wDNmDEjN7vooovCbZcuXRrmw4YNC/NNmzaF+cSJE8O8GmvWrAnzo0ePhvmIESNyszPOOCPctugchGhacEn65JNPcrMVK1aE2+7evTvMmxlLdAMIUX4gUZQfSBTlBxJF+YFEUX4gUZQfSBTH+TMjR44M8+hY/tSpxyxU9C179+4N8/PPPz/MH3zwwTCvxuLFi8O8aC6C22+/PcyjY/XROQBS8bTiM2fODPOtW/PXkLnkkkvCbSdPnhzmO3fuDPN9+/aFeT1xnB9AiPIDiaL8QKIoP5Aoyg8kivIDiaL8QKIK5+03sycl/URSl7tPzO4bJemPksZL6pA0290P1G+Y1Rs/fnyYf/bZZ2He2dmZm40ePTrctqOjI8wfeuihMC+67v344/P/GleuXBluWzTv/tdffx3mjz76aJhH50eceOKJ4bYvvvhimBedXxHNk3DrrbeG2xZdz//FF1+EeZnH+QdqIHv+pyR992yK+ZLWuvsFktZm3wMYRArL7+7rJe3/zt03Slqe3V4u6aYajwtAnVX6mX+Mu+/Jbu+VNKZG4wHQIFWv1efuHp2zb2ZzJc2t9nkA1Fale/5OMxsrSdnX3Ks33H2Zu7e7e3uFzwWgDiot//OS5mS350h6rjbDAdAoheU3s6clvSHpH81sl5n9QtKvJV1rZh9LmpF9D2AQKfzM7+4350Q/qvFY6qromvqia8uvuuqq3Ozll18Ot73tttvC/KOPPgrzm26KD6ZEx9rXrVsXbtveHn8aK1ozoMjatWtzsylTpoTbTpgwIcyL/s6i8wSeeOKJcNui1+XIkSNhPhhwhh+QKMoPJIryA4mi/ECiKD+QKMoPJIqpu2ug6JDVkiVLwrytrS3Mzz333DB/7LHHcrOiQ5xFy1yvWrUqzC+77LIwP/vss3Oz7du3h9s+9dRTYX7qqaeG+T333JObPfvss+G2RU477bQwL7pEvJ6YuhtAiPIDiaL8QKIoP5Aoyg8kivIDiaL8QKKqnsZrsCiaovrTTz8N82j67AMH4lnLX3/99TCfNm1amN9xxx1hvnr16jCPFE1hfcopp4T5xo0bw/zhhx/OzVpaWsJtJ06cGOZFlyu/9tprYV6Nw4cP1+1nNwp7fiBRlB9IFOUHEkX5gURRfiBRlB9IFOUHEpXMcf6i4/hFenp6crMdO3aE206ePDnMly9fHuZFx/Hvuuuu3KxoCW2z+NLvou2LrqmfNGlSbha9ppLU3d0d5g888ECYF51/UQ2m7gYwaFF+IFGUH0gU5QcSRfmBRFF+IFGUH0hU4XF+M3tS0k8kdbn7xOy+xZJukbQve9gCd89fD3kIGD58eMXbvvDCC2FedLy7yPz583OzRx55JNx25cqVYb5o0aIwj47jS9IJJ5yQm11//fXhts8880yYb9myJcwRG8ie/ylJM/u5/7fuPin7M6SLDwxFheV39/WS9jdgLAAaqJrP/PPMbIuZPWlmI2s2IgANUWn5n5B0nqRJkvZI+k3eA81srpltMrNNFT4XgDqoqPzu3unuPe5+VNLvJE0NHrvM3dvdvb3SQQKovYrKb2Zj+3w7S9LW2gwHQKMM5FDf05KuknS6me2S9O+SrjKzSZJcUoekX9ZxjADqwNy9cU9m1rgnayJFc99/9dVXYR5dry9JS5Ysyc2uvPLKcNuiuQh27doV5kuXLg3zWbNm5WajR48Ot73mmmvCfMOGDWGeKnePJ2nIcIYfkCjKDySK8gOJovxAoig/kCjKDyQqmam7y3To0KGqtn/llVfCPLokeP369VU9d9HhuDPPPDPMx40bl5u99NJL4bYHDx4Mc1SHPT+QKMoPJIryA4mi/ECiKD+QKMoPJIryA4nikt4hbubM/iZe/n+bN28O8wULFoT5LbfcEubR5cpFY9u3b1+YFy0vvn379jAfqrikF0CI8gOJovxAoig/kCjKDySK8gOJovxAorievwFaW1vD/OjRo2H+5ZdfhvnIkflLJRZNCz51au5iS5Kk6dOnh3nR0uWrVq3KzTZu3BhuO2zYsDAfO3ZsmCPGnh9IFOUHEkX5gURRfiBRlB9IFOUHEkX5gUQVHuc3szZJKySNkeSSlrn7o2Y2StIfJY2X1CFptrsfqN9QB6/Dhw/X9ecfOJD/shctwX3FFVeEeVtbW5ivXr06zBcuXBjmkaJzCIrOYUBsIHv+bkn3uvtFki6TdIeZXSRpvqS17n6BpLXZ9wAGicLyu/sed38nu31I0geSxkm6UdLy7GHLJd1Ur0ECqL3v9ZnfzMZLmizpTUlj3H1PFu1V78cCAIPEgM/tN7OTJT0r6Vfu/kXf+dPc3fPm5zOzuZLmVjtQALU1oD2/mZ2g3uL/wd3/lN3daWZjs3yspK7+tnX3Ze7e7u7ttRgwgNooLL/17uJ/L+kDd1/SJ3pe0pzs9hxJz9V+eADqpXDqbjObLul/JL0n6ZtrTxeo93P/SklnS/qLeg/17S/4WUzd3WAzZswI86Llv4v+fRx3XP1OFRk1alSY798f/nNL1kCn7i78zO/uf5aU98N+9H0GBaB5cIYfkCjKDySK8gOJovxAoig/kCjKDySKqbuHuAkTJoT5mjVrwrzoPIApU6aEeVdXvyd+SpJ27twZbstx/Ppizw8kivIDiaL8QKIoP5Aoyg8kivIDiaL8QKI4zj8EdHR05Gbr1q0Lty06D+D+++8P89mzZ4f5448/HuYoD3t+IFGUH0gU5QcSRfmBRFF+IFGUH0gU5QcSVThvf02fLNF5+4vmny/KW1tbw3zWrFm52RtvvBFuO2LEiDDfsGFDmBddk3/hhRdWvG29lzYfqgY6bz97fiBRlB9IFOUHEkX5gURRfiBRlB9IFOUHElV4nN/M2iStkDRGkkta5u6PmtliSbdI2pc9dIG7v1jws5I8zl+t++67L8zvvffein/2gQMHwvyGG24I887Ozqp+PmpvoMf5BzKZR7eke939HTM7RdLbZvbNSg6/dff/rHSQAMpTWH533yNpT3b7kJl9IGlcvQcGoL6+12d+MxsvabKkN7O75pnZFjN70sxG5mwz18w2mdmmqkYKoKYGXH4zO1nSs5J+5e5fSHpC0nmSJqn3ncFv+tvO3Ze5e7u7t9dgvABqZEDlN7MT1Fv8P7j7nyTJ3Tvdvcfdj0r6naSp9RsmgForLL+ZmaTfS/rA3Zf0uX9sn4fNkrS19sMDUC8D+W3/lZJ+Juk9M9uc3bdA0s1mNkm9h/86JP2yLiMcAooumz148GCYR1NzS9LKlStzs8svvzzc9rrrrgvzPXv2hDkGr4H8tv/Pkvo7bhge0wfQ3DjDD0gU5QcSRfmBRFF+IFGUH0gU5QcSxdTdQ0BLS0tu1nuOVr7u7u5aDwclY+puACHKDySK8gOJovxAoig/kCjKDySK8gOJGsj1/LX0qaS/9Pn+9Oy+ZtSsYztmXD09PSUN5RjN+ppJ6YztnIE+sKEn+Rzz5GabmnVuv2YdW7OOS2JslSprbLztBxJF+YFElV3+ZSU/f6RZx9as45IYW6VKGVupn/kBlKfsPT+AkpRSfjObaWYfmtl2M5tfxhjymFmHmb1nZpvLXmIsWwaty8y29rlvlJm9YmYfZ1/7XSatpLEtNrPd2Wu32cziecHrN7Y2M1tnZtvM7H0zuyu7v9TXLhhXKa9bw9/2m1mLpI8kXStpl6S3JN3s7tsaOpAcZtYhqd3dSz8mbGb/LOmvkla4+8Tsvv+QtN/df539xznS3f+1Sca2WNJfy165OVtQZmzflaUl3STp5yrxtQvGNVslvG5l7PmnStru7jvc/W+SnpF0YwnjaHruvl7S/u/cfaOk5dnt5er9x9NwOWNrCu6+x93fyW4fkvTNytKlvnbBuEpRRvnHSdrZ5/tdaq4lv13Sy2b2tpnNLXsw/RiTLZsuSXsljSlzMP0oXLm5kb6zsnTTvHaVrHhda/zC71jT3f2fJP2LpDuyt7dNyXs/szXT4ZoBrdzcKP2sLP13Zb52la54XWtllH+3pLY+3/8wu68puPvu7GuXpFVqvtWHO79ZJDX72lXyeP6umVZu7m9laTXBa9dMK16XUf63JF1gZuea2Q8k/VTS8yWM4xhm1pr9IkZm1irpx2q+1YeflzQnuz1H0nMljuVbmmXl5ryVpVXya9d0K167e8P/SLpOvb/x/0TSv5Uxhpxx/YOkd7M/75c9NklPq/dt4Nfq/d3ILySdJmmtpI8lvSppVBON7b8kvSdpi3qLNraksU1X71v6LZI2Z3+uK/u1C8ZVyuvGGX5AoviFH5Aoyg8kivIDiaL8QKIoP5Aoyg8kivIDiaL8QKL+D7zLSHedowqrAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "print('Counterfactual prediction: {}'.format(explanation['cf']['class']))\n",
    "print('Closest prototype class: {}'.format(cf.id_proto))\n",
    "plt.imshow(explanation['cf']['X'].reshape(28, 28))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us now use the black box model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Explanation took 8.979 sec\n"
     ]
    }
   ],
   "source": [
    "# set random seed\n",
    "np.random.seed(1)\n",
    "tf.set_random_seed(1)\n",
    "\n",
    "# define models\n",
    "cnn = load_model('mnist_cnn.h5')\n",
    "predict_fn = lambda x: cnn.predict(x)\n",
    "ae = load_model('mnist_ae.h5')\n",
    "enc = load_model('mnist_enc.h5')\n",
    "\n",
    "sess = K.get_session()\n",
    "\n",
    "# initialize explainer, fit and generate counterfactual\n",
    "cf = CounterFactualProto(sess, predict_fn, shape, gamma=gamma, theta=theta,\n",
    "                         ae_model=ae, enc_model=enc, max_iterations=max_iterations,\n",
    "                         feature_range=feature_range, c_init=c_init, c_steps=c_steps)\n",
    "cf.fit(x_train)  # find class prototypes\n",
    "start_time = time()\n",
    "explanation = cf.explain(X)\n",
    "print('Explanation took {:.3f} sec'.format(time() - start_time))\n",
    "\n",
    "sess.close()\n",
    "K.clear_session()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Which again gives a 9 as counterfactual:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Counterfactual prediction: 9\n",
      "Closest prototype class: 9\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7fd04213bd30>"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAD8JJREFUeJzt3W+sVPWdx/HPV7gNfwQDwl6RqrAV1yhEKzfEINloFioqQZsoqSb1rjHePkDdJj5Y4yoa9YFubI2PjDSa4qZr2wSrRkVxiQlRtIIiAkoFlApXuFCpCIgK3O8+uIf2Vu/5nnH+nbn+3q+EMHc+85v5MfBhzsyZc37m7gKQnuPKngCAclB+IFGUH0gU5QcSRfmBRFF+IFGUH0gU5QcSRfmBRA1t5oOZGV8nBBrM3a2S29X0ym9mc83sT2a2xcxureW+ADSXVfvdfjMbIul9SXMk7ZC0WtLV7v5uMIZXfqDBmvHKP0PSFnf/wN2/kvRbSZfXcH8AmqiW8k+UtL3fzzuy6/6BmXWZ2RozW1PDYwGos4Z/4OfuiyUtltjsB1pJLa/83ZJO6ffz97PrAAwCtZR/taQpZjbZzL4n6SeSnqnPtAA0WtWb/e5+xMxulPSipCGSHnP3jXWbGYCGqnpXX1UPxnt+oOGa8iUfAIMX5QcSRfmBRFF+IFGUH0gU5QcSRfmBRFF+IFGUH0gU5QcSRfmBRFF+IFGUH0gU5QcSRfmBRFF+IFGUH0gU5QcSRfmBRFF+IFGUH0gU5QcSRfmBRFF+IFGUH0gU5QcSRfmBRFF+IFGUH0hU1Ut0S5KZbZO0X9JRSUfcvaMekxpsRo4cGeYHDx5s0kyAytVU/sxF7v6XOtwPgCZisx9IVK3ld0nLzexNM+uqx4QANEetm/2z3L3bzP5J0ktmtsndV/a/QfafAv8xAC3G3L0+d2R2l6QD7v5AcJv6PFiL4QM/tBJ3t0puV/Vmv5mNNLNRxy5L+pGkDdXeH4DmqmWzv13SH8zs2P38r7u/UJdZAWi4um32V/Rg39HN/rINGTIkNzt69GhN933CCSfUlE+ePDk3GzFiRDh22bJlYd5IkyZNCvNt27Y1ZR7VaPhmP4DBjfIDiaL8QKIoP5Aoyg8kivIDiWJXX4WGDRuWm5155pnh2O7u7jDfs2dPmA8fPjzMJ06cmJt9+eWX4diurvib17fffnuYL1++PMyXLl2am23atCkc29vbG+avvPJKmDdS9O9Bkr744osmzeSb2NUHIET5gURRfiBRlB9IFOUHEkX5gURRfiBR7OfPjB49Osw/++yzqu971qxZYb5r164wv/POO8N8/vz5udnGjRvDsUXWr18f5uecc06YF+2rj4wZMybMt2zZEuaHDh3KzRYsWFDVnI459dRTw/yjjz6q6f5rwX5+ACHKDySK8gOJovxAoig/kCjKDySK8gOJYj9/haZOnZqbbdgQr1XS3t4e5tdcc02YF51GOtofPn369HDskSNHwnzatGlhnq3bkCv69/X++++HY/fu3RvmRecD2LFjR25WdNrw+++/P8yj7xBI0oEDB8K8kdjPDyBE+YFEUX4gUZQfSBTlBxJF+YFEUX4gUYX7+c3sMUnzJO1296nZdWMl/U7SJEnbJC1w978WPtgg3s9fi6Lj0keNGhXm99xzT5hH5xp48sknw7GrV68O86LvKHR0dIT5woULc7Oenp5w7FlnnRXm9957b5hHz8vnn38ejn355ZfDvEhbW1uYHz58uKb7j9RzP/+vJc392nW3Slrh7lMkrch+BjCIFJbf3VdK+vpXrS6XtCS7vETSFXWeF4AGq/Y9f7u778wu75IUbxsCaDlDa70Dd/fovbyZdUmKF4QD0HTVvvL3mNkEScp+3513Q3df7O4d7h5/MgSgqaot/zOSOrPLnZKers90ADRLYfnN7AlJr0n6FzPbYWbXS7pP0hwz2yxpdvYzgEGE4/krNHPmzNzs008/Dcfu378/zIu+B3DBBReE+cMPPxzmtRg/fnyY33333WH+3HPP5WZFx+PfcMMNYX7xxReHeXSugYsuuigcW3QugVbG8fwAQpQfSBTlBxJF+YFEUX4gUZQfSBS7+ppg0aJFYV60nPN1110X5k899VRu9uCDD4Zji5x++uk1jY+W0b7kkkvCsc8//3xNj33llVfmZitWrAjHFu2+bWXs6gMQovxAoig/kCjKDySK8gOJovxAoig/kKiaT+MFady4cWE+ZcqUML/pppvCvOiQ3k8++STMa7F169Ywjw51luI/2wsvvBCOLfoOytq1a8M8mvtg3o9fL7zyA4mi/ECiKD+QKMoPJIryA4mi/ECiKD+QKI7nr4OTTz45zKNTSEvSnj17wvyMM84I85UrV+ZmY8eODcfOmzcvzLdv3x7m69atC/Po39cDDzwQjr3lllvC/MILLwzz6HkpOiV50d9JK+N4fgAhyg8kivIDiaL8QKIoP5Aoyg8kivIDiSo8nt/MHpM0T9Jud5+aXXeXpBskHdsZepu713aS9UHs448/DvPjjz8+zBcuXBjmd9xxR5hHS3wXnQvg5ptvDvM5c+aE+fz588O8t7c3Nyv6jknRufVfe+21MD/xxBNzsxEjRoRjU1DJK/+vJc0d4PoH3f3c7FeyxQcGq8Lyu/tKSXubMBcATVTLe/4bzewdM3vMzPK3OwG0pGrL/7CkH0g6V9JOSb/Iu6GZdZnZGjNbU+VjAWiAqsrv7j3uftTdeyX9StKM4LaL3b3D3TuqnSSA+quq/GY2od+PP5a0oT7TAdAslezqe0LShZLGmdkOSXdKutDMzpXkkrZJ+lkD5wigAQrL7+5XD3D1ow2Yy3fW1KlTw7zoHPIzZuS+q5IkTZ8+PTd79dVXw7GXXXZZmBcd71/0ZzvuuPyNy9mzZ4djH3rooTA/fPhwmEfrGTRyrYPBgm/4AYmi/ECiKD+QKMoPJIryA4mi/ECiWKK7CT744IMwf/3118O8vb09zJctW5abnX322eHYUaNGhfmqVavCfNq0aWEeHbZ78ODBcOySJUvCHLXhlR9IFOUHEkX5gURRfiBRlB9IFOUHEkX5gUSxRHcTjBw5MsyL9nc30vnnnx/m0emvJenZZ58N83379uVmc+cOdFLov+vu7g7zouXDU8US3QBClB9IFOUHEkX5gURRfiBRlB9IFOUHEsXx/E1Q6378tra2MI9OYV10LoCvvvoqzK+99towLzoXwebNm3Oz4cOHh2PZj99YvPIDiaL8QKIoP5Aoyg8kivIDiaL8QKIoP5CowuP5zewUSY9Lapfkkha7+0NmNlbS7yRNkrRN0gJ3/2vBfSV5PH8rW716dU3ji76D0NnZmZsVrWewf//+quaUunoez39E0i3ufpak8yUtNLOzJN0qaYW7T5G0IvsZwCBRWH533+nub2WX90t6T9JESZdLOrakyhJJVzRqkgDq71u95zezSZJ+KOmPktrdfWcW7VLf2wIAg0TF3+03s+MlLZX0c3f/zOzvbyvc3fPez5tZl6SuWicKoL4qeuU3szb1Ff837v5kdnWPmU3I8gmSdg801t0Xu3uHu3fUY8IA6qOw/Nb3Ev+opPfc/Zf9omckHfsot1PS0/WfHoBGqWSz/wJJP5W03szezq67TdJ9kn5vZtdL+rOkBY2ZIooMHZr/1zhu3Lhw7N69e8P8jTfeCPOiU3uvW7cuN4vmjcYrfPbd/RVJefsN/62+0wHQLHzDD0gU5QcSRfmBRFF+IFGUH0gU5QcSxY7W74CJEyfmZi+99FI4dsqUKWG+fPnyMN+yZUuYR44cOVL1WNSOV34gUZQfSBTlBxJF+YFEUX4gUZQfSBTlBxLFfv5BYPz48WEeHbN/2mmnhWMXLVoU5j09PWG+du3aMI+MHj06zA8dOhTm0dLkKMYrP5Aoyg8kivIDiaL8QKIoP5Aoyg8kivIDiSpcoruuD5boEt3Dhg0L8+HDh4f5VVddFeaPPPLIt57TMQcOHAjz2bNnh/mkSZPCfPPmzbnZvn37wrFbt24Ncwysnkt0A/gOovxAoig/kCjKDySK8gOJovxAoig/kKjC/fxmdoqkxyW1S3JJi939ITO7S9INkvZkN73N3Z8vuK8k9/Mfd1z8f2xnZ2eYn3TSSWF+6aWX5mZF++G7urrC/MMPPwzzPXv2hPnQofmnjCg6VwCqU+l+/kpO5nFE0i3u/paZjZL0ppkdWwniQXd/oNpJAihPYfndfaekndnl/Wb2nqT8JWIADArf6j2/mU2S9ENJf8yuutHM3jGzx8xsTM6YLjNbY2ZrapopgLqquPxmdrykpZJ+7u6fSXpY0g8knau+LYNfDDTO3Re7e4e7d9RhvgDqpKLym1mb+or/G3d/UpLcvcfdj7p7r6RfSZrRuGkCqLfC8puZSXpU0nvu/st+10/od7MfS9pQ/+kBaJRKPu2/QNJPJa03s7ez626TdLWZnau+3X/bJP2sITP8Dujt7Q3zF198MczPO++8MJ85c2ZutmrVqnDs9u3bw3zTpk1hHu3Kk1iGu5VV8mn/K5IG2m8Y7tMH0Nr4hh+QKMoPJIryA4mi/ECiKD+QKMoPJIpTdzdBrfvC29rawpylqtEfp+4GEKL8QKIoP5Aoyg8kivIDiaL8QKIoP5CoSo7nr6e/SPpzv5/HZde1orrNrdZj2r+2Hz+J56wBUpnbaZXesKlf8vnGg5utadVz+7Xq3Fp1XhJzq1ZZc2OzH0gU5QcSVXb5F5f8+JFWnVurzktibtUqZW6lvucHUJ6yX/kBlKSU8pvZXDP7k5ltMbNby5hDHjPbZmbrzeztspcYy5ZB221mG/pdN9bMXjKzzdnvAy6TVtLc7jKz7uy5e9vM8pcPbuzcTjGzl83sXTPbaGb/kV1f6nMXzKuU563pm/1mNkTS+5LmSNohabWkq9393aZOJIeZbZPU4e6l7xM2s3+VdEDS4+4+NbvuvyXtdff7sv84x7j7f7bI3O6SdKDslZuzBWUm9F9ZWtIVkv5dJT53wbwWqITnrYxX/hmStrj7B+7+laTfSrq8hHm0PHdfKWnv166+XNKS7PIS9f3jabqcubUEd9/p7m9ll/dLOraydKnPXTCvUpRR/omS+i8Ts0OtteS3S1puZm+aWVfZkxlAe7ZsuiTtktRe5mQGULhyczN9bWXplnnuqlnxut74wO+bZrn7eZIukbQw27xtSd73nq2VdtdUtHJzswywsvTflPncVbvidb2VUf5uSaf0+/n72XUtwd27s993S/qDWm/14Z5ji6Rmv+8ueT5/00orNw+0srRa4LlrpRWvyyj/aklTzGyymX1P0k8kPVPCPL7BzEZmH8TIzEZK+pFab/XhZyR1Zpc7JT1d4lz+Qaus3Jy3srRKfu5absVrd2/6L0mXqu8T/62S/quMOeTM658lrct+bSx7bpKeUN9m4GH1fTZyvaQTJa2QtFnS/0ka20Jz+x9J6yW9o76iTShpbrPUt0n/jqS3s1+Xlv3cBfMq5XnjG35AovjAD0gU5QcSRfmBRFF+IFGUH0gU5QcSRfmBRFF+IFH/D8HfOAvSPxmWAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "print('Counterfactual prediction: {}'.format(explanation['cf']['class']))\n",
    "print('Closest prototype class: {}'.format(cf.id_proto))\n",
    "plt.imshow(explanation['cf']['X'].reshape(28, 28))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can include the prediction loss term again and compare the explanation time for the same amount of iteration steps:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "c_init = 1.  # set weight predict loss term to 0\n",
    "c_steps = 4  # do not update c further\n",
    "max_iterations = 500  # 4x500 = 2000"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Explanation took 893.747 sec\n"
     ]
    }
   ],
   "source": [
    "# set random seed\n",
    "np.random.seed(1)\n",
    "tf.set_random_seed(1)\n",
    "\n",
    "# define models\n",
    "cnn = load_model('mnist_cnn.h5')\n",
    "predict_fn = lambda x: cnn.predict(x)\n",
    "ae = load_model('mnist_ae.h5')\n",
    "enc = load_model('mnist_enc.h5')\n",
    "\n",
    "sess = K.get_session()\n",
    "\n",
    "# initialize explainer, fit and generate counterfactual\n",
    "cf = CounterFactualProto(sess, predict_fn, shape, gamma=gamma, theta=theta,\n",
    "                         ae_model=ae, enc_model=enc, max_iterations=max_iterations,\n",
    "                         feature_range=feature_range, c_init=c_init, c_steps=c_steps)\n",
    "cf.fit(x_train)  # find class prototypes\n",
    "start_time = time()\n",
    "explanation = cf.explain(X)\n",
    "print('Explanation took {:.3f} sec'.format(time() - start_time))\n",
    "\n",
    "sess.close()\n",
    "K.clear_session()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Counterfactual prediction: 9\n",
      "Closest prototype class: 9\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7fd04b731400>"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAD+5JREFUeJzt3X2MVfWdx/HPVxw1AZUnHQjVQauuVjCDjrpaVMwujWuaSNUYTGxYNNJEJtma/rEIGknwD7Np1fpP4xiNsOnSktRGo1ULqFF0UQFZQNwiNECZDE8ZeVQjD9/9Yy7dUed8z/U+nTv+3q+EcOd+7u/eX65+OGfuOff8zN0FID0nFT0BAMWg/ECiKD+QKMoPJIryA4mi/ECiKD+QKMoPJIryA4k6uZEvZmacTgjUmbtbOY+rastvZjeZ2V/MbLOZzanmuQA0llV6br+ZDZG0SdJUSTskfSDpTnffGIxhyw/UWSO2/FdJ2uzuf3X3LyX9TtItVTwfgAaqpvzjJP2t3887Svd9hZnNMrNVZraqitcCUGN1/8DP3bskdUns9gPNpJotf7ekc/r9/L3SfQAGgWrK/4GkC83sPDM7RdJ0SS/WZloA6q3i3X53P2pmnZJekzRE0rPu/lHNZgagrio+1FfRi/E7P1B3DTnJB8DgRfmBRFF+IFGUH0gU5QcSRfmBRFF+IFGUH0gU5QcSRfmBRFF+IFGUH0gU5QcSRfmBRFF+IFGUH0gU5QcSRfmBRFF+IFGUH0gU5QcSRfmBRFF+IFGUH0gU5QcSRfmBRFF+IFGUH0gU5QcSVfES3ZJkZlslHZR0TNJRd++oxaRQO0OGDAnzY8eONWgm315LS0uYHzlypEEz+W6qqvwlN7r73ho8D4AGYrcfSFS15XdJfzaz1WY2qxYTAtAY1e72T3b3bjM7W9JSM/tfd3+r/wNK/yjwDwPQZMzda/NEZvMlHXL3XwaPqc2LoWx84Jced7dyHlfxbr+ZDTWz00/clvQjSRsqfT4AjVXNbn+rpD+a2Ynn+S93f7UmswJQdzXb7S/rxdjtb7i2trYw37NnT5hPmDAhzA8cOBDm7e3tmdlrr70Wjv3000/DPM+IESPq9tzNrO67/QAGN8oPJIryA4mi/ECiKD+QKMoPJIpDfTVQOtch03333Rfmvb29Yb506dIwv/TSSzOzYcOGhWPHjx8f5jfeeGOY9/T0hHlnZ2dm9uGHH4Zju7u7w3z69Olhfvjw4TCPjB49Osz37m3eL7JyqA9AiPIDiaL8QKIoP5Aoyg8kivIDiaL8QKI4zl+mUaNGZWbRcXZJOn78eJhfc801YZ53rP3ss8/OzK644opw7Jo1a8L85JPjSz5cfPHFYb569erMLO9Yed6x9mXLloV5NLd77703HNvREV+FfsuWLWG+devWMK8njvMDCFF+IFGUH0gU5QcSRfmBRFF+IFGUH0hULVbpTUL0vfjoWLYkXXfddWH++uuvh/nkyZPDvLW1NTPLO48j71j6hg3xOizPP/98mL/88suZ2cMPPxyOzfs+/l133RXma9euzcyuvfbacOzGjRvDfNu2bWE+GLDlBxJF+YFEUX4gUZQfSBTlBxJF+YFEUX4gUbnH+c3sWUk/lrTb3SeU7hsp6feSxkvaKukOd//urnms+LjumWeeGY595513wjy6VoAkLVmyJMyj4+FPP/10OHbkyJFhvm/fvjBfvnx5mC9YsCAzGzp0aDj29NNPD/MzzjgjzCdOnJiZjRkzJhz7yiuvhPkpp5wS5l9++WWYN4NytvzPSbrpa/fNkbTc3S+UtLz0M4BBJLf87v6WpK8vKXOLpIWl2wslTavxvADUWaW/87e6+4l1mnZKyj6/FEBTqvrcfnf36Np8ZjZL0qxqXwdAbVW65d9lZmMlqfT37qwHunuXu3e4e3xFRAANVWn5X5Q0o3R7hqQXajMdAI2SW34zWyzpvyX9g5ntMLN7JD0qaaqZfSLpn0s/AxhEuG5/maLr1x89ejQcm3dMefjw4WGe97326DyDvOvPf/bZZ2Ged47BQw89FOYnnZS9fXniiSfCsZ2dnWH+yCOPhHkk79yKiy66KMw3bdoU5r29Xz9A1jhctx9AiPIDiaL8QKIoP5Aoyg8kivIDieLS3WVqa2vLzPIu47xz584wHzJkSMWvLUnvvvtuZpY3t6uvvjrM33zzzTC/4YYbwjyybt26MM87DJnn9ttvz8zyDsWtXLkyzM8999wwL/JQX7nY8gOJovxAoig/kCjKDySK8gOJovxAoig/kCiO85dpy5YtFY8dN25cmHd3d1eVR0477bSKx0r5l+7O8/7772dmect79/T0hPmKFSsqfu1q7dixo27P3Shs+YFEUX4gUZQfSBTlBxJF+YFEUX4gUZQfSBSX7i5TtJT1+eefH47duHFjmE+aNCnMP//88zDfvHlzZjZ37txw7K233hrm7e3tYZ73nfvo/6+8ZaxbWlrCfMqUKWEeXdJ81apV4di8cwyaGZfuBhCi/ECiKD+QKMoPJIryA4mi/ECiKD+QqNzj/Gb2rKQfS9rt7hNK982XdK+kPaWHzXX3P+W+2CA+zh9dWz/vO/PnnXdemJ966qlhPnv27DDfvn17ZjZ//vxw7AUXXBDm0TkEkjRixIgwv//++zOzBx98MBybN/cFCxaE+ejRozOzPXv2ZGaDXS2P8z8n6aYB7n/c3dtLf3KLD6C55Jbf3d+S1PzLjwD4Vqr5nb/TzNaZ2bNmFu/7AWg6lZb/N5K+L6ldUo+kX2U90MxmmdkqM4tPpgbQUBWV3913ufsxdz8u6WlJVwWP7XL3DnfvqHSSAGqvovKb2dh+P/5E0obaTAdAo+ReutvMFkuaImm0me2Q9LCkKWbWLsklbZX0szrOEUAd5Jbf3e8c4O5n6jCXpnbs2LHM7PDhw+HYDRviHaNp06aF+XPPPRfm06dPD/PI0aNHw3zmzJlhPnz48DC//vrrM7O8a98vW7YszPPOUTl+/Hhm1tbWFo7dtm1bmH8XcIYfkCjKDySK8gOJovxAoig/kCjKDySKS3fXQN4lpo8cORLmeYfLql0muxrz5s0L8+irzlL8tdw1a9aEYzs6OCm0Ely6G0CI8gOJovxAoig/kCjKDySK8gOJovxAonK/0ot8ecfx85x0Uv3+DR47dmyY5y1FvXbt2jB/6aWXwnz//v2ZWWdnZzgW9cWWH0gU5QcSRfmBRFF+IFGUH0gU5QcSRfmBRHGcvwn09sbroLa2toZ5dOnwvOP4o0aNCvMnn3wyzPMu/f3MM9lXeV+3bl04FvXFlh9IFOUHEkX5gURRfiBRlB9IFOUHEkX5gUTlHuc3s3MkLZLUKskldbn7r81spKTfSxovaaukO9z90/pNNV27du2qeGzedfWXLFkS5nnnAaxfvz7MV6xYkZmdddZZ4dgUlskuUjlb/qOSfuHuP5D0j5Jmm9kPJM2RtNzdL5S0vPQzgEEit/zu3uPua0q3D0r6WNI4SbdIWlh62EJJ0+o1SQC1961+5zez8ZImSXpPUqu7nzh3dKf6fi0AMEiUfW6/mQ2T9AdJP3f3A2b/vxyYu3vWOnxmNkvSrGonCqC2ytrym1mL+or/W3d/vnT3LjMbW8rHSto90Fh373L3Dndn1UWgieSW3/o28c9I+tjdH+sXvShpRun2DEkv1H56AOqlnN3+H0r6qaT1ZnbiOs5zJT0qaYmZ3SNpm6Q76jNFVOPkk+P/xNu3bw/zvEt/v/rqq2H+9ttvZ2Z5X1VGfeWW391XSMpa7/ufajsdAI3CGX5Aoig/kCjKDySK8gOJovxAoig/kCgu3f0d0NbWlpktXrw4HHv55ZeH+bx588L8scceC3P3Ac/6lhRfchz1x5YfSBTlBxJF+YFEUX4gUZQfSBTlBxJF+YFEWXQctuYvlnGpL8Quu+yyMB8xYkRm9sYbb4Rjp06dGub79+8P83379oX53r17M7ODBw+GY4cOHRrmBw4cCPNUuXvWV/C/gi0/kCjKDySK8gOJovxAoig/kCjKDySK8gOJ4jh/A+Qtk33JJZeE+ZVXXhnmTz31VGbW0tISjt2wYUOYz5kTL7582223hXlXV1dmtnLlynAsKsNxfgAhyg8kivIDiaL8QKIoP5Aoyg8kivIDicq9br+ZnSNpkaRWSS6py91/bWbzJd0raU/poXPd/U/1muhgNmbMmDAfPnx4mH/xxRdh/t5772VmI0eODMfOnDkzzA8dOhTmd999d5ijeZWzaMdRSb9w9zVmdrqk1Wa2tJQ97u6/rN/0ANRLbvndvUdST+n2QTP7WNK4ek8MQH19q9/5zWy8pEmSTuxndprZOjN71swGvJaUmc0ys1VmtqqqmQKoqbLLb2bDJP1B0s/d/YCk30j6vqR29e0Z/Gqgce7e5e4d7t5Rg/kCqJGyym9mLeor/m/d/XlJcvdd7n7M3Y9LelrSVfWbJoBayy2/mZmkZyR97O6P9bt/bL+H/URS/PUwAE2lnE/7fyjpp5LWm9na0n1zJd1pZu3qO/y3VdLP6jLD74Du7u6q8gceeCDMe3t7M7Nhw4aFYydOnBjmixYtCnMMXuV82r9C0kDfD+aYPjCIcYYfkCjKDySK8gOJovxAoig/kCjKDySKS3cPAlOmTKl47KRJk8L88ccfr/i50Zy4dDeAEOUHEkX5gURRfiBRlB9IFOUHEkX5gUQ1+jj/Hknb+t01WtLehk3g22nWuTXrvCTmVqlazq3N3c8q54ENLf83XtxsVbNe269Z59as85KYW6WKmhu7/UCiKD+QqKLL31Xw60eadW7NOi+JuVWqkLkV+js/gOIUveUHUJBCym9mN5nZX8xss5nNKWIOWcxsq5mtN7O1RS8xVloGbbeZbeh330gzW2pmn5T+HnCZtILmNt/Mukvv3Vozu7mguZ1jZm+Y2UYz+8jM/q10f6HvXTCvQt63hu/2m9kQSZskTZW0Q9IHku50940NnUgGM9sqqcPdCz8mbGbXSzokaZG7Tyjd9x+Set390dI/nCPc/d+bZG7zJR0qeuXm0oIyY/uvLC1pmqR/VYHvXTCvO1TA+1bElv8qSZvd/a/u/qWk30m6pYB5ND13f0vS11fkuEXSwtLther7n6fhMubWFNy9x93XlG4flHRiZelC37tgXoUoovzjJP2t38871FxLfrukP5vZajObVfRkBtBaWjZdknZKai1yMgPIXbm5kb62snTTvHeVrHhda3zg902T3f1ySf8iaXZp97Yped/vbM10uKaslZsbZYCVpf+uyPeu0hWva62I8ndLOqffz98r3dcU3L279PduSX9U860+vOvEIqmlv3cXPJ+/a6aVmwdaWVpN8N4104rXRZT/A0kXmtl5ZnaKpOmSXixgHt9gZkNLH8TIzIZK+pGab/XhFyXNKN2eIemFAufyFc2ycnPWytIq+L1ruhWv3b3hfyTdrL5P/LdImlfEHDLmdb6k/yn9+ajouUlarL7dwCPq+2zkHkmjJC2X9ImkZZJGNtHc/lPSeknr1Fe0sQXNbbL6dunXSVpb+nNz0e9dMK9C3jfO8AMSxQd+QKIoP5Aoyg8kivIDiaL8QKIoP5Aoyg8kivIDifo/CXM3OGL5JPcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "print('Counterfactual prediction: {}'.format(explanation['cf']['class']))\n",
    "print('Closest prototype class: {}'.format(cf.id_proto))\n",
    "plt.imshow(explanation['cf']['X'].reshape(28, 28))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By removing the predict loss term and having the nearest class prototype guide the counterfactual, we can speed up the search by about 100x for a simple CNN. Please note that these examples are not optimized for speed but just highlight the difference in performance."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Clean up:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "os.remove('mnist_cnn.h5')\n",
    "os.remove('mnist_ae.h5')\n",
    "os.remove('mnist_enc.h5')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}