examples/cfproto_mnist.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Counterfactuals guided by prototypes on MNIST"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This method is described in the [Interpretable Counterfactual Explanations Guided by Prototypes](https://arxiv.org/abs/1907.02584) paper and can generate counterfactual instances guided by class prototypes. It means that for a certain instance X, the method builds a prototype for each prediction class using either an [autoencoder](https://en.wikipedia.org/wiki/Autoencoder) or [k-d trees](https://en.wikipedia.org/wiki/K-d_tree). The nearest prototype class other than the originally predicted class is then used to guide the counterfactual search. For example, in MNIST the closest class to a 7 could be a 9. As a result, the prototype loss term will try to minimize the distance between the proposed counterfactual and the prototype of a 9. This speeds up the search towards a satisfactory counterfactual by steering it towards an interpretable solution from the start of the optimization. It also helps to avoid out-of-distribution counterfactuals with the perturbations driven to a prototype of another class.\n",
    "\n",
    "The loss function to be optimized is the following: \n",
    "\n",
    "$Loss$ = c$L_{pred}$ + $\\beta$$L_{1}$ + $L_{2}$ + $L_{AE}$ + $L_{proto}$\n",
    "\n",
    "The first loss term relates to the model's prediction function, the following 2 terms define the elastic net regularization while the last 2 terms are optional. The aim of $L_{AE}$ is to penalize out-of-distribution counterfactuals while $L_{proto}$ guides the counterfactual to a prototype. When we only have acces to the model's prediction function and cannot fully enjoy the benefits of automatic differentiation, the prototypes allow us to drop the prediction function loss term $L_{pred}$ and still generate high quality counterfactuals. This drastically reduces the number of prediction calls made during the numerical gradient update step and again speeds up the search.\n",
    "\n",
    "Other options include generating counterfactuals for specific classes or including trust score constraints to ensure that the counterfactual is close enough to the newly predicted class compared to the original class. Different use cases are illustrated throughout this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "tf.logging.set_verbosity(tf.logging.ERROR)  # suppress deprecation messages\n",
    "from tensorflow.keras import backend as K\n",
    "from tensorflow.keras.layers import Conv2D, Dense, Dropout, Flatten, MaxPooling2D, Input, UpSampling2D\n",
    "from tensorflow.keras.models import Model, load_model\n",
    "from tensorflow.keras.utils import to_categorical\n",
    "\n",
    "import matplotlib\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import os\n",
    "from time import time\n",
    "from alibi.explainers import CounterFactualProto"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load and prepare MNIST data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x_train shape: (60000, 28, 28) y_train shape: (60000,)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAANYElEQVR4nO3df4hd9ZnH8c9n3QTEFk0iOwxG1hr1j7iolVEWVxaX2uiKJgakJshiqTD9o0LF+CNkhQiLKLvb3T8DUxoatWvTkJjGumzqhvpjwQRHiTHRtBpJbMIkQzZgE0Rqkmf/mDPLVOeeOznn3ntu8rxfMNx7z3PvOQ9XPzm/7jlfR4QAnPv+rOkGAPQGYQeSIOxAEoQdSIKwA0n8eS8XZptD/0CXRYSnm15rzW77dtu/tf2R7ZV15gWgu1z1PLvt8yT9TtK3JR2U9Jak5RHxfslnWLMDXdaNNfuNkj6KiI8j4o+Sfi5pSY35AeiiOmG/RNLvp7w+WEz7E7aHbY/aHq2xLAA1df0AXUSMSBqR2IwHmlRnzX5I0qVTXs8vpgHoQ3XC/pakK21/w/ZsScskbelMWwA6rfJmfESctP2gpK2SzpO0NiL2dKwzAB1V+dRbpYWxzw50XVd+VAPg7EHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EASPb2VNKp55JFHSuvnn39+y9o111xT+tl77rmnUk+T1qxZU1p/8803W9aee+65WsvGmWHNDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJcHfZPrB+/frSet1z4U3at29fy9qtt95a+tlPPvmk0+2kwN1lgeQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJrmfvgSbPo+/du7e0vnXr1tL65ZdfXlq/6667SusLFixoWbvvvvtKP/v000+X1nFmaoXd9n5JxyWdknQyIoY60RSAzuvEmv3vIuJoB+YDoIvYZweSqBv2kPRr22/bHp7uDbaHbY/aHq25LAA11N2MvzkiDtn+C0mv2N4bEa9PfUNEjEgakbgQBmhSrTV7RBwqHsclvSjpxk40BaDzKofd9gW2vz75XNIiSbs71RiAzqqzGT8g6UXbk/P5j4j4r450dZYZGio/47h06dJa89+zZ09pffHixS1rR4+Wnyg5ceJEaX327Nml9e3bt5fWr7322pa1efPmlX4WnVU57BHxsaTW/yUB9BVOvQFJEHYgCcIOJEHYgSQIO5AEl7h2wODgYGm9OD3ZUrtTa7fddltpfWxsrLRex4oVK0rrCxcurDzvl19+ufJnceZYswNJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEpxn74CXXnqptH7FFVeU1o8fP15aP3bs2Bn31CnLli0rrc+aNatHnaAu1uxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kATn2XvgwIEDTbfQ0qOPPlpav+qqq2rNf8eOHZVq6DzW7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQhCOidwuze7cwSJLuvPPO0vqGDRtK6+2GbB4fHy+tl10P/9prr5V+FtVExLQDFbRds9tea3vc9u4p0+bafsX2h8XjnE42C6DzZrIZ/1NJt39p2kpJ2yLiSknbitcA+ljbsEfE65K+fF+kJZLWFc/XSbq7w30B6LCqv40fiIjJAcYOSxpo9Ubbw5KGKy4HQIfUvhAmIqLswFtEjEgakThABzSp6qm3I7YHJal4LD8kC6BxVcO+RdL9xfP7Jf2yM+0A6Ja2m/G2X5B0i6SLbR+UtFrSM5J+YfsBSQckfaebTaK6oaGh0nq78+jtrF+/vrTOufT+0TbsEbG8RelbHe4FQBfxc1kgCcIOJEHYgSQIO5AEYQeS4FbS54DNmze3rC1atKjWvJ999tnS+hNPPFFr/ugd1uxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kAS3kj4LDA4OltbffffdlrV58+aVfvbo0aOl9Ztuuqm0vm/fvtI6eq/yraQBnBsIO5AEYQeSIOxAEoQdSIKwA0kQdiAJrmc/C2zcuLG03u5cepnnn3++tM559HMHa3YgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSILz7H1g8eLFpfXrr7++8rxfffXV0vrq1asrzxtnl7ZrdttrbY/b3j1l2pO2D9neWfzd0d02AdQ1k834n0q6fZrp/x4R1xV//9nZtgB0WtuwR8Trko71oBcAXVTnAN2DtncVm/lzWr3J9rDtUdujNZYFoKaqYV8jaYGk6ySNSfpRqzdGxEhEDEXEUMVlAeiASmGPiCMRcSoiTkv6saQbO9sWgE6rFHbbU+9tvFTS7lbvBdAf2p5nt/2CpFskXWz7oKTVkm6xfZ2kkLRf0ve72ONZr9315qtWrSqtz5o1q/Kyd+7cWVo/ceJE5Xnj7NI27BGxfJrJP+lCLwC6iJ/LAkkQdiAJwg4kQdiBJAg7kASXuPbAihUrSus33HBDrflv3ry5ZY1LWDGJNTuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJOGI6N3C7N4trI98/vnnpfU6l7BK0vz581vWxsbGas0bZ5+I8HTTWbMDSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBJcz34OmDt3bsvaF1980cNOvurTTz9tWWvXW7vfH1x44YWVepKkiy66qLT+8MMPV573TJw6dapl7fHHHy/97GeffVZpmazZgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJzrOfA3bt2tV0Cy1t2LChZa3dtfYDAwOl9XvvvbdST/3u8OHDpfWnnnqq0nzbrtltX2r7N7bft73H9g+L6XNtv2L7w+JxTqUOAPTETDbjT0paERELJf21pB/YXihppaRtEXGlpG3FawB9qm3YI2IsIt4pnh+X9IGkSyQtkbSueNs6SXd3q0kA9Z3RPrvtyyR9U9IOSQMRMbnTdVjStDtYtoclDVdvEUAnzPhovO2vSdoo6aGI+MPUWkzctXLam0lGxEhEDEXEUK1OAdQyo7DbnqWJoP8sIjYVk4/YHizqg5LGu9MigE5oeytp29bEPvmxiHhoyvR/kfS/EfGM7ZWS5kbEY23mlfJW0ps2bSqtL1mypEed5HLy5MmWtdOnT9ea95YtW0rro6Ojlef9xhtvlNa3b99eWm91K+mZ7LP/jaR/kPSe7Z3FtFWSnpH0C9sPSDog6TszmBeAhrQNe0T8j6Rp/6WQ9K3OtgOgW/i5LJAEYQeSIOxAEoQdSIKwA0kwZHMfeOyx0p8n1B7SuczVV19dWu/mZaRr164tre/fv7/W/Ddu3Niytnfv3lrz7mcM2QwkR9iBJAg7kARhB5Ig7EAShB1IgrADSXCeHTjHcJ4dSI6wA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkmgbdtuX2v6N7fdt77H9w2L6k7YP2d5Z/N3R/XYBVNX25hW2ByUNRsQ7tr8u6W1Jd2tiPPYTEfGvM14YN68Auq7VzStmMj77mKSx4vlx2x9IuqSz7QHotjPaZ7d9maRvStpRTHrQ9i7ba23PafGZYdujtkdrdQqglhnfg8721yS9JumpiNhke0DSUUkh6Z80san/vTbzYDMe6LJWm/EzCrvtWZJ+JWlrRPzbNPXLJP0qIv6qzXwIO9BllW84aduSfiLpg6lBLw7cTVoqaXfdJgF0z0yOxt8s6Q1J70k6XUxeJWm5pOs0sRm/X9L3i4N5ZfNizQ50Wa3N+E4h7ED3cd94IDnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEm1vONlhRyUdmPL64mJaP+rX3vq1L4nequpkb3/ZqtDT69m/snB7NCKGGmugRL/21q99SfRWVa96YzMeSIKwA0k0HfaRhpdfpl9769e+JHqrqie9NbrPDqB3ml6zA+gRwg4k0UjYbd9u+7e2P7K9sokeWrG93/Z7xTDUjY5PV4yhN25795Rpc22/YvvD4nHaMfYa6q0vhvEuGWa80e+u6eHPe77Pbvs8Sb+T9G1JByW9JWl5RLzf00ZasL1f0lBENP4DDNt/K+mEpGcnh9ay/c+SjkXEM8U/lHMi4vE+6e1JneEw3l3qrdUw499Vg99dJ4c/r6KJNfuNkj6KiI8j4o+Sfi5pSQN99L2IeF3SsS9NXiJpXfF8nSb+Z+m5Fr31hYgYi4h3iufHJU0OM97od1fSV080EfZLJP1+yuuD6q/x3kPSr22/bXu46WamMTBlmK3DkgaabGYabYfx7qUvDTPeN99dleHP6+IA3VfdHBHXS/p7ST8oNlf7Ukzsg/XTudM1khZoYgzAMUk/arKZYpjxjZIeiog/TK01+d1N01dPvrcmwn5I0qVTXs8vpvWFiDhUPI5LelETux395MjkCLrF43jD/fy/iDgSEaci4rSkH6vB764YZnyjpJ9FxKZicuPf3XR99ep7ayLsb0m60vY3bM+WtEzSlgb6+ArbFxQHTmT7AkmL1H9DUW+RdH/x/H5Jv2ywlz/RL8N4txpmXA1/d40Pfx4RPf+TdIcmjsjvk/SPTfTQoq/LJb1b/O1pujdJL2his+4LTRzbeEDSPEnbJH0o6b8lze2j3p7TxNDeuzQRrMGGertZE5vouyTtLP7uaPq7K+mrJ98bP5cFkuAAHZAEYQeSIOxAEoQdSIKwA0kQdiAJwg4k8X98jzceoKWtgAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()\n",
    "print('x_train shape:', x_train.shape, 'y_train shape:', y_train.shape)\n",
    "plt.gray()\n",
    "plt.imshow(x_test[1]);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prepare data: scale, reshape and categorize"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x_train shape: (60000, 28, 28, 1) x_test shape: (10000, 28, 28, 1)\n",
      "y_train shape: (60000, 10) y_test shape: (10000, 10)\n"
     ]
    }
   ],
   "source": [
    "x_train = x_train.astype('float32') / 255\n",
    "x_test = x_test.astype('float32') / 255\n",
    "x_train = np.reshape(x_train, x_train.shape + (1,))\n",
    "x_test = np.reshape(x_test, x_test.shape + (1,))\n",
    "print('x_train shape:', x_train.shape, 'x_test shape:', x_test.shape)\n",
    "y_train = to_categorical(y_train)\n",
    "y_test = to_categorical(y_test)\n",
    "print('y_train shape:', y_train.shape, 'y_test shape:', y_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "xmin, xmax = -.5, .5\n",
    "x_train = ((x_train - x_train.min()) / (x_train.max() - x_train.min())) * (xmax - xmin) + xmin\n",
    "x_test = ((x_test - x_test.min()) / (x_test.max() - x_test.min())) * (xmax - xmin) + xmin"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define and train CNN model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def cnn_model():\n",
    "    x_in = Input(shape=(28, 28, 1))\n",
    "    x = Conv2D(filters=32, kernel_size=2, padding='same', activation='relu')(x_in)\n",
    "    x = MaxPooling2D(pool_size=2)(x)\n",
    "    x = Dropout(0.3)(x)\n",
    "    \n",
    "    x = Conv2D(filters=64, kernel_size=2, padding='same', activation='relu')(x)\n",
    "    x = MaxPooling2D(pool_size=2)(x)\n",
    "    x = Dropout(0.3)(x)\n",
    "    \n",
    "    x = Flatten()(x)\n",
    "    x = Dense(256, activation='relu')(x)\n",
    "    x = Dropout(0.5)(x)\n",
    "    x_out = Dense(10, activation='softmax')(x)\n",
    "    \n",
    "    cnn = Model(inputs=x_in, outputs=x_out)\n",
    "    cnn.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])\n",
    "    \n",
    "    return cnn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "cnn = cnn_model()\n",
    "cnn.fit(x_train, y_train, batch_size=32, epochs=3, verbose=0)\n",
    "cnn.save('mnist_cnn.h5', save_format='h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Evaluate the model on test set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test accuracy:  0.9887\n"
     ]
    }
   ],
   "source": [
    "cnn = load_model('mnist_cnn.h5')\n",
    "score = cnn.evaluate(x_test, y_test, verbose=0)\n",
    "print('Test accuracy: ', score[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define and train auto-encoder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def ae_model():\n",
    "    # encoder\n",
    "    x_in = Input(shape=(28, 28, 1))\n",
    "    x = Conv2D(16, (3, 3), activation='relu', padding='same')(x_in)\n",
    "    x = Conv2D(16, (3, 3), activation='relu', padding='same')(x)\n",
    "    x = MaxPooling2D((2, 2), padding='same')(x)\n",
    "    encoded = Conv2D(1, (3, 3), activation=None, padding='same')(x)\n",
    "    encoder = Model(x_in, encoded)\n",
    "    \n",
    "    # decoder\n",
    "    dec_in = Input(shape=(14, 14, 1))\n",
    "    x = Conv2D(16, (3, 3), activation='relu', padding='same')(dec_in)\n",
    "    x = UpSampling2D((2, 2))(x)\n",
    "    x = Conv2D(16, (3, 3), activation='relu', padding='same')(x)\n",
    "    decoded = Conv2D(1, (3, 3), activation=None, padding='same')(x)\n",
    "    decoder = Model(dec_in, decoded)\n",
    "    \n",
    "    # autoencoder = encoder + decoder\n",
    "    x_out = decoder(encoder(x_in))\n",
    "    autoencoder = Model(x_in, x_out)\n",
    "    autoencoder.compile(optimizer='adam', loss='mse')\n",
    "    \n",
    "    return autoencoder, encoder, decoder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "ae, enc, dec = ae_model()\n",
    "ae.fit(x_train, x_train, batch_size=128, epochs=4, validation_data=(x_test, x_test), verbose=0)\n",
    "ae.save('mnist_ae.h5', save_format='h5')\n",
    "enc.save('mnist_enc.h5', save_format='h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compare original with decoded images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABBAAAADrCAYAAADQf2U5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3deZRUxfXA8ZpB9n1TtgQEBBcWFVFU3AhxQUBBEAPmGDBxCS6J61HU/JKoJOQc1EQENSESTZCoSFBBxBxBPYnGYYkgIEKQzQUQhnVQlv79kWPl1nVe1XuP7p6e4fv569apnvdqOF1dPY+6dYsymYwBAAAAAADwKa7oAQAAAAAAgMLHAwQAAAAAABDEAwQAAAAAABDEAwQAAAAAABDEAwQAAAAAABDEAwQAAAAAABB0RJIXFxUVUfOxcGzJZDLNK3oQKAzMzcKRyWSKKnoMKAzMy4LCmgmLuVlQmJuwmJsFJXJusgOh8lpb0QMAAKCSYM0EChNzEyhMkXOTBwgAAAAAACCIBwgAAAAAACCIBwgAAAAAACCIBwgAAAAAACCIBwgAAAAAACCIBwgAAAAAACCIBwgAAAAAACCIBwgAAAAAACCIBwgAAAAAACDoiIoeQFq33Xab065du7aNu3Xr5vQNGTIk8joTJ0502v/85z9t/PTTTx/KEAEAAAAAqDLYgQAAAAAAAIJ4gAAAAAAAAIIqVQrDtGnTbOxLS9AOHjwY2Xfttdc67b59+9p4/vz5Tt+6deti3xNAdnXq1MnGK1ascPpuvvlmG//ud7/L25iAqqBu3bpO+ze/+Y2N9Rq5YMECpz106FAbr127NgejAwAAhYQdCAAAAAAAIIgHCAAAAAAAIIgHCAAAAAAAIKigz0CQZx4YE//cA50fPWfOHBu3b9/e6RswYIDT7tChg41HjBjh9I0dOzbW/QFk30knnWRjfa7Jhg0b8j0coMpo2bKl0/7Rj35kYz3XevTo4bT79+9v4wkTJuRgdEDVdvLJJzvt6dOn27hdu3Y5v//555/vtJcvX27j9evX5/z+wOFG/u05c+ZMp++GG26w8aRJk5y+AwcO5HZgCbADAQAAAAAABPEAAQAAAAAABBVcCsMpp5xi40GDBkW+7oMPPnDaAwcOtPGWLVucvl27dtm4Ro0aTt8777zjtLt3727jpk2bxhgxgHw48cQTbbx7926n78UXX8z3cIBKrXnz5jaeMmVKBY4EOLxdcMEFTrtmzZp5vb9O5R01apSNr7jiiryOBaiK9N+Tjz32WORrH330URtPnjzZ6SsrK8vuwA4BOxAAAAAAAEAQDxAAAAAAAEAQDxAAAAAAAEBQwZ2BIMtJFRUVOX3y3AOdM/bpp5/Guv6tt97qtI8//vjI177yyiuxrgkg+7p06eK0ZWmbp59+Ot/DASq1m266yWlfeumlNj711FNTX/fss8+2cXGx+38S//73v2385ptvpr4HUNUcccT/vn7369evAkdizIIFC5z2LbfcYuO6des6ffr8IQBhcp00xpg2bdpEvnbq1Kk23rt3b87GdKjYgQAAAAAAAIJ4gAAAAAAAAIIKLoXhpZdesnHHjh2dvp07d9p469atqa6vS9JUr1491XUA5Naxxx7rtOVWymnTpuV7OECl9tBDDzntgwcPZuW6gwcPLjc2xpi1a9faeNiwYU6f3jYNHE7OO+88G59++ulO37hx4/I6lsaNGzttmdpbp04dp48UBiBMl2IdM2ZM7J+VKbqZTCZrY8o2diAAAAAAAIAgHiAAAAAAAIAgHiAAAAAAAICggjsDQZL5k4fi9ttvt3GnTp28r3333XfLjQHk1x133OG05edBSUlJvocDVDqzZs2ysS6xmNYXX3zhtHft2mXjtm3bOn1HH320jf/1r385fdWqVcvKeIDKQJcllqXaVq9e7fQ9+OCDeRnT1y655JK83g+o6rp27eq0e/ToEfna/fv3O+3Zs2fnZEzZxg4EAAAAAAAQxAMEAAAAAAAQVNApDGn179/faf/iF7+wcY0aNZy+TZs2Oe277rrLxnv27MnB6ACUp127dk77lFNOcdorV660MaWkgG8655xznHbnzp1trMs2xi3jOGnSJKf92muvOe3t27fbuE+fPk6fr3TV9ddfb+OJEyfGGgtQWd1zzz1OW5YlvvDCC50+mRaUK02aNLGx/tzIVolX4HB12WWXxX6tXlMrC3YgAAAAAACAIB4gAAAAAACAIB4gAAAAAACAoCp5BoLOndbnHkjTpk1z2vPnz8/JmAD46TxMbfPmzXkaCVB5yLNDnn32WaevWbNmsa6hSya/8MILNv75z3/u9PnOBtLXueaaa2zcvHlzp2/cuHE2rlWrltP36KOP2njfvn2R9wMK2ZAhQ2zcr18/p2/VqlU2roiyxPJ8En3mwbx582xcWlqaryEBVcbZZ5/t7f/qq69s7DsrqJCxAwEAAAAAAATxAAEAAAAAAARVmRSGGTNm2Pj888+PfN2f/vQnp61L6wCoGF27dvX2yy3PAP7riCP+t4zHTVkwxk3Xu+KKK5y+LVu2pBqLTmEYO3asjcePH+/01alTx8Z6bs+cOdPGq1evTjUWoKINHTrUxvL9bowxjz32WF7HosskjxgxwsYHDhxw+u6//34bk0IExHPGGWeUG5dHliJfvHhxzsaUS+xAAAAAAAAAQTxAAAAAAAAAQTxAAAAAAAAAQZX2DISWLVs6bZlvUrNmTadP5nPK3C5jjNm1a1cORgcgjl69etl45MiRTt+iRYuc9ty5c/MyJqAq0qXiRo0aZeO0Zx6EyLMMZM61Mcb07NkzJ/cEKkrDhg2dtlzftIkTJ+Z6OA5ZUtUY97yU5cuXO31vvPFGXsYEVCVJ1rR8z/9cYAcCAAAAAAAI4gECAAAAAAAIqrQpDC+88ILTbtq0aeRrn3nmGRtTEgooHH379rVxkyZNnL5XX33Vae/duzcvYwIqq+Li6P8TOO200/I4kv8qKiqysR6bb6z/93//Z+Pvf//7WR8XkAs6fbZ169Y2njp1ar6H4+jQoUNk39KlS/M4EqBqOuWUUyL7SktLnTYpDAAAAAAA4LDAAwQAAAAAABDEAwQAAAAAABBUqc5AGDhwoI1PPvnkyNfNmzfPaf/sZz/L1ZAAHILu3bvbOJPJOH3PP/98vocDVDrXXXedjQ8ePFiBI/mmAQMG2Pikk05y+uRY9bjlGQhAZbFz506nvXjxYht369bN6ZNn/mzdujUn4znyyCNtPGTIkMjXvf322zm5P1CV9e7d22kPHz488rXbt2932hs2bMjJmPKJHQgAAAAAACCIBwgAAAAAACCIBwgAAAAAACCooM9AaNq0qdO+++67bVy9evXIn5N5Z8YYs2vXruwODEAqLVq0cNpnnXWWjT/88EOn78UXX8zLmIDKTJ4zUBGaN29u4+OPP97pk2u2z+bNm532vn37Dn1gQJ6VlZU57dWrV9v4sssuc/peeeUVG48fPz7V/bp06eK027dv77TbtWtnY33GkFRoZ6cAlYH+G7W4OPr/5OfOnZvr4eQdOxAAAAAAAEAQDxAAAAAAAEBQQacw3HrrrU67Z8+eka+dMWOGjSnbCBSmH/zgB05blpmaPXt2nkcD4FCNGTPGxqNHj479cx9//LGNr7rqKqdv3bp1hzwuoKLJ76JFRUVO38UXX2zjqVOnprr+li1bnLZOU2jWrFms6zz11FOp7g8cznylUUtLS532448/nuvh5B07EAAAAAAAQBAPEAAAAAAAQBAPEAAAAAAAQFBBn4Fwyy23xH7tDTfcYGPKNgKFqW3btpF927Zty+NIAKQxa9Ysp925c+dU11m2bJmN33777UMaE1CIVqxYYePLL7/c6TvxxBNt3LFjx1TXf/755739U6ZMsfGIESMiX6fLTwIoX5s2bWw8fPjwyNdt2LDBaZeUlORsTBWFHQgAAAAAACCIBwgAAAAAACCooFMYkmjSpImN9+3bl/o627dvj7xO9erVbdywYcPIazRq1Mhpx03FOHDggNO+8847bbxnz55Y1wAKWf/+/SP7XnrppTyOBKgaZHm44uLo/xO46KKLIvueeOIJp92qVavI1+p7HDx4MDTEcg0YMCDVzwFVweLFi8uNs+k///lPrNd16dLFaS9dujQXwwEqvTPOOMPGvvV2xowZ+RhOhWIHAgAAAAAACOIBAgAAAAAACOIBAgAAAAAACKoyZyC8//77WbnOc889Z+NPP/3U6TvqqKNsPGzYsKzcz+ezzz6z8QMPPJDz+wG50Lt3bxu3aNGiAkcCVD0TJ0608bhx4yJf9/LLLztt39kFSc41iPvaSZMmxb4mgEMnz0eRscaZB0A8TZs2jezbsmWLjR955JF8DKdCsQMBAAAAAAAE8QABAAAAAAAEFXQKw6xZs5z2JZdckvN7Dh06NNXP7d+/38a+LZ0zZ8502iUlJZGvfeutt1KNBSgkgwYNsnG1atWcvkWLFtn4zTffzNuYgKpi+vTpNr799tudvubNm+f8/ps3b7bx8uXLnb5rrrnGxjolEEBuZTKZcmMA6VxwwQWRfevWrbPx9u3b8zGcCsUOBAAAAAAAEMQDBAAAAAAAEMQDBAAAAAAAEFTQZyAMHjzYad9xxx02rl69euzrnHDCCTZOUn5x8uTJTvvjjz+OfO0LL7xg4xUrVsS+B1DV1KlTx2n369cv8rXPP/+8jQ8cOJCzMQFV1dq1a218xRVXOH2XXnqpjW+++eac3F+WGJ4wYUJO7gEguVq1akX2lZWV5XEkQOWk/9bs0KFD5Gv37t1r43379uVsTIWCHQgAAAAAACCIBwgAAAAAACCooFMYtHHjxh3yNYYPH56FkQCIordubdu2zca6jOkjjzySlzEBhwNdClW2X3vtNadPllgcMGCA0yfn6RNPPOH0FRUVOe1ly5alGyyAnBo5cqSNS0tLnb5f/vKX+R4OUOkcPHjQaZeUlNi4S5cuTt+qVavyMqZCwQ4EAAAAAAAQxAMEAAAAAAAQxAMEAAAAAAAQVKnOQABQ+PQZCGeccUYFjQTA11599VVvG0DV8t5779l4/PjxTt8bb7yR7+EAlY4uLz5mzBgbZzIZp2/BggV5GVOhYAcCAAAAAAAI4gECAAAAAAAIKtJbMLwvLiqK/2Lk2oJMJnNKRQ8ChYG5WTgymUxR+FU4HDAvCwprJizmZkFhbsJibhaUyLnJDgQAAAAAABDEAwQAAAAAABDEAwQAAAAAABDEAwQAAAAAABDEAwQAAAAAABDEAwQAAAAAABDEAwQAAAAAABDEAwQAAAAAABDEAwQAAAAAABDEAwQAAAAAABB0RMLXbzHGrM3FQJBY24oeAAoKc7MwMC8hMS8LB3MTEnOzcDA3ITE3C0fk3CzKZDL5HAgAAAAAAKiESGEAAAAAAABBPEAAAAAAAABBPEAAAAAAAABBPEAAAAAAAABBPEAAAAAAAABBPEAAAAAAAABBPEAAAAAAAABBPEAAAAAAAABBPEAAAAAAAABBPEAAAAAAAABBPEAAAAAAAABBPEAAAAAAAABBPEAAAAAAAABBPEAAAAAAAABBPEAAAAAAAABBPEAAAAAAAABBPEAAAAAAAABBRyR5cVFRUaa4+H/PHDKZjOzL3qhSXFOOJRuvM8YY+buGruO7bi7+bQ4ePLglk8k0z/qFUSmlnZtJ5kMu5GKO62um/R3TzNuDBw+aTCaT/QmPSqm4uDhyXmry/ZbveVnInxHZwpoJqbi4OFOtWjXbTvt91vdzab8Xpr1mvmXru+3+/fuZm7Aqy7pZWfk+U3TfgQMHIudmogcIxcXFpl69era9b98+G8sP4q9fKwYQe7Dy5444wh1e4JeMHPf+/fsjX+d7SFCzZs3I+x88eDDyHvpNLO+hxx13EdHj3Llz59rIH8Rhp7i42NStW9e2v/rqKxvruSnb8n1rTO4fCmryHvo9ru8vP2/0/JM/q6/j+2zw3c83b6Pm5p49e2LdC4eH4uJiU79+fdvO97z0faHyzRnJNxZjvvl7SHqeppFkzfS9bseOHayZsKpVq2aaNWtm27mYm3Hnn+8/pHzXTPIHUy4eRITW7Lhj2bRpE3MTVnFxsWnQoIFtV5bvs9n6j2TfQ5G0/5Em23G/zxpjTGlpaeTcJIUBAAAAAAAE8QABAAAAAAAEJUphMMa/1UGSW0l8Wzd8Wxy//PJLp532OnKcNWrUcPr0dhC53bmsrMzpkykNvrFkK38OSCIbczMX70Hflis5b/Uc1lvVqlevbmPfVjX9u/vSqWS7kHNNUTllMhnnfZ2NNVOLmwbn2wqZZJuk73fQY4nKMTfGv2b7tlvGxXqKkGzMzbTnh8i54UuJ9aXW+dKD9c/6xh13jS7vOlH3z9Z3ZByeKnJuxpXkO6Mv1U9/n027Nsb9uWxhBwIAAAAAAAjiAQIAAAAAAAhKnMIQtX0i7WmwemuK3ILh27olT2Q3xt0eIk/s1NfU9P1r1aoV+XP6nlLcrWI+vu1nQEiauZmPbftxtyOHTmyXc1zP27gVWpJs8YxbRgiIUlRUlPU108dXAUjPGblt0pc+5KuGpH/W9zslmXs+lO1CtmRjbqa5fpL7pU0nCr3Wt2b65rT8PEjy/SFt9QgcnvI5N9NKmzJkjPu91Pd3cLaq+knZ+jdkBwIAAAAAAAjiAQIAAAAAAAjiAQIAAAAAAAhKfAZCVNm1tDmMSc4nkLlXvXv3dvouvfRSG3fv3t3pq1u3ro1lKThjjPnHP/7htJ944gkbv//++06fPFtBlnQ0Jjc5OuSMIQmZU5V2buaC733sy+HUnw3yZ+vXr+/0lZaWRl5H/v46n1uflxJ1P5/Q2Q04fOkyjmnnZdp8Zbne+ebe3r17I/saNWrk9O3evTvusB26jJU8U8hX4irt2se8REi252YSvhLC8vulXiPlmhUqvxp37dXrovy30HNT/pyeY3HLyPF9Fj7ZWjdzwTenkpRY9JUeT3uGmW9NjfvvlmTdZAcCAAAAAAAI4gECAAAAAAAISpzCELWdwrc9wlc+Sm+zkNu69NbJe++918YXX3yx0ye3Xeg0BTnmGjVqOH1Dhgxx2j169LDx9ddf7/QtXLiw3Gsa4y+twXYt5Fomk4k9N31bsPL9XvVtv9btE044wcaPP/640zd+/HgbP/fcc06fnJvy80W3fdvItKh/J+Y6tDTz0ncN3+t866kmtzvWrl3b6bvvvvtsPHr0aKevpKTEaY8cOdLGGzdudPrkdmt9D18qo28rKHMM2ZKNuenj24os3/P6e6lci8rKypw+mW4QWjN925Hla+vUqeP0ye/Q+v6+8q++bdnMWySR7XUzLd/3QN9YQuk9vvRduW76UiF06pHv8ybN99kQdiAAAAAAAIAgHiAAAAAAAIAgHiAAAAAAAICgRGcg6NIavtJLvvwOSZ9XIEsu3n333U7fRRddFHl9WcZN52iuXbvWxi1btnT6dDlIWR5On7OwYMECG+t8El9JHjlWX14KOWI4FFH5h2lLrGaL733tK8/z5ZdfOu0LL7zQxl26dHH6atWqFXkPeR1fadgkpXSi/p0qusQQCo8vL1iK+97R70XfOizzleWZB1rXrl2dtlz75DWMMaZbt25O+9xzz7XxX/7yF6dPru86B1SOO8l6GjeXk7kIH31uUNq5mbbcr7zmqaee6vRdffXVNtZnkMj8aH3GSZLv5HLNbNq0qdMn5/y2bdsix53234W5iZBC/T7rk+R8Inl20E9+8hOn76WXXrLxnXfe6fT5zg7y/a3pk/bfiR0IAAAAAAAgiAcIAAAAAAAgKFEKQ1FRUeQ2L185Gd0nt13t3r3b6evQoYONBw8eHHnN1atXO33XXnutjT/66KNvjPtrjRs3dvqmT5/utI8++mgb6y3UstSO3tbpK63jQ9oCsiHJ3My3uCXoNP37dOrUycZbt251+uQ81lvFfGUcfduo45YHYjsmfNKmJkTR88JXqq1mzZqRr/v2t79t44ceesjpa9KkiY31WqfbRx11VOTY5PzylZjTc9a3FTvuv1NFf+6hsBUVFWVlbvquId/j+vukvKZOYTjvvPNsrEs8yp/Tc9FHr31yrg4cONDpkykNv/71r50++XvotChdVg5IK2pe+f7WTPIdLm76eJLvd75UC7mmGmPMD3/4Qxu3atXK6evZs6eN9fyXnyl6/vv+LXy/b9rvs+xAAAAAAAAAQTxAAAAAAAAAQTxAAAAAAAAAQYeUsBQ3x9BX1lDnZbVr167c1xljzLJly2ysSzyuW7fOxjrXTF5Hl6tq0KBB5Ljff/99py3zveKWqQQON76Sh74caf05IXOrjTHmuOOOs/Ezzzzj9O3Zs8fGOi9M3sNXSosSq6hIcfMwfSUP9fyS7TPPPNPpGz9+vI1bt24deU29Rutxyuvqs0nee+89G69Zs8bpKysri7yHL5eUs0mQC2lzqeOWmNPnA8jzSc466yyn77PPPrOxXNuM8Zdj1eOUY9Nr386dO22s86z79u1r4wkTJjh9ct767u8bC+spColvvvu+F+qf851Jctpppznthg0b2njHjh1O3/r16yOvGffcLt85QtnCX8EAAAAAACCIBwgAAAAAACAoZzVXfFu5fFuZXn/9dRsPGDDA6ZPbunbt2uX0ye0a9evXd/qaN29u49tuu83pa9asmdOW27pWrFjh9MUt16O3iuVjKwkOb5lMpmC27ybZDqa3LkuXXHKJ0/7Wt75l4z//+c9On9zm5SsX59uOprHNEtkWt8ySbw75tvvrPlmObfLkyU6fLOOoUw985ef0VmxZgq5Xr15O3xdffGHjn/70p07fvHnzyr2fMe7v4SsNp1M2fJ8ngI9vLYi7TvjmpkxZMMZd37p16+b0Pf744+Ve3xh3rfOVeAyNW6ZCHHnkkU6fTB+sV6+e07dly5bI68dNUyC9CJWV7/us/FtP/9135ZVXOm35eaCvM3PmzMjryLVSr435Xv/YgQAAAAAAAIJ4gAAAAAAAAIJ4gAAAAAAAAIJydgaCzNPQeYoy/0nnd8h8y0WLFjl9Mt/rq6++cvrq1q0b2XfXXXfZuHPnzk7f3r17nbYsO7Vp0yanz3eWge/3Baq6uPmOvvMJ9Ly56qqrIu+xcuXKyOvELfMG5Fvc0qC6L+5rdZ71mDFjbKzP+5HnCOm5V7t27XLHbMw3865lCSp9loEsk3zzzTc7fW+99ZaNdS6nXpcl378h8xtJROXip31f6fex/C4qS5QbY8ztt99uY72ePfbYY+Vewxh3/vnKEut+Xy5127ZtTRQ9F+XZCb6zWjhfCIUkW+9H3/ojzyA49thjnT59zon8rPj888+dvjlz5sS6v68vH+eM8JcuAAAAAAAI4gECAAAAAAAIyksZx7jlD42JvwVKb7mUpW1OOOEEp+/MM8+MvE5ZWZnTfvjhh23s20aZpDQj27WA/5LbH40xpnr16jbW5aLat2/vtGX5KD035eeI/tyQ28p8pfOYp8gn33Zj39ZgTa6Fo0aNcvr69+9vY12OUbZlqoExxmzevNnGM2bMcPpmz57ttGXaxODBg52+yy67zMa6xOO1115r44ceesjpkymJSUpTUR4OSch55Us9jbs26PefXN/Gjh3r9MkSq7feeqvTt3v3bhvrtAj53VPPDd849Wtl6cZOnTo5ffLfQqYoGRM/JVGT42aeIt98JU6TrLdxSzHrtVCuafo6sqSxMe781+TngR5nNj7DkmAHAgAAAAAACOIBAgAAAAAACOIBAgAAAAAACMrZGQhS3BJvxrg5HDqfQ15H5pYZ45a2ue2225y+Jk2aRN5flssxxpglS5ZEvlaOx5dPA1R1afOp9JyWJar69Onj9Oncz1WrVsW6ru8MBN/ZJcxp5JPv/ebLZaxVq1Zk+3vf+57TV6dOHRvrvEpfGakJEybY+Nlnn3X6tm3b5rTl2rt8+XKn75hjjrGxPgNh9OjRNtZnmkyZMsXGes7Kzwz9GcE5JsiGJPnSvu+FXbp0sXGPHj2cPlmmfPHixU6ffF/rcwV8Z6Xo18qx6TMQRowYYWN9/lBJSYmN9+zZE3lNX+445cxR0bJRutH3d6hem+R6q8/e02OR54FNnz498p6+cxb0nE5yNl82MMMBAAAAAEAQDxAAAAAAAEBQhaQw+LaOyC1Y+nVye4bcNmmMMVdffbWN9dYRueVx7ty5Tt9TTz3ltH0lMnwlatJulcnGFhsg33xbJ33vW739Ws7NIUOGOH36On/4wx9srLeVyfvrbVxxS/IA+ZRkDsk+vW1Rbpts2bKl0yfXSb1m/vWvf7WxLqO4Zs0aG+vUB30dORf1dudnnnnGxnoLd5s2bWysy9jJ0pG6jJz8t2FuIxd8W/N93wP13LznnntsrNes3//+9zbW73E5x3xj0X06pUf2t2jRwukbPnx45Nh++9vf2lj/vvKaeq2NW3KVuYl8yMbfXvr975t/ck3r2LGj06fn5kcffWTjhQsXOn2+zxhfX76xAwEAAAAAAATxAAEAAAAAAATxAAEAAAAAAARVeBlH32t1XpYse9G5c2en77rrrrOxztHcsmWLjR944AGnb+fOnU7bV65H5nfpscUt+UZpOFQFSUrQ+d7z3/72t23cu3dvp2/Tpk1O+/XXX7exzr305Y/75jRQUZLkZ/rOQGjQoEG5sTFu3uXf/vY3p+/HP/6xjeVZJMa4a6g8Y8GYb849uS6XlZU5fdOmTbOxPKfIGGO6detm45o1a0b2zZ8/P/L+aXOwAZ+453Tp1+qS4ccdd5yNN27c6PS98847Ntbvf996Jtv79++PPW69vjZr1ixybAsWLIgcm5zvrLUoZPI9mOQsPkl/n5VzTs4FY4w599xzI39Or02vvPJKudc0xp3/+uwE2ecrlZqPsuTsQAAAAAAAAEE8QAAAAAAAAEF5SWHwbZ3wlaTwbTGR5XGMMaZ+/fo21tsxZUmozZs3e+8ht4ToscntkklKPPrE3fLFdjAUEl9pKd/c+PLLL52+QYMG2bhRo0ZO35w5c5z23r17I+8v561vTgMVKe7nuG890etbq1atbKzf+7t27bLx/fff7/TJ7Ze+lKDatWs7ffoecvulvo6kS7jKe9PY2MsAAA8GSURBVOht0s2bN7ex/syQKRV6LL40QyAtOW/13JTpPnK+GWNM9erVbbxq1Sqn7+OPP7Zx48aNnT45H3RakK+Moh6b/Kzo2bOn0yfnx8qVK50+Oed8a20SlG5EZeFLfZDzXc+3c845x8Y69UCXQ54+fXrk/X1lXKPGWd5Yc40VFgAAAAAABPEAAQAAAAAABPEAAQAAAAAABFVIGUdfDpcvh+O8886zcffu3Z2+PXv22Fjncz355JM29uWvGePmUOr8El9eTNTrQuR49NhkrhlnICDffPPW9/73lXHUff3794/sk2eX6Hv65qYul+PLywYqSto1RL+/jz32WBvr9/rWrVttvGHDBqdPrjcyV9sYdy7qHGxNnqWgxzZixAgbN2zY0Onbtm1b5DU/+eSTyD7fGi1/f9ZMJOE7x8f3XUz2yXN6jHHnVevWrZ0+eW6XJq9Zr149p0+ea6DnkH5tp06dbNyrV6/I+33wwQeR9/d9TiVZazkDAZWF78wTSZ/bdcwxx9hYv9/XrVvntOUap7/7+j6L4p6JkI/5xg4EAAAAAAAQxAMEAAAAAAAQxAMEAAAAAAAQlJczEHTOhswp0bkfMoeqSZMmTt8999xjY312gcwLmzp1qtO3Y8cOG+ucrSQ1Nn19uaiNK/tkrW0gH5LUmPWdQSDrv+ucsXbt2tl4zZo1Tt+bb77ptH2fG/qecZGXiVyLOrvDt74kycE+/fTTbVy7du3Ia8o10hi31rys+677dF6zzvOW/b1793b6xowZY+O6des6ffJ3XLJkidMn275/J84NQrakPY9K1nvX7/HXX3/dxueff77T99RTT9m4pKTE6Vu8eLGN69Sp4/Q1b9683HsbY0ybNm2c9oABA2ys1175ebB06VKnT/7++rPId84I6ykKVZL3qu+zQM6bs88+2+mT819f/+9//7vTlmcL6e+z8uyUJGcg5Hv+sQMBAAAAAAAE8QABAAAAAAAEVUgKgzMAtQVLbkW+8cYbnb727dtH/pzcKjZ//nynT24P0T/nKyOpt4PIsfnKbvj6fHQpLflzvlIiQC4k2Q7lS+GRW7UGDRrk9MnSbvPmzXP6du/e7bTjlrajbCMKVdw5pT/vfaWPW7RoYWNdclHOyyOPPNLp27hxo411WoK8v14z9Ty8/PLLbfzggw86fTLVUF9Hll5++OGHnb4vvvii3N9B8/07AWn53nN6/sn0Uj2PRo8ebWP9Hu/Tp4+NzznnHKdPphDJFEBj3JRcOU/KG5ucH3puyDXzo48+cvriloNLsr0aqEhJSo/L1+p0eVm2+MILL3T65FzVnwVz5sxx2vq6km/+FdIcYwcCAAAAAAAI4gECAAAAAAAI4gECAAAAAAAIytoZCElKZPhK4shSM3379nX6ZH6XzN80xph7773Xxhs2bHD6fPlkuvybr1yi/J30z8n8Tv27y7Ifuk/mqNWvXz/y3ro8ly67BWSbr/yqJueDznWW73+Z92mMOzeXLVuWemy+s0uAQuTLbdRzTeZL6j45hzR5johe2+S81Ndo1qyZjTt16uT0XXnllU77oosusnHjxo0j76/zTKdNm2bjt99+2+mTa73+PPGVc2XuI9f02iPnle/73Q033OD0HXfccTbWJRbr1atnY32ugfx+q89A0HN87NixNu7Xr5/Tt2vXLhtv27bN6ZN53r6Sycw3FBLfmur7PqvXVHkdvf7IM4d02WL596U+V2T58uVO21d6XM7jQjrzQGMHAgAAAAAACOIBAgAAAAAACMpaCkOSUhNy64Z+ndwu0qRJE6dPbuWSJaCMMaZz58421luuZGkrvVVFt+XP6rKKcqx6q1iDBg1s3Lp1a6evZ8+eNu7atavTJ++xePFip09u8SzkbSyompJsB5P0/JOlGk899dTIe6xfv97p0/NPbqvU95f39JXg0phXyLW4JdF888tX0nfLli02Pumkk5y+o446ysbDhg1z+hYuXGhjvb154MCBNq5bt67Tp8tPybHqbZlyK/b27dudvieffNLGvpQ835qt/53ifkYBmu/94ivTLb+Xhr5fSkuXLrWxL5VWp93KPp3aqu+3aNEiG/fv39/pk/NYzlNjjCktLbWx73tA2hQG5iZyIe4c1q/1lTjV5Rhl6pFO/ZPrmPz7zZhv/s0o2zpNwvd5E/W6isAOBAAAAAAAEMQDBAAAAAAAEMQDBAAAAAAAEJS1MxB8fHlhuk/mnuzYscPpkzkjMrfTGGMmTJhQ7jWMcfPEdM7Ypk2bnPaSJUtsrEvkdOzY0cZt27Z1+mQJRp27LX9HnQcjc2++853vRI5N5ssZY8xnn31mgK8VFRXFziuUeVO+n0lSmlX26ZyxDh062Fieh6BfW1JS4r2/r/xrmt89yc8B2eA7y0C+F315j7qs26xZs2zcq1cvp0+uhTfddJPTJ8s41qlTx+nzlabTbV8JVzm/H3nkEadv3bp1Ntb/FnIN1feT/zZxP5OAfNBz0/fdT84VPd/l+1+vUb6SrvpsMHn+l/5evHPnThvLko7GxP+c0uKej8DcRC6kPbfLR88b+befPJfLGPdvxtmzZzt9+rPBV3457nlIFY0dCAAAAAAAIIgHCAAAAAAAICgvKQy+rch6W4fcLvLuu+86fUOHDrWxLjsjt2PJ2Bh3C5be/tGsWTOnLdMIfFtMfGWndN/KlSttrFMm5BYYXcZRltnSW9MAn7hpCkn4tlX5tmreeOONNm7UqJHTt3HjRht//vnnscfiSwUCCpVvXvpSGHylj19++WUbjxw50uk7/vjjbazXWpkiqNcs3xZqPTa5/XnBggVO369+9Ssb6xQluYVbl4b0rdnMdVQkX0qNj05tldfRqT+++/k+J3Rbfr/VW7FlCpPeiu0rTRm3xJwPcxghUekwufg+6yu3qu8n11Q9pz/55BMby/LK5V0nbklnX19FpzOwAwEAAAAAAATxAAEAAAAAAATxAAEAAAAAAAQlPgMhKjcjST6HzJvSucwyL/O+++5z+ubOnWtjXUZRlmfUJR5bt25t42OOOcbp0+WrNm/ebGNZZkq39XkFH374oY3Xr1/v9JWWltrYl9viyyfTOWqAFnVGQdocYt+c1u9HmcOpc6379OkTec0nn3zSxvrsEj0f5HXTlnHMRf4c4JON94qvrJosHTVq1Cin7/rrr7fxd7/7XadPniNUq1atyHvrPM9PP/3UaU+aNMnGU6ZMcfrknNafC/IzQ/9Ocu77crCBQ5FmLdTvP1/+dNQ1ymun6dNrpD7LRH4X1t+15ffS7du3O32+0qy+z6K0ZyIAPrkuHeo710S/x1u1amVj/T1Yzil9bp1eY+OeKRYaa9Q184HZDgAAAAAAgniAAAAAAAAAghKnMMitFb7tSnHLVfns3r3bac+cOdPGejuW3HKlxyXHordj6lKNcguY3oIiX+vbuuXbquLbTq7L7ABxZTKZ2HMzLXlN3/X1+3/NmjU2XrJkidP3xz/+0cY6nUjPcSnf25iTlOABpKh56dsKrd9vct3S64S8zvLly50+mcJw+umnO32yLHLHjh2dPjn3Fi1a5PQ9++yzTnvbtm021mtmLrZpRl0/yc8BadfMJNuE06YS+vp833UbN27stGX6boMGDZy+6dOn29hXYjJJudmo1+lrMk8RkqZ0Y9q56evTaQqyPGObNm2cPjlmmVZvzDfXu7SfDXHnju+a2bi+MexAAAAAAAAAMfAAAQAAAAAABPEAAQAAAAAABCU6A0HnjMWVNk9q7969Tp8sJ+MrO6XHKNuhEjxyrDqHJW4Jxq+++srp85W90aWtou5H2TiEpDlDwzf/fHNDl3aS19E/N2zYsMgxyrmir6n5cijT5kynfW1Uvjq5nZDSrpl6XfDlavvWN5nbvHDhQqdPtvUY5RkIel7q+8t+X96lLwdUzzXfZ1nc/E3WTIREzc0k7520uf1xc7B9c0GPX58j9Oqrr9r46aefdvpmz55tY/1dW58NFjW2JOe4MDeRhHzf+/5OkpJ8n/WtP7Kt/w6UZYv133pvvPFG5P30HKtRo0assfkkOWck7bz1YQcCAAAAAAAI4gECAAAAAAAISpTCUFRUFJmOELekY3nXjBLa0iz5tnH6trHoEh2S3jYjryO3n4TE3VaSZPsNoMn3q3zv6LkZd4uSb6t03NQbTZeLktfRc7GQtkDq+0X9W7M1E1JRUVHseSn51kxfmWK9ZqYtDSmvE3pPy/snSb3IxlzR18hF+VpUTb65mfa9ma2fk209p+Q6qT8nSktLnfbkyZNtrMsiy3volIW0Zdh932eZm0gizfdZ399Xvvefb03V1ywpKbHxggULnL4mTZpE/lzctKDy2mkkKRObdm4yowEAAAAAQBAPEAAAAAAAQBAPEAAAAAAAQFCiMxCMcXMlfKVmovJXdDtJqQnJdyZAkjwYX+5J2pJUSUr5+HJUKeOIuPT5JL65KV+XpByiL0/KNx/jlnbSKrokory/79+QuQmfqPeKzkn2rZlSknJU8t5Jzv/x5UDrsfk+T+Ke8ZOEb17G/TcEjIn/fdb3Hs8G3zqs5408nyR0bpDvnIO03wPSfi9nbiKuJN9n45b1Dd0v6uf0e1qeHaTPJ5FlHUOlJ+XvkY/vkL6/uzkDAQAAAAAA5AwPEAAAAAAAQFDiMo6yfOHevXttnGS7f9yyjkm2Ufm2Svq2TqUtj5itrde+9Aq5xTVJSUscfvTcLCsr8772a765qeeRr1SqfH/65ph+jyfZxhW3zFaS7WBpS6zKualLUwJfKyoqct4f2VgzfWkKvq3XvlJVPknmk289TbtNM0kZaOYl4vLNzfJe+7Uk6QaHMrao+0mh74Xy88BXxtH3nVn3yWsmSRdmbiKuJHPTt/7FnZt63ZLvf/0+9r135XX0fEvy/TLqdeVdJ0o+vs+yAwEAAAAAAATxAAEAAAAAAATxAAEAAAAAAAQVJczX32yMWZu74SCBtplMpnlFDwKFgblZMJiXsJiXBYW5CYu5WVCYm7CYmwUlcm4meoAAAAAAAAAOT6QwAAAAAACAIB4gAAAAAACAIB4gAAAAAACAIB4gAAAAAACAIB4gAAAAAACAIB4gAAAAAACAIB4gAAAAAACAIB4gAAAAAACAIB4gAAAAAACAoP8H7j+B3SXoUVUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1440x288 with 10 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "ae = load_model('mnist_ae.h5')\n",
    "enc = load_model('mnist_enc.h5', compile=False)\n",
    "\n",
    "decoded_imgs = ae.predict(x_test)\n",
    "n = 5\n",
    "plt.figure(figsize=(20, 4))\n",
    "for i in range(1, n+1):\n",
    "    # display original\n",
    "    ax = plt.subplot(2, n, i)\n",
    "    plt.imshow(x_test[i].reshape(28, 28))\n",
    "    ax.get_xaxis().set_visible(False)\n",
    "    ax.get_yaxis().set_visible(False)\n",
    "    # display reconstruction\n",
    "    ax = plt.subplot(2, n, i + n)\n",
    "    plt.imshow(decoded_imgs[i].reshape(28, 28))\n",
    "    ax.get_xaxis().set_visible(False)\n",
    "    ax.get_yaxis().set_visible(False)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generate counterfactual guided by the nearest class prototype"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Original instance:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAM3ElEQVR4nO3dXahc9bnH8d/vpCmI6UXiS9ik0bTBC8tBEo1BSCxbQktOvIjFIM1FyYHi7kWUFkuo2It4WaQv1JvALkrTkmMJpGoQscmJxVDU4o5Es2NIjCGaxLxYIjQRJMY+vdjLso0za8ZZa2ZN8nw/sJmZ9cya9bDMz7VmvczfESEAV77/aroBAINB2IEkCDuQBGEHkiDsQBJfGeTCbHPoH+iziHCr6ZW27LZX2j5o+7Dth6t8FoD+cq/n2W3PkHRI0nckHZf0mqS1EfFWyTxs2YE+68eWfamkwxFxJCIuSPqTpNUVPg9AH1UJ+zxJx6a9Pl5M+xzbY7YnbE9UWBaAivp+gC4ixiWNS+zGA02qsmU/IWn+tNdfL6YBGEJVwv6apJtsf8P2VyV9X9L2etoCULeed+Mj4qLtByT9RdIMSU9GxP7aOgNQq55PvfW0ML6zA33Xl4tqAFw+CDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJ9Dw+uyTZPirpnKRPJV2MiCV1NAWgfpXCXrgrIv5Rw+cA6CN244EkqoY9JO2wvcf2WKs32B6zPWF7ouKyAFTgiOh9ZnteRJywfb2knZIejIjdJe/vfWEAuhIRbjW90pY9Ik4Uj2ckPS1paZXPA9A/PYfd9tW2v/bZc0nflTRZV2MA6lXlaPxcSU/b/uxz/i8iXqilKwC1q/Sd/UsvjO/sQN/15Ts7gMsHYQeSIOxAEoQdSIKwA0nUcSNMCmvWrGlbu//++0vnff/990vrH3/8cWl9y5YtpfVTp061rR0+fLh0XuTBlh1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkuCuty4dOXKkbW3BggWDa6SFc+fOta3t379/gJ0Ml+PHj7etPfbYY6XzTkxcvr+ixl1vQHKEHUiCsANJEHYgCcIOJEHYgSQIO5AE97N3qeye9VtuuaV03gMHDpTWb7755tL6rbfeWlofHR1tW7vjjjtK5z127Fhpff78+aX1Ki5evFha/+CDD0rrIyMjPS/7vffeK61fzufZ22HLDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJcD/7FWD27Nlta4sWLSqdd8+ePaX122+/vaeeutHp9/IPHTpUWu90/cKcOXPa1tavX18676ZNm0rrw6zn+9ltP2n7jO3JadPm2N5p++3isf2/NgBDoZvd+N9LWnnJtIcl7YqImyTtKl4DGGIdwx4RuyWdvWTyakmbi+ebJd1Tc18AatbrtfFzI+Jk8fyUpLnt3mh7TNJYj8sBUJPKN8JERJQdeIuIcUnjEgfogCb1eurttO0RSSoez9TXEoB+6DXs2yWtK56vk/RsPe0A6JeO59ltPyVpVNK1kk5L2ijpGUlbJd0g6V1J90XEpQfxWn0Wu/Ho2r333lta37p1a2l9cnKybe2uu+4qnffs2Y7/nIdWu/PsHb+zR8TaNqUVlToCMFBcLgskQdiBJAg7kARhB5Ig7EAS3OKKxlx//fWl9X379lWaf82aNW1r27ZtK533csaQzUByhB1IgrADSRB2IAnCDiRB2IEkCDuQBEM2ozGdfs75uuuuK61/+OGHpfWDBw9+6Z6uZGzZgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJ7mdHXy1btqxt7cUXXyydd+bMmaX10dHR0vru3btL61cq7mcHkiPsQBKEHUiCsANJEHYgCcIOJEHYgSS4nx19tWrVqra1TufRd+3aVVp/5ZVXeuopq45bdttP2j5je3LatEdtn7C9t/hr/18UwFDoZjf+95JWtpj+m4hYVPw9X29bAOrWMewRsVvS2QH0AqCPqhyge8D2m8Vu/ux2b7I9ZnvC9kSFZQGoqNewb5K0UNIiSScl/ardGyNiPCKWRMSSHpcFoAY9hT0iTkfEpxHxL0m/k7S03rYA1K2nsNsemfbye5Im270XwHDoeJ7d9lOSRiVda/u4pI2SRm0vkhSSjkr6UR97xBC76qqrSusrV7Y6kTPlwoULpfNu3LixtP7JJ5+U1vF5HcMeEWtbTH6iD70A6CMulwWSIOxAEoQdSIKwA0kQdiAJbnFFJRs2bCitL168uG3thRdeKJ335Zdf7qkntMaWHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSYMhmlLr77rtL688880xp/aOPPmpbK7v9VZJeffXV0jpaY8hmIDnCDiRB2IEkCDuQBGEHkiDsQBKEHUiC+9mTu+aaa0rrjz/+eGl9xowZpfXnn28/5ifn0QeLLTuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJMH97Fe4TufBO53rvu2220rr77zzTmm97J71TvOiNz3fz257vu2/2n7L9n7bPy6mz7G90/bbxePsupsGUJ9uduMvSvppRHxL0h2S1tv+lqSHJe2KiJsk7SpeAxhSHcMeEScj4vXi+TlJByTNk7Ra0ubibZsl3dOvJgFU96Wujbe9QNJiSX+XNDciThalU5LmtplnTNJY7y0CqEPXR+Ntz5K0TdJPIuKf02sxdZSv5cG3iBiPiCURsaRSpwAq6SrstmdqKuhbIuLPxeTTtkeK+oikM/1pEUAdOu7G27akJyQdiIhfTyttl7RO0i+Kx2f70iEqWbhwYWm906m1Th566KHSOqfXhkc339mXSfqBpH229xbTHtFUyLfa/qGkdyXd158WAdShY9gj4m+SWp6kl7Si3nYA9AuXywJJEHYgCcIOJEHYgSQIO5AEPyV9Bbjxxhvb1nbs2FHpszds2FBaf+655yp9PgaHLTuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJMF59ivA2Fj7X/264YYbKn32Sy+9VFof5E+Roxq27EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBOfZLwPLly8vrT/44IMD6gSXM7bsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5BEN+Ozz5f0B0lzJYWk8Yj4re1HJd0v6YPirY9ExPP9ajSzO++8s7Q+a9asnj+70/jp58+f7/mzMVy6uajmoqSfRsTrtr8maY/tnUXtNxHxy/61B6Au3YzPflLSyeL5OdsHJM3rd2MA6vWlvrPbXiBpsaS/F5MesP2m7Sdtz24zz5jtCdsTlToFUEnXYbc9S9I2ST+JiH9K2iRpoaRFmtry/6rVfBExHhFLImJJDf0C6FFXYbc9U1NB3xIRf5akiDgdEZ9GxL8k/U7S0v61CaCqjmG3bUlPSDoQEb+eNn1k2tu+J2my/vYA1KWbo/HLJP1A0j7be4tpj0haa3uRpk7HHZX0o750iEreeOON0vqKFStK62fPnq2zHTSom6Pxf5PkFiXOqQOXEa6gA5Ig7EAShB1IgrADSRB2IAnCDiThQQ65a5vxfYE+i4hWp8rZsgNZEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoMesvkfkt6d9vraYtowGtbehrUvid56VWdvN7YrDPSimi8s3J4Y1t+mG9behrUvid56Naje2I0HkiDsQBJNh3284eWXGdbehrUvid56NZDeGv3ODmBwmt6yAxgQwg4k0UjYba+0fdD2YdsPN9FDO7aP2t5ne2/T49MVY+idsT05bdoc2zttv108thxjr6HeHrV9olh3e22vaqi3+bb/avst2/tt/7iY3ui6K+lrIOtt4N/Zbc+QdEjSdyQdl/SapLUR8dZAG2nD9lFJSyKi8QswbH9b0nlJf4iI/y6mPSbpbET8ovgf5eyI+NmQ9PaopPNND+NdjFY0Mn2YcUn3SPpfNbjuSvq6TwNYb01s2ZdKOhwRRyLigqQ/SVrdQB9DLyJ2S7p0SJbVkjYXzzdr6h/LwLXpbShExMmIeL14fk7SZ8OMN7ruSvoaiCbCPk/SsWmvj2u4xnsPSTts77E91nQzLcyNiJPF81OS5jbZTAsdh/EepEuGGR+addfL8OdVcYDui5ZHxK2S/kfS+mJ3dSjF1HewYTp32tUw3oPSYpjx/2hy3fU6/HlVTYT9hKT5015/vZg2FCLiRPF4RtLTGr6hqE9/NoJu8Xim4X7+Y5iG8W41zLiGYN01Ofx5E2F/TdJNtr9h+6uSvi9pewN9fIHtq4sDJ7J9taTvaviGot4uaV3xfJ2kZxvs5XOGZRjvdsOMq+F11/jw5xEx8D9JqzR1RP4dST9vooc2fX1T0hvF3/6me5P0lKZ26z7R1LGNH0q6RtIuSW9L+n9Jc4aotz9K2ifpTU0Fa6Sh3pZrahf9TUl7i79VTa+7kr4Gst64XBZIggN0QBKEHUiCsANJEHYgCcIOJEHYgSQIO5DEvwEvYRv57rmVLgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "X = x_test[0].reshape((1,) + x_test[0].shape)\n",
    "plt.imshow(X.reshape(28, 28));"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Counterfactual parameters:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "shape = (1,) + x_train.shape[1:]\n",
    "gamma = 100.\n",
    "theta = 100.\n",
    "c_init = 1.\n",
    "c_steps = 2\n",
    "max_iterations = 1000\n",
    "feature_range = (x_train.min(),x_train.max())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run counterfactual:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time to find prototypes each class: 10.619 sec\n",
      "Explanation took 8.659 sec\n"
     ]
    }
   ],
   "source": [
    "# initialize explainer, fit and generate counterfactual\n",
    "cf = CounterFactualProto(cnn, shape, gamma=gamma, theta=theta,\n",
    "                         ae_model=ae, enc_model=enc, max_iterations=max_iterations,\n",
    "                         feature_range=feature_range, c_init=c_init, c_steps=c_steps)\n",
    "start_time = time()\n",
    "cf.fit(x_train)  # find class prototypes\n",
    "print('Time to find prototypes each class: {:.3f} sec'.format(time() - start_time))\n",
    "start_time = time()\n",
    "explanation = cf.explain(X)\n",
    "print('Explanation took {:.3f} sec'.format(time() - start_time))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Counterfactual prediction: 9\n",
      "Closest prototype class: 9\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAQcElEQVR4nO3da2xWZbrG8eumgBwcOYvFks0xBhRBQ4xEoqIZgkYjY9SMMRNMjJ2EMRkTP2zj/oCfjG73jO5PkzDRDG7dToyjGWPG7eA5g0EphDMqSACLhaoohaBi23t/6NIU7bpXfc/w/H9J03ZdXX0f33ix3r7PWusxdxeAM9+Qeg8AQG1QdiARlB1IBGUHEkHZgUQMreWDmRlv/QNV5u420PayjuxmtszMPjSzPWZ2fzm/C0B1Wanz7GbWJOkjSb+U1C5pg6Tb3X1nsA9HdqDKqnFkv0zSHnff6+4nJf1V0k1l/D4AVVRO2c+X9Em/79uzbacws1YzazOztjIeC0CZqv4GnbuvlrRa4mU8UE/lHNkPSpra7/uWbBuABlRO2TdImm1m081suKRfS3qpMsMCUGklv4x3924zu0fSq5KaJD3p7jsqNjIAFVXy1FtJD8bf7EDVVeWkGgCnD8oOJIKyA4mg7EAiKDuQCMoOJKKm17MD/ZkNOEP0A+58XFkc2YFEUHYgEZQdSARlBxJB2YFEUHYgEUy9DdKQIfn/Lvb29ob7Dh0aP83d3d0ljWkwxowZE+ZHjx4t6/ePHTs2zFtaWnKzAwcOhPt2dXWVNCYMjCM7kAjKDiSCsgOJoOxAIig7kAjKDiSCsgOJ4O6yZ4Bhw4blZiNGjAj3PXnyZJh/++23YR6dfyAVn4OAyuPuskDiKDuQCMoOJIKyA4mg7EAiKDuQCMoOJCKZ69nPOuusMC+ab47msidMmBDuu3z58jB/6qmnwrzolstFc92Ronn0WbNmhfmJEyfC/NNPP/3ZY0J1lFV2M9sn6ZikHknd7r6wEoMCUHmVOLIvcffPK/B7AFQRf7MDiSi37C7pn2a20cxaB/oBM2s1szYzayvzsQCUodyX8Yvd/aCZnStprZl94O7v9P8Bd18tabXEhTBAPZV1ZHf3g9nnTkkvSrqsEoMCUHkll93MRpvZL77/WtJSSdsrNTAAlVXOy/jJkl7M5oCHSvpfd/+/ioyqCormk4tE14UXzaNPmTIlzOfNmxfm69evD/PIBRdcEOZ33HFHmBeN/ZlnngnzL7/8Mje77777wn03b94c5i+//HKYT506NTf75JNPwn3PRCWX3d33SppfwbEAqCKm3oBEUHYgEZQdSARlBxJB2YFEJHOJa3SJqiR99913Yb506dLcbObMmeG++/fvD/Nly5aFedGSz8ePH8/Npk+fHu579tlnh3lTU1OYr1y5Msy3b88/9aLosuKvv/46zBcujC+yLLr8NjUc2YFEUHYgEZQdSARlBxJB2YFEUHYgEZQdSARLNg/SCy+8kJv19PSE+44fPz7MH3300TC/9dZbw3zFihW52fvvvx/u+8UXX4T5tGnTwvzYsWNhft555+Vmhw4dCvd96623wnzu3LlhHs3jP/LII+G+GzduDPNGxpLNQOIoO5AIyg4kgrIDiaDsQCIoO5AIyg4k4oy5nr1oSeZybyV98803l7zvggULwrxoLnv48OFh3tHRkZt1d3eH+7799tthXnTN+YwZM8J83bp1udmll14a7nvDDTeE+UUXXRTmb7zxRm5WtAz2mYgjO5AIyg4kgrIDiaDsQCIoO5AIyg4kgrIDieB69kzR/dGvvfba3CyaSx7M73788cfDfPbs2WH+/PPP52Zbt24N9y26bnvkyJFhXnRf+tGjR+dmRffyL7qOf9GiRWEe3Te+6J7zRfcgiO7VLxWfn1BNJV/PbmZPmlmnmW3vt228ma01s93Z53GVHCyAyhvMy/i/SPrxkiX3S3rd3WdLej37HkADKyy7u78j6ciPNt8kaU329RpJyys8LgAVVuq58ZPd/fsTsg9Jmpz3g2bWKqm1xMcBUCFlXwjj7h698ebuqyWtlhr7DTrgTFfq1NthM2uWpOxzZ+WGBKAaSi37S5K+v3/xCkl/r8xwAFRL4ct4M3tW0tWSJppZu6RVkh6W9JyZ3SVpv6TbqjnIWmhubg7zAwcO5GZF16MXrd9eZNOmTWH+wQcf5GY7d+4M9+3q6grzojXSi+ab58yZk5uNHTs23LfI1KlTw/y1117LzYrm0Y8c+fF70qe/wrK7++05Uf5ZJgAaDqfLAomg7EAiKDuQCMoOJIKyA4k4Y24lXa729vaS973yyivDvOhSzClTpoT5tm3bwjya/po3b164b9ElzhdeeGGYF03NRZfItrbGZ1FfddVVYf7ZZ5+F+dNPP52bFU0Znok4sgOJoOxAIig7kAjKDiSCsgOJoOxAIig7kAjm2Sugra0tzJcsWRLmRZdTRpfXSlJPT09uFl3+KkmjRo0K8xEjRoT5hx9+GOarVq0q+XdPmjQpzB966KEwj8Y2dGj8v349bwVdLRzZgURQdiARlB1IBGUHEkHZgURQdiARlB1IBPPsGbMBV7n9QXTdd7Q0sCRt2bIlzMeNixfB7eyM1+BYv359mEeKlkUeMiQ+Htx9991hHl2z/tVXX4X7FnnuuefCvKOjI8xTw5EdSARlBxJB2YFEUHYgEZQdSARlBxJB2YFEMM+eKZpPjq4ZL1J0//Oi5aIvvvjiMJ8/f35uNnfu3HDfV199Ncy7u7vDvOgcg7179+Zm06dPD/eNllyWpGHDhoU5TlV4ZDezJ82s08y299v2oJkdNLPN2cf11R0mgHIN5mX8XyQtG2D7Y+6+IPv4R2WHBaDSCsvu7u9Iiu+bBKDhlfMG3T1mtjV7mZ97creZtZpZm5nFN2oDUFWllv1PkmZKWiCpQ9If8n7Q3Ve7+0J3X1jiYwGogJLK7u6H3b3H3Xsl/VnSZZUdFoBKK6nsZtZ/ruhXkrbn/SyAxlA4z25mz0q6WtJEM2uXtErS1Wa2QJJL2ifpt1UcY00UzaM3NTXlZkuXLg33HT58eJj39vaGedH673v27MnNiu5Jv27dujAvel5aWlrCfPfu3blZ0bkNa9euDfOi5xWnKiy7u98+wOYnqjAWAFXE6bJAIig7kAjKDiSCsgOJoOxAIrjEdZAuueSS3GzXrl3hvu+++26Yd3V1hXl0G+t6a29vD/Pov63o8toNGzaEeTQdip/iyA4kgrIDiaDsQCIoO5AIyg4kgrIDiaDsQCKYZx+ktrbq3VVr5syZYf7xxx+HeXSJbdG+R48eDfPPP/88zOfMmRPm5557bm522223hfsuWbIkzLmV9M/DkR1IBGUHEkHZgURQdiARlB1IBGUHEkHZgUQwz94AiubCR40aVXJe9LtvvPHGMH/llVfCfNasWWH+xBP5NyKeOHFiuO9HH30U5iNHjgxznIojO5AIyg4kgrIDiaDsQCIoO5AIyg4kgrIDibBa3pPczBr3BugNrGhp4hkzZuRm0f3uJenOO+8M8y1btoT5/Pnzw3zlypW5WdE5ACiNu9tA2wuP7GY21czeNLOdZrbDzH6fbR9vZmvNbHf2eVylBw2gcgbzMr5b0n3uPlfS5ZJ+Z2ZzJd0v6XV3ny3p9ex7AA2qsOzu3uHum7Kvj0naJel8STdJWpP92BpJy6s1SADl+1nnxpvZNEmXSHpP0mR378iiQ5Im5+zTKqm19CECqIRBvxtvZmdL+puke939lNX6vO9dvgHffHP31e6+0N0XljVSAGUZVNnNbJj6iv6Mu7+QbT5sZs1Z3iypszpDBFAJhS/jzcwkPSFpl7v/sV/0kqQVkh7OPv+9KiOEWlpawvzgwYO52f79+8N9x44dG+a33HJLmL/55pthzvRa4xjM3+xXSPqNpG1mtjnb9oD6Sv6cmd0lab+k+CbgAOqqsOzu/i9JA07SS7q2ssMBUC2cLgskgrIDiaDsQCIoO5AIyg4kgltJnwb27t0b5pdffnludsUVV4T7Llq0KMzb29vDfM+ePWE+efKAZ1FLkg4fPhzui8riyA4kgrIDiaDsQCIoO5AIyg4kgrIDiaDsQCKYZ6+BIUPif1N7e3vL+v3R7ZzvvffecN+iefRVq1aF+ZgxY8KcufTGwZEdSARlBxJB2YFEUHYgEZQdSARlBxJB2YFEMM9eAeecc06YF82jNzU1hfmxY8fCfPHixbnZxo0bw3137NgR5keOHAnznp6eMJ82bVputm/fvnBfVBZHdiARlB1IBGUHEkHZgURQdiARlB1IBGUHEjGY9dmnSnpK0mRJLmm1u/+3mT0o6W5Jn2U/+oC7/6NaA21kXV1dYT5jxoww7+zsDPNrrrkmzK+77rrcbMKECeG+0druknT8+PEwP3HiRJh3dHSEOWpnMCfVdEu6z903mdkvJG00s7VZ9pi7/1f1hgegUgazPnuHpI7s62NmtkvS+dUeGIDK+ll/s5vZNEmXSHov23SPmW01syfNbFzOPq1m1mZmbWWNFEBZBl12Mztb0t8k3evuXZL+JGmmpAXqO/L/YaD93H21uy9094UVGC+AEg2q7GY2TH1Ff8bdX5Akdz/s7j3u3ivpz5Iuq94wAZSrsOxmZpKekLTL3f/Yb3tzvx/7laTtlR8egEoZzLvxV0j6jaRtZrY52/aApNvNbIH6puP2SfptVUZ4Bii6XfPJkyfD/Jtvvgnz9957Lzdrbm7OzSTpscceC/OiJZlx+hjMu/H/kmQDREnOqQOnK86gAxJB2YFEUHYgEZQdSARlBxJB2YFEmLvX7sHMavdgCZk0aVJuVnR5bTRHj9OTuw80Vc6RHUgFZQcSQdmBRFB2IBGUHUgEZQcSQdmBRNR6nv0zSfv7bZoo6fOaDeDnadSxNeq4JMZWqkqO7d/cfcATL2pa9p88uFlbo96brlHH1qjjkhhbqWo1Nl7GA4mg7EAi6l321XV+/Eijjq1RxyUxtlLVZGx1/ZsdQO3U+8gOoEYoO5CIupTdzJaZ2YdmtsfM7q/HGPKY2T4z22Zmm+u9Pl22hl6nmW3vt228ma01s93Z5wHX2KvT2B40s4PZc7fZzK6v09immtmbZrbTzHaY2e+z7XV97oJx1eR5q/nf7GbWJOkjSb+U1C5pg6Tb3X1nTQeSw8z2SVro7nU/AcPMrpR0XNJT7n5Rtu0/JR1x94ezfyjHufu/N8jYHpR0vN7LeGerFTX3X2Zc0nJJd6qOz10wrttUg+etHkf2yyTtcfe97n5S0l8l3VSHcTQ8d39H0pEfbb5J0prs6zXq+5+l5nLG1hDcvcPdN2VfH5P0/TLjdX3ugnHVRD3Kfr6kT/p9367GWu/dJf3TzDaaWWu9BzOAye7ekX19SNLkeg5mAIXLeNfSj5YZb5jnrpTlz8vFG3Q/tdjdL5V0naTfZS9XG5L3/Q3WSHOng1rGu1YGWGb8B/V87kpd/rxc9Sj7QUlT+33fkm1rCO5+MPvcKelFNd5S1Ie/X0E3+9xZ5/H8oJGW8R5omXE1wHNXz+XP61H2DZJmm9l0Mxsu6deSXqrDOH7CzEZnb5zIzEZLWqrGW4r6JUkrsq9XSPp7HcdyikZZxjtvmXHV+bmr+/Ln7l7zD0nXq+8d+Y8l/Uc9xpAzrhmStmQfO+o9NknPqu9l3Xfqe2/jLkkTJL0uabek1ySNb6Cx/Y+kbZK2qq9YzXUa22L1vUTfKmlz9nF9vZ+7YFw1ed44XRZIBG/QAYmg7EAiKDuQCMoOJIKyA4mg7EAiKDuQiP8HTcQsYe7LXasAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "print('Counterfactual prediction: {}'.format(explanation.cf['class']))\n",
    "print('Closest prototype class: {}'.format(explanation.id_proto))\n",
    "plt.imshow(explanation.cf['X'].reshape(28, 28));"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The counterfactual starting from a 7 moves towards its closest prototype class: a 9. The evolution of the counterfactual during the first iteration can be seen below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "iteration c 0\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA64AAACxCAYAAADEbuChAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3daXhV1dnG8XWAQIAQJmMEEoGAohCZpVBUwFpnVFQGJ1S0WG21WG0rtGprW9DaKq30ckYFnGsrCLUgAraooIRBAQEhDGFMAiQBQiDAeb+874e36368zpYMi+T/+3hfz7XPRtYeVk+5TywejzsAAAAAAEJVp7pPAAAAAACAb8LGFQAAAAAQNDauAAAAAICgsXEFAAAAAASNjSsAAAAAIGhsXAEAAAAAQasXZTgWi/HbOVUkOTlZ5mVlZVV8JuGIx+Ox6vps1n7lqFu3rpcdPXpUziYlJcm8vLy8Qs8pRKx91GKF8Xg8rbo+nPWP6sS9H7WVtfYjbVxRdbKysmS+evXqKj4ToPKkpKR4WXFxsZw95ZRTZJ6Xl1eh5wQgKJur+wQAAGHg/yoMAAAAAAgaG1cAAAAAQNDYuAIAAAAAghaLxxP/t9f8Q+3ERC1WGjRokJdt2LBBzsZi+t/pZ2ZmynzhwoUyPxFRUlA7UEzmY+2jFsuJx+O9q+vDWf+oTtz7UVtZa59vXAEAAAAAQWPjCgAAAAAIGhtXAAAAAEDQ2LgCAAAAAIJWr7pPoCaKWiIzZMgQL7vnnnvk7He/+12Z16QSJtQ8WVlZMs/NzfWyfv36ydn58+cf93k0bNhQ5gcPHjzuYyN82dnZMl+5cmUVnwkAAN/MKmSNUqxb0/CNKwAAAAAgaGxcAQAAAABBY+MKAAAAAAgaG1cAAAAAQNDYuAIAAAAAgkarcCVITU2V+dChQ2U+bty4hI/9ySeffKtzOh7qz1NSUlLl54Hq0bhxY5kfOHAg4WOo9mDnnOvbt6+Xbd++PeHjRpWSkiLz9PR0mW/atKnSzgVVr1WrVjJPTk6W+ZIlS2TepEkTL7Pu+/n5+TIvLy+XeRT16ulH+JEjR4772AAQgjp19Hdsx44dk7m6L1bUPbFp06ZeVlxcHOkYzZo187KMjAw5u2XLFpnX5ndwvnEFAAAAAASNjSsAAAAAIGhsXAEAAAAAQWPjCgAAAAAIGhtXAAAAAEDQYvF4PPHhWCzx4cANGjRI5vPnz5f52LFjZT5hwoSEP9NqNN2/f3/Cxzj11FNlfujQIZnv2rUr4WNb0tLSvKygoOC4jxtVPB6PVfmH/q/Q137U1r2KOEZmZmbCsy1btpR5Xl6el/Xp00fOzp49W+a1AWu/Ypxxxhkyf+aZZ2Q+cOBAmffs2dPLdu7cKWcrsyW7IrRr107mATVq58Tj8d7V9eGhr3/VUOqcc0VFRVV6Hh06dJD5hg0bqvQ8LFZzuPXuFOXduDJx768eSUlJMlfr6PDhw3LWWlvqXSvKu1ptYa19vnEFAAAAAASNjSsAAAAAIGhsXAEAAAAAQWPjCgAAAAAIGhtXAAAAAEDQ6lX3CXwbsVi0kjXVDrdy5Uo5m52dLfP09HSZX3fddV72+uuvy9ko7cGW008/XeZz58497mNbrMY0VL3U1FSZW43AUZolo7baqUbgTp06yVmrobRu3bpeNmfOnEjnYcnKyvKy3NzcCjk2wjdkyBAvs+7jW7ZsiXTss88+28usRvooTj75ZJnn5+cf97Etu3fvrrRjo/JZje1V3Socent2vXr6dbesrKyKzwTOOdegQQOZq/dNq+HXWvtXXXWVl02ZMkXOWvsJ651KsdqDO3bsKPPS0lIvC/36CQnfuAIAAAAAgsbGFQAAAAAQNDauAAAAAICgsXEFAAAAAAQtpoqLzOFYLPHhCmL9g3qld+/eMl+0aFHCx4hajqE+85JLLpGzH3zwgcyjnF91yMjI8LJt27bJ2SjrKap4PB6tlasCVcfaV6JcD87pcoA1a9ZEOsZDDz0k8w8//NDLPv74Yznbpk0bmaelpXmZVSpiFTwNGzZM5lHKmVatWhUpr2qsfdugQYNkroq/rBK7Zs2aybxFixYyv/DCC71swYIFcta63vr06eNlt956q5y1StkKCgpkPmbMGC/r37+/nG3cuLHMlyxZ4mV79uyRs5UsJx6P64d7FQh9/Tdq1Ejm6l3GKlpMTk6WeShldl27dpV5cXGxzDdv3lyZp1OluPfb978bbrhB5q1bt/ayWbNmydmo79+qgPL8889P+Dycc+7VV1/1MmvN3nfffTJfvny5zGfOnOllmZmZclYVbIbEWvt84woAAAAACBobVwAAAABA0Ni4AgAAAACCxsYVAAAAABA0Nq4AAAAAgKBFqyitBkeOHPEy1erlnHOjR4+WuWoNsxq8+vXrJ/Mnn3xS5g0bNvSyO+64Q84OHjxY5meffbaXtWrVSs7u3LlT5nXq6P8N4tixYzKPYuvWrV4Wi+miOyuvzLbh2uTcc8+VudVc+otf/MLLrObSyy67TOZz5syR+c9//vOEPs85e+2/+OKLXjZ9+nQ5m56eLvOxY8fKXLWivvXWW3LWun4QjmuvvVbm11xzjczVuu3bt6+cVc8Z55xr0qSJzNU90WqeV83ezummzJYtW8rZQ4cOybxnz54yf+yxx7xMNVk659ynn36a8PlZrFbasrKyhI+Bb2Y1X1utwt27d/cyq0Haeh+y3jfKy8u9TL3HOOdcaWmpzFVD/qWXXipn9+3bJ3OrLXX8+PFeZjViR3lHslr9rfsHbElJSTJXa0u1uDvnXIcOHWSu1vPFF18sZ62/0/3798u8ffv2XpaSkiJnVbO9c87dddddXrZy5Uo5azWBHzx4UObql06sa/BExdsaAAAAACBobFwBAAAAAEFj4woAAAAACBobVwAAAABA0Ni4AgAAAACCFnyrsFJUVCTzt99+O+FjvPTSSzJv166dzEeNGiXzX/3qV172u9/9Ts5+9tlnMp8xY4aXWW1+lueee07mnTt39rLJkyfL2RdeeCHhz7Pa3NavX5/wMWqbimhcPvnkkyMdQ63F5s2by9khQ4bI3Gq1s+ajUE3Bv/zlL+XsBRdcIPMxY8bIfMKECV5mNdPOmzfPOkUEwmruXLt2rcxXrVrlZVar6s033yxzq8lVNdUfPXpUzlq5OsaVV14pZ/v06SNzq5lTtVlazZxWG3KUZ1D9+vVlTqtwxbHee6xc/eLB0KFD5azVzvuf//xH5tZ6jCItLc3LCgoKjvu4zul3qp/+9KdydtmyZTJXTd7W/aO4uDjC2cE53R5suf7662Vu3Vt79OjhZY8//rictX5lwXomqPf43bt3y1lrP6Fasi+//HI5a92HrWeTeue33uFU+7xzzuXk5Mg8FHzjCgAAAAAIGhtXAAAAAEDQ2LgCAAAAAILGxhUAAAAAELQTspxp165dMt+wYYPML774Yi+z/jH9+++/L/PU1FSZ//nPf/ayOXPmyFmrQEedt/WPva0CJfWP0Z3ThSZWyY1VWKWKGyhhis76+1dFKs7pIqbCwkI5u3DhQpmrgo7bb79dzu7YsUPmGRkZMn/wwQe9bPjw4XK2devWMt+6dauXTZo0Sc7+85//lPl7770nc1XccNNNN8nZP/zhDzJXkpOTZU4RTcVp0KCBl6lCC+ecmzlzpsxPOukkLzt27Jicta7BOnX0/7arilvGjRsnZ2+44QaZ33bbbQmfh3V/v+SSS2T+4YcfeplVTnL//fdHypWSkpKEZ/Ht9O7dW+ZW2Z56Vnz00UdytmfPnjJX707O6WIYq1jJuobUeefm5srZTZs2yfzTTz+VuXpmWaWUI0eOlLkqvbIK4ihnsql7uXP6Hmq5+uqrI31m9+7dvcwqSrKK5az3IbUGrOvKKkXKysryso8//ljOWtemVeaUnZ3tZVb5pFUYGjq+cQUAAAAABI2NKwAAAAAgaGxcAQAAAABBY+MKAAAAAAgaG1cAAAAAQNBiVtOpHI7FEh+uRFYrrtWQ1alTJy+zmrp+8pOfyHzq1KkyV63COTk5cvb000+XuWroffbZZ+Xs8uXLZW61Dd9zzz1e1qZNGzk7atQomavzXrdunZytTPF4vNoq0Kpj7au/p23bth33cV9++WWZ33zzzTKfO3euzNesWeNlVuvxq6++KnPVWvnVV1/J2XfffVfmKSkpMle2bNki86efflrmjz76aMLHttozrSbbKGrb2leaNm0qc6vRU7WWfuc735Gzv/3tb2W+aNEimau2bqtp3brfqjb5ESNGyFmrsXrYsGEyHzRokJd169ZNzubl5clcNbwuW7ZMzubn58u8guTE43FdqVsFqmP9JyUleVm/fv3krGoodc65jRs3eplat9/Ean7t3Lmzl3Xo0EHOpqWlyVydy/Tp0+XsmWeeKXNrParm19deey3Ssffu3etlVgOtaiB2zrny8nKZR1Hb7v2qWf173/uenLWaeNUxJk6cKGdPO+00mf/tb3+T+RdffOFl1ju/db21b9/eyxo3bixn1b3AOeeGDh0qc3WfKC0tlbNWU3mLFi28bP/+/XLWak6uCNba5xtXAAAAAEDQ2LgCAAAAAILGxhUAAAAAEDQ2rgAAAACAoLFxBQAAAAAErV51n8D/sdq3VOvxl19+KWd79OiRcP7KK6/I2eeff17mkydPlvn8+fO9zGr5W7JkicxV2+Pq1avl7DnnnCNz1fDqnHPjxo3zMtW45pxuEnNONwhbs3v27JE5oovSIHzRRRfJfMiQIV5mtQefddZZMm/ZsqXMFy9e7GVW++ntt98u80mTJnlZcnKynB09erTMrbZI1QppNY9bzZeNGjXyMquhryLag2Gz2oMtqgXRes5YrcKXXXaZzB988EEv69u3r5y17pUNGjTwss8//1zO3n333TK31m1GRoaX9e/fX85azdnqOuzTp4+cnTlzpszx7ag22lWrVslZ671n4cKFXmbdy61W6F69esn8r3/9q5dZrdXt2rWTuWoQVm3Yzjk3ePBgmVu/sqAaYa1Zqyk46v0GFaNVq1ZeZv0agLW2rIZrZenSpTK33qnVu3lJSYmcPXjwoMzVs8lqt27WrJnMLZmZmV5m/TLEifoezzeuAAAAAICgsXEFAAAAAASNjSsAAAAAIGhsXAEAAAAAQWPjCgAAAAAIWjCtwlb7lnLllVfK/M0335T5yJEjvcxqjJs6darMZ82aleDZOdezZ0+Zd+3aVeYdO3b0siNHjshZqy3N+u+3detWmStRmsRCbx2rbaz1MnHiRC+zmn+//vprmVvrOScnJ8Gzc27RokUyVw3X/fr1k7OdOnVK+POcc+6tt97yMuvPYjUZWw3CSvPmzWW+b98+mVt/Z6gYder4/7vsgAED5OwLL7wg83nz5sm8Xj3/0VlQUCBnreuqTZs2XrZ79245a7WtjhgxQuaqgdt6Plqtqrt27fKy3NxcOYvKp5qGnbPbzM877zwvsxpKrcbiKI2mK1askLlqiXXOuVdffdXLnnzySTmrfl3COeeGDRsmc/V+MnbsWDlrNdaiekR5Z1Vr3Dn9DtG6dWs5a/1KiWr+dU7/+oK1Prt06SJz9b5uNd5bv6ZgPcvUc2jatGly1vozho5vXAEAAAAAQWPjCgAAAAAIGhtXAAAAAEDQ2LgCAAAAAIIWTDmTRRUXWSUTzz33nMxnzJjhZcuXL5ezVm4ZOHCgl2VkZMjZOXPmyLxXr15e1qBBAzk7ffp0mVv/sPsHP/iBl2VnZ8vZ9evXy9wq80HFsMpRUlJSvMwqxfrwww9lfuedd3rZpEmT5Oy9994rc6tcY8qUKV7WrVs3OTt8+HCZP/TQQwkf48Ybb5S5RRXoWNe3KnJyzrm2bdt62ebNm+Xs3r17I5wdKopVIHPo0CEvmzx5spxduXKlzHfs2CHztLQ0L7NKRXr06CFzdX1bJXtWycf1118vc7UWrXuHVUBllYop1j3s8OHDCR8D36ykpETmixcvlrl6bltrwCqns54rilUgFovFZH7gwAEv+/zzz+XsI488InPr/NR1tH37djlbEe831vtXlNJRRLdkyRKZq7VorX2rnOvo0aMyX7NmjZc1atRIzlrX1dq1a73s4YcfjnQM9Qxyzrnx48cn9HnO6Xck58K/b/ONKwAAAAAgaGxcAQAAAABBY+MKAAAAAAgaG1cAAAAAQNDYuAIAAAAAghZ8q3BBQYGXqTZG5+y2z5deesnL0tPT5ew777wjc6sx8tJLL/Wyrl27ytmJEyfK/Mwzz/Sya665Rs6uW7dO5i1btpS5apAtLy+Xs6pB1Tm7kQwVw2pws1rwlGuvvVbmTz/9tJc1b95czvbu3VvmVjPikCFDvGzhwoVyduzYsTJfunSplxUXF8tZ65qNx+MyV63H//73v+UswteuXTuZt27dWuYbN270Mqv5uUmTJjLfvXt3wrn1XDrllFNkrtbnE088IWfr1q0rc3X9OOfc22+/7WXqOeicbneNKvQWyprss88+k7l6nlvtt1ZjsUVdL6mpqXK2c+fOMu/fv7+XTZs2Tc5effXVMj/rrLNkXqeO/53M4MGD5ay6TzgX7brglxcqjmqhtp7xpaWlMl+xYoWXWe89+fn5Ml+0aJF1ip6hQ4fKXK1D5/SvfYwePVrOFhUVJXwezulfSLDa8U9UfOMKAAAAAAgaG1cAAAAAQNDYuAIAAAAAgsbGFQAAAAAQNDauAAAAAICgxay2LjkciyU+XEEyMzO9LC8vL9Ix1J9RNZc559wXX3wh8wEDBshctVSqxjDn7LZh1chotdQNHz5c5lYznmo9Vi3Gztn/Xa0mzqoWj8f1X1oVqI61r1gtdT/72c9k/thjj3nZ9OnT5eyoUaNkfsMNN8j8sssu87KLLrpIzg4bNkzm2dnZXtatWzc5e/LJJ8t8//79Mldr32rUtqj/3seOHYt0jIrA2rfbeZOTk2U+YsQIL0tLS5OzEyZMkHlhYaHM+/bt62UzZsyQs6+88orM582b52VWe+ptt90mc9Ua7pxzY8aM8bKoaz8gOfF4XFeeV4FQ1r/V2ms1AqtWYash3nofshqB1TVnNUtb7xWqibVRo0YJzzrn3KxZs2Sufh1i2bJlctb6b1Id93mltt37VYP60aNHIx3jySef9DKrfX79+vUyj3JNzJ49W84eOXJE5s2aNfOyH/7wh3K2ffv2Mp87d67Mb7/9di/bsmWLnA2dtfb5xhUAAAAAEDQ2rgAAAACAoLFxBQAAAAAEjY0rAAAAACBo9ar7BP5P/fr1ZV5cXHzcx+7SpYuXnXfeeXJW/cNm55wbPHiwzFeuXOll6enpcnbBggUyf/75573MKluyXHDBBTJXhRxNmjSRs1YeSjkT7MIIVcJk+f3vfy9zq0DJWs+PPvpowp/ZsmVLmatyDauEySpXGD9+vMxVOZNVTGUJpaADzu3cuTPSfFFRkZdZJR8nnXSSzDt06CBzVZhhHcO6N+/Zs8fLVqxYIWf/+Mc/yvzdd9+V+QlcxASDVcJ0xhlnyHzNmjVe1r9/fzlrlepZpX+TJ0/2sq+//lrOWvnnn3/uZRdeeKGctQr4rOtl4cKFMj9eSUlJMud6qzjqHq0Km5yz14vaT1jPcmsvYJU2qfv2xx9/LGet501GRoaXWdeJVcj5wQcfyNzaS9UkfOMKAAAAAAgaG1cAAAAAQNDYuAIAAAAAgsbGFQAAAAAQNDauAAAAAICgBdMqfPjw4Uh5FGVlZV5mtdE1aNBA5lajn2oksxrGLPF43Mvmzp0rZ0eNGiXzc845J+HPs86vadOmCR8DlS8Wi3mZWivO2a29u3fv9rK8vDw5a/39W/n8+fNlrqgGSef0eh49erScfe+992R+1113ydxqxETt8Mwzz3jZLbfcImetxsmsrCyZq+vKaqe0nikDBgzwMut5l52dLfO///3vMldN4A0bNpSzmzZtkjlODKo92GK1886ePVvm6lcTnHMuPz/fy+6++245O3HiRJmra9Fqcj311FNlbv3Z1fWyatUqOWtdn+q9kfbg6tGjRw+Zf/XVVzL/5JNPvMx6h7feqSrT1q1bvcw6P+vatN6prAbmmoRvXAEAAAAAQWPjCgAAAAAIGhtXAAAAAEDQ2LgCAAAAAILGxhUAAAAAELRYlEatWCxWafVbSUlJMq/qFre2bdvKfPPmzcd97AceeEDm7dq187JDhw7JWdWW5pxzb775psxVu57VFBi6eDzu1+xWkcpc+xXBagw9ePDgcR+7b9++Mt+7d6+XDR8+XM6uXr1a5q+//rqX1auny8779esnc9Vw6Zxzubm5Mj8RsfZtjRo1knlpaamX1a9fX85audXCmpaW5mXTp0+Xs0899ZTMp0yZ4mVvvPGGnJ0zZ47MCwsLZf7+++97WbNmzeRsUVGRzAOSE4/He1fXh4ey/lNSUmRurVGlY8eOkY594YUXyrxTp05eZr0jTZs2TebLli3zMqupfunSpTJ/5JFHZK6uZ6uxu0WLFjLfs2ePzKsa9/6K0aFDB5lv2LBB5tbaV/PFxcVy1ro/q3eqRx99VM4OHDhQ5oMGDZJ5QUGBl9W0d36+cQUAAAAABI2NKwAAAAAgaGxcAQAAAABBY+MKAAAAAAgaG1cAAAAAQNB0fWc1qMz24KZNm3qZ1QJWEe3BUS1fvtzLrKbAd955J9Kxv/rqq291TjixVER7sGXRokUyv+mmm7zstNNOk7P9+/eX+bp167xswYIFctZqTs7Ly5O5aptVTbM4sVnPDtXWbtm3b5/Mrfuwam78zW9+I2fvv/9+mdetW9fLjh07JmenTp0q88svv1zmjRs39jKrrRsnhijtwRarPdhaG9b7kLrmnnjiCTn70UcfyVy9y1jvK48//rjMLVaDsBJKezAql9UebLXSW7k6zuDBg+Wsand3Trd7v/jii3L2pJNOkrl6d3LOfk+qSfjGFQAAAAAQNDauAAAAAICgsXEFAAAAAASNjSsAAAAAIGg1qq0hOTlZ5lYRUxRdunSRebNmzbwsOztbzo4fP17ms2bN8rIBAwbI2WnTpsm8fv36Mo9SUoCap1evXl526NAhObty5cpIx37ttde8bMeOHXJ2+PDhMlfr80c/+pGcta4rq5ynMgvfEA7r73nTpk1eZhVXpKamRvpMdb+1CpSSkpJkroo7Vq9eHek8Fi5cKHNV1nbgwIFIx0bNY5WQNW/eXOZvvvmmzFu1auVlqhDMOef27t0r83PPPdfLdu7cKWejyszM9DKrxA+125EjR2S+Zs0amV933XVedsstt8hZtcadc65bt25edtddd8nZwsJCmddmfOMKAAAAAAgaG1cAAAAAQNDYuAIAAAAAgsbGFQAAAAAQNDauAAAAAICg1ahW4aZNm8q8rKws4WPceeedMt+6davM165d62U33nijnP3yyy9l/txzz3nZ4MGDrVOUrPZg1X5J0/CJQTU3Wq29loKCAi8766yz5GzUVuERI0Z4mWpy/SZqLf7pT3+Ss/fdd1+kYwP/zWrULi0tjXQc1Vq6YcMGOfuvf/1L5kuXLvWy9957L9J5FBUVyTwtLc3L1L0AtYu1Ri3du3eX+fLly73sgw8+kLO7du2S+bJly7xs3LhxEc7Oplq1ASUjI0Pm27Ztk/nmzZu9TP26iHPOXXvttTKfP3++l0W9NmszvnEFAAAAAASNjSsAAAAAIGhsXAEAAAAAQWPjCgAAAAAIGhtXAAAAAEDQalSrsNXsZbXaKVdffbXMV6xYIXPVArlkyRI5+8QTT8j8mWee8bLy8nI5azVUtm7dWubbt2+XOcIXtUFY2bJli5eNGjVKzs6aNUvm1rq99957vcxat0lJSTJPTU31sgYNGsjZK664QuYzZsyQOfDfjh07JvOGDRvK/C9/+YvMX3/9dS/r06ePnJ00aZLMZ86c6WXdunWTs9bzxzrvtm3bepnVKqya552jfb4msu7D1n3b+qUGZciQITJ/4403ZJ6SkuJl6nngnHMlJSUyt54VhYWFMgf+W25ursz79u0r8/79+3tZv3795Kz1ayTr16/3svT0dDkbZf9SW/CNKwAAAAAgaGxcAQAAAABBY+MKAAAAAAgaG1cAAAAAQNBqVDnT2rVrE5696aabZG6VY3z/+9+XeTwe97J77rlHzqqiHOecKyoq8rIFCxbIWauciRKmmqdu3bpedvTo0UjH6Nmzp5dNmDAh0jEOHjwo88zMTC+z1vjZZ58t83379nlZWVmZnP3iiy+sU0QtFrVwRrnttttkbj0nbr31Vi9T12tUVglTixYtZN6xY0eZ5+fne1mTJk3krLoGceKLxWJeFuWacM65008/XeYZGRledscdd8jZBx54QOajR4/2MquEqVOnTjKP8s6H2qNOHf87OauYz2LtBcaMGeNlVgnTww8/LHNVekYJU+L4xhUAAAAAEDQ2rgAAAACAoLFxBQAAAAAEjY0rAAAAACBobFwBAAAAAEGrUa3CjRs3lnmXLl28bNmyZXK2TZs2kT5z9uzZXvbUU0/J2YEDB8pcNaA9/vjjctb6MzZs2FDmhYWFMkf4ojQI16unL+VevXp52Y9//ONIn2ddE9ZnKsOHD5d5cnKyl1mtwhs3bkz485zT53fkyJFIx0D4ojSlXnTRRTJPS0uT+dtvvy1zdU2otknnnPvHP/4h808++cTLOnfuLGdXr14tc+uaKCgokDlqD/WLB1EtXrxY5qrhferUqXK2d+/eMl+yZImXpaeny9lt27bJ3HofOnDggMxRs6SmpspcNQhbre9Wq/o555wj85ycHC9btWqVnN2zZ4/M1btWu3bt5OymTZtkXpvxjSsAAAAAIGhsXAEAAAAAQWPjCgAAAAAIGhtXAAAAAEDQ2LgCAAAAAIJWo1qFe/ToIfNf//rXXnbFFVfI2SgNlc459+yzz3qZ1Ua2ZcsWmd97771eVr9+fTlrteXRolc7tGrVSua7du2S+fPPP+9lL7zwgpx94IEHZN6hQweZq0bgl19+Wc5azaoZGRleVlGNqDQI1yyxWEzmVqN6aWmplzVp0kTOWmvOanq8/vrrvUytZeecO3z4sMwVqz24a9euMl+7dm3CxwYyMzNlnpeXJ/OsrJntUAsAAAKKSURBVCyZq1ZUq4FYtQdbrOcYoJSUlMhcrdv8/Hw5e/7558v8kksukXnLli29zGq93r9/v8zVs2nHjh1yFj6+cQUAAAAABI2NKwAAAAAgaGxcAQAAAABBY+MKAAAAAAgaG1cAAAAAQNBi8Xg88eFYLPHhapCUlCTz9u3be9m6desq5DN79uzpZffff7+cHTlypMxpP01MPB7XtaJVIJS136JFC5nXq6cLwlWTntUSPGzYMJkvWrRI5itWrPCyPXv2yNk2bdrIXLXrWY3atblxkrVfubp37y5z67pScnNzZW61Ie/evTvhY9etW1fmqt21BsqJx+O9q+vDa8P6t0yZMkXm27dv9zLVYO+ccxs2bKjQc6ptuPdHp94hrHZ361dAxo4dK3P1yw7Wu9P69eutU0QCrLXPN64AAAAAgKCxcQUAAAAABI2NKwAAAAAgaGxcAQAAAABBq1HlTBZVbFFRpRZXXXWVl+Xk5MjZvLy8CvnM2oqSAucaNmwo89TUVJmrQqO2bdvK2f3798u8pKRE5uXl5V5mFaSpWSSOtR++jh07ynzjxo0yryXFShWBcqZvQZWCRXnfc8659PR0mavnSnJyspwtKyuL9Jn4/7j3V4+0tDSZZ2VlednixYsr+3RqJcqZAAAAAAAnJDauAAAAAICgsXEFAAAAAASNjSsAAAAAIGhsXAEAAAAAQYvaKlzgnNtceacDmNrG43Fd81YFWPuoRqx91Gasf9RWrH3UVubaj7RxBQAAAACgqvF/FQYAAAAABI2NKwAAAAAgaGxcAQAAAABBY+MKAAAAAAgaG1cAAAAAQNDYuAIAAAAAgsbGFQAAAAAQNDauAAAAAICgsXEFAAAAAATtfwC0oxIQ3rNmhwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1440x288 with 5 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "iter_cf = 0\n",
    "print('iteration c {}'.format(iter_cf))\n",
    "n = len(explanation['all'][iter_cf])\n",
    "plt.figure(figsize=(20, 4))\n",
    "for i in range(n):\n",
    "    ax = plt.subplot(1, n+1, i+1)\n",
    "    plt.imshow(explanation['all'][iter_cf][i].reshape(28, 28))\n",
    "    ax.get_xaxis().set_visible(False)\n",
    "    ax.get_yaxis().set_visible(False)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Typically, the first few iterations already steer the 7 towards a 9, while the later iterations make the counterfactual more sparse."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prototypes defined by the $k$ nearest encoded instances"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the above example, the class prototypes are defined by the average encoding of all instances belonging to the specific class. Instead, we can also select only the $k$ nearest encoded instances of a class to the encoded instance to be explained and use the average over those $k$ encodings as the prototype."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "# initialize explainer, fit and generate counterfactuals\n",
    "cf = CounterFactualProto(cnn, shape, gamma=gamma, theta=theta,\n",
    "                         ae_model=ae, enc_model=enc, max_iterations=max_iterations,\n",
    "                         feature_range=feature_range, c_init=c_init, c_steps=c_steps)\n",
    "cf.fit(x_train)\n",
    "explanation_k1 = cf.explain(X, k=1, k_type='mean')\n",
    "explanation_k20 = cf.explain(X, k=20, k_type='mean')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Results for $k$ equals 1:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Counterfactual prediction: 9\n",
      "Closest prototype class: 9\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAOpElEQVR4nO3df4hV95nH8c+jUcFfQWPWDGqsiv5hlkSXiWyoWQymxY0E03+kExAXwk7/aEILRTZkSRoCCSFsazYEipYktaGbRmjz4w+xdaVEzA/JKBo1of4oI3UYHRMDMzVRoz77xxy7E3PP94z317kzz/sFw733PPfc8/TWT86993vO+Zq7C8DoN6bsBgA0B2EHgiDsQBCEHQiCsANB3NDMjZkZP/0DDebuVml5TXt2M1tlZn82s2Nm9mgtrwWgsazacXYzGyvpiKTvSDop6UNJHe7+cWId9uxAgzViz75M0jF3/4u7X5T0W0lrang9AA1US9hnSfrrkMcns2VfY2adZtZlZl01bAtAjRr+A527b5a0WeJjPFCmWvbsPZLmDHk8O1sGoAXVEvYPJS00s3lmNl7S9yW9XZ+2ANRb1R/j3f2SmT0s6Q+Sxkp62d0P160zAHVV9dBbVRvjOzvQcA05qAbAyEHYgSAIOxAEYQeCIOxAEIQdCIKwA0EQdiAIwg4EQdiBIAg7EARhB4Ig7EAQhB0IgrADQRB2IAjCDgRB2IEgCDsQBGEHgiDsQBBNnbIZrWfq1KnJen9/f5M6QaOxZweCIOxAEIQdCIKwA0EQdiAIwg4EQdiBIBhnD+7ChQtlt4AmqSnsZtYtaUDSZUmX3L29Hk0BqL967NnvcfdP6/A6ABqI7+xAELWG3SX90cz2mllnpSeYWaeZdZlZV43bAlADc/fqVzab5e49ZvYPknZIesTddyWeX/3G0BATJkxI1vkBb+Rxd6u0vKY9u7v3ZLd9kt6QtKyW1wPQOFWH3cwmmdmUq/clfVfSoXo1BqC+avk1fqakN8zs6uv8j7tvr0tXo8z48eOT9YsXLzapk28q82P6mDHpfc2VK1ea1EkMNX1nv+6NBf3O3sphLxNhb4yGfGcHMHIQdiAIwg4EQdiBIAg7EASnuA7TrbfemltbvHhxct2Ojo5k/dNP0+cRPf/888l6W1tbbm3+/PnJdYuOoHvnnXeS9e7u7mQ9ZeLEicn6F198kazza/31Yc8OBEHYgSAIOxAEYQeCIOxAEIQdCIKwA0Fw1tsw3XHHHbm1F198Mbnu7Nmzk/W+vr5kveisudRY+uHDh5PrFikaCy/q7auvvsqtnTt3rqqerhoYGEjWU8cvbNy4MbluV9fIvYoaZ70BwRF2IAjCDgRB2IEgCDsQBGEHgiDsQBCcz54ZN25csn777bfn1iZNmpRcd9u2bcn6sWPHkvXUufSStGfPntza3Llzk+tOnz49WU/9767VpUuXkvUzZ84k6zNmzEjWU/+f9vb2JtcdyePsedizA0EQdiAIwg4EQdiBIAg7EARhB4Ig7EAQYcbZi8bRi8Zs29vbq972qVOnkvWic6uLTJs2Lbf25ZdfJtctOh/9zjvvrKqnqy5fvpxbS53rLkmrV69O1js7O5P11DEEx48fT647GhXu2c3sZTPrM7NDQ5ZNN7MdZnY0u83/1wagJQznY/yvJK26Ztmjkna6+0JJO7PHAFpYYdjdfZeks9csXiNpS3Z/i6QH6twXgDqr9jv7THe/enDxKUkz855oZp2S0l+uADRczT/QubunLiTp7pslbZZG9gUngZGu2qG302bWJknZbfryqABKV23Y35a0Pru/XtJb9WkHQKMUfow3s9ckrZA0w8xOSvqppGclbTWzhySdkLS2kU3WQ9H18YvmSO/v78+tLV26NLnuK6+8kqzX6vPPP6963fPnzyfrO3furPq1JemGG/L/iRWdz37XXXcl66njCyTp4MGDubXXX389ue5oVBh2d+/IKa2scy8AGojDZYEgCDsQBGEHgiDsQBCEHQgizCmuN954Y7L+2WefJevbt2/PrT344IPJdRctWpSsl6no1N6iIckiqctgFw2tbdiwIVkfMya9r3rqqadya2fPXnu6x+jHnh0IgrADQRB2IAjCDgRB2IEgCDsQBGEHgrCiUz/rurEWvlJN0aWmU5c9njNnTnLdZ555JlkvOtXz/fffT9bfe++93NqVK1eS6/b09CTr8+bNS9bvv//+ZD11jEHR8QdFl7E+dOhQsn7PPffk1oqOqxjJ3N0qLWfPDgRB2IEgCDsQBGEHgiDsQBCEHQiCsANBMM5eB0XnVd98883J+quvvpqsF43jHzlyJLdWdN72/Pnzk/Wi89kPHDiQrC9cuDC3tnZt+grkRcc+rFixIlnftWtXsj5aMc4OBEfYgSAIOxAEYQeCIOxAEIQdCIKwA0GEuW58IxWdM3769Olkveic8KLx6H379uXWisbJFy9enKzv2bMnWb9w4UKyvmnTptxa0Tj6u+++m6zv3bs3WcfXFe7ZzexlM+szs0NDlj1pZj1mtj/7u6+xbQKo1XA+xv9K0qoKyze6+5Lsb1t92wJQb4Vhd/ddkuLNlQOMMrX8QPewmX2UfcyflvckM+s0sy4z66phWwBqVG3YfyFpgaQlknol/Szvie6+2d3b3b29ym0BqIOqwu7up939srtfkfRLScvq2xaAeqsq7GbWNuTh9ySlr+kLoHSF57Ob2WuSVkiaIem0pJ9mj5dIckndkn7g7r2FGxul57M32tSpU5P18+fP59YuXrxY07aLztWfMGFCsr579+7c2m233ZZc99577636tSPLO5+98KAad++osPilmjsC0FQcLgsEQdiBIAg7EARhB4Ig7EAQnOI6AvT395e27fHjxyfrGzZsSNaXLl2aW9u+fXtyXYbW6os9OxAEYQeCIOxAEIQdCIKwA0EQdiAIwg4EwZTNSFq9enWy/uabbybr586dy62tWlXpOqb/74MPPkjWURlTNgPBEXYgCMIOBEHYgSAIOxAEYQeCIOxAEJzPHtxNN92UrL/wwgvJ+tixY5P1bdvy5/xkHL252LMDQRB2IAjCDgRB2IEgCDsQBGEHgiDsQBCMswf3xBNPJOtz585N1o8fP56sP/7449fdExqjcM9uZnPM7E9m9rGZHTazH2XLp5vZDjM7mt1Oa3y7AKo1nI/xlyT9xN0XS/pnST80s8WSHpW0090XStqZPQbQogrD7u697r4vuz8g6RNJsyStkbQle9oWSQ80qkkAtbuu7+xm9i1JSyXtkTTT3Xuz0ilJM3PW6ZTUWX2LAOph2L/Gm9lkSb+T9GN3/9pMgz541cqKF5N0983u3u7u7TV1CqAmwwq7mY3TYNB/4+6/zxafNrO2rN4mqa8xLQKoh8KP8WZmkl6S9Im7/3xI6W1J6yU9m92+1ZAOUZNFixYl6+vWrUvWx4xJ7w+ee+65ZL1oaA7NM5zv7N+WtE7SQTPbny17TIMh32pmD0k6IWltY1oEUA+FYXf33ZIqXnRe0sr6tgOgUThcFgiCsANBEHYgCMIOBEHYgSCYsnmUO3r0aLK+YMGCZH3r1q3JekdHR7LezH9fGMSUzUBwhB0IgrADQRB2IAjCDgRB2IEgCDsQBJeSHgWefvrp3Nott9xS02tv2rQpWWccfeRgzw4EQdiBIAg7EARhB4Ig7EAQhB0IgrADQTDOPgIsX748WX/kkUdyaxMnTqxp2wMDAzWtj9bBnh0IgrADQRB2IAjCDgRB2IEgCDsQBGEHghjO/OxzJP1a0kxJLmmzu/+3mT0p6d8lncme+pi7b2tUo5HdfffdyfrkyZOrfu2i+dNPnDhR9WujtQznoJpLkn7i7vvMbIqkvWa2I6ttdPf/alx7AOplOPOz90rqze4PmNknkmY1ujEA9XVd39nN7FuSlkraky162Mw+MrOXzWxazjqdZtZlZl01dQqgJsMOu5lNlvQ7ST92935Jv5C0QNISDe75f1ZpPXff7O7t7t5eh34BVGlYYTezcRoM+m/c/feS5O6n3f2yu1+R9EtJyxrXJoBaFYbdzEzSS5I+cfefD1neNuRp35N0qP7tAaiX4fwa/21J6yQdNLP92bLHJHWY2RINDsd1S/pBQzqEuru7q173wIEDyfrKlSuT9bNnz1a9bbSW4fwav1tSpfmeGVMHRhCOoAOCIOxAEIQdCIKwA0EQdiAIwg4EYc2cctfMmN+3AaZMmZJb41LQ8bh7paFy9uxAFIQdCIKwA0EQdiAIwg4EQdiBIAg7EESzx9nPSBp6beIZkj5tWgPXp1V7a9W+JHqrVj17m+vuN1cqNDXs39i4WVerXpuuVXtr1b4keqtWs3rjYzwQBGEHgig77JtL3n5Kq/bWqn1J9FatpvRW6nd2AM1T9p4dQJMQdiCIUsJuZqvM7M9mdszMHi2jhzxm1m1mB81sf9nz02Vz6PWZ2aEhy6ab2Q4zO5rdVpxjr6TenjSznuy9229m95XU2xwz+5OZfWxmh83sR9nyUt+7RF9Ned+a/p3dzMZKOiLpO5JOSvpQUoe7f9zURnKYWbekdncv/QAMM/sXSX+T9Gt3/8ds2XOSzrr7s9l/KKe5+3+0SG9PSvpb2dN4Z7MVtQ2dZlzSA5L+TSW+d4m+1qoJ71sZe/Zlko65+1/c/aKk30paU0IfLc/dd0m6dkqWNZK2ZPe3aPAfS9Pl9NYS3L3X3fdl9wckXZ1mvNT3LtFXU5QR9lmS/jrk8Um11nzvLumPZrbXzDrLbqaCme7em90/JWlmmc1UUDiNdzNdM814y7x31Ux/Xit+oPum5e7+T5L+VdIPs4+rLckHv4O10tjpsKbxbpYK04z/XZnvXbXTn9eqjLD3SJoz5PHsbFlLcPee7LZP0htqvamoT1+dQTe77Su5n79rpWm8K00zrhZ478qc/ryMsH8oaaGZzTOz8ZK+L+ntEvr4BjOblP1wIjObJOm7ar2pqN+WtD67v17SWyX28jWtMo133jTjKvm9K336c3dv+p+k+zT4i/xxSf9ZRg85fc2XdCD7O1x2b5Je0+DHuq80+NvGQ5JukrRT0lFJ/ytpegv19qqkg5I+0mCw2krqbbkGP6J/JGl/9ndf2e9doq+mvG8cLgsEwQ90QBCEHQiCsANBEHYgCMIOBEHYgSAIOxDE/wE62qNQ0gY7GgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "print('Counterfactual prediction: {}'.format(explanation_k1.cf['class']))\n",
    "print('Closest prototype class: {}'.format(explanation.id_proto))\n",
    "plt.imshow(explanation_k1.cf['X'].reshape(28, 28));"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Results for $k$ equals 20:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Counterfactual prediction: 9\n",
      "Closest prototype class: 9\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAPIklEQVR4nO3df4xV9ZnH8c8DgkFBgcEdR8qvomhwk1IzkiWgYhqIEg2SGFP+QDdpdvpHNTVpUpFVQZMasrvdZo2mZpqa0k3XpqZUxZgUJFUkmsZRWUSEOioKODBUwm8RB57+MQcz4pzvGe5v5nm/ksncOc895z65+uHce7/3e77m7gIw+A2pdwMAaoOwA0EQdiAIwg4EQdiBIM6r5YOZGR/9A1Xm7tbf9rLO7GZ2k5ltN7NOM1tazrEAVJeVOs5uZkMl/U3SPEm7JL0habG7b03sw5kdqLJqnNlnSup09w/d/YSk30taWMbxAFRROWEfL2lnn793Zdu+xszazKzDzDrKeCwAZar6B3Tu3i6pXeJlPFBP5ZzZd0ua0Ofvb2XbADSgcsL+hqQrzGyKmQ2X9H1Jz1emLQCVVvLLeHfvMbO7Jf1Z0lBJT7n7uxXrDEBFlTz0VtKD8Z4dqLqqfKkGwLmDsANBEHYgCMIOBEHYgSAIOxBETeezA2dj9OjRyfqBAwdq1MngwJkdCIKwA0EQdiAIwg4EQdiBIAg7EASz3pBk1u8Eqq/Uc2HQSy+9NFnfs2dPjTppLMx6A4Ij7EAQhB0IgrADQRB2IAjCDgRB2IEgGGcfoJEjR+bWjh8/nty3aKx60qRJyXpnZ2eyDvTFODsQHGEHgiDsQBCEHQiCsANBEHYgCMIOBME4ewU0NTUl65999lmyPnz48GT9xIkTyfrFF1+cWzt48GBy38Es6vOSN85e1nXjzWyHpMOSTkrqcffWco4HoHoqsUjEje7+9wocB0AV8Z4dCKLcsLuktWb2ppm19XcHM2szsw4z6yjzsQCUodyX8XPcfbeZ/ZOkdWa2zd039L2Du7dLapcG7wd0wLmgrDO7u+/OfndL+pOkmZVoCkDllRx2M7vQzEadvi1pvqQtlWoMQGWVPM5uZt9W79lc6n078H/u/rOCfUK+jJ88eXKyvm/fvmT96NGjFeymskaNGpWsz549O7d2+eWXJ/ft6upK1rdt25asp64jcOTIkeS+O3bsSNYbWcXH2d39Q0nfKbkjADXF0BsQBGEHgiDsQBCEHQiCsANBVGIiTAi33357bq1o6eARI0Yk6+vWrUvWN23alKxX0/jx45P11DRSSbr++utza9u3b0/ue+zYsWR94sSJyfpLL72UW/vyyy+T+w5GnNmBIAg7EARhB4Ig7EAQhB0IgrADQRB2IAguJZ0ZN25csj516tTc2vTp05P7fvLJJ8n6/Pnzk/W33347WV+wYEFubdiwYcl9i8bJW1pakvXPP/88WS9HaplsqXia6q5du3JrK1asSO67devWZL2RsWQzEBxhB4Ig7EAQhB0IgrADQRB2IAjCDgQRZpz9sssuS9Y//fTTGnXyTatWrUrWhwxJ/5ucmjP+0UcfJfedNGlSsn7eeelLHhTNte/u7s6tXXXVVcl9i777UHQp6tSlpB9++OHkvkX1RsY4OxAcYQeCIOxAEIQdCIKwA0EQdiAIwg4EEea68fUcRy9StDzwPffck6yn5toXzQk/ePBgsn7ttdcm601NTSUfv2g++tVXX52sP/roo8l66vip8f/BqvDMbmZPmVm3mW3ps22sma0zs/ez32Oq2yaAcg3kZfxvJN10xralkta7+xWS1md/A2hghWF39w2S9p+xeaGk09/xXCXptgr3BaDCSn3P3uzuXdntPZKa8+5oZm2S2kp8HAAVUvYHdO7uqQku7t4uqV1q7AtOAoNdqUNve82sRZKy3/E+2gTOMaWG/XlJd2W375L0XGXaAVAthfPZzexpSXMljZO0V9JySc9K+oOkiZI+lnSHu5/5IV5/x2rYl/FFc6M7OztLPvaaNWuS9aJx9p07dybrw4cPz6098cQTyX2L/vsfOHAgWS+a797T05NbK5rPXjTOvmzZsmQ91duNN96Y3Hf//sL/nRtW3nz2wvfs7r44p/S9sjoCUFN8XRYIgrADQRB2IAjCDgRB2IEgwkxxLVLO0FqRV155JVmfN29esl40xTU1jHTllVcm9y0a1ps2bVqyfvz48WT95ptvzq0VDes98sgjyfqJEyeS9Yceeii3di4PrZWKMzsQBGEHgiDsQBCEHQiCsANBEHYgCMIOBME4ew1Mnz49WS8aL77//vuT9eeey7+cQFdXV25NkubPn5+s33LLLcl60SWZjx07lltrbs69mtmAjn3JJZck62vXrk3Wo+HMDgRB2IEgCDsQBGEHgiDsQBCEHQiCsANBFF5KuqIP1sCXkq6nG264IVl/+eWXk/UNGzbk1g4dOpTc97XXXkvWH3/88WR91qxZyfqtt96aWyuaKz9hwoRkffny5cn6M888k1sbOnRoct+TJ08m640s71LSnNmBIAg7EARhB4Ig7EAQhB0IgrADQRB2IAjms1dA0bzsomWNi8a6i64bf8EFF+TWvvjii+S+r776arJ++PDhZL1oznjqOwRFY/QdHR3J+rPPPpus4+sKz+xm9pSZdZvZlj7bVpjZbjPblP0sqG6bAMo1kJfxv5F0Uz/bf+HuM7KfFyvbFoBKKwy7u2+QFG+tHGCQKecDurvNbHP2Mn9M3p3MrM3MOsws/QYMQFWVGvZfSpoqaYakLkk/z7uju7e7e6u7t5b4WAAqoKSwu/tedz/p7qck/UrSzMq2BaDSSgq7mbX0+XORpC159wXQGArns5vZ05LmShonaa+k5dnfMyS5pB2Sfuju6QuUi/ns1ZIax+/p6anqY48YMSJZ37hxY25t9OjRyX2XLFmSrBd9PyGqvPnshV+qcffF/Wz+ddkdAagpvi4LBEHYgSAIOxAEYQeCIOxAEExxPQcUXfY4Nbx2/vnnJ/ctmn579OjRZL3oUtPXXHNNbu2xxx5L7svQWmVxZgeCIOxAEIQdCIKwA0EQdiAIwg4EQdiBIFiyGUlFyyI/8MADyXpqnL61NX3xos7OzmQd/WPJZiA4wg4EQdiBIAg7EARhB4Ig7EAQhB0IgvnswTU1NSXrd955Z7J+/PjxZP3FF/PX/GQcvbY4swNBEHYgCMIOBEHYgSAIOxAEYQeCIOxAEIyzD3JF15x/8sknk/UpU6Yk6x988EGy/uCDDybrqJ3CM7uZTTCzv5jZVjN718x+nG0fa2brzOz97PeY6rcLoFQDeRnfI+kn7j5d0r9I+pGZTZe0VNJ6d79C0vrsbwANqjDs7t7l7m9ltw9Lek/SeEkLJa3K7rZK0m3VahJA+c7qPbuZTZb0XUl/ldTs7l1ZaY+k5px92iS1ld4igEoY8KfxZjZS0h8l3evuh/rWvPeqlf1eTNLd29291d3TVxcEUFUDCruZDVNv0H/n7quzzXvNrCWrt0jqrk6LACqh8FLSZmbqfU++393v7bP9PyV95u4rzWyppLHu/tOCY3Ep6RqbNm1asr5t27ayjr9w4cJkfc2aNWUdH2cv71LSA3nPPlvSEknvmNmmbNsySSsl/cHMfiDpY0l3VKJRANVRGHZ33yip338pJH2vsu0AqBa+LgsEQdiBIAg7EARhB4Ig7EAQTHEdBC666KLc2tq1a8s69sqVK5P1F154oazjo3Y4swNBEHYgCMIOBEHYgSAIOxAEYQeCIOxAEIyzDwL33Xdfbm3ixIllHXv16tXJetH1ENA4OLMDQRB2IAjCDgRB2IEgCDsQBGEHgiDsQBCMs58D5syZk6zPmjUrt7Zv377kviNHjiypJ5x7OLMDQRB2IAjCDgRB2IEgCDsQBGEHgiDsQBCF4+xmNkHSbyU1S3JJ7e7+P2a2QtK/STo9kLvM3V+sVqORXXfddcn63LlzSz7266+/nqzv2bOn5GOjsQzkSzU9kn7i7m+Z2ShJb5rZuqz2C3f/r+q1B6BSBrI+e5ekruz2YTN7T9L4ajcGoLLO6j27mU2W9F1Jf8023W1mm83sKTMbk7NPm5l1mFlHWZ0CKMuAw25mIyX9UdK97n5I0i8lTZU0Q71n/p/3t5+7t7t7q7u3VqBfACUaUNjNbJh6g/47d18tSe6+191PuvspSb+SNLN6bQIoV2HYzcwk/VrSe+7+3322t/S52yJJWyrfHoBKGcin8bMlLZH0jpltyrYtk7TYzGaodzhuh6QfVqVDaOfOncn6wYMHc2ubN29O7rto0aJkff/+/ck6zh0D+TR+oyTrp8SYOnAO4Rt0QBCEHQiCsANBEHYgCMIOBEHYgSCslkvumhnr+9bYkCHpf89PnTpVo05QK+7e31A5Z3YgCsIOBEHYgSAIOxAEYQeCIOxAEIQdCKLWSzb/XdLHff4el21rRI3a21n1VeNx9EZ9zqQ4vU3KK9T0SzXfeHCzjka9Nl2j9taofUn0Vqpa9cbLeCAIwg4EUe+wt9f58VMatbdG7Uuit1LVpLe6vmcHUDv1PrMDqBHCDgRRl7Cb2U1mtt3MOs1saT16yGNmO8zsHTPbVO/16bI19LrNbEufbWPNbJ2ZvZ/97neNvTr1tsLMdmfP3SYzW1Cn3iaY2V/MbKuZvWtmP8621/W5S/RVk+et5u/ZzWyopL9Jmidpl6Q3JC129601bSSHme2Q1Orudf8ChpldL+mIpN+6+z9n2/5D0n53X5n9QznG3e9rkN5WSDpS72W8s9WKWvouMy7pNkn/qjo+d4m+7lANnrd6nNlnSup09w/d/YSk30taWIc+Gp67b5B05pIsCyWtym6vUu//LDWX01tDcPcud38ru31Y0ullxuv63CX6qol6hH28pL7rGe1SY6337pLWmtmbZtZW72b60ezuXdntPZKa69lMPwqX8a6lM5YZb5jnrpTlz8vFB3TfNMfdr5F0s6QfZS9XG5L3vgdrpLHTAS3jXSv9LDP+lXo+d6Uuf16ueoR9t6QJff7+VratIbj77ux3t6Q/qfGWot57egXd7Hd3nfv5SiMt493fMuNqgOeunsuf1yPsb0i6wsymmNlwSd+X9Hwd+vgGM7sw++BEZnahpPlqvKWon5d0V3b7LknP1bGXr2mUZbzzlhlXnZ+7ui9/7u41/5G0QL2fyH8g6d/r0UNOX9+W9P/Zz7v17k3S0+p9Wfelej/b+IGkJknrJb0v6SVJYxuot/+V9I6kzeoNVkudepuj3pfomyVtyn4W1Pu5S/RVk+eNr8sCQfABHRAEYQeCIOxAEIQdCIKwA0EQdiAIwg4E8Q93zuOrItWuUAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "print('Counterfactual prediction: {}'.format(explanation_k20.cf['class']))\n",
    "print('Closest prototype class: {}'.format(explanation.id_proto))\n",
    "plt.imshow(explanation_k20.cf['X'].reshape(28, 28));"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A lower value of $k$ typically leads to counterfactuals that look more like the original instance and less like an average instance of the counterfactual class."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Remove the autoencoder loss term $L_{AE}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the previous example, we used both an autoencoder loss term to penalize a counterfactual which falls outside of the training data distribution as well as an encoder loss term to guide the counterfactual to the nearest prototype class. In the next example we get rid of the autoencoder loss term to speed up the counterfactual search and still generate decent counterfactuals:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Explanation took 6.284 sec\n"
     ]
    }
   ],
   "source": [
    "# initialize explainer, fit and generate counterfactuals\n",
    "cf = CounterFactualProto(cnn, shape, gamma=gamma, theta=theta,\n",
    "                         enc_model=enc, max_iterations=max_iterations,\n",
    "                         feature_range=feature_range, c_init=c_init, c_steps=c_steps)\n",
    "cf.fit(x_train)\n",
    "start_time = time()\n",
    "explanation = cf.explain(X, k=1)\n",
    "print('Explanation took {:.3f} sec'.format(time() - start_time))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Counterfactual prediction: 9\n",
      "Closest prototype class: 9\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAANsklEQVR4nO3db4xV9Z3H8c9nsTUEasJgJISywjbyoGkUFMkmxQ0G2yA+GBqNKTGNm1THxLqpSeMuuJJqjMa424X1gU2o1dKVlZBY/ytbdtKE9IENo0EFlSoE7JARZDEpGA1iv/tgDs2Ic393vP+H7/uVTO6953vPud9c+XjuOefe388RIQBnv7/pdgMAOoOwA0kQdiAJwg4kQdiBJM7p5IvZ5tQ/0GYR4fGWN7Vnt73C9l7b79pe08y2ALSXG73ObnuKpD9K+o6kYUk7Ja2OiDcL67BnB9qsHXv2JZLejYj9EXFS0hZJ/U1sD0AbNRP2OZL+NObxcLXsc2wP2B6yPdTEawFoUttP0EXERkkbJT7GA93UzJ79kKS5Yx5/vVoGoAc1E/adki6yPd/2VyV9X9KzrWkLQKs1/DE+Ik7Zvk3S/0iaIunRiNjTss4AtFTDl94aejGO2YG2a8uXagBMHoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJBqen12SbB+QdFzSZ5JORcTiVjQFoPWaCnvlyog42oLtAGgjPsYDSTQb9pD0W9uv2B4Y7wm2B2wP2R5q8rUANMER0fjK9pyIOGT7AknbJf1TROwoPL/xFwMwIRHh8ZY3tWePiEPV7RFJT0la0sz2ALRPw2G3Pc32107fl/RdSbtb1RiA1mrmbPwsSU/ZPr2d/46IbS3pCi0zderUYv3jjz/uUCfotqaO2b/0i3HM3nGEPZ+2HLMDmDwIO5AEYQeSIOxAEoQdSKIVP4RJ76qrrirWb7jhhmL9k08+KdY3b95crJ86dapmbWRkpLju/fffX6zX6/3qq68u1l966aViHZ3Dnh1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkuBXbxO0f//+mrV58+Z1rpFxHD9+vGZtz549xXXPOaf8VYvSNfxeNzw8XLP24IMPFtcdGpq8o6jxqzcgOcIOJEHYgSQIO5AEYQeSIOxAEoQdSILfs0/QzTffXLN26623Ftfdtq08wvb06dOL9UsvvbRYX7ZsWc3ajh01J+iRJK1evbpYnzt3brHejHrX8D/44INiffbs2Q2/9nvvvVesT+br7LWwZweSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJLjOPkGDg4M1awcOHCiuu2/fvhZ383kzZsyoWVu4cGFx3Xrjxl9++eXF+s6dOxtev954+RdffHGxfu+99xbrfX19NWvt/m/Si+ru2W0/avuI7d1jlvXZ3m77neq29r82AD1hIh/jfyVpxRnL1kgajIiLJA1WjwH0sLphj4gdko6dsbhf0qbq/iZJq1rcF4AWa/SYfVZEnJ5E7H1Js2o90faApIEGXwdAizR9gi4iojSQZERslLRRmtwDTgKTXaOX3g7bni1J1e2R1rUEoB0aDfuzkm6s7t8o6ZnWtAOgXeqOG2/7CUnLJJ0v6bCkn0p6WtJWSX8r6aCk6yPizJN4421r0n6M37t3b83aggULiutu2bKlWK/3m/Ksrr322mJ969atxfru3btr1q688sriup9++mmxXhqrv9tqjRtf95g9Imr9S1zeVEcAOoqvywJJEHYgCcIOJEHYgSQIO5AEUza3wAQuX3aok95Tuny2fv364rrnnntusX7BBRcU69ddd13N2pNPPllcdzJjymYgOcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKhpFvgsssuK9YfeeSRYv2SSy4p1usN59xO/f39xfozz5SHMihNN/3YY48V1123bl2x/uGHHxbrpZ8lZ8SeHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeS4Dp7CwwPDxfrN910U7H+8MMPF+tHjx4t1mfOnFmztmHDhuK6d911V7F+8ODBYr3eb/lLvZ933nnFdetZtao8xWBpKOmM2LMDSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKMG3+W++ijj4r1adOmtfX177vvvpq1tWvXFtcdHBws1leuXFms15t2+WzV8Ljxth+1fcT27jHL7rZ9yPau6q/8rgPouol8jP+VpBXjLF8fEQurvxdb2xaAVqsb9ojYIelYB3oB0EbNnKC7zfbr1cf8GbWeZHvA9pDtoSZeC0CTGg37zyV9Q9JCSSOSflbriRGxMSIWR8TiBl8LQAs0FPaIOBwRn0XEXyT9QtKS1rYFoNUaCrvt2WMefk8SvyUEelzd37PbfkLSMknn2x6W9FNJy2wvlBSSDki6pY09ogntvo4+derUYn3FivEu5Iw6efJkcd3HH3+8WM96Hb1RdcMeEavHWfzLNvQCoI34uiyQBGEHkiDsQBKEHUiCsANJMJQ0mnLHHXcU64sWLapZ27ZtW3HdTZs2NdQTxseeHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSYChpFF1zzTXF+tNPP12sl4ayLv38VZJefvnlYh3ja3goaQBnB8IOJEHYgSQIO5AEYQeSIOxAEoQdSILfsyc3c+bMYv2hhx4q1qdMmVKsv/hi7Tk/uY7eWezZgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJrrOf5epdB9+wYUOxPn/+/GJ93759xfq6deuKdXRO3T277bm2f2f7Tdt7bP+4Wt5ne7vtd6rbGe1vF0CjJvIx/pSkn0TENyX9vaQf2f6mpDWSBiPiIkmD1WMAPapu2CNiJCJere4fl/SWpDmS+iWdnp9nk6RV7WoSQPO+1DG77XmSFkn6g6RZETFSld6XNKvGOgOSBhpvEUArTPhsvO3pkp6UdHtE/HlsLUZHrRx3MMmI2BgRiyNicVOdAmjKhMJu+ysaDfrmiPhNtfiw7dlVfbakI+1pEUAr1B1K2rY1ekx+LCJuH7P83yT9X0Q8YHuNpL6I+Oc622Io6Q5bsGBBsf722283tf3+/v5i/bnnnmtq+/jyag0lPZFj9m9L+oGkN2zvqpbdKekBSVtt/1DSQUnXt6JRAO1RN+wR8XtJ4/6fQtLy1rYDoF34uiyQBGEHkiDsQBKEHUiCsANJ8BPXs8CFF15YszZ9+vSmtv3CCy8U688//3xT20fnsGcHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSS4zn4WGBioPerX2rVrm9r2PffcU6zXGw8BvYM9O5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kwXX2SWDp0qXF+i233NKhTjCZsWcHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSTqXme3PVfSryXNkhSSNkbEf9q+W9LNkj6onnpnRLzYrkYzu+KKK4r1vr6+hre9b9++Yv3EiRMNbxu9ZSJfqjkl6ScR8artr0l6xfb2qrY+Iv69fe0BaJWJzM8+Immkun/c9luS5rS7MQCt9aWO2W3Pk7RI0h+qRbfZft32o7Zn1FhnwPaQ7aGmOgXQlAmH3fZ0SU9Kuj0i/izp55K+IWmhRvf8PxtvvYjYGBGLI2JxC/oF0KAJhd32VzQa9M0R8RtJiojDEfFZRPxF0i8kLWlfmwCaVTfsti3pl5Leioj/GLN89pinfU/S7ta3B6BVJnI2/tuSfiDpDdu7qmV3Slpte6FGL8cdkMTvLHvQa6+9VqwvX768WD927Fgr20EXTeRs/O8leZwS19SBSYRv0AFJEHYgCcIOJEHYgSQIO5AEYQeScCen3LXN/L5Am0XEeJfK2bMDWRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKdnrL5qKSDYx6fXy3rRb3aW6/2JdFbo1rZ24W1Ch39Us0XXtwe6tWx6Xq1t17tS6K3RnWqNz7GA0kQdiCJbod9Y5dfv6RXe+vVviR6a1RHeuvqMTuAzun2nh1AhxB2IImuhN32Ctt7bb9re003eqjF9gHbb9je1e356ao59I7Y3j1mWZ/t7bbfqW7HnWOvS73dbftQ9d7tsr2yS73Ntf0722/a3mP7x9Xyrr53hb468r51/Jjd9hRJf5T0HUnDknZKWh0Rb3a0kRpsH5C0OCK6/gUM2/8g6YSkX0fEt6plD0o6FhEPVP+jnBER/9Ijvd0t6US3p/GuZiuaPXaacUmrJP2juvjeFfq6Xh1437qxZ18i6d2I2B8RJyVtkdTfhT56XkTskHTmlCz9kjZV9zdp9B9Lx9XorSdExEhEvFrdPy7p9DTjXX3vCn11RDfCPkfSn8Y8HlZvzfcekn5r+xXbA91uZhyzImKkuv++pFndbGYcdafx7qQzphnvmfeukenPm8UJui9aGhGXSrpa0o+qj6s9KUaPwXrp2umEpvHulHGmGf+rbr53jU5/3qxuhP2QpLljHn+9WtYTIuJQdXtE0lPqvamoD5+eQbe6PdLlfv6ql6bxHm+acfXAe9fN6c+7Efadki6yPd/2VyV9X9KzXejjC2xPq06cyPY0Sd9V701F/aykG6v7N0p6pou9fE6vTONda5pxdfm96/r05xHR8T9JKzV6Rn6fpH/tRg81+vo7Sa9Vf3u63ZukJzT6se5TjZ7b+KGkmZIGJb0j6X8l9fVQb/8l6Q1Jr2s0WLO71NtSjX5Ef13SrupvZbffu0JfHXnf+LoskAQn6IAkCDuQBGEHkiDsQBKEHUiCsANJEHYgif8HvXBYH/noyN0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "print('Counterfactual prediction: {}'.format(explanation.cf['class']))\n",
    "print('Closest prototype class: {}'.format(explanation.id_proto))\n",
    "plt.imshow(explanation.cf['X'].reshape(28, 28));"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Specify prototype classes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For multi-class predictions, we might be interested to generate counterfactuals for certain classes while avoiding others. The following example illustrates how to do this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAANaUlEQVR4nO3db6xU9Z3H8c9HlybGkoiSJRdKpEt8UiQBQohJEdGmjcsTbEhMeVDYaPbWWDZt7IP1T0iNPDHrtnVjTBOIWqrVhtAaMFaEvWk0fWADIqtcTJFtMIVcwcaYin/CCt99cA/NVe/85jJnzszA9/1KbmbmfOfM+XrCx3Pm/Gbm54gQgIvfJf1uAEBvEHYgCcIOJEHYgSQIO5DEP/RyY7a59A80LCI82fJaR3bbN9v+k+0jtu+u81oAmuVOx9ltXyrpsKRvSjomaa+ktRFxqLAOR3agYU0c2ZdJOhIRf46I05J+LWl1jdcD0KA6YZ8j6S8THh+rln2G7WHb+2zvq7EtADU1foEuIjZL2ixxGg/0U50j+3FJcyc8/kq1DMAAqhP2vZKusf1V21+S9B1JO7vTFoBu6/g0PiI+tb1B0ouSLpX0eESMdq0zAF3V8dBbRxvjPTvQuEY+VAPgwkHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSKLj+dklyfZRSR9IOiPp04hY2o2mAHRfrbBXboyIv3bhdQA0iNN4IIm6YQ9Ju22/ant4sifYHra9z/a+mtsCUIMjovOV7TkRcdz2P0raI+nfIuLlwvM73xiAKYkIT7a81pE9Io5XtyclPStpWZ3XA9CcjsNu+3Lb08/dl/QtSQe71RiA7qpzNX6WpGdtn3udpyNiV1e6AtB1td6zn/fGeM8ONK6R9+wALhyEHUiCsANJEHYgCcIOJNGNL8JggC1atKhY37RpU7G+atWqYv2SS8rHi7Nnz7asbd++vbjufffdV6yPjY0V6zfeeGPL2sjISHHdjz/+uFi/EHFkB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkGGe/AEybNq1Yv+GGG1rWnnjiieK6Q0NDxXq7b0WWxtHbrb9mzZriuu3GuufOnVusr1y5smVt/fr1xXWfeuqpYv1CxJEdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5JgnP0CsGTJkmJ9167Of8G73XfCN2zYUKx/9NFHHW/76quvLtY//PDDYv2RRx4p1k+fPt2y1u6/+2LEkR1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkmCcfQAsWLCgWN+5c2fHr93u99HvueeeYn3//v0db7ud2bNnF+s7duwo1q+44opi/aGHHmpZa7dfLkZtj+y2H7d90vbBCcuutL3H9lvV7Yxm2wRQ11RO438h6ebPLbtb0khEXCNppHoMYIC1DXtEvCzpvc8tXi1pa3V/q6RbutwXgC7r9D37rIg49+HidyTNavVE28OShjvcDoAuqX2BLiLCdstfFYyIzZI2S1LpeQCa1enQ2wnbQ5JU3Z7sXksAmtBp2HdKOvdbvOsllcdIAPRd29N4289IWilppu1jkn4s6UFJ22zfLultSbc22eTFbuPGjcX6zJkzi/Xnn3++Ze2uu+4qrnvkyJFivUnXXnttsb548eJar1/ne/4Xo7Zhj4i1LUrf6HIvABrEx2WBJAg7kARhB5Ig7EAShB1Iwu2m5O3qxpJ+gm7Lli3F+m233Vast/tJ5euuu65l7dChQ8V1m1aabnr37t3FdVesWFGsv/TSS8X6TTfdVKxfrCLCky3nyA4kQdiBJAg7kARhB5Ig7EAShB1IgrADSfBT0j2wdOnSYr3dZx1OnTpVrPdzLL00ji5JmzZtalm7/vrri+u22y8PPPBAsY7P4sgOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kwzo6iefPmFet33nlnsd7up6xLxsbGivUDBw50/NoZcWQHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQYZ++Bdt83X7hwYbF+1VVXFeuvvfbaefc0Ve2mi549e3axXmdegpGRkWL9/fff7/i1M2p7ZLf9uO2Ttg9OWHa/7eO2D1R/q5ptE0BdUzmN/4WkmydZ/rOIWFT9/a67bQHotrZhj4iXJb3Xg14ANKjOBboNtl+vTvNntHqS7WHb+2zvq7EtADV1GvafS5ovaZGkMUk/afXEiNgcEUsjovyriwAa1VHYI+JERJyJiLOStkha1t22AHRbR2G3PTTh4bclHWz1XACDoe387LafkbRS0kxJJyT9uHq8SFJIOirpexFR/vKx8s7PftlllxXr27ZtK9ZXrSqPbNYZy65r9erVxfq6deta1tasWVNcd/ny5cX6K6+8Uqxn1Wp+9rYfqomItZMsfqx2RwB6io/LAkkQdiAJwg4kQdiBJAg7kETbobeubizp0FtdK1euLNbbTQldMjo6Wqy/8MILxfqjjz5arN9xxx0ta4cPHy6uu2LFimL93XffLdazajX0xpEdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5JgnB21nDlzplgv/ft6+umni+uWvh6L1hhnB5Ij7EAShB1IgrADSRB2IAnCDiRB2IEkmLIZRfPmzau1/qlTp1rWHn744VqvjfPDkR1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkmCcHUUbN26stf5zzz3XsrZ///5ar43z0/bIbnuu7d/bPmR71PYPquVX2t5j+63qdkbz7QLo1FRO4z+V9KOI+Jqk6yR93/bXJN0taSQirpE0Uj0GMKDahj0ixiJif3X/A0lvSpojabWkrdXTtkq6pakmAdR3Xu/Zbc+TtFjSHyXNioixqvSOpFkt1hmWNNx5iwC6YcpX421/WdJvJP0wIv42sRbjvyo46S8LRsTmiFgaEZ3PPgigtimF3fY0jQf9VxHx22rxCdtDVX1I0slmWgTQDW1P421b0mOS3oyIn04o7ZS0XtKD1e2ORjpEoxYsWFCsr1mzptbrv/jii7XWR/dM5T371yV9V9Ibtg9Uy+7VeMi32b5d0tuSbm2mRQDd0DbsEfEHSZP+6Lykb3S3HQBN4eOyQBKEHUiCsANJEHYgCcIOJMFXXJNbsmRJsT59+vRivd2U35988sl594RmcGQHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQYZ09u5syZxXq7cfTR0dFiffv27efdE5rBkR1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkmCcPbl169bVWv/JJ5/sUidoGkd2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUhiKvOzz5X0S0mzJIWkzRHxX7bvl/Svkt6tnnpvRPyuqUbRjEOHDhXrCxcu7FEnaNpUPlTzqaQfRcR+29MlvWp7T1X7WUT8Z3PtAeiWqczPPiZprLr/ge03Jc1pujEA3XVe79ltz5O0WNIfq0UbbL9u+3HbM1qsM2x7n+19tToFUMuUw277y5J+I+mHEfE3ST+XNF/SIo0f+X8y2XoRsTkilkbE0i70C6BDUwq77WkaD/qvIuK3khQRJyLiTESclbRF0rLm2gRQV9uw27akxyS9GRE/nbB8aMLTvi3pYPfbA9AtU7ka/3VJ35X0hu0D1bJ7Ja21vUjjw3FHJX2vkQ7RqF27dhXr8+fPL9b37t3bzXbQoKlcjf+DJE9SYkwduIDwCTogCcIOJEHYgSQIO5AEYQeSIOxAEm43JW9XN2b3bmNAUhEx2VA5R3YgC8IOJEHYgSQIO5AEYQeSIOxAEoQdSKLXUzb/VdLbEx7PrJYNokHtbVD7kuitU93s7epWhZ5+qOYLG7f3Depv0w1qb4Pal0RvnepVb5zGA0kQdiCJfod9c5+3XzKovQ1qXxK9daonvfX1PTuA3un3kR1AjxB2IIm+hN32zbb/ZPuI7bv70UMrto/afsP2gX7PT1fNoXfS9sEJy660vcf2W9XtpHPs9am3+20fr/bdAdur+tTbXNu/t33I9qjtH1TL+7rvCn31ZL/1/D277UslHZb0TUnHJO2VtDYiyhOF94jto5KWRkTfP4Bhe4WkU5J+GRHXVsv+Q9J7EfFg9T/KGRHx7wPS2/2STvV7Gu9qtqKhidOMS7pF0r+oj/uu0Net6sF+68eRfZmkIxHx54g4LenXklb3oY+BFxEvS3rvc4tXS9pa3d+q8X8sPdeit4EQEWMRsb+6/4Gkc9OM93XfFfrqiX6EfY6kv0x4fEyDNd97SNpt+1Xbw/1uZhKzImKsuv+OpFn9bGYSbafx7qXPTTM+MPuuk+nP6+IC3Rctj4glkv5Z0ver09WBFOPvwQZp7HRK03j3yiTTjP9dP/ddp9Of19WPsB+XNHfC469UywZCRByvbk9KelaDNxX1iXMz6Fa3J/vcz98N0jTek00zrgHYd/2c/rwfYd8r6RrbX7X9JUnfkbSzD318ge3Lqwsnsn25pG9p8Kai3ilpfXV/vaQdfezlMwZlGu9W04yrz/uu79OfR0TP/ySt0vgV+f+VdF8/emjR1z9J+p/qb7TfvUl6RuOndf+n8Wsbt0u6StKIpLck/bekKweotyclvSHpdY0Ha6hPvS3X+Cn665IOVH+r+r3vCn31ZL/xcVkgCS7QAUkQdiAJwg4kQdiBJAg7kARhB5Ig7EAS/w9Q6DH2lEzy9wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "X = x_test[12].reshape((1,) + x_test[1].shape)\n",
    "plt.imshow(X.reshape(28, 28));"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "# initialize explainer, fit and generate counterfactuals\n",
    "cf = CounterFactualProto(cnn, shape, gamma=gamma, theta=theta,\n",
    "                         ae_model=ae, enc_model=enc, max_iterations=max_iterations,\n",
    "                         feature_range=feature_range, c_init=c_init, c_steps=c_steps)\n",
    "cf.fit(x_train)\n",
    "explanation_1 = cf.explain(X, k=5, k_type='mean')\n",
    "proto_1 = explanation_1.id_proto\n",
    "explanation_2 = cf.explain(X, k=5, k_type='mean', target_class=[7])\n",
    "proto_2 = explanation_2.id_proto"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The closest class to the 9 is 4. This is evident by looking at the first counterfactual below. For the second counterfactual, we specified that the prototype class used in the search should be a 7. As a result, a counterfactual 7 instead of a 4 is generated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Counterfactual prediction: 4\n",
      "Closest prototype class: 4\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAN1klEQVR4nO3dXaxV9ZnH8d9PRxOlXKBYBErmMOhNkYgGcWKrwdQ2Dl5gE0PqhTiRDMZoY0kvfJnEcuGFmdiSuTA1VE2pcWwI1fgapkhE0xgbXsIoL5YXoyIeYRo1BSVB7TMXZzE56tn/fdh7rb03PN9PcrL3Xs9eez1Z+mOtvV723xEhAKe+0/rdAIDeIOxAEoQdSIKwA0kQdiCJf+jlwmxz6B9oWER4rOldbdltX2v7L7b32r67m88C0Cx3ep7d9umSdkv6oaT3JW2SdGNE7CzMw5YdaFgTW/b5kvZGxNsRcUzS7yUt6uLzADSom7BPl7R/1Ov3q2lfYXuZ7c22N3exLABdavwAXUSskrRKYjce6KdutuwHJM0Y9fo71TQAA6ibsG+SdKHtmbbPlPQTSc/W0xaAunW8Gx8RX9i+Q9J/Szpd0mMRsaO2zgDUquNTbx0tjO/sQOMauagGwMmDsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQ6HrIZp4azzz67WP/ss8961Ama1lXYbb8j6bCkLyV9ERHz6mgKQP3q2LJfHRF/reFzADSI7+xAEt2GPST90fYW28vGeoPtZbY3297c5bIAdMER0fnM9vSIOGD725LWS/ppRLxaeH/nC0MjOEB36okIjzW9qy17RByoHg9JelrS/G4+D0BzOg677Qm2Jx5/LulHkrbX1RiAenVzNH6KpKdtH/+c/4qIdbV0hRMyceLElrVLLrmkOG+73fh16/hPeqroOOwR8baki2vsBUCDOPUGJEHYgSQIO5AEYQeSIOxAEl1dQXfCC+MKuo60Oz02NDTUsvbwww8X5501a1ax3u4KujVr1hTr06ZNa1nbtWtXcd6nnnqqWB8eHi7Wr7766pa1DRs2FOc9evRosT7IGrmCDsDJg7ADSRB2IAnCDiRB2IEkCDuQBGEHkuCnpE8Cn3/+ebF+6623tqxdfHH5xsTS7bGStHfv3mL92LFjxfru3btb1qZMmVKc97bbbivWL7300mJ9wYIFLWtLliwpzvv4448X6ycjtuxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kATn2U8C7c4n33LLLS1rEyZMKM771ltvFev33HNPsb5p06ZifebMmS1rZ555ZnHea665plhvdw1B6RqADz74oDjvqYgtO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kwe/GD4DZs2cX6xs3bizWzz333Ja1dr+PftdddxXrW7duLda7UfpNeUl67rnnivV2w1GvXbu2ZW3x4sXFeU9mHf9uvO3HbB+yvX3UtHNsr7e9p3qcVGezAOo3nt3430q69mvT7pa0ISIulLSheg1ggLUNe0S8Kumjr01eJGl19Xy1pOtr7gtAzTq9Nn5KRBwfaOtDSS1/TMz2MknLOlwOgJp0fSNMRETpwFtErJK0SuIAHdBPnZ56O2h7qiRVj4fqawlAEzoN+7OSbq6e3yzpmXraAdCUtrvxtp+UtEDSZNvvS/qFpAckrbG9VNK7kk7dk5Y1OO208r+pDz74YLE+efLkYv35559vWVu+fHlx3na/C9+kiy66qFhvdx69nYceeqir+U81bcMeETe2KP2g5l4ANIjLZYEkCDuQBGEHkiDsQBKEHUiCW1x74JFHHinWly5dWqwfOXKkWL/88stb1nbu3Fmct2lnnHFGy9pLL71UnPeqq64q1l955ZVivTRk86ms41tcAZwaCDuQBGEHkiDsQBKEHUiCsANJEHYgCYZs7oHLLrusWG93rcOnn35arPfzXHrpPLok3X///S1rV155ZXHedutlxYoVxTq+ii07kARhB5Ig7EAShB1IgrADSRB2IAnCDiTBefaTQLvzzZMmtR5E9+OPP+5q2UNDQ8X67bffXqzfeeedHS97eHi4WN+2bVvHn50RW3YgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSILz7D2wY8eOYn3OnDnFershmzdu3HiiLY3beeedV6xPnTq1WO9mXIJ2vyv/ySefdPzZGbXdstt+zPYh29tHTVth+4DtbdXfwmbbBNCt8ezG/1bStWNMXxkRc6u/F+ttC0Dd2oY9Il6V9FEPegHQoG4O0N1h+41qN7/lxdm2l9nebHtzF8sC0KVOw/5rSbMkzZU0LOmXrd4YEasiYl5EzOtwWQBq0FHYI+JgRHwZEX+X9BtJ8+ttC0DdOgq77dHnW34saXur9wIYDG3HZ7f9pKQFkiZLOijpF9XruZJC0juSbo2I8s3Hyjs++1lnnVWsr127tlhfuLB8ZrObc9nd2rJlS7E+bdq0lrV25+ivuOKKYv31118v1rNqNT5724tqIuLGMSY/2nVHAHqKy2WBJAg7kARhB5Ig7EAShB1Igltce+Do0aPF+nXXXVesL1iwoFhvNyR0yfbt5Usk9u/fX6wvX768WJ83r/WFk7t37y7Ou2/fvmIdJ4YtO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4k0fYW11oXlvQW11PZnj17ivVZs2a1rD3xxBPFeW+66aaOesqu1S2ubNmBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnuZ0fR0NBQsX7BBRcU64cPH25ZW7lyZSctoUNs2YEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCc6zo+i+++7rav4XX3yxZW3r1q1dfTZOTNstu+0Ztl+2vdP2Dtt3VtPPsb3e9p7qcVLz7QLo1Hh247+Q9POI+K6kf5Z0u+3vSrpb0oaIuFDShuo1gAHVNuwRMRwRW6vnhyXtkjRd0iJJq6u3rZZ0fVNNAujeCX1ntz0k6RJJf5Y0JSKGq9KHkqa0mGeZpGWdtwigDuM+Gm/7W5L+IOlnEfG30bUY+dXKMX9MMiJWRcS8iGg9wh+Axo0r7LbP0EjQn4iIp6rJB21PrepTJR1qpkUAdWi7G2/bkh6VtCsifjWq9KykmyU9UD0+00iHaNTs2bOL9RtuuKGrz3/hhRe6mh/1Gc939u9JuknSm7a3VdPu1UjI19heKuldSYubaRFAHdqGPSL+JGnMH52X9IN62wHQFC6XBZIg7EAShB1IgrADSRB2IAlucU1u7ty5xfrEiROL9XZDfu/bt++Ee0Iz2LIDSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKcZ0/u/PPPL9bbnUffsWNHsf7aa6+dcE9oBlt2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiC8+zJLVmypKv5N27cWE8jaBxbdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IYjzjs8+Q9DtJUySFpFUR8Z+2V0j6N0n/W7313oh4salG0Yx296PPmTOnWH/vvffqbAcNGs9FNV9I+nlEbLU9UdIW2+ur2sqIeLC59gDUZTzjsw9LGq6eH7a9S9L0phsDUK8T+s5ue0jSJZL+XE26w/Ybth+zPanFPMtsb7a9uatOAXRl3GG3/S1Jf5D0s4j4m6RfS5olaa5Gtvy/HGu+iFgVEfMiYl4N/QLo0LjCbvsMjQT9iYh4SpIi4mBEfBkRf5f0G0nzm2sTQLfaht22JT0qaVdE/GrU9Kmj3vZjSdvrbw9AXcZzNP57km6S9KbtbdW0eyXdaHuuRk7HvSPp1kY6RKPWrVtXrB86dKhYf/nll+tsBw0az9H4P0nyGCXOqQMnEa6gA5Ig7EAShB1IgrADSRB2IAnCDiThdkPy1rowu3cLA5KKiLFOlbNlB7Ig7EAShB1IgrADSRB2IAnCDiRB2IEkej1k818lvTvq9eRq2iAa1N4GtS+J3jpVZ2//2KrQ04tqvrFwe/Og/jbdoPY2qH1J9NapXvXGbjyQBGEHkuh32Ff1efklg9rboPYl0VunetJbX7+zA+idfm/ZAfQIYQeS6EvYbV9r+y+299q+ux89tGL7Hdtv2t7W7/HpqjH0DtnePmraObbX295TPY45xl6felth+0C17rbZXtin3mbYftn2Tts7bN9ZTe/ruiv01ZP11vPv7LZPl7Rb0g8lvS9pk6QbI2JnTxtpwfY7kuZFRN8vwLB9laQjkn4XERdV0/5D0kcR8UD1D+WkiLhrQHpbIelIv4fxrkYrmjp6mHFJ10v6V/Vx3RX6WqwerLd+bNnnS9obEW9HxDFJv5e0qA99DLyIeFXSR1+bvEjS6ur5ao38z9JzLXobCBExHBFbq+eHJR0fZryv667QV0/0I+zTJe0f9fp9DdZ47yHpj7a32F7W72bGMCUihqvnH0qa0s9mxtB2GO9e+tow4wOz7joZ/rxbHKD7pu9HxKWS/kXS7dXu6kCKke9gg3TudFzDePfKGMOM/79+rrtOhz/vVj/CfkDSjFGvv1NNGwgRcaB6PCTpaQ3eUNQHj4+gWz2WR17soUEaxnusYcY1AOuun8Of9yPsmyRdaHum7TMl/UTSs33o4xtsT6gOnMj2BEk/0uANRf2spJur5zdLeqaPvXzFoAzj3WqYcfV53fV9+POI6PmfpIUaOSK/T9K/96OHFn39k6T/qf529Ls3SU9qZLfuc40c21gq6VxJGyTtkfSSpHMGqLfHJb0p6Q2NBGtqn3r7vkZ20d+QtK36W9jvdVfoqyfrjctlgSQ4QAckQdiBJAg7kARhB5Ig7EAShB1IgrADSfwf/KNGxKbWjZUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "print('Counterfactual prediction: {}'.format(explanation_1.cf['class']))\n",
    "print('Closest prototype class: {}'.format(proto_1))\n",
    "plt.imshow(explanation_1.cf['X'].reshape(28, 28));"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Counterfactual prediction: 7\n",
      "Closest prototype class: 7\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAN00lEQVR4nO3dXahc9bnH8d/PN9Bs0djQEGy0tfGmUfJCkAP1FTF4vIgWUepFSVFOchGlhV4cX5CKAZGDbTneFLaoTbXHUmiDG6k5jZtq6EUkMeRoYkiMIbEJ28RGpIkxGpPnXOyVstU9/7Uza97i8/3AZmbWM2vWwyK/rJn5z1p/R4QAfP2d0e8GAPQGYQeSIOxAEoQdSIKwA0mc1cuN2earf6DLIsKTLW90ZLd9s+3ttnfavr/JawHoLrc7zm77TEk7JN0kaa+kDZLuioi3C+twZAe6rBtH9qsk7YyIXRHxmaTfS7q1wesB6KImYb9Y0t8nPN5bLfsC28tsb7S9scG2ADTU9S/oImJY0rDE23ign5oc2fdJmj3h8beqZQAGUJOwb5B0ue3v2D5H0g8ljXSmLQCd1vbb+Ij43Pa9kv5X0pmSnomIrR3rDEBHtT301tbG+MwOdF1XflQD4PRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUii7fnZJcn2bkmHJB2X9HlELOpEUwA6r1HYKzdExD868DoAuoi38UASTcMekv5i+w3byyZ7gu1ltjfa3thwWwAacES0v7J9cUTss/1NSWsl3RcR6wrPb39jAKYkIjzZ8kZH9ojYV90ekLRa0lVNXg9A97QddtvTbJ9/8r6kxZK2dKoxAJ3V5Nv4mZJW2z75Ov8TEWs60hWAjmv0mf2UN8ZndqDruvKZHcDpg7ADSRB2IAnCDiRB2IEkOnEiDAbY/Pnzi/WVK1cW69dff32xPjQ0VKwfO3asZW316tXFdR966KFi/ZNPPinWFyxY0LI2Ojra6LVPRxzZgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJzno7DZx99tnF+nXXXdey9uyzzxbXnTVrVls9nVSd4txSk39fr776arF+zjnnFOvz5s1rWVuyZElx3ddee61YH2Sc9QYkR9iBJAg7kARhB5Ig7EAShB1IgrADSXA++2lg4cKFxfqaNe1fwXtsbKxYf+yxx4r1bdu2FetHjx5tWbviiiuK6954443F+k033VSsHzx4sGVtz549xXW/jjiyA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EASjLMPgLlz5xbrIyMjbb/22rVri/W6a7PXjaOfdVb5n9Bnn33Wsvbee+8V112+fHmxfuGFFxbrTz31VMva7t27i+t+HdUe2W0/Y/uA7S0Tll1ke63td6rb6d1tE0BTU3kb/xtJN39p2f2SRiPickmj1WMAA6w27BGxTtKHX1p8q6RV1f1Vkm7rcF8AOqzdz+wzI+Lkj6rflzSz1RNtL5O0rM3tAOiQxl/QRUSULiQZEcOShiUuOAn0U7tDb/ttz5Kk6vZA51oC0A3thn1E0tLq/lJJL3amHQDdUvs23vYLkq6XNMP2Xkk/l/S4pD/YvkfSHkl3drPJ013dtdWfeOKJYn3GjBnF+iuvvNKyVjdWXTfWfcYZ5ePB7Nmzi/UdO3a0rNWdz16aX30qXn755Ubrf93Uhj0i7mpRKl9ZAMBA4eeyQBKEHUiCsANJEHYgCcIOJMEprj0wPDxcrC9evLhY//jjj4v1559/vmWtbmitbjro48ePF+uloTWpfArsAw88UFy3bsiyblrldevWFevZcGQHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQYZ++BRYsWFesR5Qv4HD58uFgvXS56aGio0WvXqRunX7lyZcvaNddcU1y3br88+uijxTq+iCM7kARhB5Ig7EAShB1IgrADSRB2IAnCDiTBOPtp4PXXXy/WS+eM112u+YMPPijW33333WJ92rRpxfoll1xSrJeMjY0V65s3b277tTPiyA4kQdiBJAg7kARhB5Ig7EAShB1IgrADSTDO3gN1UwcfPXq0WK8b677hhhta1kpj8FL9NetHRkaK9fvuu69YL23/xIkTxXVHR0eL9Y8++qhYxxfVHtltP2P7gO0tE5Y9Ynuf7c3V3y3dbRNAU1N5G/8bSTdPsvxXETG/+vtzZ9sC0Gm1YY+IdZI+7EEvALqoyRd099p+s3qbP73Vk2wvs73R9sYG2wLQULth/7Wk70qaL2lM0i9aPTEihiNiUUSUr7oIoKvaCntE7I+I4xFxQtJTkq7qbFsAOq2tsNueNeHhDyRtafVcAIPBddfmtv2CpOslzZC0X9LPq8fzJYWk3ZKWR0T55OPx1ypv7Gvq3HPPLdbnzZtXrN99993F+v79+1vWLr300uK6R44cKdZ37dpVrNedD3/HHXe0rNWN8V999dXF+vr164v1rCJi0onta39UExF3TbL46cYdAegpfi4LJEHYgSQIO5AEYQeSIOxAErVDbx3dWNKht6bqpkW+7LLLWtbmzJlTXLfuNNO603OffPLJYn3FihUtazt27Ciue+211xbrdcN+WbUaeuPIDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJcCnp08CxY8eK9e3bt7dVm4rp01tecUxS/Vh46XccdVNRM47eWRzZgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJxtlRdMEFFxTrV155ZbF+6NChlrW6c+HRWRzZgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJxtlR9PDDDzdaf82aNS1rmzZtavTaODW1R3bbs23/1fbbtrfa/km1/CLba22/U92Wr3IAoK+m8jb+c0k/i4jvSfo3SStsf0/S/ZJGI+JySaPVYwADqjbsETEWEZuq+4ckbZN0saRbJa2qnrZK0m3dahJAc6f0md32tyUtkPS6pJkRMVaV3pc0s8U6yyQta79FAJ0w5W/jbQ9J+qOkn0bEPyfWYvyqgpNeWTAihiNiUUQsatQpgEamFHbbZ2s86L+LiD9Vi/fbnlXVZ0k60J0WAXRC7dt425b0tKRtEfHLCaURSUslPV7dvtiVDtFVc+fOLdZvv/32Rq//0ksvNVofnTOVz+zfl/QjSW/Z3lwte1DjIf+D7Xsk7ZF0Z3daBNAJtWGPiL9JmnRyd0k3drYdAN3Cz2WBJAg7kARhB5Ig7EAShB1IglNck1u4cGGxfv755xfrpSmZJWnnzp2n3BO6gyM7kARhB5Ig7EAShB1IgrADSRB2IAnCDiTBOHtyc+bMKdbrxtG3bt1arK9fv/6Ue0J3cGQHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQYZ09uyZIljdZ/7rnnOtQJuo0jO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kMZX52WdL+q2kmZJC0nBE/LftRyT9h6QPqqc+GBF/7laj6I6DBw/2uwX0yFR+VPO5pJ9FxCbb50t6w/baqvariHiie+0B6JSpzM8+Jmmsun/I9jZJF3e7MQCddUqf2W1/W9ICSa9Xi+61/abtZ2xPb7HOMtsbbW9s1CmARqYcdttDkv4o6acR8U9Jv5b0XUnzNX7k/8Vk60XEcEQsiohFHegXQJumFHbbZ2s86L+LiD9JUkTsj4jjEXFC0lOSrupemwCaqg27bUt6WtK2iPjlhOWzJjztB5K2dL49AJ0ylW/jvy/pR5Lesr25WvagpLtsz9f4cNxuScu70iG6atWqVcX60NBQsb5hw4ZOtoMumsq38X+T5ElKjKkDpxF+QQckQdiBJAg7kARhB5Ig7EAShB1IwnVT8nZ0Y3bvNoaOOO+884r1Tz/9tFg/fvx4J9vBFETEZEPlHNmBLAg7kARhB5Ig7EAShB1IgrADSRB2IIleT9n8D0l7JjyeUS0bRIPaW0/7OnLkyKk8fVD3mZSnt0tbFXr6o5qvbNzeOKjXphvU3ga1L4ne2tWr3ngbDyRB2IEk+h324T5vv2RQexvUviR6a1dPeuvrZ3YAvdPvIzuAHiHsQBJ9Cbvtm21vt73T9v396KEV27ttv2V7c7/np6vm0Dtge8uEZRfZXmv7nep20jn2+tTbI7b3Vftus+1b+tTbbNt/tf227a22f1It7+u+K/TVk/3W88/sts+UtEPSTZL2Stog6a6IeLunjbRge7ekRRHR9x9g2L5W0mFJv42IK6pl/yXpw4h4vPqPcnpE/OeA9PaIpMP9nsa7mq1o1sRpxiXdJunH6uO+K/R1p3qw3/pxZL9K0s6I2BURn0n6vaRb+9DHwIuIdZI+/NLiWyWdnMZllcb/sfRci94GQkSMRcSm6v4hSSenGe/rviv01RP9CPvFkv4+4fFeDdZ87yHpL7bfsL2s381MYmZEjFX335c0s5/NTKJ2Gu9e+tI04wOz79qZ/rwpvqD7qqsjYqGkf5e0onq7OpBi/DPYII2dTmka716ZZJrxf+nnvmt3+vOm+hH2fZJmT3j8rWrZQIiIfdXtAUmrNXhTUe8/OYNudXugz/38yyBN4z3ZNOMagH3Xz+nP+xH2DZIut/0d2+dI+qGkkT708RW2p1VfnMj2NEmLNXhTUY9IWlrdXyrpxT728gWDMo13q2nG1ed91/fpzyOi53+SbtH4N/LvSnqoHz206OsySf9X/W3td2+SXtD427pjGv9u4x5J35A0KukdSa9IumiAentO0luS3tR4sGb1qberNf4W/U1Jm6u/W/q97wp99WS/8XNZIAm+oAOSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJP4fjglXMA+o8fQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "print('Counterfactual prediction: {}'.format(explanation_2.cf['class']))\n",
    "print('Closest prototype class: {}'.format(proto_2))\n",
    "plt.imshow(explanation_2.cf['X'].reshape(28, 28));"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Speed up the counterfactual search by removing the predict function loss term"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also remove the prediction loss term and still obtain an interpretable counterfactual. This is especially relevant for fully black box models. When we provide the counterfactual search method with a Keras or TensorFlow model, it is incorporated in the TensorFlow graph and evaluated using automatic differentiation. However, if we only have access to the model's prediction function, the gradient updates are numerical and typically require a large number of prediction calls because of the prediction loss term $L_{pred}$. These prediction calls can slow the search down significantly and become a bottleneck. We can represent the gradient of the loss term as follows:\n",
    "\n",
    "\\begin{equation*} \\frac{\\partial L_{pred}}{\\partial x} = \\frac{\\partial L_{pred}}{\\partial p} \\frac{\\partial p}{\\partial x} \\end{equation*}\n",
    "\n",
    "where $L_{pred}$ is the prediction loss term, $p$ the prediction function and $x$ the input features to optimize. For a 28 by 28 MNIST image, the $^{\\delta p}/_{\\delta x}$ term alone would require a prediction call with batch size 28x28x2 = 1568. By using the prototypes to guide the search however, we can remove the prediction loss term and only make a single prediction at the end of each gradient update to check whether the predicted class on the proposed counterfactual is different from the original class. We do not necessarily need a Keras or TensorFlow auto-encoder either and can use k-d trees to find the nearest class prototypes. Please check out [this notebook](./cfproto_housing.ipynb) for a practical example.\n",
    "\n",
    "The first example below removes $L_{pred}$ from the loss function to bypass the bottleneck. It illustrates the drastic speed improvements over the black box alternative with numerical gradient evaluation while still producing interpretable counterfactual instances."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAANz0lEQVR4nO3dfahc9Z3H8c9n0yoxKomrG6K9mqrJH/FhkyWIsLJ2KYobIqaItfljcVE3VRtoMcKKKz6gCyFulQ1CISXSdHGVis+1pHGlrGvA6NVkY3zIgxKtIQ+KYFMwxpjv/nFP5Kp3fnOdOTNnku/7BZeZOd85c74c8sk5c86c83NECMCR7y+abgBAfxB2IAnCDiRB2IEkCDuQxLf6uTDbHPoHeiwiPNb0rrbsti+xvdn2Nts3d/NZAHrLnZ5ntz1B0hZJF0l6X9LLkhZGxBuFediyAz3Wiy37eZK2RcQ7EbFf0sOSLuvi8wD0UDdhP0XSH0e9fr+a9iW2F9ketj3cxbIAdKnnB+giYoWkFRK78UCTutmy75A0NOr1d6ppAAZQN2F/WdIM29+1fZSkH0l6qp62ANSt4934iDhge7Gk30uaIOmBiHi9ts4A1KrjU28dLYzv7EDP9eRHNQAOH4QdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4k0fGQzcB4XHTRRS1rl19+eXHeK664olifMmVKRz1Jkj3mQKdfaDe68cGDB4v1888/v1gfHh4u1nuhq7Db3i5pr6TPJR2IiLl1NAWgfnVs2f8+Ij6s4XMA9BDf2YEkug17SFpj+xXbi8Z6g+1Ftodt9/9LCoAvdLsbf0FE7LD9V5Ketf1WRDw/+g0RsULSCkmyXT7qAaBnutqyR8SO6nGPpMclnVdHUwDq13HYbU+yfdyh55IulrSprsYA1Kub3fipkh6vzld+S9J/RcTqWrpC31x55ZXF+vz584v1efPmFeuTJ09uWWt3rnvbtm3F+sqVK4v1devWtaxt2lTeLi1ZsqRYv/baa4v1dr8hOKzOs0fEO5L+usZeAPQQp96AJAg7kARhB5Ig7EAShB1IgktcjwDLli1rWVu8eHFx3qOPPrpYb3d6bMuWLcX6mjVrWtbuu+++4rzr168v1j/77LNivRsvvfRSsT5jxoxi/dZbb62znVqwZQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJNzulrm1Low71fTE7t27W9ZOOumk4ryPPPJIsX7PPfcU6xs3bizW9+/fX6wPqjPOOKNY37FjR7G+b9++Otv5RiJizB9HsGUHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSS4nv0IsHbt2pa1BQsWFOd95plnivUmbnk8CN5+++2mW6gdW3YgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSILr2Q8DM2fOLNZL91fftWtXcd7Zs2cX63v37i3WMXg6vp7d9gO299jeNGraCbaftb21epxSZ7MA6jee3fhfSbrkK9NulvRcRMyQ9Fz1GsAAaxv2iHhe0kdfmXyZpFXV81WSyr/JBNC4Tn8bPzUidlbPd0ma2uqNthdJWtThcgDUpOsLYSIiSgfeImKFpBUSB+iAJnV66m237WmSVD3uqa8lAL3QadifknRV9fwqSU/W0w6AXmm7G2/7IUnfk3Si7fcl3S5pqaTf2L5G0ruSftjLJrO77rrrivWJEye2rK1evbo4L+fR82gb9ohY2KL0/Zp7AdBD/FwWSIKwA0kQdiAJwg4kQdiBJLiV9GGgdGqtna1bt9bYCQ5nbNmBJAg7kARhB5Ig7EAShB1IgrADSRB2IAluJX0YeO+994r1yZMnt6yde+65xXm3b9/eSUsYYB3fShrAkYGwA0kQdiAJwg4kQdiBJAg7kARhB5LgevYBcOyxxxbrxx9/fLG+efPmjj/77LPPLta7tW3btpa1ffv29XTZ+DK27EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBNezD4CLL764WG837PIgW79+fcva0qVLi/M+/fTTxTrn6cfW8fXsth+wvcf2plHT7rC9w/aG6m9enc0CqN94duN/JemSMabfFxGzq7/f1dsWgLq1DXtEPC/poz70AqCHujlAt9j2xmo3f0qrN9leZHvY9nAXywLQpU7D/gtJZ0iaLWmnpJ+3emNErIiIuRExt8NlAahBR2GPiN0R8XlEHJT0S0nn1dsWgLp1FHbb00a9/IGkTa3eC2AwtD3PbvshSd+TdKKk3ZJur17PlhSStkv6cUTsbLswzrOP6cwzzyzW77///mL9448/blkrXU8+HkNDQ8X6nDlzivWzzjqr42U//PDDxfrVV19drGc9D9/qPHvbm1dExMIxJq/suiMAfcXPZYEkCDuQBGEHkiDsQBKEHUiCS1zRlYkTJxbrs2bNalm77bbbivNeeumlxfrtt99erN91113F+pGKIZuB5Ag7kARhB5Ig7EAShB1IgrADSRB2IAmGbEZXPvnkk2J9165dLWvTp0/vatkffPBBV/Nnw5YdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5LgPDu6cuGFFxbry5cvb1k755xzivO+8MILxfqDDz5YrOPL2LIDSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKcZz8CHHXUUS1r+/fvL857zDHHFOvt7s1+ww03FOuTJk1qWXvxxReL8954443F+t69e4t1fFnbLbvtIdt/sP2G7ddt/7SafoLtZ21vrR6n9L5dAJ0az278AUlLImKWpPMl/cT2LEk3S3ouImZIeq56DWBAtQ17ROyMiFer53slvSnpFEmXSVpVvW2VpAW9ahJA977Rd3bb0yXNkbRO0tSI2FmVdkma2mKeRZIWdd4igDqM+2i87WMlPSrpZxHxp9G1GBkdcsxBGyNiRUTMjYi5XXUKoCvjCrvtb2sk6A9GxGPV5N22p1X1aZL29KZFAHVouxtv25JWSnozIu4dVXpK0lWSllaPT/akwwSOO+64Yn3evHnF+umnn96ydvLJJxfnnT9/frF+2mmnFeuffvppsX7nnXe2rN17770taxKn1uo2nu/sfyvpHyW9ZntDNe0WjYT8N7avkfSupB/2pkUAdWgb9oh4QdKYg7tL+n697QDoFX4uCyRB2IEkCDuQBGEHkiDsQBJc4lqDCRMmFOtLliwp1u++++5i/a233irWZ86c2bJWuvxVkg4ePFisr1u3rli//vrri/UNGzYU6+gftuxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kIRHbjLTp4XZ/VtYHy1btqxYv+mmm3q6/AMHDrSsrV+/vjhvu1tFr169uqOe0JyIGPMqVbbsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AE17PXYO3atcX60NBQsX7qqacW68uXLy/Wn3jiiZa1dvd1Rx5s2YEkCDuQBGEHkiDsQBKEHUiCsANJEHYgibbXs9sekvRrSVMlhaQVEfEftu+Q9M+SPqjeektE/K7NZx2R17MDg6TV9ezjCfs0SdMi4lXbx0l6RdICjYzH/ueI+PfxNkHYgd5rFfbxjM++U9LO6vle229KOqXe9gD02jf6zm57uqQ5kg6NCbTY9kbbD9ie0mKeRbaHbQ931SmAroz7HnS2j5X0P5L+LSIesz1V0oca+R5/l0Z29a9u8xnsxgM91vF3dkmy/W1Jv5X0+4i4d4z6dEm/jYiz23wOYQd6rOMbTtq2pJWS3hwd9OrA3SE/kLSp2yYB9M54jsZfIOl/Jb0m6dD4vrdIWihptkZ247dL+nF1MK/0WWzZgR7raje+LoQd6D3uGw8kR9iBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUii30M2fyjp3VGvT6ymDaJB7W1Q+5LorVN19nZaq0Jfr2f/2sLt4YiY21gDBYPa26D2JdFbp/rVG7vxQBKEHUii6bCvaHj5JYPa26D2JdFbp/rSW6Pf2QH0T9NbdgB9QtiBJBoJu+1LbG+2vc32zU300Irt7bZfs72h6fHpqjH09tjeNGraCbaftb21ehxzjL2GervD9o5q3W2wPa+h3oZs/8H2G7Zft/3Tanqj667QV1/WW9+/s9ueIGmLpIskvS/pZUkLI+KNvjbSgu3tkuZGROM/wLD9d5L+LOnXh4bWsr1M0kcRsbT6j3JKRPzLgPR2h77hMN496q3VMOP/pAbXXZ3Dn3eiiS37eZK2RcQ7EbFf0sOSLmugj4EXEc9L+ugrky+TtKp6vkoj/1j6rkVvAyEidkbEq9XzvZIODTPe6Lor9NUXTYT9FEl/HPX6fQ3WeO8haY3tV2wvarqZMUwdNczWLklTm2xmDG2H8e6nrwwzPjDrrpPhz7vFAbqvuyAi/kbSP0j6SbW7OpBi5DvYIJ07/YWkMzQyBuBOST9vsplqmPFHJf0sIv40utbkuhujr76stybCvkPS0KjX36mmDYSI2FE97pH0uEa+dgyS3YdG0K0e9zTczxciYndEfB4RByX9Ug2uu2qY8UclPRgRj1WTG193Y/XVr/XWRNhfljTD9ndtHyXpR5KeaqCPr7E9qTpwItuTJF2swRuK+ilJV1XPr5L0ZIO9fMmgDOPdaphxNbzuGh/+PCL6/idpnkaOyL8t6V+b6KFFX6dL+r/q7/Wme5P0kEZ26z7TyLGNayT9paTnJG2V9N+SThig3v5TI0N7b9RIsKY11NsFGtlF3yhpQ/U3r+l1V+irL+uNn8sCSXCADkiCsANJEHYgCcIOJEHYgSQIO5AEYQeS+H9dx1zpGkBIMQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.gray()\n",
    "X = x_test[23].reshape(1, 28, 28, 1)\n",
    "plt.imshow(X.reshape(28, 28));"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "c_init = 0.  # weight on prediction loss term set to 0\n",
    "c_steps = 1  # no need to find optimal values for c"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Explanation took 6.271 sec\n"
     ]
    }
   ],
   "source": [
    "# define a black-box model\n",
    "predict_fn = lambda x: cnn.predict(x)\n",
    "\n",
    "# initialize explainer, fit and generate counterfactuals\n",
    "cf = CounterFactualProto(predict_fn, shape, gamma=gamma, theta=theta,\n",
    "                         ae_model=ae, enc_model=enc, max_iterations=max_iterations,\n",
    "                         feature_range=feature_range, c_init=c_init, c_steps=c_steps)\n",
    "cf.fit(x_train)\n",
    "start_time = time()\n",
    "explanation = cf.explain(X, k=1)\n",
    "print('Explanation took {:.3f} sec'.format(time() - start_time))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Counterfactual prediction: 6\n",
      "Closest prototype class: 6\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAPUElEQVR4nO3dXYxUdZrH8d8jiBBGIy1u04FWXMXoZEVRRJPVDTrOBL3RSQyRi42DZvBijE4yF74mGNdNyEaH7IWZpCeSYTazggm+oBl0WJwsmsjEVhRQGZuXJtOdBpYQ5UUEoZ+96ONuq33+p6k6Vafs5/tJOlV1njp1npT+OKfqf079zd0FYOw7o+oGADQHYQeCIOxAEIQdCIKwA0GMb+bGzIyv/oEGc3cbaXlde3YzW2BmfzWzHWb2cD2vBaCxrNZxdjMbJ+lTST+W1CfpXUmL3P3jxDrs2YEGa8SefZ6kHe6+y91PSFol6fY6Xg9AA9UT9umS/jbscV+27BvMbImZdZtZdx3bAlCnhn9B5+5dkrokDuOBKtWzZ++X1Dns8YxsGYAWVE/Y35U0y8wuMrMJku6StLactgCUrebDeHc/aWb3S3pD0jhJK9z9o9I6A1CqmofeatoYn9mBhmvISTUAvj8IOxAEYQeCIOxAEIQdCIKwA0E09Xp24HTMnDkzWe/t7U3Wzzgjf182ODhYQ0ffb+zZgSAIOxAEYQeCIOxAEIQdCIKwA0Ew9IbKjBs3Lllva2tL1vv707+VcvLkydPuaSxjzw4EQdiBIAg7EARhB4Ig7EAQhB0IgrADQTDOHlxHR0ey/vnnnyfrU6ZMSdYvvfTS3FpPT09y3dmzZyfrEydOTNb7+vpya/v27UuuO23atGR9z549yXqR8ePzo9eo8wPYswNBEHYgCMIOBEHYgSAIOxAEYQeCIOxAEIyzj3Fnn312sn7llVcm66mxaik9XixJixYtyq1dccUVyXWvv/76ZL2eGYjNRpzodNSvvXr16mT9mWeeSdY//PDDZL0R6gq7mfVKOizplKST7j63jKYAlK+MPftN7n6ghNcB0EB8ZgeCqDfsLulPZvaemS0Z6QlmtsTMus2su85tAahDvYfxN7h7v5n9naT1Zrbd3TcOf4K7d0nqkiQzq/0bFQB1qWvP7u792e1+SS9JmldGUwDKV3PYzWyymZ399X1JP5G0razGAJSrnsP4dkkvZeOV4yX9p7u/XkpXKM2dd96ZrD/22GPJemdnZ7Je9NvvqXrRWHbR9e5vvPFGsv7VV1/l1orG8IuuKb/66quT9fPPPz9Zr2LK6JrD7u67JKXPyADQMhh6A4Ig7EAQhB0IgrADQRB2IAir5zLB094YZ9A1xDXXXJNbe/XVV5Prtre3J+snTpxI1ot+Unnz5s25teXLl9e8rpQeWity6623JusPPfRQsl6Um1tuuSVZP3XqVLJeD3cf8fpd9uxAEIQdCIKwA0EQdiAIwg4EQdiBIAg7EAQ/JT0GpMbZzzvvvOS6RVMyP/nkk8n6qlWrkvVDhw7l1r744ovkuo20bt26ZP3TTz9N1vv7+5P1Ro6j14o9OxAEYQeCIOxAEIQdCIKwA0EQdiAIwg4EwTj7GLBgwYLc2plnnplct+i67aLr4ffu3Zusf1/t3Lmz6hZKx54dCIKwA0EQdiAIwg4EQdiBIAg7EARhB4Lgd+NbQNE150X11O+rHz16NLnuxRdfnKwfPnw4WUfrqfl3481shZntN7Ntw5a1mdl6M+vJbqeU2SyA8o3mMP53kr59itbDkja4+yxJG7LHAFpYYdjdfaOkg99afLukldn9lZLuKLkvACWr9dz4dncfyO7vlZQ7YZiZLZG0pMbtAChJ3RfCuLunvnhz9y5JXRJf0AFVqnXobZ+ZdUhSdru/vJYANEKtYV8r6e7s/t2SXimnHQCNUngYb2bPS5ovaaqZ9UlaKmmZpBfM7F5JeyQtbGSTY93g4GCyvnBh+u2dNGlSbu2tt95Krlt0vTvGjsKwu/uinNKPSu4FQANxuiwQBGEHgiDsQBCEHQiCsANB8FPSTXDWWWcl6+PGjUvW77nnnpq3/frrryfrc+fOTdY3bdqUrBddQpu6hLpoyBHlYs8OBEHYgSAIOxAEYQeCIOxAEIQdCIKwA0Ewzt4Ex48fT9ZnzJiRrE+dOjVZP3LkSG7t7bffTq5b9FPR5557brJ+6NChZB2tgz07EARhB4Ig7EAQhB0IgrADQRB2IAjCDgTBOHsTFF3PPn/+/GS96Lrvnp6e3Nr06dOT6+7YsSNZnzdvXrJeNOXzO++8k1v78ssvk+uiXOzZgSAIOxAEYQeCIOxAEIQdCIKwA0EQdiAIxtmboOh34YvGqs8555xkPXW9+1NPPZVc97PPPkvWr7322mR9woQJyfrmzZtza8uWLUuu+9prryXrx44dS9bxTYV7djNbYWb7zWzbsGVPmFm/mX2Q/d3W2DYB1Gs0h/G/k7RghOXL3f2q7O+P5bYFoGyFYXf3jZIONqEXAA1Uzxd095vZluwwf0rek8xsiZl1m1l3HdsCUKdaw/4bSRdLukrSgKRn8p7o7l3uPtfd0zMIAmiomsLu7vvc/ZS7D0r6raT0pVEAKldT2M2sY9jDn0ralvdcAK2hcJzdzJ6XNF/SVDPrk7RU0nwzu0qSS+qVdF8De/zeKxpnv+SSS5J1M0vWZ86cmVsrGqvesmVLsr579+5kfc6cOTXXV69enVx31apVyXrRvPVcL/9NhWF390UjLH6uAb0AaCBOlwWCIOxAEIQdCIKwA0EQdiAILnEtwWWXXZasb9++PVm/4IIL6tp+6lLRRx55pK7XLjJp0qRkffny5bm1++5Lj9jedNNNyfqCBSNdn/X/Xn755WQ9GvbsQBCEHQiCsANBEHYgCMIOBEHYgSAIOxAE4+wlKBpHL3Ldddcl60U/ufzss8/Wtf16tLW1Jeupy3u3bUv/DMLEiROT9a1bt9a87VOnTiXXHYvYswNBEHYgCMIOBEHYgSAIOxAEYQeCIOxAEIyzN8HixYvrWn/NmjXJ+uWXX55b6+vrq2vbs2fPTtanTMmd+UuSNG9e/vwh7e3tyXXffPPNZH3nzp3JOr6JPTsQBGEHgiDsQBCEHQiCsANBEHYgCMIOBME4exMcPHgwWX/88ceT9aKpiTs7O3NrPT09yXVTY/SSNH/+/GT9wQcfTNYnTJiQW9u0aVNy3aeffjpZx+kp3LObWaeZ/dnMPjazj8zswWx5m5mtN7Oe7DZ9dgWASo3mMP6kpF+5+w8lXS/pF2b2Q0kPS9rg7rMkbcgeA2hRhWF39wF3fz+7f1jSJ5KmS7pd0srsaSsl3dGoJgHU77Q+s5vZTElzJP1FUru7D2SlvZJGPNHZzJZIWlJ7iwDKMOpv483sB5LWSPqlux8aXnN3l+QjrefuXe4+193n1tUpgLqMKuxmdqaGgv4Hd38xW7zPzDqyeoek/Y1pEUAZCg/jzcwkPSfpE3f/9bDSWkl3S1qW3b7SkA7HgKJLMW+++eZkfWBgIFmfNWtWbm39+vXJdcePT/8vcOGFFybrx48fT9bXrVuXW7vrrruS6x4+fDhZx+kZzWf2f5T0z5K2mtkH2bJHNRTyF8zsXkl7JC1sTIsAylAYdnd/W5LllH9UbjsAGoXTZYEgCDsQBGEHgiDsQBCEHQiCS1xHafLkybm1o0ePJte98cYbk/UZM2Yk6w888ECyPnQC48iGTpPIVzR1cXd3d7K+YsWKZL2rqyu3Njg4mFwX5WLPDgRB2IEgCDsQBGEHgiDsQBCEHQiCsANBMM4+Sqmx9La2tuS6Z5yR/jd148aNyfpFF12UrKfG6Xfv3p1cd+nSpcn69u3bk/Xe3t5kHa2DPTsQBGEHgiDsQBCEHQiCsANBEHYgCMIOBGGpa6FL35hZ8zbWRIsXL65r/WnTpiXru3btStZfeOGF3FrRf9+icwC45vz7x91H/BED9uxAEIQdCIKwA0EQdiAIwg4EQdiBIAg7EEThOLuZdUr6vaR2SS6py93/3cyekPRzSf+TPfVRd/9jwWuNyXH2qVOnJusHDhxo6PYnTZqUWzt27FhDt43WkzfOPpofrzgp6Vfu/r6ZnS3pPTNbn9WWu/vTZTUJoHFGMz/7gKSB7P5hM/tE0vRGNwagXKf1md3MZkqaI+kv2aL7zWyLma0wsyk56ywxs24zS88jBKChRn1uvJn9QNJ/S/pXd3/RzNolHdDQ5/h/kdTh7vcUvAaf2RuAz+wYrq5z483sTElrJP3B3V/MXnCfu59y90FJv5U0r6xmAZSvMOw2NA3oc5I+cfdfD1veMexpP5W0rfz2AJRlNENvN0h6S9JWSV9f7/iopEWSrtLQYXyvpPuyL/NSrzUmD+OLNPoy0tTrc4lqPHmH8VzP3gSEHc3E9exAcIQdCIKwA0EQdiAIwg4EQdiBIBh6A8YYht6A4Ag7EARhB4Ig7EAQhB0IgrADQRB2IIjR/LpsmQ5I2jPs8dRsWStq1d5atS+J3mpVZm8X5hWaelLNdzZu1u3ucytrIKFVe2vVviR6q1WzeuMwHgiCsANBVB32roq3n9KqvbVqXxK91aopvVX6mR1A81S9ZwfQJIQdCKKSsJvZAjP7q5ntMLOHq+ghj5n1mtlWM/ug6vnpsjn09pvZtmHL2sxsvZn1ZLcjzrFXUW9PmFl/9t59YGa3VdRbp5n92cw+NrOPzOzBbHml712ir6a8b03/zG5m4yR9KunHkvokvStpkbt/3NRGcphZr6S57l75CRhm9k+Sjkj6vbv/Q7bs3yQddPdl2T+UU9z9oRbp7QlJR6qexjubrahj+DTjku6Q9DNV+N4l+lqoJrxvVezZ50na4e673P2EpFWSbq+gj5bn7hslHfzW4tslrczur9TQ/yxNl9NbS3D3AXd/P7t/WNLX04xX+t4l+mqKKsI+XdLfhj3uU2vN9+6S/mRm75nZkqqbGUH7sGm29kpqr7KZERRO491M35pmvGXeu1qmP68XX9B91w3ufrWkWyX9IjtcbUk+9BmslcZOfyPpYg3NATgg6Zkqm8mmGV8j6Zfufmh4rcr3boS+mvK+VRH2fkmdwx7PyJa1BHfvz273S3pJrTcV9b6vZ9DNbvdX3M//aaVpvEeaZlwt8N5VOf15FWF/V9IsM7vIzCZIukvS2gr6+A4zm5x9cSIzmyzpJ2q9qajXSro7u3+3pFcq7OUbWmUa77xpxlXxe1f59Ofu3vQ/Sbdp6Bv5nZIeq6KHnL7+XtKH2d9HVfcm6XkNHdZ9paHvNu6VdJ6kDZJ6JP2XpLYW6u0/NDS19xYNBaujot5u0NAh+hZJH2R/t1X93iX6asr7xumyQBB8QQcEQdiBIAg7EARhB4Ig7EAQhB0IgrADQfwvcefR/xr4q5oAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "print('Counterfactual prediction: {}'.format(explanation.cf['class']))\n",
    "print('Closest prototype class: {}'.format(explanation.id_proto))\n",
    "plt.imshow(explanation.cf['X'].reshape(28, 28));"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us know add the $L_{pred}$ loss term back in the objective function and observe how long it takes to generate a black box counterfactual:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "c_init = 1.\n",
    "c_steps = 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Explanation took 545.836 sec\n"
     ]
    }
   ],
   "source": [
    "# define a black-box model\n",
    "predict_fn = lambda x: cnn.predict(x)\n",
    "\n",
    "# initialize explainer, fit and generate counterfactuals\n",
    "cf = CounterFactualProto(predict_fn, shape, gamma=gamma, theta=theta,\n",
    "                         ae_model=ae, enc_model=enc, max_iterations=max_iterations,\n",
    "                         feature_range=feature_range, c_init=c_init, c_steps=c_steps)\n",
    "cf.fit(x_train)\n",
    "start_time = time()\n",
    "explanation = cf.explain(X, k=1)\n",
    "print('Explanation took {:.3f} sec'.format(time() - start_time))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Counterfactual prediction: 6\n",
      "Closest prototype class: 6\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAO30lEQVR4nO3db4xUZZbH8d9ZhMhfhSESZIjMYhOD658ZCZisWdnATBRNkDdmMDGYGBkjxpk4LzTuCwwqIZudwTXRSZqoMJtZJmNmUCQThSGTKC8ktMAiooAiBjpAixrsEfnTcPZFX0yjdZ9qqm7VreZ8P0mnq+6pp+tY+vPeqqfufczdBeDi909lNwCgOQg7EARhB4Ig7EAQhB0I4pJmPpmZ8dE/0GDubpW217VnN7PbzGy3mX1kZo/X87cANJbVOs9uZoMk7ZH0U0kHJW2RNN/ddyXGsGcHGqwRe/bpkj5y933ufkrSHyXNrePvAWigesI+QdKBPvcPZtvOY2YLzazDzDrqeC4AdWr4B3Tu3i6pXeIwHihTPXv2TkkT+9z/YbYNQAuqJ+xbJLWZ2Y/MbIikn0taW0xbAIpW82G8u/eY2cOS3pQ0SNJL7v5+YZ0BKFTNU281PRnv2YGGa8iXagAMHIQdCIKwA0EQdiAIwg4EQdiBIJp6PjtQpBEjRiTrgwcPzq19+eWXRbfT8tizA0EQdiAIwg4EQdiBIAg7EARhB4Jg6g0t6+qrr07Wz549m6zv27evyHYGPPbsQBCEHQiCsANBEHYgCMIOBEHYgSAIOxAE8+xoWd3d3cn6jBkzkvWhQ4fm1nbtyl1/VJLUzKsuNwt7diAIwg4EQdiBIAg7EARhB4Ig7EAQhB0Ignl2JE2YMCFZX7p0abJ+/Pjx3FpbW1ty7KxZs5L1alJz5WYVFzrt11hJev7555P1VatWJesdHR3JeiPUFXYz2y+pW9IZST3uPq2IpgAUr4g9+7+7+9EC/g6ABuI9OxBEvWF3SevN7F0zW1jpAWa20Mw6zKz5b1IAfKvew/hb3L3TzK6QtMHMPnT3t/o+wN3bJbVLkpldfGcXAANEXXt2d+/MfndJWiNpehFNAShezWE3s+FmNvLcbUk/k7SzqMYAFKuew/hxktZk85WXSPpfd3+jkK7QNI888kiyvmjRomR97NixyfqoUaNya4MGDUqO3bt3b7L+6quvJuubN2+uqS9Juu+++5L1e+65J1lfuXJlsl6GmsPu7vsk3VBgLwAaiKk3IAjCDgRB2IEgCDsQBGEHgrBmXjKXb9A1xvXXX59be+aZZ5Jj58yZk6xXOxV0z549yfq2bdtya8uXL695rCSdPn06WR85cmRurdplqteuXVvz35ak2bNnJ+tnzpxJ1uvh7hX/pbFnB4Ig7EAQhB0IgrADQRB2IAjCDgRB2IEgmGe/CNx77725tRUrViTHbt26NVl/4430WcvLli1L1k+dOpWst6rJkycn652dncn6iRMnimzngjDPDgRH2IEgCDsQBGEHgiDsQBCEHQiCsANBsGTzRWDevHm5tSFDhiTHvvnmm8n6kiVLauppoPv444/LbqFw7NmBIAg7EARhB4Ig7EAQhB0IgrADQRB2IAjOZx8ApkyZkqynrq8+bNiw5NjrrrsuWd+5c2eyjtZT8/nsZvaSmXWZ2c4+28aY2QYz25v9Hl1kswCK15/D+JWSbvvOtsclbXT3Nkkbs/sAWljVsLv7W5K++M7muZJWZbdXSbqr4L4AFKzW78aPc/dD2e3DksblPdDMFkpaWOPzAChI3SfCuLunPnhz93ZJ7RIf0AFlqnXq7YiZjZek7HdXcS0BaIRaw75W0oLs9gJJrxXTDoBGqXoYb2arJc2UNNbMDkpaLGmZpD+Z2f2SPpV0dyObjO7BBx9M1ocOHZpbW7duXXLs0aNHa+oJA0/VsLv7/JzSrIJ7AdBAfF0WCIKwA0EQdiAIwg4EQdiBILiU9ADQ1taWrJtVPKNRkrRx48bk2GqnwNYrdSnrgbqc80DFnh0IgrADQRB2IAjCDgRB2IEgCDsQBGEHgmCevQWk5skl6YYbbkjWu7u7c2ubNm2qeWwRmEtvHezZgSAIOxAEYQeCIOxAEIQdCIKwA0EQdiAI5tlbwIIFC5L1UaNGJeu7d+/OrfX09CTHfvbZZ8n6jBkzkvWpU6cm66tXr86tnThxIjkWxWLPDgRB2IEgCDsQBGEHgiDsQBCEHQiCsANBmLs378nMmvdkA8j69euT9dmzZyfrmzdvzq2NGTMmOfbSSy9N1idOnJisV7Nt27bc2rJly5JjX3/99WSdefrK3L3iBRKq7tnN7CUz6zKznX22PWlmnWa2PfuZU2SzAIrXn8P4lZJuq7B9ubvfmP38tdi2ABStatjd/S1JXzShFwANVM8HdA+b2Y7sMH903oPMbKGZdZhZRx3PBaBOtYb9d5ImS7pR0iFJv8l7oLu3u/s0d59W43MBKEBNYXf3I+5+xt3PSlohaXqxbQEoWk1hN7Pxfe7Ok7Qz77EAWkPVeXYzWy1ppqSxko5IWpzdv1GSS9ov6RfufqjqkzHPXtE333yTrA8aNChZT52zfvjw4eTYTz75JFk/evRosl7tfPZrr702WU95+umnk/WlS5cm61Hn4fPm2atevMLd51fY/GLdHQFoKr4uCwRB2IEgCDsQBGEHgiDsQBBcSroJbr/99mT92LFjyfrll1+erK9Zsya3tmTJkuTY1GWo+6PaKbLPPfdcbu2BBx5Ijr3jjjuS9fb29mT94MGDyXo07NmBIAg7EARhB4Ig7EAQhB0IgrADQRB2IAjm2Qtw5ZVXJuvXXHNNsm5W8YzEb73wwgvJ+qOPPpqsN1K100iHDh2aW6v2/YIPP/wwWT958mSyjvOxZweCIOxAEIQdCIKwA0EQdiAIwg4EQdiBIJhnL8BXX32VrHd2dibrn3/+ebJe5jz6JZek/xO59dZbk/WZM2fm1kaNGpUc29GRXjHs66+/TtZxPvbsQBCEHQiCsANBEHYgCMIOBEHYgSAIOxAE8+wFmDRpUrK+ffv2ZP2VV16p6/mHDBmSWzt16lRy7LBhw5L1xYsXJ+sPPfRQsj58+PDc2jvvvJMc+/bbbyfrx48fT9Zxvqp7djObaGZ/N7NdZva+mf0y2z7GzDaY2d7s9+jGtwugVv05jO+R9Gt3nyrpZkmLzGyqpMclbXT3Nkkbs/sAWlTVsLv7IXffmt3ulvSBpAmS5kpalT1slaS7GtUkgPpd0Ht2M5sk6ceSNksa5+6HstJhSeNyxiyUtLD2FgEUod+fxpvZCEl/lvQrdz/vzA93d0leaZy7t7v7NHefVlenAOrSr7Cb2WD1Bv0P7v6XbPMRMxuf1cdL6mpMiwCKUPUw3nqvc/yipA/c/bd9SmslLZC0LPv9WkM6HACqLQ382GOPJevVliZOTV9J0ujR+RMh1aan7rzzzmS92rRitUtJb926Nbc2b9685NgjR44k67gw/XnP/q+S7pX0npmdmzB+Qr0h/5OZ3S/pU0l3N6ZFAEWoGnZ33yQpbxWDWcW2A6BR+LosEARhB4Ig7EAQhB0IgrADQVjvl9+a9GRmzXuyC3TFFVck611d+d8Zuvnmm5NjX3755WS92iWRU6ewStKUKVNqHnv27Nlkfc+ePcl6tX+2Z599Nrd2+vTp5FjUxt0rzp6xZweCIOxAEIQdCIKwA0EQdiAIwg4EQdiBILiUdCY1j15NtfPRN23alKwPHjw4Wb/pppuS9d5LDlS2ZcuW5NinnnoqWV+3bl2yjoGDPTsQBGEHgiDsQBCEHQiCsANBEHYgCMIOBMH57AWYO3dusn7s2LFkvaenJ1k/cOBAsn748OHc2smTJ5NjcfHhfHYgOMIOBEHYgSAIOxAEYQeCIOxAEIQdCKLqPLuZTZT0e0njJLmkdnf/bzN7UtIDkj7LHvqEu/+1yt8asPPsV111VW7tsssuS47dsWNH0e0AufLm2ftz8YoeSb92961mNlLSu2a2Iastd/f/KqpJAI3Tn/XZD0k6lN3uNrMPJE1odGMAinVB79nNbJKkH0vanG162Mx2mNlLZjY6Z8xCM+sws466OgVQl36H3cxGSPqzpF+5+1eSfidpsqQb1bvn/02lce7e7u7T3H1aAf0CqFG/wm5mg9Ub9D+4+18kyd2PuPsZdz8raYWk6Y1rE0C9qobdei9d+qKkD9z9t322j+/zsHmSdhbfHoCi9Gfq7RZJb0t6T9K59X2fkDRfvYfwLmm/pF9kH+al/taAnXoDBoq8qTfOZwcuMpzPDgRH2IEgCDsQBGEHgiDsQBCEHQiCsANBEHYgCMIOBEHYgSAIOxAEYQeCIOxAEIQdCKI/V5ct0lFJn/a5Pzbb1opatbdW7Uuit1oV2VvuNc+bej77957crKNVr03Xqr21al8SvdWqWb1xGA8EQdiBIMoOe3vJz5/Sqr21al8SvdWqKb2V+p4dQPOUvWcH0CSEHQiilLCb2W1mttvMPjKzx8voIY+Z7Tez98xse9nr02Vr6HWZ2c4+28aY2QYz25v9rrjGXkm9PWlmndlrt93M5pTU20Qz+7uZ7TKz983sl9n2Ul+7RF9Ned2a/p7dzAZJ2iPpp5IOStoiab6772pqIznMbL+kae5e+hcwzOzfJP1D0u/d/V+ybf8p6Qt3X5b9j3K0uz/WIr09KekfZS/jna1WNL7vMuOS7pJ0n0p87RJ93a0mvG5l7NmnS/rI3fe5+ylJf5Q0t4Q+Wp67vyXpi+9snitpVXZ7lXr/Y2m6nN5agrsfcvet2e1uSeeWGS/1tUv01RRlhH2CpAN97h9Ua6337pLWm9m7Zraw7GYqGNdnma3DksaV2UwFVZfxbqbvLDPeMq9dLcuf14sP6L7vFnf/iaTbJS3KDldbkve+B2uludN+LePdLBWWGf9Wma9drcuf16uMsHdKmtjn/g+zbS3B3Tuz312S1qj1lqI+cm4F3ex3V8n9fKuVlvGutMy4WuC1K3P58zLCvkVSm5n9yMyGSPq5pLUl9PE9ZjY8++BEZjZc0s/UektRr5W0ILu9QNJrJfZynlZZxjtvmXGV/NqVvvy5uzf9R9Ic9X4i/7Gk/yijh5y+/lnS/2U/75fdm6TV6j2sO63ezzbul/QDSRsl7ZX0N0ljWqi3/1Hv0t471Bus8SX1dot6D9F3SNqe/cwp+7VL9NWU142vywJB8AEdEARhB4Ig7EAQhB0IgrADQRB2IAjCDgTx/yeCuonrRqTZAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "print('Counterfactual prediction: {}'.format(explanation.cf['class']))\n",
    "print('Closest prototype class: {}'.format(explanation.id_proto))\n",
    "plt.imshow(explanation.cf['X'].reshape(28, 28));"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Clean up:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "os.remove('mnist_cnn.h5')\n",
    "os.remove('mnist_ae.h5')\n",
    "os.remove('mnist_enc.h5')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}