examples/text_input_word_level_explanation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "6bb18fec",
   "metadata": {},
   "source": [
    "In this demo, we compute word-level explanations for the Huggingface [`textattack/roberta-base-SST-2`](https://huggingface.co/textattack/roberta-base-SST-2) model, also the setup presented in the paper.\n",
    "\n",
    "We first load required packages and the RoBERTa model. Two classes are needed to compute the explanations. `BeamSearchExplainer` implements the beam search algorithm, and `*Masker` implements the feature masking. In this demo, we use `TextWordMasker` since we need to mask out individual words from a text input. The other demos showcase other `*Maskers`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "aa183112",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Some weights of the model checkpoint at textattack/roberta-base-SST-2 were not used when initializing RobertaForSequenceClassification: ['roberta.pooler.dense.bias', 'roberta.pooler.dense.weight']\n",
      "- This IS expected if you are initializing RobertaForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
      "- This IS NOT expected if you are initializing RobertaForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n"
     ]
    }
   ],
   "source": [
    "from solvex import BeamSearchExplainer, TextWordMasker\n",
    "import torch\n",
    "from transformers import AutoTokenizer, AutoModelForSequenceClassification\n",
    "\n",
    "device = 'cuda' if torch.cuda.is_available() else 'cpu'\n",
    "name = 'textattack/roberta-base-SST-2'\n",
    "tokenizer = AutoTokenizer.from_pretrained(name)\n",
    "model = AutoModelForSequenceClassification.from_pretrained(name).to(device).eval()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "858d5fab",
   "metadata": {},
   "source": [
    "The explainer expects the function to be explained in a particular format. Specifically, it takes in a list of `N` (full or masked) inputs, and returns a `numpy` array of shape `N x C` where `C` is the number of classes. The values of the array can be anything, but most commonly the class probability, which is what we are going to do here. In addition, when masking features (i.e., words) from a piece of text, `TextWordMasker` expects the text to be a pre-tokenized list of words and returns another list of words. Thus, `sentences` is a list in which each element is a list of words and the function below needs to `join` each sentence back to a string."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "275f1456",
   "metadata": {},
   "outputs": [],
   "source": [
    "def model_func(sentences):\n",
    "    sentences = [' '.join(s) for s in sentences]\n",
    "    tok = tokenizer(sentences, return_tensors='pt', padding=True).to(device)\n",
    "    with torch.no_grad():\n",
    "        logits = model(**tok)['logits']\n",
    "    probs = torch.nn.functional.softmax(logits, dim=-1).cpu().numpy()\n",
    "    return probs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0edd223e",
   "metadata": {},
   "source": [
    "Now we are ready to explain! We instantiate the explainer, prepare the input sentence (as a list of words), and call the `explain_instance` function. The `suppression` argument passed to `TextWordMasker` tells the masker how to mask out a word. In this case, we simply delete it. The label argument to `explain_instance` specifies which `label` we want to generate the explanation for. In our case, we want to explain the positive class, which is label `1`. If it is not specified, the label with the highest function value will be used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "1c28356c",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|█████████████████████████████████| 11/11 [00:00<00:00, 25.30it/s]\n"
     ]
    }
   ],
   "source": [
    "sentence = 'A triumph , relentless and beautiful in its downbeat darkness .'.split(' ')\n",
    "masker = TextWordMasker(suppression='remove')\n",
    "explainer = BeamSearchExplainer(masker, f=model_func, beam_size=50, batch_size=50)\n",
    "e = explainer.explain_instance(sentence, label=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "661c66ed",
   "metadata": {},
   "source": [
    "The explanation `e` we get is a dictionary of keys `'exp'`, `'label'` and `'func_val'`, of type `list`, `int` and `float` respectively, as printed out below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "31ebd9ab",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'exp': [1.5, 8.5, 3.5, 2.5, 4.5, 9.5, 5.5, 7.5, 6.5, -0.5, 0.5], 'label': 1, 'func_val': 0.9996933}\n"
     ]
    }
   ],
   "source": [
    "print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "39a675ce",
   "metadata": {},
   "source": [
    "Even better, all built-in `*Masker` classes include more user-friendly explanation displays, and the `TextWordMasker` class has three. They can be called with `masker.render_result`, using different `mode` parameters. The first one is console printing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "d970a4d7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input: A triumph , relentless and beautiful in its downbeat darkness .\n",
      "Explained label: 1\n",
      "Function value for label 1: 1.000\n",
      "Word feature attribution:\n",
      "+------------+------------+\n",
      "| Word       |   Attr val |\n",
      "|------------+------------|\n",
      "| A          |        1.5 |\n",
      "| triumph    |        8.5 |\n",
      "| ,          |        3.5 |\n",
      "| relentless |        2.5 |\n",
      "| and        |        4.5 |\n",
      "| beautiful  |        9.5 |\n",
      "| in         |        5.5 |\n",
      "| its        |        7.5 |\n",
      "| downbeat   |        6.5 |\n",
      "| darkness   |       -0.5 |\n",
      "| .          |        0.5 |\n",
      "+------------+------------+\n"
     ]
    }
   ],
   "source": [
    "masker.render_result(sentence, e, mode='text', execute=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e8ae1f23",
   "metadata": {},
   "source": [
    "The second one is color rendering."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "d1e9f668",
   "metadata": {},
   "outputs": [],
   "source": [
    "masker.render_result(sentence, e, mode='color', execute=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "94186492",
   "metadata": {},
   "source": [
    "It writes an HTML snippet to a file named `explanation.html`, which is rendered below.\n",
    "<div style=\"border: solid; border-width: 1px; margin-top: 5px; margin-bottom: 5px; padding-left: 10px; padding-right: 10px; padding-bottom: 5px;\">Explained label: 1<br>Function value for label 1: 1.000<br><pre style=\"white-space: pre-wrap; margin: 0px;\"><span style=\"background-color: rgb(222, 204, 200);\">A</span> <span style=\"background-color: rgb(230, 129, 106);\">triumph</span> <span style=\"background-color: rgb(225, 183, 173);\">,</span> <span style=\"background-color: rgb(223, 194, 187);\">relentless</span> <span style=\"background-color: rgb(226, 172, 160);\">and</span> <span style=\"background-color: rgb(232, 119, 93);\">beautiful</span> <span style=\"background-color: rgb(227, 161, 146);\">in</span> <span style=\"background-color: rgb(229, 139, 119);\">its</span> <span style=\"background-color: rgb(228, 151, 133);\">downbeat</span> <span style=\"background-color: rgb(215, 217, 222);\">darkness</span> <span style=\"background-color: rgb(221, 215, 214);\">.</span> </pre></div>\n",
    "The last one is plotting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "0b5ecaf3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAArIAAAFUCAYAAADYjN+CAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAPYQAAD2EBqD+naQAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd1xT1/8/8FdACUOmgoIioKKIE8U9QByoKKK1aKUWUOvHVQetrVoFcYDaanFinTjaqnWPilpnnbirVVAEEQcuZAiKGu7vD3/ka0yAhCaE0Nfz8chDcnJy87pJkHdOzj1XJAiCACIiIiIiHaOn7QBERERERCXBQpaIiIiIdBILWSIiIiLSSSxkiYiIiEgnsZAlIiIiIp3EQpaIiIiIdBILWSIiIiLSSSxkiYiIiEgnsZAlIiIiIp3EQpaojAoKCoKjo2OJ7uvo6IigoCC15lGWsrk9PT3h6emp8vbv3r0LkUiEH3/8UfVwhTh27BhEIhGOHTumtm3+F5Wn57Gk708iKl0sZImKEBMTA5FIVOjl7Nmz2o5IZUBCQgImTJiAtm3bwtDQECKRCHfv3v1X25w+fXqh77vly5erJ3gJLVu2DDExMVrN8F/z8uVLhIWFoXv37rCysoJIJFL5NcjIyMDw4cNhbW0NExMTdOrUCZcuXVLYd/fu3WjWrBkMDQ1Rs2ZNhIWF4d27d/9qm0SaUEHbAYh0wYwZM+Dk5CTXXqdOHS2kKV5CQgL09Pg5tbScOXMGixYtgqurK+rXr48rV66obdvR0dGoVKmSTFurVq3Utv2SWLZsGapUqSI36t+xY0e8evUKBgYG2glWjj179gwzZsxAzZo10aRJE5VHvfPz8+Hj44OrV69i4sSJqFKlCpYtWwZPT09cvHgRzs7O0r779++Hn58fPD09sXjxYly7dg2zZs3CkydPEB0dXaJtEmkKC1kiJfTo0QPu7u7ajqE0sVis7Qj/Kb6+vsjIyICpqSl+/PFHtRay/fv3R5UqVdS2PU3S09ODoaGhtmOUS7a2tnj06BGqVauGCxcuoEWLFirdf+vWrTh9+jR+//139O/fHwDg7++PunXrIiwsDL/++qu07zfffIPGjRvj4MGDqFDhfZlgZmaGiIgIjBs3Di4uLipvk0hTOGRDpAZhYWHQ09PD4cOHZdqHDx8OAwMDXL16FcD/zSHcvHkzpkyZgmrVqsHExAS+vr5ITU0t9nF+/PFHtG3bFpUrV4aRkRGaN2+OrVu3yvX7eI5swRSJU6dOISQkRPo1YN++ffH06VO5++/fvx8dOnSAiYkJTE1N4ePjg3/++Ueu386dO9GwYUMYGhqiYcOG2LFjR7H7UJg3b94gNDQUzZs3h7m5OUxMTNChQwccPXq00Pv89NNPcHBwgJGRETw8PHD9+nW5PvHx8ejfvz+srKxgaGgId3d37N69u9g8ubm5iI+Px7Nnz4rta2VlBVNT02L7qVPBXGFFXy+LRCJMnz5der1gmkJiYiKCgoJgYWEBc3NzBAcHIzc3V+7+GzduRMuWLWFsbAxLS0t07NgRBw8eBPD+vfXPP//g+PHj0qkOBXNJC5sj+/vvv6N58+YwMjJClSpV8Pnnn+PBgwcyfYKCglCpUiU8ePAAfn5+qFSpEqytrfHNN99AIpEU+Vz06tULtWrVUnhbmzZtZD6Erl27Fl5eXrCxsYFYLIarq6vMKGNhCn6HPp4yUtg+nzt3Dt27d4e5uTmMjY3h4eGBU6dOyW03Pj4e9+7dK/bxxWIxqlWrVmy/wmzduhVVq1ZFv379pG3W1tbw9/fHrl27kJeXBwC4ceMGbty4geHDh0uLWAAYNWoUBEGQ+f9G2W0SaRILWSIlZGZm4tmzZzKX58+fS2+fOnUqmjZtiqFDhyI7OxsAcODAAaxcuRKhoaFo0qSJzPZmz56Nffv24bvvvsPYsWNx6NAhdOnSBa9evSoyx8KFC+Hm5oYZM2YgIiICFSpUwKeffop9+/YptR9fffUVrl69irCwMIwcORJ79uzBmDFjZPps2LABPj4+qFSpEubOnYtp06bhxo0baN++vcwf8YMHD+KTTz6BSCRCZGQk/Pz8EBwcjAsXLiiV5WNZWVlYtWoVPD09MXfuXEyfPh1Pnz6Ft7e3whHO9evXY9GiRRg9ejQmT56M69evw8vLC48fP5b2+eeff9C6dWvcvHkTkyZNwvz582FiYgI/P79ii+64uDjUr18fS5YsKdH+qEt6errM++7Fixcl3pa/vz+ys7MRGRkJf39/xMTEIDw8XKZPeHg4Bg8ejIoVK2LGjBkIDw+Hvb09jhw5AgCIiopCjRo14OLigg0bNmDDhg34/vvvC33MmJgY+Pv7Q19fH5GRkfjyyy+xfft2tG/fHhkZGTJ9JRIJvL29UblyZfz444/w8PDA/PnzsWLFiiL3a8CAAUhOTsb58+dl2lNSUnD27FkMHDhQ2hYdHQ0HBwdMmTIF8+fPh729PUaNGoWlS5cq9Rwq48iRI+jYsSOysrIQFhaGiIgIZGRkwMvLC3FxcTJ969evjy+++EJtj12Yy5cvo1mzZnJTjlq2bInc3FzcunVL2g+A3DdQdnZ2qFGjhvR2VbZJpFECERVq7dq1AgCFF7FYLNP32rVrgoGBgTBs2DDhxYsXQvXq1QV3d3fh7du30j5Hjx4VAAjVq1cXsrKypO1btmwRAAgLFy6UtgUGBgoODg4yj5Gbmytz/c2bN0LDhg0FLy8vmXYHBwchMDBQbj+6dOki5OfnS9snTJgg6OvrCxkZGYIgCEJ2drZgYWEhfPnllzLbS0tLE8zNzWXamzZtKtja2krvKwiCcPDgQQGAXG5FPDw8BA8PD+n1d+/eCXl5eTJ9Xrx4IVStWlUYMmSItC05OVkAIBgZGQn379+Xtp87d04AIEyYMEHa1rlzZ6FRo0bC69evpW35+flC27ZtBWdnZ2lbwety9OhRubawsLBi9+VDP/zwgwBASE5OVul+HwsLC1P4vit4bgueh7Vr18rd9+PcBdv68HkUBEHo27evULlyZen127dvC3p6ekLfvn0FiUQi0/fD902DBg1kXrsCHz+Pb968EWxsbISGDRsKr169kvbbu3evAEAIDQ2VtgUGBgoAhBkzZshs083NTWjevLnC56hAZmamIBaLha+//lqmfd68eYJIJBJSUlKkbR//DgmCIHh7ewu1atWSafv4/VnwO/Tx6/rxPufn5wvOzs6Ct7e3zHOWm5srODk5CV27dpW5PwCFz2VRzp8/X+hrXxgTExO5118QBGHfvn0CACE2NlYQhP97/967d0+ub4sWLYTWrVurvE0iTeKILJESli5dikOHDslc9u/fL9OnYcOGCA8Px6pVq+Dt7Y1nz55h3bp1Ml/PFfjiiy9kvoru378/bG1t8ccffxSZw8jISPrzixcvkJmZiQ4dOih9lPDw4cMhEomk1zt06ACJRIKUlBQAwKFDh5CRkYHPPvtMZhRQX18frVq1kn7N/+jRI1y5cgWBgYEwNzeXbq9r165wdXVVKsvH9PX1pQcJ5efnIz09He/evYO7u7vC/fPz80P16tWl11u2bIlWrVpJn8P09HQcOXJEOgr54Ui6t7c3bt++Lff19oc8PT0hCILMV/TasG3bNpn33S+//FLibY0YMULmeocOHfD8+XNkZWUBeD9VJD8/H6GhoXKjbB++b5R14cIFPHnyBKNGjZKZO+vj4wMXFxeF3yQoypiUlFTk45iZmaFHjx7YsmULBEGQtm/evBmtW7dGzZo1pW0f/g4VfNPi4eGBpKQkZGZmqryPH7ty5Qpu376NQYMG4fnz59L3XU5ODjp37owTJ04gPz9f2l8QhFJZruzVq1cK584XvC4F3wYV/FtY3w+/NVJ2m0SaxIO9iJTQsmVLpQ72mjhxIjZt2oS4uDhEREQUWtR9fDSvSCRCnTp1il2yae/evZg1axauXLkiM/9M2SLjwz/oAGBpaQkA0q+rb9++DQDw8vJSeH8zMzMAkBa+io5KrlevXomX31m3bh3mz5+P+Ph4vH37VtquaMUIRY9dt25dbNmyBQCQmJgIQRAwbdo0TJs2TeHjPXnyRKYYLos6duyotoO9inr9zczMcOfOHejp6ZX4w8jHCt4n9erVk7vNxcUFJ0+elGkzNDSEtbW1XEZlplMMGDAAO3fuxJkzZ9C2bVvcuXMHFy9eRFRUlEy/U6dOISwsDGfOnJGbH5yZmSnzwawkCn6HAgMDC+2TmZkpfe5Li5GRkcI5q69fv5be/uG/hfX98IOAstsk0iQWskRqlJSUJP1Ddu3aNbVu+6+//oKvry86duyIZcuWwdbWFhUrVsTatWuVPjpYX19fYXvBKFbBSNGGDRsUHliiaHRZXTZu3IigoCD4+flh4sSJsLGxkc6rvHPnjsrbK9iXb775Bt7e3gr7lNXl05RR2IeXog6MKu7117bC8imjd+/eMDY2xpYtW9C2bVts2bIFenp6+PTTT6V97ty5g86dO8PFxQULFiyAvb09DAwM8Mcff+Cnn36SGSn9mLLPd8E2fvjhBzRt2lThfT5eTq00FKx68LGCNjs7O2m/gnZ7e3u5vi1btlR5m0SaxEKWSE3y8/MRFBQEMzMzjB8/HhEREejfv7/MEb0FCordAoIgIDExEY0bNy50+9u2bYOhoSEOHDgg83Xe2rVr1bYPtWvXBgDY2NigS5cuhfZzcHAAIL8fwPs1bEti69atqFWrFrZv3y5TNISFhSnsr+ixb926JT2rWMFR7BUrVixyX3RVwYjexwdMFYyClkTt2rWRn5+PGzduFFqEAcp/A1DwPklISJAb5U9ISJDerg4mJibo1asXfv/9dyxYsACbN29Ghw4dZIqpPXv2IC8vD7t375YZnS5qZYwCyj7fBb9DZmZmZep917RpU/z111/Iz8+XmTZy7tw5GBsbo27dutJ+wPtpIR8WrQ8fPsT9+/cxfPhwlbdJpEmcI0ukJgsWLMDp06exYsUKzJw5E23btsXIkSMVLt+0fv166eoGwPsi7tGjR+jRo0eh29fX14dIJJIZAbp79y527typtn3w9vaWrhf54Vf7BQqW6rK1tUXTpk2xbt06mXmFhw4dwo0bN0r02AWjcR+ODp47dw5nzpxR2H/nzp0yc1zj4uJw7tw56XNoY2MDT09P/PzzzwpHjRQtO/YhVZbfUsWdO3dKNML8MTMzM1SpUgUnTpyQaV+2bFmJt+nn5wc9PT3MmDFDbnTyw9fFxMRErqBTxN3dHTY2Nli+fLnMV9D79+/HzZs34ePjU+KsigwYMAAPHz7EqlWrcPXqVQwYMEDmdkXvsczMTKU+DBYUqB8+3xKJRG5FhebNm6N27dr48ccf8fLlS7ntfPy+U3b5LVU8evRIbnpO//798fjxY2zfvl3a9uzZM/z+++/o3bu39MNxgwYN4OLighUrVsj8XxMdHQ2RSCRdL1aVbRJpEkdkiZSwf/9+xMfHy7W3bdsWtWrVws2bNzFt2jQEBQWhd+/eAN4vO9S0aVOMGjVKOm+zgJWVFdq3b4/g4GA8fvwYUVFRqFOnDr788stCM/j4+GDBggXo3r07Bg0ahCdPnmDp0qWoU6cO/v77b7Xsp5mZGaKjozF48GA0a9YMAwcOhLW1Ne7du4d9+/ahXbt20uWoIiMj4ePjg/bt22PIkCFIT0/H4sWL0aBBA4V/wIvTq1cvbN++HX379oWPjw+Sk5OxfPlyuLq6KtxenTp10L59e4wcORJ5eXmIiopC5cqV8e2330r7LF26FO3bt0ejRo3w5ZdfolatWnj8+DHOnDmD+/fvS9f3VSQuLg6dOnVCWFhYsQd8ZWZmYvHixQAgXSt0yZIlsLCwgIWFhcwSZ507dwaAf30KWwAYNmwY5syZg2HDhsHd3R0nTpz4V0se1alTB99//z1mzpyJDh06oF+/fhCLxTh//jzs7OwQGRkJ4H2xFh0djVmzZqFOnTqwsbFROK+6YsWKmDt3LoKDg+Hh4YHPPvsMjx8/xsKFC+Ho6IgJEyaUOKsiPXv2hKmpKb755hvo6+vjk08+kbm9W7duMDAwQO/evfG///0PL1++xMqVK2FjY6Pww86HGjRogNatW2Py5MlIT0+HlZUVNm3aJHfaVj09PaxatQo9evRAgwYNEBwcjOrVq+PBgwc4evQozMzMsGfPHmn/+vXrw8PDQ6kDvpYsWYKMjAw8fPgQwPsR5vv37wN4v7RewfzeyZMnY926dUhOTpZ+Q9G/f3+0bt0awcHBuHHjhvQsXBKJRG4Jth9++AG+vr7o1q0bBg4ciOvXr2PJkiUYNmwY6tevL+2nyjaJNEZbyyUQ6YKilt/C/1/+5t27d0KLFi2EGjVqyCxFJQiCsHDhQgGAsHnzZkEQ/m+pnt9++02YPHmyYGNjIxgZGQk+Pj4ySwQJguLlt1avXi04OzsLYrFYcHFxEdauXStdWulDhS2/df78eZl+ipadKmj39vYWzM3NBUNDQ6F27dpCUFCQcOHCBZl+27ZtE+rXry+IxWLB1dVV2L59u8Lciny8vFF+fr4QEREhODg4CGKxWHBzcxP27t0rt72CZad++OEHYf78+YK9vb0gFouFDh06CFevXpV7nDt37ghffPGFUK1aNaFixYpC9erVhV69eglbt24t8nlQZfmtgkyKLh8/Fw4ODko9PwWv69OnTwvtk5ubKwwdOlQwNzcXTE1NBX9/f+HJkyeFLr/18bYKW1JqzZo1gpubmyAWiwVLS0vBw8NDOHTokPT2tLQ0wcfHRzA1NZVZPqqw99PmzZul27OyshICAgJklk4ThPfvdxMTk0KfB2UFBARIl5pTZPfu3ULjxo0FQ0NDwdHRUZg7d66wZs0auefh4/enILx/L3Xp0kUQi8VC1apVhSlTpgiHDh1SuM+XL18W+vXrJ1SuXFkQi8WCg4OD4O/vLxw+fFim34fPX3EcHBwKfZ99mL1gKbOPX9f09HRh6NChQuXKlQVjY2PBw8ND7v+EAjt27BCaNm0qiMVioUaNGsLUqVOFN2/eyPVTZZtEmiAShDIyy5/oP+DYsWPo1KmTzCkdiYiIqGQ4R5aIiIiIdBILWSIiIiLSSSxkiYiIiEgnabWQPXHiBHr37g07OzuIRCK5ZYQEQUBoaChsbW1hZGSELl26KFw7kkhXFJz2lPNjiYiI/j2tFrI5OTlo0qQJli5dqvD2efPmYdGiRVi+fDnOnTsHExMTeHt7S09/R0RERET/XWVm1QKRSIQdO3bAz88PwPvRWDs7O3z99df45ptvALxfq7Fq1aqIiYnBwIEDtRmXiIiIiLSszJ4QITk5GWlpaTKn+DM3N0erVq1w5syZQgvZvLw8mTPI5OfnIz09HZUrV1b6tIpEREREpB2CICA7Oxt2dnYypz9WpMwWsmlpaQCAqlWryrRXrVpVepsikZGRPKMIERERkY5LTU1FjRo1iuxTZgvZkpo8eTJCQkKk1zMzM1GzZk2kpqbCzMxMi8mIiIiIqDhZWVmwt7eHqalpsX3LbCFbrVo1AMDjx49ha2srbX/8+DGaNm1a6P3EYjHEYrFcu5mZGQtZIiIiIh2hzJTQMruOrJOTE6pVq4bDhw9L27KysnDu3Dm0adNGi8mIiIiIqCzQ6ojsy5cvkZiYKL2enJyMK1euwMrKCjVr1sT48eMxa9YsODs7w8nJCdOmTYOdnZ10ZQMiIiIi+u/SaiF74cIFdOrUSXq9YG5rYGAgYmJi8O233yInJwfDhw9HRkYG2rdvj9jYWBgaGmorMhERERGVEWVmHVlNycrKgrm5OTIzMzlHloiIiKiMU6V2K7NzZImIiIiIisJCloiIiIh0EgtZIiIiItJJZXYdWSKi8sRx0j5tR1DK3Tk+2o5ARKQ0jsgSERERkU5iIUtEREREOomFLBERERHpJBayRERERKSTWMgSERERkU5iIUtEREREOomFLBERERHpJBayRERERKSTWMgSERERkU5iIUtEREREOomFLBERERHpJBayRERERKSTWMgSERERkU5iIUtEREREOqlEheyGDRvQrl072NnZISUlBQAQFRWFXbt2qTUcEREREVFhVC5ko6OjERISgp49eyIjIwMSiQQAYGFhgaioKLUHJCIiIiJSROVCdvHixVi5ciW+//576OvrS9vd3d1x7do1tYYjIiIiIiqMyoVscnIy3Nzc5NrFYjFycnLUEoqIiIiIqDgqF7JOTk64cuWKXHtsbCzq16+vllBERERERMWpoOodQkJCMHr0aLx+/RqCICAuLg6//fYbIiMjsWrVKk1kpDLAcdI+bUdQyt05PtqOQERERKVE5UJ22LBhMDIywtSpU5Gbm4tBgwbBzs4OCxcuxMCBAzWRkYiIiIhIjsqFLAAEBAQgICAAubm5ePnyJWxsbNSdi4iIiIioSCUqZAsYGxvD2NhYXVmIiIiIiJSmciHr5OQEkUhU6O1JSUn/KhARERERkTJULmTHjx8vc/3t27e4fPkyYmNjMXHiRLUFIyIiIiIqisqF7Lhx4xS2L126FBcuXPjXgT4kkUgwffp0bNy4EWlpabCzs0NQUBCmTp1a5KgwEREREZV/Kq8jW5gePXpg27Zt6tocAGDu3LmIjo7GkiVLcPPmTcydOxfz5s3D4sWL1fo4RERERKR7/tXBXh/aunUrrKys1LU5AMDp06fRp08f+Pi8XxvU0dERv/32G+Li4tT6OERERESke1QuZN3c3GS+1hcEAWlpaXj69CmWLVum1nBt27bFihUrcOvWLdStWxdXr17FyZMnsWDBgkLvk5eXh7y8POn1rKwstWYiIiIiorJB5ULWz89P5rqenh6sra3h6ekJFxcXtQUDgEmTJiErKwsuLi7Q19eHRCLB7NmzERAQUOh9IiMjER4ertYcRERERFT2qFzIhoWFaSKHQlu2bMEvv/yCX3/9FQ0aNMCVK1cwfvx42NnZITAwUOF9Jk+ejJCQEOn1rKws2Nvbl1ZkIiIiIiolShWyqnw9b2ZmVuIwH5s4cSImTZokPfVto0aNkJKSgsjIyEILWbFYDLFYrLYMRERERFQ2KVXIWlhYFLvclSAIEIlEkEgkagkGALm5udDTk11YQV9fH/n5+Wp7DCIiIiLSTUoVskePHtV0DoV69+6N2bNno2bNmmjQoAEuX76MBQsWYMiQIVrJQ0RERERlh1KFrIeHh6ZzKLR48WJMmzYNo0aNwpMnT2BnZ4f//e9/CA0N1UoeIiIiIio7SryObG5uLu7du4c3b97ItDdu3PhfhypgamqKqKgoREVFqW2bRERERFQ+qFzIPn36FMHBwdi/f7/C29U5R5aIiIiIqDAqn6J2/PjxyMjIwLlz52BkZITY2FisW7cOzs7O2L17tyYyEhERERHJUXlE9siRI9i1axfc3d2hp6cHBwcHdO3aFWZmZoiMjJSeTpaIiIiISJNULmRzcnJgY2MDALC0tMTTp09Rt25dNGrUCJcuXVJ7QCIiotLgOGmftiMo5e4cDhgRFVB5akG9evWQkJAAAGjSpAl+/vlnPHjwAMuXL4etra3aAxIRERERKaLyiOy4cePw6NEjAO9PV9u9e3f88ssvMDAwQExMjLrzEREREREppHQh279/fwwbNgwBAQHSs3w1b94cKSkpiI+PR82aNVGlShWNBSUiIiIi+pDSUwtevHgBHx8f1KxZE6GhoUhKSgIAGBsbo1mzZixiiYiIiKhUKV3IHj58GElJSRg6dCg2btwIZ2dneHl54ddff0VeXp4mMxIRERERyVHpYC8HBwdMnz4dSUlJOHToEOzs7PDll1/C1tYWo0ePxsWLFzWVk4iIiIhIhsqrFhTw8vLCxo0bkZaWhsjISGzatAmtWrVSZzYiIiIiokKpvGrBh5KTkxETE4OYmBhkZmaiS5cu6spFRERERFQklUdkX79+jY0bN8LLywvOzs5Yv349hg4diuTkZMTGxmoiIxERERGRHKVHZOPi4rBmzRps3rwZr1+/Rt++fREbG4vOnTtLl+MiIiIiIiotSheyrVu3RpMmTTBz5kwEBATA0tJSk7mIiIiIiIqkdCF74cIFNGvWTJNZiIiIiIiUpvQcWRaxRERERFSWlHj5LSIiIiIibWIhS0REREQ6iYUsEREREekkFrJEREREpJNULmQfP36MwYMHw87ODhUqVIC+vr7MhYiIiIioNKh8itqgoCDcu3cP06ZNg62tLU+GQERERERaoXIhe/LkSfz1119o2rSpJvIQERGRmjhO2qftCEq5O8dH2xFIR6k8tcDe3h6CIGgiCxERERGR0lQuZKOiojBp0iTcvXtXA3GIiIiIiJSj8tSCAQMGIDc3F7Vr14axsTEqVqwoc3t6errawhERERERFUblQjYqKkoTOYiIiIiIVKJyIRsYGKiJHIV68OABvvvuO+zfvx+5ubmoU6cO1q5dC3d391LNQURERERli8qFLABIJBLs3LkTN2/eBAA0aNAAvr6+al9H9sWLF2jXrh06deqE/fv3w9raGrdv34alpaVaH4eIiIiIdI/KhWxiYiJ69uyJBw8eoF69egCAyMhI2NvbY9++fahdu7baws2dOxf29vZYu3attM3JyUlt2yciIiIi3aXyqgVjx45F7dq1kZqaikuXLuHSpUu4d+8enJycMHbsWLWG2717N9zd3fHpp5/CxsYGbm5uWLlypVofg4iIiIh0k8ojssePH8fZs2dhZWUlbatcuTLmzJmDdu3aqTVcUlISoqOjERISgilTpuD8+fMYO3YsDAwMCp2rm5eXh7y8POn1rKwstWYiIiIiorJB5UJWLBYjOztbrv3ly5cwMDBQS6gC+fn5cHd3R0REBADAzc0N169fx/LlywstZCMjIxEeHq7WHERERERU9qg8taBXr14YPnw4zp07B0EQIAgCzp49ixEjRsDX11et4WxtbeHq6irTVr9+fdy7d6/Q+0yePBmZmZnSS2pqqlozEREREVHZoPKI7KJFixAYGIg2bdpIT4bw7t07+Pr6YuHChWoN165dOyQkJMi03bp1Cw4ODoXeRywWQywWqzUHEREREZU9KheyFhYW2LVrF27fvo34+HgA71k0t9MAACAASURBVEdJ69Spo/ZwEyZMQNu2bREREQF/f3/ExcVhxYoVWLFihdofi4iIiIh0S4nWkQUAZ2dnODs7qzOLnBYtWmDHjh2YPHkyZsyYAScnJ0RFRSEgIECjj0tEREREZZ9ShWxISAhmzpwJExMThISEFNl3wYIFaglWoFevXujVq5dat0lEREREuk+pQvby5ct4+/at9GciIiIiIm1TqpA9evSowp+JiIiIiLRF5eW3hgwZonAd2ZycHAwZMkQtoYiIiIiIiqNyIbtu3Tq8evVKrv3Vq1dYv369WkIRERERERVH6VULsrKypCdAyM7OhqGhofQ2iUSCP/74AzY2NhoJSUREZY/jpH3ajqCUu3N8tB2BiDRE6ULWwsICIpEIIpEIdevWlbtdJBLx1LBEREREVGqULmSPHj0KQRDg5eWFbdu2wcrKSnqbgYEBHBwcYGdnp5GQREREREQfU7qQ9fDwAAAkJyejZs2aEIlEGgtFRERERFQclc/slZKSgpSUlEJv79ix478KRERERESkDJULWU9PT7m2D0dnJRLJvwpERERERKQMlZffevHihczlyZMniI2NRYsWLXDw4EFNZCQiIiIikqPyiKy5ublcW9euXWFgYICQkBBcvHhRLcGIiIiIiIqi8ohsYapWrYqEhAR1bY6IiIiIqEgqj8j+/fffMtcFQcCjR48wZ84cNG3aVG3BiIiIiIiKonIh27RpU4hEIgiCINPeunVrrFmzRm3BiIiIiIiKonIhm5ycLHNdT08P1tbWMqesJSIiIiLSNJULWQcHB03kICIiIiJSSYkO9jp8+DB69eqF2rVro3bt2ujVqxf+/PNPdWcjIiIiIiqUyoXssmXL0L17d5iammLcuHEYN24czMzM0LNnTyxdulQTGYmIiIiI5Kg8tSAiIgI//fQTxowZI20bO3Ys2rVrh4iICIwePVqtAYmIiIiIFFF5RDYjIwPdu3eXa+/WrRsyMzPVEoqIiIiIqDgqF7K+vr7YsWOHXPuuXbvQq1cvtYQiIiIiIiqOUlMLFi1aJP3Z1dUVs2fPxrFjx9CmTRsAwNmzZ3Hq1Cl8/fXXmklJRERERPQRpQrZn376Sea6paUlbty4gRs3bkjbLCwssGbNGkydOlW9CYmIiIiIFFCqkP34JAhERJrmOGmftiMo5e4cH21HICL6zyrROrJERERERNqm1IhsSEgIZs6cCRMTE4SEhBTZd8GCBWoJRkRERERUFKUK2cuXL+Pt27cAgEuXLkEkEinsV1g7EREREZG6KVXIHj16VPrzsWPHNJWFiIiIiEhpKs2Rffv2LSpUqIDr169rKk+R5syZA5FIhPHjx2vl8YmIiIio7FCpkK1YsSJq1qwJiUSiqTyFOn/+PH7++Wc0bty41B+biIiIiMoelVct+P777zFlyhSkp6drIo9CL1++REBAAFauXAlLS8tSe1wiIiIiKruUmiP7oSVLliAxMRF2dnZwcHCAiYmJzO2XLl1SW7gCo0ePho+PD7p06YJZs2apfftEREREpHtULmT79OlTqqsTbNq0CZcuXcL58+eV6p+Xl4e8vDzp9aysLE1FIyIiIiItUrmQnT59ugZiKJaamopx48bh0KFDMDQ0VOo+kZGRCA8P13AyIiIiItI2lefI1qpVC8+fP5drz8jIQK1atdQSqsDFixfx5MkTNGvWDBUqVECFChVw/PhxLFq0CBUqVFB40NnkyZORmZkpvaSmpqo1ExERERGVDSqPyN69e1dhAZmXl4f79++rJVSBzp0749q1azJtwcHBcHFxwXfffQd9fX25+4jFYojFYrXmICIiIqKyR+lCdvfu3dKfDxw4AHNzc+l1iUSCw4cPw8nJSa3hTE1N0bBhQ5k2ExMTVK5cWa6diIiIiP5blC5k/fz8pD8HBgbK3FaxYkU4Ojpi/vz56ktGRERERFQEpQvZ/Px8AICTkxPOnz+PKlWqaCxUUXiKXCIiIiICSnCwV3h4OExNTeXa37x5g/Xr16slFBERERFRcVQuZIODg5GZmSnXnp2djeDgYLWEIiIiIiIqjsqFrCAICk+IcP/+fZkDwIiIiIiINEnpObJubm4QiUQQiUTo3LkzKlT4v7tKJBIkJyeje/fuGglJRERERPQxlVctuHLlCry9vVGpUiXpbQYGBnB0dMQnn3yi/oREGuA4aZ+2Iyjl7hwfbUcgIiIqs5QuZMPCwgAAjo6OGDBggMJTxl6/fp3ruxIRERFRqVB5jmxgYKBMEZudnY0VK1agZcuWaNKkiVrDEREREREVRuVCtsCJEycQGBgIW1tb/Pjjj/Dy8sLZs2fVmY2IiIiIqFBKTy0AgLS0NMTExGD16tXIysqCv78/8vLysHPnTri6umoqIxERERGRHKVHZHv37o169erh77//RlRUFB4+fIjFixdrMhsRERERUaGUHpHdv38/xo4di5EjR8LZ2VmTmYiIiIiIiqX0iOzJkyeRnZ2N5s2bo1WrVliyZAmePXumyWxERERERIVSupBt3bo1Vq5ciUePHuF///sfNm3aBDs7O+Tn5+PQoUPIzs7WZE4iIiIiIhkqr1pgYmKCIUOG4OTJk7h27Rq+/vprzJkzBzY2NvD19dVERiIiIiIiOSVefgsA6tWrh3nz5uH+/fv47bff1JWJiIiIiKhY/6qQLaCvrw8/Pz/s3r1bHZsjIiIiIiqWWgpZIiIiIqLSxkKWiIiIiHQSC1kiIiIi0kksZImIiIhIJ5WokN2wYQPatWsHOzs7pKSkAACioqKwa9cutYYjIiIiIiqMyoVsdHQ0QkJC0LNnT2RkZEAikQAALCwsEBUVpfaARERERESKqFzILl68GCtXrsT3338PfX19abu7uzuuXbum1nBERERERIVRuZBNTk6Gm5ubXLtYLEZOTo5aQhERERERFaeCqndwcnLClStX4ODgINMeGxuL+vXrqy0YEanGcdI+bUdQyt05PtqOQERE5YTKhWxISAhGjx6N169fQxAExMXF4bfffkNkZCRWrVqliYxERERERHJULmSHDRsGIyMjTJ06Fbm5uRg0aBDs7OywcOFCDBw4UBMZiYiIiIjkqFTIvnv3Dr/++iu8vb0REBCA3NxcvHz5EjY2NprKR0RERESkkEoHe1WoUAEjRozA69evAQDGxsYsYomIiIhIK1RetaBly5a4fPmyJrLIiYyMRIsWLWBqagobGxv4+fkhISGhVB6biIiIiMo2lefIjho1Cl9//TXu37+P5s2bw8TEROb2xo0bqy3c8ePHMXr0aLRo0QLv3r3DlClT0K1bN9y4cUPucYmIiIjov0XlQrbggK6xY8dK20QiEQRBgEgkkp7pSx1iY2NlrsfExMDGxgYXL15Ex44d1fY4RERERKR7VC5kk5OTNZFDKZmZmQAAKysrrWUgIiIiorJB5UL24xMhlJb8/HyMHz8e7dq1Q8OGDQvtl5eXh7y8POn1rKys0ohHRERERKVM5UJ2/fr1Rd7+xRdflDhMUUaPHo3r16/j5MmTRfaLjIxEeHi4RjIQERERUdmhciE7btw4metv375Fbm4uDAwMYGxsrJFCdsyYMdi7dy9OnDiBGjVqFNl38uTJCAkJkV7PysqCvb292jMRERERkXapXMi+ePFCru327dsYOXIkJk6cqJZQBQRBwFdffYUdO3bg2LFjcHJyKvY+YrEYYrFYrTmIiIiIqOxRuZBVxNnZGXPmzMHnn3+O+Ph4dWwSwPvpBL/++it27doFU1NTpKWlAQDMzc1hZGSktschIiIiIt2j8gkRClOhQgU8fPhQXZsDAERHRyMzMxOenp6wtbWVXjZv3qzWxyEiIiIi3aPyiOzu3btlrguCgEePHmHJkiVo166d2oIVbJuIiIiISBGVC1k/Pz+Z6yKRCNbW1vDy8sL8+fPVFoyIiIiIqCgqF7L5+fmayEFEREREpBKV58jOmDEDubm5cu2vXr3CjBkz1BKKiIiIiKg4Khey4eHhePnypVx7bm4uT0RARERERKVG5akFgiBAJBLJtV+9ehVWVlZqCaXrHCft03YEpdyd46PtCEREREQlpnQha2lpCZFIBJFIhLp168oUsxKJBC9fvsSIESM0EpKIiIiI6GNKF7JRUVEQBAFDhgxBeHg4zM3NpbcZGBjA0dERbdq00UhIIiIiIqKPKV3IBgYGAgCcnJzQtm1bVKxYUWOhiIiIiIiKo/IcWQ8PD+nPr1+/xps3b2RuNzMz+/epiIiIiIiKofKqBbm5uRgzZgxsbGxgYmICS0tLmQsRERERUWlQuZCdOHEijhw5gujoaIjFYqxatQrh4eGws7PD+vXrNZGRiIiIiEiOylML9uzZg/Xr18PT0xPBwcHo0KED6tSpAwcHB/zyyy8ICAjQRE4iIiIiIhkqj8imp6ejVq1aAN7Ph01PTwcAtG/fHidOnFBvOiIiIiKiQqhcyNaqVQvJyckAABcXF2zZsgXA+5FaCwsL9aYjIiIiIiqEyoVscHAwrl69CgCYNGkSli5dCkNDQ0yYMAETJ05Ue0AiIiIiIkVUniM7YcIE6c9dunRBfHw8Ll68iDp16qBx48ZqDUdEREREVBiVC9kPvX79Gg4ODnBwcFBXHiIiIiIipag8tUAikWDmzJmoXr06KlWqhKSkJADAtGnTsHr1arUHJCIiIiJSROVCdvbs2YiJicG8efNgYGAgbW/YsCFWrVql1nBERERERIVRuZBdv349VqxYgYCAAOjr60vbmzRpgvj4eLWGIyIiIiIqjMqF7IMHD1CnTh259vz8fLx9+1YtoYiIiIiIiqNyIevq6oq//vpLrn3r1q1wc3NTSygiIiIiouKovGpBaGgoAgMD8eDBA+Tn52P79u1ISEjA+vXrsXfvXk1kJCIiIiKSo3Ih26dPH+zZswczZsyAiYkJQkND0axZM+zZswddu3bVREYiIiKicslx0j5tR1DK3Tk+2o6gkNKFbFJSEpycnCASidChQwccOnRIk7mIiIiIiIqk9BxZZ2dnPH36VHp9wIABePz4sUZCEREREREVR+lCVhAEmet//PEHcnJy1B6IiIiIiEgZKq9aQERERERUFihdyIpEIohEIrk2IiIiIiJtUPpgL0EQEBQUBLFYDAB4/fo1RowYARMTE5l+27dvV29CAEuXLsUPP/yAtLQ0NGnSBIsXL0bLli3V/jhEREREpDuULmQDAwNlrn/++edqD6PI5s2bERISguXLl6NVq1aIioqCt7c3EhISYGNjUyoZiIiIiKjsUbqQXbt2rSZzFGrBggX48ssvERwcDABYvnw59u3bhzVr1mDSpElayURERERE2lemD/Z68+YNLl68iC5dukjb9PT00KVLF5w5c0bhffLy8pCVlSVzISIiIqLyRyR8vK5WGfLw4UNUr14dp0+fRps2baTt3377LY4fP45z587J3Wf69OkIDw+Xa8/MzISZmZlG8xIREZHm8CxY/w1ZWVkwNzdXqnYr0yOyJTF58mRkZmZKL6mpqdqOREREREQaoPQcWW2oUqUK9PX15c4g9vjxY1SrVk3hfcRisXRlBSIiIiIqv8r0iKyBgQGaN2+Ow4cPS9vy8/Nx+PBhmakGRERERPTfU6ZHZAEgJCQEgYGBcHd3R8uWLREVFYWcnBzpKgZERERE9N9U5gvZAQMG4OnTpwgNDUVaWhqaNm2K2NhYVK1aVdvRiIiIiEiLynwhCwBjxozBmDFjtB2DiIiIiMqQMj1HloiIiIioMCxkiYiIiEgnsZAlIiIiIp2kE3NkiYiIiHjGLPoYR2SJiIiISCexkCUiIiIincRCloiIiIh0EgtZIiIiItJJLGSJiIiISCexkCUiIiIincRCloiIiIh0EgtZIiIiItJJ5f6ECIIgAACysrK0nISIiIiIilNQsxXUcEUp94VsdnY2AMDe3l7LSYiIiIhIWdnZ2TA3Ny+yj0hQptzVYfn5+Xj48CFMTU0hEom0HadEsrKyYG9vj9TUVJiZmWk7jlqUt30qb/sDlL994v6UfeVtn8rb/gDlb5/K2/4A5WOfBEFAdnY27OzsoKdX9CzYcj8iq6enhxo1amg7hlqYmZnp7JuyMOVtn8rb/gDlb5+4P2Vfedun8rY/QPnbp/K2P4Du71NxI7EFeLAXEREREekkFrJEREREpJP0p0+fPl3bIah4+vr68PT0RIUK5Wc2SHnbp/K2P0D52yfuT9lX3vapvO0PUP72qbztD1A+96kw5f5gLyIiIiIqnzi1gIiIiIh0EgtZIiIiItJJLGSJiIiISCexkCUiIiIincRCVsdcv35d2xGIiIiIygQWsjogOzsbK1asQMuWLdGkSRNtxyEiov+QIUOGIDs7W649JycHQ4YM0UIi9ZJIJLhy5QpevHih7ShUAlx+qww7ceIEVq9ejW3btsHOzg79+vXDJ598ghYtWmg7mspycnIwZ84cHD58GE+ePEF+fr7M7UlJSVpKphl6enrw9PTEDz/8gObNm2s7jtJSU1MhEomkp3WOi4vDr7/+CldXVwwfPlzL6ZRjaWkJkUikVN/09HQNpyHSffr6+nj06BFsbGxk2p89e4Zq1arh3bt3WkpWMuPHj0ejRo0wdOhQSCQSeHh44PTp0zA2NsbevXvh6emp7Yhq06VLFyQlJZW7v7EfKv8r5eqYtLQ0xMTEYPXq1cjKyoK/vz/y8vKwc+dOuLq6ajteiQ0bNgzHjx/H4MGDYWtrq3ShoavWrFmDu3fvYvTo0Th79qy24yht0KBBGD58OAYPHoy0tDR07doVDRo0wC+//IK0tDSEhoZqO2KxoqKipD8/f/4cs2bNgre3N9q0aQMAOHPmDA4cOIBp06ZpK6LKFi1apHTfsWPHajCJZty+fRtHjx5V+CFXF95zH4uNjUWlSpXQvn17AMDSpUuxcuVKuLq6YunSpbC0tNRyQuVkZWVBEAQIgoDs7GwYGhpKb5NIJPjjjz/kiltdsHXrVnz++ecAgD179iA5ORnx8fHYsGEDvv/+e5w6dUrLCdWnb9++ePbsmbZjaBRHZMuQ3r1748SJE/Dx8UFAQAC6d+8OfX19VKxYEVevXtXpQtbCwgL79u1Du3bttB2FimBpaYmzZ8+iXr16WLRoETZv3oxTp07h4MGDGDFihM59qv/kk0/QqVMnjBkzRqZ9yZIl+PPPP7Fz504tJVONk5OTUv1EIpHOvUYrV67EyJEjUaVKFVSrVk3mQ65IJMKlS5e0mK5kGjVqhLlz56Jnz564du0aWrRogZCQEBw9ehQuLi5Yu3attiMqRU9Pr8hBB5FIhPDwcHz//felmOrfMzQ0RGJiImrUqIHhw4fD2NgYUVFRSE5ORpMmTZCVlaXtiKQCjsiWIfv378fYsWMxcuRIODs7azuOWllaWsLKykrbMagYb9++hVgsBgD8+eef8PX1BQC4uLjg0aNH2oxWIgcOHMDcuXPl2rt3745JkyZpIVHJJCcnazuCxsyaNQuzZ8/Gd999p+0oapOcnCwdeNi2bRt69eqFiIgIXLp0CT179tRyOuUdPXoUgiDAy8sL27Ztk/k/3MDAAA4ODrCzs9NiwpKpWrUqbty4AVtbW8TGxiI6OhoAkJubC319fS2nI1XxYK8y5OTJk8jOzkbz5s3RqlUrLFmypNx8JTBz5kyEhoYiNzdX21GoCA0aNMDy5cvx119/4dChQ+jevTsA4OHDh6hcubKW06mucuXK2LVrl1z7rl27dHJ/yqMXL17g008/1XYMtTIwMJD+X/fnn3+iW7duAAArKyudGu3z8PCAp6cnkpOT0adPH3h4eEgvbdq00ckiFgCCg4Ph7++Phg0bQiQSoUuXLgCAc+fOwcXFRcvpSFWcWlAG5eTkYPPmzVizZg3i4uIgkUiwYMECDBkyBKamptqOpzQ3NzeZr6USExMhCAIcHR1RsWJFmb66+PVheXTs2DH07dsXWVlZCAwMxJo1awAAU6ZMQXx8PLZv367lhKqJiYnBsGHD0KNHD7Rq1QrA+z9WsbGxWLlyJYKCgrQbsASKO0q84DXTFUOHDkWLFi0wYsQIbUdRG19fX7x58wbt2rXDzJkzkZycjOrVq+PgwYMYM2YMbt26pe2IJZKbm4t79+7hzZs3Mu2NGzfWUqKS27p1K1JTU/Hpp59KD25dt24dLCws0KdPHy2nI1WwkC3jEhISsHr1amzYsAEZGRno2rUrdu/ere1YSgkPD1e6b1hYmAaTkCokEgmysrJkDki5e/cujI2NdfLAjnPnzmHRokW4efMmAKB+/foYO3astLDVNX379pW5/vbtW1y/fh0ZGRnw8vLSuQ8bkZGRWLBgAXx8fNCoUSO5D7m6ePDavXv3MGrUKKSmpmLs2LEYOnQoAGDChAmQSCQqHbxXFjx9+hTBwcHYv3+/wtslEkkpJ1K/jIwMWFhYaDsGlQALWR0hkUiwZ88erFmzRmcKWdI9r169giAIMDY2BgCkpKRgx44dqF+/Pry9vbWcjgqTn5+PkSNHonbt2vj222+1HUclRR3IposHr5VHAQEBSElJQVRUFDw9PbFjxw48fvwYs2bNwvz58+Hj46PtiCqZO3cuHB0dMWDAAACAv78/tm3bBltbW/zxxx86OcL8X8ZClkrVhQsXpCNjrq6uOrXG6n9Bt27d0K9fP4wYMQIZGRlwcXFBxYoV8ezZMyxYsAAjR47UdkSV5efnIzExUeHSTh07dtRSKvVLSEiAp6enTh6UV94Utu7q8+fPYWNjo3MjmLa2tti1axdatmwJMzMzXLhwAXXr1sXu3bsxb948nDx5UtsRVeLk5IRffvkFbdu2xaFDh+Dv74/Nmzdjy5YtuHfvHg4ePKjtiKQCrlpApeL+/fv47LPPcOrUKenXNxkZGWjbti02bdoknaNE2nXp0iX89NNPAN7PIatatSouX76Mbdu2ITQ0VOcK2bNnz2LQoEFISUnBx5/ZRSKRzhUURblz547OLEwfEhKCmTNnwsTEBCEhIYX2E4lEmD9/fikmU4/Cxofy8vJgYGBQymn+vZycHGlRbmlpiadPn6Ju3bpo1KiRTh7fkJaWBnt7ewDA3r174e/vj27dusHR0VFnpxz9l7GQpVIxbNgwvH37Fjdv3kS9evUAvB9BCg4OxrBhwxAbG6vlhAS8P5ij4IDCgwcPol+/ftDT00Pr1q2RkpKi5XSqGzFiBNzd3bFv375ycyKOjws/QRDw6NEj7Nu3D4GBgVpKpZrLly/j7du30p8Lo2uvV8HcV5FIhFWrVqFSpUrS2yQSCU6cOKGTR8XXq1cPCQkJcHR0RJMmTfDzzz/D0dERy5cvh62trbbjqczS0hKpqamwt7dHbGwsZs2aBeD971J5+nD7X8GpBVQqjIyMcPr0abi5ucm0X7x4ER06dOCyXGVE48aNMWzYMPTt2xcNGzZEbGws2rRpg4sXL8LHxwdpaWnajqgSExMTXL16FXXq1NF2FLXx9PSUKfD09PRgbW0NLy8vDBkyBBUqcHxCWwrm+6akpKBGjRoya5IaGBjA0dERM2bM0LlRv40bN+Ldu3cICgrCxYsX0b17d6Snp8PAwAAxMTHSuaa6YsyYMdi7dy+cnZ1x+fJl3L17F5UqVcKmTZswb948nRxl/i/j/3hUKuzt7aUjMB+SSCQ6uxZheRQaGopBgwZhwoQJ8PLykp7W9eDBg3IfQnRBq1atkJiYqPOF7O7du9GjRw9UrFgRx44d03YcKkTBiSs6deqEHTt2lJuj4AtO5woAzZs3R0pKCuLj41GzZk1UqVJFi8lK5qeffoKjoyNSU1Mxb9486cj5o0ePMGrUKC2nI1VxRJZKxa5duxAREYGlS5fC3d0dwPsDv7766it899138PPz03JCKpCWloZHjx6hSZMm0NN7f86UuLg4mJmZ6dzXojt27MDUqVMxceJEhUs76crRyfr6+khLS4O1tXWhBxKRdn0473fChAlFTotYsGBBKSZTnzdv3iA5ORm1a9fmyD+VGSxkqVRYWloiNzcX7969k/4HWPCziYmJTN/09HRtRKQPJCYm4s6dO+jYsSOMjIwgCILOzVcEIC3EFdGlg72qVauGlStXonfv3tDT08Pjx49hbW2t7Vj0gQ9HYTt16lRoP5FIhCNHjpRisn8vNzcXX331FdatWwcAuHXrFmrVqoWvvvoK1atX16nTPRfYsGEDfv75ZyQlJeHMmTNwcHBAVFQUnJyceEIEHcOPVFQqoqKitB2BlPD8+XP4+/vj6NGjEIlEuH37NmrVqoWhQ4fC0tJS544gL/iqV9eNGDECffr0gUgkgkgkQrVq1QrtqyvFeXlz9OhRhT+XB5MnT8bVq1dx7Ngx6WmrAaBLly6YPn26zhWy0dHRCA0Nxfjx4zF79mzp74yFhQWioqJYyOoYjsgSkdQXX3yBJ0+eYNWqVahfvz6uXr2KWrVq4cCBAwgJCcE///yj7YglcuPGDblTa4pEIvTu3VuLqVQTHx+PxMRE+Pr6Yu3atYXOv+QfYVI3BwcHbN68Ga1bt4apqan0/4XExEQ0a9YMWVlZ2o6oEldXV0RERMDPz09mf65fvw5PT088e/ZM2xFJBRyRpVL15MkThQvT68pcxfLu4MGDOHDggNy6vs7Ozjq5/FZSUhL69u2La9euQSQSSdf3LJgmoUujly4uLnBxcUFYWBg+/fRT6dnXiDTt6dOnCudk5+Tk6OSUo+TkZIUHr4rFYuTk5GghEf0bhU8gI1KjixcvomHDhrC1tUXjxo3RtGlT6UUXj4Yvr3JychQWSOnp6RCLxVpI9O+MGzcOTk5OePLkCYyNjXH9+nWcOHEC7u7uOnv0f1hYGItYKlUFazEXKCheV61aJV3ZRJc4OTnhypUrcu2xsbGoX7++FhLRv8ERWSoVQ4YMQd26dbF69WpUrVpVJz/F/xd06NAB69evx8yZMwG8/4OVn5+PmJjRUAAAEVhJREFUefPmFXkAS1l15swZHDlyBFWqVIGenh709fXRvn17REZGYuzYsUUuxl+WNGvWDIcPH4alpSXc3NyK/P3hGpikbhEREejRowdu3LiBd+/eYeHChbhx4wZOnz6N48ePazueykJCQjB69Gi8fv0agiAgLi4Ov/32GyIjI7Fq1SptxyMVsZClUpGUlIRt27bp/Hqe5d28efPQuXNnXLhwAW/evMG3336Lf/75B+np6Th16pS246lMIpFIz1RWpUoVPHz4EPXq1YODgwMSEhK0nE55ffr0kY6IFxz0RVRa2rdvjytXrmDOnDlo1KgRDh48iGbNmuHMmTNo1KiRtuOpbNiwYTAyMsLUqVORm5uLQYMGwc7ODgsXLsTAgQO1HY9UxIO9qFT4+flh8ODB+OSTT7QdhYqRmZmJJUuW4OrVq3j58iWaNWuG0aNH6+SpKDt06ICvv/4afn5+GDRoEF68eIGpU6dixYoVuHjxIq5fv67tiESkRbm5uXj58iXXZdZhLGSpVDx79gyBgYFo2bIlGjZsKLcwva+vr5aSUXl24MAB5OTkoF+/fkhMTESvXr1w69YtVK5cGZs3b4aXl5e2I6qsVq1aOH/+PCpXrizTnpGRgWbNmiEpKUlLyag8k0gk2LFjB27evAng/ZH/ffr04YkRSOtYyFKp2LNnDwYPHqxwmRZdWpi+PPr777+V7lseVpdIT0+HpaWlzn49r6enh7S0NLkRpMePH8Pe3l5miTEidfjnn3/g6+uLtLQ01KtXD8D7kyJYW1tjz549aNiwoZYTqubx48f45ptvcPjwYTx58gQfl0H8e6RbWMhSqXB0dESvXr0wbdo0VK1aVdtx6AN6enoyS1MVhh84tGv37t0A3k/TWbduHczNzaW3SSQSHD58GIcOHdKpub+kG9q0aQNra2usW7cOlpaWAIAXL14gKCgIT58+xenTp7WcUDU9evTAvXv3MGbMGNja2sp9qOVazLqFhSyVClNTU1y5cgW1a9fWdhT6iCrrwzr8v/buNCbqaw0D+DOAgFAIo0JiKVwWwSoBEahVKGsj4FIoJF2Ulgq21oa4IbZpYrWCItqw2Q/WLiptjU1ttLQQErSyVK0owchaUKLSVhwJiGxSmYH7wctcBtxQmDP/4fklJM75f3mSCfLOmfc95z//Gcck9DCD1+3e70PHpEmT4ODggPT0dCxdulREPNJjkydPRnl5Odzc3DTWq6ur8cILL+DOnTuCkj0ZCwsL/P777/D09BQdhcYAm1tIK6Kjo1FUVMRCVgcNLU5LS0vh6+s7ou9NqVTizJkzLGQFGrxExNHREefPn8e0adMEJ6KJwtXVFQqFYkQhe/PmTUmeRGNnZ/fIb6BIOrgjS1qxY8cOZGVlYcmSJXB3dx8x7LV27VpByWgoQ0NDNDc3j+i/bG1thY2NDVsLiCaIofMMp06dwocffohPP/0U8+fPBwCcPXsWycnJSEtLw+LFi0XFfCKFhYVIT0/Hvn374ODgIDoOPSUWsqQVjo6OD3wmk8k4aa0jDAwMoFAoYG1trbHe0NAAHx8fyd2pro+Sk5Mf+nzLli1aSkL6bLB3ftDw652HvpbaB1y5XI6enh4olUqYmZmN2Fhpa2sTlIyeBFsLSCuuXLkiOgI9RHR0NIB7f5RWrFihcR2tSqVCZWUlfH19RcWjIY4dO6bxuq+vD1euXIGRkRGcnZ1ZyNKYKCoqEh1h3GRmZkr21BIaiYUsEakn4AcGBmBhYYHJkyernxkbG2P+/Pl47733RMWjIe53rW5HRwdWrFiBqKgoAYlIHwUGBoqOMG5WrFjxwGdSG1wjthaQlsTHxz/0+f79+7WUhB5m27ZtSEpKgrm5uegoNEpVVVV45ZVXcPXqVdFRSA+1t7fj3LlzuHnzpnrwcFBsbKygVE9m7dq12LNnz4j17u5uLF26VK93o/URd2RJK27duqXxuq+vD9XV1Whvb5fk7Ur6auvWraIj0BO6ffs2bt++LToG6aFff/0VMTEx6OrqgqWlpcbX8jKZTHKFbH5+PuRyObZt26Ze6+7uRnh4uMBU9KRYyJJWDO/rA+4dJ/TBBx/wSC4dwhtvdN/wnaSBgQE0Nzfju+++w6JFiwSlIn22ceNGxMfHIzU1FWZmZqLjPLXCwkL4+/tDLpdj/fr16OzsRFhYGIyMjFBQUCA6Ho0SWwtIqPr6egQFBaG5uVl0FAJvvJGC4SeAGBgYwNraGiEhIfj4449hYWEhKBnpK3Nzc1RVVcHJyUl0lDFTWVmJ4OBgbN26FYcPH4aJiQny8/PZViVB3JEloRobG6FUKkXHoP85deoUb7zRcTwBhLQtLCwM5eXlelXIenh4IC8vDwsXLsSLL76IvLw8jSFXkg4WsqQViYmJGq8Hvw7Nz8/HO++8IygVDccbb4houCVLlmDTpk2ora2974U2ERERgpI9vrlz5973yC0TExNcv34dfn5+6rWKigptRqOnxNYC0org4GCN10O/Do2Pjx9xJSqJwRtvpKG8vBw//vgjmpqacPfuXY1nR48eFZSK9JWBgcEDn0nlQoShg12PwqFXaWEhS0RqvPFG9/3www+IjY1FWFgYCgsLERoaioaGBigUCkRFReHAgQOiIxLpLJVKhdOnT8PDwwNWVlai49AY4DYYEallZWWJjkCPkJqaiszMTCQkJMDCwgLZ2dlwdHTE+++/j+nTp4uOR3qot7cXpqamomOMCUNDQ4SGhqKuro6FrJ7gjiyNGy8vL/z222+Qy+UP7E8axJ4kosdjbm6OmpoaODg4YOrUqSguLoa7uzvq6uoQEhLCE0BozJmammLevHkIDAxEUFAQfH19JT0Y5ePjg127duHll18WHYXGAHdkadxERkbCxMQEAPDqq68KTkOPq7GxEQcOHEBjYyOys7NhY2ODgoIC2Nvbw83NTXS8CU8ul6OzsxMAYGtri+rqari7u6O9vR09PT2C05E+OnHiBEpLS1FcXIzMzEwolUr4+PioC9uFCxeKjjgq27dvR1JSElJSUuDt7T3iyC1LS0tByehJcEeWxh17kqSjpKQEixYtgp+fH0pLS1FXVwcnJyekpaWhvLwcP/30k+iIE97y5cvh4+ODxMREpKSk4PPPP0dkZCSOHz8OLy8vDnvRuFIqlTh//jz27duHQ4cOob+/XxLDXkMNHV4b+k3hwMCAZIbX6P9YyJJWmJqaoq6ubsRh7qRbFixYgNdeew2JiYmwsLDAxYsX4eTkhHPnziE6Ohp///236IgTXltbG3p7e/Hss8+iv78fu3fvxpkzZ+Di4oLNmzdDLpeLjkh6qKGhAcXFxeqff//9FwEBAQgKCsK6detExxuVkpKShz4PDAzUUhIaCyxkSSvYkyQNzzzzDKqqquDo6KhRyF69ehXPP/88ent7RUckIi2ztbXFnTt3EBQUhKCgIAQGBsLDw+Ohcw9E2vLgw+GIxtBgT1JeXh6am5vR0dGh8UO6wcrK6r7DQhcuXICtra2ARHQ/jY2N2Lx5M5YtW4abN28CAAoKClBTUyM4Gekja2tr9PT04MaNG7hx4wYUCgXu3LkjOtZT6+npwZ9//onKykqNH5IW7siSVrAnSRqSkpJQVlaGI0eOwNXVFRUVFVAoFIiNjUVsbCwPCtcB7GMmEdrb21FaWoqSkhKUlJSgtrYWnp6eCA4Oxo4dO0THG5WWlhbExcWhoKDgvs/590haWMiSVuTk5MDOzg6GhoYa6/39/WhqauI1tTri7t27SEhIwMGDB6FSqWBkZASVSoXly5fj4MGDI94/0j72MZNIra2tKC4uRm5uLg4fPizJYa+YmBhcu3YNWVlZCAoKwrFjx6BQKLB9+3akp6djyZIloiPSKLCQJa0wNDREc3MzbGxsNNZbW1thY2Mjuf8I9V1TUxOqq6vR1dWFuXPnwsXFRXQk+h/2MZO2HT16VD3kVVtbiylTpuCll15S98vOmTNHdMRRmT59OnJzczFv3jxYWlqivLwcrq6u+OWXX7B7926cOnVKdEQaBZ4jS1ox2EIwXFdXl97cGKNP7O3tYW9vLzoG3cdgH/PwE0DYx0zjZfXq1QgICMCqVasQGBgId3d30ZGeSnd3t3pTRS6Xo6WlBa6urnB3d+flPBLEQpbGVWJiIoB7fbGffPIJzMzM1M9UKhXKysrg6ekpKh7h/+/R48jIyBjHJPQ43nzzTXz00Uc4cuQIZDIZ+vv7cfr0aSQlJSE2NlZ0PNJDgwOF+mLmzJmor6+Hg4MD5syZg3379sHBwQFffPEFr3mWIBayNK4uXLgA4N6ObFVVFYyNjdXPjI2NMWfOHCQlJYmKR/j/e/QoPGpHN6SmpiIhIQF2dnZQqVSYPXs2lEolYmJisHnzZtHxSE+pVCr8/PPPqKurAwDMnj0bkZGRkuybX7dunfp0lq1btyI8PBzff/89jI2NkZOTIzgdjRZ7ZEkr4uLikJ2dzav/iMbIX3/9haqqKnR3d2Pu3LmYMWOG6Eikpy5fvozFixfjn3/+wcyZMwEA9fX1sLOzQ35+PpydnQUnfDqDx3DZ29tj2rRpouPQKLGQJaIRLl++jMbGRgQEBGDy5MkP7HEmMb755htkZmbi0qVLAAAXFxesX78e7777ruBkpI8WL16MgYEBHDp0CFOmTAFwb1D3rbfegoGBAfLz8wUnfDS2UOkvthYQkVpraytef/11FBUVQSaT4dKlS3BycsLKlSshl8uRnp4uOuKEt2XLFmRkZGDNmjVYsGABAOCPP/7Ahg0b0NTUhOTkZMEJSd+UlJTg7Nmz6iIWAKZOnYq0tDT4+fkJTPb4hrdQVVRUQKlUqneYGxoaYGhoCG9vbxHx6CmwkCUitQ0bNmDSpEloamrCrFmz1OtvvPEGEhMTWcjqgL179+Krr77CsmXL1GsRERHw8PDAmjVrWMjSmDMxMUFnZ+eI9a6uLo25B11WVFSk/ndGRgYsLCyQk5MDuVwOALh16xbi4uLg7+8vKiI9IV5RS0RqhYWF2LVrF5577jmNdRcXF1y7dk1QKhqqr68PPj4+I9a9vb2hVCoFJCJ9t3TpUqxatQplZWUYGBjAwMAAzp49i9WrVyMiIkJ0vFFLT0/Hzp071UUscO8YrsELEUhaWMgSkVp3d7fGEWmD2traYGJiIiARDff2229j7969I9a//PJLxMTECEhE+m7Pnj1wdnbGggULYGpqClNTU/j6+mLGjBnIysoSHW/UOjo60NLSMmK9paXlvjvPpNvYWkBEav7+/vj222+RkpICAOpzSnfv3o3g4GDB6SauoYMqMpkMX3/9NQoLCzF//nwAQFlZGZqamniOLI0LKysr5Obm4vLly+rjt2bNmiXZkzKioqIQFxeH9PR0zJs3D8C936FNmzYhOjpacDoaLZ5aQERqNTU1CAkJgZeXF06ePImIiAjU1NSgra0Np0+flvwxO1L1uB8iZDIZTp48Oc5paCLQ5yn/np4eJCUlYf/+/ejr6wMAGBkZYeXKlfjss89gbm4uOCGNBgtZIgJwr/cyPDwcO3fuxPHjx3Hx4kV0dXXBy8sLCQkJvPGGaAIZ/uHpYVP+Uv3w1N3djcbGRgCAs7MzC1iJYiFLRGrW1tY4c+YMXFxcREchIh2RkZGB4uLiB075b9y4UXBCmshYyBKR2oYNG2BiYoK0tDTRUYhIR9ja2qKwsBBubm4a69XV1QgNDcX169cFJSPisBcRDaFUKrF//36cOHEC3t7eI75qk1ovHBE9PU75ky5jIUtEatXV1fDy8gJwrwduKF5RSzQxccqfdBlbC4iIiOiBOOVPuoyFLBERET0Sp/xJF7GQJSIiIiJJ4hW1RERERCRJLGSJiIiISJJYyBIRERGRJLGQJSIiIiJJYiFLRERERJLEQpaIiIiIJImFLBERERFJEgtZIiIiIpKk/wJElQZtnBaq0gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 700x350 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "masker.render_result(sentence, e, mode='plot', execute=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30f5e9f9",
   "metadata": {},
   "source": [
    "And that's it! Want to learn more? Check out the other tabs for more use cases. If you want to gain a deeper understanding of the `*Masker` classes and implement your own, check out this jupyter notebook for an example where we build one from scratch and browse the documentations. Bugs? Suggestions? Questions? Ask away on [GitHub](https://github.com/YilunZhou/solvability-explainer)!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}