docs/tutorials/training-usecase.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A complete use case"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "In this section we present a complete use case of manual training (without using the :py:mod:`~lambeq.training` package), based on the meaning classification dataset introduced in [Lea2021]_. The goal is to classify simple sentences (such as \"skillful programmer creates software\" and \"chef prepares delicious meal\") into two categories, food or IT. The dataset consists of 130 sentences created using a simple context-free grammar.\n",
    "\n",
    "We will use a :py:class:`.SpiderAnsatz` to split large tensors into chains of smaller ones. For differentiation we will use JAX, and we will apply simple gradient-descent optimisation to train the tensors.\n",
    "\n",
    ":download:`Download code <../_code/training-usecase.ipynb>`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preparation\n",
    "\n",
    "We start with a few essential imports."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings('ignore')  # Ignore warnings\n",
    "\n",
    "from jax import numpy as np\n",
    "import numpy\n",
    "\n",
    "from lambeq.backend.numerical_backend import set_backend\n",
    "set_backend('jax')\n",
    "\n",
    "numpy.random.seed(0)  # Fix the seed\n",
    "np.random = numpy.random "
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    ".. note::\n",
    "\n",
    "   Note the ``set_backend('jax')`` assignment in the above code. This is required to let :term:`lambeq` know that from now on we use JAX's version of ``numpy``.\n",
    "\n",
    "Let's read the datasets:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Input data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Read data\n",
    "def read_data(fname):\n",
    "    with open(fname, 'r') as f:\n",
    "        lines = f.readlines()\n",
    "    data, targets = [], []\n",
    "    for ln in lines:\n",
    "        t = int(ln[0])\n",
    "        data.append(ln[1:].strip())\n",
    "        targets.append(np.array([t, not(t)], dtype=np.float32))\n",
    "    return data, np.array(targets)\n",
    "\n",
    "train_data, train_targets = read_data('../examples/datasets/mc_train_data.txt')\n",
    "test_data, test_targets = read_data('../examples/datasets/mc_test_data.txt')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "nbsphinx": "hidden"
   },
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "TESTING = int(os.environ.get('TEST_NOTEBOOKS', '0'))\n",
    "\n",
    "if TESTING:\n",
    "    train_targets, train_data = train_targets[:2], train_data[:2]\n",
    "    test_targets, test_data = test_targets[:2], test_data[:2]\n",
    "    EPOCHS = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first few lines of the train dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['skillful man prepares sauce .',\n",
       " 'skillful man bakes dinner .',\n",
       " 'woman cooks tasty meal .',\n",
       " 'man prepares meal .',\n",
       " 'skillful woman debugs program .',\n",
       " 'woman prepares tasty meal .',\n",
       " 'person runs program .',\n",
       " 'person runs useful application .',\n",
       " 'woman prepares sauce .',\n",
       " 'woman prepares dinner .']"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Targets are represented as 2-dimensional arrays:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Array([[1., 0.],\n",
       "       [1., 0.],\n",
       "       [1., 0.],\n",
       "       [1., 0.],\n",
       "       [0., 1.],\n",
       "       [1., 0.],\n",
       "       [0., 1.],\n",
       "       [0., 1.],\n",
       "       [1., 0.],\n",
       "       [1., 0.]], dtype=float32)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_targets[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating and parameterising diagrams"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "First step is to convert sentences into :term:`string diagrams <string diagram>`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAzMAAAECCAYAAADKEomXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/SrBM8AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAwGklEQVR4nO3dd3wU9b7/8femFyA9pFBCkhMEAsoRRQxoBCSACDawQagiHFEURLEc4eoRKdYrCh66nHsfioriERClXkFUEJSq9HJJgUAKqZBkfn/4y96sKQSSMJnk9Xw89pHNN7Mzn918Z+f7nrJrMwzDEAAAAABYjJPZBQAAAADAlSDMAAAAALAkwgwAAAAASyLMAAAAALAkwgwAAAAASyLMAAAAALAkwgwAAAAASyLMAAAAALAkwgwAAAAASyLMAAAAALAkwgwAAAAASyLMAAAAALAkwgwAAAAASyLMAAAAALAkwgwAAAAASyLMAAAAALAkwgwAAAAASyLMAAAAALAkwgwAAAAASyLMAAAAALAkwgwAAAAASyLMAAAAALAkwgwAAAAASyLMAAAAALAkwgwAAAAASyLMAAAAALAkwgwAAAAAS3IxuwDUrMLCQqWmpiopKUnJyckOP11cXBQWFqbQ0FCFhYXZ7wcFBcnJiVwLazMMQxkZGQ59vuR+VlaWQkJCHPp/aGioQkND5eHhYXbpQLXl5+crOTm5TN9PSUlRkyZNyvT9sLAw+fr6ymazmV06UC3FxcU6c+ZMmb6flJSkwsLCcvt+06ZN5eLCELi+sBmGYZhdBC6tvJBSXmA5ffq0Sv9LnZ2dFRISotDQUBUWFio5ObnMNC4uLmratKlDwCH0oK6oLKT8ua2goMDhsX5+fgoLC1Pjxo3t609F01TW9wk9MEtFIeXPfT89Pd3hce7u7vZB2/nz58udxsPDo0yf/3PfJ/TALCUhpaI+X3I/NTVVhYWF9sfZbDYFBwcrNDRULi4u9lBfVFRUZpqK+jyhx1oIMyarbki51EYoMDBQzs7ODsu8ePGiTp8+XemG8VKhp7KVn9CDqqiJkFJZ3w8JCZGnp2eZZaanp1e6YSz5SehBbSoJKZcaqFUUUip7D64ogOTl5SklJeWSfb+iZRJ6UBNqIqRU1h+Dg4Pl6urqsMyioiKlpaVdsu9XJfSUt3xCj7kIM7XEjJBS0wg9uBKlQ8qlNlY1FVJq4zkQenAlzAgpNY3QgythRkipaYQeayLMXKbSIeVyBvh1KaTUNEJPw1AfQkpNq4nQU1nfJ/TUHfUhpNS06oae+via1EflhZTLGeDXhZBS0wg9dQth5v8jpNS+Kw09VX2NCT1XhpBS+wg9dVd5IYUBec0i9NRNhJTaV93Qc6nXmNDzh3ofZi5evKjU1NRLbqwIKXUHoadm1OQAmpByddT2/6whhR5CivUQempG6ZBS2TaUkFJ3EHqqx7JhhpCChhp6CCloyKGHkIKGGnoIKSD0lM+SYeaLL77QPffcQ0hBlVQn9OzevVtt2rQxsfqygoODdebMGYc2QgrKU53Q8+yzz2r69OkmVV6+yZMna8aMGQ5tVh+govZcaegJCgrS6dOnTaq6fPv371f79u0JKaiS6oSe5cuX66677jKv+CtgvfilPwanhmFo2bJlio6OJqSgUq6urgoPD1d4eHil05UOPRs3btQzzzyj4uLiq1Rl1RUUFGj48OEaOXIkIQWVstls8vf3l7+/v9q1a1fhdH++bmr48OG6cOHCVay0ai5cuKDw8HAtWrSIkIJL8vT0VKtWrdSqVatKpysdehYsWKDPPvvsKlVYdcXFxSoqKtLMmTMVHx9PSEGlnJ2d1bRpUzVt2lQdO3ascLrSoefQoUMaNGiQLl68eBUrrRmWDDMlevXqJR8fH7PLQD1ROvSU/tjIuig2NlZxcXFml4F6wmazyc/PT35+fmrXrl2dfl/18fHR7bffbnYZqEdKh54ff/yxToaZEl27dtUNN9xgdhmoJ0qHnsjISLPLuWJ174IAAAAAAKgCwgwAAAAASyLMAAAAALAkwgwAAAAASyLMAAAAALAkwgwAAAAASyLMAAAAALAkwgwAAAAASyLMAAAAALAkwowF2Gw2DRs27JLTTZ06VTabTceOHbO3LV68WDabTRs3brS3bdy4UTabTYsXL3Z4fFpamhITExUWFiabzab4+PjLrrW8GgAAAIDa4GJ2Aag7Jk6cqI8//lgvvPCCIiMj1bRpU7NLAgAAACpEmKlHXnzxRU2ePFnu7u5X9Phvv/1WCQkJeumll2q4MgAAAKDmcZpZPeLi4iIPDw/ZbLYrenxKSor8/f1ruCoAwKWcP3/e7BIuyQo1Amh4CDMmy8/P19SpU9W6dWt5eXnJ19dX7du316RJkyp93I4dOxQSEqK2bdvqxIkTkq78epWSxxmGoSVLlshms9mvqTl27JhsNpumTp1a4eO4Pga1peSar3Xr1unll19Wy5Yt5enpqc6dO+uHH36QJG3atEldu3aVt7e3QkND9corrzjM45tvvtH999+vyMhIeXp6ytfXV7169dKmTZvKLC8+Pl4RERFKSkrSgw8+KD8/P3l5eSkhIUEHDhy4Ks8Z1lTSV9euXaupU6eqZcuWcnd3V4cOHfTRRx85TBsREaH4+Hjt3LlTCQkJ8vHxUYcOHex/P3jwoIYMGaLQ0FC5ubkpIiJCkyZNUk5OjsN8hg0bJpvNpjNnzigxMVEBAQHy9vZWjx49tGPHjjI1vv/+++rVq5fCw8Pl5uam0NBQDR48uNz38JJrNdetW6euXbuqUaNGuvPOO+1/3759u+6++24FBgbK3d1drVu31quvvqrCwkKH+ezdu1cDBw5UeHi43N3dFRISottuu00rV668kpcZ9UxVxkAff/yx+vfvrxYtWsjd3V2BgYG66667tGvXrjLzq+ga4/KuH5akrKwsvfDCC2rTpo08PDwUEBCgrl27lllnk5OTNXbsWLVo0UJubm4KCwvT6NGjdfr06Rp5HVA9nGZmsscee0wLFy5UYmKiJkyYoMLCQh08eFDr16+v8DFr1qzRfffdpw4dOujf//53tY+m3HPPPYqOjtaQIUPUrVs3jR49WpJ08803V2u+QE2ZPHmyioqKNH78eF24cEFvvPGGevXqpQ8//FAjR47U6NGj9fDDD2vZsmV66aWX1KpVKw0ePFjSHxuxc+fOKTExUc2aNdOpU6c0f/589ejRQxs2bFC3bt0clpWTk6NbbrlFN910k6ZNm6ajR4/qnXfe0YABA7Rnzx45Ozub8RLAIp599lnl5OTob3/7myRp0aJFevDBB5Wfn+8wyDpx4oS6d++ugQMH6t5771V2drYk6eeff1b37t3l6+urRx99VOHh4fr111/1n//5n9qyZYs2bdokV1dXh2X27t1b/v7+mjp1qlJSUjR79mzdeuut2rp1q2JjY+3Tvf7667rpppv0xBNPyN/fX3v27NH8+fO1fv167d69WwEBAQ7z3b59uz777DM98sgjGjp0qL195cqV9u3GxIkT5e/vr61bt+qll17SL7/8ok8++USSdPbsWXXv3l2SNGbMGLVs2VJpaWnavn27fvzxR91xxx0198LDkqoyBpo9e7YCAgI0evRohYSE6PDhw/rnP/+puLg47dixQ3/5y1+uaNkZGRnq2rWr9u7dq/vuu09jx45VUVGRdu7cqa+++koPPPCApD/W1S5duujChQsaOXKkoqKidOjQIc2ZM0cbNmzQ9u3b5ePjUyOvB66QYUHLli0zJBkZGRlml1Jtfn5+Rp8+fSqdRpIxdOhQwzAM48MPPzRcXV2NAQMGGLm5uQ7TTZkyxZBkHD161N62aNEiQ5KxYcMGe9uGDRsMScaiRYsqXE6Jo0ePGpKMKVOmlKmrvOWV12ZF33//vSHJ2LNnj9mllNGkSRPjjTfeMLuMq6Kk/3bs2NEoKCiwt69YscKQZLi4uBjbtm2ztxcUFBghISHGTTfdZG/Lzs4uM9+UlBQjICCgzLp36623GpKMGTNmOLTPnDnTkGR8/fXXNfXU6rS2bdsaTz31lNlllPHUU08Zbdu2NbuMcpX01RYtWjhsmzIyMowWLVoYfn5+9vfsli1bGpKMefPmlZlPhw4djNatWxtZWVkO7cuXLy/zvj106FBDknH33XcbxcXF9vbt27cbNpvNSEhIcJhHeevC2rVry+3zkgxJxrfffuvQnpeXZzRt2tTo1q2bcfHiRYe/vfnmmw7bm5L19OOPPy6z3LrqjTfeMJo0aWJ2GWXs2bPHkGR8//33ZpdSo6oyBiqv3+7bt89wc3Mzxo4d69Be3jjGMMofC40dO9aQZHzwwQdlpi8qKrLf79+/vxEUFGScPHnSYZpt27YZzs7O5Y6PrCgjI8OQZCxbtszsUi4bp5mZzMfHR3v37tWePXsuOe306dM1dOhQjRgxQp999pk8PT2vQoWA+caOHSs3Nzf77yVHUzp37qxOnTrZ293c3HTjjTfq4MGD9jZvb2/7/ezsbJ09e1bOzs7q3LmzfvzxxzLLcnJy0hNPPOHQVrJ3ufR8gfKMHTvWYS+tj4+PxowZo/T0dIdTXPz9/TV8+HCHx+7evVu7du3SQw89pIKCAqWlpdlvJadSfvPNN2WW+cwzzzhcK3n99dfr9ttv19q1a+1HfKT/WxeKi4uVmZmptLQ0XXvttfLx8Sl3Xbj22mvVs2dPh7Zvv/1WqampGj58uDIyMhxq7Nu3ryTZayx5HVavXq2srKwqvX5oWKoyBirpt4ZhKCsrS2lpaQoKClLr1q3L7bdVUVxcrI8++kht2rSxn41SmpPTH8PjzMxMffXVV+rfv788PDwc+ntERISio6PLXSdxdRFmTPb2228rPT1d7du3V1RUlEaNGqUVK1aouLjYYbrly5frueee06hRozR37lxOdUGDEhkZ6fC7n5+fJKlVq1ZlpvXz89PZs2ftvx8+fFgPPPCA/Pz81LhxYwUGBiooKEirVq1Senp6mceHhYXJw8PDoa3k9JvS8wXK06ZNmzJtbdu2lSQdOXLE3hYVFVXmfXz//v2SpClTpigoKMjhFhwcrJycHKWmplZ5mUVFRTp+/Li9bf369YqPj5e3t7d8fX3t887MzCx3XYiJiSnTVlLjiBEjytR4zTXXSJK9xltvvVWJiYlavHixAgMDFRcXpylTpmjfvn1l5ouGqSpjoJ07d6pfv35q3LixfHx87P1t9+7d5fbbqkhLS1N6erquu+66Sqf7/fffVVxcrAULFpTp70FBQfr999/LXSdxdXHNjMkGDBigY8eOadWqVdq0aZPWrl2rBQsWqFu3blq7dq19b/SNN96oY8eO6dNPP9Xo0aMd9kbXpso+Ge3PF3oCtaWi8H6pUJ+dna1bbrlFOTk5evLJJ9W+fXs1btxYTk5Oeu2118q9Nq2yeRqGcXmFAxXw8vIq01bSvyZOnKjevXuX+7iSIH+5tm3bpl69eik6OlrTp09Xq1at5OnpKZvNpgceeKDMDrRL1Thr1qwKB4JhYWH2+0uWLNGkSZO0evVqfffdd3rjjTf06quv6u2339a4ceOu6Lmg/rjUGCglJUW33HKLmjRpor///e9q3bq1vL29ZbPZ9OSTTzoceazMlY5XSvr74MGDHa4bK42zZMxHmKkD/P39NXjwYA0ePFiGYWjy5MmaOXOmVqxYoYEDB0qSmjVrpiVLlqh79+7q2bOnvv76a910001XpTZJOnfuXJm/ld7LCNRF69atU1JSkhYuXFjmlJ4XX3zRpKpQn+3fv18DBgxwaCs5EvHnI4x/VnIhs7Ozc5nTuy61zD9vD/bt2ydnZ2e1bNlSkvTf//3fKioq0urVqx2OaObk5FzW3u2SGr29vatcY2xsrGJjYzVp0iRlZGSoc+fOmjx5sh577LEr/ioB1B+VjYGSkpKUnZ2tL7/8UrfddpvD486ePVvme/X8/f2rNF4JDAyUn5+ffv3110pri46Ols1m04ULFy5rncTVxWlmJioqKlJGRoZDm81mU8eOHSWVDRDh4eHatGmTwsLC1KtXL23ZsqXWa2zcuLFCQkK0fv16h73SR44c0RdffFHryweqo+Qoy5+PqHzzzTdXfK41UJk5c+YoMzPT/ntmZqbmzp0rX19f3XrrrZU+tmPHjoqNjdXcuXPL3VlUWFhY7kBt5syZDn18x44dWrt2rXr06KFGjRpJqnhdmDZtWrlHZSqSkJCg4OBgTZ8+vdxa8vLy7N9Hc+7cuTLz9vX1VatWrZSbm6v8/PwqLxf1T1XGQBX123nz5iklJaXMPGNiYrR161bl5uba29LT07Vo0SKH6ZycnPTggw9q3759WrBgQZn5lCwvICBAffv21fLly+1fB/Dn6c6cOVOFZ4vaxJEZE50/f16hoaHq37+/OnbsqODgYB09elRz5syRn5+fw2f6lwgJCdHGjRvVs2dP9e7dW1999dUlN5DVNW7cOL344ovq06eP7rrrLiUlJWnu3LmKjY3Vtm3banXZQHV07dpVISEhmjhxoo4dO6ZmzZrpl19+0dKlS9W+fXvt3r3b7BJRzwQGBqpz5872I4GLFi3SiRMnNH/+/HJP2yrNZrNp6dKl6t69uzp06KARI0aoXbt2ys3N1aFDh7R8+XK99tprZb5H4/jx40pISFD//v2VnJys2bNny9PTU7NmzbJPc/fdd+utt95S3759NXr0aLm5uenbb7/Vrl27FBgYWOXn5+3trQ8//FB33XWXWrdurREjRig6OloZGRn67bfftHz5cn3++eeKj4/Xhx9+qLfeekt33323oqOj5erqqk2bNmnNmjUaNGgQp+c0cFUZA+Xl5cnLy0tDhgzRuHHj5Ofnpy1btmjVqlWKiooqc/rYuHHjNHjwYHXv3l1DhgxRRkaG5s2bp5YtW5YJP//4xz+0fv16jRo1St988426du0qwzC0c+dOFRYWaunSpZL+2EHRtWtX3XLLLUpMTFTHjh1VXFysI0eOaMWKFUpMTCz3u/hw9RBmTOTl5aUnn3xS69ats3/qTMmK/dxzzzmcd1xacHCwNmzYoJ49e6pv37768ssv1aNHj1qr89lnn1VmZqaWLl2qjRs3qm3btlqwYIF+/vlnwgzqNF9fX61Zs0bPPPOM3n33XRUWFur666/XqlWrtGDBAsIMatyMGTP03Xff6b333lNqaqpiYmL0X//1X3rooYeq9PjrrrtOO3fu1GuvvaYvv/xSc+fOVePGjRUREaFhw4aV+17/9ddfa8KECZoyZYry8vJ00003adasWQ5fxBkXF6fPPvtMr7zyiv7+97/L09NTPXv21KZNm3TLLbdc1nNMSEjQtm3bNH36dP3rX//SmTNn5Ofnp6ioKE2YMMG+3JIvBv3qq6+UnJwsZ2dntWrVSq+//jrXy6DKY6DVq1fr+eef17Rp0+Ts7Ky4uDht2rRJ48aNK/OFrw8//LCSkpI0e/ZsTZgwQZGRkXrppZfk5ORU5mi8n5+ftm7dqmnTptlDeOPGjdW2bVs9/vjj9umaN2+un3/+WTNmzNCKFSv0r3/9Sx4eHmrevLnuvPNODRo0qNZfK1TOZljwitZPPvlEgwYNUkZGBl9UhFqxdetW3XzzzdqzZ4/atWtndjkOfHx8NGXKFE2YMMHsUlBPtWvXTgkJCXrzzTfNLsXBhAkTtGbNGu3du9fsUspYvHixhg8frg0bNig+Pv6qLHPYsGFasmQJH0xRg9588039x3/8h8OpgnXB3r17FRsbq++//15dunQxuxzUQ5mZmfL19dWyZcvs12tbBdfMAAAAALAkwgwAAAAASyLMAAAAALAkwgwAANU0bNgwGYZx1a6Xkf64TofrZQA0dIQZAAAAAJZEmAEAAABgSYQZAAAAAJZEmAEAAABgSYQZAAAAAJZEmAEAAABgSYQZAAAAAJbkYnYB1TFw4EC5urqaXQbqofT0dLNLqNS8efO0bt06s8tAPXX8+HGzS6jQ8ePHdccdd5hdBuqpI0eOmF1CpSZOnCg/Pz+zy0A9dPHiRbNLuGI2w4LfuHX+/HmNHz9eGRkZZpdSbSkpKdq6dav69OkjDw8Ps8upljNnzmjz5s3q1auXvL29zS6n2qKiojR9+nQ5OzubXYqDDz74QGvWrDG7DMs4duyYdu7cqbvvvtvsUizDyclJ06ZNU0xMjNmlODhw4ICef/55FRcXm12Kg9TUVH3//ffq3bu3PD09zS7Hweeff66OHTsqIiLC7FIcbNmyRa6urrrxxhvNLqWMhIQEPfroo2aX4aCoqEiTJ0/W4cOHzS7FMnbs2KGsrKyr+kW2Vufr66t33nlHjRs3NruUy2LJMFOfrFy5Uv369VNycrJCQkLMLqda1q9frx49eujw4cOKjIw0uxxAkrRgwQKNGjWKb0pHrVmzZo169+6tkydPqlmzZmaX48Bms2n+/PkaOXKk2aU4SEhIkI+Pj5YtW2Z2KainRo4cqX379mnr1q1ml4JaxjUzAAAAACyJMAMAAADAkggzAAAAACyJMAMAAADAkggzAAAAACyJMAMAAADAkggzsKRjx47JZrNp6tSpZpcCAKij2FYA9R9hBgAAAIAlEWYAAAAAWBJhBgAACzMMQ9nZ2WaXAQCmIMxYyOLFi2Wz2bR+/Xq9/vrrioqKkru7u2JiYrRkyRKzy7ss9em5oP6gX6K2VbePbdy4UTabTYsXL9Z7772ntm3bysPDQ6+//nq16srPz9fUqVPVunVreXl5ydfXV+3bt9ekSZOqNd/qYp1EbaOPWZ+L2QXg8j3//PPKy8vTo48+Knd3d82ZM0fDhg1TdHS04uLizC7vstSn54L6g36J2lbdPvb222/r7NmzeuSRRxQSEqLmzZtXq57HHntMCxcuVGJioiZMmKDCwkIdPHhQ69evr9Z8awrrJGobfcy6CDMWVFBQoG3btsnNzU2SdN999ykyMlKzZ8+23ApXn54L6g/6JWpbdfvYiRMn9Ntvvyk4OLhG6vn888/Vp0+fOrsnmnUStY0+Zl2cZmZBf/vb3+wrmySFh4crJiZGBw8eNLGqK1OfngvqD/olalt1+1hiYmKNBRlJ8vHx0d69e7Vnz54am2dNYp1EbaOPWRdhxoIiIyPLtAUEBOjs2bMmVFM99em5oP6gX6K2VbePxcTE1Gg9b7/9ttLT09W+fXtFRUVp1KhRWrFihYqLi2t0OVeKdRK1jT5mXYQZC3J2di633TCMq1xJ9dWn54L6g36J2lbdPubl5VWT5WjAgAE6duyYli5dqu7du2vdunW66667FB8frwsXLtTosq4E6yRqG33MuggzAABA/v7+Gjx4sObNm6cjR47omWee0XfffacVK1aYXRoAVIgwgxrj6empsLAw5eXl1eh8Sy50vXjxYo3OFw1DQUGBoqKizC4DuGxX672vqKhIGRkZDm02m00dO3aUJJ07d67GlxkcHCwnp5odgrCtQGmGYSgsLMzsMnAVEGZQY0JCQpSUlKTDhw/X6HwTExPVpk0bnTp1qkbni4bh4MGDKiwsNLsM4LJdrfe+8+fPKzQ0VPfff7+mT5+uhQsX6u9//7vGjRsnPz8/3XnnnTW+TA8PD+3bt69G58m2AqX99ttvcnV1NbsMXAWEGdSYiIgIRUVF6euvvza7FMBu9erV6tWrl9llAHWWl5eXnnzySR09elSzZs3S2LFjtXTpUvXv318//vhjrezdjo+P1+7du5Wamlrj8waysrL0008/6bbbbjO7FFwFNoMrm0y1cuVK9evXT8nJyQoJCTG7nGobP368PvroIx04cEA+Pj5ml4MG7vvvv1dcXJy++OILDRgwwOxyUE+tWbNGvXv31smTJ9WsWTOzy3Fgs9k0f/58jRw50uxSHKSkpCgsLExvvfWWxo8fb3Y5qGcWLVqkESNG6NChQ5xm3ABwZAY16umnn1Zubq5efPFFs0tBA3fx4kWNGTNGnTp1Ur9+/cwuB0ApISEhGj58uF5++WWlp6ebXQ7qkezsbL344osaNGgQQaaBIMygRjVv3lwvv/yy3nvvPX366adml4MGyjAMTZw4UXv37tUHH3xQ4UduAjDPq6++qgsXLmjMmDF15vtsYG2GYWj8+PE6e/asZsyYYXY5uEoIM6hxTzzxhB588EE98MAD+uyzz8wuBw2MYRh66qmn9O677+q9997TX//6V7NLAlCOkJAQLVmyRJ9++qkef/xxvs8D1fbcc89p4cKF+uc//6mIiAizy8FVQphBjXN2dtaSJUt0//33a9CgQZo0aZJyc3PNLgsNwPHjx3XHHXfonXfe0Zw5czRmzBizSwJQiXvuuUcffPCB3n//fQ0aNIhvW8cVyczMVGJiombMmKG33npLiYmJZpeEq4gwg1rh4uKiJUuWaNq0aZo9e7ZiY2P17bffml0W6qmioiK9/fbbateunXbv3q1///vfBBnAIkaNGqVly5Zp/fr1at++vVavXm12SbCQkn6zYsUKLVmyRE8++aTZJeEqI8yg1ri4uOjZZ5/V7t271apVK/Xq1UtDhgzR/v37zS4N9URxcbHWrVunLl26aMKECRo+fLj27t3LBf+AxQwcOFC7d+/Wtddeq759++rRRx/VyZMnzS4LdVhycrLGjx+vHj16KDo6Wrt27eKITANFmEGti46O1tq1a7Vw4UJ9/fXXatu2rW6++WbNnz9f58+fN7s8WNCJEyf08ssvKyoqSj179lR+fr62bNmid999V02aNDG7PABXICwsTKtWrdKcOXP00UcfKSIiQvfcc4/Wrl3L9TSQ9Mc1kZs2bdL999+vFi1aaMGCBXrzzTe1du1atWzZ0uzyYBLCDK4Km82m4cOH63//93+1bNky+fj4aPTo0faP59y8eTMbK1SqoKBAy5YtU+/evRUREaGZM2fqtttu0+bNm/Xrr7+qS5cuZpcIoJpsNpvGjBmjU6dO6b333tPBgwd1++23q02bNnrnnXeUkZFhdokwwfnz5zVnzhx16NBB8fHx+vXXX/XGG2/o1KlTeuqpp+TkxHC2IeO/j6vK3d1dAwcO1OrVq3X8+HFNnjxZmzZtUrdu3XTNNdfoH//4hzZv3qz8/HyzS0UdkJ6erpUrV+qJJ55QWFiY7r//fmVlZWnevHlKTk7WwoULFRcXJ5vNZnapAGpQo0aNNGbMGO3atUv/8z//o+uuu05PP/20wsPDNXr0aK1atUpZWVlml4lalJ2drTVr1uixxx5TeHi4Hn/8ccXExGjt2rXav3+/nnjiCb6cG5Ikm8HucFOtXLlS/fr1U3JyskJCQswuxxTFxcXatGmTFixYoC+++EI5OTlyc3NTp06dFBcXp7i4ON18880KCgoyu1TUIsMwdPjwYW3ZssV+27dvnyQpNDRUDz/8sEaMGKE2bdqYXCngaM2aNerdu7dOnjypZs2amV2OA5vNpvnz52vkyJFml1JtycnJmj9/vubPn68TJ07IyclJ119/veLj4xUfH6+uXbtymqmFZWdna8uWLdq4caM2btyobdu2qaioSGFhYRoxYoQeffTROrd+oW4gzJiMMOOosLBQu3btsg9mN2/erFOnTkmSYmJi7OEmLi5Of/nLX/gyRAvLz8/Xr7/+6hBeUlNTJUnt2rVz+F9HRkZy9AV1FmHm6irZ8VEy6N24caNOnTpVJtx06dJFfn5+ZpeLCmRmZuqHH34oE15CQkLs/8P4+HjFxMTw/o9KEWZMRpipnGEYOnHihMOAd9euXTIMQ66uroqIiFB0dLSioqLsP6OiotSqVSt5eHiYXX6Dl5mZqcOHD+vw4cM6dOiQw89Tp07JMAx5eHioc+fO9uDCAARWQ5gxV0XhRpKCg4PVunVrxcTEKCYmxn4/KipKbm5uJlde/128eFFHjhzRgQMH9Pvvvzv8TElJkSTCC6rNxewCgMrYbDa1bNlSLVu21EMPPSRJysrK0k8//aQDBw7YB8br16/X/Pnz7dfa2Gw2NWvWzCHgREVFKSQkRIGBgQoICJC/v79cXFgFrlR+fr7Onj2rs2fPKi0tTadOnSoTWNLS0uzT+/r62kNnXFycoqOj1bZtW3Xs2JFBBYArZrPZFB0drejoaI0aNcoebrZt22YfPP/yyy/6+OOPlZ2dLUlycnJSq1atHAJOZGSkgoKCFBwcrKCgILm7u5v8zOq+Cxcu6MyZMzp9+rTOnDmjo0ePOgSWI0eOqKioSJLk7e1tD5UloaVTp06EF1QbIzlYTpMmTdSzZ0/17NnTob24uFhJSUlljgTs2LFDn3zyiTIzM8vMy8/Pzx5uAgMDy9wv/buvr688PT3l4eEhd3f3evHmW1xcrPz8fOXn5ys3N1fnzp1TWlqaPaCU3Mr7vWRQUFpISIiio6N1zTXX6I477nA4aubv72/CMwTQ0JQON6UZhqGUlJQyRwlWrVql2bNnq7Cw0GH6xo0b24NN6ZDz57bAwEB5e3vbtw9W3DYYhqH8/Hzl5eUpNzdXaWlp9oBSOqyUvn/69OkyH8Lg7OxsD4n9+vVzCIthYWGWfG1Q9xFmUG84OTmpWbNmatasmW699VaHvxmGoXPnzunMmTPlDtBL7v/+++/2++fOnat0eR4eHvaNV+n75bWVvu/u7i5nZ2c5OTlVevPy8lJOTo6Ki4srvRUVFdkDSV5eXpn7lbVduHChwufn7OxcJtC1bNmywrDXtGlTNWrUqEb+lwBQ02w2m0JDQxUaGlpmG3Hx4kUlJSVVOHA/c+aMdu3aZb+fk5NT4XJK3u//fPPy8iq33dPTs8x2obxtRKNGjZSdnV3hdqDkfkFBgXJzc5WXl3fJW8l0lX2CqKenp0OQK7l+tbygFxYWxpF2XHWEGTQINptNAQEBCggIqPJjCgsLlZ6ebg83GRkZlx0YsrKylJqa6vC3goICh42QYRjlbpxuuOEGbd++XTab7ZLBp6Iw5evre8mgVfqnv7+/PaQ0adKEvWgAGgRXV1f7Kc1VkZubaw82aWlp9lBwqRBR8vdz5845tOfn51e4LSi5de7cWT/88EOlO8NsNpvc3d3LDU/+/v5VCldeXl4KDAy0BxVvb+9afvWB6iHMABVwcXGxv5kDAFDCy8vrssIPgNrDl2YCAAAAsCTCDAAAAABLIswAAAAAsCTCDAAAAABLIswAAAAAsCTCDAAAAABLIswAAAAAsCTCDAAAAABLIswAAAAAsCTCDAAAAABLIswAAAAAsCTCDAAAAABLIswAAAAAsCTCDAAAAABLIswAAAAAsCTCDAAAAABLIswAAAAAsCTCDAAAAABLIswAAAAAsCTCDAAAAABLIswAAAAAsCTCDAAAAABLIswAAAAAsCTCDAAAAABLIswAAAAAsCTCDAAAAABLIswAAAAAsCTCDAAAAABLIswAAAAAsCTCDAAAAABLIswAAAAAsCTCDAAAAABLIswAAAAAsCTCDAAAAABLIswAAAAAsCTCDAAAAABLIsyYrFGjRuratavy8/PNLgUAcAUKCgoUHx8vV1dXs0sBgAaHMGOypk2bavPmzfr555/NLgUAcAW2b9+uH374QUFBQWaXAgANDmHGZK1bt1b37t319NNPKycnx+xyAACX4dChQ5o1a5bGjh0rJyc2qQBwtfHOazKbzaYPPvhAKSkpGjt2rAoKCswuCQBQBWfPntXQoUMVEhKiV155xexyAKBBIszUAdHR0Zo7d64++ugjdenSRb///rvZJQEAKrFhwwZ16NBBv/32m5YuXSpvb2+zSwKABokwU0cMHTpUP/74o3JycvTXv/5Vs2bNUmZmptllAQBKOXXqlJ566in16NFDMTEx2rVrl7p27Wp2WQDQYBFm6pCOHTvq559/1tChQ/XCCy+oefPmmjhxok6cOGF2aQDQoP3yyy8aMmSIIiIitHDhQk2bNk1r165VeHi42aUBQINGmKljGjVqpPfff19Hjx7VY489poULFyoyMlL33nuvlixZotTUVLNLBIAG4fDhw5o9e7Zuu+02dezYUd99951mzpypkydPavLkyXJ2dja7RABo8GyGYRhmF4GKZWdna/HixVqyZIm2b98uSbr++uvVp08f9enTR507d2aDCgA1IC8vT5s2bdLq1au1evVqHTx4UK6urrrlllv0yCOP6N5775WLi4vZZV4Wm82m+fPna+TIkWaXAgC1gjBjIadPn9aaNWu0evVqrVmzRufOnZOfn58SEhLUp08fJSQkqGnTpmaXCQCWYBiGjhw5Yg8vGzZsUF5enpo3b66+ffuqT58+6t69uxo3bmx2qVeMMAOgviPMWFRRUZF++ukn+0a45KhNWFiYYmNjFRsbq/bt2ys2NlZt2rThk3YANGjp6enas2dPmdu5c+fk6uqqrl272gNM27ZtZbPZzC65RhBmANR3hJl6IjU1VRs3btTu3bvtG+kjR47IMAzZbDZFRkbaQ07JLSYmRm5ubmaXDgA1JicnR/v37y8TWk6dOiVJcnFxUevWre3vg9dee63i4+MtffSlMoQZAPUdYaYeK9molw44e/bsUVJSkqSyG/WSW8uWLeXq6mpy9QBQsfz8fB06dKhMaCnZiSPJvhOn5Ch1Q9yJQ5gBUN9Z60pGXBZvb2916tRJnTp1cmg/d+6c9u7dqz179tiDzjfffKP09HRJf2z8mjZtqrCwMIWHh1f409/fv96cigGgbigqKtKZM2d06tQpJSUlVfjz7Nmz9seEhoaqffv2GjBggD20tGnTRo0aNTLxmQAArgaOzEDSHxfCJicna8+ePTpx4kS5A4jTp087PMbd3V1hYWGXDD1eXl4mPSsAdYVhGMrKyrpkSElOTlZRUZH9cc7OzgoJCSnz3hIWFqZWrVopNjZWAQEBJj6zuo0jMwDqO47MQNIfG7ySAUJFLly4oJSUlAoHIrt27VJSUpKysrIcHufj42MfgAQFBcnX1/eSNx8fH7m7u9f20wZwmQzDUF5enjIzM5WRkXHJW2pqqv09Ijc312Fe/v7+9veG2NhY9erVq0xoCQ4O5uPnAQAVIsygytzc3NSiRQu1aNGi0unOnz+v5OTkcgNPSkqKfvvtN4fBTkUHBz08PC4ZeCr7u4eHR228DIClGYah3NzcSgPIpYLKxYsXy523i4tLmXU0LCxMN9xwQ5kjtqGhoayjAIBqI8ygxjVu3FiNGzdWTEzMJactLi5WdnZ2lQdTp0+f1oEDBxymK31KSmnu7u7y8fFRo0aN5OHhIU9PzzK38tovZ9qSdg8PDzk5OdX0S4kGoqioSPn5+crLy3O4lddWUXtVpi1Z1woLC8utw9XVtdwdAxEREVXameDl5cV1dACAq4owA1M5OTmpSZMmatKkySWP+JTHMAzl5ORUuhc5Jyen3IFdWlpapQPBivY+V8Td3b3S8OPh4SFXV1e5uLjIxcVFzs7O9vvl/X6l01Rn3k5OTrLZbOXeJF3230q3G4Zhv5X87/58q6i9sr8VFxersLDQfisqKqr096q21dQ0Fy9evGTgqE5fqyhkBwQElGnz9vaWn59fpUczCSMAACshzMDSbDabGjVqpEaNGqlZs2Y1Ou/L2Vtelb3iJfevZGBdUZtVdOvWTd99953ZZVRZSegsLwRebqh0dXVVYGBgjR0d5CggAAD/hzADVMDZ2Vne3t7y9vY2u5QK/fmoRHXDUXFxcY0dMSnd7u7urpEjR1720Zyq/K10iLjSI1el2wgKAABYB2EGsDAnJye5ubk1qC8BBAAAKMEuSAAAAACWRJgBAAAAYEmEGQAAAACWRJgBAAAAYEmEGQAAAACWRJgBAAAAYEmEGQAAAACWRJgBAAAAYEmEGQAAAACWRJgBAAAAYEmEGQAAAACWRJgBAAAAYEmEGQAAAACWRJgBAAAAYEmEGQAAAACWRJgBAAAAYEmEGQAAAACWRJgBAAAAYEmEGQAAAACWRJgBAAAAYEmEGQAAAACWRJgBAAAAYEmEGQAAAACWRJgBAAAAYEmEGQAAAACWRJgBAAAAYEmEGQAAAACWRJgBAAAAYEmEGQAAAACWRJgBAAAAYEmEGQAAAACWRJgBAAAAYEmEGQAAAACWRJgBAAAAYEmEGQAAAACWRJgBAAAAYEmEGQAAAACWRJgBAAAAYEmEGQAAAACWRJgBAAAAYEmEGQAAAACWRJgBAAAAYEmEGQAAAACWRJgBAAAAYEmEGQAAAACWRJgBAAAAYEmEGQAAAACWRJgBAAAAYEmEGQAAAACWRJgBAAAAYEmEGQAAAACWRJgBAAAAYEmEGQAAAACWRJgBAAAAYEmEGQAAAACW5GJ2AQAAoHYkJyerSZMmZpcBALXGZhiGYXYRAAAAAHC5OM0MAAAAgCURZgAAAABYEmEGAAAAgCURZgAAAABYEmEGAAAAgCURZgAAAABYEmEGAAAAgCURZgAAAABYEmEGAAAAgCURZgAAAABYEmEGAAAAgCURZgAAAABYEmEGAAAAgCURZgAAAABYEmEGAAAAgCURZgAAAABYEmEGAAAAgCURZgAAAABYEmEGAAAAgCURZgAAAABYEmEGAAAAgCURZgAAAABYEmEGAAAAgCURZgAAAABYEmEGAAAAgCURZgAAAABYEmEGAAAAgCURZgAAAABY0v8DCFxDFCYdsx0AAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 800x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Parse sentences to diagrams\n",
    "\n",
    "from lambeq import BobcatParser\n",
    "\n",
    "parser = BobcatParser(verbose='suppress')\n",
    "train_diagrams = parser.sentences2diagrams(train_data)\n",
    "test_diagrams = parser.sentences2diagrams(test_data)\n",
    "\n",
    "train_diagrams[0].draw(figsize=(8,4), fontsize=13)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "The produced diagrams need to be parameterised by a specific :term:`ansatz <ansatz (plural: ansätze)>`. For this experiment we will use a :py:class:`.SpiderAnsatz`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAzMAAAGjCAYAAAAYUkvcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/SrBM8AAAACXBIWXMAAA9hAAAPYQGoP6dpAAA88UlEQVR4nO3de3zP9f//8ft7mx2xg2GsNhnmsClKCRliE3LMVwcRHynlUzrol0+FDnTio4Pi04E+qT4dJJUORohK5RAZUhkRiWEOm80Or98fffb+eNvBzHi9nnvfrpfL+7Lt9X7t9X5sz+fz9X7dX6e3y7IsSwAAAABgGB+7CwAAAACAiiDMAAAAADASYQYAAACAkQgzAAAAAIxEmAEAAABgJMIMAAAAACMRZgAAAAAYiTADAAAAwEiEGQAAAABGIswAAAAAMBJhBgAAAICRCDMAAAAAjESYAQAAAGAkwgwAAAAAIxFmAAAAABiJMAMAAADASIQZAAAAAEYizAAAAAAwEmEGAAAAgJEIMwAAAACMRJgBAAAAYCTCDAAAAAAjEWYAAAAAGIkwAwAAAMBIhBkAAAAARiLMAAAAADASYQYAAACAkQgzAAAAAIxEmAEAAABgJMIMAAAAACMRZgAAAAAYiTADAAAAwEiEGQAAAABGIswAAAAAMBJhBgAAAICRCDMAAAAAjESYAQAAAGAkwgwAAAAAIxFmAAAAABiJMAMAAADASH52F1BRO3bsUEZGht1lwOEiIyMVExNjdxnnhJPHBO0AeGJMAP/DeHAOE9vCyDCzY8cONWvWTNnZ2XaXAocLDg7W5s2bjRuYp8vpY4J2ADwxJoD/YTw4h4ltYWSYycjIUHZ2tt544w01a9bM7nLgUJs3b9bgwYOVkZFh1KCsCCePCdoB8MSYAP6H8eAcpraFkWGmSLNmzdS6dWu7ywAcgzHhDLQD4IkxAfwP46FycQMAAAAAAEYizAAAAAAwEmEGAAAAgJEIMwAAAACMRJgBAAAAYCTCDAAAAAAjEWYAAAAAGIkwAwAAAMBIhBkAAAAARiLMAAAAADASYQYAAACAkQgzAAAAAIxEmAEAAABgJMIMAAAAACMRZgAAAAAYiTBzFrlcLt10002nnG/ixIlyuVzavn27e9prr70ml8ulZcuWuactW7ZMLpdLr732msfvZ2RkaMiQIapfv75cLpc6dep02rWWVAMAAADgZISZKuCee+7RO++8o1tvvVVz5szRAw88cE5fv7CwUNOmTVPTpk0VGBio888/X/fcc4+ysrLOaR1wvtdff12tWrVSUFCQ6tatqxEjRmjfvn12lwXQN4ET/Otf/9INN9ygpk2bytfXVy6Xy+6S4ABO3d4jzDjAgw8+qGPHjik2NrZCv79o0SKlpKRo/PjxGjx4sLp161bJFZbtrrvu0t13363mzZvr+eef18CBA/Xcc8/p6quvVmFh4TmtBc715ptvaujQoQoNDdWzzz6rW265RW+//bY6depk+4oQ3m3atGn0TeAEjz/+uD766CPVqVNH9evXt7scOIRTt/f8bHtluPn5+cnPr+JNsWfPHkVERFRiReW3ceNGPf/88+rfv7/ef/999/QLLrhAd9xxh95++21df/31ttQGZ3nxxRfVpk0bffHFF/L19ZUktWnTRr1799azzz6rf/zjHzZXWDUdOXJENWrUsLuMMtlZY0ZGhh588EH6phdhTJzasmXLFBMTIx8fH/Xq1Uu///67bbXAGZy8vceRmQrKycnRxIkTFR8fr+DgYIWFhSkxMVFjx44t8/fWrl2rqKgoNW/eXDt27JBU8etVin7Psiz9+9//lsvlcl9Ts337drlcLk2cOLHU36uM62P+85//yLIsjRkzxmP6zTffrODgYL3xxhtn/Br43zVUX3zxhR555BHFxsYqKChIl112mb799lv3fMOHD1dISIjq1aunRx991GMZqampGjRokBo2bKigoCCFhYUpOTlZX375ZbHX69Spkxo0aKDdu3fruuuuU3h4uIKDg5WSkqKff/65Qn9DTk6O/v73v7s3FiXp6quvVsOGDeknZShq+8WLF2vixImKjY1VQECAWrZsqbfffttj3gYNGqhTp0764YcflJKSotDQULVs2dL9/C+//KIbb7xR9erVk7+/vxo0aKCxY8cWO/pw0003yeVyad++fRoyZIhq1aqlkJAQXXnllVq7dm2xGl988UUlJycrOjpa/v7+qlevngYPHlziOqboWsIvvvhCHTp0UPXq1XX11Ve7n1+9erX69eunyMhIBQQEKD4+XpMmTVJ+fr7HcjZu3KiBAwcqOjpaAQEBioqKUufOnfXJJ5+c1v93/vz5ys7Opm8ahDFxdsdE0f/Nx4dNxNKUZxvwnXfeUe/evRUTE6OAgABFRkbq7rvvLnF5pV1jXdL105J0+PBhPfDAA2rWrJkCAwNVq1YtdejQoVj//+OPPzRq1CjFxMTI399f9evX18iRI7V3797T/pudvL3HkZkKuv322zVr1iwNGTJEd999t/Lz8/XLL79oyZIlpf7OwoULdc0116hly5b6+OOPz/hoSv/+/dWoUSPdeOONuuKKKzRy5EhJUrt27c5ouadj1apV8vHx0aWXXuoxPTAwUBdddJFWrVp1zmrxBvfff78KCgp055136vjx45o6daqSk5M1YcIESVKrVq10yy236N1339X48eN1wQUXaPDgwZL+WikeOHBAQ4YM0Xnnnaddu3bplVde0ZVXXqmlS5fqiiuu8HitrKwsdezYUW3bttXkyZO1bds2Pfvss+rTp4/S0tI8NvzK6/LLLy82rW3btvrPf/6jo0ePqnr16hX4r3iH//f//p+ysrJ02223SZJmz56t6667Tjk5OR5vgjt27FCXLl00cOBADRgwQEePHpUkrVmzRl26dFFYWJhuueUWRUdHa/369Xruuef09ddf68svv1S1atU8XrN79+6KiIjQxIkTtWfPHk2fPl1JSUlauXKlEhIS3PNNmTJFbdu21R133KGIiAilpaXplVde0ZIlS7RhwwbVqlXLY7mrV6/W+++/r5tvvllDhw51T//kk0/c67V77rlHERERWrlypcaPH69169bpvffekyTt379fXbp0kSTdeuutio2NVUZGhlavXq3vvvtOPXv2LPf/tWgdRd80D2Pi7IwJnFp5tgGnT5+uWrVqaeTIkYqKitLWrVs1Y8YMSX/1ydatW1fotTMzM9WhQwdt3LhR11xzjUaNGqWCggL98MMPWrBgga699lr3a1x++eU6fvy4/va3vykuLk6//vqrZsyYoaVLl2r16tUKDQ0t9+s6envPMtCaNWssSdaaNWtsqyE8PNy66qqrypxHkjV06FDLsizr9ddft6pVq2b16dPHys7O9phvwoQJliRr27Zt7mmzZ8+2JFlLly51T1u6dKklyZo9e3apr1Nk27ZtliRrwoQJxeoq6fVKmlYeCQkJVp06dUp8buDAgZYkKzc397SWWVmc0E8qS1F/aNWqlcf/88MPP7QkWb6+vh5/a25urhUVFWW1bdvWPe/Ro0eLLXfPnj1WrVq1ivXlpKQkS5L15JNPekx/6qmnLEnW559/Xu7ai9pBUrG+b1mWNXbsWEuStWXLlnIv06nORp8ravuYmBgrMzPTPT0zM9OKiYmxwsPD3f/X2NhYS5L18ssvF1tOy5Ytrfj4eOvw4cMe0+fNm1dsvTJ06FBLktWvXz+rsLDQPX316tWWy+WyUlJSPJZRUt9avHhxiX2oqC8sWrTIY/qxY8esunXrWldccYWVl5fn8dw///lPj/VhUb9/5513ir3u6erVq9c575tVad10KoyJ/zFlTJysZ8+e1tncXDRxPJRnG7CkPjB37lxLknXNNdd4TC9pO86ySt4WHDVqlCXJ+te//lVs/oKCAvf3vXv3tmrXrm3t3LnTY55Vq1ZZvr6+JW4fltUWTt7e4xhiBYWGhmrjxo1KS0s75bxPPPGEhg4dquHDh+v9999XUFDQOajw3MjOzlZAQECJzwUGBrrnQeUYNWqU/P393T8XHU05cY+gJPn7++vSSy/VL7/84p4WEhLi/v7o0aPav3+/fH19ddlll+m7774r9lo+Pj664447PKYV7fk7cbmno6S+Qj8pn1GjRnnsRQsNDdWtt96qgwcPepyCEBERoWHDhnn87oYNG/Tjjz/q+uuvV25urjIyMtyPDh06KCQkRKmpqcVe87777vO4i9HFF1+sbt26afHixe6929L/+lZhYaEOHTqkjIwMXXjhhQoNDS2xb1144YXq2rWrx7RFixbpzz//1LBhw5SZmelRY48ePSTJXWPR/+Gzzz7T4cOHy/X/K01Rv6NvmocxcXbGBE6tPNuARX3AsiwdPnxYGRkZCgsLk6RybTuWpLCwUG+//baaNWvmPhvnREWnBh46dEgLFixQ7969FRgY6NF3GjRooEaNGpXYv8vi5O09wkwFPfPMMzp48KASExMVFxenESNG6MMPPyx2N4d58+Zp3LhxGjFihGbOnFmhU3OcLDg4WLm5uSU+l5OT454HlaNhw4YeP4eHh0uSoqOji80bHh6u/fv3u3/eunWrrr32WoWHh6tGjRqKjIxU7dq19emnn+rgwYPFfr9+/fruFVSRolMjTlzu6Sipr9BPyqdZs2bFpjVv3lySlJ6e7p4WFxdXbD2zefNmSdKECRNUu3Ztj0edOnWUlZWlP//8s9yvWVBQoN9++809bcmSJerUqZNCQkIUFhbmXvahQ4dK7FtNmjQpNq2oxuHDhxersWnTppLkrjEpKUlDhgzRa6+9psjISLVv314TJkzQpk2bii33VIr6HX3TPIyJszMmcGrl2Qb84Ycf1KtXL9WoUUOhoaGqXbu2O7AeOXKkQq+bkZGhgwcP6qKLLipzvi1btqiwsFCvvvpqsb5Tu3ZtbdmypcT+XRYnb+9xzUwF9enTR9u3b9enn36qL7/8UosXL9arr76qK664QosXL3bvPb/00ku1fft2zZ07VyNHjtQll1xyTuor657wJ180eCbq16+vTZs2KTc3t1hi37VrlyIjIz2OJODMlBaGT3Wh5tGjR9WxY0dlZWVpzJgxSkxMVI0aNeTj46PHH3+8xGu9ygrelmWdXuH/tWvXLjVq1KjYNJfLxe0/K0lJbyZF7XXPPfeoe/fuJf5eUTA+XatWrVJycrIaNWqkJ554QhdccIGCgoLkcrl07bXXlni7zrJqfPrpp0t9oz6xj/z73//W2LFj9dlnn2nFihWaOnWqJk2apGeeeUajR48ud/1Fy6RvVl2MidMbEzi1U20D7tmzRx07dlTNmjX10EMPKT4+XiEhIfr111912223lfs2xhXdXivqO4MHD/a4ButEp3uWkJO39wgzZyAiIkKDBw/W4MGDZVmW7r//fj311FP68MMPNXDgQEnSeeedp3//+9/q0qWLunbtqs8//1xt27Y9J7VJ0oEDB4o9d+IeqzPVpk0bpaam6vvvv/e4gDwnJ0fr1q1Tx44dK+21UHFffPGFdu/erVmzZhU73eLBBx88Z3WsXLmy2Abjt99+q/j4eC6wPoXNmzerT58+HtOK9rqefMTuZI0bN5b0V0A9+VSWU73myeurTZs2ydfX1/25WG+99ZYKCgr02Wef6YILLnDPl5WVVeIe6FPVGBISUu4aExISlJCQoLFjxyozM1OXXXaZ7r//ft1+++3l/pC/Nm3a6KWXXqJvGogxUVxljAmUT1nbgLt379bRo0f10UcfqXPnzu7fOfnGDycuqzzba5GRkQoPD9f69evLrK1Ro0ZyuVw6fvz4afXvsjh5e4/TzCqgoKBAmZmZHtNcLpdatWolqXiAiI6O1pdffqn69esrOTlZX3/99VmvsUaNGoqKitKSJUs89qKnp6dr/vz5lfY6gwYNksvl0jPPPOMx/eWXX1Z2drZuuOGGSnstVFzRUZaTj6ikpqaWeP722RAQEKDp06eroKDAPe3jjz9Weno6/aQcZsyYoUOHDrl/PnTokGbOnKmwsDAlJSWV+butWrVSQkKCZs6cWeLOjPz8/BLfSJ966imPPrN27VotXrxYV155pXsDv7S+NXny5NP6ELWUlBTVqVNHTzzxRIm1HDt2zH1qxoEDB4otOywsTBdccIGys7PdpzyUR58+fRQUFETfNBBj4uyMCZStPNuApfWBefPmlbjMJk2aaOXKlR7XnBw8eFCzZ8/2mM/Hx0fXXXedNm3apFdffbXYcoper1atWurRo4fmzZvn8fENJ863b9++U/ylnpy8vceRmQo4cuSI6tWrp969e6tVq1aqU6eOtm3bphkzZig8PNzj/vBFoqKitGzZMnXt2lXdu3fXggULTrmyPVOjR4/Wgw8+qKuuukp9+/bV7t27NXPmTCUkJFTaLfQSExN1++23a/r06erfv7969OihzZs367nnnlNSUhIfmOkQHTp0UFRUlO655x5t375d5513ntatW6c5c+YoMTFRGzZsOOs1jBo1Ss8884y6du2q6667Trt27dLUqVPVtGnTYvetR3GRkZG67LLL3EfWZs+erR07duiVV1455XnKLpdLc+bMUZcuXdSyZUsNHz5cLVq0UHZ2tn799VfNmzdPjz/+eLHPOfjtt9+UkpKi3r17648//tD06dMVFBSkp59+2j1Pv379NG3aNPXo0UMjR46Uv7+/Fi1apB9//FGRkZHl/vtCQkL0+uuvq2/fvoqPj9fw4cPVqFEjZWZm6qefftK8efP0wQcfqFOnTnr99dc1bdo09evXT40aNVK1atX05ZdfauHChfq///u/0zp9onbt2nr00Ud177330jcNw5g4O2NC+ivMF+39//XXXyVJjz32mKS/QpI3n7ZWnm3AY8eOKTg4WDfeeKNGjx6t8PBwff311/r4449LXObo0aM1ePBgdenSRTfeeKMyMzP18ssvKzY2Vnv27PGY97HHHtOSJUs0YsQIpaamqkOHDrIsSz/88IPy8/M1Z84cSX+F/Q4dOqhjx44aMmSIWrVqpcLCQqWnp+vDDz/UkCFDSvwswtI4envvHN89rVLYfRu/3Nxc6/7777fatGljRUREWP7+/lZsbKw1bNgw6+eff3bPpxJutZeRkWFddNFFVnBwsLV48WLLss7OrZkty7Ly8vKssWPHWlFRUVZAQIDVqlUr66OPPqrUWzNblmXl5+dbU6ZMsZo0aWL5+/tb9evXt+666y7ryJEjp72symR3P6lMJfWHIpLct5c98W8tupVokfXr11spKSlWWFiYVb16dSspKclavnx5sfks669bM8fGxhZ7rbJu+V2aE9th9uzZVsuWLa2AgACrdu3a1rBhw6w///yz3MtyurN5G9pFixZZ48ePt84//3zL39/fSkhIsN58802PeWNjY62kpKRSl7V9+3brlltusWJjY61q1apZERERVuvWra3777/f2rFjh3u+oj6xd+9ea/DgwVZERIQVFBRkde7c2Vq9enWx5X7wwQdW69atreDgYKtWrVrWoEGDrN9++63EekpbXxXZsGGDdcMNN1j169e3qlWrZtWpU8e6/PLLrUceecTav3+/ZVmW9cMPP1hDhgyx4uLirODgYKtGjRpWy5YtrSlTplg5OTmn/qeW4Fz2zaq0bjoVxoSZY6Lo7y3pUdJ7w5kwbTyUdxvwyy+/tNq3b29Vr17dCg0NtXr06GG98847liSrXr16xZb71FNPWTExMZa/v7/VtGlT69VXXy31vf/gwYPW2LFjrbi4OHe/7dChQ7Fbc+/bt8+69957rcaNG1sBAQFWaGiolZCQYN1xxx3Wxo0bi9VwqrZw6vYeYQZVljf1Eyf/rU6urbKdzQ23koLs2VJSwEXlYUycGcZE1cJ4cA6n11carpkBAAAAYCSumUExx48fL/Fiw5PVrl27yn1uDsqvoKCgXBcQ5uXlnYNqgP8pb9+MiIjg1vHwCowJlOTk7b2MjAz31xOv1XH69h5hBsV88803HrcSLM22bdvUoEGDs18QHGnnzp0etx0tzb/+9a9zUA3wP+Xtm0uXLlWnTp3OfkGAzRgTKElp23spKSkePzt9e48wg2IuvPBCLVq06JTzRUVFnYNq4FRRUVHl6id+fqxmzsRNN91U7I5KZ9trr72m11577Zy+ZmUqb9+88MILz0E1qGyMidPHmEBJTt7e++WXX3TbbbfpxRdfdH/OkeT87T22MlBMeHh4pX3IEqquwMDAcvWTtWvXnoNqgP8pb98EvAVjAiU5eXuv6APXL7vsMrVu3dqusk4bNwAAAAAAYCTCDAAAAAAjEWYAAAAAGIkwAwAAAMBIhBkAAAAARiLMAAAAADASYQYAAACAkQgzAAAAAIxEmAEAAABgJMIMAAAAACMRZgAAAAAYiTADAAAAwEiEGQAAAABG8rO7gDOxefNmu0uAg3lj/3Di3+zEms42b/ybUX7e2D+88W9G+Xhj33Dq3+zUuk7FyDATGRmp4OBgDR482O5S4HDBwcGKjIy0u4yzzuljgnYAPDEmgP9hPDiHiW3hsizLsruIitixY4cyMjIqZVn33nuvcnJyNH369EpZHipm3rx5mjRpktasWVNpy4yMjFRMTEylLc/JKmtM7NmzRz179tT06dN1+eWXV0JltENFDRgwQO3atdM999xTKctDxUydOlXffPON3n///UpbJmPi9K1cuVKjR4/WJ598oqioqEqoDBV18cUX64EHHlD//v0rZXmMh4oZPXq0AgMDNWXKlEpZnmRmWxh5ZEaSYmJiKu2fHR4eruzsbLVu3bpSloeKKQoxtEPFVNaY2LlzpySpcePGtEUFVOa6KSgoSHXr1qUdbFa3bl0FBQXRDhVUWWOiaAMwMTFR559//hkvD2cmNjaWMVEBlfkeERoaquDgYK9vB24AAAAAAMBIhBkAAAAARiLMAAAAADASYQYAAACAkQgzAAAAAIxEmAEAAABgJMIMAAAAACMRZgAAAAAYiTADAAAAwEiEmTL8/PPPGj9+vNq2bavatWurRo0auuiiizRp0iRlZWXZXZ5XoS2cgXZwBtrBGWgH56AtnIF2cAZvawfCTBlmzZqladOmKS4uTuPHj9fTTz+t+Ph4Pfjgg2rXrp2OHTtmd4leg7ZwBtrBGWgHZ6AdnIO2cAbawRm8rh0sWP3797e6d+9ebPqqVauszMzMYtMfeOABS5L1/PPPn4vyvMZLL71kldYlaYtzZ8eOHZYka+HChcWeox3OrWbNmll33XVXsem0w7l11113Wc2aNSs2nXY4txYuXGhJsnbs2FHsOdri3JJkvfTSS8Wm0w7nVvfu3a3+/fsXm+5t7cCRmTJccsklCg0NLTZ90KBBkqS0tLRzXZLXoi2cgXZwBtrBGWgH56AtnIF2cAZvawfCTAX8/vvvkqS6devaXAloC2egHZyBdnAG2sE5aAtnoB2coaq2A2HmNBUUFOjRRx+Vn5+frr/+ervL8Wq0hTPQDs5AOzgD7eActIUz0A7OUJXbwc/uAkwzZswYrVy5UpMnT1Z8fLzd5Xg12sIZaAdnoB2cgXZwDtrCGWgHZ6jK7cCRmdPw0EMPafr06Ro5cqTGjRtndzlejbZwBtrBGWgHZ6AdnIO2cAbawRmqejsQZspp4sSJeuyxxzRs2DDNnDnT7nK8Gm3hDLSDM9AOzkA7OAdt4Qy0gzN4QzsQZsph4sSJevjhhzV06FC98sorcrlcdpfktWgLZ6AdnIF2cAbawTloC2egHZzBW9qBMHMKjzzyiB5++GHdeOONmjVrlnx8+JfZhbZwBtrBGWgHZ6AdnIO2cAbawRm8qR24AUAZXnjhBU2YMEExMTHq2rWr3nrrLY/n69atq27dutlUnXehLZyBdnAG2sEZaAfnoC2cgXZwBm9rB8JMGVatWiVJ2rFjh4YOHVrs+aSkpCrVGZyMtnAG2sEZaAdnoB2cg7ZwBtrBGbytHaruMadK8Nprr8myrFIfy5Yts7tEr0FbOAPt4Ay0gzPQDs5BWzgD7eAM3tYOhBkAAAAARiLMAAAAADASYQYAAACAkQgzAAAAAIxEmAEAAABgJMIMAAAAACMRZgAAAAAYiTADAAAAwEiEGQAAAABGIswAAAAAMBJhBgAAAICRCDMAAAAAjESYAQAAAGAkwgwAAAAAIxFmAAAAABiJMAMAAADASIQZAAAAAEYizAAAAAAwEmEGAAAAgJEIMwAAAACMRJgBAAAAYCTCDAAAAAAjEWYAAAAAGIkwAwAAAMBIhBk4hp+fn+rUqWN3GV6voKBAderUka+vr92lAI4QGBiomjVr2l2G17Msi/cIh6AdnCEoKEghISF2l2E7wgwcw8/PT3v37lVOTo7dpXi1nJwc7d27V/7+/naXAjjGn3/+aXcJXi87O1t79+5VUFCQ3aV4vSNHjig7O9vuMrze/v37ZVmW3WXYjjADxwgPD5ckZWZm2luIlyv6/4eFhdlaB+AU4eHhrJccoKgNQkND7S0ECgsLY0w4QGZmJu/VIszAQYoGJCtIexFmAE9hYWE6dOiQCgsL7S7Fq2VmZiokJETVqlWzuxSvR5hxBsLMXwgzcIy4uDhJ0rp16+wtxMutW7dOISEhqlu3rt2lAI7QqFEjWZalDRs22F2KV1u3bp0aNWpkdxnQX2OC92p7ZWRkaOfOnYwJEWbgIPXq1VNiYqJSU1PtLsWrpaamqnPnzlwzA/xXu3btFBwczLrJRpZlKTU1VcnJyXaXAknJycn6+uuvdfToUbtL8VqLFy+WZVnq1q2b3aXYjjADR0lOTlZqaioXtNkkKytLX331FRsMwAkCAgLUqVMnwoyN0tLStGfPHtZNDpGcnKy8vDx9+eWXdpfitVJTU5WQkKD69evbXYrtCDNwlJSUFO3atUubNm2yuxSvtGzZMuXl5SklJcXuUgBHSUlJ0YoVK7iDk00WLlyooKAgdejQwe5SIKlx48Zq0KCBFi5caHcpXqnoSCXv1X8hzMBROnTooMDAQPaA2iQ1NVWxsbFq3Lix3aUAjpKcnKzc3FwtX77c7lK8UmpqqpKSkhQYGGh3KZDkcrncZ1Lg3Nu8ebN27drFkcr/IszAUYKCgtSxY0e99dZb3DnoHMvOztYHH3yg5ORkuVwuu8sBHCU+Pl7nn3++3nzzTbtL8To7d+7U8uXL2XBzmOTkZG3ZskWrV6+2uxSv88YbbygwMFBXXHGF3aU4AmEGjvPQQw9p9erVmjlzpt2leJWJEydq7969Gjt2rN2lAI7jcrk0fvx4vfHGG/riiy/sLsdrWJal2267TbVq1dLw4cPtLgcn6N27ty688EKNHDlS+fn5dpfjNTZu3KgpU6Zo7NixfIDsfxFm4DgdOnTQLbfcovvvv187d+60uxyvsHbtWk2dOlUTJ07kFDOgFH/729+UlJSkW265hWtnzpF3331XCxYs0IsvvsiHZTpMtWrV9PLLL2v9+vWaNm2a3eV4hYKCAo0YMUJxcXF64IEH7C7HMQgzcKQnn3xS1atX1+23386dzc6y/Px8jRgxQomJibrnnnvsLgdwLJfLpZdeekm///67Hn74YbvLqfIOHDigO+64QwMGDFCfPn3sLgclaNOmje68806NHz9eW7dutbucKm/GjBn69ttv9fLLLysgIMDuchyDMANHCg0N1QsvvKCPP/5Y7733nt3lVGnTpk3T+vXr9fLLL/PJ2sApNGnSROPHj9fUqVP1ww8/2F1OlXbvvfcqNzdXzz//vN2loAyPPvqooqKidMstt7Dz8SzauXOnxo0bp1tvvZW7+p2EMAPH6tevn/r3769hw4ZpwYIFdpdTJb3yyisaN26cxowZozZt2thdDmCEsWPHqkWLFurZs6fWrl1rdzlVTmFhoe6//37Nnj1bU6ZMUb169ewuCWUICQnRzJkz9cUXX+imm27S8ePH7S6pyklPT1e3bt0UGhqqJ554wu5yHIcwA0ebM2eOkpOT1adPH7344ot2l1NlFBYW6oEHHtDNN9+skSNH6sknn7S7JMAY1apV08KFCxUdHa2OHTvq008/tbukKiMnJ0fXX3+9nnrqKU2dOlV/+9vf7C4J5ZCSkqI333xTb7/9trp3767MzEy7S6oyvvvuO7Vt21YFBQVaunQp146VgDADRwsODtbcuXP197//XbfffrvGjh3LLZvPUG5urgYPHqzJkyfr6aef1gsvvCA/Pz+7ywKMEhUVpWXLlunKK6/U1Vdfzd0XK8H+/fvVrVs3ffjhh3rvvfd09913c5t4g1x//fVatGiR1q1bp/bt2+u3336zuyTjffDBB+rcubMaN26slStXcoOeUhBm4Hi+vr565pln9Mwzz2jq1KkaNGiQsrKy7C7LSBkZGUpOTta8efP07rvv6t5772VjAaigkJAQzZs3T7fffrtGjRql++67j1vUVtAvv/yidu3a6aefftKSJUs0YMAAu0tCBXTs2FHffPONjh07prZt2+r777+3uyQjWZaladOmacCAAerVq5cWL16syMhIu8tyLMIMjHHnnXdq3rx5+uSTTxQXF6fp06crNzfX7rKMcOTIET366KOKi4vTxo0btWTJEg0cONDusgDj+fr66rnnntO0adM0ZcoUtWjRQu+88w5HkMtpz549uuOOO5SQkCDLsrRy5UpdfvnldpeFM9C0aVOtXLlSMTExatu2rYYMGaL09HS7yzKCZVn6/PPP1aZNG919992699579fbbb/N5MqdAmIFR+vbtq02bNumqq67SnXfeqSZNmmj27NnsDS1FTk6Opk2bpoYNG+qxxx7T8OHDtWnTJrVr187u0oAqZcyYMVqzZo0aNWqka6+9Vq1bt9Ynn3zC3Z1KceDAAY0bN05xcXGaM2eOJkyYoLVr16pRo0Z2l4ZKULduXa1YsULTp0/XokWLFB8fr1GjRmnXrl12l+ZYK1asUFJSkq666ioFBARo6dKleuqpp+Tjw6b6qfAfgnEaNGig2bNnKy0tTZdeeqmGDx+uhIQEvfvuu+wN/a+8vDy99NJLatSokcaOHat+/frp119/1bRp01SnTh27ywOqpFatWumTTz7RV199pdDQUPXq1Uvt27fXsmXL7C5N+fn5yszMtH3Hz9GjRzVp0iQ1bNhQzz//vMaMGaP09HT94x//UPXq1W2tDZXL399ft912m7Zu3apJkybp3Xffdb8nZWRk2F2eY6xdu1Y9evRQx44ddeTIEfc6pFOnTnaXZgzCDIzVrFkzvffee1qzZo0aNmyoQYMGKSYmRqNHj9bixYuVl5dnd4nn1LFjx/TRRx9p+PDhql+/vm699VYlJSVp8+bNeumll3T++efbXSLgFYoCzMKFC5WXl6fOnTuradOmGjdunL777rtzttOlsLBQCxYsUPfu3eXv76/w8HD5+/ure/fuWrBgwTmr4+DBg3rzzTc1cOBARUVF6ZFHHtFNN93k3sgNDw8/J3XAHsHBwbrvvvuUnp6u++67TzNnzlR0dLR69uypV155RX/++afdJZ5TlmXpxx9/1COPPKLWrVvr4osvVnp6ut59912tWbNGPXr04FrW0+SyOAauAQMGKDs7W5999pndpeAMfPfdd3r77bf1wQcf6LffflNYWJh69uypvn37qnv37lVyr9+BAwe0YMECzZ8/XwsXLlR2draaNWumfv366frrr1eLFi3sLhFnoHnz5urevbv++c9/2l0KKsiyLC1cuFBz587VRx99pH379qlevXrq06eP+vbtq86dO8vf37/SXzcrK0vXXHONPv/8c/n6+qqgoMD9XNHP3bt319y5cxUSElLpr//777/rww8/1Pz587Vs2TLl5+fr0ksvVd++fTV48GB2rnixjIwMvfnmm/rggw+0YsUKWZaldu3aqW/fvurbt2+VPNWwoKBAX3/9tebPn6/58+dr27Ztqlmzpnr27Kn+/furb9++3FX0DBBmRJipaizL0vr1690rjfXr1ysgIEBdunRRmzZtlJiYqISEBDVq1Miolcfx48e1ZcsWbdiwQWlpafr222+1fPlyFRQU6PLLL1ffvn3Vp08fxcfH210qKglhpmopKCjQN9984143paenq2bNmkpOTlarVq3c66bY2NgzOk++sLBQPXv2VGpqaplHX3x8fJSSkqIFCxac0etlZ2dr06ZN7nXTihUrtGrVKvn5+alLly7q27evevfurejo6Aq/BqqmjIwMjx1yOTk5atGihTp37uweDwkJCapZs6bdpZabZVnavXu3ezysW7dOCxcuVEZGhurVq+cObZ06dTorOzK8EWFGhJmqbtu2bfrwww/16aef6scff3Qf0g4ICFCzZs08VpiJiYmqV6+erSEnLy9PO3bsUFpamtLS0twrxC1btrjPdz/vvPN00UUXqVevXurduzefkF1FEWaqLsuylJaWpvnz5ys1NVUbNmzQoUOHJEnVq1dXixYt3OumxMREtWjRQrVr1y5X6FiwYIGuvvrqcteyYMEC9ezZ85Tz5ebmauvWrR7rpbS0NG3dulWWZcnlcqlhw4a6+OKL1adPH/Xo0UNhYWHlrgPeLSsrS4sWLdL8+fP13Xff6ZdffnEfUYyJifEYDwkJCWrcuLGCg4Ntq9eyLO3fv1+bN292j4eir0UfGhoSEqIWLVq4Q32bNm24oP8sIMyIMONt9u3bV2zFk5aWpqNHj7rnCQsLU61atRQZGalatWoVe0RGRio8PFwBAQHy8/PzeBSdwpGfn6+8vDzl5+crPz9fubm52r9/f7FHRkaGx8+HDx/2qCMxMbHYSpwNBO9AmPEelmVp165dxdZNmzZtct+C3sfHR+Hh4adcNz3++ONau3Ztua6J8fHxUatWrXTXXXeVuV7av3+/x+d7RUVFeayTEhMT1bx587Nyyhq8U05Ojn766SeP8bBhwwbt3LnTPU9QUFCp4+DEn0NCQjzep6tVqyZfX19J8nifLnrfPnToUJljoehRFLb8/PwUHx9f7P36TI+yonwIMyLM4K/TMoqOhuzZs6fM0HHgwAGP889PdsUVV2jFihWlPl+tWrUSV74nPqKjo5WQkKD69etzIaAXI8wgPz9fW7du1aZNm7R3794yd4YcPHjwjG4FHRgYeMp1U9Eecj7AD3bJzMzUxo0blZ6eXmrIKHpkZ2eXupwLLrhA27ZtK/O1inZslhaWIiMj1bRpU8XHx3PKmI3MuWAAOIt8fHzUoEEDNWjQ4JTzWpalQ4cO6eDBg8rLy/PYq1N0BzWXy+Xe+1O0JyggIEARERGqXr06AQVAuRTt8S3PtXAFBQX67bffFBcXd9qvs2vXLtWvX78iJQLnVFhYmNq3b6/27dufct6cnBzt379fx44dc79Xn/ieffJ7ddHXmjVrKjw83Kjrar0ZrQScJpfLpbCwME71AuAovr6+iomJkcvlOq0jND4+Pnz+FKqkwMBAbjzhBTiRDwCAKsLPz0/Jycnu6wFOxdfXV8nJyeyBBmAswgwAAFXI6NGjy7yu70QFBQUaPXr0Wa4IAM4ewgwAAFVIjx491L1791PeRcnHx0dXXXWVrrrqqnNUGQBUPsIMAABViI+Pj+bOnauUlBRJKnbKWdHPKSkpeu+997h1LACjsQYDAKCKCQkJ0YIFC7RgwQJ169bNfQdFHx8fdevWzf0cnwsDwHRc8QcAQBXk4+Ojnj17qmfPnnr++ed17733Kisri4v9AVQpHJkBAKCKKzq1jCADoKohzAAAAAAwEmEGAAAAgJEIMwAAAACMRJgBAAAAYCTCDAAAAAAjEWYAAAAAGIkwAwAAAMBIhBkAAAAARiLMAAAAADASYQYAAACAkQgzAAAAAIxEmAEAAABgJMIMAAAAACMRZgAAAAAYiTADAAAAwEiEGQAAAABGIswAAFDFVatWTXFxcXaXAQCVjjADAEAVl5eXp61bt9pdBgBUOsIMAAAAACMRZgAAAAAYiTADAAAAwEiEGQAAAABGIswAAAAAMBJhBgAAAICRCDMAAAAAjESYAQAAAGAkwgwAAAAAIxFmAAAAABiJMAMAgBf7+eefNX78eLVt21a1a9dWjRo1dNFFF2nSpEnKysqyuzwAKBNhBgAALzZr1ixNmzZNcXFxGj9+vJ5++mnFx8frwQcfVLt27XTs2DG7SwSAUvnZXQAAALDPNddco3Hjxik0NNQ97dZbb1Xjxo01adIkvfrqqxo9erSNFQJA6TgyAwCAF7vkkks8gkyRQYMGSZLS0tLOdUkAUG6EGQAAUMzvv/8uSapbt67NlQBA6QgzAADAQ0FBgR599FH5+fnp+uuvt7scACgV18wAAAAPY8aM0cqVKzV58mTFx8fbXQ4AlIojMwAAwO2hhx7S9OnTNXLkSI0bN87ucgCgTIQZAAAgSZo4caIee+wxDRs2TDNnzrS7HAA4JcIMAADQxIkT9fDDD2vo0KF65ZVX5HK57C4JAE6JMAMAgJd75JFH9PDDD+vGG2/UrFmz5OPD5gEAM3ADAAAAvNgLL7ygCRMmKCYmRl27dtVbb73l8XzdunXVrVs3m6oDgLIRZgAA8GKrVq2SJO3YsUNDhw4t9nxSUhJhBoBjcRwZAAAv9tprr8myrFIfy5Yts7tEACgVYQYAAACAkQgzAAAAAIxEmAEAAABgJMIMAAAAACMRZgAAAAAYiTADAAAAwEiEGQAAAABGIswAAAAAMBJhBgAAAICRCDMAAAAAjESYAQAAAGAkwgwAAAAAIxFmAAAAABiJMAMAAADASIQZAAAAAEYizAAAAAAwEmEGAAAAgJEIMwAAAACMRJgBAAAAYCTCDAAAAAAjEWYAAAAAGIkwAwAAAMBIhBkAAAAARiLMAAAAADASYQYAAACAkfzsLsAJGjZsqP3799tdBgB4aNKkierVq2d3GQAAOBZHZiRlZWVpzZo1dpcBAG6WZWnZsmWyLMvuUgAAcCzCjKQLL7xQaWlp2r17t92lAIAkafXq1Tp06JAuvPBCu0sBAMCxCDOSBg0aJH9/f82ZM8fuUgBAkjRr1ixFR0era9eudpcCAIBjEWYkhYWFacCAAXr11VeVl5dndzkAvNyhQ4f0n//8RzfddJN8fX3tLgcAAMcizPzXmDFjtH37dt155512lwLAixUUFOiGG26QJI0cOdLmagAAcDbCzH9dcsklevHFFzVjxgzNmDHD7nIAeKkHHnhAn332md555x3FxMTYXQ4AAI7GrZlPMGLECP3444+64447VKNGDQ0ePNjukgB4Ccuy9PTTT+vJJ5/U1KlTlZKSYndJAAA4HmHmJP/85z91+PBh3XjjjVq+fLmeffZZBQUF2V0WgCrswIEDuummm/Txxx9r3Lhxuuuuu+wuCQAAI3Ca2Un8/Pw0e/Zsvfrqq5ozZ47atm2rn376ye6yAFRR3377rVq3bq2vvvpKH3/8sSZPniyXy2V3WQAAGIEwUwKXy6Xhw4fr+++/V25urhITEzVq1Cjt2rXL7tIAVBE///yzrrvuOl1++eWqV6+e1q1bp169etldFgAARiHMlCExMVFr167V5MmT9e6776pRo0YaO3asMjIy7C4NgKF27NihESNGqHnz5lqxYoVeeuklLV++nIv9AQCoAMLMKQQHB2vs2LFKT0/Xfffdp5kzZ6phw4b6+9//rh9++MHu8gAYwLIsrVixQsOGDVPjxo310UcfacqUKfr111918803q1q1anaXCACAkQgz5RQaGqqHH35Y6enpGj16tObOnavWrVurVatWev7557V//367SwTgMLt27dLjjz+uJk2aqGPHjlq+fLl7PTJmzBgFBgbaXSIAAEZzWZZl2V2EifLz8/X5559r1qxZ+vjjj+Xj46Orr75avXr1Urdu3RQdHW13iQBssHXrVi1atEgfffSRFi5cqICAAF1zzTUaPny4OnbsKB8f9iHh3HvxxRd11113KTc31+5SAKBScWvmCvLz81OvXr3Uq1cv7d27V2+88YbeeustDR8+XJZlqXnz5kpOTla3bt2UlJSkkJAQu0sGcBZkZmZqyZIlSk1N1aJFi5Seni5fX1+1a9dOM2bM0KBBgxQaGmp3mQAAVEkcmalkGRkZ+uKLL7Ro0SKlpqZq586dqlatmtq3b+8ON61bt2bvLGCovLw8fffdd+7w8v3336uwsFBNmjRRt27dlJycrE6dOqlmzZp2lwq4cWQGQFVFmDmLLMvSzz//7N7oWbp0qY4ePapatWopKSlJLVu2VEJCghITExUXFydfX1+7SwZwgry8PG3ZskVpaWnasGGD1q9fr+XLl+vIkSOKiIhQ165d1a1bN3Xr1k2xsbF2lwuUijADoKoizJxDeXl5+vbbb7Vo0SJ99dVXSktL0759+yRJgYGBat68uTvcFH2tX78+H6AHnGWFhYXasWOHNmzY4A4uaWlp+umnn5SXlydJio6OVkJCgjp27Kjk5GS1atWKHRAwBmEGQFVFmLHZ3r17i21ApaWlKSsrS5IUHh5eLOAkJCQoLCzM3sIBQ+3bt6/EMXf06FFJf925MDEx0WPMtWjRQhERETZXDlQcYQZAVUWYcaDCwkJt377dY2Nrw4YN2rJli/Lz8yVJ5513nhISEpSQkKAGDRooOjpa9evXV3R0tOrWrSs/P+7tAO90/Phx7dmzR7t27dKuXbu0e/dupaenu8fR3r17JUkBAQElHg2Njo7maCiqHMIMgKqKMGOQ48ePe5y/n5aWpo0bN2rnzp3uU2EkycfHR3Xr1lV0dLT7URR0Tvw+NDSUjTYYw7IsHThwwCOklPR9UVgpEhAQoNjY2GKhJS4ujtAPr0GYAVBVEWaqgMLCQu3fv9+9MVfahl5GRobH7wUHB5cadIq+j4iIUPXq1bn7Gs6awsJCHT582N2HSwspu3fv9tgQc7lcqlOnTpmBPTo6WuHh4YR2eD3CDICqit2SVYCPj49q166t2rVr66KLLip1vtzcXP3xxx8lhp7ff/9d33//vXbt2qVjx455/J7L5VLNmjUVGhqq0NBQj+9PfpT2XI0aNdgLXgXl5eXp8OHDOnTokMejpGmlPXfkyJFiy61evbo7jDRo0EDt27cvFlKioqJUrVo1G/5qAADgFGxdepGAgAA1aNBADRo0KHUey7KUmZnpDjkHDx4sdeP0jz/+0E8//eTx3Imnu50sJCSkzABUs2ZNBQQEyN/fX/7+/qf1fUnT/Pz8vGqPvGVZysvL0/Hjx3X8+HHl5uZ6fC3P9ydOy8nJ0eHDh8sMJicH3xMFBASUGHCjoqJKDL8RERHuoFKjRo1z+J8DAACmIszAg8vlUnh4uMLDw9WiRYvT+l3LspSTk1OuPfRF0w8cOKBt27a599CfvEF9pn9LUcA5VfApevj4+MjHx0cul8v99cTvyzOtbt26+vPPP1VYWCjLsmRZlvv7sqad/FxhYeFph5IzdfL/6OQgEhMTU+4jcgEBAWdcDwAAQFkIM6g0LpdLQUFBCgoKUlRU1Bkvr7QjDeU5qlDR7wsLC5Wfn3/KwFFWOGnSpIl+/vnnEgNPWWGopK+hoaGVcpSqPN9725EsAABgPsIMHOvEIysAAADAybhFFQAAAAAjEWYAAAAAGIkwAwAAAMBIhBkAAAAARiLMAAAAADASYQYAAACAkQgzAAAAAIxEmAEAAABgJMIMAAAAACMRZgAAAAAYiTADAAAAwEiEGQAAAABGIswAAAAAMBJhBgAAAICRCDMAAAAAjESYAQAAAGAkwgwAAAAAIxFmAAAAABiJMAMAAADASIQZAAAAAEYizAAAAAAwEmEGAAAAgJEIMwAAAACMRJgBAAAAYCTCDAAAAAAjEWYAAAAAGIkwAwAAAMBIhBkAAAAARiLMAAAAADASYQYAAACAkQgzAAAAAIxEmAEAAABgJMIMAAAAACMRZgAAAAAYiTADAAAAwEiEGQAAAABGIswAAAAAMBJhBgAAAICRCDMAAAAAjESYAQAAAGAkwgwAAAAAIxFmAAAAABiJMAMAAADASIQZAAAAAEYizAAAAAAwEmEGAAAAgJEIMwAAAACMRJgBAAAAYCTCDAAAAAAjEWYAAAAAGIkwAwAAAMBIhBkAAAAARiLMAAAAADASYQYAAACAkQgzAAAAAIxEmAEAAABgJMIMAAAAACMRZgAAAAAYiTADAAAAwEiEGQAAAABGIswAAAAAMBJhBgAAAICRCDMAAAAAjESYAQAAAGAkwgwAAAAAIxFmAAAAABiJMAMAAADASIQZAAAAAEYizAAAAAAwEmEGAAAAgJEIMwAAAACMRJgBAAAAYCTCDAAAAAAjEWYAAAAAGIkwAwAAAMBIhBkAAAAARiLMAAAAADASYQYAAACAkQgzAAAAAIxEmAEAAABgJMIMAAAAACMRZgAAAAAYiTADAAAAwEiEGQAAAABGIswAAAAAMBJhBgAAAICRCDMAAAAAjESYAQAAAGAkwgwAAAAAIxFmAAAAABiJMAMAAADASIQZAAAAAEYizAAAAAAwEmEGAAAAgJEIMwAAAACMRJgBAAAAYCTCDAAAAAAjEWYAAAAAGIkwAwAAAMBIhBkAAAAARiLMAABQxcXFxal///52lwEAlc5lWZZldxEAAAAAcLo4MgMAAADASIQZAAAAAEYizAAAAAAwEmEGAAAAgJEIMwAAAACMRJgBAAAAYCTCDAAAAAAjEWYAAAAAGIkwAwAAAMBIhBkAAAAARiLMAAAAADASYQYAAACAkQgzAAAAAIxEmAEAAABgJMIMAAAAACMRZgAAAAAYiTADAAAAwEiEGQAAAABGIswAAAAAMBJhBgAAAICRCDMAAAAAjESYAQAAAGAkwgwAAAAAIxFmAAAAABiJMAMAAADASIQZAAAAAEYizAAAAAAwEmEGAAAAgJEIMwAAAACMRJgBAAAAYCTCDAAAAAAjEWYAAAAAGIkwAwAAAMBIhBkAAAAARiLMAAAAADASYQYAAACAkQgzAAAAAIxEmAEAAABgJMIMAAAAACMRZgAAAAAYiTADAAAAwEiEGQAAAABG+v+rux1PTNqR0gAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 800x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Create ansatz and convert to tensor diagrams\n",
    "\n",
    "from lambeq import AtomicType, SpiderAnsatz\n",
    "from lambeq.backend.tensor import Dim\n",
    "\n",
    "N = AtomicType.NOUN\n",
    "S = AtomicType.SENTENCE\n",
    "\n",
    "# Create an ansatz by assigning 2 dimensions to both\n",
    "# noun and sentence spaces\n",
    "ansatz = SpiderAnsatz({N: Dim(2), S: Dim(2)})\n",
    "\n",
    "train_circuits = [ansatz(d) for d in train_diagrams]\n",
    "test_circuits = [ansatz(d) for d in test_diagrams]\n",
    "\n",
    "all_circuits = train_circuits + test_circuits\n",
    "\n",
    "all_circuits[0].draw(figsize=(8,4), fontsize=13)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating a vocabulary\n",
    "\n",
    "We are now ready to create a vocabulary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.5488135 , 0.71518937])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create vocabulary\n",
    "\n",
    "from sympy import default_sort_key\n",
    "\n",
    "vocab = sorted(\n",
    "   {sym for circ in all_circuits for sym in circ.free_symbols},\n",
    "    key=default_sort_key\n",
    ")\n",
    "tensors = [np.random.rand(w.size) for w in vocab]\n",
    "\n",
    "tensors[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training\n",
    "\n",
    "### Define loss function\n",
    "\n",
    "This is a binary classification task, so we will use binary cross entropy as the loss."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def sigmoid(x):\n",
    "    return 1 / (1 + np.exp(-x))\n",
    "\n",
    "def loss(tensors):\n",
    "    # Lambdify\n",
    "    np_circuits = [c.lambdify(*vocab)(*tensors) for c in train_circuits]\n",
    "    # Compute predictions\n",
    "    predictions =  sigmoid(np.array([c.eval(dtype=float) for c in np_circuits]))\n",
    "\n",
    "    # binary cross-entropy loss\n",
    "    cost = -np.sum(train_targets * np.log2(predictions)) / len(train_targets)\n",
    "    return cost"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "The loss function follows the steps below:\n",
    "\n",
    "1. The :term:`symbols <symbol>` in the training diagrams are replaced with concrete ``numpy`` arrays.\n",
    "2. The resulting :term:`tensor networks <tensor network>` are evaluated and produce results.\n",
    "3. Based on the predictions, an average loss is computed for the specific iteration.\n",
    "\n",
    "We use JAX in order to get a gradient function on the loss, and \"just-in-time\" compile it to improve speed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "from jax import jit, grad\n",
    "\n",
    "training_loss = jit(loss)\n",
    "gradient = jit(grad(loss))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "We are now ready to start training. The following loop computes gradients and  uses them to update the tensors associated with the :term:`symbols <symbol>`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 10 - loss 0.1838509440422058\n",
      "Epoch 20 - loss 0.029141228646039963\n",
      "Epoch 30 - loss 0.014427061192691326\n",
      "Epoch 40 - loss 0.009020495228469372\n",
      "Epoch 50 - loss 0.006290055345743895\n",
      "Epoch 60 - loss 0.004701168276369572\n",
      "Epoch 70 - loss 0.0036874753423035145\n",
      "Epoch 80 - loss 0.0029964144341647625\n",
      "Epoch 90 - loss 0.0025011023972183466\n"
     ]
    }
   ],
   "source": [
    "training_losses = []\n",
    "\n",
    "epochs = 90\n",
    "\n",
    "for i in range(epochs):\n",
    "\n",
    "    gr = gradient(tensors)\n",
    "    for k in range(len(tensors)):\n",
    "        tensors[k] = tensors[k] - gr[k] * 1.0\n",
    "\n",
    "    training_losses.append(float(training_loss(tensors)))\n",
    "\n",
    "    if (i + 1) % 10 == 0:\n",
    "        print(f\"Epoch {i + 1} - loss {training_losses[-1]}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluate\n",
    "\n",
    "Finally, we use the trained model on the test dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on test set: 0.8666666666666667\n"
     ]
    }
   ],
   "source": [
    "# Testing\n",
    "\n",
    "np_test_circuits = [c.lambdify(*vocab)(*tensors) for c in test_circuits]\n",
    "test_predictions =  sigmoid(np.array([c.eval(dtype=float) for c in np_test_circuits]))\n",
    "\n",
    "hits = 0\n",
    "for i in range(len(np_test_circuits)):\n",
    "    target = test_targets[i]\n",
    "    pred = test_predictions[i]\n",
    "    if np.argmax(target) == np.argmax(pred):\n",
    "        hits += 1\n",
    "\n",
    "print(\"Accuracy on test set:\", hits / len(np_test_circuits))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Working with quantum circuits"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "The process when working with :term:`quantum circuits <quantum circuit>` is very similar, with two important differences:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. The parameterisable part of the circuit is an array of parameters, as described in Section [Circuit Symbols](training-symbols.ipynb#Circuit-symbols), instead of tensors associated to words.\n",
    "2. If optimisation takes place on quantum hardware, standard automatic differentiation cannot be used. An alternative is to use a gradient-approximation technique, such as [Simultaneous Perturbation Stochastic Approximation](https://en.wikipedia.org/wiki/Simultaneous_perturbation_stochastic_approximation) (SPSA)."
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "More information can be also found in [Mea2020]_ and [Lea2021]_, the papers that describe the first NLP experiments on quantum hardware.\n",
    "\n",
    ".. rubric:: See also:\n",
    "\n",
    "- `Classical pipeline with Pytorch <../examples/classical-pipeline.ipynb>`_\n",
    "- `Quantum pipeline with tket <../examples/quantum-pipeline.ipynb>`_\n",
    "- `Quantum pipeline with JAX <../examples/quantum-pipeline-jax.ipynb>`_"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}