Spring2021/penalty_admm.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Метод штрафов, модифицированной функции Лагранжа и ADMM"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## На прошлом семинаре\n",
    "\n",
    "- Методы внутренней точки\n",
    "- Метод внутренних барьеров (логарифмических)\n",
    "- Прямо-двойственный метод"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Метод внешних штрафов\n",
    "\n",
    "**Идея:** если внутренние штрафы (барьеры) предотвращали выход точки из множества, то внешние штрафы существенно увеличивают целевую функцию при выходе точки из множества, но позволяют этому случиться."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Задача с ограничениями-равенствами\n",
    "\n",
    "\\begin{equation*}\n",
    "\\begin{split}\n",
    "& \\min f(x)\\\\\n",
    "\\text{s.t. } & g_i(x) = 0, \\; i=1,\\ldots,m\n",
    "\\end{split}\n",
    "\\end{equation*}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Квадратичная штрафная функция\n",
    "Исходную задачу можно преобразовать к задаче безусловной оптимизации\n",
    "\n",
    "$$\n",
    "\\min_x Q(x, \\mu),\n",
    "$$\n",
    "\n",
    "где \n",
    "\n",
    "$$\n",
    "Q(x, \\mu) = f(x) + \\frac{\\mu}{2}\\sum\\limits_{i=1}^mg^2_i(x), \\quad \\mu > 0\n",
    "$$\n",
    "- Если ограничение не выполняется, то значение целевой функции увеличивается пропорционально парамеру $\\mu$\n",
    "- При увеличении параметра $\\mu$, решение задачи всё точнее и точнее удовлетворяет ограничениям"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Общая схема\n",
    "\n",
    "```python\n",
    "def QudraticPenaltyEquality(Q, gradQ, x0, get_mu, get_tau, **kwargs):\n",
    "    \n",
    "    while True:\n",
    "        \n",
    "        # Stop when norm of gradient of Q is less than current tau\n",
    "        \n",
    "        x = MinimizeQ(Q, gradQ, x0, get_tau)\n",
    "        \n",
    "        if global_convergence(x, Q, **kwargs):\n",
    "            \n",
    "            break\n",
    "            \n",
    "        mu = get_mu()\n",
    "        \n",
    "        Q, gradQ = UpdateQ(mu)\n",
    "        \n",
    "        x0 = UpdateStartPoint(x, Q)\n",
    "        \n",
    "    return x\n",
    "```\n",
    "\n",
    "- Параметр $\\mu$ нужно изменять в зависимости от сложности решения подзадачи: если известно, что задача решается долго, то $\\mu$ нужно увеличивать плавно, например $\\mu_{k+1} = 2\\mu_k$. Eсли подзадача решается быстро, то допустимо более резкое увеличение $\\mu$: $\\mu_{k+1} = 15\\mu_k$.\n",
    "- При увеличении $\\mu$ решение подзадачи становится сложнее из-за плохой обусловленности гессиана. Об этом подробнее ниже."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Сходимость\n",
    "\n",
    "**Теорема 1.** Пусть для каждого $\\mu$ безусловная задача имеет конечное глобальное решение. Тогда предельная точка последовательности решений безусловных задач при $\\mu \\to \\infty$ является глобальным решением исходной задачи.\n",
    "\n",
    "- Проблема: глобальный оптимум для каждой подзадачи не всегда может быть достигнут!\n",
    "\n",
    "**Теорема 2.** Пусть $\\tau_k \\to 0$ и $\\mu_k \\to \\infty$ и $\\| Q'(x^*_k, \\mu_k) \\| \\leq \\tau_k$. Тогда \n",
    "- если $x^*_k \\to x^*$ и $x^*$ - недопустима, то $x^*$ стационарная точка функции $\\| g(x) \\|^2_2$;\n",
    "- если $x^*$ - допустима и градиенты ограничений в этой точке линейно независимы, тогда $x^*$ - точка, в которой выполнены KKT.\n",
    "\n",
    "Для любой подпоследовательности $x^*_k \\to x^*, \\; k \\in \\mathcal{C}$ выполнено\n",
    "\n",
    "$$\n",
    "\\lim_{k \\in \\mathcal{C}} \\mu_k g_i(x_k) = \\lambda_i^*\n",
    "$$\n",
    "\n",
    "для всех $i = 1,\\ldots,m$, где $\\lambda_i^*$ множители Лагранжа, удовлетворяющие ККТ."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Вырожденность гессиана\n",
    "\n",
    "**Пример.**\n",
    "\\begin{equation*}\n",
    "\\begin{split}\n",
    "& \\min -5x_1^2 + x_2^2\\\\\n",
    "\\text{s.t. }& x_1 = 1\n",
    "\\end{split}\n",
    "\\end{equation*}\n",
    "Штрафная функция примет вид\n",
    "\n",
    "$$\n",
    "Q(x, \\mu) = -5x_1^2 + x_2^2 + \\frac{\\mu}{2}(x_1 - 1)^2\n",
    "$$\n",
    "\n",
    "**Наблюдение** При $\\mu < 10$ функция $Q(x, \\mu)$ неограничена снизу по $x$, и подзадача не имеет конечного решения"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Общий вид гессиана\n",
    "\n",
    "$$\n",
    "Q''(x, \\mu) = f''(x) + \\mu\\sum_{i=1}^m g_i(x) g''_i(x) + \\mu(g'(x))^{\\top} g'(x),\n",
    "$$\n",
    "\n",
    "где $g'(x)$ - якобиан вектор-функции ограничений-равенств.\n",
    "\n",
    "Около точки минимума в силу теоремы 2 справедливо следующее приближение\n",
    "\n",
    "$$\n",
    "Q''(x, \\mu) \\approx L''(x, \\lambda^*) + \\mu(g'(x))^{\\top} g'(x)\n",
    "$$\n",
    "\n",
    "- гессиан лагранжиана не зависит от $\\mu$\n",
    "- произведение $(g'(x))^{\\top} g'(x)$ ранга $m \\ll n$\n",
    "\n",
    "**Итог:** некоторые собственные значения гессиана $Q''(x, \\mu)$ имеют порядок $\\mu$, а некоторые от $\\mu$ не зависят, что приводит к вырожденности при увеличении $\\mu$\n",
    "\n",
    "**Следствие:** поиск направления в методе Ньютона очень неточный"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Как искать направление в методе Ньютона?\n",
    "\n",
    "$$\n",
    "Q''(x_k, \\mu)p = -Q'(x_k, \\mu)\n",
    "$$\n",
    "\n",
    "Введём новую переменную $\\xi = \\mu g'(x) p$ и запишем эту систему в виде\n",
    "$$\\begin{bmatrix} f''(x) + \\mu \\sum\\limits_{i=1}^m g_i(x)g_i''(x) & (g'(x))^{\\top}\\\\ (g'(x)) & -\\frac{1}{\\mu} I \\end{bmatrix} = \\begin{bmatrix} -Q'(x, \\mu) \\\\ 0 \\end{bmatrix}$$\n",
    "\n",
    "- Обусловленность этой системы уже не растёт с ростом $\\mu$\n",
    "- Размерность увеличилась на $m$\n",
    "- Проблема неадекватности квадратичной аппроксимации остаётся"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Задача с ограничениями типа равенств и неравенств\n",
    "\n",
    "\\begin{equation*}\n",
    "\\begin{split}\n",
    "& \\min_x f(x)\\\\\n",
    "\\text{s.t. } & g_i(x) = 0, \\; i=1,\\ldots,m \\\\\n",
    "& h_j(x) \\leq 0, \\; j = 1,\\ldots,p\n",
    "\\end{split}\n",
    "\\end{equation*}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Штрафная функция\n",
    "\n",
    "$$\n",
    "\\min_x f(x) + \\frac{\\mu}{2}\\sum\\limits_{i=1}^mg^2_i(x) + \\frac{\\mu}{2}\\sum\\limits_{j=1}^p (\\max(0, h_j(x)))^2,\n",
    "$$\n",
    "где $\\mu > 0$\n",
    "- Отличие от задачи без ограничений-неравенств: вторая производная штрафной функции разрывна и, как следствие, штрафная функция уже не дважды непрерывно дифференцируема"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Эксперимент\n",
    "\n",
    "Рассмотрим задачу\n",
    "\\begin{equation*}\n",
    "\\begin{split}\n",
    "& \\min \\|x\\|_2 \\\\\n",
    "\\text{s.t. } & Ax = b\n",
    "\\end{split}\n",
    "\\end{equation*}\n",
    " \n",
    "и решим её методом штрафных функций."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "----------------------------------------------------------------------------\n",
      "\tSCS v1.2.6 - Splitting Conic Solver\n",
      "\t(c) Brendan O'Donoghue, Stanford University, 2012-2016\n",
      "----------------------------------------------------------------------------\n",
      "Lin-sys: sparse-indirect, nnz in A = 221, CG tol ~ 1/iter^(2.00)\n",
      "eps = 1.00e-04, alpha = 1.50, max_iters = 2500, normalize = 1, scale = 1.00\n",
      "Variables n = 21, constraints m = 31\n",
      "Cones:\tprimal zero / dual free vars: 10\n",
      "\tsoc vars: 21, soc blks: 1\n",
      "Setup time: 9.12e-04s\n",
      "----------------------------------------------------------------------------\n",
      " Iter | pri res | dua res | rel gap | pri obj | dua obj | kap/tau | time (s)\n",
      "----------------------------------------------------------------------------\n",
      "     0|      inf       inf       nan      -inf       inf       inf  1.08e-03 \n",
      "    40| 4.27e-06  3.32e-06  1.31e-06  1.34e+00  1.34e+00  4.50e-17  1.27e-03 \n",
      "----------------------------------------------------------------------------\n",
      "Status: Solved\n",
      "Timing: Solve time: 1.28e-03s\n",
      "\tLin-sys: avg # CG iterations: 2.59, avg solve time: 2.44e-06s\n",
      "\tCones: avg projection time: 7.07e-08s\n",
      "----------------------------------------------------------------------------\n",
      "Error metrics:\n",
      "dist(s, K) = 1.5845e-16, dist(y, K*) = 0.0000e+00, s'y/|s||y| = -4.8403e-17\n",
      "|Ax + s - b|_2 / (1 + |b|_2) = 4.2672e-06\n",
      "|A'y + c|_2 / (1 + |c|_2) = 3.3176e-06\n",
      "|c'x + b'y| / (1 + |c'x| + |b'y|) = 1.3096e-06\n",
      "----------------------------------------------------------------------------\n",
      "c'x = 1.3414, -b'y = 1.3414\n",
      "============================================================================\n",
      "Optimal value from CVXPY = 1.341372001979229\n"
     ]
    }
   ],
   "source": [
    "import cvxpy\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "plt.rc(\"text\", usetex=True)\n",
    "\n",
    "num_iters = 30\n",
    "n = 20\n",
    "m = 10\n",
    "A = np.random.randn(m, n)\n",
    "b = np.random.randn(m, 1)\n",
    "\n",
    "# Initialize problem\n",
    "x = cvxpy.Variable(shape=(n, 1))\n",
    "f = cvxpy.norm(x, 2)\n",
    "\n",
    "# Solve with CVXPY.\n",
    "cvxpy.Problem(cvxpy.Minimize(f), [A @ x == b]).solve(solver=cvxpy.SCS, verbose=True)\n",
    "cvxpy_f = f.value\n",
    "print (\"Optimal value from CVXPY =\", f.value)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimal value from method of augmented Lagrangian = 1.3415368101550718\n"
     ]
    }
   ],
   "source": [
    "resid = A@x - b\n",
    "mu = cvxpy.Parameter(value=1, nonneg=True)\n",
    "penal_f = f + (mu/2.)*cvxpy.sum_squares(resid)\n",
    "res_conv = []\n",
    "f_conv = []\n",
    "for t in range(num_iters):\n",
    "    cvxpy.Problem(cvxpy.Minimize(penal_f)).solve(solver=cvxpy.SCS)\n",
    "    mu.value = mu.value * 2\n",
    "    res_conv.append(np.linalg.norm(resid.value))\n",
    "    f_conv.append(f.value)\n",
    "\n",
    "print(\"Optimal value from method of augmented Lagrangian =\", f.value)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhAAAAGJCAYAAADbgQqfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3X90W+d5J/jvQ1KkZJESSEmW7CS2DDmO89MySKft2bQrxaCn6UymHZeU2pnunv0REe452+xuNyWs7Mzu7JyzlcH0dGf+mElIdWbn7JyeiiLtdJLOtI3oRJmZzjSVCNm1I9lOCdmOLcmkREIEJRMSyWf/uO+FLsELEhcEcC+A7+ccHBIX9168fHVtPHjf532uqCqIiIiIvGjyuwFERERUexhAEBERkWcMIIiIiMgzBhBERETkGQMIIiIi8owBBBEREXnGAIKIiIg8YwBBREREnjGAICIiIs9a/G6ATUT6AKRVdaLI/aMAegHcAHAAwKSqjjheDwEYADAOYBZAF4AYgDPFvgcRERG5kyCUsjbBwBiA/mI+3M3+cO4rIpMARlV1yDwPA5hyHJYGcExVx4tt1+7du3X//v3F7r6hW7duYfv27WU7X71gv7hjv7hjv7hjv7hjv7gr1C+Tk5PXVXVPMefwdQTCfMjHAUzCGiUoVgzAcN62CbN9yLGtF8B5AF2qmvLavv379+P8+fNeDyvo7NmzOHToUNnOVy/YL+7YL+7YL+7YL+7YL+4K9YuIvFPsOXzNgVDVlKrGnFMPHvS6bEu7vEe6lOCBiIiICgtMDoQXqtrvsrkPa0cliIiIqAJqMoDIJyIDAJJ2/oND2CRnAlYS5ayXHAgiIiJyF4gkSgAQkSkAMS8rJExw0AsAqhrLey0E4EjeyowxWImWBYMIE4wMAMDevXu7T5065envWM/CwgLa29vLdr56wX5xx35xx35xx35xx35xV6hfDh8+PKmqPcWco6YDCMexIQAvw1plkVxnvyiAYVU9UMx5e3p6lEmUlcd+ccd+ccd+ccd+ccd+cbdOEmXRAURdFJJS1TSs/IeXN9g1BWtaI1T5VhEREdWvugggjAkAIbtGhIgMuuxjLxUNV61VREREdajmAggRCYvInIhECuwSMvUlEuanU5f5yWWdREREm1BzAQSAEKwAID8IsIOFpKn7EHOp/xA1r6+pF0FERETFC1IA0QUrOFhFREIiMmVWR8AkSY66HB8HMOQIGmadIxAm7yEG4FjZW05ERNRg/C5lHQJwHNboQQjWtEMvrBteOZdadjmPU9UhERkQkQO4dzOtMeeSTVUdF5E+s9Rzlzl/P6tSEhERbZ6vAYSZSogXsU+ny/YNy1+zaBTVi5UVxbIqVlSxsgIsq2J5RaHm57IqVIHlFWsfVVgPKFYUUFUozE+FtQ3279ZP4N4x1k9rf8D+3frt3mv2Mff2gWO72RvOF52LxvNXkCvWLil3W2X+2swS5K0Z136qxLL0YCx039hrM0vQN6f9bkbg1Hu/7Njagu6HuzbesQLqohJlTfjT53Hwjf8IXOYK0nwH0+nA94v9oWp/WOc+0B0f5iv2z1X7WB/+Cjg+2POf532Yw9q2VRV/+X2f//ASSIHfy+UpAHitAieucZ8H2C8u6r1f3t/+GLrj/58v780AghraiiruLK/g7vIK7i6r9XNpxWwzz81jxcNX0SYRNAnQ3CRoEoEIICJogvkpwBYRCO69JgBEkNs/m81i29atAMwHsQBiPpJF1tlmiNx7Jo5f1n7Ar9645kNf3LeLW3hQ3KaCXPfN25jJLKCjg5UF8y1kFtDOflmj3vvlsT27fXtvBhDV8qUX8Mo2VkRz80oFK8WpKmYWsnjr2gLeuDaPtz7I4O0bt3E9k8XMQhaZxSXX4zrv24I9HW3Wo70Nu9vb0Lm9Fe1tLbivtRn3tbbgvrZmbG+1nm9va8H21mbc19aCbVua0dy0+e/erKDnbpL94or94o79UjkMIKhuZBbv4q0PFvDmtQze+iBjAoYFzN66k9tnd3srwrvb8ckHd+AX2u8FCHs6rCBhT0cbdrW3YktzkBYoEREFDwMIqknXbi7iL1M38OYHGbx5zXq8n/4w9/p9rc14bG8HnvnUXjy2twOP7+vAY/s6sLu9zcdWExHVDwYQVDNuLGTxp69fw3dfvYK/ensWqkBLk+DAnnZ0P9yJv/8zD+ETezvwiX0d+EhoG5rKMI1ARETuGEBQoM0v3sX3fvwBvvPqFfzF31zH8oriwJ7t+F+efgy9n9qLR+9vR2sLpxuIiKqNAQQFzod3lvHyGx/gO69cwdk3Z3BneQUf7dyGgV8I48ufexCffKBj1QoDIiKqPgYQFAjZpWX8h7eu47uvXsHEpQ9w+84y7u9owz/42Yfwd594EAc/FmLQQEQUIAwgyFdvXJvHv3wti6+encD84hJC923BLx/8CP7uEw/i8490lWU5JBERlR8DCPLF8opi+D9M4f858xaaofilJz6CLz/xIL7w6G4uoSQiqgEMIKjq3rlxC799+lVMvjOHL31mH37p/nl8+ZmDfjeLiIg8YABBVaOq+MMfvYvf/feX0Nwk+KdHD+KXDz6IH/7wh343jYiIPGIAQVXxwfwiBsf/Gj98awZfeHQ3hvo+hwdD2/xuFhERlYgBBFXcd1+9gn/4x68ju7SMf/LLn8Zv/MzDLPJERFTjGEBQxaRv38E/+rc/xndfvYKDHwvh9488gfCe+r0rHhFRI2EAQRVx9s1pDI7/NWZv3cHXnnkMz/3XB9DC1RVERHWDAQSV1a3sEn7331/CH/7oXTy2tx3/6r97Cp/5yE6/m0VERGXGAILKZvKdWfz26Vfx7uxtDPxCGL/d+xi2bmn2u1lERFQBDCCoLP7wR+/gH/3x63gwtA2njv0sfia8y+8mERFRBTGAoE177b2b+Mff+TF+/uN78M//QQTtbbysiIjqHbPaaFMWskv4rT9KYnd7G/7Zrx1k8EBE1CD4f3valP/j376Od2dv49TAzyF0X6vfzSEioirhCASV7NsX3sNLyffx1ac/js8/0uV3c4iIqIoYQFBJLl+/hX/47dfx+f1d+J8OP+p3c4iIqMoYQJBnd5ZW8NU/uoCW5ib80187yAJRREQNiDkQ5NnQn72B196/iZH/pps3xCIialD86kie/ODNafzBf7qM//bnHsYzn97nd3OIiMgnDCCoaNPzi/ja6Vfx+L4OfP2XPul3c4iIyEecwqCirKwo/tfTr+DWnSWM/v2fZYlqIqIGxwCCivLNH07hL/7mBl549rN49P4Ov5tDREQ+4xQGbSj57hx+/8xb+NufewBHn/qY380hIqIAYABB67r54V189Y8u4IGdW3Hi2c9CRPxuEhERBQCnMKggVcXXv/0art5cxNhzP4cdW7f43SQiIgoIjkBQQaPnfop/99dX8b898xgiD3X63RwiIgoQBhDk6icfZPCPv/tjfOHR3XjuFw743RwiIgoYBhC0xuLdZfzWH13A9tYW/P6RJ9DUxLwHIiJajTkQtMb//e8u4Y1rGfy///1TuH/HVr+bQ0REAcQRCFrl5Usf4N/85Tv4yhceweFP3O93c4iIKKACMwIhIn0A0qo6UeT+UQC9AG4AOABgUlVH8vaJAOgBkAIQBpAq9vyNaviHKezfdR8Gf/Fxv5tCREQBFogAwgQDJwH0e9gfqhp3bJsUkZCqDpnnYQAJVe117DMmIilVTZX1D6gT7964jb96exa/87c+gdYWDk4REVFhvn5KiEhYRIZhjQ7Mejg05rJtIm97DMBw3j7DABKeGtlAvn3hfYgAv/LkR/xuChERBZyvAYSqplQ1lj/1UKRel21px+99AJJ5r5832ymPquKlC+/h58K78JHQNr+bQ0REAReIKQyvVNVtqqMPZsRBREJwGdVQ1bSIQETCnMZYLfnuHN65cRu/9cWP+90UIiKqAXUx0S0iAwCSdv4DgC7AChgKHBKuSsNqyIvJ97FtSzN+8TP7/G4KERHVgJocgbCZlRu9wJpRiZA/LapNi3eX8SevXsEvfmYf2ttq+pIgIqIqEVX1uw0AABGZAhArZZmlmbJ4GcAxVU2a5ZuTqrqmhKKIKIDeQu9jRjMGAGDv3r3dp06d8tqcghYWFtDe3l6285XLX11bwr94JYuv9WzFZ3Y3V/39g9ovfmO/uGO/uGO/uGO/uCvUL4cPH55U1Z5izlEXXzdNbsMwrCAid9cns6yz0DRGoXONABgBgJ6eHj106FDZ2nn27FmU83zl8m/+9Tns2zGP33z2i2j2oWx1UPvFb+wXd+wXd+wXd+wXd+Xol7rIgTAmAIRMjQg7QbLLuYMZqYDj9YZ3fSGLs2/N4Fee/IgvwQMREdWmmgsgTO2IOTNN4cYedUhhbS5EF6xqlwwgjO+8cgXLK4pnI6z9QERExau5AAJWUJDC2lEEe2WFXfthAlYZa6eI2U7GSxfew2c/shOP7e3wuylERFRDghRAdMFl9YSIhERkyiQ3QlWTAEZdjo8DGHKMLsSxtjR2zGwnAG9ey+D19+c5+kBERJ75mkRpchKOwxo9CAFIiEgvgDOqOu7YdVUug6oOiciAiBzAvZtpjTkrWprEyriIJACcM++R4PTFPS9deA8tTYIvP/Gg300hIqIa42sAYXIV1h0RMPt0umzfsPy1Ga3IL2dNAJZXFH984X0c+sQe7G5v87s5RERUY4I0hUFV9J+nruOD+SyejXzU76YQEVENYgDRoF5Kvo8dW1vwxcfv97spRERUgxhANKCF7BL+7PVr+DtPPIitW6pfeZKIiGofA4gG9GevX8OHd5fxq1x9QUREJWIA0YBeSr6Hh3fdh8hDa3JTiYiIisIAosG8n/4Q/yV1A88++VGIsHQ1ERGVhgFEg/njC+9DFSweRUREm8IAooGoKl5MvofPP9KFj3Xd53dziIiohjGAaCCvvncTqZlbTJ4kIqJNYwDRQF5Kvoe2liZ86bMP+N0UIiKqcQwgGsSdpRV859UreObT+7Bj6xa/m0NERDWOAUSD+MGb00jfvsvkSSIiKgsGEA3ipeR72N3ehp9/dLffTSEiojrAAKIBzN26g++/MY1fOfggWpr5T05ERJvHT5MG8Cd/fQV3l5V33iQiorJhANEAXky+j8f3deBTD+7wuylERFQnGEDUuamZBbzy0zR+laMPRERURgwg6ty3k++jSYBfPvig300hIqI6wgCijq2sKL594X38/Mf34P4dW/1uDhER1REGEHXsR5dn8X76Q9Z+ICKismMAUcdeTL6H9rYWPPOpfX43hYiI6gwDiDp1+84S/vS1q/jbn30A21qb/W4OERHVGQYQdep7P/4At+4sc/qCiIgqggFEnfqPP7mO3e2teGp/l99NISKiOsQAok5dvDqPTz+4E01N4ndTiIioDjGAqEN3llbwN9MZVp4kIqKKYQBRh34yncHdZcWnHmAAQURElcEAog5dvDIPAPgkAwgiIqoQBhB16NLVDLZuacIju7f73RQiIqpTDCDq0MWrN/H4vh1oZgIlERFVCAOIOqOquHhlngmURERUUQwg6sz76Q8xv7jEBEoiIqqollIOEpEdAMIA0gBmVXW+rK2ikjGBkoiIqqGoAEJEjgHoB9ADQAGkAMyZl8Mi0mm2jQIYV9W3y99UKsalqxmIAI/v6/C7KUREVMcKBhBmlOHrAJ4EMAagX1VvrncyEXkawPMmoBhW1e+Xs7G0sYtXb+KRXduxva2kwSUiIqKiuH7KiMiTAAYADKnq5WJPpqovA3jZnOOYiERU9ffK0lIqysWr8/jcR0N+N4OIiOrcmiRKEXkEQFhVf9NL8JBPVU8COCkiz26mgVS8mx/exU9nP2QCJRERVdyaEQgTNJQcOOSd6yaAl8pxLtrYG1etBEoGEEREVGlcxllHLtkBBGtAEBFRhTGAqCMXr85j1/ZW3N/R5ndTiIiozpUlgBCR085cBxF52qzi8HKOPhGJetx/UESGReSMiPTlvR4yr4fN72ERSXh5j1pz8apVgVKEJayJiKiyyjUCMayquVwHsxqjp9iDzYf6SQ/79wFIqeqQqsZg1ahIiMiAY7cuAAkAU7BqVkwCOKeqE8W+Ty25u7yCt64tMP+BiIiqolwBREpE9gOAiHxPRM4B6N3oIDMqMAyrquWsh/cLq2rSfqKqaVjBwnDefr0AOgEcUNVOVR338B41ZWpmAXeWV1iBkoiIqqLkAEJEfiIi3xSRL5qVGyEzjTEMIKqqxzc6h6qmVDWmqiMe3jcE4Kj56TRhXg/nvUdaVVPFnr9WMYGSiIiqaTPlCl8EcA7AETOKcBnAIyiiYuVmqGraBAlhAMmN9m8UF6/Mo7WlCeHd2/1uChERNYCSAwhVfd78+iKQK0AVBfB1EYkAOKOqv7n5Jrq+d6fL5iiA/NGGsCO5sgvWjb/qchrj4tV5PL6vAy3NXFhDRESVJ6pamROLPOKlkqWITAGIlZrkKCKTAEZVdcg8DwE44pweEZExs0/BIMIkYg4AwN69e7tPnTpVSnNcLSwsoL29vWzns6kqfuv7txHZ24L/4TO1t4SzUv1S69gv7tgv7tgv7tgv7gr1y+HDhydVtbhFEKrq6QFgh9djijzvFKzciVKOHYA14rHRflEAU8Wet7u7W8vpBz/4QVnPZ7uSvq0Px/9E//VfXK7I+SutUv1S69gv7tgv7tgv7tgv7gr1C4DzWuRnpKfxbhE5DWDOXnHh2P5FL+cpJ5MPEVPVDVd9wLrleNglAbOmMYGSiIiqzeuE+RlY0wJv522fFJGveS0eVSYJAE/nbxSRQZd97aWiYZfXatbFK1YA8fi+Dp9bQkREjcJrABGCFUSsoqo31bpt95GytKpIZvVHXK06EM7tYViFpfIDhS7zs66WdV68Oo+Hd92Hjq1b/G4KERE1CE+rMFT1GyLy5yKSBPA9Vf1Bhdq1IZPsmFDHqgtT0TKlqikRiena+g9RAMn8gKPWXbwyzwqURERUVV5zIL4FQGBVeHxZRJZF5JwpKHUCQPcm2tIFa4Qj/z1DIjLlLFPtWJoZEpGIeURh1aCwg4ZZ5wiEyXuIATi2iTYGzkJ2CW/fuM0KlEREVFVe60BMqepz9hPzoW0/FC65COsxH+rHYeUkhGBNO/TCWlHhXGrZlXfMWIFT5kYcVHXc3HCrD8Auc/5+l1GJmvbmNZNAyQCCiIiqyGsAsWroX62aDXYJ6Sdh5UD8QbEnM1MJ8SL26cx7XtTtJrVOi0Y52QmUXIFBRETV5DWJckJEvlLgtbnNNoa8u3h1HqH7tuCBnVv9bgoRETUQTwGEWpUlx0TkK84lm6aMdQqby4GgEtgJlCJFDcoQERGVhed7Yah1o6w/yNt22eQu1FV+QdAtLa/gjWsZ/MbPPux3U4iIqMFs5m6cq6jqy+U6FxXn7Ru3kF1aYQIlERFV3ZopDBF5RESeLcfJRWTHOjkTtEk/ZgIlERH5ZE0AYfIcLpvaDvtLPbGIHANwXFWLXpVB3ly8Oo/W5iYc2MM7zRERUXW5JlGq6gUAzwN4zlSeXJU0WYiIHBSRb4nIn8OqGXG8zO0lh4tX5vHxve1obfG6mIaIiGhzCuZAmGTJ5wFARH4VwB+ISARWwag07t2Y6gCsOg0pAOcBDJsAhCrs0tV5HPrE/X43g4iIGlBRSZSq+iKAF+3nIrITVvXINIBZE2xQFU1nFnF94Q4TKImIyBclrcIwAQNHGXzECpREROSngpPnIvI7JpeBAujiVSuA4E20iIjIDxtl393Mqzh5sMLtoSJdvDKPj3Zuw85tW/xuChERNaD1AohdAL6iqvOObbEKt4eKdPHqPEcfiIjIN+vlQIwCeFtEbgBIAjgDx221yT+37yzh8vVb+PLnHvS7KURE1KDWW8Z5AUCXiAwAiMBa0hkWkT5YSzaTAM6Zn+fzRiqogt68loEqEyiJiMg/G67CUNUR+3cROQ3gBIAeWHfe/DUAQwBURNIAJmCNXEwwoKgcO4GSSziJiMgvXksYnlPVC6p6UlWfU9UeVW0C8CiAAQCXAXwdwJyInCh3Y8ly8co8Ora24KOd2/xuChERNShPdSBU9RsFtl+GFTw4i009LSLfVNXf3FwTKZ+dQCkifjeFiIgaVEVuomCWe0YBTFbi/I1seUXx5rUMpy+IiMhXlboLk8Ba8hmq0Pkb1js3buH2nWUmUBIRka9KKmW9EXsFRyXO3eiYQElEREHA+0DXmItX5tHSJPj43na/m0JERA2MAUSNuXh1Ho/e3462lma/m0JERA2MAUSNuXR1ntMXRETkOwYQNeT6QhYfzGeZQElERL5jAFFDLjGBkoiIAoIBRA25eMUKIHgXTiIi8hsDiBpy8eo8Hti5FZ3bW/1uChERNTgGEDWECZRERBQUZQsgROSgiHzFlLGmMlu8u4ypmVtMoCQiokAo5wjE1wGMABgXkZ+IyLNlPHfDe+uDDJZXlCMQREQUCOUsZR0HcMKUsYaI7CzjuRuenUDJEQgiIgqCsgUQ5pbezuc3y3VushIot7c242Od9/ndFCIiIu9TGCKyX0ROiMifi8gOs22niHzTfk7ld+nqPD75wA40NYnfTSEiIvIWQIjI0wCmAPQCiMLccdOMNhwA8EK5G0jAyori0tUMpy+IiCgwvI5AJACMq2oPgPyvwiMAjpalVbTKT+duYyG7xARKIiIKDK8BRATAaIHX5gCENtcccsMESiIiChqvAUQSQE+B12IAJjbXHHJz8eo8mgR4bG+H300hIiIC4H0VxgsATouIAFAAO03hqASsnIjuMrePYCVQHtjTjq1bmv1uChEREQCPAYSqjovIEVj5DgJrREIApAEcUdVXSm2IiPQBSKtqUaMYZv8wrOTNMIBhVR3P2ycCa8QkZfZJFXv+ILl4ZR5PPdLldzOIiIhyPNeBMB/S4yLyJO59KF/YTCNEJArgJID+IvfvM+87bp6HAEyKSJeqjphtYQAJVe11HDcmIilVTW2mvdU0d+sOrtxcZAIlEREFitdlnLlPMVW9oKovbiZ4EJGwiAzDCkRmPRwaVtWkoy1pWNMow459YnnPYZ4nSmyuLy5dZQIlEREFj9ckyrSI/JWI/I/lKBqlqilVjdmjBsUwow1HzU+nCfN62DzvgzXF4nTebK8ZF00A8UmOQBARUYB4DSCeN8ecBDBnqlH+vfI3qzAz2hA2D1cmuFgzqmGOdQYZgff2jVvYuW0Ldre3+d0UIiKiHE8BhKoOmSJSBwAcB7ALwIsisiwio9UKJlS10zmFYURhJWGmcK9CZrrAKWomgJiez2LvDgYPREQULKKqmzuBddfNKKyA4klVLWmtoYhMAYiVukpCRCYBjKrqkFl9Mamqa24cISIKoLfQ+4jIAIABANi7d2/3qVOnSmmOq4WFBbS3t3s65p/8lw+xrQX4nae2la0dQVNKvzQC9os79os79os79ou7Qv1y+PDhSTNQsKFN3Y1TRJ6FlawYxb1lnVVnPvRnVXVos+cy+RgjANDT06OHDh3a7Clzzp49C6/n+9//8vt44qEuHDp0sGztCJpS+qURsF/csV/csV/csV/claNfSrkb57Mm92EZwDis6YznARwoNmopJ5PPEHMu13S8VtOltVUVM5ks7u/gFAYREQWLpxEIEVmBVYHyJoBvwCredLkSDfMgAeDpvG12nYcuWEWuAKwKKGqiDsTND+/izvIK9jCAICKigPE6hXESwLc2WziqXEwNiXh+sqSqpkUkhbU39+rCvUTLwJvOZAEA9+/Y6nNLiIiIVvO6CiMWoOBhAFalyZRjW9SxRHMCa2/8FUEN3fBrxgQQe7iEk4iIAqZgACEi3xORpbxtK2bJZqHHUqHzFaELLrcDF5GQiEyZgMHeZheDColIxDyiAPodAUUca0tjx8z2mjCdWQQA3M9lnEREFDDrTWFMwJE/YHwDVg5EWZichOOw6jKEACREpBfAmbwbY3XlHTNW4JS50QgzjREXkQSAc+Y9ErUyfQFYNSAAMImSiIgCp2AA4bYkUlXL+u3d5C6se06zT2fe8zX1HQocm4RPS0vLYSaTxbYtzWhv29RqWyIiorLzvIyTqmc6k8WejjaIFBUvERERVY3Xu3HeEJH9BV77noj8bjkaRZbpzCKnL4iIKJC8jkB0rvPaMNYmLdImTGeyTKAkIqJA2nBy3Yw4RBybnhaROZddB1BDN6mqBTOZLH7+0d1+N4OIiGiNYrLzngMwaH5XWMWkCtn0vSjIsnh3GZnFJRaRIiKiQNowgFDV50XkBKyVD7MAerG2FPSsqt6sQPsalr2Ek2WsiYgoiIpaH2gHByIyDuCcqs5XtFV0r4gUAwgiIgogr0mUJ2AVflpFRHaaO3TuL0ejyFHGmgEEEREFkNcA4gW4JEqaEQqBdWdMKoPcjbQ6mANBRETB4zWAeArAmQKvjWH1ag3ahOnMIpqbBLu2t/rdFCIiojVKqUQ5u852LuMsk5lMFrvbW9HUxCqUREQUPF4DiPOw6j24OYoaulV20NllrImIiILI612angdwXkT+DFa+wxysO2XGAUQBdJe3eY1rej6LfTuZ/0BERMHkKYBQ1aSIPAOrbPXLsApLCay6EM+o6ivlb2Jjms5k8bmP7vS7GURERK483ydaVScAHBCRR2DlPKRU9XLZW9bAllcUs7c4hUFERMHlOYlSRPabypTfhFVU6rKpA/FNEdlR/iY2nhsLWawoi0gREVFweb2d99MApmCVs+6Flf9g14E4AKtOBG3SdK6IFHMgiIgomLyOQCQAjKtqD6zcB6cRWCsxaJPsKpS8lTcREQWV1wAiAmC0wGtzAEKbaw4B9+6DsaedAQQREQWT1wAiCaCnwGsxsA5EWfBOnEREFHReV2G8AOC0iAisJZw7ReQgrKkN1oEok+lMFju3bcHWLc1+N4WIiMiV1zoQ4yJyBFa+g8AakRAAaQBHWAeiPGYyWa7AICKiQCulDsQ4gHEReRL36kBcKHvLGth0ZpHTF0REFGieAwibCRoYOFTAdCaLnoc7/W4GERFRQQUDCBF5FkBYVX/Pse1rRZ43DWBCVd/eXPMaj6piOpPF/TtYA4KIiIJrvRGI5wA8DeBtspx8AAAeU0lEQVT3HNuGPJxbRWRAVf9lSS1rUPOLS7iztMIcCCIiCrT1lnHGkLdkU1WbinnAqlD5ErwFHARgxq4BwQCCiIgCrGAAoaqXS02OVNU0gFMlt6qBsQYEERHVAs830/LgKKxbfpMHMwumjDXvg0FERAHmeRWGiOwHEIc1vREGkAJwBsALqjpv76eqR8rTxMbCEQgiIqoFpd6NMwbr3hdj5ufzAFImuKBNmM4soq2lCTu2lrzCloiIqOK8fkoNwxpxeEZVL9sbRSQEa7riWwB+sXzNazzWEs42WNXCiYiIgslrDkQXgIQzeABySZNxAL3lalijsspYM/+BiIiCzWsAcR7AbIHXpgBcLvAaFWk6k+VtvImIKPC8BhBxAMcLvPYCgMHNNYem5xdx/w4GEEREFGzrlbJegXXLbpeXZNltO4A+ALwHdYkW7y5jfnGJVSiJiCjw1kui/AbcAwiqkJkMa0AQEVFtKBhAqGq8mg0hK/8BYA0IIiIKvkpWoiSPeB8MIiKqFUXXgRCRg7CqT3abTZMAUqr6/XI0RET6AKRVdaIcx5naFAMAxmGtHOmCVQDrjNf3qJbcFAaTKImIKOA2DCBM4HASQARWoqSTisgkgGOq+mqpjRCRqHmP/jIe1wUgYR4AkDbtDGTwAFhTGE0C7NrOAIKIiIJt3SkMEXkWQBJAJ4AjAA6Y23V3whqJOA7g4wCSIvL3vL65iIRFZBjWPTUK1ZfYzHG9pq0HVLVTVce9trGapuez2NXehuYmVqEkIqJgKxhAiMgjsIb/R1T1UVV90a5Aqao3VfWCqg6paieAlwCMi8jDXt5cVVOqGlPVkUodp6ppVU15Ob9fpjOLXMJJREQ1Yb0RiASASVV9bqOTqGo/gAuw7pVBJZpZyDKAICKimrBeDkQUwO96ONcorLtyBknYJFkCVk7EbJCnMabns/jUAzv8bgYREdGG1gsgQrASD4uVNscExSwAOAMGERkTEQQxiFheUVxf4I20iIioNoiqe7FJETkP4G9U9deKOpHIaQCPqOpTJTVEZApArIRlnEUfZ1ZtDKvqgXX2GYC1/BN79+7tPnXqlJfmrGthYQHt7e2ur93MKv7nH9zGb3yyFdGHt5TtPWvBev3SyNgv7tgv7tgv7tgv7gr1y+HDhydVtaeYc6w3AvECgNMiMqqq317vJCJyDMCvwuMyTB+kYE1rhMwtyNcwiZkjANDT06OHDh0q25ufPXsWhc734ys3gR/8J3yh+zM49JkHyvaetWC9fmlk7Bd37Bd37Bd37Bd35eiXgkmUZpj/JKzVFf9CRNZMzovIQREZBfAtACdV9aVNtaaMRMTtzqD2ks9wNdtSDJaxJiKiWrJuISlVjYlIGsDvALB/t5dERsxPATCkqoFJoBSRMICEiIznLeHsMj8Dt6xzZp430iIiotqx4b0wzE21OgH8Hqzy1QfM42VYd+w8EKTgAbDqRMDKi8gPFKIAkoWmL/w0s8ARCCIiqh1F3QtDVW8CqPTdObvgsorD3NNiEkCiQOEo1+MAzIpI2A4izHliAI6Vr8nlMz2/iI6tLdi6pdnvphAREW2o6JtpVYL5UD8OKychBGvaoRfWDa+cSy27vB6nquMi0mfqQOwy+/UHtSrldIZFpIiIqHb4GkCYqYR1RzbMPp1ejzP7Ba7eQyFWAMH8ByIiqg0b5kBQdcxksryNNxER1QwGEAGgqpjOLGJPOwMIIiKqDQwgAiCTXcLi3RWOQBARUc1gABEAMxnWgCAiotrCACIApnNFpDgCQUREtYEBRABMZxYBsIgUERHVDgYQAcApDCIiqjUMIAJgJpNFa0sTdmzztSwHERFR0RhABMB0Jos97W0QEb+bQkREVBQGEAEwnVnkEk4iIqopDCACYHqe98EgIqLawgAiAGYWeB8MIiKqLQwgfJZdWkb69l0u4SQioprCAMJn95ZwMoAgIqLawQDCZ7kAgkmURERUQxhA+GyaRaSIiKgGMYDwmR1AMAeCiIhqCQMIn83ML0IE2LW91e+mEBERFY0BhM9mFrLYtb0NLc38pyAiotrBTy2fTc9nOX1BREQ1hwGEz6YzrEJJRES1h7d/9NlMJovH93X43QwiqnPZbBazs7PIZDJYXl72uzlVs3PnTly6dMnvZviiubkZHR0d6OrqQltb+b+oMoDw0cqK4vpCljUgiKiistks3n33XXR2dmL//v3YsmVLw9z9N5PJoKOj8b6kqSru3r2L+fl5vPvuu3jooYfKHkRwCsNHs7fvYGlFsaedAQQRVc7s7Cw6Ozuxe/dutLa2Nkzw0MhEBK2trdi9ezc6OzsxOztb9vdgAOGj6Xm7CiWLSBFR5WQyGezYscPvZpBPduzYgUwmU/bzMoDw0cwC74NBRJW3vLyMLVu2+N0M8smWLVsqkvfCAMJH0/OLAFjGmogqj9MWjatS//YMIHzEMtZERFSrGED4aCaTRUdbC7a1NvvdFCIiIk8YQPhoJpPFHi7hJCKiGsQAwkfTmUUu4SQiqrBkMgkRWfPo7u7G0NCQb+2Kx+M1nZvCAMJH05ksl3ASEVVJIpHA1NQUpqamMDk5iaNHjyIej6O7u9vvptUkBhA+muF9MIiIqiYUCiEcDiMcDiMSiWBwcBCTk5NIJpO+jkQ4xeNxpFIpv5tRFAYQPlnILuH2nWUGEEREPopEIgiHwzhz5ozfTUE6ncbQ0BCSyaTfTSkKAwif2DUguISTiMhfoVCoIqWe6x0DCJ/YNSBYRIqIyD+pVArJZBLRaDS3bXx8HN3d3blEy4mJidxr8XgcnZ2dSCaT6O3tze2TP2qQSqXQ39+Pzs5OiAh6e3vXnZqw97V/F5HcczvZMp1OrzpmaGjIdXu18G6cPpmxAwgu4yQiH/1f3/0xLl6Z97sZ6/rUgzvwf37502U9Zzqdxvnz59Hf349wOIxEIgHACh76+/sxODiI48ePY3R0FL29vZiamkI4HM4d29/fj0Qigf7+fsTjcfT392Nqaip3/uHhYQDAyZMnEQqFcsmac3Nzru05efIkYrEYent7MTw8vCqgicViGBoawsjICAYHB3PbR0dHEY1GEQqFyto3xeIIhE/ujUAwgCAiqoZYLJZbwtnZ2Yne3l5Eo1FMTk7m9jl27BgGBweRSCTQ19eHsbExRCKRXEBgs18fGBjAwMAAUqnUqpGARCKBsbEx9PX1IRqN4uTJk0in0xgfH3dtm53gCQBdXV25ZE8AuaRPZxvS6TSSyST6+/vL1j9ecQTCJ9OZRbQ2N2HnNt7ghoj8U+5v9kE2ODiIWCyWe25/QNuSyWQukTF/VUZXV9eq584RggMHDmz43pFIBABKXmFx/Phx9Pf3I5lMIhKJYGRkBAAwMDBQ0vnKgQGET2bms9jT0VbTRUSIiGrJrl271gQNTvaH++Tk5JppgfwAophpg/HxcYyOjiKVSm16aWZfXx9CoRBGR0cRiUQwOjqKvr6+TZ1zswITQIhIH4C0qk5suHORx4lIBEAPgBSAMICU1/NXysxCliswiIgCxBlcrBdoFMNOmozH4+jp6QGATResOnLkCEZGRnD8+HEkk8lc3oZfApEDISJRACfLeZyIhAEkVHVEVSdUdQRAzGz33fQ8AwgioiCJRCK5hMd8XlY6pFIpTExMIJFIYGBgIFdrYiP5oxz54vE40uk0RkZGEAqFVk2j+MHXAEJEwiIyDGt0oOhFuEUeFwMwnLdtGIC/IZsxnVlkAiURUcCcPHkSExMT6O/vx8TEBMbHx9Hb2+saVBQSDodzgcj4+HjufBuxp0WGh4dz75t/3kgkghMnTuDIkSPe/rAK8DWAUNWUqsbM6EC5j+sDkF/O67zZ7qs7SyuYu32XNSCIiAKmr68Pk5OTSKfT6O3txbFjx1Yt8yzWyZMnMTs7u2qZZzEGBwcxMTGBeDyeS7x0isViSKfTq5JB/RKYHIhyEpEQXEYnVDVtlvCEVdW3YuPXF1gDgoioWiKRCFTV0/6FSlsnEok1wYS9lNOpr69vTZJj/j5u53Lb5pROp3MjEX4LRA5EBXQBVsBQ4HVf8yDsGhC8lTcREXkxPDwciNEHoE5HIACUXJZLRAYADADA3r17cfbs2XK1CQsLCzh79iySHywBAH76k9dxdvpS2c5fq+x+odXYL+7YL+7W65edO3cik8lUt0EBsby8XPN/++XLl/Hqq68imUwilUrh13/91z3/TYuLi6uuj3L8d1SvAUTJTF7FCAD09PTooUOHynbus2fP4tChQ3j/R+8AF17Hlw79V9i3k3kQdr/QauwXd+wXd+v1y6VLl9DR0VHdBgVEJpOp+b99aWkJX/3qVwEAY2Nj+NjHPub5HFu3bsWTTz6Ze16O/47qOoAQkdA60xi+mZ7PQgTY1d7qd1OIiCjgIpFIwXto+KlecyDsBMlVi2pNcqXzdV9MZ7Louq8VW5rrtfuJiKje1eUnmBl1SGFtLkQXrKqVvgYQMxkWkSIiotpWlwGEMQGrjLVTxGz31UxmEffvYO4DERHVriAFEF1wWT0hIiERmTKrI4o+DkAcQH7ljpjZ7qvpTJZLOImIqKb5mkRpchKOw6rLEAKQEJFeAGdU1XnTdLdchnWPM0Wj4iKSAHDO7Jvwe/piZUUxk8myiBQREdU0XwMIk6uw7oiA2afT63FmvyTWlrP2VfrDu1haUd4Hg4iIalqQpjAawnRmEQB4HwwiIqppDCCqbHrelLHmCAQREdUwBhBVZt8Hg1MYRERUyxhAVNlMhnfiJCKi2scAosqmM4tob2vBfa11XUWciCiQkskk+vv70dnZCRFBZ2cnent7MTFhlQiKxWIQEaRS6y/Yi8Vi6OzsRCqVgoigu7u74L7d3d3o7OxEOp3OtUFE1jy6u7sxNDRUvj+2whhAVNk0q1ASEfliZGQE3d3dSCaTOH78OMbGxnD8+HHMzs5ieHgYAHK3yrafr3euI0eOIBwOY3h4GMlkEvH42sWBQ0NDSCaTOHnyJEKh1SWLEokEpqamMDU1hcnJSRw9ehTxeHzdYCRI+DW4yljGmoio+iYmJhCLxRCNRnHmzJlVrw0ODuZGByKRCMLhMEZGRpBIJFzPNT5ulSmyg42BgQGMjY1haGgIR48eRSQSAQCkUinE43H09fWhr69vzXlCoRDC4XDueSQSQTQazY1EDA4Obv4PryCOQFTZTCbLBEoioiqLxWIIh8Nrggebc3QgFoshnU7npjXyjY6OIhwO5wIFwLrNdigUQn//vQLI/f39CIVCOHnyZNHttAOYQu0MEgYQVTY9v8gaEEREVTQ+Pp4bDSiGPVowNja25rV0Oo3x8fHc6IMtFAphbGws9z721IUdWHgRCoUwOzvr6Rg/MICoosUlxa07y5zCICKqonPnzgEAjhw5UtT+9ujC6dOn17xmb3ObkohGoxgYGMDQ0BDi8TgGBgYQjUY9tTWVSiGZTOaOi8fjEJHcFIttaGjIdXs1MQeiim5mFQBrQBBRgPzp88C11/xuxfr2fRb40gslH26vqPAyEhCLxRCLxTAxMbEqCBgbG8tNM7gZHh7G6dOnkU6nC+ZQuEmn0zh//jz6+/sRDodzx8ZiMQwNDWFkZGRVTsTo6Cii0ajn0Y1y4ghEFaXtAII1IIiIqqaUb+n2aIVzNYadF5E/feE0Pj6ee7+RkZF138NeMupcThqNRjE5OZnbxx4NyW+HvRzVTxyBqCJ7BIJTGEQUGJv4Zl8rIpEIJiYmkE6ni/7GHgqF0NfXl1txAdwLCApNhaTTafT396Ovrw/hcBjxeBzRaHRVsqXT4ODgqmCk0KjG8ePH0d/fj2QyiUgkkmvHwMBAUX9LpXAEoopyIxBMoiQiqpqnnnoKAFxzGtZz9OhRAPeWbY6OjqKvr69gEOJcdZFIJBAOh9cdJdi1axfC4XDuUYj9nqOjo6va4TcGEFV0M6vY0izovG+L300hImoY9oiAl5wE+zj7g9ueNrCDinwjIyOYmJhYVTDKXpWx3pRHsY4cOYKRkZFcO8pxzs1iAFFF6axiT3sbRMTvphARNZTh4WGkUqmCIwLJZNJ1+5EjRzA+Pr7u6ot0Op0rUuV8PRKJYHBwECMjI6umQkoRj8eRTqcxMjKCUCjkeXVHJTCAqKKbWWX+AxGRD6LRKMbGxjA+Po7u7u7ch/rQ0BB6e3vx9NNPux5nf9O3l2W6sYMSt7oR9lTGsWPHNrXk0k6mPHHiRNHLUSuNAUQV3byj2MP8ByIiX/T19WFqaiqX4Njf348TJ04gFAqtWvngZC/ZtEcZ8tlTF8PDwwVzI8bGxnIJlpthV8gMwvQFwFUYVZXOrnAJJxGRj8LhsOtIwXqmpqYKvjYwMLDhaohIJAJVLfi8WOl0ek0JbT9xBKJK7i6vIHOHRaSIiKg0w8PDgRl9ADgCUTXXF7IAWAOCiIiKZ5e2PnfuHFKplO+1H5wYQFTJTMYKIFgDgoiIipVOp3Hs2DEAKOnGXJXEAKJKpuftAIIjEEREVJxIJIK5uTm/m+GKORBVMp3hFAYREdUPBhBVYk9h7G5nAEFERLWPAUSV7NvZhif2NKO1hV1ORES1jzkQVXL0qYew91bK72YQUYNSVZbRb1Cl1JwoBr8OExHVuebmZty9e9fvZpBP7t69i+bm5rKflwEEEVGd6+jowPz8vN/NIJ/Mz8+jo6Oj7OdlAEFEVOe6urowNzeH69ev486dOxUb0qbgUFXcuXMH169fx9zcHLq6usr+HsyBICKqc21tbXjooYcwOzuLt99+G8vLy343qWoWFxexdWtjFvBrbm5GR0cHHnroIbS1lX8FIAMIIqIG0NbWhgceeAAPPPCA302pqrNnz+LJJ5/0uxl1iVMYRERE5BkDCCIiIvKMAQQRERF5xgCCiIiIPGMAQURERJ4xgCAiIiLPGEAQERGRZwwgiIiIyDNhSdPCRGQGwDtlPOVuANfLeL56wX5xx35xx35xx35xx35xV6hfHlbVPcWcgAFEFYnIeVXt8bsdQcN+ccd+ccd+ccd+ccd+cVeOfuEUBhEREXnGAIKIiIg8YwBRXSN+NyCg2C/u2C/u2C/u2C/u2C/uNt0vzIEgIiIizzgCQURERJ4xgCAiIiLPWvxuQCMQkQiAHgApAGEAKVWd8LdV/hKREIABAOMAZgF0AYgBONNofSMifQDSbn93I187hfqlka8d0ydhAAfMz2FVHc/bp+GumY36pVGvGRGJAugFcANW30yq6kjePqVfL6rKRwUf5h/kTN62MQBhv9sWgH5Rx2MOQJ/f7fKhH6Lmb48W6KOGvHaK6JeGu3YA9AGIOJ6HAEwBGGjka8ZDvzTUNWP+G4rmbZsEMFiu64VTGJUXAzCct20YQMKHtgRNL4BOAAdUtVPzvknVMxEJi8gwrP+AZwvs1nDXTpH9AjTmtRNW1aT9RFXTsK4F5zXScNcMiusXoPGumZjLtom87Zu6XhhAVF4fgGTetvNme8NT1bSqpvxuR7WpakpVY5o3nJin4a6dIvvF3rdhrh0zBH/U/HSaMK+HzfOGumY89AuAxrpmjF6XbWnH75u6XhhAVJC5qNd8kzIR8pqLm8jGa4eczL972DxcNeI1U0y/NCpV7VfVeN7mPgCjQHmuFyZRVlYXcO8fxEUYVuJKowqb5CfA6qvZBhhWLBavnfU13LWjqp0um6OwEk1T9v/wG+2a2ahfHNsa7ppxEpEBAElVHTKbNv3/GAYQlZU/rEb3zAKArs6UHhMRNNJ/1OvgtVMYr517YgBOmN95zdzj7Begga8ZEzT1AtaohOOlTV8vnMIgX5i5yPx57npP9qIy4LVjMd8oZx3fKAnu/dLI14yqjqtqDEBcRCbNss2yYABRBS4JPuQuBWuYkf1lsC+K1lDXjpmuiKnqmiS5RukDN+v1i4uGumbMVMUwgJed2zfz9zOAqCx7/qjLudHxD1Z385HFEpFBl812Mg8TonjtFMRrB4D1zfnpvG28Ztz7hdfMPRMAQqbA1KavFwYQFWQivhTWzjV1YW2CT8Mw3xISLlm+9oXckP3ixGvHHa8dwNTJiOcnvzX6NVOoXxrxmjH1VObWma4IleN6YQBReROwyoQ6Rcz2hmQuzJjLBRqFlSVcKCu40fDaydPo146Z3084/34RiTo+HBvymlmvXxr0mgnBCg7y/2b7OrFrP2zqemEAUXlxAP1522JmeyObdX4jMMNmMQDH/GuSb7rgnhHd6NdOoX5pyGvHsQQxJCIR84gC6Hd8ODbcNVNkvzTUNWMqc466vBQHMFSu60VM7WuqIDOMdBTAOVgRYFLr+AYuxXLcAGcXrA+KRL0Ps9rM/8COw/r7+2B9U5iAVZfeudSsoa4dD/3SUNeO6Ze5Ai+nVPWAY9+GuWY89ktDXTNAbmTmADa+mVZJ1wsDCCIiIvKMUxhERETkGQMIIiIi8owBBBEREXnGAIKIiIg8YwBBREREnjGAICIiIs8YQBAREZFnDCCIimDK4qp5DLi8Hi70WoXblRCRwBdzEZFhU5t/3T4SkUEROVPNtnlVC210MtfmGdP3w363h+pHi98NIKpBcQAjG+5FAAARGYNVX98uG7xe9b+nYN2jIMhqoY05ptpirwk0aybwoeDjCASRNyMAwo76+7SxPlh3SRw3j2ShHVW1X1XFuc2MslT9lsuF3tetjUHnuCtjXZa0Jn8wgCDyZgzWN+jjfjekFjg+gEu646G518EgrBGMqvHrfSsoCuveEPV450nyCQMIIu8SACKOb3VEQdcLjj5QmTGAIPLI3M0uDSuQKMgtwVFEBkwyWyhvvzlzG+JJ8/qkSdx0JsBNFZo6MceeMeeZMrczzt+nL//8+W0wvw+bc7jdSrvQ+9rnHHC8lgAwZZ7a+6w79ePsM5M7Yd9pccwcP5e3v+e/yfTpmCOp80zerZ7Xfd8C/64F+yG/PS77RvL26zNtVbf2lSgKk/9g+mDQPNa9honWwwCCqDQnAETLODcfgjU9cgJADNZtdcccj36z31iB408CGIaV4AlYH9jOD8U+c+yEOVcqfx+z3ySAHgDDGw13m3NOmnP1AhgFMOzI9D9htsO064DzltxFOOY4PgbrdsTdZfibYnnn7zJ/R1Hvm6+IfnCy/52Hsfrf2T6X/XzcvKf9d20YzK3Tvlz+gwkKj6jqkPm7mMtDpVNVPvjgY4MHrG9wCiBqnofM82HzPGyeDziOSVj/ia06z4DZL5S/n33uvG2Djm19Zltkg/cIO9tmts0BSOTtN2lvc7zfGQ99Mud8jwL9ZLelr8hzrvp71ju+XH8TrDyHVe+xwfvmt3HDfshrT5/LtlDev3FovTZ7vHYHYY0EhfKuz1A534ePxntwBIKoBGp9kx0BMFDMUH+Rzjt+t4f+nfPW9vLHrg3aljLH9QC5b6AhAIOOYXGF9cGZn8cRQxHMVEEI1jdp53tPwJre6Xc7rlzK+TfpvVUhnkeTSuwH57/pVIHXXjZTDOXIs7FHU46oNf1mtzGtTKqkTWAAQVQ6e/64LMWjCvzPvNT/wadw7wPR/tkNa9ja+Vj1AWeCj2LY53Tb3/nelbKpv8nkGYyZHIS5/NdLaEfR/bDeh7Z5zf7ATwCYlM0XrYrCTIm45VwQlYqFpIhKpKopERmHtaQzaBnuYdz7UMt9uHkIEDZinycMIL+uQxirR1MqoeS/yXwgh2F9QNvtnCx8RFHtKFs/mNGLbjOyNQAgISKDauUteOIIFk6oalpEBmHlWBzwei6ifByBINqcE7CGsI8WuX/BZLxyMR88UZigxgzRu64a2cT0i/3BuGp6wCQU2omC5TDrtrHUv8kkKUZhFbYaMedxC0Bc39dFxfrBTDEMwQpMnirxNPn1H1IwU2BmNQaTKKlkHIEg2gRVTYpIElaiWr4pwLp3AqwP8yjKNN2Rzyw9jOPeN+s0rODGdgzWkkR7BUAI1odeCkXmPTiZb7MxWKsNumCtPLDfe9x8i9408z4AELN/qqo9xO/5bzKjRmlY3+oBq5/iLvut9775+5WtH8zyz7j5e+wpkIh5bgdACVgjCgUrejq41X+wA6ZVORFEXnEEgmjz1nwAGadhfXtMwPomugslfFgXaRTWOv8zsL49dzvn2tVaPtkN60P2DKxln6l12r4h8+HTi3tLEWOwvtmXO4FyCFbwlYBjmmATf9MxWN/Cx3Dv36bo981X5n44DStfIeY415Djgz4Ea6VGT5Hn64IjwdP0WcoEtcWOshC5EtXA38iPiIgMEZkC0F/kCARRxXAEgoioxjB4oCBgAEFEVCNMgmjJ005E5cQpDCIiIvKMIxBERETkGQMIIiIi8owBBBEREXnGAIKIiIg8YwBBREREnjGAICIiIs8YQBAREZFnDCCIiIjIs/8fPqM7PFs6LuIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 576x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(8, 6))\n",
    "fontsize=20\n",
    "plt.plot(f_conv, label=\"Penalty\")\n",
    "plt.plot(np.ones(num_iters) * cvxpy_f, label=\"CVXPy\")\n",
    "plt.legend(fontsize=fontsize)\n",
    "plt.xlabel(\"Number of iterations, $k$\", fontsize=fontsize)\n",
    "plt.ylabel(\"Objective, $f(x_k)$\", fontsize=fontsize)\n",
    "plt.xticks(fontsize=fontsize)\n",
    "plt.grid(True)\n",
    "_ = plt.yticks(fontsize=fontsize)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhMAAAGJCAYAAAAwtrGcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzs3Xl8VOXZ//HPlQWCbCEgVJQtoCiKQAiySCxYcEGxKmFxXyFoBWq1oO3T1uf31GqwVQEVAm7VVlm1VRGVoGiQRQiIFDckoFXcISgqKHD//pgTHcJMyGQmOZPM9/16zSvkzJkzV24OzJV7uW5zziEiIiJSVUl+ByAiIiK1m5IJERERiYqSCREREYmKkgkRERGJipIJERERiYqSCREREYlKit8B1DQzywRygbVAFjDTOVfqb1QiIiK1lyVanQkzK3bO9fT+nA7Mcs4N9zksERGRWqtW9kyYWS5Q6pwrDPFcFpANlACZQEnZed5z28vOdc6VmtmgmolaRESkbqp1yYT34T8LOKg3wRvCyHfODQ46Ns/MSpxzZclF+SGN7WaW5ZxbW51xi4iI1FW1JpnwEoVJQDFBvQvl5AEF5Y4VAPkEko+Mqr5/ixYtXPv27av68oN88803NGzYMGbXqyvULqGpXUJTu4SmdglN7RJauHYpLi7+wjl3eGWuUWuSCa9nIQ/AzCaFOS2Xg5OJNcBi78/bgfRyz1cqwWjfvj1r1qypXLCVsHTpUgYMGBCz69UVapfQ1C6hqV1CU7uEpnYJLVy7mNn7lb1GnVka6k2mzKRcr0XZSg2vZ6OEEMmDhjhERESqrs4kE3hJQgXLPDPLJw1egnHQJE4RERGpvFozzFEJ5YcvwhltZhP5qc7E6OoLSUREpO6rlXUmzGwzkBe8NNRb9lnsnLMQ5ztgcKilpId4nzHAGIBWrVr1nD17dnSBB9m1axeNGjWK2fXqCrVLaGqX0NQuoaldQlO7hBauXQYOHFjsnMuuzDXqUs8EEJg7EauKls65mcBMgOzsbBfLiTuaCBSa2iU0tUtoapfQ1C6hqV1Ci0W71KU5EyXe1wMmWHoTM4OfFxERkRiqM8mE1xtRQuiln6Xe0lIRERGJsTqTTHgKCZTSDpaFVmyIiIhUm9qaTGQQevXGJA4us53nHRcREZFqUGsmYHpzH24mUJgqHcg3s8HAYufcfPhx465JZpYPrPbOzdcQh4iISPWpNcmENyfikD0MXmEqVbQUERGpIbV1mKNW2/3DPlZ/spfaWONDRESkPCUTPvjHyve59/U9TJj9Ol/t/sHvcERERKKiZMIHV5zcgfOPTmXhho8ZMqWI4vfD7aguIiIS/5RM+CA5yTinYz3mje2LGYwoWMmUwk3s3bff79BEREQipmTCR1ltm/Hs+ByGnngEdxW+ywWzVvJR6Xd+hyUiIhIRJRM+a5yWyt2jenDXyG689fHXnHH3Kzzzxja/wxIREak0JRNx4rweR7FwfH86Ht6I6x5bx2/nreebPXv9DktEROSQlEzEkXbNGzJvbF+uG9iJ+Ws/5Oxpy3jjw5hsgCoiIlJtlEzEmdTkJG48vTOPj+7D7h/2cf59y5nx8mb271dNChERiU9KJuJUn8zmLJqQw6DjWnH7ore55MFVfPrVbr/DEhEROYiSiTiWflg9pl+cxe3nd2Xt+6WccfcrvLDxE7/DEhEROYCSiThnZow6qS1Pj+vPEU0bMObRYv747/+w+4d9focmIiICKJmoNTq1bMSTv+rH1f078MiK9zn33lfZ9OnXfoclIiKiZKI2qZ+SzP+c3YWHLu/F51/vYeg9y3hs1QfaMExERHylZKIWGnhsSxZNyCG7XQa/e3IDv3psLTu/1YZhIiLiDyUTtVTLJmk8cuVJ3HTmsbyw8VOGTC1izVZtGCYiIjVPyUQtlpRkjP15R+Zf04/kJGNEwQqmLtnEPtWkEBGRGqRkog7o3iadheP7c/aJrblz8btcOGslH+/UhmEiIlIzlEzUEY3TUpkyqjt/Hd6NDR/t5MwpRapJISIiNULJRB1iZuT2PIpnxvXnyHTVpBARkZqhZKIOyjy8EU9cq5oUIiJSM5RM1FGhalI8/ppqUoiISOwpmajjympS9GzXjJuf2MB1j61j53eqSSEiIrGjZCIBtGySxqNX9ua3p3fmuY2fcNbUItZ+sMPvsEREpI5QMpEgkpKMXw3sxNy8vjgHw2es4L6l77FfNSlERCRKSiYSTM92zXh2Qg5nHP8zJj/3Dpc++Bqffb3b77BERKQWUzKRgJo2SOWeC3vwl/O6snrrdoZMKeLldz/3OywREamllEwkKDPjwt5teXpcfzIa1uOyB1/jtmff4vu9+/0OTUREahklEwnumFaNeeq6/lzUuy0Fr5QwfMZyPvjyW7/DEhGRWkTJhJCWmsyt53Vl+kVZbPniG4ZMLeLfr3/kd1giIlJLRJ1MmFmTWAQi/juz6xE8OyGHzj9rzITZrzNx/nq+/X6v32GJiEicq3IyYWYdzOw9oNTMvjSz84KeG2Zmm2ISodSoo5odxpwxfbhuYCfmFX/I2dOW8ea2r/wOS0RE4lg0PRO3A3lAM2AkcIGZnQ/gnFsAdIw+PPFDSnISN57emX9e1Ztdu/dy7n2v8siKrSrFLSIiIUWTTKxxzi1xzu10zhU650YAzc3sVO95ffLUcv06tWDRhBxO7ticP/57I3mPFlP67fd+hyUiInEmmmSitPwB59wswMysRxTXlTjSvFF9HrisF/9z1nG89M5nDJlSxOqt2/0OS0RE4khUPRNmNtrMNplZ+7KDzrkl3h93RhOYxI+kJOPqnEwWXNOP1JQkRhasYOqSTexTKW4RESGKZMI5tw6YC4xwzm0N8VzP6EKTeHPiUek8M64/Q7u15s7F73LR/Sv59CuV4hYRSXRRLQ315kusC/PclmiuLfGpcVoqd4/szh25J7L+vzs5c0oRL779qd9hiYiIj1S0SiJmZgzPbsPT4/rTqkkaVz68hv975k327N3nd2giIuKDlFAHzWwGkE30KzIM2OycGxnldSQOdWrZiCev7cfti97mgWVbeG3LdqZd0IP2LRr6HZqIiNSgkMmEc25sTQcitVNaajK3nHM8/To2Z+KCNzhrahG3nteVc3sc6XdoIiJSQzTMITFx2vE/49nxORzfuim/nvM6N8xdzzd7VIpbRCQRKJmQmGmd3oDHRvdm/C+O5ol1HzJ02jI2btMKYRGRui7cnInbgQ4xeo8S59zNMbqWxLmU5CR+M/gY+mY259dz1nHevcv5/VnHcWnfdpiZ3+GJiEg1CJlMALcBGTF6D5VLTEB9OzZn0YRTuHHeev701EaWvfcFd+SeSPph9fwOTUREYizcBMydqIKlRCmjYT0euCybB1/dyu2L3mLIlCKmXNCDXu1jlaeKiEg80JwJqVZmxlX9O/DENSdTT6W4RUTqpIiSCTNrYmZzzWyf91htZjdUV3BSd3Q9qinPjM/hHJXiFhGpcyLtmZgMrAZGADcBW4A7zOxLM+sW6+CkbmlUP4W7VIpbRKTOCTcBM5xib5vxA5hZLjDfzAaX3/RLJFhZKe4ebZsx7vF1XPnwGq7u34GJZxxLvRSNuomI1EZh//c2s+lmdp6ZNQk6HHKg2zk3H+gF5MU4PqmjykpxX9a3Hfcv28Kw6cvZ+sU3foclIiJVUNGvgnnAAmCHmW0ys+lAupndFupk51wpsLkaYpQ6Ki01mf/95QkUXNKTD7Z/y1lTi1ixTVUzRURqm4qSiULnXBKBHoeZQEcCcyYmlSUXZna1mXUHMLP2QM9qjjdqZpZlZmPMbKKZzTOzTL9jSnSnH/8znp2Qw3FHNKHgjT38dt56vv1eSYWISG1xqJ4JnHNrnXN3OOdO85KLngSSi+YEkou1ZrYPmAfkV3fA0TCzdCDbOTfTOTcZKAAW+xyWAEemN2D2mD4M7ZjK/LWBUtxvbvvK77BERKQSwiYTzrktYY6v85KLEc65DKCZcy7ZOdcrmsmXZpZrZoPCPFfWmzCo7GsV3yYTmBT0/Rog00syxGcpyUkMO7oe/7yqN1/v3su5973Koyu24pxqUoiIxLOop8971TKj4iUHB60S8Z7LBPK93oRC59xMIK8qwxPOubXA4KBD2UCpN99D4kS/Ti14dkIO/To25w//3sjYfxSz89sf/A5LRETC8HUtnpllmlkBgR6DcHt45BEYjghWQBWHVJxzJeWuPboq15Hq1aJRfR68rBe/H3IcS976jCFTi1izVdu8iIjEo0jrTMSU98GeB2Bmk8KclsvBycQaguY6mNkYAhNEw1nsnCsMPuC9Zo63rFXiUFKSMfqUTE7qkMG4x9cxcuZKfjP4GMb+vCPJSdqBVEQkXviaTByKN5fhoF4L51ypmWFmmc65Em/oI5LrDiKwNXrhIU8W33Vrk87C8f353ZP/4Y7n32H55i+4a0R3WjZJ8zs0EREh/jf6yoAfa1iEEvG8CTPLAraXJRJe9U6Jc43TUpk6qjv5w7pS/P4OzpxSxNJ3PvM7LBERASxeZsqb2WYgL7i3wPvgL3bOHdSnbWYOGBxJ74I3abMYCF69UeKcCzlE4g2FjAFo1apVz9mzZ1f2rQ5p165dNGrUKGbXqysq0y4f7drP9Nd38+Eux5kdUhl2dCopdXzYQ/dLaGqX0NQuoaldQgvXLgMHDix2zmVX5hpVHuYws6bAaOfcX6t6jZrmzdFoFsH5MwnU1CA7O9sNGDAgZrEsXbqUWF6vrqhsuww7fR9/Xvgm/1j5Adv2NmTaqB60bX5Y9QfoE90voaldQlO7hKZ2CS0W7RLNMEcGgeqY1U51IKS8tNRk/nxuV6ZflEXJ57s4a2oRz7yxze+wREQSUrzPmShbxpkRfDAouShBEtqZXY/g2fE5HN2qEdc9to6bn3iD777f53dYIiIJJa6TCW/iZQkHznGAQHJRWq5mhCSoNhmHMSevL9cO6Mjs1f/ll/cu491Pv/Y7LBGRhBHXyYSnkEClymBZ3nERAFKTk5h4xrE8cuVJbP/mB4ZOW8Zjqz5QKW4RkRoQT8lEBgf3QEBgL43h5Y7lceAeGyIA5Bx9OIsm5HBShwx+9+QGrntsHTu/UyluEZHq5Hc57XQzyzezeQQSiXwzKwiu/eANdUzyzss1s4kE9urQEIeEdHjj+vz9ipO46cxjeX7jJ5w1tYh1H+zwOywRkTrL73LapVSih8HboGtt9UckdUVSkjH25x05qUMG4x9fx/AZK7jx9M6MyckkqY7XpBARqWnxNMwhEnNZbZuxcHwOpx3fitsXvc1lD73G51/v8TssEZE6RcmE1HlNG6Ry74VZ/OW8rry2ZTtnTili2aYv/A5LRKTOUDIhCcHMuLB3W566rj/NDkvlkgdXMfm5t/lh336/QxMRqfWiTSY0+Cy1SuefNeap6/ozqlcb7lu6mZEFK/hwx7d+hyUiUqtVOZlwzm1ByzOlFmpQL5nbzj+RaRf0YNOnuxgypYhFGz72OywRkVorqp4JL6EQqZWGdmvNwvE5dGjRkGv+uZb/+dcGdv+gUtwiIpHSnAlJaG2bH8a8sf3IOyWTf6z8gHPvfZX3PlMpbhGRSCiZkIRXLyWJm4ccx8NX9OLzr/cwdNqrzF39X5XiFhGpJCUTIp4BnVuyaEIOPdqmM3HBG0yY/Tpf71YpbhGRQ1EyIRKkZZM0Hr2qN789vTMLN3zM2dOW8caHpX6HJSIS15RMiJSTnGT8amAn5ozpww979zNs+nLuLyph/34Ne4iIhKJkQiSM7PYZPDshh1OPbcmfF77FVX9fzZe7VIpbRKS8mCQTZjbXzM4P+v4XZtYkFtcW8VP6YfWYcXFP/u+Xx/Pq5i8ZMrWIFZu/9DssEZG4EqueiQLn3BNl3zjnlgDZMbq2iK/MjEv6tudf155Mw/opXHj/Su584R32qhS3iAgQu2SixMzaA5jZC2a2Ghgco2uLxIUurZvwzLj+DMs6iqkvvseFs1axrfQ7v8MSEfFdlZMJM9tkZtPN7FSvEma6N9RRAAxyzt0csyhF4sRh9VL46/Bu3DWyGxu37WTI1CIWv/mp32GJiPgqmp6JBUAhMMLMNgGTgXxgs3NuZyyCE4lX5/U4imfG53BUswaMfmQNtzy1kT17VYpbRBJTSlVf6Jy7yfvjAgAz6wAMAn5nZlnAYufcNdGHKBKfOrRoyIJr+pG/6B0efHULq7du554Ls+jQoqHfoYmI1KiYLQ11zm1xzs1yzo1wznUi0FMhUqfVT0nmj0O7cP+l2XxU+h1nTy3iyXUf+h2WiEiNijiZqOyST+0oKolkUJdWLJqQw/Gtm3L9nPXcMHc93+zZ63dYIiI1IqJkwszmAjvKVm4EHT81hjGJ1EpHNG3AY6N7M+EXR/PEug8ZOm0ZG7dp+pCI1H2R9kwsBkY457aWO15sZjeqUJUkupTkJK4ffAyPXd2Hb77fy3n3LeeRFVu1A6mI1GmRJhPpBBKKAzjndjrn/gqMiElUIrVc347NeXZ8Did3bM4f/72RvEeLKf32e7/DEhGpFhElE865O4B5ZnabmQ2spphE6oTmjerzwGW9+J+zjuOldz5jyJQi1mzd7ndYIiIxF+mciRmAEahuucTM9pnZaq941W1Az+oIUqS2Skoyrs7JZP7YfqQkJzFy5krueXET+7QDqYjUIZEOc2x2zp3mnMt2ziUBpwNLgF4EakxMinWAInVBtzbpLBzfnyFdj+CvL7zLpQ+u4rOvdvsdlohITESaTJQGf+OcK3TO3eScywbGoDkTImE1Tktl6qjuTB52IsXv7+DMKUUsfeczv8MSEYlapMlEoZldHea5HdEGI1LXmRkjerXh6ev6c3jj+lz+0Gpue/Ytvt+rHUhFpPaKdALmFgITMK8OXgbqldIuQXMmRCrl6FaN+devTubiPm0peKWE4QUr+ODLb/0OS0SkSiKugOktA73fOfdV0LEtBCZlqoS2SCWlpSbz53O7ct9FWZR8vouzphax8I2P/Q5LRCRisdybY4lKaItEbkjXI3h2fA6dWjXiV4+t5eYnNvDd99qBVERqj5glEwBmNj2W1xNJFG0yDmNuXl+uGdCRx1/7gF/eu4x3P/3a77BERColqmTCzJqYWXczO9WbmKnVHCJVlJqcxKQzjuWRK09i+zffc849y3j8tQ9UiltE4l5KuCfMrDuQAWQCHQmU0s4MOpZe/iWA/tcTidIpxxzOsxNyuGHuem5+YgOvvvcFQw7XPy0RiV9hkwkCkykHEUgQzDtW4j2WEEgoCoDtBOpPbAHmVlukIgmkZeM0/n7FScx4ZTN/e+FdVtWH1seW0r1N+RxeRMR/YYc5nHOnERi2mAU0c84lOec6eRUwRwBrnHOznHMLvMmXJUBhDcUtUuclJRnXDujE3Ly+7HeQO305BS9vZr9KcYtInKlwzoRzbr5zbiww3Bv2OODpEOffFMvgRAR6tmvG/zu5AYO7tOK2RW9z+cOr+WLXHr/DEhH5UaUmYDrn7gd2mtmp1RyPiITQMNW476Isbj3vBFaVfMmZU4pYtukLv8MSEQEiWM3hnNvinHvRW73RoTqDEpGDmRkX9W7Hv687maYNUrnkwVXc8fzb/LBPpbhFxF9VqYD5ulecam1wSW0RqRnH/qwJT1/Xn1G92nDvS5sZWbCC/25XKW4R8U+V60w452YBzb0aE00AzOz8mEUmImE1qJfMbeefyLQLerDp010MmVrEog0qxS0i/oiqaFXZ0Acw2MxmEFj5ISI1ZGi31iwcn0Nmi4Zc88+1/P7JDez+QaW4RaRmxaSctnNuATCTgwtZiUg1a9v8MOaN7UfeKZn8c9UH/PKeV9mkUtwiUoNiudHXWgLFrESkhtVLSeLmIcfx8BW9+GLXHobes4zZKsUtIjUkpht9AXkxvp6IRGBA55YsmpBDz3bNuOmJDYx7fB1f7f7B77BEpI6LaTKhLchF/NeySRqPXNmb357emUX/+YSzpy5j/X9L/Q5LROqwWPdMiEgcSE4yfjWwE3Pz+rBvv2PY9OXMeqVEpbhFpFqE3OjLW5mRTfS7gBqw2Tk3MsrriEgV9GyXwcLx/Zm04A1uffYtXt38BX8b3o3mjer7HZqI1CEhkwlvPw4RqQPSD6vHjIt78o+V7/N/C9/izClF3D2yO/06tfA7NBGpIzTMIZIAzIxL+rbnX9eeTKO0FC56YBV/e+Ed9qoUt4jEgJIJkQTSpXUTnhnXn2FZRzHtxfe4YNZKtpV+53dYIlLLhZszcTsQq828SpxzN8foWjFlZgXOOS1nlYRyWL0U/jq8Gyd3as7/PPkfzpxSxB25J3La8T/zOzQRqaVCJhPAbUBGjN5je4yuE1NmNggYg2pjSII6r8dRdG/TjHGPr2XMo8Vc3q89N515LGmpyX6HJiK1TLgJmDuBnTUZiJnlAqXOucIQz2URWF1SAmQS6O046LwI3iudQJKjxfeS0Dq0aMiCa/qRv+gdHnx1C69t2c49F/Yg8/BGfocmIrVIXMyZ8HoJQm4SZmaZQL5zbqZzrtA5NxPI845X1SCv/LdIwqufkswfh3bh/kuz2bbzO86etown1n7od1giUov4mkyYWaaZFRDobQg3HJIHFJQ7VgDkV/E9BwFV7tUQqasGdWnFogk5nNC6Kb+Zu54//vs//KDVHiJSCeHmTNQI51wJ3pwFM5sU5rRcDk4m1gCLy74xszFAxwrearFzrrBseMM5p+ENkRCOaNqAx0b3Jv+5t5lVtIW3P/ma+y7KooWKXIlIBXxNJg7F+/A/qNfCOVdqZphZpnOuxBv6qIxBQIaZZXvfp3uJSKGX2IgkvJTkJH5/VheOb92USQve4Jxpyyi4JJuuRzX1OzQRiVNxMWeiAhkQSB7CPB/RvAnn3Hxv7sXMsgTE+7MSCZFyzu1xJAuu6YeZkTtjueZRiEhY5lx8bPxjZpuBvOBVGt4qjmLnnIU43wGDq7Kqw+vxGENg3sUkYH6ohMLrtRgD0KpVq56zZ8+O9K3C2rVrF40aacZ8eWqX0Pxsl6++d9z3+m7e3r6f09ulMKJzPZKTDvon6QvdL6GpXUJTu4QWrl0GDhxY7JzLDvGSg8T1MEd18Xo6JnuPis6bCcwEyM7OdgMGDIhZDEuXLiWW16sr1C6h+d0uZ/5iP7cufIuHl2/l65Sm3HNhFhkN6/kWTxm/2yVeqV1CU7uEFot2ifdhDuDHngQR8UlqchK3nHM8d+SeyJr3dzB02jI2bqvRUjQiEsfiPZkoG3o4oBpnUHKhuQ4iNWh4dhvm5vVl337HsOnLeWr9Nr9DEpE4ELNkwsy6m9nVZtY9Vtf0hiNKgPI9ExkEqmUqmRCpYd3bpPPUuJM5oXVTxj++jtsWvcW+/fEx90pE/BHLnonfEZhfMN/MNpnZ+TG6biGBUtrBslDhKRHftGycxmOj+3BR77YUvFzC5Q+9Rum33/sdloj4JJbJxCSgp3Ouk3PuaGBJhK/P4OAeiLLrDi93LM87LiI+qZeSxK3ndeUv53VlZcmXnHPPq7z9yVd+hyUiPohZMuGc2+KcWxf0/SFnZ5lZupnlm9k8AolEvpkVeJt+lV2nFJjknZdrZhMJ7NWhIQ6ROHBh77bMHtOH737Yx3n3LufBZVs07CGSYCJaGmpm7YF059zrQcduBHoBLzjnHojkemWJQiXOWwtoYy6RONWzXQbPjOvPpAVv8P+eeZOFGz4mf9iJdGqpNf0iiSDSnokCvL00AMzsBQKFnzoCM83sLzGMTURqkVZN0njo8l7cOaIb7322iyFTi7hv6Xvs1WZhInVepMnEYLwNtsysB4G9LiZ5FbJu5uC5DSKSQMyM87OOYvFvTmFg58OZ/Nw7nHffct76WHMpROqyqsyZKJurMAhweBUigc1EuFeGiNRNLRunMePintx7YRbbSr9j6LRl3LX4Xb7fq14Kkboo0mSiBBhhZk2AkcBa51zZrxyZqIiUiHjMjLNOPILFv/k5Z594BFOWbGLotGW88WG4fftEpLaKNJm4yXvsIFDrIXjyZB4wP0ZxiUgdkdGwHneP6sEDl2VT+t33nHvvq9y26C12/7DP79BEJEYiSiacc/OBTsAIoJlz7sWgp/OB22MYm4jUIb84rhUvXP9zRmS3oeDlEoZMKWL11u1+hyUiMRDxnAnnXIlzbkH5OhLOuVmVqS0hIomraYNUbh92Io9edRJ79u5nRMEKbnlqI9/s2et3aCIShbDJhJntN7N9ET70P4KIHFLO0YfzwvWncFnf9jy8fCtnTHmF17aol0KktqqoaNUdBFZriIjEXMP6KdxyzvEM6XoEN85bz8iZKxj7845cP+gY6qXE+4bGIhIsbDLhnNPeFyJS7U7qkMGiCTn8eeGbTF+6maXvfM7dI7vT+WeN/Q5NRCpJ6b+I+K5h/RRuO/9E7r80m8+/3s3Qe5Zxf1EJ+7XHh0itENHeHGW87cVDFqhyzv01qohEJGEN6tKK59qewk0LNvDnhW+x5K3P+OuIbhyZ3sDv0ESkApFu9NUBWAM08w45wIJOcYCSCRGpshaN6jPr0p7MW/Mh//v0Rs64+xX+3y+P59zuR2Jmh76AiNS4qmz0tQXo6ZxLAnYSKF6VARQStAmYiEhVmRkjerVh0YRT6NyqMdfPWc91j62j9Nvv/Q5NREKINJkYBEx0zq3zvi+BH7cSz0fJhIjEUNvmhzEnry8Tz+jMC29+wml3vcLL737ud1giUk6kyUQp0DTo+xIg2/tzOoFeChGRmElOMq4d0Iknrz2Zpg1SuezB1/jTv//Dd9+rHLdIvIg0mSgERgV9PxeY5E3IzCeQbIiIxNwJRzbl6XH9uap/B/6+4n3OmlbElp1KKETiQaTJxCQCcyaAH/fq2Epgg68MYHTMIhMRKSctNZk/nN2Fx67uzXff7+PPK3dz70vvsU9LSEV8FelGX1ucczeVOzYY6Oicy3DOPRHT6EREQujXqQXP/foUslslc8fz73DhrJVsK/3O77BEElZMilY557Yc+iwRkdhp2iCVsd3q87fh3fjPRzs54+5XWPjGx36HJZKQIkomzKzJoR7VFaiISHlmxrCeR/HshBwyD2/Erx5by2+1FXRuAAAgAElEQVTnrdcupCI1rCqrOXYc4iEiUqPaNW/IvLF9GXdqJxas/ZCzphbx+n81H1ykpkRaTnssoXcSzSYw+XJM1BGJiFRBanISN5zWmZyjD+f6Oa+TO3051w8+hrE/70hykipnilSniJIJ59zMME/NMrP3gI7RhyQiUnUndcjg2Qk5/P7JDdzx/Du88u7n3DWyO621v4dItYnlrqGFQG4MryciUiVNG6Qy7YIe/FWTM0VqRCyTiUGE2UlURKSmmRm5PY9i4fgcOmhypki1inQ1x34z2xfqQaAC5trqCVNEpGrat2jI/LF9uW5gJ+ZrcqZItYh0AuYdhJ6ACbCZQHltEZG4kpqcxI2ndybn6BY/Ts684bTO5J2SSZImZ4pELdIJmJOqKxARkerWO7M5iyacwu+e3ED+c2/z6ntfcOeIbrRskuZ3aCK1WiznTIiIxL2mh6Vyz4U9uP38rqx5fztnTCnixbc/9TsskVotbDJRmWqXqoApIrWRmTHqpLY8M64/rZqkceXDa7jlqY3s/kG7kIpURUXDHKWEnx9RkeQqxiIiUqM6tWzMk9f24/ZFb/Pw8q2s2rKdaRd0p1PLxn6HJlKrVJRMhKp2mU5g1cYkAslGmU7Ab4GJMY1ORKSapaUmc8s5x3PKMS24cd4bnD1tGbcMPZ6RvdpgpsmZIpURNpkIVe3SzKYDY5xz94d47gugeWzDExGpGace24rnJuRw/dzXuemJDRRt+oK/nN+Vpg1S/Q5NJO5FOgFzJLA9zHNrUQVMEanFWjZJ49ErezPpjGN5fuMnDJlSxJqt4f7LE5EyVVnNEW4zr+FARhSxiIj4LinJuGZAR+Zf04/kJGNEwQqmLtnEvv1VmUImkhgiTSZuB04zs+fM7Dwz625mp5rZHAJJRriNwEREapXubdJZOL4/53RrzZ2L3+WCWSvZVvqd32GJxKWIkgnn3GTgJuAkYAFQTGCDr8HAJOfczTGPUETEJ43TUrl7VA/uHNGNjR/t5MwpRTy/8RO/wxKJOxEPczjnJjvnMoBsYATQ0zmX4Zy7I+bRiYjEgfOzAhuGtc04jLxHi/nDv/6jmhQiQapcAdM5t9Y5t8A5ty6WAYmIxKP2LRqy4Jp+jM7pwKMr3+fce19l06df+x2WSFyoqALmC2a2t9yxsLuGeg/t7SsidVa9lCR+f1YXHrqiF59/vYeh9yzjsVUf4JwmZ0piq6hoVSEHFqaCincNFRFJCAM7t2TRr3P4zZz1/O7JDbz6nmpSSGKrqGjV5BDHtGuoiAjQsnEaj1x5EgWvlPC3F97h9f+WMvWC7vRspxXykngimjNhZu3NrHu5Yzea2Rwzuyq2oYmIxLeymhTzxvYlKQlGFKzknhdVk0IST6QTMAuAvLJvzOwFAnt1dARmmtlfYhibiEit0KNtMxaOz2FI1yP46wvvcvH9q/j0q91+hyVSYyJNJgYDiwHMrAcwiEB9iWzgZgJVMEVEEk6TtFSmjurO5NwTef2/pZxx9ysseetTv8MSqRFVWRpa4n0dRGAyZlnVy81AZiyCEhGpjcyMEdlteHpcf37WtAFX/X0Ntzy1kT17VZNC6rZIk4kSYISZNSGw6dda59xX3nOZ/JRoiIgkrE4tG/Hktf24vF97Hl6+lfPuXU7J57v8Dkuk2kSaTNzkPXYAWUDw6o48YH6M4hIRqdXSUpO55Zzjuf/SbD7e+R1nT1vGguIP/Q5LpFpEujfHfAKTLUcAzZxzLwY9nU9gIzAREfEM6tKKZyfkcMKRTblh3np+M+d1vtmj+n5St1RlzoQjsC/HHG+4AzNrSqCnQuuhRETKOaJpAx4f3YcJvziaf73+EWdPW8Z/Ptrpd1giMRNpnYlfEJhoOdh7ZAA453YS6LGoFT0TZpYb/PA7HhGp+5KTjOsHH8M/r+7Dt9/v5fz7lvPQq1tUilvqhEh7JvKB+d5SUCv33EwCkzLjmplNhB+HbAoJLGkVEakRfTs2Z9GEU+h/dAv+9+k3Gf1IMTu++d7vsESiEmkykQXMCfPcDiC9qoF4vQSDwjyXZWZjzGxQ2dcqvkc6cLOXSOCcK3XO9axqzCIiVZHRsB4PXJbNH87uwsvvfsaQqUW8tmW732GJVFmkycRaAvMlQskj8Jt+xLzkYFaY5zKBfOfcTOdcoXNuJpDnHY9UNlBSlriY2cQqXkdEJCpmxlX9O/DENSdTLyWJUTNXMHWJSnFL7RRpMnE7cJOZ3UZgsmVTM+tuZs8DwzhwqeghmVmmmRUQqFERLi3PI1DGO1gBgSGXSGUS6F0pdM4VEhiaWVyF64iIxETXo5ryzLj+DO3WmjsXv8tF969UKW6pdSragvwgzrn5ZjaCwIewEeipMAJblY9wzr0e4fVK8Pb6MLNwiUguBycTawhKAsxsDIEJoOEs9pKHEqDEOVfqvX+pl9BkerGIiNS4xmmp3D2yO/07teCP/97ImVOK+OvwEzn12FZ+hyZSKRElE2bWxJtvMN/bmyOTwIfzuuoIzpvjcFCvhZcEUJYEeEMflREqYSiNNk4RkWiZGcOz29CjbTOue2wtVz68hqv7d2DiGcdSL6Uqq/hFak6kd2ixmd0A4Jxb55xbUF2JhKds6Wm4D/yI5jt4vQ+lXpJSlqyUqFdCROJFp5aN+NevTuaSPu24f9kWcmcsZ+sX3/gdlkiFIk0mXgRGVUcgYVR5dUgFhgM3e0Mj2ulUROJOWmoy/3fuCcy4uCdbv/iGs6ct49+vf+R3WCJhWaQFU8xsDfAFkOecez9mgZht9q5ZGHQsCyh2zpWvaYGZOWBw8Pmx5iUcYwBatWrVc/bs2TG79q5du2jUqFHMrldXqF1CU7uElgjt8sV3+5mxfg/vle4n58gULj6uHvVTDvov8QCJ0C5VoXYJLVy7DBw4sNirK3VIkc6ZKJu7kEVgiWX5U5xzLqJrVvJ90ysY6qg23lyMmQDZ2dluwIABMbv20qVLieX16gq1S2hql9ASpV3OPW0/dxdu4t6l77Ht+/pMuyCLLq2bhD0/UdolUmqX0GLRLpF+8M+iZvffKJvLkEHQRMmyOQ9oy3MRSQApyUnceHpn+nVszoQ5r3Pufa/yh7OO4+I+7QjxS51IjYt0aWhEdSSi5a3aKOHguRMZQKkmTopIIunXqQWLJuRw47z1/OHfG1n23hfkDzuR9MPq+R2aJLjasN6okIOrbmZRxWqbIiK1WYtG9Xnwsl78fshxLHnrM4ZMKWLNVpXiFn/FUzKRQejVG5M4eMVFHhFW2xQRqSuSkozRp2Sy4Jp+pCQnMXLmSu55UaW4xT++JhNmlm5m+WY2j0AikW9mBcHbgnsTLyd55+V6u37ma4hDRBJdtzbpLBzfnyFdj+CvL7zLJQ+s4jOV4hYfxHzlRSTKEoVKnLeWQOluEREJ0jgtlamjupPTqQV/fOo/nDmliMuONQb4HZgklHga5hARkSowM0b0asMz4/pzeOP63Fm8h9sXvc0P+/b7HZokCCUTIiJ1RKeWjfnXr05mQJsUZry8mQtmruTjnd/5HZYkgLDJhJldbWbdazIYERGJTlpqMpcfX58po7rz1sdfMWRKES+985nfYUkdV1HPRAHlNtIys/OrNxwREYmFX3Y/kqfH9adVkzSueGg1+c+9zV4Ne0g1qSiZ2MnB1S7nmVn7aotGRERiJvPwwA6kF5zUlulLNzNKwx5STSpazTEPmOyVat3pHTNgmJltCfci59wTsQtPRESikZaazG3nd6VPZga/e2IDQ6YUcefI7gzs3NLv0KQOCZtMOOfyvPoPC4IPA3dUcD0HJMcoNhERiZFfdj+Srkc25dp/ruWKh1ZzzYCO3DD4GFKSNQ9foldhnQnn3HAAM+tAoFfiPWAw2mBLRKTWKRv2+N+n32T60s2s2bqdqRf04IimDfwOTWq5SqWkzrktXsXJycBq7/uQj+oNV0REolE27DFlVHfe3PYVZ01dxlKt9pAoRdS/5Zy7yTn3lZmdambTzex57+vA6gpQRERi75fdj+Spcf1p2bg+lz+0msla7SFRiHiwzMymE9ixcyTQ3PtaaGazYxybiIhUo45Bqz3uW7qZC2ZptYdUTUTJhJmNJrBj52DnXIZzLts5l0EgoRhhZjdUR5AiIlI9yg97qMiVVEWkPRN5wETn3JLgg865+cBNwNhYBSYiIjWnbNijrMiV9vaQSESaTGQRfvfOzZSrmCkiIrVH2bDHhb3bMuPlQJGrbaUa9pBDizSZKARywzw3Em0TLiJSq6WlJvOX8wLDHm9//BVDphbx4tuf+h2WxLlIk4mbgLFmdp+ZdTezJt7X6cAwYFLsQxQRkZpWtrfHEU0bcOXDa7jt2bc07CFhRbo0dC0wAhgFFAM7CPRG5AFjnXMvxjxCERHxRebhjXjy2n5c1LstBa+UMLJgBR9p2ENCiHhpqHNuvreC43QCEy4HA82cc7NiHZyIiPgrLTWZW8/ryrQLevDup7sYMqWIwjc17CEHqnJRdudcoXNulnNuiXNu56FfISIitdXQbq15Zlx/jmrWgKsfWcOtC9/UsIf8SDu8iIhIpbRv0ZAF1/Tjkj7tmFW0heEzVvDhjm/9DkvigJIJERGptLTUZP7v3BO498Is3vssMOzxwsZP/A5LfKZkQkREInbWiUfwzLj+tG1+GGMeLeaWpzayZ+8+v8MSnyiZEBGRKikb9ri8X3seXr6V8+9bTsnnu/wOS3ygZEJERKqsfkoyt5xzPLMuzeaj0u84e9oynlj7od9hSQ1TMiEiIlEb3KUViybkcELrpvxm7npumLueb/bs9TssqSEpVXmRmXUHMkI9p8JVIiKJ6YimDXhsdG+mvvge017cxLoPdjDtwh4c37qp36FJNYsomTCzYcDcsm9DnOKA5GiDEhGR2iklOYnfDD6GPpkZXD/ndc67dzm/P+s4Lu3bDrNQHxtSF0Q6zJEPbCFQUrtniEd2TKMTEZFaqV/HFjw7PoeTOzXnT09tJO/RYkq//d7vsKSaRDrMkQnkOueeqI5gRESk7mjeqD4PXNaLB1/dQv5zbzNkShFTLuhBr/YhR8mlFqvKFuS6C0REpFKSkoyrczKZP7YfKclJjCxYwbQlm9i33/kdmsRQpD0TecBcM2sKLAFKy5/gnNsag7hERKQO6dYmnYXj+/O7J//D3xa/y4qSL7l7ZHdaNknzOzSJgUh7JjKBjsBkAluQbw7xEBEROUjjtFSmjurO5GEnsvaDHZx+9ys8tX4bzqmXoraLtGci3/t6B0ocREQkQmbGiF5tyGqXzg1z1zP+8XU8s34bfz73BPVS1GKRJhNZwBjn3P3VEYyIiCSGTi0bs+CafjywbAt/W/wug+58mT8OPZ5hWUdqCWktFOkwx1pge3UEIiIiiSUlOYm8n3dk0YQcjmnVmBvnreeKh1ezrfQ7v0OTCEWaTIwG8s1soJk1qY6AREQksXQ8vBFz8/ryp6FdWFWyndPueoXHX/tAcylqkUiTiSUEJmAWAjvMbF+5hwqxi4hIxJKSjCtO7sDzvz6Frkc25eYnNnDxA6v47/Zv/Q5NKiHSOROzCJTMFhERibm2zQ/jn1f35vHVH3Dbs29z2l2vMOmMzlzatz1JSZpLEa8iTSZudc59VS2RiIiIEOiluKh3OwZ0bsnNT2zglqffZOGGj5mc240OLRr6HZ6EEOkwR7GZ3VAtkYiIiAQ5Mr0Bf7+iF3fknsg7n3zNGXe/wqxXSlQ9Mw5Fmky8CIyqjkBERETKMzOGZ7dh8W9+Ts7Rh3Prs29x7r2v8vK7n2uCZhyJKJlwzuUBZmbPmVm7aopJRETkAK2apDHr0p5MvaAHX+7aw2UPvkbujBUs2/SFkoo4ENGcCTMrqzGRBZSEKCzinHORzsMQERE5JDPjnG6tOf34Vsxd8yH3vfQeFz+wil7tm3H94GPo17GF3yEmLK3mEBGRWqV+SjKX9GnHiOyjmLP6v9z70ntcOGsVvTtkcP3gY+iT2dzvEBNORMmEc25SdQUiIiISifopyVzatz0jstsw+7UPuHfpZkbNXEnfzOZcP/gYTuqQ4XeICSPSCZgAmNmpZjbdzJ43s/vMbGCsAxMREamMtNRkLj+5A0UTB/KHs7uw6bNdjChYwcX3r6L4fe0AURMiTibMbDqBCpgjgeYEVncUmtnsGMcmIiJSaWmpyVzVP5BU/M9Zx/HWx18xbPoKLnlgFWs/2OF3eHVaRMmEmY0G8oDBzrkM51y2cy6DQGIxQjUoRETEbw3qJXN1TiZFkwZy85nHsnHbV5x/33LuKt7NB1+qPHd1iLRnIg+Y6JxbEnzQOTcfuAkYG6vAREREonFYvRTyft6RookDmXhGZ97Zvo/Bd73MfUvf44d9+/0Or06JNJnIIrANeSibgczowhEREYmthvVTuHZAJ/6S04CBnVsy+bl3OHvqMorf19BHrESaTBQCuWGeG0n4RENERMRXGWlJzLikJ7Muzebr3T8wbPpyfv/kBnZ+94PfodV6kdaZuAlY4xWrmgmUEOiNyAOGAYNjGp2IiEiMDe7Sin4dm3Pn4nd56NUtPL/xU/40tAtnn3gEIYoxSiVEWmdirZmNIJBI5HmHy1o+zzn3YiyDqw5mlgVke9+mA/OdcyU+hiQiIjWsYf0U/nB2F87rcSQ3P7GBcY+vY37xh/z53BNok3GY3+HVOhEvDXXOzfdWcJxOYMLlYKCZc25WrIOrJoOcczO9x2RAhbhERBLUCUc25V+/Opk/De3Cmq3bGXzXy0xfulkTNCNU5X00nHOFsQzEzHKB0lDXDepNKBtWKYni/fPMbKZzrrTq0YqISF2RnGRccXIHzjjhZ9zy1Ebyn3ubf7/+Ebee15We7Zr5HV6tEDaZMLOrq3JB59z9kb7GzAYR2PdjeIjnMoF859zgoGPzzKykisMT+cAWMyvrkVDPhIiIcETTBhRcks0LGz/hT09tJHfGci7q3Zbfnn4sTRuk+h1eXKuoZ2JmBNdxQV8rnUx4icIkoBgIV/M0Dygod6yAQFJwUPJxKM65mWaWzk9zPgoB9VKIiAgApx3/M/p1asGdL7zLw8u38OJbn/GPq3uTeXgjv0OLWxUlEx0reY10AstCJ0b65l7PQh5AUE9BebkcnEysARaXfWNmY6g43sVlwyJmNtGbKzHZe93iQ7xWREQSTKP6KfxxaBfO6d6aqx5ezciZK3ns6t4c3aqx36HFpbDJhHNuS0UvNLMmwBjgZqAZgRoTMR0y8HoQMinXa+GcKzUzzCzTOVfinKtUL4o3nPJjLQyvl6KjmWU551QjQ0REDtC9TTpz8vpw4axVjJy5kn9c1ZsurZv4HVbcqcpGX+29zb52AJOBJUBPb5+OJRW/OmIZEEgewjwfacXN7QSqeB5AiYSIiITTqWVj5uT1pX5KEhfMWskbH2pkvDxzzh36LMDMuhOYpzCIQG2JmQQmRlbYg1HpQMw2E6hVURh0LAsods4dVEXEzByBDcciWtXhrRop2+Q+HSgMl0x4wyBjAFq1atVz9uzYbYy6a9cuGjXS+Ft5apfQ1C6hqV1CU7uEFm27fP7tfiav3s2uHxw39EyjU7PkGEbnn3DtMnDgwGLnXHaIlxzMOVfhAzgfWA3sI/Cb/W+Bpod6XaQPAnt7DCp3LCsQYsjzXfnzq/PRs2dPF0svvfRSTK9XV6hdQlO7hKZ2CU3tElos2uWjHd+6AXe85I77wyK3YvMX0QcVB8K1C7DGVfIzMuwwh5ldbWabgHkE5kSMdYFtx+9wzu2sXL4TG97cCREREV+1Tm/AnDF9aJ3egMsfeo2iTZ/7HVJcONTSUOd9nQdgZqce6oIutiW1y+pIZBC0fDMouVAZbBERqVEtm6Qxe0wfLr5/FVf9fQ0zLs7i1GNb+R2Wrw5VAdMILN0cE/Q9/FRXovy5DojZIJILrNooITC3IVgGgWqZSiZERKTGtWhUn8dH9+HSB18j79Fipl2QxRkn/MzvsHxTUTIRLzuAFhIopR08STLLOy4iIuKLZg3r8Y+re3PFQ6/xq8fWctfI7pzTrbXfYfmiojoTsV7meSgZHNwDAYHaFfM4sCJnHj9VsBQREfFF0wapPHJVb658eDW/nr2O7/fuJ7fnUX6HVeMirjMRS2aWbmb5ZjaPQCKRb2YF3vJN4McaE5O883LNbCKBJaka4hAREd81qp/C3684iX4dW/Db+et5bNUHfodU46q8a2gslCUKlThvLQcOc4iIiMSNBvWSuf+ybK75RzG/e3ID3+/dx+Und/A7rBrja8+EiIhIXZGWmsyMS3pyWpdW3PL0m9y39L2yukh1npIJERGRGKmfksy9F2UxtFtrJj/3Dtf+cy07v/vB77CqnZIJERGRGEpNTmLKyO7cfOaxLH7zU86aWsTr/63b+3komRAREYmxpCQj7+cdmTu2L85B7vTlzHqlhP376+awh5IJERGRapLVthnPjs/hF8e15NZn3+LqR9aw/Zvv/Q4r5pRMiIiIVKOmh6Uy4+Ke/L9fHs+yTV8wZEoRr23Z7ndYMaVkQkREpJqZGZf2bc8T1/YjLTWJUTNXMG3JJvbVkWEPJRMiIiI15IQjm/LM+ByGdmvN3xa/y6UPruKzr3f7HVbUlEyIiIjUoEb1U7h7ZHcmDzuR4vd3MGRKUa3fylzJhIiISA0zM0b0asNT1/Wn2WH1uPTB17jj+bfZu2+/36FViZIJERERnxzTqjFPXdefkdltuPelzVwwayXbSr/zO6yIKZkQERHxUYN6ydw+7ESmjOrOm9u+4uxpy1j7wQ6/w4qIkgkREZE48MvuR/L0uP40TkvhgpkreX7jJ36HVGlKJkREROJE5uGNeOKafhx3RBPG/qOYvy/f6ndIlaJkQkREJI40b1Sfx0f3YdBxrfjTUxv5y7NvxX0ZbiUTIiIicaZBvWRmXNyTS/u2Y+YrJYyfvY7dP+zzO6ywUvwOQERERA6WnGT87znHc2R6A25b9Daffb2HmZf0JP2wen6HdhD1TIiIiMQps8Duo1Mv6MHrH5QybPpy/rv9W7/DOoiSCRERkTh3TrfWPHLVSXz+9R7On76c/3y00++QDqBkQkREpBbok9mcBdf0o15yEiMKVvDSO5/5HdKPlEyIiIjUEke3asyT1/ajQ4uGXP33Ncx+7QO/QwKUTIiIiNQqLZukMSevL/07teCmJzZw5wvv4Jy/S0eVTIiIiNQyjeqncP9l2YzMbsPUF9/jhnnr+X6vf5uEaWmoiIhILZSanMTtw7rSOr0BdxW+y2df7aHgkp40rF/zH+1KJkRERGopM2PCoKNpnZ7GC29+Sv0UfwYclEyIiIjUcsOz25Db8yjMzJf315wJERGROsCvRAKUTIiIiEiUlEyIiIhIVJRMiIiISFSUTIiIiEhUlEyIiIhIVJRMiIiISFSUTIiIiEhUlEyIiIhIVJRMiIiISFSUTIiIiEhUlEyIiIhIVJRMiIiISFTMOed3DLWCmX0OvB/DS7YAvojh9eoKtUtoapfQ1C6hqV1CU7uEFq5d2jnnDq/MBZRM+MTM1jjnsv2OI96oXUJTu4SmdglN7RKa2iW0WLSLhjlEREQkKkomREREJCpKJvwz0+8A4pTaJTS1S2hql9DULqGpXUKLul00Z0JERESiop4JERERiYqSCREREYlKit8BJBozywKygRIgEyhxzhX6G5W/zCwdGAPMB7YDGUAesDjR2sbMcoHSUD93It874dolke8dr00ygY7e1wLn3Pxy5yTcPXOodknUe8bMBgGDgS8JtE2xc25muXOqfr845/SooYf3l7O43LF5QKbfscVBu7igxw4g1++4fGiHQd7PPihMGyXkvVOJdkm4ewfIBbKCvk8HNgNjEvmeiaBdEuqe8f4NDSp3rBiYGKv7RcMcNSsPKCh3rADI9yGWeDMYaAZ0dM41c+V+w6rLzCzTzAoI/GPeHua0hLt3KtkukJj3TqZzbm3ZN865UgL3QvA9knD3DJVrF0i8eyYvxLHCcsejul+UTNSsXGBtuWNrvOMJzzlX6pwr8TuOmuacK3HO5blyXY7lJNy9U8l2KTs3Ye4dr5t+pPc1WKH3fKb3fULdMxG0C5BY94xncIhjpUF/jup+UTJRQ7wb/KDfsLzM+aAbXaSM7h0J5v29Z3qPkBLxnqlMuyQq59xw59ykcodzgTkQm/tFEzBrTgb89JcTQiaBSS+JKtObOAWBttqeAF2PlaV7p2IJd+8455qFODyIwCTVkrL//BPtnjlUuwQdS7h7JpiZjQHWOucme4ei/j9GyUTNKd/1Jj/ZDuAOnHE9z8xIpH/gFdC9E57unZ/kAbd5f9Y985PgdoEEvme8BGowBHorgp6K+n7RMIf4zhu7LD8uXtcnikkM6N4J8H7T3B70m6YQul0S+Z5xzs13zuUBk8ys2FsKGhNKJmpYiMlBEloJga5ItZdHbVFpCXXveEMaec65gybYJUobhFJRu4SQUPeMN5xRACwJPh7Nz69kouaUjTdlBB8M+surc+OXlWVmE0McLpsIpMlUunfC0r0DBH6j/kW5Y7pnQreL7pmfFALpXjGrqO8XJRM1xMsESzh4bCqDgycHJQzvt4f8ELOFy27qhGyXYLp3QtO9A14djknlJ84l+j0Trl0S8Z7x6rXsqGBIIz0W94uSiZpVSKBUabAs73hC8m7SvBA36yACs43DzS5ONLp3ykn0e8ebD5Af/POb2aCgD8qEvGcqapcEvWfSCSQK5X/msvukrLZEVPeLkomaNQkYXu5Ynnc8kW0P/k3B61rLA0b7F5JvMgg9szrR751w7ZKQ907QssZ0M8vyHoOA4UEflAl3z1SyXRLqnvEqgs4J8dQkYHKs7hfz6m9LDfG6mkYCqwlkhmtdHd5cprKCNudpTuBDI7+ud8WW8f4zu5nAz59L4DeIQgJ18oOXryXUvRNBuyTUveO1y44wT5c454tLVcIAAAYoSURBVDoGnZsw90yE7ZJQ9wz82GPTkUNv9FWl+0XJhIiIiERFwxwiIiISFSUTIiIiEhUlEyIiIhIVJRMiIiISFSUTIiIiEhUlEyIiIhIVJRMiIiISFSUTIhHySvM67zEmxPOZ4Z6r5rjyzSzuC8eYWYG3V0CFbWRmE81scU3GFqnaEGMw795c7LV9gd/xSN2R4ncAIrXcJGDmIc8SAMxsHoF6/2WliyuqOtiLwJ4J8aw2xPgjr8rjYC/prDVJkMQ/9UyIVN1MIDNoPwA5tFwCuznO9x5rw53onBvunLPgY17vS41vEx3ufUPFGO+Cdo+sk2W1xR9KJkSqbh6B36xv9juQ2iDow7hKOzN6ey9MJNCzUWP8et9qNIjAXhV1cYdM8YmSCZHo5ANZQb/ticS7wahXQmJMyYRIFLxd90oJJBVhhZocaWZjvIlw6eXO2+FtnVzsPV/sTfoMnjy3Odzwivfaxd51NntbMJc/J7f89cvH4P25wLtGqO2/w71v2TXHBD2XD2z2vi07p8LhoeA28+ZalO0IOc97/Y5y50f8M3ltOi9oQujicttTV/i+Yf5ew7ZD+XhCnJtV7rxcL1YXKr4qGoQ3X8Jrg4neo8J7WKQiSiZEoncbMCiGY/npBIZQbgPyCGwFPC/oMdw7b16Y188CCghMDoXAh3fwB2Su99pC71ol5c/xzisGsoGCQ3WJe9cs9q41GJgDFAStGLjNO44XV8fgbcQrYXTQ6/MIbKHcMwY/U16562d4P0el3re8SrRDsLK/5wIO/Hsuu1bZ9/O99yz7uQ6Z2FUQ34/zJbwEcYRzbrL3c2nuj1Sdc04PPfSI4EHgNzsHDPK+T/e+L/C+z/S+HxP0mvzAP7cDrjPGOy+9/Hll1y53bGLQsVzvWNYh3iMzODbv2A4gv9x5xWXHgt5vcQRtsiP4PcK0U1ksuZW85gE/T0Wvj9XPRGBexAHvcYj3LR/jIduhXDy5IY6ll/s7Tq8o5gjv3YkEeojSy92f6bF8Hz0S76GeCZEoucBvuDOBMZUZDqikNUF/LhseCB7nLltSmXGI2Eq812XDj7+ZpgMTg7rOHYEP0fLzPvKoBG84IZ3Ab9jB711IYAhoeKjXxUosfyb30+qSiHuZqtgOwX+nm8M8t8QbhojFvJyyXpYRLjBEVxZjqdOETImCkgmR2Cgbb45Joaow/7FX9T/7En76cCz72pNA13bw44APOy8RqYyya4Y6P/i9q0tUP5M3L2GeN2dhR/nnqxBHpduhog9w77myD/98oNiiL5A1CG/YJNQcDZGqUtEqkRhwzpWY2XwCy0TjbaZ8Jj99wP34QRdBsnAoZdfJBMrXjcjkwF6W6lDln8n7cM4k8GFdFmdx+FdUKo6YtYPXq9HT6/EaA+Sb2UQXmOcQkaDE4TbnXKmZTSQwJ6NjpNcSKU89EyKxcxuBbu6RlTw/7ES+WPE+hAbhJTheN37I1SdRDNGUfUgeMITgTUYsm2QYC9tDHazqz+RNcBxEoIjWTO86oZKRkO8bQrW1gzcMMZlAktKripcpX1+iBG+YzFvVoQmYUmXqmRCJEefcWjNbS2CSW3mbIbCXA4EP9kHEaEikPG854yR++o27lECiU2Y0gWWOZSsJ0gl8AJZQyXkSwbzfcvMIrFrIILCCoey953u/XUft/7d3RzcNw1AUQF82QLAJZQVGYASUETpDugllhM5CZ2CD8PEcNZRQ0jpIrXTOT1UpShr3w9fPdlKuExHRDp993w/TAGffU6kmfUaO9iOyndYTx5267vFxi7VD2VK6LvczTJM8lu9DGOoiKw2/Pkl0ZOr5EkN4+raGAs6lMgHL+tEZFdvIUWUXOUJ9iAs67pneIp8jsIscVa/Gc/N9bslcRXa4u8itpPsTv/1PpSN6jsP2xjZyxL/04stNZBDrYjSVUHFPr5Gj8/c4/Dezr3ts4XbYRq5vaEfn2ow6/bvIHR9PM893H6PFoaXN9iXgzq2+wKSm76/+JYMATGia5iMiXmZWJuDfqEwA3DBBgmsgTADcoLK49OKpKViSaQ4AoIrKBABQRZgAAKoIEwBAFWECAKgiTAAAVYQJAKCKMAEAVBEmAIAqXxvf6nbzsv4gAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(8, 6))\n",
    "fontsize=20\n",
    "plt.semilogy(res_conv)\n",
    "plt.xlabel(\"Number of iterations, $k$\", fontsize=fontsize)\n",
    "plt.ylabel(\"Norm of residuals, $\\|Ax_k - b\\|_2$\", fontsize=fontsize)\n",
    "plt.xticks(fontsize=fontsize)\n",
    "plt.grid(True)\n",
    "_ = plt.yticks(fontsize=fontsize)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Pro & Contra\n",
    "\n",
    "Pro\n",
    "\n",
    "- автоматический способ превращения произвольной задачи *условной* оптимизации в задачу *безусловной* оптимизации\n",
    "- начальную точку искать не надо\n",
    "- часто ограничения \"нестрогие\" и допустимо незначительное нарушение ограничений\n",
    "- простота реализации общей схемы\n",
    "\n",
    "Contra\n",
    "\n",
    "- решение задачи безусловной оптимизации не всегда является решением исходной задачи\n",
    "- если целевая функция определена только на допустимом множестве, то метод внешних штрафов не применим\n",
    "- квадратичная аппроксимация $Q(x, \\mu)$ может быть неадекватной"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Двойственная задача: напоминание\n",
    "\n",
    "- Исходная задача\n",
    " \n",
    "\\begin{align*}\n",
    "& \\min f(x) \\\\\n",
    "\\text{s.t. } & Ax = b\n",
    "\\end{align*}\n",
    "\n",
    "- Лагранжиан\n",
    "\n",
    "$$\n",
    "L(x, \\lambda) = f(x) + \\lambda^{\\top}(Ax - b)\n",
    "$$\n",
    "\n",
    "- Двойственная задача\n",
    "\n",
    "$$\n",
    "\\max_{\\lambda} g(\\lambda), \n",
    "$$\n",
    " \n",
    "где $g(\\lambda) = \\inf_x L(x, \\lambda)$\n",
    "\n",
    "- Восстановление решения исходной заадчи\n",
    "\n",
    "$$\n",
    "x^* = \\arg\\min_x L(x, \\lambda^*)\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Решение двойственной задачи\n",
    "\n",
    "- Градиентный подъём, так как задача без ограничений\n",
    "\n",
    "$$\n",
    "\\lambda_{k+1} = \\lambda_k + \\alpha_k g'(\\lambda_k)\n",
    "$$\n",
    "\n",
    "- При этом градиент двойственной функции \n",
    "\n",
    "$$\n",
    "g'(\\lambda_k) = A\\hat{x} - b,\n",
    "$$\n",
    "\n",
    "где $\\hat{x} = \\arg\\min_x L(x, \\lambda_k)$\n",
    "\n",
    "- Объединим два шага в один и получим \n",
    "\n",
    "\\begin{align*}\n",
    "& x_{k+1} = \\arg\\min_x L(x, \\lambda_k)\\\\\n",
    "& \\lambda_{k+1} = \\lambda_k + \\alpha_k (Ax_{k+1} - b)\n",
    "\\end{align*}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Важный частный случай\n",
    "\n",
    "- Функция сепарабельна\n",
    "- Обновление $x$ распадается на параллельные задачи по каждой координате"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "plt.rc(\"text\", usetex=True)\n",
    "import cvxpy as cvx\n",
    "\n",
    "def dual_ascent(update_x, A, b, alpha, x0, lambda0, max_iter):\n",
    "    x = x0.copy()\n",
    "    lam = lambda0.copy()\n",
    "    conv_x = [x]\n",
    "    conv_lam = [lam]\n",
    "    for i in range(max_iter):\n",
    "        x = update_x(x, lam, A, b)\n",
    "        lam = lam + alpha * (A @ x - b)\n",
    "        conv_x.append(x.copy())\n",
    "        conv_lam.append(lam.copy())\n",
    "    return x, lam, conv_x, conv_lam"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Модельный пример\n",
    "\n",
    "\\begin{align*}\n",
    "& \\min \\frac{1}{2}x^{\\top}Px - c^{\\top}x\\\\\n",
    "\\text{s.t. } & Ax = b\n",
    "\\end{align*}\n",
    "\n",
    "- Лагранжиан $L(x, \\lambda) = \\frac{1}{2}x^{\\top}Px - c^{\\top}x + \\lambda^{\\top}(Ax - b)$\n",
    "- Обновление прямых переменных \n",
    "\n",
    "$$\n",
    "x_{k+1} = P^{-1}(c - A^{\\top}\\lambda_k)\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.035276707573235914\n",
      "-----------------------------------------------------------------\n",
      "           OSQP v0.6.0  -  Operator Splitting QP Solver\n",
      "              (c) Bartolomeo Stellato,  Goran Banjac\n",
      "        University of Oxford  -  Stanford University 2019\n",
      "-----------------------------------------------------------------\n",
      "problem:  variables n = 20, constraints m = 10\n",
      "          nnz(P) + nnz(A) = 410\n",
      "settings: linear system solver = qdldl,\n",
      "          eps_abs = 1.0e-05, eps_rel = 1.0e-05,\n",
      "          eps_prim_inf = 1.0e-04, eps_dual_inf = 1.0e-04,\n",
      "          rho = 1.00e-01 (adaptive),\n",
      "          sigma = 1.00e-06, alpha = 1.60, max_iter = 10000\n",
      "          check_termination: on (interval 25),\n",
      "          scaling: on, scaled_termination: off\n",
      "          warm start: on, polish: on, time_limit: off\n",
      "\n",
      "iter   objective    pri res    dua res    rho        time\n",
      "   1  -1.1152e-01   2.23e+00   1.87e+03   1.00e-01   1.77e-04s\n",
      "  50   8.3513e+00   3.51e-09   2.52e-08   1.00e-01   3.28e-04s\n",
      "plsh   8.3513e+00   1.51e-15   2.05e-14   --------   4.58e-04s\n",
      "\n",
      "status:               solved\n",
      "solution polish:      successful\n",
      "number of iterations: 50\n",
      "optimal objective:    8.3513\n",
      "run time:             4.58e-04s\n",
      "optimal rho estimate: 6.14e-02\n",
      "\n",
      "1.7377739072730912e-15\n",
      "8.351275526841468\n"
     ]
    }
   ],
   "source": [
    "m, n = 10, 20\n",
    "A = np.random.randn(m, n)\n",
    "b = np.random.randn(m)\n",
    "P = np.random.randn(n, n)\n",
    "P = P.T @ P\n",
    "c = np.random.randn(n)\n",
    "spec = np.linalg.eigvalsh(P)\n",
    "mu = spec.min()\n",
    "print(mu)\n",
    "\n",
    "x = cvx.Variable(n)\n",
    "obj = 0.5 * cvx.quad_form(x, P) - c @ x\n",
    "problem = cvx.Problem(cvx.Minimize(obj), [A @ x == b])\n",
    "problem.solve(verbose=True)\n",
    "print(np.linalg.norm(A @ x.value - b))\n",
    "print(problem.value)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3.318634760560637e-09\n",
      "8.351275508748442\n"
     ]
    }
   ],
   "source": [
    "x0 = np.random.randn(n)\n",
    "lam0 = np.random.randn(m)\n",
    "\n",
    "max_iter = 100000\n",
    "alpha = 1e-3\n",
    "\n",
    "def f(x):\n",
    "    return 0.5 * x @ P @ x - c @ x\n",
    "\n",
    "def L(x, lam):\n",
    "    return f(x) + lam @ (A @ x - b)\n",
    "\n",
    "def update_x(x, lam, A, b):\n",
    "    return np.linalg.solve(P, c - A.T @ lam)\n",
    "\n",
    "x_da, lam_da, conv_x_da, conv_lam_da = dual_ascent(update_x, A, b, alpha, x0, lam0, max_iter)\n",
    "print(np.linalg.norm(A @ x_da - b))\n",
    "print(0.5 * x_da @ P @ x_da - c @ x_da)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmIAAAH5CAYAAAAiH1L7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3XtwY9lh5/ffAUmgH0A3iAGn56Fpz4Cy5dHKepC0HMl6UCtyEzuO1lmT0/YmUeLYTSSuxLXZOI2dSm1tXJXdXnIdx3FqdwN2vHFqy163mvEqVmW9dlM2bVn2WtOENJatkSUPRpoZjR7NZmOa7AefJ3/ce9GXIEgCxONeAN9PFYvgxcG9B30Gwx/POfccY60VAAAA2i8SdAUAAAB6FUEMAAAgIAQxAACAgBDEAAAAAkIQAwAACAhBDAAAICAEMQAAgID011LIGDMhaVLSLUnDkpattfMVZUYkjUkqSspIKlprF+stAwAA0CuODGJuCJO1Nuc7tmyMSVpr59yfM5JmrbWTvjLXjDFFa22x1jIAAAC9pJahyWyVY4sVx7OS8hVl8pJm6ywDAADQM2qdIzZZ5VjJ93hKUqHi+Rvu8XrKAAAA9AxznL0mjTEvS8pba+eMMUlJtyUNWmtLFeWsnDllq0eVYXgSAAD0mrrvmjTGzEgqePPDJKUkqTJg+WRqLAMAANBTarprUpKMMVNyhyittdO+p5I1vLyWMtWuOSNpRpJOnjw5+tRTTx3nNDXb3d1VJMKKHmFCm4QT7RI+tEk40S7h0642+cpXvrJirR06qlzNQcxauyBpwRiTNMYsS7pora2c89VU7hIZ85I0NjZmb9y40crLaWlpSePj4y29BupDm4QT7RI+tEk40S7h0642McZ8vZZydUdCd3gxL+nTFRc8sterljIAAAC94rh9c4uSku4aY94k+5S/gC90FWssAwAA0FMODWLGmIwx5ra7In41SbeHrKj988BSkkrW2mItZY5RdwAAgI52VI9YUnt7tDzeXY7eHLFFOVsX+Y24x1VHGQAAgJ5xaBBzJ+NfrfJUTtKcrycrJ2m6okzWPa46ygAAAPSMI++adBdtnTHGDOvhpt/X/Jt+W2tLxpicMWZW0gtyesxm/UOOtZQBAADoJTUtX+EPXYeUKWj/FkZ1lwEAAOgVrDIHAAAQkJoXdAUAoNU2Nja0urqqtbU17ezsBF2dhp09e1YvvfRS0NWAz3HapK+vT4lEQqlUSrFYrKn1IYgBAEJhY2NDr776qgYHB/X0009rYGBAxpigq9WQtbU1JRKJoKsBn3rbxFqrra0t3blzR6+++qrOnz/f1DDG0CQAIBRWV1c1ODiodDqtaDTa8SEM3cEYo2g0qnQ6rcHBQa2urjb1/AQxAEAorK2t6cyZM0FXAzjQmTNntLa21tRzEsQAAKGws7OjgYGBoKsBHGhgYKDpcxcJYgCA0GA4EmHWiv8+CWIAAAABIYgBAAAEpGOWr7BWerC1s+9Y+bHsvuN2T1nf8wedw/8DAABAi3VMEPvzN97U9/79f9vSa3zgyX595CMtvQQAAEdaWFjQ5cuXVSgUlEwmNTExodnZWWUymX1lc7mc5ubmjuxMKBaLymazymazmpqaalXVy+bm5nT9+nVdv3695dfqZB0TxM6dOaFL/8HbJElGDyfL+efNmarHqpd9eMw5+IkXXtNra3ebVl8AAI4jm81qfn5eU1NTymazKpVKunr1qoaHh3Xt2rVjh6hSqaTFxUVNTk42ucbVvfDCC1pcXGzLtTpZxwSxRxMx/cz4W1t2/r/81h397hfXW3Z+AACOksvlND8/vy9wXbp0SdlsVtPT01peXtbIyEjd5x4ZGWnJFJxcLqdsNruvt+7atWtNv1Y3YrK+Kx2P6c6m1e4u88QAAO1XKpU0NzenmZmZqr1e+XxemUxGFy9eDKB21Xl1LhQKQVelYxHEXEOJmHatVLq/FXRVAAA96PLly5Kk2dnZA8vMzs6qUCgQfLoIQcyVjjsbeN5c2wi4JgCAXuRNzE8mkweW8YYkb9y4UfX1k5OTGhwc1PDwcNX5WcYYzc3N7Tm2sLCg0dFRGWM0Ojq673X+8w4ODmpyclKFQkHT09MaHByUJE1PT8sYU/5ZcoYsvXnY2Wy26mKoc3NzMsaoVCrVXJ9u0zFzxFrNC2Ir6xt6m2rflR0A0Ho//6m/0JfeuBN0NQ719ifO6B/8R3/t2K+/ceNG1bsi/bznl5eX9z138eJFPf/885qentbs7KwmJyf18ssvH3rOhYUFTU9P69KlS3r++ed19erVPa/zJvdPTEzoypUrWl1d1fXr13X16lVduXJF2WxWk5OTyufzmpiYOPA63g0ICwsLe4Zdr169qpGRkXL4PKo+3Ygg5hpKPAxiAACEkb/nqJI/nE1MTGh4eFizs7PK5/MHvubixYu6dOlSeTh0ampKo6Ojyufzmp2dVTab1cTExJ4lKGZmZsqPvXCUSqUODUpe2Mrn8+UgViwWVSgU9tTvqPp0I4KYa4ihSQAIrUZ6mjrF2NjYkcNwxWJRkjQ6OnpouUwmo4mJiapDmJ5CoVCebF85XJlKpVQsFlUsFg8NcvWYmZnZc52FhQVJ0nPPPVdTfboVQcx15mS/+o10kx4xAEAARkZGtLi4qEKhcODyFF5Qq2WYLpPJHBrEvFC3vLy8b15aKpUqv7ZZISibzWpubq48PHn16lVNTEyUr31UfboVk/VdxhidiRmtrG0GXRUAQA/KZrOSHt49Wc3ly5c1MjJy6HwsT7FYPDSw+Z/LZDJ7vpLJZPn51dXVWt/Cobxz5/N5lUolFQqF8nuupT7diiDmczZq6BEDAAQik8no0qVLWlhY0Pz8/L7np6enVSqVdOXKlSPP5a2if1hg8+Zt5XK5qq/3QlC1uVneXLV6e6qy2awWFxfLPXv+iftH1adbMTTp4/SIEcQAAMGYnZ1VqVRSNpvV9evXNTk5Wd7iqFAo6Nq1awcOW3p3SxaLReVyOSWTST3//POHXu/KlSuanp7W9PR0eTslb+HYfD6vfD6vyclJTU9P68KFC1pdXS2vmH/9+vVyT5U3jyyfzx+6t+TU1JRyuZwuX75cddHao+rTjegR8zkbM9w1CQAIVD6f17Vr18qbdF++fFmpVEovv/zyoftMXrhwQZOTk5qcnFQqlao616rS1NSUlpeXVSqVNDk5qYsXL+7pBZuYmCg/7wU97+5Hz6VLl7S4uKhcLnfk1kuZTEYjIyMqFAq6cOFC3fXpRqYV+061wtjYmD1s0mEz/Dfzv6vf/tq2vvo//5AikSo7hKPtlpaWND4+HnQ1UIF2CZ9uaJOXXnpJzz77bNDVaKq1tTUlEuFZm9IYo9nZWV26dCnoqgSm0Tap9b9TY8yytXbsqHL0iPmcjRrt7FrdvseEfQBAd/HmWXXzxPdORBDzORtzesGYsA8A6BbT09NaWFgobxZeyx2XaB8m6/uccYPYytqm9FjAlQEAoEm8EOZNfEd4EMR8zkbdIEaPGACgS3h3OSKcGJr08XrE2OYIAAC0A0HM51S/FO2L0CMGAADagiDmY4zRUCLGZH0AANAWBLEK6XiUoUkAANAWBLEK6XhMK+usIwYAAFqPIFZhKBFjjhgAAGgLgliFdDymW+sb2tntjK2fAABA5yKIVUjHo9q1YpsjAADQcgSxCkOJE5JY1BUAALQeQaxCOh6VxKKuAACg9QhiFdKJmCR6xAAA7bWwsCBjTM1fvS6Xy3XFvwN7TVYY8oLYGnPEAADtMzExoevXr+85ls/ntbCwwGbdXYwgViER61e0P8Lq+gCAtkomk5qYmNhzrFAoaGFhQRMTE6EMYrlcTtlsNpR16xQMTVYwxmgoHtMKc8QAADhQqVTS3NycCoVC0FXpaASxKtLsNwkAANqAIFbFEPtNAgBCLpfLaXBwUJKUzWY1PDysUqmkYrGo6elpDQ4OyhijyclJFYvFfa+fn5/X8PCwjDHKZrPlc3jnlJwbCEZHR2WM0ejoqBYXFyWpfH7vsTFmz+sOks1mq06wn5ubkzFGpVJJkmp+D5X/HpXnnp+f33Peo95XEJgjVkU6HtMXXnsz6GoAADy//fekb30x6Foc7rHvk37oH7f9sqOjo5KckJNMJnX58mVJ0pUrV5RMJpXL5TQ6Oqrbt2+XX7OwsKBsNqt8Pq9UKqVcLidJunbt2p4y09PTunTpkp5//nldvXpVk5OTevnll3XlyhVls1lNTk4qn8/vm9t2kGw2q/n5eS0sLGhqaqp8/OrVqxoZGVEymZTk3KRw1Hs4rk9+8pP6+Mc/XvV9BTHXjSBWxVAiptW7zjZHfZHOvzUWANCdSqWSUqnUnrstZ2dn95S5cuWKRkdH94Sfy5cva2pqSjMzM+Vy09PTGhkZKf988eJFXbp0qXy+qakpjY6OKp/Pa3Z2thxaUqlUzQHGC1v5fL5cl2KxqEKhUA5ftb6H4/rZn/3ZQ99XuxHEqkjHY9q10urdzfJyFgCAAAXQ09Qp/AGmGi9c+Yf2vABXqVQqKZlMqlAolCfjz83N7SlT7XX1mJmZ2XPOhYUFSdJzzz1X13s4jla+r+MiiFWRjj9c1JUgBgAIs2q9UQsLC7p69aqKxWLV8DIxMaFPfOIT5aUn8vn8nqFB7zXLy8vlY55GA0s2m9Xc3Fy5d+vq1auamJjYd52j3sNxtPJ9HRdBrIohVtcHAHQob2J7LpfT2NiYpIfzyDzDw8NKpVLl4yMjI/r0pz9dft4f7po9byqTyZTD38TEhAqFwp65abW+h+Neu9rjIBHEqmC/SQBAJyoWi1pcXNS1a9fKc6kq7xiUnMnxXhDyhiP9vN6xXC63b7V/r3wjPUjZbFa5XK58t6J/3let76EWy8vLe36u5X21G0GsCnrEAACdKJPJlIOG5KzWX20CeiaTUS6X04ULF8rHRkZG9tz9eOXKFU1PT2t6elrZbFalUqm81VI+n993h2M+n98Xbg4yNTWlXC5XvmngOO+h0vDwsCRnKYyJiQktLi5qfn5+X7lf/uVf1sc//vED31e7sY5YFfFYv2L9Ea2ss98kAKCzXLlyRaurq5qenlYul9P09PS+MqlUSoVCQblcrvw1OTmp4eHh8jyqqakpLS8vq1QqaXJyUhcvXlQmk9kTii5duqTFxUXlcrk9d1weJZPJaGRkRIVCYU8YrOc9VHruuec0MjJSLn/r1q2qwepHf/RHj3xf7WSstYFcuF5jY2P2xo0bLb3G0tKSxsfHJUk/+I9/T+99JqX/9cK7W3pNHM7fJggP2iV8uqFNXnrpJT377LNBV6Op1tbWlEgkgq7GHouLi5qcnNw39Hfjxg1NTk5qampq35ytbtJom9T636kxZtlaO3ZUuZqGJo0xU5Iykobd73lr7YLv+aSkGUkLklYlpSRlJV231i76yo1IGpNUdM9T9D8fJkOJGEOTAICu4+/x8ngbjntzxtA+RwYxN4QVveDlhq5lY0zKWusNvqYkzbpfklSSdLEihGUkzVprJ33Hrhljitba5tyX2kTpeEyv374XdDUAAGiq5557rrxSfTab1djYmIrFoq5fv67FxcWa53mhOWqZI5ax1pa3VrfWluQErsqB10lJg5KGrbWD/h4zV7bKa/J6GN5CZSgRpUcMANB1ksmkXnnlFY2NjWl2dlajo6O6ePGiVldXtby8XPN2RWiOQ3vE3N6vC8aYeTeAeRbd5zP+3iy3zEF9mlPaH8RuSApl9B6Kx7R6d5NtjgAAXcfbZgjBO7RHzA1WGffr2NxAl5Ezf6zy/N6wZaikE842R7fu0isGAABa48g5YtbawSqHJySVKuZ2Zdz5ZJIzZ2zVNzyZcs91UG9ZRs4E/tAob3O0tqlHEycCrg0AAOhGx13QNSvpsu/nVUmquJPymjHGO9b+pWobxKKuAACg1eoOYsaYGTm9XeVty92ersrla/PuV+Wk/XqvNSNJ586d09LS0nFPVZP19fXyNb51d1eS9JkXvqDdNwZael0czN8mCA/aJXy6oU3Onj2rO3fuyJjumZe7s7OjtbW1oKsBn0baxFqrBw8eNPWzVlcQc+dyZa21tey8WZQzXFnuDTPGJA8ZntzHXR5jXnIWdG31YoX+BRHXHmzp733md5V+S0bjHx5u6XVxsG5YpLIb0S7h0w1t8pWvfEUnTpxQNBoNuipNE8YFXXtdI22yubmp06dP6z3veU/T6lPvFkezkj5aedAYc6lKWW9ivn/+154dQn0hLVTzwyRnm6MTAxGGJgGgTRKJhO7cuRN0NYAD3blzp+nBuuYgZozJS8pV9mh5C7VWufPRC11F9zVF7Z8rltL+Sf+hYIxROh7TzTWCGAC0QyqV0u3bt7WysqLNzU11yhZ86G7WWm1ubmplZUW3b99WKpU6+kV1qHWLoxk5q+IXfccm5ISsojEmWyVMTUgq+ILbopztjQq+MiPu8VBKx2Ns/A0AbRKLxXT+/Hmtrq7qa1/7mnZ2doKuUsMePHigEye48z5MjtMmfX19SiQSOn/+vGKxWFPrU+sWR5KUdPeKlJyerGlrbdb9edW/uKs75JiVdNF3qpyka9o7qT/rfoXSUCKm11bZ5ggA2iUWi+nxxx/X448/HnRVmmJpaamp84nQuLC1SS0r6x+0Bbt/Rf0FY8yUG9oekTMEOV256r4xJmeMmZX0gpy5Y7NhHJb0pOMxFb5+O+hqAACALnVoEHOHFWu6j7jK3pLVyhS0d2gy1IbiUa3e29T2zq76++q9rwEAAOBwpItDDCVislZavcc8MQAA0HwEsUN42xxx5yQAAGgFgtgh0uVtjugRAwAAzUcQO8RQeeNvesQAAEDzEcQO4fWI3WR1fQAA0AIEsUOcjvbp5EAfPWIAAKAlCGKHMMYonYiy3yQAAGgJgtgR0vEYQ5MAAKAlCGJHGIrHtLLGXZMAAKD5CGJHSCdiDE0CAICWIIgdIR2Plbc5AgAAaCaC2BHK2xzdZXgSAAA0F0HsCEPxqCTWEgMAAM1HEDsC+00CAIBWIYgdYYj9JgEAQIsQxI5AjxgAAGgVgtgRTsf6nW2OmCMGAACajCBWgyHWEgMAAC1AEKtBOh5laBIAADQdQawG6Tg9YgAAoPkIYjVwhia5axIAADQXQawG6XhMq3c3tcU2RwAAoIkIYjVIu2uJsc0RAABoJoJYDYZYSwwAALQAQawGQwn2mwQAAM1HEKuBt7r+Cj1iAACgiQhiNSgHMe6cBAAATUQQq8HpWL9ORfuYIwYAAJqKIFYjtjkCAADNRhCrEavrAwCAZiOI1Yj9JgEAQLMRxGrE0CQAAGg2gliN0vGYbt/bYpsjAADQNASxGnlLWNxiCQsAANAkBLEaDSW8tcQYngQAAM1BEKuR1yPGNkcAAKBZCGI1YuNvAADQbASxGqXdjb8ZmgQAAM1CEKvRqWi/Tkf7tLLGZH0AANAcBLE6pBMx5ogBAICmIYjVYSge0wpzxAAAQJMQxOrAfpMAAKCZCGJ1SCeiDE0CAICmIYjVYSh+QqV7W9rcZpsjAADQOIJYHbwlLG7dpVcMAAA0jiBWB291fZawAAAAzUAQqwP7TQIAgGYiiNWBbY4AAEAzEcTqwMbfAACgmQhidTgZ7VM81s/QJAAAaAqCWJ3S8ShDkwAAoCkIYnUaSrC6PgAAaA6CWJ2cbY5YvgIAADSOIFandDzG0CQAAGgKglidhhIxvXmfbY4AAEDj+mspZIyZkpSRNOx+z1trFyrKjEgak1R0yxSttYv1lgk7bwmLW3c39PjZkwHXBgAAdLIjg5gbwope8DLGJCUtG2NS1tp591hG0qy1dtL3umvGmKK1tlhrmU6Qjjv7Td5cI4gBAIDG1DI0mbHWFrwfrLUlSbOS8r4y2Yqf5f48W2eZ0GObIwAA0CyHBjG39+uC+91v0X0+4/48JalQUeaGe1x1lAk9Nv4GAADNcmgQc3u/Mu5XVW5Iy0harfJaGWMytZQ5TuWD4PWIsc0RAABo1JFzxKy1g1UOT0gqWWuLXojyQlUV5ZB1RJmOmCd2YqBPiVg/S1gAAICG1XTXZBVZSZfdx5XDltXUUmYfY8yMpBlJOnfunJaWlo5zmpqtr6/XdI1TfTv6UvE1LS3dbGl9UHuboL1ol/ChTcKJdgmfsLVJ3UHMDUer1tq5FtRnD/euzHlJGhsbs+Pj4y293tLSkmq5xvkv/7GMMRoff19L64Pa2wTtRbuED20STrRL+IStTepa0NUdhsz6l6DwPXdkr1ctZToB+00CAIBmqHdl/VlJH6045s3tSvkP+kJXscYyHYP9JgEAQDPUHMSMMXlJucoJ9+7PRe2fB5aSO6G/ljJ11zxA6bizzdHG9k7QVQEAAB2spiDmzgub9QcmY8yEb9mJRTlbF/mNuMdVR5mO4C1hcYteMQAA0IAjg5i7xZEkJY0xI+7XhKRpXzDLSZqueGnWPa46ynSE8qKuzBMDAAANOPSuSXcO17UDni73jllrS8aYnDFmVtILctYF29ODVkuZTuHfbxIAAOC4Dg1i7twuU8uJ3P0oK7cwqrtMJ2C/SQAA0Az13jUJ+YcmmSMGAACOjyB2DGxzBAAAmoEgdkxDiRgbfwMAgIYQxI4pHY9phR4xAADQAILYMdEjBgAAGkUQO6Z0PEqPGAAAaAhB7JjS8ZjuPNjWgy22OQIAAMdDEDum8jZHd1nCAgAAHA9B7JjKa4kxPAkAAI6JIHZMabdHjLXEAADAcRHEjoltjgAAQKMIYsf0yGln42+CGAAAOC6C2DGdGOhT4gTbHAEAgOMjiDVgKBFj428AAHBsBLEGpOOsrg8AAI6PINaAIfabBAAADSCINYD9JgEAQCMIYg1Ix6NaY5sjAABwTASxBpRX16dXDAAAHANBrAEPF3XlzkkAAFA/glgD2G8SAAA0giDWgPJ+kwxNAgCAYyCINSAdd7c5okcMAAAcA0GsAbH+Pp050c9kfQAAcCwEsQalWUsMAAAcE0GsQc7q+tw1CQAA6kcQa1A6EWNoEgAAHAtBrEFD8ZhuMlkfAAAcA0GsQUOJmNY22OYIAADUjyDWoPISFgxPAgCAOhHEGuRtc8TwJAAAqBdBrEEPN/7mzkkAAFAfgliDHgYxesQAAEB9CGINesSdI8bQJAAAqBdBrEGx/j6dPTlAjxgAAKgbQawJ0vEoPWIAAKBuBLEmGGJ1fQAAcAwEsSZIx2PcNQkAAOpGEGuCNNscAQCAYyCINcFQIqZ1tjkCAAB1Iog1wVCc1fUBAED9CGJNkE64a4kxYR8AANSBINYEQ/ETkqQVesQAAEAdCGJN4PWIceckAACoB0GsCR45zRwxAABQP4JYE0T7I0qeYpsjAABQH4JYkziLuhLEAABA7QhiTcJ+kwAAoF4EsSYZSpygRwwAANSFINYk6XiUuyYBAEBdCGJNko472xzd32SbIwAAUBuCWJMMJZwlLBieBAAAtSKINUl5v0mCGAAAqBFBrEm8HjHunAQAALXqr7WgMWZKUslau1hxPClpRtKCpFVJKUlZSdf9ZY0xI5LGJBUlZSQVK8/VydJxhiYBAEB9agpixpgJSVckTVd5OiVp1v2SpJKkixUhLCNp1lo76Tt2zRhTtNYWj1v5MHkk7u43ucadkwAAoDaHDk0aYzLGmLycHqzVQ4pOShqUNGytHbTWLlQ8n5WUrziW18Pw1vEG+iIaPDWgm+sPgq4KAADoEIcGMWtt0VqbtdbOH3Uia23pkN6tKUmFimM33ONdIx2P0SMGAABq1vLJ+u4csn09atbakvt8ptV1aBf2mwQAAPWoebL+ETLuZH7JmTO26hueTEkPg1e118qZwN/xhhIxvfj6QW8TAABgr2YEsVVJ8s8Lcyfie8eSxz2xMWZGzh2ZOnfunJaWlhqs6uHW19cbusaD0oa+XdpueT17SaNtgtagXcKHNgkn2iV8wtYmDQcxt6ercg5Z3v2qnLRf77nnvXOPjY3Z8fHxRk53pKWlJTVyjS/pr/S7X/9Lvff9H9CpaLM6G3tbo22C1qBdwoc2CSfaJXzC1iatmiNWlDNcWe4N8z/uVt7q+kzYBwAAtWg4iBljLlU57E3M98//SlW8zgtmXTE/TJLSCbY5AgAAtWsoiHkLtVa589ELXUV36LKo/XPFUnJW6u+aIFbeb5JtjgAAQA0aCmJuiMpWCVMTkgq+OyUX5Wxv5DfiHu8a3n6TLGEBAABqUU8QS6n6HZCr/h4xd8gxK+mir0xO+7dHyrrHu0bqtLvNEUEMAADU4NBb+9xQ9bycuV5JOcOQk3I29F6QnGUrjDFT7jpij7jlpv29ZNbakjEmZ4yZlfSCe77ZbhqWlHzbHDE0CQAAanBoEHOHFo/staqyt2S1MgXt3+ao6wwlWF0fAADUpuVbHPUaZ5sjlq8AAABHI4g1WToeY2gSAADUhCDWZAxNAgCAWhHEmiwdj+ne5o7ubmwHXRUAABByBLEmS8dZwgIAANSGINZkLOoKAABqRRBrsnR5myPunAQAAIcjiDXZo2z8DQAAakQQa7LU6aiMkVZYwgIAAByBINZk/X0RDZ6KMkcMAAAciSDWAkMs6goAAGpAEGuBdIIeMQAAcDSCWAuw3yQAAKgFQawFGJoEAAC1IIi1QDoR0/0ttjkCAACHI4i1gLeoK/PEAADAYQhiLeBtc8TwJAAAOAxBrAXY+BsAANSCINYCQ95+k9w5CQAADkEQawFvmyOGJgEAwGEIYi3Q3xdRim2OAADAEQhiLZKOx9j4GwAAHIog1iJDiZhu0iMGAAAOQRBrkXScoUkAAHA4gliLpN1tjqy1QVcFAACEFEGsRYYSMT3Y2tXdzZ2gqwIAAEKKINYi5W2OmLAPAAAOQBBrkbS3zRHzxAAAwAEIYi0yRI8YAAA4AkGsRdIJ9psEAACHI4i1yCOnY4qwzREAADgEQaxF+iJGqdNRNv4GAAAHIoi1UDoeY2gSAAAciCDWQkOJGEOTAADgQASxFqJHDAAAHIYg1kLefpNscwQAAKohiLWQt83R+sZ20FUBAAAhRBBrofI2R9w5CQAAqiCItdDDIMY8MQAAsB9BrIWGvP0muXMSAABUQRBrIXrEAADAYQhiLZQ6HVUonjxcAAAgAElEQVTEsPE3AACojiDWQs42RzHdpEcMAABUQRBrsXQ8qptr3DUJAAD2I4i12FCC1fUBAEB1BLEWG4qz3yQAAKiOINZiabdHjG2OAABAJYJYi6XjUW1ss80RAADYjyDWYizqCgAADkIQazH2mwQAAAchiLUYq+sDAICDEMRajKFJAABwEIJYiw2ecrc5okcMAABUIIi1WF/E6BHWEgMAAFX011rQGDMlqWStXazy3IikMUlFSRlJxcpytZTpVuk4q+sDAID9agpixpgJSVckTVd5LiNp1lo76Tt2zRhTtNYWay3TzdLxqG5y1yQAAKhw6NCkMSZjjMnL6cFaPaBYVlK+4lhe0mydZbrWUCKmFYYmAQBAhUODmLW2aK3NWmvnDyk2JalQceyGe7yeMl1rKB7TTbY5AgAAFRqarG+MSapKb5m1tuQ+n6mlTCN16ATpeEyb27taY5sjAADg0+hdkynpYaiqIlNjma7GWmIAAKCaRoNYskllulp5dX2CGAAA8Kl5+YogGGNmJM1I0rlz57S0tNTS662vr7fkGq+t7UqS/uBzn9f9V0P9Tx46rWoTNIZ2CR/aJJxol/AJW5s0JRUYY5KHDD3WXKaSe5PAvCSNjY3Z8fHx41eyBktLS2rFNW6tb+jvf3ZRjz41rPEffKbp5+9mrWoTNIZ2CR/aJJxol/AJW5s0OjTprQGW8h90J+h7z9dSpqsNnoqqL2K0wlpiAADAp6EeMWttyRhT1P55YCk5q/B7C7oeWaabRSJGqdNRVtcHAAB7NGOvyUU5Wxf5jbjH6ynT1YbYbxIAAFSoJ4ilVP0OyJz2b32UdY/XU6arpRPsNwkAAPY6dGjSncf1vJy1vpKSZo0xk5KuW2sXpPLwZM4YMyvpBbfsrH/IsZYy3S4dj+rl76wHXQ0AABAihwYx9y7HI3utrLUF7d/CqO4y3Wwo4QxNWmtljAm6OgAAIASaMUcMNRiKx7S5s6s7D9jmCAAAOAhibVJeXZ95YgAAwEUQaxP2mwQAAJUIYm1CjxgAAKhEEGuTdDwqiY2/AQDAQwSxNvG2ObpJjxgAAHARxNokEjF65HRUK2vsNwkAABwEsTYaYnV9AADgQxBro3Q8xtAkAAAoI4i1UToeY7I+AAAoI4i1kTM0uSlrbdBVAQAAIUAQa6N0POpsc3SfbY4AAABBrK3Kq+szTwwAAIgg1lZDcbY5AgAADxHE2iidYJsjAADwEEGsjdhvEgAA+BHE2ih5ckD9EcPQJAAAkEQQa6tIxOiReJQeMQAAIIkg1nbpuLOWGAAAAEGszYYSMYYmAQCAJIJY2zk9YgQxAABAEGs7L4ixzREAACCItdlQIqatHas3728FXRUAABAwglibpeNRSawlBgAACGJt93CbI+6cBACg1xHE2oyNvwEAgIcg1mblbY5YwgIAgJ5HEGuzs+42R8wRAwAABLE2i0SM0nEWdQUAAASxQKQT7DcJAAAIYoEYYr9JAAAgglggGJoEAAASQSwQ6URMt+6yzREAAL2OIBaAoTjbHAEAAIJYINLeoq4MTwIA0NMIYgHw9ptkdX0AAHobQSwAj7o9Ytw5CQBAbyOIBSAdZ2gSAAAQxAJx9uSABvrY5ggAgF5HEAuAMc42R2z8DQBAbyOIBSQdjzFZHwCAHkcQC0g6zn6TAAD0OoJYQIYSbHMEAECvI4gFJB2P6db6pnZ32eYIAIBeRRALSDoe0/Yu2xwBANDLCGIBGfK2OWKeGAAAPYsgFhBvUVeWsAAAoHcRxAIylGC/SQAAeh1BLCBD8ROS2OYIAIBeRhALyJmT/Yr2Rdj4GwCAHkYQC4gxRo+wqCsAAD2NIBYgFnUFAKC3EcQClI7H6BEDAKCHEcQCxH6TAAD0NoJYgIYSMa2wzREAAD2rvxknMcYkJc1IWpC0KiklKSvpurV20VduRNKYpKKkjKSi//lek47HtLNrVbq/pdTpaNDVAQAAbdaUICYneM26X5JUknSxIoRlJM1aayd9x64ZY4rW2mKT6tFRvG2OVtY3CGIAAPSgZg5NTkoalDRsrR201i5UPJ+VlK84ltfD8NZzvG2OuHMSAIDe1NQ5Ytba0iG9W1OSChXHbrjHe1J5v0km7AMA0JPaMlnfnUOWkTN/rMxaW3Kfz7SjHmHjDU3SIwYAQG9q1hwxScoYY7zerZSkVd/wZEp6GLyqvVbOBP6ecuaEs80RG38DANCbmhXEViXJPy/MnYjvHUse56TGmBk5d2Pq3LlzWlpaakJVD7a+vt7ya1SKD1j9+Vdf1dLJb7f1up0iiDbB0WiX8KFNwol2CZ+wtUlTgpjb0zVfcTjvflVO2q/nvPPeecfGxuz4+PhxT1WTpaUltfoalZ768z9S/6moxsff29brdoog2gRHo13ChzYJJ9olfMLWJq2cI1aUM1xZ7g3zP4YjHWe/SQAAelVTgpgx5lKVw97EfP/8r1TF67xg1nPzwzzsNwkAQO9qOIh5C7VWufPRC11Fd+iyqP1zxVKSDlvyousNJWK6dZdtjgAA6EUNBzE3RGWrhKkJSQXfnZKLcrY38htxj/esdDyqnV2r2/c2g64KAABos2bNEVv194i5Q45ZSRd9ZXKSpitel3WP96x0eZsjghgAAL2mWXdNLhhjptx1xB6RMwQ57e8ls9aWjDE5Y8yspBfkzB2b7eVhSUl67MwJSdJffWddb3ssEXBtAABAOzVtQdcqe0tWK1PQ/m2Oetq7n0rqyeRJ/d9/8jX9h+98POjqAACANmrLFkc4WH9fRP/lB57R515Z1RdeO2jjAQAA0I0IYiFw4fufUuJEv658pqdHaQEA6DkEsRCIx/r1t3/gvH77i9/Ua6v3gq4OAABoE4JYSPzk+59RxBj9i8++EnRVAABAmxDEQuKxsyf0sXc9oasvvKY3720FXR0AANAGBLEQ+ekPZnRvc0e//rlXg64KAABoA4JYiLz9iTP6wFvT+tU/fkWb27tBVwcAALQYQSxkLn4oo2/f2dCnXnwj6KoAAIAWI4iFzIe+O623nUvoymeKspaNwAEA6GYEsZAxxuinP/iMvvytNX3mqytBVwcAALQQQSyEPvbuJ/RoIsYCrwAAdDmCWAjF+vv0n7//aX3mqyt66Zt3gq4OAABoEYJYSP0nP3Bep6J99IoBANDFCGIhlTwV1XNjT+lTL76hb735IOjqAACAFiCIhdhPfeAZ7exa/eoffy3oqgAAgBYgiIXYU6lT+qF3PK5f+9Ova31jO+jqAACAJiOIhdxPf/AZrT3Y1ideeC3oqgAAgCYjiIXce84P6vufHtSv/NEr2t5h2yMAALoJQawDXPxgRt8o3de//YtvBV0VAADQRASxDjDx7Dk9kz6tK3/ItkcAAHQTglgHiESMfuoDz+jF19/U515ZDbo6AACgSQhiHeLHRt6iwVMDuvKZV4KuCgAAaBKCWIc4Ge3Tf/a+p7X40rf18s31oKsDAACagCDWQT7+vu9StD+i/5NeMQAAugJBrIOk4zH92MiT+s3C61pZ3wi6OgAAoEEEsQ7zUx/IaGN7V//yT74edFUAAECDCGId5q2PxvXR731U//LffV0PtnaCrg4AAGgAQawDXfxQRqt3N/X/FF4PuioAAKABBLEO9APPpPTOt5zVr3zmFe3ussArAACdiiDWgYwx+ukPZlRcuatPf/k7QVcHAAAcE0GsQ/3wOx7Tk8mTuvKHxaCrAgAAjokg1qH6+yL6yR98Wp/72qq+8Fop6OoAAIBjIIh1sB9/73klTvTrymfoFQMAoBMRxDpYPNavv/0D5/XbX/ymXlu9F3R1AABAnQhiHe6/eP/TihijX/kjtj0CAKDTEMQ63ONnT+pj73pCn7jxmt68txV0dQAAQB0IYl3gpz+Y0b3NHf3a59j2CACATkIQ6wJvf+KMPvDWtH71s19jrhgAAB2EINYl/s7Ed+vOgy199Bf/QP/kd76suxvbQVcJAAAcgSDWJcaeTun3/vtx/fA7HtM//f2X9ZFfWNLC8utsgQQAQIgRxLrIE8mT+qUff49+82fer8eTJ/Vz117Uj/6zz+rG11aDrhoAAKiCINaFRs4P6l//1+/XLz73Ln37zgNN/R9/ov/2X31e3yjdD7pqAADAhyDWpSIRo7818hb9/s+N62f/+lv1u3/xLX30f1nSL17/iu5tMn8MAIAwIIh1uVPRfv3dv/E2/d7PjWvy7Y/plz/9Vf31X/gD/evPM38MAICgEcR6xJPJk/rff+I9Wviv3qdHz8T03119UX/rn/+x/rR4S9YSyAAACAJBrMeMPZ3SJ3/mB/UL0+/SG6X7ujD/7/Tv/9If6v/67CuszA8AQJsRxHpQJGI0NfoWLf0P45r9se/TyWi/fv5TX9J7/9Gi/u4nvqDlr6/SSwYAQBv0B12Bmm3elV77XEsvcebNL0uvnWrpNcLklKQLj0kXPjag4spJXf/St/WHf/FX+oef39b51GlNvv1Rffh7HlX8RHD/mfRam3QK2iV8aJNwCqpdrLXa2t3Vzo60Kyu7K+1aq11Z7e5aWSvtWueYtQ+fc8q5Px/2B7mt/HHvAf9LK89SWfag17XKnde/Imm89ReqkemUno+xJ/rsjZl40NUAAACd7n96s+WXMMYsW2vHjirXOT1ijwxL/+k/a+klXvyzP9O73vnOll6jk3xt5Z5+/y+/oz8p3tKDrR29JXlK428b0g++Na3Tsb621IE2CSfaJXxok2BYK61vbOvO/S29+WBLb953Ht95sKW7G9t649srGjiV0N3NHd3b2Na9zR3d3djWTg2dIBFjFBuIKNYf0Yn+PsX6I4oN9Km/z2ggYtTfF1G/7/tAxc/9fUZ9xigSMYoYIyMpEpGMvGPuYyMZ77hxfo4YSe5r/IypPOJ7bl9Z/3MVzx58msOeaopXXinqb7b4GvXonCAWOyO9daKll7j9er/01vGWXqOTPP1W6Sf/Pem5jW196sU39K8+96p+/U/e1Ikbu/qRdz6mn3jveY2cTx76wWwUbRJOtEv40CbNs7trdfveplbWN3VzbUMr687XzfUNraxtln9eWd/QrfVNbe9aOVOuY+6X1BcxOntyQAP2CT128ozOnBnQ2ZPO13n3+5mTD4+djvXrdLRPp2L9OjXQp1OxPkX7Ii39/2uvWtVS0FXYo3OCGAJzOtavH3/vef34e8/rz7/xpn79c6/q//38N7Sw/Lq+97GEfuK95/Wj735SZ08NBF1VAKhqZ9fq1t39QWplfVMra27IWneeW727qZ0q6ywO9Bml4zGl4zE9mojp7Y+fUToRc49FNZSIach9/uzJAUUiRktLSxof/0AA7xidgiCGurzjybP6R//x9+l//OFn9VtuL9k/+K2/0D/8/17Sh982pI+96wlNPHtOJ6PtGboE0Ls2tne0sr6pW/5Q5fZa3bq7UX68sr6h1XubVSeCR/sjbniK6omzJ/TOJ88qnYg6x8ohywlYZ07200OFpiOI4VhOx/r1E+89r59we8k++flv6FN/9oauf+nbOhXt0994+zn9zXc/qQ98d1oDfaySAuBw1lrdebCtN+9t6fa9Td2+t6k372/p9t1N3b63pVt3nWFAf+Bae1B9u7ZT0T6l4zE9Eo/qqdQpjXxXcl+wSsejSidiSsQIVwhWW4OYMWZE0pikoqSMpKK1drGddUDzvePJs3rHk2f1/A8/q8+9sqrfevEN/ZsvflOf/MIbGjw1oI9876P68PcM6UPfPaTB09GgqwugBXZ3re5ubmt9Y1vrD7a1trGttQfO4/WNLeexe6x0b0tv3ncC1u17m3rz3pZK97eqDgd6kqcGygHq7U+c0VA8pkdOR8vh6pG404v1SDyqU1H6GNA52vZfqzEmI2nWWjvpO3bNGFO01hbbVQ+0Tl/E6H3Dj+h9w4/o5z/21/SZr97Up158Q7//5e/oNwvfkDHSu96S1PjbhvSh7xnS9z15lt6yHrO7a7W5s6vNnV1tbXvf3WPbu9racb42ved2bPn4tvtL2ltyp/wre896RV6Zh8eMce4+63PvHItEnDvJ+iLVj0cicp83Mu738nH3sSnfWea91vfYqPw67/FRZethfes+WT1cB2pjx+re5rZ2rbTt/ttt7exqe8f5993e9T12n9tyy227bbKxtasH2zu6v7mj+1vO18bW7p6fH7hf97eccvc2d5ywtbld0xpQp6J9Sp4cUPJUVIOnB/TsY2eUPDWg5KkBDZ6KKnkqquTJAQ2eHig/PntyQP38vwJdqp1/NmQl5SuO5SXNSppuYz3QBtH+iD767Dl99Nlz2tm1+rPXS/qDr9zU0l/e1P/26a/qlxa/qlh/RO948qze9Zak3n0+qXe/JamnUie7cphgd9dZXHF3V9pxF0rc3bXa2bXacX+R7rg/W+uUcR7b8uNdd0HGHfe1u+5rvIUXvcc73sKN7rl3rSqu5ZR5+Hj/db3zb+8+DEUPg5LV6998oF979UY5NG3t7GrTF5r2HPeFqsN6PHpZxKgc/CLGCZJW+0PXkUHn+u80tV7R/ohODvTp5ECfTgxEdGKgTyejzs+JEwM6MRDR6Wi/4if6lTgxoETMeRyP9StxwvmKxwbKx+KxfvVFuu/zDTSinUFsSvuD2A1J19tYBwSgL2L0nvODes/5Qf2die/R7bub+uzLK/rCqyV94bWSfu1Pv65/8dlXJDl/LT+TPq1n0qeVSZ/W/Ztb6v/qinM3UiKmpHsnUqO2d3b1YHtX9za3dX9zR3c3dnR/a1t3N5y/8O9tbld8dx+7z9/1Xre54/Y6uD0Luw97G7Z3rbbdY52aP4yRon0RRfsjivZFNOA+3trY1ZruKdbvHBvoi+hU1HvelMsOuK+L9kc00GcU7evTgPt81Pda5/ymyrGHj52eqP31c74/fMJUPOcF0cqwelCIPer4zq4thyR/8LVe6PUF5V332KFlffXzjhn3PRl3PSdvrSdvXaeI/zn38SvFot46PCxjpP6I828/4K0t1ef8m/f3Oe3g/Rs/PG7UH4noZLRPJ/q9731N+awBOFxbgpgxJilnTtiq/7i1tuT8T8RkGJ7sHYOno/qRdz6hH3nnE5KkrZ1d/eW31vTi6yX91XfW9crKXX3xG2/q33zxm9q10pUv/mn5tf0Ro/iJfp2O9ut0rE+nov0PfxnJ+eVrZCTj/PJ7sLXrDKVs7+j+5q423MdbO/UloxMDEZ2K9utUtM/9ch4nTw0o2h9RfyTiLrLofncXVOyrOBbxDYmVh8W8oS+jPcNje4bNvCGyiDvsVR46O3gYbe/rdeAQXPlY5OEQmlf2oN4L55b8Dx37vwE035Je1/iHh4OuBoA6tatHLCU5weuA5zNyJvCjBw30RcoT/v02t3f1m7+zpMzb362baxv6ztoD3VzbcCYDb2zrrrtKtTOM4wzh+B9HIkbpeNQZThnoU8w3xOJ8d4ZZ/MFqz2M36J0c6GM4BQDQEm3Za9K9W3LZWrvvt5kxxkqarHb3pDFmRtKMJJ07d270N37jN1paz/X1dcXj7GcZJrRJONEu4UObhBPtEj7tapOPfOQjnb/XpLV2XtK8JI2Njdnx8fGWXs8ZbmntNVAf2iScaJfwoU3CiXYJn7C1SVvvB3bnigEAAEDtC2Le/K+U/6AvmDE/DAAA9Jy2BDF3kn5RUmWPWEpSiTsmAQBAL2rn0OSinO2N/Ebc4wAAAD2nnUEsp/0r6Gfd4wAAAD2nbXdNuou35owxs5JekLN22CzDkgAAoFe1dfkKa21BUqGd1wQAAAgrtrMHAAAICEEMAAAgIAQxAACAgBDEAAAAAkIQAwAACAhBDAAAICAEMQAAgIAQxAAAAAJCEAMAAAgIQQwAACAgxlobdB1qYoy5Kenrks5KevOAYgc9V+14tWNpSSsNVLMRh72vVp6j1tccVa5V7dLpbXLc89TymqDaRAquXcLeJrWU47PSvPOEvU0kPiuNlKu3XWptq3a1yXdZa4eOLGWt7agvSfP1Plft+AHHboTxfbXyHLW+5qhyrWqXTm+TVrZLUG0SZLuEvU2CbJde/KyEvU2CbJde/KzU2lZBflaqfXXi0OSnjvFcteOHnScIzajPcc5R62uOKteN7dKsurSqXWiT9p6Hz8rBgmoX2uRgvfhZqaetQqNjhibbwRhzw1o7FnQ98BBtEk60S/jQJuFEu4RP2NqkE3vEWmk+6ApgH9oknGiX8KFNwol2CZ9QtQk9YgAAAAGhR+wIxpiMMeaSMWbC/Z4Muk54yBhzPeg6wGGMGTHGzLifk2vGmEzQdUK5XSaMMVPGmDztEi7GmHzQdYDDGDNrjLHGmNvGmOvt+qz0t+MiHe6atXZUcsaVJV2RNB1slWCMmZCUkTQRdF0guX+gjFlr592fJyRdlzQcaMUgSZ+W9Iy1tmSMSUm6Jmk04DpB5c/JjKRs0HWBJOlla61p90V7IogZY6Yklay1i1WeG5E0Jqko5xd70SvnPrfqlXX/R8Yv/iY5brtIkq+N+GuyiRpok4yknB7OvbghKWOMSVprS62veXdr5LMiN4S5j1crX4/jabBNvD9eViXx+WiiRtslEEGvn9HqLzk9JrclTVR5LiPpesWxa5Iy7uMpOT1i/udfljQS9Pvq9K9G2qXiuA36vXTLV6NtUvF4QtLtoN9TN3w167PiPne92nn4an+bSJpyv/M5CUm7SLrk/t6fkDQrKdmOenftHDF3bldezj/+QX8FZiVV9qjk5TSAJKVaVL2e1aR2QRM1q02stcWK8hebWc9e08zPijfXVc4vouB7ADpUs9rEHVmhHZqkiZ+VeWvtgvsZuSpnWL/leuKuSWPMy5Kylf8Dco9P+n+BuN3Ft621xu3izFprJ33P35b0UWttoU3V71rHbZeKsrbyGI6vSW0yI2nVWrvQjjr3gma0i/vcjKRp///TcDwN/F5JyumFKbjP3bbWDraz7t2sWZ8V93kradC2eHpF1/aIHcX7MKgiPXv/4O7dEkVV6RUjhLVOje2CNqqnTdy/9IuEsNarpV18PWGeT0ia4HPUGjV+ViYkjbl3GM9ISrqPaZMWqfGzMmKMWa58batDmNQjk/UPkJIO/UfOWGsXjXkYlN0PCt3JrXVku8gJyGifmtrEu7nF95f+FIGspWppF0l6pOJYqWIYGc1Ty++VPZ8JY0zeuncbo2Vq+azckG/o0v2jsi3//+rlIFbremAX3b8oC5JGxLyXVqupXdxf+hPu41kx96WVjmwT94+UT8v56947XFSb/kfWo45sF/ePyaTb8yJJk5I+2tpq9bSa15l0e2lm3MeXJC0QkFumls9KyRhT9H1WhtWm3/e9HMRq4v517w1F8os+JHztMhd0XVCeqM88lxCq6IGh5yUk3N6ZOfH/sNAI6o/5np0j5mGl/HCiXcKHNgkn2iV8aJNwCmu79HIQ87qA90zG9zUUXcTBoF3ChzYJJ9olfGiTcAp1u/RsEHO7hYvaP3acEpNZA0O7hA9tEk60S/jQJuEU9nbp2SDmWpSz3YHfiJgLFjTaJXxok3CiXcKHNgmn0LZLrwSxlKrfNZHT/g28s+5xtB7tEj60STjRLuFDm4RTx7VL166s7479Pi9nfZApOd2Si3KWOVjwlRuRdEHSC27ZAssgtA7tEj60STjRLuFDm4RTp7dL1wYxAACAsOuVoUkAAIDQIYgBAAAEhCAGAAAQEIIYAABAQAhiAAAAASGIAQAABIQgBgAAEBCCGAAAQEAIYgBaxhiTr/h5yhgzUcfrLxljrje/Zs3TCXUEEF4EMQCtlKn4eVLO9iO1+n5JNQe3gHRCHQGEFEEMQEu4+79Vhq6MtbbmIGatnbbWmorzzhpjKgNeyx103Wp1BIBaEcQAtMqEpGXvBzeYlRo5oXuOS5JGGqtaZ1wXQPcjiAFolUlJi76fxyS9EFBdACCUCGIAWqVyGLIymB3JHQ607uNrkm67T10zxlhjzO2K8lPGmGX3uWX/jQHuuW67j/PGmJeNMUljTMYYc80Yc9t93XX/EORR1/XX0XdsxD2PV4+ZKu/rdpVyIxXlptx62mp1A9D5CGIAmu6AYcgRa22hgdNelBPmJCkraVjSqO+aU5KuyQl703Lmp+0LLsaYZTm9c3lrbck9l//8KfmGVI+6biW3Hsvu9SclXZWUr7yDVFLSrW/ePW/G/dk7j/fzgns97z0lD7o2gM7TH3QFAHQ+N2T4A09KUrJiWYexip9nrbU195BZa0vGGK+HbbXKpP8rkuastTn35wU3dGUleceS7mu9YCVfee+9XJS0bIyZstYu1HDdSlckzVtrvYC3aIwpyAmF1yrec85au+Bed1jSJWNM0g2IXu/YZffngpxQBqCLEMQANMwXOiSVg1nODRByhwgz1tr5VlzfHdJLygkylyqeXq34OatDWGsLxhhp/9IbtdRjwq3Hnt4va+2iMaYkp1fLH8T8j1+uOJ333KeNMVclLTbYowgghAhiAFoh44Uw17Sk2VZez/0+qv1DonuCWLUeLXc48YJ7nkbmYHmvrdZrVqw8d8W/kSqfM8ZMyvl3m3XruejvzQPQ+QhiABpWMTRZdVhSUsbtafLUNTR5hHLwqWedMkly65mRE3ZuuIeXD35FTfXIyBlK9Mv4zl8T999n1J1zNyNp1hhzyVo7d8z6AQgZghiAhvmHJt2hwYIXstxJ57nK4ctjqhxm9K5fcIf+ZvVwYr1Xn+RBPU9u3SYkTfvmalWbDF/1ulV4QSsr3xCo2+PmTc6vm1v/OWPMBTkr+QPoEgQxAM32/RU9NhOSmrIXoztcJ0lZ77tvqO6inOUlvDsRk3LCUFEHzAuz1ha9AOeer6SHE/trvW5luaycuyRTcu6Y9HrbFurpAXSXvMi578Ub1hxRxfwzAJ2N5SsANFtlj9Ko6lw/7AhzcsLdrHzDf26P1qh7/ety7l4sqkqwqnBRznDqNb3MFvAAAACcSURBVPecB/VaVb1uJfeGhEk9XI4iK6dHcPqIelT6hJy7JLO+88y16oYHAMEw1tqjSwFADaoNQxpjrjPBHACqo0cMQDPtGYZsxv6SANDNCGIAmqlyG6OmzQ8DgG5EEAPQTKsVdyhm1Nz5YQDQVZgjBgAAEBB6xAAAAAJCEAMAAAgIQQwAACAgBDEAAICAEMQAAAACQhADAAAIyP8PkvxH2nsITFAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(10, 8))\n",
    "plt.plot([f(x) for x in conv_x_da], label=\"Objective\")\n",
    "plt.plot(problem.value * np.ones(len(conv_x_da)), label=\"Traget value\")\n",
    "# plt.yscale(\"log\")\n",
    "plt.xscale(\"log\")\n",
    "plt.legend(fontsize=20)\n",
    "plt.xlabel(\"\\# iterations\", fontsize=20)\n",
    "plt.xticks(fontsize=20)\n",
    "plt.yticks(fontsize=20)\n",
    "plt.grid(True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 0, '\\\\# iterations')"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAETCAYAAADTbHYBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAF9VJREFUeJzt3c9vG+l9x/HP11rbiRE7kmxtu03XMShsEaRNUVBctAnQHAL6GBRFpfWhhxZFLbWXXlpI6CnoodhK6D8guYceChRe6VCglxSiL7m06VJCWqTbIoC4m80PJJVNMXZ3HVsrf3vgM9KIoiyRMyTFR+8XIJDzcDjzjEh9+OiZh/OYuwsAEKcLg64AAKB3CHkAiBghDwARI+QBIGKEPABEjJAHgIgR8gAQMUIeACLWccibWTF938zczLbCz3IoXwy3s/lVFQDQqVc6WdnMypKWJU2GonF3t/BYUVIjlM+a2bSkuZO2eePGDb9161Yn1QCAc29jY+Ohu0+ctF5HIe/uFTOrpZdTD5fcfSXcv+vua6fZ5q1bt1StVjupBgCce2b2/dOsl0uffGjhv5MqKphZ2czm89g+AKA7eZ14ve3uSVeN3H0ptPKvhw8AAMAA5BXy6ZOxSX+8JD2SVGhdOaxTNbPq9vZ2TlUAALTKHPJm1hriVUlJX/1kWD7E3VfcveTupYmJE88bAAC61FHIhxZ6KdVST6RPxm5KeiussxWWAQAD0OnomjVJay1lNbUMlUyNsgEADBDfeAWAiA1tyH/zuz/RvW/VTl4RAM6xjrprzpLKf/9U/7r1SHe/emTwDpCbZ8+eqV6v68mTJ9rb2xt0dRCpkZERXb16VePj47p8+XKu2x7akAd67dmzZ/rwww81NjamW7du6eLFizKzQVcLkXF37e7u6vHjx/rwww918+bNXIN+aLtrgF6r1+saGxvTjRs3dOnSJQIePWFmunTpkm7cuKGxsTHV6/Vct0/IA8d48uSJrl27Nuhq4By5du2anjx5kus2CXngGHt7e7p48eKgq4Fz5OLFi7mf+yHkgZegiwb91Iv3GyEPABEj5AEgYoQ8AESMkAeAiBHyANqqVCoyM62snK/rDS4tLen27duDrkZuCHkASHn33XdVqVROXnFIcFkDAEhZXV0ddBVyRUseACLWccibWbFleTHczqbKps2sbGbz2asIAOhWR901ZlaWtKzm3K2JZOLuubBOUZLcvWJmBTMrMgUgYvNX//xfeu/HjwddjZf64i9d0ze+/qs930+tVtPCwoIqlYoajYbK5bKWl5dVKBy+DPjKyooWFxdVq9U0O9tsE1YqFdXrde3s7GhhYUErKyva2dnR3NycKpWKNjY2VK/XT9x+8twHDx7sr1ssFnXv3j0Vi8WO67G0tCR3P/XxdbL/fuuoJe/uFaXmcw3uuvtkeEyS7khqhPs1SeVsVQRwli0vL0uS7t27p/X1ddXrdU1NTR1aZ21tTXNzc1pYWNDq6qoqlYoqlYpWV1f14MGDQ+tOTU2pWq1qbm5Oo6Ojp9q+JDUaDc3MzGhubk7Ly8uq1WqamZnpuh6dHN9p9z8Q7t7Rj6T1luV5NYN8PiwvSyqG+2VJiy/b3tTUlHfjz9/5jn/l7QddPRc4jffee2/QVRio9fV1l+TLy8sdPW9jY8Ml+erq6n5ZsVj06enp/eXV1VVvxs+B+fl5l+Tlcrnj7SfPbVe2s7PTVT16sf/TOO37TlLVT5HZmU+8uvuSN1vx10N3DoBzLOmeqNUO/ulvNBoaHx8/sm6j0ThSlrScO9l+olw+iKDJyckjj3dSj17sfxAyhbyZJf3xkvRIUkHNrprktzgayts9r2pm1e3t7SxVAHAGrK2taWZmRlNTUxobGzvyeLlc1jvvvKPNzU01Gg0tLy+rWCxqdHT0yLqtffmn2X6i3fa6rUcv9j8IWcfJV3XQRz+pZldNVVIplBUkHflWgbuvSFqRpFKp5BnrAGCAbt++vX9yslRq/um39llPTk5qfHx8v7xYLB7bB97N9k+rm3rkuf9B6HR0zbSkkplNu/uau2+GVnld0paHUTRmVgpdNw1nZA0QrVqttn/icnq6+U99u66P+/fva3l5WeVyWY1G49Qt3tNu/7Q6rUfe+x+EjkLe3dckrbWUHbmwRbsyAMNpY2NDa2trR8rL5bIKhYJGR0e1sLAgqdldsbi4eGTdQqGghYUF3blzZ7+sWCwe6sNu57TbP61O65H3/geByxoAeKmVlZW2FylbX19XuVzWvXv3dPfuXc3MzKhYLO6PcU8bHx/X2tqaNjcP/2NfKBS0vr7eth8+cZrtn1Y39chz/wNxmiE4vfxhCCXOqvM+hDIvyVDM9PDCnZ2d/fL0kMbzUI+TnLkhlADwMslQw6RPW2p2e5TL5f2+8fNUj36juwZAT7311ltaWFjQ1NSU5ubmVCqVVKvVtL6+rkqlovX19XNVj34j5AH01OjoqN5//30tLCzsXzMmaUFvbGz07douZ6Ue/UbIA+i59DVoqEd/0ScPABEj5AEgYoQ88BLuXHUD/dOL9xshDxxjZGREu7u7g64GzpHd3V2NjIzkuk1CHjjG1atX9fjx2Z79CXF5/Pixrl69mus2CXngGOPj49rZ2dHDhw/1/Plzum7QE+6u58+f6+HDh9rZ2Wl7vfssGEIJHOPy5cu6efOm6vW6PvjgA+3t7Q26SojUyMiIrl69qps3b+ry5cu5bpuQB17i8uXLeu211/Taa68NuipAV+iuAYCIEfIAEDFCHgAi1nHIm1mxZXk2/CymyhaTx7JXEQDQrY5CPszbutqyXPHmdH+FsCxJs2a2pYNJvgEAA9BRyLt7RYeDuyApCfZaWJaku+4+GdYHAAxIpiGUfnjC7qKk++F+0qovuvtSln0AALqXy4nX0E+/6e6bkuTuS6EVfz3VhZNef9bMqmZW3d7ezqMKAIA28hpdU3b3BWk/wJNJFB/poAtnn7uvuHvJ3UsTExM5VQEA0CpzyJvZbNIlE1rtVUlJX/xkWAYADECno2umJZWSlnoI9UUz2zKzHUkKXTZvhXW2ki4cAED/dXTi1d3XJK2lliuSxtqst9JaBgDoP77xCgARI+QBIGKEPABEjJAHgIgR8gAQMUIeACJGyANAxAh5AIgYIQ8AESPkASBihDwARIyQB4CIEfIAEDFCHgAiRsgDQMQIeQCIWMchHybtTi9Pm1nZzOZfVtYL7t7LzQPA0Ot0+r+ypNXUclHanyGqYWbFdmU51vegLr3YKABEpqOQD8FdSxXdkdQI92uSyseUAQAGIGuf/Kikemr5+jFlAIABGMiJVzObNbOqmVW3t7cHUQUAOBeyhnxD0ni4Pyrp0TFlh7j7iruX3L00MTGRsQoAgOO8kvH59yWVwv2CpEq4364MANBnnY6umZZUCrdy981QXpbUcPfNdmU51xkAcEodteTdfU3SWkvZSpv1jpQBAPqPb7wCQMQIeQCIGCEPABEj5AEgYoQ8AESMkAeAiBHyABAxQh4AIkbIA0DECHkAiBghDwARI+QBIGKEPABEjJAHgIgR8gAQMUIeACJGyANAxDKFvJkVzczNbCv8LIfyxXA7m0clAQDdyTqR97i7m9QMfEmNUD4b5oGdy7h9AEAGmVry7l5JLZbcvRbu33X3yZbHAQB9lkufvJmVJb2TKiqYWdnM5o9Zf9bMqmZW3d7ezqMKAIA28jrxetvdk64auftSaMVfDx8Ah7j7iruX3L00MTGRUxUAAK3yCvlicie00qfD4iNJhZz2AQDoUOaQN7PWEK9KSvriJ8MyAGAAso6uSSQnXOXum6E1X5e05e6bOe0DANChzCEfRtTMtZStZN0uACA7vvEKABEj5AEgYoQ8AESMkAeAiBHyABAxQh4AIkbIA0DECHkAiBghDwARI+QBIGKEPABEjJAHgIgR8gAQMUIeACJGyANAxAh5AIhYHtP/LYbb2VTZtJmVzWw+6/YBAN3LoyU/a2ZbClMAmllRkty9IqmRLAMA+i+PkL/r7pMh1CXpjqRGuF+TVM5hHwCALuQR8oWWrplRSfXU49dz2AcAoAuZQ97dl0Ir/rqZnarVbmazZlY1s+r29nbWKgAAjpEp5ENYT4fFR5IKanbVjIey0VB+iLuvuHvJ3UsTExNZqgAAeImsLfmqpKQvfjIs31cz7BVuK22eBwDog1eyPNndN0Nrvi5py903JcnMSqHrppGUAQD6L1PIS82ul9OUAQD6j2+8AkDECHkAiBghDwARI+QBIGKEPABEjJAHgIgNdcj7oCsAAGfc0Ia82aBrAABn39CGPADgZIQ8AESMkAeAiBHyABAxQh4AIkbIA0DECHkAiBghDwARyxzyYWaoWTNbTJUtJo9l3T4AoHtZJ/IuS6qEmaAKYVmSZs1sS1ItawUBAN3L2pIvSEqCvaaDCbzvuvukuzOJNwAMUNaJvNNzuRYl3Q/3k1Z90d2XsuwDANC9XE68mllR0qa7b0qSuy+FVvz1VBdOev1ZM6uaWXV7ezuPKgAA2shrdE3Z3Rek/QCfDuWPdNCFs8/dV9y95O6liYmJnKoAAGiVy+iapEsmtNqrkpK++MmwDAAYgDxG1yya2ZaZ7UhS6LJ5K7Tmt5IuHABA/2U98VqRNNamfKXN6gCAPuMbrwAQMUIeACJGyANAxAh5AIgYIQ8AESPkASBihDwARIyQB4CIEfIAEDFCHgAilumyBufBixeun3+yp90914sXrk9euF64a+9F6scPPyZJ4eaQpMx1dB3fX8fblLWWAIjBG79wVdc+dbGn+zhXIf/skz39cOepflD/WD/Yeaqf/Oypdj7e1c5Hz7Xz8XM1Pt7V0909/Xx3T0+f7+nnn7zQ809eDLraACJ1f/a39JuF6z3dR9QhX//oub71vW19+/1H+s8f/kzf++kT7e4dtIZHLpjGrlzU6JVLGrtyUa+PX9GVSyP69MURferiiC5fvLB//5ULplcumEYumC6E+xesubz/YyYzk1lz++FGlhQcKjt823ys5YlttpF6CMCQ+8IvXuv5PqIL+Wef7Omb3/2J/vHfP9S336/LXfrspy/q13/5s/rj3y7ojVc/o9fHr+j1sSt69eplXbhAbAKIVzQh7+76p+/8SH/7L9/TjxpPdXP8iv7sa2/oa194VV/63GcJcwDnUhQhv/PRc/3F6n/owf/8r770uc/qr3/31/TVNyYIdgDnXk9CPswK1ZBUTKYG7JXtJ8/0+3/3b/rg4cf6xte/qD/48i3CHQCC3EPezIpSc9YoMyuYWbFXUwDu7rn+9B829IP6U/39H72pr0ze6MVuAGBo9eLLUHfUbMVLUk1SuQf7kCQ9/L9nqn5/R3/ze18i4AGgjV6E/Kikemq5J4NAk/HrX/2VCf3Ob3yuF7sAgKE3kMsamNmsmVXNrLq9vd3VNt79YEeS9Idf+XyeVQOAqPQi5BuSxsP9UUmPWldw9xV3L7l7aWJioqud/PhnTyVJb7x6tctqAkD8ehHy9yUVwv2CpEoP9rF/PZfrn7nUi80DQBRyD/lkJI2ZlSU1ejWyJnHlUhRD/QGgJ3qSkO6+0ovtAgA6w/XkASBihDwARIyQB4CIEfIAEDFCHgAiRsgDQMQIeQCIGCEPABEj5AEgYoQ8AESMkAeAiBHyABAxQh4AIkbIA0DECHkAiBghDwARI+QBIGKZQ97MZsPPYqpsMXks6/YBAN3LFPJhHtdKmO6vEJYladbMtiTVslYQANC9rC35gqQk2GthWZLuuvuku1cybh8AkEGmibxbJuwuSrof7iet+qK7L7U+L3TjzErSzZs3s1QBAPASuZx4NbOipE1335Qkd18KrfjrqS6cfe6+4u4ldy9NTEzkUQUAQBsntuSPOXlaa+mKKbv7Qmr9uruvSXqkgy4cAECfnRjyLV0yR5jZbNIlE1rtVR2ccJ2UtJy1ku2s/cmXVdv+qBebBoBo5DG6ZtHMtsxsR5JCl81bZjYtaSvpwslb6da43nrz9V5sGgCikfXEa0XSWJvyl7b+AQD9wTdeASBihDwARIyQB4CIEfIAEDFCHgAiRsgDQMQIeQCImLn7YCtgti3p+10+/YakhzlWZxhwzOcDxxy/rMf7eXc/8eJfAw/5LMys6u6lQdejnzjm84Fjjl+/jpfuGgCIGCEPABEb9pA/j9fI4ZjPB445fn053qHukwcAvNywt+QxxMxsPnV/2szKeZUB/RZmyEsv5/qe7vZ9nulSw4MUrlff0DHzyA6T1Oxbk6kZto4cX5aysybMRXBb0lLyx+HuFTMrpP9Yuinr1RwGWYS6FiQpzJp2Hl7jpI6F5PLjsR5zeD8vqzlRkvJ+T7crO+37fChb8ulfoKRG6yfoMAlvjkr4IyiET+ojx5elbBDH1aE7av7xSs1ZxcoZy86ivwzhXsj6eg7DaxzqlEwTWov9mJPjTBXl/Z7u+n0+lCGv4fnDPo2CDupfC8tRh15ohaTnCB6VVE8tX89YdqaEFui70v4k95uK/DUOFsNt4RwdcyLv93TX7/NhDfkz/4d9Wu6+kppJq6jmHLlRh56k8UFXoM/elHQ9tEiT/tSoX+MQ6rUwLWhS16iP+awa2j752IR/PzfdfdPMBl2dnmnTipeaLbQk+EclPQr3s5SdNY/Ca1sOLfuomdmomq/r25LumdmZO0/SY714T3f1Ph/WkD/uFzjMyslJV8UdegUzK6hZx/Hw4XZfUvL17oKk5EMgS9lZ8kgH/bUNNVv2Mb/GkjQr6W13b5hZTVJy0jTmY07rxXu6q/f5sIb8cb/AoWRms6kRBGVFHHqpkSWzav6xKrRwS+HYG8mogSxlZ8yamiEnNY/5XTVDP8rXuJW7r4XXu6JIjzn8d1Yys2l3X+vFe7rb9/nQfhkqvGlqSg3PGkbhRVtVs79xXNJMGCZ15PiylGGwwmtSl/Rmaphs1K9xOP9QkzSex/ENwzGfRUMb8gCAkw3r6BoAwCkQ8gAQMUIeACJGyANAxAh5AIgYIQ8AESPkMRBmttyyPB2+M3Da58+b2Xr+NcvPMNQR8SPkMSiFluXbOnyp1pO8qbN/JcJhqCMiR8ij78LFq1oDveDupw55d59x90NXcjOzxXBdnL46br/t6gj0GyGPQShL2kgWUlcs7FrYxryal2vum0HtFzgtQh6DcFuHLzBVUphUA0C+CHkMQmvXTGvonyh0kXi4vyppJzy0amYeJqtIrz9tZhvhsY30Sd6wrZ1wf9nMtsxsNMyluWpmO+F56+lumZP2m65jqqwYtpPUY7bl8cWwv9b12k0SvRUeP1I3IEHIo6+O6ZrJOvn2XTU/KCRpTs3JlKdS+5xW80qfFUkzap4POBKKZrah5n8Vy+7eCNtKb39cqW6mk/bbKtRjI+z/tpqXlF5uHWmk5uWIV9WcGHpOzZPUq6ntJMtrYX/JMY0et2+cX8N6PXkMiRBg6TAdlzTaMrSw1LK82Gb2qGOlJqaQpHqbE7j3JC2lJmVZC4E+JykpGw3PTUJbqfWTY7kraSN1zfCT9tvqnqQVd08+PCphxqR1M1ttOeaF1LX3JyXNm9lo+PBJWvVvh+VNNQMfOIKQR0+lAk3SfugvhHBKrqffs+uDh26OUTVDcr7l4XrL8pxeIjU1Y8fdIuE4R9Vsnae3WTGzhpqt8XTIp+9vtWwueeyBmd2XVDmjk6XgDCDk0W+FJOCDGUmLvdxfuJ3S0W6iQyHfriUeuljuhO1k6fNOntuutV9r3XbL70itj5nZbTV/b4uhnpX0fyFAgpBHT7V017TtqlFz3tf00zrqrjnBfqh2Mg5fkkI9C2oGaTUUbxz/jFPVo6Bm90paIbX9Uwm/n6lwjmNW0qKZzSfTSAIJQh49le6uCd0lm0mAhxOIC61dOl1q7XpJ9r8ZukMWdXCSNKnP6HEt5lC3sprTMSZ94+1ObLbdbxtJiM8p1S0U/lNITrR2LNR/yczuqPkNW+AQQh799GZLS7MsKZdru4QuDEmaS25T3Rd31RzimIxYGVUzaGs6ph/e3WvJh0PYXkMHJ2lPu9/W9ebUHE0zrubImuS/hLVO/nMJwy4XwrEkXT1FtfT3AxJDKNFfrS3hKXU4Pv4ES2p+cCwq1SUSWuJTYf/rao5yqalNaLe4q2YX02rY5nGt7bb7bRVOLt/WwZDIOTX/k5k5oR6t3lFzNM1cajtLTG6NdpjIG33RrmvGzNY5WQj0Fi159Muhrpk8rlcD4GSEPPql9dIFufXHAzgeIY9+qbeMZCko3/54AG3QJw8AEaMlDwARI+QBIGKEPABEjJAHgIgR8gAQMUIeACL2/3QI4CJdl164AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot([L(x, lam) for x, lam in zip(conv_x_da, conv_lam_da)], \n",
    "         label=\"Lagrangian\")\n",
    "plt.legend(fontsize=20)\n",
    "plt.xlabel(\"\\# iterations\", fontsize=20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAEcCAYAAADgJkIVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3Xl8VNXh///XyUJAA4QABkS2CaC4E4KIsiSQqF21Nmir1daqibvWVlI//Xz7+X4+n35/7WBbqXuCe6tiSXfbWhNIWFxJ4lIVK2RwoSoWQwSqsp7fH/dOmEz25GbW9/Px4BHmzp2558wk855zzj3nGmstIiIi/ZUS7QKIiEhiUKCIiIgnFCgiIuIJBYqIiHhCgSIiIp5QoIiIiCcUKCIi4gkFioiIeEKBIiIinkiLdgG8MmrUKDtp0qQ+Pfbf//43hx9+uLcFinGqc3JQnZNDf+rc0NCw3Vo72otyJEygTJo0ifr6+j49tq6ujoKCAm8LFONU5+SgOieH/tTZGPO2V+VQl5eIiHhCgSIiIp5QoIiIiCcUKCIi4gkFioiIeEKBIiIinlCgiIiIJ5I+UHbv2c+z7+2PdjFEROJewkxs7Ksf/O7v/OGVPXxhfgsnHpUV7eJIAtuzZw/Nzc3s2rWLAwcOROSYw4cPZ+PGjRE5VqxI5jqnpqYydOhQsrOzycjIiHg5kj5Q3v/4MwD+vScyf+CSnPbs2cM777zDiBEjmDRpEunp6RhjBvy4u3btYujQoQN+nFiSrHXOzMxk37597Ny5k3feeYcJEyZEPFSSvssr+CdtsVEthyS25uZmRowYwahRoxg0aFBEwkSSizGGQYMGMWrUKEaMGEFzc3PEy6BAOZQoIgNm165dDBs2LNrFkCQxbNgwdu3aFfHjKlDQN0UZeAcOHCA9PT3axZAkkZ6eHrFxulBJHyhBaqDIQFM3l0RKtH7XYm5Q3hiTB+QDWcAsoNxaGxi44zk/rRJFRKRfYqqFYozJAvKttZXW2qVABVA9sMd0fmpQXmTgtLS0UF5e3mZbWVlZlEoTfYn6egxIoBhjSowxRZ3cl2eMKTXGFAV/htztA0Jf5XrA5wbNgAiOoaiFIjJwmpubCQTadjT05YJ4xcXFNDY2elWsAREIBCguLsYY02lIePV6xBrPu7zcgFgOLO7gPh/gt9YWh2xbaYwJWGsD1tpGY0xxyEPygRZrbYvX5Tx0fOen8kQktlVVVVFTUxOV02F7w+fzUV1djTGG4uLi7h+QQDxroRhjfMaYCpxWRmfveBlON1aoCsAfvBE2XlIGXO5VGUUkflVXO73f4d/sY1GwFVVU1GFHTcLyLFDcFkaZtbayi91KgPD2ar27vQ1jTCnwuLW2yqsydsWqz0skZlVWVuL3O987m5qaolya7tXU1ODz+cjKSq7lnCI2KO+Og7RrvQS7s9zusOC+RUAgEmESPL1OcSISmwKBAPn5+a0fzi0tA9YD7pnq6uqka51AZE8bzoZDAdIBHxBwTxtuttY2gjPAP5DB0nq2thJFJCYFAoHWD2efz9dpl1cgEKCmpoZt27axdetWKioqWLp0KVlZWVRXV7Ny5cqIlbmmpga/309lpdNh09DQgN/vT/gWSyQDpdtX0m2lrAKyQibmBIAOA8XtFisFyMnJoa6urteF2tHsLA758iuvYD6IuWk5A2b37t19er3iWTTrPHz48KgshXHgwIGoHDfc7t272b9/f5uyHDx4sNuyvfTSS8yePbt1v4kTJ7J9+/Z2j2tpaeGJJ57gkksu4cCBA5x77rksXLiQhx56iLq6Oqqqqnj33Xcj8oH+0ksvAfD+++9z5ZVXArBt2zYuueQSHn74YaDvr0dnOnqfP/vss4j/vsfUJ6g7ID+iF/tXApUA+fn5tqCgoNfHfGjLC7D9X5xwwgkUTM/p9ePjVV1dHX15veJZNOu8cePGqKyAGysr72ZmZpKWltamLCkpKd2Wbdy4cW32GTlyJC+++GK7xz322GNcd911gFPnjz/+mKKiIsaPH89FF13E6aefzvjx4z2sUeeeffZZfD4ft956a+u26dOn88Mf/rC13H19PTrT0fs8ePBgZsyY0afn66uIB4oxJmsgTwPuLS2HIdH23396jdff2zkgz33gwAFSU1P7/TzHHjmM//rScR6UqOcqKytpaGhosy0QCHQ4hnLeeee1ud3Y2Ng6iA9OV1lXenPmWHZ2dpctnQ0bNrQbP+nLmWmNjY3U19fT0tLChg0b8Pv93dYj2iIZKMFXNBto/Y0ImbQY1XMBdZKXSOwIBAKcd955lJaWttleWVlJWVkZLS0tbT7UQ/9fW1sL9PyU3UAg0G7WeldmzZrFkiVLOr0/OH4SasOGDeTl5fX4GC0tLdTX17fWv6amhuLi4pg/wy1igWKtbTHGBGg/lpKNM3kxKoGi1esl2gbym3+sdHn1ViAQ6PDbeHZ2duv9nX1A19bW9urD2+fzeTZg39LSQktLC/n5+W22V1VVtQuZrgQCAfx+f2ug5Ofnt7bOYnlgP9JredXgzH4Pleduj4pDi0MqUkRiQVVVVbsP5KDgh2n4bPnQLqW6uro2j29paaGmJjIfMcHyhQZaVVUVWVlZXbZqwuXl5bVO5ARnWZasrKyYDhMYuEDJpuOzusppvyRLGW3X74owzUMRiRWNjY1UV1d3+sEZbLWEBkhNTQ25ubkEAoHWGeqhj6+srIzonJC8vLzW8gUXgVy1alWvnye0hVZRUcHy5cs9K+NA8azLyx0LuRlnPkkW4HfX5aoOziNxu73KjTF+YIO7rz9a3V1OuZ2faqCIRE8gEKCsrIyamhqysrIoLi6mpKTtAhrl5eWtLY3y8nIaGhqoqKjA5/NRWlraet8f//hHfvSjH7XOAQkfhxloK1euxO/3M3PmTJqamqiuru7XYHplZSXnn39+u9cjFnkWKO6ZW922NNwJizGzXKjO8RKJvuCCil3pbAzC5/NRUXFoicBdu3a1uR1p4eXpj+ASLvEy6z6mrocSXWqiiEjsaGxsJDs7uzVMqqoisqxhv8TUxMZoUJeXiMSaQCDAokWL2sy58fl8Md/tpUDRoLyIxBifz8eOHTuiXYxeS/ouL7VQRAZednZ2u4HpWJ/1PZAS9fVQoOia8iIDLisrq92geiRX/401ifp6KFB0npeIiCeSPlCC1OUlItI/CpTWLi8REemPpA+UscMGAzAoNelfChGRfkn6T9H500YDMCpzUJRLIolOC5BKpETrdy3pAyUjzXkJ9u4/GOWSSCJLTU1l37590S6GJIl9+/Z5cmG13kr6QBkyyHnRP9l7IMolkUQ2dOhQdu4cmKsyioTbuXNnVK6Do0BJdwLl030KFBk42dnZ7Nixg+3bt7N37151f4nnrLXs3buX7du3s2PHjtaLkUVS0i+9EmyhKFBkIGVkZDBhwgSam5t56623OHAgMr9vn332GYMHD47IsWJFMtc5NTWVoUOHMmHCBDIyMiJeDgWK20L5TIEiAywjI4OxY8cyduzYiB2zrq6OGTNmROx4sUB1jh51eWkMRUTEE0kfKIPT3C4vBYqISL8kfaCkpBgGpcAne/dHuygiInEt5sZQjDE+oATnMsF5QKV7eeEBkz3E8E7zJwN5CBGRhBdzgQKstNbOBDDG1APLgcUDeUDf8FSeafqIXZ/tY+jg9IE8lIhIwhqQLi9jTIkxpqiT+/KMMaXGmKLgz9D7gObgbbdl0uHzeOmMiWns3rOfn1e/OdCHEhFJWJ4HihsQyzu5zwf4rbWV1toaa20lUOZuB/AB4d1bzW7QDJhJw1O56NSJPPjMWzz12gcDeSgRkYTlWaAYY3zGmAqcUGjuZLcyoCJsWwUQvHRZ5Kd2um7+3HROHDec61a8yOo3tkWrGCIiccuzQLHWBqy1ZW6rozPBwfZQ9e52cIIoK+z+iITMkEGp3PetWUw5IpPLH27gnjVNHDyo5TFERHoqYqcNG2Oy6KD1EjyDy+32CtBBgFhrw0NoQIzKzGBF6RyKp+fwk7++wUX3P8+7OvtLRKRHIjkPJRsOBUgHfOHB4YZMzUAXLFRmRhp3fyOPn5x7Ai++00LxrWu4q24z+w5oeXsRka6YgVj11BjTBJRZa2tCtuUBDdZa08H+Fii21ta4+xXRg3koxphSoBQgJydn5ooVK/pU3t27d5OZmdlu+0efHuTRN/bSsO0A4zINF07P4NiRkb/GwEDorM6JTHVODqpz7xQWFjZYa/M9KYi11vN/QBNQFLYtzzlch/vb8P17+2/mzJm2r2pra7u8v+b1D+zpP1llJ5Y/YS998AXb9OGuPh8rVnRX50SkOicH1bl3gHrr0Wd/xJdeccdS4sqi6TnU3LiA8rOO4blAM2fcupb//tNrtHyyN9pFExGJGZEMlID7s82ge0jABIhhg9NTubIgl9rvFbA4fzwPPfMWC26p4951AS19LyJCBAPFOuMgATo+LbjFWhvTgRI0emgGPz73BP5y/TxOPGo4P/rzRhb+tI5fb3iX/Rq4F5EkFukurxogfPAnjwifyeWFY8YM45eXzuaRy2YzemgGS37zCmcuW8uTr76vy7uKSFIaqEDJpn1LBKCc9gs9lrnb49LpU0bx+6tP555vzATgil81cs5dz/DM5u1RLpmISGR5ttqwOxZyM87kxSzAb4wpBqqttVXgdHsZY8qNMX5gg7uvP166uzpjjOGs48dQNP0IfvviP1lW/SYX3Ps886aO4rtnHM3J4+PuPAQRkV7zLFDcMZJuWxrWmbwYkZnvkZaWmsJ5+eP58klH8sjz73Bn7WbOufNpFh1zBN8pnsbx44ZHu4giIgMm6a/YOBAGp6dy6dzJrF1SyE1nHk392zv44u3rKftlPRvf3xnt4omIDAgFygDKzEjj6sIprCsv5DtF03im6SM+94t1XP1II29u2xXt4omIeEqBEgHDBqdzfdFU1i9ZyHULp7DmzX9x5rK1XPfYizT9a3e0iyci4gkFSgQNPyydG884mnVLCrliQS7Vr2+j+OdruPHXL/HW9n9Hu3giIv2iQImCEYcPovysY1hXXsilcyfz51feZ9HP11Be9YqWyxeRuKVAiaJRmRn84AvHsm5JIRedOpHfvfRPCn9ax82//TtbdyhYRCS+KFBiwBHDBvN/v3wca24q4OunTOA3DVsVLCISdxQoMWTs8CH87znHU3dTAV+b5QRLwS0KFhGJDwqUGHRklhMsa5YUcMHs0GDRGIuIxC4FSgwbO3wI/3N2aLAEx1gULCISexQocUDBIiLxQIESR0KD5cKQYPn+bxQsIhJ9CpQ4NHb4EP47JFh+26hgEZHoU6DEMQWLiMQSBUoCCAbL2iWFbYJFM+9FJJIUKAlkzPDBrcHyjZCZ9woWEYkEBUoCGjPcmXm/9qb2wfLORwoWERkYnl2xUWJPMFiuWJDLPWuaePSFd6hq3MpX88aRf9jBaBdPRBJMzAWKMSYPyMe5Lv0soDzerzkfbcFgubIgl7vr3GA5cJD6T17mmsKpTBh5WLSLKCIJIKa6vIwxWUC+tbbSWrsUqACqo1yshJHjLkK5bkkhiyak8fuX3qPwZ3UsqXpZXWEi0m89DhRjTIkxpqiT+/KMMaXGmKLgzz6WxweUh9yuB3xu0IhHcoYN5sLpGa3L5geD5aaVL/P2R7rQl4j0TY+6vNyAWA4s7uA+H+C31haHbFtpjAn0tqvKWttojCkO2ZQPtFhrW3rzPNIzwRZLaFfYb1/8J+fOGMc1C6cwceTh0S6iiMSRLlsoxhifMaYCp+XQ3MluZThdU6EqAH9fChQWQmXA5X15Hum50K6wi+dM5A8vv8fCn61Ri0VEeqXLFor74V4GYIwp72S3EtoHSj0hYx/GmFIgt4tDVVtra0I3uI953Fpb1VUZxTs5wwbzX18KOSvseafF8pUZ47imcAqTRqnFIiKd69dZXu7YRrvWi7W2xRiDMcZnrQ1Yayt7+bxFQCA8ZCQygsFy5YJc7lkT4JHn3+Z3L/6Tc04ex7ULFSwi0rH+nuWVDU6AdHK/r7dP6J423BwME2NMSd+LJ/1xxLDB/PBLzjXvv3XaJJ545T0W/qyOG3/9Elu2qytMRNoy1tqe7WhME1AW2mpwP/wbrLWmg/0tUNybVoY7wN+AMwclKGCt7bC7zO0WKwXIycmZuWLFip4eqo3du3eTmZnZp8fGq77UuWXPQf66ZR+17+xn30GYc2QaX85NZ8zhMXX2eaf0PicH1bl3CgsLG6y1+V6UI6YmNrpjNiN6sX8lUAmQn59vCwoK+nTcuro6+vrYeNXXOp8DfLjrMyrXBPjV82/z3PufcvbJzllhuaNj+49Y73NyUJ2jx5OvlponklyOGDqY//zisaxbspBL507mr6++T/HP13DDihfZ/OHuaBdPRKKkv4ESPMU3O3RjSMBoyZQENnpoBj/4ghMsl83z8bfXtlF86xquV7CIJKV+BYo7GB+g7ZgHOAHTojW4ksPooRn8x+ens668kNJ5Pp5yg+W6x15k84e7ol08EYkQL7q8anBmtIfKc7dLEhmVmcHNn5/O+vJCSuf7qNm4jeJb13LtYy+yaZuCRSTR9SZQsmnfEgFn7a3wJVnKaLsmlySRkZkZ3Py56axbUkjZ/FxWbdzGGcvWcs2jjbypYBFJWF2e5eWOhdyMM58kC/C7a21VB2ewu5MYy40xfmCDu69f3V0yMjOD73/uGErn+1i+LsBDz7zFn//+Pp8/YSzXL5rKtJyh0S6iiHiou6VXWuhBS8Na2wg0elUoSSzZhw+i/KxjuHyej3vdYPnL39/n88eP5bpFUzl6jIJFJBHEx4w0SQjZhw9iyVnHsL58IVcV5FL3jw85c9larnqkgTc+2Bnt4olIPylQJOJGHD6Im850guWawimsfXM7Zy1bp2ARiXMKFImaEYcP4ntnHs368kKuXXgoWK78VQMb31ewiMSbmFp6RZJT1mGD+O4ZR3Pp3Mncv34LDzz9Fn999QPOOm4M1y2ayrFHDot2EUWkBxQoEjOyDhvEjWcczaVzfdz39BYeWL+FJ1/7gDOPy+G6RVM57sjh0S6iiHRBgSIxZ/hh6dxYPI1LT5/M/U9v4f71W/jba9s441gnWI4fp2ARiUUKFIlZww9L5zvF0/h2MFie3sJTr2+j+NgcrlewiMQcBYrEvNZgmTuZB57ewn3rt1D9+jaKpudwQ5GCRSRW6CwviRvDh6RzQ9E01pcv5DtF03hhy0d88fb1XPbQBv6+9eNoF08k6SlQJO4MH5LO9UVTWf/9hdxYPI0XtjTzpTvWc+mDG3hla2dXoxaRgaZAkbg1bHA61y1yguW7xdOof3sHX77jab794AZeflfBIhJpChSJe8MGp3PtoqmsLy/ke2dMo/GdHZx959Nc8sALvKRgEYkYDcpLwhg6OJ1rFk7lm6dN4uFn32b5ugDn3Pk0BUeP5vpFU6NdPJGEp0CRhDN0cDpXF07hm6dN4qFn3mL5ugBfuesZThiVyjDfDvImjIh2EUUSkrq8JGFlZqRxdeEU1pcvZMlZR7Pl4wOce9czXHz/CzS8vSPaxRNJOGqhSMLLzEjjqoIpTN7/Lm+lTaBybRNfvfsZ5k0dxQ1F05g5US0WES+ohSJJY0ia4cqCXNaXL+T7nzuG197byVfvfoaL7nuehrebo108kbinQJGkc3hGGlcsyGXdkkJubg2WZ7novuepf0vBItJXMR0oxpiKaJdBEtfhGWmULchlfXkh//H5Y3j9vZ2U3PMs37j3eTYoWER6LWYDxRhTBJRGuxyS+A4blEbp/FzWlRfyg89P540PdrL4nme58N7neGGLgkWkp3o8KG+MKQFarLU1HdyXB+QDAcAHBDrarxfHygKaAc1Kk4g5bFAal8/38Y1TJ/LI829zz5omzqt4ltNyR3L9oqnM9o2MdhFFYlqPWihua2F5J/f5AL+1ttJaW2OtrQTK3O19VWStbezH40X6bMigVC6b52PdkoX85xem8+a23Zxf+Rxfr3yO5wIfRbt4IjGry0AxxvjccQwfTouhI2VA+FhHBeDvS4Hc8Opz60bEK4eCpZD//MJ0Nn24m69VPsfXKp/l2SYFi0i4Lru8rLUBnMDAGFPeyW4ltA+UeqA6eMMYUwrkdnGoamttTbCry1qrri6JGcFguXD2RB594R3uWdPE15c/x+zJ2dxQNI05ueoKE4F+Tmx0A6Bd68Va22KMwRjjs9YG3G6wnigCso0x+e7tLDeMatxwE4maIYNSuXTuZC6cPYFHn28bLNcXTWWObyTGmGgXUyRq+jtTPhucAOnkfh/OQH2PWGurQm8bYyp6EUYiETE4PZVvz53MBbMn8NgL73B3XRMXLH+eUyZnc8OiqczJVbBIcjLW2p7taEwTUBZ69pZ7dleDtbbdX48xxgLFfTnby235lOKMw5QDVR21UNzWSylATk7OzBUrVvT2UADs3r2bzMzMPj02XqnO3tl7wLJm637+HNhHyx7LtBEpnDNlENOzU6IeLHqfk0N/6lxYWNhgrc3vfs/uxWSg9EV+fr6tr6/v02Pr6uooKCjwtkAxTnX23mf7DvD4hne5q24z23buYdakEVy/aBqnT4lei0Xvc3LoT52NMZ4FiicTG90WhUhSG5yeyjdPm8Samwr5n7OP493mT/nGfc+z+J5nWbfpX/T0y5tIvOpvoAS7obJDN4YEjAbSJekMTk/l4jmTqLupgP89+zi27viUi+57gZJ7nmXtmwoWSVz9ChR3MD4AhLdQsnFm1StQJGkNTk/lojmTWLOkgP8953jea/mUi+9/ga/e/YyCRRKSF11eNTjLroTKQ5MTRQDISEvlolMnUndTAT8653g++PgzLr7/Bc69+xnWKFgkgfQmULJp3xIB5yysxWHbytztIuLKSEvlG6dOpPamAv7fV45n28ef8c37X+Ardz1D3T8+VLBI3OtyHoo7FnIzznySLMBvjCnGmdleBa2TGMuNMX5gg7uvX91dIh3LSEvlwtkTWTxzPFUNW7mzdjPfemADJ43P4oaiqRRMGx31041F+qK7pVda6EFLw13IUYs5ivTCoLQULpg9gZKZR/Gbxq3csXozlzywgZOOGs4NRdMoOFrBIvElZq+HIpIsBqWl8PVTJlD7vQJ+fO4JbN+9l0se3MDZdz7N6je2qStM4oYCRSRGhAbLT849geZ/7+XbD9Zz9p1Ps2qjgkVinwJFJMYMSkvha26w+L96Ajs+2culD9Xz5TuepuZ1BYvELgWKSIxKT03h/FkTWP3dApZ+9UQ+/nQflz1cz5fuWE+1gkVikAJFJMalp6Zw3qzxrPruApaWnMjOT/dz+cP1fPH29Tz12gcKFokZChSROJGemsJ5+U6w3FJyIrv37Kf0lw188fb1/E3BIjFAgSISZ9JTU1icP55VNy7gp4tPYvee/ZT9soEv3LaeJ1/9gIMHFSwSHQoUkTiVlppCycyjWHXjAn62+CQ+2bufK37VwBduX8+Tr76vYJGI6+8VG0UkytJSU/jqzKM4++Qj+ePL73H76s1c8atGjhkzlEVj9jP/oCUlRRMkZeCphSKSINJSUzg37yiqvzOfW88/ib37D3LnS3v4/G3r+Mvf1WKRgadAEUkwaakpfGXGUVTfuIDSEzPYu/8gVz3SyOd+oWCRgaVAEUlQqSmG045Mo/rGBfziayez7+ChYPnzKwoW8Z4CRSTBpaYYzj55HNXfcYJl/8GDXP1oI2f9Yi1PvPKegkU8o0ARSRLBYHnqOwu47eszOGjhmkdf5Mxla/nTy+9xQMEi/aRAEUkyqSmGL590JH+7YT63f30GFrj2MSdY/qhgkX5QoIgkqdQUw5dCgsUA1z32ImfcuoY/vPRPBYv0mgJFJMmFBssdF8wgNcVw/YqXFCzSawoUEQEgJcXwxROP5Mnr53PnBXmkpaRw/YqXKL51Db9/UcEi3YvJmfLGmJLQ28Hr14vIwEtJMXzhxLF87vgxPPnaB9y2ahM3PP4St63exLULp/ClE48kLVXfRaW9mPutMMYsgdYQqQFujm6JRJJTSorh8yeM5S/XzePuC/MYlJrCdx5/mTNuXctvG7ey/8DBaBdRYkyPWyhuq6HFWlvTwX15QD4QAHxAoKP9enCMLOBma+0IAGttCzCzt88jIt5JSTF87oSxnHncGJ56/QOW1Wzixl+/zO2rN3NN4RTOPlktFnH06LfAGFMELO/kPh/gt9ZWWmtrrLWVQJm7vbfygYAxpsQYU2SMWdLH5xERj6WkGM463mmx3PONmQxOT+W7K1+m6OdrqGpQi0W6CRRjjM8YU4HT6mjuZLcyoCJsWwXg70N5fEAeUOO2cCqB6j48j4gMECdYxvDna+dScdFMDhuUxvdWvsyin69hZf27CpYk1mWXl7U2gBMYGGPKO9mthPaBUk9IEBhjSoHcLg5V7QZIAKe7rMU9fosbaj63LCISI1JSDGceN4Yzjs2h+vVtLKvZxE1Vr3BH7WauLpzCV2aMI11dYUmlX2d5uWMe7VovbhAQDAK3G6wnOgqNlv6UUUQGljGGM44bQ/GxOdRs/JBlNW+ypOoV7li9mWsWKliSSX/f5WxoHTzvSK/GP9xWSIsbVMHACqh1IhL7jDEUH5vDE9fO5d6L8xk2JI0lVa+w8Gd1/HrDu+xTV1jCM9b2bLKSMaYJKAs9e8s9u6vBWtvucnDGGAsU9/ZsL3cQvgxowukmq+gsUNyutFKAnJycmStWrOjNoVrt3r2bzMzMPj02XqnOySGadbbW8vK/DvD7zft4a+dBRg8xfDE3ndOPTCNtAK8gqfe5dwoLCxustflelCPmAqWv8vPzbX19fZ8eW1dXR0FBgbcFinGqc3KIhTpba6n9x4csq9nEK1s/5qgRQ7imcArn5h3FoDTvu8Jioc6R1p86G2M8CxRP3s1gF5WISDhjDAuPyeEPV5/OA9+axcjDB/H93/6dwp/W8dgL77B3v7rCEkV/AyXYFZUdujEkYDT2ISKAEyyFxxzB791gGTU0g5vdYHn0eQVLIuhXoLiD8QEgvIWSjTOrXoEiIm20BstVp/HAJbMYPTSD//idEyyPPP+2giWOedHlVYMzwz1UnrtJf14xAAAQ4UlEQVRdRKRDxhgKjz6C3111Gg+6wfKD371KwS21/Oq5t9mz/0C0iyi91JtAyaZ9SwSgHFgctq3M3S4i0iVjDAVusDz07VPIGT6Y//z9qxTcUscvFSxxpcuJjcHFGnHmk2QBfmNMMc7M9iponcRYbozxAxvcff3q7hKR3jDGsGDaaOZPHcW6TdtZVvMm/+f3r3JX7WauKsjlvFnjyUhLjXYxpQvdLb3SQg9aGtbaRqDRq0KJSPIyxjB/2mjmTR3F+s3bWVazif/zh9e4q66JKwtyOS9/PIPTFSyxSOshiEhMMsYwb+poqq6Yw68unc24rCH88A+vUXBLHQ898xaf7VNXWKxRoIhITDPGMHfqKFZeMYdHLpvN+Owh/NcfX2PBLbU8+PQWBUsMUaCISFwwxnD6lFH8umwOj142m4nZh/N///Q6C26p5QEFS0yIyWvKi4h0xhjDaVNGMSd3JM8GPmJZzSb++0+vc3ddE1csyOWoAz1bTkq8p0ARkbhkjOG03FGcljuKZ5s+YlnNm/zPE68zPMOwNWMLF8yeoMH7CFOXl4jEvTm5I3m8bA6PXX4qYw83/M8TrzNvaS33rVdXWCQpUEQkYczJHcn3TxnCitJTmTI6k/994nXm+mu5d12AT/cqWAaaAkVEEs6pvpE8Vnoqj5eeyrScTH70543MW6pgGWgKFBFJWLN9I3n08lP5ddkcjh4TDJbVLF8b4JO9+6NdvISjQBGRhHfK5GweuSwYLEP5f3/ZyDx/LZVrmxQsHlKgiEjSCAbLyivmMH3sMP6/v7zBPH8tFWsULF5QoIhI0pk1KZtfXTabqivmcOyRw/jxX99grr+We9Y08e89Cpa+UqCISNLKn5TNLy+dzW+unMNxRw7jJ399g3lLa7m7TsHSFwoUEUl6MycGg+U0jh83HP+TbzDXv5q76jazW8HSYwoUERHXzIkjePjbp/Dbq07jxKOyWPrkP5jnX82dtQqWnlCgiIiEyZswgoe+fQq/u+o0ThqfxS1/+wdz3WDZ9dm+aBcvZilQREQ6MWPCCB685BR+f/XpzHCDZd7SWu5YvUnB0gEFiohIN04en8UDbrDkTRjBT596k7n+Wm5fpWAJFXOrDRtj8oB892YWUKXr04tILDh5fBb3f2sWL7/bwm2rNvGz6je5d/0WLps7mW+ePolhg9OjXcSoirlAAYqstUuDN4wxFUBZFMsjItLGSeOzuO9bs3hl66FgWb4uwGXzfHwriYOlx11expgSY0xRJ/flGWNKjTFFwZ/9KFOZMSarH48XEYmIE4/K4t5vzuJP18zllMkj+Xn1m8z9yWp+UbOJjz9Nvq6wHrVQ3IBYDizu4D4f4LfWFodsW2mMCfSxq8oPbDHGlLu3y7vaWUQk2k44ajj3fjOfV//5Mb9YtYlba97k3vUBLp07mUtOn8zwIcnRYumyhWKM8bldTj6guZPdyoCKsG0VOMHQa9baSuDH7vOWAdl9eR4RkUg7ftxwll+czxPXzmWObyTLajYx17+aW6vfTIoWS5ctFLeFUQYQ0mIIV0L7QKkHqoM3jDGlQG4Xh6q21ta4+y5xx1CWuo+r7uaxIiIx5fhxw6m82Gmx3LZqE79YtYn712/hkrmTufT0yQw/LDFbLP0alHfHOtq1Xqy1LcYYjDE+a23AbXX05PmKgMaQ56k0xuQaY/KstY1dPFREJOYEg+W195xguW3VJh5Yv4VLTp/Et+dOJuuwQdEuoqf6Ow8lG5wA6eR+Xy+frxnIC9+oMBGReHbckcOpuCifv1w3j9OnjOK21ZuZ66/lZ0/9g5ZP9ka7eJ7p72nDnp6NZa1tdMdtSkOe/3EvjyEiEi3HHjmMey6aycb3d3Lbqk3cvnozDzz9Ft86bRKXzYv/Foux1vZsR2OagLLgWIe7LQ9osNaaDva3QHHo/l5zg6cUICcnZ+aKFSv69Dy7d+8mMzPTy6LFPNU5OajOse3dXQf5w+a91G87wOBUKJqYzlmT0skc1O4jtUv9qXNhYWGDtTa/+z2758nERmNMVhfdXgPGHZupBMjPz7cFBQV9ep66ujr6+th4pTonB9U59l0EvPHBTm5ftZk/v/o+tVsP8s3TJnHZPB/Zh/esxRIrde7vGEpwnkmbU3tDJiZqyRQRkW4cM2YYd16Yx5PXz6fgmCO4e00T8/yr8T/5Bs3/jp8xln4FitsqCdB+LCUbaNEaXCIiPXf0mKHceUEef7thPoXHHME9a5qY61/NT/76Bh/t3hPt4nXLi9WGazi0mGNQnrtdRER6aVrOUO64II+nbpjPouk5VKxtYt7SWn78140xHSy9CZRsOj6rq5z2S7KUoSVTRET6ZWrOUG7/+gyeumE+RdNzqFwbYK6/lh//ZSPbYzBYuhyUd8dCbsaZT5IF+I0xxTgz26ugdRJjuTHGD2xw9/Wru0tExBtTc4Zy29dncN2iKdy+ejOV6wI8/OzbXDxnIpfP7+10v4HT3dIrLfSgpeFOPNTkQxGRATTliKH84mszuHbhVO5YvYnlbrAsnppKQbQLh67YKCISd6Yckcmyr82g+sYFnHX8GEYP6d28lYGiQBERiVO5ozO59fyTOWF0bFwrUYEiIiKeUKCIiIgnFCgiIuIJBYqIiHhCgSIiIp5QoIiIiCcUKCIi4gkFioiIeKLHV2yMdcaYfwFv9/Hho4DtHhYnHqjOyUF1Tg79qfNEa+1oLwqRMIHSH8aYeq8ugRkvVOfkoDonh1ips7q8RETEEwoUERHxhALFURntAkSB6pwcVOfkEBN11hiKiIh4Qi0UERHxhAJFREQ8ERtXZYkSY0wekA8EAB8QsNbWRLdU7RljSnDKl+v+rLDWVoXt021dIrmPl4wxWYDfWls2EGWNpTq7xzkf+AgYifNeB6JRn0jU2RhTBOS5N0cCTdbayrB94rrO7t9vS0fPE2t163f9rbVJ+c99sarDtq0EfNEuW1iZSoC8kNtZQBNQ2pu6RHKfAXgNKoCVvX3/4q3O7ntdEV73RK0zTpAUdfAaJMzvNlAE7AivZyzWzYv6e/bHEG//AD9Q0sGbvzIa5eminEs62FYK2N7UJZL7eFx/n/tLHR4oCVVnnC8KO8K2leB8Y0/UOld0sj0q9fGyzu7vbYX7t9pEx4ESU3Xzov6e/GLE4z/3TfaFbcsi5IM62v/c8jQAWWHbfYANlr8ndYnkPh6/BqXuv/BASag6u3/M/g62+3pTjjirc0P4Mdzt1b0pR6zXmc4DJabq5kX9PfnFiLd/wReJsA9q9z4b/qJGuaw7COnycre1BkpP6hLJfTyue5F7zDaBkoh1dt/nki7uT8Q6+90PsdAu3dYur0SpMx0ESqzVzav6J+ugfDaAtbalk/t9OINSUWetHdHB5iKcQb6AMcbn7tdVXYjgPl6+blnW2hZjTPj2nrx/RHAfL+qcBQSMMaVAM04dm+2hky8Srs7W2nJ3UL7BGFMefE57aFA+4eocItbq1pN9uq1/sgZKVrQL0E9lwI/d//ekLpHcxxPGmBIbdiZbL8sRN3UOfinA+RZYGbLdb4zJdrclVJ2DrLUzjTHVOK2VRmBRL8sRd3XuxfFibZ9uaR5KnAl+g7XWLo12WQaKe5pwZ9+UElFnf8yP43zQJiz397kcKMb5FtwQErASZ5I6UNwPrrjh/qGVWWuLO7iv27pEcp9+Os/24Nz3BKpzIOwnANbaRiAr9AM2geqMMaYCqLHWNrrv92Sc16Cht+WIlzr35Xixtk9XkjVQgn+42aEbQ17MmBg/6YCftl0C0LO6RHKffnEnVtV3s1tC1Tmk37on/dcJUefgc9mQSZvW2hb3y1LAHVtJqDqHibW6eVL/pBxDcQd6A7TvasjGHeyOQrG65H6bKw8fNOtpXSK5Tz/lA7nGmPNDtuUBPmOMH9hgra1KsDpDyMzkju5L0Pe5qZP7Kjh0QkYi1blVLNbNi/onawsFoAbnlzpUnrs9prj9zH7bdgmOopCukJ7UJZL79Jm1ttJaWx76D6gGGt3bwYH6hKmzq4JDS5AAra210D/mRKpzAGcpoY5k4QzQ97Qc8VLncLFWt/7Xv6/nVsf7P5xf2vBlBqqJoTkobplKcOZh5IX8K6Ltkhzd1iWS+wzAa+Cn/cTGhKqze4ymDo5R4nVZY6jOFbSfn+EjZIJnItSZTuYYxVrdvKh/Ul8PJWQhvg04v8jBwcGY4PZf7ujk7oC1Njdk327rEsl9vOC2wMqB83B+2StxgrQx0vWJRJ1D6tuE8+19ZTTrE6E6l+LU9SN3U4vteHHIuKqz+7d7s/scJTgtshqcD+yqkP1iqm79rX9SB4qIiHgnmcdQRETEQwoUERHxhAJFREQ8oUARERFPKFBERMQTChQREfGEAkVERDyhQBEREU8oUCSuuYtmht4ucVeq7enjl7gXeIpZ8VBGEVCgSPwLvxhTMb1banwWztposSweyiiiQJH45a6XFB4ePtuLpcattYuttW0uWu9eejfiVw3s7LgdlVEkFilQJJ4VEXJ1Py8uHew+xxLClpIfaNE6roiXFCgSz4ppe62GfJxVUkUkChQoEs/Cu7fCA6ZbbjeTdf+/kkOXC1hpjLHGmB1h+5cYYxrc+xpCTwBwn2uH+/8KY0yTMSbLGOMzxqw0xuxwH1dt2l4nvsvjhpYxZFue+zzBcpR2UK8dHewXfhGvErectqOyifSGAkXiUifdW3nBa6X00eU4oQRQhnOdjpkhxywBVuKE1mKc8Zt2H8DGmAac1lKFdS7ZXBb2/NmEdNV1d9xwbjka3OMXA48DFeFnvOFcQ2YlzoWsynBOYFgZ8jzB21Xu8YJ1Cr8MrEiPJOU15SX+uB+WoR/c2UBW2Om0+WG3/b25OJA9dJ1vgOYOBveXA0utc1ligCo3PMpwLowFzodxs7U2GBCE7B+sy+VAgzGmxFpb1YPjhlsOVFprg0FVY4xpxAm38ItytV422RiTCywxxmS5QRdsrfzYvd2IEy4ifaJAkbgQ8uEJtAZMuftBiNv15Au/2p9X3K6iLJwP5CVhdzeH3S6jC9baRmMMtD/luSflKHLL0aY1Yq2tMca04LQywq8THtQU9nTB+1YZYx4HavrZwpMkp0CReOULholrMc615wfseO7PmbTvamsTKB21MNxuqvPd5+nPGEXwsR21YgLhzx32GhF+nzGmGOd187vlrAltXYn0hgJF4kJYl1eH3V2Az/3mH9SrLq9utH6A92aeC4BbTh/Oh3a9u7mh80f0qBw+nC6qUL6Q5+8R9/WZ6Y5JlQJ+Y8wSa+3SPpZPkpgCReJCaJeX2+XUGAwLd3C5PLxbrI/Cu6+Cx290u5T8HBpAD5Ynq7OWgFu2ImBxyFhGR4PeHR63A8HAKCOka81tAQUH4XvNLf9SY8z5ODPzRXpNgSLxaFbYN+giwJO1rtxuIICy4M+QLqDLcU7rDZ45lYXzoR6gk3ETa20gGETu87VwaAC/p8cN368M56yubJwzvIKtn6retMjcU43L3boEu8vyCBufEekpnTYs8Sj8G/5Mejn/pBtLcULKT0i3ktvCmOkevxrnbKsAHQREmMtxuulWus/ZWSuiw+OGc088KObQacBlOC20xd2UI9yvcc7qKgt5nqUDdWKDJD5jre1+L5EY0VH3ljGmWgPJItGnForEmzbdW16s3yUi3lCgSLwJX17Fs/ETEekfBYrEm+awM6p8eDt+IiJ9pDEUERHxhFooIiLiCQWKiIh4QoEiIiKeUKCIiIgnFCgiIuIJBYqIiHji/wcvs66n0dx2ZAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.semilogy([np.linalg.norm(A @ x - b) for x in conv_x_da], label=\"$\\|Ax - b\\|_2$\")\n",
    "plt.legend(fontsize=20)\n",
    "plt.xlabel(\"\\# iterations\", fontsize=20)\n",
    "plt.xticks(fontsize=20)\n",
    "plt.yticks(fontsize=20)\n",
    "plt.grid(True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Метод модифицированной функции Лагранжа\n",
    "**Мотивация:** для метода штрафных функций решения подзадач могут нарушать ограничения, и известно только, что\n",
    "\n",
    "$$\n",
    "g_i(x^*_k) \\approx \\frac{\\lambda^*_i}{\\mu_k} \\to 0, \\quad \\mu_k \\to \\infty\n",
    "$$\n",
    "\n",
    "Можно ли изменить $Q(x, \\mu)$ так, чтобы избежать этого нарушения ограничений?\n",
    "\n",
    "**Идея:** добавлять штраф не к целевой функции, а к функции Лагранжа. Аналог прямо-двойственного метода, так как за одну итерацию происходит обновление как прямых, так и двойственных переменных"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Задача с ограничениями-равенствами\n",
    "\n",
    "\\begin{equation*}\n",
    "\\begin{split}\n",
    "& \\min f(x)\\\\\n",
    "\\text{s.t. } & g_i(x) = 0, \\; i=1,\\ldots,m\n",
    "\\end{split}\n",
    "\\end{equation*}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Модифицированная функция Лагранжа\n",
    "\n",
    "$$\n",
    "M(x, \\lambda, \\mu) = f(x) + \\sum\\limits_{i=1}^m\\lambda_i g_i(x) + \\frac{\\mu}{2}\\sum\\limits_{i=1}^mg^2_i(x)\n",
    "$$\n",
    "\n",
    "Необходимое условие минимума $M(x_k, \\lambda^k, \\mu_k)$\n",
    "\n",
    "$$\n",
    "f'(x_k) + \\sum\\limits_{i=1}^m (\\lambda^k_i + \\mu_k g_i(x_k) ) g'_i(x_k) \\approx 0\n",
    "$$\n",
    "\n",
    "Отсюда получаем выражение для $\\lambda^{k+1}$\n",
    "\n",
    "$$\n",
    "\\lambda^{k+1}_i = \\lambda^k_i + \\mu_k g_i(x_k)\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Нарушение ограничений\n",
    "\n",
    "$$\n",
    "g_i(x_k) \\approx \\frac{\\lambda^*_i - \\lambda^k_i}{\\mu_k} \\to 0\n",
    "$$\n",
    "- В методе штрафных функций сходимость к 0 была порядка $1/\\mu_k$\n",
    "- В методе модифицированной функции Лагранжа сходимость к 0 быстрее, поскольку не только знаменатель возрастает, но и числитель стремится к 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Задача с ограничениями типа равенства и неравенства\n",
    "\\begin{equation*}\n",
    "\\begin{split}\n",
    "& \\min_x f(x)\\\\\n",
    "\\text{s.t. } & g_i(x) = 0, \\; i=1,\\ldots,m \\\\\n",
    "& h_j(x) \\leq 0, \\; j = 1,\\ldots,p\n",
    "\\end{split}\n",
    "\\end{equation*}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Модифицированная функция Лагранжа\n",
    "\n",
    "$$\n",
    "M(x, \\mu, \\lambda, \\nu) = f(x) + \\sum\\limits_{i=1}^m \\left[\\lambda_ig_i(x) + \\frac{\\mu}{2}g^2_i(x)\\right] + \\frac{1}{2\\mu}\\sum\\limits_{j=1}^p\\left[ (\\nu_j + \\mu h_j(x))_+^2 - \\nu^2_j \\right],\n",
    "$$\n",
    "где $\\lambda$ - двойственные переменные для ограничений-равенств, $\\nu$ - двойственная переменная для ограничений-неравенств."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Обновление двойственных переменных для неравенств\n",
    "\n",
    "$$\n",
    "\\nu^{k+1}_j = (\\nu^k_j + \\mu_k h_j(x_k))_+\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Общая схема \n",
    "\n",
    "1. Формируется функция $M$\n",
    "2. По необходимым условиям находится выражение для $x_k(\\lambda, \\nu, \\mu)$ и проверяется по достаточным условиям на минимум\n",
    "3. Взятием предела по $\\mu \\to \\infty$ при фиксированных двойственных переменных определяются прямые переменные, являющиеся решением исходной задачи\n",
    "4. Выражения для $x^k$ подставляются в выражения для $\\lambda$ и $\\nu$\n",
    "5. Взятием предела по $\\mu \\to \\infty$ определяются двойственные переменные для найденных оптимальных прямых переменных"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Сходимость\n",
    "\n",
    "- Локальная сходимость при выборе начального приближения достаточно близко к минимуму по прямым и двойственным переменным\n",
    "- Бесконечно увеличивать параметр $\\mu$ не нужно: для некоторого достаточно большого $\\mu$ решения вспомогательных задач гарантировано будут сходиться к решению исходной задачи\n",
    "- Решение вспомогательных задач проще, чем в методе штрафов, так как параметр $\\mu$ ограничен сверху"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Практические замечания\n",
    "\n",
    "- Метод модифицированной функции Лагранжа предпочтительнее, поскольку решение подзадачи не сложнее, чем в методе штрафов, но сходимость более быстрая\n",
    "- Метод штрафов становится предпочтительным как регуляризация в подходе *последовательного квадратичного программирования* ([SQP](https://en.wikipedia.org/wiki/Sequential_quadratic_programming))\n",
    "- Пакеты [LANCELOT](http://www.numerical.rl.ac.uk/lancelot/blurb.html) и [MINOS](https://en.wikipedia.org/wiki/MINOS_(optimization_software)) по-разному реализуют метод модифицированной функции Лагранжа\n",
    "- В [статье](https://www.dropbox.com/s/rdqcn0fppumreto/MinosvsLancelot.pdf?dl=0) приведено сравнение этих пакетов, однако они оба разработаны в начале 90х и написаны на Fortran'e"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "def augmented_lagrangian(update_x, A, b, rho0, x0, lambda0, max_iter):\n",
    "    x = x0.copy()\n",
    "    lam = lambda0.copy()\n",
    "    conv_x = [x]\n",
    "    conv_lam = [lam]\n",
    "    rho = rho0\n",
    "    for i in range(max_iter):\n",
    "        x = update_x(x, lam, A, b)\n",
    "        lam = lam + rho * (A @ x - b)\n",
    "        conv_x.append(x.copy())\n",
    "        conv_lam.append(lam.copy())\n",
    "    return x, lam, conv_x, conv_lam"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.3955287025942437e-15\n",
      "8.35127552684147\n"
     ]
    }
   ],
   "source": [
    "def update_x_al(x, lam, A, b):\n",
    "    return np.linalg.solve(P + rho * A.T @ A, c - A.T @ lam + A.T @ b)\n",
    "\n",
    "rho = 10\n",
    "max_iter = 1000\n",
    "x_al, lam_al, conv_x_al, conv_lam_al = augmented_lagrangian(update_x_al, A, b, rho, x0, lam0, max_iter)\n",
    "print(np.linalg.norm(A @ x_al - b))\n",
    "print(0.5 * x_al @ P @ x_al - c @ x_al)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Objective')"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAETCAYAAADKy1riAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3VtwY/d9H/DvDxcS5C4JkNxdAbK92gUjW1F8xVJ2m6QZNwId220fYmOXM5lJO5NWS08fOm2nXmpf2ulLJFKeyfghiUm1nT56STp9yCSyvVDi9DqxuJQvim8ysZLsaFd7IQ+5V16AXx/O/5AgFiAPwQMc4OD7mcEA538ODv4HBPHD/y6qCiIiIjdCfmeAiIjaB4MGERG5xqBBRESuMWgQEZFrDBpEROQagwYREbkW8fPFRSRrHo6q6oRJywGwAGRUdapWGhERNZ9vJQ0TMM6qah5ARkQyIpIBAJNm1UrzK89ERJ3Ot6ChqnlVHTebaVVdBDAGu0QBAAUA2RppRETkA1+rpwBARC4AcIJHAsBy2e6hGmk1HTt2TE+dOuVlFomIAu/KlSu3VPX4fsf5HjRUdUpE5kRkod5ziMh5AOcB4OTJk1hYqPtUREQdSUTednOcn20ambL2iQLsL30LwKBJSwC4XSNtF1WdUdURVR05fnzfQElERHXys6SRBbBoHicAvAYgD2DEpKXNNmqkERFRk/k5TmMGQNpULUFV501juNOzylLVxWppvuWYiKjD+VbSUFULduCoTHeVRkREzccR4URE5BqDBhERucagQURErvk+TsNrN9bW8dX8mwAAxc5StuWr2u5a4LZsx7G+bvz+P3gCItLgXBIRtafABY337jzEH+V/Vvfzf+vJ4zh17IiHOSLqbOvr61heXsadO3dQLBb9zk5HCIfD6Ovrw+DgILq7uz09d+CCxkfeF8d3//Dz29uVhYZapYj/u3QLv/fy3+Ld1QcMGkQeWV9fxzvvvIOBgQGcOnUK0WiUJfkGU1Vsbm5ibW0N77zzDk6ePOlp4Ahc0ACAUOjgH8pUvAcAcH31odfZIepYy8vLGBgYwLFjx/zOSscQEXR1dW2/58vLy0ilUp6dnw3hRrI/BgC4xqBB5Jk7d+6gv7/f72x0rP7+fty5c8fTczJoGD1dYSR6oyxpEHmoWCwiGo36nY2OFY1GPW9HYtAok+yP4drqA7+zQRQobMPwTyPeewaNMql4jNVTRER7YNAok4z3sHqKiGgPDBplUvEYbt/bwMNN9iUnIqqGQaNMMm73oLqxtu5zToiIWhODRplU3Ol2y8ZwIjq8xcVFiMgjt4GBAYyPj8OyrH3PMTw8DBHB4mJrLCXEoFFme4DfGts1iMg7k5OTWFpawtLSEq5cuYKLFy9idnYWp0+f3jMYLC4uolAoAAAuXbrUrOzuiUGjTDLOAX5E5L1EIoF0Oo10Oo1MJoMLFy7g6tWrSKfTePbZZ2s+79KlS8hkMsjlcpiZaY216Bg0yhztjqAvFmEPKiJquEQigZdffhmWZWFqaqrqMTMzMxgbG8Po6Cgsy2qJKioGjQr2WA22aRBR42UyGWSzWUxMTDyyb3FxEZZlIZfL4dy5cwCA6enpZmfxEYGcsPAwkvEeVk8RNcF//vO/w4/eXfM7G3t6+vF+/Kd/9msNfY3R0VHk83kUCgWk0+nt9Onp6e0qLcAOMLOzs74HDpY0KqT6OSqciJrHCQpOg7djdnYWuVxue3tsbAyWZSGfzzc1f5VY0qiQjMdw6+46NrZK6IowphI1SqN/wbeLRCIBALu63+bzeViWhdHR0e20XC6HiYkJzM3NIZvNNj2fDn4rVkjFY1AFbtxhaYOIGs9p3M5kMttpc3NzSCQSu4KDU1Xldy8qBo0KTrdb9qAiomZ47bXXAGBXe8bs7Cwsy3pkUKBTheVnFRWDRgVngB/bNYio0QqFAubn53HhwoXtNKdqam5ubntAoHO7fPkyAH97UbFNowJLGkTUDJZl4ezZs0gkErh48eJ2+vT0NBKJxK5GcIczOHB+fr6ZWd3F15KGiJw3t8mytElnX1laTkSyInKh2nm81B+L4EhXmCUNIvKMZVkoFAooFApYXFzE1NQUzpw5g0KhgFdffXW7MRwA5ufncf78+ZrnGh8f3z7OD74FDRHJAsir6gyAtNkGgPMisgSgYI7LAICq5gFYznYD84VkPIbraxzgR0TemJiYwPDwMIaHh3HmzBm88MILyGazuHr16q4GcCcQjI2N1TyX3wP9/KyeSpvbDOwA4bQCPaeq5SF0DMBl87gAIAugoWPpUxzgR0QeyGQyUFXXx+dyuX2PTyQSBzqn13wraajqjCllAEAGwIJ5nK6oikoAWC576lDluUwV14KILNy8efPQeUvGY2zTICKqwvfeU6a6aVFVFwFAVadMVdRQWZXVnkwAGlHVkePHjx86T6l4DO+tPcRWsXTocxERBYnvQQNAVlUngO0Sg9Nl4DbsKisLwKBJS5j0hkrGYygpcPMuV/AjIirne+8pVZ0yj7Owq6icUSvDZvsSdto70mX7GybFdTWIiKryu/fUpIgsicgKAJgqqnOmtLGkqtvVVuZ4y9lupGS/WcGPQYOIaBffek+ZdouBKumPTKxSLa2RWNIgIqquFdo0Wk6iN4ruSAjXuRgTEdEuDBpViAgeT3CsBhFRJQaNGpL9HKtBRFSJQaMGe61wBg0ionIMGjUkzQC/Usm/4fpERK2GQaOGVDyGrZLiFgf4ERFtY9CoIcnFmIiIHsGgUQPHahARPYpBo4adFfw4VoOIvDM8PAwRweJi9ckt8vk8RAQzM00d0+wag0YNg71d6AqHcG2NJQ0i8sbi4iIKhQIA4NKlSz7npj4MGjWEQoLH4t0cq0FEnrl06RIymQxyuVzLliT2w6Cxh1Q/R4UTkXdmZmYwNjaG0dFRWJZVs4qqlTFo7CGV4KhwIvLG4uIiLMtCLpfzfZ3vw/BzjfCW5yz7qqoQEb+zQxQsrzwPXP+h37nYW/IjwOde9ORU09PTSKfTSKft5YEymQxmZ2fbLnCwpLGHVH8MG8USlu9t+J0VImpzs7OzyOVy29tjY2OwLAv5fMPXlfMUSxp7KB/gN3S02+fcEAWMR7/g20E+n4dlWRgdHd1Oy+VymJiYwNzcHLLZrI+5OxiWNPbAAX5E5IW5uTkkEoldwcGpqmq3XlQMGntIcYAfEXlgdnYWlmVBRHbdnDEb7VRFxaCxh6Gj3YiEhCUNIqqbUzU1NzeHpaWlXbfLly8DaK9eVGzT2EM4JHiMizER0SFMT08jkUjsagR3pNNpZDIZzM/P+5Cz+jBo7CPJxZiI6BDm5+dx4cKFmvvHx8cxPj6O+fn5XYHlypUrVYNJNptFIpFoSF7dYNDYRzIew4/eXfM7G0TUhpwv/bGxsZrHnDt3DuPj45ient4VNGZmZqo2kl++fNnX3lYMGvtI9cfw6o/f4wA/IjqwXC4H1b1X/0wkEruOyWaz+z7HT2wI30cq0YOHmyWsPtj0OytERL5j0NgHx2oQEe3wNWiIyHlzmyxLy4lIVkQu7JXWLDuLMTFoEBH51qYhIlkAeVUtiMic2V4GAFXNi0haRDLO8eVpqtq0+YSdksa7HOBHRORrSSMNwOkCUDDbYwCssrRsjbSmOX60GyFhSYOICPCxpKGq5X3JMgAuATgDU9owhgAkqqQ1TSQcwok+jtUgIgJaoCHcVEEtHqbKybSLLIjIws2bNz3Mnc1ZV4OIDq6Vu48GXSPee9+DBoCsqk6YxxaAQfM4AeB2jbRdVHVGVUdUdeT48eOeZzAVj+Ea2zSIDiwcDmNzk93V/bK5uYlwOOzpOX3vPaWqU+ZxFnYVVdrsTgPI10hrKmcqEf5iIjqYvr4+rK1xRgW/rK2toa+vz9Nz+hY0TJCYFJElEVkBAKeKyuyzVHWxWlqz85qKx3B/o4g761vNfmmitjY4OIiVlRXcunULGxsb/OHVBKqKjY0N3Lp1CysrKxgcHNz/SQdQd0O4iJwCMA67Efusqq6JSBzAiwAmVHXPnxeqmgcwUCX9kclWqqU1U8qs4Hd99SH6Y1E/s0LUVrq7u3Hy5EksLy/jrbfeQrFY9DtLHSEcDqOvrw8nT55Ed7e3q47WFTRE5FkA3wbwOoBPwG5zWFPVVREZhh04/rVnufRZ+ajwDz7mbVGPKOi6u7uRSqWQSqX8zgp5oN7qqUkA86o6AqByFr8Z2GMrAiPJFfyIiADUHzSccRXVrMDu5RQYJ/piEOH8U0RE9QaNRQAjNfaNw4ceTo3UFQnh2NFuXLMYNIios9XbEP4igFmxF5hQAHER+Tjsaqss7JHdgZKKx3BtjUGDiDpbXUFDVedF5Bzs9guBXfIQ2APxzqnq97zLYmtI9sfw1u17fmeDiMhXdY/TUNV5VR2EXao4B+CMqg6q6jc8y10LSXGtcCKiurvc9jvjMFT1ddhdbwMtGe/BnYdbuLu+haPdXCWXiDpTvSUNS0S+KyL/UkT6Pc1Ri0pxMSYiorqDxvPmuS8DWBGRb4nI73qXrdbDFfyIiOoMGqo6ZQb2DQO4CHuNi2+ISFFELgUxgDxuphLhbLdE1MkONWGhql4tCyADsBvEhwHMe5G5VnKi356/hSUNIupknrToisgXYA/qy2KnC26gxKJhDB3p4lgNIupodZc0ROQLpi2jCLtkMQy7rWPYlDwCJxmP4ZrF6iki6lz1drktwR4JvgrgJQDTqnrVy4y1olQ8hl+uMGgQUeeqt3rqZQBfM2M0OkYyHsPC2yt+Z4OIyDf1TiMy7nVG2kEq3gPr/iYebBTR0+XturtERO1g3zYNEfm2iGxVpJVM99pat0Cui5rsN2M12BhORB3KTUkjD3siwnIvwW7T6Cg7K/g9wOljR3zODRFR8+0bNFR1qkraRGOy09o4KpyIOt2hBvd1mtT2qHAGDSLqTHUFDRG5LSKnauz7toj84WEy1ap6usJI9EZZ0iCijlVvSWNgj33TAM7Wed6Wl+znuhpE1Llcd7k1JYtMWdKzIlJt0MJ5AOnDZat1peIxXF/jAD8i6kwHGafxJQAXzGOFPcCvlkcaz4MiGe/BD3656nc2iIh84bp6SlWfh10tNQh7UsJR2PNNld8GVDVkjnVFRDIV25Pm/nxZWk5EsiJyofL5zZaKx3D73gYebhb9zgoRUdMdqE1DVVdV1YI9QeFrZmr08tuBfoKLSBbAXEXyeRFZAlAwx2TMa+dhrxiYgY+cbrc31tb9zAYRkS/qbQh/AfbiS7uISNzMfHvKzUlMIChUJD+nqsNmHwCMYWdwYQH29Ou+KR/gR0TUaeoNGi+iSmO3KWkIgMlD5CldURWVALBctn/oEOc+tO21wjmVCBF1oHqDxjMALtfYN4fdvawOxKwEmAcwZKqv9iUi50VkQUQWbt68We9Lu5LkAD8i6mCHGRG+vEd6XV1uzZd/zmzeNuexYDe+A3ap43bl81R1RlVHVHXk+PHj9by0a0e7I+jrjnCAHxF1pHqDxgLs8RjVjMGe5LDe8zrPHTbbl7AThNKHOLdnUokY2zSIqCPVuwjT8wAWROSbsNsvVmCXBiZgN1SfcXMSU6oYEZGcqs6r6qIpbSwDWFLVRXPciKmqspw0PyXjPSxpEFFHqncRpkUR+QzsKUNehT3YT2D3bvqMqn7P5XnmYXffLU+bqXLcI2l+SvXH8JNra35ng4io6eotaTjdZYdF5DTsaqNCJ6wTDthjNW7eXcfGVgldEU4UTESdo+5vPBE5JSIvAPhTmIF+ZpzGn4pIv3dZbD2peAyqwI07rKIios5S79TozwJYgj2VyChM7yYzTmMY9jiOwOJiTETUqeotaUwCmFfVEdhtGeVmYPegCiwuxkREnareoJGB3RW2mhXY4ykCiyUNIupU9QaNRQAjNfaNowXGUjRSfyyC3q4wSxpE1HHq7T31IoBZERHY3W3jIvJx2NVWrsdptCsRQZKLMRFRB6p3nMa8iJyD3X4hsEseAnvKj3Nux2m0s1Scy74SUeepu8utGcE9CLtUcQ7AGVUdVNVveJa7FpbiqHAi6kB1D+5zqOrrAF73IC9tJRWP4caddWwVS4iEOcCPiDrDvkFDRL4AIK2qXylL+w8uz28ByKvqW/Vlr3Ul4zEUS4pbdze2e1MREQWdm5LGlwA8C+ArZWlTB3gNFZHzqvpfD5SzFucsxvTu6gMGDSLqGG7qVcZR0b1WVUNubrBHiv8ZDhZk2kKy3x7gx3YNIuok+wYNVb1q2i0OTFUtAF+v57mtbmetcAYNIuoczWjBHYM9fXqgJHqj6I6EcJ2LMRFRB6m795SInIK96NIIzNTosNcNf1FVtxebUNVzh8tiaxIRjtUgoo5z2Flux2HPNTVn7p8HUDABJfCS8RjbNIioo9Rb0pjGzip92wsviUgCdlXU1wB89vDZa22peA++e3XZ72wQETVNvW0agwAmK1fqMw3fE7DX2Ai8ZDyG99YeolRSv7NCRNQU9QaNBQC1fmIvAeiIZV8fj8ewVVLcurfud1aIiJqi3qAxAeBijX0vArhQ53nbSjLOsRpE1FncTCNSgj39eZVdUqyWDiAHIHzIvLW87VHh1kN89P0+Z4aIqAncNIS/hOpBo+PtrODHsRpE1Bn2DRqqOtGMjLSjwd4udIVDuLbG6iki6gyc0/sQQiHBY/FutmkQUcc48DgNs6zrCHaWdL0CoKCqf1VPBkQko6qLZds52FOqZ1R1qlZaq0j193BUOBF1DNdBwwSLlwFkYDd2l1MRuQLgOVX9/gHOmYU9UHDYbGcAQFXzIpJ2tivTyoOM35LxGL73C8vvbBARNYWr6imzENMigAHYS7sOm6nPB2CXOC4CeBLAooj8rtsXV9U87JHljjHYJQqY9GyNtJaRMlOJqLKvABEF375BQ0ROA5gHMKOqv6Kq33BGgqvqqqq+rqpTqjoAe+2MeRF5os78JLB70OBQjbSWkYzHsFEsYfneht9ZISJqODcljUkAV1T1S/sdqKpnYa8XPn3YjLWLlBngx3YNIuoEboJGFsClA5zzEoBn6ssOLNjzWgF2CeN2jbRdROS8iCyIyMLNmzfrfOn6pLbHajBoEFHwuQkaCey0KbhhmefU4xLstTlg7vM10nZR1RlVHVHVkePHj9f50vXZWcGPA/yIKPjcBI1FHKzxedQ8Z1+mK+2IuYfTK8r0qrJUdbFa2gHy0nBDR7sRCQmrp4ioI7jpcvsigFkRuaSq/2OvA0XkOQBfBHDWzYur6jzsRvbytJkqxz2S1irCIcFj/VyMiYg6w74lDfPF/jLsXlF/IiL9lceIyMdF5BLsxZdeVtU/8z6rrSvJZV+JqEO4GtynquMiYgH4MgDnsTO+whmAJwCmVPV577PZ2pLxGH707tr+BxIRtTnXc0+ZiQsHAHwF9tQhw+b2KuyZcIc7MWAAQKo/hmurDzjAj4gC70BzT6nqKuwFmKhMMh7Dw80SVh9sItHb5Xd2iIgahrPceoAD/IioUzBoeCDJAX5E1CEYNDzweMIZ4MegQUTBxqDhgeNHuxESLvtKRMHHoOGBSDiEE30xvMuSBhEFHIOGR5JxjgonouBj0PBIKh7jpIVEFHgMGh5xphLhAD8iCjIGDY+k4jHc3yjizvqW31khImoYBg2PJM0AP7ZrEFGQMWh4ZGcxJgYNIgouBg2PJPudUeFsDCei4GLQ8Mhj/TGIsKRBRMHGoOGRrkgIx452s02DiAKNQcNDqThHhRNRsDFoeCjZH2ObBhEFGoOGh1JcK5yIAo5Bw0PJeA/uPNzCXQ7wI6KAYtDwUIqLMRFRwDFoeIgr+BFR0DFoeGhnVDgbw4komBg0PPRYP0saRBRsDBoeikXDGDzShWtrDBpEFEwtFzREZNLcny9Ly4lIVkQu+Jczd1JcwY+IAqzlggaA8yKyBKAAACKSAQBVzQOwnO1WxbEaRBRkrRg0nlPVYRMkAGAMgGUeFwBk/cmWO0ku+0pEAdaKQSNdURWVALBctn+o8gkicl5EFkRk4ebNm03JZC2peA+s+5t4sFH0NR9ERI3QckFDVadMKWNIRFyVKlR1RlVHVHXk+PHjDc7h3rbX1WBjOBEFUEsFDVNiyJnN2wDSsKumBk1awqS3LI7VIKIga6mgAWABgNOWMWy2L8EOHjD3+SrPaxnOqPBrFksaRBQ8LRU0VHURwDlT2lhS1UWTBlNVZTnbrerxRA/6uiP4yx9e8zsrRESei/idgUqqOuMmrVXFomF86dPDeOlbP8XfFm7jU+lH2u2JiNpWS5U0guIPfuM0kv0xvPDKT6CqfmeHiMgzDBoN0NMVxr8f/SC+9wsLr7xx3e/sEBF5hkGjQb545v344GNH8dK3forNYsnv7BAReYJBo0HCIcHEZ5/C1Vv38PXvvuN3doiIPMGg0UC//dQJfPL0IL766ptcApaIAoFBo4FEBBc/9xRu3d3AzP8s+J0dIqJDY9BosE+cHMDnP5LEf/lfBdy4wwF/RNTeGDSa4Mu/8xQ2tkr4av5Nv7NCRHQoDBpNcPrYEfzep07i66/9Aks37/qdHSKiujFoNMm/efZJxCIhvPTNn/qdFSKiujFoNMmxo904/1vD+ObfXceVt1f8zg4RUV0YNJroX/2j0zh2tBsvvvJjTi9CRG2JQaOJjnRH8G+zT+K1t1aQ//ENv7NDRHRgDBpNNvbMB5A+dgST3/wJtji9CBG1GQaNJouGQ7jw2Q/h5zfuYu7KL/3ODhHRgTBo+OB3fi2JzMkE/ujyz3B/g9OLEFH7YNDwgYjg4ud/FTfurOO//e+rfmeHiMg1Bg2fPHNqEKNPP4av/U0Bcwu/4PTpRNQWGDR89B//6dM4OdiLL8//AJ9+6Tv47//nKh5sFP3OFhFRTRK08QIjIyO6sLDgdzZcU1V856c38cd//XMsvL2CoSNd+IPfPI3f/4dPoD8W9Tt7RNQhROSKqo7sexyDRuv47tVl/PFf/xx/87Ob6OuO4J//+hP4F79+Cif6Yn5njYgCjkGjjb3x96v4k+/8HK+8cR0C4FOnh/BPPprCZz+cxLGj3X5nj4gCiEGjnZRKwPoq8HAVeGABDy3g4Spu3HwPb7x1HT99dxlrdx8gIkWcSnThgydiGB6MoTeiQGkLKG4CpU1gzz/lHjv3/Azs8/kI2OcnsE48Bfzmv/M7F77Z2Crh/sYW7m8Ut+/vrRfxYHML99aL2NgqYatUwkZRsWkebxYVm8USNoslbBUVG8USVO0qZYX90VcoSur8G6jZb6erwt4H3fffyCuf+0gKo08/Vtdz3QaNSF1np4NRBe7fBm4vActLu++tt4GHa6j2qToB4LfNDU7zxl1g804YxaUQ7oeiCEeiiEajCIW7ANmnX4PIXjvr2uXyAPJd+wd3VcXawy2s3t/Eyv0NrNzfwOqDTazc24D1YBOWSbfub8K6v5N2f2MLm8X6rz8cEkTDgmgoBBG7y7yI/akPmcfATpp9b2+HzP+c/Twv3oW9feJkouGvwaDhpVLRDgTXfwDcenN3gHi4unOchIHESWBoGPjAJ4HeISCWAGJxoMfcO9tdR4BQBAhHgVAEKmEs3biLv/zBNfzFD69h6eY9hAT45OlBfObpJP7xUydw+tgR/94Don2oKu5vFGGZL/zVB04Q2MSquS//4ncCweqDTRRLtb/8+2IRDPR2IdEbRaK3C08MHUGiN4oj3REc6Qqjp8u5D+NIVwS93WH0dkXQ2xVGdySEaDiESFjQFd55HA2FEArxR1G5tqieEpEcAAtARlWn9jq2adVTWxvAzR8D174PXPuBff/eG8DmfXOAAPEPAENpYHDYDhBDv2I/TpwEIl2HzoKq4mfv3cVf/PAaXvnhNbx5w17g6fSxI/j0h47jN4aP4cwTAxg4cvjXIgKAUkmxbqp67q5v4c7DLdxbtx9v3x7ufrxqfvFbD5zAsImNPcYl9XaFMdDbhXhPFANHokj0OIEgaoJCFxJmX7ynCwO9UcR7ooiEOYLgMALTpiEiGQBpVZ0XkfMAFlR1sdbxDQkaG/eA62/YgeG6CRI3fmy3IwBA11Eg+VEg9dGd+6EngWhzez29c/s+vvOzG/irn9zA/1u6jfUt+x/zyRNHceaJAfxqqh9PJfvwoWQfEr3+B5KtYgkbxRI2txQldW52MCwpttN0+zHMdtn+EiqO2TlHsWTfNkuKoqmjLpbseupiSbFVVGyVFFulknlcsreddHOc/bySfZ6iYrNU/vzSrvMATpWFIGSqJ0KmSmNn2666CId2HodEdlV3OFUeIald3eE8Z3eViWxXg5TM9W+V7PfFeT+KJUWxbLukul1nv75ZwsOt4q779bJtt9U8IsDR7giOdkcQ77G/1MtLAXYAiJYFAPs+3htFdyTs+WeN9hekoDEJ4LKq5kUki31KG4cOGveX7eolp/TgVDU5dcK9Q0DqYyY4fMy+DZwGQq31K+fhZhHf/4WFhbdX8Npby3j9HQurDza39/fFInhfogfJeAzxnij6YhH0x6Loi0URi4ZMY55p9DONeVslxcZWaedW3Hm8Xva4cp/zeH2rhI2t4vb2HjUNLSESElOfHdqu1w6HBJGQXXURDtnVF+X7RKQi+FUPbI8Gwop77DSuOudy0krmj6PYeY5WNMICdlAJh3ZuEROkytPCZdtdkRC6IyHEonZ1Tbe5j1W574mG7aAQi6CvO4IjFY97u8KQZlTik2eC1BCeALBctj2059E3fgR89WPYbpzd/uC62N64B6yVzTwb/4AdHD78xZ1A0f94c1q0DikWDeNT6SF8Km2/XaqK99bW8ZPra3jzvbv45cp9/L31ANfXHuLqrXtYe7CJOw+3sOXimzxq6n27Iju3aDiErrD9pdMVCSEWDaE/FjH7w9vHO/vLnx8NhxAWIBSq9Qt9J03Kfq0/8gs+9OjxkZCYL/jQ9uNI5eOwc5xJD+0EACLarR2Cxr5MtdV5APjw+44C7/+k2WO+ALdLU/tsR7qBE0/vlCB6Bxud9aYRESTjMSTjMXz6QyeqHqOqeLBpV0WETFeQnR4h9pdpV5gNg0SdrB2ChgXA+fZOALhdeYCqzgCYAezqKXzx5eblLkBExPT2kKcUAAAHTElEQVQm8TsnRNSqWqsivrpLANLmcRpA3se8EBF1tJYPGk5PKdMIbu3Vc4qIiBqrHaqnnOonIiLyWcuXNIiIqHUwaBARkWsMGkRE5BqDBhERucagQURErrX83FMHJSI3AbwNIA6gbD7yXdu19h0DcMujrFS+Rr3H1dpfLd3tNZc/5jXXz+31ujnW7TXv9x50wjW7/Zu3yzUf9HNdue3VNT+hqsf3PUrNrKFBuwGYqbVdax/sGXQb8vr1Hldrf7V0t9dc8ZjX3ODr9fKa93sPOuGaD/A3b4trPujnuhnXvNctyNVTf77H9l77GvX69R5Xa3+1dLfX3IjrPch5g3LNBzmnV9e833vQCdd8kP9tLzT6mg/6ua7cbtT/c1WBq546DBFZUBdTAwcJr7kz8Jo7QzOuOcgljXp04shzXnNn4DV3hoZfM0saNYhIAsAI7Jl186pq+ZylpjArJQ6qakdMDCkiadh/4yyAeVUt+Jylhiv7bGdg14d3xGcbAERkWlXH/c5HM5jPtrOInWfBpGNLGubLsXw7JyJZEblgki6aL848zFod7c7FNQPAGOwv0UBwcc0ZAAXYf+dcs/PXCC6ueQTAAuxlB9KVz29Hbj7blce0O5f/z895GTCADg0aZsbcubLtDACYIGGZ7YRJswAM+5FPL7m8ZsCeij4Q3Fyzqs6bv3EWwLw/OfWOy2t2SpEJDcCs0W6u2fzqtsyt7bn8f14GkDbBxLMfgh0ZNMwbW14NMYadD1MB9heIJSIJ82YvNTmLnnN5zYHi9prNP2A+CFVTbq5ZRM6bQLlY8au0Lbn8OzvVkGkTQNqay2seMT8KCvCwtqQtpkZvgmrrkE9j54s0iA1qtdZezwIYEpEgtuM8cs0mYEwAKIjIZVVt+9JGhWp/57y57jQCULqq4pFrVtW8+QEYmKrXCtX+zgVT4hiBh39nBo0azK/Otv/leVCqOuV3HpqprN2qYwShSqoe5kfQqN/5aJaykrOnf++OrJ6qYt91yAOI18xrDipecwOvmUHD1onrkPOaec1BxWtu4DV3ZNAQkRyAEXO/XVwP8jrkvGZeM3jNgeHnNXNwHxERudaRJQ0iIqoPgwYREbnGoEFERK4xaBARkWsMGkRE5BqDBhERucagQURErjFoUCCIyHTFds4MdHL7/Asictn7nHmnHfJIwcegQUFROd31KA424eQzaP3p4dshjxRwDBrU9syU15UBIn2Q9TFU9ayqSsV5J/1Ye6HW61bLI1GzMWhQEGQBXHE2TBA51Fog5hwXYC8H2zR+vS6RWwwaFASj2D2r5wiA13zKC1GgMWhQEFRWRVUGkX2ZKiE1j+cArJhdcyKiIrJScXxORK6YfVfKG93NuVbM42kRWTJLB6dFZE5EVszzLpdXQ+33uuV5LEvLmPM4+ThfsX/SvF7lcZmK43Imn1otb0QOBg1qazWqojKHnBr6Oeys8DYOYBjAmbLXzAGYgx2YzsJuT3nkS1ZErsAu9UybVePGK84/iLJqtf1et5LJxxXz+qOw11SYruxJBntRnjnYSxiPw+40MFd2Hmd73ryec01BXRqVDoHLvVJbMV+I5V/OgwASFV1RRyq2J82yrq6oqiUiTslluUqD+ssAplR1wmzPmwAxDnu9ccCs2ayq28uLlh3vXMtzAK6ISE5V5128bqWXAcyoqhOM8iKyCDuAzVVc84Sz/rmIDAO4ICIJE8ycUscLZnsRwVw7nDzAoEFtpewLEsB2EJkwX3bOIjRpVZ1pxOubap0E7C/dCxW7lyu2x7EHVV0UEeDR7sJu8pE1+dhVqlDVvIhYsEsL5UGj/PFSxemcfa+KyCUA+SAuXETeYNCgdpd2AoZxFsBkI1/P3J/Bo9Viu4JGtZKCqVIaM+c5TJuB89xqpZFC5bkr3iNU7hORUdjv26TJZ768lETkYNCgtlJRPVW1agpA2vyCdxyoemof21/SBxkHAgAmn2nYX8wLJvlK7We4ykcadnVSuXTZ+V0x788Z00Z0HsCkiFxQ1ak680cBxaBBbaW8espUDy06AcE06E5UVmHVqbKqyXn9RVP9M4mdRmsnP4lav+hN3rIAzpa1LVRraK76ulU4QWEcZdVgpiTjNHwfmMn/lIiMwR6BTrQLgwa1s2cqfglnAXgyN5OpsgGAcee+rLrmOdhdYp0eSQnYX9wF1GjHUNWCE2zM+SzsNJq7fd3K48Zh95YahN1zyinFzB+kZGW66U6Ya3GqtjKoaC8hAtjlltpb5S/1Mzjg+Ix9TMEORJMoqwIyJYUz5vUvw+7FVECVIFDhOdhVanPmnLVKA1Vft5Jp7B/FThfacdglrbP75KPSLOzeUuNl55lqVGcCam+iqvsfRdRiqlVFichlNt4SNRZLGtSudlVFeTHfFBHtj0GD2lXlVCGetWcQUW0MGtSulit6KqXhbXsGEVXBNg0iInKNJQ0iInKNQYOIiFxj0CAiItcYNIiIyDUGDSIico1Bg4iIXPv/hjI+qLQ6JeEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot([f(x) for x in conv_x_da], label=\"DA\")\n",
    "plt.plot([f(x) for x in conv_x_al], label=\"AL\")\n",
    "# plt.yscale(\"log\")\n",
    "plt.xscale(\"log\")\n",
    "plt.legend(fontsize=20)\n",
    "plt.xlabel(\"\\# iterations\", fontsize=20)\n",
    "plt.ylabel(\"Objective\", fontsize=20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 0, 'Lagrangian')"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAETCAYAAADTbHYBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3XtwXNdh3/HvwWvxxgIEiCUpQeSCpPWAbGcFW04kyskIrNs8aieBiHYySTttBbgzqeNJpmTU8UwnbaY21CR9TJMGcJqmaTMZEMiMU3diO4STjCg5dgTCT1kPCiBFPUgKILAE+MD79I97Fli8yF3u++7vM7Oze8/evXsuAf5w9tyz5xhrLSIi4k8lua6AiIhkjkJeRMTHFPIiIj6mkBcR8TGFvIiIjynkRUR8TCEvIuJjCnkRER9LOuSNMZH4x8YYa4yZcLcBV97v7nvTV1UREUlWWTI7G2O6gAGg3RU1WWuNey4CRF15rzGmG+i72zGbm5vtwYMHk6mGiEjRO3fu3LS1tuVu+yUV8tbaUWPMZPx23NOd1tpB9/hZa+1IIsc8ePAgY2NjyVRDRKToGWPeSmS/tPTJuxb+6biisDGmyxhzMh3HFxGRe5OuC6/HrbWxrhqstc+7Vv4e9wdARERyIF0hH38xNtYfD3ANCG/d2e0zZowZm5qaSlMVRERkq5RD3hizNcTHgFhffbvb3sRaO2it7bTWdra03PW6gYiI3KOkQt610DvjWuox8Rdjx4ETbp8Jty0iIjmQ7OiaEWBkS9kkW4ZKxo2yERGRHNI3XkVEfEwhLyLiY0l114iIvy0uLjIzM8P8/Dyrq6u5rk7RKC0tpa6ujqamJgKBQFqPrZAXEcAL+EuXLtHY2MjBgwcpLy/HGJPravmetZbl5WXm5ua4dOkSbW1taQ16ddeICAAzMzM0NjbS3NxMRUWFAj5LjDFUVFTQ3NxMY2MjMzMzaT2+Ql5EAJifn6e+vj7X1Shq9fX1zM/Pp/WYCnkRAWB1dZXy8vJcV6OolZeXp/1aiEJeRNapiya3MvHvr5AXEfExhbyIiI8p5EVEfEwhLyLiYwp5ERFgfHwcY8y2W2NjI319fUSj0bseo729HWMM4+P5M/muQl5EJE5/fz8TExNMTExw7tw5nnvuOU6fPs2hQ4fuGN7j4+NMTnqzrg8NDWWrunelkBcRiRMMBgmHw4TDYSKRCCdPnuTChQuEw2GefvrpXV83NDREJBKhu7ubwcH8mW1dIS8ichfBYJAvfvGLRKNRnn/++R33GRwcpKenh+PHjxONRvOmyybpkDfGRLZs97v73riybmNMlzHmZOpVFBHJvUgkQldXF6dOndr23Pj4ONFolO7ubk6cOAHAwMBAtqu4o6RmoTTGdAEDeGu3xsQW7u5z+0QArLWjxpiwMSaiJQBFCttvfPkVfvjeXK6rcUcP76/n3/7MIxl9j+PHjzM6Osrk5CTh8Mby1gMDA+tdPOD9QTh9+nReBH1SLXlr7Shx67k6z1pr291zAD1A7DL0JNCVWhVFRPJDLMRjF1hjTp8+TXf3xtLXPT09RKNRRkdHybV0zCcfdi38iLX2eSAIxM+VuScN7yEiOZTpFnKhCAaDAJuGU46OjhKNRjl+/Ph6WXd3N6dOnWJ4eJiurty2c1O+8Gqtfd614ve4sBcR8aXYxdRIZOPS5PDwMMFgcFOYx7pu8mGUTUohb4yJ9ccDXAPCeF01Ta4s6Mp3et2YMWZsamoqlSqIiGTNyy+/DLCpP/706dNEo9FtX6KKdenkussm1Zb8GBA7g3a3PYQX9rj7bWdorR201nZaaztbWlpSrIKISOZNTk4yMjLCyZMbgwZjXTXDw8PrX6CK3c6cOQPkfpRNsqNruoFOY0y3tXbEWjvuWuUzwERsFI0xptN13UQ1skZECl00GuWZZ54hGAzy3HPPrZcPDAwQDAY3XXSNiX2ZamRkJJtV3SapkLfWjgAjW8q2dTrtVCYiUgii0eh6V0tshMzAwAAzMzN8/etfX7/4Cmxr2W/V19dHX18fIyMjO/4hyAZ941VEJM6pU6dob2+nvb2dxx57jM9//vN0dXVx4cKFTRdcYy30np6eXY+VD1+MSscQShGRgheJRLDWJrx/d3f3XfcPBoNJHTMT1JIXEfExhbyIiI8p5EVEfEwhLyLiYwp5EREfU8iLiPiYQl5ExMcU8iIiPqaQFxHxMYW8iIiPKeRFRHxMIS8i4mMKeRERH1PIi4j4mEJeRGQX7e3tGGPWF/DeanR0FGNMXizYvZukQ94YE9my3etu/XFl/bHnUq+iiEj2jY+Pr68QNTQ0lOPa3LukQt6t2zq8ZXvULfcXdtsAvcaYCWAybTUVEcmioaEhIpEI3d3ded1Sv5ukQt5aO8rm4A4DsWCfdNsAz1pr293+IiIFZ3BwkJ6eHo4fP040Gt21yybfpdQnb60djFu0OwKMucdhY0yXMWb3FW5FRPLU+Pg40WiU7u7uvFinNRVpWePV9dOPW2vHAay1z7vy48aYrq0tetdX3wvQ1taWjiqISCZ95dfhyvdzXYs7Cz0K/+ALaTnUwMAA4XCYcNjrnIhEIpw+fboggz5do2u6rLWnYP1CbLcrv8ZGF8469wmg01rb2dLSkqYqiIikx+nTp+nu7l7f7unpIRqNMjpaeD3QKbfkjTG9cS33Lrwum1i/fTtQeH/6RGSzNLWQC8Ho6CjRaJTjx4+vl3V3d3Pq1CmGh4fp6uq6w6vzT7Kja7qBzlhL3YV6vzFmwhgzC+C6bE64fSZiXTgiIoVgeHiYYDC4KcxjXTeFOMomqZa8tXYEGInbHgUad9iv8P4lRETwumqi0SjGmB2fHx0dLajWvL7xKiLixLpqhoeHmZiY2HQ7c+YMUHijbNIyuiYldg2Wbu7wxA5/Rbf+ZS0NQIn+TolIegwMDBAMBjdddI0Jh8NEIhFGRkZ2eGX+yn3IX/4u/If99/bath+Ff/bV9NZHRIrWyMgIJ0/u/vWevr4++vr6GBkZ2fSH4Ny5czuGf1dXF8FgMCN1TVTuQ77+ABz/tY1ta3fYaYeyt74B5894nwIqajJWPREpDrGQ7unp2XWfEydO0NfXx8DAwKaQHxwc3PGi7JkzZ3Lef5/7kK/dC0/8SvKvaz4K5/8Srv4Q7v9I+uslIkWlu7sbu2Mjc0MwGNy0T1dX111fk2uF26Hd2uHdX83zb+GJiORQ4YZ8sA0CDXDlB7muiYhI3irckDcGWh+Bqwp5EZHdFG7IA4Q64OorsLaW65qIiOSlwg751g5YugGzF3JdExGRvFTYIR+KXXxVl42IyE4KO+T3PgymRBdfRdIk34cD+l0m/v0LO+TLq2DPYbXkRdKgtLSU5eXlXFejqC0vL1NaWprWYxZ2yIPXL6+WvEjK6urqmJuby3U1itrc3Bx1dXVpPWbhh3yoA65fgtvRXNdEpKA1NTUxOzvL9PQ0S0tL6rrJEmstS0tLTE9PMzs7S1NTU1qPn/tpDVIV+qB3f/UVOPhEbusiUsACgQBtbW3MzMxw8eJFVldXc12lolFaWkpdXR1tbW0EAoG0HjvpkDfGROJXe3IrQEWBSNwygNvKMqY1boSNQl4kJYFAgH379rFv375cV0XSJNnl/7qA4bjtCKyvEBU1xkR2KktjfberC0H1nvxfSV5EJAeSCnkX3JNxRT14LXZcedcuZZljjNea1wgbEZFtUr3wGgRm4rb37FKWWaFH4f1XYXUl428lIlJIcjK6xhjTa4wZM8aMTU1NpX7A1g5YWYCZidSPJSLiI6mGfBSIjfcJAtd2KdvEWjtore201na2tLSkWAU2pjdQv7yIyCaphvwQEHaPw8DoLmWZ1fwBKClXyIuIbJHs6JpuoNPdExtK6UbdRK214zuVpbnO25VVQMsHdPFVRGSLpMbJW2tHgJEtZdtWr92pLONaO2Dyb7L+tiIi+azwpzWICXXAjStwczrXNRERyRv+CflWXXwVEdnKPyEfetS7V7+8iMg6/4R8TTPU7dO0wyIicfwT8qDpDUREtvBXyIc6YOp1WFnKdU1ERPKCv0K+tQPWlmH69VzXREQkL/gr5GMXX9UvLyIC+C3km9qhrFLDKEVEHH+FfGkZ7H0IrirkRUTAbyEPXr/8lR+AFiEWEfFhyIcehdszMH851zUREck5/4X8+vQGuvgqIuLDkH/Eu1e/vIiID0O+KgjBNrXkRUTwY8gDtD6q6Q1ERPBryIc64NqbsHw71zUREcmplELeGBMxxlhjzIS7Dbjyfnffm45KJq21A+wavP/DnLy9iEi+SLUl32StNdbaduAZoN+V9xpjJoDJFI9/b0IaYSMiAimGvLV2NG6z01obC/VnrbXtW57PnuBBqKhVv7yIFL209MkbY7qA03FFYWNMlzHm5C779xpjxowxY1NTU+mowmYlJd5QSs1hIyJFLl0XXo9ba6OxDWvt864Vv8f9AdjEWjtore201na2tLSkqQpbtHbA1Vc0vYGIFLV0hXwk9sC10rvd5jUgnKb3SE6oAxbnIPpWTt5eRCQfpBzyxpitIT4GxPri29129rVqbnkRkXS15NdH0Vhrx4ETrjU/4bazr/VhwOjiq4gUtbJUD+BG1PRtKRtM9bgpq6iBprAuvopIUfPnN15jQpreQESKm89DvgNmL8LCXK5rIiKSE/4O+djFV01vICJFyt8hvz69gfrlRaQ4+Tvk6w9AZVD98iJStPwd8sZ4F181Vl5EipS/Qx42pjdYW811TUREss7/IR/qgJXbMJObWY9FRHLJ/yHfqouvIlK8/B/yLQ+CKdXFVxEpSv4P+fJKaD6qi68iUpT8H/Lg9curJS8iRahIQv5RmHsXbs3kuiYiIllVHCEfu/iq1ryIFJniCPmQFhARkeJUHCFfuxdq9qolLyJFJx3L//W7+964sm5jTJcx5mSqx0+bUIfGyotI0UlHS77XGDOBWwLQGBMBsNaOAtHYds61dsDUa7C6nOuaiIhkTTpC/llrbbsLdYAeIOoeTwJdaXiP1IUehdUlmH4j1zUREcmadIR8eEvXTBCIH6u4Jw3vkbr16Q3ULy8ixSPlkLfWPu9a8XuMMQm12o0xvcaYMWPM2NTUVKpVSEzzESitgKvqlxeR4pFSyLuw7nab14AwXldNkysLuvJNrLWD1tpOa21nS0tLKlVIXGm5N4+NWvIiUkRSbcmPAbG++Ha3PYQX9rj70R1elxuhRzWMUkSKSkohb60dB0641vyEtXbcleG6bqKx7bzQ2gE3p2D+aq5rIiKSFWWpHsBaO5hIWV6IffP16vehrjW3dRERyYLi+MZrTEgjbESkuBRXyFc1Qv196pcXkaJRXCEPbnoDhbyIFIeU++RT9dqVeZ74wl9hjLe9fo/Zsh173qxvf/RQE1/4+Q8m94atHXD+DCwveKtGiYj4WM5DvjZQxuNhN6zebrrDWru+bbc899a1mwyNvc3Jv/8gTTUVib9hqAPsKky9Cvt/JNXqi4jktZyH/H2NVfzOiQ8n/bpvX5rlZ3/vG7z05jQ/86H9ib+wNW5ueYW8iPhcwfbJf/C+IPWVZZw9n+S0CE2HoLxaF19FpCgUbMiXlhiePNLM2fPT6906CSkphb0P6+KriBSFgg15gGNHWrh8fYGJqRvJvTDU4X0hKpk/DiIiBaigQ/7Jw80AvPDGdHIvbO2Ahetw/Z0M1EpEJH8UdMjf31RNuLkm+X759ekN1GUjIv5W0CEPcOxIM9+cnGFxZTXxF7U+4t2rX15EfM4HId/C7eVVzr01m/iLAnXQeEgLiIiI7xV8yH+sfQ9lJYaz55Psl9f0BiJSBAo+5GsDZUQeaEy+X771UZiZhKWbmamYiEgeKPiQB3jqSDOvvDfHtRuLib8o1AFYuPrDjNVLRCTXUg55t85rrzGmP66sP/ZcqsdPxLEjLVgLL01sW052d62xueW/l5lKiYjkgVQX8u4CRt1KUGG3DdBrjJkAJlOtYCI6DjQQrC7n7BtJdNkE2yDQoGGUIuJrqbbkw0As2CfZWMD7WWttu7U2K4t4l5YYnjic5BQHxnhDKXXxVUR8LNWFvAfj1nONAGPucdgY02WMOZlS7ZLw1JFmrswt8Ob7SUxxEOqAq6/A2lrmKiYikkNpufBqjIkA49bacQBr7fOuFb8nrgsnfv9eY8yYMWZsairJUTG7ePJICwAvJDOUsrUDlm/C7IW01EFEJN+ka3RNl7X2FKwHeLcrv8ZGF8469wmg01rb2dLSkpYKHAhWEW5JcoqD2MLe6pcXEZ9Ky+gaa+3z7nEXXpdNrC++nY0unIx76kgL35y8lvgUB3sfBlOifnkR8a10jK7pN8ZMGGNmAVyXzQnXmp+IdeFkw7EjzSwsr3HuYoJTHJRXwZ4jasmLiG+ltPyf63dv3KF8cIfdM+5j4T2UlxpeOD/Nj7lpiO8q1AFvv5zZiomI5IgvvvEaUxMoI9KW5BQHrR1w/RLcTmKCMxGRAuGrkAd46mgLr7w3x3SiUxw88IR3/8efhOnzmauYiEgO+C7kjx3xumleejPBoZRtj0PPn0D0Egw8Bef+l5YFFBHf8F3IP7K/gcbq8uSWBHzop+FffgPu64QvfwZO/xLcmslcJUVEssR3Ib8xxcFU4lMcANTvh1/8c+j6DXj9L+D3n4SLL2auoiIiWeC7kAdvvPz784u8cTWJKQ4ASkrgyc/CPz8DZZXwRz8NX/93sLqcmYqKiGSYL0P+Sdcvn/RCIjEHItD3AvzIL8DZ34Y//IS3wIiISIHxZcjvD1ZxeG9tcvPYbBWohU/+LjzzR3DtTfj9Y/CdP9VFWREpKL4MefBG2Xxr8hoLywlOcbCbR34WPv0S7PsQfOnT8Cfd8K1BuPw9WEvx2CIiGZbSN17z2VNHWvifL11k7OLsevfNPQveD//ky/DSf4a/+wN4003NE6iH+z8KbR+Dth+FA495UyWIiOQJ34b84+EmyksNZ89PpR7yACWlcOzX4Mlfhetvw6VvwqW/9e7/6jfdPuVQvw+qm6GmBWqaoXqPd19WBaXl3q2kfMvjMu/e7PLBypidCpPYd5f9k9lXsqvhADTcl+taZMzqmmV5dY3FlTWWV91txbJmYzcA737NWtbWwGKxbjt2H9svUzLdO3s0VEd9ZXlG38O3IV9dUcZjDzRy9vw0z6XzwMZ4SwcG2+CDJ7yyWzPwzste4F9/B25Nw/xlb+Kzm1OwupTOGkgx+PHn4Md/Pde12GRheZXorWVmby0xe3OJWff4+u1lbiyucHNxhZuLq9790sb2jcUVFpZXWXJhvrSy5sJZhno/xuPhPRl9D9+GPHgLfP/Hr73O1PwiLXWBzL1RdRMc/YR328paWLoBK4te2K8uw9qKd7+6BGvLsLpyhz8EO/xv2LV5sUv5jvsns69kXdOhrLzN4soqU/OLXJ1bZGp+gatzi7y/fr/I9Pwi0VteoN++w/Wt8lJDTaCMmooyagKl649b6gLUVJRRWVFKRWkJgbISyku9W0VZCeWlhoqyEipKSygrLaG0BErcJ8wSY9zNa1sZt23wRjub2ONdP5GmRyYP/2CoPnMHd3wd8k+5kH/pzWk+9SMHclMJYyBQ591EsmRxZZX3XVC/P7fA+/OLXI27n3L3s7e2fwektMTQUhugtT5AqKGSh/bV01hdTmNNBcHqcpqqKwhWV9BYU05jdQUNVeVUlpfm4CwlEb4O+Uf2e7+cL5yfyl3Ii6TR2ppl+uYi788tcuX6Alddq/uqe3zluhfkMze3fzIsLTHsrQuwty7A/U3VPPZAI631leytC3j39QH21lXSVFNBaYmuy/iFr0O+pMTw5JEWzp6fxlqLyfDHOpF7Za1lbmGF9+e80L4yt+C1vOcW3OONFvjKlg5tY6C5NkCovpL7GqvWw7u1PsDeuBBvqq6gROFddDIS8m5VqCgQiS0NmCvHjjTz5e++x+tX57PS/yUSEwvuqflFpm+42/wiUzcWmZ5fWi/znl9iaXVt2zHqKssI1VfSWl9Je3vzehfK3rpKQg1ekLfUBigr9e1XXiRFaQ95Y0wEvFWjjDFhY0wkm0sAbhWbevjsG9MKeUnJwvKqG1myvH4xMn6kiVe2xMzNpTsGd2mJoammgubaAC11Adr31tJSG6C5NsDe+oBrhXsBXl3h6w/bkgWZ+A3qAc64x5NAF5CzkN/XUMWRvbW8cH6KZ58K56oakmPWWm4vrzK/sML8wrK7X1nfvrG4wlzscax8cXlToN9pdEltoIxgtXchsqmmgsN762iuq1gP75Y67765toJGdZtIFmUi5INA/GTsmR0EmoBjR1r4k2+9xcLyqkYBFJi1Ncut5VVuLKxwY9EL59iY7NjjG+5+Pu7xjYUV5lx4x/ZbTWBwdm2gjLrK2K2cfW50SVNNuTeipLpifaRJ7HFDdTmBMv1eSX7KyWdBY0wv0AvQ1taW8fc7drSZP3zpAi9fnOHYkZaMv5/szFrL9dvLTM17/dAzroV8Pa7r47q7j95aJnrba0Un8sWZQFkJdZVl1AbKqK30xmjf11gdF9heaMdCvL6y3NvflddVllFbUaYWtvhOJkI+CjS5x0Hg2tYdrLWDwCBAZ2dnxr998/ihJipKSzh7flohnwFLK2tM3/DGZE/Ne1+m8e4X1++n3eOd+qgBqitKaaz2xmEHq8vZF6yisbqcYFUF9VVl1AbKvUB2IV4b2LjVBMqoKNOFR5GdZCLkh4BO9zgMjGbgPZJSXVFG58FGXnhjin/zkw/lujoF5dbSCleue+Ovr8wtcNk9vnzdG+J3+frCroumN9VUsLfOXVxsqaGlzhuH7fVPV7CnJqDuDpEMS3vIW2vHjTGdxpguIJrLkTXxjh1pof+rr/H+3AJ76ytzXZ28cGtphXdmb/Ne9Pa24Pa2bzO3sLLtdQ1VXl91qKGSR/bXu6F8leuBHrvIWK5hfSI5l5E+edcdk1eOHWmm/6vw4pvT/FzEv7P7bXX91jKvXZnjwvRN3p69xaWZ27w9c4t3Zm8xfWPztyJjX6rZ11DJA3uqeTzcRKihkn0uxPc1VBGqr6SqQq1ukUJRNINwH95Xz56aCs6e92/Ivxu9zdjFGV55b47Xrszz+pU5rs5tdKWUlhgOBKu4v6mKrodaub+pmvsaq7ivsYpQQxV769T6FvGbogn5khLDE4ebOXt+mrU164tRFFPzi3z91av87eQ1xi7O8m70NgAVZSUcbqnlifZmPhCq42iojsMttexrqNQ3I0WKTNGEPHhdNv/3u+/x2pV5Ht5fmN9+fevaTb72yhW+9spVxi/NYi3srQvwkUNNPHvsEJ0Hm3gwVKcwFxGg6ELeGz559vxUQYX8/MIyf/6d9xh6+W2+/+51wOt++uzTR/lERysfaK3T5GsisqOiCvlQQyVHW2s5e36avo+357o6d3Vx+iZ/8OIkf3buXW4vr/JgqI7P/dRDfOKREPc3Vee6eiJSAIoq5MFrzf/vb+b3FAc/ePc6v/c3b/LVH1yhrKSET354P7/wsQf40H0NarGLSFKKMOSb+R8vXuDvLszw1NH8+vbrxemb/NZfvs7/+95l6irL+PTH2/mnP3ZQ4/pF5J4VXcg/fmiPm+JgKm9CfvbmEr9z5g3+9O8uUV5awi//xGF6Px7O+CruIuJ/RRfyVRWlfORQI2fPT+e6KqytWYbPvc0XvvIa8wsr/OOPtvGvnj7M3jq13EUkPYou5MHrl//CV3I7xcEP35vjc1/6PuOXonz0YBP//lMdfCCkxb5FJL2KcjD1+mpROWjNL62s8Vtfe52f+W8v8ta1W/z2Mx9iqO9jCngRyYiibMk/FKqnubaCs+en+PnHsjfFwauX5/jV09/l1ctzdD92H5/7qYcIVldk7f1FpPgUZciXlBiePNzMi29mZ4qDldU1Bs9O8p/OvEFDVQVf/KVOjj/cmtH3FBGBIg158Prlv/Sd93j1yhyP7G/I2PtcunaLzw59m/FLUX7y0RC/+alHaapR611EsqOIQ36jXz5TIf+lb7/L5770A4yB//KPPsw//NB+fZlJRLKqKC+8Auytr+TBUB1nz0+l/djzC8v86tB3+OzQd3gwVMdXfuUYn/zwAQW8iGRd0bbkAZ483Mwf/+1b3F5aTdtCGN95O8pn/vTbvDN7i892HeGXf+KwZoQUkZxJOX2MMb3u1h9X1h97LtXjZ9Kxoy0sra7xrQvb1hpP2vLqGr/712/S/d+/weqaZajvR/ls11EFvIjkVEoJ5NZxHXXL/YXdNkCvMWYCmEy1gpn00YNNVJSVpDxe/uWLM/z0f32R//i11/nEIyH+4jPH+MjBpjTVUkTk3qXaXRN2t0G8QA+78mettSMpHjvjqipK+ejBpnvul5+5ucTn/+JVhs+9w4FgFYO/+Bh/75FQmmspInLvUgr5LQt2R4Ah9zjWqo9Ya5/f+jrXjdML0NbWlkoVUnbsSDOf/8pr/PqffY+H99fz0L56PhCqu+PkYGtrltNjb/OFr77GjYUVPv3xdj7z9GGqK4r6EoeI5CFjrU39IMZEgB5r7akt5f3AGWvt6G6v7ezstGNjYynX4V5dvn6bfz38Pb7/7nWu315eLz8QrOKhffU8tK+OB0Pe/QN7anj9yvzGnDOHmvjNT3VwtFVTEohIdhljzllrO++2312bnrtcPJ3cEtxdsYB3+8+47pprbHTh5KV9DVX8n3/xONZarswt8NrleX54eY7Xrszz2uU5/vr191ld8/4QVpaXsLxqCVaV89vPfIifi2hYpIjkt7uG/JYumW2MMb2xLhnXRTPGxgXXdmAg1UpmgzGGfQ1V7Guo4ice3LtevrC8ypvv3+DVy3O8enmeQHkJfU+FNeeMiBSElDqRXaj3G2NOAU3AM9bacTekcgaYsNaOp6OiuVJZXkrHgQY6DmRu6gMRkUxJ9cLrKNC4Q/kdW/8iIpId+qaOiIiPKeRFRHxMIS8i4mMKeRERH1PIi4j4mEJeRMTHFPIiIj6WlrlrUqqAMVPAW0ADcD3uqfjt3R43A6nNE7z9mKnuu9vzdzq/ncqK4ZwT/ZkXyjknUlao55zoz3inMp3zzuec6vk+YK1tuete1tq8uAGDu23f4fHHmL/LAAAFXElEQVRYJt47lX13e/5O51es55zEz7wgzjmRskI950R/xjrnxM85Xed7t1s+ddd8+Q7buz3O1Hunsu9uz9/p/HYqK4ZzTvRnni6ZPudEygr1nBP9Ge9UpnPO/DnvKufdNakwxozZBKba9BOdc3HQOftfts43n1ry96IY58jRORcHnbP/ZeV8C7olH88YEwQ6gSDeurPRHFcpK9yCLU32Dguz+IkxJoz3M+4CRqy1eb2OcDrE/W5H8Ppzi+J3G8AYM2Ct7ct1PbLB/W7HFlpK2x+AgmnJuzCL3+42xnQZY066oudc0I3ilhYsdAmcM0APXuj5QgLnHMFbr2AU6M52/TIhgXPuxFunIUqeL8KTqER+t7fuU+gS/P/8bDoDHgok5N289cNx2xFYn+o46raDriyKt1hJQUvwnGFjXd2Cl8g5W2tH3M+4C8j7xeLvJsFzjn1KC9oCX58BEjtn16qNulvBS/D/8wze+tjd7tNbWhREyLt/iPiP5T1s/PAn8f7DR40xQfePM5HlKqZdgufsK4mes/sPM+qHrppEztmtvhYFxre0+gpSgj/nWLdc2AV+QUvwnDvdH/FJ0tgbkdKiITkUxPurF7MHb5nBWPD58QLOTucM3jnvMcb48TrEtnN2AX8KmDTGnLHeWsJ+stPPedSddxgffHrZwbZzttaOugabb7oit9jp5zzpWvSdpPHnXKghv41r1RV8yy5Z1q2vWyzirrsUDT900dwL12g5nut6ZEvcJ9O0/rwLortmB1G8NWXB+4t4LYd1yRads87Zr3TOGTznQg35ITZGGYQpjpadzlnn7Fc65wyec0GEvDGmG+h09+sfX10/ZdSPH2d1zjpndM6+kctz9s2XoUREZLuCaMmLiMi9UciLiPiYQl5ExMcU8iIiPqaQFxHxMYW8iIiPKeRFRHxMIS9Z5ebPtsYYX8z5nyhjzEljzJlc10OKj0JeJDs+gg+nh5b855tZKEXymbX2mVzXQYqTWvIiIj6mkBcR8TGFvOQlY0zYGDNsjJl1F2rP7LQMnDGm1xgz4fYZcLcJY8yse74/7nHsuWAix4+91q05esbtd26nBaYTrIeN2z+t7y+yG4W85Ks+d/8s3upATcC5+B3ctK0DQD/wDN6FzS73+Okt+57DW1ZtwK04dNfjO0G8BZgH3GvCxC3InGw9kjm/RN9f5I6stbrplrUbXvhZoDfJ10Xc67rjys4Bw3Hb3d6v9KbX9bvXnbmH4/ffoSx4L/XIxPvrptudbmrJS0GwG4sqxHdpbF0MGQC3APRWfTuU3e34MfGr9kzs8Hwy9cjE+4vsSiEvecsY0+36rc/F+ra3GAVOuD7rIF6Qj1uvO2YTu7FIcjLHj7122/HutR4Zen+RXWmcvOQl9+3QMF73xJgr3tpnPYHXgo6Vj7N7H/i9HD9RSdcjze8vsiuFvOQdN8qkC3jGWjviynbq+ugB+qy1o8aYYKIt3iSOn6ik6pGB9xfZlUJecuWx2KLGW4xaayeNMVGg3xgDEAVO7bDvpNtnCMDtO26tHd1h33VJHD9RSdUjA+8vsiuFvORKr7ttdRyvj/tZ4It4wwXH8YYQbp37ZQZvJMumcePGmEng+E798HESOX6i7qUe6Xx/kV0Za+3d9xLJM8aYLuAM27s8Ol35iM3CfDH5Ug+R3Wh0jRSqMEAsWN3jqOsiGcUb1lhM9RDZkVryUpBca/kCXn/4AN4IlTBed08vXjfJHfvm/VQPkd0o5KVguYDtx+vLDuNdwBwFPh/35aKiqYfIThTyIiI+pj55EREfU8iLiPiYQl5ExMcU8iIiPqaQFxHxMYW8iIiP/X9z6/y1btKWuQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot([L(x, lam) for x, lam in zip(conv_x_da, conv_lam_da)], \n",
    "         label=\"DA\")\n",
    "plt.plot([L(x, lam) for x, lam in zip(conv_x_al, conv_lam_al)], \n",
    "         label=\"AL\")\n",
    "plt.legend(fontsize=20)\n",
    "plt.xscale(\"log\")\n",
    "plt.xlabel(\"\\# iterations\", fontsize=20)\n",
    "plt.xlabel(\"Lagrangian\", fontsize=20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([1.e-02, 1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03, 1.e+04, 1.e+05,\n",
       "        1.e+06, 1.e+07]), <a list of 10 Text xticklabel objects>)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaoAAAEcCAYAAACWHPCeAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3Xl8VOd18PHf0b4hjQaQ2BEjDNgstiXhNbYBS45TO5sjm6RxmjSJkZsmbbZCeJtma1MH4jZ987aJJZzGrbMAwlmdeJFs8BYvINnYgG0Cw2Zss0lCCKH9ef+4d8RImkHbzNxZzvfzmc/M3PU8Gpgzz3KfK8YYlFJKqWiV5HQASiml1IVoolJKKRXVNFEppZSKapqolFJKRTVNVEoppaKaJiqllFJRTROVUkqpqKaJSimlVFTTRKWUUiqqpTgdQDyYNGmSKSoqGtO+Z8+eJTs7O7QBRTktc2LQMse/8Za3oaHhpDFm8nDbaaIKgaKiInbs2DGmfbdt28ayZctCG1CU0zInBi1z/BtveUXk0Ei206Y/pZRSUU0TlVJKqaimiUoppVRU00SllFIqqmmiUkopFdU0USmllIpqmqiUUkpFNb2OKgSOtXbw73V7A6805oL7nny7m0lHT3Px1FySkyQM0SmlVGzTRBUCx8908sMn/hx0vVwg/xgDv3jjWSakp1BWlM+VnolcOcfNoul5pCZrhVcppTRRhcDi6Xns+N4tY9r3V48+SfKU+bx4oIkXvafY+uYJALLSkimdnc+Vc9xcMWcil87MIz0lOZRhK6VUTNBE5TB3RhLLLpvOBy+bDsCJM528dKCJlw6c4sUDTdz7uNWkmJaSxOUzXVzpmchVc9xcPiufzDRNXEqp+KeJKspMnpDOLUumcsuSqQA0n+1i+8Emq8Z14BT/+eSf+aGB1GRhyQyXXeNyU1bkJiddP06lVPzRb7Yol5+dxk0Lp3DTwikAtHZ003CwmRcOnOJFbxPVT3v50bb9pCQJV3rc3HTJFCouKWSaK9PhyJVSKjQ0UcWY3IxUli8oYPmCAgDOdvbQeLiZZ/edpG73Mb75u91883e7WTIjj5suKeSmhVO4qCAHudCIDqWUimKaqGJcdnoK1100mesumsza913MvuNtPL7nXR7ffYx7H9/LvY/vpWhiFjcttGpaJbPydRi8UiqmaKKKM3MLcphbMJfPLZvLsdYO6vYc4/E9x/jpcweoedrLxOw0ls0vYGlRPqWz8ymenEOSJi6lVBTTRBXHCnMzuPOq2dx51WxaO7rZ9uYJHtv9Lk++cYyHGt8CIDcjhZLZ+ZTOshLXpTNdZOugDKVUFNFvpASRm5HKBy6dxgcunYYxhgMnz9JwqJnGw800HGpmm339VnKSMHtiFsWTc5hbkOP3nM2EjFSHS6GUSkSaqBKQiOCZnINncg63l80E4HR7Ny8faabxcAt73z3D/hNtbHvzON2956eAKsxNH5DAfK8Lc9N1sIZSKmw0USkA8rJSWTa/gGXzC/qXdff2caSpnX3H29h/4qz93MavG49yprOnf7uc9BSKJ2dTXJDDJVNzWTw9j4XT8/S6LqVUSOg3SQAiUmeMqXA6DqelJif117z8GWM4caaTfSfa2O+XxJ7bd5JfNR4FrPkN50zMZtH0PBZPz2PJjDwunekiI1Vn01BKjY4mKj8iUg54gHKnY4lmIkJBbgYFuRlcUzxpwLoTZzrZdfQ0rx09za6jp9lxsInf7XwbgLTkJJbMyGNKchdm6nFKZ+eTq/1eSqlhxF2iEpFKoMUYUx9gXQlQBnixEpLXfzvfaxGpjlC4cWfyhPQBFyQDnGzr5JXDLf1TQT1yuJuHvdtJErh4ai5XzHFz5ZyJXOVx48pKczB6pVQ0iqtEZdeINgC3B1jnAdb5N+mJSK2IeI0x3giGmXAm5aRTfkkh5ZcUAvBY/VZyihbbk+828cuXDvPT5w4iApdMzeWa4olcXTyRpUVuHWmolIqPRGUnoTVAA9AUZLMqYHBNqRpYR4DEpsInPUW4du4krp1rNRt29fSx860W/rTvFM97T/I/fzrEhmcOkJwkLJ6e15+4yma7dcZ4pRJQXCQqu0ZUBSAia4JsVsnQRLUDqAtjaGoE0lKSWFrkZmmRm7/nIjq6e2k81Mzz3lP8af8pauyJd1OThctn5nPpzLz+QRpFE7N1Zg2l4lxcJKrhiIgLq09qQG3LGNMiIoiIR5v/okdGajLXzJ3ENXMn8RWsiXe3H2ziea81Y/z/PH+Irp4+4PzQ+KJJ2RRNzMYzOZtZ7iym5GUwOSedFL1LslIxLyESFeAGKzEFWe/BGmCholB2esqAa7y6e/v487E2dh09za63T+M9cZYdB5v53c63MeevT0bE6h+bkptBYW46hbkZFOZmMCUvgym+57wMJqSn6AXLSkUxMf7/s+OAiOwHqvxH89mj/RqMMUO+jUTEABXGmHp7u3Ksfqv1QF2g0YP2fquAVQCFhYWlGzduHFO8bW1t5OTkDL9hHAlXmbt6DSfOGU6099HSaWjuMDR3Glrs5+aOPtq6h+6Xngz56UJ+hpCfkdT/emKmUJCVxORMIS15fIlMP+fEkGhlHm95ly9f3mCMKRtuu0SpUY2IMaYRaMRKUsNtWwPUAJSVlZlly5aN6Zzbtm1jrPvGKifL3NHdy/HWTt5t7eCd0+c41trBu6c7ebf1HO+e7uDg6Q5eereTnr6BP+Cm5GYwa2IWs91ZzJ6YxayJ2RRNzGL2xGzyMocfmaifc2JItDJHqrwJlahExHWB5j+VADJSk5k1MYtZE7OCbtPXZzjZ1slbLec4fKqdQ6faOdR0lsOn2tm29wQnznQO2H5idhpFk7KZYz+KJtrPk7LISkuo/2JKhUWi/C/y9T+5gf5EZQ+y8F+vFElJ52feKJmVP2R9e1cPR5rOcfDUWQ6ePMsB+/HMn0+wpeGtAdsW5qYzZ1I26V2dvCn7+5PZrIlZpKfoUHulRiIhEpU9us8LuAatcmPNYjG+RHVqHzxw65h2nd+RBpdMgYIF4wpBRU5WWgrzp0xg/pQJQ9ad7eyxE1g7B062ceBkOwdPnWX3sR6eeuuN/u2SBKa5MgfUwKxaWDYz8jNJ1dGKSvVLiERlq8eaPqnRb1mJvXx8jIG+3rHsSMHx7fCjq+CSD8L1/wBTFo07HOWc7PQUFk7LY+G0vAHLt23bxuVXXjugBnbwlPX8m1eOcqbj/Gz0SQKurDRcWam4s9JwZaXhzk7FlZVGdloK2enJZKenkJWWTE56Cll+y3zrs9JSSNbry1SciMdE5WZozQmsmStqsQdA2Krsx/hMugg+/ciYdn3h8d9xbfIr8GIN7PkNLLjVSljTLht3WCq65GWmculMF5fOHPjP0xjDqbNd/UnscFM7TWe7aGnvpulsF281t/Pa0S6a27v7rx8bicxUO3mlJ5OdlkKO/TorPYWctBSy01PIsd/7Xmf3L08hLzOV3MxUcjNS9Ho05ai4SFR2X9NarOuhXMA6EanAGl6+Bfqb/9aIyDpgu73tOqcv9O1Oy4Vl34BrvgAv3Acv/hjeeBjm3QzXr4YZpU6GpyJARJiUk86knHTKitwX3La7t4/2rl7au3o429nD2c5eznZZz+1dPbR19tDe2Ws9d/XQ1tlrb9fD2a4eTrZ1cfZUe/8+Z7t6GMkVKjnpKeRmpFiJKzOVPPuRm2E/Z6acf59lLTvdaeju7dNmTDVucZGo7JF8waZO8t/ON/w8+mTmw/K1cPXn4KUaeP6/4P4VUHwj3LAaZl3ldIQqCqQmJ5GXmTSiIfEj0ddn6OixEttZv6R2pqOH1o5uWs91c/pcD6fPddPa0W09n+vmSFM7u89109phJcdg/n7rI+RmpODOTiM/Ow13lv2cnUa+3aRpPZ9fn5eZqtNiqQHiIlHFlYw8q+nvyrth+0/gT/8P/vu9MOd6uGENFL3H6QhVHElKErLSrH4uho4NGZGe3j5aO3rspGYltJb2bl7auZtJ04pobu+i6WwXze1dvNvawevvtHLqbBedQZoxkwTc2ekUTEinMDedggnWzCKTczMonJBOgT3TyKScdK2tJQhNVNEqfQK854twxV2w46fwpx/CA7fArGusGpZnmTVHkFIOS0lOwm3XkvxNaN7LsmUXBd3vXFcvTe1dNJ89n8ia7NcnznRy/Ewnx1o72PV2KyfbOoc0UYpAwYR0ZuZnMdOdxYz8TGbm28/2fI+ayOKDJqpol5YN13weln4GGv8Xnv0PePBDMGOpVcOaW64JS8WkzLRkpqdlMt2VOey2Pb19nDrbxbHWDo63Wkns3dYO3m45x5Gmdl460MRvXzmH/4QiyUnCLHcWxZNzmFtw/lE8OVvvcxZjNFHFitRMuLIKSj8FL/8Mnv0B/LwSpl1uDbqY/z5NWCpupSQn9U8qHEx3bx/vnu7gSFM7bzWf43BTO/tPtLHveBtP7T1Od+/5LDYlN4OF03JZaN8uZtH0XKbkZujkxFFKE1WsSUm3aleXfwJe3QjP/Bts/BhMWQw3flNrWCphpSYnMdNtNQMO1tPbx+GmdvYdb2Pfibb+2fe3vnm8vxY2MTuNRdPzKJ2dz9IiN5fNdOmNOqOEJqpYlZIGJX8Fl/4lvFYLT62zalhzroeK71g1LaUUYNXIPJNz8EzO4Sa/5e1dPbz+zhl2v32aXUdPs/PIaX5QvxdjIDVZWDQ9j6VFbq72TOQqz0RNXA7RRBXrklPgso/Boo9Aw0+thFWzDBbfDiu+DvlFTkeoVNTKSkuhdHY+pbPPz+l4ur2bhsNNvHSgmR0Hm3jguYPUPO0lLSWJK+e4uWHeZJbNL6B4crY2FUaIJqp4kZJm9WFd+lF47v/C8z+CPb+FK1bBdV+BrAtfSKqUsuRlpbJiQSErFhQC1q1hXjrQxFN7T/DU3hP8yx9e51/+8DpFE7P4i8VTuWXJVC6ZmqtJK4w0UcWbjDy48Ruw9LOw9V/hhR9B44Nw3ZetRJY6/AgrpdR5GanJXD9vMtfPm8w/AUea2nlq7wke2/0u1U97+dG2/RRNzOKWJVOZOYoprtTIaaKKV7nT4IP/CVd9Duq/BfXfhJc2WM2BS+6AJG1rV2osZrqzuPOq2dx51Wyaznbx2O53+eNr73DfU156+wy/eut5VpbN5C8WT9U+rRDRq+HiXeEl8PHN8MnfQ85k+M3dsGEFnNjrdGRKxTx3dhofu2IWD37mSp7/2goq56VyvLWDr9Tu5Irv1vMvD+/haMs5p8OMeZqoEsWc6+GzT8Jt90PLYai5wbqAeCQzkiqlhlWQm8GtnjS2fnUZG1ddxQ3zJ/PTPx3k+vVb+cIvX+a1t047HWLM0qa/RJKUBEtuh6Jr4ddV8LsvwL4n4P3/FzID3RlFKTVaIsJV9nD2oy3n+OmzB9i4/Qi/3/k2Ny4o4EsV81g0PW/4A6l+WqNKRLnT4BO/sS4Qfv33cN91cPhFp6NSKu5Md2Xy9Vsv4fm1K/iH985n+8Embv1/z/K5nzew7/gZp8OLGZqoElVSsjUS8DOPWzNZ/PR98NT6Md6pWCl1IRMyUvnb5XN5Zs0K/m7FXJ568wQ3/8cz/PPDe2jt6HY6vKiniSrRzSiDu5+FRbfB1u/C/7wfWt92Oiql4lJeZipfvmk+T69ezu1lM/jv5w6w4t6nqN1xBKP9xUFpovIjIiUiskpEVotIrYh4nI4pIjJy4bYN8KH74J2dcH85HH/D6aiUilsTc9K557Yl/PZvr2WmO5N/2PIqd/7kRd5qbnc6tKikicpm386+zBhTY4xZD1QDdQ6HFTki1lRMf/0I9PVYN2vUfiulwmrJDBcP3X0N3/3wIl453MJ7f/A0P3/xkNauBomJRCUilSJSHmSdrxZU7nse42k8DLyd/Q7AYyewxDF1idVvlTUR/veD8OYjTkekVFxLShI+fuVsHv3i9Vw2y8U//noXn35gO81nu5wOLWpEfaKyE8+GIOs8wDq7FlRvjKkBqsbSZGeMaQQq/BaVAS3GmJaxxB3T8ousZFWwADZ+3Lr/lVIqrGa6s/jZZ67kW++/hGf3neSWHz5Dw6Fmp8OKClGbqETEIyLVWDWdpiCbVWE10fmrBtaN5ZzGGO+gY981luPEhexJ8MmHwXMD/PZvrfteaXOEUmElInzq2jk89DfXkJwsrKx+np8+dyDhmwKj9oJfO2lUAYjImiCbVTI0Ue3Ar29JRFYBxRc4VZ0xpt5/gb3PJmPMltHGHVfSc+Bjm6xE9cR3oPucNVegUiqslsxw8fAXruMrm3fy7d/vYf+JNr71/oWkJEdt3SKsojZRDcfuOxpS2zLGtIgIIuIxxnjt5sDRHLcc8A5OXgkrJQ0+XA2pGfD09yFrElx1t9NRKRX38jJTqflEKesee4Pqp7wcOtXOf328hNyMVKdDi7hYTs9usBJTkPWj7qcSkRKgyZekRKRy7OHFkaQkuOUHsOBWeHQNvLrZ6YiUSghJScLa913M+o8s4fn9p7jjvuc5cabT6bAiTmKh7VNE9gNV/rUcO6k0GGOG3K1MRAxQMZpakT0AowHwH+XnNcYEbDa0mwdXARQWFpZu3LhxpKcaoK2tjZycnDHtG2lJvV0sfu075J3ew65F/0jTxNIxHSeWyhwqWubEEM4y7zrZyw9f7sCdLvzD0gwmZjpfzxhveZcvX95gjCkbdkNjTNQ/gP1A+aBlJVb4Abc3g7cP56O0tNSM1datW8e8ryPOnTbmvuuM+edCYw6/OKZDxFyZQ0DLnBjCXebtB06ZRd941FxzzxPmwIm2sJ5rJMZbXmCHGcF3rPMpeZwS7jonp2Xkwscfsia2/fntcPLPTkekVMIoK3Lzy1VXca67l7/c8ELCzGQRy4nKN5Tc7b/QL3F5UeGRMxk+8WtrYttNn4DONqcjUiphLJqex4OfuYIznT3cef+LHD/T4XRIYRezicpYgyi8DOxTAitxtZiB10SpUMufDZX/DSfftO5rFQN9nUrFi4XT8njgr6/g+JlOPnH/S3E/i0XMJipbPdYMEv5K7OUq3DzLYMU/we5fwYv3OR2NUgmldHY+9/9VGQdOnqXqwQY6e+L3Fj3jTlQikhuKQIbhZmjNCay5+W4ftKyKgXP2qXB6z5dg/i3w+Nfh0J+cjkaphHLN3El8//YlvHSwibUPvRa3M1iMOVGJyBwR2Qe0iMgpEfmw37qPiMi4etlFxCUi60SkFitJrRORav9rm+zmvzX2dpUishpr7j9t9osUEfjwj8E1C2o/BWeOOR2RUgnlg5dN58sV8/jVy0f5zyf3OR1OWIxnZorvYdVedgBLgVUiIsaYXxljHrITzJj5ktAItmsEGsdzLjVOGXmw8mewYQX8/u/gYxutBKaUiogvrJjLwZNn+be6vcybMoH3LpzidEghNZ6mvx3GmCeMMaeNNXP5HcBEEVlhr4/POqgKrHAh3PgN2PsovPILp6NRKqGICPd8ZDFLZuTx1dqdHDp11umQQmo8iWrI1EXGmA2AiMjl4ziuilVX/g3MvhYe/RqcfsvpaJRKKOkpyfzXX5aQJMLdP2ukozt+BleMq0YlIneJyJ9FpMi30BjzhP3y9HgCUzEoKQk++F/Q12vNuB6nHbtKRauZ7ix+sPJSXn+nlW/+drfT4YTMmBOVMeZlYDNwhzHmYIB1Y5sITsU29xy46Z/Buw12/MTpaJRKOCsWFPK5ZcVs2nGER3e943Q4ITGu4el2/9TLQdYdGM+xVQwr+zR4lkPdN6E1Pv6jKBVLvlQxj8XT81j7q9fiYuaKWL/gV0UjEbj136G3G+r+yelolEo4qclJ/GDlpbR39fK1OLi+KuDwdBG5D2vGh/GWToD9xpiV4zyOijVuD1z79/D0eij9FBS9x+mIlEoocwsmsPZ9C/jW7/ewafsRPnrFLKdDGrOAicoYo7dwVeP3ni/BqxvhD1+Fu5+B5MS7M6lSTvqrq4t4bPcxvvvH11lxcQEFEzKcDmlMtOlPhU9aFtz8PTjxOrxU43Q0SiWcpCThux9eRGdPH9/5/R6nwxkzTVQqvOb/BVx0E2y9B86edDoapRKOZ3IOX1g+l4dffYetbxx3OpwxCdZH9T1gTojO4TXGrA3RsVSsEYGb/gV+dBU8829w8z1OR6RUwqm6oZjf7Xybr/9mF3Vfvp6stPHMnhd5waK9h0E3JByHphAdR8WqyfPhso/D9vvhqr9xOhqlEk5aShLf/fBi7qh+npqnvXyxfJ7TIY1KsMEUp9GZJVQoLVsLr9XC1n+F/I86HY1SCeeKOW5uWTKV+57az8qlM5mal+l0SCOmfVQqMvKmwxWrYOdGstsOOh2NUgnpazcvoM/A+kffdDqUURlVohKRXBHZLCK99mO7iHwlXMGpOPOeL0FGLnMO/MzpSJRKSDPdWdx13Rx+/fJRXj7c7HQ4IzbaGtV6YDtwB/A14ADwffvGiZeGOrhIs2/AaESkWUTqRMTjdExxJcsNV3+BSae2wzs7nY5GqYT0N8vmMnlCOv/6x9djZsaK0SaqBmPM940xD9nPdxhjkrBuoLjFfxb1GLXfGCPGmHxjTIXeKTgMrriLnuQsawSgUirictJT+LsbL2L7wWae2nvC6XBGJGiiEpEfi8iHRSTXb3HA9GuM2YJ1l9+qEMfni6VSRMqDrCsRkVUiUu57DkcMKkQyXRydfgvs+R0cf8PpaJRKSCvLZjIjP5N/e3xvTNSqLlSjqgIeAprte079GHCJSMALYexbx+8PdYB24tkQZJ0HWGeMqbHvMlwDVI2jyc7lS4p2M6BrrHGr4N6a8QFI1VqVUk5JS0ni72+8iNeOnuax3cecDmdYF0pU9Xaz3lKgBijG6qNa40tcIvJZEbkMwG72C9k9qETEIyLVgIfg12JVAdWDllUD68Z42hpjzBZjTD2wCXhiuB3U6HWn5cLST8OuLXAq5L9tlFIj8OHLp+OZnM2/171Jb19016oudHlyFYAxphFoBL4PYN9mvhwrga3HqoUYe5vbQxWY3T9UZZ9zTZDNKhmaqHYAdb43IrIKK8kGU2cnJl+t0Hf+RrtZ0eW/XIXI1Z+HF6vhxfvgL77vdDRKJZyU5CS+VD6PL/zyZR7Z9Q63LpnmdEhBBU1UwW58aN8osf9miSKSZ18gHFF2s9yQ2pYxpkVEEBGPMcZrNweO5HglwAZjzIBaoSapMJkwBRZVwss/g+X/BzLznY5IqYTzF4un8oO6vfx4235uWTwVEXE6pIDGfcGvE0nK5rbPHyyRjLafyotf7czuG9syttDUiFz9Oehuh4b/cToSpRJScpJw9w3F7H67lWf+HL2TRsfWzIQDhXSgg10T89pNhWA1F94VbHt7u1UAhYWFbNu2bUznbWtrG/O+scq/zJe6lpD5zA95sWsRJimW/zleWKJ/zokiFsuc32fITxf+9Tc76LtidNMqRay8xpiof2CNJiwftKzECj/g9mbw9uF8lJaWmrHaunXrmPeNVQPK/Oajxnwz15hXax2LJxIS/nNOELFa5g1P7zez1zxsGg41jWq/8ZYX2GFG8B0b83P96RDyGDe3AtzF8FLAKxCUUhHwsStmkZeZyn3bonMUbiwnKt+sEQNuR+KXuHRWiViQlARln4YjL8Cx3U5Ho1RCyk5P4c6rZlH3+jGONLU7Hc4QMZuojDWIwsvQvio30GJ0+qPYcdlfQnI67PhvpyNRKmHdedVskkR48IVDTocyxJgTlYjkichXQxnMGNQDZYOWldjLVazIcsPCD8POTdDZ5nQ0SiWkqXmZ3LxwChtfOkx7V4/T4QwwnhqVG+ui30hwE3iU3xqGXmRcZS9XsaTs09B1xpqtQinliE9dW0RrRw+/efltp0MZIGqb/kTEZc+3V4uVpNaJSLWIVPq2sZv/1tjbVYrIaqy5/7TZL9bMvAIKFkLDA05HolTCKpudzyVTc3ngTweiarLaqL1wxZeERrCdb4onFctE4PI74bG1cPx1KLjY6YiUSjgiwqeuLWL1lld5wdvE1cUTnQ4JiOIalUpAS+6ApBRrWiWllCPev2QaEzJS2LT9sNOh9NNEpaJH9iSYdzO8uhl6u52ORqmElJmWzIcum84fd73L6fbo+H+oiUpFl8s+DmePwz4duKmUU1YunUlXTx+/3XnU6VAATVQq2lxUAdmTYecvnY5EqYS1aHoeC6fl8suXjkTFoApNVCq6JKfCJR+CvY/rNVVKOeijS2fy+jut7Dra6nQomqhUFFp0G/Scg72POh2JUgnrA5dNJz0liY1RMKhCE5WKPjOvggnTYNevnI5EqYSVl5nKzYum8PCr79DV0+doLONNVNF5O0gV25KSYOGHYF8ddDh1X06l1Icum87pc91se/O4o3GMOVEZ61b1OlWRCo+Ft0FvF7zxR6cjUSphveeiSbiz0/jtK85OqTSuGpWdrJQKvRllkDcLdmvzn1JOSU1O4tYlU6l//RhnOpy7pkr7qFR0ErGa//Y/Ce1NTkejVML64GXT6ezp47HdxxyLQROVil6LboO+HnjjYacjUSphlcxyMdOdyW9fce7iX01UKnpNvQzy5+joP6UcJCJ88NLpPLfvJMfPdDgSgyYqFb1ErFrVgafh7Emno1EqYb3/0mn0GRxr/ht1ohKR3HAEolRAF38ATC/sfczpSJRKWPMKc/BMyubRXe84cv5RJSoR2Qw0i0jRoOUrQhiTUudNvdS6+HfvI05HolTCEhFuXjSFF7xNNJ/tivj5R1ujqgPuMMYcHLS8QUS+Guu1LREpEZFV9mO1iHicjinhicC898L+rdDT6XQ0SiWs9y2aSm+foe71yDf/jTZRubCS1QDGmNPGmHuBO0ISlXPKjTE19mM9ekFzdJj/Puhqg4PPOh2JUglr0fRcprsyeXTXuxE/96gSlTHm+0CtiNwjIsvDFNMQIlIpIuVB1vlqQeW+53GcqkpEXOPYX4XDnOshJVMnqVXKQb7mv2f/fDLiF/+Oto/qPqz5/SqAJ0SkV0S2i8iPReQeoDTUAdqJZ0OQdR5gnV0DqjfG1GAlm7E22a0DDvia/9AaVXRIzQTPMnjzUYiCe+Molajet2gKXb19PPlGZOf+G23T335jzE3GmDJjTBIlFSgaAAAeiElEQVTwXuAJYClQTgi/2EXEIyLVgAcINjVBFVA9aFk1VsIZNTvR3WMftwpwj+U4Kgzm3wynD8PxPU5HolTCKpmVT8GEdB7bHdnmv5RRbt/i/8YYUw/UA4jI5Vh9VPeHIjBjjBcrWSAiwRJgJUMT1Q78+tHsmlHxBU5VZ5cDEVlt902tt/erG2ZfFSnzbrae9z4KhQudjUWpBJWUJNx4cQEP74zsrT9Gm6jqReSzxphAyag5FAGNlN2XNKS2ZYxpERFExGOM8dq1pJEcrxxo9DtOjYgUi0iJMabxAruqSJgwBaZdbjX/XfcVp6NRKmEtn1/AL186wo6DkZuDc7SDKQ5gDab4rP9QdBGZA3gJQx/VBbjtmFqCrB9tP1UTUDJ4oSapKDLvZnhrO7SdcDoSpRLWtXMnkZacFNF+KjEh6pwWkRsBbzhu/SEi+4EqXxOdvawEaDDGDLl5o4gYoMJ/+xGep5Lz/VIuoD5YorKbBlcBFBYWlm7cuHE0p+rX1tZGTk7OmPaNVWMtc86ZfZQ1fIXXF3yRY1MiNug0JPRzTgyJUuZ7d3Rwsr2Pr5f0jau8y5cvbzDGlA233Wib/oIyxjwRqmM5xRizZRTb1gA1AGVlZWbZsmVjOue2bdsY676xasxl7rseXv9XLk4/xsUx9jfTzzkxJEqZD6Ye4Fu/30ObZHFrBMob0klpReTHoTzeCM+p1z0liqQkmHODNUuFDlNXyjErFhQCsPNEb0TON65EJSK5InKZiKwQkc8S2ZkpvPbzgCHkfonLi4o/nmXQ9i6ceNPpSJRKWLMmZjG3IIdXT/RE5HxBm/5E5DKsJODBGqLtG2XnWza4JiNAxH7m2qP7vAHicAMt9vB2FW+K7b4p71YoWOBsLEolsBsXFHD/M17aOnvISQ9ZL1JAF6pRrce6Rqoa60LeKqyE1Yx1ke/LwN1YtaibgLn2skiqBwZ3xJXYy1U8cs0CdzF4tzkdiVIJbdn8AnoNNBwK/5VJQdOgMeYmexRcObDGGHPaf72I3GeM2TBoWbgShJuhNSewEmgt9qAGm29WCRWvPMvg1U3Q2w3JqU5Ho1RCKivK5/vXZ3LDvMlhP9cF+6iMMVuMMXcDt9tNgQNWB9j+a6EKTERcIrJORGqxktQ6Eam2k6fvfC3AGnu7ShFZjTX3nzb7xbPi5dZs6m9tdzoSpRJWanISk7Mic5P4ETUsGmPuF5E5IrLCGPNkuIOyz9nCCOYOtK9z0otyE0nRdSBJVvPf7GucjkYpFWYjTofGmAPGmCftUX5zwhmUUheU6YJpJdYwdaVU3Bt1vc0Y84o9+0RjrN/RV8UwzzI42gAdrU5HopQKszE3MNoDKSba11DlAojIbSGLTKkLmXMdmF448qLTkSilwmxcPWG+5kCgwr6pYsAbHCoVcjOWQlIKHHrO6UiUUmEWkiEbxpiHsIaI63RGKjLSsq3bfhz6k9ORKKXCLGRjC+3RdzE/Ma2KIbOvhaON0NXudCRKqTAK9SB4vdBWRc7sa6GvG47ucDoSpVQYhXSCpnDci0qpoGZdCQgcfA7mXO90NCpKdHZ20tTUxJkzZ+jtjczs3j55eXm8/vrrET2nk/zLm5yczIQJE3C73aSnp4f0POGdSVCpcMrIgymLdUCF6tfZ2cnhw4fJz8+nqKiI1NRURIbcWzVszpw5w4QJEyJ2Pqf5ymuMobu7m9bWVg4fPsysWbNCmqwCJip7BF8Z458NXYD9xpiV4zyOUoHNvhYafgo9XZCS5nQ0ymFNTU3k5+czadIkp0NJKCJCWlpa/9+9qamJqVOnhuz4AROVPb+fUtFv9jXw4o/h7ZftpkCVyM6cOUNRUZHTYSS03NxcDh48GNJEFZkZBZUKF99cf4eedTYOFRV6e3tJTdUZ9Z2Umpoa8r5BTVQqtmVPgokXwWGdoUJZItknpYYKx98/WB/V94BQTTzrNcasDdGxlBpq5hXw5iNgDOiXlFJxJ9iov3uwblYYCk0hOo5Sgc1YCq/8HJq8MLHY6WiUUiEWbDDFaeB0oHVKRZ2ZV1jPb23XRKVUHErYPioRqRvLOhWFJi+AtAlw5CWnI1HKUY2NjYjIkEd+fj5VVVW0tLQMe4zi4mJEhMbG6LkfbcIlKhEpF5FVQPlo1qkolpQM00vgLU1USgGsW7eO/fv3s3//fhoaGli7di2bN29mzpw5F0xAjY2NeL1eADZt2hSpcIcVNYlKRCpFJGCCEJESEVnlSyTBthsJY0y9MaZmtOtUlJt5BRzbDV1nnY5EKce5XC48Hg8ej4eSkhJWr17NgQMH8Hg83HjjjUH327RpEyUlJVRWVlJTEz1fhVGRqOzEE/BeViLiAdYZY2r8EkmVvVwpy4wrwPRZs6krpYZwuVxs2LCBlpYW1q9fH3CbmpoaVq5cSUVFBS0tLVHT/OdoohIRj4hUAx6Cjw6sAqoHLasG1oUzNhVjZpRZz9r8p1RQJSUllJeXs2bNmiHrGhsbaWlpobKykjvuuAOA6urBX73OcHRSWmOMF/vWICIy9C9nqWRootoB9A94sPuVLjTcq84YUz+OUFW0y3JbF/6+pbf8UOpCKioqqK+vx+v14vGcb5iqrq7uby4EK6lt3rw5KpJVVM+eLiIuAtS2jDEt9mgWjzHGq/1KCrCup9pXpxf+qoC+/fvd7Hm7Nazn6O3tJTk5ecz7XzItl2++f2EIIxrKl4gGJ6rNmzezatWq/vcrV65kzZo11NfXU17u7PiyqOijugA3WIkpyHrtp1LnTS+Bsyeg9ajTkSgVtVwuF8CAoer19fW0tLRQUVHRv6yyshKA2trayAYYQFTXqABXqA8oIiXYw89FZB1+zYIXWhfgOKuAVQCFhYVs27ZtTPG0tbWNed9YFa4yT2g1lAK7Hn+Qk5OvDvnxx0M/58jIy8vjzJkzAdd9edmssJ9/vDUqIGj8I3H2rDXqtaOjI+hxnn/+eQDmzZvXv80vfvELXC4XV155Zf+yyZMnU1RURE1NDffee2/AY/X29gY8T0dHR2g/e2NMVDyA/UD5oGUlVogBtzeDt3fqUVpaasZq69atY943VoWtzF3njPm225j6b4fn+OOgn3Nk7NmzJ+Ln9Nfa2uro+RsaGgxgqqurg25TWVlp7O/Vfi6Xy9jfqQEfdXV1AY8VrLwj/RyAHWYE37HR3vQH9PdVKXVhqRlQcIkOUVcqCK/Xy5YtW1i9enX/Ml+zX21tbf9Fwr5HXZ01Zs3pARXR3vTntZ/dQH+Dql/i8g7ZQyW2aZfDnt/qgAqlBmlpaeH222/H5XKxdu35G1pUV1fjcrn6+6T8+S4Y3rJlSyRDHSKqa1TGGkThZWhflRtoMdbwdqXOm3Y5dLRA80GnI1HKMS0tLXi9XrxeL42Njaxfv57S0lK8Xi9PPPFE/4AKgC1btgwY7TdYVVVV/3ZOiepEZasHygYtK7GXKzXQtMut57dfdjYOpRy0Zs0aiouLKS4uprS0lHvuuYfy8nIOHDhASUlJ/3a+5LNy5cqgx4qGi3+jqenPTeBRfmuAWsD/Wqkq+6HUQAWXQHIavN0Ii25zOhqlIqqkpMQ32GxEKisrh93e5XKN6pjh4Giisvua1mJdD+UC1olIBdaw8C3Qf3HvGnu4+HZ723Xa7KcCSkmDKYvh7VecjkQpFSJOT6HUglVjGm67RkCHcqmRmXY57NwEfX2QFAut20qpC9H/xSr+TFkCXWeg+YDTkSilQkATlYo/UxZbz8d2ORuHUiokNFGp+FNwMUgSvKuJSql4oIlKxZ/UTOuWH+++5nQkSqkQ0ESl4tOUxdr0p1Sc0ESl4tOURXD6CJxrdjoSpdQ4aaJS8ck3oEL7qZSKeZqoVHwq1JF/SsULTVQqPk0ohOzJOqBCqTigiUrFrymLNVEpFQc0Uan4VbgITrwBvd1OR6KUGgdNVCp+TVkMvV1wcq/TkSilxkETlYpfOvJPKYqLixERGhsDz+tdX1+PiFBTUxNwfTTQRKXi18SLIDkdjmk/lUpMjY2NeL3WHZE2bdrkcDRjp4lKxa/kFGvePx1QoRLUpk2bKCkpobKyMqprTMNJ2EQlInXDrHfuvssqdKYs0qY/lbBqampYuXIlFRUVtLS0BG3+i3YJl6hEpFxEVgHlF9oGWBW5qFTYFCyE9pPQdsLpSJSKqMbGRlpaWqisrOSOO+4AoLo6Nn9/R02iEpFKO0EEWlciIqt8SSbYdiNhjKk3xgStA4uIC2gCWsZ6DhVFCi62nk+87mwcSkVYdXU1Ho8Hj8eDy+WipKSEzZs3Ox3WmDh6K3ofO/FsAG4PsM4DrDPGVPgtqxURrzHGG4Zwyo0xW0QkDIdWEVdwifV8bA/Mud7ZWJSzHvla2PsrM3t7rL7RsZqyGN73vZDEsnnzZlatOt8wtHLlStasWUN9fT3l5WP+re8IR2tUIuKx+4I8WLWYQKqAwfXVamBdGOIpB+pDfVzloJwCyHTD8T1OR6JUxNTX19PS0kJFRf/veyorKwGora11Kqwxc7RGZdeIqgBEZE2QzSoZmqh2AP2DIew+p+ILnKrOGHPBBORr8jPGaJNfPBGxalXHtekv4YWopnIh586cYcKECWE/z3Bqa2txuVwDak6+ZsCampqY66uKiqa/YOzkMaS2ZYxpERFExGOM8V6oz2kUygG3iJTZ7112AqwPUxOjipSCi2HnRjDGSlxKxbnNmzfT0tJCsC6MWGv+i+pEBbjBSkxB1nuAkCQRY8wW//ciUh2iBKicVnAxdJ2B02+Ba6bT0SgVVr5mv9raWkpKSgas83q9VFRUUF1drYkqhFyhPqCIlGAPTReRdQxqFrRrcavs16uBLVqjinG+ARXHX9dEpeJedXU1Lperv0/Kn8fjoaSkhC1btgTYM3pFe6IKOWNMI9AIrA+yvsVeF3C9j90suAqgsLCQbdu2jSmetra2Me8bqyJd5pTuNt4D7H/hYY68nRax8/rTzzky8vLyOHPmTETP6a+3t9fR8wNs2bKFL37xi0Hj+OQnP0ljYyMPPvggH/rQh2hvbwfghRdeIDMzc8j2y5Ytw+UKXGcIVt6Ojo7QfvbGmKh4APuxhob7LysBDOAKsL0ZvL1Tj9LSUjNWW7duHfO+scqRMt+7wJgtn438eW36OUfGnj17In5Of62trY6ev7a21gCmoaEh6DbNzc0GMOXl5cYYY+rq6oz9fRrwUVdXF/RYwco70s8B2GFG8B0b7TUqX5ObG78LcO3mOf/1Sl3Y9BI42uB0FEqFVWVlpe+HfFAul2vANuXl5cPu47SomZkiEGM1w3kZ2lflBlqM9h2pkZpRBk374ewppyNRSo1SVCcqWz1QNmhZCXphrhqNGUut56M7nI3DKR2nrSH6ITlWK5zcZw33VyoCoqnpz03gUX5rgFrAf6h4lf1QamSmXQ5JKXDwWSi+EbrbISN35PsbA29th9a3ofF/4OMPQdKg33mdZwCB5FSQZGsqnb5eeP6/SOn2WNuc3AeZ+ZA9ceC+PV3QfAAmzx8+luOvQ9YkyJlsvd+/1ZoeKin5fKxvPmIdK6cA7plxft+8GXDgaZhzA/z6bnjPF+EPX4bST1ll+/Pj1nZf3QcZedbrl/8XUrNg6mXw46uHxlN0Hdzxv5CeC6ePQPc5OPgMmHnDl0WpEXA0Udl9TWuxrodyAetEpAJryPgW6L+4d409lHy7ve06bfZTo5KWbX2Z79wI+56wrqv6/A5ISR+67caPQ95MayYD30XCux6Chz5zfpsjL8Lsq6GvD9qOWV/wT3zbSkwdLTDraitZHXoWgKLpt0DvzfCfpdb+32gCBOq/AUcb4dBz1vLppdby00dgbjlc/XnImw6mD07+2Uq0T3zb2ja7ANpPgem1truowprL7p2d8FqQaXIeuMV6fsqegewPX7aeGx4YuN29c0f+tz34DKyfM2Tx/Ck3wvLlIz+OUkE4PYVSC1aNabjtfEPKlRq7iz8A+5+Es8et96/8HMo+bb1uPgi1n4Jb/g3eeNha1nHaSkIfr4Xdvx54rJ/eDMvWwrZ7Ap/r8PMD3s44+gf4Z79a1PeL4Vzz0P38B3y88nPrEYyvHADP/6f1iCKTTr7gdAgqTkRT059S4bXoNjj8Alz2MXh0LdR9Ew48A8lp8Krdf/P4N85vv/MX1vN33Nbz7GvP13wgeJIaiUBJKs6k9px1OgQVJ2JhMIVSoZGRB7dVg2cZFK+AzlbY/SurWW/mVdY2h56FpFTInQETpsF7/ZLRoo/AJx+GKUsCH/+SDw1d9rkXQ10KpRKOJiqVmGZfaz0vuBW+fhw+8xjMu9laNu+98OlHrMfVn4OST1pJa8EtMOc6WPUULP/6wOMtvA1ufwDK7H6sj/wEbv0PKFgAdz3Ja4v+D6w5CF/3u9PwjCvg+tXW6799aeCxBie9pBRY+bPz77MnBy6Xb3SjT+V/D/eXiDvRfk1QvAvH31+b/lRimltu9TEt/ez50Xtln7FGt634Orhmnd/21v+wBlT4ZqJOSoJrvgDzb4bCRQNnZH/feuu4OX6JZHoppyadsUb7AUy+2Lrj8F8/Yo3UW/hha4TeR38BG/8SLr/TGoyxbK21vO04TCi09l1wK3ifgn/YB9/KO3+Oa/4O/vRD+MRvrMEV35sFt91v1QILFsLT37fuyeW7L9e1fw+93bC40hpgcvmdMPVSa932n5wfZDEOuy/5KgvHfZTRSU5Opru7m7Q0Z6bKUtDd3U1ycnJIj6mJSiWmlDRY9rWBy+bdZD0GGzwMHSA1w7ob62DJKQOTVCB//UdriLnvTrCF9qS5C26Bb50+v13BAuvZl6QAPhpgcMWcG2DFP8FN/3x+2eDjVP7Eev2Hr8LLD0LFd86vn1468HhLPwO50+GXK88v+/wOOLnXSqQf+Yk1MOWBWyDTBbdtgHWzh4R1YvK1Qf4A4TNhwgRaW1uZNGlSxM+tLK2trSG/J5cmKqUiLcsNRSH8Ev/k70a+7S33Wo/hzL8ZPvUH6OmEuTdayyZdNDABfrbu/OvPN1h9gM0H4SflVvKUyPcsuN1uDh8+DEBubi6pqalB78mkQscYQ3d3N62trTQ3NzNr1qzhdxoFTVRKxaov7rKuowqXoveMfNtJ9nVXWROtfreyT0Pjm+GJ6wLS09OZNWsWTU1NHDx4kN7e3oiev6Ojg4yMjIie00n+5U1OTmbChAnMmjWL9PQA1yeOgyYqpWKVa2b03V8rKQlW/KP9JvKJCqxkNXXqVKZOnRrxc2/bto3LL7884ud1SqTKq6P+lFJKRTVNVEoppaKaJiqllFJRTROVUkqpqKaJSimlVFTTRKWUUiqqaaJSSikV1TRRKaWUimqiMw2Pn4icAA4BeYDfHDMD3gd7PQk4GYIwBp97PNsGW3+h8gValghlHulnHitlHsmyWC3zSD/jQMu0zIHLPN7yzjbGDDM5JtYcTfoIzQOoCfb+Aq93hOPc49k22PoLlS9RyzyKzzwmyjySZbFa5pF+xlrmkZc5VOUd7qFNf6H1+wu8D/Y6XOcez7bB1l+ofIGWJUKZR/qZh0q4yzySZbFa5pF+xoGWaZnDX+agtOnPYSKywxhT5nQckaRlTgxa5vgXqfJqjcp5NU4H4AAtc2LQMse/iJRXa1RKKaWimtaoopSIeERktYiU288up2OKBBGpG36r+CAiJSKyyv58a0XE43RM4WaXuVxEKkWkOhHK7E9Eqp2OIRJEZJ2IGBFpFpG68X7Oej+q6FVrjCkFqx0Y2ADc7mxI4SMi5YAHKHc6lkiwf3iUGWNq7PflQB1Q7Ghg4fcEMMcY0yIibqAWKHU4poiwP+NVQJXTsUTAfmNMyG6trIkqjESkEmgxxtQHWFcClAFerC9or287e12Tb1v7P3XUf4GPtbwAfmWPqV+c4yizB1jD+Tb+HYBHRFzGmJbwRz524/mcsZOU/bpp8P7Rapxl9v0waQKi+rP1N94yh1QkxsAn4gOrZtAMlAdY5wHqBi2rBTz260qsGpX/+v1AidPlCkd5By03TpclUmUe9LocaHa6TJH6nO11dYGOE22PUJQZqLSfo/4zDkWZgdX291g5sA5wjSce7aMKMbtvqRrrwwz2i7EKGFxzqMb6QAHcYQov5EJU3pgSqjIbY7yDtr8rlHGGUig/Z1//K9aXXfh+hY9TqMpst4ZEbTn9hfBzrjHGbLE/301YTb5j53TmjucHVi0o0C+S/Qz9xeXCrk1g/RIZ/IulmSiuUY2nvIOWD1kWzY8QlXkV9i/uWHiEosx+5a4LZWzRVmb7dYnfupioUYXyc7bXG8ZRq9IaVYTZbdVDfq0Yu93eHh3jJUCtyhjTGIkYQ2mE5Y0roymz/Wvba4zZEtEgQ2wkZfarSflsBspj9d/ACD/ncqDMHt25CnDZr+O2zPbIzobB+5px9L3qYIrIc8MFPzSPMaZe5PyAGfsfdUw0HQQwbHmxEnM8GVGZfYNmfD9ARKQyhhPWSMoMMHHQshYzsAk0lozk//KAz1NEqo090jNGjeRz3oFf06D9Y2xc/641UUXeSK+Husv+9dkIlBDF/RfDGFF57S/tcvv1OqK8/2IYw5bZ/vHxBNYvbN9iL+P8D+2gYcts/wBz2TULgArgxvCGFVYjvrbRromssl+vBrbEaIIeyefcIiJev8+5mHF+f2miilL2r2xfU1+sfmGPmF951zsdSyTYX1L5TscRaYNqGLFcsxgVuwaynsT59x3S7yzto3JIosw04ZNo5QUtc6LQMoefJqrI81X3BwyW8PvgY7E54EISrbygZe6nZY47jpRZE1WE2U0AXoa29bqJ7Y7lgBKtvKBlHrRKyxxHnCqzJipn1GNNP+KvhPjti0q08oKW2UfLHH8iXmZNVOHlJvAomTUMnWC2yl4eyxKtvKBl9qdltmiZQ0zvRxVidlvtWqzrCSqxqsn1WMOtt/htVwKsBLbb2zbG4nDsRCsvaJnRMmuZI1xmTVRKKaWimjb9KaWUimqaqJRSSkU1TVRKKaWimiYqpZRSUU0TlVJKqaimiUoppVRU00SllFIqqmmiUkopFdU0USk1RiJSPeh9pX0305Huv1pE6kIfWejEQowq/mmiUmrsPIPeVzC62xwsxb6rcRSLhRhVnNNEpdQY2HOiDU5KntHc5sAYc7sxRvyXicg6+zb1ERXsvIFiVCrSNFEpNTblQIPvjZ24WsZzQPsYq7FumRAxTp1XqZHSRKXU2FQw8P47ZVgzSSulQkwTlVJjM7iZb3DiGpbd3Gbs17VAs72qVkSMiDQP2r5SRBrsdQ3+AzfsYzXbr6tFZL+IuETEIyK1ItJs71fn38Q33Hn9Y/RbVmIfxxfHqgDlag6wXcmg7SrtOE2g2JTy0USl1CgFaeYrMcY0juOwd2ElO7BuQlcMlPqdsxKoxUqGt2P1jw35YheRBqzaXbV92/CqQcd349dkOdx5B7PjaLDPXwFsAqoHj4DEuuFeLVBtH9djv/cdx/d+i30+X5kC3ahPJbgUpwNQKtrZX8L+CcENuAYN2y4b9H7daG4kZ4xpERFfDa0pwKCMDcB6Y4zvLqpb7KTkf2dVl72vL/Hgt72vLHcBDSJSaYzZMoLzDrYBqDHG+BJgvYg0YiXN2kFlXuO72Z6IFAOrRcRlJ1Bf7eoe+30jVtJSaghNVEoNw+9LGehPXGvsL1jsJjiPMaYmHOe3m8xcWF/0qwetbhr0vooLMMY0iggMHVo/kjjK7TgG1J6MMfUi0oJVK/JPVP6v9w86nG/dEyKyCagfZ41UxTFNVEqNnseXpGy3A+vCeT77uZShTY4DElWgGpHdXLfSPs54+oB8+waqdXkHH3vQ34jB60SkAuvvts6Os96/NqiUjyYqpYYxqOkvYLMf4LFrKj6javobRn9iGM11WgB2nB6sZLDDXtwQfI8RxeHBaqrz5/E7/ojYf59Su89vFbBORFYbY9aPMT4VpzRRKTUM/6Y/u+mt0ZeE7EEBawY3D47R4GY83/kb7aa1dZwf+OCLxxWs5mLHVg7c7tdXFGiwQsDzBuBLRFX4NTHaNTbf4IlRs+NfLyIrsWbCUGoATVRKjc7SQb/4y4GQzIVnN4cBVPme/ZrC7sIaPu4bSefCShZegvRLGWO8vgRnH6+F8wMvRnrewdtVYY3yc2ON+PPV1raMpgZpD2lfY5fF12xYwqD+L6VAh6crNVqDaySljPL6qWGsx0p+6/BrXrNrRKX2+euwRt95CZB4BrkLq7my1j5msFpPwPMOZg8YqeD8cPMqrBrl7cPEMdhmrFF+VX7HWR+uASkqtokxZvitlFIBm/lEpE4HACgVXlqjUmrkBjTzhWJ+P6XU8DRRKTVyg6dJCln/lFIqOE1USo1c06ARdh5C2z+llApA+6iUUkpFNa1RKaWUimqaqJRSSkU1TVRKKaWimiYqpZRSUU0TlVJKqaimiUoppVRU+//Hi/wLHI38NAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.semilogy([np.linalg.norm(A @ x - b) for x in conv_x_da], label=\"DA\")\n",
    "plt.semilogy([np.linalg.norm(A @ x - b) for x in conv_x_al], label=\"AL\")\n",
    "plt.legend(fontsize=20)\n",
    "plt.xscale(\"log\")\n",
    "plt.xlabel(\"\\# iterations\", fontsize=20)\n",
    "plt.ylabel(\"$\\|Ax - b\\|_2$\", fontsize=20)\n",
    "plt.grid(True)\n",
    "plt.yticks(fontsize=20)\n",
    "plt.xticks(fontsize=20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Существенная проблема\n",
    "\n",
    "- Слагаемое $\\|Ax - b\\|_2^2$ сделало лагранжиан НЕсепарабельным!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Сделаем его сепарабельным и получим ADMM\n",
    "\n",
    "Задача станет такой\n",
    "\n",
    "\\begin{align*}\n",
    "& \\min f(x) + I_{Ax = b} (z)\\\\\n",
    "\\text{s.t. } & x = z\n",
    "\\end{align*}\n",
    "\n",
    "Для неё модифицированный лагранжиан примет вид\n",
    "\n",
    "$$\n",
    "L_{\\rho}(x, z, \\lambda) = f(x) + I_{Ax = b} (z) + \\lambda^{\\top}(x - z) + \\frac{\\rho}{2}\\|x - z\\|_2^2\n",
    "$$\n",
    "\n",
    "- Теперь метод примет вид\n",
    "\n",
    "\\begin{align*}\n",
    "& x_{k+1} = \\arg\\min_x L_{\\rho}(x, z_k, \\lambda_k)\\\\\n",
    "& z_{k+1} = \\arg\\min_z L_{\\rho}(x_{k+1}, z, \\lambda_k) \\\\\n",
    "& \\lambda_{k+1} = \\lambda_k + \\rho (x_{k+1} - z_{k+1})\n",
    "\\end{align*}\n",
    "- Обновление $z$ эквивалентно $\\pi_{Ax = b}(x_{k+1} + \\frac{\\lambda_k}{\\rho})$ "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "def admm(update_x, update_z, rho0, x0, z0, lambda0, max_iter):\n",
    "    x = x0.copy()\n",
    "    z = z0.copy()\n",
    "    lam = lambda0.copy()\n",
    "    conv_x = [x]\n",
    "    conv_z = [z]\n",
    "    conv_lam = [lam]\n",
    "    rho = rho0\n",
    "    for i in range(max_iter):\n",
    "        x = update_x(x, z, lam, A, b)\n",
    "        z = update_z(x, z, lam, A, b)\n",
    "        lam = lam + rho * (x - z)\n",
    "        conv_x.append(x.copy())\n",
    "        conv_z.append(z.copy())\n",
    "        conv_lam.append(lam.copy())\n",
    "    return x, z, lam, conv_x, conv_z, conv_lam"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "def update_x_admm(x, z, lam, A, b):\n",
    "    n = x.shape[0]\n",
    "    return np.linalg.solve(P + rho*np.eye(n), -lam + c + rho * z)\n",
    "\n",
    "def update_z_admm(x, z, lam, A, b):\n",
    "    x_hat = lam / rho + x\n",
    "    return x_hat - A.T @ np.linalg.solve(A @ A.T, A @ x_hat - b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "8.35127552680348\n"
     ]
    }
   ],
   "source": [
    "z0 = np.random.randn(n)\n",
    "lam0 = np.random.randn(n)\n",
    "rho = 10\n",
    "\n",
    "x_admm, z_admm, lam_admm, conv_x_admm, conv_z_admm, conv_lam_admm = admm(update_x_admm, \n",
    "                                                                         update_z_admm, \n",
    "                                                                         rho, x0, z0, lam0, \n",
    "                                                                         max_iter=100)\n",
    "print(f(x_admm))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([1.e-02, 1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03, 1.e+04, 1.e+05,\n",
       "        1.e+06, 1.e+07]), <a list of 10 Text xticklabel objects>)"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnkAAAH5CAYAAAD9USjkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzs3Xt022l97/vPI9mSY0uObEu5MEwu8gDDbZdRQrnT0Nhsdtu92OU4yenZ7V6rhYmhXNtCzJRSejudceDs1bPWXru1B3rd7Sa2S0tpC+14wGfoZSCJS7kNDESZZJgLE8dRYifxTXrOH/pJlmXZlmVJP0l+v9bSkv27PuE3ZD7zfX7P8xhrrQAAANBYPG43AAAAAOVHyAMAAGhAhDwAAIAGRMgDAABoQIQ8AACABkTIAwAAaECEPAAAgAbU5HYDjDE9knolXZXULem8tXY475iYpMOS4pKikuLW2vHNHgMAALBduBrynIAna+1AzrbzxpiQtfa083tU0qC1tjfnmFFjTNxaGy/2GAAAgO3E7e7a/gLbxvO290sayjtmSNLgJo8BAADYNtwOeVK6qzZfIufnPkmTefvPOds3cwwAAMC2YWpt7VpjzAVJQ9ba08aYkKRrkjqstYm846zS7/BNb3QMXbYAAGC7qYVKXpYx5qSkycz7eJI6JSk/vOWIFnkMAADAtuL66FpJMsb0yem2tdYey9kVKuL0Yo4pdM+Tkk5K0o4dOw7deeedpVymaKlUSh5PTWXqbY9nUpt4LrWHZ1KbeC61p1rP5PHHH5+y1kY2Oq4mQp61dkzSmDEmZIw5L+lea23+O3blvuewpGFJOnz4sD137lwlb6eJiQkdOXKkovfA5vBMahPPpfbwTGoTz6X2VOuZGGMuFXNcTf0ngNPlOiTp4dztzrt56yrmGAAAgO2ipkKeY1xSyJlDLzNgojP3gJxAFy/yGAAAgG3FtZBnjIkaY645K1UUEnIqe3Gtfu+uU1LCWhsv5piyNhwAAKAOuFnJC2llJS4jMxo2807euNLLleWKOdu1iWMAAAC2DddCnjOw4kyBXQOSTudU4AYkHcs7pt/Zrk0cAwAAsG24OrrWmfD4pDGmW9JVpSc3HnVGvmaOSRhjBowxg5LOKl3pG8zthi3mGAAAgO3E9SlUcgPdOsdMavWyZZs+BgAAYLuoxdG1AAAA2CLXK3kAADSC+fl5TU9Pa2ZmRslksuL327lzpx577LGK3wfFK+WZeL1eBYNBdXZ2yu/3l7U9hDwAALZofn5ely9fVkdHhw4cOKDm5mYZYyp6z5mZGQWDwYreA5uz2WdirdXi4qJu3Lihy5cva9++fWUNenTXAgCwRdPT0+ro6FA4HJbP56t4wENjMMbI5/MpHA6ro6ND09PTZb0+IQ8AgC2amZlRe3u7281AHWtvb9fMzExZr0nIAwBgi5LJpJqbm91uBupYc3Nz2d/lJOQBAFAGdNFiKyrxzw8hDwAAoAER8gAAABoQU6hIslaaW0yu2pb9WXbVdrvi2Jz9a10j9xcAAIAKI+RJ+ubT13X3R79Q0Xu8/o4mvelNFb0FAACumJyc1KFDh1ZtD4VCOn78uAYHBxUKhda9Rnd3t+LxuM6fP69YLFappm4rhDxJu9tbdOotL5IkGS2/+Jj7DqQpuK3wscvb0htHzj6pJ2dulq29AADUosHBQfX19UmSEomExsfHdf/992tkZEQPP/zwmuFtcnJS8XhcknTmzBlCXpkQ8iTtCvr1i0fuqtj1v/vsDf3jN2Yrdn0AAGpBKBRSNBrN/h6LxXTy5EkdPXpUR48e1bVr1wqelwl20WhUw8PDGhwcrFaTGxoDL6ogHPDrxoJVKsV7eQCA7SUUCunBBx9UIpHQ6dOnCx4zPDysEydOqLe3V4lEQpOTk1VuZWMi5FVBJOhXykrXbi243RQAAKouFoupp6dHAwMDq/ZNTk4qkUior69Px48flyQNDQ1Vu4kNiZBXBeFAerHhqVlCHgBge+rt7ZWk7Lt3GUNDQ4pGo4pGowqFQorFYhoZGXGjiQ2Hd/KqIBJMh7wrM/N60Z6gy60BAFTTb37uW/r20zfKft1kMimv11uWa73kee362H9+aVmutZbMu3rxeHzFe3sjIyM6efJk9vcTJ05oYGBA4+Pj6unpqWibGh2VvCpYruTNu9wSAADckZlCJZFIZLeNj48rkUhkq3ySsqNzR0dHq9vABkQlrwoylTxCHgBsP5WqkM3MzCgYrJ/eocxgitzpUUZHRxUKhVZU7DJdt8PDw7ybt0VU8qqgvaVJTSbdXQsAwHZ09uxZSVrVVZtIJGSMWfHJvLc3Pj7uSlsbBSGvCowxavcbXaGSBwDYhuLxuMbGxnTq1KnstkxX7ejoqC5cuLDi89BDD0lilO1W0V1bJTv9hkoeAGDbSSQSOnbsmEKhkO67777s9qGhIYVCoew7eLmi0ahisZjGxsaq2dSGQyWvStp9hilUAAANLZFIKB6PKx6Pa3JyUqdPn9ahQ4cUj8f18MMPr1i/dmxsbMWo2nz9/f3Z41AaQl6V7PQbBl4AABrawMCAuru71d3drUOHDun+++9XT0+PLl68uGLARSa4nThxYs1rMTHy1tFdWyU7fUZXZ+eVTFl5Pcbt5gAAUDaxWEzWFr90Z19f34bHh0KhTV0Tq1HJq5J2v2FpMwAAUDWEvCrZ6U9X7xh8AQAAqoGQVyU7femQx3t5AACgGgh5VZKp5BHyAABANRDyqqTdR3ctAACoHkJelexokvxNHubKAwAAVUHIqxJjjMIBP5U8AABQFYS8KgoH/byTBwAAqoKQV0URKnkAAKBKCHlVFAn6qOQBAICqIORVUSTg1/TNBSVTLNMCAAAqi5BXReGgXykrXb1JNQ8AAFQWIa+KwgG/JGlqhmlUAABAZRHyqigSTIe8K7yXBwAAKoyQV0XLlTxCHgCgsXV3d8sYo8nJyYL7x8fHZYzR8PBwlVu2fRDyqihTyWOELQCgkU1OTioej0uSzpw543Jrti9CXhW1+bxqafYwVx4AoKGdOXNGsVhMfX19VOpcRMirImOMIqx6AQBocMPDwzpx4oR6e3uVSCTW7LJFZRHyqiwc8DPwAgDQsCYnJ5VIJNTX16fjx49LkoaGhlxu1fZEyKuycMDPFCoAgIY1NDSkaDSqaDSqUCikWCymkZERt5u1LTW53YDtJhL0a/LSNbebAQCols9/WHr2G2W/7I7kkuQt07/G97xc+k8PlOVSIyMjOnnyZPb3EydOaGBgQOPj4+rp6SnLPVAcKnlVFg74NX1rQUvJlNtNAQCgrMbHx5VIJNTb25vd1tfXJ0kaHR11q1nbFpW8KosE/bJWmr65oF3tLW43BwBQaWWqkOW7PTOjYDBYkWuXanR0VKFQaEXFLtN1Ozw8zLt5VUYlr8oiAZ8k6TmmUQEANJiRkRElEgkZY1Z8MnPmjY+Pu9zC7YWQV2XZVS8YYQsAaCCZrtrR0VFduHBhxeehhx6SxCjbaqO7tsqWV71ghC0AoHEMDQ0pFApl38HLFY1GFYvFNDY25kLLti9CXpVlKnmsegEAaCRjY2M6derUmvv7+/vV39+vsbGxFUHw/PnzBcNfT0+PQqFQRdq6XRDyqqzN36RWn5fuWgBAw8iEtBMnTqx5zPHjx9Xf36+hoaEVIW94eLjg0mcPPfQQU65sESHPBeGAn0oeAKBh9PX1yVq77jGhUGjFMT09PRueg61xPeQZY/okRSV1O99D1tqxnP0hSScljUmaltQpqV/SQ9ba8ZzjYpIOS4o714nn7q8l4YCPSh4AAKgoV0OeE/DimVDnBLrzxphOa22mdtspadD5SFJC0r15AS8qadBa25uzbdQYE7fWxqvxZ9mMSNCv+JWbbjcDAAA0MLenUIlaayczv1hrE0qHufwx1r2SOiR1W2s7cit9jv4C5wxpORjWlHDATyUPAABUlGshz6nanXC+c407+6O5G621iXWqcn2SJvO2nXO215xI0K9rtxa1yNJmAACgQlwLeU7VLup8SuaExKjS7+vlX39VWKwFmWlUrjJXHgAAqBBX38mz1nYU2NwjKb9qF3Xe35PS7+hN53TZdjrXSqxxm6jSgzFqxvKEyPPas5P1awEAQPm5Prq2gH5J9+f8Pi1JeSNuR40xmW0lzZRojDmp9Khd7d69WxMTEyU3uBizs7PZezx5LSlJevhfzmoqUouPYHvIfSaoHTyX2sMz2djOnTs1MzNT1Xsmk8mq3xPr2+ozmZubK+v/12oqYTjBa9paezqzzanQ5c+SOOR8Sl4fxRm9OyxJhw8ftkeOHCn1UkWZmJhQ5h7Rq7f0O1/5kvYefJGOHL6zovfF2nKfCWoHz6X28Ew29thjjykYDFb1njMzM1W/J9a31WfS0tKie+65p2ztcXt0bZbz7lx/7jQo64gr3YWbreIVGMBRs8JBnyQxwhYAAFRMzYQ8pac7OZq/0RhTaCG8zCCL3PftOvPOy4S+mnofT5JafU1q83lZ9QIAAFRMTYQ8Y8yQpIH8wROZSY4LjJDNBLq4c05cq9/N69TqARw1IxL0a4rRtQAAoEJcD3nOe3iDuWHMGNNjjIk62/oLBLUeSZM5oXBc6SXNcsWc7TUpvX7tnNvNAAAADcrVkJczLUrIGBNzPj2SjuUEu+ncSp7TDdsv6d6cSw1IOpZ3+X5ne01Kr3pBJQ8AAFSGa6NrnbA2usbubOXOWjtmjOlzAmGX0t2yuSFQ1tqEMWbAGDMo6azS7+oN1mpXrZTurn304lW3mwEAABqUqyteWGvNGp/uvGPHrLWnrbUD1tpC3bey1k46+zPH1mxXrZSu5CVuLWphiaXNAACNp7u7W8YYTU7mrzqaNjk5KWPMqk9HR4f6+/uVSBRe4yD3vPHx9f9Vnznu9OnszGxbPr+euP5O3naVWfXi6k1G2AIAGsvk5KTi8XQ95syZM+seOzg4qAsXLujChQs6f/687rvvPo2MjOjgwYNrBsSM0dG1OgSlsbGNp9Ld6vm1jpDnknAgPVce06gAABrNmTNnFIvF1NfXp+Hh/PUMVgqFQopGo4pGo4rFYjp16pQuXryoaDSqo0dXzayW1dPTo5GRkTX3Dw0Nqaenp2Ln1wNCnkty168FAKCRDA8P68SJE+rt7VUikdiwIpcvFArpwQcfVCKRWLOr9NixY0okEgW7XDPbjx3LH5NZvvPrASHPJeFAOuRRyQMANJLJyUklEgn19fXp+PHjktJVsc2KxWLq6enRwEDhiTI6OzsVi8UKXjtToVuvErfV8+sBIc8ly5U8plEBADSOoaGhbPdrKBRSLBZbt1t0Pb296ZVOM+/35Ttx4kTBd+dGR0fV09Ojzs7OAmeV7/xcAwMDam9vX1W17O7uVnd39xpnVZZrU6hsdy3NXgX9TVTyAKDBDX51UN+Z/k7Zr5tMJuX1estyrbs779bAj5ZnatmRkRGdPHky+/uJEyc0MDCg8fHxTVfGotH0NLnxeDz7c66+vr5V1850tRZTPdzq+bkGBwd19uxZHT16VBcvXlQoFFJ/f7/i8bguXLiwqWuVC5U8F4WDfl3hnTwAQIMYHx9XIpHIVuCkdJCS1h/JupZQKL1i6VrTqWQqhrmBLFM1zHQVr2er5+f7kz/5E3V2durYsWMaGxvT8PCwRkdHCwbUaqCS56JIwK8pKnkA0NDKVSHLNzMzo2AwWJFrl2p0dFShUGhFxS4TpIaHhzddHct0fcZisTWP6evrWzE4I9PVGgqF1gyH5Tw/VygU0ujoqA4dOqTx8XGdPHkyG3LdQCXPReGgj0oeAKBhjIyMKJFIrJrgOPNO3UaTD+c7e/asJK1bCevv789eO9PVmtlWjK2eny8zdYykNQeNVAuVPBeFA35NzUy53QwAALYsE5JGR0dXVd7i8bh6e3s3NfdcPB7X2NiYTp06te5xmUrh6OhotvK2merZVs/PNzY2prGxMfX09OjYsWM6f/58ydfaKkKeiyIBv27MLWl+KSl/U3lengUAwA1DQ0MKhUIFA1JmouNiV5FIJBI6duyYQqGQ7rvvvg2P7+/vz3YFlxLQtnp+xsWLF3Xs2DENDQ3p+PHjOnjw4IprVxvdtS4KM40KAKBBjI2NrRhVmy/TBZof9BKJhOLxuOLxuCYnJ3X69GkdOnRI8XhcDz/8cHbwxXr6+voUj8ezkzBv1lbPz3jrW9+qWCymkydPZt/PGx4e3nDVj0oh5Lko4kyIzOALAEA9ywS39QLSWhMjDwwMZOeSO3TokO6//3719PTo4sWL6w64yJXpcpVKq8Rt9XwpHWKfeOKJFaOIe3p6dOrUKfX392961Y9yoLvWRZlKHnPlAQDqWV9fn6y16x4TCoVWHBOLxTY8p5C1zis0F13+Pctx/lqGhob0iU98YtWI58HBQQ0ODhZ1jXKjkuci1q8FAACVQshzUVebTxIhDwAAlB8hz0UtzV4FW1jaDAAAlB8hz2WRoJ/RtQAAoOwIeS4LB/xU8gAAQNkR8lyWruQR8gAAQHkR8lwWoZIHAAAqgJDnsnDAp5n5Jc0tJt1uCgAAaCCEPJcxVx4AAKgEQp7LwgFWvQAAAOVHyHPZciWPaVQAAED5EPJcRiUPAABUAiHPZV0BljYDAADlR8hzmb/Jq507mgl5AACgrAh5NSAc8NFdCwBoKN3d3TLGaHJysuD+yclJGWNWfTo6OtTf369EIrHheePj4+u2IXPc6dOny3Z+PSHk1QBWvQAANJLJyUnF43FJ0pkzZ9Y9dnBwUBcuXNCFCxd0/vx53XfffRoZGdHBgwfXDIgZo6Oja+4bGxvbsJ1bPb/WEfJqAOvXAgAayZkzZxSLxdTX16fh4eF1jw2FQopGo4pGo4rFYjp16pQuXryoaDSqo0ePrnleT0+PRkZG1tw/NDSknp6eip1fDwh5NSBdyWMKFQBAYxgeHtaJEyfU29urRCKxYUUuXygU0oMPPqhEIrFmV+mxY8eUSCQKdrlmth87dmzNe2z1/HpAyKsB4YBfs/NLur3A0mYAgPo2OTmpRCKhvr4+HT9+XFK6KrZZsVhMPT09GhgYKLi/s7NTsVis4LUzFbr1KnFbPb8eEPJqQCTA0mYAgMYwNDSU7X4NhUKKxWLrdouup7e3V5Ky7/flO3HiRMF350ZHR9XT06POzs51r7/V83NNTk6qvb294GCSjbqsK6XJlbtihcyqF1dm53VnZ6vLrQEAlNOzv/u7mn/sO2W/7lIyqWmvtyzX8r/4bu351V8ty7VGRkZ08uTJ7O8nTpzQwMCAxsfHN10Zi0ajktIhL/Nzrr6+vlXXznS1FlM93Or5uWKxmB555BG1tbVlt2XeKcxUNKuNSl4NYNULAEAjGB8fVyKRyFbgpHSQktYfybqWUCgkSWtOp5KpGOYGskzVsJhgtdXz873iFa9QLBZTLBbTmTNnlEgkNDo6mv1zVBuVvBqwvH4tIQ8AGk25KmT5ZmZmFAwGK3LtUmUCTW7FLhOkhoeHN10dywzYiMViax7T19e3YnBGpqs1FAqtGQ7Lef5a7T59+rROnTrl6nt9VPJqQGZpMyp5AIB6NjIyokQiseqdtMw7dRtNPpzv7NmzklSwqzajv78/e+1MV2tmWzG2en4hx44dUzQa1eDg4Jaus1VU8mpAs9ejUCtLmwEA6lcmJI2Ojq6qvMXjcfX29m5q7rl4PK6xsTGdOnVq3eMylcLR0dFs5S3TRVyMrZ6fr7+/X/F4XBcuXCj5GuVCyKsRkYBfUzPMlQcAqE9DQ0MKhUIFA1JmouNiV5FIJBI6duyYQqGQ7rvvvg2P7+/vz3YFlxLQtnp+xl//9V9nu6XXqz5WC921NSIc8OsKlTwAQJ0aGxtbMao2X6YLND/oJRIJxeNxxePx7Ltshw4dUjwe18MPP1zUoIW+vj7F4/HsJMybtdXzpfSf433ve1+2Mjg+Pp79rDUFTKUR8moE69cCAOpVJritF5DWmhh5YGBA3d3d6u7u1qFDh3T//ferp6dHFy9eXHfARa5MsJJKq8Rt9XwpvcpHJrD29vau+JQyGXQ50F1bI1i/FgBQr/r6+mStXfeYUCi04phYLLbhOYWsdV6hd+Dy71mO89dy6tQpvetd76qpEc9U8mpEJOjXrYWkbi0sud0UAADQAAh5NSLsTKPC4AsAAFAOhLwaEc4ubTbncksAAEAjIOTViEh2aTMqeQAAYOsIeTUikq3kMfgCAABsHSGvRnS2+WSMNMUIWwAAUAaEvBrR7PWoo9XHXHkAAKAsCHk1JBzwMVceANSpUuZ8AzIq8c8PIa+GsOoFANQnr9erxcVFt5uBOra4uCiv11vWaxLyagjr1wJAfQoGg7px44bbzUAdu3HjRtlXyyDk1ZBIwK+pmQVK/gBQZzo7O3Xt2jVNTU1pYYG/x1Eca60WFhY0NTWla9euqbOzs6zXd33tWmNMn6SopG7ne8haO5Z3TEzSYUlx55i4tXZ8s8fUunDQr9uLSd1cSCrgd/3RAACK5Pf7tW/fPk1PT+uJJ55QMpms+D3n5ubU0tJS8fugeKU8E6/Xq2AwqH379snv95e1Pa4mCSfgxTOhzhgTknTeGNNprR12tkUlDVpre3POGzXGxK218WKPqQdhZ0LkqZl5Qh4A1Bm/36+9e/dq7969VbnfxMSE7rnnnqrcC8WptWfidndt1Fo7mfnFWpuQNChpKOeY/rzf5fw+uMljal5mQmQGXwAAgK1yLeQ5VbsTzneucWd/1Pm9T9Jk3jHnnO3axDE1LxzwSRLTqAAAgC1zLeQ5Vbuo8ynICYBRSdMFzpUxJlrMMeVteeVQyQMAAOXi6otf1tqOApt7JCWstfFMQMsEtgKyAW6DY+rivbzO1vTSZlTyAADAVrn9Tl4h/ZLud37O78otpJhj6kKT16OuNp+uzC643RQAAFDnamoIpzHmpKRpa+3pKt3rpCTt3r1bExMTFb3f7OxsUfdo0aK+88RTmpi4WtH2oPhnguriudQenklt4rnUnlp7JjUT8pyu2X5r7aEC+0LrdMcWfUwuZ4qWYUk6fPiwPXLkyCZbvDkTExMq5h4Hvv8Vzc4v6ciR11W0PSj+maC6eC61h2dSm3gutafWnkktddcOSjqaty3zLt2KKaBzRuTGizymbrB+LQAAKIeaqOQZY4YkDeRX4qy1CWNMXKvfu+uUMzjDOX/DY+pFOODTlZl5WWtljHG7OQAAoE65Xslz3o0bzA1jxpienKlPxpVerixXzNmuTRxTFyJBv+aXUpqdX3K7KQAAoI65GvKcZc0kKWSMiTmfHknHckLfgKRjeaf2O9u1iWPqQnZpM0bYAgCALXCtu9Z5Z250jd3Zqp7TZTtgjBmUdFbpee9WVP6KOaZeZELelZl5HQy3udwaAABQr1wLec77d0W9dOasb5u/bNmmj6kHrHoBAADKwfV38rBSbiUPAACgVIS8GtPZ5pPHUMkDAABbQ8irMV6PUWcbc+UBAICtIeTVoEjQT3ctAADYEkJeDQoHfLrCFCoAAGALCHk1KBLwa4pKHgAA2AJCXg2KBP26Mpte2gwAAKAUhLwaFA74tbCU0o05ljYDAAClIeTVICZEBgAAW0XIq0HZ9Wt5Lw8AAJSIkFeDwkGfJOkKlTwAAFAiQl4NilDJAwAAW0TIq0EdrT55PYZKHgAAKBkhrwZ5PEZdbT5NzTAhMgAAKA0hr0aFA6xfCwAASkfIq1GZCZEBAABKQcirUWGWNgMAAFtAyKtR4aBPU7MLLG0GAABKQsirUZGAXwvJlG7cZmkzAACweYS8GpVZ2oz38gAAQCkIeTUqMyHyFd7LAwAAJSDk1aiwU8ljGhUAAFAKQl6NClPJAwAAW0DIq1GhHc1q8hgqeQAAoCSEvBrl8Rh1BXxU8gAAQEkIeTUsEmRpMwAAUBpCXg1Lr1+74HYzAABAHSLk1bBwwE93LQAAKAkhr4ZFgn5dvTmvVIqlzQAAwOYQ8mpYOODXYtLq+u1Ft5sCAADqDCGvhkWYEBkAAJSIkFfDwgGfJNavBQAAm0fIq2G7gqx6AQAASkPIq2GZpc2YRgUAAGwWIa+G7dzRrGavoZIHAAA2reSQZ4w5YIy53xjzD8aYdmfbTmPM72d+x9YYY5wJkQl5AABgc0oKecaYo5IuSOqV1COpU5KstdcldUt6oFwN3O4IeQAAoBSlVvIGJY1Zaw9LMnn7hiWd2FKrkBUJsuoFAADYvFJDXkzSmTX2XZMUKvG6yBMO+KjkAQCATSs15E1KOrzGvn5J4yVeF3nS3bULLG0GAAA2panE8x6QNGKMMZKspJ3GmFco3Y3bI+lQmdq37UWCfiVTVonbi+ps87ndHAAAUCdKCnnW2jFjzHGl378zSlf2jKSEpOPW2q+Vr4nbW2auvCsz84Q8AABQtFIrebLWjkkaM8bcIykqKW6t/beytQySVq5f+yIFXW4NAACoFyWFPGNMu7X2hiQ5wY5wVyHLq14w+AIAABSv1IEXCWPMV40xb2fi48qKsH4tAAAoQakh78POuQ9KuuasevHT5WsWMtpbmuTzenSFSh4AANiEkkKetfa0MxFyt6T7JHVJ+ktjTNIYc4bAVz7ppc18VPIAAMCmlLx2rSRZay/mBL4OSceVDn5j5Wgc0iLB9Fx5AAAAxSp5dG0uY8zblJ4EuUfLU6qgTMIBv565Pud2MwAAQB0puZJnjHmb8y5eUunKXbfS7+p1O5U9lEkk6OedPAAAsCmlTqGSUnqli+uSPi5pyFp7sZwNw7JwwK/pmwtKpqy8HuN2cwAAQB0otbv2QUl/wOTH1REO+JRMWV27tZCdNw8AAGA9pS5r1l/uhmBtkWCLpPSEyIQ8AABQjA3fyTPG/KMxZilvW8qZLmWtz9Ja18PmhQPpNWunZhhhCwAAilNMJW9cUiJv28eVfievLIwxfZIS1trxvO0hSSeVHtgxLalT6VG8D+Uea4yJSTosKa7ldXRXXKueZVe9mGWELQAAKM6GIc9ae7rAtoFyNcAY06P0O37HCuzulDTofKR02Lw3L+BFJQ1aa3tzto0aY+LW2ni52ummsBPyqOQBAIBibWky5K0wxkSNMUNKV96m1zm0V+mJlrswAjtsAAAgAElEQVSttR3W2vyJlvslDeVtG9JyMKx7QX+TfE0sbQYAAIpXUsgzxlw1xhxYY98/GmN+d6NrWGvj1tp+a+1wEccm1qnK9Wn15MvnnO0NwRijSMCvKZY2AwAARSq1ktexzr4hFe56LTvnnb1VlUBrbcLZH61GO6ohzITIAABgE4qeQsWp3MVyNh01xlwrcOhJpYNXuUSdgRlS+h296Zwu205pOdQVOlfpwRh1LxLw6wfXbrndDAAAUCc2M0/eOyWdcn62Sg+WWMuqwRolmpak3PfwnEEVmW2hMt2n5kWCPn3tybWyLAAAwErG2uJnQjHG7JRklA5fvVpdJZu21l7fdCOMuSCpv5hpT5zRuEPW2m5n6pTz1tpVa30ZY6yk3rWuaYw5qXTVUbt37z706U9/erPN3pTZ2VkFAoGSz//M9xb0uQuL+tR/bJXHsLRZOWz1maAyeC61h2dSm3gutadaz+RNb3rTeWvt4Y2O29SKF5kAZ4wZk3TWWnujxPZtRVzpLtxsFc8YE1qny7YgZ8DHsCQdPnzYHjlypKyNzDcxMaGt3OOy/wn9zYVv6eWHX5udNw9bs9VngsrgudQenklt4rnUnlp7JqUOvLhf0n35G40xO40x/7DWyNvNMsacKrA5M8gi9327zrzzMgGwId7Hk5RdzuwKI2wBAEARSg15D6jA4Aqn0mdUhjnqMpMcFxghmwl0cad6F9fqd/M6lV5Bo2FCXqZ6N8UIWwAAUIRSQ94rJT20xr5RrRyFWxInoPUXCGo9kiZzumfHlV7SLFfM2d4wMpU8Qh4AACjGVla8WGuVimltfgqVThUeKTudW8lzumH7Jd2bc8yAVs/L1+9sbxjZ9WvprgUAAEXY1MCLHOeUHpn6mQL7TqiIKpoT2O5TOhCGlO6a7ZX0UGbKFGvtmDGmz5knr8s57lhudc9amzDGDBhjBiWdda432EhdtZLU5vOqpdlDJQ8AABSl1JD3YUnnjDFfUPr9u2tKV+MGlO5OPbTRBZzu1g2rbQXWqi10zKRWL23WUIwxCgf8VPIAAEBRSgp51tpJY8yblV7C7GGlJ0c2Sg+CeLO19mvlayIyIkG/pmYX3G4GAACoA6VW8uRMMtxtjDkoZzoTa+3FsrUMq4QDfj05zdJmAABgYyUPvDDGHDDG3C/p95WeGPmiM0/e7xtj2svXRGREgnTXAgCA4pQU8owxRyVdUHpps145c9c58+R1Kz2PHsosHPBr+taClpIpt5sCAABqXKmVvEFJY866afkLqQ4rPcIWZRYJ+mWtNH2T9/IAAMD6Sg15MUln1th3TYXnvMMWRQI+SdIVplEBAAAbKDXkTWr1KhMZ/Wqw1SZqxfKqF1TyAADA+kodXfuApBFjjFF6+pSdxphXKN2NW9Q8edg8Vr0AAADFKnWevDFjzHGl378zSlf2jKSEpOPMk1cZrF8LAACKtZV58sYkjRlj7tHyPHn/VraWYZU2f5NafV4qeQAAYEMlh7wMJ9gR7qokHPBTyQMAABvaMOQZY94mKWqt/UTOtg8Wef2EpHFr7ROlNQ/5wgEflTwAALChYip575R0VNIncrad3sQ9rDHmpLX2U5tqGQqKBP26OHXT7WYAAIAaV8wUKv3Kmy7FWusp5qP0Shif0eZCIdaR7q5lChUAALC+DUOetfZiqQMqrLUJSZ8u5VwUFgn6NX1zQYssbQYAANZR6mTIm3FC0sNVuM+2kJlGhaXNAADAekoOecaYA8aY3zfGnDXGXHW+f9cY0557nLX2uLX2+NabCokJkQEAQHFKCnnGmKOSLij9vt41SaPO94clxY0xB8rUPuTJVPJYvxYAAKyn1HnyhiTFJb3ZWnsxs9EYE1K6a/YPJL1l681Dvkhm1QsqeQAAYB2ldtd2ShrMDXhSdqDFgKTerTYMhYWDPklU8gAAwPpKDXnnJE2vse+CpItr7MMWtfqa1ObzamqGgRcAAGBtpYa8AUn3rbHvAUmnSrwuihAJ+qnkAQCAdRWzrFlKki28yyQLbZfUJ8m7xbZhDeGAn3fyAADAuooZePFxFQ55cEk44NeFK7NuNwMAANSwDUOetXagGg1B8SJBvx69eNXtZgAAgBpWjRUvUGbhgF+JW4taWGJpMwAAUNim58kzxrxC0mFJh5xN5yXFrbVfLGfDsLbMqhdXb85r784dLrcGAADUoqJDnhPuHpQUU3pwRS5rjDkv6V5r7b+XsX0oIBxIz5U3NbNAyAMAAAUV1V1rjHmbpElJHZKOS+q21nqc3w8pPZ3KCyRNGmN+ukJthSOcWb92ds7llgAAgFq1YcgzxhyUNCZp2Fp7l7X2LzMrXVhrr1tr/81ae9pa2yHpM5LGjDH7K9vs7W15aTMmRAYAAIUVU8kblHTeWvvOjQ601h6T9G9Kr22LColkK3nMlQcAAAorJuT1SDqziWuekfTK0pqDYrQ0exX0N+kKEyIDAIA1FBPyQpISm7hmwjkHFRQO+jVFJQ8AAKyhmJA3qXQ1r1i9zjmooEjATyUPAACsqZiQ94Ck48WMmjXG3Cvp/5B0/1YbhvWFgz4qeQAAYE0bhjxr7ZjS8+ONGWP+pzGmPf8YY8wrjDFnJP2BpAettZ8pf1ORKxzwa2qW0bUAAKCwoiZDttb2G2MSkj4kKfNz3Nkdc76NpNPW2g+Xv5nIFwn4df32ouaXkvI3ed1uDgAAqDFFr11rrR1QevLjTyi9lFm383lY0seVniCZgFclmQmRr1LNAwAABWxq7Vpr7XVJAxVqCzYhMyHylZl5PS/E0mYAAGCloit5qC2ZSh6DLwAAQCGEvDoVDvgkEfIAAEBhhLw6Fc7prgUAAMhHyKtTLc1eBVuamEYFAAAURMirY5Egq14AAIDCCHl1LBzw6wrv5AEAgAIIeXUsEvRrikoeAAAogJBXxyJU8gAAwBoIeXUsHPBpZm5Jc4tJt5sCAABqDCGvjkWYEBkAAKyBkFfHMnPlMY0KAADIR8irY5lKHtOoAACAfIS8OrZcySPkAQCAlQh5dawrs34tlTwAAJCHkFfH/E1e7dzRzDQqAABgFUJenQsHfHTXAgCAVZrcboAkGWP6JCWsteMF9sUkHZYUlxSVFM8/rphjGhXr1wIAgEJcD3nGmB5JD0o6VmBfVNKgtbY3Z9uoMSZurY0Xe0wjCwf8+tbTN9xuBgAAqDGuddcaY6LGmCGlK2/TaxzWL2kob9uQpMFNHtOwwgHWrwUAAKu5FvKstXFrbb+1dnidw/okTeZtO+ds38wxDSsS9GtmnqXNAADASjU78MIYE1KBKp+1NuHsjxZzTHVa655IgAmRAQDAajUb8iR1SsuBrYBokcc0tOyqF4ywBQAAOVwfeLGOUJmOKcgYc1LSSUnavXu3JiYmSr1UUWZnZytyj0vX0920E/96Xjfitfw4a0+lngm2hudSe3gmtYnnUntq7Zls21TgvAs4LEmHDx+2R44cqej9JiYmVIl7PHt9Tr/xrw9r94EX6Mir9pf9+o2sUs8EW8NzqT08k9rEc6k9tfZMarm7VlL23bwtH9Oolpc2W3C5JQAAoJbUcsjLzHHXmbsxJ9DFizymoTV7PQq1NrPqBQAAWKFmu2uttQljTFyr37vrVHp1jMxkyBse0+giAVa9AAAAK9VyJU+SxpVerixXzNm+mWMaWjjgp5IHAABWqJWQ16nCI2UHtHq5s35n+2aOaWiRoJ8pVAAAwAquddc6783dp/RcdiFJg8aYXkkPWWvHpGyX7YAxZlDSWefYwdxu2GKOaXQsbQYAAPK5FvKcCYw3rLZZaye1etmyTR/TyMJBn24uJHVrYUmtvpp9zRIAAFRRrXTXYgsyS5sxjQoAAMgg5DWAMEubAQCAPIS8BpCp5DGNCgAAyCDkNYCIU8ljGhUAAJBByGsAnW0+GUPIAwAAywh5DaDZ61FHq4/uWgAAkEXIaxDhgI9KHgAAyCLkNYhIkPVrAQDAMkJeg0ivX8s8eQAAII2Q1yAiASp5AABgGSGvQYSDft1eTOrm/JLbTQEAADWAkNcgwgHmygMAAMsIeQ0iMyEyXbYAAEAi5DWMcMAniUoeAABII+Q1CCp5AAAgFyGvQXS2ppc2u8I0KgAAQIS8htHk9airjVUvAABAGiGvgYSZKw8AADgIeQ0kveoFIQ8AABDyGgrr1wIAgAxCXgMJB9Lv5Flr3W4KAABwGSGvgUSCfs0tpnRzIel2UwAAgMsIeQ0ks7QZXbYAAICQ10BYvxYAAGQQ8hoIq14AAIAMQl4DoZIHAAAyCHkNpLPNJ4+RpqjkAQCw7RHyGojXY9TZ5tcVKnkAAGx7hLwGEw74dGVmwe1mAAAAlxHyGkwkSCUPAAAQ8hpOJODnnTwAAEDIazSZSh5LmwEAsL0R8hpMOODXwlJKM/NLbjcFAAC4iJDXYDITItNlCwDA9kbIazCsXwsAACRCXsMJB32SpKlZplEBAGA7I+Q1mEi2kjfncksAAICbCHkNpqPVJ6/HUMkDAGCbI+Q1GI/HqKvNpykmRAYAYFsj5DWgcMDPwAsAALY5Ql4DCgf9VPIAANjmCHkNKEIlDwCAbY+Q14DCQZ+mZhdY2gwAgG2MkNeAIgG/FpIp3ZhjaTMAALYrQl4DyixtRpctAADbFyGvAWUmRGbwBQAA2xchrwGFqeQBALDtEfIaUJhKHgAA2x4hrwGFdjSryWOo5AEAsI0R8hqQx2PUFWBpMwAAtjNCXoOKBP2aml1wuxkAAMAlhLwGxfq1AABsb4S8BhUOsH4tAADbGSGvQaW7a+dZ2gwAgG2qye0GbMQYE5J0UtKYpGlJnZL6JT1krR3POS4m6bCkuKSopHju/u0mHPBrMWl1/faiQq0+t5sDAACqrOZDntKhbtD5SFJC0r15AS8qadBa25uzbdQYE7fWxqva2hqRWdpsanaekAcAwDZUL921vZI6JHVbazustWN5+/slDeVtG9JyMNx2woF0sHuOwRcAAGxL9RLyZK1NrFOV65M0mbftnLN9W1pev5ZpVAAA2I7qJuStxXlnL6r0+3pZ1tqEsz/qRrvcFmH9WgAAtrV6eCdPkqLGmExVrlPSdE6Xbae0HOoKnav0YIxtZeeOZjV7DdOoAACwTdVDyJuWpNz38JxBFZltoVIuaow5qfSoXe3evVsTExNlaOraZmdnK36PfMFm6Zvfu6SJlmeret964cYzwcZ4LrWHZ1KbeC61p9aeSc2HPKdCN5y3ecj55A/A2Mx1hzPXPXz4sD1y5EiplyrKxMSEKn2PfHd845/kDfh05MiPVvW+9cKNZ4KN8VxqD8+kNvFcak+tPZN6fScvrnQXbraKl/sz0jITIgMAgO2n5kOeMeZUgc2ZQRa579t15p2XCX3b7n28jHDAx8ALAAC2qZoOeZlJjguMkM0EurjTnRvX6nfzOiWtN+1KwwsH/Lo6u6BUiqXNAADYbmo65DkBrb9AUOuRNJkzonZc6SXNcsWc7dtWJOjXUsoqcXvR7aYAAIAqq+mQ55jOreQ53bD9ku7NOWZA0rG88/qd7dtWOLC8tBkAANhe6mF07Zgxps+ZJ69L6W7ZY7nVPWttwhgzYIwZlHRW6Xf1BrdzV60k7W5vkSTFr8zqhbuDLrcGAABUU82HPGnlHHnrHDOp1UubbWuvuDOkPe0t+rNHL+ktL9vrdnMAAEAV1UN3LUrka/Lo5193QP/8/av65lPX3W4OAACoIkJeg/uZV+1TwN+kT355W/dcAwCw7RDyGlx7S7P+z1feqc99/Rk9nbjtdnMAAECVEPK2gZ9//UFJ0h/980WXWwIAAKqFkLcN3BHaoZ98+V79768+qRtzzJkHAMB2QMjbJu59Q1Sz80v69Fcvu90UAABQBYS8beLlz9+p10S79If/9IQWllJuNwcAAFQYIW8bOfnGqJ69Mae/+8bTbjcFAABUGCFvG/mxF0b0gl0BDT9yUdZat5sDAAAqiJC3jXg8Rve+IarHnrmhf/7+VbebAwAAKoiQt8289Z7nKRzwa5jJkQEAaGh1sXZtpc099pi+e+hw+hdj1v02mZM2OG75O/3V/oIXyr7udTLNzWVv/2b4m7z6+dcd0Mf/4bt67JkbevHedlfbAwAAKoOQJ8nb0aFQX5+k9Htq2ffVMq+tZX/P+151fOHzkjM3lPz8F/TUBz+kOz7xcdeD3n991T79jy9+X5/88kX9P8d/xNW2AACAyiDkSWres0e77/twRe/x1bY2aewv9ZQxuuPjp10NeqFWn0688k79+Vcu6UP/8UXas7PFtbYAAIDK4J28KrnV06Ndp05p5gtf0FMfOiW7tORqe37hdQeVTFn98b884Wo7AABAZRDyqqjrF35euz70ISfofcjVoLevq1X/6WV79edfuaTZeXcDJwAAKD9CXpV1vf0XtOtDH9TM57+gp0+5W9F7xxsOamZuSWfOPulaGwAAQGUQ8lzQ9fa3a9cHf0U3/v7zevrUgGtB7559HfrRA536w3+6qKUkS50BANBICHku6XrHOxT5lV/Wjb//ez098GHXgt69b4zqqcRt/f03n3Xl/gAAoDIYXeui8L33Sla68t//u2SMnvfA/TJN1X0kR+/epWi4TcOPXNB//g97ZYzZ+CQAAFDzqOS5LHzyXkV+6Zd042//Vk9/+D7ZZLKq9/d4jN7xhqi++dQNPRqfruq9AQBA5RDyakC4/6QiH/iAa0HvbbE71NXm0ydZ6gwAgIZBd22NCL+zX5J05fd+TzLS8+6/X8brrcq9W5q9+rnX7NfvjX9P339uRnftClblvgAAoHKo5NWQ8Dv7FfnA+3Xjbz6nZ371V6ta0fu5V++Xv8mjT375YtXuCQAAKoeQV2PC73ynIu9/n65/9m/0zK9+pGpBryvgV9+h5+szk0/puZm5qtwTAABUDiGvBoXf9S6F3/deXf/sZ/XMR36takHv7a8/qMVUSn/2r5eqcj8AAFA5hLwaFfnFX1T4ve/R9b/+az3zax+tStCLRgLqffFu/dmjl3RrgaXOAACoZ4S8GhZ597sVfs97dP2v/krPfPTXZVOVX5Xi5BujStxa1Nj5H1T8XgAAoHIIeTUu8p53K/zud+v6Zz6TruhVOOgd2t+he/aF9MkvX1QyZSt6LwAAUDmEvDoQee97FP7FX0wHvY9WNugZY3TyDVFdnr6lf/wWS50BAFCvmCevToTf+x5JVlP/8/clY7T3t35LxlOZjP7ml+7Rvs5WDT0S11tetoelzgAAqENU8uqEMUbh975XXe96p66P/aWe/djHKlbR83qM3vGGg/rakwmdv3StIvcAAACVRcirI8YYRd73PnW9s1+J0TE9+7HfqFjQ6zv0fIVamzX8CEudAQBQj+iurTPGGEXe/37JSleHhiRjtOc3Plb2rttWX5N+7tX79T++9H3Fr8wqGgmU9foAAKCyqOTVIWOMIh94v7pOnlRiZETP/uZvVaSi999ec0DNHo8+9U8sdQYAQL0h5NUpY4wiv/QBdd17rxJnzujZ3yp/0IsE/Xpb7A6Nnf+Brs7Ol/XaAACgsgh5dcwYo8gv/5K67n2HEp8+o2d/+7dlbXnntnvHGw5qfimlP3uUpc4AAKgnvJNX59JB75cla3X1k5+SJO359V8v27Qnd+0K6ujdu/Sn/3pJb3nZHt29p70s1wUAAJVFyJP05MyT+uWJX5ZROhgZY5Z/dr5ltGJ/Zl/+ttxzco+7fu26nvvuc9rfvl/72/drV+sueUx5CqnGGEV+5VdkrdX0p/5Qxhjt/uhHyxb0PtDzQv3sp76in/h/v6z/61X79Es9L1RXwF+WawMAgMog5EmaT87rQuKCJMnKruryzN1m5Xxbm/05e1zOtvzjpm9N64uPfjF7rN/r153BO7UvuE/72/drX7vzHdynXa27Nh3QjDHa9cEPSlaa/sM/lGS0+6O/Vpag9/Ln79TEB4/o98Yf1//6ymV99mtP6/1HX6D/9poD8jXR4w8AQC0i5Em6K3SXPvtfPlvRe3zxS1/Ui1/5Yl2auaTLNy7r0o1LujxzWU/ceEJffurLWkwtZo9t8bbozvY7tT+4Mvztb9+v8I7wmsHNGKNdH/qgZK2m/+iPJGO0+9c+Upag19Hm02++9WX62Vfv12//3WP6nb97TH/+lcv6yE+8WEdfvPlQCgAAKouQVyUe49HewF7tDezVq/e+esW+ZCqpZ289mw5+TgB8cuZJfT/xfU38YEJLqaXssTuadmhfcN+q8LevfZ+6WrrSQe/Uh9JB74//WLJWXSfvVdOuXWWZS+8Fu4P601/4UX3pu8/pd/7223rHn57T6+8K69d+6sW8rwcAQA0h5NUAr8erOwJ36I7AHXrt8167Yt9SaknP3HxmRfi7dOOSHr/2uL50+UtasssBsK25LRsA9/XeqUOJ10t/8Re69hd/IdPSIt++ffLt3y/fgf3p7/371bx/v5oikU1X4t70ol16/V1h/a9HL+n3xr/H+3oAANQYQl6Na/I06c7gnbozeKded8frVuxbTC3qmdlnsl2/l29c1qWZS/r21W9r/NK4Hrx7SXf/rFcvutai2MJeRWf8Cnz/e5qZmJAWl7uHPa2tat6/HPxyg6C3s3PNANjs9ejnX3dQ/+UVd/C+HgAANYaQV8eaPc3pql37vlX7FlOLemrmKT1+7XFNPDmhTzw5oZnFGbU2teqNe3v05paYXjG/W96nntPCpUtauPSE5h77tmYeekhKJrPX8QQC6Yrf858vTzAgT2urPDta09/Ox9vWqg91tOq/HgnowXPP6o/OPKK//eLXde9P/ojefM9+NXsJewAAVBshr0E1e5p1YOcBHdh5QG8+8GYtJhd19tmzGr88ri9e/qK+MPeQmj3Nes3zXqOjrzqqI3e+U50tnbKLi1p86ikn+F3SwhPp7/nHH1fq1q3sJzcI5voF5yNJ+t/S13ytSnVF1LH/DgWev1fNu3arac9uNe/erSbn4w2FGLgBAECZEfK2iWZvs157x2v12jteq4+86iP6+tTXNX5pXA9ffliP/OAReYxHsV0x9ezv0dF9R7XnwI+teS1rrezCQjrw3bwle/vWigCYunVLS7M39f3v/UDxxy5q7pkfqus7l7X3G99S260bMnlT1Bi/3wl+e9S8d4+adu9J/75nr5r37JY3HFZTKCTj81X6fyYAABoGIW8b8nq8umfXPbpn1z364OEP6rvXvpsNfA989QE98NUH9NKul6pnf49+fN+PK7ozuuJ8Y4yM3y+P3y91dKx5ny5Jr5L0dOK2Rs5e1sfPXtT1xHW90DOrn36eR68PJRWcmdbic1Naem5Ki1emdevRuBanrhWsFHp2tMjb3iZvsC377WnxyeNvlvE1y/ia5PE1yTQ3acljNa+k5k1Sc1rSbS1pQSktpZa0qJQWbFJT167qr84+qJRSsrJKFlgSLj0xtuSRkccYeYwn/bOMPB6PPPJktxsp59tIMs66gbmTZ6/PZr+LWZ6uvEvY1Yq5hUVp94zbzUCO8JVvSouvlppb3G4KXGCt1VLKajGZ0sJSSgvO91LSKmUzHymZcn5OSSmb/jvVWqtkKr3POttSNr1/+QZ591s1B23hnwucumKe29X7NvfnLsW3frikI5W/TdFMudc6rUeHDx+2586dq+g9JiYmdOTIkYreoxyeuP6EHr78sL54+Yv6+tTXJUnRnVG98flv1PPa9irs3aGwmtRlpXAyqdb5WenW1ZzPtPO5Ks3PSEtz0tJ8+rvIUGKttDTn0dItrxZveXV73qtbC17dXvRoYcGrpXmP7LxHnnkjs2jkTUreJcmbosu3EVx4UVI/dc8P3W4G8n3we1Jgl9utQI78f68sJVO6Mbek67cXs58btxd1c35JtxaSurWQ+U6mty0mdWt+STcXkrrt7M8EuMWkzQa6xWSqKgGpUTzxwE9W/B7GmPPW2sMbHUclDysc2HlAb7/rbXp7+4v17A8e1Zee+Rc9nHhCf5b4IyULvDe3I5VSVzKpcMqqS00Ke1rU1dyqrq7davN1K+lpUtLjdb49zs9e3VySLl6bU3z6tmaSizK+lEI7jVpbpXmzoJml25pZuq0byduaz5koOpfXeLSzKaAOX0A7mwPq9LSpS23q0A4FPTsUMH61enzOt18tnmb5TLP8nib5TLO+/93v6eUveZk8xiOv8axZZ7M2paRNOf8VmlJKKSVTSee/VFNK2aTz7Rwn69QGJTkrnhRaSWUtucvhbXhsEcfUG/8zz0lveavbzUCOs+fO6ZU71q7ao/ySKatrtxY0NTuvqRnne3ZeV2bnNT27oOu3F3Xpmdt64GuP6IYT6G4uFH5XOleTx6jN36RWn9f5pH+OBP3a0dwqX5NHPq9HzU1GPq9XzU1Gfq9HzV6PfE3L3z6vR01eI6/HyBgjrzHyGMnjSfd6eD3K2W7k8cjZnv7dGK342yv/vez8v9lyd+f/vbfeK935+yr9d+a5c2crev3NIuRtZ4u3pSvflZ57THruW+nvH35bmnlakrRH0s/4d+pndr9ES13dSrSGdLW5RVNNTZoyVleV1FRqXlNLtzS9cF1P3L6qc3NTuj6fkBYTUuFstlJI8hivPHaHLs/5ZW/u0E5/UAc779SP3LFLO/1B7fTvVGdLp0L+UPa7o6VDQV9wS+v/PmYn1P76IyWfj8p4dmJC2vNyt5uBHDcDVyVvs9vNqHtLyZSmby7oyuy8pmYXNDUznw1vU7PpIHdlJv3z9M15pQr8d6HP61Fnm0+h1mZZSXd2tGrn85q1c0fm06Sdrcu/t7c0q83fpDZfk3b4vExtVWHPtXvdbsIKhLztIJWUpi/mBDnne/qCZFPpY7w+KfIi6eAbpF0vkXa/VNr1Yqn9DskYNUkKO58XbXC7xeSirs5d1e2l22oyTfJ6vPIYj5o8TfKa5Z/TFTSvmj3NMsbo2etzGjn3pM6cfVL/8t3bejzgU6fRzNwAABLNSURBVN+hO/Xjh5+vaCRQ2f+NAKAEC0spXb25XG27kgltOdW3TIi7dmuhYLdnS7NH4YBf4YBfz+9o1T37Qtnf0x+fwsH0z+0tTdmqV7q7dsMeO2xjhLxGdP0H0uVHpSe/Ij35VenKd5x34iTJSJ0H00HupT8t7X6JtOulUmdU8pbnH4dmb7P2tO3Z9Hl7drbofUdfoHe/6S498vgV/cVXL+vBL8f1B//fBb3sjna99Ufu0E/9yF7t3bmjLO0EgHzWWt1aSK6qsF2dXSi47frtwl0WrT5vNqAd6GrT4QOdCgf8igR86e1BvyLOd5vPyzRSqAhCXr1LJdOVuSe/kg52lx+Vbvwgva+5TbojJh1+uxPmXpKu1vna3G3zBrweozfdvUtvunuXfnhjTp/796f1N//+tP7vv39Mv/v5x/TKA5166yuep5942V51tDGtCoC1zS8llbi1qGu3FpS4taiE830t5+erN5eD29TsvOYWUwWv1d7SlK2o3b0nqHDAr642v8JBX7bqlg5uPrX6+Ncr3Ncw/xQaY2KSDkuKS4pKiltrx91tVQXMz0pPnZMuf0V68lHpybPSgjPdRHCvdOerpH3vlfa9Str98rJV59yyu71F73hDVO94Q1QXp27qc//+tD77taf0kb/6pj722W/plQc69WMviujHXhjR3XuC/Ncw0EBSKavZhSXNzi1pdn5JM3NLmplb1Oz88rYbc5mf09vTQS4d4K7dWtTtxbUHI/iaPArtaFZnm0+RoF8Hw23qakt3jWa+IwG/ugI+dbX5eZ8Ndae+E4DDGBOVNGit7c3ZNmqMiVtr4y42betuPL3c9Xr5UenZb0g2KcmkK3P/4Zi07zXpcBfat/4wozp3MNym9x19gd7743fp28/c0N9+/Rl96TvP6YHPf0cPfP472t3u14+9MKI3vjCiVx7o1O525vRqVKmU1WJqeaqH3Pm7FrNTQKS0sGTT2zK/Jwufk3TecM+8L5U7T9fytrwNkuSMKPQaI48nPXLQ64wc9HqWRxKut92TuYbHuYZZ3mZy9mV+9pjl0Ym55+cem9nvyblWZtSjtek/XcpayUopm/7zpqwz0blN/xFTOcc9PZvS9344IytpMZmeH20xufy/41Iq/b/1Uiq1cnvOcfNLSd1eTGpuwfleTDnf6ek75pacb2f7bee4YgT8TQq2/P/t3V9vG1d6x/HfQ4qyJDsRrY27WTfJdpmLdOO2KSSlGxS9SLAy0NsC8uYVWOpVi95Y6Cso5Hcgp28gkYFe9GZRKWizLdACsYTN9k/Q7VqLZjdoduNacmMjcSzx9OKcIYcjyqJEDudw9P0AhIYzwzOHejTiw3PmnBnT+XNjmp6s6XJ9Qq9eflb1yZounh/X9GRNF6fGdXGqpukpv1yfqmmyRjcpyq0USZ6kZUlrmXVrklYlXRt+dU6peeAHRPziX9otdXuf+G1jk9IL89If/YVP6l6Ylybrxda3IGamK5endeXytFb++Lf1q//7Sh/89HN98J+f64f/9pneu+O7q781PaHXXqjrtRfr+v0X63rl+Wd0cao2kv/U/YSi7QlJ9w/88n6z/UHaDJOONsO+7Z+d65pNhQlJnZrN1Pr0ZKZJWUk5Hfuqa9npyU4Pmu06H4TEIVlOytw/cHqcScCe7Dvd2/1SNz/6x5AcJImY60je9rsNOxyi5E/oTM0d9k8/6ruIc2MVTY5XNVnzj3O1qiZrfl19qhaeh8d4VRO1qp6d8AnchXM1XZgYayV0z4Tl8+NjqlRG75wGhqEsSd6iDid5dyRtFFCX3n39SPp0q7Pr9fEDv+3CN33r3Pf+VHrpDen532MKgyN889kJ/WD+Rf1g/kXtHzT1k08f6Mef7OnHv9jTR7/c0w///bPWvtOTNX3nufNqPHde7ouv9dnUJ7r0zDldSkauTdY0Vaue6EPDOafH+009ftJst0bs+wlHvwyTjn755ECPHvvJRr/8+iBMPtqehPRRmKQ0WU5e9+QgaR0ZnWwiabVqzYsVWrkqHS1PptqY+Tm30vNvVSuaqpmer090zMtVq1Z0bqyiWtU65+vq2Mfac3xVK6qNJduPf41vZUvmJ/Q65uV6yheDZjNJbDOJdSqpbc3y33Qdye5B+u4AzfYdAtKvT1rVkoQ6uYNAM71v6jXZfZPkPUm6kxa/pBXSlLQEpu7wkmoJNJM+/vhjXXn1VZlJYxX/Ox2rJHOmtZeT+IxVLcyjlopZtUIyBgzZyCd5ZlaXvwbvfnq9c27P/D/uRjRdtl/8KrTShcdnP5Ga+37bpe9Kv/Mn0otv+KTu4m+Vuus1L2PVimZfuqjZl9oTt+4++lof/XJPP/v1Q/383iP9/N4j/fPO/+p/HjzR3/zsX7uWk0wS2vrwU3vyTjNT0zl9Fbqcvto/OHGLzni1oqlzVU3VqppKTUx66Zlz+vb4VOv442MVjVXMf1hWTNWqqVbxH6KtdRX/IZp09aUnIU26ANNdislEpUmy1dl1qM4uw0pnYtbqdqyoVb5Zern/v1k/LcTrfZczLJWKv81dLa7psQZqeu+/9OZrl4uuBoATGvkkT9KM5JO6I7Y35AdjHO3eT6W/XlDrO3y472h7Wf55erlj2/H7fe/T/5D+IbQojU1Ivzkn/eGfhevpXpeYTT43F8+P681XfkNvvtJ5S6a/e//vdWXuDX3+xePWw1+8fdC6DVDrGiUlP30XnZk0WatqolbRRK2aelTCet/dNFWr6vw5PwlpMhnp1HhVtSoXcAMA8jXy964No2q3nHOHmhDMzEm62m2UrZktSVqSpCvfmpj70cofhC1Oduh34pRcdm3u8LpWmS69znWsf1R9Vo++8bt6MP1dPbzQkKvQ9Vq0hw8f6sIFJlmODXGJDzGJE3GJz7Bi8tZbb3Hv2qdxzt2SdEuS5ufn3cyff5Dr8bYzN5JG8bI390YciEt8iEmciEt8YotJafqMwrV5AAAAUDmSvOR6u5n0ylTSF8egCwAAgCEa+SQvDLjYkZRtyZuRtBfNyFoAAIAhGvkkL9iUv6VZ2mxYDwAAcOaUJclb0eE7WyyH9QAAAGdOKUbXhomPV8xsVdKH8nPjrdJVCwAAzqpSJHmS5JzblrRddD0AAABiUJbuWgAAAKSQ5AEAAJQQSR4AAEAJkeQBAACUEEkeAABACZHkAQAAlBBJHgAAQAmR5AEAAJQQSR4AAEAJkeQBAACUEEkeAABACZlzrug6FM7MPpf035KmJT04YrejtnVb323dc5Lu9VHNfjztfeVZRq+vOW6/vOIy6jE5bTm9vKaomEjFxSX2mPSyH+fK4MqJPSYS50o/+500Lr3Galgx+bZz7tKxeznneISHpFsn3dZt/RHr7sT4vvIso9fXHLdfXnEZ9ZjkGZeiYlJkXGKPSZFxOYvnSuwxKTIuZ/Fc6TVWRZ4r3R5013b621Ns67b+aeUUYRD1OU0Zvb7muP3KGJdB1SWvuBCT4ZbDuXK0ouJCTI52Fs+Vk8QqGnTXDomZ3XHOzRddD7QRkzgRl/gQkzgRl/jEFhNa8obnVtEVwCHEJE7EJT7EJE7EJT5RxYSWPAAAgBKiJa9AZtYwsxtmthB+1ouuE9rMbKPoOsAzs1kzWwrnybqZNYquE1pxWTCzRTNbIy5xMbO1ousAz8xWzcyZ2a6ZbQzrXBkbxkFwpHXn3Jzk+/ElvSPpWrFVgpktSGpIWii6LpDCl59559yt8HxB0oaklwutGCTpfUnfcc7tmdmMpHVJcwXXCWqdJ0uSlouuCyRJd51zNuyDkuT1ycwWJe055za7bJuVNC9pRz5p2En2C9vuJ/uGf5IkFQNy2rhIUipGfAseoD5i0pC0ova1LnckNcys7pzby7/m5dbPuaKQ4IXl+9nX43T6jEnyxei+JM6PAeo3LoUoeg6XUX7It/TsSlrosq0haSOzbl1SIywvyrfkpbfflTRb9Psa9Uc/ccmsd0W/l7I8+o1JZnlB0m7R76kMj0GdK2HbRrdyeAw/JpIWw0/Ok0jiIulG+NxfkLQqqT6MenNN3imEa+nW5AN71LfXZUnZlqA1+eBK0kxO1TuzBhQXDNCgYuKc28nsf32Q9TxrBnmuJNcWy3/IFd9yMaIGFZPQI0QcBmSA58ot59ztcI68K3+pQ+4YXdsnM7sraTn7zy2sv5r+cApN6LvOOQvNvsvOuaup7buSvu+c2x5S9UvrtHHJ7Ouy63B6A4rJkqT7zrnbw6jzWTCIuIRtS5Kupf+n4XT6+Fypy7cebYdtu865i8Ose5kN6lwJ252kiy7nS05oyctBcqIpk/UnwQyjanbUpTWPBC8/PcYFQ3SSmIQWih0SvPz1EpdUC17iPUkLnEf56PFcWZA0H0aiL0mqh2VikpMez5VZM9vKvjbvBE9i4EVeZqSnBrDhnNs0ayf44SSkiT1fx8ZFPvnG8PQUk2SgUqqFYpFkL1e9xEWSvpFZt5fpWsfg9PK50nFOmNmaC6PSkZtezpU7SnXnhi+sQ/n/RZKXj17nu7sevglvS5oV1xnlrae4hIRiISyvimuN8nRsTMIXoPflWyWS1Tsa0j/JM+rYuIQvqvXQYiRJVyV9P99qnWk9z6MaWpeWwvINSbdJvnPTy7myZ2Y7qXPlZQ3p854kr0ChVSLpniWJiEQqLjeLrgtagy64rihCmZYjWowiEVqVbor/YdEoqqGAa/JyxB0s4kRc4kNM4kRc4kNM4hRrXEjy8pE0i3cMrEj9EdBsXgziEh9iEifiEh9iEqeo40KSl4PQVL6jw331M+LC5MIQl/gQkzgRl/gQkzjFHheSvPxsyt/iJG1WXHtXNOISH2ISJ+ISH2ISp2jjQpLXvxl1H12zIulaZt1yWI/8EZf4EJM4EZf4EJM4jVxcuOPFKYS+9r+Un/9mUb6pdlN+qo3bqf1mJb0t6cOw7zZTceSHuMSHmMSJuMSHmMRp1ONCkgcAAFBCdNcCAACUEEkeAABACZHkAQAAlBBJHgAAQAmR5AEAAJQQSR4AAEAJkeQBAACUEEkeAABACZHkARhJZraWeb5oZgsneP0NM9sYfM0GZxTqCCBeJHkARlUj8/yq/C2HevW6pJ6TwoKMQh0BRIokD8DICfeTzCZ0Dedcz0mec+6ac84y5a6aWTZ5zN1Rx+1WRwDoFUkegFG0IGkreRKSvr1+Cgxl3JA021/VRuO4AMqPJA/AKLoqaTP1fF7ShwXVBQCiRJIHYBRlu2azSd+xQhepC8vrknbDpnUzc2a2m9l/0cy2wrat9CCPUNZuWF4zs7tmVjezhpmtm9lueN1Gulv2uOOm65haNxvKSeqx1OV97XbZbzaz32Kop+tWNwCjjyQPwEg5omt21jm33Uex1+UTRUlalvSypLnUMRclrcsnktfkrwc8lBSZ2ZZ8q+Kac24vlJUuf0apbubjjpsV6rEVjn9V0ruS1rIjjSXVQ33XQrmN8DwpJ3l+OxwveU/1o44NYPSMFV0BAHiakMCkk6kZSfXM1CLzmeerzrmeW/acc3tmlrQM3u8ygOMdSTedcyvh+e2Q0C1LStbVw2uTpE2p/ZP3cl3SlpktOudu93DcrHck3XLOJcnjppltyyec65n3vOKcux2O+7KkG2ZWD8ln0qr3V+H5tnzCB6BESPIARC2V0EhqJX0rITlR6DZtOOdu5XH80M1Zl0+SbmQ23888X9ZTOOe2zUw6PP1LL/VYCPXoaLVzzm2a2Z58a1w6yUsv380Ul2x738zelbTZZ0sogAiR5AEYNY0kwQuuSVrN83jh55wOdxN3JHndWuJCF+vboZx+rnlLXtuttW8nW3bmd6TsNjO7Kv97Ww313Ey3QgIYfSR5AKKW6a7t2lUrqRFayBIn6q49RiupOsk8fJIU6tmQT6TuhNVbR7+ip3o05LtX0xqp8nsSfj9z4RrHJUmrZnbDOXfzlPUDEBmSPABRS3fXhu7S7SSBCwMIVrJduqeU7XpNjr8dukNX1R4kkdSnflSLWajbgqRrqWvjug1s6HrcLpIkblmpbuHQUpgMtDixUP+bZva2/B02AJQESR6AUfJ6pqVpQdJA7u0aujAlaTn5meq+vC4/xUkyYrUun2jt6Ijr8JxzO0lyGMrbU3uQRq/Hze63LD+adkZ+ZG3SSnj7JC2XYdqVlfBekq7eWWWu9wMw2phCBcAoybaEzemE8+Md46Z84riqVJdoaImbC8ffkB/luqMuSVvGdfku5vVQ5lGtbV2PmxUGl1xVe0qUZfmWzGvH1CPrPfnRtMupcm7mNXgFQDHMOXf8XgBQsG5ds2a2wWABAOiOljwAo6Kja3YQ96sFgDIjyQMwKrK3LhvY9XgAUEYkeQBGxf3MSNaGBns9HgCUCtfkAQAAlBAteQAAACVEkgcAAFBCJHkAAAAlRJIHAABQQiR5AAAAJUSSBwAAUEL/D6/JnXd1vDlfAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(10, 8))\n",
    "plt.plot([f(x) for x in conv_x_da], label=\"DA\")\n",
    "plt.plot([f(x) for x in conv_x_al], label=\"AL\")\n",
    "plt.plot([f(x) for x in conv_x_admm], label=\"ADMM x\")\n",
    "plt.plot([f(z) for z in conv_z_admm], label=\"ADMM z\")\n",
    "# plt.yscale(\"log\")\n",
    "plt.xscale(\"log\")\n",
    "plt.legend(fontsize=20)\n",
    "plt.xlabel(\"\\# iterations\", fontsize=20)\n",
    "plt.ylabel(\"Objective\", fontsize=20)\n",
    "plt.grid(True)\n",
    "plt.yticks(fontsize=20)\n",
    "plt.xticks(fontsize=20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaoAAAEcCAYAAACWHPCeAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzs3Xl8lNW9+PHPmUky2TOZrOxhgoDshIC4gwaXam1rA7TaVq9a0mvb2/a2JeXXfbVge9vb21YTbNXb9ioEbetSF4LgvgBRFESRTBBEgYRkspL9/P54noTs62Semcn3/XrNa5Jnmed7GCbfOctzjtJaI4QQQgQqm9UBCCGEEAORRCWEECKgSaISQggR0CRRCSGECGiSqIQQQgQ0SVRCCCECmiQqIYQQAU0SlRBCiIAmiUoIIURAC7M6gFCQnJysMzIyRnRufX09MTExvg0owEmZxwcpc+gbbXn37t1bobVOGew4SVQ+kJGRwZ49e0Z07q5du1ixYoVvAwpwUubxQcoc+kZbXqXU+0M5Tpr+hBBCBDRJVEIIIQKaJCohhBABTRKVEEKIgCaJSgghRECTRCWEECKgSaISQggR0OQ+Kh84WdPIf20/1PdOrQc8t+LDFpKPV3PuhHjsNjUG0QkhRHCTROUDp2qb+N2O9/rdrwbIP1rD/73zAnGOMLIzEjnPncR5013Mm5RAuF0qvEIIIYnKB+ZPSmDPL68Z0bkPP/kM9vRZvFpWyaue0+x8txyA6Ag7S6Ylct50F8umJ7FwSgKOMLsvwxZCiKAgicpirkgbKxZN4hOLJgFQXtvEa2WVvFZ2mlfLKvnV00aTYkSYjcVTnJznTmL5dBeLpyYSFSGJSwgR+iRRBZiUOAfXLJjANQsmAFBV38zuI5VGjavsNL9/5j1+pyHcrlgw2WnWuFxkZ7iIdcjbKYQIPfKXLcAlxkRwxdx0rpibDkBNYwt7j1TxStlpXvVUUvCchz/uKiXMpjjP7eKKOemsmpPGRGeUxZELIYRvSKIKMvGR4aycncrK2akA1De1UnK0ihcOV7D9wEl++MgBfvjIARZMTuCKOWlcMTedc1JjUQON6BBCiAAmiSrIxTjCuPicFC4+J4UNV5/L4VN1PP32CZ4+cJJfPX2IXz19iIykaK6Ya9S0sqYmyjB4IURQkUQVYmakxjIjdQa3r5jByZpGtr99kqffPsm9L5ZR+JyHpJgIVsxKZWlGIkumJZKZEotNEpcQIoBJogphafGRfG75ND63fBo1jS3serecpw6c4Jl3TvJQyQcAxEeGkTUtkSVTjcS1cIqTGBmUIYQIIPIXaZyIjwznuoUTuW7hRLTWlFXUs/f9KkqOVrH3/Sp2mfdv2W2KaUnRZKbEMiM1tstzDHGR4RaXQggxHkmiGoeUUrhTYnGnxLI6ewoA1Q0tvH6sipKjXg6dqKW0vI5d756ipe3sFFBp8Y5uCazj57R4hwzWEEKMGUlUAoCE6HBWzEplxazUzm0tbe0cq2zg8Kk6Ssvrzec6/l5ynNqm1s7jYh1hZKbEkJkay5wJ8cyflMDcSQlyX5cQwifkL0kflFLbtdarrI7DauF2W2fNqyutNeW1TRwur6O0SxJ78XAFD5ccB4z5DacnxTBvUgLzJyWwYHICC6c4iQyX2TSEEMMjiaoLpVQO4AZyrI4lkCmlSI2PJDU+kgsyk7vtK69tYv/xat46Xs3+49XsOVLJI/s+BCDCbmPB5ATS7c3oCadYMi2ReOn3EkIMIuQSlVIqF/BqrYv72JcFZAMejITk6Xpcx89KqQI/hRtyUuIc3W5IBqioa+KNo97OqaCeONrCY57d2BScOyGeZdNdnDc9ieVuF87oCAujF0IEopBKVGaNaDOwuo99bmBj1yY9pVSRUsqjtfb4McxxJznWQc6cNHLmpAHwVPFOYjPmm5PvVvLAa0e598UjKAVzJsRzQWYS52cmsTTDJSMNhRChkajMJJQP7AUq+zksD+hZUyoANtJHYhNjxxGmuHBGMhfOMJoNm1vb2feBl5cOn+ZlTwX3v/Q+m58vw25TzJ+U0Jm4sqe5ZMZ4IcahkEhUZo0oD0Apld/PYbn0TlR7gO1jGJoYgogwG0szXCzNcPE1zqGxpY2S96t42XOal0pPU2hOvBtuVyyeksjCKQmdgzQykmJkZg0hQlxIJKrBKKWcGH1S3WpbWmuvUgqllFua/wJHZLidC2Ykc8GMZL6JMfHu7iOVvOwxZoy//+X3aW5tB84Ojc9IjiEjKQZ3SgxTXdGkJ0SSEusgTFZJFiLojYtEBbjASEz97HdjDLAQASjGEdbtHq+WtnbeO1nH/uPV7P+wGk95PXuOVPHIvg/RZ+9PRimjfyw9PpK0eAdp8ZGkxUeSnhBJesdzQiRxjjC5YVmIAKZ01092CFBKlQJ5XUfzmaP99mqte/01UkppYJXWutg8Lgej32oTsL2v0YPmeeuAdQBR06KWzPrxrBHFG6kimeiYyKTwSUyMMJ7TI9IJV6E7iKCuro7Y2NjBDxym5jZN+RlNeUM73iZNVaOmqknjNZ+rGtupa+l9nsMOiQ5FYqQiMdLW+XNSlCI12kZKlCLCPrpENlZlDmRS5tA32vKuXLlyr9Y6e7DjxkuNaki01iVACUaSGuzYQqAQYMqcKfoL874womu+XfY2tZG1vOR9iabaJgDsys60+GnMTJzJOYnnMDNxJjMTZzIhZkJIfPPftWsXK1assOTajS1tnKpp4kRNIx9Vn+FkTSMnqps4UXOGE9WNHKlu5LUTTbS2d/8Clx4fydSkaKa5opmWFM3UpBgykqKZlhRDQtTgXyqsLLNVpMyhz1/lHVeJSinlHKD5b8TSotP4xpJvjOjcXbXGG93W3sax2mMcqjrU+Xir4i2ePPJk57Gx4bG9ktc5iecQEx7jq6KEvMhwO1OTopmaFN3vMe3tmoq6Jj7wnuHo6QbeP93A+5X1HD3dwK5D5ZSbXyg6JMVEkJEcw3TzkZFkPidHEx0xrj5iQoyJ8fIp6uh/cgGdicocZNF1v2XsNjsZCRlkJGRwRcYVndvrW+p5r+q9zuT1XtV7PO55nC0tWzqPmRQ7iVmJs5ifMp+FKQuZmzSX6PD+/xCLgdlsZ2feyJqa2Gt/Q3MrxyrPcOR0PUcq6ikzH8+/V862vR90OzYt3sH05BgczU28q0o7k9nUpGgcYTLUXoihGBeJyhzd5wGcPXa5MGaxGF2iOn0Y7rt2RKfOaoyAOemQOrvP/THhMSxKXcSi1EWd27TWfFT/UbcE9k7lOzxz7BnAaDqcmTiTBSkLWJiykEUpi5gcNzkkmg0DQXREGLPS45iVHtdrX31Tq5nAGiirqKOsooEjp+s5cLKVZz94p/M4m4KJzqhuNTCjFhbD5MQowmW0ohCdxkWiMhVjTJ9U0mVblrl9dLSG9raRnEjqqd3wx+Uw5xNwybchfd6gZymlmBg7kYmxE7l0yqWd272NXt6seJN95fvYV76PR0sfZcu7Rs3LFenqTFwLkhdwbtK5xEX0/kMrRifGEcbciQnMnZjQbfuuXbtYfN6F3WpgR04bz/944zi1jWdno7cpcEZH4IwOxxUdgTM6AldMOM7oCGIiwohx2IlxhBEdYSfWEUZ0l20d+6MjwrDL/WUiRIRionLRu+YExswVRZgDIEx55mN0ks+BW54Y0amvPP0IF9rfgFcL4e1/wOxrjYQ1cdHgJ/fgjHRyyeRLuGTyJQC0tbdx2Hu4M3G9Wf4mu47t6jx+StwU5iTN4VzXuZ3Pzsi+/umELyREhbNwipOFU7r/G2utOV3f3JnEjlY2UFnfjLehhcr6Zj6oauCt481UNbR03j82FFHhZvJy2ImJCCPW/DnaEUZsRBgxjjBizd87fo7p3B5GQlQ48VHhxEeGyf1owlIhkajMvqYNGPdDOYGNSqlVGMPLt0Fn81++UmojsNs8dqPVN/q2RMTDih/ABV+FV+6GV++Cdx6DmVfBJeth8pIRv7bdZmeWaxazXLNYM2sNYNS69p/ezzuV7/D26bfZX7Gfp4481XnOxJiJXDDpAq7OuJolaUuw26QfZawppUiOdZAc6yA7wzXgsS1t7TQ0t9HQ3Ep9Uyv1TW3UNxvPDc2t1DW10tDUZjw3t1LX1GYe10p9cysVdc3Un27oPKe+uZWh3KES6wgjPjLMSFxR4SSYj/hI8zkq7Ozv0ca26iZNS1u7NGOKUQuJRGWO5Otv6qSux3UMPw88UYmwcgOcfzu8Vggv/wHuuQwyL4dL18PU5T65jDPSyUWTLuKiSRd1bqtuquZg5UEOnj7IWxVv8bjncbYd2kZKVApXZlzJVdOvYkHyAunjCgDhdhsJUbYhDYkfivZ2TWOrkdjquyS12sZWahpbqDnTQvWZVqrPtFDT2GI8n2nhWGUDB860UNNoJMf+fG3nE8RHhuGKiSAxJgJXtPkcE0Gi2aRpPJ/dnxAVLtNiiW5CIlGFlMgEo+nvvC/B7j/BS/8Df74Spl8Cl+ZDxkWDv8YwJTgSWD5hOcsnGMnwTOsZnv3gWZ4se5Kt727lrwf/yqTYSaydtZY1s9bIcPgQYrMpoiOMfi5G2GXZ2tZOTWOrmdSMhOZtaOG1fQdInphBVUMzlfXNVDU0c6KmkYMf1XC6vpmmfpoxbQpcMQ5S4xykxTtIjTNmFkmJjyQtzkGqOdNIcqxDamvjhCSqQOWIg4u+Dsu+CHvuhZd+B/ddA1MvMGpY7hXGHEFjICosiqsyruKqjKuoba5l57GdPHL4Ef5r739xz1v3cMO5N3Dj7BulP0sAEGa34TJrSV3FVR1ixYpz+j3vTHMblQ3NVNWfTWSV5s/ltU2cqm3iZE0j+z+soaKuqVcTpVKQGudgSmI0U1zRTE6MYkqi+WzO9yiJLDRIogp0ETFwwVdg6a1Q8r/wwm/hL5+EyUuNGtaMnDFLWABxEXFcl3kd12Vex/6K/dzz1j3cve9u7j9wP6tnrua2+beRGNn7XiMhBhMVYWdSRBSTnFGDHtva1s7p+mZO1jRyqsZIYidqGvnQe4ZjlQ28VlbJP984Q9cJRew2xVRXNJkpscxIPfvITImRdc6CjCSqYBEeBeflwZKb4fW/wgu/gb/lwsTFxqCLWVePacICmJc8j9+u/C2Hqw7z5/1/5m8H/8bfD/+d2xfeztrZawm3yYdfjI0wu61zUuH+tLS1c6K6kWOVDXxQdYajlQ2Ultdx+FQdzx46RUvb2SyWHh/J3InxzDWXi5k3KZ70+Ejphw1QkqiCTZjDqF0t/jy8+SA8/2t48LOQPh8u/+GY17AAZiTO4BcX/4Jb59/Kxtc2snH3RrYe2sq3s7/NxZMvHtNrC9GfcLuNKS6jGbCn1rZ2jlY2cPhUHYfL6zpn39/57qnOWlhSTATzJiWwZFoiSzNcLJrilIU6A4QkqmAVFgFZX4CFN8BbRfDsRqOGNf0SWPUTo6Y1xjKdmRSsKuDZD57lzt13cvuO27lw0oV8O/vbZDozx/z6QgxVmN2GOyUWd0osV3TZ3tDcysGPajnwYTX7j1ez71g1vyk+hNYQblfMm5TA0gwX57uTWO5OksRlEUlUwc4eBos+C/M+DXvvNRJW4QqYvxou+x4kZozp5ZVSrJiyggsmXsAD7zxAwb4CPv3Ip8mdmcvti27HFTnwfUFCWCk6Iowl0xJZMu1sP2t1Qwt7j1byWlkVe45Uct+LRyh8zkNEmI3zpru4dGYKK2alkpkSI02FfiKJKlSERRh9WAs/Ay/+N7z8R3j7n7BsHVz8TYge24QRYY/gprk38fHMj3PXG3dRdKiIR0of4Vr3tdww+wZmJM4Y0+sL4SsJ0eFcNjuNy2anAcbSMK+VVfLsoXKePVTOzx4/yM8eP0hGUjQfmz+BaxZMYM6EeElaY0gSVaiJTIDLfwBLb4Odv4BX/gglf4GL/9NIZOGDj7AaDVeki+8u/y6fnf1Z7n/7fh4pfYSiQ0UsTV/KdZnXkTM1Z0yvL4SvRYbbuWRmCpfMTOH7wLHKBp49VM5TB05Q8JyHP+4qJSMpmmsWTGDKMKa4EkMXciv8WiE7O1vv2bNnROeO+cJjJ9+G4h/Be09B/GSjOXDBGvDT1EhVjVU89N5DPPzewxyrPYbD7mBB5AK+ufKbzE2a65cYAsF4W1APxkeZK+ubeerACf711ke8VHqatnbNsuku1mZP4WPzJ4R8n9Zo32Ol1JBW+JW74UJd2hy4cSvc9CjEpsA/vgSbL4PyQ365fGJkIrfNv43HP/U4f/3YX/nkjE/yZsObfOaxz3DTEzfx+qnX/RKHEGPBFRPBZ5dN5S+3nsfL37mM3JnhnKpp5JtF+1j282J+9tjbHPeesTrMoCeJaryYfgnc9gxcfw94j0LhpcYNxH6qUSulWJiykO8t/x4/nfxT1i9dzwd1H/CFJ77Ad1/4LpWNlX6JQ4ixkhofybXuCHZ+awUPrlvOpbNSuPelI1yyaSdffeB13vqg2uoQg5YkqvHEZoMFq+HfX4TJ2fDIV6HoZjjjHfRUX4qyRfH5OZ/n0U8+yq3zbuVfZf9i9aOr2Ve+z69xCDEWlFIsdyfx+xuyeG79Sv7tggx2vnOKj//+BW69bzf7j0vCGi5JVONR/ET4/D+MG4QPPgp3XwxHX/V7GNHh0Xx9ydd54JoHiLBFcPOTN/P39/7u9ziEGCuTnFF879o5vLzhMr595Sx2H6nk2v95gdv/tpfDp2qtDi9oSKIar2x2YyTgrU8bM1ncezU8u2mEKxWPzmzXbB689kGWpS/jBy/9gL8d/JvfYxBiLMVFhvPllTN4Pv8y/uOyGTz7bjlX/fZ5fvrY29Q0tlgdXsCTRDXeTc6GL70A866HnT+H+z8ONR/6PYwERwK/v+z35EzN4Zev/ZL7D9zv9xiEGGsJUeH85xWzeG79SlZnT+bPL5Zx2a+epWjPMWQEdv8kUXWhlMpSSq1TSq1XShUppdxWx+QXkfFw/Wb45N3w0T64JwdOveP3MMLt4dx56Z1cMe0KfrXnV/zz8D/9HoMQ/pAU6+CO6xfwzy9fyBRXFN/e9iaf+9OrfFDVYHVoAUkSlclczj5ba12otd4EFADbLQ7Lf5QypmL6tyegvdVYrNGCfqswWxh3XHwHyycs54cv/ZDnPnjO7zEI4S8LJjt56EsX8PNPzeONo16u/M1z/O3V96V21UNQJCqlVK5Sqs8pDbrUgnI6nkd4GTfdl7PfA7jNBDZ+TFhg9FtFJ8H/fgLefcLvIUTYI/jtyt8yyzWLb+76Jm+Wv+n3GITwF5tNceN503jy65ewaKqT7/59P7fct5uq+marQwsYAZ+ozMSzuZ99bmCjWQsq1loXAnkjabLTWpcAq7psyga8Wmv/jt0OBIkZRrJKnQ0P3misf+VnMeEx/OHyP5ASncKXd3yZw1WH/R6DEP40xRXNX289jx99fA4vHK7gmt89z973q6wOKyAEbKJSSrmVUgUYNZ3+7gbNw2ii66oA2DiSa2qtPT1e+4sjeZ2QEJMMNz0G7kvhn1821r3yc3NEclQyd+fcTbgtnFufvpX3qt7z6/WF8DelFDdfOJ2H/v0C7HbF2oKXuffFsnHfFBiwiUpr7dFa55m1pP7kAiU9tu0xtwNgNgduHODRq6lQKbUO2KK13uaTwgQrRyx8dgvMXwM7fmKMCvSzqfFT+fOVf8au7Nz85M3sPrHb7zEI4W8LJjt57KsXs2JWKj9+9G2+/8/9tLaN3wlvg3b2dLPvqFdtS2vtVUqhlHKbyW6gRNfX6+YAHq11sQ/DDV5hEfCpAgiPhOfuhOhkWP4lv4aQkZDB/Vffz5d3fJl129fx/eXf5/pzrvdrDEL4W0JUOIWfX8LGp96h4FkP759u4A83ZhEfGW51aH4XsDWqIXCBkZj62T/sfiqlVBZQ2ZGklFK5g5wyPthscM1vYPa18GQ+vLnV7yFMiZvCX67+C0vTlvLDl37Ixtc20mbBzclC+JPNpthw9bls+vQCXi49zZq7X6a8tsnqsPwuKJb5UEqVAnldazlmUtmrte61WplSSgOrhlMrMgdg7AW6jvLzaK37XFPdbB5cB5CWlrbkwQcfHOqluqmrqyM2NnZE5/qbra2Z+W/9hITqt9k/77tUJi0Z0euMpsxtuo1/VP2DXbW7WBS9iJuSbyJM9d8woJTC4XAQERFh6cJ2Wutxt7Ce1WXWWtPc3ExTU5Pf+njG8vO8v6KN373eiMuh+PbSSJKirK9njLa8K1euHNIyH2itA/4BlAI5PbZlGeH3ebzuefxYPpYsWaJHaufOnSM+1xJnqrW++2Ktf5qm9dFXR/QSvijzffvv0/Pum6e/9szXdHNbc5/HNDY26kOHDuny8nLd1NSk29vbR33dkaqpqbHs2laxsszt7e26qalJl5eX60OHDunGxka/XHesP8+7y07reT94Ul9wxw5dVl43ptcaitGWF9ijh/A31vqUPErj7j4nq0XGw40PGRPb/m01VFgzEu+muTfxnWXfYcfRHfzslZ/1+Y25srKSxMREkpOTLa9RCf9SShEREUFycjKJiYlUVobGMjLZGS4eWLecMy1t3LD5lXEzk0UwJ6qOoeSurhu7JC4PYmzEpsDn/25MbLvl89BUZ0kYN557I+sWrOPh9x7m3gP39tpfW1tLfHy8BZGJQBIfH09tbejMVD5vUgJ/uXUZtU2tfO6eVzlV22h1SGMuaBOVNgZReOjepwRG4vLq7vdECV9LnAa5f4aKd411rSzq6/zKoq9wZcaV/HfJf/Pyhy9329fW1kZ4+PgbISW6Cw8Pp60ttAbezJ2YwH3/toxTtU18/p7XQn4Wi6BNVKZijBkkusoyt4ux5l4Bl30fDjwMr95tSQhKKX5ywU9wJ7jJfy6fk/Une+0X41uo/h9YMi2Re76QTVlFPXl/2UtTa2gl465GnaiUUv5oW3HRu+YExtx8q3tsy6P7nH1iLF30DZh1DTz9PXj/JUtCiA6P5tcrfk1jWyP5z+fT2t5qSRxC+NsFM5K5c/UCXjtSyYaH3grZGSxGnKiUUtOVUocBr1LqtFLqU132fVopNapedqWU05w5oggjSW1UShV0vbfJbP7LN4/LVUqtx5j7T5r9/EUp+NRd4JxqLGtfe3LQU8aCO8HN95d/n70n9/KHN/5gSQxCWOETiybxn6tm8vDrx/n9M6E5J+ZoZqb4JUbtZQ+wFFinlFJa64e11g+ZCWbEOpLQEI4rofc0SsKfIhNg7V9h82Xw6H/AZx80EpiffTzz4+w9uZd73rqH+cnzmcAEv8cghBW+etkMjlTU8+vth5iZHseVc9OtDsmnRtP0t0drvUNrXa2NmcvXAElKqcvM/aFZBxV9S5sLl/8ADj0Jb/yfZWFsOG8Dc5Pm8v9e+H+0tI/vJb5LSkowpxPr9khMTCQvLw+vd/CFATIzM1FKUVIi3wUDmVKKOz49nwWTE/hW0T7eP11vdUg+NZpE1et/udZ6M6CUUotH8boiWJ337zDtQnjyO1D9gSUhOOwOfrvyt0SHRVPZWDnukxXAxo0bKS0tpbS0lL1797Jhwwa2bt3K9OnTB0xAJSUleDxGK/qWLVv8Fa4YIUeYnT/ckIVNKb701xIaW0JncMWoalRKqS8qpd5TSmV0bNRa7zB/rB5NYCII2WzwiT9Ae5uxNIhFHbvpMen8z+X/Q7tu52jN0XE/J6DT6cTtduN2u8nKymL9+vWUlZXhdru5/PLL+z1vy5YtZGVlkZubS2HhsOZ2FhaZ4ormN2sXcvCjGn74zwNWh+MzI05UWuvXga3AGq31kT72jWwiOBHcXNPhip+CZxfs+ZNlYcxNmkuiI5HG1kaO1x0P2dFQI+V0Otm8eTNer5dNmzb1eUxhYSFr165l1apVeL1eaf4LEpfNTuP2FZls2XOMJ/d/ZHU4PjGq4elm/9Tr/ewrG81riyCWfQu4V8L2H0KNdR+UyLBI0mPSqW2u5VTDKcviCFRZWVnk5OSQn997zFJJSQler5fc3FzWrFkDQEFBzzVKRaD6xqqZzJ+UwIaH3wqJmSuC/YZfEYiUgmv/C9paYPv3LQ3FFekiMTKRijMV1DTVWBpLIFq1ahVAZ19Uh4KCgs7mQqfTSVZWFlu3+n95FzEy4XYbv1m7kIbmNr4TAvdX9Tk8XSl1N8aMD6MtnQJKtdZrR/k6Iti43HDh1+C5TbDkZsi4yJIwlFKkx6RzpvUMH9Z9SFRYFOH2cH786AHe/tB/iautrQ273T6sc+ZMjOeHH587RhEZ3G5j2TaPx9P5M8DWrVtZt25d5+9r164lPz+f4uJicnJ6LYotAtCM1Dg2XD2bHz36Nlt2H+Mzy6ZaHdKI9Vmj0lp/SWudrbVeOspHtiSpceyibxg3Aj/+LaN2ZRGbsjE5bjIaLf1VPTidxoQvXYeqFxcX4/V6O2tbALm5xn32RUWjuj1S+NkXzs/gfHcSP//XwaBuAgzapehFEIiIhqt+CQ/eAK8VwvlftiwUh91BWkwaH9V9RFVT1ZjXVHqqra0lLi7Or9ccio4BEllZWZ3bioqKcDqd3WpOHc2AhYWF0lcVRGw2xc8/NY+r/vt5fvLo2/z+hqzBTwpA0kclxtasj8E5V8DOO6C+wtJQEh2JxITHcLL+JC0W1vACye7duwF6Nft5vd5eNwp39GMVF8ucz8HEnRLLV1fO4LE3P2LnO8E5qKi/PqpfAtN9dA2P1nqDj15LBBul4IqfwR+Xw/O/hqvusDAUxYTYCZR6SznRcIIpcVMsiyUQeDwetm3bxvr16zu3dTT7FRUVdatldRy/atUqCgoKpJ8qyORdmskj+z7ke//Yz/b/vIToiOBqTOsv2jvosSDhKITG0ppi5FJmwaIbYfc9sPzfLQ3FYXeQHJVMeUM5dY46YiNiLY3HKl6vl9WrV+N0Otmw4ez3yIKCApxOZ2efVFcdNwxv27bNn6EKH4gIs/HzT81nTcHLFD7n4es5M60OaVj6G0xRrbUu89FDZqgQsGIDKBvs/IXVkZAclUyEPYKP6j+iXbdbHc6Y83q9eDwePB4PJSUlbNq0iSVLluDxeNixY0fngAqAbdu2dRvt11NeXl7ncSK4LJvu4poFE7j72VIO4g1MAAAgAElEQVQ+qj5jdTjDIn1Uwj8SJsGydbDvQWLqjlgaik3ZSI9Jp7mtmdNnTlsaiz/k5+eTmZlJZmYmS5Ys4Y477iAnJ4eysrJuzXsdyWft2v4H6srNv8HtO1fNpl3DpifftTqUYRlWQ6W5SOI9wKfNTSXAg1rrX/s6MBGCLvoGlNzP9LK/AjdbGkpcRBxxEXFUnKnA6XASbg+9JeuzsrKGNRQ/Nzd30OOdTqcM7w9iU1zRfPHi6fxhZylfOH8ai6cmWh3SkAy3RrUJ2A2sAb4DlAF3mgsnLvR1cP5mLsColVJVSqntSin34GeJIYt2wflfJfn0bvhon9XRkB6TjkZzssGaxR6FsMK/r5hBSpyDX/zrYNB86Rhuotqrtb5Ta/2Q+bxGa23DWEBxW9dZ1INUqdZaaa0TtdarZKXgMbDsi7Tao40RgBaLsEeQFJlEdVM1Z1qCq81eiJGKdYTxH5efw+4jVTx7qNzqcIak30SllLpLKfUps7mvQ5/pV2u9DWOV3zwfx9cRS65Sqs/xsEqpLKXUOqVUTsfzWMQgfCTKyfFJ18Dbj8Cpd6yOhuSoZMJsYZxoOBE03y6FGK212VOYnBjFr58+FBT/7weqUeUBDwFV5ppTdwFOpVSfN8KYS8eX+jpAM/Fs7mefG9iotS40VxkuBPJG0WTn7EiKZjOgc/BTxHB9MPk6CA+MWpXdZiclOoWGlgZqm2utDkcIv4gIs/G1y8/hrePVPHUg8Ju+B0pUxWaz3lKgEMjE6KPK70hcSqnblFKLAMxmP5+tQaWUciulCgA3/d+LlQf0HH5UAGwc4WULtdbbtNbFwBZgx2AniOFriYiHpbfA/m1w2uffbYYt0ZGIw+7gZMPJoPh2KYQvfGrxJNwpMfzX9ndpaw/s//cDjfrLA9Bal2CM7rsTwFxmPgcjgW3CqIVo85jVvgrM7B/KM6/Ze8EcQy69E9UeYHvHL0qpdRhJtj/bzcTUUSvsuH6J2azo7Lpd+Mj5X4FXC+DVu+Fjd1oailKK1OhUjtUew9vkJTEyOEZCCTEaYXYb38iZyVcfeJ0n9n/EtQsmWh1Sv/pNVP0tfGgulNi5WKJSKsGKm3rNZrletS2ttdecm8yttfaYzYFDeb0sYLPWulutUJLUGIlLh3m58PpfYeX/gyhrk0NcRBxRYVGUN5ST4EjApuQWQxH6PjZ/Ar/Zfoi7dpVyzfwJKKWsDqlPo/40WjjzhMu8fn+JZLj9VB661M7MvjG5/X4snX87tDTA3vutjqSzVtXS3oK3Sb6biPHBblN86dJMDnxYw/PvWTtp9ECCa2bC7nw60MGsiXnMpkIwmgu/2N/x5nHrANLS0ti1a9eIrltXVzfic4NV1zIvdC4g6vnf8WrzPLTNt/8dExISqK0d+gAJrTURKoJT9acIaw7z6bfLtra2YcUSCgKpzI2NjX75nAXj5zmxXZPoUPziH3toXxY1rHP9Vd5gTlQ+19FXNcRjCzEGmZCdna1XrFgxomvu2rWLkZ4brLqVeeL34P/WcGmKF+b3ngh1NA4ePDj8NaCa4FjtMdod7TgdvvsuFKjrUY2lQCpzZGQkixcvHvPrBOvn+cvhHn72+EHi3QvJGsZsFf4qb9A3xMsQ8iA3YxW4MuG1Pu9A8Lu4iDgcdgcVZypkBKAYNz67bCoJUeHcvcv6Ubh9CeZE1TFrRLflSLokLplVIhjYbJB9Cxx7BU4esDoalFIkRSXR1NpEfUu91eEI4RcxjjA+t3wq2w+e5Fhlg9Xh9BK0icocROGhd1+VC/DK9EdBZNENYHfAnj9bHQkACY4E7DY7lY2ylJoYPz63fBo2pfjLK+9bHUovI05USqkEpdS3fBnMCBQD2T22ZZnbRbCIdsHcT8G+LdBUZ3U02JQNV6SL2uZamtuarQ5HCL+YkBDFVXPTefC1ozQ0t1odTjejqVG5MG769QcXfY/yy6f3TcZ55nYRTLJvgeZaY7aKAJDoMDqUQ61WlZmZiVKKkpKSPvcXFxejlKKwcEi3H4oQc/OFGdQ0tvKP1z+0OpRuArbpTynlNOfbK8JIUhuVUgVKqc6hYWbzX755XK5Saj3G3H/S7BdspiyD1Lmw9z6rIwEg3B5OvCMeb5M3ZFYBLikpweMxPhpbtmyxOBoRiLKnJTJnQjz3vVQWUIOJAjZRaa29Wut8rfVqc+mNTK11njlTe9fjSszjtmmtNw1niLkIIErB4s/Bh6/DqYNWRwMYtaq29jZqmmqsDsUntmzZQlZWFrm5uVJjEn1SSnHzhRkcOlnHK57AaU0I2EQlxqEFa8AWZkyrFABiwmOIsEdQ2RQ4H9jRKCwsZO3ataxatQqv19tv858Y3z6+YCJxkWFs2X3U6lA6SaISgSMmGWZeBW9uhbYWq6NBKUViZCJnWs7Q2NpodTijUlJSgtfrJTc3lzVr1gBQUNBzPmchICrCzicXTeJf+09Q3WD95xAkUYlAs+hGqD8FhwOjBdfpcKKUCvpBFQUFBbjdbtxuN06nk6ysLLZu3Wp1WCJArV06hebWdv6577jVoQAyhZIINOesgpgU2PcAzLp67K7zxHfgxFuDHhYGZLY10apb0WHRKEY2/19UWyvYh/lxS58PV/9yRNfraevWraxbt67z97Vr15Kfn09xcTE5ObIotuhu3qQE5k6M54HXjvH55dMsn1VdalQisNjDYc4n4dDTAXFPFUCYLQw0tLYH1r0lQ1VcXIzX62XVqlWd23JzjcGzRUVFVoUlAtxnlk7h4Ec17D9u/WAiqVGJwDPveti9GQ496fOJajsNo6Zi05qPqktRKDKdA63B2b8zFk7QWlRUhNPp7FZz6mgGLCwslL4q0afrFk3iZ48f5MHdR5k/eb6lsUiNSgSeKcshbiLsf9jqSABjUIXL4aKxtZEzLWesDmfYtm7ditfrxVxQtPPRcU9VcXFg9AeKwJIQFc5V89J57M2PaG619l7C0SaqwFwOUgQ3mw3mfhIOb4dGq9bl7K5j1d9gG6re0exXVFREaWlpt8f27dsBGf0n+vfJRZOoPtPCrndPWRrHiJv+tNZlSimZqkiMjbnXwyt/hHf+BYs+a3U02G124h3x1DTVkB6djt1mtzqkISkoKMDpdHb2SXXldrvJyspi27bAmLZKBJ6LzknGFRPBP9/4kCvmplsWx6hqVFrrMl8FIkQ3k7MhYSocCIzmPzBmqmjX7VQ3B0Ytbyi2bdvWbbRfT3l5eZ3HdbV37162bdvW6+H1esc0XhFYwu02rl0wgeKDJ6lttO6eKumjEoFJKaP5r/QZaAiM5raosCgcYQ6qGqusDmVIOpLP2rVr+z2mv5t/CwsLWb16da/Hnj17xi5gEZA+sWgSTa3tPHXgpGUxyKg/EbjmXQ8v/Q7eeQyyvmB1NMZMFY5ETtSfoLG1kciwSKtDGlBubu6gE4s6nc5ux+Tk5ATUZKTCellTnUxxRfHPN46Tu2SyJTFIjUoErgmLIHF6wIz+A2NQhVIKb5M0gYnxQSnFJxZO4sXDFZyqtWYqMUlUInApZdSqyp6D+gqrowGMm3/jwuNCavkPIQbz8YUTaddY1vw37ESllIofi0CE6NO514Fug0NPWR1JpwRHAm3tbdS31FsdihB+MTMtFndyDE/u/8iS6w8rUSmltgJVSqmMHtsv82FMQpw1YaFx8++hJ6yOpFNsRCx2m53qpuAZ/SfEaCiluGpeOq94Kqmqb/b79Ydbo9oOrNFaH+mxfa9S6lvBXttSSmUppdaZj/VKKbfVMY17SsHMK6F0J7Q2WR0NADZlIz4inprmGtra26wORwi/uHreBNraNdsP+r/5b7iJyomRrLrRWldrrX8FrPFJVNbJ0VoXmo9NgNzQHAhmXQ3NdXDkBasj6ZTgSEBrTV1LYEycK8RYmzcpnknOKJ7cf8Lv1x5WotJa3wkUKaXuUEqtHKOYelFK5Sql+lyLoEstKKfjeRSXylNKOUdxvhgL0y+BsChjktoAER0WTZgtTJr/xLjR0fz3wnsVfr/5d7h9VHdjzO+3CtihlGpTSu1WSt2llLoDWOLrAM3Es7mffW5go1kDKtZaF2Ikm5E22W0Eyjqa/5AaVWAIjwL3Cnj3SQiQe3yUUsQ74qlrqZPmPzFuXD0vnea2dp55x79z/w236a9Ua32F1jpba20DrgR2AEuBHHz4h10p5VZKFQBuoL+pCfKAnjNqFmAknGEzE90d5uvmAa6RvI4YA7OuguqjcOptqyPplBBhNP/VttRaHYoQfpE1NZHUOAdPHfBv899wZ6bodpej1roYKAZQSi3G6KO6xxeBaa09GMmCASa/zaV3otpDl340s2Y00CJC281yoJRab/ZNbTLP2z7IucJfZl5lPB96EtLmWhuLKSosijBbGDVNNTgd0mIsQp/Nprj83FQe2+ffpT+Gm6iKlVK3aa37SkZ+nQDN7EvqVdvSWnvN9XbcWmuPWUsayuvlACVdXqdQKZWplMrSWpcMcKrwh7h0mLjYaP67+JtWRwOYzX8R8VQ1VdGu27EpuX9ehL6Vs1J54LVj7Dnivzk4hzuYogxjMMVtXYeiK6WmAx7GoI9qAC4zpv7mshluP1UlkNVzoySpADLzKvhgN9SVWx1Jp7iIOGP0X7OM/hPjw4Uzkomw2/zaTzXsSWm11tX0aN4z16ZahZGs/MWnbS1a6xKzX6xjTQQnsKW/483j1gGkpaWxa9euEV23rq5uxOcGq5GWObY2mWw0Bx/7PSfTBx90mpCQQG3t2PYfaa2NBRXrK1HN/a8j2tbWNuaxBJpAKnNjY6NfPmfj5fM8M1HxWMkRFmW1+6W8Pps9XWu9w1evZRWt9ZBXkDObFAsBsrOz9YoVK0Z0zV27djHSc4PViMvcfgkc/AXnOk5y7hDOP3jwIHFxccO/zjDV1NZQ11JHbGwsSvWdrGpra/0SSyAJpDJHRkayePHiMb/OePk8Hwkv40ePvk2diuZaP5TXp43qSqm7fPl6Q7ym9GKPFzYbTL/UmKUiQIapg9H819bexpnWM1aHMqjMzEyUUpSU9N2iXVJSgtnH2+2RmJhIXl5evwsndj2vuLh4wBg6jtu0aZPPzhf+ddnsNAD2lfvn1oxRJSqlVLxSapFS6jKl1G34d2aKjmbGbkPIuyQufzZDCn9xr4C6E1D+rtWRdIoNj0WhqG0OjGau/pSUlODxGB+LLVv6bdUGYOPGjZSWllJaWsrevXvZsGEDW7duZfr06f0muQ5FRUX97hvKsvejPV+MvalJ0cxIjeXN8la/XK/fRNU1AZkzUdyllHrKvMH3tFKqDWOk316MYdyF+LjfaCDmIApPH9d0AV5zeLsINZlm35Rnp7VxdGG32YkOjw74+6m2bNlCVlYWubm5FBYOPBjW6XTidrtxu91kZWWxfv16ysrKcLvdXH755f2el5OTw9atW/vdX1BQQE5O/5PHjPZ84T+Xz07lncp26prGPlkNVKPahHGPVAHGjbx5GPcUVWHc5Ps68CWMWtQVwAxzmz8VA9k9tmWZ20Uock4FVyZ4dlkdSTexEbE0tTbR0ubfqWWGo7CwkLVr17Jq1Sq8Xu+gNaOenE4nmzdvxuv19tvstnr1arxeb5/Ndx3bV69e3e81Rnu+8J8Vs1Jp07D3/bG/M6nfRKW1vgIjCW0GErXWNq31DHNmijXAHq31Zq31Q1rrHWYNZqwShIu+a2v5QM//tXnI1Eehzb3CmKA2gJJCbHgsQMDWqkpKSvB6veTm5rJmjdFCX1DQ8175wWVlZZGTk0N+ft8fMZfLRVZWVp+v3VFTGqhGNNrzhf9kZyRy5yVRXDozZcyvNWAfldZ6m9b6S8BqpdSinrv7OP47vgpMKeVUSm1UShVhJKmNSqkCpVRul+t5gXzzuFyl1HqMuf+k2S+UZa40ZlP/YLfVkXRy2B2E28ID9n6qgoKCzqY8p9NJVlbWgE1sA1m1ahVAZ39XT2vXru2zL6moqIicnBxcroFnJhvt+cI/wu02UqL9c5P7kIana63vUUpNV0pdprV+ZqyDMq/pZQg1I/OGXLkpdzzJuBiUzWj+m3bBiF5i42sbeafyHZ+G1dzWTGt7K9Hh0b32tbW1Ybfbh/V6s12zyV/mm8aBrVu3sm7dus7f165dS35+PsXFxcOuobjdxr30Ho+n8+eucnNzyc/PZ+fOnVx33XXA2Wa7odTiOs7vGttwzhehZ8jpUGtdprV+xhxkMX0sgxJiQFFOmJhlDFMPIHZlR6Np04E1m3pxcTFer7ezJgRGMoCBR9j1x+k0WuH7G6reUXO79957O7d11N46mh0H0nF+16Q0nPNF6BnJzBRvACilSpRS8VrrGt+HJcQg3Cvghd9AYw1EDn9haV/VVLpqbW/l3cp3SYlOITU6tds+K29+LSoqwul0dqs5dSSDwsLCYddSOgZhZGX1mnGsU25ubrcBFx3Ndk6ns98E58vzRWgZcQOj1nozkGQOYY8HUEpd77PIhBjI9ItBt8GxV62OpFOYLYyosKiAW/V369ateL3eXjfxdvQxDXaDbU+7dxt9g301+3XIy8vrfO2OZruObUMx2vNFaBnVFErmJLVlSqlPm3P9rQYe9klkQgxk8lKwhcH7L8I5qwY/3k9iwmOoOFNBW3sbdtvw+qTGQscf+qKiol41II/Hw6pVq4Z1b5LH42Hbtm2sX79+wOPcbjcZGRkUFRV11oA6mhuHoqPGN9LzRWjxyVx/WuuHlFJlwBd98XpCDCoixlj24/2XrI6km9iIWCrOVFDfUk+8Y/hNkr5WUFCA0+ns8498x828Q53twev1snr1apxOJxs2bBj0+FtuuYX77rsPGFmSycvL62yWlCQ1vvlsbKE5+i7oJ6YVQWTahXC8BJobrI6kU1RYFEop6lvqrQ4FMKYc6jrar6eO5rSeycrr9eLxePB4PJSUlLBp0yaWLFmCx+Nhx44dnQMqBvKJT3wCj8fTeaPxcOXm5o7qfBE6fD0IXhqRhf9MuxDaW+D4Hqsj6WRTNmLCYwIiUXUkn4H+yPd3829+fj6ZmZlkZmayZMkS7rjjDnJycigrKxtwEEVX06dP7+zHGkmNqKP5b6Tni9Dhs2U+oLPPSgj/mHoeoODIizD9Equj6RQTHsPJ5pO0tLUQbg+3LI7c3Fz0ILPMO53ObsdkZWUNek5f+juvtLR00Gv64nwR2mTtbBG8IhMgfb4xoCKAxITFANDQGjhNkkIEsz5rVEqpuzEmex3t1xYFlGqtpYFZjI1pF8Lee6G1GcIirI4GgMiwSGzKRn1LPQmOBKvDESLo9ZmozPn9hAh80y6AV++CD183mwKtp5QKmH4qIUKBNP2J4NYx19/7L1gbRw8x4TE0tzUH9LIfQgQLSVQiuMUkQ9I5cDRwZqgAOiemlVqVEKPXXx/VLwFfTTzr0VoPfnegECM1ZRm8+wRoDUpZHQ0AkfZI7MpOfWs9Tv8tfC1ESOpvePodGIsV+kKlj15HiL5NXgpv/A0qPZCUaXU0gNFPFR0eTUOLjPwTYrT6G0xRDVT7ORYhRmbKMuP5g929EpXWGmVRLSs6PJra5lrpp7KY3HMV/MZtH5VSavtI9okAlDIbIuLg2GvdNtvtdlparEsSMeHG/VT1rdJPZaWWlpZhL1opAotPZ6YIBkqpHMAN9JoueqB9IoDZ7DApCz7onqji4uKoqakhOTnZkrAi7cb9VA0tDcRhzVpUAmpqaixbC0z4RsDUqJRSuWai6GtfllJqnVIqp+N5pNfRWhdrrQuHu08EuCnL4OQBaD5be3G5XFRVVVFRUUFzc7Pfm4A6+6lkhgq/01rT3NxMRUUFVVVVuFy+6nIXVgiIGpWZeDZjrGfVc58b2Ki1XtVlW5FSyqO19vgxTBHIJi8D3W7Mpj79YgAcDgdTp06lsrKSI0eO0Nbm/yXia5trqW2upSashqjIKL9f30qNjY1ERkZadn273U5cXBxTp07F4XBYFocYPUsTlZmE8oG99D86MA/ouVZ2AbCRPhKbGKcmZxvPH7zWmajASFYTJkxgwoQJloS158Qevv7U11mXso6vfuyrlsRglV27drF48WKrwxAhwNJEZdaI8gCUUvn9HJZL70S1B+gc8KCUWgcMNC55u9Z6eOtti+AS7TJu/P0gcJb8AJiXPI8wWxieJqn8CzFSAdH01x+llBNjcEO32pbW2quUQinl1lp7pF9JAMb9VIe3B9aNv2GRXDP9GsIqA/qjJkRAC5jBFP1wgZGY+tnv9mMsItBNyoL6cqg5bnUk3fzsop+xIn6F1WEIEbQC/Wuez+eeUUplYQ4/V0ptpEuz4ED7+niddcA6gLS0NHbt2jWieOrq6kZ8brAaqzLH1WiWAPuf/gsVKef7/PVHQ97n8WG8ldlv5dVaB8QDKAVyemzLMkLs83jd83irHkuWLNEjtXPnzhGfG6zGrMzNZ7T+sUvr4h+PzeuPgrzP48N4K/Noywvs0UP4GxvoTX9AZ1+VEAMLj4TUOcYQdSFEyAj0RNUxVKrb3XpdEpcMpRLdTVxsLKIo87sJETICOlFpYxCFh959VS7Aq+WGX9HTxMXQ6IWqI1ZHIoTwkYBOVKZiILvHtixzuxDdTTRvMP3wdWvjEEL4TCAlKhd9j/LLp/cMFHnmdiG6S50D9gj4UPqphAgVVk+h5AQ2YNwP5QQ2KqVWYQwL3wadN/fmm8PFd5vHbpRmP9GnsAhInw8fvmF1JEIIH7F6CiUvQ6gZaa1LAPmKLIZm4mLYtwXa28EWSI0GQoiRkE+xCD3pC6C5FqrKrI5ECOEDkqhE6Emfbzyf3G9tHEIIn5BEJUJP6rmgbHBCEpUQoUASlQg94VHGkh8n3rI6EiGED0iiEqEpfb40/QkRIiRRidCUPg+qj8GZKqsjEUKMkiQqEZo6BlRIP5UQQU8SlQhNaTLyT4hQIYlKhKa4NIhJkQEVQoQASVQidKXPl0QlRAiQRCVCV9o8KH8H2lqsjkQIMQqSqEToSp8Pbc1QccjqSIQQoyCJSoQuGfknREiQRCVCV9I5YHfASemnEiKYSaISocseZsz7JwMqhAhq4zZRKaW2D7K/wF+xiDGUPk+a/oQIcuMuUSmlcpRS64CcgY4B1vkvKjFmUudCQwXUlVsdiRBihCxd4bcrpVQu4NVaF/exLwvIBjwYS9F7+jpuKDrO66/GpJRyApWAdySvLwJM6rnGc/lBiE2xNhYhxIgERI3KrMFs7mefG9iotS7UWhdrrQuBPHP7WMjRWsuy96EidY7xfPJta+MQQoyYpYlKKeU2azZujFpMX/KAnrWfAmDjGMSTA4yopiYCVGwqRLnglCQqIYKVpU1/WmsPRiJCKZXfz2G59E5Ue4DOwRBmn1PmAJfaPlhTYUeTn9ZamvxCiVJGrerUQasjEUKMUMD0UfXFTB69altaa69SCqWUW2vtMZsDRysHcCmlss3fnWYCLDYTqghWqefCvgdBayNxCSGCSkAnKsAFRmLqZ78bY4DFqGmtt3X9XSlV4KMEKKyWei4010L1B+CcYnU0QohhCojBFANw+voFlVJZSqn15s8bzX6prvudXfavH8NBG8JfOgZUSPOfEEFJaa2tjgEApVQpkNe1L8kclr5Xa92rvUYppYFVIx2mPlpms+A6gLS0tCUPPvjgiF6nrq6O2NhYX4YW8Pxd5rCWOi568UZK3TdxbOr1frtuV/I+jw/jrcyjLe/KlSv3aq2zBzsu0Jv+AKOWE2iDHMxmwUKA7OxsvWLFihG9zq5duxjpucHKkjK/OZHMmDNkWvRvLe/z+DDeyuyv8gZ6019H/5Or60ZzkEXX/UIMbFIWHN9rdRRCiBEI6ERl1qI89O6rcmHMYiGJSgzN5GyoLIX601ZHIoQYpoBOVKZijOmTuspCbswVwzF5qfF8fI+1cVilsdoYou+T16qBisPGcH8h/CCQ+qhc9D3KLx8owuwPMuWZDyGGZuJisIXBkRcg83JoaYDI+KGfrzV8sBtqPoSS++HGh8DW43teUy2gwB4Oym4sM9LeBi//gbAWc/BoxWGISoSYpO7ntjZDVRmkzBo8llMHITr57NyFpTth+iVgs5+N9d0njNeKTYU7Jp89N2EylD0H0y+Fv38JLvo6PP6fsORmo2zvPW0c963DEJlg/Pz6/0J4NExYBHed3zuejIthzf+CIx6qj0HLGTjyPOiZg5dFiCGwNFGZfU0bMO6HcgIblVKrMGaS2AadN/fmK6U2ArvNYzdKs58YlogY44/5vgfh8A7jvqqv7IEwR+9jH7wREqbA1b88e5Pw/ofgoVvPHnPsVZh2PrS3Q91J4w/8jh8bianRC1PPN5LV+y8AkDHpGmi7Cn6/xDj/B5WAguIfwPESeP9FY/ukJcb26mMwIwfO/wokTALdDhXvGYl2x4+NY2NSoeE06DbjuHNWGWtvfbQP3irq+9/hvmuM52fNGcge/0/jee993Y/71Yyh/9seeR42Te+1eVb65bBy5dBfR4h+WD2FkhejxjTYcSWATBQrRufc66D0Gag/Zfz+xt8g+xbj56ojUHQzXPNreOcxY1tjtZGEbiyCA3/v/lr3XgUrNsCuO/q+1tGXu/06+fjj8NMutag7M+FMVe/zug74eONvxqM/HeUAePn3xiOAJFe8YnUIIkQEUtOfEGNr3vVw9BVY9Fl4cgNs/yGUPQ/2CHjT7L95+gdnj9/3f8bzT8xBp9MuPFvzgf6T1FD0laRCTHhrvdUhiBARDIMphPCNyAS4vgDcKyDzMmiqgQMPG816U5Ybx7z/AtjCIX4yxE2EK7sko3mfhpseg/QFfb/+nE/23nb7q74uhRDjjiQqMT5Nu9B4nn0tfO8U3PoUzLzK2DbzSrjlCeNx/u2QdZORtGZfA9MvhnXPwsrvdX+9udfD6vsg2+zH+vSf4NrfQups+OIzvDXv/0H+Efhel5WGJy+DS9YbP3/5tcdEb3kAAAsbSURBVO6v1TPp2cJg7V/P/h7TzyKQHaMbO+T+ebB/CSECnjT9ifFpRo7Rx7T0trOj97JvNUa3XfY9cE49e+y1vzUGVHTMvG6zwQVfhVlXQdq87jOyX73JeN2uqwlPWsLp5FpjtB9AyrnGisP/9oQxUm/up4wRep/5P3jwBlj8OWMwxooNxva6UxCXZpw7+1rwPAvfPgw/Sjh7jQv+A176HXz+H8bgil9OhevvMWqBqXPhuTuNNbk61uW68GvQ1gLzc40BJos/BxMWGvt2/+nsIItRODDnW8wd9asIIYlKjFdhEbDiO923zbzCePTUcxg6QHgkpM/vvd0eNviS9//2L2OIud38+KWZk+bOvgZ+VH32uNTZxnNHkgL4TB+DK6ZfCpd9H6746dltPV8n90/Gz49/C17/C6z6ydn9k5Z0f72lt0L8JHhg7dltX9kDFYeMRPrpPxkDU+67BqKccP1m2DitV1jlKRf28w8gxPBIohLC36JdkOHDP+I3PTL0Y6/5lfEYzKyr4ObHobUJZlxubEs+p3sCvG372Z+/stfoA6w6An/KMZKnkp4F4RuSqIQIVl/fb9xHNVYyLhr6scnmfVfRSUa/W/YtUPLu2MQlxh1JVEIEK+eUwFsI0maDy75r/iKJSviG1M2FEEIENElUQgghApokKiGEEAFNEpUQQoiAJolKCCFEQJNEJYQQIqBJohJCCBHQJFEJIYQIaEprbXUMQU8pVQ68DyQAXeaY6fZ7fz8nAxU+CKPntUdzbH/7BypfX9vGQ5mH+p4HS5mHsi1YyzzU97ivbVLmvss82vJO01oPMjkmoLWWh48eQGF/vw/w856xuPZoju1v/0DlG69lHsZ7HhRlHsq2YC3zUN9jKfPQy+yr8g72kKY/33p0gN/7+3msrj2aY/vbP1D5+to2Hso81PfcV8a6zEPZFqxlHup73Nc2KfPYl7lf0vRnMaXUHq11ttVx+JOUeXyQMoc+f5VXalTWK7Q6AAtImccHKXPo80t5pUYlhBAioEmNKkAppdxKqfVKqRzz2Wl1TP6glNo++FGhQSmVpZRaZ76/RUopt9UxjTWzzDlKqVylVMF4KHNXSqkCq2PwB6XURqWUVkpVKaW2j/Z9lvWoAleR1noJGO3AwGZgtbUhjR2lVA7gBnKsjsUfzC8e2VrrQvP3HGA7kGlpYGNvBzBda+1VSrmAImCJxTH5hfkerwPyrI7FD0q11spXLyaJagwppXIBr9a6uI99WUA24MH4A+3pOM7cV9lxrPmhDvg/4CMtL0CXsgfVN85RlNkN5HO2jX8P4FZKObXW3rGPfORG8z5jJinz58qe5weqUZa544tJJRDQ721Xoy2zT/ljDPx4fGDUDKqAnD72uYHtPbYVAW7z51yMGlXX/aVAltXlGovy9tiurS6Lv8rc4+ccoMrqMvnrfTb3be/rdQLt4YsyA7nmc8C/x74oM7De/DuWA2wEnKOJR/qofMzsWyrAeDP7+8aYB/SsORRgvKEArjEKz+d8VN6g4qsya609PY7/oi/j9CVfvs8d/a8Yf+zG7lv4KPmqzGZrSMCWsysfvs+FWutt5vu7BaPJd+Ssztyh/MCoBfX1jaSU3t+4nJi1CYxvIj2/sVQRwDWq0ZS3x/Ze2wL54aMyr8P8xh0MD1+UuUu5t/sytkArs/lzVpd9QVGj8uX7bO7XjKJWJTUqPzPbqnt9W9Fmu705OsZDH7UqrXWJP2L0pSGWN6QMp8zmt22P1nqbX4P0saGUuUtNqsNWICdY/w8M8X3OAbLN0Z3rAKf5c8iW2RzZubfnuXoUfa8ymML/XDDgm+bWWhcrdXbAjPmfOiiaDvowaHkxEnMoGVKZOwbNdHwBUUrlBnHCGkqZAZJ6bPPq7k2gwWQon+Vu76dSqkCbIz2D1FDe5z10aRo0v4yN6v+1JCr/G+r9UF80v32WAFkEcP/FIIZUXvOPdo7580YCvP9iEIOW2fzysQPjG3bHZg+j/EBbaNAym1/AnGbNAmAVcPnYhjWmhnxvo1kTWWf+vB7YFqQJeijvs1cp5enyPmcyyr9fkqgClPktu6OpL1j/YA9Zl/JusjoWfzD/SCVaHYe/9ahhBHPNYljMGsgm/n97d3ucRhLEYfzfVRcApRDWGSA7AykDVIpAJgOpLoIrlIHsCHwoA3ER3EEGVgiSMpj7MD3yMCxeFvGywPP7Ii0sM9PYRWt6h53T+f+90c8srlHtyancaSI5tXglYj4VxLx9JKrdS9P9ucUS2T/8IZYDfufU4pWI+R0xH529xEyi2jEvATxrsdZ7psO+sFzr1OKViLl4ipiPyL5iJlHtx0Tx9iO5vo73WtSpxSsRc0LMx2fnMZOotutM9atk7rR4g9mhP37ITi1eiZhzxBwR84axH9WGea32T8XvEwwUp8kTxeXWj9l5fUnXkv71c2eHuBz71OKViFnETMw7jplEBQDoNEp/AIBOI1EBADqNRAUA6DQSFQCg00hUAIBOI1EBADqNRAUA6DQSFQCg00hUwJrM7KE4Hvhupqu+/tbMnjY/ss05hDHi+JGogPVVxfGl2m1z8EW+q3GHHcIYceRIVMAa/J5oZVKq2mxzEEK4CiFY/piZjXyb+p1a1m/dGIFdI1EB67mQNE0HnrjePtKgt3GruGXCzuyrX2BVJCpgPZea33/ns+KdpAFsGIkKWE9Z5isTVyMvtwX/fSzp1Z8am1kws9fi/IGZTf25ab5ww9t69d8fzOynmfXMrDKzsZm9+uue8hJfU7/5GLPH+t5OGsfXmrhea87rF+cNfJyhbmxAQqICWlpS5uuHEGYfaPZGMdlJcRO6T5LOsz4HksaKyfBK8frYwge7mU0VZ3cPvm34sGj/TFnJsqnfko9j6v1fSvoh6aFcAam44d5Y0oO3W/lxaicdP3p/Kaa6jfpw4v7Y9wCArvMP4TwhnEnqFcu2PxfHozYbyYUQ3swszdBeahZlfJd0H0JIu6g+elLKd1bt+WtT4lF2forlRtLUzAYhhMcV+i19l/QthJAS4MTMZopJc1zEfJc22zOzT5JuzaznCTTNrv7y45li0gIWkKiABtmHsqT3xHXnH7DyElwVQvi2jf69ZNZT/KC/LZ5+KY6H+o0QwszMpMWl9auM48LHMTd7CiFMzOxNcVaUJ6r8959Fc+m5f8zsh6TJB2ekOGIkKqC9KiUpdyVptM3+/Oe5FkuOc4mqbkbk5bprb+cj14DSa+tmXc9l28V7pPI5M7tUfN9GPs5JPhsEEhIV0KAo/dWW/SRVPlNJWpX+Grwnhjbf05IkH2elmAz+84eny1+x0jgqxVJdrsraX4m/P+d+ze+rpJGZ3YYQ7tccH44UiQpokJf+vPQ2S0nIFwXcleXBNZVlvNT/zEtrI/1a+JDG01s2c/GxXUi6yq4V1S1WqO23RkpEQ2UlRp+xpcUTrfn4783sWvFOGMAcEhXQzpfiL/4LSRu5F56XwyRpmH5mpbAbxeXjaSVdTzFZPGvJdakQwnNKcN7em34tvFi13/K8oeIqvzPFFX9ptvbYZgbpS9rvPJZUNuyruP4FSCxPB9oqZyTnavn9qQb3islvpKy85jOic+//SXH13bNqEk/hRrFcOfY2l816avst+YKRS/1abj5UnFFeNYyj9LfiKr9h1s79thak4LBZCKH5LAC1ZT4ze2IBALBdzKiA1c2V+TZxfz8AzUhUwOrK2yRt7PoUgOVIVMDqXooVdpU2e30KQA2uUQEAOo0ZFQCg00hUAIBOI1EBADqNRAUA6DQSFQCg00hUAIBO+x9xvAO8JBFJKAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.semilogy([np.linalg.norm(A @ x - b) for x in conv_x_da], label=\"DA\")\n",
    "plt.semilogy([np.linalg.norm(A @ x - b) for x in conv_x_al], label=\"AL\")\n",
    "plt.semilogy([np.linalg.norm(A @ x - b) for x in conv_x_admm], label=\"ADMM\")\n",
    "plt.legend(fontsize=20)\n",
    "plt.xscale(\"log\")\n",
    "plt.xlabel(\"\\# iterations\", fontsize=20)\n",
    "plt.ylabel(\"$\\|Ax - b\\|_2$\", fontsize=20)\n",
    "plt.grid(True)\n",
    "plt.yticks(fontsize=20)\n",
    "plt.xticks(fontsize=20)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaoAAAETCAYAAABnSkJLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3Xl8VOXZ//HPlYSwBQgRCCgoDCIighKCggokNXHXagXXWrVKgku1rtQ+fX61T/s8Nqi4VhtwadUqCmpd6kaAgBtCANeqLAF33CAqKlLg/v0xZ3CYJGSb5JyZ+b5fr7zInDlzznUzw1zcuznnEBERCao0vwMQERHZGSUqEREJNCUqEREJNCUqEREJNCUqEREJNCUqEREJNCUqEREJNCUqEREJNCUqEREJtAy/AwgiM5vjnCtu7Pk9evRw/fv3b9a9vv32Wzp37tys1yYqlTk1qMzJr6XlXbp06RfOuZ4NnadEFcXMioAQUNSU1/Xv35+qqqpm3bOyspKCgoJmvTZRqcypQWVOfi0tr5m915jzki5RmdkEoMY5V1HHc3lAPlBNOCFVR58X+d3MytsoXBERaUBSJSqvRjQDmFjHcyGgLLpJz8xmmVm1c666DcMUEZEmSIrBFGYW8mpBIWB9PaeVArE1pXKgrDVjExGRlkmKROWcq3bOlTrnpu/ktAnAsphjVd5xEREJqKRIVA0xs2zqqG0552q850N+xCUiIg1LiUQF5MCPiakOSlQiIgGVVIMpdiK7MSd5owKLvN/LgDl1jR70ni8BSgByc3OprKxsVmAbN25s9msTlcqcGlTm5NdW5U2VRNUozrllhPuxpjbi3OnAdID8/HzXnLkEc9/+lOfXvE7ZMU1/bSJLtbkmoDKnilQrc1uVN1Wa/oDtfVWBUfnu5zy6ajPfbd7idygiIoGVKokqMk8qJ/pgVOLyZR7VkcN6s3krLHj3cz9uLyKSEFIiUXmDKKqp3VeVQ3gVC18S1QH9c+iSCU+9uc6P24uIJISUSFSeCsLLJ0XL8477IiM9jZG9Mpj39qds+s9Wv8IQEQm0ZExUOdQ9ym8KtZdWKvWO+ya/dwbfbt7KwhVq/hMRqUtSjPrz+pquIjwfKhsoM7NiwsPLZ0O4+c/MpnjDzpd455b5vc7f3jlpZHdqx9NvruOwob39DEVEJJCSIlF5fVAN1oyihp8HRkaaUTwkl2feXMcPW7bSPiPd75BERAIlGZv+Es5Rw/rwzQ9beHHVF36HIiISOEpUAXDQnrvQpUMGT72h0X8iIrGUqAKgfUY6xfvk8uyb6/hm03/8DkdEJFCUqALirIP6880PW7h3UaN2ZhYRSRlKVAExvG824/bqyZ3Pr+H7zZpTJSISoUQVIBcW7smX325m5pL3/Q5FRCQwlKgC5IABORzQP4fpC6vZvGWb3+GIiASCElXAXPCTPfnkq008suxDv0MREQkEJaqAGTeoB8N268Yt81bx2deb/A5HRMR3SlQBY2b84adDqfluMxPLX+bDDd/5HZKIiK+UqAIob/fu3HvugWz4djMn/fVl1nzxrd8hiYj4RokqoPJ2784DJaPZtGUbJ97+Eo+9+hHOOb/DEhFpc0pUATZ0127MmjyGfjmduHjmq5xx52LWqnYlIilGiSrgBvbM4pHzDuKPPx3Kax/UcORNzzP37U/9DktEpM0oUSWA9DTjjDH9qbhsPINys5h0TxUzF2tSsIikBiWqOpjZHL9jqEtu1w48MGk0Ywf15DePvMGNFSvUbyUiSU+JKoqZFZlZCVDkdyz16dw+gzvOzGfCyL7cWLGSaXNW+B2SiEirSogdfs1sAlDjnKuo47k8IB+oJry9fHVd5zVG5HVmVt6CcFtdu/Q0pp44nIw045Z5q2ifkcaFPxnkd1giIq0i8InKzIqAGcDEOp4LAWXOueKoY7PMrNo5V92GYba5tDTjf08Yxg9btnHdcyton5HOpHEhv8MSEYm7wDb9mVnIq9mEgPX1nFYKxNZ+yoGy1owtKNLTjGsnDOfoYX3436fe5h+vaC8rEUk+ga1ReTWiUgAzm1LPaROonaiqgO2DIbw+p4E7udWc5jYVBkFGeho3nrI/3/9nK7/755t06dCO4/bb1e+wRETiJrCJqiFmlk0dtS3nXI2ZYWYh51y1c266PxG2nXbpafzltDzOvHsxlz74Kl3aZ1C4dy+/wxIRiYvANv01Qg6EE1M9z6dUh03HzHTuPDOfvft0YfJ9SzUpWESShiXCPBwzWw2URjfReaP9ljrnrI7zHVDc1CY975pFhPu4prKTZkGvSbEEIDc3d+TMmTObcqvtNm7cSFZWVrNeW5dvNjumVW1i7dfbOHNoJgX92sXt2vES7zInApU5NaRamVta3sLCwqXOufwGT3TOBf4HWA0UxRzLC4df5/ku9vzW/Bk5cqRrrvnz5zf7tfXZuOk/7sy7XnF7THnSXf/cu27r1m1xv0dLtEaZg05lTg2pVuaWlheoco34jk3kpj9ge1+VROncPoMZv8hn4si+3Dx3Jcff9iJVa+sbOCkiEmyJnKgi86Ryog9GJa6knkfVkHbpaUydMJwbTt6Pz77+gQl/fZkL7l/G25987XdoIiJNkrCj/lx4dF81EFujyiG8ikVKJyoI7xZ8woi+HD60N+ULqilfuJp/vf4Jo0M5nHXQAIr3ySU9rVYXn4hIoCRyjQqggvDySdHyvOPi6ZSZwSXFe7HoqkO56si9+WD990y+bymH3bCAR5d/yJat2/wOUUSkXomSqHKoXXMCmELtpZVKveMSI7tTJqXjB7LgigJuPW0E7dLTuOTB1yiatoBn31rnd3giInUKbNOf19d0FeH5UNlAmZkVEx4yPhu2N/9NMbMyYIl3bpma/XYuIz2NY4bvylH79mHO258y7bkVlN67lKIhufzhp0PZLbuj3yGKiGwX2ETlwhN5G6wZOeeWActaP6Lkk5ZmHD60Nz/Zuxd3vbCGGytWUjxtAb8/dh9Oyu+HmfqvRMR/idL0J62oXXoapeMH8twl4xixezZTHn6Dy2e9zvebt/odmoiIEpX8qF9OJ+755YFcdOggHln+Icf/5UXWfvGt32GJSIpTopIdpKcZlxbvxd/PPoDPvtnEibe/xOsf1recoohI61OikjqN26snD593EB0z0zll+iIWrPjc75BEJEUpUUm9Qj2zeOS8g+i/S2fO+dsSHn/tY79DEpEUpEQlO9WrawceLB1N3h7d+fXM5Tz5upKViLQtJSppUJcO7bj7rFHk75HDxTNf5ak3PvE7JBFJIUpU0iid22dw99mjGNEvm189sJxn3tRKFiLSNpSopNE6t8/gb788gOF9u3HRA8t5YeUXfockIilAiUqaJKt9Bn876wBCPTtTcm8Vy97f4HdIIpLklKikybp1asc95xxAzy7tOeuuxdrjSkRalRKVNEuvLh2475wD6ZSZwc/veIWVn37jd0gikqSUqKTZ+uV04v5JB5KWZpw64xVWfbbR75BEJAkpUUmLhHpm8cCk0QCcNmMRqz9XshKR+FKikhbbs1cWD0w6kK3bHCf85UUWarklEYkjJaooZpZnZkVmNsHMys0s5HdMiWJQbhcePf9gds3uyFl3L2b6wtU45/wOS0SSgBLVjuYCVd4OwkuBWT7Hk1B236UTD593EEfs25v/e+odJt1TxXtfapsQEWmZhEhUXg2nqJ7n8sysxKsJldR3XiMN8HYWBljfguukrM7tM/jLaXn87ughvLT6S4qnLeSap9/mm03/8Ts0EUlQLd6K3sy6OudabSKNl3hmABPreC4ElDnniqOOzTKzaudcdVPvFZWkAEqBKc0IOeWZGeeODXHcfrsy9dl3KV9QzczFH3DmmD046+ABfocnIgmm2TUqMxtgZquAGjP70sxOiHruRDNb2ZLAzCxkZuVAiPprN6VAecyxcqCshfe9EpjjnKto7nUkvPL6dRP344kLD2F0KIeb563i4D/P4/63f+DTrzf5HZ6IJIiW1Kj+TDhRVAGjgBIzM+fcI865h82sRf07Xo2oFMDM6qvZTKB2oqoC5kQemFkJMHAnt9ohIXn3neo1I86Jrq1J8wzr243yM/JZ+ek33F65mn+++hGVU+dzyqh+nFcwkD7dOvodoogEWEsSVZVzbq73ewVQYWaTzOwnzrl5QKsO+TKzbOqobTnnaswMMws556qdc9Mbeb0QMME5N9U79BBQHrlOXINPUYNyuzDt5P0Z3WU9y77vwf2vvM/MJR/wy4MHcH7hQLp2aOd3iCISQC0ZTFETe8A5NwMwMxvRgus2Vo53z1pxeJo6tDwE7BLzuEZJKv56dUrjzycOp/KKAo4Z1ofyhasZP3U+f3txDVu2bvM7PBEJGGvuXBcvGeUDVwLFzrm1Mc/Ndc7lxCVIs9VAaXQTnZnlAUudc1bH+c6LqUl9TGY2AS8BAsXANc65ZfWcWwKUAOTm5o6cOXNmU2613caNG8nKymrWaxNVbJnf+3orD727mbe+3MbuXdL4xdBM9sxO9zHC+NP7nBpSrcwtLW9hYeFS51x+gyc655r9A3QDRtTz3ICWXDvmWquBophjeeHw6zzfxZ7fmj8jR450zTV//vxmvzZR1VXmbdu2uade/9gd+L8Vbo8pT7rfPPyaq/luc9sH10r0PqeGVCtzS8tLuAupwe/YFs2jcs595ZxbXs9za1py7cby+qokwZkZRw7rQ8Vl45k0dgAPLvmAw25YwJx/f+p3aCLis4SY8FuPSN/RDs2LUYlLfUsJKKt9Bv919D7884KD6d4pk0n3VPGrB5az/tvNfocmIj6pc9Sfmf2VcP9TS0fuGbDaOXdyC69TiwuP7qsGYmtUOWgQRMIb3jebxy88hNsrV3Pr/JW8vPoL/nT8vhyxbx+/QxORNlZnonLOTW7rQJqpgnBCjR7wkOcdlwSXmZHGxUWDOGxoLpfPeo3J9y3jmOF9+MNxQ9klq73f4YlIG0mUpr8catecILzEUezSSlr6KMkM6dOVf15wMJcW78Wzb62j+IaFPP7ax1qdXSRFBDZRmVm2mZV5K1xkA2Xe1hsTIue48ByqKd55E7ylj8rU7Jd82qWncdGhg3jyV2Ppl9OJix5YzqR7qvio5nu/QxORVlZfH9WfgXitHlrtnLuqqS+KJKFGnLeMHZv+JIkN7t2FR847iLteWMO0OSsonraAS4r24qyD+9MuPbD/7xKRFqhvCaVriBlN1wLaLkPiKj3NmDQuxBH79ubqx9/if596m4eXfcjVxw1ldGiXhi8gIgmlvsEUXwFftXEsIk3SL6cTd5yZz3P//pT/eeLfnDJ9EUcP68NVR+1N3+6d/A5PROKkxftRifjJzDh8aG/G79WT8gXV3L5gFRVvf8o5hwzgvIKBdNFCtyIJT436khQ6tEvn4qJBzLusgKOG9eG2ytUUXFvJvS+v5T9a6FYkoSlRSVLZNbsjN5y8P09ceAh79srivx97i8NvWMgzb36i4ewiCSouicrMHjKzn0U9PtTMusbj2iLNMaxvN2aWjObOM/NJTzMm37eMn93+ElVrNbZHJNHEq0ZV7px7JPLAhTdUbHjpdpFWZGYcOiSXpy8eS9mJw/i45nsm/PVlSu+tovrzjX6HJyKNFK9EVW1m/QHM7DkzW0J4PycR32Wkp3HyqN2Zf3kBlxbvxQsrv+CwGxZy9eNvsUGL3YoEXrMTlZmtNLPbva3n1wDZXvNfOeG9oJo8yVekNXXKzOCiQwdReUUhJ43qxz0vr2X8tfO54/lqNm/RgAuRoGpJjephwou/nmRmK4GpQBnh1dI1B0sCq2eX9vzfCcN4+uJx7Ncvmz/9622OuGkh89/9zO/QRKQOzU5UzrnfOOceds5Nds4NIrwY7FTgt2a2ysxuj1uUIq1gcO8u3PPLA7jrrHxwcPbdSzj77sWs+eJbv0MTkShxm/DrNf/N8H4ws3itFSjSasyMn+ydyyF79uTvL63lprkrOfyGhZwzdgAXFu5J5/aaEy/itybVqMzsCjP7SR3Hax1rq63oReIhMyONSeNCzLt8PMfs14fbK1dz6PULeELbiYj4rkmJyjl3LfAbMzs35qkNZvas5k5JouvVpQPTTtqfh88bwy5ZmfzqgeWcfscrrPrsG79DE0lZTa1RdSW8pcYGM7s8ctw5txxYjtfsJ5LoRu6Rw+MXHsL//HQob370FUfc+DzXPPU23/6wxe/QRFJOUwdTzAW+cM49DKyJqVktAYriFpkPvA0YnZltMLM5ZhbyOybxT3qa8Ysx/Zl3eQHHj9iN8oXVFE1bwFNvaDkmkbbU1EQ1EKgG8JKVRSWrEDArjrH5YbVzzpxz3Z1zxdopWAB6ZLXnuon78fB5Y8julMn5/1jGGXcuZtVnWt1CpC00NVH9Bjg58sA5NwPo7iUr55ybHM/gIrxt5uusrZlZnpmVmFlR5M/WiEFk5B45PHHhwVx97D689mENR960kD8//Y6aA0VaWVMHU0wH/hw9aMIbYGFEJbB48hJPnX1fXtNcmXNuunOuwouvtAVNdtmRpOg1A2Y3N25JThnpaZx18ADmXVbAT/ffjb8uCI8OfFyjA0VaTZMn/Drnljvnvo45NoPwaMBr4hWYmYXMrJxwk2J9S16XEl6yKVo54RUymmO6c262c64CeJBwn5xILT27RJoDD6JHl0wuemA5p0xfxNuffN3wi0WkSeI54Xcucfxi9/qHSgHMbEo9p02gdqKqAuZEHphZCeG+tfrM8RITzrmaqPsv85oVs6OPi0QbuUd3HrvgEGYueZ/rnn2Xo29+ntMO3J3LigfTvXOm3+GJJIWEnXbvNcvVqm0552rMDDMLOeeqvebAxlwvD5jhnBsZe724BS1JKT3NOP3APTh6WB9umLOC+155nyde+4RLigZx+ug9aJeu/UlFWiKR/wXlwE4TSVP7qaqJqp15fWOzmxeapKLsTpn84af78tRFY9l3t65c/cS/OfKm56nUYrciLWKJ0AFsZquB0kgTnXcsD1jqnLM6zndAcfT5jbxPET8muIHANfUlQq9JsQQgNzd35MyZM5tyq+02btxIVlZWs16bqFKhzM45ln+2lZnvbuaz7xxDuztOH9qJXbMS+f+GTZMK73OsVCtzS8tbWFi41DnX4Ca7SlRxkJ+f76qqqpr12srKSgoKCuIbUMClUpl/2LKVv7+0lhuee4fN24yfH7g7vy7aKyX6r1LpfY5ItTK3tLxm1qhElbB9VBEa7CBB1j4jnZJxA+m96X0Wf9eDexe9x6PLP+KiQwfxizH9ycxInRqWSHMl8r+SyKoROdEHo+Y+aVUJCYyumcafjh/GM78ex/67d+dP/3qbw25YwDNvrtP8K5EGtGQr+m7RC9O2Na8WVQ3ETsrNAWq0/JEE0V654c0a/3b2KNqlpzH5vqWcPH0Rb3yoTbFF6tOSGlUOMCpegTRTBRDbvpnnHRcJrILBvXj64rH86fh9Wf3ZRo699QUufehV1n21ye/QRAInUZr+cqhdcwKYAkyMOVbqHRcJtIz0NH4+eg/mX1FA6fgQT772CYXXVXJjxQq+37zV7/BEAiOwicrMsr319mYRTlJlZlZuZhMi53jNf1O88yaY2ZWE1/5Ts58kjK4d2nHVkUOouHQ8hXv35MaKlRReV8k/l3+k/isRAjzqL5KEGnHeMsKbOYoktN136cRtp49k8Zr1/M+Tb/HrB1/l7y+v5ffHDmX/flofWVJXYGtUIqnqgAE5PH7BIUydMJwPN3zP8X95kcseeo3Pvlb/laQmJSqRAEpLM07K78f8ywuYPH4gT7z2MYXXVXJb5Sp+2KL+K0ktSlQiAZbVPoPfHLk3z10yjjEDezD1mXc5/IaFzHvnU79DE2kzSlQiCaB/j87ccWY+f//lAaSlGb/8WxVn372YNV9863doIq1OiUokgYzfqyfPXDyO/zpqCEvWbuDwGxZy7bPv8N3mLX6HJtJqlKhEEkxmRhqTxoWYd9l4jhneh7/MX82h1y/gqTc+0XB2SUotTVS1Vi4XkbbRq2sHpp28P7MmjyG7Uybn/2MZZ9y5mFWfbfQ7NJG4anaics6tQStAiPhuVP8cnrjwYP5w3FBe+7CGI29aSNkzag6U5NGiGpWXrETEZxnpaZx5UH/mXVbAcfvtxu2Vqym6fgFPqzlQkoD6qESSSM8u7bn+pP2YNXkMXTu247x/LOPMu5dodKAkNCUqkSQ0qn8OT/7qEH5/7D4sfy88OvD6597VYreSkJSoRJJURnoaZx88gLmXjeeoYb25Zd4qiqYt4Nm3tFmjJBYlKpEk16trB248ZQQzS0aT1T6D0nuXcvbf1BwoiUOJSiRFjA7twpMXHcLvjh5ClSYLSwKJS6Iys4fM7GdRjw81s67xuLaIxE+79DTOHbvjZOGi6xfwr9c1OlCCK141qnLn3CORB865udTeIj7wzGyDmbmYnyv9jksk3qInC3frlMkF9y/j1BmLeGfd136HJlJLvBJVtZn1BzCz58xsCVAcp2u3CTPLBiY65yzyA0xxzk31OzaR1hIZHfjH4/flnXXfcPTNL/D/HnuTmu82+x2ayHbN3uHXzFYCFcAs59w8M9vfa/4rByqcc1/FK0hv+/ka51xFHc/lEa69VQMhoLqu8xoj+nXePWc3L2KRxJGeZpwxeg+OGdaHaXNWcN+i93j8tY+5pGgvTj9wdzLS1ZUt/mrJJ/BhwonqJC9pTQXKgNVxTlJFwIx6ngsBZc656c65CufcdKDUO94kzrmaqOtmAznOuermxi2SaLp3zuSPx+/LUxePZZ8+Xfn9429x5E3PU/nuZ36HJimuJWv9/cY597BzbrJzbhBQSjhZ/dbMVpnZ7S0JzMxCZlZOuJa0vp7TSgnX4KKVE06YLXEV8FALryGSkPbu3ZV/nHsg5WeMZPPWbZx19xLOunsxKz/9xu/QJEU1u+kvlrfu3wzvBzMb0MLrVRNORJhZfYvfTqB2oqoC5kQemFkJMHAnt5pTR1NhkXNOC+5KyjIzDh/am8LBvbjn5bXcNHclR9z0PKeM6sevi/aiZ5f2focoKaTJicrMujrnGhwa1NoL1nrNc7VqW865GjPDzELOuWqvObAp1y2KvaZIqsrMCA9n/1leX26eu5L7Fr3HY69+zOTxIc45JETHzHS/Q5QU0KSmPzN7CNgQGeEXdfwncYypsXJgx76lGE3up/LkAfVdUyQl5XTO5OrjhvLcJeMYM3AXrntuBYXXVfJQ1Qds3ab5V9K6mtpHNQc4yTm3Nub4UjO7vI0n+Wa34rU1iEKkDqGeWcz4RT4PlY4ht1sHrpz9Okff/Dzz3/1ME4al1VhTPlxmdgXhyb11Nv2Z2bnOuTviFVzUdVcDpTHDx/OApd58p9jzHVDc3GHqjYypBCgByM3NHTlz5sxmXWfjxo1kZWXFM7TAU5mTg3OOxeu2MnvFZj7/3jEkJ42TBmcyoFu4OTAZy9yQVCtzS8tbWFi41DnX4OIQTeqjcs5da2bPmtky4Dnn3PxmRxgnZpa9k+a/VuP1fU0HyM/PdwUFBc26TmVlJc19baJSmZNHIXDJlm3c/8p73DxvFX94eRPHDO/DFYcPZs0bS5KyzDuTrO9zfdqqvE1KVGb2V8AIrzoxxau5LCM80q6GcHNc3GtU9Yg0z+UQ1afkDbKIfl5EWlFmRhpnHTyAE0f2ZfrCau54fg3PvLmO8X3T2Tf/B3pkaYSgtExT+6hWO+cOc87lO+fSgMOBucAooAhosyHdXi2qmtp9VTmEV7FQohJpQ106tOOywwaz4IoCThrVj/kfbGH81PncMGcFG3/QCu3SfE0dnr5DE5vXB1QBYGYjgJNouxoV3r3zCdfqIvIiMYlI2+vVtQP/d8Iwhmd+zsKabtzkDWu/6NBBnHrA7mRmaEkmaZqmfmIqzOzcep7b0NJgdiKHukf5TQEmxhwrpQ1rdiJSt96d07jt9JH884KDGZSbxe8ff4uiaQt4/LWP2aYh7dIETUpU3iTeWWZ2bvRQdG8VimpgZLwCM7NsMyszs1mEk1SZmZV7i8VG4qkh3FdWZmYTvC05ytTsJxIc+/fL5oFJo/nb2aPolJnORQ8s57i/vMALK7/wOzRJEE1emcJbcPaOmGNrzKyYOA5giCShRpy3jB2b/kQkYMyMgsG9GDuoJ4+9+hHXP7eCn9/5CmMH9WDKEXuz727d/A5RAixujcXOubmtvWySiCS29DTjZ3l9mXvZeH539BDe+OgrjrnlBS6euZwP1n/nd3gSUOrVFJE216FdOueODbHgikLOLxjIM2+u49DrF/DHJ//Nhm+1aaPsqM6mP2++VD7Q0h5PIzyk/eQWXkdEklC3ju248oi9+cWY/twwZwV3v7iGh6o+4PyCPTn74P50aKdFb6WeROWcm9zWgYhI6urdrQNlE4ZzztgBlD39DmXPvMM9L6/l0uK9+FleX9LTaq2UJilETX8iEhh75XbhzrNGMbNkNL26tOcKb9HbBSs+9zs08ZESlYgEzujQLvzzgoO55dQRfLt5C2fetZgz7nyFtz7+yu/QxAf19VH9GWjRDr1Rqp1zV8XpWiKSIsyMY/fblcOG5vKPRe9z87yVHHPLC5wwYjcuP2wwu2Z39DtEaSP1zaO6Bm9jwjjQbrki0mztM9L55SHhRW9vq1zF3S+u5V+vf8I5hwzgvIKBdOnQzu8QpZXVN5jiK0B1bBEJjG4d23HVkUM4Y/QeXPfsu9xWuZoHl3zAr4sGccoBu9MuXT0ZyUrvrIgklL7dO3HjKSN44sJD2LNXFv/92FscfuNC5vz7U+0ynKSUqEQkIQ3r242ZJaOZ8YvwBrGT7qni1BmLeONDNQYlGyUqEUlYZkbxPrk8++tx/PGnQ1nx6UaOvfUFLn3wVT6u+d7v8CROlKhEJOG1S0/jjDH9qbyigPMKBvLkG59QeF0l1z77jjZtTAJKVCKSNLp2aMeUI/Zm3mXjOWLf3vxl/moKrp3P/a+8z5at2/wOT5pJiUpEkk7f7p246ZQR/POCg+m/S2d+++gbHKUVLhKWEpWIJK39+2Uza/IYbj89j03/2caZdy3mzLsWs+LTb/wOTZpAiSqKmeWZ2ZXebsFlZpaEyJM1AAAQY0lEQVTtd0wi0jJmxpHD+jDn0nH87ughLH9/A0fcuJDfPvoGn3/zg9/hSSM0eYffZOUlpVnOuYHe42VAGVDqa2AiEhftM8J7YJ2Y15eb5q7kvkXv8firH3NewUDOOWSAthQJsISoUXk1nKJ6nsszsxIzK4r82czbFAHVkQfOuWqgpJnXEpGA6t45k6uPG8qzl4xjdCiHa599l0OvX8Djr32sCcMBFfhE5SWeGfU8FwLKnHPTnXMVzrnpQKl3vKlqqGN9w2ZeS0QCbmDPLO44cxT3n3sgXTu246IHlnPi7S+x/P0NfocmMQKbqMwsZGblQIj6F7YtBcpjjpUTbrJrEudchXffbO/PSM1M/VQiSeygPXvw5K8OYeqJw/lgw/eccNtLXDxzOR9pwnBgBLaPymt6KwUwsyn1nDaB2omqCpgTeWBmJcDAndxqTiRJOedGes2M1fzYDFhd/0tFJBmkpxknjerHUcP78NfK1cx4vppn3lzHpLEhzisYSOf2gf2qTAkJ+7fv1Xxq1bacczVmhpmFnHPVXnNgoznnZnvXDxHeS6smbkGLSKBltc/g8sMHc+qBu1P29DvcOn8VD1Z9wBWHD2ZCXl/S0szvEFNSYJv+GiEHwompnueb3LdkZtGN06VAfTU5EUliu2V35OZTR/DI+QexW3ZHrpz9Osfe+gKLqr/0O7SUZIkwysXMVgOlkSY671gesNQ5V+u/OGbmgOLo8xt5nxLCNbQcYH2kdrWTc0sAcnNzR86cObMpt9pu48aNZGVlNeu1iUplTg3JUmbnHIs+2cqsFZtZv8kxMjedkwdn0qtT7f/nJ0uZG6ul5S0sLFzqnMtv6DwlqjjIz893VVVVzXptZWUlBQUF8Q0o4FTm1JBsZf5+81bueL6a2ypXs3Wb4+xD+nNh4Z477DCcbGVuSEvLa2aNSlSJ3PQH/DhKT0SkNXXMTOdXhw6i8ooCjt1vV8oXVFN4XSUPLH6frduC/x/+RJbIiSoyGm+HuU9RiUuj9UQk7nK7duD6k/bj8QvDC95e9cgbHH3z87y0+gu/Q0taCZuovEEU1dSe55QD1HjD20VEWsXwvuEFb289bQTfbNrCaTNe4eZlm3jvy2/9Di3pJGyi8lQAse2bed5xEZFWZWYcM3xX5l42nisOH8xbX26leNpCrnn6bb7Z9B+/w0saiZKocqh7hYgpwMSYYxpWLiJtqkO7dC4o3JOysR05bv8f+69mqv8qLgKbqMws29tqYxbhJFVmZuVmNiFyjtf8N8U7b4KZXUl47T81+4lIm8vukMZ1E3/sv/rNI29w7C2af9VSgV2ZIpKEGnHeMmBZ60ckItI4kf6rf73xCdc89Q6nTF/EEUN789ujhrD7Lp38Di/hBDZRiYgkskj/VdGQXGYsDM+/mvfOZ3XOv5KdC2zTn4hIMujQ7sf5V8fs10f9V82gRCUi0gZyu3Zg2kn789gFB7NHVP/Vy6vVf9UQJSoRkTa0X79sZk8ewy2njuCr7//DqTMWUXpvleZf7YQSlYhIGzMzjt0vPP/qsuK9WLjiC82/2gklKhERn0T3X0XmXxVcW8n9r6j/KpoSlYiIz3K7dtg+/yrUszO/fTS8fuCLq7R+IChRiYgExvC+2TxUOobbTs9j4w9bOP2OVzj370tY/flGv0PzlRKViEiAmBlHDetDxaXjmXLE3iyqXs/hNyzk6sffoua7zX6H5wslKhGRAOrQLp3zCgYy//ICThrVj3teXsv4ayu584U1bN6yze/w2pQSlYhIgPXs0p7/O2EYT108luF9u/HHJ//N4Tcu5Lm31pEIO7THgxKViEgC2Lt3V+755QHcffYo0tOMknuXcuqMRbz50Vd+h9bqlKhERBKEmVE4uBfPXDyWPx6/Lys+3cixt77AZQ+9xrqvNvkdXqtRohIRSTAZ6WmcMXoPKq8ooGRsiCde+5iC6+Yzbc4Kvv1hi9/hxZ0SlYhIguraoR1XHTWEuZeNp2hILjfPXUnBdZU8uCS5JgynbKIysznNeU5EJGj65XTi1tPyeOT8g+jXvSNTHg5PGH5+5ed+hxYXKZeozKzIzEqAoqY8JyISdHm7d+fh8w7i1tNG8O3mLZxx52LOvGsxKz79xu/QWiQwGyd6W8zXOOcq6nguD8gHqoEQUF3XeY0ReZ2ZlTflORGRRBDZsLF4n1z+/tJabpm3iiNuXMjJo3bnkuJB9OrSwe8QmywQicrMioAZwMQ6ngsBZc654qhjs8ys2jlX3YZhiogkjPYZ6ZSMG8jEkf24ae5K7lv0Ho+/+hGTxw/k3LEhOmam+x1io/na9GdmIa/2EgLW13NaKRBbwykHylozNhGRZNC9cyZXHzeUOZeOZ+ygnlw/ZwWF11Uyq+qDhBlw4WuNyqsRlQKY2ZR6TptA7URVBWwf8OD1Kw3cya3mNLepUEQkGQzo0Zm/njGSJWvX86d/vc0Vs1/n7hfX8l9HD+HgPXv4Hd5OBaLprz5mlk0dtS3nXI2ZYWYh51y1c266PxGKiCSWUf1zePS8g3jyjU8oe/odTr/jFQoH9+Sqo4awV24Xv8OrU9BH/eVAODHV83yoDWMREUkKaWnGcd4Ow789am+q3tvAETcu5KpHXuezb4K3woUFZVFDM1sNlEY30Xmj/ZY656yO8x1Q3NQmPe+aRYT7uKYS1Sy4s+fquE4JUAKQm5s7cubMmU0JY7uNGzeSlZXVrNcmKpU5NajMiWPjZsdjqzcz7/0tZKTBUQPacUT/drTPqPXVu+PrWljewsLCpc65/IbOS7lE1Rry8/NdVVVVs15bWVlJQUFBfAMKOJU5NajMiWfNF98y9Zl3ePrNdeR2bc9lhw3mxLy+pKfVnbBaWl4za1SiCnrTH7C9r0pERFrRgB6duf3nI5k9eQx9unXkytmvB2KFi6Anqsg8qZzog1GJS/OoRETiLL9/Do+efxC3nDqCjT/8uMLFu+v8WeEi0InKG0RRDcTWqHIIr2KhRCUi0grMjGOjBlwsf38DR97kDbj4um0HXAR6eLqngvDyScuijuV5x0VEpBVFr3Bx6/xV3PPyWh579WNKxw1kb9pmjEOQalQ51K45AUyh9tJKpd5xERFpA907Z/Lfx+zDnEvGUzC4JzdUrGDK89+z9L0NrX5vX2tUXl/TVYTnQ2UDZWZWTHhY+GzYPrl3ipmVAUu8c8vU7Cci0vb69+jMbaePZOl76/l/D71C/106tfo9/V5CqYZG1Iycc8vYselPRER8NHKPHK4Y1ZFdstq3+r2C1PQnIiJSixKViIgEmhKViIgEmhKViIgEmhKViIgEmhKViIgEmhKViIgEmhKViIgEWmD2o0pkZvY58F4zX94D+CKO4SQClTk1qMzJr6Xl3cM517Ohk5SofGZmVY3ZOCyZqMypQWVOfm1VXjX9iYhIoClRiYhIoClR+W+63wH4QGVODSpz8muT8qqPSkREAk01KpFWZGZXRv0+wcyKoo+JJAozy4t5XOvz3FqfcSUqH6XKF5eZlXg/ZVHHkr7sZlYEFHu/5wE45yqAmth/9MnAzPK893VC1LGkfp+jyldSx7GkKbP3WZ4V9bjW57k1P+NKVD5JhS8u2P4Br3DOTQdC3j/glCh7jJOBGu/3aqDIx1hay1Xeztyh1v7iCgKvPNVe+aqTucyRMkYdquvz3GqfcSUq/6TCFxdAiB/LVu09Tvqym1me9487IhtYH/V4lzYOqVV5taglAM65qd6u3En/PgORVoJQCpUZ6v48t9pnXInKP0n9xRXhnJvu1aYA8oAqUqPsOX4H0MZGAbt4tYpIk1dSv89eYqo2sw38WM6kLrNflKikTXhNIMu8f9xJrY7aFIT/lx1JXtnAl20bVZv4MvL+RvdTJSszyyb8vl4DzDCzkM8htaW6Ps+t9hnPiNeFpMlS4YsrWpFzbor3e7KXPeR9aeUAOV6SfhCILDUTAmITWaL7kh/7MGoI17CS/X0uAa5xztWYWTUwgeQvc0R9n+dW+YyrRuWfBwm/mZCcX1zbmVmJc26q93sRSV5259xsb1ABhL+siKppFAE1SViznM2P72k24f6qpH6fo3nvdw1JWmavhpwfqSnX9Xluzc+4Jvz6yBvSWk24IzYpZ7RHDWtdT/h/mhOdcxWpUPZU472n64FRkdpzsr/PXn9cNZATKV+yl9kPSlQiIhJoavoTEZFAU6ISEZFAU6ISEZFAU6ISEZFAU6ISEZFAU6ISEZFAU6ISaSYzK495PMGbN9bY119pZnPiH1n8JEKMkvyUqESaL3Ztt2J23AqhIaMI/uraiRCjJDklKpFm8BYkjU1KIedcoxOVc26ic85irlvmx+Km9d23rhhF2poSlUjzFAFLIw+iVtJuNu8aVxLeDqXN+HVfkcZSohJpnmJ2XHA0H2/jQBGJLyUqkeaJbeaLTVwN8prbnPf7LGCD99QsM3PehnzR508ws6Xec0ujB25419rg/V5uZqvNLNvMQmY2y8w2eK+bE93E19B9o2OMOpbnXScSR0kd5dpQx3l5MedN8OJ0dcUmEqFEJdJE9TTz5bVwW4NJhJMdQCkwEBgZdc8JhFehrwAmEu4fq/XFbmZLCdfuyp1zNd61oq+fQ1STZUP3jeXFsdS7fzHhbS3KY0dAEt7qYxZQ7l035D2OXCfyeLZ3v0iZsuu7t6QubZwo0gDvSzg6IeQA2THDtvNjHpfVsctvvaI23wNYX8egjBnA1KjNJ2d7SakUiBzL9l4bSTxEnR8pyyRgqZlN8PbNaui+sWYA051zkQRYYWbLCCfNWTFlnhLZl8vMBgJXmlm2l0AjtatrvMfLCCctkVqUqEQaEPWlDGxPXFO8L9jInluttveQ12SWTfiL/sqYp9fHPC5lJ5xzy8wMag+tb0wcRV4cO9SevP3FagjXiqITVfTvq2MuF3lurpk9CFQk4WaSEidKVCJNF4okKc9EoKw17+f9OZLaTY47JKq6akRec93J3nVa0gcUeW1dta7q2GvH/B0R+5yZFRP+eyvz4qyIrg2KRChRiTQgpumvzmY/IOTVVCKa1PTXgO2JoSnztAC8OEOEk0GVd3hp/a9oVBwhwk110UJR128U7+9npNfnVwKUmdmVzrmpzYxPkpQSlUgDopv+vKa3ZZEk5A0KmBLbPNhMsc14kfsv85rWyvhx4EMknuz6ai5ebEXAxKi+oroGK9R53zpEElEpUU2MXo0tMniiybz4p5rZyYRXwhDZgRKVSNOMivkffxEQl7XwvOYwgNLIn1FNYZMIDx+PjKTLJpwsqqmnX8o5Vx1JcN71avhx4EVj7xt7XinhUX45hEf8RWprs5tSg/SGtE/xyhJpNswjpv9LBDQ8XaSpYmskI2ni/KkGTCWc/MqIal7zakQjvfvPITz6rpo6Ek+MSYSbK2d516yv1lPnfWN5A0aK+XG4eSnhGuXEBuKI9RDhUX6lUdeZ2loDUiSxmXOu4bNEpM5mPjObowEAIq1LNSqRxtuhmS8e6/uJSMOUqEQaL3aZpLj1T4lI/ZSoRBpvfcwIuxDx7Z8SkTqoj0pERAJNNSoREQk0JSoREQk0JSoREQk0JSoREQk0JSoREQk0JSoREQm0/w+Qom17YIVY0wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.semilogy([np.linalg.norm(x - z) for x, z in zip(conv_x_admm, conv_z_admm)])\n",
    "plt.grid(True)\n",
    "plt.xlabel(\"\\# iterations\", fontsize=20)\n",
    "plt.ylabel(\"$\\|x_k - z_k\\|_2$\", fontsize=20)\n",
    "plt.yticks(fontsize=20)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Учтём, что все свойства сохранятся при аффинных преобразованиях\n",
    "\n",
    "- Тогда наша задача в общем виде может быть записана как\n",
    "\n",
    "\\begin{align*}\n",
    "& \\min f(x) + g(z)\\\\\n",
    "\\text{s.t. } & Ax + Bz = d\n",
    "\\end{align*}\n",
    "\n",
    "- Модифицированный лагранжиан для неё будет\n",
    "\n",
    "$$\n",
    "L_{\\rho}(x, z, \\lambda) = f(x) + g(z) + \\lambda^{\\top}(Ax + Bz - d) + \\frac{\\rho}{2}\\|Ax + Bz - d\\|_2^2\n",
    "$$\n",
    "- В этом случае сепарабельность по $z$ и $x$, но не внутри этих переменных\n",
    "- В итоге, после внесения линейного слагаемого в квадратичное получим\n",
    "\n",
    "\\begin{align*}\n",
    "& x_{k+1} = \\arg\\min_x \\left( f(x) + \\frac{\\rho}{2}\\|Ax + Bz_k - d + u_k \\|_2^2 \\right)\\\\\n",
    "& z_{k+1} = \\arg\\min_z  \\left( g(z) + \\frac{\\rho}{2}\\|Ax_{k+1} + Bz - d + u_k \\|_2^2 \\right)\\\\\n",
    "& u_{k+1} = u_k + x_{k+1} - z_{k+1},\n",
    "\\end{align*}\n",
    "где $u_k = \\lambda_k / \\rho$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Как это всё использовать?\n",
    "\n",
    "- Часто приводить вашу задачу к стандартному виду с предыдущего слайда неудобно\n",
    "- Поэтому лучше для конкретной задачи приводить её руками к виду, который допускает применение ADMM\n",
    "- Выписать аналитически все решения вспомогательных задач\n",
    "- Реализовать их вычисления наиболее оптимальным образом (сделать факторизации матриц, которые не меняются с итерациями)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Задача линейного программирования\n",
    "\n",
    "\\begin{align*}\n",
    "& \\min c^{\\top}x\\\\\n",
    "\\text{s.t. } & Ax = b\\\\\n",
    "& x \\geq 0\n",
    "\\end{align*}\n",
    "\n",
    "- Модифицированный лагранжиан\n",
    "\n",
    "$$\n",
    "L_{\\rho}(x, z, \\lambda) = c^{\\top}x + I_{z \\geq 0}(z) + \\lambda^{\\top}(x - z) + \\frac{\\rho}{2}\\|x - z\\|_2^2,\n",
    "$$\n",
    "\n",
    "где $c^{\\top}x$ определена на множестве $Ax = b$.\n",
    "\n",
    "- Шаг обновления по $x$ примет вид\n",
    "\n",
    "$$\n",
    "x_{k+1} = \\arg\\min_{x: \\; Ax = b} c^{\\top}x +\\lambda^{\\top}x + \\frac{\\rho}{2}\\|x - z\\|_2^2\n",
    "$$\n",
    "\n",
    "- Получим систему из условий оптимальности\n",
    "\n",
    "$$\n",
    "\\begin{bmatrix}\n",
    "\\rho I & A^{\\top} \\\\\n",
    "A & 0\n",
    "\\end{bmatrix}\n",
    "\\begin{bmatrix}\n",
    "x_{k+1}\\\\\n",
    "\\mu\n",
    "\\end{bmatrix}\n",
    "= \n",
    "\\begin{bmatrix}\n",
    "-\\lambda_k - c + \\rho z_k\\\\\n",
    "b\n",
    "\\end{bmatrix}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     con: array([-5.55111512e-17, -1.11022302e-16, -5.55111512e-17, -1.11022302e-16,\n",
      "       -2.22044605e-16, -1.11022302e-16, -1.11022302e-16, -3.33066907e-16,\n",
      "        0.00000000e+00, -5.55111512e-17])\n",
      "     fun: 0.3506633242985203\n",
      " message: 'Optimization terminated successfully.'\n",
      "     nit: 22\n",
      "   slack: array([], dtype=float64)\n",
      "  status: 0\n",
      " success: True\n",
      "       x: array([0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.004794  , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.16781554, 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.09238484, 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.22680145, 0.        ,\n",
      "       0.09523994, 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.05256825, 0.        , 0.27356996, 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.03652139, 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.39543046, 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.        , 0.        ,\n",
      "       0.        , 0.        , 0.        , 0.05843656, 0.        ])\n"
     ]
    }
   ],
   "source": [
    "import scipy.optimize as scopt\n",
    "\n",
    "m, n = 10, 200\n",
    "A = np.random.rand(m, n)\n",
    "b = np.random.rand(m)\n",
    "c = np.random.rand(n)\n",
    "\n",
    "scipy_linprog_conv = []\n",
    "def callback_splin(cur_res):\n",
    "    scipy_linprog_conv.append(cur_res)\n",
    "    \n",
    "res = scopt.linprog(c, A_eq=A, b_eq=b, \n",
    "                    bounds=[(0, None) for i in range(n)],\n",
    "                    callback=callback_splin, method=\"simplex\")\n",
    "print(res)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "def update_x_admm(x, z, lam, A, b):\n",
    "    n = x.shape[0]\n",
    "    m = A.shape[0]\n",
    "    C = np.block([[rho * np.eye(n), A.T], [A, np.zeros((m, m))]])\n",
    "    rhs = np.block([-lam - c + rho * z, b])\n",
    "    return np.linalg.solve(C, rhs)[:n]\n",
    "\n",
    "def update_z_admm(x, z, lam, A, b):\n",
    "    x_hat = lam / rho + x\n",
    "    return np.clip(x_hat, 0, np.max(x_hat))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.35159598712261525 0.3506633242985203\n"
     ]
    }
   ],
   "source": [
    "x0 = np.random.randn(n)\n",
    "z0 = np.random.randn(n)\n",
    "lam0 = np.random.randn(n)\n",
    "rho = 10\n",
    "\n",
    "x_admm, z_admm, lam_admm, conv_x_admm, conv_z_admm, conv_lam_admm = admm(update_x_admm, \n",
    "                                                                         update_z_admm, \n",
    "                                                                         rho, x0, z0, lam0, max_iter=10000)\n",
    "print(c @ x_admm, res.fun)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.000932662824094932 0.008932164763089577\n"
     ]
    }
   ],
   "source": [
    "print(c @ x_admm - res.fun, np.linalg.norm(x_admm - res.x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([-20.,   0.,  20.,  40.,  60.,  80., 100., 120.]),\n",
       " <a list of 8 Text xticklabel objects>)"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAoAAAAH4CAYAAADaVFwSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3WtwXOd95/nf0xcAvIBsgpRoWTIlNZxYtsexDNKOvcmMKQvI1kx2dnZTgJRUkn0xFRHZZLd2U3GR4dZUpWamdhjQrtrZ2p2qBTSVqZ1JpkSCTs0mdmY8hGwmlZSTkIQviS3f2NRlIlkSBbYIkmig+5xnX5xzGo1G497nnKfR308VqhunT3c/xAMCP/yfyzHWWgEAAKB7ZNJuAAAAAJJFAAQAAOgyBEAAAIAuQwAEAADoMgRAAACALkMABAAA6DIEQAAAgC7TFQHQGFMwxpw2xhTD+0VjzIQxZjjttgEAACTNdMNG0MaYoqQbDYfKkp6z1l5KqUkAAACpyaXdgASNSLomacBaW0q7MQAAAGnppgAoa21ZQfUPAACga3XFHEAAAAAs66YKYNEYMxreH5A0t5k5gEeOHLGPPfZYrA27d++e9u3bF+t7YHvoGzfRL+6ib9xEv7ir3X1z/fr1W9baBzY6r1sC4JwkNQY+Y8y0MUatQqAx5pSkU5J09OhRff7zn4+1cXfv3tX+/ftjfQ9sD33jJvrFXfSNm+gXd7W7b5566qlXNnNeV6wCbiXcAmbSWju43nknTpyw165di7UtV65c0cmTJ2N9D2wPfeMm+sVd9I2b6Bd3tbtvjDHXrbUnNjqvm+cAlhQMCxfSbggAAECSuiIAGmNOtzg8F94Wk2wLAABA2nZ9AAw3gZ4IbxsNhLfsCQgAALrKrg+A4abP4y02fx6WNBvuDQgAANA1dn0ADM01VgDDeX/jkp5Lr0kAAADp6IptYKy1l4wxo+E+gIclFSSNcUk4AADQjboiAEpqud8fAABAN+qaAAgAQCdaXFzU3Nyc5ufn5Xnetl7j4MGDeumll9rcMrTDen2TzWbV39+vgYEB9fb2tvV9CYAAADhqcXFRr776qg4dOqTHHntM+Xxexpgtv878/Lz6+/tjaCF2aq2+sdaqWq3qzp07evXVV3Xs2LG2hsBuWQQCAEDHmZub06FDh3TkyBH19PRsK/yhMxlj1NPToyNHjujQoUOam5vb+ElbQAAEAMBR8/PzOnDgQNrNQMoOHDig+fn5tr4mARAAAEd5nqd8Pp92M5CyfD6/7fmfayEAAgDgMIZ9Ecf3AAEQAACgyxAAAQAAugwBEAAAoMsQADtE6e27+shvf1k3b91LuykAAKDDEQA7xMvv3NP8Yk3ffeNO2k0BACA1g4ODMsZodna25eOzs7Myxqz6OHTokMbHx1Uulzd83szMzLptiM47f/58256fNAJgh6hUfUnSm3cqKbcEAIB0zM7OqlQqSZIuXLiw7rkTExO6ceOGbty4oevXr+vs2bO6ePGiHn/88TXDY2R6enrNxy5durRhO3f6/CQQADvEwlKw/89b84sptwQAgHRcuHBBQ0NDGh0d1dTU1LrnFgoFFYtFFYtFDQ0N6fTp07p586aKxaKefvrpNZ83PDysixcvrvn45OSkhoeHY3t+UgiAHaJSIwACALrb1NSUnn32WY2MjKhcLm9YyWtWKBT0/PPPq1wurzn8OjY2pnK53HIYNzo+Nja25nvs9PlJIQB2CCqAAIBuNjs7q3K5rNHRUT3zzDOSgmraVg0NDWl4eFhnzpxp+fjAwICGhoZavnZU2VuvgrfT5ycll3YDsDmLtWAO4FvMAQQASPqnf/Rtfef1zS0M9DxP2Ww25hat9qH3HtBv/8MPt+W1Jicn60O6UhDkLl68uK0QODIyopmZGZVKpfrrNXr22WdbBsTp6WkNDw9rYGBg3dff6fOTQAWwQ1SqVAABAN3r4sWLGh0drX/+7LPPrjnUupEo9EULSppF79P42lsZvt3p85NABbBDREPAc/eWtFTz1ZMjuwNAN9tKZW1+fl79/f0xtiZeMzMzKpfLGhkZqR8bHR3VmTNn6lW1rSgUCpK05pYwUaWxccFGNHwbDT+vZ6fPTwIpokNEi0Ak6dZdqoAAgO4xPT2tQqGwIuhFIWuj1cCtRItHhoaG1jxndHR0xZYtUdCMwuNGdvr8uBEAO8TCkl+/z16AAIBucvHiRZXL5VWbO0dDuFsdBr569aoktZz/FxkfH6+/djR8Gx3bjJ0+P24MAXeIxgog8wABAN0iClDT09OrKnalUkkjIyNb2luvVCrp0qVLOn369LrnRRXG6enp+lBx4xzEjez0+XEjAHaIypKnB/p79fb8IgEQANA1JicnVSgUWoanaJPnzV5do1wua2xsTIVCQWfPnt3w/PHx8foq4+2Et50+P04MAXeISs3Tw4U9yhi2ggEAdI9Lly7p1KlTaz4eDas2h8ByuaxSqaRSqaTZ2VmdP39ex48fV6lU0osvvripuXijo6MqlUr1Dai3aqfPjxMBsEMsLHna35vT4f29eusOFUAAwO4Xhbr1wtNam0KfOXNGg4ODGhwc1PHjx3Xu3DkNDw/r5s2b6y7+aNS47+B2Kng7fX6cGALuEJWqr4F9GT3Y36u35qkAAgB2v9HRUVlr1z2nUCisOGdoaGjD57Sy1vNu3Lix4Xvu5Pnz8/Nbbms7UAHsEJWqp758VkcP9DEHEAAA7AgBsENEAfDB/l69yRAwAADYAYaAO8RC1dOefFaH9ub1zr1F1TxfuSz5HQAAbB0JokNUqr768hk9cKBP1krv3FtKu0kAAKBDEQA7gLW2XgE82t8riauBAACA7SMAdoDFWnAZuN58Vg8e6JMktoIBAADbRgDsAIvVIADuCReBSFwODgAAbB8BsAMsVIPrAPflszqyPwqADAEDAIDtIQB2gEoYAPf0ZNSTy+jwvh62ggEAANtGAOwA9QpgLitJeqC/V29TAQQAANtEAOwAUQWwrycIgA9yNRAAALADBMAO0FwBfLC/l1XAAABg2wiAHSBaBdyXD7rr6IFevX13UZ6/9YtdAwAAEAA7wEJ9EUhUAeyT51vNcTUQAEAXKJfLGhsb0+DgoIwxGhwc1Pj4uEql0qZf4/z58xoZGYmxlZ2FawF3gEqLIWAp2ArmgfA+AAC7UalU0uDgoIrFosbHx1UoFHTjxg2dP39ekjQ5Obmp17l69apmZmbibGpHIQB2gFUVwIargXz4vak1CwCA2I2Pj6tYLOrGjRsrjp89e3ZLFcDp6el2N62jMQTcASrRHMAWFUAAAHaza9euaXh4eNXxQqGgoaGhFFq0OxAAO8DyNjBBd0XDvqwEBgDsdgMDA7p27Vrazdh1CIAdoFL1ZIzUkw26qy+f1cE9efYCBADsehMTE5qdndXIyMiGc/ii8w4dOqRDhw5pZGREs7OzkqQzZ87IGFM/N/q88TmDg4O6dOnSqnPK5fKK9zl//nzL452EOYAdoFL1tCefXfGNe/RAr968wxAwAHSt//hb0o/+elOn7vFqUjaFX/nv+Yj0939nRy8xOjqqyclJjY+P1wPg6OioJiYmVCwW6+fNzMxoZGREw8PDev755zU3N6fLly/rwoUL6w4VP/fccxofH9fIyIjOnTunsbEx3bhxo77o5Pz585qamtLp06frz7lw4YKGh4dVKBR29G9LEwGwAyxUPfXlsyuOPdjP1UAAAN3h1KlTOnXqlGZmZjQ9Pa2LFy9qcHBQly9frs8PHB8f1/DwsC5fvrzieRu5fv16/f7w8LCOHz+uiYkJTU5OqlgsamhoSJOTk/UAWC6XNTs7u+nVx64iAHaAStXXnlUBsFc3b91LqUUAgNRtobK2MD+v/v7+GBuTjOHhYQ0PD2tiYkLHjx/X2NiYbt++rVKppFKptONQNjQ0pKGhoRVzDs+ePauxsTHNzs5qaGhIU1NTkjYXLl3GHMAOsFD11Jtf2VUPHOjVW/MVWcvVQAAA3aVQKOjMmTP1aly0HczAwMCOX/vEiRMrtpcZHR1VoVDQhQsXJAXDv6Ojozt+n7QRADvAYjgHsNHR/j5VPavb96sptQoAgPQ0zv+L7s/Nze34da9du7bitSXpmWee0dTUVD1wjo+P7/h90kYA7AAt5wAeYC9AAMDuNz4+3nK17cTERH0vwGKxqGKxqImJiVXnbbRSt/Hx2dlZzc7Ortp3MKo2Tk1NqVAotNyXsNMwB7ADtJ4DuHw1kCfek0arAACIV7lc1sWLFzU1NaXR0VF9/OMf1zvvvKNLly6pVCqtuLrH5OSkRkZGNDY2pmeffVZzc3P1xxsXhjR7+umnNTExoVKpVL/U3NmzZ1ecEy0GOXfunJ555pl4/rEJowLYARaWPPU1zQGMrgbCVjAAgN2qUCjo5s2b9YB27tw5TU1NqVgs6vr16yvm4g0PD+v69esql8saGxurVwg3WhgyMTGhM2fO1FcRX79+veX2LlElcjcM/0pUADtCpeapd80hYLaCAQDsXoVCQadPn16xD99ahoaG1qz2TUxMtBwijkLfRsrlcr0SuBtQAewAlaXVi0D29uTU35vT2wRAAABiF21GvVtQAewAlZq/aghYWt4KBgAAtF+pVNLs7KyuXr2qUqnU8Xv/NSIAdoCFFhVAKZgH+OYdKoAAAMShXC7rueeekyRNT0939KXfmjEE7DhrrSq11dvASNLRA31UAAEA2IaJiYkNL6YwNDSk27dv6/bt27ti8+dGBEDHLdZ8WauWAfDB/l69dWeRq4EAAIAtIQA6brHqS1orAPZpsebrzkIt6WYBAIAORgB0XKXmSVLrOYBcDQQAAGwDAdBxC0tBAGy1Crh+NRC2ggGAXYtpPojje4AA6Lj1KoAPHQwC4I237ybaJgBAMrLZrKrVatrNQMqq1aqy2dU5YCcIgI5brgCu7vhHD+/VE+/p1wt/9Rp/IQLALtTf3687d+6k3Qyk7M6dO+rv72/raxIAHVdZZxGIMUa/9MlH9Z037ujrr5UTbdfLt+7pt77wLb30Bj+YACAuAwMDun37tm7duqWlpSX+2O8i1lotLS3p1q1bun37tgYGBtr6+mwE7bhKde05gJL0333sYf3Of/yufu9rr2jo2KHE2vX/fu1lvXD1NV289pp+4RPH9Js/8wEN7OtJ7P0BoBv09vbq2LFjmpub08svvyzP87b1OpVKRX19fW1uHdphvb7JZrPq7+/XsWPH1Nvb29b3JQA6bjkAth7739+b088NPawX/uo1/ZP/5kPbDmG+b/XuQlXv3FvSwL6edV/HWqsXX3pLnywO6In3HNC/+4tX9EfffF3/6/CP65c/9ajyWQrLabPWasnzVfWsqjVfS56vpZqvmm9Vbbpf9Xx5vlXNs/X7nrX1Y9HnNd/K84Ln1fzwePjhh4/7/vL5vm/lW8mzVtZa+b7k2+DYmlUMIxkZZYyUMUaZTHCbyxjlshnlMkbZjNHfvrakb9sfKp81ymcz6sll1JvLan9vTgf6curvy6u/L6f+vpz29uTUl8/IGJNsJ3Qxa62sDfq7saejHjDGyEgyRvTLJvT29uqhhx7SQw89tO3XuHLlij72sY+1sVVol7T6hgDouIXq2otAIr/0yUf1b7/2ii5ee02/+unBTb/2v/rqD/WH33hd79xb0u37S/L84Ef1sYG9uvLZk8pkWv9gvvH2Xb06d1+n/l5Rv/TJR/WLP3lM/+yL39E/++J39Ht/+Yp+/uPv08/+xHv1cGHPFv6l3WGp5mu+UtXdxZrmKzXdqVR1t1LTQtVTperp/pIX3F/yVKn5qoTHK1VfizVPizVfi433a8H9pZq//BEGvyRljJTNGGVMENCyxiiTMSuOBx/hL38T/PJvZq2Wg0N46zUEzprvq+YFn+vG9zbdPmOkvfms9vTktLcnq55cRj3ZjPK5jHqypv55Ty6jnly2fr83l1FfPqu+fEZ78ln15bPqzWXUm8+oJ5utnxMcy9bv9+WDx/LZTD2k5jImtrBjrVXVs/Wgv1gLvmcq1eD7ZPn7aPn4QsP9Si38nmvxvRV8vwV/KCyFfzBEfywEfxSs7B/fSvpPf7zlf4OJQn/4R0D0PVK/r+Ww2HhuxgRf12xG9e+zbPiHQuP3X+M5xhhl639krP4+jZ6bMUa5rFE2E/RfxpgVf3Tks8H3Tz6bUTYb/qGSyYTPMeEfLBllMwpuG9qQzQTvnQ3vG6OG/zct/u9kgsej0Nz8tYq+Lo3hWgoel6TbFV8/enfllmFWwf+z6KdFFNyD+8sB3jbehudH/0eDz1vcb3je8vfpynff4DtixffGiv7Xyu8NNX2eMY3fM41fkxbfXw3HMiZ48Ux4fvS1Nxmt/LzhPRp/rnUaAqDj1psDGPnxo/36yccH9Pt/+Yqe+7tFZdcIbo2+/O0f6XNf/p5OPHpIQ48e0uGw6vd6eUH/+s9u6tort/WJx1vPN3jxpbckSZ954kFJ0o8d7de//cef0Fe++5b+5cwP9C/++Lv6F3/8XQ0dK+hnf+K9+tmPPKT3HHRz6MHaoPL59vyiKlVf1ShghL/cvvOOp/5X5tSbC3659+QyqvlW9xc93V+q6f5SENruLQZhbr5SCz+qulOp6t2Fqsr3l2+jQL8ZffkwfOSy6s1n1JcLgkhvLqt9vTkN7FuufC2Hl8zKMNPwiyofBp3oF1RPNriNPs9lwl9k9V9cwWOZjOr3cxlT/0WXXfHLK9kffl/96lf1U3/300EgCUNvpeqt+PpHt/ernhbCfrq/5GlhqRYGJVt/fqUabKi+1FAtXawHKa9tgTofBoN8JrPq62iafqEs/1Je/qXqrwjDwfdpFPx2oieXUV897C5/n0Vht78v1xB4gttcdvn7Jdvw73jt1Vf02GOPrfolXA8WWh0UFAaF5eDfGCqaQ0pwnhRVlK08X2HFebn67NnlY9E5Nno8fN2oeu37UtXzw/th5bqpsh193aOqeWPg7hhXXky7BbtacyBcDo3Lf/hO/fIJfWrwcNpNlUQAdN5mKoCS9MufelT/07//uv7k+2/pM08cXffcN+9U9Ftf+JY+8vBB/fvnPqme3PKQ7b3Fmn7vL1/RH37zb9cOgN99Sx966IDe21DhM8bo6Q8e1dMfPKqXb93Tl/76DX3pW2/on3/xO/rnX/yO9vfmdHh/jw7v69GR/b06vL9Xh/bmdWhvjwrh7aF9efX35bW3J6t9PTnt7Q2CzWbDhbVWizVf85Wa7i3W6lW2uXtLmru3qLl7Vc3dW9Ste0t6605Fb95Z1Jt3Klrc6Af41a9t6v0j+3qy6u/L68CenAp7e/S+gb36yJ68CnvzOlAfmlx5u6cnqz358KMn+MXbiX9RJsUYUw+7au+0mJY839YraIu15YAYVduaq2XRsWpUKfN8LTUMsQdD676qYQWtcWjchu8Xhafo32sUBvFsQziPvg5ZsyL89zaEuL7wj4fepkpmY0VzrWr/dly58oZOnvzxtr2e66xdGcqDiqi/YppEqykTq6ZK+LYeUIP7yxXweoBVw/dJ9Bw1VNoagnP4ad33vvc9feADH1jV/sZqWnQgCi6rqrANoaYx3EsrQ0/j89TwfSytrJSt9V23smq4/G8LbhqrjY3VyfDR5gqlGv+ICML+qmMN1Us/fDG/aSQium3848T3l/+/Nr5m9IeIorb4y8+LLuDgAgKg46I5gL1rLAKJ/Ncffo8e6O/Vv/vaK+sGQN+3+s2L31Sl6utf/vyTK8KfJO3rzWn4g0f1pW+9od/+hx9eNZ+vfH9J11+5rV87ufZQ82NH9unXn3q/fv2p9+vG23f1lZfe0hvvVvTOvUW9c3dJr87d1+yrZZXvLwVDeevIZYx6c5kV879y4S+raC5aNCS1FP5yjWTk62Hztgq6p4Pmng7qno72LOgjPUvq781p/76c+g/ntL83p319PVo68Jgqh5+Q139MuVxWuYzRtdmv60N/5ydW/ILPZY32hkOJe3qy9cB6oC+v/X25TVVg0VmyGaN9vTnt6+VHJlYy9WHhtFuyviv3Szr5iWNpNwMO4aeZ4ypVT8ZIvbn1A2A+m9EvfOKY/q+v/ECvvnNfxw7vbXne7/75Tf3ZD2/p3M99RIMP7G95zn/70ffqi996Q3/2w1t66gMPrnjsT77/tjzf1od/NzL4wP4138daq7uLNZXvV3X7/pJu3w/mw91bCip495c83V0MhuWi+UVR9cRqeTgtlwmGN/fbeyouvqRj9/9G73n3Wzpc/pZytXur33gp/Jhfo9E9/dLRD0lHP6yB+UUV370p9RWkvYekPYek/Q9K+w9LWf77AAA6E7/BHFepeurLZTc1HPgLn3if/tVXf6jf/6tXdPbvf3DV499+/V2d/0/f08986Kh+/uPvW/N1Pv2BB3SgL6c/+sbrqwLgiy+9pSP7e/TRRwobN37pnvRfrkmvz0pL9yXrr/gwkvol9Ruj9ykcM6guSJV3pcodafGOVCkHz/WWpFpFqi0GH34teI/618VIfrhbvslIRz8sPfnz0nuflPY9EAS4voPSnoLUeyA4p5FflW79QHrzb6Qf/Y305relv/mCipV3pZu/3+IfZ6T9R6UDD0n975V6+1evath/VBr5pxt/nQAASFjXBEBjzJCkE5JKkoqSStbamXRbtbFK1deens2NLTx0cI9GPnhUF6++pt8Y/vEVC0cWljz9Ly98Q4W9ef3Oz31ExlsKwlZtUaqFt9UFqVZRr+/pV4u39Cff/oEqr1r19fZIJquqV9Pffu+qfrE4oMyPvhkEOb8medXg1q8Gwe2/XJNe/Zr0xjcl27DowWQkkw1vw+VWCidfRLf5vUFQ6zsQ3B54ROrZK+X6pFyvlO0NbjM5LU8KCW979kmPnJAePh4Esq165ETw0eBPvnJZn/7ER6WF28HH/Tnp7pvSndel+delO29It29KSy0ux3fosa23AQCABHRFADTGFCVNWGtHGo5NG2NK1tpSik3b0ELVU98Gw79auie9+7fSu6/ps0e+qy9976pe/Nzv6kj2ng5qXvv9efXW7uhS7Z76c56yn1/URkvwf03Sr2Uk/e7ysbykL0hBhJ5a58m5PunhE9JP/4Z07FNBqNqziYqhg2wmHw75bm7IGwCATtAVAVDSuKTJpmOTkiYkjSXfnM2rVD31NVcAvap080+l7/wH6ftfDipSofdL+p9zGc3X+jXv9etd9esVW9Bt+4iOPfygfuKxo1JuT1BFy4e3uT1Svi+ssvVJmay8Wk2fvXBd7z/Sp1//e49K1tMffONH+sp3b+lzzz6pPfl8uOY9H8yFy+SDqlx+j/TAB4LXBQAATuqWADiq1QHwmqTLKbRlS6I5gFq6J73859JL/5/03S8Fw5E9+6Uf+xnpPX9HOvi+8OMRZfof0sFsTgclPbLN981KKjz5kP7Pv3hVv1Qc1sE9ef3fX76ihx/foz0f+ck2/gsBAEDSdn0ANMYUFMz5m2s8bq0tBxszmqKTw8D356RX/0L//dtf0I9V/lo698NgPl3vAekD/0D60D+SBj8TVO5i8o+efFj/5s9f1pe//SN9/LEBld6+p//hk4/G9n4AACAZuz4AShqQgsC3xuNFBbPa0lG6osEf/hvpnd8LhnLn35Tu/ihYCSvpZ5TTjZ4npE/9hvTofyU99tOJDa9+9JGDevTwXv3hN17XfCVYdfv0B9ffZBoAALjPrHlR9l0iXP173Vq7ah8VY4yVNNK8GtgYc0rSKUk6evTo8RdeeCG29j368gUde/WSlnoGtNRT0FLPIS31HFKl74juHHhCp7/9iPb29eo3jqdzKbUvfH9JXyxV9Uh/Rr61+t9/uvX+grvV3bt3tX9/630MkR76xV30jZvoF3e1u2+eeuqp69baExud1w0VwC2z1k4pXOd64sQJe/LkyfjezPtpXfnTZ3Tyqae0p9XDN67okYcO6OTJofjasI6HPzivP/o//lSvzfv61U8P6uTJJ1JpR1quXLmiWPsf20K/uIu+cRP94q60+maD/UV2j3AuoHuyudUbCDdYrPrBIpCU/NjRfj3xnmBPveEPshUKAAC7QTcEwGh+30DjwYZA6N4CkAYLVU97etLtpn/8U4/ro48c1MeOHUq1HQAAoD12/RBwuNq3JKm5AjggqezkCuAG9W1gUvTMx9+nZ9a5dBwAAOgs3VABlKQZBZeBazQUHneWtTasAKYbAAEAwO7SLQHwjFZf8WM8PO6sJc+XtVpxTV8AAICd2vVDwFJ9GPiMMWZC0lUFe/9NuD/860siAAIAgPbqigAoSdbaWUmzabdjKypVT5LUl++WQi0AAEgCycJhUQDcQwUQAAC0EQHQYQv1CiABEAAAtA8B0GHRHEAqgAAAoJ0IgA5bWAoqgL3MAQQAAG1EsnBYpcYQMAAAaD8CoMMqSywCAQAA7UcAdBgVQAAAEAcCoMMWllgEAgAA2o8A6DA2ggYAAHEgWTiMfQABAEAcCIAOW6x6MkbqzdFNAACgfUgWDluoeurLZWWMSbspAABgFyEAOqxS9Zn/BwAA2o504bBK1WMFMAAAaDsCoMMWqh4LQAAAQNsRAB0WDAETAAEAQHsRAB1WqXrMAQQAAG1HunBYhSFgAAAQAwKgwxZYBAIAAGJAAHQYFUAAABAHAqDDWAQCAADiQAB0GItAAABAHEgXDmMOIAAAiAMB0FHWWuYAAgCAWBAAHbXk+fKttKeHAAgAANqLAOioStWXJPXm6CIAANBepAtHLVY9SVQAAQBA+xEAHbUQBsC+HAEQAAC0FwHQUdEQMBVAAADQbgRAR9UrgOwDCAAA2ox04agKQ8AAACAmBEBH1SuADAEDAIA2IwA6apEKIAAAiAkB0FELbAMDAABiQgB0VLQKmEUgAACg3UgXjlpYCiuAXAsYAAC0GQHQUZVatA0MARAAALQXAdBRlbACyLWAAQBAu5EuHFWp+erLZ2SMSbspAABglyEAOqpS9Zj/BwAAYkEAdNTCksf8PwAAEAsCoKMqNZ8KIAAAiAUB0FELS556CYAAACAGBEBHLdY8NoEGAACxIGE4amGJRSAAACAeBEBHVWosAgEAAPEgADqKCiAAAIgLAdBRlaqvXuZe6JwZAAAdH0lEQVQAAgCAGJAwHMVG0AAAIC4EQEdVqswBBAAA8SAAOshaqwUqgAAAICYEQAdVPSvfin0AAQBALEgYDlqoepLEEDAAAIgFAdBBiwRAAAAQIwKggypVX5KYAwgAAGJBAHQQQ8AAACBOBEAHVeoBkO4BAADtR8JwUFQBZAgYAADEgQDooKgC2EsABAAAMSAAOqhCBRAAAMSIAOigaBUwcwABAEAcSBgOYggYAADEiQDooKpvJUn5rEm5JQAAYDciADqo5gVDwPkM3QMAANqPhOGgmhdUAHNUAAEAQAwIgA6q+mEFMEv3AACA9iNhOKhaCyuAGSqAAACg/QiADqr5voyRsgRAAAAQAwKgg6qeVT6TkTEEQAAA0H4EQAfVPJ8FIAAAIDYEQAfVfMv8PwAAEBsCoIOqns8KYAAAEBtShoNqnmUIGAAAxIYA6KCq7yvHVUAAAEBMdnXKMMYUjDGnjTHF8H7RGDNhjBlOu23rqXmW6wADAIDY7OoAKGlA0oSkG5JuS7ou6aq1dibVVm2g6vnKMQcQAADEJJd2AxIwIumapAFrbSntxmxG1bMsAgEAALHphgAoa21ZUjntdmxWzfcZAgYAALEx1tq02xAbY0xRUnGrQ77GmFOSTknS0aNHj7/wwgtxNK/u7t272r9/f/3zz11d0KIn/ZNP7on1fbGx5r6BG+gXd9E3bqJf3NXuvnnqqaeuW2tPbHReNwTAYUlz4aEBSXPW2kubfY0TJ07Ya9euxdG8uitXrujkyZP1z5+d/JqspIvjn4r1fbGx5r6BG+gXd9E3bqJf3NXuvjHGbCoA7vYh4DlJagx8xphpY4y2EgKTVvOt+vLMAQQAAPHY1SnDWlu21k41HZ5UsDLYWTWPfQABAEB8OqICGA7lbtZcuOhjLSVJRWNMYYPzUlNlH0AAABAj5wNgGP62UrG7Kul8+NzT1trzTY9H8wGLkmZ33sL2q3ElEAAAECPnA2C4d9/YVp8XBUdjzKWm/f8Gwltn9wSsci1gAAAQo11bZgpD33iLzZ+HJc26OvwrBVcC6WEjaAAAEJPdnjLmGucPGmMKksYlPZdekzZWowIIAABi5PwQ8E5Yay8ZY0aNMaOSDksqSBpz/ZJwNZ9rAQMAgPjs6gAoyen9/tZS9azyGSqAAAAgHpSZHFTzqAACAID4kDIcVPWZAwgAAOJDAHRQzfOVZx9AAAAQE1KGYzzfyreiAggAAGJDAHRM1fMlSXnmAAIAgJiQMhxT860kcS1gAAAQGwKgY2phBZBrAQMAgLiQMhxT9agAAgCAeBEAHVPzwwogcwABAEBMSBmOqYUVwBxXAgEAADEhADqGVcAAACBupAzHRKuA2QcQAADEhQDomCqrgAEAQMzakjKMMReNMT/X8PnTxpgD7XjtbsMqYAAAELd2lZkmrbV/EH1irX1R0ok2vXZXqTEHEAAAxKxdKaNkjHlMkowx/9kYc1XSSJteu6tEFUDmAAIAgLjktvtEY8wPJM1ImrbWfsUY82Q4DDwpacZa+267GtlNon0AqQACAIC4bDsASvqCpKuSnjHGTEq6KelxSWOEv+1jH0AAABC3bQdAa+1vhXe/IEnGmMclDUv634wxQ5IuW2v/x503sbuwDyAAAIjbTiqAK1hrb0p6PvyIAiG2iH0AAQBA3LZcZtrs9i5hIMQWsQ8gAACI25ZShjHmoqTb0YrfhuOfaWObulqNfQABAEDMtlpmuizpGWvty03HrxtjPsvmzztXrwAyBxAAAMRkqymjoCAErmCtfdda+3lJz7SlVV2s6lMBBAAA8drSIhBr7eeMMV82xsxK+s/W2q/G1K6uVb8SCHMAAQBATLYUAI0x/48ko+AqH2eMMVbSrKRrksoKKoT/ut2N7CY1rgQCAABittVtYG5Ya381+sQYM6xg779hSVbS021sW1eqciUQAAAQs60GwHLjJ9baGQWXg5Mx5mMK5gBSAdwBrgQCAADittUy04wx5lfWeOz2ThuD5TmAWQIgAACIyZYCYLi587Qx5lcat3wJr/pRknS8ze3rOlXfKp81MoYACAAA4rHlS8FZa99V0zCvtfamMWZEQQjEDtQ8n6uAAACAWLXzWsAvtuu1ulnVs6wABgAAsaLU5Jiq56uHFcAAACBGq5KGMeazaTQEgRoVQAAAELNWpaYjibcCdVWfOYAAACBeJA3H1DzLdYABAECsWgVAm3grUFfzfeWYAwgAAGLUahXwSLgHndHqMBgdO2etvRNz27pS1bNcBQQAAMSqVQCcsdaeTbwlkBTsA8h1gAEAQJwYAnZMzWcVMAAAiBelJscs1XzlWQUMAABi1CppUH5KUc23yufoAgAAEJ9WAfDcek8wxjzZ4thjbWpP1+NawAAAIG6rkoa19t21TjbGPCfpK8aY5pB4iCuItEeVfQABAEDMWq0C3sjx5gPW2q8bY2SMedJa+402tKtr1bgSCAAAiNlWk4a11t601t5s8cDXJQ23p1ndi2sBAwCAuG01AL7fGHNgncfXHD7G5lR99gEEAADx2mrSOCfpRWPMR9d4nD0Ed6jGlUAAAEDMthQAwwUiZyV93Rhz1Rjzm8aYzxhjHjPG/IpazA/E1lQ9y7WAAQBArLa8CMRaO2OMGZD0vKTPRYclzUh6po1t60pVz2cVMAAAiNV2VgHLWluWNCZJxpjHJc2tt30MNo9rAQMAgLhtKwA2arUiGNtX5VrAAAAgZpSaHFPzuBYwAACIF0nDIb5v5VtRAQQAALEiADqk6vuSxBxAAAAQK5KGQ2pesI0i+wACAIA4EQAdUg+AVAABAECMSBoOWR4CpgIIAADiQwB0SNULAmCOVcAAACBGJA2HREPAVAABAECcCIAOiSqArAIGAABxImk4pOZHi0CoAAIAgPgQAB3CHEAAAJAEkoZDmAMIAACSQAB0SC3cBoZ9AAEAQJxIGg6pRhVArgQCAABiRAB0CFcCAQAASSBpOKS+CIQ5gAAAIEYEQIdEAbCHCiAAAIgRScMh7AMIAACSQAB0CPsAAgCAJJA0HMI+gAAAIAm5tBvQDsaYUUlla+1Mi8eGJJ2QVJJUlFRqdZ4L2AcQAAAkoeMDoDFmWNLzksZaPFaUNGGtHWk4Nm2MKVlrSwk2c1PYBxAAACShY0tNxpiiMWZSQVVvbo3TxiVNNh2blDQRZ9u2q+ZRAQQAAPHr2KRhrS1Za8ettVPrnDYqabbp2LXwuHNYBQwAAJJgrLVpt2HHjDE3JI03zu0zxhQk3ZZ0yFpbbjrfShpcaxjYGHNK0ilJOnr06PEXXnghtrZL0t27d7V//359sbSkS9+vanJkr3oJgU6I+gZuoV/cRd+4iX5xV7v75qmnnrpurT2x0XkdPwdwHQOS1Bz+GhQVLAxZJawqTknSiRMn7MmTJ+NoX92VK1d08uRJfcv7gfT97+vpk59mGNgRUd/ALfSLu+gbN9Ev7kqrb3Zzyiik3YCtiuYAZlkEAgAAYrSbA2DHqfpW+ayRMQRAAAAQn9SHgMOtWjZrbp0h3bVev7DV56Sl5vlcBQQAAMQu1QAY7dO3hadclXR+k+dG8/sGJNUDYLg4pPFxZ1Q9ywpgAAAQu1QDYLgKd9UGzm167bIxpqTVcwEHFFw1xLkAWPN95Vn8AQAAYrbb08aMgsvANRoKjzun5lnlWAACAABitlsC4IBar/o9o9UVxvHwuHOqnqUCCAAAYpf6IpDtCufynVWwn19B0oQxZkTSZWvtJak+DHzGGDOhYP5gdG1g54Z/Janq+cwBBAAAsevYABiu7N2wkmetndXqy8E5iTmAAAAgCaQNh1SZAwgAABJAAHRIzaMCCAAA4kfacEjNZx9AAAAQPwKgQ6qerzxXAgEAADEjbTikxpVAAABAAgiADqn6VjnmAAIAgJiRNhxS83zlWQUMAABiRgB0CBtBAwCAJBAAHVLjUnAAACABpA2HVLkSCAAASABpwyE1rgQCAAASQAB0SNVjFTAAAIgfacMhNd9XnkUgAAAgZgRAhwRDwHQJAACIF2nDIVWPCiAAAIgfAdAhNZ9LwQEAgPgRAB3h+1aezxAwAACIH2nDEVXflyT15OgSAAAQL9KGI2qelST2AQQAALEjADqiHgDZBxAAAMSMtOGIaAiYVcAAACBuBEBHLA8B0yUAACBepA1HVL2gAsg2MAAAIG4EQEfU/KACyBAwAACIGwHQEbWoAsgQMAAAiBlpwxFLHotAAABAMgiAjogWgeTZBgYAAMSMtOGImh8tAqFLAABAvEgbjqhGFUCuBAIAAGJGAHQEVwIBAABJIW04ouqzDyAAAEgGAdAR9UUgbAMDAABiRtpwRI0rgQAAgIQQAB1R5UogAAAgIQRAR1Rr0UbQdAkAAIgXacMR7AMIAACSQtpwBPsAAgCApBAAHbG8CIQuAQAA8SJtOKLmRxtBUwEEAADxIgA6oso+gAAAICGkDUewDyAAAEgKAdAR0T6AORaBAACAmBEAHVHzfOUyRsYQAAEAQLwIgI6oej6bQAMAgESQOBxR9Szz/wAAQCIIgI6o+VQAAQBAMkgcjqh5lgUgAAAgEQRAR1Q9SwUQAAAkgsThiJrvMwcQAAAkggDoCIaAAQBAUgiAjmAbGAAAkBQShyNqPtvAAACAZBAAHUEFEAAAJIXE4Yiq5yufoTsAAED8SByOqHElEAAAkBACoCOqvlWOIWAAAJAAEocjap6vPNvAAACABBAAHcEQMAAASAoB0BFV32cIGAAAJILE4YiaZxkCBgAAiSAAOqLmUQEEAADJIHE4YsmzbAQNAAASQeJwRM33lWcRCAAASAAB0BE1zyrHlUAAAEACSByOCK4FTAUQAADEjwDoiJrPPoAAACAZBEAHWGvl+QwBAwCAZJA4HODZ4JYhYAAAkAQCoAM8P7hlH0AAAJAEEocDamEFMMeVQAAAQAIIgA6IKoA9OboDAADEj8ThAM8GJUAWgQAAgCTsisRhjBk1xgy3OF4wxpw2xhTD+0VjzESrc9NUq88BZAgYAADEr+MDYBjmnl/j4QFJE5JuSLot6bqkq9bamYSatymsAgYAAEnKpd2A7TLGFCWdURDq5tY5dUTSNUkD1tpSEm3bKq++CKTj8zgAAOgAHRsAwzA3LknGmDMbnFuWVE6iXdvh+UECpAIIAACSQMnJAVQAAQBAkjq2ArgFRWPMaHh/QNKctfbSek8wxpySdEqSjh49qitXrsTawPl7C5KMvvPtv1burZdifS9szd27d2Pvf2wd/eIu+sZN9Iu70uqb3R4A5ySpMfAZY6aNMVovBFprpyRNSdKJEyfsyZMnY23k9/7gRUkVHf/Yk/qp9x+J9b2wNVeuXFHc/Y+to1/cRd+4iX5xV1p9s6vHHK215TDMNZpUsDLYGdE2MHkuBQcAABKQegUwXM27WXPhgo6dKCkYFi604bXaor4RNItAAABAAlINgGH420o17qqk81t4/dPW2ubzoy1jipJmt/DesanvA8giEAAAkIBUA2C4lctYHK8dhUtjzKWm/f8Gwltn9gTkSiAAACBJu7bkFO0T2GLz52FJs64M/0pcCQQAACRrtwTAAUmFFsfnGucYGmMKCjaPfi6phm1GtBE0+wACAIAkpL4IZLvCMHdWwVy+goLh3hFJl6MtXqy1l4wxo+E+gIfD88ZcuyRcfSNoKoAAACABHRsAwyHcdS8BF5637qbPLvDYBgYAACSIxOGA5UvBUQEEAADxIwA6oL4RdI7uAAAA8SNxOCDaCJp9AAEAQBJIHA5gEQgAAEgSAdAB0SIQ5gACAIAkEAAd4Nkg/BlDAAQAAPEjADqg5jP8CwAAkkMAdIBnLQtAAABAYkgdDvAsFUAAAJAcAqADPF/KcRUQAACQEFKHA2q+1EMABAAACSF1OMCzliFgAACQGAKgA6JtYAAAAJJAAHSA50t5hoABAEBCSB0OYBUwAABIEgHQAZ4v5dgHEAAAJITU4QDPWuWpAAIAgIQQAB0QLAKhKwAAQDJIHQ7gWsAAACBJBEAHeJaNoAEAQHJIHQ7wfDaCBgAAySEAOqBmuRYwAABIDqnDAZ4v5bkSCAAASAgB0AEeFUAAAJAgUocDPCv2AQQAAIkhADrA8y37AAIAgMSQOhzAtYABAECSCIAO8HwpzxxAAACQEFKHA2rMAQQAAAkiAKbMWiufawEDAIAEkTpSVvWsJCqAAAAgOQTAlNV8XxL7AAIAgOSQOlIWVQBzXAkEAAAkhACYspoXVABZBQwAAJJC6khZzQ8rgMwBBAAACSEApqwaVQBZBQwAABJC6khZzaMCCAAAkkUATFmVOYAAACBhpI6UsQ8gAABIGgEwZfV9AJkDCAAAEkLqSFmVOYAAACBhBMCUsQ8gAABIGqkjZfV9ALkSCAAASAgBMGXRKmCuBQwAAJJC6khZjVXAAAAgYQTAlLEKGAAAJI3UkbKlsALYk6MCCAAAkkEATFm0CpgKIAAASAqpI2VcCxgAACSNAJiyqs8+gAAAIFmkjpTVK4DsAwgAABJCAEwZ+wACAICkkTpSFl0JhH0AAQBAUgiAKWMVMAAASBqpI2VVrgQCAAASRgBM2YMHejV4MCNjCIAAACAZubQb0O1+8Scf1cMLN9NuBgAA6CJUAAEAALoMARAAAKDLEAABAAC6DAEQAACgyxAAAQAAugwBEAAAoMsQAAEAALoMARAAAKDLEAABAAC6DAEQAACgyxAAAQAAugwBEAAAoMsQAAEAALoMARAAAKDLEAABAAC6DAEQAACgyxAAAQAAugwBEAAAoMsYa23abXCaMeZtSa/E/DZHJN2K+T2wPfSNm+gXd9E3bqJf3NXuvnnUWvvARicRAB1gjLlmrT2RdjuwGn3jJvrFXfSNm+gXd6XVNwwBAwAAdBkCIAAAQJchALphKu0GYE30jZvoF3fRN26iX9yVSt8wBxAAAKDLUAEEAADoMgRAAACALpNLuwHdzBgzJOmEpJKkoqSStXYm3VZ1H2PMqIKv/2B4O2mtvdR0Dn2VImNMQdKEtXa86Tj9kpLwa/+spHckHVbw/6bU9Dh9kyBjzLCkofDTw5JuWGunms6hXxIQ/l4pt/rabqYPkugnAmBKjDFFBb/QRhqOTRtjSo0/RBGv8D9pKQp8YdC4bowZiH5w0ldOmJA00HiAfklP+P9mpDGQG2MmJY2H9+mbhIWBQdba8w3HRo0xp/hZlqwwiD8vaazFYxv2QVL9xBBwesYlTTYdm1Twiw7JKVprZ6NPrLVlBX3Q2Df0VYrCH4YDLR6iX1IQ/pH0fFP4G5U03HAafZO88eYKUfiH7UjjOaJfYmOMKYZ/CBUlza1x2mb6IJF+YhVwSowxNxT8Bd04ZFKQdNtaa9JrWfcIv94vSno6DH7R8aKkG5IGrbUl+ipdxphT4d0Ra+1Yw3H6JQXGmAlJstaeaTpebKhg0DcJM8ZclzTWXCEyxlyOKkn0S3LCr/WqUL6ZPkiqn6gApiDsyFV/IUQhJAwgiFn49S6GHy3RV+kKh1IutjhOv6TnlKSrzQcbwh99k44ZSZejoWCpXpmdDu/TLynbTB8k2U8EwHQMSMsd2gL/ERNirT3UOAQcGlYwebck+ipthTW+9vRLegqSSsaYU9EcszBoROibFIQV2bKCOcynoz5pWARCv6RvM32QWD+xCCQdhbQbgHWNSzoX3qevUmKMGW1ejd2AfklBQ/Wh2Li61Bgz0bBwir5JibX2uDHmsoK5YrOSnm54mH5J32b6ILF+ogIINAjnm801rqRD8sJhkLX+AkZ61vrldEEsJEhd+PPrjIKFH0UF1UAqe2iJAJii8JccHBH+oBxvXHrf8Bh9laxnNrPnFf2SuFLTrSQpnEZRaAwb9E2ywtWnM9ba2fD/zuMK+ul603n0S8o20wdJ9BMBMB3RD8/mfc0KTY8jWRNaOWQi0VeJCyexX9vgNPolBQ3zktabn0TfJCz62jauGrXWlsM/ZkvhYir6JX2b6YPE+ok5gCmw1paNMSWtHk4Z0PLiAyQo/Ov5TPPEW/oqFSckDRpjnm04NiSpGG5BctVae4l+SU39ygStHuP/TCpOKNi6qpVJhYup6Jd0bbYPkuonKoDpmVHwn7bRUHgcCQrnzUw07bk03DCcRV8lyFo7Za090/gh6bKk2fDzaGEI/ZKOSS1fbkxSvWrb+MuJvklWScGlLFspKFgQItEvLthMHyTSTwTA9JzR6svEjIfHkZCG7SsKxpih8GNYKzdUpa/Sd7jFMfolHVMKL/nWYELScw2f0zcJaqgcNV6NJZrXfJifZakYUOtFU5vpg0T6iSuBpMgsX0z9qoIhlWjyLhIQ7ay+xsMla+1gw7n0VQrCX2BnJD2j4IfplKTJaO9G+iUdDf1yQ0HlaXqNi9nTNwkKRzMGJb0THio3btcTnkO/xCT8nXJWwdd1VEFldkbS5cYtrTbTB0n0EwEQAACgyzAEDAAA0GUIgAAAAF2GAAgAANBlCIAAAABdhgAIAADQZQiAAAAAXYYACAAA0GUIgAAAAF2GAAhg1zHGTDZ9Ptp8mawNnn/aGHO5/S1rn05oIwB3EQAB7EbFps9HFFyWabM+LmnTgTElndBGAI4iAALYVcLrcTaHvaK1dtMB0Fo7Zq01Ta87EV4DN1FrvW+rNgLAZhEAAew2w5KuR5+EgbC8kxcMX+O0pKGdNa0z3hfA7kcABLDbjEiaafj8hKSrKbUFAJxEAASw2zQP9zYHwg2Fw642vD8t6Xb40LQxxhpjbjedP2qMuR4+dr1xwUn4WrfD+5PGmBvGmIIxpmiMmTbG3A6fd7lxqHej921sY8OxofB1onacavHvut3ivKGm80bDdtpWbQPQ+QiAAHaNNYZ7h6y1szt42ecUhEhJGpc0KOl4w3uOSppWEDLHFMw/XBWYjDHXFVQjJ6215fC1Gl9/QA1D1xu9b7OwHdfD9x+RdEHSZPOKaEmFsL2T4esWw8+j14k+vxS+X/RvKqz13gA6Ty7tBgDAdoXhpjFoDUgqNG2PcqLp8wlr7aYrgtbasjEmqijOtVhM8ryk89baM+Hnl8KwNy4pOlYInxsFOjWcH/1bnpN03Rgzaq29tIn3bfa8pClrbRQsZ4wxswrC6HTTv/mMtfZS+L6Dkk4bYwphMI2qgefCz2cVhEEAuwgBEEDHagg7kuqB8EwYXBQOxRattVNxvH84dFpQEKBONz081/T5uNZhrZ01xkirt7DZTDuGw3asqPZZa2eMMWUFVbzGANh4/0bTy0WPvWiMuSBpZocVVAAOIgAC2E2KUfgLjUmaiPP9wtvjWj30vCIAtqrghcO2z4avs5M5dtFzW1UJS82v3fQ1UvNjxpgRBV+3ibCdM43VSwCdjwAIoGM1DQG3HP6VVAwra5EtDQFvoB64trLPoCSF7SwqCFnXwsPX137GptpRVDBk26jY8PqbEn59jodzKk9JmjDGnLbWnt9m+wA4hgAIoGM1DgGHQ7CzUbgLFzOcaR4m3qbm4dzo/WfDIdYJLS/YiNpTWKvSFrZtWNJYw1y8VossWr5vC1HAG1fDUHNYYYwWfWxZ2P7zxphnFVx5BMAuQQAEsFt8vKlCNSypLdfKDYdFJWk8um0YEn1OwTYt0cragoIQVtIa8/6staUoOIavV9bygpHNvm/zeeMKVv0OKFgBHFUXL22l4hluHXMm/LdEw8dDappfCKCzsQ0MgN2iuYJ2XFvc/28D5xWEygk1DLOGFbzj4ftfVrAat6QWga7JcwqGrafD11yrStfyfZuFC11GtLyty7iCCujYBu1odlHBqt/xhtc5H9dCGgDpMNbajc8CAIe1Gu41xlxm4QIAtEYFEMBusGK4tx3X/wWA3YwACGA3aL7cW9vm/wHAbkQABLAbzDWtuC2qvfP/AGBXYQ4gAABAl6ECCAAA0GUIgAAAAF2GAAgAANBlCIAAAABdhgAIAADQZQiAAAAAXeb/B0HnJFOozAF5AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(10, 8))\n",
    "plt.plot([c @ x for x in conv_x_admm[:100]], label=\"ADMM\")\n",
    "plt.plot([c @ res.x for res in scipy_linprog_conv], label=\"Scipy\")\n",
    "plt.legend(fontsize=20)\n",
    "plt.grid(True)\n",
    "plt.xlabel(\"\\# iterations\", fontsize=20)\n",
    "plt.ylabel(\"$c^{\\\\top}x_k$\", fontsize=20)\n",
    "plt.yticks(fontsize=20)\n",
    "plt.xticks(fontsize=20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Комментарии\n",
    "\n",
    "- Сходимость по итерациям медленнее, но стоимость одной итерации также меньше\n",
    "- Основной выигрыш при использовании ADMM в получении не очень точного решения **параллельно** и очень быстро\n",
    "- Различные способы представления задачи в виде, пригодном для использования ADMM, порождают различные методы, которые имеют различные свойства\n",
    "- Например в [этой](https://papers.nips.cc/paper/6746-a-new-alternating-direction-method-for-linear-programming.pdf) статье предлагается альтернативный способ решения задачи линейного программирования через ADMM\n",
    "- [Метод SCS](https://stanford.edu/~boyd/papers/pdf/scs_long.pdf), используемый по умолчанию в CVXPy, основан на применеии ADMM к коническому представлению исходной задачи "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Выводы\n",
    "\n",
    "- Метод внешних штрафов\n",
    "- Метод двойственного подъёма\n",
    "- Метод модифицированнй функции Лагранжа\n",
    "- ADMM\n"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3 (cvxpy)",
   "language": "python",
   "name": "cvxpy"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}