HEBO/doc/source/mo_constrained.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Advanced optimisation\n",
    "\n",
    "In this section, we show how HEBO can be used for constrained and multi-objective optimisation\n",
    "\n",
    "$$\n",
    "\\begin{array}{lll}\n",
    "\\text{minimize} && (f_1(x), \\dots, f_m(x)) \\\\\n",
    "\\text{s.t.}     && c_1(x) < 0 \\\\\n",
    "        ~       && c_2(x) < 0 \\\\\n",
    "                && \\dots \\\\\n",
    "                && c_n(x) < 0\n",
    "\\end{array}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Objective function\n",
    "\n",
    "We use benchmark functions from pymoo "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAATgElEQVR4nO3df6zddX3H8ee7pUgXkS7rXSKX1mJWiEwSy24QQ7KhqFRMoNFNwBDnQmx0w8wfIylxcQazgCFzzozNdRtxmigwY5qbgGmiSEiIsF5SBKmpq4hyL2RUpfwhVUp5749zLp6e3nvP95zzveec7/f7fCRNzjnfb8/38+W2Lz59fz7fzycyE0lS9a0ZdwMkSeUw0CWpJgx0SaoJA12SasJAl6SaOGVcF964cWNu2bJlXJeXpEp6+OGHf56ZU0sdG1ugb9myhbm5uXFdXpIqKSJ+utwxSy6SVBMGuiTVhIEuSTVhoEtSTRjoklQTPQM9Im6PiGcj4gfLHI+I+GJEHIqIRyPigvKb2bJn/wIX33IvZ++6m4tvuZc9+xdW61KSVDlFeuhfBravcPxdwNb2r53Avw7frJPt2b/Ajd98jIUjR0lg4chRbvzmY4a6JLX1DPTMvB/45QqnXAl8JVseBDZExGvLauCiW/ce5Oix4yd8dvTYcW7de7DsS0lSJZVRQ58Gnup4P9/+7CQRsTMi5iJi7vDhw31d5OkjR/v6XJKaZqSDopm5OzNnMnNmamrJJ1eXdeaG9X19LklNU0agLwCbOt6f1f6sVDdcdi7r16094bP169Zyw2Xnln0pSaqkMgJ9FvhAe7bLRcDzmflMCd97gh3bprn5PeczvWE9AUxvWM/N7zmfHduWrO5IUuP0XJwrIr4OXAJsjIh54O+AdQCZ+SXgHuBy4BDwAvAXq9XYHdumDXBJWkbPQM/Ma3ocT+CvSmuRJGkgPikqSTVhoEtSTRjoklQTBrok1YSBLkk1YaBLUk0Y6JJUEwa6JNWEgS5JNWGgS1JNGOiSVBM913KZVHv2L3Dr3oM8feQoZ25Yzw2XnevCXZIarZKBvri/6OKWdIv7iwKGuqTGqmTJxf1FJelklQx09xeVpJNVMtDdX1SSTlbJQHd/UUk6WSUHRRcHPp3lIkm/VclAB/cXlaRulQ30Ts5Jl6QaBLpz0iWppZKDop2cky5JLZUPdOekS1JL5QPdOemS1FL5QHdOuiS1VH5Q1DnpktRS+UCHk+ek79m/wMW33GvAS2qUWgR6J6cxSmqqytfQuzmNUVJT1S7QncYoqalqF+hOY5TUVIUCPSK2R8TBiDgUEbuWOL45Ir4bEfsj4tGIuLz8phaz1DTGoFVLv/iWe9mzf2E8DZOkVdYz0CNiLXAb8C7gPOCaiDiv67S/Be7KzG3A1cC/lN3QonZsm+bm95zPdLtHHkC2jy0OkBrqkuqoSA/9QuBQZj6RmS8CdwBXdp2TwGvar88Ani6vif3bsW2aB3a9jekN618J80UOkEqqqyKBPg081fF+vv1Zp88A10bEPHAP8NGlvigidkbEXETMHT58eIDm9scBUklNUtag6DXAlzPzLOBy4KsRcdJ3Z+buzJzJzJmpqamSLr08B0glNUmRQF8ANnW8P6v9WafrgLsAMvN7wGnAxjIaOAwHSCU1SZFA3wdsjYizI+JUWoOes13n/Ay4FCAi3kAr0Fe/ptKDA6SSmqRnoGfmS8D1wF7gh7RmszweETdFxBXt0z4JfCgivg98HfhgZnaPR46FA6SSmqLQWi6ZeQ+twc7Ozz7d8foAcHG5TSuXA6SS6q52T4ouZ7mB0ATr6ZJqoTGBvtQA6SLr6ZLqoDGB3j1A2s16uqSqa0ygw28HSGOZ49bTJVVZowJ9kfV0SXXUyEC3ni6pjhoZ6NbTJdVRIwMdetfTXR5AUtU0NtAXrbRQl+UXSVXS+EBfqZ4Oll8kVUehR//rbMe21tLut+49yILLA0iqsMb30OHEBbyW4nRGSVVgoHdwOqOkKjPQOzidUVKVGehdnM4oqaoM9GU4nVFS1Rjoy3A6o6SqMdCX0aueDpZfJE0WA30FvaYzguUXSZPDQC/A8oukKjDQC7D8IqkKDPSCLL9ImnQGep8sv0iaVAZ6nyy/SJpUBvoALL9ImkQG+hAsv0iaJAb6ECy/SJokBvqQLL9ImhQGekmKlF8+ducj9tYlrZrGb0FXliJb2cFve+udv0eSylCohx4R2yPiYEQciohdy5zzvog4EBGPR8TXym1mNRQpv4CDpZJWR89Aj4i1wG3Au4DzgGsi4ryuc7YCNwIXZ+YfAh8rv6nV0av8Ag6WSipfkR76hcChzHwiM18E7gCu7DrnQ8BtmfkcQGY+W24zq6XI7BdwsFRSuYoE+jTwVMf7+fZnnc4BzomIByLiwYjYvtQXRcTOiJiLiLnDhw8P1uKKWCy/fOGqN/UcLP3kXd/n7F1322OXNJSyZrmcAmwFLgGuAf49IjZ0n5SZuzNzJjNnpqamSrr0ZCvSWz+eSWKPXdJwigT6ArCp4/1Z7c86zQOzmXksM38C/IhWwIvig6XggKmkwRUJ9H3A1og4OyJOBa4GZrvO2UOrd05EbKRVgnmivGbWQ5HBUnDAVNJges5Dz8yXIuJ6YC+wFrg9Mx+PiJuAucycbR97Z0QcAI4DN2TmL1az4VXUOVf96SNHWRPB8cwlz3W+uqR+RS4TKKttZmYm5+bmxnLtSbFn/wI3fvMxjh47vuJ50xvWc8Nl5xrskoiIhzNzZqljPik6Rj5dKqlMruUyZv08XepaMJJWYqBPiH4GTJ3aKGkpBvqEKPp0Kdhbl7Q0A32CFH26dJG9dUmdDPQJZG9d0iAM9Allb11Svwz0Cddvb91lA6TmMtAroJ/eussGSM3lg0UV4oNIklbio/8V5bIBUjP56H8N2VuX1M0aeoW5bICkTgZ6DbhsgCQw0GvBB5EkgYOitVN0sBQggMSBU6lKHBRtkKKDpdAKc3DgVKoLe+g11k9vfZG9dWmy2UNvqH5664vsrUvVZQ+9IeytS/VgD10n9dYXB0RXYm9dqhZ76A21Z/9C4VLM2ghezuRMe+zS2K3UQ3ceekP1s4Lj8UwSH0ySJp2B3nD9PJQEPpgkTTIDXX3vjgT21qVJZKDrFZ299aBVO1+JvXVpsjgoqmW5jIA0eZy2qIG4jIBULfbQVYgPJkmTwR66huYyAtLks4euvtlbl8Zn6AeLImJ7RByMiEMRsWuF894bERkRS15M9dA9d33luTAtC0eO8vE7H2HLrrudFSOtkp4ll4hYC9wGvAOYB/ZFxGxmHug673Tgr4GHVqOhmiw7tk2/0tsuuoyAA6fS6irSQ78QOJSZT2Tmi8AdwJVLnPdZ4HPAr0tsnypgkAeTnMMula9IoE8DT3W8n29/9oqIuADYlJl3r/RFEbEzIuYiYu7w4cN9N1aTrd9lBMAnTqUyDf2kaESsAT4PfLLXuZm5OzNnMnNmampq2EtrAtlbl8anSKAvAJs63p/V/mzR6cAbgfsi4kngImDWgdFmc+BUGr0i89D3AVsj4mxaQX418P7Fg5n5PLBx8X1E3Af8TWY6J7HhHDiVRqtnDz0zXwKuB/YCPwTuyszHI+KmiLhitRuoerAUI60+HyzSyPWzW9IiF/+SWtyxSBNlkN56dynGHrt0MgNdYzPIwClYipGW4+JcGqtBBk4XOXAqncgauiaOi39Jy1uphm6gayJ19tYXB0R7WbcmePVpp3DkhWOcacCrplwPXZUzSCnm2MvJcy8cAyzHqJnsoasyBinFgOUY1YslF9XGIHPYwXnsqg8DXbUzaG8dDHdVmzV01U7nHqdPHznKGevX8asXX+LY8d4dFNeLUV3ZQ1dtDFqOAXvrqg5LLmqUQcsxlmJUBZZc1Cid5Zh+5rFbilHV2UNX7Q3ykNIie+uaNJZcpDaX7lXVuXyu1ObSvaozA12N5NK9qiNLLhKWYlQd1tClgpzyqEnntEWpIKc8qsrsoUsr8OlTTRpLLtKQLMVoUhjoUgmGeUAJ3FFJ5bCGLpVguV2Uioa7OypptdlDl4Y0TJ0dLMeoP5ZcpBFw0w2NgoEujciwdXYw3LUyA10ag8Vw73dHpU6Gu7o5KCqNQecgKgxWa/eBJfXDHro0YsPU2sHeetMNvXxuRGyPiIMRcSgidi1x/BMRcSAiHo2I70TE64ZttFRXg670uGjhyFE+fucjbNl1t6s+6gQ9e+gRsRb4EfAOYB7YB1yTmQc6znkr8FBmvhARHwEuycyrVvpee+hSy7ADqdbZm2WoQdGIeAvwmcy8rP3+RoDMvHmZ87cB/5yZF6/0vQa6dDLDXb0MOyg6DTzV8X4eePMK518HfGuZhuwEdgJs3ry5wKWlZlnuadSiHERttlJ3LIqIa4EZ4Naljmfm7sycycyZqampMi8t1c4g2+V1cnel5ikS6AvApo73Z7U/O0FEvB34FHBFZv6mnOZJchBVRRWpoZ9Ca1D0UlpBvg94f2Y+3nHONuAbwPbM/N8iF7aGLg3GOnuzDf2kaERcDnwBWAvcnpl/HxE3AXOZORsR3wbOB55p/5afZeYVK32ngS4Nz3BvHh/9lxpg2FUfDfdqGPrBIkmTb9hB1O4ZMtbaq8ceulRDZaz6uDaClzPdXWnCWHKRGswlfevFQJcEGO51YKBLOonhXk0GuqQVdW7GsSaC4wPkguE+Gga6pMKGXa8dDPfV5I5FkgpbDN9hyjEuEjYe9tAlraiMWjvYWy+LJRdJpXCpgfEz0CWVznAfDwNd0qoy3EfHQJc0Mi4StroMdEkj5/TH1WGgSxqLsmbIgOG+yECXNHZlhvu6NcGrTzuFIy8ca9xqkAa6pIlSZrhDs3rvBrqkiWW498dH/yVNrB3bpl8J3TLCvcnLDthDlzSRyuy516m3bslFUqV1Lu97xvp1/OrFlzh2vL/sqkspxkCXVCtNfjLVQJdUW00LdwNdUiM0YdkBA11So9R52QEDXVLj1HXZAQNdUqPVKdwNdElqq3q4G+iStIQqhruBLkk9VGU1SANdkvowyQuGDR3oEbEd+CdgLfAfmXlL1/FXAV8B/gj4BXBVZj650nca6JKqYNLCfajVFiNiLXAb8A5gHtgXEbOZeaDjtOuA5zLzDyLiauBzwFV9tVKSJlCVVoNcU+CcC4FDmflEZr4I3AFc2XXOlcB/tV9/A7g0IqKUFkrShNixbZoHdr2NJ295N/941ZuY3rAeaPW6B3H02HFu3XuwtPYVWQ99Gniq4/088OblzsnMlyLieeD3gJ93nhQRO4GdAJs3bx6wyZI0fkv13AdZDfLpAZcpWMpIN7jIzN3AbmjV0Ed5bUlaLZ3hDv2VZs5s9/LLUCTQF4BNHe/Pan+21DnzEXEKcAatwVFJapyidff169Zyw2XnlnbdIoG+D9gaEWfTCu6rgfd3nTML/DnwPeBPgXtzXPMhJWmCLFeaKXt+OhQI9HZN/HpgL61pi7dn5uMRcRMwl5mzwH8CX42IQ8AvaYW+JKlDd2mmbIVq6Jl5D3BP12ef7nj9a+DPym2aJKkfRaYtSpIqwECXpJow0CWpJgx0SaqJsa22GBGHgZ8O+Ns30vUUagN4z83gPTfDMPf8usycWurA2AJ9GBExt9xqY3XlPTeD99wMq3XPllwkqSYMdEmqiaoG+u5xN2AMvOdm8J6bYVXuuZI1dEnSyaraQ5ckdTHQJakmJjrQI2J7RByMiEMRsWuJ46+KiDvbxx+KiC1jaGapCtzzJyLiQEQ8GhHfiYjXjaOdZep1zx3nvTciMiIqP8WtyD1HxPvaP+vHI+Jro25j2Qr82d4cEd+NiP3tP9+Xj6OdZYmI2yPi2Yj4wTLHIyK+2P7v8WhEXDD0RTNzIn/RWqr3x8DrgVOB7wPndZ3zl8CX2q+vBu4cd7tHcM9vBX6n/fojTbjn9nmnA/cDDwIz4273CH7OW4H9wO+23//+uNs9gnveDXyk/fo84Mlxt3vIe/5j4ALgB8scvxz4Fq0tSS8CHhr2mpPcQ2/i5tQ97zkzv5uZL7TfPkhrB6kqK/JzBvgs8Dng16Ns3Copcs8fAm7LzOcAMvPZEbexbEXuOYHXtF+fATw9wvaVLjPvp7U/xHKuBL6SLQ8CGyLitcNcc5IDfanNqbtXhj9hc2pgcXPqqipyz52uo/V/+Crrec/tf4puysy7R9mwVVTk53wOcE5EPBARD0bE9pG1bnUUuefPANdGxDyt/Rc+OpqmjU2/f997Gukm0SpPRFwLzAB/Mu62rKaIWAN8HvjgmJsyaqfQKrtcQutfYfdHxPmZeWScjVpl1wBfzsx/iIi30NoF7Y2Z+fK4G1YVk9xD72dzamqyOXWReyYi3g58CrgiM38zoratll73fDrwRuC+iHiSVq1xtuIDo0V+zvPAbGYey8yfAD+iFfBVVeSerwPuAsjM7wGn0VrEqq4K/X3vxyQH+iubU0fEqbQGPWe7zlncnBrqsTl1z3uOiG3Av9EK86rXVaHHPWfm85m5MTO3ZOYWWuMGV2Tm3HiaW4oif7b30OqdExEbaZVgnhhhG8tW5J5/BlwKEBFvoBXoh0faytGaBT7Qnu1yEfB8Zj4z1DeOeyS4xyjx5bR6Jj8GPtX+7CZaf6Gh9QP/b+AQ8D/A68fd5hHc87eB/wMeaf+aHXebV/ueu869j4rPcin4cw5apaYDwGPA1eNu8wju+TzgAVozYB4B3jnuNg95v18HngGO0foX13XAh4EPd/yMb2v/93isjD/XPvovSTUxySUXSVIfDHRJqgkDXZJqwkCXpJow0CWpJgx0SaoJA12SauL/AVMzwdoIAnn6AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import torch\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "from pymoo.factory import get_problem\n",
    "from pymoo.util.plotting import plot\n",
    "from pymoo.util.dominator import Dominator\n",
    "\n",
    "\n",
    "problem = get_problem(\"zdt1\", n_var = 5)\n",
    "plot(problem.pareto_front())\n",
    "\n",
    "dim = problem.n_var\n",
    "num_obj = problem.n_obj\n",
    "num_constr = problem.n_constr"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We define a wrapper for our BO algorithm, the wrapper function takes in a `pd.DataFrame` and output an `np.ndarray` where left columns are objectives and constraint values are in right columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def obj(param : pd.DataFrame) -> np.ndarray:\n",
    "    names = ['x' + str(i) for i in range(problem.n_var)]\n",
    "    x   = param[names].values\n",
    "    out = {}\n",
    "    problem._evaluate(x,out)\n",
    "    o = out['F'].reshape(x.shape[0], num_obj)\n",
    "    if num_constr > 0:\n",
    "        c = out['G'].reshape(x.shape[0], num_constr)\n",
    "    else:\n",
    "        c = np.zeros((x.shape[0],0))\n",
    "    return np.hstack([o,c])\n",
    "\n",
    "def extract_pf(points : np.ndarray) -> np.ndarray:\n",
    "    dom_matrix = Dominator().calc_domination_matrix(points,None)\n",
    "    is_optimal = (dom_matrix >= 0).all(axis = 1)\n",
    "    return points[is_optimal]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Design space"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>x0</th>\n",
       "      <th>x1</th>\n",
       "      <th>x2</th>\n",
       "      <th>x3</th>\n",
       "      <th>x4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.248171</td>\n",
       "      <td>0.632608</td>\n",
       "      <td>0.240098</td>\n",
       "      <td>0.490050</td>\n",
       "      <td>0.732430</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.931607</td>\n",
       "      <td>0.034736</td>\n",
       "      <td>0.701038</td>\n",
       "      <td>0.516294</td>\n",
       "      <td>0.121251</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.645922</td>\n",
       "      <td>0.065483</td>\n",
       "      <td>0.469201</td>\n",
       "      <td>0.127420</td>\n",
       "      <td>0.123271</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.160319</td>\n",
       "      <td>0.797957</td>\n",
       "      <td>0.901810</td>\n",
       "      <td>0.912018</td>\n",
       "      <td>0.235120</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.188969</td>\n",
       "      <td>0.337747</td>\n",
       "      <td>0.181045</td>\n",
       "      <td>0.068357</td>\n",
       "      <td>0.539323</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         x0        x1        x2        x3        x4\n",
       "0  0.248171  0.632608  0.240098  0.490050  0.732430\n",
       "1  0.931607  0.034736  0.701038  0.516294  0.121251\n",
       "2  0.645922  0.065483  0.469201  0.127420  0.123271\n",
       "3  0.160319  0.797957  0.901810  0.912018  0.235120\n",
       "4  0.188969  0.337747  0.181045  0.068357  0.539323"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from hebo.design_space.design_space import DesignSpace\n",
    "\n",
    "lb,ub  = problem.bounds()\n",
    "params = [{'name' : 'x' + str(i), 'type' : 'num', 'lb' : lb[i], 'ub' : ub[i]} for i in range(dim)]\n",
    "space  = DesignSpace().parse(params)\n",
    "space.sample(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bayesian optimisation\n",
    "\n",
    "The `GeneralBO` class can be used to perform multi-objective bayesian optimisation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "from hebo.optimizers.general import GeneralBO\n",
    "conf = {}\n",
    "conf['num_hiddens'] = 64\n",
    "conf['num_layers'] = 2\n",
    "conf['output_noise'] = False\n",
    "conf['rand_prior'] = True\n",
    "conf['verbose'] = False\n",
    "conf['l1'] = 3e-3\n",
    "conf['lr'] = 3e-2\n",
    "conf['num_epochs'] = 100\n",
    "opt = GeneralBO(space, num_obj, num_constr, model_conf = conf)\n",
    "for i in range(50):\n",
    "    rec = opt.suggest(n_suggestions=4)\n",
    "    opt.observe(rec, obj(rec))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll plot the Pareto front given by BO algorithm, and compare it with random search, we can see that BO gives much better PF than random search"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x29010f9d308>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAEICAYAAAB25L6yAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAkXUlEQVR4nO3de5xVdb3/8dfHGWREERQ4HRIQLRXxCg4omh4ntZ8gWto5aP5ELXNqLC8ni7JzJg3qICT+ypOhni6mloqZHtPqpDJeMC8zHLEDeck8kKDmgKGYkICf3x/ftZg1m7337Jl9WzPzfj4e85i91l577e/al/f+7s/67rXM3RERkfTaodoNEBGR/BTUIiIpp6AWEUk5BbWISMopqEVEUk5BLSKScgrqEjCzG83sG1W6bzOzH5nZX8zsqWq0IY3M7Dozay7Tut3MPliG9a4ws2NLvd4s97OfmS0zsw1mdlG570+K1yeD2sxWmtnrZrZzYt6nzeyhKjarXD4EnACMcvfJmVea2blmttXM3jazt6I36PRSN8LMxkYBVlvqdRdw3+ea2ZLkPHf/rLvPqXRbiuHuB7j7Q4UsG73Gj+/hXc0CWtx9sLtf08N19Fj0Ovlr9Jpca2a3mtnQxPVmZl8ysz+Y2UYz+5OZzTWzgd24j0PN7FEze9PMVic/tM3sCDO738zeMLN2M7vDzEaWeDNLqk8GdaQGuLjajeguM6vp5k32BFa6+1/zLPO4u+8CDAV+ACwys9262a6KB7CUzZ7AilxX9uA12BOHRK/JvYHdgCsS110DNAJnA4OBqcBxwKJurP+nwCPA7sA/ABeY2SnRdbsBNwBjCY/FBuBHPdyOynD3PvcHrAS+ArwBDI3mfRp4KLo8FnCgNnGbh4BPR5fPBR4D/h+wHngJODKa/zLwOnBO4rY3AtcB9xOe9IeBPRPXj4uuewN4HpiRcduFwC+BvwLHZ9me9wP3RLd/ETg/mn8esAnYCrwNfD3Lbc8FliSmd462vR74JPBs1OaXgM8kljsWWA18GXgNuJnwwf4V4I/AOsIbZ/do+T9F6307+psSLf+vwKroMbsJGJLneTs/2r43ou19f+I6By6K2rkW+Fa0/v0zHoP1icf1GxnbMitqx6vAx4BpwAvR/X01cV+Tgcej5/5V4LvAjhlt+WCObXgImAs8BbwF/Gf8GEXXn0IIyfXRsvtnvG6Pjy5fET2+N0XPzwqgPrruZuA9YGO0zbOAOuCW6HlZD7QC78vSvsXRY7Upuu2+ZHkNRo/rQ9G6VgCnZLxmvwf8KlrHY8DfA98G/gI8B0zI8zx3evyAC4DfRJf3ido3OeM2o4G/AR8uMAPeAcYnpu8ALsux7ERgQ7VzK+/2VLsBZdmo6AUP/DzxZu1uUG8hBFkN8A1CEF0LDAQ+Er15dkm8cDcAx0TXf4coHAnB+HK0rlpgAiFoxidu+yZwFCF46rJszyPRG6MOOBRoj1+wZARxltuem2hLLeFbxgZgCHAS8AHACL2Od4CJ0bLHRo/BvGibdopu+wQwKpp3PXBrnsf0U4Tg3RvYJXo+bs7Rzg9Hj8vEaN3/DjySuN6BFkIPaQwhYD+d6zFg+6DeAnwNGED4QGgn9LoGAwcQQm+vaPnDgCOix2ss4cPskoy25AvqNcCB0XN/J3BLdN2+hCA8IWrHrOjx2TH5uo0uX0EI02mE1+Bc4InM13hi+jPAL4BB0fKHAbvmaeOnMx6r5GtwcNSurwI7Rs/NBmC/xPJro/uoI4T//xJ6wPH7pSXPa3Lb40fo3f4GmB1NfxZYleN2DwNzo8vfA76X5z7+Dbgyepz3I3xQT8qx7CXJxzaNf1VvQFk2qiOoD4xegCPoflD/IXHdQdHy70vMWwccmnjh3pa4bhdCr2A0cDrwaEb7rgcuT9z2pjzbMjpa1+DEvLnAjYm2dhXUWwg9o7WEoN2u1x4tezdwcXT5WOBdEh8chMA6LjE9EthMR6BlPqYPAhckpveLl89y3z8A5mc8hpuBsdG0Aycmrr8AeDDXY8D2Qb0RqImmB0frOzyx/FLgYzkel0uAuxLTXQX1lYnp8dHjWAM0A4sS1+1ACPVjk6/b6PIVwAMZ69mY+RpPTH8K+C1wcAHvj4fYPqhvSkwfTfgWtUNi3q3AFYnl/yNx3YXAsxnvl/V57t8J3zbWE17bzwF7RNf9KzlCE7gteb9dbOORhA+bLdH9bfdtM1ruYMI3qqMLWW+1/vpyjRp3Xw7cS/i63l1/TlzeGK0vc94uiemXE/f7NuHJfz+hBna4ma2P/4D/S/iquN1ts3g/8Ia7b0jMWwXsUfim8IS7D3X34e5+hLs/AGBmU83siWinynpC72144nbt7r4pMb0ncFdiO54lvNHel6ftqzLaXZtj+U7LRo/huoztTD5Oq6LbFGqdu2+NLm+M/md9Ps1sXzO718xeM7O3CL2z5OPSlcx2Dohun7mN70XL5nouX0tcfgeoy7Ov4Gbgv4DbzOwVM5tvZgN62Ob3Ay9H7UtuR7KdmY9dvvdGNhPdfSihR74QeNTM6gidiVw79kZG1+dlZrsDvwZmR+sfDfwfM7sgY7kPEso3F7v7o12tt5r6dFBHLid81U2+yOIdb4MS85LB2ROj4wtmtgvhK/orhDfAw1FQxn+7uHtT4raeZ72vALub2eDEvDGEnliPRXvQ7wSuInxTGEqoUVqedr0MTM3Yljp3X5NjG14hhHuy3Vvo/KbOumw0YmcYnbdzdOLymOg22dpZrIWEXt4+7r4roQRg+W/SSWY7NxMCJnMbLVq2J89lp212983u/nV3H0/oTU4nlCJ6sr5XgNFmlsyHol9zWe/UfTPwfWAvwjfgxdF9dxrBZGajCeWoBwtY7d7AVne/yd23uPtqQm98WmJ9ewIPAHPc/eaSbEwZ9fmgdvcXgdsJO6Liee2EF91ZZlZjZp8i1GqLMc3MPmRmOwJzCL3Ylwk9+n3NbKaZDYj+JpnZ/gW2/2XCV9q5ZlZnZgcTdiLeUmR7dyTUgtuBLWY2lVB7z+c64JvRixwzG2FmH42uayfs4No7sfytwD+b2V7Rh9e/Abe7+5Ys674V+GQ0rGpgtOyT7r4yscyXzGy36E17MeF5hRD8o6LHvhQGE76av21m44CmLpbPdJaZjTezQYRe3c+i3vwi4CQzOy7q7V5K2EH22x608c8kHmszazCzg6IRG28RPhzey3XjLjxJ6MHPil6vxwInE8KupKL2fpLQC3/J3V8gvM5+Eg2jqzGzAwidigfib4NdeCGs2s40sx3M7O8JJcjfRfe5B+ED4bvufl2pt6kc+nxQR2YTduwknQ98ifD1+gB69mZJ+imh9/4GYSfLWQBRyeIjwBmEnsprdOygK9QnCDXgV4C7CPXtQl6wOUXtuogQHn8BziSMtMjnO9EyvzGzDYR69+HR+t4Bvgk8FpVGjgB+SPhK/ghhZ9MmQj0zW3seINRw7ySMtPgA4TFL+k9CLXkZcB+hrg3hTbcCeM3MuvxqXIAvEh6PDcB/0PGBUKibCXXc1whfvS8CcPfnCa+Lfyf0sE8GTnb3d3vQxrnAv0aP9RcJ3wh/RgjpZwk73nrUU4zaczJhWNxawo67s939uZ6sL4dnzOxtwmvvHOBUd38juu7zhF72LYRRJb8m1NU/Ht/Ywg+asoasu78FnAb8c7T+ZcBywk5OCPur9gauiMZyvx21JbUsKqiLpJqZOaEU8WK125KPhR9V3eLu3692W6Tv6C89ahGRXktBLSKScip9iIiknHrUIiIpV5YD7QwfPtzHjh1bjlWLiPRJS5cuXevuI7JdV5agHjt2LG1tbeVYtYhIn2Rmq3Jdp9KHiEjKKahFRFJOQS0iknI6a4eIVNXmzZtZvXo1mzZt6nrhPqCuro5Ro0YxYEDhBzdUUItIVa1evZrBgwczduxYwgEF+y53Z926daxevZq99tqr4NulqvQxfz60tHSe19IS5otI37Rp0yaGDRvW50MawMwYNmxYt789pCqoJ02CGTM6wrqlJUxPmlTddolIefWHkI71ZFtTVfpoaIBFi0I4NzXBwoVhuqGh2i0TEameVPWoIYRyUxPMmRP+K6RFpNxqamo49NBDOeSQQ5g4cSK//W3H4emXLFnC5MmTGTduHOPGjeOGG26oePtSF9QtLaEn3dwc/mfWrEWkHyvTjqyddtqJZcuW8cwzzzB37lwuu+wyAF577TXOPPNMrrvuOp577jmWLFnC9ddfz3333VfU/XVXqoI6rkkvWgSzZ3eUQRTWIgJUZEfWW2+9xW677QbAtddey7nnnsvEiRMBGD58OPPnz+fKK68s2f0VIlU16tbWzjXpuGbd2qoSiIhQth1ZGzdu5NBDD2XTpk28+uqrLF68GIAVK1ZwzjnndFq2vr6eFStWFHV/3ZWqoJ41a/t5DQ0KaRFJSO7Iam4uSUDEpQ+Axx9/nLPPPpvly5cXvd5SSVXpQ0SkS2XekTVlyhTWrl1Le3s748ePZ+nSpZ2uX7p0KQcccEBJ77MrCmoR6T0qsCPrueeeY+vWrQwbNozPfe5z3Hjjjdt62+vWrePLX/4ys7J9/S+jVJU+RETyKtOOrLhGDeFn3j/+8Y+pqalh5MiR3HLLLZx//vls2LABd+eSSy7h5JNPLsHGFE5BLSK9R5l2ZG3dujXndccccwytra1Frb9YKn2IiKScglpEJOUU1CIiKaegFhFJOQW1iEjKKahFRFJOQS0i/V58mNMDDzyQk08+mfXr15dkvTfeeCOf//zni16PglpEeo1yna4vPtbH8uXL2X333bn22muLW2GJKahFpNeoxOn6pkyZwpo1awB46qmnmDJlChMmTODII4/k+eefB0JP+bTTTuPEE09kn3326fST8h/96Efsu+++TJ48mccee6wkbdIvE0Wk1yj36fq2bt3Kgw8+yHnnnQfAuHHjePTRR6mtreWBBx7gq1/9KnfeeScAy5Yt4+mnn2bgwIHst99+XHjhhdTW1nL55ZezdOlShgwZQkNDAxMmTCi6XQUFtZmtBDYAW4Et7l5f9D2LiPRAGY5yuu1YH2vWrGH//ffnhBNOAODNN9/knHPO4Q9/+ANmxubNm7fd5rjjjmPIkCEAjB8/nlWrVrF27VqOPfZYRowYAcDpp5/OCy+8UHT7ulP6aHD3QxXSIlJN5TjKaVyjXrVqFe6+rUbd3NxMQ0MDy5cv5xe/+AWbNm3adpuBAwduu1xTU8OWLVuKb0gOqlGLSK9R7qOcDho0iGuuuYYFCxawZcsW3nzzTfbYYw8g1KW7cvjhh/Pwww+zbt06Nm/ezB133FGSdhUa1A78xsyWmlljSe5ZRKSb8h3ltFQmTJjAwQcfzK233sqsWbO47LLLmDBhQkE95pEjR3LFFVcwZcoUjjrqKPbff/+StMncveuFzPZw9zVm9nfA/cCF7v5IxjKNQCPAmDFjDlu1alVJGigifduzzz5bskDrLbJts5ktzVVaLqhH7e5rov+vA3cBk7Msc4O717t7fVxIFxGR4nUZ1Ga2s5kNji8DHwHSc9ZHEZE+rpDhee8D7jKzePmfuvuvy9oqEelX3J0oY/q8QsrNmboMand/CTikJw0SEelKXV0d69atY9iwYX0+rN2ddevWUVdX163b6ZeJIlJVo0aNYvXq1bS3t1e7KRVRV1fHqFGjunUbBbWIVNWAAQPYa6+9qt2MVNMPXkREUk5BLSKScgpqEZGUU1CLiKScglpEJOUU1CIiKaegFhFJOQW1iEjKKahFRFJOQS0iknIKahGRlFNQi4ikXDqCev787c9O2dIS5ouI9HPpCOpJkzqfSjg+1fCkSdVtl4hICqTjMKfxqYRnzICmJli4sPOphkVE+rF09KgBGhqYf8hPaJnzaAjrKKRVARGR/i49Qd3SwqTW7zFj0L20fOd30NKiCoiICGkJ6iiRG+6+mEX37swMW8TXTlrKjFPfVQVERPq9dAR1a+u2mnRDAzRdtCNzNn6Rpvo2hbSI9HvpCOpZszrVpBcuhOZmWPjMkduN2hMR6W/SEdSRuCa9aBHMnt0xEERhLSL9WaqCOlEBATpG7bW2VrddIiLVZO5e8pXW19d7W1tbydcrItJXmdlSd6/Pdl2qetQiIrI9BbWISMoVHNRmVmNmT5vZveVskIiIdNadHvXFwLPlaoiIiGRXUFCb2SjgJOD75W2OiIhkKrRH/W1gFvBergXMrNHM2sysrb29vRRtExERCghqM5sOvO7uS/Mt5+43uHu9u9ePGDGiZA0UEenvCulRHwWcYmYrgduAD5vZLWVtlYiIbNNlULv7Ze4+yt3HAmcAi939rLK3TEREAI2jFhFJvW6disvdHwIeKktLREQkK/WoRURSTkEtIpJyCmoRkZRTUIuIpJyCWkQk5RTUIiIpp6AWEUk5BbWISMopqEVEUk5BLSKScgpqEZGUU1CLiKRcKoN6/nxoaek8r6UlzBcR6W9SGdSTJsGMGR1h3dISpidNqm67RESqoVuHOa2UhgZYtCiEc1MTLFwYphsaqt0yEZHKS2WPGkIoNzXBnDnhv0JaRPqr1AZ1S0voSTc3h/+ZNWsRkf4ilUEd16QXLYLZszvKIAprEemPUhnUra2da9Jxzbq1tbrtEhGpBnP3kq+0vr7e29raSr5eEZG+ysyWunt9tutS2aMWEZEOCmoRkZRTUIuIpJyCWkQk5RTUIiIpp6AWEUm5LoPazOrM7Ckze8bMVpjZ1yvRMBERCQo5KNPfgA+7+9tmNgBYYma/cvcnytw2ERGhgKD28IuYt6PJAdFf6X8lIyIiWRVUozazGjNbBrwO3O/uT2ZZptHM2sysrb29vcTNFBHpvwoKanff6u6HAqOAyWZ2YJZlbnD3enevHzFiRImbKSLSf3Vr1Ie7rwdagBPL0hoREdlOIaM+RpjZ0OjyTsAJwHNlbpeIiEQKGfUxEvixmdUQgn2Ru99b3maJiEiskFEfvwMmVKAtIiKSRb/5ZeL8+dufIaalJcwXEUmzfhPUkyZ1Pp1XfLqvSZOq2y4Rka4UUqPuE+LTec2YEc5qvnBh59N9iYikVb/pUUMI5aYmmDMn/FdIi0hv0K+CuqUl9KSbm8N/ndVcRHqDfhPUcU160SKYPbujDKKwFpG06zdB3drauSYd16xbW6vbLhGRrlg4OF5p1dfXe1tbW8nXKyLSV5nZUnevz3Zdv+lRi4j0VgpqEZGUU1CLiKScglpEJOUU1CIiKdfvg1oHaxKRtOv3Qa2DNYlI2vWbgzLlooM1iUja9fseNehgTSKSbgpqdLAmEUm3fh/UOliTiKRdvw9qHaxJRNJOB2USEUkBHZRJRKQXU1CLiKScglpEJOUU1CIiKaegFhFJuS6D2sxGm1mLmf3ezFaY2cWVaJiIiASFHOtjC3Cpu/+3mQ0GlprZ/e7++zK3TUREKKBH7e6vuvt/R5c3AM8Ce5S7YSIiEnSrRm1mY4EJwJNZrms0szYza2tvby9R80REpOCgNrNdgDuBS9z9rczr3f0Gd6939/oRI0aUso29ik5EICKlVlBQm9kAQkj/xN1/Xt4m9W46EYGIlFqXOxPNzIAfAM+6+9Xlb1LvphMRiEipFdKjPgqYCXzYzJZFf9PK3K5eTSciEJFS6rJH7e5LAKtAW/qMzBMRNDQorEWk5/TLxBLTiQhEpNQU1CWmExGISKnpxAEiIimQ/hMHaPCxiEhO6QhqDT4WEcmpkIMylZ8GH4uI5JSOHjVo8LGISA7pCerMwccazyYiAqQlqDX4WEQkp3QEtQYfi4jkpHHUIiIpkP5x1CIikpOCWqSE9NstKQcFtUgJ6bdbUg7pCGp1Q6SPSP5262tf6xjMpJ8FSDHSEdTqhkgfot9uSamlI6jVDZE+RL/dklJLR1CDuiHSJ+i3W1IO6Tgo0/z5UFvbuRsydChs2QKzZlW7dSIFy/fbLfU9pKfSEdS1tfDFL8JVV8EXvhBCOp4W6UWy9St0zkwpVjqCesuWEMpz58L69aFHfdVVYb6I9Brz54cxAMkPppaW8I1CX457Lh1BHT+D69eHGnVzc+hZi0ivEg/giss/yZq99Fx6diZqV7lIr5drAFdrq34qUYx0BLV2lYv0GdkGcOmnEsVJR1DrMKcifUa2L8f6qURx0lWjTtKucpFeJ/nlOH4LJ6fjnnZzs97e3dFlj9rMfmhmr5vZ8ko0SER6r3xfjrUbqucK6VHfCHwXuKm8TRGR3i7Xl2PI39OW/LrsUbv7I8AbFWiLiPRR2g1VnIJOxWVmY4F73f3APMs0Ao0AY8aMOWzVqlWlaqOISJ9XkVNxufsN7l7v7vUjRowobmU6PrWIyDbpGJ6XSYMuRUS2ScfwvEzJQZdNTWEXsfY6iEg/1WVQm9mtwLHAcDNbDVzu7j8od8NyDrqMj/ryrW9BTU3HMUFaW8NR+G6/HT7+cR0BRkT6jEJGfXzC3Ue6+wB3H1WRkIbcgy7jssjYsXDffTB1Kpx6KqxcGQ6N+vTTIbAz16X6toj0UumsUec79kdcFrnjDjj+ePjb32DDhhDmO+0EV14ZDpeq+raI9BHpDOquBl3GZZH774fRo+G998L8Sy8NpZDMgwqcdlrHuuMRJcletnrcIpJiBY2j7q76+npva2sr+Xq3iXvJEyaEsN5hhxDWgwbBvfeGIP/a1zrXt5MHxf3Yx+Ddd+Gb3wzrSF6nI5yLSBVUZBx1xcQh/U//BA88AAMHwuDBoYe9cSNMnw5XX925vg2dyydmIdybm0NoxyEdl0imTQvrSLr66jBfRKTCel9Qx2WRlSvhpJPgV7+Cu+4KOxevugrGjAm96cz6NnSMIrnoolAmeeedcLqvzEN+HX982DEZh/XVV4fp44+v1laLSH/m7iX/O+yww7xq5s1zX7y487zFi90bG92HD3dvbnYfMsR9113D5Z12codwOWnBAncz96OPDv8XLKjYJohI/wO0eY5M7X096q7MmpX9hzE//3lHj9k9lD+GDoUddwyjRa65pvPP1r/wBfjQh+DRR8N/ncNRRKqk7wV1NslRJK2tcPfdoTzS3BzKJvfdB6ef3vln61dfDUuWwNFHh/+ZNetSyHVMk2nTdKwTEemQq6tdzF9VSx+FylUimTevo+wRlzsyp0tl8eJQjonbEU8vWOC+887u06eHecn5jY2hjSLSp5Cn9NF/gzqfqVO3D+UFC8L8UotDuLm5c2jHHw477hhq6gsWdNTWMz9gssn3QSQiqaOgTrvm5uw7NGfODPNra90HDSo8pN3Dcjvv3PGBk+yVz5sXPnTiHntswQL3yZMV5iJVoKBOs1w96nj+0UeHpylbkHcl7pXPnNkR0vF9xNcNGtQxDWFeU1Pn9k2fXp5vEyKyjYI6rfLVqOP/Q4aE8gd0hGp3xL3yo4/ufF/uHWFdWxuWGTjQva6uox4f98qT9fl580KdPLmexYtDT7yxcfvtU+9cpCAK6rTKVUeOa+TJmvSCBSFEu1v+SPbKZ87cfpnMHnscznHJJXMn6uLFoQ1DhnTs6Nx113CbZNsaG8N0MtQbGzum583r2HGbDHOFu/RTCureKFfPtdBRH5m985kztw/dzB513GOPa+ZxTzzbunfdNSy/006dQzsu48RhnvntIBnoydE0cWgPGOB+xBGhh97UFD60mprC9Lhx4X8yzBXs0kcoqPujOPgyR5Ika9KZNWqzjvJHrh51LBnmydp5csdoHNzxh8TMmR0BHtfkMz9ITjkl/K+pCes54IDwP57eYYdwOR6qOGRIR5DH31Di3rpGvUgvoqDur/KFVbZRH01NIQghd406XkdXPer4AyIO7rjE0ty8/SiXeDre6XnCCR0fAvEOTgjz4w+RAQNC2+LpeIRLfL+56v/J0lLy8YmHXyrMpUoU1FKYefNCzzQzlJOjPgqtUSeXK6RHHU/HOz9Hj+4c1vF0c3NHkMcBngz5zFJRvjHqTU1hflNT5+nMD7dkTz25fcn5IkVSUEvpFDrqY/HiEMjTpxdWo3bvmD7ssNw96jjok0Ge7K1nyjVGPb6vgw7qWHe2kI63JXM0TuZwx9jUqaGWnvywmz49tHfq1I5wV51dMuQL6t554gBJv/gkxK2t4X9DA3zmM+G6M87oOFtPbW041Gx8Lswjj4R77gknLt66FQ44AFas6Jiuqwu327Sp44QRADNnhkPeJs8MFB++NteZ7I85Jhx0a/RoePnlcOyX2bOzb0+8rqlT4ZZb4Kyztr8/CMeEufTScHnBgvA/nm5qCsdRP/XU8BFz991hfnyI3dZW+OMfO9Z1xhnh/223dcz7wAfC//gxTbZPJ73o1fKdOEA9akmHuKcZ185zjfoYM6ajRh0PWexujdq9ez3qWLZ6ezbxj4eSf3FZJTkiJtuPnOIy0qBB4X/8N2jQ9vsCcm2bdqT2Sqj0IX1GcidoHEjJn74XMuqjuzXq+LaZI1jyBXtyfHo8xDFZhslVkonDeuDAjtvHx3sppP6evC5bkGc+LvHjEe9ITT5OCveKUlCLJHV31Ed3atTxunrSo44lhz7mO3xArrBPtjnXoQly7VjtatsKlfyGlDzeTPKDQTpRUIsUozujPpIhvWBB5+mmpo6drPEO1Wy932J71LF8vfbkbTPHsnf1baEQ8X1kfluJp7ONze/nJRsFtUilFDPqo1Q16uS8rnrtmWPZu6q/d0fchng45UEH5T+ue67tSo4yyvb4xYcmSGps7PgVa3L9KT6eu4JapDeIhz4mj4kSh0v8V8ivLrsK8kr0qGPxB0A8nDLbIQkytyPzAybZ/mzj+DOP055tbH+8XF1d5w+KFNXoiw5q4ETgeeBF4CtdLa+gFqmifEFeqRp1fJ/d6VHHspVskgGePDl1ZphnzhsyJPx6Nj6ee7bDKnR3+zNLYcmDixUR8kUFNVAD/BHYG9gReAYYn+82CmqRlKrUqI+e1KiTt8tWsulq1Ey+ecn5xX6jyNy5nLmNPfyQKzaopwD/lZi+DLgs320U1CL9XE9GfeQr2ZSqRx2vu9gafeZwzRNOKLpsVGxQ/yPw/cT0TOC7WZZrBNqAtjFjxnS7kSLSz+Uq2TQ2lq5GnSx/FFujzwz3InfEViSok3/qUYtIySQDvNhRHyed1PlMST2t0aewR63Sh4j0DaWo0VehRt3lQZnMrBZ4ATgOWAO0Ame6+4pct9FBmUSkz8o84Fhra+eDiyXnd+MgWfkOylTQ0fPMbBrwbcIIkB+6+zfzLa+gFhHpnnxBXVvICtz9l8AvS9oqEREpyA7VboCIiOSnoBYRSTkFtYhIyimoRURSriznTDSzdmBVD28+HFhbwub0Btrmvq+/bS9om7trT3cfke2KsgR1McysLdcQlb5K29z39bftBW1zKan0ISKScgpqEZGUS2NQ31DtBlSBtrnv62/bC9rmkkldjVpERDpLY49aREQSFNQiIilXtaA2sxPN7Hkze9HMvpLl+oFmdnt0/ZNmNrYKzSyZArb3C2b2ezP7nZk9aGZ7VqOdpdTVNieW+7iZuZn1+qFchWyzmc2InusVZvbTSrex1Ap4bY8xsxYzezp6fU+rRjtLxcx+aGavm9nyHNebmV0TPR6/M7OJRd9prgNVl/OPAk6YC1wAXBddPgO4vRptreD2NgCDostNvXl7C93maLnBwCPAE0B9tdtdged5H+BpYLdo+u+q3e4KbPMNQFN0eTywstrtLnKbjwEmAstzXD8N+BVgwBHAk8XeZ7V61JOBF939JXd/F7gN+GjGMh8Ffhxd/hlwnJlZBdtYSl1ur7u3uPs70eQTwKgKt7HUCnmOAeYA84BNlWxcmRSyzecD17r7XwDc/fUKt7HUCtlmB3aNLg8BXqlg+0rO3R8B3sizyEeBmzx4AhhqZiOLuc9qBfUewMuJ6dXRvKzLuPsW4E1gWEVaV3qFbG/SeYRP5N6sy22OvhKOdvf7KtmwMirked4X2NfMHjOzJ8zsxIq1rjwK2eYrgLPMbDXhuPYXVqZpVdPd93uXCjpxgFSOmZ0F1AP/UO22lJOZ7QBcDZxb5aZUWi2h/HEs4VvTI2Z2kLuvr2ajyuwTwI3uvsDMpgA3m9mB7v5etRvWW1SrR70GGJ2YHhXNy7pMdN7GIcC6irSu9ArZXszseOBfgFPc/W8Valu5dLXNg4EDgYfMbCWhlndPL9+hWMjzvBq4x903u/v/Es5Huk+F2lcOhWzzecAiAHd/HKgjHLyoryro/d4d1QrqVmAfM9vLzHYk7Cy8J2OZe4Bzosv/CCz2qFLfC3W5vWY2AbieENK9vW4JXWyzu7/p7sPdfay7jyXU5U9x9958ss1CXtd3E3rTmNlwQinkpQq2sdQK2eY/EU6OjZntTwjq9oq2srLuAc6ORn8cAbzp7q8WtcYq7jmdRuhN/BH4l2jebMKbFcKTeQfwIvAUsHe19/aWeXsfAP4MLIv+7ql2m8u9zRnLPkQvH/VR4PNshJLP74H/Ac6odpsrsM3jgccII0KWAR+pdpuL3N5bgVeBzYRvSOcBnwU+m3iOr40ej/8pxetaPyEXEUk5/TJRRCTlFNQiIimnoBYRSTkFtYhIyimoRURSTkEtIpJyCmoRkZT7/2oHwHoMb1hKAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "feasible_y = extract_pf(opt.y)\n",
    "rand_y     = extract_pf(obj(space.sample(opt.y.shape[0])))\n",
    "plt.plot(feasible_y[:,0], feasible_y[:,1], 'x', color = 'r', label = 'BO')\n",
    "plt.plot(rand_y[:,0], rand_y[:,1], 'x', color = 'b', label = 'Rand')\n",
    "plt.title('Number of Pareto optimal points from BO: %d' % feasible_y.shape[0])\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model accuracy checking"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x29010eeb488>]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW0AAAD4CAYAAAAn3bdmAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAABKJklEQVR4nO3deXzU5bX48c/znZmskBAmMTAhGJMIUdJgZAlGUIQWtUgr/tpaEVpckNJ6acvtpra9t71V22ppy7UFd1ogtrWCvUpVWhAE0bBFIkuATAyEJGASMCvJbM/vj5n5ZiEJQRKycN6vly+ZbyYzz6CcPJzvec5RWmuEEEL0D0ZvL0AIIUTXSdAWQoh+RIK2EEL0IxK0hRCiH5GgLYQQ/Yi1J140NjZWJyUl9cRLCyHEgLR79+5KrXXcuZ7XI0E7KSmJXbt29cRLCyHEgKSUOtqV50l6RAgh+hEJ2kII0Y9I0BZCiH5EgrYQQvQjErSFEKIfkaAthBAXYMUWJ9udla2ubXdWsmKLs0feT4K2EEJcgIwR0TyYk2cG7u3OSh7MySNjRHSPvF+P1GkLIcSlIjsllqfmZPJgTh5zs0ayOvcYT83JJDsltkfeT3baQghxgbJTYpmbNZJlmwqZmzWyxwI2dDFoK6W+q5Tar5Tap5R6SSkV1mMrEkKIfma7s5LVucdYPC2V1bnHzspxd6dzBm2lVAKwGBivtU4HLMBXe2xFQgjRjwRz2E/NyWTJjNFmqqSnAndX0yNWIFwpZQUigLIeWY0QQvQz+cerW+Wwgznu/OPVPfJ+qiszIpVS3wYeBc4AG7TWd7fznAeABwBGjhw57ujRLvU+EUIIASildmutx5/reV1Jj8QAXwSuABxApFJqbtvnaa2f0VqP11qPj4s7Z3dBIYQQn0JX0iOfBT7SWldord3AWiC7Z5clhBC9b8UWJw+tzW+Vn97urOShtfk9dnjmXLoStI8Bk5RSEUopBUwHDvbssoQQovdljIjm9fxyFq7azXZnJdudlSxctZvX88t77PDMuXQ1p/0z4E7AA+QB92utmzp6/vjx47UMQRBCDATBQO32+gCwWQyenjeu22uxuy2nDaC1/i+tdZrWOl1rPa+zgC2EEANBsKdIdkos92Qn0ej20ej2kZEQ3aOHZ85FTkQKIUQ7gj1Fnt3q5MXtxdgsCoA9x0736OGZc5GgLYQQ7chOiWXR1GQeW19Ak9tLmM3CIzPTsFoMM8fdGyRoCyFEB7w+uNoRhcuruSc7iQVTUnh63jhuyxje+vCMqwF+mw6/SevxNUmXPyGE6EDGiGieeafI7CkyKcVOdkps65z25l/B5scu2pokaAshRDta9hTJTollUoq91WNOfAgrJjd/w7j5MOv3Pb4uCdpCiEvGii1OMkZEk3+8mowR/iqQ7c5K83H+8Wq+cWMK0HFPkX3HKsj+581QVdj8wj/4CCKGXpTPIDltIcSA0tH4r/kv7sBiwIM5eea/H1mXz30rd5mPWx6Y+caNKWeV9mV//Dce2DKpOWDPeRn+u/qiBWyQoC2EGGA6Gv91faqd5ZuLWDQ1maUbjuCIDmNNbgnjLh9iXu+wM1/lEfjvaHjrIf/j9P8H//UJjJpxcT5UC5IeEUIMKJ2N/xrj8Af0kfYI9pXVkBgTzrbCKmZnJrB8cxFPzcls/WJeD7wwA0p3N1/7z0MweNjF/VAtyE5bCDHgdDT+KzsllhtHxXHoRC2JMeGUnD5DWvwgXs0rZdHU5NbpkD1/hv+xNwfsL73oT4X0YsAG2WkLIfq54M3F7JRYVmxxcrSqHkPBX3ceZ3amgxe3F7OvrJqJV9gpOVXPurxSJqfaebewinRHFPvKapgcSJ2McUSTbW+A332m+Q1SpsHdr4DRN/a4ErSFEP1aMId985h4kuMieWX3cVxezfS0ON7af5JGt5e3CypQwMaCCu7OSiRxaCSX2yPIyS1hcqqd3Uc/YcnnUhixfi6ceq/5xb/zIQwZ2WufrT0StIUQ/Vowhx3sxOfTEGJRbC2swuXxd+abnGpny+FKHpmZxoIpKWx3VvLMO0U8PDMNrw9+csUhRm+6s/lFv/C/cO3XeukTdU6CthCi3wt24lu2qZAwm0FGQjQ7ik8DMDEphm2FVSyelsqCKW1qsC/zwm9Gma9zcnA68d/ZApa+Gxr7RpJGCCEuwHZnJatzj7F4Wio+n2ZH8WmshsJmUewsPs3szARW5x4zywC/cUMy2Xu+1ypg8+Au4v/z3T4dsEGCthCij+vosExw3FfL4+aDw624vP7BLobyDywIsxn8++BJFk1N5sGcPA688zL8bAjsX+d/sZsf81eFxF55MT/WpyZBWwjRZ7QXoC0G3Ldy11mHZYKnF4OpDoClG44wLS2OR2amcZUjCouhWDJjFLdlDMfaVM0e75e4etP9/he2p8KPP4brvnXxPmA3OOe4MaXUaOCvLS4lAz/VWv+uo++RcWNCiE+j5a45/3g1FgPztOLyzUXcOCqWN/ed5Pn54886Yt6y9K/l6+Ufr+YbtU/Brhean7xwKwzPuFgfq0u6Om7snMkbrfUh4JrAi1qAUmDdhS5QCCHaClaC3LdyF+MuH8K7hVU8HKj42HKognV5ZczOdLQ77ivY6KnV6xkHyX77tuYLUx+CqT/qyY/Q48434z4dcGqtj/bEYoQQl562O+TslFjGXR7DtsJK89DLlkMVbCusMkv3grMbO9RUC0+OAneD/3FkHHw7H0IiLsIn6lnnG7S/CrzU3heUUg8ADwCMHNm3itGFEH1Ly0AdPByzaGoyL+WWcEVsBO8WVpJkj+Ddwiqiwq1mwP7mTam8tresVQqlbUrk+Ms/YMT+p5vf7L5/QeLEXviUPeOcOW3ziUqFAGXAGK31yc6eKzltIURn2uauS07Vk5NbwuX2CIqrGshMjCZteBQv7SgBIDrcii8QqhZPT6Woop7L7ZFmwH9qTibZYUfh2WnNbzLpW3DLxZsoc6G6mtM+n6D9ReBbWutz9iKUoC2EOJdg4L5xVCyv5pVxfaqdbYVVJAUCd9wgGxV1bgAshiLEolBKYTEUT88bZ+6u3z90nCtemkI8pwDwGTaMHzghLLrD9+6Luhq0z6fk7y46SI0IIcT5CnbiW5dXxvWpsbxbWMXEpBiOVjUQNyiEijo3iTFhzM5MwOvTuL0arbU5VHe7sxK2/oZJL40xA/Zj9scxflrZ7wL2+ehSTlspFQl8DljYs8sRQlwKVmxxYjFgde4xZmcm8GpeKWMSothRfJr0hCj2lfp7XR8/fYbKunImJsWwo/g0o4cN4vE7Mvhg17tcs+pa8/XWMZWHvAuxVVmYeq6blP1cl4K21roesPfwWoQQlwiLAY+uLzA77k1Li2NjQQXDokLZV1rD9LQ4LosK49W8Us64fXxYWs3sTAfr845x6olruabeab7Wdd5nqLMM4YX54wBaD98dgOREpBCiW7V3qvGhtfk8tDbffOz1wfS0ONbklrDtSAWbCiqYnhbHx7VNTE+LI6/EP/ZryYxRhNsMRg0bzG+v2MXhsK8xNBiwv5rDnIS3KHcP4p7sJLJTYs067w7Hhg0AErSFEN2qvRmNr+eX83p+uXktY0Q0eSXVTA7cfBwdP4i8kmpW35/F8/Mn8tScTC63R+L1wZo74vhHxUz45/cAqEqcwYqpu9luy6LgRC2Lp6W2agaVnRLb7kGbgaJvt7MSQvQ7bWc0Prv1I65LGcp1KXbz2ovbi0mNi2T30U/MfPXszIRWB2yyr4iBlTPhWIuhBEsOYo9ykNGiZDA7JZZJgdceyGmRINlpCyG6VbD73lXDBrNsUyHJcRFsO1LJE28d5urh/mv1TR7ySqq541oHhRX15s3IZ7cGUh8fvAQ/H9ocsO941t+JL8oBtOiH3SLID/S0SFCX67TPh9RpC3Hp2u6sZOGq3Xi8PjTg82lcXo2hwKdBARrITIzm6KkzZvB9dquTNRveZ7Plm80vljQFvvYPMCy99Gkunp6o0xZCCODcPa4BrBaDW9OH424RsMEfsCenxlJwoo5b0+P9u2OtWVDycOuAvfgDmP/6JRGwz4cEbSHEeWvvZmOwx3X+8WqenjeOe7KTWJdXyoSkGDNgB+05dpo7rnWwdk8ZU33v+4cSHH7T/8WZS/2pkKFXXNwP1U/IjUghxHlre7Nxde4xbh4TD/hbpAbHf01OjWVbYfOOPHhEvcHl5c3cfRwMWwRb/F+ri7maQQ++AxZbb3ykfkN22kKITyV4DH3ZpkLmZo1k1lgHD+bk8exWJw/m5HFrejzvFlaSGhcJgEVBcVUD6cMHs9T2R3aHLTJfK2/Wm6y+ZjXbi1vfSGybchEStIUQXdQ2j/3Q2nye3VrE9Sl2VuceA+CaxGh++cYhbhwVx9o9ZTw8M40JVwxlWlocX5mQyDTbh7x+ehZ3WLYB8Lj7LpZm7yBz3HWdplxEM0mPCCE6HNX1zDtFPHBDsjn6K9j3uqiinld2H8fl1YTaDBZNTTYrRiwK1uWVkp1iZ4wjmuWbi1j+pRSy/poJgXuKJb44bvE+idcIxba9mEkp9nZTLpdC3fX5kpI/IYS5q715TDyzxjp4bW8Zr+eXs3h6Kss2FjIhKYadxacZHGqlvLqRy+0RHK1qYE5WImv3lOL2akKsBpOSh/J+0SkaXF5CrAahVoMNo19j+KFV5nvNdj9KnvcKwm0GS2aMoqiinrf2nzQD9NINh1i2qZDF01JZMmN0L/6uXFxS8ieE6FTLdEdwl/tqXilfe34Hr+aV4vL4KK6sB+A9ZxVNHh9l1Y2APzd9e6aDmRkONODxaRrdXrYVVmExFLMzE8jwHuBDvmIG7GeYzVXev+BzXMMjM9OwWgyWbSxk1liHeTAmeAOz7dF00UzSI0JcojJGRHPfyl0smXElC6b4e3UEA7BPa7JTYlmTW8LkVDs7ik/j8vjMemuLAW/sO8Gb+07g88HszAT+b28ZLo+PKSPD+dnBzxMV6g/4dSqSv0x+i/dLzmAtPs0Pb0kjOyWWMY5oXttb5p+WHugVcqkeTT8fkh4R4hL27FYnj60v4PbMBP598CQerw9PYOAAwOhhgzl0ohZoPsk4OdXO+0Wn8ASKrzMToymsqKfJ7eVB9TcWW9eZr/9MylM8vn8ot2cmsOVwRacBuKO8esugPpB1NT0iO20hLjErtjg5WlXPrLEOFkxJ4UBZLevySgH/WK+vThjB2j3+PtbBgA2gFNgMxY6PTuFtcVpGA6O8Tl6xPWRe+5Pnc3zwmUf47Z2ZHPzrB6zLK2XxtNROd8ztBeZgu1XRTHLaQlxiMkZE83p+OQtX7ebZrU7+fbB5TnfsoBBycku449oEQqzN4SEmwobWMMYRhcuruSwqFIBww8Oyj+/hFUtzwJ7ofY4PPvMIb+47ybNbnWw5XCE56m7UpaCtlBqilPq7UqpAKXVQKXVdTy9MCNEzslNieXreODxeH4+uL+CMywP4TyuerGnimsRoXt5ditvjA/z56/omL9enxpJXUk26I4qTNU08Fv82B0O+xkj1MQD3e39Equslzlii+PL4RJbMuJLH1hewaGoyS2aMNsv5JHBfmK7utH8PvKm1TgPGAgd7bklCiO7S0RSZ1/aWce3IGAA8PkiK9Qfs4NQYl8ffoS/dEcWq+7IwDNhWWMnkVDu64hDFYXOYU/0sAOu815PUuIbN3rH88NbRPD1vHA/m5FFUUc/DM9Pw+mP/JdU+tSedM6etlIoGbgDmA2itXYCrZ5clhOgOwVOGwRuAwSkynkAktRoKn9YUVzaYATsyxEK9y0vcoBDKqhvZX1aNzWKQMNjKj44tJN1SbL7++MblqEFx4HYRYjUY44huFZyDVSlBkqO+cF3ZaV8BVAAvKqXylFLPBaazCyH6uJanDJduOMSDOXlMSIqhyePjjNvHrLHDiQy1YjVgY0EFTW6PGbAr61xkJkbz2PoC/mfETjY2fIl0oxiAP8T9lM/wN9JSk6moczE704HFULy2t8x830uh4qM3dCVoW4FrgeVa60ygHvhR2ycppR5QSu1SSu2qqKjo5mUKIT6NYLOllo2dgiwGrMsr47NXXUaI1X++vN7lIzEmHK+GOVmJHDq0n4/C5nD78Sf833TlzWyfW8jOiCksnp7KgXL/jMYthytZPD2Vy+2yn+tp56zTVkoNA97XWicFHk8BfqS1ntnR90idthC9K1jzDLBw1W4APntVPP/8sJwQq8EXxg4nJ7cEw/BPRrca/tz25FQ7B8prWXRjEmM3zWci+5pf9Dv7YEgi0HzsvWXaRQ7CXJhuO8autT4BlCilgk0ApgMHLnB9Qohu1vKm49Gqehau2s36fP8pxUaXl9f2luLzabw+zf/tLef6VLt5k9Djg7uzEll9/yT+cl0JCzaNNwP2m6k/5VrL33lo02nz9YMzGoPvKzcZL54unYhUSl0DPAeEAEXAPVrr0x09X3baQlx8X/zDNo6crOO5r/s3a/et3MkZt4+YcBu1TW5zJ504NIIthyooq27EUKCUwqLgtitg6fG7zNfbo0fROPd1sq+MN+c+Ajw9b5zsrntAtzaM0lp/oLUer7XO0Frf3lnAFkJcHG3L+W7LGE6Dy8vXX9jB+84qc8TX6TPNAfvdwipKTjWYjZ/umphIRIjBMstvWwXsJ0fnsPuzfyG/rA7w76wXT0/ltozh5k3Nhat2c/OYeAnYF5kcYxeinwnmq1s2fBrjiKaoop5wm8EZt49lmwoBzAZPNovimzelEjc4lHV5ZSTZI7h70kj2v/1XPuRX5vbtZ56v47j522Q7olm4aje3ZQxnu7OSo1XN7VPjBoWybFMhYTaDWWMdvfg7cWmShlFC9BMtby4G0xLr88tYk1uCzaIIs1n4wtjhrMktMb/HaihmjR3Ovw9+TKPLi9WiuCV9GO/vO8J7lgXm84oYwc2Nj+FVVgylMJS/D0lMhL/07/u3+Ptev55fTqPbi9YQHmIxUyXiwkk/bSEGiBVbnDy0Np8dH1WZeeVFU5P5+gs7eGlHCQpwezUJ0WFmwI4O8/8l2mIovjw+kcXTU3EHuvf9yP3HVgH7W1FPMb3x13iVFZ/2t2Z1ebXZP1trzbKNhSTHRdLo9uL2akYPG2yefJRj6ReXBG0h+rhgg6f3nFV4fZqvPb+DX75RgNurzd7WhoKCk/78891Zibi8mulpcTR5fPzk1X14ffCH62ooDLmL+MK/AfCE+ytkh67lnx8PZU5WImE2S6v39WlIjAnH49N4vD5ezSvD7dXYLIpjpxoApGKkF0h6RIhe1l4f6YfW5gPw+B0ZgL8u+r6VO2ny+GjRFRWbRaEAl7f5YohFsfLeiYC/guSaywxe+mQueP3dJz6xxpJV9yQTUoez++gnZKcMZWthFe5Av5G2kuwRlH1yBpdXt3ptqRzpXpIeEaKfaDuF/KG1+byaV8rr+eXmtf1l1ebOuqXRwwb75zNaFGMcUYA/gL+8q4QHc/J4Jn4dL1V9xQzYr47/E9fULSMiPJLdR0+TNmwQmwoqsClaBWxDNf+6uKrBHHhgBL4gddm9R3baQvQBwZrnuVkjeXF7MQCLp6eyfHMRN46KZV1eGRYF3nb+uKYnROH8uM7MNR8sr2EsR1gX+l/mc/aO/Bpzj92GN5DqaLkzT7JHUFzVYD62GmAYBi6Pz596QeH2aWZnOvj3QX8bVrkB2f1kpy1EP5KdEmv2B7knO4mn541j+eYiUuMiWZdXhsIfsEOtBo/MTMPSYie8r7SGJo8Pi6EorzzFjpBFZsB2qVBy7/yAe0q/wMih4TS4vMzMcGALvIDCv5M2lL/S5O6sREDh8viYnekgyR6JUopHZqYxelgUT88bB2A2hhIXn9RpC9EHtJ1CPinFzo2j4liXV8qImHBOVDei8actXs0rPWvH7dPwgP47/2n83bw21/1jdqp07jhYa+7WJybFsC6vlBCrQdwgGxV1/rTJgzelMinFzsJVu7FZFFlXDGXL4UpuHhPP/9zuaLWrfnreOEmL9CIJ2kL0gpY3Hx9am8/r+eUsnp7Ku4VVLJqazPwXduDyamYHBu7eMCqWncWnaXB52V/mn9sYbjP4TEI0tUc/4I3Q5nFf+fG3c/fJOWCD701P5T1nFW8XVJBkD2dH8Wmshn8nfbrBbaZcVmxxMjjcHw5uz0xg1lgHr+0t4639J886QCM9sXuXpEeEuAhaHjlfscXJjo+quG/lTrNKxOXx8fg/C6hucLF0w2FcXk1qXCRXOwbT4PLynrOKiUkx+FreifS6+GX5va0C9njXM3zh6Fdwe31mKuM95ynmZCVytOoMCn8dts2i8Pg0IVaDzMRoXF7Nr944xOLpqcwa6+DBnDxmjXXIzcY+SIK2EBdBsOvedmclGSOi2XqkkjNuHy/vOk7+8WqzlC+/tAaPV2M14KOqBh5dX8BXJ4zAp/1DCmwWRZI9gnstb3Aw5Gskq3IA3hr7O1JcOVT6Bpnvub+smqUbjrBkxpUkDo3kakeUWSESajGwGoqEIeEUnKhjelocGng1r4z7Vu4yS/lkmEHfI0FbiB62YouT5LhImjw+5j6Xyx/fLsQbSEp7fJr9ZTUA/lSFT+PVGq/P/+sQq4E9MhSPz99DNYlyNtffzk9tqwB43ZvFVd6XOBZ7AxYVuLkY6Dfy6PoCsy/J0ap6jp1qIMxmYCioc3mJibBRWFFPdspQ8kqquS55KPvLarglfZikP/owKfkToocFy/luTY9v1RdEwVmHWRJjwik5fcb8dXl1oz+doby8FvkL0jyHzOdmNf2BCh1DWIiFRrcXn4bMxGj2l9fi8vgIsRpMTrXzXqDj34v3TGB/WTWPri8w3zs4Vuz61FjeLazk9swEthyukEMzvUBK/oToI4IHUf6y43ir6+1tl4IBWwV+rYEvWzZzJHSeGbD/w/UgSY05nLbYeWhmmhmwk+wRfD5juBmw0ZqtRyrxavjezaMAWL65yD9RHYiwGVTUuYgdZGNbYSW3Zzr47Z3XmDMlpadI3yRBW4ge0rbftQ6E6ZY11p1xUIkz5C6esD0DwFZvOsmNq3nNlw34b14+t/UjfBpiImyUVzfyxJuHmJYWh9b+pk9ur2Zyqp0FU1LIP17NrenxvFtYxeRUO2fcPpLsEVTUuUl3RLHlcCXbnZVy2rGPk6AtRA+Y9uRmdn5UZe5Yf/LqPvMIenunGltS+HjR9iu2hy02r033LuPNa5djGM1NnWwGnKxpIibChlKKxJhwXF7N6XoX7nbeJGNENGv3lDEnK5ED5bWBipIGpqfF4ayoZ9HUZHO9cgOy75KcthCfUrDWOv94NUer6s165mfeKaK6wUVeSTWpcZGUnD5Dk8d/I7G9PHZLnzfe548hy8zHv7J9k+W1k7EacLk9kvLqRnxa0+T2mSmOBrfPnEpzeeBIusUAm8VAAVaLYR6ICa43+G9LYLBv2+sSsC++rua0uzojshioBbyA51wvLEFbXAqCNxgXTU1m2cbCQF8PzeAwC1X1bjMgGoAPf8CG9oN2HJ+wM+yb5uO9vmS+5P4ZHm1hTEIU+0prMBREhlqZmBTDxoIK8wdAuiOKfWU1xA0KoaLOZV5fPC2VweFWnnjzMP9vXILZMVD0TV0N2udzIvImrbXcmRAiIJj7vW/lLsZdPoRthVVYDUVVvRsAhSLE0tw2tWWwtgYCukazPOwP3Mp282vTm54gfPhVeMpquNwewf7SGuKjQjlZ08TlQyPYWFDB9LQ4JqXYefyfBewra/568H1mZzrMxlPfv2WUOXVd9H+S0xbiAmSnxJIcF8m2wipzYEBQcAJMkLVFv1OlFDNseykOu9sM2L/yzSO5KYcSYwSHP67j+lQ7xVUNjHFEUXPGw+xMBwfKa8hMjOadI5UUV9YTGWrFovy57eANzhCL4o19J/AEIvUYR7SkOwaQru60NbBBKaWBp7XWz7R9glLqAeABgJEjR3bfCoXoQ1ZscZr56/zj1by8q4SjVfVmiV5HgrlngGjq2Gt7wPzaR754bnb9Gm2EYLP4byierG1iW2EVSfYI9pXVcHdWIo/OzuBqRxSPrS9gWloca3JLmJxq5/2iU6A1SinunjiC4soG3nVWcX2KnW9NSyX/eLXUXA8gXQ3ak7XWpUqpy4B/KaUKtNbvtHxCIJA/A/6cdjevU4het2KLE4sBr+eX83p+OV8YOxxnRX2XvrfB7UMBvw5/kS/rf5nXZzY9xn6dBIA93EpVvZvCwGs6osPM6o41uSX8+8DHVNW7mJOVSOJQ/wnLbYVVZlvV8BALSbGRvLHvpNktEJBd9gDTpfSI1ro08O+PgXXAxJ5clBB9UcaIaJZvLmLx9FQAXtpRco7vaJalDvJR2BwzYP/ecwdJjTkc0ElYAn8Kg7lw8N+0LKtuZFpaHPdNScZiKE7WNpE2bDBv7DtJyal63i2sQuE/sj4p2c7i6ak8tr6ARVOTWTJjtBySGaDOudNWSkUChta6NvDrGcDPe3xlQvSylu1TV2xx8sa+cjITo1m64QifSYhiR/FpwD+nsb26aIDBnOH90G8RqRoBOKUH8/2E1Wws8u+mNRA7qPkmIkCYzaAxkErZVFDB1kL/QN9gqiTdEcWa3BJiwm2cPuNmcuAIeohV8fDMNPOmY8tDMpIeGTi6stOOB7YppfYCO4D1Wus3e3ZZQvSuYCokuFPNGBFNfkk1GwsqiI8KNQM20GHA/oH1L3wYdp8ZsH805AmubXqajUX1WA1/hQfQKmADNAZSKeAP6i6Pj/jBoRRXNRAfFcq+shqiw6yBgG1n9f1ZPDwzjfecp8666SiHZAaec+60tdZFwNiLsBYh+oRgwF6+ucg8JXj18Oa2pi3nKbYnQzn5v9CfmI9f8NzCmphFfHVcImp9QeB1FOvzyzECHfnaanvpZG0TRqBKJNhUKt0RxYHyWrY7K1kwJYUxjmjZVV8C5ESkuOS1TIOA/9DMwlW7uWxwKB/XNpEQHUbByTqS7BEcO9XQbpAFGBbuY61vMQ51CgAvBpmNTzPUHkdMhI28kmrCbQYWQ1HX5AX8NxubPN5W+ezOBA/STE6N5UB5DYumJrN8c5F05RsAeuJwjRADRssj6ME0yDWJ0cRHhfl7X7t9lFc3csblpaCxjsSYsHZ32BE2gzNuH98KeY3v6ZfMY49zXQ+xzfcZ4qP8aY2S0wpD+VMfLWP+GZeX02f8ATsmwsbpho6D9+AwK4dO1jI70D510dRkvD4kb32JkaAtLjkt89XBnerQCBubCiowFGgND89MY837x8xAXVbdnHdWgMXwj+sa4TnKhrAfml972XMDf4r7PvvK/XMcT9Y0+Sep+3S7QTkYsO2RNqrq3VgN/zu0PKQTPJZe2+jhkZlpLJiSYh6hbzlhRlwa5ESkuOQES/eCAfvq4YPN2mifBosB/8grMwN2iNXA2yKIamBUXChvhvyQDaHNAfvr9hwe1ovYV15LZIil1fMTY8I73UVX1bsxFPzw1jRsbXq3aiA1LpKIEAtFgXVK+9RLl+y0xSWj5WnGYA3zZYNC2FZYhc0At4/A4FvYFxgBlpkYbQZ0V6BT31zLv/hF9YvmlucB13cpsk+lsLQe0FgMRb3L2+q9W56WbK/TnwIe+nyaGZQtgRuULW9+/vDW0a16iMgO+9IkO21xycgYEc3r+eXMf2Enz20twhG4wQj+gA1nB9MD5bUsnp6Ky+NjpDpJcdgcfmF7EYAN3nEkNa7BO+rzZmAHWu3K2xoUYmm3y59SUFRRz+X2SK5LsRNiNRgUZmXxtFQiQiwo/F+X8j0hO21xychOieW2jOGs3VPKpoKKLn1Pk8fHr/55gL+G/IIso8C8PqnxfzmBHYAth/2vFcxLt8dQMHJohJlyGRRqMStIwN9y9XJ7JN+4MYWjVfVmD+zslFgmpdhZuGr3p/rMYuCRnba4pMwa6+B8ylxnG1spDJ1rBuzvuhYxzvKyGbDBn06ZnZlATERIh68THAlmMfypkLomL+mOKBT+sr/aRo95nP1ye6QZsMH/w+bpeeO43B553p9XDDyy0xYDStua6xVbnOz4qIr4qDBzCEAn2QvAf4x8iLuC98P+w7yW60vjP8P+BxVuper0mVZd+wBezy/r8GQk+HPjeSXV2AxFRJiB1pj11laL4qa0OF7KLWHBlJR2UyCSvxZBErTFgJIxItoshcs/Xk3JqXreLqggzGaQHBfJk28dxuPTHZ5ENJTmKX7NZ8P2mNduavotH+l4cLkBN5EhlrNuNHYWsAHKqxuxGQpvYFSY26tJd0SxrbCSu7MSeWPfSZ6ak9kdvwVigJOgLQaElpUhN4+JZ+Gq3YwdMYRthZVckxjN/rIaHl3fnJP26ZbTY/xuNnbydMhvzef82H0Pa7yfO+vGYduA3VbLBlIKuCwqlBM1TVgMfwtVt1czJnCyMT0hipzcEh6emSY7adElktMWA0KwMmThqt3+E40eH9sKK7EYikMnaltVdCjlvzFoMQyUAjvVFIfNMQP2Ad/lXMsadsbd0SpgKyBuUMd56+BzFP7pMeD/gXC63kViTDhen3+azahhg9hXVsPoYYPYV1rD7ZkJMg5MdJkEbdGvrdji5KG1+by2t8zsc/2rNw6ZNdVen6bJ42uVCtEaLErR5PHyhHU5u8MWmV/7XNOv+bzrcU41+oN9S9en2qmoc3W6Ho1/JqTFUGYXP5dXU1nXxOxMBz4Nh07UkRY/iMMn6swj6Rkjorvhd0NcCiQ9Ivql+S/u4PpUO0er6vnHB2VorXF5fNisRqsj4NA6dx082DKJvawK+6V5/Zfur7LC+4VO33NbYVWHX2s5WDf4nidrmgi1GjR5fFyXYudqRxTr8sowFBw6WcecQC472ElQmj6JrpCgLfqVaU9u5rqUoVyfauex9QXMyUrE7fHhDkRmr7vzPMNg6vkg9AEM5X9+qbYzrek3NBFCalxkq0MynWl7I7PlYN0hETY+aXCz3VlFdoqdm9LiKKqoZ+mGIzwy03/q8WRNoxmwpemTOB8StEW/cl3KUNbklpCZGM2crETW5HZ95Nd/Wf/EPda3zMezmn7BhzrZfFxYUU90mIXqxs5vNAbL99ryaogIsTByaARV9dWMcURRcKKWB6el4vXB8/PHtwrK252V5B+vNkv8JGCLrpCgLfqUtnXW4A9uz7xTxAM3JPPobH+t9ZrcknYDZ3vGqUO8Evoz8/EfPF/gCc9X233uuQI2QF5JNYaCm0bHsbHNycpQq0FeSTXT0+J4fv7Es7rxtSS11+LT6PKNSKWURSmVp5R6vScXJC5twTrr4DDaYNC7PtVuXk8cGomtg/9zW/bHC6eRvNAHzIBdo8MZ0/h8q4BtOY9b8TERNvPXw6LC2FhQQai19QucbnCT7oji+fn+2dfSjU90ty5PrlFKLQHGA1Fa69s6e65MrhEXYruzkvtW7uKW9GFsOVzR6qDM33YdZ2hkyFlzFdv6rvVlvm1dZz7+StNP2KGvuuC1xUeFUlnnIsSi8Gl/blsphdfno8nj/7M0OMza6hi6EF3R1ck1XdpnKKVGADOB5y50YWJgWLHFae6Gg7Y7K1mxxXnBr52dEsst6cNYl1eKzVDsL/NPl8nJLWFQqLXTgD1GFVMcNscM2Ks8nyWpMeecAdvahT8JhoIxjigiQiz4NFw9fDBKKVweL00ebdZwxw8ObfW3BSG6U1f/cvg74AdAh7fmlVIPKKV2KaV2VVR0rYOa6L86SmN82nrjlj8EHlqbz78PnmRyaiwna5t4dH0Bj/+zgJS4yA4HCYTgZkvId1gf+rB5bWzjM/zEc2+X3t/TSdGJoSDJHkHwL6VPzxtHiNXgVL0bj9eHxwd3ZyWy88ef4+6sRAor6rk1PV5SIqJHnDM9opS6Dfi81vqbSqmpwPckPSKgOVDPzRrJ6txjF1Rn/NDafF7PL8dmUdSccaOUwlAKm6GoO8ex8fst6/mxbY35eL7rB2z2XfOp1tFWMEf+8Mw0AJZuOMLz88fzvrOKZZsKuXxoBHOvG8mCKc1Nnp7d6uTdwipW3jOxW9YgLg3dOdj3euALSqnPA2FAlFJqtdZ67oUuUvRv2SmxzM0aybJNhSyelnpBOdxZYx28nl/OJw1uf64YjRtNZ5nrFFXKxtDvm4//4c3m2+5v0fp25IUJjvoKBuExjmhe21vGW/tPsnhaKqtzjzHG0fpvFwumpLQK4kJ0p3OmR7TWD2mtR2itk4CvApskYAvw77RX5x4zg9eF5HDzj1ezeHoqhvIH3JbZCpvROghb8PJayMOtAvaExj/ybfeDdFfAzkxsDsTFVQ08cENzPfdb+/0d+ZbMGG2OLZP8tbhYpPeI+FRa1h+fK3h1dNPy2p9vYPYftrHdWUnGiGiWbSxsd1SXu8W1r1o24Qybx2eMYgAWub7NFY05VDCkWz6Xwt+JL6+kmhCLYlpaHF8eP8LMT+cfr26VBpKSPnGxdbnk73xITnvg6+gQTMsTfi2vtzxgEsxfxw8OpbCinlCrwZfGJZCTW9Lu/ESAEepjtoV+x3z8tncs97q/j+7GfYdFQdrwKPYHhvpmp9jJWTCp215fiM50NactQVv0uBVbnFgMWL65iLlZI3lxezEer7+J0rYjlbjaGSAQnI5u4OPPtseZbNlvfi27cRll+H9YOKLDKKtuvOA1xg0KISrcxrFTDbi9mhCrQajVkHprcdF0a522EOerZUokY0Q0v3rjEFFhVpZtKuSzV12G1WKw7Uhlu3MV46NCcftgbuROisLmmgH7++4HSGrM4YRqDqJdDdhWo+NctwIq6lx8VFmP26t5ZGYaK++ZAMDCVbslXy36FAnaolsFg3XbOu7LBodSXNVA3KAQXs0rY2JSDC6v5mTt2fUhuuYExWFz+IXXP5Rgjy+VlMZVvOydCvi7692dldjlNRnKP3wgpJ0TNIaC5LhIs2tfuiOKBVNSzGG6t2UMl3y16FMkPSIuSNvc9nZnJfNf3MmgUAu3pg/j5d2leDw+fPgH5ja6fUSFWahptzGT5o+23/N5yw7zyk1Nv+EjPRzwV5EEb0oG+2J3ROEf+xVMvdgjQ6iqd5lfU8rfP6S8uhGbRWExFPFRoRw/3cif75soKRFx0Ul6RFwUR6vqzRTCii1O9pdV4/VpTtW7WZNbgjsQsAEa3T5CLeqsgK2A6cZuisPuNgP2z9zzSGrMMQM2+KtIgj2rz7XVCE6QAbBH2syAnZkYzaAwK1+8JoHy6kampcWBUtyemcDm70/jz/dNlBI+0adJa1Zx3lrurmeNdfDKnlLueXEnQ8JtnKxtwmqAxVB4ffqs4NrU5qbjEGr5IGyh+bjQ5+BW1y9xd/C/5jmGnp9lepq/fWpwZ361I4opV8axbFMhszMTON3gYuU9E9ot4ZPdtuiLJGiL87Jii5MdH1Xxh7cLzcqKKal2NhZUmPlpfx+Pc0fXx63Pcpf1bfPxLU2/pECP7Nb1biyoICLEwnNfH8/6/DLW5JYQEWIxDwRJn2vR30h6RJyXjBHR7Cw+jcfrY+Gq3Xz3r3lnDQI4l+uM/RSHzTED9m/cXyKpMeeCA3bL+hBLi2qRywaHkp0Sy8wMBxEhFq6MHySnGUW/JTtt0aH2DtAA3JYxnLV7Sqlv8rAurwyAEIvC49O0PdBoMcAbSGoPpYY9Yd8wv1aho5nS9DsaCe2W9Qbf2qL87VEr6ppwezXHTjWYB3+e+/p4SYWIfk122qJDHbVfnTXWwYSkmFYBesqVsWcFbGgO2P8MeahVwJ7d9DMmNC0/K2B3Uk59TlZDMT0tDqUUZdWNWA3FIzPTuHNCotk2tr1USNsTnEL0ZbLTFh0K7kQfzMnjqmGD2Xn0NN+/eRT7y6p5t7AKi2q+MbixoIKUuEg+qqxvFbxvMXawIuR35uODvkRudf2qw/dsL/B3JjUukqLAe34mIYoJV9hpdPt411lF5sgYs9verLGVsqMWA4IEbdGhFVucHK2q58ZRcf4pMhbFY+sL0DSP2TJoTokcrfIHz0EhFnDVsi/s/lavd23jCk4RdUFralufXVbdyJ0TEkkOtE/NGBHNM+8Uteo8GLyxKAFbDAQStEWHjlbVs3ZPKU0eH2McURw5WWsGTJ8GrVuX9AWnv/xR/4Ibwj40r//AvYC/eW/61OsInlaE5oDtr7120+DykhwXyYIpKYxxRLdqTDUpxd7hJHQh+ivJaYsOJcdF0uTxEWo1OFhec1Zjp7aZjOuNDykOm8MNFn/ArtBRJDXmXFDABn/AVrSuDqlp9JhzHf+ZXw5I21RxaZCdtjDTILPGOlodR39hWzGZidEcPFHbaa45FBeHwua3utayE193CLMZOIaEc/z0GTw+H26vZvG0VKrqm/jbruNsd1a2e0NR0iJioJGdtiBjRDSv55ebx9G3OytZuGo3nzS4+LC0BlcnU29/Z3uqVcB+1D2HpMacbg3Y6QlRPD9/Al8en8j1qXYshmHmrGdmOPjTvRNlNy0uGRK0L2HBjnzBjnYA81/YwdzncvF4fSyZMQqFPz0RGWJp9b2Z6gjFYXO43bLdvJbUuIZnvZ3OfO6ylqmQA4GhBBkjovmgpJqV90xodTgGkLI9cck4Z9BWSoUppXYopfYqpfYrpX52MRYmel7LZk/ZKbGMHBqBy+uvBnF7Nevzy/FqzaBQC/WBiehWPBSHzWFd6H+ZrzOt6UmSGnP4tPMZ29ZmW5Q/Xx7MWQ+LCuPBnDxe21smOWtxyTtna1allAIitdZ1SikbsA34ttb6/Y6+R1qz9g/BNAjAyKHh7C+rBfyHVAxFqxuPYVaDH7CSe61vmtee8nyRJz13XvA6guWD3kAXP8NQWA3FkhmjeM9ZxXvOUyyZcSVen+yoxcDVba1ZtV9d4KEt8E/3N+EW3aajQbortjhbPQfg6XnjaHB5zYDtiA4jPMTSKmCPUiUUWL/aKmAnN64+r4Dd3knHQYGUS8aIaFbdN5GcBVlYLAZTrozl+fkT8PrghfkTeX7+eAnYQgR0KaetlLIopT4APgb+pbXObec5DyildimldlVUnF8DIdG92h4/f2htPgtX7SZjRDTzX9zBs1v9MxvvW7mL/WXV+FqUhpRVNxIWyEsofBSHzWFD6A/Nr89seoykxhx8XbwdMiwq1Kyzbhu3G9xe7s5K5NipM4A/3bHynglMvMLe6ni5HDUXotl5Ta5RSg0B1gH/obXe19HzJD3Su9obpNvo9jLlyliuS7Hz6PoCIkIsXJc81OzQ1/ak4WLLWpbY/m4+zvFM42FP6xOO58NQYCiFYSjiB4dScvoMClizIAug3SnuQlxKupoeOa86ba31J0qpt4FbgA6Dtrj4gh358o9XmwH76uGDWbapkMmpdt4vOsXbBRVEh9uICLHQ4PK2aqmqgZgIG4PPHOed0O+2eu0rG//c4VCCrjCU/7Vrm7x8/+ZRLN9cxORUO9sKq3huaxEvzJfxXkJ01Tn/JCql4gB3IGCHA58DOu74I3rF0ap6/vB2IROSYthZfJqJSTFsLKggOtzKtsIqrkmM5orYSNbllTF62GCcH9fhCaRFhkWFcqKmkfe9dxEa6jFf88tNP2WnTuuW9X1yxs1XJySyfHORWQHy7FYnSzccMatXhBDn1pXqkQzgT4AFfw78b1rrn3f2PZIe6X7t9bbe7qzkmXeKeOCGZAAWrtqNx+vD5dV4fdrMJVsMRYhFYbX4TxUeOuG/6Rj8+tcsb/Fz25/M133DO4FF7ta77XMJ9gLpyCMz0/D6aPczSGpEiG5Mj2it84HMblmV+NSCNxeDdck7PqpiZ/FpFk9P5cGcPBZNTcZmKOqafAR/Dgdv/nl9Go8Cx5BQiivrzdfMHNLIK2fubfU+VzW+wBnCznt9g8NsVJ/xmLv3ltPPQyyKoop6Hr8j46zvk2PmQpwf6T3ST7TsbX3jqFg2FVRgsyjec1Zxa3o8j64vwFDQ9i9OmuZd8EeBgJ2ZGM3yqnsZduak+bz5ru+z2Xf+P5tV4D2LqxrMa4aCqnoXVqO5858QonvIMfZ+JDsllrlZI1mXV8bsTAdo2FRQwZrcEpTqeIBAVb3bTIV8f1ge6ypmMsznD9g7fKNJbsr5VAHbZihsbQqwp6fFEWbz11/bLAZ3ZyWilOL1/HKZxShEN5Cddj+y3VnJ6txjZrOkrOShbCusAs7eYbcVrWvJC1sIp5uvZTQ+gyUiBp+r41x0Z24YFcu7zirwamIibGSOHMJ25ymuSxnKdSl2iirqSRwayYv3TOC1vWUyOUaIbiBBu58IzmcMVl7sK6tmU0EFFsN//Lsz60J+SqZRaD5e7HqQ//Nl+x80uFsNGTgfmwoqCLM1d9y7f0oy909JJv94tTnmK0iCtRDdQ9IjfVzwSPoz7xSxaGqyWSq3+ZC/xjoipOP/hJ81dlMcNscM2B/54klqzGkO2AHnG7Anp9qxGAoNpA0bLB33hLiIZKfdxwWrRhZNTWb55iIAHv9nATER/puLtY3es74ngkYOhLWuCpnQ+EcqGPKp1xFiUbi8mszEaCZfGUfi0AjW7ikleJ+xZcc92VUL0XMkaPch7dViA9w8Jp7lm4sYHh3GY+sLuCwqlJM1TTiiwyirbmz13GdtT/I5yx7z8Y/d97Da+7lO3zcypLn1ansc0WGcbnCT6RhE9RmPuZOeNdbRqi2qlO8J0fMkPdKHtG30FMxjA9w4Ko79ZTUYBpysaUJBq4A9yThAcdgcM2DX6nCSGtecM2ADNLQI2G2bOhnK/z7ZKUO5OX04m7431fyaNHIS4uKTnXYf0rIWe27WSFbnHuOpOZnsL6vm0fUFpCdEsa+0ptWNwxDcHA77eqvXmdz0O47ry7r8vrqDXwebPFkMOFBWy/PzJ376DyeE6Bay0+5jgrXYyzYVMjdrJOBv/jQ9LY59pTWE2QwzYM+1/KtVwP61+yskNeZ0KWB39B8+WHZ9d1Yid05IJDzEgtVQTE2Lu5CPJYToJrLT7mO2Oyt5dutHzM50sDr3GBV1TSQODedUvYuYCBunG9yMVCfP6sR3ReNq9Pn8DA70Yg2zKBoDAw8MBVNHx3Fdit1s7DRrrIPX9pZxuT2yGz+lEOLTkqDdhwRz2NelDOXfBz9m8fRUlm44QkyEjbLqRqzKx19DfkGWUWB+z6TG/+UEdvOxzVD48Jfi7Q8MxA2KGxRCRZ2/H4hP+4O0q0W9n9VQxEeFsWBKCmMc0WYjJ7m5KETfIemRPiRYi33/FH/XvmUbC0m5LJKy6kZuN7ZRGDrXDNhLXN8gqTGnVcAGcPs0U0fFkjEiGke0v/GTzaIIsRpU1LnM9IclkBf3aUh3RPHIzDRocdxcbjIK0TfJTrsbdNQ2tb2Woy2HFQS/J/jc61Pt/OqNQ9wwKpbF01N54s1DVJZ+RHHYf5jfn+tL4y7Xjzsc9+WIDjObSVktBuE2g+tS7OwsPo0Cmjw+Qq0GTYFOToaCo6caGOOIZqUcNxeiz5OddjfoqFQvY0R0h8+1GPBgTh7PbnWaj5dvLuLOCSN4u6CCX75xkD8YT/J+i4B9Y9NS7nT9tMOAHSwDtFkUiUMjuDJ+EM/P989cfHreOO64NoFpaXEMD+zA4waFEBlqZUJSjFla+PgdGbLDFqIPO68ZkV11KQ5B2O6s5L6Vu7glfRhbDleYPULa23EHg/rw6DD2l9UwOTWWA+U13DwmnuS4SA5v+QtPeH5tPr8rB2RaGuOIory60VxDS89udfLo+gImp9o5UF5rnrRcNDVZJp4L0Yu6OgRBdtrdJDslllvSh7Eur5QbR8WZAbvtjnvFFicAc7NG+g/LKNhWWMnw6DCujnaxYOO1ZsA+6EsktfHPZsBOjYukZSfUjv7j7S+rMfuUtLTdWcnSDUd4ZGYaq++fxFNzMiVgC9HPdGVGZCLwZyAe/9mLZ7TWv+/phfU3252VbDlcwezMBF7NKwU0Ww5XnrXbPVpVz+//fRirxWB2ZgLrAs+9p+LXfGnrO+bzbnX/moPeEebj1LhISj9pJG5QKCdrmwDwAdHhVqrP+Oc6JtkjKK5qwGIolm0sZIyjdZ49/3g1z88fb15r2S9EArYQ/UNXbkR6gP/UWu9RSg0Gdiul/qW1PtDDa+s32rZNBW0OKmi7202Oi6TR7SNEwxv7yplq289Ky6Pm13/tvpMVvi+26rxnKCisqG8RlMEb6NRUfcaDAqwWRdknjSyelsqL24uZkBRz1g3F9gKz9AsRon85Z3pEa12utd4T+HUtcBBI6OmF9Sf5x6tb5bDf3HeSyal23tx3stXNyRVbnHh98PDMNCJ8dew37jIDdpkeylVNK/mjt3XAVviPksdE2MyRXt42I7wshsLt1Uy+0s6SGaN5et44PiipbvdGqBCifzuvnLZSKgn/kN/cHllNPxU8gBLccS+ZcSUHymtZMuPKVhUiGSOi+caNKcwsXUZeyP1YlD863+H+BTerFWSPbv2zMNTi71nt8WlOB4YVAKQnRJnPUcr/9czEaF4I9AZpmfYQQgwsXQ7aSqlBwCvAd7TWNe18/QGl1C6l1K6KioruXGO/kX+8mpvHxDPGEW3e5LtxVBxPvHmYm8fEk21zwn9H4yhYCcA/Bt1JclMOe7zJpMZFmqPDgpq8msgWQw5CrP6Zi/tLawgJHJjR2t/rurCivtUMRjkcI8TA1KWgrZSy4Q/Ya7TWa9t7jtb6Ga31eK31+Li4S7O50DduTGHWWIdZ8+wfwltKpNHIzw99AV6YAUCdDmPljVspn/BDbhrt/73ae7wad5u8hwLqXc3XPF6NT8NNaXFMvjKWUKvB9Sl2Qm0WM4cthBjYulI9ooDngYNa66U9v6T+LZiauOfFnWit+V7IKzxovAL+gg8ej3+S2PRpLN1whJS4SPaV1ZCZGM0HJdW0rZgPPrYYEGq1oLXm9fxyFk9PZfnmIp6eN65VWiZ4/F0IMXB1Zad9PTAPmKaU+iDwz+d7eF39RnCGY/DfAPvLqhnlK+Kw7S5/wAb+qj9LUmMOsenTWDAlhXGXx7CvrIYkewR5bQK2zVDmMAKLghCLwRevcfD8/AncljGcdwurWpUSSg5biEuHnIi8QMFd7qKpySzdcIQvXxPLfflzuFydNJ9z55CXyKtQXO0YzKETdYy7fAjvFlYxxhHFvhad+CJsBg1uf28QQ4F9UCjHT59hjCOK9Yun9MbHE0JcJHIi8iIJ7nKXby7ix0M38vMPp5sB+42x/8uV7pfIPaGZeMVQjp06Q8pl/huOYxKiKKtuZFCoBWgO2OkJUTR5fKQNG0yDy8vszAQOlNXw7FZnb35MIUQfIV3+PoW2Xf2yo06xx/slCGQn/s97HasdP+HDPTWgNfGDQ9lRfIovj0tgTW4JiTHh7CutISbCRl2Tl/jAoN7JqXa2FVaRmRhNwYk68/Ti1Y7BLN1w5KwTjkKIS4/stD+Fo1X1LFy1m+1HTsAzU+EPE8yvTWj8I08M+j47jn7CGbcPm9Xg/huuwKIgJ7eEyal2Sk6fwVBwusFNkj0Ct1fzyMw0DpTXcndWIvvKalgy40ozQC+YksLz88dLzloIITvt9nTW8zp4yvD/8W+y13zF/J7FnsX8S2XjMTTu081T0mdnOlgwJYWiinrW7inl/aJThFgNXB4f6Y4oCk7U8sNbR7eaFvOneyeeFaDluLkQAmSn3a7Oel4/uuYtHs+fwn/zDACbvWNJda3mLbK549oELJbmNnzpCVGs3VPGdmcls8Y6zNOLhoJHZqZRVt3ID28dzfLNRa2mxcjBGCFERyRot6PlzcUbR8Xy2PoCpl5p5zMb57Pe903zeZObfs989w+xWqx87+ZRvLyrlCa3/zBMiEVxtKqBJTOuJP94Na/tLcNmMchOsWOzGOapSa8PKdcTQnSZBO0OZKfEctWwwazLK+Pb8XtZWnATk8gH4O8jfsRn+BsfG5cB4PNplm44jNvrQ+OfubjyXn8fkGUbC7EY8Nb+kzw9bxw5Cybx9Lxx5qlJ2VkLIc6H5LTbCOazAcpKiykOW2hWhezlSr7OzzHKrTS5Pay8dyL7y6p5bH0BLq+/3j3d4S/lA3h63jhe21vW6WEYyVMLIc6HBO02MkZEs3DVLn7Db3mb983rN7t/Q4llBBZDccXQCD4oqWZ/WTULpqTwzJYiKupcRIZYeH3xlFb9tR+/I6Pd95Ebi0KIT0OCdhvZnl18yJ3m41/xdU5cdS+H8koZEx9JxohoLrdH8vmM4Ty2voA/by+mos5F3KAQKutcPLvVyYIpKbKTFkL0iEsmaLc9EAO0HrrbcAp+fYX5tZMhI5lc8wsS7FF8vP8EYxxRHCir4fZMh1maNyImjJLTjSTGhLH1h9N5dquTx9YXAP7aagnYQojudsnciAyW8bWcJPNgTh5Hq+o5sfqBVgF7tu9XTKn/NVmpwyiuaqDB5aWooo6HZ6axbGMhC1ftpuRUvRmwj59uNHfYD89M4902fbGFEKK7DNid9llHzVNiWTQ1mftW7uKyqFAqapv4281u0v/V3IhpbfTXqBr/HfLWFxARotBaYzUUHp/G49PUBgbopsRFkpNbwiMz01gwJeWsHfaCKVIJIoToGQN2p93eznr55iJuSR9GVVUle9Rc0v81F4Ba61DSGl9k7eC5LN1whEdmpnH/5Ct411mF1aKYnelAa1i2qZB7spMYEhHCw4GADcgOWwhx0Qzo1qzBFMjcrJGszj3GoqnJhGz6GV/X/zCf88jQpeSUDeP61Fi2FVayeFoqk1LsLFy1G7fXh81isHh6Kss2FpqPg8MHhBCiu3S1NeuADtoASzccYtmmQu64rJylNf9pXi+76l6mfPBZvBpGDxtERa2LuVkjeXF7MQC3ZQxn1liHWYf98Mw0xjiieW1vGW/tP9mq7loIIS5UV4P2gM1pAzy0Np9/5xezb/B3GFTjT114lY0/Tf43zhoDry4hOtzKoRN13J2VyJIZo6moa+L1/HJmjXWQnRJL/vFqHp6ZhtfXXFs9a2yllPMJIXrFgA3a252VDM//Azv5C7j91+7xPsJm9xjG7K9lX1kNd2clkjg0kpJT9eTklpAUG8njd2Qwa6zDDMrtHS+XgzFCiN7SlcG+LwC3AR9rrdN7fkmfnlkxEnmC7FXXkx24/nffTRy7/pfseu8oV8dFsK+shsmpdh6d3XxaMSk2kncLq8z6agnKQoi+qCvVIyuBW3p4Hd1i7PBwhq+6AVZcb16bZrzAu2P+m2VvO/nsVfGUVzeyeFoqB8przcoS8FeArLxnYm8sWwghuuycQVtr/Q5w6iKs5YJsW/Mo1+VcxRWUAvBq2pN8hr+RNDKRLYcrmJ2ZwKt5pSyamsySGaN5ak5mq5JAIYToD7qtTlsp9YBSapdSaldFRUV3vey5VTnhv6OZfOTX/ocjb2bpdbl85wMHjW4vO4tP89ScTEYPG8zDM9NaDRyQPtZCiP6m225Eaq2fAf84l/Hjx3d/HWFbPi+8eCuU5JqXdv6/7dz7ynHcH31EmM3Ap2Hx9NRWOepg35DgNcldCyH6k/5ZPZK3Bv7RPEGGO56DjC/jdlbi9h6j0e0zD8k8mJPXaoq5BGohRH/Wv4J29XH47Zjmx1fcAPP+AYY/yxMc6fXAlGRW5x5jUopdWqQKIQaUrpT8vQRMBWKVUseB/9JaP9/TC2vF54OX7oQjG5qvfXsvxCSZD7c7K82RXtkpseYu+6k5mTLKSwgxYJwzaGut77oYC+nQ/lfh5a83P77ttzD+3rOeln+8WkZ6CSEGvD7Te+SsIQV1H8OTVzY/YfhYuH8jWGzduFIhhOgbutp7pE+0Zl2xxYnFwF83XVgBr9zfOmB/MxcWviMBWwhxyesTNyKDva8f/cwJslenmtdzU79N1tyf9+LKhBCib+kTQTuYf85e5b9heNR3GU9dtYon7prUyysTQoi+pU8EbfAH7hcS/oe/FtmIvnwshYW15slFIYQQfn0ipw3w7FYn/+NM4eprrqOwop5FU5OlN4gQQrTRJ3ba252VLN1wxJy7GBwTtmhqspTsCSFEC31ip51/vJrn5483B+UGc9xeH3IwRgghWugzddpCCHEp61d12kIIIbpGgrYQQvQjErSFEKIfkaAthBD9iARtIYToR3qkekQpVQEc7fYX7ppYoL+cyOlPa4X+tV5Za8/oT2uF/rXe0Vrrwed6Uo8crtFax/XE63aFUmpXV8pm+oL+tFboX+uVtfaM/rRW6F/rVUp1qU5a0iNCCNGPSNAWQoh+ZCAG7Wd6ewHnoT+tFfrXemWtPaM/rRX613q7tNYeuREphBCiZwzEnbYQQgxYErSFEKIfGTBBWyn1glLqY6XUvt5ey7kopRKVUm8rpQ4opfYrpb7d22vqiFIqTCm1Qym1N7DWn/X2ms5FKWVRSuUppV7v7bWci1KqWCn1oVLqg66WfPUWpdQQpdTflVIFSqmDSqnrentN7VFKjQ78fgb/qVFKfae319URpdR3A3+29imlXlJKhXX6/IGS01ZK3QDUAX/WWqf39no6o5QaDgzXWu9RSg0GdgO3a60P9PLSzqKUUkCk1rpOKWUDtgHf1lq/38tL65BSagkwHojSWt/W2+vpjFKqGBivte7zB0CUUn8Ctmqtn1NKhQARWutPenlZnVJKWYBSIEtr3VsH/jqklErA/2fqaq31GaXU34B/aq1XdvQ9A2anrbV+BzjV2+voCq11udZ6T+DXtcBBIKF3V9U+7VcXeGgL/NNnf9IrpUYAM4HnenstA4lSKhq4AXgeQGvt6usBO2A64OyLAbsFKxCulLICEUBZZ08eMEG7v1JKJQGZQG4vL6VDgXTDB8DHwL+01n12rcDvgB8Avl5eR1dpYINSardS6oHeXkwnrgAqgBcDqafnlFKRvb2oLvgq8FJvL6IjWutS4EngGFAOVGutN3T2PRK0e5FSahDwCvAdrXVNb6+nI1prr9b6GmAEMFEp1SfTT0qp24CPtda7e3st52Gy1vpa4FbgW4E0X19kBa4FlmutM4F64Ee9u6TOBVI4XwBe7u21dEQpFQN8Ef8PRQcQqZSa29n3SNDuJYH88CvAGq312t5eT1cE/jr8NnBLLy+lI9cDXwjkif8CTFNKre7dJXUusNNCa/0xsA6Y2Lsr6tBx4HiLv2X9HX8Q78tuBfZorU/29kI68VngI611hdbaDawFsjv7BgnavSBwc+954KDWemlvr6czSqk4pdSQwK/Dgc8BBb26qA5orR/SWo/QWifh/2vxJq11p7uW3qSUigzciCaQapgB9MnqJ631CaBEKTU6cGk60OdunLdxF304NRJwDJiklIoIxIXp+O9xdWjABG2l1EvAe8BopdRxpdR9vb2mTlwPzMO/EwyWJX2+txfVgeHA20qpfGAn/px2ny+l6yfigW1Kqb3ADmC91vrNXl5TZ/4DWBP4f+Ea4LHeXU7HAj8EP4d/59pnBf7m8ndgD/Ah/pjc6XH2AVPyJ4QQl4IBs9MWQohLgQRtIYToRyRoCyFEPyJBWwgh+hEJ2kII0Y9I0BZCiH5EgrYQQvQj/x/qfqIcvyAIsAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "params = space.sample(1000)\n",
    "values = obj(params)\n",
    "with torch.no_grad():\n",
    "    py,ps2 = opt.model.predict(*space.transform(params))\n",
    "plt.plot(values[:,1], py[:,1], 'x')\n",
    "plt.plot(values[:,1], values[:,1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Copyright statment\n",
    "\n",
    "Copyright (C) 2020. Huawei Technologies Co., Ltd. All rights reserved.\n",
    "\n",
    "This program is free software; you can redistribute it and/or modify it under\n",
    "the terms of the MIT license.\n",
    "\n",
    "This program is distributed in the hope that it will be useful, but WITHOUT ANY\n",
    "WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A\n",
    "PARTICULAR PURPOSE. See the MIT License for more details."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}