С2W4_Градиентный_бустинг.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "С2W4 Градиентный бустинг.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lQy99l3tiUjO",
        "colab_type": "text"
      },
      "source": [
        "# Постановка задачи\n",
        "\n",
        "В рамках текущего задания необходимо реализовать упрощенную модель градиентного бустинга, придерживаясь ряда условий, описанных в ТЗ,  в частности это долен быть градиентный бустинг над регрессионными деревьями для случая квадратичной функции потерь.  \n",
        "Проверку нужно произвести используя классический бостонский датасет из библиотеки sklearn, а полученные результаты соизмерить с классической линейной регрессией\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ujwbtM-wixIA",
        "colab_type": "text"
      },
      "source": [
        "## Подготовка данных и  окружения"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rchEi0MRiM2S",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "from sklearn.datasets import load_boston\n",
        "from sklearn.tree import DecisionTreeRegressor\n",
        "from sklearn.metrics import mean_squared_error\n",
        "from sklearn.linear_model import LinearRegression\n",
        "from xgboost import XGBRegressor \n",
        "from sklearn.model_selection import cross_val_score\n",
        "import matplotlib.pyplot as plt\n",
        "import warnings\n",
        "warnings.filterwarnings('ignore')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CjeU2z3omaH8",
        "colab_type": "code",
        "outputId": "32db2a03-c9af-4911-dd82-36b035d7edd3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "df = load_boston()\n",
        "X = df.data\n",
        "y = df.target\n",
        "size = len(df.target)\n",
        "print(size, size == len(df.target))"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "506 True\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MtQohrvyuyu3",
        "colab_type": "text"
      },
      "source": [
        "Разобъем данные на выборки для обучения и тестирования, в сооотношении 75/25 %, с сохранением порядка"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6fSzd2hlu7It",
        "colab_type": "code",
        "outputId": "cdd6e6e1-08e4-498f-bb0a-6bbbea7804e5",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "source": [
        "X_train, X_test, y_train, y_test = (\n",
        "    X[:int(0.75*size)], X[int(0.75*size):],\n",
        "    y[:int(0.75*size)], y[int(0.75*size):])\n",
        "\n",
        "print(X_train.shape, y_train.shape)\n",
        "print(X_test.shape, y_test.shape)"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(379, 13) (379,)\n",
            "(127, 13) (127,)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KYSOUKd6LiuK",
        "colab_type": "text"
      },
      "source": [
        "Аналогично минимизации функций методом градиентного спуска, в градиентном бустинге мы подправляем композицию, изменяя алгоритм в направлении антиградиента ошибки.  \n",
        "Воспользуйтемся формулой из лекций, задающей ответы на обучающей выборке, на которые нужно обучать новый алгоритм (фактически это лишь чуть более подробно расписанный градиент от ошибки), и получите частный ее случай, если функция потерь L - квадрат отклонения ответа композиции a(x) от правильного ответа y на данном x\n",
        "\n",
        "$$\\sum_{i=0}^{l} \\mathbb{L}(y_i, a_{N-1}(x_i) + \\xi_i)  =  \\sum_{i=0}^{l} (y_i - (a_{N-1}(x_i) + \\xi_i))^2 \\to \\min_{\\xi}$$  \n",
        "\n",
        "$$\\xi_i = 2(y_i - a_{N-1}(x_i).$$"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GCjZuiJ2n9oB",
        "colab_type": "text"
      },
      "source": [
        "## Реализация алгоритма\n",
        "Определим  функцию, которая будет вычислять прогноз построенной  композиции деревьев на выборке `X` и функцию расчета градиента (согласно условию  - 2 не берем в расчет)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0te3DaLRoQRE",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def gbm_predict(X):\n",
        "  '''\n",
        "  base_algorithms_list - список с базовыми алгоритмами\n",
        "  coefficients_list - список с коэффициентами перед алгоритмами\n",
        "  '''\n",
        "  return [sum([\n",
        "      coeff*algo.predict([x])[0] for algo, coeff\n",
        "      in zip(base_algorithms_list, coefficients_list)\n",
        "  ]) for x in X]\n",
        "\n",
        "\n",
        "def get_grad():\n",
        "  return [y - a for a, y in zip(gbm_predict(X_train), y_train)]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-yRbHeH3t-i5",
        "colab_type": "text"
      },
      "source": [
        "В качестве базового алгоритма для градиентного бустинга будет использовано регресионное дерево решений из библиотеки sklearn. Воспроизводимость ответов на платформе coursera предписывает установить для него параметры `max_depth=5` и `random_state=42`  \n",
        "В бустинге зачастую используются сотни и тысячи деревьев, но мы ограничимся 50, чтобы алгоритм работал быстрее, и его было проще отлаживать (т.к. цель задания разобраться, как работает метод)  \n",
        "\n",
        "Определим ф-цию, обучающую 50 базовых алгоритмов.  \n",
        "\n",
        "Попробуем при этом для начала всегда брать коэффициент равным 0.9. Обычно оправдано выбирать коэффициент значительно меньшим - порядка 0.05 или 0.1, но т.к. в нашем учебном примере на стандартном датасете будет всего 50 деревьев, возьмем для начала шаг побольше."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SddL9iA-x1nG",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "base_algorithms_list, coefficients_list = [], []\n",
        "\n",
        "def gboost_fit(coef, minimize_coef=False):\n",
        "  for i in range(50):\n",
        "    coef = coef/(1.0 + i) if minimize_coef else coef\n",
        "    base_model = DecisionTreeRegressor(max_depth=5, random_state=42)\n",
        "    base_model.fit(X_train, get_grad())\n",
        "    base_algorithms_list.append(base_model)\n",
        "    coefficients_list.append(coef)\n",
        "\n",
        "gboost_fit(0.9)  "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6xU_0iokCn9z",
        "colab_type": "code",
        "outputId": "78807d57-d61c-4bc8-f96d-2d0c56c438b8",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "def get_RMSE():\n",
        "  MSE = mean_squared_error(\n",
        "      y_test,\n",
        "      gbm_predict(X_test)\n",
        "  )\n",
        "  return MSE**0.5\n",
        "\n",
        "\n",
        "get_RMSE()"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "5.455565103009402"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "t3NllaOhoUn1",
        "colab_type": "text"
      },
      "source": [
        "А теперь попробуем уменьшать вес перед каждым алгоритмом с каждой следующей итерацией"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "inuL0K3hNCmb",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "base_algorithms_list, coefficients_list = [], [] # Переопределим\n",
        "gboost_fit(0.9, True)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6JY-UXsBOCvP",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "a3accc8b-924f-4ebd-b019-857f59e61f6c"
      },
      "source": [
        "get_RMSE()"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "4.409771514361549"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uaScU15JRc-a",
        "colab_type": "text"
      },
      "source": [
        "## Оценка путем сравнения с другими регресионными моделями  \n",
        "Сравним реализованный алгоритм с тем, что предлагает библиотека XGBoost "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YIAzpo0VSFX0",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        },
        "outputId": "da340a10-10d0-4b64-8c74-da6ea941bddd"
      },
      "source": [
        "xbg = XGBRegressor(n_estimators=50, max_depth=5)\n",
        "xbg.fit(X_train, y_train)\n",
        "xbg_pred = xbg.predict(X_test)\n",
        "mean_squared_error(y_test, xbg_pred)**0.5"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[08:30:10] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "4.923855118818414"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f1gnm-PqVXTE",
        "colab_type": "text"
      },
      "source": [
        "Получили соизмеримую величину ошибки, отчасти из-за некоторой синтетичности в реализации.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9-P_k0_oWHus",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "ab647698-4b9f-4918-b1ef-708beea8ca0c"
      },
      "source": [
        "linear_model = LinearRegression()\n",
        "linear_model.fit(X_train, y_train)\n",
        "linear_model_pred = linear_model.predict(X_test)\n",
        "mean_squared_error(y_test, linear_model_pred)**0.5"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "8.254979753549401"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "y2lRZvSuWYZi",
        "colab_type": "text"
      },
      "source": [
        "По причине того, что модель проста и никак не адаптирована - получили требуемое по заданию и ожидаемое ухудшение качества"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "67Of4LNlWkFQ",
        "colab_type": "text"
      },
      "source": [
        "## Анализ влияния параметров градиентного бустинга на качество\n",
        "\n",
        "Для чистоты эксперемента сравнение будем производить на широко используемом алгоритме XGBRegressor, вместо лабораторного"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "P_laA-lrXajQ",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "9b2e3920-2f6f-4593-e845-52d1c4435e78"
      },
      "source": [
        "trees, errors = np.arange(50, 1000, 50), []\n",
        "for tree in trees:\n",
        "  errors.append(\n",
        "      cross_val_score(XGBRegressor(n_estimators=tree),\n",
        "                      X,\n",
        "                      y,\n",
        "                      scoring='neg_mean_squared_error').mean()\n",
        "    )\n",
        "plt.plot(trees, errors)\n",
        "plt.xlabel(\"n_estimators\")\n",
        "plt.ylabel(\"error\")"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[08:30:10] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:10] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:10] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:10] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:10] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:10] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:10] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:10] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:10] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:10] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:11] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:11] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:11] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:11] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:11] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:11] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:11] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:11] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:11] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:11] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:12] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:12] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:12] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:12] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:12] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:12] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:13] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:13] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:13] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:13] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:13] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:14] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:14] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:14] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:14] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:14] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:15] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:15] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:15] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:15] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:16] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:16] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:16] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:17] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:17] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:17] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:17] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:18] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:18] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:18] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:19] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:19] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:19] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:20] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:20] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:20] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:21] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0, 0.5, 'error')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAELCAYAAAD+9XA2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xl81PW97/HXJztJICErCUvCKgoE\n1CBgEUHUUqVaT12P1r1ab3usbdVW7e2tPaebWq299rZSW61ia+tecUHABbWCAkJAZFF2MhACSYDs\ny/f+Mb9AxBCyzGSSmffzwTzym98y85lhMp98d3POISIi0lVRoQ5ARETCgxKKiIgEhBKKiIgEhBKK\niIgEhBKKiIgEhBKKiIgEhBKKiIgEhBKKiIgEhBKKiIgEREyoA+hOGRkZLj8/P9RhiIj0KsuXLy91\nzmUe67yISij5+fksW7Ys1GGIiPQqZra1PeepyktERAJCCUVERAJCCUVERAJCCUVERAJCCUVERAJC\nCUVERAJCCUVERAJCCSXCrNhWxra9VaEOQ0TCkBJKBFmzs4JLH17CD58tCnUoIhKGlFAixIGaer7z\ntxXUNTaxdPNeSg7UhDokEQkzSigRwDnHnc+vYXtZNT+/YCxNDl5bsyvUYYlImFFCiQBPfbidl1YV\n8/2zRnH5pDxGZSczb5Uv1GGJSJhRQglzn/j289N/fcxpIzO46fThAMwuyOXDrfvYVaFqLxEJHCWU\nMFZZ28B3/raCfn1iuf/iCURFGQDnFuTgHLy8WqUUEQkcJZQw9pMXP2ZTaSUPXjKBzL7xh/YPz0zm\n+Jx+zCsqDmF0IhJulFDC1DPLd/Dsih3cfMZITh2R8YXjswty+GhbOTvKNCZFRAJDCSUMfVpygP/9\nwhomD0vj5pkjWz3nqwW5ALyiai8RCRAllDBTXdfIt5/8iMS4aB689ESivXaTIw1JT6RgUArzipRQ\nRCQwlFDCzM/mfcz63Qe4/5IJZPdLaPPc2QU5FO2oYOveym6KTkTCmRJKGHlx5U7+/sF2bpo+nNNH\nZR7z/HPG5QColCIiAaGEEiY2l1Zy53OrKczrzw/OGtWuawb1T+TEIam8rIQiIgGghBIGauob+faT\nK4iNieJ3l51ITHT7/1tnF+Sy1refTXsOBjFCEYkESihh4BevfMJa337uu3A8ual9OnTtueNyMFO1\nl4h0nRJKL/fqah+Pv7+V66cO5cwTsjt8/YCUBCbmpWmQo4h0mRJKL7ZtbxW3P1vE+MGp3D5rdKcf\n59yCHDbsPsiG3QcCGJ2IRBollF6qrqGJ//r7CgAeuuxE4mI6/1/5lXEDiFK1l4h0kRJKL/Xr19ax\nakcF915YwOC0xC49VlbfBCYNTWdeUTHOuQBFKCKRRgmlF1qwdjd/fnczV03JY9bYnIA85uzxOWza\nU8knPlV7iUjnKKH0MjvLq7n16VWMHdiPO889PmCPO2vMAKKjTI3zItJpSii9SH1jEzf//SMamxwP\nXXYS8THRAXvs9OR4Th2ezrwin6q9RKRTlFB6kQcWbGD51jJ+8R/jyM9ICvjjzy7IYdu+Ktbs3B/w\nxxaR8BeShGJm95rZOjMrMrPnzSy1xbECM3vfzD42s9Vm9oUZDtu6Plwt37qPP7z9GZdOHMx543OD\n8hxfHjOAGFV7iUgnhaqEsgAY65wrADYAdwCYWQwwF/iWc24MMB2ob+/14aq6rpFbny4iN6UPP559\nQtCeJzUxjtNGZqjaS0Q6JSQJxTn3unOuwbu7BBjkbZ8NFDnnVnnn7XXONXbg+rB0z/x1bC6t5N6L\nCkiOjwnqc51bkMvO8mo+2l4e1OcRkfDTE9pQrgVe9bZHAc7M5pvZCjO7vYPXh50lm/by6HtbuGpK\nHqcO/+JSvoF29phs4qKjmLdKgxxFpGOCllDMbKGZrWnldn6Lc+4CGoAnvV0xwFTgcu/nBWY2s43n\nOPL61s65wcyWmdmyPXv2BOCVdZ/K2gZue2YVeemJ/PArnZ9apSP6JcQybVQmr6z20dSkai8Rab+g\n1Z84585s67iZXQ3MBma6wxX2O4DFzrlS75xXgJOARe28vrU45gBzAAoLC3vVN+QvX/2EHWXV/PPG\nKSTGBbeqq6Wvjs9h4Se7Wb6tjIn5ad32vCLSu4Wql9cs4HbgPOdcVYtD84FxZpboNdCfDqztwPVh\n492Npcxdso3rvjS027/UZx6fTXxMFPNWqbeXiLRfqNpQHgL6AgvMbKWZ/RHAOVcG3A98CKwEVjjn\nXgYws0fMrLCt68PFgZp6fvhsEcMyk7j1y8d1+/Mnx8cw47gsXlmzi0ZVe4lIO3VfPUoLzrkRbRyb\ni7/r8JH7r2/P9eHg5y9/gq+immdvOpWE2MCNhu+I2eNzeO3jXSzdvLdbOgOISO/XE3p5SQtvri/h\nqQ+3c+PpwzlxSP+QxXHG6Cz6xEZrvXkRaTcllB6koqqeHz1bxKjsZG45c2RIY0mMi2Hm8Vm8tmYX\nDY1NIY1FRHoHJZQe5O55H1N6sI7fXDQhoBM/dtbsglz2Vtbx/qa9oQ5FRHoBJZQeYsHa3Ty3Yiff\nnjGCcYNSQh0OANOPyyQpLlqDHEWkXZRQeoCyyjrueG41J+T04zszek5/g4TYaM46IZvXPt5FXYOq\nvUSkbUooPcBP/vUxFdV13HfR+C6tDR8Mswtyqaiu573PSkMdioj0cD3r2ysCvbLax0urirn5jJGc\nkNsv1OF8wWmjMuibEKNqLxE5JiWUECo9WMuPX1hDwaAUbpo+PNThtCo+JpqzTxjA62t3UdvwhYmf\nRUQOUUIJEeccP35+DQdrGvjNReOJie65/xWzx+dwoKaBxRtU7SUiR9dzv8XC3L9WFfPax7v4/tmj\nGJndN9ThtGnqiAxSE2O1kqOItEkJJQRK9tfwkxc/5qQhqXzztGGhDueYYqOjmDVmAAvX7qamXtVe\nItI6JZRu5pzjjudWU1PfyH0XjSc6ykIdUrvMLsilsq6Rt9aXhDoUEemhlFC62TPLd7BoXQm3zxrN\nsMzkUIfTbpOHpZGeFMdLmttLRI5CCaUb+Sqq+dlLazklP41rTs0PdTgdEhMdxayxA3jjkxKq6hpC\nHY6I9EBKKN3EOcftzxTR0OS496IConpJVVdLswtyqa5vZNEnqvYSkS9SQukmTy/bwTsbS7nznNHk\npSeFOpxOOWVoGpl94zWlvYi0Sgmlm7xUVMzIrGQun5QX6lA6LTrKOHdcDm+uL+Fgraq9ROTzlFC6\nia+ihuGZyb2yqqulcwtyqG1oYuHa3aEORUR6GCWUbrKrooYBKQmhDqPLTh7Sn4GpfXhy6dZQhyIi\nPYwSSjfYX1PPwdoGclN7f0KJijKunTqUD7eUsWJbWajDEZEeRAmlG/jKawDISekT4kgC49KJg+mX\nEMOctzeFOhQR6UGUULpBcUU1QFiUUACS4mO4YnIe89fuYnNpZajDEZEeQgmlG4RbCQXg6lPziY2K\n4pF3VEoRET8llG7gq6gmyiCrb3yoQwmYrH4JXHDiQJ5ZvoPSg7WhDkdEegAllG5QXF5Ddr+EHr3m\nSWd8c9pQahuaePx99fgSESWUbuGrqCYnDLoMH2lEVl/OPD6LJ97fQnWdprUXiXRKKN3AV1ETVu0n\nLd0wbThlVfU8vXx7qEMRkRBTQgky51zYllAAJub3Z8LgVB55ZzONTS7U4YhICCmhBFl5VT019U3k\npIZnCcXMuHHaMLbtq+K1NbtCHY6IhJASSpAdGoMSpiUUgLPHDCA/PZE5iz/DOZVSRCKVEkqQHRqD\nEqYlFPDPQnzdacNYtaOCpZv3hTocEQkRJZQg80VACQXgopMHkZYUx5zFGugoEqlCklDM7F4zW2dm\nRWb2vJmltjhWYGbvm9nHZrbazI76TWxmPzAzZ2YZ3RN5xxVX1BAbbWQkh8+gxtYkxEZz5ZQ83lhX\nwsbdB0IdjoiEQKhKKAuAsc65AmADcAeAmcUAc4FvOefGANOB+tYewMwGA2cD27oj4M7ylVeT3S+h\n16+D0h5XTsknITZKpRSRCBWShOKce90517zk3xJgkLd9NlDknFvlnbfXOXe0EXMPALcDPboVuLii\nhtwwHYNypLSkOC46eTAvrNzJ7v01oQ5HRLpZT2hDuRZ41dseBTgzm29mK8zs9tYuMLPzgZ3Niacn\nC5eFtdrr+tOG0tjkePS9LaEORUS6WdASipktNLM1rdzOb3HOXUAD8KS3KwaYClzu/bzAzGYe8biJ\nwJ3AT9oZxw1mtszMlu3ZsycAr6z9mpocuypqyAmTaevbIy89iVljB/Dk0q1ad14kwgQtoTjnznTO\njW3l9iKAmV0NzAYud4cHL+wAFjvnSp1zVcArwElHPPRwYCiwysy24K8uW2FmA44SxxznXKFzrjAz\nMzPgr7MteyvrqGtsipgqr2Y3TBvOgZoGnvqgRzdviUiAhaqX1yz87R/neYmj2XxgnJkleg30pwNr\nW17rnFvtnMtyzuU75/LxJ6GTnHM9bph2c5fhcJ125WgmDE7llKFp/OXdzdQ3NoU6HBHpJqFqQ3kI\n6AssMLOVZvZHAOdcGXA/8CGwEljhnHsZwMweMbPCEMXbKcXeoMbcMB7UeDQ3ThtGcUUN84qKQx2K\niHSTmFA8qXNuRBvH5uLvOnzk/uuPcn5+4CILrEgtoQDMOC6LEVnJPPz2Jr42YSBm4d9tWiTS9YRe\nXmHLV1FDfEwUaUlxoQ6l20VFGTecNox1uw7wzsbSUIcjIt1ACSWIisv909ZH6l/n55+YS1bfeA10\nFIkQSihBFM4La7VHfEw0V38pn3c/LWXNzopQhyMiQaaEEkS7Kmoisv2kpcsn5ZEUF82f3lEpRSTc\nKaEESWOTY9f+yBrU2JqUPrFcesoQ5hX52FFWdewLRKTXUkIJkj0HamlschFd5dXs2qlDMeAv724J\ndSgiEkRKKEFyaKXGCC+hAAxM7cNXx+fy1IfbqKhqdfJoEQkDSihBcmilRpVQAPjmacOoqmtk7tKt\noQ5FRIJECSVIDq/UqIQCcEJuP04bmcFj/95CbcPRViQQkd5MCSVIistrSIyLpl+fkExG0CPdOG04\new7U8sJHO0MdiogEgRJKkPgqIntQY2u+NCKdE3L6MWfxJpqaevS6aCLSCUooQRLpgxpbY2bcePow\nPttTyRvrSkIdjogEmBJKkDSXUOTzzhmXw8DUPpqORSQMKaEEQX1jEyUHasmJwGnrjyU2Ooprpw7l\ngy37+NeqYlV9iYQRJZQg2L2/BucgVyWUVl06cTB56Ync/PePmPGbt/jT4k2UV9WFOiwR6SIllCDw\nVXhjUFRCaVVSfAyvf28aD146gay+8fz8lU+Y9ItF3Pr0Kop2lIc6PBHppGP2aTWzaOBm59wD3RBP\nWCgubx6DohLK0cTHRHP+hIGcP2Egn/j2M3fJVp7/aCfPLN/B+EEpXDE5j6+OzyUhNjrUoYpIOx2z\nhOKcawQu64ZYwoZKKB1zfE4/fn7BOJbcOZO7zxtDZV0jtz1TxKRfLOLnL69l697KUIcoIu3Q3lF3\n75nZQ8A/gEO/3c65FUGJqpfzlVfTNyGG5HgNauyIfgmxXHVqPldOyWPJpn3MXbKVR9/bwp/e2czp\nozL5xuQ8ZozOIjpKY3tEeqL2fuNN8H7+rMU+B5wR2HDCg6+iRlOudIGZMWV4OlOGp7N7fw1//2Ab\nf/9gG9c/voyBqX34z0lDuGTiYDKS40Mdqoi00K6E4pybEexAwomvooYBaj8JiOx+Cdxy5ii+PWME\nC9fu5oklW7l3/noeXLiRc8YN4DtnjGREVnKowxQR2plQzCwF+D/ANG/X28DPnHNa17UVvopqxg7s\nF+owwkpsdBRfGZfDV8bl8GnJAeYu2cYzy3fwUpGPb0zO47szR9I/KS7UYYpEtPZ2G/4LcAC42Lvt\nBx4NVlC9WW1DI6UH6zTtShCNyOrLT88bw9u3TeeyUwbz+PtbmH7fWzz63mbqG5tCHZ5IxGpvQhnu\nnPs/zrlN3u1uYFgwA+utdjX38FKVV9ClJ8fzP18bx6vfnUbBoBTufmktX/7tYt5YtxvnNAJfpLu1\nN6FUm9nU5jtm9iWgOjgh9W7F3sJaueoy3G2OG9CXx689hb9cXQgOrn1sGVf+5QM27D4Q6tBEIkp7\ne3l9C3jca0sBKAOuCk5IvVvzwloqoXQvM+OM0dlMHZHJ3CVb+e3CDXzlwXf4z1OG8L2zRpGm9hWR\noGvPSPko4Djn3Hgz6wfgnNsf9Mh6qUODGtWGEhJxMf7JJy84cSC/XbiBuUu38cLKnXx35kiunJJP\nXIxmGxIJlvaMlG8Cbve29yuZtK24vJr+ibH0idOUIaHUPymOu88fy2vfPY2T8/rzPy9/wpd/u5gF\na9W+IhIs7f1zbaGZ3Wpmg80srfkW1Mh6qV1aWKtHGZndl8euOYVHr5lIdJTxzceXccWfl/KJT38X\niQRae9tQLvF+frvFPod6en1BcUWNJoXsgWYcl8XUERn8bek2Hli4gXN/9w6XTBzCD84epRH3IgFy\nzBKK14ZyhXNu6BE3JZNW+CqqyUlVQumJYqOjuOrUfN6+dQZXnzqUp5dtZ8a9b7F0095QhyYSFtrb\nhvJQN8TS61XXNVJeVa8qrx4uJTGWn3z1BOZ/bxqZ/eK5ce5yNpdqRmORrmpvG8oiM/u6mQVkmlcz\nu9fM1plZkZk9b2apLY4VmNn7Zvaxma02s1b/3Dez//Ie42MzuycQcXVVsddlOFcllF5heGYyj119\nClFmXPPoB5RVatVIka5ob0K5EfgnUGtm+83sgJl1pVVzATDWOVcAbADuADCzGGAu8C3n3BhgOlB/\n5MVmNgM4HxjvnXdfF2IJGF+5ugz3NkPSE/nTlSdTXFHDjU8sp7ahMdQhifRa7U0oKcDVwP845/oB\nY4CzOvukzrnXnXMN3t0lwCBv+2ygyDm3yjtvr7fA15FuAn7lnKv1zivpbCyBdKiEooTSq5ycl8Z9\nF43ngy37+NGzq9WtWKST2ptQfg9M5vDKjQcIXLvKtcCr3vYowJnZfDNbYWa3H+WaUcBpZrbUzN42\ns4kBiqVLmkso2SnqNdTbnDc+lx+cNYrnP9rJg4s2hjockV6pvd2GJznnTjKzjwCcc2Vm1uZcFma2\nEBjQyqG7nHMveufcBTQAT7aIZyowEajC33az3Dm3qJW40/AnuYnAP81smGvlT0szuwG4AWDIkCHt\nerGdtWt/NRnJ8cTHaFBjb/SdM0awZW8Vv124kfz0JL524sBQhyTSq7Q3odSbWTT+sSeYWSbQ5jzh\nzrkz2zpuZlcDs4GZLRLBDmCxc67UO+cV4CTgyISyA3jOu+4DM2sCMoA9rcQxB5gDUFhYGNS6jOLy\nGs3h1YuZGb/8j3HsLK/i9meKyE3twylDNX5XpL3aW+X1O+B5IMvMfg68C/yis09qZrPwT+dynnOu\nqsWh+cA4M0v0GuhPB9a28hAvADO8xxoFxAGlnY0nUHwV1UoovVxcTBR/vOJkBvXvw41PLGOLuhOL\ntFu7Eopz7kn8CeCXgA/4mnPu6S4870NAX2CBma00sz96z1MG3A98CKwEVjjnXgYws0fMrNC7/i/A\nMDNbAzwFXNVadVd385XXaNr6MJCaGMej1/ib5a597EPKq9SdWKQ92lvlhXNuHbAuEE/qnBvRxrG5\n+LsOH7n/+hbbdcAVgYglUA7U1HOgtkEllDCRl57EnCsLufxPS7nxieU8cd0kzVQscgz6DQmQQ9PW\nq4QSNibmp3HvRQUs3byPHz1XpO7EIsfQ7hKKtK24vHkMikoo4eT8CQPZUlrFAws3MDQ9if+aOTLU\nIYn0WEooAaISSvi6eeYItu6t5DcLNjAkPZHzJ6g7sUhrVOUVIL7yaqIMsvtqUGO4MTN++fVxnDI0\njdueLmLZln2hDkmkR1JCCRBfRQ1ZfROIidZbGo7iY6J5+IqTGdi/Dzc8sZyte9WdWORI+vYLEF9F\nDQPUfhLW+ifF8ZerJ9LkHNc89iEVVV+Yt1QkoimhBEhxRbWmrY8AQzOSmPONQnbsq+bGucuoa2hz\nwgiRiKKEEgDOOXzlWks+UpwyNI1fXziOJZv2cefzmp1YpJl6eQVARXU91fWNGtQYQS44cRBbSqt4\ncNFGhmYk8e0ZRx2rKxIxlFACoNibtl7TrkSWW84cyda9ldw7fz3vfVrKdVOHMuO4LKKiArKwqUiv\no4QSAD5vYS2VUCKLmXHPheM5Pqcfj/17C9f9dRnDMpK4dupQvn7SIPrEaRkDiSxqQwmA4gqVUCJV\nXEwUN54+nMW3z+DBSyeQnBDDj19Yw5RfLeK++esp2V8T6hBFuo1KKAGwq6KamCgjI1mDGiNVbHQU\n508YyHnjc1m2tYxH3tnE79/6lIcXf8ZXx+dy3dShjMlNCXWYIkGlhBIAvvIasvslEK2684hnZkzM\nT2Nifhpb91by6Htb+Oey7Ty3YienDk/n+tOGMn1U4NpZmpocO8ur2VhygD0HaknpE0tqYhz9E+NI\nTYwlNTFWK4hKt1FCCYBiLawlrchLT+Kn543he2eN4qkPtvHYv7dw7WPLGJaZxLVf6lg7S2OTY/u+\nKjaWHGRjyQE+3X2QjSUH+bTkINX1jW1e2yc2mv6J/kSTmhj7uWTTPzGOlD7+n/2TYslLT1JJWzrN\nIqkPfWFhoVu2bFnAH/f0e9+kYFAq//eyEwP+2BI+6hubeGW1jz+/u5miHRWkJsZyxaQ8rpySR1Y/\n/x8kDY1NbN1XxcbdB/m05AAbvMTx2Z6DnxtEmZOSwMjsvozMSvbfspPJ6pvAgZoGyqvqKKuqp7y6\njvKqesoq6yivrj+8v8q/v7y6nsamz//+x0VHccXkPL5zxgjSkuK69f2RnsvMljvnCo91nkooXeSc\nw1dRw6wxKqFI29pqZ5k8LJ2S/bVsKj1IfePhL/lB/fswMiuZ00ZmMMJLHiOykumbENvleJqaHAfr\nGiivrKesqo6yqjpeW7OLx/69maeXbefG04dx7dShJMbpa0LaR5+ULtpbWUddQ5OqvKTdWmtnee/T\nUvLSE5kxOutQiWN4ZjJJ8cH7FY2KMvolxNIvIZYh6YkATD8ui+tPG8o9r63nvtc38Nf3t3LLmSO5\nuHAwsZr4VI5BCaWLfOVaB0U6r7mdpScZkdWXOVcWsnzrPn75yjruen4Nf35nM7d9+ThmjR2AmTqf\nSOv0J0cXFVc0r9SohCLh5eS8NJ7+1hQeubKQ6CjjpidX8LX/92+WbNob6tCkh1JC6SKft/RvjmYa\nljBkZpx5Qjav3TKNey4soGR/DZfOWcI1j37AJ779oQ5PehgllC7y7a8hLjqKdPWIkTAWHWVcXDiY\nN2+dzp3njGbFtnLO+d07fP+fK9lRVhXq8KSHUELpIl+5f2Et1StLJEiIjeaGacNZfNsMbpw2nJeL\nfJxx39v897y17KusC3V4EmJKKF3k06BGiUApibH86Cujeeu26Vxw4kAefW8zp9/zJr9/81Oq6hpC\nHZ6EiBJKFxWX12hSSIlYOSl9+PWFBcy/ZRqTh6dz7/z1fOlXb/Dr19apKiwCqdtwFzQ2OXbvr1EJ\nRSLeyOy+/Mnrajxn8SYefvszHn77M2Yen81VU/L50oh0VQtHACWULig9WEtDk9MYFBHPyXlpPPyN\nNHaWV/O3pVt56oPtLFi7m2GZSVw5OY+vnzwoIKP8pWdSlVcXFJc3j0FRCUWkpYGpfbjty6P59x1n\n8MAl4+mXEMtPX1rLpF8s4scvrGbD7gOhDlGCQCWULvB5C2vlaFCjSKviY6K54MRBXHDiIIp2lPP4\n+1v557IdzF2yjcnD0rhqSj5nnZBNjKZ1CQtKKF3gO7RSo0ooIsdSMCiV+y5K5c5zjucfH25n7pKt\n3PTkCnJSEvjPU4Zw6SlDyOyrqfN7MyWULvCVV9MnNpqUPqoTFmmvtKQ4bpo+nBumDeONdSU8/v4W\nfrNgA797YyPnjMvhyin5nDQkVY34vZASShf4Kvw9vPTBF+m46CjjrBOyOeuEbD7bc5An3t/Ks8t3\n8OLKYoZnJjHz+GxmHJdFYX5/zXTcS4QkoZjZvcBXgTrgM+Aa51y5d6wAeBjoBzQBE51zNUdcPwH4\nI5AANAD/yzn3Qfe9Ar/iimrN4SUSAMMzk/npeWO47cvH8cLKnby6ehePvreZOYs30TchhmkjM5kx\nOovpx2VqRckeLFQllAXAHc65BjP7NXAH8EMziwHmAt9wzq0ys3SgvpXr7wHuds69ambnePend1Ps\nh/jKa5g6MqO7n1YkbCXFx3D5pDwun5THwdoG3t1YypvrSnhzfQkvr/ZhBuMHpXLG6CzOGJ3FmNx+\nqiHoQUKSUJxzr7e4uwS40Ns+Gyhyzq3yzjvaPNkOfwkGIAUoDkacbWlobKLkQI26DIsESXJ8DLPG\nDmDW2AE0NTnW+vbzxroS3lhXwgMLN3D/gg1k9Y1nxnFZzBidxdSRGSR3cUGyxibH/mr/Cpbl1fXU\nNzThgOaV0h0O75//vvP2Hdr2r+LqP/fw60jtE0tKYiypfeKIiwnf6rue0IZyLfAPb3sU4MxsPpAJ\nPOWcu6eVa24B5pvZffjH0px6tAc3sxuAGwCGDBkSsKB3H6ilyWlhLZHuEBVljB2YwtiBKdw8cySl\nB2t5a/0e3lxXwiurffxj2XZio41JQ9OZ4ZVesvvFU1ZVT1llHRXVzcsc11Ne6f30lj0uq6o/dLyi\nuv5Q8giWpLhoUhPjSOkTS2qi/5bSJ86/fWhfnLcdR//EWDKS44mK6vklsaAlFDNbCAxo5dBdzrkX\nvXPuwt8G8mSLeKYCE4EqYJGZLXfOLTriMW4Cvuece9bMLgb+DJzZWhzOuTnAHIDCwsKAfVQOrYOi\nEopIt8tIjufCkwdx4cmDqG9sYtmWMt5c7y+9/Pe8tfz3vLVtXt/8pd4/KZb+iXEMTksktU8s/b0v\n8/5J/tJEfHNpwsDwf6GbgcGhqrbm+83bYLSshausbaC8qp7yan8yK6+up7yqnorqOsqr6tmw+6D/\neFUdDU2tf0XFx0QxNCPp0G1YZrL/Z0YS/XvQ0hlBSyjOuVa/4JuZ2dXAbGCmc4f+JtgBLHbOlXrn\nvAKcBByZUK4CvuttPw08EqBH83OfAAAMAklEQVSw26340BgUlVBEQik2Ooopw9OZMjydO885nu37\nqnhrfQkHaxsPJ4jEWPonNZcCema1k3OOqrpGyqsPl6rKq+rZV1nLtn1VbC6tZP2uAyxYu/tziSc1\nMfZQohnuJZqhGUnkpyfRJy66W19DqHp5zQJuB053zrWcknQ+cLuZJeLvAXY68EArD1HsHXsLOAPY\nGNSAW7GrQiUUkZ5ocFoi35iSH+owOszMSIqPISk+hoFt/KFa39jEjrJqNpceZNOeSjaVVrJ5TyX/\n/nQvz63Y+blzc1MSDpVmrpicx3ED+gb1NYSqDeUhIB5Y4BUblzjnvuWcKzOz+4EP8bdpveKcexnA\nzB4B/uicWwZ8E3jQ6xVWg9dG0p2Ky2tIjo/RRHci0q1iow9Xf50x+vPHKmsb2LK3kk17Ktlc6r9t\nKq3khZU7mV2QE/TYQtXLa0Qbx+bi7zp85P7rW2y/C5wcnOjaRwtriUhPkxQfw5jcFMbkpnxuvwt2\nTwNPT+jl1Sv5KmrUw0tEeoXuGqvT81qmeonico1BERFpSQmlE2obGik9WKtp60VEWlBC6YTdFbUA\nmsdLRKQFJZROKK5oXqlRJRQRkWZKKJ2wq3mlRpVQREQOUULphGINahQR+QIllE7wldeQ0ieWxDj1\nuhYRaaaE0gka1Cgi8kVKKJ1QXF6jSSFFRI6ghNIJKqGIiHyREkoHVdc1UlZVrxKKiMgRlFA6yKce\nXiIirVJC6SBf8xgUDWoUEfkcJZQO8h1aqVElFBGRlpRQOqh5LfnsfkooIiItKaF0UHFFDelJcSTE\ndu9azSIiPZ0SSgf5Kqo1h5eISCuUUDrIV16jBnkRkVYooXRQcUW1VmoUEWmFEkoHHKxt4EBNg9aS\nFxFphRJKBzT38NKgRhGRL1JC6YDDY1BUQhEROZISSgdo2hURkaNTQumA4vIazDSoUUSkNUooHeCr\nqCYzOZ7YaL1tIiJH0jdjB/gqatTDS0TkKJRQOqC4XGNQRESORgmlnZxz/hKKRsmLiLRKCaWd9lc3\nUFXXqGnrRUSOQgmlnYoPdRlWCUVEpDUhSShmdq+ZrTOzIjN73sxSvf2Xm9nKFrcmM5vQyvVpZrbA\nzDZ6P/sHO+ZdzSs1qoQiItKqUJVQFgBjnXMFwAbgDgDn3JPOuQnOuQnAN4DNzrmVrVz/I2CRc24k\nsMi7H1TNJZRclVBERFoVkoTinHvdOdfg3V0CDGrltMuAp47yEOcDf/W2/wp8LbARfpGvvIboKCOz\nb3ywn0pEpFfqCW0o1wKvtrL/EuDvR7km2znn87Z3AdnBCKyl4opqsvvGEx1lwX4qEZFeKSZYD2xm\nC4EBrRy6yzn3onfOXUAD8OQR104Cqpxza471PM45Z2aujThuAG4AGDJkSPtfwBF85RrUKCLSlqAl\nFOfcmW0dN7OrgdnATOfckQnhUo5eOgHYbWY5zjmfmeUAJW3EMQeYA1BYWHjUxHMsvopqxg5M6ezl\nIiJhL1S9vGYBtwPnOeeqjjgWBVzM0dtPAP4FXOVtXwW8GIw4mzUPatS09SIiRxeqNpSHgL7AAq97\n8B9bHJsGbHfObWp5gZk9YmaF3t1fAWeZ2UbgTO9+0OyrrKO2oUnT1ouItCFoVV5tcc6NaOPYW8Dk\nVvZf32J7LzAzKMG1onlhLQ1qFBE5up7Qy6vHO7xSo0ooIiJHo4TSDs0rNQ5QlZeIyFEpobRDcXkN\nsdFGRpIGNYqIHI0SSjtkJMcxbWQmURrUKCJyVCFplO9trj9tGNefNizUYYiI9GgqoYiISEAooYiI\nSEAooYiISEAooYiISEAooYiISEAooYiISEAooYiISEAooYiISEDYF9e2Cl9mtgfYGuo4eoAMoDTU\nQfQgej8O03vxeXo//PKcc5nHOimiEor4mdky51zhsc+MDHo/DtN78Xl6PzpGVV4iIhIQSigiIhIQ\nSiiRaU6oA+hh9H4cpvfi8/R+dIDaUEREJCBUQhERkYBQQgkzZjbYzN40s7Vm9rGZfdfbn2ZmC8xs\no/ezv7ffzOx3ZvapmRWZ2UmhfQXBYWbRZvaRmc3z7g81s6Xe6/6HmcV5++O9+596x/NDGXegmVmq\nmT1jZuvM7BMzmxLJnw0z+573e7LGzP5uZgmR+tkIBCWU8NMA/MA5dwIwGfi2mZ0A/AhY5JwbCSzy\n7gN8BRjp3W4A/tD9IXeL7wKftLj/a+AB59wIoAy4ztt/HVDm7X/AOy+cPAi85pwbDYzH/55E5GfD\nzAYCNwOFzrmxQDRwKZH72eg655xuYXwDXgTOAtYDOd6+HGC9t/0wcFmL8w+dFy43YBD+L8ozgHmA\n4R+sFuMdnwLM97bnA1O87RjvPAv1awjQ+5ACbD7y9UTqZwMYCGwH0rz/63nAlyPxsxGom0ooYcwr\nkp8ILAWynXM+79AuINvbbv6larbD2xdOfgvcDjR599OBcudcg3e/5Ws+9H54xyu888PBUGAP8KhX\n/feImSURoZ8N59xO4D5gG+DD/3+9nMj8bASEEkqYMrNk4FngFufc/pbHnP9PrIjo3mdms4ES59zy\nUMfSA8QAJwF/cM6dCFRyuHoLiLjPRn/gfPyJNhdIAmaFNKheTgklDJlZLP5k8qRz7jlv924zy/GO\n5wAl3v6dwOAWlw/y9oWLLwHnmdkW4Cn81V4PAqlmFuOd0/I1H3o/vOMpwN7uDDiIdgA7nHNLvfvP\n4E8wkfrZOBPY7Jzb45yrB57D/3mJxM9GQCihhBkzM+DPwCfOuftbHPoXcJW3fRX+tpXm/Vd6PXom\nAxUtqj96PefcHc65Qc65fPwNrm845y4H3gQu9E478v1ofp8u9M4Pi7/YnXO7gO1mdpy3ayawlgj9\nbOCv6ppsZone703z+xFxn41A0cDGMGNmU4F3gNUcbjO4E387yj+BIfhnXL7YObfP+0V6CH9Rvwq4\nxjm3rNsD7wZmNh241Tk328yG4S+xpAEfAVc452rNLAF4An/b0z7gUufcplDFHGhmNgF4BIgDNgHX\n4P/DMiI/G2Z2N3AJ/t6RHwHX428ribjPRiAooYiISECoyktERAJCCUVERAJCCUVERAJCCUVERAJC\nCUVERAJCCUVERAJCCUUkyMxsgpmd0+L+eWb2o7au6cBj32JmiYF4LJGu0jgUkSAzs6vxT5H+nSA8\n9hbvsUs7cE20c64x0LGIqIQi4jGzfG/RqT95iy69bmZ9jnLucDN7zcyWm9k7Zjba23+Rt1jTKjNb\n7C3O9DPgEjNbaWaXmNnVZvaQd/5jZvYHM1tiZpvMbLqZ/cWL47EWz/cHM1vmxXW3t+9m/JMavmlm\nb3r7LjOz1V4Mv25x/UEz+42ZrQKmmNmvzL8IW5GZ3Recd1QiTqjnz9dNt55yA/LxT8Exwbv/T/zT\nbrR27iJgpLc9Cf+8TuCf8magt53q/bwaeKjFtYfuA4/hn+bD8M98ux8Yh/+PveUtYknzfkYDbwEF\n3v0tQIa3nYt/fqpM/DMLvwF8zTvm8E+pAv4p19dzuIYiNdTvvW7hcVMJReTzNjvnVnrby/Enmc/x\nlgY4FXjazFbiX4gqxzv8HvCYmX0T/5d/e7zknHP4k9Fu59xq51wT8HGL57/YzFbgn1tqDHBCK48z\nEXjL+WfPbQCeBKZ5xxrxz0AN/nU8aoA/m9l/4J+nS6TLYo59ikhEqW2x3Qi0VuUVhX8RpglHHnDO\nfcvMJgHnAsvN7OQOPGfTEc/fBMSY2VDgVmCic67MqwpLaMfjtlTjvHYT51yDmZ2Cf3bdC4Hv4J/W\nX6RLVEIR6SDnX7Bss5ldBP4lA8xsvLc93Dm31Dn3E/yrIw4GDgB9u/CU/fAvhlVhZtn413pv1vKx\nPwBON7MMM4sGLgPePvLBvBJWinPuFeB7+NeWF+kylVBEOudy4A9m9mMgFn87yCrgXjMbib9NZJG3\nbxvwI6967JcdfSLn3Coz+whYh38J2vdaHJ4DvGZmxc65GV535De953/ZOffiFx+RvsCL3nTsBny/\nozGJtEbdhkVEJCBU5SUiIgGhKi+RNpjZ7/GvM97Sg865R0MRj0hPpiovEREJCFV5iYhIQCihiIhI\nQCihiIhIQCihiIhIQCihiIhIQPx/ZEsr/H1wShQAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9pO9U0EuX1j4",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 760
        },
        "outputId": "ebc61854-2b1a-44dd-d27c-b4f6dff80a2a"
      },
      "source": [
        "depth, errors = np.arange(2, 20, 2), []\n",
        "for d in depth:\n",
        "  errors.append(\n",
        "      cross_val_score(XGBRegressor(max_depth=d),\n",
        "                      X,\n",
        "                      y,\n",
        "                      scoring='neg_mean_squared_error').mean()\n",
        "    )\n",
        "plt.plot(depth, errors)\n",
        "plt.xlabel(\"max_depth\")\n",
        "plt.ylabel(\"error\")"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[08:30:21] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:21] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:21] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:21] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:21] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:21] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:22] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:22] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:22] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:22] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:22] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:22] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:22] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:22] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:22] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:22] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:23] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:23] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:23] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:23] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:23] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:23] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:23] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:24] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:24] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:24] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n",
            "[08:30:24] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0, 0.5, 'error')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 12
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAELCAYAAAD+9XA2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xl8VPW9//HXJwsJhIR9CTsIyBIR\nJVXrruBSpSqtXa3VWq/13q7aW1tre7vaRWu1vfbWql1s9dfNVm1dQbq4VGvBCmQAQQMImbBjJhBC\nlvn8/pgTjDGBBGZyZnk/H4885sxZZj4JYd455/s936+5OyIiIocrL+wCREQkOyhQREQkKRQoIiKS\nFAoUERFJCgWKiIgkhQJFRESSIpRAMbObzWy1mS03swfMbGC7bbPM7Dkzi5jZCjMr7uT4wWa2yMzW\nBo+Devc7EBGRjsI6Q1kEVLj7LGANcD2AmRUA9wJXu/tM4HSguZPjvwAsdvcpwOLguYiIhCiUQHH3\nhe7eEjx9HhgTLJ8NLHf3ZcF+O9y9tZOXuBC4J1i+B7golfWKiMjBFYRdAHAF8NtgeSrgZvYEMAz4\njbvf1MkxI9y9NljeDIzo6sXN7CrgKoCSkpI506ZNS1rhIiK5YOnSpdvdfdjB9ktZoJjZk8DITjbd\n4O4PBfvcALQA97Wr52TgbUADsNjMlrr74q7ex93dzLocP8bd7wTuBKisrPQlS5YcyrcjIpKzzGxD\nd/ZLWaC4+7wDbTezy4H5wFx/Y0CxTcBT7r492OdR4FgS7STtbTGzcnevNbNyYGtSixcRkR4Lq5fX\nucB1wAXu3tBu0xPAUWbWL2igPw1Y2clL/Am4LFi+DHgolfWKiMjBhdXL63agFFhkZi+Z2R0A7r4L\n+D7wL+Al4EV3fwTAzO42s8rg+O8AZ5nZWmBe8FxEREIUSqO8u08+wLZ7SXQd7rj+ynbLO4C5qalO\nREQOhe6UFxGRpFCgiIhIUihQREQkKRQoGWxb/T6er94RdhkiIoACJaPd/pe1fOCu51m28fWwSxER\nUaBksuU1dbjDFx9YQUtrPOxyRCTHKVAyVGvcWVUb44hhJUSiMX75XLdGRhARSRkFSoaq3rabxuY4\n/3X6ZE6bOoxbFr7M5rrGsMsSkRymQMlQkWgMgJmjy/j6hTNpiTtffzgSclUikssUKBkqEq2jT0Ee\nRwzrz/ghJXzyzMk8umIzf12tcTJFJBwKlAxVVRNj+shSCvMT/4RXnXoEk4f358sPVbG3qbM5yURE\nUkuBkoHcnUi0jhmjBuxf16cgj29eVMGmXXv537+sDbE6EclVCpQMtGnXXmKNLcwcVfam9SdMGsK7\njx3DnU9Vs2ZLfUjViUiuUqBkoLYG+YrRA96y7YvnTaN/cQFfeqCKeLzLiSxFRJJOgZKBItE68vOM\naSNL37JtSP8irn/HNF5Yv5P7X9wUQnUikqsUKBkoEk3c0FhcmN/p9vfMGUvl+EF8+9FV7NzT1MvV\niUiuUqBkoEi0jpmj3nq5q01ennHjgqOob2zh24+u6sXKRCSXKVAyzLb6fWyJ7XtLg3xHR44s5cpT\nJvH7pZv4p0YkFpFeoEDJMJFoHcABz1DafGruZEYP7MuXHqyiqUWDR4pIailQMkxbD68ZBzlDAejX\np4CvXziTtVt3c/cz1akuTURynAIlw6yMxhg7uC8D+hZ2a/+500dw7syR/HDxWjbubEhxdSKSyxQo\nGSYSraOiG5e72vvKBTPIN+PLD1XhrntTRCQ1FCgZJNbYzPodDQdtkO+ofEBfrjlrKn97eRuPV21O\nUXUikusUKBlkVduQ9T08QwG4/MQJzCgv46t/jlDf2Jzs0kREFCiZpP0cKD1VkJ/HjQsq2Fq/j+8v\nWpPs0kREFCiZpCpax7DSIoaXFh/S8ceMG8Qlx4/jnn+sp6qmLsnViUiuU6BkkJXRWI/bTzr63DnT\nGFxSxBcfWEGrBo8UkSRSoGSIxuZW1m7dfdiBMqBvIV+eP53lm+q4758bklSdiIgCJWOs2VJPa9x7\n3GW4MxccPYqTJw/l5sdfZmusMQnViYgoUDJGVc2h9/DqyMz4xkUV7GuN841HNHikiCSHAiVDRKJ1\nlBYXMHZw36S83sShJXz89Mn8eVmUp9ZsS8prikhuU6BkiEg0xozyMswsaa959emTmDS0hC8/VEVj\nc2vSXldEcpMCJQO0tMZZvTnW6ZS/h6OoIJ9vXlTBhh0N/N9fX0nqa4tI7lGgZIDq7XtobI4fdg+v\nzpw4eSgLjhnNj//+Kq9s3Z301xeR3KFAyQA9mQPlUHzxvOn0LcznSw+u0OCRInLIFCgZIFITo6gg\njyOGlaTk9YeVFvH5d0zj+eqdPPDvmpS8h4hkv4KwC5CDq4rWMa28jIL81OX/B942jvuXbuLGR1Zx\n5rThDOzXJ2XvJSKHLh53Glta2dcc3/+4ryVOY3PrAR/PmTmCMYP6pbS2UALFzG4G3gk0Aa8CH3H3\n14Nts4CfAGVAHHibuzd2OP6rwH8Abf1dv+juj/ZO9b3L3VkZjTH/6FEpfZ+8POPGi47inbc/w3cf\nX8233zUrpe8nkm3icad6+27qG1ve9EH+puXOPuzbBUNXj/vaPW9uPbTL0pOGlWRnoACLgOvdvcXM\nvgtcD3zezAqAe4FL3X2ZmQ0Buhpr/VZ3/14v1RuaTbv2EmtsSUmDfEczRpVxxUkTuOvpdVw8Zwxz\nxg9O+XuKZDJ3Z9mmOh5eFuXRFbVE67o38oQZFBfkU1yYR1Enj/2LChhSkk9RYR7FBZ0/FhXkUVyY\n363HooJ8SoryU/zTCClQ3H1hu6fPAxcHy2cDy919WbDfjt6uLd20NcgnY8iV7vjMvKk8sryWGx6o\n4s+fPJnCFF5mE8lE7k4kGuPh5bU8siLKxp17Kcw3Tps6jM+cNZVhpUVdfqi3BUZhviX1nrJ0kQ5t\nKFcAvw2WpwJuZk8Aw4DfuPtNXRz3CTP7MLAE+Ky77+psJzO7CrgKYNy4cUktvDdU1cTIzzOOHFna\nK+9XUlTAVy+YyVW/WsrPnlnHx047olfeVySduTsvb6nnkeW1PLy8lnXb91CQZ5w0eSifOnMKZ88c\nyYC+hWGXGbqUBYqZPQmM7GTTDe7+ULDPDUALcF+7ek4G3gY0AIvNbKm7L+7wGj8GvgF48HgLiWB6\nC3e/E7gToLKyMuP6xEaidUwe1p/iwtSfrrY5e+ZI5k0fwW1PruX8WeUpv+4qkq5e2bqbh5dHeXh5\nLa9s3U2ewYlHDOVjp07inJkjGVSizivtpSxQ3H3egbab2eXAfGCuv3HzwybgKXffHuzzKHAs8KZA\ncfct7V7nLuDh5FWeXiLRGCdPHtrr7/vVC2Zw1vef4qt/Wsndl1X2+vuLhGX99j08sqKWPy+Lsnpz\nPWZw3ITBXHZRBe+oGMnQ/kVhl5i2wurldS5wHXCauze02/QEcJ2Z9SPRA+w04NZOji9399rg6QKg\nKsUlh2JrfSNb6/cxM8lDrnTHmEH9uOasKXzr0dU8EdnMOTM7O9kUyQ4bdzbwyIpaHl4e3T+y95zx\ng/jKO2dw3lHljCg7tFlSc01YbSi3A0XAoqBh6nl3v9rdd5nZ94F/kbic9ai7PwJgZncDd7j7EuAm\nM5sd7LMe+FgI30PK7Z9Dvhd6eHXmIydN5I8v1vDVP0U4efJQSorSoclNJDlq6/bubxN5aePrABw9\ndiBfOn865x1VzqiByRnZO5eE1ctr8gG23Uui63DH9Ve2W740RaWllZVBoMwIKVAK8/O4cUEF7/7x\nc9z25BpuOH9GKHWIJMvWWCOPrkiEyJINiX48M0eV8flzpzF/VjljB6u98HDoT840FonWMX5IP8qK\nw+s9Mmf8YD5w3Dh+9ux6FhwzJrRwEzlU23fv4/GqzTy8PMo/1+3EHaaNLOW/z57K+bNGMXFoaoY0\nykUKlDRWVROjYnT4H+CfP/dIFkY2c8ODK/jD1SeSl5eZ/efdnabWOEUFvddjTsLxekNTECK1PFe9\ng9a4c8SwEj515hTeeXQ5k4f3Tjf8XKNASVOxxmZe29nA+942NuxSGNivDzecP51rf7eMX//rNS45\nfnzYJfXIvpZW/vRSlLufXser23bzoRPG88kzJzNEvXWySqyxmYWRLTy8PMoza7fTEnfGD+nH1adN\nYv6sUUwbWZqVNxOmEwVKmgq7/aSjBceM5vdLNvHdx1Zz9oyRDCtN/w/juoZm7nthA794dj1b6/cx\nbWQpF8wexa+e38D9Szdx9WmT+OjJk+jbR2csmWr3vhaeXJkIkafWbKepNc7ogX356CkTeeesUcwc\nldxZTuXAFChpqq2HV28NuXIwZsY3F1Twjtue5sZHVnLb+48Ju6QubdzZwE+fWcfvlmykoamVU6YM\n5Zb3Hs3Jk4diZvzX6ZO56fHVfG/hGn71/AaumTeVi+eMSelozpI8za1xFka28OdlUf768lb2tcQZ\nWVbMpW8fz/xZ5cweO1AhEhIFSpqK1NQxvLQorc4EjhjWn6tPm8QP//IK76kcy0kh3HB5IMs2vs6d\nT1fz2Ipa8sy4YPYo/uOUSUwvf/NZ3uTh/bnzw5UsWb+Tbz26ii/8cQU/fWYdnz93GnOnD9eHUZpq\naY3z4EtRfrh4La/tbGBYaREfOG4c588qZ864QRnbtpdNLJdm6KusrPQlS5aEXUa3nHPrU4waWMzP\nP3Jc2KW8SWNzK+fc9hR5Zjz26VN6dUiYzsTjzl9Wb+XOp6t5Yd1OSosK+OAJ47j8xAmUDzj4fQTu\nzhORzXz38ZdZt30Px00czBfPm87ssQN7oXrpjnjceWRFLbc+uYbqbXuYOaqMa+ZN5Yxpw8lXiPSK\nYAisgw6ZoTOUNNTY3Mor23Zz1owRYZfyFsWF+Xzjwgo+/LMX+Mnfq/n0vCmh1NHY3MoD/67hrqer\nqd62h9ED+/Kl86fzvreNpbQH3azNjHMrypk7fQS/eeE1frB4LRf96FnOn1XOdeccyfgh6lIaFndn\n4cot3LpoDas31zN1RH/u+NAczpk5QmeRaUqBkoZe3lxPa9zTostwZ06dOox3Hj2KH/3tFS6Y3bv9\n+HfuaeLe5zfwy+fWs313ExWjy/jB+2dz3lHlhzXUfmF+Hpe+fQILjh3DnU9Vc9dT1TxRtVk9wkLg\n7vxtzTZuXbSG5ZvqmDS0hB+8fzbzZ43SGUmaU6CkoapgDpSZadIg35kvnz+dv63eypcfrOJXHz0u\n5X8xrt++h58+s47fL91IY3OcM6cN5z9OmcQJkwYn9b37FxVw7VlT+dDx47ht8dr9PcL+8/QjuOKk\nieoRlmL/eHU7tyxcw9INuxgzqC83XzyLBceMVoeJDKFASUORaIyy4gLGDErfsYSGlxVz3blH8uWH\nIvxpWZQLZ49Oyfss3bArcbawcjOFeXksOGY0V54ykSkjUntj2vCyYr614CiuOGkC3338ZW5+4mV+\n+dx6rj1rKhfPGau/lJNsyfqd3LJwDc9V72BkWTE3LqjgPXPG0qdAQZJJ1Cifhi780bP0K8zn11ed\nEHYpB9Qad971f89S83ojiz97WtImGGqNO4tWbuGup6tZumEXA/oWcukJ4/nwieMZXhrOqK//CnqE\n/fu115k6oj9feMc0zjhSPcIO1/JNr3PLwjX8fc02hvYv4uNnHMEHjhsXemcPebPuNsorUNJMS2uc\nmV95gktPGM+X5qf/YIxVNXVccPszfPD4cXzzoqMO67X2NrVy/4ub+OnT1azf0cDYwX258uRJvKdy\nDP36hH8y7e48XrWZ7z6+mvU7Gjg+6BF2tHqE9djqzTG+v3ANC1duYWC/Qq4+7Qg+/PbxafHvLG+l\nXl4Z6tVte9jXEmdmmjbId1QxegCXnziRn/9jHe8+dgzHjBvU49fYvnsfv3xuA796bj27Gpo5euxA\n/u/caZwzc2RaXVoyM95xVDnzZozg1y+8xg+eXMuFP3qW+bPK+Zx6hHXLK1t3c9uTa3hkRS39+yTa\nqz5y0oQe9cyT9KVASTORDGiQ7+jas6fy6Ipabnigij994qRuN6C+um03dz+9jj+8uInm1jjzpo/g\nqlMnUTl+UFpfSirMz+PDb5/AgmNGc9dT1dz19DqeiGzmkuPH86m5UxisaWHf4rUdDdy2eA0P/ruG\n4sJ8Pn76ZP7jlEkM6KcgySYKlDQTicYoLsxjUgYNqd2/qICvvHMG/3nfi/ziH+u58pRJXe7r7ryw\nbid3PV3Nk6u2UlSQx8VzxvDRkydyxLD+vVj14SstLuTas4/kkhPGc9uTa/jlc+v5w9JNXK0eYftF\nX9/L//7lFX6/ZCP5ecZHT57I1acdoW7YWUqBkmaqauqYNrIs47pJnlsxkjOOHMb3F63pdLa7ltY4\nj0c2c9dT1SzbVMfgkj58eu4ULn37+Iyfo3tEWTHfftcsrjhp4v4eYb96bgPXnjWVd88Zk1aX7XrL\n1lgj//e3V/l//3wNgEuOH8fHz5jMcE2lm9XUKJ9G3J1ZX1vIBUeP4sYFh9fAHYaNOxs469a/c/rU\n4dxx6RwA9uxr4XdLNvLTZ9axaddeJg4t4aMnT+Tdx47J2r/gX1iX6BH20sbXOXJEKV94xzROP3JY\nWl/GS5Ydu/fxk6eq+eVz62ludd5bOYZPnDmF0ZpON6OpUT4Dbdy5l/rGloxqP2lv7OB+fGruFG56\n/GV+96+NrN+xh3uf30CssYXK8YP48vwZzJs+Iuv/Yj9u4mAe+K8TeaxqMzc9vpqP/OJfnDAp0SNs\n1pjs7BFWt7eZu5+u5mfPrGNvcysXHTOaT8+doo4KOUaBkkbaGuTTdciV7rjy5Ek88GIN1/1hOWZw\n7syRXHnKJOaM73nvr0xmZpx3VDlntesRdsHtz/LOo0fxubOPZNyQ7Ji7fPe+Fn7+zDrufLqa+sYW\n5s8q5zPzpjJ5eGa1h0lyKFDSSFW0jvw8Y2qK7wJPpT4FefzvB4/h4WW1vKdyTM7/hdq+R9idT1Vz\n99PreLyqNhgjLHN7hO1tauWXz63njr+/yq6GZs6aMYJrz5r6lqkCJLcoUNJIJBpjyvD+GX+X8LSR\nZUwbqQ+W9kqLC/ns2UfyoaBH2D3/WM/9SzKvR1hjcyu/fuE1fvTXV9m+ex+nTR3GtWdN1c2dAihQ\n0kokGuPUKcPCLkNS6M09wlbv7xH2sdMmMXpgX/r1KaBvn3z6BV+J5QL6FuaH2vbU3Brn90s28b9/\nWUttXSNvnzSEOz50LJUTBodWk6QfBUqa2BprZFv9PmamyRzyklpTRpRy92Vv45/VO/jWY6v52p9X\nHvSYooK8IGgK3ho4ffLpV9i27o3tbfu+KaQK33p8V2HVNkviDxavYePOvcwZP4hb3nM0J6bZbJ2S\nHhQoaaJtDnkFSm45ftIQHvyvE9mwo4Hd+1poaGqloamFvU2tieXmVvY2tbBnXyt7mxPbGppa92/f\n29TK1vrGt6xrao33qI4+BXmUtA+nPvn0Lcxnc6yRDTsaqBhdxtc/UsHpU3Oj+7McGgVKmmjr4TVD\ngZJzzIwJSR4Zobk1zt7mRLjsCYIqEUiJgGpoFz6J4Gp5UyC1BdeYQX354nnTOXuGZkmUg1OgpIlI\nNMaEIf00SJ4kRWF+HoX5eZTp90l6UWaN75HFqqJ1GXtDo4gIKFDSQt3eZjbu3KvLXSKS0RQoaWCl\nGuRFJAsoUNJAJs6BIiLSkQIlDUSiMUaUFTGsNLOHcReR3KZASQMRNciLSBZQoIRsb1Mrr2zdTYXa\nT0QkwylQQrZ6c4y4wwydoYhIhlOghExDrohItgglUMzsZjNbbWbLzewBMxsYrL/EzF5q9xU3s9md\nHD/YzBaZ2drgMWNnb4pEYwzoW8iYQZoiVUQyW1hnKIuACnefBawBrgdw9/vcfba7zwYuBda5+0ud\nHP8FYLG7TwEWB88z0spoHTNHlWmcJBHJeKEEirsvdPeW4OnzwJhOdvsA8JsuXuJC4J5g+R7gouRW\n2DuaW+Os2lyvy10ikhXSoQ3lCuCxTta/D/h1F8eMcPfaYHkzMKKrFzezq8xsiZkt2bZt2+FVmmSv\nbttNU0tcXYZFJCukLFDM7Ekzq+rk68J2+9wAtAD3dTj2eKDB3asO9j7u7oAfYPud7l7p7pXDhqXX\nbIiRGjXIi0j2SNnw9e4+70DbzexyYD4wNwiF9t5P12cnAFvMrNzda82sHNh6WMWGJBKNUVyYx6Rh\n/cMuRUTksIXVy+tc4DrgAndv6LAtD3gvXbefAPwJuCxYvgx4KBV1plpVtI7p5WWhzhUuIpIsYbWh\n3A6UAouC7sF3tNt2KrDR3avbH2Bmd5tZZfD0O8BZZrYWmBc8zyjxuLMqGtPlLhHJGqHM2Ojukw+w\n7W/ACZ2sv7Ld8g5gbkqK6yUbdzVQv6+FCjXIi0iWSIdeXjmpan+DvAJFRLKDAiUkkWgdBXnG1JFq\nkBeR7HDQQDGzfDO7pjeKySWRaIzJw/tTVJAfdikiIklx0EBx91YSd61Lkrg7kWgdFaN1uUtEskd3\nG+WfNbPbgd8Ce9pWuvuLKakqy22t38f23U3q4SUiWaW7gdI24u/X261z4MzklpMbNIe8iGSjbgWK\nu5+R6kJySduQK9PLS0OuREQkebrVy8vMBpjZ99sGWTSzW8xMf14fokg0xsShJZQWF4ZdiohI0nS3\n2/DPgHoSQ6K8F4gBP09VUdmuKlrHDLWfiEiW6W4byhHu/u52z79mZp1NfCUHUdfQzKZde/ng8ePC\nLkVEJKm6e4ay18xObntiZicBe1NTUnaL1CYa5DXkiohkm+6eoVwN/LJdu8ku3hjtV3pAc6CISLY6\naKAEw8kf6e5Hm1kZgLvHUl5ZlopE6xhZVsyQ/kVhlyIiklTduVM+TmLuEtw9pjA5PBENWS8iWaq7\nbShPmtl/m9lYMxvc9pXSyrLQ3qZWXt22m5kackVEslB321DeFzx+vN06ByYlt5zstmpzjLir/URE\nslN321A+5O7P9kI9WS0SVYO8iGSv7rah3N4LtWS9ldE6BvQtZPTAvmGXIiKSdN1tQ1lsZu82M0tp\nNVkuEo1RMboM/RhFJBt1N1A+BvwO2GdmMTOrNzP19uqB5tY4q2vrNcKwiGSt7jbKDwAuASa6+9fN\nbBxQnrqyss8rW3fT1BpX+4mIZK3unqH8CDiBN2ZurEftKj3yRoO8zlBEJDt19wzleHc/1sz+DeDu\nu8ysTwrryjqRaB19C/OZOLQk7FJERFKiu2cozWaWT+LeE8xsGBBPWVVZKFITY3p5Kfl5apAXkezU\n3UD5IfAAMNzMbgSeAb6VsqqyTDzurKyN6XKXiGS17k4BfJ+ZLQXmAgZc5O6rUlpZFnltZwO797VQ\nMVoN8iKSvbrbhoK7rwZWp7CWrFUVTcyBojMUEclm3b3kJYchEo1RkGdMGdE/7FJERFJGgdILItEY\nU0aUUlSQH3YpIiIpo0BJMXcnUlNHhW5oFJEsp0BJsS2xfezY06Q75EUk6ylQUizS1iCvSbVEJMsp\nUFIsEo1hBtPLdYYiItlNgZJikWgdE4eU0L+o2z20RUQykgIlxapqYsxQ+4mI5AAFSgq93tBEzet7\ndUOjiOQEBUoKrQyGrNeQKyKSC0IJFDO72cxWm9lyM3vAzAYG6y8xs5fafcXNbHYnx3/VzGra7Xde\n738XB6chV0Qkl4R1hrIIqHD3WcAa4HpIDELp7rPdfTZwKbDO3V/q4jVubdvX3R/tnbJ7JhKNUT6g\nmMElmjpGRLJfKIHi7gvdvSV4+jwwppPdPgD8pveqSr5INKYbGkUkZ6RDG8oVwGOdrH8f8OsDHPeJ\n4JLZz8xsUFc7mdlVZrbEzJZs27btcGvttoamFqq37dblLhHJGSkLFDN70syqOvm6sN0+NwAtwH0d\njj0eaHD3qi5e/sfAEcBsoBa4pas63P1Od69098phw4Yd7rfVbatq64k7OkMRkZyRsrvt3H3egbab\n2eXAfGCuu3uHze/nAGcn7r6l3evcBTx86JWmxkoNuSIiOSaU27fN7FzgOuA0d2/osC0PeC9wygGO\nL3f32uDpAqCrM5nQRKIxBvUrZNSA4rBLERHpFWG1odwOlAKLgm6/d7Tbdiqw0d2r2x9gZnebWWXw\n9CYzW2Fmy4EzgGt6peoeSDTID8DMwi5FRKRXhHKG4u6TD7Dtb8AJnay/st3ypampLDmaW+O8vLme\nj5w0IexSRER6TTr08so6a7fspqk1rjG8RCSnKFBSoG0OlAo1yItIDlGgpEAkGqNfn3wmDikJuxQR\nkV6jQEmBSLSO6eVl5OWpQV5EcocCJcnicWelhlwRkRykQEmyDTsb2NPUSoWGXBGRHKNASbKqmkSD\nvHp4iUiuUaAkWSQaozDfmDqiNOxSRER6lQIlySLROqaOKKVPgX60IpJb9KmXRO5qkBeR3KVASaLN\nsUZ27GnSHCgikpMUKEkUqYkBmgNFRHKTAiWJItEYZjC9XIEiIrlHgZJEVdE6Jg4toaQolEGcRURC\npUBJopXBHCgiIrlIgZIku/Y0UfP6XrWfiEjOUqAkycraRIO8hlwRkVylQEmStiFXdIYiIrlKgZIk\nkWiMUQOKGVTSJ+xSRERCoUBJkki0jpmaoVFEcpgCJQkamlqo3r5Hl7tEJKcpUJJgVW0Md9RlWERy\nmgIlCSJRDbkiIqJASYJITYzBJX0oH1AcdikiIqFRoCRBVbSOmaPKMLOwSxERCY0C5TA1tcRZs6Ve\nU/6KSM5ToBymtVvraW51NciLSM5ToBymtgb5Cp2hiEiOU6AcpkhNHSV98pkwpCTsUkREQqVAOUyR\naIzp5WXk5alBXkRymwLlMMTjzqraGBUackVERIFyONbv2MOeplb18BIRQYFyWKp0h7yIyH4KlMMQ\nidZRmG9MGV4adikiIqFToByGldEYR44spU+BfowiIvokPETuTlVNHTPL1SAvIgIhBoqZ3Wxmq81s\nuZk9YGYDg/WFZnaPma0ws1Vmdn0Xx080s3+a2Stm9lsz69WpEmvrGtnV0MzM0Wo/ERGBcM9QFgEV\n7j4LWAO0Bcd7gCJ3PwqYA3zMzCZ0cvx3gVvdfTKwC/hoyituR0PWi4i8WWiB4u4L3b0lePo8MKZt\nE1BiZgVAX6AJiLU/1hLD+p6Vly6AAAAJ5UlEQVQJ3B+suge4KOVFtxOJ1mEG08sVKCIikD5tKFcA\njwXL9wN7gFrgNeB77r6zw/5DgNfbBdImYHRvFNqmqibGpKEl9OtT0JtvKyKStlL6aWhmTwIjO9l0\ng7s/FOxzA9AC3BdsOw5oBUYBg4CnzexJd68+xBquAq4CGDdu3KG8RKdWRuuonDA4aa8nIpLpUhoo\n7j7vQNvN7HJgPjDX3T1Y/UHgcXdvBraa2bNAJdA+UHYAA82sIDhLGQPUdFHDncCdAJWVld7ZPj21\na08T0bpGKtQgLyKyX5i9vM4FrgMucPeGdpteI9E+gpmVACcAq9sfG4TPX4GLg1WXAQ+luuY2bzTI\nq8uwiEibMNtQbgdKgUVm9pKZ3RGs/xHQ38wiwL+An7v7cgAze9TMRgX7fR641sxeIdGm8tPeKrwq\nWgeoh5eISHuhtSgH3X07W7+bRNfhzrad1265mkR7S6+LRGOMHtiXgf169dYXEZG0li69vDJKJFqn\nsxMRkQ4UKD20Z18L67bvUfuJiEgHCpQeWlUbw13tJyIiHSlQemh/Dy91GRYReRMFSg9FonUMKenD\nyLLisEsREUkrCpQeqqqJMWNUGYnhxEREpI0CpQeaWuKs3VqvBnkRkU4oUHpgzZZ6mltdQ66IiHRC\ngdIDKzXkiohIlxQoPVAVraN/UQHjB/cLuxQRkbSjQOmBSDTG9PJS8vLUIC8i0pECpZta486q2pgu\nd4mIdEGB0k3rd+yhoalVd8iLiHRBgdJNVTVtQ9brDEVEpDMKlG5aGY3RJz+PKSP6h12KiEhaUqB0\nUyQa48iRpRTm60cmItIZfTp2g7tTpTlQREQOSIHSDdG6Rl5vaFagiIgcgAKlGyJtDfKj1SAvItIV\nBUo3RKIx8gymj9QZiohIVxQo3VBaXMCZ04bTt09+2KWIiKStgrALyARXnjKJK0+ZFHYZIiJpTWco\nIiKSFAoUERFJCgWKiIgkhQJFRESSQoEiIiJJoUAREZGkUKCIiEhSKFBERCQpzN3DrqHXmNk2YMMh\nHj4U2J7EcpJFdfWM6uoZ1dUz6VoXHF5t49192MF2yqlAORxmtsTdK8OuoyPV1TOqq2dUV8+ka13Q\nO7XpkpeIiCSFAkVERJJCgdJ9d4ZdQBdUV8+orp5RXT2TrnVBL9SmNhQREUkKnaGIiEhSKFBERCQp\nFCgHYWZjzeyvZrbSzCJm9umwa2rPzPLN7N9m9nDYtbQxs4Fmdr+ZrTazVWb29rBrAjCza4J/wyoz\n+7WZFYdUx8/MbKuZVbVbN9jMFpnZ2uBxUJrUdXPw77jczB4ws4HpUFe7bZ81MzezoelSl5l9MviZ\nRczspnSoy8xmm9nzZvaSmS0xs+NS8d4KlINrAT7r7jOAE4CPm9mMkGtq79PAqrCL6OAHwOPuPg04\nmjSoz8xGA58CKt29AsgH3h9SOb8Azu2w7gvAYnefAiwOnve2X/DWuhYBFe4+C1gDXN/bRdF5XZjZ\nWOBs4LXeLijwCzrUZWZnABcCR7v7TOB76VAXcBPwNXefDfxP8DzpFCgH4e617v5isFxP4sNxdLhV\nJZjZGOB84O6wa2ljZgOAU4GfArh7k7u/Hm5V+xUAfc2sAOgHRMMowt2fAnZ2WH0hcE+wfA9wUa8W\nRed1uftCd28Jnj4PjEmHugK3AtcBofQs6qKu/wS+4+77gn22pkldDpQFywNI0e++AqUHzGwCcAzw\nz3Ar2e82Ev+h4mEX0s5EYBvw8+BS3N1mVhJ2Ue5eQ+KvxdeAWqDO3ReGW9WbjHD32mB5MzAizGK6\ncAXwWNhFAJjZhUCNuy8Lu5YOpgKnmNk/zezvZva2sAsKfAa42cw2kvh/kJIzTQVKN5lZf+APwGfc\nPZYG9cwHtrr70rBr6aAAOBb4sbsfA+whnMs3bxK0SVxIIvBGASVm9qFwq+qcJ/ryp1V/fjO7gcTl\n3/vSoJZ+wBdJXLpJNwXAYBKXxz8H/M7MLNySgMSZ0zXuPha4huAKQrIpULrBzApJhMl97v7HsOsJ\nnARcYGbrgd8AZ5rZveGWBMAmYJO7t53F3U8iYMI2D1jn7tvcvRn4I3BiyDW1t8XMygGCx16/VNIV\nM7scmA9c4ulx49oRJP4wWBb8/o8BXjSzkaFWlbAJ+KMnvEDi6kGvdxjoxGUkfucBfg+oUT4MwV8X\nPwVWufv3w66njbtf7+5j3H0Cicblv7h76H9xu/tmYKOZHRmsmgusDLGkNq8BJ5hZv+DfdC5p0Fmg\nnT+R+E9P8PhQiLXsZ2bnkriseoG7N4RdD4C7r3D34e4+Ifj93wQcG/zuhe1B4AwAM5sK9CE9Rh+O\nAqcFy2cCa1PyLu6urwN8ASeTuPywHHgp+Dov7Lo61Hg68HDYdbSrZzawJPiZPQgMCrumoK6vAauB\nKuBXQFFIdfyaRDtOM4kPw48CQ0j07loLPAkMTpO6XgE2tvvdvyMd6uqwfT0wNB3qIhEg9wa/Yy8C\nZ6ZJXScDS4FlJNqA56TivTX0ioiIJIUueYmISFIoUEREJCkUKCIikhQKFBERSQoFioiIJIUCRURE\nkkKBIpJmzGz9oQ7HbmaXm9moZLyWSE8pUESyy+UkxioT6XUKFJEumNmEYKKkX5jZGjO7z8zmmdmz\nwURYxwVfzwUjK/+jbciZYDKvnwXLRwWTevXr4n2GmNnCYEKmuwFrt+1DZvZCMDHST8wsP1i/28xu\nDY5ZbGbDzOxioBK4L9i/b/AynzSzF81shZlNS+XPTHKbAkXkwCYDtwDTgq8PkhjG4r9JjHi7GjjF\nEyMr/w/wreC4HwCTzWwB8HPgY971WFhfAZ7xxIRMDwDjAMxsOvA+4CRPTIzUClwSHFMCLAmO+Tvw\nFXe/n8SQN5e4+2x33xvsu93djwV+HNQtkhIFYRcgkubWufsKADOLkJhV0c1sBTCBxGRF95jZFBJj\nvhUCuHs8GKV3OfATd3/2AO9xKvCu4LhHzGxXsH4uMAf4VzACel/eGIU4Dvw2WL6XN0aS7UzbtqVt\n7yOSCgoUkQPb12453u55nMT/n28Af3X3BcEEbH9rt/8UYDeH3qZhwD3u3p3JkA40KF9bza3o/7yk\nkC55iRyeAUBNsHx528pgKuQfkjj7GBK0b3TlKRKX0jCzdwCDgvWLgYvNbHiwbbCZjQ+25QFtr/lB\n4JlguR4oPYzvR+SQKVBEDs9NwLfN7N+8+a//W4EfufsaEsOHf6ctGDrxNeDU4JLau0jM3YK7rwS+\nBCw0s+XAIqA8OGYPcJyZVZGY3+LrwfpfAHd0aJQX6RUavl4kA5nZbnfvH3YdIu3pDEVERJJCZygi\nvcTMPgJ8usPqZ93942HUI5JsChQREUkKXfISEZGkUKCIiEhSKFBERCQpFCgiIpIU/x/+OFf5ZO0s\n+AAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    }
  ]
}