2021/seminars/sem13_rf.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "contemporary-rouge",
   "metadata": {},
   "source": [
    "## ИАД: Введение в анализ данных – 2021\n",
    "\n",
    "## Семинар 13: Случайный лес"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "compressed-costume",
   "metadata": {},
   "source": [
    "### 1. Bias-Variance decomposition"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "legendary-variable",
   "metadata": {},
   "source": [
    "Вспомним, что функцию потерь в задачах регрессии или классификации можно разложить на три компоненты: смещение (bias), дисперсию (variance) и шум (noise). Эти компоненты позволяют описать сложность алгоритма, альтернативно сравнению ошибок на тренировочной и тестовой выборках. Хотя такое разложение можно построить для произвольной функции потерь, наиболее просто (и классически) оно строится для среднеквадратичной функции в задаче регрессии, что мы и рассмотрим ниже. \n",
    "\n",
    "Пусть $(X, y)$ – некоторая выборка. Обучим интересующий нас алгоритм на этой выборке и сделаем предсказания на ней. Обозначим предсказания как $\\hat{y}$. Тогда \n",
    "\n",
    "$$\n",
    "\\mathrm{bias} := \\mathbb{E}(\\hat{y}) - y.\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\mathrm{variance} := \\mathbb{E}[\\mathbb{E}(\\hat{y}) - \\hat{y}]^2\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\mathrm{noise} := \\mathbb{E}[y - \\mathbb{E}(y)]^2\n",
    "$$\n",
    "\n",
    "Ожидаемую среднеквадратичную ошибку на тренировочной выборке можно разложить как\n",
    "\n",
    "$$\n",
    "\\mathrm{E}[y - \\hat{y}]^2 = \\mathrm{bias}^2 + \\mathrm{variance} + \\mathrm{noise}.\n",
    "$$\n",
    "\n",
    "**Задание для самых смелых:** покажите, что это разложение корректно. Проверьте себя [здесь](https://github.com/esokolov/ml-course-hse/blob/master/2020-fall/lecture-notes/lecture08-ensembles.pdf).\n",
    "\n",
    "**Техническое замечание:** все математические ожидания в разложении выше берутся по объектам тренировочной выборки, то есть это разложение верно для среднеквадратичной ошибки на тренировочной выборке, которую иногда называют MSE for estimator. Тем не менее, нам интересна и величина ошибки на ненаблюдаемых данных, которую иногда называют MSE for predictor. В этом случае математическое ожидание ошибки следует брать по ненаблюдаемым объектам. Для решения этой проблемы зачастую предполагается, что тренировочная и тестовая выборка имеют одинаковое распределение, и математическое ожидание берётся по всевозможным вариациям тренировочной выборки. Суть разложения при этом не изменится, однако запись его станет более громоздкой. Посмотреть на это можно [здесь](https://towardsdatascience.com/mse-and-bias-variance-decomposition-77449dd2ff55). \n",
    "\n",
    "Заметим, что так как на практике мы считаем оценки математических ожиданий и зачастую имеем доступ к тестовой выборке, то проблем с расчётом **оценок** MSE for estimator и MSE for predictor не возникает."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "gross-dover",
   "metadata": {},
   "source": [
    "Разберёмся с интерпретацией компонент. \n",
    "\n",
    "- $\\mathrm{Bias}$ – показывает отклонение среднего ответа алгоритма от ответа идеального алгоритма. $\\mathrm{Bias}$ отражает ошибку модели, возникающую из-за простоты модели. Высокое смещение обычно является показателем того, что модель недообучена.\n",
    "\n",
    "\n",
    "- $\\mathrm{Variance}$ – показывает разброс ответов алгоритмов относительно среднего ответа алгоритма. Показывает, насколько сильно небольшие изменения в обучающей выборке скажутся на предсказаниях алгоритма. $\\mathrm{Variance}$ отражает ошибку модели, возникающую из-за чрезмерной сложности модели. Высокая дисперсия обычно является показателем того, что модель переобучена.\n",
    "\n",
    "\n",
    "- $\\mathrm{Noise}$ – ошибка идеального классификатора, естественный неустранимый шум в данных. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "amber-witch",
   "metadata": {},
   "source": [
    "Посмотрим наглядно на примере полиномиальной регрессии."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "documented-silver",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlsAAAGbCAYAAADzxVVYAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAACMpklEQVR4nOzdd1xV9f/A8ddhyQZREUUBJw622xyYiaWWexRmZu7R+FXf6tu3st0369vSzK2puVdlucWcORFEBReouFAEQTac3x9XbuIEufeeC7yfj4cP5dxzz+cNR/HN5/M577eiqipCCCGEEMI4LLQOQAghhBCiPJNkSwghhBDCiCTZEkIIIYQwIkm2hBBCCCGMSJItIYQQQggjstI6gAepWrWq6uPjo3UYZcbNmzdxcHDQOgxxG7kn5knui/mRe2J+5J6U3IEDB66qqlrtzuNmnWz5+Piwf/9+rcMoMyIiIggNDdU6DHEbuSfmSe6L+ZF7Yn7knpScoigJ9zouy4hCCCGEEEYkyZYQQgghhBFJsiWEEEIIYUSSbAkhhBBCGJEkW0IIIYQQRiTJlhBCCCGEEUmyJYQQQghhRJJsCSGEEEIYkSRbQgghhBBGJMmWEEIIIYQRSbIlhBBCCGFEkmwJIYQQQhiRJFtCCCGEEEZkpXUAQgghyqbVhxKZtD6WCymZ1HS1482uvvQK9tQ6LCHMjiRbQgghSmz1oUTeWRlNZl42WOSRmALvrIwGkIRLiDvIMqIQQogSm7Q+lszcfGxrLsGh7reg5JKZm8+k9bFahyaE2ZFkSwghRIldSMnEwvY81s7RWFinYF15l/64EKKoip1sJZ+Gi1FaRyGEEGVOTVc7KlXbiJpnT97NulSqGgEWur1bQoiiKm6ypaowvw9seFfrSIQQoswZ2C4fK8dYcpI7kH25B4plJg7VtvNmV1+tQxPC7FTcZEtRIOg5OPMXXE/QOhohhChTom4uwcHKlaoFj6Nm18QqsxmVquzkMV8brUMTwuxU3GQLIPBZQIHDi7SORAghyoy9F/fy96W/GRc8kl1vPcWZL7rz63Mfo5LPtKhpWocnhNmp2MmWa22o2xEiF0JBgdbRCCGE2VNVlcmRk3G3c2eA7wD98drOtenbsC8r4lZw7sY5DSMUwvxU7GQLIGgwpJyFhB1aRyKEEGZv14VdHLpyiJEBI6lkWanIa6MCRmFtac3kyMkaRSeEeZJkq1F3qOQMhxZqHYkQQpg1VVWZfGgyNR1q0qdBn7ter2ZfjcGNB/PHmT84nnxcgwiFME+SbNnYg18fOLoGsm5oHY0QQpitiHMRHLl2hFGBuhmsexnqNxRnG2e+P/i9aYMTwoxJsgW6pcS8TIhZpXUkQghhlgrUAqZETsHLyYun6z193/OcbZwZ7j+c7Ynb2X9pvwkjFMJ8SbIFUKs5VG2o2ygvhBDiLpsSNhF7PZbRgaOxtrj3rFahZxs9i7udO98d/A5VVU0UoRDmS5ItuFVzKxzO/Q1XT2gdjRBCmJX8gnymRE6hrktdutXp9tDzba1sGRM0hsikSCLORRg9PiHMnSRbhQIHgWIps1tCCHGHP+P/5HTqacYGjcXSwrJY7+lVvxc+zj58f+h78gvyjRyhEOZNkq1CTh5Q/wk4vBjkG4MQQgCQV5DHT4d/omHlhnTx7lLs91lZWDE+eDwnU06y9sxaI0YohPmTZOt2weGQdhFObdE6EiGEMAu/nfqNhBsJjAsah4VSsv8yunh3oUmVJkw5NIWc/BwjRSiE+ZNk63YNnwI7Nzi0QOtIhBBCc7n5ufx0+CeaVmlKp9qdSvx+C8WCV0Je4cLNCyyLW2aECIUoGyTZup2VDQQMgNg/ICNZ62iEEEJTq06u4sLNC4wPHo+iKI90jTY12tDKoxXTo6ZzM/emgSMUomyQZOtOQeGQnwPRy7WORAghNJOdn820qGkEVQvisZqPPfJ1FEXhlZBXSM5K5uejPxswQiHKDkm27lQjADz85alEIUSFtix2GVcyrjAheMIjz2oV8q/mzxNeTzAvZh7JWbJqICoeSbbuJWgwXIyEyzFaRyKEECaXmZfJzOiZtPRoScsaLQ1yzQnBE/TXFaKikWTrXvz7g4W1NKcWQlRIi48v5lrWNcYHjzfYNeu61qVnvZ4sPr6Yi+kXDXZdIcoCSbbuxaEK+D4FUUsgP1fraIQQwmRu5t5k9pHZPFbzMYLdgw167TGBY1BQ+PHwjwa9rhDmTpKt+wkeDBlXIW691pEIIYTJLDi6gJTsFIPOahWq4ViDQY0G8eupXzmVcsrg1xfCXEmydT/1OoOjh2yUF0JUGKnZqcyLmUdo7VD8qvoZZYzh/sOxs7Ljh0M/GOX6QpgjSbbux9IKAgfqZrbSr2gdjRBCGN3PR38mLTeN8UGGn9UqVNm2MkObDmXz2c1EJUUZbRwhzIkkWw8SNBjUfN3eLSGEKMeuZ11nwdEFhHmH4evma9SxhjQZgputG98e/BZVVY06lhDmQJKtB6nWEGq10D2VKN8QhBDl2JyYOWTmZTI2aKzRx7K3tmdUwCj2XdrH7gu7jT6eEFqTZOthgsIh6RhcOKh1JEIIYRRXM6+y6NgiutXtRj3XeiYZs3/D/ng6evLtwW8pUAtMMqYQWpFk62H8+oCVndTcEkKUW7OiZ5FbkMuYwDEmG9Pa0ppxQeM4lnyMDQkbTDauEFqQZOthbF2g8dNwZDnkZmkdjRBCGNSlm5dYGruUZ+o9g7ezt0nH7lanGw0qN2Dyocnkq/kmHVsIU6qwydbqQ4k89sUW6ry9lse+2MLqQ4n3Pzk4HLJS4fjvpgtQCCFMYEbUDAooYFTgKJOPbWlhySvBr5BwI4E96XtMPr4QplIhk63VhxJ5Z2U0iSkZqEBiSibvrIy+f8Ll0wFcakvNLSFEuZKYnsjKkyvp26Avno6emsTQoVYHgt2D+TP1TzLzMjWJQQhjq5DJ1qT1sWTm5lGp+hps3LYBkJmbz6T1sfd+g4UFBD4Lp7ZC6nkTRiqEEMYz7fA0LLBghP8IzWJQFIVXQ14lNT+VRccXaRaHEMZUIZOtCymZQAGKZSaVqv+JTdUNgHrr+H0EPQeocFi+GQghyr6EGwn8eupXBvgOoLpDdU1jCakeQlO7psyMnklqdqqmsQhhDBUy2arpagdYknVhIDkpzalUbQuV3NdSw9X2/m9yqwPe7SDyF6m5JYQo86YenoqNpQ0v+b+kdSgAPO36NOk56cyNmat1KEIYXIVMtt7s6oudtSVgQfbFPuQkt8Wmyg58m64nv+ABT8QEh0PyaTgrRfiEEGXXqZRT/HH6DwY1GkRVu6pahwOAp40n3ep2Y8HRBSRlJGkdjhAGVSGTrV7Bnnzexx9PVzsULKiSPYAO7oPYn/wn7+58l7yCvHu/sUlPsHGUmltCiDLtx8gfsbOy48WmL2odShHjgsaRV5DHtKhpWocihEFZaR2AVnoFe9Ir+PanbzozI8qd7w99T1ZeFl92+BIbS5uib7JxgKa94MgqeOq/UMnRlCELIUSpHU8+zoaEDYwKGEVl28pah1NEbafa9GvYj+VxyxnSZAhezl5ahySEQRhkZktRlNmKolxRFOXIfV5XFEX5XlGUk4qiRCmKEmKIcQ1tRMAI3m75NpvPbublrS/f+zHkoMGQexOOrjF9gEIIUUpTIqfgZOPEkKZDtA7lnkYFjsLa0prJkZO1DkUIgzHUMuJc4MkHvP4U0ODWr5HAVAONa3DhjcP5sO2H7ErcxdhNY7mZe7PoCV6twa2e1NwSQpQ5R64eIeJcBC80eQFnG2etw7mnqnZVGdx4MH+e+ZPjyce1DkcIgzBIsqWq6l9A8gNO6Qn8rOrsAVwVRalhiLGNoU+DPnzR/gsOXTnEiA0jij6KrCi6MhAJO3Wb5YUQooyYfGgyrpVcGdxksNahPNCLfi/iUsmF7w5+p3UoQhiEohqojIGiKD7A76qq+t3jtd+BL1RV3XHr483AW6qq7r/HuSPRzX5RvXr1ZosXLzZIfI8iKiOKOUlzqG5dnXHVx+Fk6QRApayrtN4zggTvfsTXCdcsvjulp6fj6Cj7yMyJ3BPzVBHvy+ms03xz+Rt6uvbkCZcntA7nLnfek82pm1mdspqXq79MA9sGGkZWcVXEfyel1alTpwOqqja/87ipNsgr9zh2zyxPVdXpwHSA5s2bq6GhoUYM68FCCaX5hea8suUVpt+Yzsywmf8U/0v6BZ+kXfi88BNYWGoW4+0iIiLQ8usl7ib3xDxVxPsyf/18qthW4d3u72JnZad1OHe58560zmvNrlW72FawjeEdh6Mo9/pvRBhTRfx3YiymKv1wHqh928e1gAsmGrtU2tZsy09dfiIpM4kX1r3A+bRb7XqCw+HGeTjzl7YBCiHEQ/x98W/2XtrLcP/hZplo3YutlS1jA8dyOOkwEecitA5HiFIxVbL1KzDk1lOJrYFUVVUvmmjsUmtWvRkzw2aSnpvOC+te4HTqafDtDrYuslFeCGHWVFVl8qHJuNu709+3v9bhlEjP+j3xcfbh+0PfP7jgtBBmzlClHxYBuwFfRVHOK4rykqIooxVFGX3rlD+A08BJYAYw1hDjmpJfVT9md51NXkEeL657kdi0BPDvD8d+g8wUrcMTQoh72nlhJ5FJkYwKGEUly0pah1MiVhZWTAiewMmUk6w9s1brcIR4ZIZ6GvFZVVVrqKpqrapqLVVVZ6mq+pOqqj/del1VVXWcqqr1VFX1v9fG+LKgYeWGzHtyHtYW1ry4/kWi6rSGvCyIWal1aEIIcZfCWS1PR0961++tdTiPpIt3F5pUacKUQ1PIyc/ROhwhHkmFbNdTGj4uPsx7ah4uNi6MOPQV+zx8pX2PEMIsbT23lZhrMYwK0BUKLYsUReHVkFe5cPMCy+KWaR2OEI9Ekq1H4Onoybyn5uHh4MEY+xx2JB+BpFitwxJCCL0CtYApkVPwdvbm6XpPax1OqbSp2YZWNVoxPWr63YWmhSgDJNl6RO727sx5cg51nOswoXo1Nu/+UuuQhBBCb0PCBuKuxzE6cDRWFmW/De6rIa+SnJXMz0d/1joUIUpMkq1ScLN1Y1a3eTSxsOf15L/5/eSvWockhBDkF+QzNXIq9Vzq8ZTPU1qHYxB+Vf3o4t2FeTHzSM56UMMSIcyPJFul5GzjzIxmb9MsK5t/7/yP7CkQQmjujzN/cDr1NGODxmJpJkWXDWF88Hgy8zKZGT1T61CEKBFJtgzAvtHTTElTaWfhyEe7P+LnGJnmFkJoI7cgl6mHp+Jb2ZcnvM2vLU9p1HWpS6/6vVh8fDEX08tMqUYhJNkyCEtrbAMG8t2ZOLp4dmDS/klMOzwNQ/WdFEKI4vrt1G+cSzvH+ODxWCjl71v8mMAxKCj8ePhHrUMRotjK379ErQSFY12Qy5cOTXim3jNMjpzMtwe/lYRLCGEyufm5TDs8Df+q/nSs1VHrcIzCw8GDZxs9y6+nfuVUyimtwxGiWCpusrVwIfj4gIWF7veFpayVVb0J1AzGKnIRHz/2MQMaDmD2kdl89vdnFKgFhohYCCEeaOWJlVy4eYFxQePKdePm4f7Dsbey54dDP2gdihDFUjGTrYULYeRISEgAVdX9PnJk6ROuoHC4HI3FpWj+0/o/DG06lMWxi3l/5/vS10sIYVRZeVlMj5pOiHsIbWu21Toco3K1dWVo06FsPruZw0mHtQ5HiIeqmMnWu+9CRkbRYxkZuuOl4dcXLG3g0EIUReH/mv0fYwPHsubUGt7a/ha5+bmlu74QQtzHsrhlXMm8wvjg8eV6VqvQ802ex83WjW8PyHYNYf4qZrJ19mzJjheXvRs06g7RSyEvG0VRGBM0hjeav8H6+PW8FvEa2fnZpRtDCCHukJGbwczombTyaEULjxZah2MS9tb2jAoYxf7L+9l1YZfW4QjxQBUz2fLyKtnxkggaDJnXIfZP/aEXmr7Ae63fY9v5bYzbPI6M3IwHXEAIIUpm0fFFJGclMz54vNahmFT/hv3xdPTku4Pfyd5YYdYqZrL16adgb1/0mL297nhp1esETjUhsuj+rwG+A/i03afsu7SPURtHkZaTVvqxhBAVXnpOOnNi5tDOsx1B7kFah2NS1pbWjAsax7HkY2yI36B1OELcV8VMtsLDYfp08PYGRdH9Pn267nhpWVhC4CA4uQluFC2690y9Z/iq41ccuXaEl9a/xPWs66UfTwhRoS04toDU7FTGB1WsWa1C3ep0o0HlBvxw6AdyC2RfrDBPFTPZAl1iFR8PBQW63w2RaBUKCge1AKIW3/VSF+8ufNfpO06nnmbY+mEkZSQZblwhRIWSmp3KzzE/06l2J5pWbap1OJqwtLDkleBXOJt2llUnVmkdjhD3VHGTLWOqWh9qt4ZDC3WlJe7QoVYHfuz8I4npiQxdN5QL6Rc0CFIIUdbNi5lHWm4a44LGaR2KpjrU6kCwezA/Hf6JzLxMrcMR4i6SbBlLcDhcOwHn993z5ZY1WjK9y3SuZ13nhXUvkHAjwcQBCiHKsuSsZBYeW0hXn674uvlqHY6mFEXh1ZBXScpM4pdjv2gdjhB3kWTLWJr2Bmt7OLTgvqcEuQcxq+sssvOyGbpuKCeunzBhgEKIsmzOkTlk5WcxNnCs1qGYhZDqIXSo1YFZR2aRmp2qdThCFCHJlrFUcoImPeHISsi5f6mHxlUaM+fJOSgoDFs/jJhrMSYMUghRFl3NvMri44vpXqc7dV3rah2O2Xg5+GXd05lH5mgdihBFSLJlTEHhkJMGx39/4Gn1XOsx78l52FvZM3z9cA5dOWSiAIUQZdHM6JnkFuQyOnC01qGYFV83X7rV7cbCYwu5knFF63CE0JNky5i8HwNX7wcuJRaq7VybeU/No6pdVUZtHMWei3tMEKAQoqy5dPMSS2OX0rN+T7ycDVCIuZwZFzSOPDWPaYenaR2KEHqSbBmThYVuduvMX5Dy8FZAHg4ezHlyDp6OnozbNI6IcxFGD1EIUbZMj5qOisqogFFah2KWajvVpn/D/qw4sUIePBJmQ5ItYwt6Vvd75KJinV7Vripzus6hQeUGvLb1NdbFrzNicEKIsuR82nlWnVhF3wZ9qelYU+twzNbIgJHYWNow5dAUrUMRApBky/hcvaBOB137noLi9e5ytXVlZthMAqoF8NZfb0mhPiEEAD8d/gkLxYIR/iO0DsWsVbWryvNNnufP+D85du2Y1uEIIcmWSQQPhpQESNhZ7Lc42jgy9YmptPJoxfu73mfR8eLNjAkhyqf41Hh+O/0bAxsNpLpDda3DMXtDmw7FpZIL3x36TutQhJBkyyQa9YBKznc1p34Ye2t7JneeTKfanfjs78+YFT3LSAEKIczd1MNTqWRZiZf8XtI6lDLBycaJEf4j2Jm4k32X7l1cWghTkWTLFGzswa8PHF0D2Wkle6ulDV+Hfs1TdZ7i24Pf8sOhH1Dv0QJICFF+nbh+gj/P/MmzjZ6lil0VrcMpMwb6DqS6fXW+PfitfN8UmpJky1SCBkNuBsSUfP+VtYU1n7f7nD4N+jA9ajqT9k+SbxxCVCBTD0/F3tqeF5u+qHUoZYqtlS1jg8YSlRTF1nNbtQ5HVGCSbJlKreZQtaGuOfUjsLSw5IM2HxDeOJz5R+fz4e4PyS/IN3CQQghzczz5OBsTNvJ8k+dxtXXVOpwy55l6z+Dj7MP3B7+X75lCM5JsmYqi6GpundsDV08+0iUsFAveavEWI/xHsOLECt7d+S55BXkGDlQIYU6mHJqCk40Tzzd5XutQyiQrCyteDnmZU6mn+P30g7t5CGEskmyZUuAgUCxLvFH+doqi8HLIy7wS8gprT6/ljW1vkJOfY8AghRDmIiopiojzEQxtOhRnG2etwymznvB6gqZVmjIlcop8vxSakGTLlJw8oP4TcHgxlHI6e7j/cN5u+Tabz27m5S0vk5mXaaAghRDmYkrkFCpXqkx443CtQynTFEXh1WavcvHmRZbGLtU6HFEBSbJlasHhkHYBTpV+s2Z443A+avsRuy7sYsymMWQVZBkgQCGEOThw+QC7LuximN8wHKwdtA6nzGtdozWta7RmetR0bube1DocUcFIsmVqDZ8COzeIfHhz6uLo3aA3/+3wXyKvRDL58mRSs1MNcl0hhHZUVWXyoclUtavKwEYDtQ6n3Hgl5BWuZ1/n55iftQ5FVDCSbJmalQ3494fjayEj2SCXfKrOU/wv9H8k5iQybP0wrmVeM8h1hRDa+PvS3+y/vJ/h/sOxs7LTOpxyw6+qH128uzA3Zi7JWYb5/itEcUiypYXgcMjPgSMrDHbJx70eZ5T7KM7eOMvQdUO5dPOSwa4thDCdwlmt6vbV6dewn9bhlDsTgieQnZ/NjKgZWociKhBJtrRQIxCq+8MhwywlFmpk14ifuvxEUmYS/dcMps2kpdR5ey2PfbGF1YcSDTqWEMI4tidu53DSYUYGjKSSZSWtwyl36rjUoVf9XiyJXcKF9AtahyMqCEm2tBIcDhcj4XKMQS/brHoznvf+nOtZadyo/B2KzRUSUzJ5Z2W0JFxCmDlVVZkSOQVPR0961++tdTjl1ujA0Sgo/Bj5o9ahiApCki2t+A8AC+tHrij/IL9sV8lIGAlKAXbe08Eik8zcfCatjzX4WEIIw9lybgtHrx1ldOBorC2ttQ6n3PJw8OC5xs/x2+nfOHn90YpMC1ESkmxpxaEK+D4JUUsgP9egl76QkklBtgeZ54ZiYZWOTeVd+uNCCPNUoBYw+dBkfJx96FG3h9bhlHsv+b2EvZU9Pxz6QetQRAUgyZaWggZDxlWIW2/Qy9Z01T29VJBVi7y0Rti47QQlW39cCGF+NsRv4GTKScYEjsHKwkrrcMo9V1tXXvR7kS3ntnA46bDW4YhyTpItLdV/Ahyrl6p9z7282dUXO2tLALKvPo5ilYF91b282dXXoOMIIQwjryCPKZFTqO9anyfrPKl1OBXG4MaDqWJbhW8PfIuqqlqHI8oxSba0ZGkFAQN1M1vpVwx22V7Bnnzexx9PVzvULC8ssxvi4rGLJ/2rGGwMIYTh/HHmD+JvxDM2aCwWinxbNhV7a3tGBY5i/+X97LqwS+twRDkm/6q1FjwY1HyIMmy/rl7Bnux8+3HOfNGdmc+8Q3redVaeWGnQMYQQpZdbkMvUyKk0dmtMZ6/OWodT4fRr0A9PR0++O/gdBWqB1uGIckqSLa1V8wXP5rqlRCNNYzf3aE6Iewizj8yWjvdCmJlfT/7K+fTzjAsaJ7NaGrC2tGZ88HiOJR9jQ/wGrcMR5ZT8yzYHweFw5ShcOGS0IUYFjOJyxmV+PfWr0cYQQpRMTn4OP0X9REDVADrU6qB1OBVWtzrdaFi5IT8c+oHcAsM+HS4ESLJlHvz6gpWtwTfK365NzTb4VfFjZvRM8gryjDaOEKL4VpxYwaWblxgXPA5FUbQOp8KyUCx4JeQVzqadZdWJVVqHI8ohSbbMga0LNH4aopdBbpZRhlAUhZEBI0lMT+TPM38aZQwhRPFl5WUxI2oGIe4htKnRRutwKrz2nu0JcQ/hp8M/kZknNQmFYUmyZS6CwiErFWLXGm2I0Nqh+Fb2ZUb0DPIL8o02jhDi4ZbGLiUpM4nxweNlVssMKIrCKyGvkJSZxC/HftE6HFHOSLJlLup0BJfaRmnfU0hRFEYEjOBM6hk2nt1otHGEEA+WkZvBrCOzaFWjFS08WmgdjrglpHoIHWt1ZNaRWaRmp2odjihHJNkyFxYWEPgsnNoCqcZrGP2E1xPUcanD9Kjp8pizEBr55fgvJGclMz5ovNahiDtMCJ5Aek46c47M0ToUUY5IsmVOgp4DVDi8yGhDWFpYMsJ/BCeun2DbuW1GG0cIcW9pOWnMOTKH9p7tCXIP0joccQdfN1+61+3OwmMLuZJhuGLTomKTZMucuNUB73ZGrbkF8FSdp6jlWIvpUdOlRYUQJrbg6AJu5NxgXPA4rUMR9zE2aCx5ah7TDk/TOhRRTki3U3MTHA6rx8DZPeBtnCeUrCysGO4/nIm7J7L7wm7aerY1yjhCiKJSs1P5+ejPdPbqTNMqTbUOp/SSz8DG9+HcXnCoeutXNbC/7c8O1Yq+ZuMIZv5AQG2n2vRv2J+lsUsZ0nQI3s7eWockyjhJtsxNk57wx5sQucBoyRbAM/WeYerhqUyLmkabmm3kaSghTGBezDxu5t5kbNBYrUMpnZwM2PEN7PwOLKygcQ/IToObSXA9AW5ehZy0e7/XyvYeyVgVal+6AZEXbiVrVf5J0qztTPu53TIyYCSrT65m8qHJTOo4SZMYRPkhyZa5sXGApr0gZjU89aXuYyOwtrRmmN8wPt/7Ofsv75cnooQwsuSsZBYcW0BXn640rNxQ63AejapCzCrY8B7cOA/+A6DLh+Bc8+5zczN1SVfGVd3vN5Nu/brj46TjkH6FevnZcHre3dexcQKHKkVnyezvM2tmXwUsrQ3yqVa1q0pLt16si1/Eyq0N8LCtz5tdfekV7GmQ64uKxSDJlqIoTwLfAZbATFVVv7jj9VBgDXDm1qGVqqp+ZIixy6WgwXBoARxdc2vTvHH0adCH6VHTmR41XZItIYxsdvRssvOzGRM0RutQHs3lGPjzLYjfDh7+0Hfmg2ffre3Atbbu18OoKts3/0n7kEZw89ptiVkSZNz2ccpZSDyo+7N6n1qBdpVvS8bu83vh63aVdU+C38PqQ4ls3tMYSy97bKqtJ/GcJ++sjAaQhEuUWKmTLUVRLIEpQBfgPLBPUZRfVVU9esep21VV7VHa8SoEr9bgVldXc8uIyZatlS1Dmw7l6wNfczjpMIHVAo02lhAVWVJGEotjF9Ojbg/qutTVOpySybwOWz+HfTPB1hm6/w+aDQULS8ONoSjkW9nrvu+5FePrU1AAWSm3zZzda9bsKiTFQsJOyEgG7vEwkGJx25LmbbNl9lWJ3X6N9rm2nEhuylX3fVjYniMzqzaT1sdKsiVKzBAzWy2Bk6qqngZQFGUx0BO4M9kSxaUouiRryye6DahudYw21ADfAcw6MosZUTOY3Hmy0cYRoiJafSiRSetjuVppMTaVc6ln1UvrkIqvIB8OzYfNH+kSrubDoNO7YO+mdWS62Sh7t1uxFGNJNj8PMpNvS8hu+/32Zc4Lh3Qza9mpvAVgA2kZCh3VWjRxWcuRrNFcSJFWPqLklNI++q8oSj/gSVVVh9/6+Hmglaqq4287JxRYgW7m6wLwhqqqMfe53khgJED16tWbLV68uFTxlVWVspJovWcECd79ia8TXqz3pKen4+joWOKx1qeu5/eU3/lXjX9R26YYU/6i2B71ngjjMsV9ScnMJfF6JjcKrrM4/xsaKsGEWvXGs7IdrnaG2VdkLM6px2lwYjpO6adIcWnCiQYjuelovB/6wLz+rSgFuZxLTMQqJxW7/Bv8pq4gwSqbkepQkh2b4OvhpHWIJmFO96Ss6NSp0wFVVZvfedwQM1v3eoztzgzuIOCtqmq6oijdgNVAg3tdTFXV6cB0gObNm6uhoaEGCLGMSvoFn6u78ekw7b77Cm4XERHBo3y9muU0Y9vybRyyOcTzoc8/QqDifh71ngjjMsV9eeyLLSSmWFLJ4y+sXWD/qSfYl2eJp6slO9827tiPLO0SbPwAohaDU03oOwtXv760MMHTyub2b2X1oUTeWRlNZm4+zs5ZqJ6/4pH4Iy6NlxH6WEutwzMJc7snZZkhipqeB26fDqmFbvZKT1XVG6qqpt/68x+AtaIoVQ0wdvkWFA6p5+CMcSu9O9k48WzjZ9mYsJGT108adSwhKooLKZkolmlYu+4nN6UFap6r/rjZycvRlXH4oRnErIT2r8P4feDfz+xrYhlLr2BPPu/jj6erHWnpISiqJTudbQg7/IquzIUQJWCIZGsf0EBRlDqKotgAg4Bfbz9BURQP5VYhJ0VRWt4a95oBxi7fGvUAWxddRXkjG9x4MHZWdsw8MtPoYwlREdR0tcPadT+KUkDO9ceKHDcrJzbB1Da64qQ+7WHsHuj8PlSS5aNewZ7sfPtxznzWlw6127Glag3UpFhYMVy3p02IYip1sqWqah4wHlgPHAOWqqoaoyjKaEVRRt86rR9wRFGUw8D3wCBV+sQ8nLUt+PWDY79BZopRh6psW5lBvoP488yfnL1x1qhjCVERvB5WH5vKe8m7WQ81pxoAdtaWvNnVV+PIbkk+DYuehYV9dfWzwpfDc4uhSj2tIzNLYT5hXMy+TnTo/0HcOtj0gdYhiTLEIL0RVVX9Q1XVhqqq1lNV9dNbx35SVfWnW3+erKpqU1VVA1VVba2q6i5DjFshBIdDXpZuat/IhjQdgrWFNTOjZXZLiNKq5p6AYn0dp5z2KICnqx2f9/HXvmxAzk3Y/DFMaQ1n/oInPtTNZjXoom1cZi60dihWFlZssLeFFiNg1w9wcL7WYYkyQirIm7uaIVCtsa7mVvNhRh2qql1V+jboy9LYpYwOHE1Nx3tUhRZCFMuyuGW42bqxacLLWBuoqnmpqKruh7YN78GNRAgYqEu0nGtoHVmZ4GzjTNuabdmYsJHXe/2Ocu0E/P6ari6Yz2MPv4Co0AwysyWMSFF0s1uJ+3UF+ozsRb8XQYHZR2YbfSwhyqtLNy+x7fw2etfvbR6J1qUjMLcHLB+ma2kzbD30mS6JVgmFeYdx4eYFjlw/Dv3nQmUfWDJYVw9RiAeQZKssCBgIiqWuhY+ReTh40LNeT1adWMWVjCtGH0+I8mjViVUUqAX0bdhX20AykmHtGzCtPVw5Cj2+hZERui4VosQ6eXXSLSUmbNC1+nluCagFsGgQZN3QOjxhxiTZKgsc3aFhV4haoquEbGQv+b9EvprPvJh7NIUVQjxQXkEey08s57Gaj1HbSaMiwQX5sH+2rpTD/lnQ/CWYcACav2jYNjsVjLONM21qtGFD/AZUVdU9TDBwPlw7qZs1lCcUxX1IslVWBIVD+mU4tdnoQ9V2qk33ut1ZFreM5Kxko48nRHmy/fx2rmRcoX/D/toEcHYPTA/V7SdybwKjtkP3r8yjzU45EOajW0qMuXarCUqdDtBtEpzcqNsPJ8Q9SLJVVjTsqmuYaoKlRNDNbmXlZTH/qDxtI0RJLItbRjW7anSo3cG0A9+4CCtHwuyukHEN+s2Gob+Dh59p4yjnOtW+tZQYv+Gfg82HQavRsGcKHJirWWzCfEmyVVZYWuv2bsX+qWuUamR1XeoS5hPGouOLSM1ONfp4QpQHiemJ7EjcQZ8GfbC2MNHG+Lxs2PEtTG4OMaug/Ru66u9+fSts9XdjcqnkQusardmQcGspsVDYp1CvM6x9Hc5s1y5AYZYk2SpLgsOhIBeil5lkuBH+I7iZe5Nfjv9ikvGEKOtWxK1AURT6NjDRxvgTG+HHNroCm3U6wLi/ofN7YONgmvErqDDvMBLTEzl67eg/By2toP8ccKsHS5/XFY0V4hZJtsqS6k2hRhBEmmYp0dfNl061O7Hg6AJu5t40yZhClFW5BbmsOrmK9p7tqeFo5JIK107BLwNh4a3eheEr4NlFuppPwuge93ocK8WK9Qnri75g66Krwg+6+5MlqwJCR5KtsiZ4MFyKhotRJhluZMBIbuTcYEnsEpOMJ0RZFXEugquZV427MT47HTZ9CD+2hvgd0OVjGLMbGjxhvDHFXVwqudCqZqt/nkq8nVtdGLhAN7O17EWTPEEuzJ8kW2WNX1+wtDFJc2oAv6p+PFbzMebFzCMzL9MkYwpRFi2NXYqHgwftPNsZ/uKqCtHLYXIL2PE/3feBCQfgsZfBysbw44mH6urdVbeUmHz07hd92kH3/+meHt/wrumDE2ZHkq2yxt4NGnWHqKWQl2OSIUcGjCQ5K5kVcStMMp4QZc3ZG2fZc3EPfRv0xdLQdawuRcOcbrDiJXCsBsM2QO+fwMnDsOOIEilcSizyVOLtmr0ArcfB3z/pap6JCk2SrbIoaDBkJkPcnyYZLqR6CM2rN2fOkTnk5JsmwROiLFl+YjmWiiV9GvQx3EUzknVPtk3rAFdj4envYMRW8GpluDHEI3Op5EKrGq1YH7/+7qXEQmEfQ4Mw+ONNOL3NtAEKsyLJVllUrxM41dQ1pzaRUYGjuJJ5hdUnV5tsTCHKgpz8HFafWE1o7VDc7d1Lf8GCfNg3C34Igf1zoMUI3ZJhs6FS/d3MhPmE3X8pEXT3q+8sqFIflg7RPdggKiRJtsoiC0sIHKSrWJx2ySRDtvJoRUC1AGYfmU1uQa5JxhSiLNh8djPXs68bZmN8wm6Y3hHW/h9U94PR26Hbl7o+fMLsPF77cSwVy/svJQLYOsOzi3Xft38ZAJnXTRegMBuSbJVVQeG6BqiHF5tkOEVRGBUwisT0RNaeXmuSMYUoC5bGLsXT0ZM2Nds8+kVuXIAVw2HOk5BxHfrPhRd+05V7EWbL1daVVjXu81Ti7dzq6J5QvJ4Ay4ZCvvzAWtFIslVWVa0PtVvrnkp80D9yA2rv2Z5Gbo2YGT2TfGm4KgSnU0+z//J++jXsh4XyCN9O87Jh+//gh+Zw9Ffo8C8Yvxea9pbq72VEV5+unE8/z7HkYw8+0bstPP0tnI6Ade+YIjRhRiTZKsuCw+FqHJzfb5LhFEVhZMBIEm4ksCHhAdPmQlQQy+OWY6VY0at+r5K/OW69rl7W5g91+zDH/Q2PvyvV38uYYi0lFgoeDG0nwL4ZsHeG8YMTZkOSrbKsSS+wsjNZRXmAzl6dqedSj+lR0ylQC0w2rhDmJisvizUn19DZuzNV7aoW/43XTsHCAbr9O4olDF4JgxbqlppEmaNfSryzV+L9PPEhNHwS/nwLTm01foDCLEiyVZbZOkOTnnBkJeRkmGRIC8WCEQEjOJlykq3n5BuFqLg2JmzkRs6N4m+Mz06HTRN1s1kJuyDsExizC+p3NmqcwvjCvMM4l3aO48nHH36yhSX0nQnVfGHZC3D1hPEDFJqTZKusCw6H7Btw/HeTDdnVpyteTl5MOzyteD/JCVEOLY1dio+zDy09Wj74RFWFqGUwuTns+Ab8+ulKObSdINXfy4nHvW4tJRZ3e0Ulp1tPKFrreihmJBs3QKE5SbbKOu924OoFh0y3lGhlYcVw/+EcSz7GjsQdJhtXCHMRdz2OyKRI+jXsh/Kgjexpl2He07ByODhWh5c2Qu+p4FTddMEKo6tsW5mWHi0f/lRikTd565aPU8/pZrjkCcVyTZKtss7CQlcG4sxfVMq6YrJhe9TtQQ2HGkyLktktUfEsi12GtYU1z9R75v4nJR6E6aGQeOCf6u+1HzILJsqsMJ8wzqadJfZ6bPHf5NVa93fjzF/w579M9mS5MD1JtsqDwGcBFY9LW0w2pLWlNcP8hnE46TD7Lu0z2bhCaC0jN4PfT/9OmE8YlW3vU2w0ejnMeUq3P+elDbeqv8u32/Kss1fn4j+VeLug5+CxV3X9E+UJxXJL/vWXB5W9oU4HXbJVYLonBHs36E01u2pMj5pusjGF0Nr6+PWk56bfe2N8QQFs/kjXNLpmiG42y8Pf9EEKk6tsW5kWHi2K/1Ti7Tp/AL7dYd1bcHKzcQIUmpJkq7wIGoxd1mVI2GmyIStZVmJo06H8felvIq9EmmxcIbS0NHYp9VzqEeIeUvSF7DRYEg7bv4aQF2DIGnCspk2QQhNhPmEk3Egg7npcyd5oYQF9poN7U1j2IiSVYClSlAmSbJUXjZ8mz9JeV1HehPo17EflSpWZFjXNpOMKoYWj145y5NoR+vv2L7oxPvkMzOyiK1T61CTdPhx50rDCKVxKXB+/vuRvruQIzy7S/b2RJxTLHUm2ygsbe664t4Oja3Q/YZuIvbU9Q5oOYUfiDmKuxZhsXCG0sCxuGbaWtvSo2+Ofg2f+ghmdIO0iPL8SWo2UVjsVlJutG809mrMxYeOjPTjkWhsG/QI3EmHpEMjLMXyQQhOSbJUjlzw6Q24GxKw26biDfAfhZOPEjCjZ3CnKr5u5N/nj9B909emKSyUX3cG9M+DnXuDgDiO3Qt1QLUMU97NwIfj46JbrfHx0HxtJmHcY8TfiS76UWKh2S3hmMsRvhz/ekCcUywlJtsqRG86+UKWByZcSHW0cGdx4MJvPbubEdamGLMqntafXkpGXQX/f/roZh99f0/1nWP8JGL4J3OpqHaK4l4ULYeRISEjQJS4JCbqPjZRwdfbqjIViUbr+sYEDof3rcHAe/P2T4YITmpFkqzxRFF1F+bO7df3XTCi8cTj2VvYyuyXKJVVVWRa3DN/KvgTYecL83rpH9du9pttnY+usdYjift59FzLuaGeWkaE7bgRV7KrQonqLkhU4vZdO/4FGPWD9v+HERsMFKDQhyVZ5EzAIFAuTz265VHJhUKNBrItfx5nUMyYdWwhjO3L1CMeTj9Pf4zGUmZ3g/D7oMwOemKirpSXM19mzJTtuAGE+uqXEEymlmOkvfEKx+q0nFK8cM1yAwuQk2SpvnGvoljUiF0FBvkmHHtJkCJUsKzErepZJxxXC2JbGLcXOwobum77StVUZ9icEDNA6LFEcXl4lO24A+qXEkhY4vZONg66HorWd7gnFm9cME6AwOUm2yqOgcEi7AKe3mnTYKnZV6NewH7+f/p3E9ESTji2EsdzITmXdqd/olpKMYzVfXaFSz2ZahyWK69NPwd6+6DF7e91xI6liV4Xm1Zs/WoHTO7nU0i1Vp12Cpc/LE4pllCRb5ZHvU2BXGQ6ZdikRYGjToVgoFsyOnm3ysYUwuJwMfl8xiCw1n/4ebWHoH7rZY1F2hIfD9Ong7a3b1+rtrfs4PNyow4Z5h3Em9QwnU06W/mK1mkOvH3VFq9e+Jk8olkGSbJVHVpXAfwAcXwuZ1006dHWH6vSu35tVJ1dx+eZlk44thEGlnkedHcaym2doWqkqTfstAGtbraMSjyI8HOLjde2U4uONnmgBdPY2wFOJt/PvBx3+BYcWwO4phrmmMBlJtsqr4HDIz9Y1xDWxYf7DKFALmBsz1+RjC2EQZ/+G6aFE3jzPSRtr+oeMl0KlokSq2lWlWfVmpd+3dbvQd6BJT9jwH4hdZ7jrCqOTZKu8qhEI1f11PwWZeMrZ09GTHnV7sDxuOVczr5p0bCFK7dACmNsdKjmxNKA7jtaOPFXnKa2jEmVQmHcYp1NPc/K6AZYSQfeEYq+fdN/fV7wEl48a5rrC6CTZKs+aD4WLkbDjG5MPPdx/ODkFOcw/Ot/kYwvxSPLzYN2/Yc048HmMlCGr2HBpD93rdsfe2v7h7xfiDk94P4GCYrilRAAbe92GeRtHWDQQbsoPtGWBJFvlWbNh4N8fNn8Ikb+YdGgfFx+6+nRl8fHFpGanmnRsIUos8zr80h/2TIFWYyB8BWsSt5FTkEP/hv21jk6UUUZZSgRwrgnP/gLpV2DJYMjLNuz1hcFJslWeWVhAzx91/drWjDd5FeIR/iPIyMtgwbEFJh1XiBJJioMZneHMdnjmB3jqC1QLS5bHLSewWiC+br5aRyjKsDCfME6lnuJUioG7eng2g15TdR1DfpcnFM2dJFvlnZUNDJgP1ZvousgnHjDZ0A0qN6CzV2cWHltIWk6aycYVothObISZnSErFV74DUKGALD/8n7ib8TLrJYotS7eXXRLiYae3QLw66PbNB+5EHZ9b/jrC4ORZKsisHWG8BXgUA0WDjBp38QRASNIy0ljSewSk40pxEOpKuz6AX4ZAJW9YWQEeLfRv7w0dilONk509emqXYyiXKhqV5WQ6iGG3bd1u45vQdM+sPEDOP6HccYQpSbJVkXhVB0GrwRUXRPd9CsmGbZplaa082zHzzE/k5Gb8fA3CGFsuVk0Ov6d7vH5xk/DsPXgWlv/8rXMa2w6u4me9XpiayV1tUTphXmHcTLlJKdTThv+4oqiK3haMwhWDIdLRww/hig1SbYqkqr14bmlcDMJFvaDbNMs7Y0KGMX17OssjzN9zS8hiki7BHO743F5K4T+G/rP0/Wfu83qk6vJK8iTJURhMIVLiesT1htnAGs7GLRIt4qxaJDJfpgWxSfJVkVTq7nuP5hLR2CJafpsBbkH0cqjFXNj5pKdL0/NCI0kHoTpneDKUY40fRtC37qrUGmBWsDyuOU0q96Muq51NQpUlDfV7KsR7B5snH1bhZxr6EpC3Lyqe0IxN8t4Y4kSk2SrImoYBs98r2tUvWacroWFkY0MGElSZhKrTqwy+lhC3CV6Ocx5Ciys4KUNXK3W5p6n7bm4h/Pp52VWSxhcmI8RlxIL1QyG3j/Bub/ht1fkCUUzopS6I7kRNW/eXN2/f7/Rrv/hbzEcvXDDaNc3tZSUFFxdXYt9fu+0RQxKn8evDv1Y6DzceIEBKirxNl+Sp1ynfvYnKFgZdTxzUdJ7IgxLUQsYmDaP3jeXcNTGn/+5vkuapet978s566lkWJygQfZ/scDa9AFXYOX930ouKZyo9BbV8p6mWn4Po47VJ+0XBqb/zC9OL7LGceAjX6c83ZMmNZ354OmmRh9HUZQDqqo2v/O4zGxVYKscB7HevgfP3FxOt5vGnXFSUKia141cJZkUyz1GHUsIALuCm7xx/UN631zCJvun+MTtM9IsXe97fi4ppFkcxjW/rSRawuCsccVOrccNS+NNIBRa6fgsO2xDeS5tDs2zdhl9PPFwFXpmq7yJiIggNDS0ZG8qyIdlL8Cx36DfbPDra5TYAFRVZdDaQaTnpLOm1xqsLMr/7NYj3RNResmnYdFzcDUOnvovtBheZH/Wve7LtMPTmBw5md97/463s7eJAxYV4d/KwmML+WLvF6zptYa6LkbeE5ibqevxeeU4DFsHNQJKfImKcE8MTWa2xL1ZWEKfmeDVFlaNhtPbjDaUoiiMDBjJ2bSzrIuXjvXCSE5vgxmPQ/oleH4VtBxx10b4O+UX5LPixApa1WgliZYwmie8ngAw7kb5QtZ2MOgXsHOFRc9C2mXjjynuS5ItAda2uj5bbvVgcThcijbaUJ1qd6K+a31mRM2gQDX+xnxRgagq7J2hqyPnWB1GbIG6HYues3Ah+PjAgQO63xcuBGDnhZ1cvHlRNsYLo6ruUJ0QdyMWOL2Tk4fuCcXMZFj8nDyhqCFJtoSOXWUYvFxXp2VBP7ieYJRhLBQLRgaM5HTqaTaf3WyUMUQFlJej6w/3xxvQoAu8tBHc7limWbgQRo6EhFt/txMSdB8vXMiy2GVUsa3C47UfN33sokIJ8wnjxPUTnEk9Y5oBawRCn+mQuB9+HS9PKGpEki3xD5daMHgF5GXCgr6QkWyUYcK8w/Bx9mF61HTMec+gKCNuXoX5veDAHGj3f7qlE1vnu897913IuKOLQUYGl/77H/5K/IveDXpjbSkb44VxmXQpsVDjp6Hz+xC9DLZ/ZbpxhZ4kW6Io98bw7GJIOavrG5dj+BY7lhaWDPcfzvHk42xP3G7w64sK5NIRXaHSxAO6vYdPfKDbh3gvZ8/e8/CK+hmoqkrfBsZ7OESIQtUdqusKnJpqKbFQu/+DgIGw5RM4+qtpxxaSbIl78G4LfWfC+f2wfBjk5xl8iG51u+Hp6Mm0w9Nkdks8mmO/wawwKMiFF/+AgIfst/LyuutQngWs7FSVtp5tqeVUy0iBClFUmHcYcdfjiE+NN92gigJPfw+1WsCqUXAh0nRjC0m2xH00eQa6fwVxf8La1wy+zm9tYc0wv2FEXY1iz0WpuyVKQFVh25e6liTujWFkBHg2e/j7Pv0U7O2LHNrWqipXnC1kY7wwqSe8by0lmnp2y9r21hOKbreeULxk2vErMIMkW4qiPKkoSqyiKCcVRXn7Hq8riqJ8f+v1KEVRQgwxrjCyFsOh/Rtw8GeI+MLgl+9Vvxfudu5Mj5pu8GuLcirnJiwbCls/hYBBMHSt7omr4ggPh+nTwftWaQdvb5aNbou7nTsda3V88HuFMCAPBw+CqgWZdt9WIUd3eG4xZKXqEq7cTNPHUAGVOtlSFMUSmAI8BTQBnlUUpckdpz0FNLj1ayQwtbTjChN5/D8QPBi2fQH7Zxv00jaWNrzo9yL7L+/nwOUDBr22KIdSzsHsrnB0DXT5WNcDztq2ZNcID4f4eGjWjPPRO9ilnqFPwz4VosCuMC9hPmHEXo8l4YZxnvx+IA9/3VaRC4dg9Vh5QtEEDDGz1RI4qarqaVVVc4DFQM87zukJ/Kzq7AFcFUWpYYCxhbEpCvT4FhqEwdrX4fhag16+b8O+uNm6MSNqhkGvK8qZs3tgRiddSZLnlsJjLz+0UOnDrDixAkVRZGO80EQX7y6AiZ9KvF2jbvDERIhZqVuWF0ZliB/nPIFzt318HmhVjHM8gYt3XkxRlJHoZr+oXr06ERERBgixYkhPTzfa18vC4yWCLp3BYelQDgd+xA2Xxga7dnvb9qy5sIa56+fiU8nHYNc1B8a8JxWFx8VNNIybSpatO0cCJpJxwQYuRJTqmilpKSw5t4Qmtk04vu84xzlumGDFI6uI/1bqVKrDyiMraZDcQJsA1EAaVe+ER8RnHElSuVqtdZGXK+I9MRZDJFv3+vHyzjnJ4pyjO6iq04HpoOuNKH2Zis/ofaxaN4dZYYQc+wJe2gDVfA1y2Ra5Ldi6fCsHrA8wNHSoQa5pLqS3WCnk58GG/0DsVKjbCfv+c2hpV9kgl/5m7TekFaQxpu0YOtTqYJBritKpiP9WzsacZdL+SdQNqYuX891Py5pE+7YwrQN+19dB/6JbriviPTEWQywjngdq3/ZxLeDCI5wjzJ1DVXh+JVjawPw+cMMwt9DB2oHBTQYTcS6C2ORYg1xTlHGZ12FhP/h7KrQeC+HLdV0ODGRH+g5qONTgsZqPGeyaQpRUmE8YoMFTibezqgQhQ3T7t5LitIujnDNEsrUPaKAoSh1FUWyAQcCdFdN+BYbceiqxNZCqqupdS4iiDKjso2vrk5Wqa+uTmWKQyz7X6DkcrB2YES17tyq8pFhdI+n4HfDMZHjyc7A03Ab2hBsJxGXF0bdBXyzvVwBVCBPwcPAgoFqAdvu2Cvn1A8UCohZrG0c5VupkS1XVPGA8sB44BixVVTVGUZTRiqKMvnXaH8Bp4CQwAxhb2nGFhmoEwsD5cDVO17jaAM1NXSq58Fyj59gQv4HTqacNEKQok+I2wMwnIDsNhv4OIc8bfIjlccuxwII+DfoY/NpClFSYdxjHko9x7sa5h59sLE7Vod7jELUUCgq0i6McM0idLVVV/1BVtaGqqvVUVf301rGfVFX96dafVVVVx9163V9V1f2GGFdoqF4n6DUVEnbAqpFQkF/qSw5uMhhbK1tmRs00QICiTFFV2PmdrkVUZW8YsRW8Wj/8fSWUk5/D6pOr8bf3p5p9NYNfX4iSCvPWLSWuT1ivbSABgyD1HJzdpW0c5ZRUkBePLqA/hH2iq3u07p1S12pxs3Wjf8P+/HHmD21/yhOmlZsFq0bDxvehSU8Yth5caz/8fY9gY8JGUrJTaOfYzijXF6KkajjWIKCqGSwlNuoONo5wWJYSjUGSLVE6bSdAm/Gwdxrs/LbUl3uh6QtYKpbMOjKr9LEJ83c9AeZ20+0V6fQu9J8LNg5GG25Z3DJqOdaioW1Do40hREmF+dxaSkzT8IdMG3to/Izuh2epKm9wkmyJ0uvysW6D5aaJELmoVJdyt3end4PerDm1hks3pW9XuZWXA9u/himtdBviBy6Ajv8qdaHSBzmVcooDlw/Qr2E/LBT51ifMh+YFTgsFDoTsGxD7h7ZxlEPyHUeUnoUF9PoR6nSEX8fDiU2lutwwv2GgwpwjcwwUoDAr8Tvgp3aw+SNo8ASM+xsaP230YZfHLcfKwope9XsZfSwhSqKmY038q/prWwICwKc9OHvC4SXaxlEOSbIlDMOqkm52wr0xLB0CiY/e67CmY02eqf8MK06s4GrmVQMGKTR18yqsGgNzu0Nepq7tzsAF4FLL6ENn5WWx5tQanvB6gip2VYw+nhAlFeYdxtFrR7VdSrSwBP/+cHITpCdpF0c5JMmWMBxbZ13xSYcqsHAAXDv1yJd6ye8lcgtymRczz4ABCk0UFMD+OfBDM4heBu1fh7F/Q8OuJgthffx60nLSGOA7wGRjClESXXx0S4kbEzZqG0jgIFDz4cgKbeMoZyTZEobl5AGDV4FaAAv6QvqVR7qMl7MXT9V5iiWxS7iedd3AQQqTuRQNs7vC769CdT8YsxM6v6/bjGtCy+KW4ePsQ/PqzU06rhDF5enoiV8VP+33bbk3Bo8AKXBqYJJsCcOrWh/Cl0H6ZVjYH7LTH+kyI/xHkJmXyYJjCwwcoDC67DRY/y5M6wjJp6HXT7oipQbqp1kSscmxHE46TL+G/VCMuAFfiNIK8wkj5loM59POaxtI4CC4cAj7mxrHUY5IsiWMo1Zz3WP8l6Jh6fO6p89KqJ5rPbp4d+GXY79wI+eG4WMUhqeqcPRX3VOGuyfrKsCP3wdBzxr1ScMHWRa3DBsLG3rW66nJ+EIUV+FTiZovJd5q31P98lZt4yhHJNkSxtOwKzz9HZzaAr9OeKSipyMDRpKem87i4zKlbfaux8MvA3XJtZ0bvLRJd//t3TQLKSM3g99P/06YTxiutq6axSFEcdRyqkXTKk21X0q81b6n+uVt0r7HQCTZEsYV8jx0+o9u/X/TxBK/vZFbIzrW6sj8o/PJyM0wfHyi9PJy4K+vYEprSNgJXT+DkRFQu4XWkfHnmT+5mXtTNsaLMiPMJ4wj146QmJ6obSABg7DNTtL9mxalJsmWML4Ob0Dzl3QV5vf8VOK3jwgYQUp2Cktjlxo+NlE6Z7bramZt+RgadIFxe6HNOLC00joyQLeEWN+1PkHVgrQORYhiKeyVuDFe46XERt3Js7SVjfIGIsmWMD5FgW6ToFEPWPc2HFlZorcHVgukdY3WzI2ZS1ZelpGCFCWSnqTrZzivB+RlwXPLYOB8cPHUOjK9mGsxxFyLoX/D/rIxXpQZtZxq0aRKE+0LnNrYk1StrW4PprTvKTVJtoRpWFhC35ng1RpWjYIzf5Xo7SMDRnIt6xorT5QsURMGVlgza3JziF5+q2bWHmgYpnVkd1kWuwxbS1t61OuhdShClEiYdxjRV6O5kH5B0zguV+8k7XsMRJItYTrWdvDsInCrC4vD4dKRYr+1hUcLQtxDmH1kNrn5uUYMUtzXpWiYHaarmeXhr1nNrOJIz0nnjzN/8GSdJ3G2cdY6HCFKJMzn1lKixk8lprj6SfseA5FkS5iWXWUYvAJsHHVFT1POFvutowJGcTnjMmtOrTFigOIu2Wmw7t+3amadgd7T4IXfNKmZVVxrT68lMy+TAQ1lY7woe2o71aaxW2Ptn0pULKR9j4FIsiVMz6WWLuHKy9QlXBnJxXpbm5pt8Kvix8zomeQV5Bk5SKGrmbUGJreEPT9CyBCYsF9X8NCM90CpqsqyuGU0cmuEX1U/rcMR4pGE+YQRdTVK86VEad9jGJJsCW1UbwKDFsH1BF1tppyHl3VQFIWRASNJTE/kzzN/miDICiz5jK76/9IhYF8FXtoIT3+rm5k0c1FXo4i9Hisb40WZ1tVb1ztU66VEad9jGJJsCe34PKbbNH9+H6x4CfIfPlsVWjuUhpUbMiN6BvkF+SYIsoLJy4a/JsGPreHsbuj6udnUzCquZbHLsLeyp3vd7lqHIsQjq+1sJkuJoG/fQ1Ks1pGUWZJsCW01eUZXFiL2D1j7fw+tMl84u3Um9Qybzm4yUZAVhL5m1ie66v/j9kKbsWZTM6s4UrNTWRe/jm51u+Fg7aB1OEKUSuFS4sX0i9oGcqt9D4dldutRSbIltNdyhK6EwMF5sO2/Dz39Ca8nqONSh+lR0ylQpZVEqaUnwcpRt2pmZetqZg342axqZhXX76d/Jzs/WzbGi3KhsMCp5jW3nKpDvc4QvUza9zwiSbaEeXj8PQgKh4jPdXWcHsDSwpIR/iOIux5H629/oM7ba3nsiy2sPqRxe4uypqAA9s+Gyc10m1/bv2G2NbOKQ1VVlsUuw6+KH42rNNY6HCFKzcvZS7eUqHWyBbqlxNRz0r7nEUmyJcyDouiaFtfvoltOPP7gIno5qYGouW6k261DRSUxJZN3VkZLwlVcF6NgVhf4/TXd5tcxu6Dze2ZZM6u4Dl45yKnUU9IHUZQrYT5hRCVFcenmJW0D8e0GNk6yUf4RSbIlzIelNQyYBzWCYPmLcPbv+576vw0nyb4aiqXdeaycdMVRM3PzmbReNnA+UHYarHsHpneE6/HQe/qtmlkNtY6s1JbFLcPR2pGuPl21DkUIg9EvJWq9Ud7GXrfHVtr3PBJJtoR5sXGA8GW6qsWLBkJS3D1Pu5CSSW5KCPlZNbGtuRQLuwT9cXEPqgoxq2FyC9gzFZoNvVUza6BZ18wqrutZ19kQv4EedXtgb112Z+eEuJOXsxeN3BqZx1JiwEBp3/OIJNkS5sehqq7oqYU1LOgDN+5+Eqemqx1gRebZF1HznLGvPRcLm8u3josikk/Dwn6w7AXd13b4JujxTZmomVVcv576ldyCXPr79tc6FCEMLsw7jMNJh7VfSvRpL+17HpEkW8I8udWBwcsh87ouUchKLfLym119sbO2RM13IuPsS6gFVth7z2ZEJ1dt4jVHedmwbRL82AbO7tHVzBoRAbWaax2ZQRVWjA+qFkTDymV/OVSIO5lLr0Qsbm/fc0XbWMoYSbaE+aoRCAPn6wrpLQ7XJQ+39Ar25PM+/ni62kGuG04pY7GrlM+KCx+QnFW89j/l2pm/YOpjsPVWzazx+8pczazi2ntpLwk3EmRjvCi3vJ298a3sq/2+LZD2PY9Iki1h3uo9Dr2mQvx2WDWqSI2XXsGe7Hz7cc580Z09bwxmetiPXLp5iTGbxnAz96aGQWso/QqsHAnznoaCXAhfoauZ5VxT68iMZlncMpxtnOni3UXrUIQwmjCfMCKTIrVfSnRvrPtBWAqclogkW8L8BfSHsE8gZhWsf+e+VeaD3YP5OvRrYpNjeWXLK2TnZ9/zvHKpIB/2zYTJzeHISujwpq5mVoMntI7MqK5mXmVzwmZ61u+JrZWt1uEIYTSFTyVuSjCDzhkBg+BipLTvKQFJtkTZ0HYCtB4Hf/8EO7+772kdanXg48c+5u9Lf/P2X29XjP6JFw/ramatff2fmlmP/wesy//DAqtPriZPzaNfw35ahyKEUfm4+NCwckPzeCrRvx8oljK7VQKSbImyI+wT8OsLmz544D/yp+s9zZvN32TT2U188vcnqA/pt1hmZd2AP9+G6aGQchb6zCg3NbOKo0AtYHnccppXb05dl7pahyOE0YV5h3HoyiEu37ysbSCO7rotHtK+p9gk2RJlh4WFbv9WnQ6wZpzuiZj7GNJ0CMP9h7M8bjk/HPrBhEGagKrqllSntNTN9DUbqtsAHzCgXNTMKq7dF3aTmJ4oG+NFhVH4VOKms2awlCjte0pEki1RtlhVgoELoVpjWDIEEg/e99SXg1+mb4O+zIiewYKjC0wYpBHpa2YNBYdq5bJmVnEti1tG5UqV6ezVWetQhDCJOi51aFC5gXk8lSjte0qk/D0HLso/W2ddDa5ZXWBhfwh5HqxsdYmYlS1Y2oCVLYpVJd5zDSHV7QT/3fdfXG5e4+laof+ca1npn/dYVQILS60/s/vLy9btVdv+ta7Y65NfQIsR5bKUQ3FcybhCxLkIhjQdgo2ljdbhCGEyYd5h/Bj5I1cyruBu765dIIXte2LWwFOTynRfVVOomN+pRdnn5AGDV8IvA2H3FMjPuedplsAXCoyt7s77MTNw2fIZHTKz7n1NC6uiSdu9EjIrW7CyuePjfxK8oudWotqVU3D85m2J4J3Xq1T0NYt7TDaf3qZrzn3tJDTpBU9+Xq5LORTHyhMryVfz6ddANsaLiiXMJ4wpkVPYmLCR8Mbh2gYTMBAiF+ra9/jLv8UHkWRLlF1VG8DLt5YRCwogPxvysnSzQPpfWVTKy+a7rOsMO/wNr1vaML3B8wTbVr91bs5t77n1e5Hr3HG9rNR/juff8d6CvLtCbApwtASfk6VN0YTM0krXMLqyj65mVjkv5VAc+QX5rDixgtY1WuPl7KV1OEKYVF2XutR3rc+G+A3aJ1s+7cG5FkQtkWTrISTZEuWDhQVY2N233IEjMLVWM15Y9wLjzixj3pPzaFC5gWFjyM+7laj9k+jt3b2dlsEBdydmt51zd6J3x/HgwdBmfIUo5VAcOxJ3cOnmJf7V4l9ahyKEJsJ8wpgaOZWkjCSq2VfTLhALC10dxJ3f6woqO2q4rGnmJNkSFUYVuypM6zKNIX8MYfTG0fzc7Wc8HT0NN4Clle6XjYP+UIZDAtQMMtwYgmVxy6hqV5XQ2qFahyKEJrp6d+XHyB/ZmLCR5xo/p20wAYNgxze69j2tx2gbixmTpxFFheLp6MlPXX4iKz+LkRtGci3zmtYhiRK4mH6R7Ynb6V2/N9YW1lqHI4Qm6rreWko0hwKn7o2kfU8xSLIlKpwGlRswpfMUrmRcYcymMaTnpGsdkiimFSdWoKqqVIwXFV6YdxgHLx8kKSNJ61CkfU8xSLIlKqQg9yD+F/o/Tlw/wctbX65YfRTLqNyCXFaeWMljno9R07FiP40pRJhPGCqqeRQ4lfY9DyXJlqiw2tdqz8ftPmbfpX289ddb5N3jaUJhPv469xdJmUkMaCgV44Wo51qPei71zKPAaWH7nqil0r7nPiTZEhVaj7o9eKvFW2w+u5mP93xcfvsolgNL45bibu9O+1rttQ5FCLMQ5hPGgcsHuJp5VetQdO17bpyHhB1aR2KWJNkSFd7gJoMZ4T+ClSdW8v2h77UOR9zDubRz7Lqwi34N+mFlIQ9RCwG6fVsqKpsSzGApsbB9z+ElWkdiliTZEgKYEDyB/g37MzN6Jj/H/Kx1OOIOK+JWYKFY0LtBb61DEcJs1K9cn7oudc3jqUQbe2jSE46ugZwMraMxO5JsCQEoisK7rd6li3cXJu2fxK+nftU6JHFLbn4uq06uomOtjng4eGgdjhBmxbyWEgdCTpqufY8oQpItIW6xtLDki/Zf0KpGK97f+T7bzm3TOiQBbD63meSsZPo37K91KEKYnTDvMArUAjYnbNY6FPBu90/7HlGEJFtC3MbG0obvOn2Hr5svr297nYOXD2odUoW3LHYZNR1q0rZmW61DEcLs1HetTx2XOuaxlFjYvufkZl37HqEnyZYQd3CwdmDqE1Op4VCD8ZvHE5sshfq0Ep8az95Le+nXsB+WFpZahyOE2VEUhTDvMPZf3m8eHTECBoGar2vfI/Qk2RLiHtxs3ZjWZRp21naM3jSa82nntQ6pQloetxwrxUo2xgvxAGE+t5YSz5rBUqK077knSbaEuI+ajjWZ9sQ0cvJzGLlxpHlsQK1AsvOzWX1qNZ28OlHVrqrW4Qhhthq4NsDH2cc8CpzCP+17rhzXOhKzIcmWEA9Qv3J9fnziR65mXmXMpjGk5aRpHVKFsTFhI6nZqbIxXoiHUBSFMJ8w9l3eZx5LiYXte6JkdquQJFtCPERgtUD+F/o/Tl4/yctbpI+iqSyLXUZtp9q0qtFK61CEMHv6pxLNYSnR0R3qd4aoZdK+5xZJtoQohnae7fi03accuHyAN7e9KX0Ujezk9ZMcvHKQ/g37Y6HItykhHqZh5YZmtpQ4UNr33Ea+iwlRTN3qduOtlm+x9dxWPtr9kfRRNKJlccuwtrCmZ/2eWociRJlgdkuJjbpL+57bSLIlRAmENw5nVMAoVp1cxTcHv9E6nHIpMy+T3079xhPeT+Bm66Z1OEKUGWa1lGhtJ+17blOqZEtRFDdFUTYqinLi1u+V73NevKIo0YqiRCqKsr80YwqhtXFB4xjQcABzjsxh7pG5WodT7qyPX09abppsjBeihPRLieZQ4BSkfc9tSjuz9TawWVXVBsDmWx/fTydVVYNUVW1eyjGF0JSiKPy71b/p6tOVrw98zZqTa7QOqVxZFruMOi51aF5dvlUIURKKotDFuwv7Lu0jOStZ63Ckfc9tSpts9QTm3frzPKBXKa8nRJlgaWHJZ+0+o3WN1nyw6wMizkVoHVK5cDz5OFFXo+jfsD+KomgdjhBlTlefruazlCjte/SU0mzyVRQlRVVV19s+vq6q6l1LiYqinAGuAyowTVXV6Q+45khgJED16tWbLV4sdTqKKz09HUdHR63DqFCyC7L5/vL3XMy9yFj3sdS3rV/kdbknJbPk2hL2pO/hk1qf4GDpYLRx5L6YH7knhqGqKh9f+Bg3KzfGVx9fqmsZ4p7Y3zxLy30TOFnvJc7XfqZU1yoLOnXqdOBeK3gPTbYURdkEeNzjpXeBecVMtmqqqnpBURR3YCMwQVXVvx4WdPPmzdX9+2WLV3FFREQQGhqqdRgVzvWs6wz5cwhXM68y98m5+Lr56l+Te1J8GbkZPL7scTp7debTdp8adSy5L+ZH7onhfH/we2Yfmc3WAVupbHvPrdTFYrB7Mq2D7vdRD/1vv8xTFOWeydZDlxFVVX1CVVW/e/xaA1xWFKXGrQFqAPecJ1RV9cKt368Aq4CWpflkhDAnlW0rM73LdBysHRi1cRTnbpzTOqQy6Y8zf3Az96ZsjBeilMJ8wshX881jKREg8Fm4eLhCt+8p7Z6tX4EXbv35BeCuncKKojgoiuJU+GcgDDhSynGFMCs1HGswvct08tQ86aP4iJbGLqVB5QYEVgvUOhQhyjTfyr54OXmZT4FTP2nfU9pk6wugi6IoJ4Autz5GUZSaiqIUPutZHdihKMphYC+wVlXVdaUcVwizU9e1Lj92/pFrWdcYvXE0N3JuaB1SmRFzNYZjycdkY7wQBlBY4HTvpb1cz7qudTjgWK3Ct+8pVbKlquo1VVU7q6ra4NbvybeOX1BVtdutP59WVTXw1q+mqqoadzOGEBoKqBbAt6Hfcir1FBM2TyCnIEfrkMqEpXFLsbOyo0fdHlqHIkS5EOatW0rccnaL1qHoVPD2PVJBXggDa+vZls/bfc6hK4eYe3Wu9FF8iLScNP488ydP1XkKJxsnrcMRolxo5NaI2k61zafAaQVv3yPJlhBG8GSdJ/l3q38TnRnNxF0TpY/iPaw+lMhjX2yh2Tf/JTMvk2pqR61DEqLcUBSFMO8w/r74NylZKVqHc1v7ntUVsn2PJFtCGMmgRoN4yuUp1pxaw/8O/E/rcMzK6kOJvLMymsSUDKxc/yY/05Mp67JZfShR69CEKDcKn0rccs5MlhIDB0JOeoVs3yPJlhBG9JTLUwzyHcTcmLnMOTJH63DMxpfro8mxOUal6muwtL1EbkpLMnMLmLQ+VuvQhCg3Grs1ppZjLfN5KrGwfc/hivdUopXWAQhRnimKwjut3iElO4X/HfgfrpVc6d2gt9ZhmZyqqsRdj2PXhV3surCLG9X3Y2+Rh1pgSW5aE3JTgwG4kJKpcaRClB+FTyX+HPMzqdmpuFRy0TYgCwsIGAA7v9O173F01zYeE5JkSwgjs1As+KzdZ9zIucHE3RNxqeTC416Pax2W0V3NvMruC7vZdWEXuy/s5lrWNQDqu9anUkY7UpLrkJ9RB1Qb/XtqutppFa4Q5VKYTxizj8xmy9kt5vGDXuAg2PE/iF4ObcZqHY3JSLIlhAlYW1rzTeg3jNgwgje3vclPXX6ihUcLrcMyqKy8LA5eOahPsOKuxwFQuVJlWtdsTduabWlTow3VHarr92xlqvn699tZW/JmV9/7XV4I8QiauDXB09GT9QnrzSPZquYLNYJ0BU4l2RJCGJq9tT1TOk/hhXUv8PKWl5nddTaNqzTWOqxHpqoqJ1JO6JOrA5cPkJ2fjZWFFSHuIbwS8gpta7alkVsjLJSi20N7BXsCMGl9LBdSMqnpasebXX31x4UQhlG4lDg/Zr55LCWCbnZr3du69j3ujbSOxiQk2RLChFxtXZnWZRrP//k8ozeNZv5T8/Fy9tI6rGK7lnmNPRf36JcGkzKTAKjrUpf+DfvTpmYbmldvjr21/UOv1SvYU5IrIUygq3dX5hyZYz5LiX79YP27utmtJyZqHY1JSLIlhIl5OHgwrcs0XvjzBUZuHMn8p+ZTzb6a1mHdU05+DoeuHNInV8eSjwHgUsmFNjXa6JYGa7bBw8FD40iFEPfTpIpuKXFDwgbzSLb07XuWwuPv6zbOl3OSbAmhgbouuj6KL214iVGbRjH3ybk42zhrHRaqqnI69bT+qcH9l/aTlZ+FlWJFkHsQLwe/rF8atLSw1DpcIUQxFBY4nX/MjJYSAwbCipcgfjvULf8FjSXZEkIj/tX8+a7Td4zdPJYJmyfwU5efsLMy/dN417Ou65cGd13YxZWMKwD4OPvQp0Ef2tZsS3OP5jhYO5g8NiGEYYT5hDEnZg5bz22lV/1eWofzT/ueqCWSbAkhjKtNzTZ80f4L3tz2Jm9ue5NvOn2DtYW1UcfMzc8lMilSn1wdu3YMFRVnG2da12itXxqs6VjTqHEIIUynaZWmuqXE+A3mkWxZ20HTnhCzGrp9BTYP3+dZlkmyJYTGuvp0JTU7lY/3fMzEXRP5+LGP73p6rzRUVeXMjTP6pwb3XdpHZl4mVooVAdUCGBc0jrY129KkShNZGhSinFIUhS7eXVhwbAE3cm6YxbYFAgbBoQW69j3+/bSOxqgk2RLCDAzwHUByVjJTIqfgWsmVN5q/gaIoj3y9lKwU9lzao0+wLt28BIC3szc96/Wkbc22tPBogaONo6E+BSGEmQvzDmNuzFy2nt1Kz/o9tQ4HvB8Dl9q69j2SbAkhTGFUwCiSs5L5+ejPVLatzHD/4cV+b25+LoeTDuufGoy5FoOKipO1E61rtmZkwEja1GhDLadaRvwMhBDmzK+qHzUdarIhYYN5JFsWFuDfv0K075FkSwgzoSgKb7d8m5TsFL47+B3xl2Hr/rr3LPqpqioJNxL0ydXeS3vJyMvAUrEkoFoAY4LG0LZmW5pWaYqVhfwzF0L8s5S48PhC81lKrCDte+S7sKh4Fi6Ed9+Fs2fByws+/RTCw7WOCtD1Ufz0sU85kXSF1ee/Jys/HBU/ElMyeWf13xxJySGv0nF2X9jNhZsXAKjtVJun6z1Nm5ptaOnREicbJ40/CyGEuQrzCWPe0XlEnIvgmXrPaB3OP+17Di+SZEuIcmPhQhg5EjIydB8nJOg+BrNJuKwtrbl8ciAFzknY1lxM7vXWWNonYGF7niVnVRytHWlVoxUv+b9EmxptqO1cW+uQhRBlhH9Vf2o41GBD/AbzSLbgtvY9x8C97LYwe5DyX7ZViNu9++4/iVahjAzdcTNy8XoBGeeGUpBTBWu3naAq5Fx9nIz4MWwftJ1vO33LAN8BkmgJIUqkcClx14VdpOWkaR2Ojl8/UCx1G+XLKUm2RMVy9mzJjmukpqsdFNiTET+e9LgPyEgYS87VLnhUaiR7sIQQpRLmE0ZuQS4R5yK0DkWnsH1P9DIoKNA6GqOQZEtULF73afp8v+MaebOrL3bWlqBaQ4EtAHbWlrzZ1VfjyIQQZV1A1QA8HDzYEL9B61D+ETgIbiTq2veUQ5JsiYrl00/B/o5Kxfb2uuNmpFewJ5/38cfT1Q4F8HS14/M+/vqnEYUQ4lEVLiXuvLDTfJYSfbtBJWdd+55ySNYjRMVSuAneTJ9GvF2vYE9JroQQRtHVpyvzj84n4lwET9d7WutwdO17mjxTbtv3yMyWqHjCwyE+Xrc3ID7eLBMtIYQwJrNcSgwYBDnpuvY95YwkW0IIIUQFY5ZLibe37ylnJNkSQgghKqAwbzN7KrGwfc+pzZB2WetoDEqSLSGEEKICCqgWgLN1Vd7f9At13l7LY19sYfWhRG2DChwEagEcWa5tHAYmyZYQQghRAf0aeZGUpMbk2hxHtcjStQVbGa1twqVv31O+lhIl2RJCCCEqoEnrY8lM8UOxyMPGbQeKVQqZuXlMWh+rbWCBz8KlKF37nnJCSj8IIYQQFdCFlExUapOf5UGlapuoVG0Tar4t17Nq8Nnfe1DTVNyS3KjvWh97axOWYvDrC+v/rZvd6vKh6cY1Ikm2hBBCiAqopqsdiSmZZMSPxcL2IpaVLmJhewk7h8v8eupXbubeZPEfi1FQ8HL2omHlhjSo3ICGlRvSsHJDPB09sVCMsEDmWA3qP6Fr39P5A93G+TJOki0hhBCiAnqzqy/vrIwmM9eGgkxvCjK9sbO2ZGJHf54JqsHKTStx83Uj7nqc/temhE2oqAA4WDvQwPWf5KuhW0MauDbA0cax9MEFDoTl63Xte+p2LP31NCbJlhBCCFEBFXaomLQ+lgspmdR0tePNrr7641WtqxLqFcrjXo/r35ORm8GplFNFErA/4/9kadxS/Tmejp7/JGC3ftV2qo2lhWXxg7u9fY8kW0IIIYQoq0raFsze2h7/av74V/PXH1NVlcsZl/9JwJLjiL0ey7bz2yhQCwCwtbSlvmt9fN18iyxFulRyufdA5ax9T5lLtnJzczl//jxZWVlah2J2XFxcOHas/Dy9Yc5sbW2pVasW1tbWWocihBCaUhQFDwcPPBw86FCrg/54dn72XbNgW85uYcWJFfpzqttXx9fNt8gsmLezN1YWVrr2PYcWwPG1ENBfi0/NYMpcsnX+/HmcnJzw8fFBURStwzEraWlpODk5aR1GuaeqKteuXeP8+fPUqVNH63CEEMIsVbKsRJMqTWhSpYn+mKqqXM28WiQBi70ey67EXeSpeQDYWNhQz7UeDVwb4Otem4aHZ9OwYWfcbN20+lRKrcwlW1lZWZJoCU0pikKVKlVISkrSOhQhhChTFEWhmn01qtlX4zHPx/THc/NzOZ16mrjrcZy4fkKXgF3cxa8OChSchyUdqWpXlYaVG+Jb+Z+lyLoudbG2NP8VhjKXbAGSaAnNyd9BIYQwHGtLa3zdfPF18y1y/Nr5fZxY0J24gF7EObsTdz2OBccWkFuQC4CVYkUd1zr6JKxwKbKqXVUURWH1ocT7PgBgSmUy2RJCCCFE+VelVguquDWh9dljMHo6AHkFeSTcSCiyFLn/0n7Wnl6rf1/lSpWpbO3NyfNOZKvVUWxrkJhSg3dWRgOYPOGSZOsRnD9/nnHjxnH06FEKCgro0aMHkyZNwsbGRuvQhBBCiPIlYBCse0vXvse9MVYWVtRzrUc913o8Vecp/Wmp2an65OvE9ROsitmP4hyDnWsuaoEV6bEfkZmbz6T1sSZPtsp+WVYTU1WVPn360KtXL06cOEFcXBzp6em8++67WocmhBBClD9+fUGxfGhzapdKLrTwaEF443Amtp3IjVNjSI/9kPRTr5N5/gUKU54LKZkmCLqoMj2z9eFvMRy9cMOg12xS05kPnm5639e3bNmCra0tL774IgCWlpZ888031KlThw8//JAvv/wSR0dH3njjDeLj4+nRowdHjhwhPz+ft99+m4iICLKzsxk3bhyjRo0iIiKCr776it9//x2Ar776ivT0dCZOnEhoaChfffUVzZs3v288oaGhXLx4ETs7OwoKCoiJiSE/P5+IiAjef/99qlSpQmxsLB06dODHH3/EwsKCRYsW8dlnn6GqKt27d+e///0vAOvWrePf//43+fn5VK1ald9//502bdoAcPToUXx9fbG0tGTmzJlcvHiRTz75hJycHKpUqcLChQupXr06ycnJ9OnTh+TkZLKzs6lRowYRERFFYp47dy4vvvgix44do1GjRhw7dowmTZowZ84chg4dyubNm3njjTfIy8ujRYsWTJ06lUqVKgHg5+dHQUEBNjY2nDx5kvT0dAA2bNjABx98QHZ2NvXq1WPOnDk4OuqqGPv4+ODk5ER+fj4WFhYcOXKEiRMn6u+TEEIIM1bYvidqKXR+H4pZHLWwHZGaU438nGpFjpuazGyVUExMDM2aNStyzNnZGS8vL06ePImFhQWqqt71vlmzZuHi4sK+ffvYt28fM2bM4MyZMwaJaeHChURGRrJz507s7P75S7R3716+/vproqOjOXXqFCtXruTChQu89dZbbNmyhcjISPbt28fq1atJSkpixIgRrFixgsOHD7Ns2TLs7OyIjIwkMjKSmjVrsnXrViIjI2nevDnt2rVjz549HDp0iEGDBvHll1/qY/Hz8yMqKoqFCxfeN+aWLVsye/ZsAGbPnk2rVq0A3dOmQ4cOZcmSJURHR5OXl8fUqVP178vPz2fdunVERkbqj129epVPPvmETZs2cfDgQZo3b87//ve/Iu/Ztm0bf/zxh0G+3kIIIUwscCCkXdC17ymmN7v6YmddNDGzs7bkza6+93mH8ZTpma0HzUAZi6qq93wSrfB4rVq12LRp012vb9iwgaioKJYvXw5AamoqJ06cwMbGhu3btxMUFASgT3oKhYeHY2dnh5eXFzNnzsTd3b3YsbZs2ZK6desC8Oyzz7Jjxw6sra0JDQ2lWrVq+uv/9ddfWFpa0qFDB33dKDe3B9czOX/+PAMHDuTixYvk5OTo32dpaUlaWtpDY2vRogWHDh0iKytLn8ABxMbGUqdOHRo2bAjACy+8wJQpU3j11VcBSE9Pvyu2PXv2cPToUR57TPcYcU5Ojn5GDiAzMxNbW1tu3Cg6C/rNN9+wYMECHBwc+Prrr2nduvVD4xZCCKGBwvY9h5dA3dBiveVh7YhMSWa2Sqhp06bs37+/yLEbN25w7tw56tWrx8CBA8nMzMTPz49u3brpz1FVlR9++EE/U3TmzBnCwsIAaN++vf74a6+9VuTahbNWAQEBfPvttyWK9c6kUFGUe866FcZXknIGEyZMYPz48URHRzNt2jR9Rf/nn3+e5ORkPDw8CA8Pf+A1nnzySSZMmMBTT/2zwfF+8YFu1iszM1O/PHj7e7p06aL/Gh49epRZs2bp31NQUIC9/d2tHl577TUiIyP58MMP+b//+79if+5CCCFMrLB9z7FfISej2G/rFezJzrcf58wX3dn59uOaJFogyVaJde7cmYyMDH7++WdAt0T1+uuvM3ToUOzt7XFwcGDVqlUcOXKkyLJV165dmTp1Krm5utogcXFx3Lx5s9jjVqlShZycnBLFunfvXs6cOUNBQQFLliyhXbt2tGrVim3btnH16lXy8/NZtGgRHTt2pE2bNmzbtk2/tJmcnPzAa6empuLpqftLO2/ePP1xR0dHrKys+Pnnnx+4jAi6xGzXrl0MHjxYf6xRo0bEx8dz8uRJAObPn0/HjrompKtWreLJJ5+86zqtW7dm586d+vdkZGQQFxcHwPLly4vMct3Lo3xthRBCmFjgs5CTrmvfU8aU6WVELSiKwqpVqxg7diwff/wxBQUFdOvWjc8+++yB7xs+fDjx8fGEhISgqirVqlVj9erVDx1v+PDh+pmchyUvd2rTpg1vv/020dHRdOjQgd69e2NhYcHnn39Op06dUFWVbt260bNnTwCmT59Onz59KCgowN3dnY0bN9732hMnTqR///54enrSunVrfZI2adIkAgICCAsLu2sG8E7u7u7ExMQUOWZra8ucOXPo37+/foP86NGj2b9/Py+99BJubm76JdfMzEzef/99PvroI+bOncuzzz5LdnY2AJ988gkxMTFMnTqVuXPn3nP8KVOmsHr1ajIyMvj888+L8yUVQgihFa+24FIbohaXuV6JyoOWbbTWvHlz9c7/sI8dO0bjxo01isi83d4b8c6nHMu6iIgIIiIimDhxov5Yeno648ePv28yZWzF+bsYERFBaGioaQISxSb3xfzIPTE/ZnlPNn8EO76B/zsOTtW1juYuiqIcUFX1rhICsowoyoQmTZrQr1+/IsdsbW0ZM2aMRhEJIYQwuYBBoBbAkeVaR1IikmyVU6GhoeVmVgt0S45+fn5FjllZWelLRgghhKgAqjWEmsFweJHWkZSIJFtCCCGEKDsCBsGlaLh8VOtIik2SLSGEEEKUHYXte6Ie3L7HnEiyJYQQQoiyQ9++ZxkU5GsdTbFIsiWEEEKIsiVwUInb92hJkq1HYGlpSVBQEIGBgYSEhLBr1y6tQxJCCCEqDt+n/mnfUwZIsvUIChs0Hz58mM8//5x33nlH65CEEEKIisPaDpr0LHH7Hq2UKtlSFKW/oigxiqIUKIpyVxGv2857UlGUWEVRTiqK8nZpxiziz7dhTnfD/vqzZOHduHGDypUr33U8Pj4eRVH46aefAF1bH09PT4YOHQroGk737duXFi1a0KJFC3bu3AnoKrM///zzPP744zRo0IAZM2YAutY2a9as0V8/PDycX3/99a5xv/zyS/z9/QkMDOTtt//5XEJDQ/H19SUoKEhfkT4rK4sXX3wRf39/goOD2bp1KwDr16+nc+fO5OTkEBERQY8ePQBYunQpvXv3pqCggLlz5zJ+/Hj99W8vLvrRRx/RokUL/Pz8GDlypL7f4d69ewkMDCQoKAhPT88iBUoLDR06lFq1apGfr1uHnzp1KoqiEB8fD8D//vc//Pz88PPzK9Ir8urVq9jY2BAUFET9+vX1MYOuqn2LFi0ICAjggw8+KHKP7OzsCAoKwsvLS//5hIaGPrT6vRBCCI0FDioz7XtKO7N1BOgD/HW/ExRFsQSmAE8BTYBnFUVpUspxNZWZmUlQUBCNGjVi+PDhvPfee/c8r379+vqWPOvWraN27dr611555RVee+019u3bx4oVKxg+fLj+taioKNauXcvu3bv56KOPuHDhAsOHD2fOnDmAri/hrl27ijS6BtiwYQOrV6/m77//5vDhw/zrX//Sv1bYBzEyMlJ/bMqUKQBER0ezaNEiXnjhBbKysujatSv9+vVjxIgR+nP//vtvvv32WxYsWICFxYP/2owfP559+/Zx5MgRMjMz9fW+/vvf//Lee+/ds+H27Tw9PVm/fj0Aa9asoX79+gAcOHCAOXPm8Pfff7Nnzx5mzJjBoUOH9J9frVq1iIyMZObMmUW+JidOnGDv3r1ERkZy4MAB/vrrL/17GjRoQGRkJB999NEDPychhBBm5vb2PWauVL0RVVU9Brp+gQ/QEjipqurpW+cuBnoCpS+Q8dQXpb7EoyhcRgTYvXs3Q4YM4ciRI3d9HSpVqkT9+vWJiYlh/vz5DB48WD9jsmnTJo4e/edLcOPGDdLS0gDo2bMndnZ22NnZ0alTJ/bu3UuvXr0YN24cV65cYeXKlfTt2xcrq6K3LyIighdffBF7e3sA3Nzc9K9lZmZia2tb5PwdO3YwYcIEQNcA2tvbm7i4OAICAhgxYgSfffYZly9f5vLly/Ts2ZOvv/4aBwcH/fuXLFnCjh07AEhMTKR5c93k5tatW/nyyy/JyMggOTmZpk2b8vTTT2Npaan/HB/k+eefZ/78+Xh5edGgQQPOnz+vj7d37976GPr06cP27dsJDg4mPT29yOdbaMOGDWzYsIHg4GBA1+LnxIkTdOjQ4Z5fk0Lh4eHY2dnh5eXFzJkzcXd3f2jcQgghTMjCAgIG6Nr3pF0CJw+tI7ovUzSi9gTO3fbxeeC+Zb8VRRkJjASoXr06ERERRV53cXEp1n/YxlYYg5+fH0lJSZw5c4Zq1arpX09PT6egoIABAwbw6aefkpWVhYuLC7m5uaSlpZGfn8+GDRuws7Mrct3s7GxUVdVfPzc3l6ysLNLS0hgwYACzZs1ixYoV/Pjjj3d9HQoKCsjOzr7n1ycxMRFHR0f9a2lpaeTm5pKRkaE/lp+fz82bN0lLS2Py5MkMGDCAo0ePEhcXx4wZM/j8888JCwvD1taWrKwsevfuzddffw3A66+/TlZWFklJSYwZM4Zt27ZRq1YtPvvsM1JTU0lLS+PNN99kyJAhfPzxx2RlZfHCCy/cFWtubi4uLi5kZmby2WefMWbMGDZv3kx6ejqZmZlFPr/s7Gz91yYmJgYPDw/S0tLIyMggLy+PtLQ0srOzee211xg2bNhd9+/kyZO4u7uTlpZGVlYWOTk5+nszffp0QkJC+Oijj/jyyy+LLD8WysrKuuvv553S09Mfeo4wPbkv5kfuifkpC/fEPqsuLdUCTq7+L+dr99Q6nPt6aLKlKMom4F7p4ruqqq65x/G7LnGPY/ftfq2q6nRgOugaUd/ZBPPYsWP6ZstaKozh+PHjFBQU4O3tjaWlpf51R0dHLCws6NChA59//jnjx48nMzMTa2trnJyc6Nq1K/PmzePNN98EIDIykqCgICpVqsTq1auZOHEiN2/eZOfOnXz99dc4OTkxatQoWrZsiYeHBy1btrwrps6dO/PVV18xbNgw7O3tSU5Oxs3NjR07duDm5oaXl1eR+B9//HFWrVpFjx49iIuLIzExkZCQEK5du8aiRYv4+++/+fvvv7G0tOS5557j/PnzTJ48mQ8//BBbW1tsbGz0XwcbGxtsbW2xtrZGURR8fHzIz8/nt99+o1+/fjg5OVGvXj2cnZ1ZtWoVv/zyC+np6XfdS2tra+zs7BgxYgSTJ0+mffv2WFhY4OjoSFhYGEOHDuWDDz5AVVX++OMP5s+fj5OTE2vXrqVXr144OTlhb2+PlZUVTk5OPP3007z33nu89NJLODo6kpiYiLW1Ne7u7qxdu5aOHTvi5ORU5POxtLTEwcEBJycnatasSWJi4j3/ztna2upnzO7HLBu5CrkvZkjuifkpM/ckcSb1b+6jfug3WkdyXw9NtlRVfaKUY5wHat/2cS3gQimvqanCPVsAqqoyb968IonWnf78808Ali//p3Hm999/z7hx4wgICCAvL48OHTroN9O3bNmS7t27c/bsWd577z1q1qwJ6Gb6GjduTK9eve45TpcuXYiLi6N58+bY2NjQrVs3evfuzcsvv8zs2bPvOn/s2LGMHj0af39/rKysmDt3LpUqVeLVV1/l008/vWvW7bXXXqN169YMHjz4vp+rq6srI0aMwN/fHx8fH1q0aKH/Og0dOpTPPvtM//k8SPfu3enevXuRYyEhIQwdOlSfaA4fPpzg4GB+/PFHpk+fzrZt25g8eTLp6ekkJSXx66+/8swzz3Ds2DHatGkD6JLgBQsW8NVXX3Hz5k3GjRt3z/GHDx+uf5Bg4cKFD41XCCGERgKfhT//pWvfU908t4QrhU+KleoiihIBvKGq6l2PcCmKYgXEAZ2BRGAf8JyqqjEPu27z5s3VO58KO3bsGI0bNy51zOZq4sSJODo68sYbb9z1WkZGBv7+/hw8eBAXF5e7Xk9LSzOLWT9TmzhxIqGhoUV+Avv999+5evWq/ulPYyjO38Uy85NhBSP3xfzIPTE/Zeae3LwKX/tCm3HQRduHnRRFOaCq6l3VGUpb+qG3oijngTbAWkVR1t86XlNRlD8AVFXNA8YD64FjwNLiJFqiqE2bNtGoUSMmTJhwz0SrIuvXrx9NmhT9aSYkJISOHTtqFJEQQgiTcahq9u17Svs04ipg1T2OXwC63fbxH8AfpRmrorhX7SmAJ554grNnz5o2mDLCz8/vrmPFWaoUQghRTgQMhLh1uvY9dUO1juYuUkFeCCGEEGWbmbfvkWRLCCGEEGVbYfueo2sg56bW0dxFki0hhBBClH2BgyD3plm275FkSwghhBBlX2H7nsPm175Hkq1HcP78eXr27EmDBg2oV68er7zyCjk5OVqHJYQQQlRche17Tm/Vte8xI5JslZCqqvTp04devXpx4sQJ4uLiSE9P591339U6NCGEEKJiCxgEagFEL3/4uSZUppOt/+79Ly+ue9Ggv/67978PHHPLli3Y2try4osvAmBpack333zD7NmzycjIYOLEiXz11VcAxMfH68sS5Ofn8+abb9KiRQsCAgKYNm0aoCsa16NHD/31v/rqK335h9DQUO4s6nqnkydP8sQTTxAYGEj79u05deqU/rouLi4EBQXh4eGhj2nz5s0EBwfj7+/PsGHDyM7OJjc3ly5durB27doi42ZmZtK+fXt27twJgI+PD1evXgXg6tWr+Pj46D/P9u3bExISQkhICLt27dLHN3r0aBo3bkxQUNA9q+zHx8ejKIq+en5+fj6enp76YqQJCQl07tyZgIAAOnfuXKT8xfjx4/Hy8iIoKAhHR0f91+rUqVM8+eSTNGvWjPbt23P8+HH9e4YOHUqdOnUICgrCxsaGq1ev3nUPhBBClFHVGkLNEIgyr6XEMp1saSEmJoZmzZoVOebs7IyXlxcnT57EwsKCe1XlnzVrFi4uLuzbt499+/YxY8YMzpw5U+p4wsPDGTduHIcPH2bjxo3UqFED0CUtHTt2JDIyktGjRwO6xslDhw5lyZIlREdHk5eXx9SpU7G2tmbp0qVMnDiRqKgoQDeDN2TIEEaNGsVjjz32wBjc3d3ZuHEjBw8eZMmSJbz88ssAREdHs2vXLmJiYoiMjLyr/U+h+vXrs3r1agDWrVtH7dr/dHcaP348Q4YMISoqivDwcP21Cz/HTz75hMjISJo3/6dg78iRI/nhhx84cOAAX331FWPHji3ynq+//prIyEipxSWEEOVR4CC4FK1r32MmSlXUVGtvtXzL5GOqqoqi3N1bu/B4rVq12LRp012vb9iwgaioKH1/xNTUVE6cOIGNjQ3bt2/X91pMSkpixIgR+veFh4djZ2eHl5cXM2fOxN3dXf9aWloaiYmJ9O7dG9A1Rra3twd0/RttbW2LxBAbG0udOnVo2LAhAC+88AJTpkzh1VdfpXLlygwfPpwePXrg5OTEe++9x5kzZ1i6dGmRa3Tq1AlLS0vy8/+p0pubm8v48eOJjIzE0tKSuLg4QDfrl5OTQ05Ozl2x3K5SpUrUr1+fmJgY5s+fz+DBg/WzVLt372blypUAPP/88/zrX//Svy89PR03N7ci10pPT2fXrl30799ffyw7O1v/53t9XQD9PVAUhQkTJjBs2LD7xiuEEMKM+fWF9f/WzW5p3L6nkMxslVDTpk3vWtq7ceMG586do169egwcOJDMzEz8/Pzo1k1fRB9VVfnhhx+IjIwkMjKSM2fOEBYWBkD79u31x1977bUi1164cCGRkZEEBATw7bffFnntQX0tL1y4cNfMzYPOT0tLY/LkyXz77becPHkSV1dXwsLCmDlzZpHztm7dSmRkJFu3btUf++abb6hevTqHDx9m//79+ocFmjRpwoABA3B3dycoKIjMzMz7jv/iiy/y5ZdfkpeXh4eHx33Puz3RPXPmDLVq1SryekFBAa6urvqvZ2RkJMeOHXvg1wX+uQcbN27kX//6FxkZGfeNQQghhBnTt+9ZajbteyTZKqHOnTuTkZHBzz//DOiWpV5//XWGDh2Kvb09Dg4OrFq1iiNHjvDHH/90KOratStTp04lNzcXgLi4OG7eLH7htSpVqtz1xKOzszO1atXSL8FlZ2eTkZFBfn4+K1euvGv5r1GjRsTHx3Py5EkA5s+fr+8f+MEHHzB27Fj69OlDmzZteOONN/j000/57rvv9Pu07ic1NZUaNWpgYWHB/Pnzi8x6ubi48MorrzxwGRGgWbNmXLlyRb8XrlDbtm1ZvFi39r5w4ULatWsH6PZyXbx4kcDAwLu+JnXq1GHZsmWALsE8fPgwoNvfFh8ff1cfxds5OTlhZWVV5HMQQghRxgQMhLSLcOYvrSMBJNkqMUVRWLVqFcuWLaNBgwY0bNgQW1tbPvvsswe+b/jw4TRp0oSQkBD8/PwYNWoUeXl5Dx1v+PDhtGvXjhUrVjBhwoS7Xp8/fz7ff/89AQEBdOnShUuXLvH888/ToEED+vbtW+RcW1tb5syZQ//+/fH398fCwoLRo0cTHR3N7t27GTVqVJHznZ2d+fe//81bbz14uXbs2LHMmzeP1q1bExcXh4ODAwC7du1iw4YN9+33eKc///yT7t27Fzn2/fffM2fOHAICApg/fz7fffcdAC1atODatWsEBwcTFBTE/v37efPNNwFdUjZr1iwCAwNp2rQpa9as4cKFC/Ts2ZPp06djY2Nz19i7du2iXbt2tG7dmtdeew0nJ6dixSyEEMIMFbbviTKP9j3Kg5aWtNa8eXP1ziW7Y8eO0bhxY40iMm9paWkVJkkIDQ0lIiKiyLF+/frp98SZQnH+LkZERBAaGmqagESxyX0xP3JPzE+ZvydrxsORlfDmCbBxMMmQiqIcUFW1+Z3HZWZLlEnvv//+Xcfu3O8mhBCiAgt81mza90iyJcqkxx9//K5jDytRIYQQogLxagMuXmbRvkeSLSGEEEKUP2bUvkeSLSGEEEKUT4Hm0b5Hki0hhBBClE9VG+ja92i8lCjJlhBCCCHKr6DnwN4NstM1C6FMt+sxtWvXrtG5c2cALl26hKWlJdWqVQNg796996zfJAzv7NmzvPvuu8TGxpKRkUFERARVq1bVOiwhhBDmqMVwaDni4ecZUblPtlYfSmTS+lgupGRS09WON7v60ivY85GuVaVKFSIjIwGYOHEijo6OvPHGGwaMVjxMVlYWzz77LJ9++ikdO3a8Z59KIYQQQs8M/p8o18uIqw8l8s7KaBJTMlGBxJRM3lkZzepDiQYdJz4+nkaNGvHCCy8QEBBAv3799L31Dhw4QMeOHWnWrBldu3bl4sWLgK4o5+0FWx0dHQFdEbkePXoAkJycjIuLC1999ZX+Pa+++ipt27bFz8+PvXv36s/r1asXbdq0oXXr1kRFRQG6hNDT05OAgAAaNWrEli1bAF2rm86dOxMQEEDnzp05e/bsXTHNnTuX8ePHAzB06NAixUL9/PyIj48HoFevXjRr1oymTZsyffp0/TmzZs2iUaNGBAUF4eLiclcBUgAfHx/8/f1p1KgRYWFh+vZFhV+L2xWOuWXLFjIzMxk/fjz+/v5FqtsvWrQIf39//Pz8ihx3dHTk9ddfJyQkhM6dO5OUlFTk883Pz+eZZ55hzpw5AMyYMYMWLVoQGBhI3759pU+iEEKIUinXydak9bFk5hbtcZeZm8+k9bEGHys2NpaRI0cSFRWFs7MzP/74I7m5uUyYMIHly5dz4MABhg0bxrvvvlvsa37++ed4e3sXOXbz5k127drFjz/+yLBhwwBdX8Pg4GB2797NZ599xpAhQ/Tnv/baa0RFRfHSSy/x+++/AzB+/HiGDBlCVFQU4eHhvPzyywBYWFg8sFn1vcyePZsDBw6wf/9+vv/+e65duwbA22+/zV9//UVkZCTt27e/7/u3bt1KTEwMly9f5tSpUw8dLykpicTERH1D7H379rF69WouXLjAW2+9xZYtW4ocL/yahYSEcPDgQTp27MiHH35Y5JqjRo2idevW+r6Mffr0Yd++fRw+fJjGjRsza9asEn1NhBBCiNuV62XECymZJTpeGrVr19YX1Rw8eDDff/89Tz75JEeOHKFLly6Arml1jRo19O8JDw/XN2fOzCwaU2JiInv27KF3795Fjj/77LMAdOjQgRs3bpCSksKOHTtYsWIFoCv2ee3aNVJTUwH45ptvmD17NleuXGHr1q0A7N69m5UrVwLw/PPP869//QuAWrVqcejQIVq0aHHX5/fmm2/yySefABRJir7//ntWrVoFwLlz5zhx4gRVqlTBwsKCtLQ03N3dH/h169SpE9euXcPPzw9/f3/91yIoKAhVVenYsSPffvut/nxVVenatat+r1x4eDh//fUXiqIQGhp61/FevXphYWHBwIED9femT58++utNnDiRvXv3cu7cOf2xI0eO8J///IeUlBTS09Pp2rXrAz8HIYQQ4kHK9cxWTVe7Eh0vjTv3DimKgqqqNG3alMjISCIjI4mOjmbDhg36cxYuXKh/rTDpKvThhx/y3nvv3fO69xrnfvG89tprHD16lMWLF/P6668/MPZ///vfzJ49m8DAwLva4UyaNEkfa7169QDdkuemTZvYvXs3hw8fJjg4mKysLACmTp2qX+7cvn37vb9o6Ga2EhMTqV69OosWLQLAzs6OyMhIDhw4QFRUFJs2bdKf7+zsfM/rlGRG7vavYaVKlRg1ahSffvqp/tjQoUOZPHky0dHRfPDBB/rPSQghhHgU5TrZerOrL3bWlkWO2Vlb8mZXX4OPdfbsWXbv3g3o9g61a9cOX19fkpKS9Mdzc3OJiYl56LVOnTpFfHw8YWFhd722ZImug/mOHTtwcXHBxcWFDh06sHDhQgD9k3l3JiXOzs5cvXoVgLZt27J4sa7myMKFC2nXrh0AjRo1Ys+ePRw+fJiPPvrooXGmpqZSuXJl7O3tOX78OHv27NG/VrNmTQIDAzl8+PADlxFBl/w4OTnp4ytkZWWFi4sLOTk5+mPNmjVjy5YtXL16lfz8fBYtWkTHjh1p1aoV27Ztu+s4QEFBgX7P2S+//KL/fAHeeecd3nvvPX799Vf9vUlLS6NGjRrk5ubqv65CCCHEoyrXy4iFTx0a6mnEB2ncuDHz5s1j1KhRNGjQgDFjxmBjY8Py5ct5+eWXSU1NJS8vj1dffZWmTZs+8FrHjx/Xb9a+U+XKlWnbti03btxg9uzZgG4p7MUXX6RNmzY4Ojoyb948/fnffPMNCxYsIC8vT7/R/vvvv2fYsGFMmjSJatWq3Xesh3nyySf56aefCAgIwNfXl9atWwO6Ehkvv/wyv/76K5aWlg+8RqdOnVAUherVq/PZZ58BumXEdu3akZubi4+PD127duXtt98GwNvbm4kTJ9KhQwcsLS3p3r07PXv2BHR73Dp16oSqqnTr1k1/3MHBgZiYGJo1a4aLi4s+YS1kY2PDlClTGDlyJNu3b+fjjz+mVatWeHt74+/vT1pa2iN9fYQQQggApaQbok2pefPm6u1P7AEcO3aMxo0baxTRvcXHx9OjRw+OHDli1HFCQ0P56quvaN68+T1fT0tLw8nJyagxlEWOjo6kpxu+mF1x/i5GREQQGhpq8LFF6ch9MT9yT8yP3JOSUxTlgKqqd/0nXa6XEYUQQgghtFaulxFNxcfHx+izWsA9a1WJhzPGrJYQQghRXDKzJYQQQghhRJJsCSGEEEIYkSRbQgghhCifFi4EHx+wsND9rlE5H9mzJYQQQojyZ+FCGDkSCvvbJiToPgYIDzdpKDKzVQLXrl0jKCiIoKAgPDw88PT01H98e+HN8iw/P58vv/yStm3bEhISwowZM7QOSQghhLjbu+/+k2gVysjQHTex8j+ztXCh7gt79ix4ecGnnz5yRlulShUiIyMBXSFRR0dH3njjDQMGa/4mTpyIhYUFmzdvvqvFkBBCCGE2zp4t2XEjKt8zW4VTiAkJoKr/TCEaYc12wYIFtGzZkqCgIEaNGkV+fj4A69atIyQkhMDAQDp37qxvshwUFISNjQ3+/v4EBQWxf/9+EhIS6Ny5MwEBAXTu3Jmzt/5CDB06lDp16uDn50dAQIC+zMSMGTNo0aIFgYGB9O3bl4xbGfypU6f0sdSpU4ehQ4feFe/EiRPx9PQkICCARo0asWXLFv1Yha1tCn311VdMnDjx1pd0Idu3b6dly5ZFYnxQ7KNHj6Z9+/Y0bNiQ33//HYC5c+cyfvx4ABYvXkzXrl3Jzc0lPj6e9u3bExISQkhICLt27TLULRJCCFGReHmV7LgRle9ky0RTiMeOHWPJkiXs3LmTyMhILC0tWbhwIUlJSYwYMYIVK1Zw+PBhli1bpm+yHBkZSc2aNdm6dSuRkZE0b96c8ePHM2TIEKKioggPD+fll1/WjzFp0iSOHDlChw4d9IlRnz592LdvH4cPH6Zx48b8/PPPAPz4448MGDCAyMhIJk2adN+4X3vtNaKionjppZf0SdDDnDlzhhdeeIHo6OgiMT4o9vj4eLZt28batWsZPXp0kcbOmzdv5rvvvmP58uVYW1vj7u7Oxo0bOXjwIEuWLClyHSGEEKLYPv0U7O2LHrO31x03sfK9jGiiKcTNmzdz4MABWrT4//buLTTK/Izj+PeJGwjYSjFrbNioVZCgNAUhHlc8trJtwxZURN2KQe+ssAY30nTViOYu0ohEPCDxLKGxlkKltBaMVyrNbrVxjdH11LptjUa0emEamacXE4fEHDa2Tv7vzPw+N8n7zsw7v+FJmGf+//8771Qgfm2/vLw8Ll68yJw5cxg/fjwAI0eOHPA4Fy5c4PTp0wCsWrWKTZs2JW4rLy+noqKCjo4OLl26BMDVq1fZvHkzT5484fnz5yxYsACAYcOGDep6fjU1NdTV1dHW1sa5c+d6PFdVVRW5ubns3bu3x2OysrJYuXJlr4wDZV+2bBlZWVlMnDiRCRMmcP36dQCam5s5evQoR44cSVxmqLOzk/Xr1yea1hs3bnzt6xAREenl1ZKht7SU6P+R3iNbQzSE6O6sXr06MWLV2trKtm3bcHfM7H8+bvfHVldXc/PmTbZu3UplZSUQn6Krra2lubmZysrKxIjRhg0bOHv2LGPHjqW8vLzf45eVlXHt2jXq6+vZuHFjj+e6fPkyK1asSEwfvvL6tRf7e33d979+n1fbLS0tnDx5skf2mpoaRo8ezZUrV2hqasqYEw9ERCQJPvoI7t6FWCz+M0CjBenebA3REOLChQs5deoUbW1tADx+/Jh79+4xc+ZMzp8/z507dxL7BzJr1izq6+uB+Nqo2bNn97rPiBEjePToERC/8HR+fj6dnZ2c6LYOLTc3l+zsbM6cOTPgNGJfx+wuNze3V7MzderUPjMOlL2hoYFYLMatW7e4ffs2hYWFQHzEq6SkhKVLl7J9+3YAnj59Sn5+PllZWRw7diyx9k1ERCRVpfc04hANIU6ePJmqqioWLVpELBYjOzubPXv2MGPGDA4cOMDixYuJxWKJ9Uj92b17N2vWrKG6uppRo0Zx6NChxG2vpvbMjIMHDwKwY8cOpk+fzrhx4ygqKko0c2VlZZSWllJUVERra2u/z1dTU8Px48d5+fIlO3fuTOzfsmULu3btoqOjg/379yfWiAHU1taydu1aqqurycvLo66u7muzFxYWMnfuXB48eMC+ffvIycnpkaOiooJp06axfPly1q1bx5IlS2hoaGD+/PkMHz58MCUQERGJLHP30Bn6VVxc7E1NTT32tbS0MGnSpECJou3Zs2e9pvlCKy0tTYxepZvB/C02NjYyb968oQkkg6a6RI9qEj2qyZszs8/cvfj1/ek9jSgiIiISWHpPI0pwhw8fDh1BREQkqJQc2Yry1KdkBv0NiojIYKVcs5WTk0N7e7ve7CQYd6e9vb3XQn8REZG+pNw0YkFBAffv3+fhw4eho0TOixcv1AAMkZycHAoKCkLHEBGRFJByzVZ2dnbiG9mlp8bGRqZMmRI6hoiIiHSTctOIIiIiIqlEzZaIiIhIEqnZEhEREUmiSH+DvJk9BO6FzpFC3gV6X+RQQlJNokl1iR7VJHpUkzc3zt1Hvb4z0s2WvBkza+rrMgESjmoSTapL9Kgm0aOavD2aRhQRERFJIjVbIiIiIkmkZiu9HAgdQHpRTaJJdYke1SR6VJO3RGu2RERERJJII1siIiIiSaRmS0RERCSJ1GylITP7xMzczN4NnUXAzKrN7LqZ/dXMfmNm3wqdKVOZ2Qdm1mpmX5rZz0PnETCzMWZ2zsxazOwLM/s4dCaJM7NhZvYXM/td6CypTs1WmjGzMcAPgL+FziIJZ4Hvuvv3gBtAReA8GcnMhgF7gB8Ck4EVZjY5bCoBXgIb3X0SMAP4meoSGR8DLaFDpAM1W+mnBtgE6MyHiHD3P7r7y67Ni0BByDwZbBrwpbvfdvf/APXATwJnynju/k93/7zr92fE39zfC5tKzKwA+DFwMHSWdKBmK42Y2YfAV+5+JXQW6dca4PehQ2So94C/d9u+j97UI8XMvgNMAS4FjiKwi/gH91jgHGnhndAB5M2Y2Z+Ab/dx06fAL4BFQ5tIYOC6uPtvu+7zKfEpkxNDmU0SrI99GgGOCDP7BvBrYIO7/zt0nkxmZiVAm7t/ZmbzAsdJC2q2Uoy7f7+v/WZWBIwHrpgZxKeqPjezae7+ryGMmJH6q8srZrYaKAEWur7cLpT7wJhu2wXAPwJlkW7MLJt4o3XC3U+HziO8D3xoZj8CcoARZnbc3X8aOFfK0peapikzuwsUu7uu2B6YmX0A/BKY6+4PQ+fJVGb2DvETFBYCXwF/Bla6+xdBg2U4i386PAI8dvcNgePIa7pGtj5x95LAUVKa1myJJF8t8E3grJldNrN9oQNloq6TFNYDfyC+CPtXarQi4X1gFbCg6//jcteIikja0MiWiIiISBJpZEtEREQkidRsiYiIiCSRmi0RERGRJFKzJSIiIpJEarZEREREkkjNloiIiEgSqdkSERERSaL/AkNAPFFEJPwKAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x504 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.linear_model import LinearRegression\n",
    "\n",
    "# Выборка\n",
    "np.random.seed(42)\n",
    "N = 10\n",
    "X = np.linspace(-5, 5, N).reshape(-1, 1)\n",
    "y = np.sin(X) + np.random.normal(0, 0.2, size = N).reshape(-1, 1)\n",
    "\n",
    "X_test = np.linspace(-5, 5, N // 2).reshape(-1, 1)\n",
    "y_test = np.sin(X_test) + np.random.normal(0, 0.2, size = N // 2).reshape(-1, 1)\n",
    "\n",
    "# Очень простая модель (регрессия на константу)\n",
    "too_simple_model_predictions = np.mean(y) * np.ones_like(y)\n",
    "\n",
    "# В меру сложная модель\n",
    "X_ok = np.hstack([X, X ** 2, X ** 3])\n",
    "ok_model = LinearRegression()\n",
    "ok_model.fit(X_ok, y)\n",
    "ok_model_predictions = ok_model.predict(X_ok)\n",
    "\n",
    "# Очень сложная модель\n",
    "X_compl = np.hstack([X, X ** 2, X ** 3, X ** 4, X ** 5, X ** 6, X ** 7, X ** 8, X ** 9, X ** 10])\n",
    "compl_model = LinearRegression()\n",
    "compl_model.fit(X_compl, y)\n",
    "compl_model_predictions = compl_model.predict(X_compl)\n",
    "\n",
    "plt.figure(figsize = (10, 7))\n",
    "\n",
    "plt.scatter(X, y, label = 'Тренировочная выборка')\n",
    "plt.scatter(X_test, y_test, c = 'r', label = 'Тестовая выборка')\n",
    "plt.plot(X, too_simple_model_predictions, label = 'Очень простая модель')\n",
    "plt.plot(X, ok_model_predictions, label = 'В меру сложная модель')\n",
    "plt.plot(X, compl_model_predictions, label = 'Очень сложная модель')\n",
    "plt.grid()\n",
    "plt.legend();"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "parliamentary-occasion",
   "metadata": {},
   "source": [
    "- Очень простая модель имеет большое смещение (bias), но малую (нулевую) дисперсию (variance). Модель явно недообучена.\n",
    "\n",
    "- В меру сложная модель имеет небольшое смещение (bias) и небольшую дисперсию (variance).\n",
    "\n",
    "- Очень сложная модель имеет небольшое смещение (bias), но большую дисперсию (variance). Модель явно переобучена.\n",
    "\n",
    "**Задание:** пользуясь определениями выше, объясните, почему это так."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "noble-outline",
   "metadata": {},
   "source": [
    "**Задание:** прокомментируйте величину смещения и дисперсии для следующих моделей:\n",
    "\n",
    "1. Линейная регрессия, обучаемая на большой выборке без выбросов и линейно зависимых признаков, в которой признаки сильно коррелируют с целевой переменной.\n",
    "2. Решающее дерево, которое строится до тех пор, пока в листах не окажется по одному объекту.\n",
    "3. Логистическая регрессия, относящая все точки к одному классу."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aboriginal-enemy",
   "metadata": {},
   "source": [
    "### 1.A. Bias-Variance tradeoff"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "certain-conclusion",
   "metadata": {},
   "source": [
    "Из описания выше можно заметить, что при обучении моделей возникает выбор между смещением и дисперсией: недообученная модель имеет низкую дисперсию, но высокое смещение, а переобученная – низкое смещение, но высокую дисперсию. Этот выбор можно отобразить на картинке ([источник](https://www.bradyneal.com/bias-variance-tradeoff-textbooks-update)). Здесь Total Error – ошибка на тестовой выборке (generalization error)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "national-trail",
   "metadata": {},
   "source": [
    "![](https://www.bradyneal.com/img/bias-variance/fortmann-roe-bias-variance.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "considerable-documentary",
   "metadata": {},
   "source": [
    "Вывод из неё очевиден: строить следует оптимальные по сложности модели. \n",
    "\n",
    "Возникает ли такой выбор при обучении любой модели? Последние исследования показывают, что поведение ошибки при обучении некоторых (современных) моделей не соответствует такой U-образной форме. Например, было показано, что ошибка на тестовой выборке продолжает убывать при расширении (увеличении числа слоёв) нейронных сетей:\n",
    "\n",
    "<img src=\"https://www.bradyneal.com/img/bias-variance/neyshabur.jpg\" alt=\"drawing\" width=\"400\"/>\n",
    "\n",
    "В таких моделях поведение ошибки приобретает сложный вид:\n",
    "\n",
    "\n",
    "<img src=\"https://www.bradyneal.com/img/bias-variance/double_descent.jpg\" alt=\"drawing\" width=\"800\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dried-recorder",
   "metadata": {},
   "source": [
    "### 3. От деревьев к случайному лесу"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "configured-purple",
   "metadata": {},
   "source": [
    "#### 3.1 Решающее дерево"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "official-investor",
   "metadata": {},
   "source": [
    "Мотивацию построения алгоритма случайного леса (Random Forest) удобно рассматривать в терминах смещения и дисперсии. Начнём с построения решающего дерева."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 161,
   "id": "reduced-willow",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Пример отсюда: http://rasbt.github.io/mlxtend/user_guide/evaluate/bias_variance_decomp/\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "from mlxtend.data import boston_housing_data\n",
    "\n",
    "X, y = boston_housing_data()\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y,\n",
    "                                                    test_size = 0.3,\n",
    "                                                    random_state = 123,\n",
    "                                                    shuffle = True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "accepting-jacksonville",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.tree import DecisionTreeRegressor\n",
    "from sklearn.metrics import mean_squared_error\n",
    "\n",
    "# TODO: обучите решающее дерево без ограничений на тренировочной выборке\n",
    "\n",
    "\n",
    "# TODO: рассчитайте MSE на тренировочной и тестовой выборках\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 163,
   "id": "centered-yemen",
   "metadata": {},
   "outputs": [],
   "source": [
    "from mlxtend.evaluate import bias_variance_decomp\n",
    "\n",
    "# TODO: воспользуйтесь функцией bias_variance_decomp и выведите среднее смещение и среднюю дисперсию модели\n",
    "# на тестовой выборке\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "helpful-murder",
   "metadata": {},
   "source": [
    "Как мы обсуждали на предыдущем семинаре, такое дерево окажется сильно переобученным (высокая дисперсия и низкое смещение). Постараемся исправить это. На лекции мы обсуждали, что один из способов борьбы с переобучением – построение композиций моделей. На этом семинаре мы рассмотрим построение композиций при помощи бэггинга."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "headed-password",
   "metadata": {},
   "source": [
    "#### 3.2 Бэггинг\n",
    "\n",
    "Вспомним суть алгоритма:\n",
    "\n",
    "1. Обучаем много деревьев на бутстрапированных подвыборках исходной выборки независимо друг от друга. Бутстрапированную подвыборку строим при помощи выбора $N$ (размер исходной выборки) наблюдений из исходной выборки с возвращением. \n",
    "\n",
    "2. Усредняем предсказания всех моделей (например, берём арифметическое среднее). \n",
    "\n",
    "Можно показать, что модель, построенная при помощи бэггинга, будет иметь **то же смещение**, что и у отдельных деревьев, но значительно **меньшую дисперсию** (при выполнении некоторых условий). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 167,
   "id": "unexpected-pacific",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.ensemble import BaggingRegressor\n",
    "\n",
    "base_tree = DecisionTreeRegressor(random_state = 123)\n",
    "\n",
    "# TODO: обучите бэггинг с 20 деревьями, каждое из которых строится без ограничений\n",
    "\n",
    "\n",
    "# TODO: выведите среднее смещение и среднюю дисперсию модели на тестовой выборке\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "complicated-harvard",
   "metadata": {},
   "source": [
    "Как мы видим, по сравнению с единичным деревом смещение практически не изменилось, но дисперсия уменьшилась в несколько раз!\n",
    "\n",
    "Посмотрим, как это отразилось на среднеквадратичной ошибке."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "random-statistics",
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO: рассчитайте MSE на тренировочной и тестовой выборках для бэггинга\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bibliographic-breast",
   "metadata": {},
   "source": [
    "Среднеквадратичная ошибка на тренировочной выборке больше не равна 0, а на тестовой – уменьшилась, что говорит о том, что мы успешно победили переобучение единичного решающего дерева. \n",
    "\n",
    "Можем ли мы снизить переобучение ещё сильнее? Можем!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "harmful-personal",
   "metadata": {},
   "source": [
    "#### 3.3 Случайный лес"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "floating-stable",
   "metadata": {},
   "source": [
    "При построении каждого дерева в бэггинге в ходе создания очередного узла будем выбирать случайный набор признаков, на основе которых производится разбиение. В результате такой процедуры мы уменьшим корреляцию между деревьями, за счёт чего снизим дисперсию итоговой модели. Такой алгоритм назвывается **случайным лесом** (Random Forest). \n",
    "\n",
    "По сравнению с единичным деревом к параметрам случайного леса добавляются:\n",
    "- `max_features` – число признаков, на основе которых проводятся разбиения при построении дерева.\n",
    "\n",
    "- `n_estimators` – число деревьев. \n",
    "\n",
    "Естественно, все параметры, относящиеся к единичному дереву, сохраняются для случайного леса."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "altered-bundle",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.ensemble import RandomForestRegressor\n",
    "\n",
    "# TODO: обучите случайный лес с 20 деревьями, каждое из которых строится без ограничений\n",
    "\n",
    "\n",
    "# TODO: выведите среднее смещение и среднюю дисперсию модели на тестовой выборке\n",
    "\n",
    "\n",
    "# TODO: рассчитайте MSE на тренировочной и тестовой выборках для случайного леса\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "authorized-portuguese",
   "metadata": {},
   "source": [
    "Как мы видим, по сравнению с бэггингом смещение вновь осталось практически неизменным, а дисперсия немного уменьшилась. Конечно, если подобрать хорошие гиперпараметры, то получится снизить дисперсию ещё больше. \n",
    "\n",
    "Ошибка на тренировочной выборке увеличилась, а на тестовой – уменьшилась, что означает, что мы добились нашей цели в борьбе с переобученными деревьями!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "straight-appeal",
   "metadata": {},
   "source": [
    "### 4. Особенности случайного леса"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "consecutive-verse",
   "metadata": {},
   "source": [
    "#### 4.1 Число деревьев и \"Случайный лес не переобучается\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "heard-confusion",
   "metadata": {},
   "source": [
    "В своём [блоге](https://www.stat.berkeley.edu/~breiman/RandomForests/cc_home.htm#remarks) Лео Бриман (Leo Breiman), создатель случайного леса, написал следующее:\n",
    "\n",
    "> Random forest does not overfit. You can run as many trees as you want.\n",
    "\n",
    "**Обратите внимание:** как говорилось на лекции, случайный лес не переобучается именно с ростом числа деревьев (за счёт совместной работы бэггинга и использования случайных подпространств), но не в принципе. Посмотрим на поведение случайного леса при росте числа деревьев."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 234,
   "id": "wooden-dining",
   "metadata": {},
   "outputs": [],
   "source": [
    "X, y = boston_housing_data()\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y,\n",
    "                                                    test_size = 0.3,\n",
    "                                                    random_state = 123,\n",
    "                                                    shuffle = True)\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.3, random_state = 123)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 236,
   "id": "freelance-threshold",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAl4AAAGpCAYAAABcXji6AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABD+ElEQVR4nO3deZyU1YHv/++prbt6YWmWZpNFZBGQRVtBSbTR4B7NYiZq4oaO14wm5mYbJ/O7JsZgnLneTCaJmYwmRmOIxBhHjYrGYNqVqKCIICKyiEjLjtB7Lef3x3mqu2i6oZd6qqq7P+/Xq15P7c+pPtVV3zrnPOcYa60AAADgv0CuCwAAANBXELwAAACyhOAFAACQJQQvAACALCF4AQAAZEko1wXoiMGDB9uxY8f6uo/a2loVFxf7ug90HfWTv6ib/Eb95C/qJr91p35WrFixy1o7pK3bekTwGjt2rJYvX+7rPqqqqlRZWenrPtB11E/+om7yG/WTv6ib/Nad+jHGvN/ebXQ1AgAAZAnBCwAAIEsIXgAAAFnSI8Z4AQCA7onFYtq6dasaGhpyXZQeoX///lq7du1h71NYWKhRo0YpHA53+HkJXgAA9AFbt25VaWmpxo4dK2NMrouT9w4cOKDS0tJ2b7fWavfu3dq6davGjRvX4eelqxEAgD6goaFBgwYNInRliDFGgwYN6nQLIsELAIA+gtCVWV35exK8AAAAsoTgBQAAkCUELwAAkBXGGF122WXNl+PxuIYMGaLzzz9fkrR9+3adf/75mjFjhqZMmaJzzz1XkrR582ZFo1HNnDmz+fTb3/623f3cdtttXSrfNddco7fffrtLj+0ojmoEAABZUVxcrNWrV6u+vl7RaFTPPPOMRo4c2Xz7zTffrPnz5+vGG2+UJK1atar5tvHjx2vlypUd2s9tt92m7373u4dcb62VtVaBQNvtTr/61a868Wq6huAFAEAfc8uf1+jtbfsz+pxTRvTT9z499Yj3O+ecc/TEE0/ooosu0gMPPKBLLrlEL7zwgiSpurpaZ555ZvN9p0+f3uly3HTTTaqvr9fMmTM1depULVy4UOecc47mzZunZcuW6ZFHHtHtt9+u1157TfX19brooot0yy23SJIqKyt1xx13qKKiQsOHD9eNN96oxx9/XNFoVI8++qjKy8s7XZ7W6GoEAABZc/HFF2vx4sVqaGjQqlWrNHv27Obbrr/+el199dWaN2+eFi5cqG3btjXftmHDhoO6GlNhrbXbb79d0WhUK1eu1KJFiyRJ69at0+WXX6433nhDY8aM0cKFC7V8+XKtWrVKzz333EEtaym1tbWaM2eO3nzzTZ166qm6++67M/L6afECAKCP6UjLlF+mT5+uzZs364EHHmgew5Vy1llnaePGjXrqqae0ZMkSzZo1S6tXr5bUua7G1saMGaM5c+Y0X37wwQd11113KR6Pq7q6Wm+//fYhrWuRSKR57NkJJ5ygZ555pkv7bo0WL0mq26Pimk1SMpnrkgAA0OtdcMEF+ta3vqVLLrnkkNvKysp06aWX6v7779eJJ56o559/vtv7Ky4ubj6/adMm3XHHHVq6dKlWrVql8847r81JUMPhcPM8XcFgUPF4vNvlkAhezsrf68TlX5eaDuS6JAAA9HoLFizQzTffrOOOO+6g65999lnV1dVJckv2bNiwQaNHj+7084fDYcVisTZv279/v4qLi9W/f39t375dS5Ys6fwL6AaClySFo24bq89tOQAA6ANGjRrVfORiuhUrVqiiokLTp0/XySefrGuuuUYnnniipEPHeP30pz9t9/mvvfZaTZ8+XV/60pcOuW3GjBmaNWuWpk6dqgULFmju3LmZe2EdYKy1Wd1hV1RUVNjly5f7t4OVv5ce+Yr0tZVSWccXukT2VFVVqbKyMtfFQBuom/xG/eSvbNfN2rVrdeyxx2Ztfz3dkRbJTmnr72qMWWGtrWjr/rR4SbR4AQCArOCoRkkKecErTvACAKCnmD17thobGw+67v777z9k7Fg+IXhJtHgBANADvfLKK7kuQqfR1SilBa9DDycFAADIFIKXlBa86nJbDgAA0KsRvKS0MV60eAEAAP8QvCRavAAAQFYQvCTGeAEAkAXGGF122WXNl+PxuIYMGdK8JuL27dt1/vnna8aMGZoyZUrzWo6bN29WNBo9aALV3/72t+3u57bbbutyGe+9996DFufONI5qlGjxAgAgC4qLi7V69WrV19crGo3qmWee0ciRI5tvv/nmmzV//vzmWe1XrVrVfFtnFsm+7bbb9N3vfrdLZbz33ns1bdo0TZo0qUuPPxKClySFCt2WMV4AgL5gyU3SR29l9jmHHSedc/sR73bOOefoiSee0EUXXaQHHnhAl1xyiV544QVJUnV1tc4888zm+06fPr3TxbjppptUX1+vmTNnaurUqVq0aJF+97vf6ac//amampo0e/Zs/eIXv5AkXX311Vq+fLmMMVqwYIGOOuooLV++XF/60pdUUFCgV155RdFotNNlOBy6GiXJGCUCEVq8AADw2cUXX6zFixeroaFBq1at0uzZs5tvu/7663X11Vdr3rx5Wrhw4UFdfq3XakyFtdZuv/12RaNRrVy5UosWLdLatWv1hz/8QS+99JJWrlypYDCoRYsWaeXKlfrwww+1evVqvfXWW7rqqqt00UUXqaKiQosWLdJLL72U8dAl0eLVLBkoUJAxXgCAvqADLVN+mT59ujZv3qwHHnigeQxXyllnnaWNGzfqqaee0pIlSzRr1iytXr1aUue6GtMtXbpUK1asaF5su76+XkOHDtWnP/1pbdy4UV/96ld13nnnHdTS5ieClycRjCjMzPUAAPjuggsu0Le+9S1VVVVp9+7dB91WVlamSy+9VJdeeqnOP/98Pf/88zrhhBO6vC9rra644gr96Ec/OuS2N998U08//bTuvPNOPfjgg7rnnnu6vJ+OoqvRkwwUsFYjAABZsGDBAt18882HrKn47LPPqq7ODfs5cOCANmzYoNGjR3f6+cPhsGKxmCTpjDPO0EMPPaQdO3ZIkvbs2aP3339fu3btUjKZ1Oc//3ndeuutev311yVJpaWlOnDgQHde3mHR4uVJBgpYqxEAgCwYNWpU85GL6VasWKEbbrhBoVBIyWRS11xzjU488URt3ry5eYxXyoIFC/S1r32tzee/9tprNX36dB1//PFatGiRfvjDH+rMM89UMplUOBzWnXfeqWg0qquuukrJZFKSmlvErrzySl133XW+Da431tqMPqEfKioq7PLly33dx8c/Pkn9B4+QLn/E1/2ga6qqqlRZWZnrYqAN1E1+o37yV7brZu3atTr22GOztr+e7sCBAyotLT3i/dr6uxpjVlhrK9q6P12NHlq8AACA3+hq9CSCjPECAKAnmT17thobGw+67v777z9k7Fg+IXh5koGIFPs418UAAMA31loZY3JdjIx55ZVXcrr/rgzXoqvR47oamccLANA7FRYWavfu3V0KCziUtVa7d+9WYWFhpx5Hi5cnEWTmegBA7zVq1Cht3bpVO3fuzHVReoSGhoYjhqrCwkKNGjWqU89L8PIwuB4A0JuFw2GNGzcu18XoMaqqqjRr1qyMPy9djZ5kIOIG19MECwAAfELw8iSCBZJNSommXBcFAAD0UgQvTzIQcWfobgQAAD4heHkSwQJ3huAFAAB84lvwMsYUGmNeNca8aYxZY4y5xbu+zBjzjDFmvbcd6FcZOqO5xYtJVAEAgE/8bPFqlHS6tXaGpJmSzjbGzJF0k6Sl1toJkpZ6l3MuGaDFCwAA+Mu34GWdGu9i2DtZSRdKus+7/j5Jn/GrDJ3R0tXIJKoAAMAfvs7jZYwJSloh6RhJd1prXzHGlFtrqyXJWlttjBnazmOvlXStJJWXl6uqqsrPoqqgMSFJWvnay9r33gFf94XOq6mp8f09gK6hbvIb9ZO/qJv85lf9+Bq8rLUJSTONMQMk/Y8xZlonHnuXpLskqaKiwlZWVvpSxpTXH10nSZo5bbI0wd99ofOqqqrk93sAXUPd5DfqJ39RN/nNr/rJylGN1tp9kqoknS1puzFmuCR52x3ZKMORtHQ1smwQAADwh59HNQ7xWrpkjIlK+pSkdyQ9JukK725XSHrUrzJ0Rss8XozxAgAA/vCzq3G4pPu8cV4BSQ9aax83xiyT9KAx5mpJWyR9wccydBgtXgAAwG++BS9r7SpJh6wuaa3dLekMv/bbVS3zeNHiBQAA/MHM9Z6Webxo8QIAAP4geHmSgbAkwxgvAADgG4JXijFSOEqLFwAA8A3BK12okDFeAADANwSvdOEi1moEAAC+IXilCxcSvAAAgG8IXunCUYIXAADwDcErXSgqxQleAADAHwSvdLR4AQAAHxG80hG8AACAjwhe6QheAADARwSvdKEo83gBAADfELzSMXM9AADwEcErXTjKWo0AAMA3BK90qRYva3NdEgAA0AsRvNKFCiVZKdGU65IAAIBeiOCVLlzktozzAgAAPiB4pQsXui3jvAAAgA8IXulo8QIAAD4ieKULeS1ezOUFAAB8QPBK19zixez1AAAg8whe6ZrHeBG8AABA5hG80tHiBQAAfETwStc8xovgBQAAMo/glS4cdVtavAAAgA8IXukIXgAAwEcEr3QELwAA4COCV7qQF7wY4wUAAHxA8EoXKpBkaPECAAC+IHilM8Z1NxK8AACADwherRG8AACATwherYWirNUIAAB8QfBqLRyVYnW5LgUAAOiFCF6thQulGC1eAAAg8wherYWLaPECAAC+IHi1FipkjBcAAPAFwas1WrwAAIBPCF6tMcYLAAD4hODVWriIebwAAIAvCF6thQpZqxEAAPiC4NUaM9cDAACfELxaS3U1WpvrkgAAgF6G4NVauFCSleKNuS4JAADoZQherYWL3JZxXgAAIMMIXq2FCt2WcV4AACDDCF6tpVq8CF4AACDDfAtexpijjDF/M8asNcasMcbc6F3/fWPMh8aYld7pXL/K0CVhWrwAAIA/Qj4+d1zSN621rxtjSiWtMMY84932H9baO3zcd9c1j/Fi9noAAJBZvgUva221pGrv/AFjzFpJI/3aX8Y0j/FivUYAAJBZxmZhvipjzFhJz0uaJukbkq6UtF/ScrlWsb1tPOZaSddKUnl5+QmLFy/2tYw1NTUqKSlR6f53dcLr39aq4/6P9gyq8HWf6LhU/SD/UDf5jfrJX9RNfutO/cybN2+FtbbNEOF78DLGlEh6TtJCa+3DxphySbskWUm3ShpurV1wuOeoqKiwy5cv97WcVVVVqqyslLavkf7rFOkL90lTP+PrPtFxzfWDvEPd5DfqJ39RN/mtO/VjjGk3ePl6VKMxJizpT5IWWWsfliRr7XZrbcJam5R0t6ST/CxDp4WjbssYLwAAkGF+HtVoJP1a0lpr7Y/Trh+edrfPSlrtVxm6JOQFL8Z4AQCADPPzqMa5ki6T9JYxZqV33XclXWKMmSnX1bhZ0v/ysQydl2rxitHiBQAAMsvPoxpflGTauOlJv/aZEWFavAAAgD+Yub61YEQyAcZ4AQCAjCN4tWaMG+fFzPUAACDDCF5tCRO8AABA5hG82kLwAgAAPiB4tSUcleIELwAAkFkEr7aECmnxAgAAGUfwaku4iOAFAAAyjuDVljAtXgAAIPMIXm0JFzHGCwAAZBzBqy2M8QIAAD4geLUlXMRajQAAIOMIXm0JF7JWIwAAyDiCV1vCUdZqBAAAGUfwakso6lq8rM11SQAAQC9C8GpLOOq28cbclgMAAPQqBK+2pIIX47wAAEAGEbza0tzixTgvAACQOQSvtoRSLV7M5QUAADKH4NWWMMELAABkHsGrLQQvAADgA4JXW5rHeBG8AABA5hC82sIYLwAA4AOCV1voagQAAD4geLUlXOi2BC8AAJBBBK+2hIvcljFeAAAggwhebQnR4gUAADKP4NWWVIsXwQsAAGQQwastwbBkAgQvAACQUQSvthjjWr1YqxEAAGQQwas9oUIpVpfrUgAAgF6E4NWecJEUo8ULAABkDsGrPWFavAAAQGYRvNoTjjLGCwAAZBTBqz2hKC1eAAAgowhe7QlHGeMFAAAyiuDVnnCUebwAAEBGEbzaE6arEQAAZBbBqz0hBtcDAIDMIni1hxYvAACQYQSv9oQLGVwPAAAyiuDVnnCRFK+XrM11SQAAQC9B8GpPqNBtGecFAAAyhODVnnCR2zKlBAAAyBCCV3vCXosXwQsAAGQIwas9tHgBAIAMI3i1p3mMF8ELAABkhm/ByxhzlDHmb8aYtcaYNcaYG73ry4wxzxhj1nvbgX6VoVto8QIAABnmZ4tXXNI3rbXHSpoj6XpjzBRJN0laaq2dIGmpdzn/lJa77b4tuS0HAADoNXwLXtbaamvt6975A5LWShop6UJJ93l3u0/SZ/wqQ7cMmSwFC6Rtb+S6JAAAoJcwNgsThBpjxkp6XtI0SVustQPSbttrrT2ku9EYc62kayWpvLz8hMWLF/taxpqaGpWUlBx03fErvqVEsFBvzvyhr/vGkbVVP8gP1E1+o37yF3WT37pTP/PmzVthra1o67ZQt0rVAcaYEkl/kvR1a+1+Y0yHHmetvUvSXZJUUVFhKysrfSujJFVVVemQfRz4hLT6YVWedprUwXLDH23WD/ICdZPfqJ/8Rd3kN7/qx9ejGo0xYbnQtcha+7B39XZjzHDv9uGSdvhZhm4ZMVNq/FjauynXJQEAAL2An0c1Gkm/lrTWWvvjtJsek3SFd/4KSY/6VYZuGz7DbbetzGkxAABA7+Bni9dcSZdJOt0Ys9I7nSvpdknzjTHrJc33LuenoVOkQFiqfjPXJQEAAL2Ab2O8rLUvSmpvYNQZfu03o0IFUvkUqXplrksCAAB6AWauP5LhM1yLVxaO/gQAAL0bwetIhs+U6vcykSoAAOg2gteRDJ/ptnQ3AgCAbiJ4HUn5VMkEGWAPAAC6jeB1JOFCaeixTCkBAAC6jeDVEcNnMsAeAAB0G8GrI4bPkOp2Sfs/zHVJAABAD0bw6ogRM92WcV4AAKAbCF4dUT5NMgHGeQEAgG4heHVEpEgaPIkWLwAA0C0Er44aPoO5vAAAQLcQvDpqxEypZru0vzrXJQEAAD0Uwaujmmewp7sRAAB0DcGro4YdJ8nQ3QgAALqM4NVRBSXS4Am0eAEAgC4jeHXG8Bk9d0qJeJO0e0OuSwEAQJ9G8OqM4TOlA9ukmh25LknnPf0v0s9PlLauyHVJAADoswhenTF8htv2tO7G+r3Syt9LNiE9cp0Uq891iQAA6JMOG7yMMV9OOz+31W03+FWovDViphSKSm8uznVJOuf1+6VYnTT/B9Kud6Vnf5jrEgEA0CcdqcXrG2nnf9bqtgUZLkv+KyiV5lwnrX5I+mh1rkvTMcmE9Ord0pi50twbpROukpbdKb2/LNclAwCgzzlS8DLtnG/rct9wytekgv7S3xbmuiQds26J9PEWafb/cpfPvFUacJT0yFekptrclg0AgD7mSMHLtnO+rct9Q1GZNPer0ronpQ9ey3VpjuyVX0r9j5ImnecuF5RKF/5C2rtJ+uv3c1o0AAD6miMFr8nGmFXGmLfSzqcuT8pC+fLT7K9IRYOlZ3+Q65Ic3vY10uYXpBOvkYKhluvHfVKafZ306l3SxudyVz4AAPqY0BFuPzYrpehpCkqkU78lPXWTtLFKOroy1yVq2yu/dAcDHH/5obed8T1p/TPSozdI/7TMvSYAAOCrw7Z4WWvfTz9JqpF0vKTB3uW+64SrpH6jpKU/kGwe9rrW7ZFWPShN/wfXPdpapEi68E43/uvl1sdNAAAAPxxpOonHjTHTvPPDJa2WO5rxfmPM1/0vXh4LF0qV/yx9uMKN98qG137tjlD84FWpqe7w9339Pine0DKovi1jTpamflZ6+afS/urMlhUAABziSF2N46y1qXkTrpL0jLX2cmNMqaSXJP3Ez8LlvRmXSi/+xM2LNfFsKRD0b197NklPpM3uYQLS4IluUtdRJ0pjTpGGHCsFAlIiLr36K2nsJ6XyqYd/3jO+J619XKq6TbqAli8AAPx0pOAVSzt/hqS7Jclae8AYk/StVD1FMCSd/q/SQwuk5fdIJ/2jf/vaWOW2V/xZaqxxs+dXv+kGx6/6g7stOlAafYpUMkTav1U699+P/Lxl41y5X/mlO2igfIpvLwEAgL7uSMHrA2PMVyVtlRvb9ZQkGWOiksI+l61nmPJZaexvpCe/JX30lnT2j6RIceb3s7FK6jfStWIZI00+111vrbTvfen9l6XNL0nvvySt2yQNHOda4Tri1G9LKxdJz9wsffmhzJcdAABIOnLwulrSDyR9StIXrbX7vOvnSPqNj+XqOQIB6csPuwlVX/pPF4Au+nXLuo6ZkExKm56TJp3rQlc6Y6SBY91p5qXuuv3bpGCk412fRWXSJ78lPfN/pA1/k8bPy1zZAQBAsyMd1bjDWnudtfZCa+1f0q7/m7X2Dv+L10OEItL8W6TLH5GaaqS7z5Be/rkLTJnw0Sq30HVHp63oN0IqHty5fZx0rdR/tAtfmSo3AAA4yGFbvIwxjx3udmvtBZktTg93dKX0lZelx74q/eVfXWD63F3df97U+K5xp3b/udoTLpQ+9T3pT1e7MWMzL/FvXwAA9FFH6mo8WdIHkh6Q9Ir66vqMnVFUJn3xd+5IxxfukKZ8pmU8VldtrHJHLJYOy0QJ2zf1c9Kyn0vP3ir1HyWFCl1rXqhQChVIpcPdFgAAdMmRgtcwSfMlXSLpUklPSHrAWrvG74L1aMZIlTdJ7zwhLfmOdPRpXR9wH2uQtixzE7b6LRCQzlwo3Xe+O7VmgtLgCW6KiqFT3HbsJ5n1HgCADjps8LLWJuSOZHzKGFMgF8CqjDE/sNYy6dPhBMPSp38i3XOWVHW7dOatXXuera+6iVCztSzR2LnSDculj7dKiSYp3iglGl0A3LNR2vG2tPU1afWf3P3Lj5P+cSktYQAAdMCRWrzkBa7z5ELXWEk/lfSwv8XqJUbPceskLrtTmv5Fadi0zj/HxirX0jR2bsaL165B493pcBr2uxa9R65zyyadtTA7ZQMAoAc70pJB90l6WW4Or1ustSdaa2+11n6YldL1Bp+6RYoOkB7/3107WnBjlZuZvqA00yXrnsJ+bgD+ide4cWGpAwAAAEC7Dhu8JF0maaKkGyW9bIzZ750OGGP2+1+8XqCoTDrzh67L8PX7OvfY+r3Stjey183YFfNvdUsX/c9X3MLcAACgXUeaxytgrS31Tv3STqXW2n7ZKmSPN+MSacwnpL9+T6rZ2fHHbX5Rssn8Dl6RIulzd0u1O6XHv+5m0gcAAG064hgvZIAx0vn/If3XKdIfvuyOBow3SvF6ty3s71rFisoOftzGKilSIo2qyEmxO2zETLdm5V+/L735QMsM+gAA4CAEr2wZMlGa/wOp6kfSrnelcNSbH6tQ2r1e2v2edNkjrgUpZWOVNOYUd4Rkvjvla9L6Z6Qnvy2NPtktvg0AAA5C8Mqmk//JnVpb84j0xyvd6eJFLmjt+8CFsYoFWS5kFwWC0mf/W/qvue51XP6IFB2Y61IBAJBXjjS4Htkw9TPSef9PWv+0W24otSi2lN/ju1obcJT0uf92c3395lxpf3WuSwQAQF6hxStfnHi1VLtLqrrNLXB94COpeIibIb4nmXSOdOmD0uIvSfec6bpPjzQnGAAAfQQtXvnktO9IJ/6j9PLPpDX/41q7TA9cHnP8POnKP0tNtW7m/uo3c10iAADyAsErnxgjnfNv0tTPSsl4z+pmbG3kCdJVT0nBAune893UGAAA9HG+BS9jzD3GmB3GmNVp133fGPOhMWaldzrXr/33WKlB6v/wW2n6xbkuTfcMmShd/bRUOlz63eelXetzXSIAAHLKzxaveyWd3cb1/2GtnemdnvRx/z1XqECacqEU7AVD8PqPkq54zL2mx77WtWWTAADoJXwLXtba5yWxhgyk0mHSmQulLS9LK+7JdWkAAMgZY31c4sUYM1bS49baad7l70u6UtJ+ScslfdNau7edx14r6VpJKi8vP2Hx4sW+lVOSampqVFJS4us++jRrNePNm1V6YL1eO/Hnaiwc3KmHUz/5i7rJb9RP/qJu8lt36mfevHkrrLVtLjuT7eBVLmmXJCvpVknDrbVHnCG0oqLCLl++3LdySlJVVZUqKyt93Ueft2eTWzZp3KnSJYs7dcQm9ZO/qJv8Rv3kL+omv3Wnfowx7QavrB7VaK3dbq1NWGuTku6WdFI2948cKxsnzftX6d2npNV/ynVpAADIuqwGL2PM8LSLn5W0ur37opea8xU31cSS70i1u3NdGgAAssrP6SQekLRM0iRjzFZjzNWS/t0Y85YxZpWkeZL+t1/7R54KBKULfiY1fCw9/S+5Lg0AAFnl23wF1tpL2rj6137tDz1I+VTpE9+Qnv93acdaN2Hs1M9IZUfnumS5F29yobRhn9uWHS0VleW6VACADOkFE0WhRzrtO1LRIGn1Q9LSW9xp+EwXwioWSIX9cl3C7KnbIy2+1C2tFKs7+LZomXThndJk5hoGgN6A4IXcCIalOde5074t0tuPuvUp//o9adUfpC/90U2+2tvFm6Q/XCZ9uEI68RrXulU4wJ3ChdJz/y4tvsSt4XnmrVI4eujj33lcql4pHTVHGvuJzofWpjopGOkdE/YCQJ7jkxa5N2C0dMpX3WnD36QHL5fuPkP60oPS8Bkdf554o1vjMlLsX1kzyVrpzzdK778ofe5X0vQvHHqfCWdKS38gLfu59P5L0kX3SEOPlfZullbcJ71xv1S7U5KR9J9SICSNOkkaf7o7jZglBdoZynngI+nF/5CW/0bqN1z61C1uxYSeuDA7kGnJpLRrnbRlmbTl79LW19xnTCDo/s9SpwFjpKNOkkbPcf9vrX8cAa0QvJBfxs+TFjwtLfqCdM850hfulSae2f7992yS3vur9N5SadPzkqybsmL2dfnfgvPCHdKbv5cqv9t26JLcUktnLZSOnic9cp10V6U06kS36Lgx0sRzpIqrXEvX1tekDc+68Pq3H7pT6XBp8vnSsZ+Wxsx1f5OaHdKLP5GW/1pKxKTp/+C6Of94hWs1O+s2adQJ2fxLIJOslT7eKn20SvpotUZ8uEvaNVIadIy/obpmp7RjjbT9bbfd8Y5UMlSa/kVp4tmuBbezrJXq90r7P5Q+/lCq2S71GyENmexaxDP1eur3urVkd73rTjvWSh+86sZaSlJJuQtXhf2lZML9wEvG3f/PznXSu0vc/QJh92NxyGT3AzBSJIW9baS45XzYXS6q/cC1WocimXkdfkgmpY8/OPjvs2u9lGhy9Vs6TCoZJpWWS8VDpehAKTrAbQsHtP/arHV/v3i9FKuXmmqlA9WuByT9FKuXTODgUzLu7t9U421rJZtww1eKBknFg6WiwVLxEPejst8IqXSEtx2e87+3rxOoZgoTqPZB+6ul3/+DtH2NdN4dqjowRpUzxrl/+J3rvF+if5d2v+fuP2CMNGG+tO8Daf3T7pfnBT+Thh3Xsf1ZK731kAsc2Rjkv/pP0kML3JfSZ/+7Y18gNTtcC9mOt90C6sdfLvUf2fZ9a3e5MPrOn6X1f3UfbtEyacwpLpzFG9xznPZt93qTCdd69uxCqXaHNO0iqfJfpMHHHLFYh/zvWOvK2FgjJWPuwzWZcB+Mw6a7D0J0XFOd+0Lav821Uh7Y5t4LycTB90vGpd3rpepVUn0bq7X1GyUdXel+3Aw6xn2hxWrd88fqpMYD7n1Tt0uq2+3ON+6XQtGDwoJChe4Lr36fCyz1e93+Gj5u2VfRYNcyu/s9V/bC/tK0z0szLpVGVbj3ezLhWpDiDe6xeza60+4N0p4N7kfV/g8PHfeYEimVhkxy+xl6rDs/ZLLUb2T7/0/xJmnnO+5zZftq6aO3XMiq3dFyn2DE/X1GVUijT3YtWQPHHf5/tHa39MErLae977f8bZOx9h8nubA2ZLI0bJpUPs1tBx3jgkJ7rdXpkkmp5iP399qz0YWQ5kBU7k6BkLRzrXu9H612rz312RkIu1a8YNjdL1UnsXq3jTccvL/CAe5vHSp078Oaj9x7oD2hQheWJEnG/R2tdc9rE+08yLiANOAoKVIi2aS7r7XuvAlIBaVeuC129zHGjZdNvYdrd7negLbeP5//tXTcRUf80/o1gSrBy0PwykONNdJDV0nr/6KkCStg0z7Aioe4wfjHfMqdBo1v+Yde87D05Hfcr9W5N0qnfufwv7atdfOKvXqX+wc+78fSjC/697o+eFW693xp5PHS5Y+6Vi0/NdW6ELb2z65VcNyp0mn/3HaoajzgWsOW/dx9MA45Vpp0jjTpXDf/WhtfBM3/OzU7pZWLpNd/67442zOyQpp8njsNnpibrk1rpY1/c60y8j7MUx/q6VJlMwHXhRQu9rZFbizdsOMy27VtrbR3k/tRsWWZtOUV9yOjtVDUBYSDyioXEIZPdwF3+AypfKr+vvQxzRlcJ22skjY9d3BAakt0oNdyMNi9xnhDSzhrqnWXIyUtrRqplo2ycdLQKe6o5ZKh7rmSCbffNxe791+83pU9GXNBsS3hYvdjoGyc1P8o9+Oin3cqGerC2I61LkDtWOtOdbtaHh8pccGgcIArb6y2pVWkbnfLfkOFLvCUT5OGTJQGT5IGT3A/4jLZWp6IeeWo8/6OqfLU6e3XX9SUMuuFwNUuxKQEI64sA8e6UyDkniNW723rXBDfs8n9XTsqXOzqaMhEyQQPbr1Lxt1+w4WunsLeqd8I9786eKJ7b7T+n403utbI2p2tAvk+qdF7v7XOGqGClv+lUKHblg5zQ0/6j8rM56K17v1+oNq9b/Z7P2Cmfsa9R46A4EXw6psScWnZz7Rl3UqNnvUp759/wpGnWKjbI/3l/3NBYNAx0oW/kEbPPvR+6aGr4mr3Ib7lZWnGJdK5/9f9qkoXa5A2LHUffhPPOvT2tjTsd90+21a6Lr31T7vWp2uWSsWDOvynyKr91dLbj0jvPCG9/7L7tVk8VBr3SfchXDLM+0U9VKtef1XT429K6550H9yjT5FmXuruFwi1/JK2SWnzC9I7T0rbXnf7GXSMa51MdREUlbkv/METXSuGH6Hsg9fcQRzvv9T95wqEXMAZc4rryh09xwWRjkgmXcj66K2W1pdtb7gvMMm1Eh01xwXeAUe5FoBUV0knDqA46LMtmXDvwwPVB3eDhYvcezla5l8XfcN+ae1j7n8sVOC+bIMRty0ocaFx0HjXQtPZeq/d5VrCd77Tsm2qaWkNSbXWFQ1yoWPYcVLZ+JwPRzjke6d2l3sv7Nnk3ht7NrnxnPvedwvtpYJQuMiFo+Kh7m9WNs4Lq0e711uz3Z0ObHdhLt7otaod5/7OHWlJA8GL4NW3dbl+NjwrPXajtH+rG7xf+d2W1q/00HXyDdKZP3RfTC/cIT33b+4D6qJfS0OnutaR1Q+7INJ0wD0+FHWtQcdd5FrdQgXu8Tvf8bocXnXjrlJN+pLrPhh5vDT/B+4Dsyeo3yutf8a99m2vuw/zROPB94mWubB1/BXul/SRfPyhC2rrlrjukbo9Lb+MU8rGu8H+Uy504aa7IWzXejdtydo/uxbT0/7ZdX8dNH7EyDUdSe6bzpOMu9Cd3uJQu0va+qoLph+ucGNeJPeLfcjklq6vwZNcK9HezWlfpptceZpq3GNM0IXN4TNceBs9xz0uA1+QfLblL+omv/kVvPJ89DHQTeNPl77ykmv9euk/pXeflj77S9dN+eS3pdfudoFs/q3uSzcYkipvksZ+Unr4H6VfzXctAg0fuxaIqRdKUz/nfnGufshNgbHmYXdb+XGuZatxv9t38RA3EH76xdKIme5LNdUF05NEB7oB+NP/wV221nXjHnC/qle9sULTL7y+c10D/UdKJ/2jO6XEm9xYodpdLrC+/Yirsxd/7LpajpkvDRzjWn36jXQtP8WDXUtKapxRqoujsSZt8G2N+/X/zpOutaDyu9LJ17tWls4o7H/odZPOdttYvQtfW/7udYOtkzY+d2hANUHXejVwrAuqw45zpyHHdm3wOYAeh+CF3q+wn3TBT92RfY991U1VMfpkN41DeuhKN3audN2L0l+/71oypn7WHVmYfjTMmJOls293Y1je+qM72ue4i6SjZrsjoI40ILenMqZlbM/QydqzxWZmPEYo4rovS4e5AcYVV7lBy+uecPO8rfy9Gx/TGcEC18VUUOLmSTv121LJkO6XtbVw1B1ZOvYTLdcl4q6LaOc6F6oGjnVjloLhzO8fQI9B8ELfMWG+9E/LpCX/7CZpbS90pRSVucB2OMGwe94J8zNfXrgxcMdf7k6HDJTd5gZLF/RzdRUd6Lo8owNbjnjKZcgJhlx3ck/pUgaQFQQv9C3RgdLn7nKThTKtQc9ijHck3QA38B4AeiAObUDfROgCAOQAwQsAACBLCF4AAABZQvACAADIEoIXAABAlhC8AAAAsoTgBQAAkCUELwAAgCwheAEAAGQJwQsAACBLCF4AAABZQvACAADIEoIXAABAlhC8AAAAsoTgBQAAkCUELwAAgCwheAEAAGQJwQsAACBLCF4AAABZQvACAADIEoIXAABAlhC8JG3YWaOXPowpmbS5LgoAAOjFCF6Snlu3U3e/1aT9DbFcFwUAAPRiBC9JA4vDkqQ9tU05LgkAAOjNCF6SBhZFJEl762jxAgAA/iF4KS140eIFAAB8RPCSVFacavEieAEAAP8QvCQNKHJjvAheAADATwQvSSUFIQUNY7wAAIC/CF6SjDEqiRjGeAEAAF8RvDylYboaAQCAvwhenuKw0d5auhoBAIB/CF6e0oihxQsAAPiK4OUpCRO8AACAv3wLXsaYe4wxO4wxq9OuKzPGPGOMWe9tB/q1/84qiRjtrYvJWhbKBgAA/vCzxeteSWe3uu4mSUuttRMkLfUu54WSsFEiabW/IZ7rogAAgF7Kt+BlrX1e0p5WV18o6T7v/H2SPuPX/jur1E1er310NwIAAJ+Esry/cmtttSRZa6uNMUPbu6Mx5lpJ10pSeXm5qqqqfC1YMNEoyeivL/xd4wcEfd0XOq+mpsb39wC6hrrJb9RP/qJu8ptf9ZPt4NVh1tq7JN0lSRUVFbaystLX/W14ZKmkBh09+ThVTm43DyJHqqqq5Pd7AF1D3eQ36id/UTf5za/6yfZRjduNMcMlydvuyPL+21USNpKkPcxeDwAAfJLt4PWYpCu881dIejTL+29XScQFL6aUAAAAfvFzOokHJC2TNMkYs9UYc7Wk2yXNN8aslzTfu5wXikJSMMBcXgAAwD++jfGy1l7Szk1n+LXP7jDGaGBRWHvrWDYIAAD4g5nr0wwoimgvY7wAAIBPCF5pyooidDUCAADfELzSDCgKa28tXY0AAMAfBK80ZcW0eAEAAP8QvNIM8LoaWSgbAAD4geCVpqw4rFjCqqaRhbIBAEDmEbzSDCxyK2XvY0oJAADgA4JXmlTwYtkgAADgB4JXmoHFLngxwB4AAPiB4JVmYFFYEsELAAD4g+CVpizV4sVcXgAAwAcErzT9CsMKGFq8AACAPwheaQIB0zyXFwAAQKYRvFph2SAAAOAXglcrLJQNAAD8QvBqZUBRhHm8AACALwherZQVh5m5HgAA+ILg1crAooj2sFA2AADwAcGrlYHFETXFk6qPJXJdFAAA0MsQvFpJzV7POC8AAJBpBK9WUgtlM84LAABkGsGrldRC2bR4AQCATCN4tZJq8WIuLwAAkGkEr1ZSY7z20uIFAAAyjODVSv9oWMZIexnjBQAAMozg1UooGFC/wjBdjQAAIOMIXm0oK47Q4gUAADKO4NWGAUVhxngBAICMI3i1oawoQlcjAADIOIJXGwYURWjxAgAAGUfwakNZcZgxXgAAIOMIXm0YUBRRfSyhBhbKBgAAGUTwakNZMbPXAwCAzCN4tSE1ez3rNQIAgEwieLUhtV7jPsZ5AQCADCJ4tWGg19VIixcAAMgkglcbWlq8CF4AACBzCF5tGNA8xouuRgAAkDkErzaEgwGVFoY4qhEAAGQUwasdA1k2CAAAZBjBqx0DiyPMXg8AADKK4NWOgUVh1msEAAAZRfBqRxldjQAAIMMIXu0YUBShxQsAAGQUwasdZcVh1TYl1BhnoWwAAJAZBK92pGavZ9kgAACQKaFc7NQYs1nSAUkJSXFrbUUuynE4qdnr99Y1qbxfYY5LAwAAeoOcBC/PPGvtrhzu/7CG9Xdha+POWk0e1i/HpQEAAL0BXY3tmD6yv/oVhvS3d3bkuigAAKCXMNba7O/UmE2S9kqykv7bWntXG/e5VtK1klReXn7C4sWLfS1TTU2NSkpKDrrul2826O3dCf1kXpECxvi6fxxeW/WD/EDd5DfqJ39RN/mtO/Uzb968Fe0No8pVV+Nca+02Y8xQSc8YY96x1j6ffgcvjN0lSRUVFbaystLXAlVVVan1Pj4e8KFuXLxSZcfM0syjBvi6fxxeW/WD/EDd5DfqJ39RN/nNr/rJSVejtXabt90h6X8knZSLchzJaROHKGCkZ9duz3VRAABAL5D14GWMKTbGlKbOSzpT0upsl6MjBhRFVDGmTH9dyzgvAADQfblo8SqX9KIx5k1Jr0p6wlr7VA7K0SGnHztUb1fvV/XH9bkuCgAA6OGyHrystRuttTO801Rr7cJsl6Ezzpg8VJL0LEc3AgCAbmI6iSM4ZmiJjiqL6lm6GwEAQDcRvI7AGKMzJpfrxfd2qb6JdRsBAEDXEbw64PTJQ9UYT2rZxrydaB8AAPQABK8OmH10mYojQS2luxEAAHQDwasDCkJBfXLCED37zg7lYqZ/AADQOxC8Ouj0Y4eq+uMGra0+kOuiAACAHorg1UHzJqWmlWAWewAA0DUErw4aUlqgGUcN0FLm8wIAAF1E8OqEMyYP1coP9mlXTWOuiwIAAHogglcnnD55qKyVbvnz2/q4Ppbr4gAAgB6G4NUJU0f009fOmKAn36rW/B8/p7+s+SjXRQIAAD0IwasTjDH6xvyJevT6uRpUUqBr71+hG37/Ol2PAACgQwheXTBtZH89dsNcfXP+RP1lzXbN//FzWvJWda6LBQAA8hzBq4vCwYC+esYEPfG1T2h0WZG+suh13fLnNWqKJ3NdNAAAkKcIXt00obxUf7zuFF01d6x+89JmffGuZdq2r77Dj48nknRVAgDQR4RyXYDeIBIK6HufnqqKMWX6zkNv6ryfvqCfXDxLp00c0ub999U16bl3d2rp2h167t2d+rg+pvOmD9c/nzVZowcVZbn0AAAgWwheGXTe9OGaPLxU//S713Xlb17VvElDFQ0HZYwUMEYBI23b16Dl7+9R0kqDiiOaP6VcZcUR/XbZZv1lzUe6bM5YffX0YzSwOJLrlwMAADKM4JVh44eU6JHr52rhk2/rlY17ZCUlrZW1btuvMKzr5x2j0ycP1YxRAxQIGEnS1Z8Ypx//5V3d+/Im/XHFB7p+3jH63KyRGtqvMLcvCAAAZAzBywfRSFA//MxxnXpMeb9C/dtF07XgE+P0oyVrdfuSd3T7knc0qbxUn5gwWJ+cMFizxw1SNBL0qdQAAMBvBK88M2lYqe696iS9vW2/nl+/Uy+s36n7//6+fv3iJkWCAX2hYpS+eeYkldEVCQBAj0PwylNTRvTTlBH9dN1p41XflNBrm/doyeqPtPi1D/T4qmp9Y/5EfWn2aIWChx6Yar2uzVQ3JgAAyA8Erx4gGgnq1IlDdOrEIbrylLH6weNr9L3H1uj3r2zR9y6Yomkj++vND/bpjS379MaWvXrjg32qaYhrxICoRg2MauSAqEYNLNLoQVEdM6RU44cWqyhC1QMAkG18+/Ywk4aV6ndXz9bTa7brh0+8rUvvfkXGSNZKxkgThpborCnDVFYS0Yd767V1b52eX79T2/cfPFfYqIFRTRhaoonDSnXS2DKdOK5M/QrDOXpVAAD0DQSvHsgYo7OnDVPlpCH6/StbVNsY16zRAzX9qP7thqeGWEIf7KnTeztqtD512n5AL723W//93EYFjFsKac7RgzTn6DLNHjdIxQW8PQAAyCS+WXuwwnBQCz4xrsP3nVBeqgnlpTon7fqGWEJvbNmnZRt36+8bd+velzbrruc3Khw0mjV6oE6dMFifnDBE00b2V5AxYwAAdAvBq48rDAd18vhBOnn8IEkuiK14f69efG+XXli/U3f85V3d8Zd3NaAorJOPHqRTjhmsueMHadzgYhlDEAMAoDMIXjhIYTiouccM1txjBuufz56s3TWNXgjbpZff26Ulqz+SJA3vX6iTxw/StBH9Nax/ocr7Fai8X6GGlhYqEnJHWiaTVrFkUk3xpIIBw4B+AECfxzchDmtQSYEunDlSF84cKWut3t9dp5c27NLLG3arat1OPfz6h4c8JhoOKpZIKp60B10/amBUk4eVavKwfpo8vFTjh5QoGDCKJZJKJK1iCaukteofDWtQcUQDiiJ0bwIAehWCFzrMGKOxg4s1dnCxvjR7jKy12lcX00f7G/TR/gbt2N+gjz5uVE1jTJFQQOGgOxWEAmqIJbRue43WfbRff1u3U4lWoawtASOVFRdocElETfX1+smal2SMZLyyGLllmJLWm7tM7rbUfsOhgCLBgMJBo4AxB62ZaYxRImmVsFbWWiWS7nkKw0GVFIRUWhhScSSkksKQBkTDGlQS0aDiAg0qiaisOKLCcNdXEGiIJdSUSMomU+V3+y4IB1QcCRE2AaAXI3ihy4wxGlgc0cDiiI4d3q/Dj2uMJ/Tejhpt3lUnSQoGjMJBo2DABaT9DTHtOtCo3bVN2lXTpF01jareUafSQvd2tVaycpPEBgOmOYQFjJS0UjyZVCxuVVcfUyyeVCyRdOtleo9NhZ2gMQoEjILGNIedxnhSNY1x1TTEVR9LtPsaiiNBDSiKaEBRWAO9bb9oWNFwUNFwUIXhgArDQQWMUfXH9dq6N3Wq09662GH/PtFwUMUFIZUUBFUUCam41TYSCigVzVLD7EKBgPoVhtS/KKL+0bAGRF15IqGAQgHjhVG3TQXLtibfBQD4i+CFrCsIBTV1RH9NHdG/w4+pqqpSZeVsH0t1qHjChbB9dTHtrm3U7pom7a5t0u6aRu2pjWlffZP21cW0r65J2/bVa39DTA2xpOpjiYNa9ApCAY0a6CaxnT6qv0YMiKogFGhufQsEXHBsiLn91TbGVdsUV01jQvVNcdU2JrSvrkkf7kuovimhxrgLhDat0bDJK6s9ckNis5KCkPoVhtTPC2mpwNY/GtaAorAKw8HmBd5ToTUVeJNeC2HSWm3e3KTVyfUHtXJGggFFI0EVRYLeNuTOh91lF06DbbbupVZeMF7LJLoumXQtuocTTyRVF0uoMZaUtS31mnoPh4MBhbzQHvHOt1UrAe+HDIDDI3gB7QgFA16rVkRjBxd36rGxhBfAElYDisJZCRCJpFVNQ1z76pv0cX1M++vjiiWSakokFU/Y5vM1DXHtb3C372+IefeN6YM9dVpdH9O+uthhW/vatOHdLpU5EgwoEJCSSSmR9mWfEmjuHnatktFIUIWhgApTIc4LcKkWxsJwUAWpgzu8LuhUN3IoYBQOtQSISCigpkRSBxri3immAw1x1TclmruhE8mWU0p6kAmnhc1Ui2LL1p1PtSzGvbGMce/5jDHutXjlj4aDCgUDinvjI1PjHhPJlsvpW9scir2tlepiCdV6LbY1XoC3VoosXaKCUMA7BWWtVV0sobqmhJriyS7VXVsCxrW+BgNGoYBRQTjQErojQRVHQq3qK6DCUNC14nr/Ikam5Xzqx0mqVTtgmp87YIxCXkt5akhDJBX8Qy4gplq1U88RDJjmHwgR7zHtLbuWev8k01vJvf2HAwEFg24ruTpIv38wYJqHOfj1v596b1up+QdS0ntvHjy0omV4xeGeK/XeCgcDDHfwGcEL8EHqizebggGj/kVh9S/q/goEjXH3hZzqxnVj6w4eJ5f6cK+qqtLcT56mWCLZHO4aY0k1eF/s7hRXXVNC9bGEGmNuW9/kwqm1tvnLMZjWatLyRea+zOKJZHOLYn0soQbv+eqa4tpT6/bXEEuoIZ5s/pJOD24uyLjyNcXdNhx0XbSlhWGVFrou2IFFEYW8L1jXFZ1qlXTlSo0ztHJliiWsmlKvPe5Cbk3chd64d5tRWiDxwkIyabU9llRD3Ct3LKl4IqlQKrCl3T8UaLmcKleg1XhHY6T+0bBGDihUSUHI664Oacv772vYqKPUFE+qMZ5sDlrNrZFh141dEAp4z9vyRW0kxVN/t7gLgLE2glrqyz8VCtP/LnWNLXVf15TQ7tomNabVVUOsJfy5llWbdr7bb+WcMkbNYTccNJL3P5Sqr6bGJhUuW3rI41L/W+nhKZZw9dcYS7htFwJzMHDw/24y6YZmtB5yG/ICaqrsze87738hmAp0B/2PudvCARd6Q8GAwt7jUsNI0j8/goFA8xAT9/8WUDjkAmvEG6Ob+gxt/vGRSI3LlffYg5/DmJZyGuPK6cb6tvwYCgWNJpaXanBJQRdqNDMIXgAOURAKqiDUsQMIjHEf0qlpRJBfqqqqVVl5bK6L0WWtW51SYSGR1vqXCtJN8ZZQHU/Y5scmvMcm0kJyozf+M+b9wGgtFSSag6iMktYqlrRKeK2SsYRtvm8qVEiurOlBt9ErkwuSLS2U26qrNXz44FavVwe1YqVeQyhovP/LgArCARUEXWtdKmSkgk3qOZI27fUnD/1bpMbIpn5khLxAH09YNcZd17MLeAmvhVXNLcFuqMGhrYKJtJBe25RobuVNpt2n+XmStrkeYwkvrHt16befXzpL508f4ft+2kPwAgDkLddyIQUPGlnW9aOK80lV1R5VVs7IdTHyirUt3exN8aRk1BwOg15rm+QCZDzhwpvb2kNCYNK650m1xrtQaDWhvCSnr5HgBQAA8oLxxu6FglI00n7ADsjIzerT80I4fQMAAABZQvACAADIEoIXAABAlhC8AAAAsoTgBQAAkCUELwAAgCwheAEAAGQJwQsAACBLCF4AAABZQvACAADIkpwEL2PM2caYdcaY94wxN+WiDAAAANmW9eBljAlKulPSOZKmSLrEGDMl2+UAAADItly0eJ0k6T1r7UZrbZOkxZIuzEE5AAAAsspYa7O7Q2MuknS2tfYa7/JlkmZba29odb9rJV0rSeXl5ScsXrzY13LV1NSopKTE132g66if/EXd5DfqJ39RN/mtO/Uzb968FdbairZuC3WrVF1j2rjukPRnrb1L0l2SVFFRYSsrK30tVFVVlfzeB7qO+slf1E1+o37yF3WT3/yqn1wEr62Sjkq7PErStsM9YMWKFbuMMe/7WippsKRdPu8DXUf95C/qJr9RP/mLuslv3amfMe3dkIuuxpCkdyWdIelDSa9JutRauyarBTm0XMvbaxZE7lE/+Yu6yW/UT/6ibvKbX/WT9RYva23cGHODpKclBSXdk+vQBQAAkA256GqUtfZJSU/mYt8AAAC5wsz1Le7KdQFwWNRP/qJu8hv1k7+om/zmS/1kfYwXAABAX0WLFwAAQJYQvAAAALKE4CUW7c4nxpijjDF/M8asNcasMcbc6F1fZox5xhiz3tsOzHVZ+ypjTNAY84Yx5nHvMnWTJ4wxA4wxDxlj3vH+h06mfvKHMeZ/e59rq40xDxhjCqmf3DHG3GOM2WGMWZ12Xbv1YYz5Fy8nrDPGnNXV/fb54MWi3XknLumb1tpjJc2RdL1XHzdJWmqtnSBpqXcZuXGjpLVpl6mb/PGfkp6y1k6WNEOunqifPGCMGSnpa5IqrLXT5KZTuljUTy7dK+nsVte1WR/e99DFkqZ6j/mFlx86rc8HL7Fod16x1lZba1/3zh+Q++IYKVcn93l3u0/SZ3JSwD7OGDNK0nmSfpV2NXWTB4wx/SSdKunXkmStbbLW7hP1k09CkqLeROJFcqu2UD85Yq19XtKeVle3Vx8XSlpsrW201m6S9J5cfug0gpf7Uv8g7fJW7zrkmDFmrKRZkl6RVG6trZZcOJM0NIdF68t+Iuk7kpJp11E3+eFoSTsl/cbrCv6VMaZY1E9esNZ+KOkOSVskVUv62Fr7F1E/+aa9+shYViB4dXDRbmSXMaZE0p8kfd1auz/X5YFkjDlf0g5r7YpclwVtCkk6XtJ/WWtnSaoV3VZ5wxsrdKGkcZJGSCo2xnw5t6VCJ2QsKxC8urBoN/xljAnLha5F1tqHvau3G2OGe7cPl7QjV+Xrw+ZKusAYs1muS/50Y8zvRN3ki62StlprX/EuPyQXxKif/PApSZustTuttTFJD0s6RdRPvmmvPjKWFQhebpHuCcaYccaYiNzgucdyXKY+yxhj5MaorLXW/jjtpsckXeGdv0LSo9kuW19nrf0Xa+0oa+1Yuf+TZ621XxZ1kxestR9J+sAYM8m76gxJb4v6yRdbJM0xxhR5n3NnyI1hpX7yS3v18Ziki40xBcaYcZImSHq1Kztg5npJxphz5caupBbtXpjbEvVdxphPSHpB0ltqGUf0XblxXg9KGi33AfYFa23rQZHIEmNMpaRvWWvPN8YMEnWTF4wxM+UOfIhI2ijpKrkf2NRPHjDG3CLpi3JHb78h6RpJJaJ+csIY84CkSkmDJW2X9D1Jj6id+jDG/KukBXL193Vr7ZIu7ZfgBQAAkB10NQIAAGQJwQsAACBLCF4AAABZQvACAADIEoIXAABAlhC8APQqxpgfGWMqjTGfMca0OXO7MeY6Y8zl3vkrjTEjsltKAH0VwQtAbzNbbt630+TmhDuEtfaX1trfehevlFvCpcO8RY4BoNOYxwtAr2CM+b+SzpJbC2+DpPGSNkl6yFr7g1b3/b6kGkmbJd0r6UNJ9ZJOljRF0o/lJrbcJelKa221MaZK0stySyc9Zq39f36/JgC9D7/aAPQK1tpvG2P+KOkySd+QVGWtnXuExzxkjLlBbhb+5d46oT+TdKG1dqcx5ouSFsrNVi1JA6y1p/n4MgD0cgQvAL3JLEkrJU2WW6ewsyZJmibpGbecnoKSqtNu/0M3ywegjyN4AejxvDUK75U0Sq57sMhdbVZKOtlaW9/Rp5K0xlp7cju313avpAD6OgbXA+jxrLUrrbUzJb0rN0brWUlnWWtndiB0HZBU6p1fJ2mIMeZkSTLGhI0xU30qNoA+iOAFoFcwxgyRtNdam5Q02Vrb0a7GeyX90msdC0q6SNK/GWPelOu2PCXzpQXQV3FUIwAAQJbQ4gUAAJAlBC8AAIAsIXgBAABkCcELAAAgSwheAAAAWULwAgAAyBKCFwAAQJb8/58D6yAJOi0BAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x504 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "n_trees = 100\n",
    "train_loss = []\n",
    "test_loss = []\n",
    "\n",
    "for i in range(1, n_trees):\n",
    "    rf = RandomForestRegressor(n_estimators = i, random_state = 123)\n",
    "    rf.fit(X_train, y_train)\n",
    "    train_loss.append(mean_squared_error(y_train, rf.predict(X_train)))\n",
    "    test_loss.append(mean_squared_error(y_test, rf.predict(X_test)))\n",
    "    \n",
    "plt.figure(figsize = (10, 7))\n",
    "plt.grid()\n",
    "plt.plot(train_loss, label = 'MSE_train')\n",
    "plt.plot(test_loss, label = 'MSE_test')\n",
    "plt.ylabel('MSE')\n",
    "plt.xlabel('# trees')\n",
    "plt.legend();"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bacterial-swiss",
   "metadata": {},
   "source": [
    "Как и ожидалось, по достижении некоторого числа деревьев обе ошибки практически не изменяются, то есть переобучения при росте числа деревьев не происходит.\n",
    "\n",
    "Однако практика показывает, что при изменении какого-нибудь другого параметра на реальных данных переобучение может произойти: [пример 1](https://datascience.stackexchange.com/questions/1028/do-random-forest-overfit), [пример 2](https://mljar.com/blog/random-forest-overfitting/). Например, случайный лес с ограниченными по глубине деревьями может предсказывать более точно, чем лес без ограничений. \n",
    "\n",
    "В нашем же случае случайный лес, скорее, лишь страдает от регуляризации. Например, посмотрим на поведение модели при изменении максимальной глубины деревьев (поэксперементируйте с другими параметрами)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 271,
   "id": "designing-wilson",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAl4AAAGqCAYAAADaykoUAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABCC0lEQVR4nO3deZxcdZ3v/9e3q1e6O52EdNoskA47CYQEGsOi2BFBQRRRXACVdVAHHeY3ymPUO4PboDDDXO/1XpfBDcQM0euAMiiODBhZRDCBEBLCGsIasxCSdCedXr+/P051V5P03l1Vvbyej0c9qurUcj79sSTvx/d8z/eEGCOSJEnKvoJ8FyBJkjRRGLwkSZJyxOAlSZKUIwYvSZKkHDF4SZIk5UhhvgsYiGnTpsXa2tqs7mPXrl2Ul5dndR9jhb3IsBcZ9iJhHzLsRYa9yLAXsHLlyq0xxuqeXhsTwau2tpYVK1ZkdR/Lly+nvr4+q/sYK+xFhr3IsBcJ+5BhLzLsRYa9gBDCC7295qFGSZKkHMl68AohpEIIj4YQ7kg/nxpCuCuE8Ez6fkq2a5AkSRoNcjHidSWwrtvzzwN3xxgPBe5OP5ckSRr3sjrHK4QwG3g3cA3wd+nNZwP16cc3AcuBv89mHZIkTXStra28/PLL7NmzJ6v7qaqqYt26df2/cRwoLS1l9uzZFBUVDfgzIZvXagwh/AL4BlAJfC7GeFYIYXuMcXK397weY9zncGMI4XLgcoCamprjli1blrU6ARobG6moqMjqPsYKe5FhLzLsRcI+ZNiLjLHQi4qKCmpqaqiqqiKEkLX9tLe3k0qlsvb9o0WMkR07drBp0yYaGxvf8NqSJUtWxhjrevpc1ka8QghnAZtjjCtDCPWD/XyM8QbgBoC6urqY7TMkPAsjw15k2IsMe5GwDxn2ImMs9GLdunXMnj07q6ELoKGhgcrKyqzuY7SorKyksbGRuroeM1aPsnmo8WTgvSGEM4FSYFII4afAphDCjBjjxhDCDGBzFmuQJElp2Q5dE81Q+pm1yfUxxi/EGGfHGGuBjwD3xBg/CtwOXJh+24XAr7JVgyRJ0miSj3W8rgVOCyE8A5yWfi5JkjTu5SR4xRiXxxjPSj9+LcZ4aozx0PT9tlzUIEmS8iuEwMc+9rGu521tbVRXV3PWWWcBsGnTJs466yyOOeYY5s2bx5lnngnAhg0bKCsrY+HChV23n/zkJ73u5+tf//qQ6rvssst44oknhvTZgRoTlwySJEljX3l5OWvWrKGpqYmysjLuuusuZs2a1fX61VdfzWmnncaVV14JwOrVq7teO/jgg1m1atWA9vP1r3+dL37xi/tsjzESY6SgoOdxpx/84AeD+GuGxuAlSdIE85X/XMsTr+4c0e+cN3MSX3rP/H7fd8YZZ/DrX/+ac889l1tuuYXzzjuP++67D4CNGzdy+umnd713wYIFg67j85//PE1NTSxcuJD58+dzzTXXcMYZZ7BkyRIefPBBfvnLX3Lttdfy5z//maamJs4991y+8pWvAFBfX8/1119PXV0dFRUVXHnlldxxxx2UlZXxq1/9ipqamkHXszev1ShJknLmIx/5CMuWLWPPnj2sXr2axYsXd712xRVXcOmll7JkyRKuueYaXn311a7XnnvuuTccauwMa3u79tprKSsrY9WqVSxduhSAp556io9//OM8+uijzJkzh2uuuYYVK1awevVq/vCHP7xhZK3Trl27OOGEE3jsscc45ZRT+P73vz8if78jXpIkTTADGZnKlgULFrBhwwZuueWWrjlcnd75zneyfv16fvvb33LnnXeyaNEi1qxZAwzuUOPe5syZwwknnND1/Oc//zk33HADbW1tbNy4kSeeeGKf0bXi4uKuuWfHHXccd91115D2vTdHvAA62inZszXfVUiSNCG8973v5XOf+xznnXfePq9NnTqV888/n5tvvpnjjz+ee++9d9j7Ky8v73r8/PPPc/3113P33XezevVq3v3ud/d4GaWioqKudbpSqRRtbW3DrgMMXok/fosT/3QpNDfkuxJJksa9Sy65hKuvvpqjjz76Ddvvuecedu/eDSQr4D/33HMceOCBg/7+oqIiWltbe3xt586dlJeXU1VVxaZNm7jzzjsH/wcMg8ELYPKc5P71F/JbhyRJE8Ds2bO7zlzsbuXKldTV1bFgwQJOPPFELrvsMo4//nhg3zle3/rWt3r9/ssvv5wFCxZwwQUX7PPaMcccw6JFi5g/fz6XXHIJJ5988sj9YQPgHC+AKengtf0FeNNR+a1FkqRxau+LSUNyJmHndS6vuuoqrrrqqn3eU1tbS1NT04D3c91113Hdddd1Pe+cJ9bpxhtv7PFzy5cv77HWc889l3PPPXfA+++LI14AU+Ym969vyGsZkiRpfHPEC6BsCm2p/Sg0eEmSNGYsXryY5ubmN2y7+eab95k7NpoYvABCYE9pDRXO8ZIkacx46KGH8l3CoHmoMa2prMZDjZIkKasMXml7SmuSyfUx5rsUSZI0Thm80prKaqBtDzRuyncpkiRpnDJ4pe0pTV/40nlekiQpSwxeaXtK35Q8cJ6XJElZEULgYx/7WNfztrY2qquru66JuGnTJs466yyOOeYY5s2b13Utxw0bNlBWVvaGBVR/8pOf9Lqfr3/960Ou8cYbb3zDxblHmmc1pu0pnZ482O6IlyRJ2VBeXs6aNWtoamqirKyMu+66i1mzZnW9fvXVV3Paaad1rWq/evXqrtcGc5Hsr3/963zxi18cUo033ngjRx11FDNnzhzS5/tj8ErrSBVD5QxHvCRJ49+dn4e/PD6y3/mmo+GMa/t92xlnnMGvf/1rzj33XG655RbOO+887rvvPgA2btzI6aef3vXeBQsWDLqMz3/+8zQ1NbFw4ULmz5/P0qVL+elPf8q3vvUtWlpaWLx4Md/5zncAuPTSS1mxYgUhBC655BIOOOAAVqxYwQUXXEBZWRkPPvggZWVlg66hLx5q7G5KrcFLkqQs+shHPsKyZcvYs2cPq1evZvHixV2vXXHFFVx66aUsWbKEa6655g2H/Pa+VmNnWNvbtddeS1lZGatWrWLp0qWsW7eOn/3sZzzwwAOsWrWKVCrF0qVLWbVqFa+88gpr1qzh8ccf5+KLL+bcc8+lrq6u6/WRDl3giNcbTZ4DG+7PdxWSJGXXAEamsmXBggVs2LCBW265pWsOV6d3vvOdrF+/nt/+9rfceeedLFq0qOs6i4M51Njd3XffzcqVK7sutt3U1MT06dN5z3vew/r16/nMZz7Du9/97jeMtGWTI17dTamFna9AW3O/b5UkSUPz3ve+l8997nOcd955+7w2depUzj//fG6++WaOP/547r333mHtK8bIhRdeyKpVq1i1ahVPPfUUX/7yl5kyZQqPPfYY9fX1fPvb3+ayyy4b1n4GyuDV3ZQ5QIQdL+e7EkmSxq1LLrmEq6++ep9rKt5zzz3s3r0bgIaGBp577jkOPPDAQX9/UVERra2tAJx66qn84he/YPPmzQBs27aNF154ga1bt9LR0cEHPvABvva1r/HII48AUFlZSUNDw3D+vD55qLG7KbXJ/evPw/4H57UUSZLGq9mzZ3edudjdypUr+fSnP01hYSEdHR1cdtllHH/88WzYsKFrjlenSy65hL/5m7/p8fsvv/xyFixYwLHHHsvSpUv5p3/6J04//XQ6OjooKiri29/+NmVlZVx88cV0dHQA8I1vfAOAiy66iE9+8pNZm1xv8Opu8pzk3kVUJUkacY2Njftsq6+vp76+HoCrrrqKq666ap/31NbW0tTUNOD9XHfddVx33XVdzz/84Q/z4Q9/eJ/3dY5ydfeBD3yAD3zgAwPe12B5qLG7yhmQKvbMRkmSlBWOeHVXUACTD3QRVUmSxoDFixfT3PzGE+JuvvnmfeaOjSYGr725lpckaZyKMRJCyHcZI+ahhx7K6/5jjIP+jIca92bwkiSNQ6Wlpbz22mtDCgvaV4yR1157jdLS0kF9zhGvvU2eA3t2QNPrUDYl39VIkjQiZs+ezcsvv8yWLVuyup89e/YMOoyMVaWlpcyePXtQnzF47a1rSYkXDF6SpHGjqKiIuXPnZn0/y5cvZ9GiRVnfz1jloca9TUkvKeEEe0mSNMIMXnvrGvHakM8qJEnSOGTw2ltpFZROdhFVSZI04gxePfHMRkmSlAUGr54YvCRJUhYYvHoyZQ7seAk62vNdiSRJGkcMXj2ZUgvtLdCwMd+VSJKkcSRrwSuEUBpCeDiE8FgIYW0I4Svp7V8OIbwSQliVvp2ZrRqGbHJ6SQkn2EuSpBGUzQVUm4G3xxgbQwhFwP0hhDvTr30zxnh9Fvc9PN2XlKg9OZ+VSJKkcSRrwSsmF4NqTD8tSt/GxgWiqg4AgouoSpKkEZXVOV4hhFQIYRWwGbgrxth5GfFPhxBWhxB+FEIYfdflKSyGqtme2ShJkkZUyMVVykMIk4HbgM8AW4CtJKNfXwNmxBgv6eEzlwOXA9TU1By3bNmyrNbY2NhIRUVF1/OFj/4PQmzn0WOvzep+R6O9ezGR2YsMe5GwDxn2IsNeZNgLWLJkycoYY11Pr+UkeAGEEL4E7Oo+tyuEUAvcEWM8qq/P1tXVxRUrVmS1vuXLl1NfX5/Z8Msr4Nn/hs89ldX9jkb79GICsxcZ9iJhHzLsRYa9yLAXEELoNXhl86zG6vRIFyGEMuAdwJMhhBnd3nYOsCZbNQzLlFpo/Au0NuW7EkmSNE5k86zGGcBNIYQUScD7eYzxjhDCzSGEhSSHGjcAn8hiDUM3Jb2kxPYXofrw/NYiSZLGhWye1bgaWNTD9o9la58jqvuSEgYvSZI0Aly5vjcuoipJkkaYwas3FdOhsMwlJSRJ0ogxePUmhGSel4uoSpKkEWLw6suUWke8JEnSiDF49aUzeOVorTNJkjS+Gbz6MnkOtDTC7m35rkSSJI0DBq++dF9SQpIkaZgMXn3pWkR1Q17LkCRJ44PBqy9da3ltyGsZkiRpfDB49aWkAvab5iKqkiRpRBi8+uOSEpIkaYQYvPrjIqqSJGmEGLz6M6UWtr8E7W35rkSSJI1xBq/+TKmF2A47X853JZIkaYwzePWn68xGDzdKkqThMXgBG7bu4o+vttHR0cOlgVxEVZIkjRCDF3DPk5u5YXUzO5pa931x0iwIKSfYS5KkYTN4AdMqSwDY0ti874upQph8gCNekiRp2AxeQHVFOng19BC8IJnn5RwvSZI0TAYvoLqyn+DlIqqSJGkEGLwYSPCaA7u3QnNjDquSJEnjjcELmFRaSGEBbO1pjhdkzmx0gr0kSRoGgxcQQqCqOPR9qBE83ChJkobF4JVWVRJ6PqsRYHJtcu8Ee0mSNAwGr7Sqkj5GvPabCsWVjnhJkqRhMXil9Rm8Qkgm2DvHS5IkDYPBK62qOLBtdwut7R09v8ElJSRJ0jAZvNKqSgIxwrZdLT2/oXMR1djD9RwlSZIGwOCVVlUSgH4WUW1rgsbNuStKkiSNKwavtKridPDqby0vDzdKkqQhMnil9T/iNSe5d4K9JEkaIoNX2qT+gtfkA5N7R7wkSdIQGbzSSlKBypLC3oNXURlUvMlFVCVJ0pAZvLqprizpfY4XuKSEJEkaFoNXN9MqSnof8QIXUZUkScNi8OqmurKErX0Gr1rY8TK09bLWlyRJUh8MXt1UV/Yz4jV5DhBhx0s5q0mSJI0fBq9uqitLaGhuY09re89vcC0vSZI0DFkLXiGE0hDCwyGEx0IIa0MIX0lvnxpCuCuE8Ez6fkq2ahis6ooSoJ/V68HgJUmShiSbI17NwNtjjMcAC4F3hRBOAD4P3B1jPBS4O/18VKiuTAev3s5srJwBqWIn2EuSpCHJWvCKicb006L0LQJnAzelt98EvC9bNQxWV/DqbcSroCBZSNURL0mSNARZneMVQkiFEFYBm4G7YowPATUxxo0A6fvp2axhMPoNXpBMsHcRVUmSNAQhxpj9nYQwGbgN+Axwf4xxcrfXXo8x7jPPK4RwOXA5QE1NzXHLli3Lao2NjY2U7lfOX/1uN+89uIhzDi3u8X2HPv09pm++jwfesjSr9eRTY2MjFRUV+S5jVLAXGfYiYR8y7EWGvciwF7BkyZKVMca6nl4rzEUBMcbtIYTlwLuATSGEGTHGjSGEGSSjYT195gbgBoC6urpYX1+f1RqXL19OfX09Ux64i/Jpb6K+/uie31j0GLx6J/WLF0LZ5KzWlC+dvZC96M5eJOxDhr3IsBcZ9qJv2TyrsTo90kUIoQx4B/AkcDtwYfptFwK/ylYNQ1Hd7+r1tcm9E+wlSdIgZXPEawZwUwghRRLwfh5jvCOE8CDw8xDCpcCLwAezWMOgVVeWsLW/6zVCMsF+xjG5KEmSJI0TWQteMcbVwKIetr8GnJqt/Q5XdWUJGzbs6v0Nk+ck906wlyRJg+TK9XvpvGxQrycdlE2G0skuKSFJkgbN4LWX6ooSmts6aGhu6/1NU+Y4x0uSJA2awWsvA1rLa0qtI16SJGnQDF57GfAiqttfhI6OHFUlSZLGA4PXXgY84tXeAg0bc1OUJEkaFwxee5lWMZDglT6z0XlekiRpEAxee5lcVkRhQehnLa+5yb3zvCRJ0iAYvPZSUBCY1t/q9VUHAMHgJUmSBsXg1YPqyhK29DXiVVgMk2a5iKokSRoUg1cPOhdR7ZNLSkiSpEEyePWg3wtlg4uoSpKkQTN49aC6soTXdrXQ3tHLZYMgGfFq2AitTTmrS5IkjW0Grx5UV5bQ3hF5fXdL72/qvFj29pdyU5QkSRrzDF49GNhaXrXJvfO8JEnSABm8etC5en3fa3nVJvfb1me/IEmSNC4YvHowoMsGVUyHsimwZV2OqpIkSWOdwasHAwpeIcD0ebDpiRxVJUmSxjqDVw/Ki1OUFaX6X1Ji+jzYvA5iH2c/SpIkpRm8ehBC6H/1eoCaedDSADs8s1GSJPXP4NWLAa1eP31ecu/hRkmSNAAGr14MaPX66Ucm95sNXpIkqX8Gr14M6FBjaRVMmm3wkiRJA2Lw6sW0ihK2726lpa2j7zfWpCfYS5Ik9cPg1YvOJSVe2zWAw41bnoL21hxUJUmSxjKDVy8GtJYXwPT50NEKrz2bg6okSdJYZvDqxYCDV036zEbneUmSpH4YvHox4OA17TAIKZeUkCRJ/TJ49WJaRTEwgOBVWAL7H+IEe0mS1C+DVy9KClNUlRX1v6QEJBPsN6/NflGSJGlMM3j1YUCr1wPUzIfXN0BzY9ZrkiRJY5fBqw/VFSVsHdCIV3qC/ZansluQJEka0wxefZg20BGvrksHebhRkiT1zuDVhwFdrxFgylwoLHOCvSRJ6pPBqw/VlSXsamlnV3Nb328sKIDpR8AmR7wkSVLvDF596FzLa2DzvOY74iVJkvpk8OrDgBdRhWSe167NsGtrlquSJEljlcGrD9UVgwhenZcO8nCjJEnqhcGrD10jXgM91AgebpQkSb3KWvAKIRwQQvh9CGFdCGFtCOHK9PYvhxBeCSGsSt/OzFYNwzW1vJiCAFsHMuJVMR3KprqkhCRJ6lVhFr+7DfhsjPGREEIlsDKEcFf6tW/GGK/P4r5HRKogMLW8ZGAjXiEkK9g74iVJknqRtRGvGOPGGOMj6ccNwDpgVrb2ly0DvmwQpK/ZuA46OrJblCRJGpNCjDH7OwmhFrgXOAr4O+AiYCewgmRU7PUePnM5cDlATU3NccuWLctqjY2NjVRUVOyz/foVe9jVGvnSiWX9fseMV/+Lw5/+Dn9afAN7ymqyUWZO9NaLicheZNiLhH3IsBcZ9iLDXsCSJUtWxhjrenot68ErhFAB/AG4JsZ4awihBtgKROBrwIwY4yV9fUddXV1csWJFVutcvnw59fX1+2z/7M8f48HntvLHL5za/5e89DD88DQ4bxkcfsbIF5kjvfViIrIXGfYiYR8y7EWGvciwFxBC6DV4ZfWsxhBCEfAfwNIY460AMcZNMcb2GGMH8H3gzdmsYbiqK5M5XgMKqNVHJPcuKSFJknqQzbMaA/BDYF2M8X922z6j29vOAdZkq4aRUF1ZQmt7ZEdTa/9vLp0EVQc6wV6SJPUom2c1ngx8DHg8hLAqve2LwHkhhIUkhxo3AJ/IYg3D1n31+sn7Fff/gelHwuYnslyVJEkai7IWvGKM9wOhh5d+k619ZkPX6vWNzRxaU9n/B2rmwXN3Q1sLFA4gqEmSpAnDlev7MajrNUKygn1HG7z2bBarkiRJY5HBqx+Dul4jJIcawcONkiRpHwavfkwqK6Q4VTCw1esBph0GBYUGL0mStA+DVz9CCINbvb6wGPY/BDYZvCRJ0hsZvAZg2mCCF8D0eY54SZKkfRi8BqC6YgjBa/sL0NyQvaIkSdKYY/AagOrKErYOdI4XJEtKAGx5KjsFSZKkMcngNQDVlSW8tquFtvaOgX1gejp4eekgSZLUjcFrAKorS4gRtu1uGdgHJs+BonLneUmSpDcweA3AoNfyKiiA6UcYvCRJ0hsYvAagujK59M/gJtgf6ZISkiTpDQxeA1BdUQoMNnjNh91boXFLlqqSJEljjcFrAKZ1jngN5czGzU6wlyRJCYPXAOxXXEhFSeHg1/IC2LwuO0VJkqQxx+A1QIO6bBBAxXTYb5pLSkiSpC4GrwEa9Or1kEyw98xGSZKUZvAaoEGvXg9QMx82PwkdA1x4VZIkjWsGrwEa9KFGSEa8Wncl122UJEkTnsFrgKorS9i5p409re0D/9D0+cm9E+wlSRIGrwGbVpEsKTGow43Tj0juXVJCkiRh8Bqw6spBXjYIoKQSJh/oCvaSJAkweA3YkFavh+Rwo4caJUkSBq8B6xrxGuyZjdOPhNeegbaWLFQlSZLGEoPXAO1fMYQLZUOypERHWxK+JEnShGbwGqCiVAFTy4sHv5ZX56WDnOclSdKEZ/AahCGtXr//IVBQ6Ar2kiTJ4DUYQ1pEtbAYph1m8JIkSQavwZhWUTz4yfWQTLD3UKMkSROewWsQOke8YoyD++D0ebDjRdizMzuFSZKkMcHgNQjVlSXsae2gsbltcB+sSV86aMuTI1+UJEkaMwxegzCk1eshOdQIzvOSJGmC6zN4hRA+2u3xyXu99ulsFTVaDXn1+qoDobjCeV6SJE1w/Y14/V23x/9nr9cuGeFaRr3OEa+tjYNchb6gAKqPcMRLkqQJrr/gFXp53NPzcS9zqHHP4D9cMy8JXoOdmC9JksaN/oJX7OVxT8/HvcllRRQWhCEuKTEPdr8GjZtHvjBJkjQmFPbz+hEhhNUko1sHpx+Tfn5QVisbhQoKAtOGsno9ZC4dtPkJqKwZ2cIkSdKY0F/wOjInVYwh0yqLhxa8OpeU2PwEHLxkZIuSJEljQp/BK8b4QvfnIYT9gVOAF2OMK7NZ2GhVXVEytEON5dOgvNoJ9pIkTWD9LSdxRwjhqPTjGcAakrMZbw4h/G0/nz0ghPD7EMK6EMLaEMKV6e1TQwh3hRCeSd9PGZk/JTeGdL3GTtPnuaSEJEkTWH+T6+fGGNekH18M3BVjfA+wmP6Xk2gDPhtjPBI4AbgihDAP+Dxwd4zxUODu9PMxo7qyhK2NLXR0DOHcgunzktXrOzpGvjBJkjTq9Re8Wrs9PhX4DUCMsQHoMz3EGDfGGB/p9v51wCzgbOCm9NtuAt436KrzqLqihPaOyPam1v7fvLeaedC6G7ZvGPG6JEnS6Bf6uuBzCOE/gd8BLwM/IhkB2x5CKANWxBjnD2gnIdQC9wJHkcwPm9zttddjjPscbgwhXA5cDlBTU3PcsmXLBvo3DUljYyMVFRX9vu/hv7TxnVXN/NPJZcyuHNwVlyp3Ps1xj1zFmvlfYGv1CUMtNesG2ouJwF5k2IuEfciwFxn2IsNewJIlS1bGGOt6eq2/sxovBb4KvAP4cIxxe3r7CcCPB7LzEEIF8B/A38YYd4YwsHVXY4w3ADcA1NXVxfr6+gF9bqiWL1/OQPax3/Pb+M6qB6k9YgFvOXTa4HbSXAePXMVR01Pwtv73lS8D7cVEYC8y7EXCPmTYiwx7kWEv+tbfWY2bgU/2sP33wO/7+/IQQhFJ6FoaY7w1vXlTCGFGjHFjesL+mFpRtGv1+sYhrF5fUgFTamHz2pEtSpIkjQl9Bq8Qwu19vR5jfG8fnw3AD4F1Mcb/2e2l24ELgWvT978acLWjQOayQUM8s/GAxfDUb6G5MQlikiRpwujvUOOJwEvALcBDDO76jCcDHwMeDyGsSm/7Ikng+nkI4VLgReCDgyk438qLU5QWFQw9eB3/V7D6Z/DYLfDmvxrZ4iRJ0qjWX/B6E3AacB5wPvBr4JYYY7/HymKM99N7UDt1MEWOJiGE4a3ldcDxMOs4eOh7UHcpFAxugr4kSRq7+vxXP8bYHmP8bYzxQpIJ9c8Cy0MIn8lJdaPUkFev73TCX8Nrz8Kz/z1yRUmSpFGv3+GWEEJJCOH9wE+BK4BvAbf2/anxrbqyhK0NLUP/gnlnQ+UMeOi7I1eUJEka9fq7ZNBNwB+BY4GvxBiPjzF+Lcb4Sk6qG6WqK4c54pUqguMvhefugc1PjlxhkiRpVOtvxOtjwGHAlcAfQwg707eGEMLO7Jc3OlVXlLJtVwut7cO49M9xF0NhaTLXS5IkTQj9zfEqiDFWpm+Tut0qY4yTclXkaNO5pMRrjcM43Fg+DY7+IDy2DHZvG6HKJEnSaOYpdUMw7LW8Op3wKWhrgkdu6v+9kiRpzDN4DcG0imJgiKvXd1czH+aeAg//ANrbRqAySZI0mhm8hmDERrwAFn8Kdr4MT/7n8L9LkiSNagavIZhWMYLB67B3wpS58CeXlpAkabwzeA1BaVGKSaWFbB3O5PpOBSlY/Al46SF45ZHhf58kSRq1DF5DNKzLBu1t4QVQXOnSEpIkjXMGryEa0eBVOgkWfRTW3AoNfxmZ75QkSaOOwWuIqitLh7d6/d4WXw4dbfDnH47cd0qSpFHF4DVE1RUjOOIFMPUgOOxdsOJH0DrMZSokSdKoZPAaourKEhqb29jdMoLrb53wKdi9Fdb8YuS+U5IkjRoGryHqXER1a8MInNnYae4pMH0e/Ol7EOPIfa8kSRoVDF5D1LWI6nBXr+8uBFj8Sdj0OLzwwMh9ryRJGhUMXkOUWb1+BEe8ABZ8CMqmuqCqJEnjkMFriKZXlgKwcUfTyH5xURnUXQxP/hq2PT+y3y1JkvLK4DVE0yqKmVpezJMbG0b+y4+/LFnR/uHvj/x3S5KkvDF4DVEIgXkzJrF2446R//JJM2He++DRm6E5C8FOkiTlhcFrGObPnMTTf2mktb1j5L/8hE9B805YdcvIf7ckScoLg9cwzJs5iZb2Dp7Z1DjyXz67DmbVJddv7MhCsJMkSTln8BqG+TOrAHhi487s7OCET8G25+DZu7Lz/ZIkKacMXsMwd1o5ZUUp1r6ahXleAPPOhsqZLi0hSdI4YfAahlRB4IgZlax9NUsjXqkiOP5SWP972LwuO/uQJEk5Y/AapvkzJ7Hu1Z10dGTpEj/HXQyFpclcL0mSNKYZvIZp/swqGprbeOn13dnZQfn+yWr2j/0MtjyVnX1IkqScMHgN0/yZkwB4IluHGwFOuQpKKuHmc2D7i9nbjyRJyiqD1zAdVlNJqiBkb54XwOQD4WO3QUsj/ORsaNycvX1JkqSsMXgNU2lRikOqK7J3ZmOnNx0FF/wCGv4CN78fmrZnd3+SJGnEGbxGwPyZk7I74tXpgDfDR5bClifh3z8ELbuyv09JkjRiDF4jYN7MSWxuaGZLQ3P2d3bw2+HcH8LLf4affQzaWrK/T0mSNCIMXiMg6yvY723e2fCeb8Fzd8OtfwUd7bnZryRJGhaD1wiYNyM5szHr87y6O/ZjcPo18MQv4Y6/hZildcQkSdKIKcx3AeNB1X5FzJ5Slpt5Xt2d9GnYsx3u/RconQynfRVCyG0NkiRpwAxeI2T+zEnZXcurN0v+R3KG4x+/BWWT4a2fzX0NkiRpQDzUOELmz6xiw2u7aGxuy+2OQ4Az/hmO/hDc/VX48w9yu39JkjRgWQteIYQfhRA2hxDWdNv25RDCKyGEVenbmdnaf67NnzmJGOHJXE2w766gAN73HTjsDPj15+DxX+S+BkmS1K9sjnjdCLyrh+3fjDEuTN9+k8X959S8mZ0T7PMQvABSRfDBH8Ock+G2T8DT/5WfOiRJUq+yFrxijPcC27L1/aPNmyaVMrW8OLdnNu6tqAzOuwXedDT8/OOw4YH81SJJkvYRYhaXIQgh1AJ3xBiPSj//MnARsBNYAXw2xvh6L5+9HLgcoKam5rhly5ZlrU6AxsZGKioqhvUd//LnJhpb4SsnlY1QVUNT1LKThau+QEnzazx38CVsqT6RtqLKAX9+JHoxXtiLDHuRsA8Z9iLDXmTYC1iyZMnKGGNdT6/lOnjVAFuBCHwNmBFjvKS/76mrq4srVqzIWp0Ay5cvp76+fljf8Y071/Hj+zew9qvvpCiV5/MWdrwCS8+FzU9AQREcvATmvx+OOBNKq/r86Ej0YtRob4OVP4a1v4RjPgwLL4CC1IA/Pq56MUz2ImEfMuxFhr3IsBcQQug1eOV0OYkY46bOxyGE7wN35HL/2TZvxiRa2jt4ZlNj15yvvKmaBZ/6I2xcBWtuTYLHM5+EVDEc8o4khB3+LigZ+EjYmPP8fXDn38PmtVDxJrj9M/Cn7ybrnR3yDtc8kyTlXE6DVwhhRoxxY/rpOcCavt4/1nReOmjtqzvyH7wgCRYzFyW3074KL6+Atbclt6d+A4WlcOhpSQg77J1QXJ7b+lp2QWFZclbmSNr+EvzuH5JV/asOhA/9BI58LzzxK/jvLycjgQfVw2lfgxkLRnbf/YkRWndDcyM0N0BLQ3Lf1gL7HwSTa0e+H5KkUSNrwSuEcAtQD0wLIbwMfAmoDyEsJDnUuAH4RLb2nw9zp5VTVpRi7as7+WC+i9lbCHDA8cnt9H+Clx5KAtgTv4R1/wlF+8Fh74L551DQXpK9Ona9ButuT/a94T6YNAuOOQ8Wng9T5w7vu1ub4IFvwf3fTJ7XfxFO/pvkpAOA+e+Dw8+EFT+EP1wH/3ZKsu+3/0MyQjhcr2+AJ38Dm9YkYaq5AVrSAat70IodvX9HUTnUzIOa+VBzVHI/fV6yOO5Y096W/O4GcWhXksa7rAWvGON5PWz+Ybb2NxqkCgJHzqjM3cWyh6qgAOacmNze9Q144Y+w9lZ44nZYeytvCUXw8okw9xSYW5+MmKWG8VPZvQ2evCMJW+v/ALEdph4MJ346CSn3/gvc+89Q+9ZkDta89w5u9C3GJMz91z/Ajhdh/jnJaNbkA/Z9b2ExnPCpJHDd96/w0PeSv/3EK+Dkv4XSQYxUxgivPpqMHj75m+SQJkDljGQeXUklFFckz0sqM89LKqGkAoorM48LiuC1Z+Ava2DT2uTQ8MobM/uqOiAdxuZnQtnUg5P/XbpG0dIBr6V70GvcKwQ2Jv0vnZyEubIpmVtp5/PJUNhH+O7ogKZt0LARdm5M7hv+Ag2vJvc70/e7tgAx6UXZ1Dfuq69bSUVy4ffYnr7vSO472vbd1vU8dxeKn/z6Y7A+Z7sb1exFhr3IGPW9mD4PKqbnbfdZnVw/UsbK5HqAf/zlGn756Cs89qXTKSgYY3OI2ttgw3289Psfc0Dretj0eLK9uBJqT04HsbclP9r+Doc1vZ6EkbW3wfrfJ/9oTqlNDmvOPydZ8qJzjtWOl+GxW+DRpfD688n+5r8PFn0UDljc91ysTU/Ab/8enr8Xps+HM66DuW8d+N/8+gtwz9fg8f8H+02D+s/DcRcl66LRw++irSUZqXvy1/DUnUnYCAVw4InJaNoRZ8LUgwa+/97EmASYTWuTcLppbXKixNank14CpEqSgNTS2PcoWpeQBL1QAHt2kAw896Ko/I3BrLiCHX9ZT1VoSoJWR+u+n9lvWhIyJ82Ayjcl8+pCQfJb6Lpt6/Z4e981SFI2nPtjOOr9Wd3FqJlcPxHMmzmJm//0Ai+9vps5++d4ztRwpQrh4CU891LggPr65LDghnuTUarn74Wnf5u8b79pSbiZ+7YkjE09KAlHe3YkYWTtbfDs3ck/zpMPTEaT5r8fZhzTc4iqmg2nXAVv/Ry8+GASwNbcCo/eDPsfkoyCHXNe8g96p6bX4fffSC6RVFIJZ14Px108+JG5KXPgAz+AE/4afveP8JvPJaNgp301CVKQBIRn/zsJW8/+NzTvTA7NHvx2OOIf4dB3Qvn+Q+l470JIDn9WzYLDTs9sb2tOwtemtcmtvTUZIeo+mrbPqFpFsq24PNP/jg5o3pH8bZ1BaM/2N4ai7q/teJmOghI4cFESqroCVreQVVg8uL+xq4bX37jf5obk8GRIvfF+n20F6fvCJODl6GSJRx99lEWLFuVkX6OdvciwFxmjvhfTDsvr7g1eI2x+txXsx1zw2lv5/sno1Pxzkuc7Xk4C2Po/wPN/SAIWJIfBph6UhKb2luT5CZ9MPjfz2IH/gxgCzDkpuZ1xXTL/7NGlcPdXklGpg0+FRRck/zjf/dUkKBx3cXKh8OEGn1nHwkV3JOHyrqth2flw4IksaNgN965NRpnKq9PzxN4NB70tM3cslwpLktHCNx09vO8pKMiMZjGwuXWPjfQp4m+oYezY8XxL8huVvejGXmTYi74ZvEbYYTWVpAoCa1/dwZlHz+j/A2NJ1exkEvzC85NDYa89mwSw9X9IHh//V0nYml03/NGHkorkUOOij8Jrz8Gqf09u/++i5PUD0+FsJM9KDAEOPwMOOQ0euQnuvZ7StpDMRTvi3TCrzjMOJUnDYvAaYaVFKQ6dXsET+bpmY66EANMOTW7HX5bdfe1/MJz6j7Dki7B+eTKfKZvrcKUK4fhL4fhLediFACVJI8jglQXzZk7i/me25ruM8acgBYecmu8qJEkaMo+bZMG8GZPY3NDMlobmfJciSZJGEYNXFnRfwV6SJKmTwSsLOi8XNOoXUpUkSTll8MqCqrIiDphaxtrxPsFekiQNisErS+bNmDT+z2yUJEmDYvDKkvkzq3h+6y4am9vyXYokSRolDF5Z0rmC/TrneUmSpDSDV5Z0ntno4UZJktTJ4JUlNZNK2L+82CUlJElSF4NXloQQmDdzkmc2SpKkLgavLJo3cxJPb2qgpa0j36VIkqRRwOCVRfNnVtHaHnl2c2O+S5EkSaOAwSuLOs9sdJ6XJEkCg1dW1e5fTllRynlekiQJMHhlVaogcOSMSpeUkCRJgMEr6+bPrOKJjTvp6Ij5LkWSJOWZwSvL5s+cRGNzGy+9vjvfpUiSpDwzeGVZ5wr2zvOSJEkGryw7tKaCVEHwzEZJkmTwyrbSohSHTq9wxEuSJBm8cmHezEme2ShJkgxeuTB/ZhWbG5rZ0tCc71IkSVIeGbxywBXsJUkSGLxy4sgZncHLw42SJE1kBq8cqCor4oCpZc7zkiRpgjN45cj8GckK9pIkaeIyeOXI/JmTeH7rLhqb2/JdiiRJyhODV47MS0+wX+eolyRJE5bBK0e6Lh30imc2SpI0URm8cqRmUgn7lxc7z0uSpAnM4JUjIQTmzZzkkhKSJE1gBq8cmj+ziqc3NdDS1pHvUiRJUh5kLXiFEH4UQtgcQljTbdvUEMJdIYRn0vdTsrX/0WjezEm0tkee2dyQ71IkSVIeZHPE60bgXXtt+zxwd4zxUODu9PMJI3PpIA83SpI0EWUteMUY7wW27bX5bOCm9OObgPdla/+j0dz9y9mvOOUK9pIkTVAhxpi9Lw+hFrgjxnhU+vn2GOPkbq+/HmPs8XBjCOFy4HKAmpqa45YtW5a1OgEaGxupqKjI6j4ArvlTEy0d8JWTyrK+r6HKVS/GAnuRYS8S9iHDXmTYiwx7AUuWLFkZY6zr6bXCXBczUDHGG4AbAOrq6mJ9fX1W97d8+XKyvQ+AF0s2cPWv1lJ10DEsOnB0TnHLVS/GAnuRYS8S9iHDXmTYiwx70bdcn9W4KYQwAyB9vznH+8+79x87m4qSQm7644Z8lyJJknIs18HrduDC9OMLgV/leP95V1FSyAfrZvPrxzeyuWFPvsuRJEk5lM3lJG4BHgQODyG8HEK4FLgWOC2E8AxwWvr5hPPxE2tpbY/8+0Mv5rsUSZKUQ1mb4xVjPK+Xl07N1j7HirnTyqk/vJqlD73IX9cfQnGh69hKkjQR+C9+nlx4Ui1bGpq5c83GfJciSZJyxOCVJ287tJq508qdZC9J0gRi8MqTgoLAx0+cwyMvbmf1y9vzXY4kScoBg1cenXvcbMqLU9zoqJckSROCwSuPKkuL+MBxs7njsY1sbWzOdzmSJCnLDF559vETa2lp72DZwy4tIUnSeGfwyrNDplfw1kOncfOfXqC1vSPf5UiSpCwyeI0CF51Uy6adzfzX2r/kuxRJkpRFBq9RoP7w6Rw4dT+XlpAkaZwzeI0CqfTSEn/e8DprX92R73IkSVKWGLxGiQ/WHUBZUcpRL0mSxjGD1yhRVVbE+4+dxS9Xvcq2XS35LkeSJGWBwWsUufCkWlraOlj2Z5eWkCRpPDJ4jSKH1VRy0sH789MHX6DNpSUkSRp3DF6jzIUn1fLqjj3897pN+S5FkiSNMIPXKPOOI2uYNbmMHz+wId+lSJKkEWbwGmU6l5Z46PltrNu4M9/lSJKkEWTwGoU+fPwBlBYV8JMHN+S7FEmSNIIMXqPQ5P2Ked/CWdz26Cts3+3SEpIkjRcGr1HqwpNq2dPawc9XvJTvUiRJ0ggxeI1SR86YxOK5U/nJgy/Q3hHzXY4kSRoBBq9R7KKTann59SbudmkJSZLGBYPXKHbavBpmVpVyk5PsJUkaFwxeo1hhqoALTpjDA8++xjObGvJdjiRJGiaD1yh33psPpLiwgBv/uCHfpUiSpGEyeI1yU8uLOfuYmdz6yCts3NGU73IkSdIwGLzGgL9ecggFAT7100dobmvPdzmSJGmIDF5jwNxp5fzrh45h1Uvb+fLtT+S7HEmSNEQGrzHiXUfN4K/rD+aWh19k2cMv5rscSZI0BAavMeSzpx/OWw+dxtW/Wsuql7bnuxxJkjRIBq8xJFUQ+NZHFjF9Ugmf+ulKtjY257skSZI0CAavMWZKeTHf++hxbNvVwqf//RHa2jvyXZIkSRogg9cYdNSsKr7x/qP50/ptXHvnk/kuR5IkDVBhvgvQ0Lz/2Nk89tJ2fnD/8xw9u4qzF87Kd0mSJKkfjniNYf/j3fM4vnYKf/8fq1m3cWe+y5EkSf0weI1hxYUFfPv8Y5lUWsQnbl7Jjt2t+S5JkiT1weA1xk2fVMp3P3osG3c0ceXPHqWjI+a7JEmS1AuD1zhw3JypXP2e+Sx/agv/6+5n8l2OJEnqRV4m14cQNgANQDvQFmOsy0cd48lHFx/IYy9t51t3P8PRs6o4bV5NvkuSJEl7yeeI15IY40JD18gIIfBP7zuKo2ZN4u9+tor1WxrzXZIkSdqLhxrHkdKiFN/76HEUpgKfuHkljc1t+S5JkiR1E2LM/WTsEMLzwOtABP4txnhDD++5HLgcoKam5rhly5ZltabGxkYqKiqyuo9cWbu1netX7OG4mhRXLCwhhDCoz4+nXgyXvciwFwn7kGEvMuxFhr2AJUuWrOztiF6+FlA9Ocb4aghhOnBXCOHJGOO93d+QDmM3ANTV1cX6+vqsFrR8+XKyvY9cqQcKq5/jG3c+yfKd1fzjWfMoTA18cHM89WK47EWGvUjYhwx7kWEvMuxF3/ISvGKMr6bvN4cQbgPeDNzb96c0GJefchBbGpr5wf3P89yWXfzf8xcxeb/ifJclSdKElvM5XiGE8hBCZedj4HRgTa7rGO9CCPzDWfP453MX8PDz2zj72w/w9KaGfJclSdKElo/J9TXA/SGEx4CHgV/HGH+bhzomhA/VHcAtl5/A7pZ2zvn2A9z1xKZ8lyRJ0oSV8+AVY1wfYzwmfZsfY7wm1zVMNMfNmcLtnz6Zg6dXcPnNK/i/9zxDPk6qkCRponM5iQliRlUZP//EiZx9zEyu/93TfPqWR9nd4nITkiTlksFrAiktSvHNDy/kC2ccwW8e38i5332QV7Y35bssSZImDIPXBBNC4BNvO5gfXXg8L23bzXv/z/08/Py2fJclSdKEYPCaoJYcMZ3brjiZqrIiLvjBn7jl4RfzXZIkSeOewWsCO2R6BbddcTInHTyNL9z6OFf/ag2t7R35LkuSpHErXyvXa5SoKiviRxcdzz//9kn+7d71PL2pgQtqPeNRkqRsMHiJVEHgC2ceyREzKvn7/3icJ16ONFS9yIfqZg/qUkOSJKlv/quqLucsms2tnzqJmv0CX7ztcc743/dxz5ObXPNLkqQRYvDSGxw1q4ovLi7lex89ltb2Di65cQUX/OAh1ryyI9+lSZI05hm8tI8QAu86aga/+//expffM491G3dy1v+5n7/72SrX/ZIkaRgMXupVcWEBF508l+VXLeGTbzuYOx7fyJLrl3Pdb59k557WfJcnSdKYY/BSv6rKivj8GUdwz2ffxruPnsF3lz9H/b8s5ycPbnD5CUmSBsHgpQGbPWU/vvnhhfznp9/CYTUVXP2rtbzzm/fyX2v/4gR8SZIGwOClQTt6dhW3/NUJ/PDCOkKAT9y8knO/9yA3/XEDL7y2K9/lSZI0armOl4YkhMCpR9bwtsOqWfbnl/j+fev50u1rAThoWjlvO7ya+sOns3juVEqLUnmuVpKk0cHgpWEpTBXw0RPm8NET5vD81l0sf2ozv39qC0sfepEfP7CB0qICTjp4GvWHV1N/2HQO3H+/fJcsSVLeGLw0YuZOK2futLlcfPJcmlra+dP617qC2D1PbgbWclB1OfWHTaf+8Gre7GiYJGmCMXgpK8qKUyw5YjpLjpjOl2NMj4ZtYfnTW/jpQy/woweep6woxYLZVRw1q4r5Mydx1KwqDppW7mWKJEnjlsFLWRdC4KDqCg6qruCStySjYQ+u38q9T2/lsZe3s/ShF9jTmixLUVpUwJEzJnHUzCqOmjWJ+TOrOKymkuJCw5gkaewzeCnnyopTvP2IGt5+RA0Abe0drN+6izWv7GDNKztZ8+oObnv0FW7+0wsAFKUCh9VUdoWxg6dXML2ylOrKEiaVFhJCyOefI0nSgBm8lHeFqQIOq6nksJpK3n9ssq2jI/Litt2seTUJY2tf3cHvnvgLP1vx0hs+W1xYQHVFCdWV3W49PJ9WUUJpUYEhTZKUVwYvjUoFBYHaaeXUTivnrAUzAYgx8uqOPTy/ZRdbG5vZ2tjMlob0rbGZl7bt5pEXXue1XS09fmeqILBfcYr9ilOUFxeyX0mK/YqS+/Liwq7X9isppLw4RVlxIS+91MqOVa9QWpTqej15XEhZUYqy4hRlRalBHQrt6Ii0dUTaOyJtHR20px+XFKUoLSxwjpskjWMGL40ZIQRmTS5j1uSyPt/X2t7Btl0tmVDW0MzWXc3sbm5nV0tb131TS3K/bVcLL23bnX7ezq7mNto6Mivx37h2Vb+1FRaErhBWlCpIh6p0sGp/Y9Dq6GeR/6JUoLQo1RXsSgtTlBanKCsqoKwo1fVaaXGKksICSouS+5LC9H1RAaWFKUqKum0rLEiCXVEBhQWB9g66Al97jJnHHZGOmNTbsdfrq19tY+vKl2lt76C1vYOWtg5a22PmeXsHrW1vfB4jBIAAgUAIyfPQ/Xn6DZ2vFYRAqiB5nup6HEgVJM9Detve78mV5za08ux963O2v9HMXmTYi4zR3ou3HzGdg6or8rZ/g5fGnaJUATWTSqmZVDrk72hp62B3Sxv33Hs/xxz3Zppa2mlqbWd3S3v6cRtNLcl79nRub01ea2nvoKiggFQqUFgQKCwooDCVhIPCdGAoShW84XlBCLS0dSTf0drOnvStqaWdPa2Z7dt2tXQ9bmrpoLmtnea2JATlxOrHetwcAhSnCihOFVBUWEBRKvm7CwogxuQGyahlJL2NmL7vfD153hEjHTEZGWyPSRDs6KDr8ai4OtWT6/JdwehhLzLsRcYo7sWbqkoNXtJoU1xYQHFhMVNLCzg4j/8HHaiOjkhLewfNrUkY29OaCWXNbe00t3awJ33f2hG7RomSG6QKCkiFQEF6VKkwFbpGnjpvj65YwVtOOoGiVBKsigrTQSsdInMlxsxIXIx0Pc6V+++7n7e89S05299oZi8y7EXGaO9FaWF+1480eEnjQEFBoLQglV6Qtigr+/hLRQEHTM3/lQdCOhjm6z9e+xUFJpVmp8djjb3IsBcZ9qJvzuKVJEnKEYOXJElSjhi8JEmScsTgJUmSlCMGL0mSpBwxeEmSJOWIwUuSJClHDF6SJEk5YvCSJEnKEYOXJElSjhi8JEmScsTgJUmSlCMGL0mSpBwxeEmSJOWIwUuSJClHQowx3zX0K4SwBXghy7uZBmzN8j7GCnuRYS8y7EXCPmTYiwx7kWEvYE6MsbqnF8ZE8MqFEMKKGGNdvusYDexFhr3IsBcJ+5BhLzLsRYa96JuHGiVJknLE4CVJkpQjBq+MG/JdwChiLzLsRYa9SNiHDHuRYS8y7EUfnOMlSZKUI454SZIk5YjBS5IkKUcMXkAI4V0hhKdCCM+GED6f73ryKYSwIYTweAhhVQhhRb7ryaUQwo9CCJtDCGu6bZsaQrgrhPBM+n5KPmvMhV768OUQwivp38WqEMKZ+awxV0IIB4QQfh9CWBdCWBtCuDK9fUL9Lvrow4T7XYQQSkMID4cQHkv34ivp7RPqNwF99mLC/S4GY8LP8QohpICngdOAl4E/A+fFGJ/Ia2F5EkLYANTFGCfc4nchhFOARuAnMcaj0tv+GdgWY7w2HcqnxBj/Pp91Zlsvffgy0BhjvD6fteVaCGEGMCPG+EgIoRJYCbwPuIgJ9Lvoow8fYoL9LkIIASiPMTaGEIqA+4ErgfczgX4T0Gcv3sUE+10MhiNe8Gbg2Rjj+hhjC7AMODvPNSkPYoz3Atv22nw2cFP68U0k/9iMa730YUKKMW6MMT6SftwArANmMcF+F330YcKJicb006L0LTLBfhPQZy/UB4NX8h+Pl7o9f5kJ+h+UtAj8LoSwMoRweb6LGQVqYowbIfnHB5ie53ry6dMhhNXpQ5Hj/jDK3kIItcAi4CEm8O9irz7ABPxdhBBSIYRVwGbgrhjjhP1N9NILmIC/i4EyeEHoYdtETuwnxxiPBc4ArkgfdpK+CxwMLAQ2Av+a12pyLIRQAfwH8Lcxxp35ridfeujDhPxdxBjbY4wLgdnAm0MIR+W5pLzppRcT8ncxUAavZITrgG7PZwOv5qmWvIsxvpq+3wzcRnIodiLblJ7f0jnPZXOe68mLGOOm9H9gO4DvM4F+F+m5K/8BLI0x3prePOF+Fz31YSL/LgBijNuB5SRzmibcb6K77r2Y6L+L/hi8ksn0h4YQ5oYQioGPALfnuaa8CCGUpyfOEkIoB04H1vT9qXHvduDC9OMLgV/lsZa86fwHJe0cJsjvIj15+IfAuhjj/+z20oT6XfTWh4n4uwghVIcQJqcflwHvAJ5kgv0moPdeTMTfxWBM+LMaAdKnuv4vIAX8KMZ4TX4ryo8QwkEko1wAhcC/T6RehBBuAeqBacAm4EvAL4GfAwcCLwIfjDGO64nnvfShnuSwQQQ2AJ/onM8ynoUQ3gLcBzwOdKQ3f5FkftOE+V300YfzmGC/ixDCApLJ8ymSwYufxxi/GkLYnwn0m4A+e3EzE+x3MRgGL0mSpBzxUKMkSVKOGLwkSZJyxOAlSZKUIwYvSZKkHDF4SZIk5YjBS5IkKUcMXpIEhBA2hBCmDfGzF4UQZo7Ed0ka3wxekjR8FwEz+3uTJBm8JI0qIYTaEMKTIYQfhBDWhBCWhhDeEUJ4IITwTAjhzenbH0MIj6bvD09/9u9CCD9KPz46/fn9etnP/iGE36W/49+A0O21j4YQHg4hrAoh/FsIIZXe3hhC+NcQwiMhhLvTl0w5F6gDlqbfX5b+ms+k3/d4COGIbPZM0thh8JI0Gh0C/G9gAXAEcD7wFuBzJJeqeRI4Jca4CLga+Hr6c/8LOCSEcA7wY5JLlezuZR9fAu5Pf8ftJJd6IYRwJPBh4OQY40KgHbgg/Zly4JEY47HAH4AvxRh/AawALogxLowxNqXfuzX9vu+m65YkCvNdgCT14PkY4+MAIYS1wN0xxhhCeByoBaqAm0IIh5JcD64IIMbYEUK4CFgN/FuM8YE+9nEK8P70534dQng9vf1U4Djgz8m1oSkDNqdf6wB+ln78U+DWPr6/87WVnfuRJIOXpNGoudvjjm7PO0j+u/U14PcxxnNCCLXA8m7vPxRoZGBzrnq6WG0AbooxfmGIn+/UWXM7/rdWUpqHGiWNRVXAK+nHF3VuDCFUkRyiPAXYPz3/qjf3kj6EGEI4A5iS3n43cG4IYXr6takhhDnp1wqAzu88H7g//bgBqBzG3yNpgjB4SRqL/hn4RgjhASDVbfs3ge/EGJ8GLgWu7QxQPfgKcEoI4RHgdOBFgBjjE8A/AL8LIawG7gJmpD+zC5gfQlgJvB34anr7jcD39ppcL0n7CDH2NVIuSeoUQmiMMVbkuw5JY5cjXpIkSTniiJekcS2EcDFw5V6bH4gxXpGPeiRNbAYvSZKkHPFQoyRJUo4YvCRJknLE4CVJkpQjBi9JkqQc+f8BaARYpRLa3LoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x504 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "max_depth = 40\n",
    "train_loss = []\n",
    "test_loss = []\n",
    "\n",
    "for i in range(1, max_depth):\n",
    "    rf = RandomForestRegressor(n_estimators = 20, max_depth = i, random_state = 123)\n",
    "    rf.fit(X_train, y_train)\n",
    "    train_loss.append(mean_squared_error(y_train, rf.predict(X_train)))\n",
    "    test_loss.append(mean_squared_error(y_test, rf.predict(X_test)))\n",
    "    \n",
    "plt.figure(figsize = (10, 7))\n",
    "plt.grid()\n",
    "plt.plot(train_loss, label = 'MSE_train')\n",
    "plt.plot(test_loss, label = 'MSE_test')\n",
    "plt.ylabel('MSE')\n",
    "plt.xlabel('max_depth')\n",
    "plt.legend();"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "determined-journalism",
   "metadata": {},
   "source": [
    "Переобучение не наблюдается. Вообще же, как обычно, гиперпараметры случайного леса стоит подбирать на кросс-валидации."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dental-mississippi",
   "metadata": {},
   "source": [
    "#### 4.2 Out-of-bag-ошибка"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "neither-merchandise",
   "metadata": {},
   "source": [
    "Как мы обсудили выше, при построении случайного леса каждое дерево строится на бутстрапированной подвыборке, полученной из исходной обучающей выборки случайным набором с повторениями. Понятно, что некоторые наблюдения попадут в такую подвыборку несколько раз, а некоторые не войдут в неё вообще. Для каждого дерева мы можем рассмотреть объекты, которые не участвовали в обучении и использовать их для валидации.\n",
    "\n",
    "Усреднённая ошибка на неотобранных образцах по всему случайному лесу называется **out-of-bag-ошибкой**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 340,
   "id": "bibliographic-california",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8760889947613861"
      ]
     },
     "execution_count": 340,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X, y = boston_housing_data()\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y,\n",
    "                                                    test_size = 0.3,\n",
    "                                                    random_state = 123,\n",
    "                                                    shuffle = True)\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.3, random_state = 123)\n",
    "\n",
    "# oob_score_ = R2 на невиденных наблюдениях.\n",
    "rf = RandomForestRegressor(n_estimators = 100, random_state = 123, oob_score = True)\n",
    "rf.fit(X_train, y_train)\n",
    "rf.oob_score_"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cooked-evening",
   "metadata": {},
   "source": [
    "#### 4.3 Важность признаков"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "sound-resort",
   "metadata": {},
   "source": [
    "Как и решающие деревья, случайный лес позволяет оценивать важность признаков."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 341,
   "id": "enhanced-oasis",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Просто чтобы подгрузить названия признаков\n",
    "from sklearn.datasets import load_boston\n",
    "data = load_boston()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 342,
   "id": "joint-monitor",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlMAAAGbCAYAAADgEhWsAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAaLElEQVR4nO3dfbQtZX0f8O/Pi1g1Whu5VQOEi0qkJFFrbtAmptUYK76sog2NoNGQxhJM0UTFSJoVl4lNgukyagsJUiXGtPHGLBtL5Cq2vqdq5ar4Aoq9IMqVWK+apaFBEX36x8yRYXNe9r3PedlwP5+1zjp7Zp4z8zsze2Z/55nZe1drLQAAHJw7bXUBAAC3Z8IUAEAHYQoAoIMwBQDQQZgCAOhw2FYt+Igjjmg7duzYqsUDAMztwx/+8Jdba9uXm7ZlYWrHjh3Zs2fPVi0eAGBuVfW5laa5zAcA0EGYAgDoIEwBAHQQpgAAOghTAAAdhCkAgA7CFABAB2EKAKCDMAUA0EGYAgDoIEwBAHQQpgAAOghTAAAdhCkAgA7CFABAB2EKAKDDYVtdALD5dpxzyZYt+9pzn7hlywbYCHqmAAA6CFMAAB3mClNVdVJVXVVVe6vqnGWmP6qqvlZVl48/L17/UgEAFs+a90xV1bYk5yd5bJJ9SS6rqotba1fONH1fa+1JG1AjAMDCmqdn6sQke1tr17TWbkqyK8nJG1sWAMDtwzxh6sgk102G943jZv2TqvpYVb21qn5wuRlV1RlVtaeq9uzfv/8gygUAWCzzhKlaZlybGf5IkmNaaw9J8p+SvHm5GbXWLmyt7Wyt7dy+ffsBFQoAsIjmCVP7khw9GT4qyfXTBq21r7fWbhgf705y56o6Yt2qBABYUPOEqcuSHFdVx1bV4UlOTXLxtEFV3beqanx84jjfr6x3sQAAi2bNd/O11m6uqrOSXJpkW5KLWmtXVNWZ4/QLkpyS5NlVdXOSG5Oc2lqbvRQIAHCHM9fXyYyX7nbPjLtg8vi8JOetb2kAAIvPJ6ADAHQQpgAAOghTAAAdhCkAgA7CFABAB2EKAKCDMAUA0EGYAgDoIEwBAHQQpgAAOghTAAAdhCkAgA7CFABAB2EKAKCDMAUA0EGYAgDoIEwBAHQQpgAAOghTAAAdhCkAgA7CFABAB2EKAKCDMAUA0EGYAgDoIEwBAHQQpgAAOghTAAAdhCkAgA7CFABAB2EKAKCDMAUA0EGYAgDoIEwBAHQQpgAAOghTAAAdhCkAgA7CFABAB2EKAKCDMAUA0EGYAgDoIEwBAHQQpgAAOghTAAAdhCkAgA7CFABAB2EKAKCDMAUA0EGYAgDoIEwBAHQQpgAAOghTAAAdhCkAgA7CFABAB2EKAKCDMAUA0EGYAgDoIEwBAHQQpgAAOghTAAAdhCkAgA7CFABAB2EKAKCDMAUA0GGuMFVVJ1XVVVW1t6rOWaXdj1bVt6vqlPUrEQBgca0ZpqpqW5Lzkzw+yQlJTquqE1Zo97Ikl653kQAAi2qenqkTk+xtrV3TWrspya4kJy/T7jlJ3pTkS+tYHwDAQpsnTB2Z5LrJ8L5x3HdV1ZFJnpLkgtVmVFVnVNWeqtqzf//+A60VAGDhzBOmaplxbWb4lUle1Fr79mozaq1d2Frb2VrbuX379jlLBABYXIfN0WZfkqMnw0cluX6mzc4ku6oqSY5I8oSqurm19ub1KBIAYFHNE6YuS3JcVR2b5AtJTk3ytGmD1tqxS4+r6nVJ3iJIAQCHgjXDVGvt5qo6K8O79LYluai1dkVVnTlOX/U+KQCAO7J5eqbSWtudZPfMuGVDVGvt9P6yAABuH3wCOgBAB2EKAKCDMAUA0EGYAgDoIEwBAHQQpgAAOghTAAAdhCkAgA7CFABAB2EKAKCDMAUA0EGYAgDoIEwBAHQQpgAAOghTAAAdhCkAgA7CFABAB2EKAKCDMAUA0EGYAgDoIEwBAHQQpgAAOghTAAAdhCkAgA7CFABAB2EKAKCDMAUA0EGYAgDoIEwBAHQQpgAAOghTAAAdhCkAgA7CFABAB2EKAKCDMAUA0EGYAgDoIEwBAHQQpgAAOghTAAAdhCkAgA7CFABAB2EKAKCDMAUA0EGYAgDoIEwBAHQQpgAAOhy21QUAALdvO865ZEuXf+25T9zS5euZAgDoIEwBAHQQpgAAOghTAAAdhCkAgA7CFABAB2EKAKCDMAUA0EGYAgDoIEwBAHQQpgAAOghTAAAdhCkAgA7CFABAB2EKAKDDXGGqqk6qqquqam9VnbPM9JOr6uNVdXlV7amqR65/qQAAi+ewtRpU1bYk5yd5bJJ9SS6rqotba1dOmr0jycWttVZVD07yxiTHb0TBAACLZJ6eqROT7G2tXdNauynJriQnTxu01m5orbVx8O5JWgAADgHzhKkjk1w3Gd43jruVqnpKVX06ySVJ/vVyM6qqM8bLgHv2799/MPUCACyUecJULTPuNj1PrbW/aK0dn+TJSV663Ixaaxe21na21nZu3779gAoFAFhE84SpfUmOngwfleT6lRq31t6b5AFVdURnbQAAC2+eMHVZkuOq6tiqOjzJqUkunjaoqgdWVY2PH5bk8CRfWe9iAQAWzZrv5mut3VxVZyW5NMm2JBe11q6oqjPH6Rck+ekkz6yqbyW5MclTJzekAwDcYa0ZppKktbY7ye6ZcRdMHr8sycvWtzQAgMXnE9ABADoIUwAAHYQpAIAOwhQAQAdhCgCggzAFANBBmAIA6CBMAQB0EKYAADoIUwAAHYQpAIAOwhQAQAdhCgCggzAFANBBmAIA6CBMAQB0EKYAADoIUwAAHYQpAIAOwhQAQAdhCgCggzAFANBBmAIA6CBMAQB0EKYAADoIUwAAHYQpAIAOwhQAQAdhCgCggzAFANBBmAIA6CBMAQB0EKYAADoIUwAAHYQpAIAOwhQAQAdhCgCggzAFANBBmAIA6CBMAQB0EKYAADoIUwAAHYQpAIAOwhQAQAdhCgCggzAFANBBmAIA6CBMAQB0EKYAADoIUwAAHYQpAIAOwhQAQAdhCgCggzAFANBBmAIA6CBMAQB0EKYAADoIUwAAHYQpAIAOwhQAQAdhCgCggzAFANBhrjBVVSdV1VVVtbeqzllm+tOr6uPjz/ur6iHrXyoAwOJZM0xV1bYk5yd5fJITkpxWVSfMNPtskn/WWntwkpcmuXC9CwUAWETz9EydmGRva+2a1tpNSXYlOXnaoLX2/tba34yDH0xy1PqWCQCwmOYJU0cmuW4yvG8ct5JfSPLW5SZU1RlVtaeq9uzfv3/+KgEAFtQ8YaqWGdeWbVj16Axh6kXLTW+tXdha29la27l9+/b5qwQAWFCHzdFmX5KjJ8NHJbl+tlFVPTjJa5I8vrX2lfUpDwBgsc3TM3VZkuOq6tiqOjzJqUkunjaoqu9P8t+SPKO19pn1LxMAYDGt2TPVWru5qs5KcmmSbUkuaq1dUVVnjtMvSPLiJPdO8gdVlSQ3t9Z2blzZAACLYZ7LfGmt7U6ye2bcBZPHz0ryrPUtDQBg8fkEdACADsIUAEAHYQoAoIMwBQDQQZgCAOggTAEAdBCmAAA6CFMAAB2EKQCADsIUAEAHYQoAoIMwBQDQQZgCAOggTAEAdBCmAAA6CFMAAB2EKQCADsIUAEAHYQoAoIMwBQDQQZgCAOggTAEAdBCmAAA6CFMAAB2EKQCADsIUAEAHYQoAoIMwBQDQQZgCAOggTAEAdBCmAAA6CFMAAB2EKQCADsIUAEAHYQoAoIMwBQDQQZgCAOggTAEAdBCmAAA6CFMAAB2EKQCADsIUAEAHYQoAoIMwBQDQQZgCAOggTAEAdBCmAAA6CFMAAB2EKQCADsIUAEAHYQoAoIMwBQDQQZgCAOggTAEAdBCmAAA6CFMAAB2EKQCADsIUAEAHYQoAoIMwBQDQQZgCAOggTAEAdJgrTFXVSVV1VVXtrapzlpl+fFV9oKq+WVVnr3+ZAACL6bC1GlTVtiTnJ3lskn1JLquqi1trV06afTXJc5M8eSOKBABYVPP0TJ2YZG9r7ZrW2k1JdiU5edqgtfal1tplSb61ATUCACysecLUkUmumwzvG8cBABzy5glTtcy4djALq6ozqmpPVe3Zv3//wcwCAGChzBOm9iU5ejJ8VJLrD2ZhrbULW2s7W2s7t2/ffjCzAABYKPOEqcuSHFdVx1bV4UlOTXLxxpYFAHD7sOa7+VprN1fVWUkuTbItyUWttSuq6sxx+gVVdd8ke5LcM8l3qupXkpzQWvv6xpUOALD11gxTSdJa251k98y4CyaPv5jh8h8AwCHFJ6ADAHQQpgAAOghTAAAdhCkAgA7CFABAB2EKAKCDMAUA0EGYAgDoIEwBAHQQpgAAOghTAAAdhCkAgA7CFABAB2EKAKCDMAUA0EGYAgDoIEwBAHQQpgAAOghTAAAdhCkAgA7CFABAB2EKAKCDMAUA0EGYAgDoIEwBAHQQpgAAOghTAAAdhCkAgA7CFABAh8O2ugCA25Md51yypcu/9twnbunygdvSMwUA0EGYAgDoIEwBAHQQpgAAOghTAAAdhCkAgA7CFABAB2EKAKCDMAUA0EGYAgDoIEwBAHQQpgAAOghTAAAdhCkAgA7CFABAh8O2uoCNtuOcS7Zs2dee+8QtWzYAsDn0TAEAdBCmAAA6CFMAAB2EKQCADsIUAEAHYQoAoIMwBQDQ4Q7/OVOwFbby880Sn3EGsJmEKQA2hZMM7qhc5gMA6KBnClgoei/YKp57HCw9UwAAHYQpAIAOLvMBwO3AVl6GdAlydXqmAAA6CFMAAB1c5mNF3tkCtz8uBcHmm6tnqqpOqqqrqmpvVZ2zzPSqqv84Tv94VT1s/UsFAFg8a4apqtqW5Pwkj09yQpLTquqEmWaPT3Lc+HNGkj9c5zoBABbSPJf5Tkyyt7V2TZJU1a4kJye5ctLm5CSvb621JB+sqntV1f1aa3+97hXfgbiMdvCsOwAWRQ35Z5UGVackOam19qxx+BlJHt5aO2vS5i1Jzm2t/dU4/I4kL2qt7ZmZ1xkZeq6S5EFJrlqvf2SDHJHky1tdxCrUd/AWubZEfT0WubZksetb5NoS9fVY5NqSxa8vSY5prW1fbsI8PVO1zLjZBDZPm7TWLkxy4RzLXAhVtae1tnOr61iJ+g7eIteWqK/HIteWLHZ9i1xbor4ei1xbsvj1rWWeG9D3JTl6MnxUkusPog0AwB3OPGHqsiTHVdWxVXV4klOTXDzT5uIkzxzf1feIJF9zvxQAcChY8zJfa+3mqjoryaVJtiW5qLV2RVWdOU6/IMnuJE9IsjfJ3yX5+Y0reVMt+iVJ9R28Ra4tUV+PRa4tWez6Frm2RH09Frm2ZPHrW9WaN6ADALAyXycDANBBmAIA6HDIhamqum9V7aqqq6vqyqraXVU/UFU3VtXl47jXV9Wdx/aPGj9HK1V1elW1qnrMZH5PGcedskH1PmWsa/rznap69rjc50zanldVp6/z8m8Yf+9YbXlV9bqq+mxVfayqPjOuwyNn5zMZPr2qzhsfP6iq3j3+b5+qqgO+dr7Kdv3kTLuXVNXZk+HDqurLVfW7M+2eVFUfHf+fK6vqFw+0phXqbFX18snw2VX1ksnwGVX16fHnQ1X1yHH886vqtZN2T6+qDf3k0qr69rhNPllVf1lV9xrHLz0XXjppe0RVfWtpm25wXUv73PGTcSeOz6H/U1UfqapLquqHx2kvqaovzOxD99rA+pbW2xXj8+f5VXWncdr0eHKfqnrL5Dm2e6NqmqnrVttzMv1jVfWGmXGr7tfrWNu9J9vmizPb6z7jc+sXJ+3vMe7rx43Dd66qT1TVww9wudN18udVdeQqdRx+IOuwqn5+8rc3jfVdXlXn1uT4N7Zddr9fD5OaPzbuGz+2XvNeYXk3LDPuNsf4qnrcZP3cUMNX1l1eVa8f/+ZW+3lV/e9x+uerav/kb3ds5P8zt9baIfOT4fOwPpDkzMm4hyb5iSSfHIe3JXlnkqePw49K8pbx8elJPp7kNZO//7Mklyc5ZZP+hzOSvCfJ/ZP83ww3/R8+TjsvyenrvLwbxt87VltektctrYNxPT8vyWcmbW+Yme/pSc4bH1+a5OTJtB9e7+06Gf+SJGdPhp+Q5H8luTq33EN45wwf7XHUOHyXJA9ap/X5jSSfTXLEOHx2kpeMj5+U5MOTaQ9L8vkk983wZpHLk/x4knuN87j/Bj/Xbpg8/uMkvz55Llyd5KOT6c8e6ztvE/aBNyZ532S93SfJtUl+bNLmkUmevNw234T6puvtHyb5n0l+cxx+VG45nrw6yS9P2j54K7bnOPyPknwiyReS3H0yftX9eoPqnN1Hf2nc3u+eafczSd4+Pv61JK/uXCf/NcnzV6rjYNfhOO3apf16HD49txz/VtzvN2C7Py7JezbreTYZt+oxPsm7k+ycGXer/Xy5dbdIP4daz9Sjk3yrDe9ATJK01i5Pct1k+NtJPpRkpbOv9yU5cTwT+p4kD8zwIrLhquoHkrw4yTOSfCfJ/iTvSPJzm7H8eZfXBq9I8sUM39u4lvtl+Kyypb//xAHWteZ2XcVpSV6V4eD1iHHcPTKEl6+M8/pma229Pq3/5gzvWnneMtNelOSFrbUvj8v9SIYD9r9trd2c4UXl/CS/l+FdtdesU03z+EBuvU/cmORTVbX0IXtPzXDw21DjPvfjSX4hw8e0JMlZSf64tfb+pXattb9qrb15o+tZS2vtSxlOgM6qqtkPN5593n98E0ub3Z5PS/InSd6e5F8s9wcHsV+vl9OSvCDJUdNesdbaG5N8p6p+NcmZGQJVj/dlOJ7P64DX4QpW3O8PYB7zumeSv9mA+a7lgI7xK+znC+1QC1M/lOEMYEVV9feSPDzJ21Zo0jKcaT4uw3cSzn7m1oao4bLjn2Y4U/r8ZNK5SV5QwxdSb4YDWd5Hkhy/ZqvkFUneWVVvrarnzXadz2G17fqASXfw5RkOukmSqrprksckeUuSN2Q4aKe19tUM2/VzVfWGGi6pree+cn6Sp1fV358Z/4PL/B97xvEZw8KnkvxUhkC1KcZt/Zjc9rm+K8mpVXVUkm9ncz6o98lJ3tZa+0ySr1bVwzKsn4+s8XfPmzwP3rXRRU6NofdOGXqpps5P8tqqeldV/XpVfd9m1LPC9nxqhl727+4Hq5h3v+5WVUdn6KH5UIaw/tSZJr+S5GVJ/v243x7scg7LEBDnOpFbh3U4tep+vw7uOj7vP53kNUleutYfbIADPcY/ObfdzxfaoRamVvOA8cX2K0k+v8ZZ4q4MafnUDDvOZnhpkitaa7umI1trn83Qk/a0zSjiAJe33NcM3Wp24zz/KEMX+Z9nuAzywaq6S0eZU1e31h669JPkgsm0JyV5V2vt75K8KclTlkJiG76L8jEZ/tezk1y0TvWktfb1JK9P8tw5mlfG9TSere3McBly2e+HWmd3newT35vkf8xMf1uSx2Z44fizTagn47KW9oFdWeZFa7y34lNV9arJ6FdMngeP3oxCZ8uaHdFauzTD5fr/nCGcfLSqNnK7Lrs9q+pHk+xvrX0uQ8/zw6rqH6wyn7X26/V0am7p8Vxue5+U5K8znFAdjKV1sidD7/RrV2++butwLd/d79fBjePz/vgM6+v1y/SSbqiDOMavuZ8vmkMtTF2R5EdWmHb1+GL7wCSPqKoVu2nHs6QfynCN+zPrXuWMqnpUkp/OcDljOb+Toat4s7bnvMv7xxl6UpLkxho+QX/J92bypZattetbaxe11k7OcCnsQA6Oq23X1ZyW5Keq6toMZ4b3znDJcKmmT4yXNR6bYf2vp1dm6MK++2Tclbnt//GwcXyS/GaS/5LktzOc6W20G8d94pgkh2fmskNr7aYM6+0FGcLohqqqeyf5ySSvGbfZCzP0BlyRYT0t1fXwJL+RZLbnb0tU1f0z9Nx9aXZaa+2rrbU/ba09I8O3TfzTDSxlpe15WpLjx3V6dYZLQas936f79UY7LcnpY20XJ3lI3XLT+fdlOCE5MckTqurBBzH/Gych+znjc3rN9ulfh1Nr7ffrprX2gQxfKLwZJ2Ozy57rGL/Sfr7ZAfBAHWph6p1J7lJV/2ZpxHhGcczScBu+BuecrH39/deS/LuNKHJqPLv5oyTPbK397XJtWmufzrDjPWmj65lneTV4bobr5EuXS9+T5GfH6XfNcPPou8bhk+qWd0/eN0Oo+cIBlLTmdl2mxntmuEn5+1trO1prOzIcGE+rqu8ZA+yShyb53AHUs6bxksQbMwSqJb+X5GXjwSRV9dAMN1v+QQ3vTHtihksaFyY5pqoeu541rVLr1zK8aJ29tJ0mXp7kRa21r2xCKackeX1r7Zhxmx2d4Ub8t2d4wZ2+S+lum1DPmsaepgsy3DDbZqb9ZFXdbXx8jyQPyNA7sqFmtuddkvyrDDe/L+0HJ2f5Hr/l9usNU1UPynAj95GT2n43t9xD84okv9Na25fk+UnO36wX3INdhytYcb9f57JTwzvjtmW8H3SzHOAxfqX9fN3e4bgR1vw6mTuS1lqrqqckeWVVnZPhnVXXZrjuPvXmJC+pqp9YZV5v3aAyZ52Z4V6LP5w5TsxeXvztJB/dpJpWWt5/qKrfyPBC9sEkj56c6f1yklePB+PKsLO8d5z2z5O8qqq+MQ6/sLX2xXkLOYDtOvUvk7yztfbNybj/nuHA9vwkv1pVr85wo/X/y3BwW28vz6S3sbV2cQ032L6/qlqSv80QQL+YoXv8ea21byRJVf1Shu76h85xNt2ttfbRqvpYhhey903GX5GhZ2gznJbhnr2pN2W45PzUDC9IR2boAfpykt+atHteVf3sZPjJrbVrN6jOpUtBd85wBv4nSX5/mXY/kuS8qro5w4nta1prl21QTbcy2Z4/k+QLrbXpC9t7k5xQVfcbh1fbrzfSaUn+Ymbcm5LsqqoPJvn+jJflWmt/OZ5MPTPDzdsbbt512Nb4ntqV9vu1/u4ALD0fk+HY+3NteKPVRrlbVe2bDP9+kqMy/zF+tf38fbdtvhh8nQwAQIdD7TIfAMC6EqYAADoIUwAAHYQpAIAOwhQAQAdhCgCggzAFANDh/wMskrUSdFZMBAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x504 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize = (10, 7))\n",
    "plt.bar(data['feature_names'], rf.feature_importances_);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "intimate-breeding",
   "metadata": {},
   "source": [
    "Будьте осторожны с сильно коррелирующими признаками. Посмотрим, что произойдёт с важностью, если добавить в выборку линейно зависимый признак."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 347,
   "id": "indoor-burner",
   "metadata": {},
   "outputs": [],
   "source": [
    "RM_mc = (X_train[:, 5] * 2 + 3).reshape(-1, 1)\n",
    "X_train_new = np.hstack((X_train, RM_mc))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 349,
   "id": "sensitive-slovakia",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RandomForestRegressor(oob_score=True, random_state=123)"
      ]
     },
     "execution_count": 349,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rf.fit(X_train_new, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 353,
   "id": "imposed-mechanism",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlkAAAGcCAYAAADqNbXdAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAdjElEQVR4nO3de7QlV10n8O+PDmEQYVDSPMyDjhjJRIVM7AkuwUdEMDzWBEZGEhGMIxOjRhSMGselC8VHcAZRF5EQMSrOYMTRaICG4AgKDjCmgfBIIEwTAjQRkwALzYiEyG/+qLrm5OZ239Pdd6dvt5/PWnfdU7t2Ve1z6lSdb+2qU6e6OwAAbKx7HOwGAAAcjoQsAIABhCwAgAGELACAAYQsAIABjjjYDVjLUUcd1du2bTvYzQAAWNc73vGOW7p76+ryTRmytm3blp07dx7sZgAArKuqPrJWudOFAAADCFkAAAMIWQAAAwhZAAADCFkAAAMIWQAAAwhZAAADCFkAAAMIWQAAAwhZAAADCFkAAAMIWQAAAwhZAAADCFkAAAMIWQAAAwhZAAADHHGwGwAAbKxtF7z2bl/mDRc+6W5f5manJwsAYAAhCwBgACELAGAAIQsAYAAhCwBgACELAGAAIQsAYAAhCwBgACELAGAAIQsAYAAhCwBgACELAGAAIQsAYAAhCwBgACELAGAAIQsAYAAhCwBgACELAGAAIQsAYAAhCwBgACELAGAAIQsAYAAhCwBgACELAGAAIQsAYAAhCwBgACELAGAAIQsAYIClQlZVnV5V11XVrqq6YI3xZ1TVe6rq6qraWVWPWXZaAIDD0RHrVaiqLUkuSvK4JLuTXFVVV3T3tQvV/jzJFd3dVfWIJK9KcuKS0wIAh7ltF7z2bl/mDRc+6W5f5qJlerJOTbKru6/v7tuSXJbkjMUK3X1rd/c8eJ8kvey0AACHo2VC1tFJPrYwvHsuu5OqempVfSDJa5P8p32Zdp7+nPlU486bb755mbYDAGxay4SsWqOs71LQfXl3n5jkKUlesC/TztNf0t3bu3v71q1bl2gWAMDmtUzI2p3k2IXhY5LcuKfK3f3mJA+rqqP2dVoAgMPFMiHrqiQnVNXxVXVkkjOTXLFYoaq+oqpqfnxKkiOTfHKZaQEADkfrfruwu2+vqvOSXJlkS5JLu/uaqjp3Hn9xkm9P8qyq+nySzyZ5+nwh/JrTDnouAACbxrohK0m6e0eSHavKLl54/MIkL1x2WgCAw507vgMADCBkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAwgZAEADLBUyKqq06vquqraVVUXrDH+GVX1nvnvrVX1yIVxN1TVe6vq6qrauZGNBwDYrI5Yr0JVbUlyUZLHJdmd5KqquqK7r12o9uEk39Tdn66qJyS5JMmjFsaf1t23bGC7AQA2tWV6sk5Nsqu7r+/u25JcluSMxQrd/dbu/vQ8+PYkx2xsMwEADi3LhKyjk3xsYXj3XLYn35vkdQvDneQNVfWOqjpnTxNV1TlVtbOqdt58881LNAsAYPNa93RhklqjrNesWHVappD1mIXiR3f3jVX1wCR/VlUf6O4332WG3ZdkOs2Y7du3rzl/AIBDxTI9WbuTHLswfEySG1dXqqpHJHl5kjO6+5Mr5d194/z/piSXZzr9CABwWFsmZF2V5ISqOr6qjkxyZpIrFitU1XFJ/jjJM7v7gwvl96mq+648TvL4JO/bqMYDAGxW654u7O7bq+q8JFcm2ZLk0u6+pqrOncdfnORnkjwgyW9UVZLc3t3bkzwoyeVz2RFJXtndrx/yTAAANpFlrslKd+9IsmNV2cULj5+d5NlrTHd9kkeuLgcAONy54zsAwABCFgDAAEIWAMAAQhYAwABCFgDAAEIWAMAAQhYAwABCFgDAAEIWAMAAQhYAwABCFgDAAEIWAMAAQhYAwABCFgDAAEIWAMAAQhYAwABCFgDAAEIWAMAAQhYAwABCFgDAAEIWAMAAQhYAwABCFgDAAEIWAMAAQhYAwABCFgDAAEIWAMAAQhYAwABCFgDAAEIWAMAAQhYAwABCFgDAAEIWAMAAQhYAwABCFgDAAEIWAMAAQhYAwABCFgDAAEIWAMAAQhYAwABCFgDAAEcc7AYAG2/bBa+925d5w4VPutuXCbCZ6ckCABhAyAIAGEDIAgAYQMgCABhgqZBVVadX1XVVtauqLlhj/DOq6j3z31ur6pHLTgsAcDhaN2RV1ZYkFyV5QpKTkpxVVSetqvbhJN/U3Y9I8oIkl+zDtAAAh51lerJOTbKru6/v7tuSXJbkjMUK3f3W7v70PPj2JMcsOy0AwOFomZB1dJKPLQzvnsv25HuTvG4/pwUAOCwsczPSWqOs16xYdVqmkPWY/Zj2nCTnJMlxxx23RLMAADavZXqydic5dmH4mCQ3rq5UVY9I8vIkZ3T3J/dl2iTp7ku6e3t3b9+6desybQcA2LSWCVlXJTmhqo6vqiOTnJnkisUKVXVckj9O8szu/uC+TAsAcDha93Rhd99eVecluTLJliSXdvc1VXXuPP7iJD+T5AFJfqOqkuT2uVdqzWkHPRcAgE1jqR+I7u4dSXasKrt44fGzkzx72WkBAA537vgOADCAkAUAMICQBQAwgJAFADCAkAUAMICQBQAwgJAFADCAkAUAMICQBQAwgJAFADCAkAUAMICQBQAwgJAFADCAkAUAMICQBQAwgJAFADCAkAUAMICQBQAwgJAFADCAkAUAMICQBQAwgJAFADCAkAUAMICQBQAwgJAFADCAkAUAMICQBQAwgJAFADCAkAUAMICQBQAwgJAFADCAkAUAMICQBQAwgJAFADCAkAUAMICQBQAwgJAFADCAkAUAMICQBQAwgJAFADCAkAUAMICQBQAwgJAFADCAkAUAMICQBQAwgJAFADDAUiGrqk6vquuqaldVXbDG+BOr6m1V9bmqOn/VuBuq6r1VdXVV7dyohgMAbGZHrFehqrYkuSjJ45LsTnJVVV3R3dcuVPtUkuckecoeZnNad99ygG0FADhkLNOTdWqSXd19fXffluSyJGcsVujum7r7qiSfH9BGAIBDzjIh6+gkH1sY3j2XLauTvKGq3lFV5+xL4wAADlXrni5MUmuU9T4s49HdfWNVPTDJn1XVB7r7zXdZyBTAzkmS4447bh9mDwCw+SzTk7U7ybELw8ckuXHZBXT3jfP/m5Jcnun041r1Lunu7d29fevWrcvOHgBgU1omZF2V5ISqOr6qjkxyZpIrlpl5Vd2nqu678jjJ45O8b38bCwBwqFj3dGF3315V5yW5MsmWJJd29zVVde48/uKqenCSnUnul+QLVfUjSU5KclSSy6tqZVmv7O7XD3kmAACbyDLXZKW7dyTZsars4oXHn8h0GnG1v0vyyANpIADAocgd3wEABhCyAAAGELIAAAYQsgAABhCyAAAGELIAAAYQsgAABhCyAAAGELIAAAYQsgAABhCyAAAGELIAAAYQsgAABhCyAAAGELIAAAYQsgAABhCyAAAGELIAAAYQsgAABhCyAAAGELIAAAYQsgAABhCyAAAGELIAAAYQsgAABhCyAAAGELIAAAYQsgAABhCyAAAGELIAAAYQsgAABhCyAAAGELIAAAYQsgAABhCyAAAGELIAAAYQsgAABhCyAAAGELIAAAYQsgAABhCyAAAGELIAAAYQsgAABhCyAAAGELIAAAYQsgAABhCyAAAGWCpkVdXpVXVdVe2qqgvWGH9iVb2tqj5XVefvy7QAAIejdUNWVW1JclGSJyQ5KclZVXXSqmqfSvKcJP9tP6YFADjsLNOTdWqSXd19fXffluSyJGcsVujum7r7qiSf39dpAQAOR8uErKOTfGxhePdctoylp62qc6pqZ1XtvPnmm5ecPQDA5rRMyKo1ynrJ+S89bXdf0t3bu3v71q1bl5w9AMDmtEzI2p3k2IXhY5LcuOT8D2RaAIBD1jIh66okJ1TV8VV1ZJIzk1yx5PwPZFoAgEPWEetV6O7bq+q8JFcm2ZLk0u6+pqrOncdfXFUPTrIzyf2SfKGqfiTJSd39d2tNO+i5AABsGuuGrCTp7h1Jdqwqu3jh8ScynQpcaloAgMOdO74DAAwgZAEADCBkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAywVMiqqtOr6rqq2lVVF6wxvqrq1+fx76mqUxbG3VBV762qq6tq50Y2HgBgszpivQpVtSXJRUkel2R3kquq6oruvnah2hOSnDD/PSrJS+f/K07r7ls2rNUAAJvcMj1ZpybZ1d3Xd/dtSS5LcsaqOmckeUVP3p7k/lX1kA1uKwDAIWOZkHV0ko8tDO+ey5at00neUFXvqKpz9rSQqjqnqnZW1c6bb755iWYBAGxey4SsWqOs96HOo7v7lEynFH+wqr5xrYV09yXdvb27t2/dunWJZgEAbF7LhKzdSY5dGD4myY3L1unulf83Jbk80+lHAIDD2roXvie5KskJVXV8ko8nOTPJd66qc0WS86rqskwXvH+mu/+mqu6T5B7d/ffz48cn+bmNaz7A/tl2wWsPynJvuPBJB2W5wN1v3ZDV3bdX1XlJrkyyJcml3X1NVZ07j784yY4kT0yyK8k/JPmeefIHJbm8qlaW9crufv2GPwsAgE1mmZ6sdPeOTEFqsezihced5AfXmO76JI88wDYCABxy3PEdAGAAIQsAYAAhCwBgACELAGAAIQsAYAAhCwBgACELAGAAIQsAYAAhCwBgACELAGAAIQsAYAAhCwBgACELAGAAIQsAYAAhCwBgACELAGAAIQsAYAAhCwBggCMOdgMOlm0XvPZuX+YNFz7pbl8mAHBw6MkCABhAyAIAGEDIAgAYQMgCABhAyAIAGEDIAgAYQMgCABhAyAIAGEDIAgAYQMgCABjgX+zP6sBG8RNNAKxFyAJgjxxEwP5zuhAAYAA9WcBwB6M3JNEjcrjSu8ahQk8WAMAAQhYAwABCFgDAAEIWAMAAQhYAwABCFgDAAEIWAMAA7pMFAAfAfeDYEz1ZAAADCFkAAAM4Xcge6QIHgP0nZAFsEn6TDw4vThcCAAywVMiqqtOr6rqq2lVVF6wxvqrq1+fx76mqU5adFgDgcLRuyKqqLUkuSvKEJCclOauqTlpV7QlJTpj/zkny0n2YFgDgsLPMNVmnJtnV3dcnSVVdluSMJNcu1DkjySu6u5O8varuX1UPSbJtiWmZuR5jfS7GB+BQUVMu2kuFqqclOb27nz0PPzPJo7r7vIU6r0lyYXf/1Tz850l+IlPI2uu0C/M4J1MvWJI8PMl1B/bUhjoqyS0HuxELNlt7Em1a1mZr02ZrT6JNy9psbdps7Um0aVnatO8e2t1bVxcu05NVa5StTmZ7qrPMtFNh9yVJLlmiPQddVe3s7u0Hux0rNlt7Em1a1mZr02ZrT6JNy9psbdps7Um0aVnatHGWCVm7kxy7MHxMkhuXrHPkEtMCABx2lvl24VVJTqiq46vqyCRnJrliVZ0rkjxr/pbh1yX5THf/zZLTAgAcdtbtyeru26vqvCRXJtmS5NLuvqaqzp3HX5xkR5InJtmV5B+SfM/eph3yTO5em+205mZrT6JNy9psbdps7Um0aVmbrU2brT2JNi1LmzbIuhe+AwCw79zxHQBgACELAGAAIWtWVQ+uqsuq6kNVdW1V7aiqr6yqz1bV1XPZK6rqnnP9b57vD5aqOruquqoeuzC/p85lT9vANj51bsvi3xeq6vvnZf3QQt2XVNXZB7i8W+f/2/Y2/6r6nar6cFW9u6o+OL9OR6+ez8Lw2VX1kvnxw6vqL+bn8v6qWvq8+17W2ftW1Xt+VZ2/MHxEVd1SVb+0qt6Tq+pd8/O4tqq+b9m2rNG2rqoXLQyfX1XPXxg+p6o+MP/9dVU9Zi5/XlX91kK9Z1TVht+Btar+aX7N31dVr66q+8/lK+v6BQt1j6qqz6+sswFtWdlWTlwoO3V+X/zfqnpnVb22qr5mHvf8qvr4qu3g/gPatfIaXTO/J55XVfeYxy1u/w+qqtcsvG92DGrHndbVwvh3V9Xvryrb6za5AW16wMJr/4lV6+NB8/vl+xbq33feTk+Yh+9ZVe+tqkctsazF5/+HVXX0XpZ95L68XlX1PQvT3ja36eqqurAW9lNz3TW32QN4DVfa+e75Pf71BzK/PSzj1jXK7rLPrapvW3gdbq3pp/CurqpXzNPcaRutqv8zj/9oVd28MO22jX4Oh7zu/hf/l+l+Xm9Lcu5C2clJviHJ++bhLUnemOQZ8/A3J3nN/PjsJO9J8vKF6f8gydVJnjaw3eck+cskX57kbzN98eDIedxLkpx9gPO/df6/bW/zT/I7K89zfi2fm+SDC3VvXTXfs5O8ZH58ZZIzFsZ9zUats4Xy5yc5f2H4iUn+d5IP5Y7rEu+Z6fYix8zD90ry8AN47f4xyYeTHDUPn5/k+fPjJyd5x8K4U5J8NMmDM30Z5eokj05y/3keXz7gvXPrwuPfTfJTC+v6Q0netTD+++c2vWTQ+/hVSd6y8Po8KMkNSb5+oc5jkjxlrfU56m/Va/TAJP8ryc/Ow9+cO7b/lyX54YW6j7g71tU8/G+SvDfJx5PcZ6F8r9vkBrdv9fb1A/P6/ItV9b4jyRvmxz+Z5GX78fz/R5Ln7WnZ+/t6zeNuWNkm5+Gzc8d+ao/b7Aat129L8pcj38MLZXvd5yb5iyTbV5XdaRtd6zVapx3/NO9D3pfk1UnuP5dvy3TvzBcs1D0qyeeXme+h8Kcna3Jaks/39E3JJEl3X53kYwvD/5Tkr5Ps6WjwLUlOnY/QvjjJV2R6Uw1RVV+Z5GeSPDPJF5LcnOTPk3z3oEUuNf+evDjJJzL9ZuV6HpLpPmsr0793yfasu8724qwkv5ZpJ/l1c9l9MwWcT87z+lx3H8ivDtye6dswz11j3E8k+bHuvmVe1jszfRj8YHffnulD6qIkv5zpG7nXH0A7lvG23Pl9/dkk76+qlRv/PT3TTnbDzdvKo5N8b6ZbvCTJeUl+t7vfulKvu/+qu/9kRBuW0d03ZTqoOa+qVt9kefV7+D0Dm7J6XX1nkt9L8oYk/36tCfZjmzxQZyX50STHLPaedferknyhqn48ybmZgta+ekumfeuy9vn12oM9brP7MI+9uV+ST2/QvNazT/vcPWyj++qz3X1yd391kk/lzq/b9ZlC7Ir/mORwuAtBEqcLV3x1pqOUPaqqf5XkUUlev4cqnelI99sy/T7jsPuB1XTK8pWZjuA+ujDqwiQ/WtMPc4+wL/N/Z5IT162VvDjJG6vqdVX13NVd+3uxt3X2sIXu66sz7dCTJFV17ySPTfKaJL+f6QMh3f2pTOvsI1X1+zWdpjvQ7eOiJM+oqn+9qvyr1mj7zrk8c7h4f5JvzRS0hpnX5WNz1/frZUnOrKpjMh2FjrqJ8FOSvL67P5jkU1V1SqbX4Z3rTPfchXX8pkFtu5M57N4jU6/WoouS/FZVvamqfqqqvmzE8vewrp6eqdf8n9/Le7HsNrnfqurYTL07f50pmD99VZUfSfLCJD8/b3P7Mu8jMoXEpQ7ENuD1WrTXbXY/3Xt+/34gycuTvGC9CTbIvu5zn5K7bqMH4oAP6mo6Hf7SeZu7vqq+qaounU9//s5CvdPnU7Hvrunn/u52Qtb6HjZ/UH8yyUfXOUq9LFPSPzPTRjzKC5Jc092XLRZ294cz9bZ954iF7uP81/pJpTvNbp7nb2fqwv/DTKdg3l5V9zqAZibJh+ajppO7++QkFy+Me3KSN3X3PyT5oyRPXQmNPf3G5mMzPcfzk1x6II3o7r9L8ookz1miemV+TeYjx+2ZTmHe5bewNsi9F97XX5rkz1aNf32Sx2X6IPqDQW3IPP+V9/FlWeODb77+4/1V9WsLxS9eWMenDWzfXZqzuqC7r8x0yv43M4WYd1XVRq63NddVVf27JDd390cy9TKfUlVfsi9tH+DM3PEBudb6PD3J32Q6SFrWyvPfman3+bf2Xn3DXq/1/PM2u59WendOzPS6vGKNXtINtx/73HW30WVt8EHdlyT5lkxnC16dKTx+VZKvqaqT523wN5N8e3c/MlMP2d1OyJpck+Rr9zDuQ/MH9Vck+bqq2mMX83z09tWZztt/cMNbmemC2yTfnumUylp+MVPX9qh1u+z8/22m3pgk+WxNd/xf8aVZ+KHP7r6xuy/t7jMynWZbZge8t3W2N2cl+daquiHTkekDMp16XGnLe+dTK4/L9DofqF/N1M1+n4Wya3PXtp8ylyfJzyb570l+IdOOY4TPzu/rh2b6+as7nfbo7tsyvT4/mimMbriqekCmneTL5/XxY5mOYq/J9HqstOVRSX46yeoewbtVVX15pg+Am1aP6+5Pdfcru/uZmX7p4hs3cNF7WldnJTlxfu0+lOmU097es4vb5ChnJTl7btMVSR5Zd1zs/mWZDjhOTfLEqnrEkvP87EKg/qH5vblu/Rz467VovW32gHT32zJdizTqoGr18pba5+5pG92PMDjioO7V3d2Zejb/dt53fyHT/mNbpktB3jx3DqycrbjbCVmTNya5V1X955WC+ajnoSvDPf1M0AVZ/zqCn0zyX0Y0cj7q+u0kz+ruv1+rTnd/INOG/+S1xh+o9eZfk+dkOu+/cmr1L5N81zz+3pkugH3TPHx63fGNzQdnCj0fX6Ip666zNdp2v0wXUB/X3du6e1umHfBZVfXFc4BdcXKSjyzRjr2aN+xXZQpaK345yQvnHViq6uRMF5D+Rk3foHtSplMqlyR5aFU97kDbsZf2fSbTB9/5K+thwYuS/ER3f3LQ4p+W5BXd/dB5fRyb6UL/N2T6oF78ttUXDWrDUuaj4oszXYzbq8Z9S1V90fz4vkkelqnHZUOtWlf3ynRk/oiF9/IZWbsncK1tcsNV1cMzXUx+9EKbfil3XMfz4iS/2N27kzwvyUUje2729/Xagz1usxvR1pq+tbcl8zWhI+3jPndP2+i+frNyxEHd5+b/X1h4vDJ8RA68p3FDLPMD0Ye97u6qemqSX62qCzJ9M+yGTNcPLPqTJM+vqm/Yy7xeN6iZyXRt0QOTvHTVvmn1qclfSPKuge1Ya/7/tap+OtOH4duTnLZwxPnDSV427+gr00b75nnc45P8WlX94zz8Y939ifUasA/rbNF/SPLG7l7cIP800w70eUl+vKpelukagf+XaSe6EV6UhZ7H7r6ipguC31pVneTvM4XQT2Tqwn9ud/9jklTVD2Q6jXDyEkfw+6W731VV7870YfiWhfJrMvYC1LMyXee36I8ynY5+eqYPtaMz9RzdkuTnFuo9t6q+a2H4Kd19wwa3b+Xo+56ZjvZ/L8mvrFHva5O8pKpuz3Tg+vLuvmqD25LkTuvqO5J8vLsXPxzfnOSkqnrIPLy3bXKEs5Jcvqrsj5JcVlVvT3Jc5lN93f3q+QDpWZkuIB9i2ddrPoje23zW3GbXm24dK++vZNovfndPX7DaSF9UVbsXhn8lyTFZfp+7t230LXetvnfd/Zn5c+BPq+qlq0a/KNM3LD+5Qdn7bZmC/PHd/eGq+tKD0ZvlZ3UAgGGq6tbu/uKF4VfnjttCvGb+1uFi/bMz3UZizctiarq4/TXd/T9rujfXP89j1bgnZLrE5R5JburuYWcF9kTIAgAYwDVZAAADuCYLANh0quqnctdbL/xhd//CwWjP/nC6EABgAKcLAQAGELIAAAYQsgAABhCyAAAG+P9d64AnKKEFtgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x504 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize = (10, 7))\n",
    "names = list(data['feature_names'])\n",
    "names.append('RM_mc')\n",
    "plt.bar(names, rf.feature_importances_);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "instrumental-absence",
   "metadata": {},
   "source": [
    "Важности перераспределились между линейной зависимыми признаками `RM` и `RM_mc`. Не забывайте учитывать корреляции между признаками, если вы используете этот метод для отбора признаков. Также обратите внимание на предупреждение в документации `sklearn`: не стоит использовать этот метод и для признаков, в которых есть много уникальных значений (например, категориальные признаки с небольшим числом категорий). "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "czech-consultation",
   "metadata": {},
   "source": [
    "### 5. Тестирование случайного леса на разных данных"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "personal-rebecca",
   "metadata": {},
   "source": [
    "Ниже представлены шаблоны для сравнения случайного леса и других моделей на данных разных типов. Проведите побольше экспериментов, используя разные модели и метрики. Попробуйте подобрать гиперпараметры случайного леса так, чтобы достичь какого-нибудь порога качества. \n",
    "\n",
    "**Внимание:** в этой части вам предстоит скачивать объёмные наборы данных. Не забудьте удалить их после семинара, если не планируете использовать их в дальнейшем, чтобы они не занимали лишнее место на вашем компьютере.\n",
    "\n",
    "**! Случайный лес может обучаться достаточно долго.**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "needed-cleveland",
   "metadata": {},
   "source": [
    "#### 5.1 Бинарная классификация на примере [Kaggle Predicting a Biological Response](https://www.kaggle.com/c/bioresponse/data?select=train.csv)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 290,
   "id": "increasing-trout",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Загрузка данных\n",
    "!wget  -O 'kaggle_response.csv' -q 'https://www.dropbox.com/s/uha70sej5ugcrur/_train_sem09.csv?dl=1'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 293,
   "id": "occupational-mileage",
   "metadata": {},
   "outputs": [],
   "source": [
    "data = pd.read_csv('kaggle_response.csv')\n",
    "X = data.iloc[:, 1:].values\n",
    "y = data.iloc[:, 0].values\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.3, random_state = 123)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "quiet-newfoundland",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.metrics import roc_auc_score\n",
    "\n",
    "# TODO: обучите логистическую регрессию и случайный лес с дефолтными параметрами\n",
    "# Сравните их AUC ROC на тестовой выборке\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "skilled-complexity",
   "metadata": {},
   "source": [
    "#### 5.2 Изображения на примере [Fashion MNIST](https://github.com/zalandoresearch/fashion-mnist)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 308,
   "id": "scenic-network",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Загрузка данных\n",
    "import torchvision\n",
    "\n",
    "fmnist = torchvision.datasets.FashionMNIST('./', download = True)\n",
    "X = fmnist.data.numpy().reshape(-1, 28 * 28)\n",
    "y = fmnist.targets.numpy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 310,
   "id": "current-ocean",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7f93e885a880>"
      ]
     },
     "execution_count": 310,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAUDklEQVR4nO3da2yc1ZkH8P8z4/ElzjiJc3FCcAmXUJLCEqhJgFSUkkJDtNqQUioQYkFCG7QL3bbLBxDtquyXFUILCC277RrIElaFqlVBUBRRgrlkgZLGhJTcNgQSk5tjOzGxHcdjz+XZDx5aE3ye18w7M+/A+f8ky/Y8PjPHM/77nZnznnNEVUFEX36xqDtAROXBsBN5gmEn8gTDTuQJhp3IE1XlvLFqqdFa1JfzJom8ksIgRnRYxquFCruILAfwMIA4gMdU9T7r52tRjyWyLMxNEpFho7Y5awU/jReROID/AHA1gIUAbhCRhYVeHxGVVpjX7IsBfKCqe1R1BMCvAKwsTreIqNjChH0ugP1jvj+Qv+xTRGS1iLSLSHsawyFujojCCBP28d4E+My5t6raqqotqtqSQE2ImyOiMMKE/QCA5jHfnwrgULjuEFGphAn7JgDzReR0EakGcD2A54vTLSIqtoKH3lQ1IyJ3APg9Rofe1qjq9qL1jIiKKtQ4u6quA7CuSH0hohLi6bJEnmDYiTzBsBN5gmEn8gTDTuQJhp3IEww7kScYdiJPMOxEnmDYiTzBsBN5gmEn8gTDTuSJsi4lTRGQcVcV/ouQG3vGpzea9Y+/c7az1vDU26FuO+h3k6qEs6bpkXC3HVbQ42Ip8DHjkZ3IEww7kScYdiJPMOxEnmDYiTzBsBN5gmEn8gTH2b/kJB4365rJmPXYInuvzp23TbbbD7lricHFZtuqoZxZT7zUbtZDjaUHjeEH3K8Q+zgapm9SZcTWeDh5ZCfyBMNO5AmGncgTDDuRJxh2Ik8w7ESeYNiJPMFx9i85c0wWwePs+78z1azfeMn/mvU3e85w1j6qmW221TqzjKpvX2LWz/7Pg85apmOffeUBc8aD7rcg8WnT3MVs1myb7e93F41uhwq7iHQAGACQBZBR1ZYw10dEpVOMI/u3VPVIEa6HiEqIr9mJPBE27ArgJRF5R0RWj/cDIrJaRNpFpD2N4ZA3R0SFCvs0fqmqHhKRWQDWi8j/qeqGsT+gqq0AWgGgQRrDrW5IRAULdWRX1UP5z90AngVgT2MiosgUHHYRqReR5CdfA7gKwLZidYyIiivM0/gmAM/K6LzfKgBPqeqLRekVFU0ulQrVfuSC42b9e1PsOeW1sbSz9nrMnq9+8JVms579K7tvHz2YdNZy715qtp2+zR7rbni306wfuWyuWe/5uvsVbVPAcvrTXv7QWZNed6QLDruq7gFwfqHtiai8OPRG5AmGncgTDDuRJxh2Ik8w7ESeEA25Ze/n0SCNukSWle32vGEtexzw+B7//sVm/eqfvmbWF9QeMusDuVpnbUTDncD5yK5vmvXBPVOctdhIwJbJAeVsk70UtKbt4+i0ze7fvW5ll9lWHp3prL3X9jCO9+4ft/c8shN5gmEn8gTDTuQJhp3IEww7kScYdiJPMOxEnuA4eyUI2B44lIDH99x37P/3351mT2ENEjfWNh7UarPtsWx9qNvuybinuKYDxvgf221PgT1ujOEDQCxjP6ZXfutdZ+3axk1m2/vPPM9Z26ht6NdejrMT+YxhJ/IEw07kCYadyBMMO5EnGHYiTzDsRJ7gls2VoIznOpxs9/FZZv1ow2Szfjgz1axPj7uXe07Ghsy28xL2fqE9Wfc4OgDEE+6lqkc0brb9l6/9zqynFiTMekLspagvNdYBuG7H35pt67HHrLvwyE7kCYadyBMMO5EnGHYiTzDsRJ5g2Ik8wbATeYLj7J6bWWNve1wr7i2XAaBaMmb9UHqas7Z76Ktm2/f77XMAljdtN+tpYyzdmmcPBI+Tn5L42Kyn1B6Ht+7VpU32OPoWs+oWeGQXkTUi0i0i28Zc1igi60Vkd/6z+xElooowkafxTwBYftJldwNoU9X5ANry3xNRBQsMu6puANB70sUrAazNf70WwDXF7RYRFVuhb9A1qWonAOQ/O19cichqEWkXkfY0hgu8OSIKq+Tvxqtqq6q2qGpLAjWlvjkicig07F0iMgcA8p+7i9clIiqFQsP+PICb81/fDOC54nSHiEolcJxdRJ4GcDmAGSJyAMDPANwH4NciciuAfQCuK2Unv/QC1o2XuD33WjPuse74NHtU9JtTt5r1nmyDWT+WnWTWp8ZPOGsDGffe7QDQO2Rf9zk1nWZ984l5ztrManuc3Oo3AHSMzDDr82sOm/X7u9z7JzTXnvx++Kdlll3mrOnGPzhrgWFX1RscJe72QPQFwtNliTzBsBN5gmEn8gTDTuQJhp3IE5ziWgkClpKWKvthsobe9t+6wGx7xSR7yeS3UnPN+syqAbNuTTOdU9Nntk02pcx60LBfY5V7+u5Ats5sOylmn9od9HtfWG0vg/3jly901pLnHjXbNiSMY7QxissjO5EnGHYiTzDsRJ5g2Ik8wbATeYJhJ/IEw07kCY6zVwBJVJv1XMoeb7bM2Dpi1o9k7SWPp8bsqZ7VAUsuW1sjX9q412zbEzAWvnnodLOejLu3hJ4Zs8fJmxP2WPfWVLNZXzd4llm/9a9fdtaebr3SbFv94lvOmqj78eKRncgTDDuRJxh2Ik8w7ESeYNiJPMGwE3mCYSfyxBdrnN1Yclmq7PFiiQf8X4vZ9VzKmN+cs8eag2jaHgsP4+H/esSs789MNeuH03Y9aMnlrDHB+u2hKWbb2pi9XfTMqn6z3p+zx+ktAzl7mWtrnj4Q3Pe7pu921p7p+7bZtlA8shN5gmEn8gTDTuQJhp3IEww7kScYdiJPMOxEnqiocfYw66MHjVWrPewZqaGVi836/mvscfwbL/ijs3Y4kzTbvmtsawwAU4w54QBQH7C+ekrd5z8cGrG3kw4aq7bWhQeAWcY4fFbt49zBtN23IEHnHxzIGGva/409137qkwV1KfjILiJrRKRbRLaNuexeETkoIlvyHysKu3kiKpeJPI1/AsDycS5/SFUX5T/WFbdbRFRsgWFX1Q0AesvQFyIqoTBv0N0hIu/ln+Y7X+CIyGoRaReR9jTs13dEVDqFhv3nAM4EsAhAJ4AHXD+oqq2q2qKqLQnUFHhzRBRWQWFX1S5VzapqDsCjAOy3k4kocgWFXUTmjPl2FYBtrp8losoQOM4uIk8DuBzADBE5AOBnAC4XkUUAFEAHgNuK0RlrHD2sqjmzzXr69Caz3rvAvRf4idnGptgAFq3YadZvafpvs96TbTDrCTH2Z09PN9teMKnDrL/St9CsH6mabNatcfpL691zugHgWM7ef/2Uqo/N+l0ffM9Za5pkj2U/dpo9wJTWnFnflbZfsvbl3PPh/3Hhq2bbZzHTrLsEhl1Vbxjn4scLujUiigxPlyXyBMNO5AmGncgTDDuRJxh2Ik9U1BTX4asvMuuzfrLHWVvUcMBsu7DuDbOeytlLUVvTLXcMzTXbnsjZWzLvHrGHBfsy9hBUXNzDQN0j9hTXB/bayxa3Lf6FWf/pofHmSP1FrE6dtaNZe9ju2sn2UtGA/Zjd9pUNztoZ1d1m2xcG55j1QwFTYJsSfWZ9XqLHWftu8n2zbaFDbzyyE3mCYSfyBMNO5AmGncgTDDuRJxh2Ik8w7ESeKO84u9jLRS/5101m82XJ7c7aCbWnFAaNoweNm1qmVNnLBg+n7bu5O21PYQ1yds1hZ21Vwxaz7YZHlpj1b6R+YNY/vMKents25J7K2ZOxf+/r915h1jfvazbrF8/b66ydlzxotg06tyEZT5l1a9oxAAzm3H+vb6fs8w8KxSM7kScYdiJPMOxEnmDYiTzBsBN5gmEn8gTDTuQJUXXPNy62utnNeuZN/+Sst97+72b7p3ovdtaaa+3t6E6rPmLWp8ft7X8tyZg95vrVhD3m+sLgqWb9tWPnmPWvJzuctYTY2z1fPukDs37Lj+8065laexnt/nnu40mm3v7bazj/qFn/wVmvmPVq43c/lrXH0YPut6AtmYNYaxAkY/Y22Q+sWOWs/aHjCfQNdY77oPDITuQJhp3IEww7kScYdiJPMOxEnmDYiTzBsBN5oqzz2WNpYFKXe3zxhf5FZvsz6txrbR9J2+uj//74eWb91Dp7+19r6+GzjPnkALAlNdWsv9jzNbN+Sp29fnpXeoqzdjRdb7Y9YcyrBoDHH3rQrD/QZa87v6pxs7N2frU9jn4sZx+LdgSstz+Qq3XWUmqvb9AXMA6fNP4eACCtdrTixpbPU2P2GH7/ee5tuLNd7tsNPLKLSLOIvCoiO0Vku4j8MH95o4isF5Hd+c+Fr/5ARCU3kafxGQB3quoCABcDuF1EFgK4G0Cbqs4H0Jb/nogqVGDYVbVTVTfnvx4AsBPAXAArAazN/9haANeUqI9EVASf6w06EZkH4AIAGwE0qWonMPoPAcAsR5vVItIuIu2Z4cGQ3SWiQk047CIyGcBvAfxIVYN23PszVW1V1RZVbamqsd8sIqLSmVDYRSSB0aD/UlWfyV/cJSJz8vU5AOxtMYkoUoFDbyIiAB4HsFNVx47DPA/gZgD35T8/F3Rd8ZEckvuHnfWc2tMlXzninurZVDtgtl2U3G/Wd52wh3G2Dp3irG2u+orZti7u3u4ZAKZU21Nk66vc9xkAzEi4f/fTa+z/wdY0UADYlLJ/t7+f+ZpZ35dxD9L8bvBss+2OE+77HACmBSzhvbXf3f5Ext5GezhrRyOVsYdyp9TYj+lFjR85a7tgbxfdc74xbfhNd7uJjLMvBXATgK0isiV/2T0YDfmvReRWAPsAXDeB6yKiiASGXVXfAOA65C4rbneIqFR4uiyRJxh2Ik8w7ESeYNiJPMGwE3mivFs2Hx9C7PV3neXfvLTUbP7PK3/jrL0esNzyC4ftcdH+EXuq58xJ7lN9G4xxbgBoTNinCQdt+VwbsP3vxxn3mYnDMXsqZ9Y50DLq8LB7+iwAvJmbb9bTOfeWzcNGDQg+P6F3ZIZZP6Wuz1kbyLinvwJAx0CjWT/SZ2+rnJpkR+uN7JnO2vLZ7q3JAaCu2/2YxYw/FR7ZiTzBsBN5gmEn8gTDTuQJhp3IEww7kScYdiJPlHXL5gZp1CVS+ES5vhvdWzaf8Q+7zLaLp+4165v77Xnb+4xx13TAkseJmHvZYACYlBgx67UB483Vcfec9BjsxzcXMM5eH7f7FjTXvqHKPa87GbfnfMeMbY0nIm787n/smxfqupMBv3dG7b+JS6Z86Kyt2Xup2XbKCvc22xu1Df3ayy2biXzGsBN5gmEn8gTDTuQJhp3IEww7kScYdiJPlH+cPX6V+wdy9hrmYQxeu8SsL7lnk11PusdFz6nuMtsmYI8X1waMJ9fH7LHwlPEYBv03f2Oo2axnA67hlY8XmPW0Md7cdaLBbJswzh+YCGsfgqFMwJbNQ/Z893jMzk3qNXuu/fQd7nMnatbZf4sWjrMTEcNO5AuGncgTDDuRJxh2Ik8w7ESeYNiJPBE4zi4izQCeBDAbQA5Aq6o+LCL3Avg7AD35H71HVddZ1xV2PnulkovsNemHZteZ9Zqj9tzogdPs9g0futeljw3ba87n/rTTrNMXizXOPpFNIjIA7lTVzSKSBPCOiKzP1x5S1X8rVkeJqHQmsj97J4DO/NcDIrITwNxSd4yIiutzvWYXkXkALgCwMX/RHSLynoisEZFpjjarRaRdRNrTsJ+uElHpTDjsIjIZwG8B/EhV+wH8HMCZABZh9Mj/wHjtVLVVVVtUtSUBez81IiqdCYVdRBIYDfovVfUZAFDVLlXNqmoOwKMAFpeum0QUVmDYRUQAPA5gp6o+OObyOWN+bBWAbcXvHhEVy0TejV8K4CYAW0VkS/6yewDcICKLACiADgC3laB/Xwi6aatZtydLBmt4q/C24RZjpi+Tibwb/wYw7uLi5pg6EVUWnkFH5AmGncgTDDuRJxh2Ik8w7ESeYNiJPMGwE3mCYSfyBMNO5AmGncgTDDuRJxh2Ik8w7ESeYNiJPFHWLZtFpAfAR2MumgHgSNk68PlUat8qtV8A+1aoYvbtNFWdOV6hrGH/zI2LtKtqS2QdMFRq3yq1XwD7Vqhy9Y1P44k8wbATeSLqsLdGfPuWSu1bpfYLYN8KVZa+RfqanYjKJ+ojOxGVCcNO5IlIwi4iy0Vkl4h8ICJ3R9EHFxHpEJGtIrJFRNoj7ssaEekWkW1jLmsUkfUisjv/edw99iLq270icjB/320RkRUR9a1ZRF4VkZ0isl1Efpi/PNL7zuhXWe63sr9mF5E4gPcBXAngAIBNAG5Q1R1l7YiDiHQAaFHVyE/AEJHLABwH8KSqnpu/7H4Avap6X/4f5TRVvatC+nYvgONRb+Od361ozthtxgFcA+AWRHjfGf36Pspwv0VxZF8M4ANV3aOqIwB+BWBlBP2oeKq6AUDvSRevBLA2//VajP6xlJ2jbxVBVTtVdXP+6wEAn2wzHul9Z/SrLKII+1wA+8d8fwCVtd+7AnhJRN4RkdVRd2YcTaraCYz+8QCYFXF/Tha4jXc5nbTNeMXcd4Vsfx5WFGEfbyupShr/W6qqFwK4GsDt+aerNDET2sa7XMbZZrwiFLr9eVhRhP0AgOYx358K4FAE/RiXqh7Kf+4G8Cwqbyvqrk920M1/7o64P39WSdt4j7fNOCrgvoty+/Mowr4JwHwROV1EqgFcD+D5CPrxGSJSn3/jBCJSD+AqVN5W1M8DuDn/9c0AnouwL59SKdt4u7YZR8T3XeTbn6tq2T8ArMDoO/IfAvhJFH1w9OsMAH/Kf2yPum8Ansbo07o0Rp8R3QpgOoA2ALvznxsrqG//A2ArgPcwGqw5EfXtGxh9afgegC35jxVR33dGv8pyv/F0WSJP8Aw6Ik8w7ESeYNiJPMGwE3mCYSfyBMNO5AmGncgT/w866iIlnq8zVgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(X[0, :].reshape(28, 28))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 311,
   "id": "breeding-george",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.3, random_state = 123)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 317,
   "id": "composed-shipping",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "# TODO: обучите случайный лес и kNN с дефолтными параметрами\n",
    "# Сравните их доли правильных ответов на тестовой выборке\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "completed-magic",
   "metadata": {},
   "source": [
    "#### 5.3 Тексты на примере бинарной классификации твитов из семинара 10\n",
    "\n",
    "Скачиваем куски датасета ([источник](http://study.mokoron.com/)): [положительные](https://www.dropbox.com/s/fnpq3z4bcnoktiv/positive.csv?dl=0), [отрицательные](https://www.dropbox.com/s/r6u59ljhhjdg6j0/negative.csv)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 324,
   "id": "rocky-fifteen",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "from sklearn.preprocessing import MaxAbsScaler\n",
    "\n",
    "# Предобработка из семинара 10\n",
    "positive = pd.read_csv('positive.csv', sep=';', usecols=[3], names=['text'])\n",
    "positive['label'] = 'positive'\n",
    "negative = pd.read_csv('negative.csv', sep=';', usecols=[3], names=['text'])\n",
    "negative['label'] = 'negative'\n",
    "df = positive.append(negative)\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(df.text, df.label, random_state=13)\n",
    "\n",
    "vec = CountVectorizer(ngram_range=(1, 1))\n",
    "bow = vec.fit_transform(X_train)\n",
    "bow_test = vec.transform(X_test)\n",
    "\n",
    "scaler = MaxAbsScaler()\n",
    "bow = scaler.fit_transform(bow)\n",
    "bow_test = scaler.transform(bow_test)\n",
    "\n",
    "X_train = bow\n",
    "X_test = bow_test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 337,
   "id": "expired-windows",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.tree import DecisionTreeClassifier\n",
    "\n",
    "# TODO: обучите случайный лес с числом деревьев 100 и макс. глубиной дерева 20 \n",
    "# и решающее дерево с макс. глубиной 20\n",
    "# Сравните их доли правильных ответов на тестовой выборке\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}