3.linear_regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "afIgLBWb689l"
   },
   "source": [
    "# Линейная регрессия\n",
    "__Суммарное количество баллов: 10__\n",
    "\n",
    "__Решение отправлять на `ml.course.practice@gmail.com`__\n",
    "\n",
    "__Тема письма: `[ML][MS][HW03] <ФИ>`, где вместо `<ФИ>` указаны фамилия и имя__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "bYKrc1lO689l"
   },
   "outputs": [],
   "source": [
    "from sklearn.datasets import make_blobs, make_moons\n",
    "from sklearn.model_selection import train_test_split\n",
    "import numpy as np\n",
    "import random\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib\n",
    "import copy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "VOGx85I5689m"
   },
   "outputs": [],
   "source": [
    "def read_data(path=\"boston.csv\"):\n",
    "    dataframe = np.genfromtxt(path, delimiter=\",\", skip_header=15)\n",
    "    np.random.seed(42)\n",
    "    np.random.shuffle(dataframe)\n",
    "    X = dataframe[:, :-1]\n",
    "    y = dataframe[:, -1]\n",
    "    return X, y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "id": "tte75xx7689m"
   },
   "outputs": [],
   "source": [
    "def generate_synthetic(size:int, dim=6, noise=0.1):\n",
    "    X = np.random.randn(size, dim)\n",
    "    w = np.random.randn(dim + 1)\n",
    "    noise = noise * np.random.randn(size)\n",
    "    y = X.dot(w[1:]) + w[0] + noise\n",
    "    return X, y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Aab8Oyui689m"
   },
   "source": [
    "### Задание 1 (1 балл)\n",
    "Для начала нужно понять, какую метрику для ошибки будем использовать. В нашем случае нам подойдет стандартная метрика MSE. Также чтобы оценить качество модели нам понадобится метрика $R^2$. Реализуйте обе эти метрики."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "RDUbABpp689m"
   },
   "outputs": [],
   "source": [
    "def mse(y_true:np.ndarray, y_predicted:np.ndarray):\n",
    "    return np.mean(np.square(y_true - y_predicted))\n",
    "\n",
    "def r2(y_true:np.ndarray, y_predicted:np.ndarray):\n",
    "    rss = sum(np.square(y_true - y_predicted))\n",
    "    tss = sum(np.square(y_true - np.mean(y_true)))\n",
    "    return 1 - rss / tss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Qqlg3KeW689m"
   },
   "source": [
    "### Задание 2 (3 балла)\n",
    "Теперь реализуем линейную регрессию при помощи явного решения задачи минимизации. \n",
    "\n",
    "#### Методы\n",
    "`fit(X, y)` - решает задачу минимизации $\\arg\\min_{w, b}\\sum ((w\\cdot x + b) - y)^2$. \n",
    "\n",
    "`predict(X)` - строит предсказание `y` для объектов из `X`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "id": "zTGLaghB689m"
   },
   "outputs": [],
   "source": [
    "class NormalLR:\n",
    "    def __init__(self):\n",
    "        self.w = None\n",
    "    \n",
    "    def fit(self, X:np.ndarray, y:np.ndarray):\n",
    "        X_ = np.hstack((np.ones((X.shape[0], 1)), X))\n",
    "        self.w = np.linalg.inv(X_.T @ X_) @ X_.T @ y\n",
    "    \n",
    "    def predict(self, X:np.ndarray) -> np.ndarray:\n",
    "        X_ = np.hstack((np.ones((X.shape[0], 1)), X))\n",
    "        return X_ @ self.w"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "id": "hgS8c_sM689m"
   },
   "outputs": [],
   "source": [
    "X, y = generate_synthetic(1024)\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, train_size=0.8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "id": "9DjhMRug689m"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MSE: 0.009077361058275618, R2: 0.9969602533378904\n"
     ]
    }
   ],
   "source": [
    "regr = NormalLR()\n",
    "regr.fit(X_train, y_train)\n",
    "y_pred = regr.predict(X_test)\n",
    "print(f\"MSE: {mse(y_test, y_pred)}, R2: {r2(y_test, y_pred)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rPaByJZN689m"
   },
   "source": [
    "### Задание 3 (2 балла)\n",
    "Теперь реализуем линейную регрессию с использованием градиентного спуска с learning rate `alpha` в течение `iterations` итераций. В задании необходимо использовать регуляризацию Лассо с коэффициентом `l`.\n",
    "\n",
    "#### Методы\n",
    "`fit(X, y)` - приближает решение задачи минимизации $\\arg\\min_{w, b}\\sum ((w\\cdot x + b) - y)^2$ при помощи градиентного спуска. \n",
    "\n",
    "\n",
    "`predict(X)` - строит предсказание `y` для объектов из `X`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {
    "id": "oBmZJrhL689m"
   },
   "outputs": [],
   "source": [
    "class GradientLR:\n",
    "    def __init__(self, alpha:float, iterations=10000, l=0.):\n",
    "        self.alpha = alpha\n",
    "        self.iterations = iterations\n",
    "        self.l = l\n",
    "        self.w = None\n",
    "    \n",
    "    def fit(self, X:np.ndarray, y:np.ndarray):\n",
    "        X = X.copy()\n",
    "        X = np.hstack((np.ones((X.shape[0], 1)), X))\n",
    "        def grad(w):\n",
    "            return 1 / X.shape[0] * X.T @ (X @ w - y) + self.l * np.sign(w)\n",
    "        \n",
    "        def gradient_descent(x, max_iter=10000, tol=1e-8):\n",
    "            start_grad = np.linalg.norm(grad(x)) ** 2\n",
    "            for _ in range(max_iter):\n",
    "                g = grad(x)\n",
    "                if np.linalg.norm(g) ** 2 / start_grad < tol:\n",
    "                    break\n",
    "                d = -g\n",
    "                x = x + self.alpha * d\n",
    "            return x\n",
    "        \n",
    "    \n",
    "        self.w = gradient_descent(np.array([1 for _ in range(X.shape[1])]), max_iter=self.iterations)\n",
    "\n",
    "\n",
    "    def predict(self, X:np.ndarray):\n",
    "        X = X.copy()\n",
    "        X = np.hstack((np.ones((X.shape[0], 1)), X))\n",
    "        return X @ self.w\n",
    "    \n",
    "    @property\n",
    "    def coef_(self):\n",
    "        return self.w[1:]\n",
    "    \n",
    "    @property\n",
    "    def intercept_(self):\n",
    "        return self.w[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {
    "id": "OPG6AR_b689m"
   },
   "outputs": [],
   "source": [
    "def build_plot(X_train, y_train, X_test, y_test):\n",
    "    xs = np.arange(0.0, 0.002, 0.00002)\n",
    "    errors = []\n",
    "    for x in xs:\n",
    "        regr = GradientLR(0.1, iterations=10000, l=x)\n",
    "        regr.fit(X_train, y_train)\n",
    "        errors.append(mse(y_test, regr.predict(X_test)))\n",
    "    plt.figure(figsize=(9, 4))\n",
    "    plt.xlim(xs[0], xs[-1])\n",
    "    plt.grid()\n",
    "    plt.plot(xs, errors)\n",
    "    plt.xlabel(\"Коэффициент регуляризации\")\n",
    "    plt.ylabel(\"MSE\")\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {
    "id": "6D5_9sSc689m"
   },
   "outputs": [],
   "source": [
    "X, y = generate_synthetic(1024)\n",
    "X, X_val, y, y_val = train_test_split(X, y, train_size=0.9, shuffle=True)\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, train_size=0.8, shuffle=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {
    "id": "OzRIkE7P689m"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkwAAAEKCAYAAADgjQovAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3Xl8VfW1///XSkLCGJAQICTMowiIgoBz1IqIAzhV1FZb7bW1pb23t7ZXb6vXq+391fb29ttqa2tbxzrPiODQahyQKcyDosiQkRkCATKv3x9nRw8xyWHIyUly3s/HIw/28Nmfs1Y2IYu9P/uzzd0RERERkYYlxDoAERERkZZOBZOIiIhIBCqYRERERCJQwSQiIiISgQomERERkQhUMImIiIhEoIJJREREJAIVTCIiIiIRqGASERERiSAp1gG0Nt26dfMhQ4bEOoyY2L9/P506dYp1GDETz/kr9/jMHeI7/3jOHeIn/yVLluxw9/RI7VQwHaFevXqRm5sb6zBiIicnh+zs7FiHETPxnL9yz451GDETz/nHc+4QP/mb2ebDaadbciIiIiIRqGASERERiUAFk4iIiEgEKphEREREIlDBJCIiIhKBCiYRERGRCFQwiYiIiESgeZhEREQk7pRVVvPC0oLDbq+CSUREROLG7v0VPL5gM49+uImd+ysO+7io3pIzsylmts7M1pvZbfXsTzGzZ4L9C81sQLA9zczeMbNSM7u/zjHjzGxVcMzvzcyC7XeZWaGZLQ++pgbbB5jZwbDtf4rUl4iIiLQt+bsOcNesNZz2y7f5v7c+YXRWV578l4mHfXzUrjCZWSLwB+B8oABYbGaz3H1tWLObgN3uPsTMZgD3AlcDZcAdwKjgK9wDwM3AAmAOMAWYG+z7rbv/bz3hfObuY+vZ3lhfIiIi0sqtLizhz+9tYM6qYhIMLj0xk5vPGsTw3l2OqJ9o3pKbAKx39w0AZvY0MA0IL5imAXcFy88D95uZuft+4AMzO+Qtt2aWAaS6+/xg/TFgOkdR5DRlXyIiItJyuDvvfrKdB9/bwIef7aRzShI3nTGQb54+gIyuHY6qz2gWTJlAfth6AVD32tfnbdy9ysxKgDRgRyN9ho/QKgi21ZppZtcDucCP3H13sH2gmS0D9gI/c/f3D6MvERERaUUqqmp4dUURf3l/Ax9v2Uev1BRuv3AE10zsR2r7dsfUdzQLpvrGA/lRtDnc9g8A9wTr9wC/AW4EioF+7r7TzMYBL5vZCUfy2WZ2M6Fbd6Snp5OTk9NIiG1XaWlp3OYO8Z2/cs+JdRgxE8/5x3Pu0LryP1Dp5BRU8tamKnaXO1mdjW+NTmZSRiJJns/SBfmRO4kgmgVTAdA3bD0LKGqgTYGZJQFdgV0R+syqr09331q70cz+AswOtpcD5cHyEjP7DBjWWF91ufuDwIMAw4cP9+zs7EZCbLtycnKI19whvvNX7tmxDiNm4jn/eM4dWkf+xSUHeXjeJp5cmEdpeRWnDU7j5rMGcfawdJr6Oa5oFkyLgaFmNhAoBGYA19ZpMwu4AZgPXAm87e4NXmFy92Iz22dmk4CFwPXAfRAak+TuxUHTy4DVwfZ0YJe7V5vZIGAosMHddzXUl4iIiLRca4v28tf3NzBrRREOTB2dwbfPGsSozK5R+8yoFUzBmKSZwBtAIvCQu68xs7uBXHefBfwNeNzM1hO6sjSj9ngz2wSkAslmNh2YHDxhdwvwCNCB0ADt2kHavzKzsYRuq20Cvh1sPwu428yqgGrgO+5eexWrob5ERESkBXF3Pli/gwff28D7n+6gY3Ii1586gG+ePoC+3TtG/fOjOnGlu88h9Lh++LY7w5bLgKsaOHZAA9tz+fJUA7j71xto/wLwwpH0JSIiIi1DRVUNs1cW8eB7oYHc6V1S+MmU4Vw3oT9dOx7bQO4joZm+RUREpMXZW1bJUwvzeHjeJrbsLWNoz8786ooxTDupDylJic0ejwomERERaTEKdh/g4XmbeGZxPqXlVZw+JI1fXjE6KgO5j4QKJhEREYm5VQUl/OX9Dby2KvT81sVjMviXM6M7kPtIqGASERGRmKipcd5Zt42/vL+BBRt20TkliRtPH8A3Th9IZrejm5E7WlQwiYiISLMqq6zmpWWF/PX9DXy2fT8ZXdvz06nHc/WEvsc8I3e0qGASERGRZrGztJy/L8jj8QWb2FFawQl9UvndjLFMHZ1Bu8SEWIfXKBVMIiIiElWfbS/lbx9s5IUlBZRX1XDuiJ5868yBnDooLaYDuY+ECiYRERFpcu7Owo27+Ov7G/jHR9tITkrg8pMyuemMgQzt1SXW4R0xFUwiIiLSZCqra5izqpi/vr+RVYUldO+UzA/OG8rXJ/UnvUtKrMM7aiqYRERE5JjtLavk6UV5PDJvE0UlZQxK78QvLhvFFSdn0b5d80802dRUMImIiMhRy99VO9FkHvsrqjl1UBr3TB/FOcN7kpDQOsYnHQ4VTCIiInLElubt5m/vb2Tu6mISzLh4TAbfakETTTY1FUwiIiJyWKprnDfXbOGvH2xkyebdpLZP4uazBnPDaf3J6NqyJppsaiqYREREpFGl5VU8uzifhz/cSP6ug/Tr3pG7LhnJVeP70iklPkqJ+MhSREREjljhnoM8+uEmnlqYx77yKk4ZcBw/nTqS80f2IrENjU86HCqYRERE5BDL8/fwx+VlLHnzHQCmjs7gpjMGMrZvtxhHFjsqmERERORL45M6JMGNpw/khtMGkHVcx1iHF3MqmEREROLYvrJKns0t4OF5GynYfZC+3Ttw58UjySjbxIVfGRnr8FoMFUwiIiJxKH/XAR75cBPPLM6nNBif9LOLjuf8kb1JTDBycjbHOsQWRQWTiIhInHD30PxJH2zk9dVbSDDjojGh8UljsuJ3fNLhUMEkIiLSxlVW1zB39Rb+9sFGVuTvIbV9Ev9y1iC+cdqANj9/UlNJiGbnZjbFzNaZ2Xozu62e/Slm9kywf6GZDQi2p5nZO2ZWamb31zlmnJmtCo75vZlZsP0uMys0s+XB19Rg+/lmtiQ4ZomZnRvWV04QX+0xPaP5/RAREWlOJQcq+dO7n3HWr97hB08tY+/BSu6ZdgIL/vM8br/weBVLRyBqV5jMLBH4A3A+UAAsNrNZ7r42rNlNwG53H2JmM4B7gauBMuAOYFTwFe4B4GZgATAHmALMDfb91t3/t077HcAl7l5kZqOAN4DMsP3XuXvusWUrIiLScmzYXsrD8zbx/JICDlZWc9rgNH7eBt/v1pyieUtuArDe3TcAmNnTwDQgvGCaBtwVLD8P3G9m5u77gQ/MbEh4h2aWAaS6+/xg/TFgOl8UTF/i7svCVtcA7c0sxd3LjyU5ERGRlsTd+fCznfztg428/fE2khMTuHRsH248fSAj+6TGOrxWL5oFUyaQH7ZeAExsqI27V5lZCZBG6KpQQ30W1Okz/GrRTDO7HsgFfuTuu+scfwWwrE6x9LCZVQMvAD93d4+YmYiISAtRVlnNrOVFPDRvIx9v2Udap2T+9byhfG1Sf9K7pMQ6vDYjmgVTfdf86hYjh9PmcNs/ANwTrN8D/Aa48fMDzU4gdMtvctix17l7oZl1IVQwfR147EsfanYzoduApKenk5OT00iIbVdpaWnc5g7xnb9yz4l1GDETz/m39Nz3lNfwdl4V7+RXsq8CsjobN41KZmJGEsmJRaxZUnRM/bf0/JtbNAumAqBv2HoWUPfs1bYpMLMkoCuwK0KfWfX16e5bazea2V+A2WHrWcBLwPXu/lntdncvDP7cZ2ZPErqN+KWCyd0fBB4EGD58uGdnZzcSYtuVk5NDvOYO8Z2/cs+OdRgxE8/5t9TcVxeW8NC8jby6ooiqGue8ET258fSBnDo4jeA5qCbRUvOPlWgWTIuBoWY2ECgEZgDX1mkzC7gBmA9cCbzd2C0xdy82s31mNglYCFwP3Aeh8U3uXhw0vQxYHWzvBrwG3O7u82r7Cgq0bu6+w8zaARcD/zjGnEVERJpcdY3z1tqtPDRvI4s27qJjciLXTujHDacNYFB651iHFxeiVjAFY5JmEnoqLRF4yN3XmNndQK67zwL+BjxuZusJXVmaUXu8mW0CUoFkM5sOTA6esLsFeAToQGiwd+2A71+Z2VhCt+Q2Ad8Ots8EhgB3mNkdwbbJwH7gjaBYSiRULP2lqb8PIiIiR2tvWSXPLs7n0fmbyN91kMxuHfjp1OP56il96dqhXazDiytRnbjS3ecQevQ/fNudYctlwFUNHDugge25fHmqAdz96w20/znw8wZCHNfAdhERkZjZuGM/j364iedy89lfUc0pA47jPy88nvNH9iIpMapTKEoDNNO3iIhIC+DuzFu/k4fnbeTtddtISjAuOTE0LcCozK6xDi/uqWASERGJoYMV1by0rJBHPtzIJ1tLSeuUzPfPHcrXJvWjZ5f2sQ5PAiqYREREYqBoz0EeX7CZpxblsedAJSf0SeV/rzqRi8dk0L5dYqzDkzpUMImIiDQTd2dp3m4emreJ11dvwd2ZPLI33zx9ABMGdm/SaQGkaalgEhERibKKqhpeW1XEw/M2sbKghNT2Sdx0xkC+Pqk/fbt3jHV4chhUMImIiETJ9n3lPLFwM08szGP7vnIGp3finumjuOLkTDom61dwa6KzJSIi0sRWFZTw8LyNzF5ZTEV1DecMT+cbpw/kzCE9SEjQbbfWSAWTiIhIE6isruGNNVt4ZN4mcjfvplNyItdO7Mf1p/bXbNxtgAomERGRY7BrfwVPLcrj8fmb2bK3jP5pHbnj4pFcNT6L1PaajbutUMEkIiJyFNYUlfDoh5t4eXkRFVU1nDm0B7+4bBTZw3uSqNtubY4KJhERkcNUVV3Dm2u38si8TSzatIsO7RK5clwW3zxtAEN7dYl1eBJFKphEREQi2LW/gqcX5/H3+ZspKimjb/fgJbjj+9K1o267xQMVTCIiIg2oe9vttMFp/Pe0UZw7Qrfd4o0KJhERkTCV1TW8uWYrv1t4kE9e/4AO7RK5alwWN5w2gGG67Ra3VDCJiIgAO0vLeXpx/udPu6V3MH520fFcNU633UQFk4iIxLnVhSU8PG8Tr64M3XY7Y0gP7pk+isStazn3zEGxDk9aCBVMIiISdyqqapi7uphHP9zE0rw9dExO5OrxfbnhtP4M6Rm67Zaz7aMYRyktiQomERGJG9v2lfHkwjyeXJjHtn3lDEjryJ0Xj+RKTTIpEahgEhGRNs3dWZq3h8fmb2LOqmIqq53s4ence9oAzh6arne7yWFRwSQiIm1SWWU1r64o4rH5m1lVWEKXlCS+Nqk/1586gIE9OsU6PGllVDCJiEibUrD7AE8szOPpRXnsPlDJ0J6duWf6KC4/KZNOKfq1J0cnIZqdm9kUM1tnZuvN7LZ69qeY2TPB/oVmNiDYnmZm75hZqZndX+eYcWa2Kjjm92Zmwfa7zKzQzJYHX1PDjrk9aL/OzC443PhERKR1cHc+XL+Dmx/L5axfvcOf3/2MCQO788S3JvLmD8/i65P6q1iSYxK1vz1mlgj8ATgfKAAWm9ksd18b1uwmYLe7DzGzGcC9wNVAGXAHMCr4CvcAcDOwAJgDTAHmBvt+6+7/WyeOkcAM4ASgD/APMxsW7I4Un4iItGCl5VW8tLSAR+dvZv22Uo7r2I5vnz2Y6yb2I+u4jrEOT9qQaJbbE4D17r4BwMyeBqYB4QXJNOCuYPl54H4zM3ffD3xgZkPCOzSzDCDV3ecH648B0/miYKrPNOBpdy8HNprZ+iA2DiM+ERFpgdZvK+XvCzbz/JICSsurGJ3ZlV9fOYZLTuxD+3aJsQ5P2qBoFkyZQH7YegEwsaE27l5lZiVAGrCjkT4L6vSZGbY+08yuB3KBH7n77mD/ggaOiRSfiIi0ENU1zj8/2spj8zfzwfodJCcmcNGYDK4/tT9j+3YjGKEhEhXRLJjq+5vrR9HmcNs/ANwTrN8D/Aa4sZFj6hu/Ve9nm9nNhG4Dkp6eTk5OTiMhtl2lpaVxmzvEd/7KPSfWYcRMS8h/X4XzbkEl7+RVsbPM6d7euHxoO7Kz2pGasoeSDXt4d0PTf25LyD2W4j3/uqJZMBUAfcPWs4CiBtoUmFkS0BXYFaHPrPr6dPettRvN7C/A7MOII1J8BH0/CDwIMHz4cM/Ozm4kxLYrJyeHeM0d4jt/5Z4d6zBiJpb5L88PzZ00e2UxFVU1nDY4jetP7c9Xju9FUmJUn1kCdO7jPf+6olkwLQaGmtlAoJDQwOtr67SZBdwAzAeuBN529wavMLl7sZntM7NJwELgeuA+CI1vcvfioOllwOqwz3jSzP6P0KDvocAiQleeIsUnIiLNqHbupMcXbGZlQQmdgleWXH9qf4b26hLr8CSORa1gCsYkzQTeABKBh9x9jZndDeS6+yzgb8DjwUDsXYSKFgDMbBOQCiSb2XRgcvAE2y3AI0AHQoO9awd8/8rMxhK6rbYJ+HYQxxoze5bQYO4q4HvuXh18xpfii9K3Q0REGpG/6wB/X7iZZxbns+dAJUN6dubuaSdw2UmZdNErS6QFiOqkFO4+h9Cj/+Hb7gxbLgOuauDYAQ1sz+XLUw3g7l9vJI5fAL84nPhERKR51NQ47366ncfnb+adddtIMOP843tx/an9OXVwmgZxS4uiWbxERKRZ7TlQwXO5Bfx94WY27zxAj84pzDxnCNdO7EdG1w6xDk+kXiqYRESkWawqKOGx+ZuYtaKI8qoaThlwHD+aPJwpJ/QmOSn6g7hFjoUKJhERiZqyympeW1nMYws2syJ/Dx2TE7liXBZfm9ifkX1SYx2eyGFTwSQiIk0ub+cBnli0mWcX57P7QCWD0ztx1yUjuXxcFqkaxC2tkAomERFpEtU1zrufbOPx+ZvJ+WQ7CWZMHtmLr03qz2kaxC2tnAomERE5JjtLy3k2t4AnFm6mYPdB0ruk8P1zh3LthH707to+1uGJNAkVTCIicsTcnaV5e/j7gs28trKYiuoaJg3qzm0XjuCCE3rTrhlm4hZpTiqYRETksO0vr+KV5aGZuD8q3kuXlCSumdCXr03STNzStqlgEhGRiD7duo+/L9jMC0sLKS2v4viMVH5x2Simj82kU4p+lUjbp7/lIiJSr4qqGhYUV/HHP89n0cZdJCcmcNGYDL42qT8n9+umQdwSV1QwiYjIIQp2H+CpRXk8szifHaUV9O2ewH9MGcFXx2eR1jkl1uGJxIQKJhERobrGee+T7TyxcDNvf7wNgHNH9GRMxxJmXnEOCQm6miTxTQWTiEgc21FazrO5+Ty5MI+C3Qfp0TmF72YP4ZqJ/cjs1oGcnBwVSyKoYBIRiTvuzqKNu3hiYR5zVxdTWe2fTwkweaTe6yZSHxVMIiJxYm9ZJS8uKeCJhXl8uq2ULu2TuG5if742qR9DempKAJHGqGASEWnjVhWU8MTCzbyyvIiDldWcmNWVX105hkvG9KFDcmKswxNpFVQwiYi0QQcqqnh1RRFPLMxjZUEJHdolMm1sH66b2J/RWV1jHZ5Iq6OCSUSkDVm3ZR9PLtzMi0sL2VdexbBenbl72glMPymT1PbtYh2eSKulgklEpJUrq6xm7upinliQR+7m3SQnJnDh6N58bVJ/xvc/ThNMijQBFUwiIq3Uhu2lPLkwj+eXFrDnQCUDe3Tip1OP54pxWXTvlBzr8ETaFBVMIiKtSEVVDW+u3cITC/KYv2EnSQnG5BN6cd3E/pw6KE1zJolESaOTbZjZ18KWT6+zb2akzs1sipmtM7P1ZnZbPftTzOyZYP9CMxsQbE8zs3fMrNTM7q9zzDgzWxUc83urc63ZzG41MzezHsH6j81sefC12syqzax7sG9T0NdyM8uNlI+ISKzk7TzAva9/zGm//Cczn1xG/u4D/PiC4Xx4+7n88bpxnD6kh4olkSiKdIXp34G/B8v3ASeH7bsRuP9LRwTMLBH4A3A+UAAsNrNZ7r42rNlNwG53H2JmM4B7gauBMuAOYFTwFe4B4GZgATAHmALMDT6zb/B5ebWN3f3XwK+D/ZcAP3T3XWH9nePuOxr/NoiINL/K6hr++dFWnliYx/uf7iDB4Lzje3HdxH6cNTRdBZJIM4pUMFkDy/Wt1zUBWO/uGwDM7GlgGhBeME0D7gqWnwfuNzNz9/3AB2Y25JAPNMsAUt19frD+GDCdoGACfgv8BHilgZiuAZ6KELeISEwV7D7A04vyeSY3n+37ysno2p4ffmUYV5/Sl95d28c6PJG4FKlg8gaW61uvKxPID1svACY21Mbdq8ysBEgDGrrikxn0E95nJoCZXQoUuvuK+p4IMbOOhK5Ghd9KdOBNM3Pgz+7+YIScRESioqq6hrc/3saTi/J495PtAGQPS+e6if3JHp5OUqJeVyISS5EKphFmtpLQ1aTBwTLB+qAIx9Z3BapukXU4bSK2D4qhnwKTGzn2EmBendtxp7t7kZn1BN4ys4/d/b0vfajZzYRuA5Kenk5OTk4jH9N2lZaWxm3uEN/5K/ecqPW/82AN7xVU8V5BFbvLnW4pxiWD2nF2VhJpHQ7Ato/4YNtHUfv8SHTuc2IdRszEe/51RSqYjj+GvguAvmHrWUBRA20KzCwJ6ArsomEFQT91+xwMDARqry5lAUvNbIK7bwnazqDO7Th3Lwr+3GZmLxG6jfilgim48vQgwPDhwz07O7uRENuunJwc4jV3iO/8lXt2k/ZZVV3DO+u289SiPHLWbcOBs4elc82Efpw3omeLupqkc58d6zBiJt7zr6vRgsndN4evm1kacBaQ5+5LIvS9GBhqZgOBQkIFy7V12swCbgDmA1cCb7t7g1eY3L3YzPaZ2SRgIXA9cJ+7rwJ6hsW5CRhfO5jbzLoCZwPhT/11AhLcfV+wPBm4O0JOIiJHrXDPQZ5ZnM+zi/PZsreMnl1S+N45Q/jq+L707d4x1uGJSCMaLZjMbDZwm7uvDgZcLwVyCd2ee9Dd/19DxwZjkmYCbwCJwEPuvsbM7gZy3X0W8DfgcTNbT+jK0oywz94EpALJZjYdmBw8YXcL8AjQgdBg77lEdhnwZjCYvFYv4KXgilQS8KS7v34YfYmIHLbasUlPLcojJxibdNbQdP572gmcO6In7VrQ1SQRaVikW3ID3X11sPxN4C13v97MugDzgAYLJgB3n0Po0f/wbXeGLZcBVzVw7IAGtufy5akGGj3W3R8hVGSFb9sAnNhYPyIiR6tg9wGeXRx60m3r3nJ6dklhpq4mibRakQqmyrDl84C/AAS3sWqiFpWISCsUmjdpG08vPvRJt3um9ePcFjY2SUSOTKSCKd/Mvk9osPXJwOsAZtYB0GuvRUSA/F0HeHpxHs/lFrBtXzm9U9vz/XOH8tXxWWQdp6tJIm1BpILpJkIDob8CXO3ue4Ltk4CHoxmYiEhLVlFVw1trt/L04i9m4T53RE+umdCPs4dp3iSRtibSU3LbgO/Us/0d4J1oBSUi0lJt2F7KM4vzeX5JATv3V5DZrQP/fv4wrhqfRUbXDrEOT0SiJNJTcrMa2+/ulzZtOCIiLU9ZZTWvr97CU4vyWLhxF4kJxleOD11NOnNoOol6p5tImxfpltyphF5d8hSheY/0r4KIxI1Ptu7jiY/K+bf3/smeA5X0696RH18wnKvGZdEzVe90E4knkQqm3sD5hF5aey3wGvCUu6+JdmAiIrFwoKKK2SuKeWpxHsvy9pBocOHoDK6Z0I9TB6WRoKtJInEp0himakJPxr1uZimECqccM7vb3e9rjgBFRKLN3VlVWMJTi/J5dUURpeVVDE7vxM8uOp5eBzdzyeSTYx2iiMRYpCtMBIXSRYSKpQHA74EXoxuWiEj0lRys5JXlhTy1KJ+PivfSvl0CF43uwzUT+jKu/3GYGTk5ebEOU0RagEiDvh8lNKv2XOC/w2b9FhFpldydRRt38czifF5bVUx5VQ0n9EnlnumjuPTEPnTtoCnmROTLIl1h+jqwHxgG/CB47xqEBn+7u6dGMTYRkSazfV85Lywt4NnF+WzYsZ/OKUlcOS6Layb0Y1Rm11iHJyItXKQxTJp5TURareoa571Pt/PMonz+8dFWqmqcUwYcx3fPGcLU0b3pmBxxVIKICHAYY5hERFqb/F0HeC43n+eWFFBcUkZap2RuPGMgXx3flyE9O8c6PBFphVQwiUibUF5VzZtrtvJsbj4frN8BwFlD07nz4pGcd3wvkpN0wVxEjp4KJhFp1T7espdnFufz0rJC9hyoJLNbB/7tvGFcOT6LzG56VYmINA0VTCLS6uwrq+TVFcU8k5vPivw9JCcmcP4Jvbh6fF9OH9JDryoRkSangklEWgV3Z/Gm3TyzOJ85q4o5WFnN8F5duOPikVx2UibdOyXHOkQRacNUMIlIi7ZtbxkvLC3kudwvpgOYflImV5/SlxOzuhI23YmISNSoYBKRFqeyuoZ3Pt7Gs7n5vLNuO9U1zvj+x3FL9mAuGpOh6QBEpNnpXx0RaTHWbyvluSX5vLCkkB2l5fTonMK3zgxNBzA4XdMBiEjsqGASkZgqLa9izspins3NJ3fzbhITjHOG9+TqU/qSPTyddomaDkBEYi+qBZOZTQF+ByQCf3X3X9bZnwI8BowDdgJXu/smM0sDngdOAR5x95lhx4wDHgE6AHOAf3V3D9t/K/BrIN3dd5hZNvAKsDFo8qK733048YlIdLg7Szbv5tncfGavLOZARTWD0jtx24UjuPzkTHp2aR/rEEVEDhG1gsnMEoE/AOcDBcBiM5vl7mvDmt0E7Hb3IWY2A7gXuBooA+4g9OLfUXW6fgC4GVhAqGCaQujlwJhZ3+Dz6r5e/H13v/go4hORJlR3AHfH5EQuHpPB1af05eR+x2kAt4i0WNG8wjQBWO/uGwDM7GlgGhBekEwD7gqWnwfuNzNz9/3AB2Y2JLxDM8sAUt19frD+GDCdoGACfgv8hNAVpaaIT0SOUUVVDW9/vI3nl3wxgHvCgO58J3swF43OoFOKRgaISMsXzX+pMoH8sPUCYGJDbdy9ysxKgDRgRyN9FtTpMxPAzC4FCt19RT3/Sz3VzFYARcCt7r7mMOMTkaO0bss+nssNzcBgax82AAAecElEQVS9c38FPbukcPNZg7hqXBaDNIBbRFqZaBZM9V1b96NoE7G9mXUEfgpMrmf/UqC/u5ea2VTgZWDokXy2md1M6DYg6enp5OTkNBJi21VaWhq3uUN853+4ue+vdBYWV/F+YRUbS2pINDipZyLXj0hhVFoiiQlbyFuz5Uv3zFuyeD7vEN/5x3PuoPzrimbBVAD0DVvPInSFp742BWaWBHQFdkXoM6uePgcDA4Haq0tZwFIzm+DuW2obu/scM/ujmfU4zPhqj3sQeBBg+PDhnp2d3UiIbVdOTg7xmjvEd/6N5V5T48z7bAfP5Rbw+potVFTVMKJ3F+44sy/Tx/YhrXNK8wbbxOL5vEN85x/PuYPyryuaBdNiYKiZDQQKgRnAtXXazAJuAOYDVwJvhz/xVpe7F5vZPjObBCwErgfuc/dVQM/adma2CRgfPCXXG9jq7m5mE4AEQk/k7TmM+ESkAXk7D/D8knxeWFpI4Z6DdO3Qjhmn9OWr4/tyQp9UDeAWkTYlagVTMCZpJvAGocf2H3L3NWZ2N5Dr7rOAvwGPm9l6QleWZtQeHxQ9qUCymU0HJgdPsN3CF9MKzOWLAd8NuRK4xcyqgIPAjKAoqze+pslepG06UFHFnFVbeC43n4Ubd2EGZw5N57YLR3D+yF60b5cY6xBFRKIiqo+nuPscQo/+h2+7M2y5DLiqgWMHNLA9ly9PNdDgse5+P3D/4cYnIodyd9btqmbO8yt4bWUx+yuqGZDWkR9fMJzLT84ko2uHWIcoIhJ1ep5XROpVtOcgLy4t4PklBWzaWUan5GIuGpPBVeP7Mr6/5kwSkfiigklEPldWWc0ba7bw/JICPli/A3eYNKg75/ep4odXnaOX3opI3NK/fiJxzt1ZmreH55cUMHtFEfvKq8js1oHvnzuUK0/Ool9aR3JyclQsiUhc07+AInFqS0kZLy4L3XLbsH0/7dslMHVUBleOz2LSwDQSEnTLTUSklgomkThSVlnNm2u3hm65fbqdGocJA7rz7bMGMXV0Bl3at4t1iCIiLZIKJpE27pBbbiuL2FcWuuU285whXH5yFgN6dIp1iCIiLZ4KJpE2qrjkIC8uLeSFJQVs2LGfDu0SuXBUb64Yl8Wpg3TLTUTkSKhgEmlDDlZ88ZTbvM9CT7lNGNid72QPZuroDDqn6EdeRORo6F9PkVbO3Vm0cRcvLC1gzqotlJZX0bd7B35w7lCuCJ5yExGRY6OCSaSVytt5gBeWFvDisgLydx2kU3IiU0dncMW4LCYM6K5bbiIiTUgFk0grsq+skjmrinlhSSGLNoXe5Xba4DR++JVhTBnVW3MliYhEif51FWnhqmucD9bv4MWlBbyxZgtllTUM6tGJH18wnMtOyqRPN73LTUQk2lQwibRQn2zdxwtLC3h5WSFb95bTtUM7rhyXxRUnZzG2bze9y01EpBmpYBJpQXaWljNrRREvLC1gdeFekhKM7OHp3HVJFuce35OUpMRYhygiEpdUMInEWHlVNW9/tI0XlhaSs24bVTXOqMxU/uuSkVxyYh96dE6JdYgiInFPBZNIDIRm397NC0sLmb2iiL1lVfTsksJNZwzk8pOzGN67S6xDFBGRMCqYRJpR3s4DvLSskJeWFbBp5wHat0tgygm9ufzkLE4f0oNETQUgItIiqWASibKSg6GpAF5cWsDiTbsxg0kD0/juOUM0+7aISCuhf6lFoqCyuoZ3123nxWUF/OOjbVRU1TA4PTQVwPSTMsnUVAAiIq2KCiaRJuLurCgo4aWlBby6sphd+yvo3imZayf04/KTMxmd2VVTAYiItFIqmESOUf6uA7yyvJAXlxWyYft+kpMSOH9kLy4/KZOzhqXTLjEh1iGKiMgxUsEkchRKDlYyd1UxLy4rZNHGXQBMGNidm88cxIWjM+jaoV2MIxQRkaYU1YLJzKYAvwMSgb+6+y/r7E8BHgPGATuBq919k5mlAc8DpwCPuPvMsGPGAY8AHYA5wL+6u4ftvxX4NZDu7jvM7DrgP4LdpcAt7r4iaLsJ2AdUA1XuPr5pvwPSllTVOG+t3cpLYeOSBqV34tbJw5g2NpO+3TvGOkQREYmSqBVMZpYI/AE4HygAFpvZLHdfG9bsJmC3uw8xsxnAvcDVQBlwBzAq+Ar3AHAzsIBQwTQFmBt8Zt/g8/LC2m8Eznb33WZ2IfAgMDFs/znuvqMJUpY2yN1Zlr+Hl5YW8tKSA5RW5pIWjEu67KRMxmRpXJKISDyI5hWmCcB6d98AYGZPA9OA8IJpGnBXsPw8cL+ZmbvvBz4wsyHhHZpZBpDq7vOD9ceA6QQFE/Bb4CfAK7XHuPuHYV0sALKaJDtp0zbu2M/Lywp5eXkhm3ceICUpgbE9Evn2lJM4c6jGJYmIxJtoFkyZQH7YegGHXtk5pI27V5lZCZAGNHTFJzPoJ7zPTAAzuxQodPcVjfyP/ya+KK4AHHjTzBz4s7s/WN9BZnYzoatapKenk5OT01D/bVppaWmbzn1vhbOouIr5RVV8VlKDAcenJXDTqGTG906iumw/CVs+Yt6Wj2IdarNr6+e+MfGcO8R3/vGcOyj/uqJZMNVXtfhRtInY3sw6Aj8FJjd4oNk5hAqmM8I2n+7uRWbWE3jLzD529/e+9AGhQupBgOHDh3t2dnYjIbZdOTk5tLXcD1ZU84+PtvLyskLe/WQ7VTXOiN5duP20TKaNzaR31/aft22L+R8u5Z4d6zBiJp7zj+fcQfnXFc2CqQDoG7aeBRQ10KbAzJKArsCuCH2G31Kr7XMwMBCovbqUBSw1swnuvsXMxgB/BS509521B7t7UfDnNjN7idBtxC8VTNK2VNc48z/byUvLCnljzRZKy6vondqem84cyPSxmRyfkRrrEEVEpIWJZsG0GBhqZgOBQmAGcG2dNrOAG4D5wJXA2+FPvNXl7sVmts/MJgELgeuB+9x9FdCztl3w9Nv44Cm5fsCLwNfd/ZOwNp2ABHffFyxPBu4+1qSlZXJ31hTt5eVlhcxaUcS2feV0SUniwlG9uezkTCYOTNN73EREpEFRK5iCMUkzgTcITSvwkLuvMbO7gVx3nwX8DXjczNYTurI0o/b4oOhJBZLNbDowOXjC7ha+mFZgLoeOSarPnYTGRf0xuPpUO31AL+ClYFsS8KS7v94UuUvLUTup5MvLi1i/rZR2iUb28J5cdlIm547oSft2ibEOUUREWoGozsPk7nMIPfofvu3OsOUy4KoGjh3QwPZcvjzVQIPHuvu3gG/V02YDcGJj/UjrtHt/BbNXFfPKskJyN+8G4JQBx/Hz6aO4eEwG3TomxzhCERFpbTTTt7QJByuqeeujrbwSNnh7aM/O/PiC4Vx6Yh9NKikiIsdEBZO0WlXVNcz7bCevBIO391dU0zu1PTeeMZBpY/swMiNVk0qKiEiTUMEkrYq7szx/D68sL2L2yiJ2lFbQpX0SF4/pw7ST+mjwtoiIRIUKJmkV1m8rZdbyQl5ZUcTmnQdITkrgvBE9mTY2k3NGpJOSpMHbIiISPSqYpMUqLjnI7BXFvLy8kDVFe0kwOHVwGt87ZwhTRvUmtX27WIcoIiJxQgWTtCh7DlQwd/UWXlleyMKNu3CHE7O6csfFI7lkTAY9U9tH7kRERKSJqWCSmKt9wm3W8tATbpXVzqAenfjX84YybWwmA3t0inWIIiIS51QwSUxUVtfw/qfbmbW8iDfXbuVARTW9UlP4xmkDmDY2kxP66Ak3ERFpOVQwSbOpqXEWb9rFrBVFzFlVzO4DlXTt0I5pY/twyYl6wk1ERFouFUwSVbXvcHtleSGzVxZTXFJGh3aJnD+yF5ee2IezhqWTnJQQ6zBFREQapYJJouKz7aXMWl7EqyuK2LBjP+0SjbOHpXPbhSP4yvG96JSiv3oiItJ66LeWNJnCPQeZvaKIV5YXsbZ4L2YwcWB3/uWsQVw4qrfe4SYiIq2WCiY5JjtKy5mzqphXVxSxeFPoRbcn9u3Gzy46novH9KF3V00DICIirZ8KJjliJQcreWPNFl5dUcS89TuocRjaszO3Th7GJSf2oX+apgEQEZG2RQWTHJYDFVUsKK7iicdyeXfddiqqa+jXvSO3ZA/m0hMzGd67S6xDFBERiRoVTNKg8qpq3l23nVdXFvOPtVs5WFlNr9Q9fG1Sfy4d24cTs7pqriQREYkLKpjkEJXVNXz42U5eXVHEG2u2sK+siuM6tuPykzPp59v4l+nnkqC5kkREJM6oYBKqa5xFG3cxe2URc1dvYdf+CrqkJHHBqN5cPCaD04f0oF1iAjk5OSqWREQkLqlgilPuztK8PbwazLq9bV85Hdol8pWRvbhkTAZnDUunfbvEWIcpIiLSIqhgiiPuzurCvcxeWcTslcUU7jlIclIC5wxP55IT+3DuiJ50TNZfCRERkbr027GNc3c+3rLv8yJp884DJCUYZw7twY8mD+P8kb3o0r5drMMUERFp0aJaMJnZFOB3QCLwV3f/ZZ39KcBjwDhgJ3C1u28yszTgeeAU4BF3nxl2zDjgEaADMAf4V3f3sP23Ar8G0t19h4Ue4/odMBU4AHzD3ZcGbW8AfhYc+nN3f7SJvwUxs37bPl5dUczslUV8tn0/iQnGaYPT+G72YC44QbNui4iIHImoFUxmlgj8ATgfKAAWm9ksd18b1uwmYLe7DzGzGcC9wNVAGXAHMCr4CvcAcDOwgFDBNAWYG3xm3+Dz8sLaXwgMDb4mBsdPNLPuwH8B4wEHlgTx7W6a70Dz27hjP7NXFPHaqmI+3rLv81eTfPP0gVw4qjdpnVNiHaKIiEirFM0rTBOA9e6+AcDMngamAeEF0zTgrmD5eeB+MzN33w98YGZDwjs0swwg1d3nB+uPAdMJCibgt8BPgFfqfMZjwVWoBWbWLegnG3jL3XcFfb1FqPh6qglybzZ5Ow8we1URr60sZk3RXgDG9z+Ouy4ZydTRGfRM1atJREREjlU0C6ZMID9svYDQFZ5627h7lZmVAGnAjkb6LKjTZyaAmV0KFLr7ijqTKdYXR2Yj27/EzG4mdFWL9PR0cnJyGgiveew4WMPiLdUsKq5i494aAAZ1TeCaEcmM75VIWocKqNzM2qWbD6lOj1VpaWnMc4+leM5fuefEOoyYief84zl3UP51RbNgqm/CHj+KNhHbm1lH4KfA5COI47A/290fBB4EGD58uGdnZzcSYnQU7TnInFXFzF5ZzPL8PQCMzuzK7adnMHV0Bn27d4x6DDk5OcQi95YinvNX7tmxDiNm4jn/eM4dlH9d0SyYCoC+YetZQFEDbQrMLAnoCuyK0GdWPX0OBgYCtVeXsoClZjahkTgKCN2WC9+eEzmt5lNccpA5q7YwZ1UxSzaHhlaNykzlP6aM4KLRGfRLi36RJCIiItEtmBYDQ81sIFAIzACurdNmFnADMB+4Eng7/Im3uty92Mz2mdkkYCFwPXCfu68Ceta2M7NNwPjgKblZwMxgDNVEoCTo5w3gf8zsuOCwycDtx5z1Mdq6t4w5q4p5bWUxuUGRdHxGKrdOHsZFY/owsEenGEcoIiISf6JWMAVjkmYCbxCaVuAhd19jZncDue4+C/gb8LiZrSd0ZWlG7fFB0ZMKJJvZdGBy8ITdLXwxrcBcvhjw3ZA5hKYUWE9oWoFvBvHtMrN7CBV2AHfXDgBvblv3ljF3VTGvrQoVSe4woncXbp08jKmjMxiU3jkWYYmIiEggqvMwufscQgVL+LY7w5bLgKsaOHZAA9tz+fJUAw0eG1yx+l4D7R4CHmqsr2jZtreMuau38NrKYhZv3oU7DO/VhR9+JVQkDempIklERKSl0Ezfzaj2StKcVVs+L5KG9erMv503jIvG9GZIzy6xDlFERETqoYIpyraUlDF3dTFzw4qk4b26qEgSERFpRVQwRUFxyUHmBk+31Q7cHtFbRZKIiEhrpYKpiRTuORjcbitmaV5onqQRvbvwo/OHMXVMBoM1cFtERKTVUsF0DPJ3HWDu6mJeW7WFFcFkkif0SeXHFwznwlG99XSbiIhIG6GC6QhV1cAfc9Yzd9UWVhWWAKHJJH8yZThTR2UwQPMkiYiItDkqmI5QQWkNv3p9HWP7duM/p47gwlHN81oSERERiR0VTEeoe3tj3m3nktmtQ6xDERERkWaSEOsAWpvUZFOxJCIiEmdUMImIiIhEoIJJREREJAIVTCIiIiIRqGASERERiUAFk4iIiEgEKphEREREIlDBJCIiIhKBCiYRERGRCMzdYx1Dq2Jm+4B1sY4jRnoAO2IdRAzFc/7KPX7Fc/7xnDvET/793T09UiO9GuXIrXP38bEOIhbMLDdec4f4zl+5x2fuEN/5x3PuoPzr0i05ERERkQhUMImIiIhEoILpyD0Y6wBiKJ5zh/jOX7nHr3jOP55zB+V/CA36FhEREYlAV5hEREREIoiLgsnMppjZOjNbb2a31bM/xcyeCfYvNLMBYftuD7avM7MLIvVpZgODPj4N+kyO9BnR1My5PxFsX21mD5lZu2B7tpmVmNny4OvO6GZ9SH7Nmf8jZrYxLM+xwXYzs98H7Vea2cnRzbrxOMP2N2Xu74flXWRmLwfb29q5f8jMtpnZ6jp9dTezt4Kf+7fM7Lhge1s69w3l/msz+zjI7yUz6xZsH2BmB8PO/Z+il/Eh8TRn7neZWWFYjlMj9RVtzZz/M2G5bzKz5cH2mJz7qHP3Nv0FJAKfAYOAZGAFMLJOm+8CfwqWZwDPBMsjg/YpwMCgn8TG+gSeBWYEy38CbmnsM9pY7lMBC76eCss9G5gdB+f+EeDKeuKYCswNvi+TgIVtLfc6/b4AXN/Wzn2w7yzgZGB1nb5+BdwWLN8G3NuWzn2E3CcDScHyvWG5D6jbtg3mfhdwaz1xNNhXW8q/Tr+/Ae6M1blvjq94uMI0AVjv7hvcvQJ4GphWp8004NFg+XngPDOzYPvT7l7u7huB9UF/9fYZHHNu0AdBn9MjfEY0NVvuAO4+xwPAIiAryvlF0qz5N2Ia8FjwrVkAdDOzjKZIsBExyd3MuhD6GXg5Snkdrmjkj7u/B+yq5/PC+6r7c98Wzn2Dubv7m+5eFawuILY/98193hvSYF9RFpP8g+O/Sug/ym1WPBRMmUB+2HpBsK3eNsEPfgmQ1sixDW1PA/aE/eMR/lkNfUY0NWfun7PQrbivA6+HbT7VzFaY2VwzO+FoEzpCscj/F8Gtid+aWcoRxNHUYnLugcuAf7r73rBtbeXcN6aXuxcHfRUDPY8gjqbW3LmHu5HQFbVaA81smZm9a2ZnHkE/RysWuc8MfuYfsuBW7DH0daxide7PBLa6+6dh25r73EddPBRM9V3FqftoYENtmmr74cbR1Joz93B/BN5z9/eD9aWEpp4/EbiP5rv60Nz53w6MAE4BugP/cQRxNLVYnftrOPR/mW3p3EcrjqYWk9zN7KdAFfBEsKkY6OfuJwH/DjxpZqmH09cxaO7cHwAGA2MJ5fubI4gjGmL1977uz30szn3UxUPBVAD0DVvPAooaamNmSUBXQpcfGzq2oe07CF1yT6qzvbHPiKbmzJ2gj/8C0gn9kADg7nvdvTRYngO0M7Mex5LYYWrW/N29OLj1Ug48zBeX4A8njqYWi3OfRijn12q3tbFz35ittbfagj+3HUEcTa25c8fMbgAuBq4LbskT3NrZGSwvITQmZthR5HMkmjV3d9/q7tXuXgP8hdj+zB/u5zb1uU8CLgeeqd0Wo3MffU05IKolfhF6X94GQoPYagfBnVCnzfc4dBDcs8HyCRw6CG4DoUF1DfYJPMehg76/29hntLHcvwV8CHSo8xm9+WLOrwlAXu16G8s/I/jTgP8H/DJYv4hDB/4uamu5B8d9B3i0rZ77sOMG8OXBv7/m0EHfv2pL5z5C7lOAtUB6ne3pfDFoeBBQCHRvY7lnhC3/kNAYoIh9tZX8w87/u7E+983xFfMAmiXJ0JMqnxCqcn8abLsbuDRYbk+o0FlPaLDyoLBjfxoctw64sLE+w/5yLAr6eg5IifQZbSj3qmDb8uCr9omJmcCa4IdxAXBaGz33bwOrgNXA34HOwXYD/hC0XwWMb2u5B/tygCl1trW1c/8UodsNlYT+R35TsD0N+CfwafBn9zZ47hvKfT2hsS+1P/e1v4yvCDv3S4FL2mDujwfndSUwi0MLqHr7akv5B/seAb5TJ4aYnPtof2mmbxEREZEI4mEMk4iIiMgxUcEkIiIiEoEKJhEREZEIVDCJiIiIRKCCSURERCQCFUwibZyZlYYtZ5jZZ2Z2yTH2mWNm4xvZn21ms4/lM0REWhIVTCJxIngx7hxCb5N/NdbxiIi0JiqYROJA8ELkF4FZ7v5g2PZrzGyVma02s3uDbeeZ2ZLg6wkz6xpsH2Nma83sTULvyvtxcNz/hPV3n5mtAr4LZJjZO8GLd4cG+x8xsyuD5W+ZmZtZDzMbYGarw/q50sweCZY/v1plZt3NrMTMbg3WB5vZ60Gs75vZiLqfE6yvDj7j12a23My2mFlhsHx3ne/VADP72MweDV6q+ryZdQz2jQteJrrEzN4Iex1KjpmtC/pbbmaDzOxTM0sP9ieY2fra18IE8W0M2laEbf+TmX0UbK8Oy/89M3sp+P7/ycwSgn3hVw/fD/s+fSOIcZWZvRIW/8vB9jVmdnPYseH9jDeznLDvxerav0NmtsHM7m/sXDb291CkNVPBJBIfHgLOJuwFmWbWB7gXOJfQy0NPMbPpwAfuPs7dxxG6IvX/BYf8H6GZu68AUoE3guPOC4qpM4DRwInAB0AnQrMO3wH8MjwYM2tP6FUq2zgytwObw9YfBL4fxHoroRc/N8jdf+zuYwm9tui37j7W3e+sp+lw4EF3HwPsBb4bFJ33AVcGn/cQ8IuwY64L+hvr7hsIzfZ+XbDvK8AKd98RrCcCPwpiKQIws9HAaYReZTEWOBjW9wTgR4S+v4MJvbvrc2Z2EaF3gtXm+UgQ41hCxW3t2+JvDLaPB35goff/Ha6bgdK6G4/hXIq0KiqYRNq+ToR+aX6D0Gs6ap0C5Lj7dnevfcv8We5ebmaPmdly4D+AC4L2Jwft9wGbgJXBce8R+oV+CvC2h15EuhJY7+4HCb0qZGKdmL4HPMqhRUGjzCyT0PvYXgrWOxMqMJ4LYv0zkBF2SO3VpOWEiowjke/u84LlvwNnECqiRgFvBX3+jNALShvyEHB9sHwjoRcy1+oAlNVpX03o/V/J9fS1yN03uHs1oaL3jNodZmaEXmnxP+EHmNmdwBZgH/B+sPkHZlb7mpq+wNBG4g/vqyPwTeCBenYf8bkUaY1UMIm0feXAV939SaDSzGqvelhDB7j79cFVjkuBw3l/kjXSn9fZlwpcQ6jAORL/BdwTFk8CsCfsqs5Ydz8+rP2Pa7cTej/Wkaibc20Oa8I+a7S7T26wA/d8YKuZnUuoYJwbtrsPdd4E7+5rgWeBbUFB1iFCPLWuIfQevy11+rubUAFZDpxkZtmErnSd6u4nAssIvVfscPwboat5dYuioz2XIq2OCiaRtq/K3fcHyzOBX1hoXNJC4OxgDFEioV9875pZupklBe1/APwjWF4OZFto8PgAYExw3NnAYiAXODcYXzMGGGJmHQj9kl4cFs8Pgd+7e8UR5DAYGODub9ZucPe9wEYzuwpCV1rM7MQj6LMx/czs1GD5GkK3GNcB6bXbgzE9J0To56+ErlA9G1wdwsyGEPr+ra2nfQnwu/puyZnZwOB7e3UQD4T+Df8h8KvwTsysW7BYE/w5jNAtu93ufiAY6zUpQuy1ugLTCV0xq+tozqVIq5QUuYmItBXuvt7MHgb+x92/Z2a3A+8Qunoyx91fCa6I/C745byW0NgVCI0RegLII/SLfTKhcTWvufty4P9v745Z4giDOIw/A6YQrEw+QOxSJJ1VIHCtR9pUFgkWIekPtLDIRwhpJJWthSCGYGmrRbCw8AoLv0LqcDApZnN3HB4rdyE55fmVu8s7y27z551ZlojoU38ov6LmXU6AJ8CbsdsIKkRMWouIP0HgMbAaERtUcHhGtYQmbQJ7EbELPAIOmvrz6gNvI+IrcA3sZeavZsj5SxM4l4DP1F/Zp/lGteL2YTg3dgy8nwwZEfGSeqbdW9Y5o+bAXlAt0KPm+DJwmJk/qzM3tB0RXWoH6ZLauRoAHyLikgp/52PXL489+xXqXWwBp1TbsZeZg4kaMP1dSg9OZN5lt12SRpqvqHqZ+WPK+U5z/vUcNd5BDTDPusaMdZ8C3zPz+V9Ya50aLn/VevH0NTrM+SxnrNnJzE//qqa06NxhkrSoLv73DcwjInaAj4y+lLtPbhi18yThDpMkSVIrh74lSZJaGJgkSZJaGJgkSZJaGJgkSZJaGJgkSZJaGJgkSZJa/AYU72nL0yHSZgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 648x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "build_plot(X_train, y_train, X_val, y_val)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {
    "id": "e3EYauPK689m"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MSE: 0.010623679209131955, R2: 0.9985263268229801\n"
     ]
    }
   ],
   "source": [
    "regr = GradientLR(0.1, iterations=10000)\n",
    "regr.fit(X_train, y_train)\n",
    "y_pred = regr.predict(X_test)\n",
    "print(f\"MSE: {mse(y_test, y_pred)}, R2: {r2(y_test, y_pred)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "S_WtOSFG689m"
   },
   "source": [
    "### Задание 4 (2 балла)\n",
    "Протесируйте оба метода на данных `boston.csv`, для градиентного спуска постройте график зависимости ошибки от коэффициента регуляризации. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {
    "id": "_einEvle689m"
   },
   "outputs": [],
   "source": [
    "X, y = read_data()\n",
    "X_train, X_val, y_train, y_val = train_test_split(X, y, train_size=0.8, shuffle=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [],
   "source": [
    "def normalise(X_train, X_test, y_train, y_test):\n",
    "    maxs = np.amax(X_train, axis=0)\n",
    "    mins = np.amin(X_train, axis=0)\n",
    "    y_max, y_min = np.max(y_train), np.min(y_train)\n",
    "    for j in range(X_train.shape[1]):\n",
    "        for i in range(X_train.shape[0]):\n",
    "            X_train[i][j] = (X_train[i][j] - mins[j]) / (maxs[j] - mins[j])\n",
    "    for j in range(X_test.shape[1]):\n",
    "        for i in range(X_test.shape[0]):\n",
    "            X_test[i][j] = (X_test[i][j] - mins[j]) / (maxs[j] - mins[j])\n",
    "    \n",
    "    for i in range(y_train.shape[0]):\n",
    "        y_train[i] = (y_train[i] - y_min) / (y_max - y_min)\n",
    "    for i in range(y_test.shape[0]):\n",
    "        y_test[i] = (y_test[i] - y_min) / (y_max - y_min)\n",
    "    \n",
    "    return X_train, X_test, y_train, y_test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_val, y_train, y_val = normalise(X_train, X_val, y_train, y_val)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {
    "id": "jBswiVjC689m"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MSE: 0.010995709175721886, R2: 0.6872797475661063\n"
     ]
    }
   ],
   "source": [
    "regr = NormalLR()\n",
    "regr.fit(X_train, y_train)\n",
    "y_pred = regr.predict(X_val)\n",
    "print(f\"MSE: {mse(y_val, y_pred)}, R2: {r2(y_val, y_pred)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {
    "id": "dLUWno_N689m"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAj8AAAEKCAYAAAD98zS0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3Xl8FdX9//HXh4QEAoQ17EvY90VR3FqNu5YqtmpFW5dvbfm21trWWn/6rVq1tXWpX2uL31qq1mqta6uiUtGqEVBAUFEg7HvCviSQhCTc3M/vjzvYa0wIhFxuknk/H4/7yMyZM2fOJ5OQDzNn5pi7IyIiIhIWzZLdAREREZEjScmPiIiIhIqSHxEREQkVJT8iIiISKkp+REREJFSU/IiIiEioKPkRERGRUFHyIyIiIqGi5EdERERCJTXZHUimdu3a+YABA5LdjaQoKSmhVatWye5G0oQ5/jDHDuGOP8yxQ7jjD0vsH3744XZ3z6qtXqiTny5dujB//vxkdyMpcnNzycnJSXY3kibM8Yc5dgh3/GGOHcIdf1hiN7N1B1NPt71EREQkVJT8iIiISKgo+REREZFQUfIjIiIioaLkR0REREJFyY+IiIiEipIfERERCRUlPyIiItLoLdm0+6DrhvolhyIiItK4FZdH+N2by/nL+2sPeh8lPyIiItLouDvTF2/mjlfy2FRUxqXjenH3Qe6r5EdEREQalQ07S/nF1MW8vXQrQ7q2YfJlRzO2T3slPyIiItK0VEadx99fy2+nL8MMbhk/lKtOzCY15dCGMCv5ERERkQZv2eY9/L9/fMqCDYXkDM7irq+NpEe7lnVqS8mPiIiINFjlkUoeemcVf8xdSZsWzXlw4hjOH90dM6tzm0p+REREpMEp21fJ8/M38PC7qyko3MsFY7pz23nD6dAq7bDbVvIjIiIiDcbeikqemruOKTNWs3VPOUf3bsfdF47kywOz6u0YCX3JoZmdY2bLzGylmd1UzfZ0M3s22D7XzLKD8o5m9o6ZFZvZ5Cr73GVmG8ysuJr2vmFmeWa22Mz+nqi4REREpH5FKqP85b01fOmet/nVa0vol9WKv3/nOP7x/RPrNfGBBF75MbMU4CHgTCAfmGdmU909L67a1cAudx9gZhOBe4BLgDLgVmBE8In3CjAZWFHleAOBm4GT3H2XmXVOQFgiIiJSzz7NL+Tmfy5k8cbdfGlAJ350xkCOze6QsOMl8rbXOGClu68GMLNngAlAfPIzAbg9WH4BmGxm5u4lwCwzG1C1UXefE7RXddN3gYfcfVdQb2v9hSIiIiL1bU/ZPu5/YzlPzF5Lp9bp/N83j+bcEV0PazDzwUhk8tMD2BC3ng8cV1Mdd4+YWRHQEdheh+MNAjCz94AU4HZ3f71qJTObBEwCyMrKIjc3tw6HavyKi4tDGzuEO/4wxw7hjj/MsUO442+IsX+4JcKTeRUUlTun9U7lwoEpZOxYxrvvLkv4sROZ/FSXtnkd6hysVGAgkAP0BGaa2Qh3L/xc4+5TgCkAgwcP9pycnDoernHLzc0lrLFDuOMPc+wQ7vjDHDuEO/6GFPv24nJ+8fJiXlu4iaHdMnn86yMZ06vdEe1DIpOffKBX3HpPYGMNdfLNLBVoC+w8jOPNcfd9wBozW0YsGZpXx/ZERESknrg7Uz/ZyO1TF1NSXsnPzh7MpJP70fwQ385cHxKZ/MwDBppZX6AAmAhcVqXOVOBKYDZwEfC2u9f1ys9LwKXA42bWidhtsNV1bEtERETqyZbdZfz8xUX8e8kWxvRqx30XjWJglzZJ60/Ckp9gDM+1wHRiY3Aec/fFZnYnMN/dpwKPAk+a2UpiV3wm7t/fzNYCmUCamV0AnOXueWZ2L7EkKsPM8oFH3P324DhnmVkeUAn8zN13JCo+ERERObCi0n38acYqHn9/LZVR55bxQ/mvk/qS0iyxA5prk9CXHLr7NGBalbLb4pbLgItr2De7hvIbgRurKXfg+uAjIiIiSVJcHuGxWWv488zV7CmLcN7o7vz0zEFkd2qV7K4BesOziIiI1JPSighPzl7Hw++uYlfpPs4c1oXrzxzE0G6Zye7a5yj5ERERkcNSXB7hidlreWTmGnaWVPDlgZ244azBjD7CT3EdLCU/IiIiUie7y/bxxPtreWTWGgpL93HKoCyuO30AY/sk7u3M9UHJj4iIiBySSGWUp+au54F/L6ewdB+nD+nMD08feMTf11NXSn5ERETkoM1Yvo1fvprHiq3FnNi/IzefO5SRPdsmu1uHRMmPiIiI1Gr1tmLuem0Jby3dSp+OGUy5fCxnDuuS8Hm4EkHJj4iIiNSoMupMmbGaB95cTlpqM246dwj/dVI26akpye5anSn5ERERkWqt3lbMT5//hI/XF3LuiK7cMWE4ndu0SHa3DpuSHxEREfmcaNT56+y13PP6UtJTU3hw4hjOH929Ud7iqo6SHxEREfnM1t1l/OiZBcxevYPThnTmN18fSZfMxn+1J56SHxEREQHgo/W7+P7fPmT33gj3XDiSbxzTq8lc7Ymn5EdERER4dt56bn1pMV3apvPPa05scFNS1CclPyIiIiFWEYly56uL+duc9Xx5YCf+cOlRtMtIS3a3EkrJj4iISEit31HKT59fwLy1u/jvk/vxs7MHk5rSLNndSjglPyIiIiFTEYny55mr+f1bK0htZjw4cQwTxvRIdreOmISmd2Z2jpktM7OVZnZTNdvTzezZYPtcM8sOyjua2TtmVmxmk6vsc5eZbTCz4irlV5nZNjNbEHy+k8jYREREGqO5q3fwld/P5L7pyzh1cGf+/dNTQpX4QAKv/JhZCvAQcCaQD8wzs6nunhdX7Wpgl7sPMLOJwD3AJUAZcCswIvjEewWYDKyo5rDPuvu19RuJiIhI47e9uJx7/rWU5z/Mp0e7ljx65TGcPrRLsruVFIm87TUOWOnuqwHM7BlgAhCf/EwAbg+WXwAmm5m5ewkwy8wGVG3U3ecE7SWw6yIiIk1DRSTK62v28cN3ctm7r5LvndKf604fQEZaeEe+JDLyHsCGuPV84Lia6rh7xMyKgI7A9joe80IzOxlYDvzE3TfUtoOIiEhT9c7Srfzy1TxWb68gZ3AWt4wfxoDOrZPdraRLZPJT3aUZr0Odg/UK8LS7l5vZ94C/Aqd9oVNmk4BJAFlZWeTm5tbxcI1bcXFxaGOHcMcf5tgh3PGHOXYIV/xbS6P8La+CT7dX0rWV8f1hznG9S8nPm09+Xu37N3WJTH7ygV5x6z2BjTXUyTezVKAtsLMuB3P3HXGrfyY2fqi6elOAKQCDBw/2nJycuhyu0cvNzSWssUO44w9z7BDu+MMcO4Qj/mjUeWL2Wu6ZvYzUZsYt44dyxQnZvD9rRpOP/VAkMvmZBww0s75AATARuKxKnanAlcBs4CLgbXev05UfM+vm7puC1fOBJXXqtYiISCO0fkcpP3vhE+au2ckpg7K4+8KRdGvbMtndapASlvwEY3iuBaYDKcBj7r7YzO4E5rv7VOBR4EkzW0nsis/E/fub2VogE0gzswuAs9w9z8zuJZZEZZhZPvCIu98OXGdm5wORoK2rEhWbiIhIQxGNOn+bu467/7WUFDPuvXAUFx/TUw8GHUBCh3q7+zRgWpWy2+KWy4CLa9g3u4byG4Ebqym/Gbj5MLorIiLSqGwuKuOG5z9h1srtfHlgJ+65cBTd2+lqT23C+5ybiIhIIzZt4SZu/udCKiJR7vraCC4b11tXew6Skh8REZFGZE/ZPu54JY8XPsxndM+2PHDJGPpl6fH1Q6HkR0REpJF4f9V2/t8/PqVg116uO20APzx9IM1DMBFpfVPyIyIi0sBt3V3GXdOW8PKCjfTukMFz/30Cx2R3SHa3Gi0lPyIiIg1UpDLKk3PW8b9vLKc8EuW60wZwzakDaNE8Jdlda9SU/IiIiDRAH67bya0vLSZv025OHpTFHecPp2+nVsnuVpOg5EdERKQBKSjcy93/Wsorn2yka2YL/u+bR3PuiK56kqseKfkRERFpAPZWVPLwu6v404xVuMN1pw3gezn9Qz37eqLoOyoiIpJE7s60hZv51Wt5bCoqY/yobtx87hB6ts9IdteaLCU/IiIiSZK/q5TbXl7M20u3MqxbJg9OPIpxffUUV6Ip+RERETnCIpVRHn9/Lfe/sRwzuGX8UK46MZtUvbPniFDyIyIicgQtzC/i5hc/ZVHBbk4b0pk7JwzXLa4jTMmPiIjIEVBYWsFv31jGU3PX07FVOpMvO4rxI7vpKa4kUPIjIiKSQNGo88KH+dz9+lIKSyu48oRsfnLmINq2bJ7sroWWkh8REZEEWVRQxG0vL+Kj9YUc06c9d044jmHdM5PdrdBT8iMiIlLPNuws5bdvLOPlBRvp1DqN+y8ezdeP7qFbXA1EQoeVm9k5ZrbMzFaa2U3VbE83s2eD7XPNLDso72hm75hZsZlNrrLPXWa2wcyKazjmRWbmZnZMImISERGpyY7icu54ZTGn3Z/L9MWb+cGp/Xn7hhwuHNtTiU8DkrArP2aWAjwEnAnkA/PMbKq758VVuxrY5e4DzGwicA9wCVAG3AqMCD7xXgEmAyuqOWYb4Dpgbj2HIyIiUiN359FZa/jdv1dQWhHhkmN78eMzBtEls0WyuybVSORtr3HASndfDWBmzwATgPjkZwJwe7D8AjDZzMzdS4BZZjagaqPuPidor7pj/hK4F7ihnmIQERE5IHfnnteX8fC7qzhtSGf+5ytDGdC5dbK7JQeQyNtePYANcev5QVm1ddw9AhQBHetyMDM7Cujl7q/WZX8REZFD5e7c9doSHn53Fd86vjePXHGMEp9GIJFXfqq7NON1qFP7gcyaAQ8AVx1E3UnAJICsrCxyc3MP9XBNQnFxcWhjh3DHH+bYIdzxhzl2qP/43Z2/L63gzXURzuidyulttzNjxrv11n59Cvu5ryqRyU8+0CtuvSewsYY6+WaWCrQFdtbhWG2IjQ3KDW6HdQWmmtn57j4/vqK7TwGmAAwePNhzcnLqcLjGLzc3l7DGDuGOP8yxQ7jjD3PsUL/xR6PObVMX8ea69Vz9pb7cMn5ogx7QHPZzX1Uib3vNAwaaWV8zSwMmAlOr1JkKXBksXwS87e6HfOXH3YvcvZO7Z7t7NjAH+ELiIyIicriiUefnLy3kb3PW89+n9GvwiY98UcKSn2AMz7XAdGAJ8Jy7LzazO83s/KDao0BHM1sJXA989ji8ma0F/he4yszyzWxYUH6vmeUDGUH57YmKQUREJF5l1LnhhU94+oMN/ODU/tx0zhAlPo1QQl9y6O7TgGlVym6LWy4DLq5h3+waym8EbqzluDmH2FUREZED2lcZ5frnPuGVTzZy/ZmDuO70gcnuktSR3vAsIiJSi4pIlB8+/RHTF2/hpnOH8L1T+ie7S3IYlPyIiIgcQNm+Sq556iPeXrqVX5w3jP86qW+yuySHScmPiIhIDcr2VfLdJ+Yza+V2fv21kVx2XO9kd0nqgZIfERGRakQqo/zw6Y+ZuWI79140im8c06v2naRRSOjEpiIiIo1RNOrc+MKnvJm3hTsnDFfi08Qo+REREYnj7tz+ymL++XEBPzt7MFeckJ3sLkk9U/IjIiIS5/43lvPE7HX898n9uCZHT3U1RUp+REREAn96dxWT31nJpeN6cdO5eoFhU6XkR0REBHhi9lp+86+lfHVUN351wUglPk2Ykh8REQm9Zz5Yz20vL+aMoV144JIxpDRT4tOUKfkREZFQ+8eH+dz84kJyBmfx0DePonmK/jQ2dTrDIiISWq98spGfvfAJJ/XvxMPfGkt6akqyuyRHgJIfEREJpdcXbebHzy7gmOwOTLliLC2aK/EJCyU/IiISOi99XMAPn/6I0T3b8thVx5KRpgkPwkRnW0REQsPd+eO7q7j39WUc368DU644htbp+lMYNjrjIiISClF3bn15EX+bs57zR3fnvotHaYxPSCU0+TGzc4AHgRTgEXe/u8r2dOAJYCywA7jE3deaWUfgBeBY4HF3vzZun7uAK4D27t46rvx7wA+ASqAYmOTueYmMT0REEsvd2VRUxsKCItZsL6GkPEJpRSWlFbGvBhzfryOnDulMl8wWNbazt6KSP3xczsdb1/O9U/pz49mDaabH2UMrYcmPmaUADwFnAvnAPDObWiUhuRrY5e4DzGwicA9wCVAG3AqMCD7xXgEmAyuqlP/d3R8Ojn0+8L/AOfUblYiIJFJFJMoHa3Yye/V2FhbsZnFBETtKKj7bbgYZzVPISE8lIy2FvRWVvLRgIwDDu2dy2pDOnNCvIxWVUXaVVrCzZB+7SirIXb6VxVsruXPCcM3VJQm98jMOWOnuqwHM7BlgAhCf/EwAbg+WXwAmm5m5ewkwy8wGVG3U3ecE7VUt3x232grw+glDREQSqbC0gtxl2/j3ki28u2wbe8ojpDYzBnZpw2lDOjOyZ1tG9GjLoC5taJWW8rl//92dpZv38PbSrbyzdCsPvbOSP7y98nPtNzPoktmCa49KV+IjAJh7YnIEM7sIOMfdvxOsXw4cV+UW1qKgTn6wviqosz1Yvwo4Jn6fuH2L4297BWU/AK4H0oDT3L3q1SHMbBIwCSArK2vsc889Vx/hNjrFxcW0bt269opNVJjjD3PsEO74kx27u1NY7mzYEyW/OBr7uscpKI4SdchMM8Z0TuGozikM65BCeuqh35YqrnDWFFXSItVok2a0bm5kNIdmZkmPP5nCEvupp576obsfU1u9RF75qe6ntmqmdTB1Dpq7PwQ8ZGaXAbcAV1ZTZwowBWDw4MGek5NT18M1arm5uYQ1dgh3/GGOHcIdf7JiLyjcy4sf5fPPjwpYvb30s/KumS0Y3K0NF/Rsy2lDOjO6Z7uEjsPRuc9JdjcajEQmP/lAr7j1nsDGGurkm1kq0BbYWQ/Hfgb4Yz20IyIidVBcHmH6os3846N8Zq/egTsc17cDl5/Qh2HdMhnctQ3tMtKS3U0JqUQmP/OAgWbWFygAJgKXVakzldjVmdnARcDbXsf7cGY2MO4213i+OCBaREQSZF9llE/zC5m1YgfvrdzOR+t3EYk6vTtk8OPTB/H1o3vQq0NGsrspAiQw+XH3iJldC0wn9qj7Y+6+2MzuBOa7+1TgUeBJM1tJ7IrPxP37m9laIBNIM7MLgLPcPc/M7iWWRGWYWT6xR+hvB641szOAfcAuqrnlJSIi9acy6sxYsY0X5ufz7vJtFJdHMIORPdry3ZP7cdqQzhzTp/0XHlARSbaEvufH3acB06qU3Ra3XAZcXMO+2TWU3wjcWE35jw6nryIicnA27CzlufkbeOHDfDYVldGhVRrnje7Olwd24sT+HXU7Sxq8AyY/ZvYtd/9bsHySu78Xt+1ad5+c6A6KiEhylUcqWbC+kNmrY7e05q3dhRmcPDCLW786jDOGdiEtVVNFSuNR25Wf64G/Bct/AI6O2/ZtYi8bFBGRJiRSGeXTgiLmrN7B7FU7mLd2J2X7opjBiO5tuf7MQVw0tifd27VMdldF6qS25MdqWK5uXUREGiF3J2/Tbmau2M6c1TuYt2YnJRWVAAzu0oaJx/bmxP4dOa5vR9pmNE9yb0UOX23Jj9ewXN26iIg0InvK9vHygo08/cF6Fm+MvSR/QOfWfO3oHpzQrxPH9etAp9bpSe6lSP2rLfkZYmafErvK0z9YJljvl9CeiYhIvXN3Pskv4um565n6yUb27qtkWLdMfjlhOGeP6ErnNjVPDirSVNSW/Aw9Ir0QEZGEKimPkLthH/f9YRaLN+4mIy2FCWO6c+m43ozq2VaPo0uoHDD5cfd18etm1hE4GVjv7h8msmMiInJ49o/lefqD9bz08UaKyyMM6ZrOLy8YwQVjutOmhcbvSDjV9qj7q8BN7r7IzLoBHwHzid0Cm+LuvzsSnRQRkYNTEYnywZqdvLV0C28t2cr6naWkpzbjq6O6MzRtO1dP+LKu8kjo1Xbbq6+7LwqW/wt4092vMLM2wHuAkh8RkSTbsruMd5dtI3f5VmYs305xeYT01GacNKATk07ux1dHdaNdRhq5ublKfESoPfnZF7d8OvBnAHffY2bRhPVKRERqVLR3H0s27ebd5dvIXbaNJZtiT2p1bpPOeaO7cdqQLpw0oCMZaQl9ib9Io1Xbb8YGM/shsdnXjwZeBzCzloBuFouIJNjKrcXkLtvKqm3FrNpWwuptJWwvLgcgtZkxtk97bjxnMDmDOjO0Wxtd2RE5CLUlP1cDdwJnAJe4e2FQfjzwl0R2TEQkrLYXl/PKJxt58eMCPs0vAqB9RnP6Z7XmtCFZ9MtqzYCs1ozr14FMDVoWOWS1Pe21FfheNeXvAO8kqlMiImFTWFrBW0u28sqnG5m5YjuVUWd490xuGT+Ur47qTte2ev+OSH2p7WmvqQfa7u7n1293RETCY+vuMt7I28L0xZuZvWoHkajTo11LJp3cj68d1YNBXdoku4siTVJtt71OADYATwNz0XxeIiKHJX9XKa8v2szrizbz4fpduEPfTq347sn9OHdEV0b20AsHRRKttuSnK3AmcClwGfAa8LS7Lz6Yxs3sHOBBIAV4xN3vrrI9HXgCGAvsIDauaG3wMsUXgGOBx9392rh97gKuANq7e+u48uuB7wARYBvw7aovaRQRSYaCwr289HEBry/azMKC2Bieod0y+fHpgzh3ZFcGdm6thEfkCKptzE8lsSe8Xg8SlUuBXDO7093/cKB9zSwFeIhY8pQPzDOzqe6eF1ftamCXuw8ws4nAPcAlQBlwKzAi+MR7BZgMrKhS/jFwjLuXmtn3gXuDtkREjjh3Z/66XfzlvTW8vmgzUYfRvdpx07lDOGd4V7I7tUp2F0VCq9aXQARJz3hiiU828HvgnwfR9jhgpbuvDtp5BpgAxCc/E4Dbg+UXgMlmZu5eAswyswFVG3X3OUF7VcvjB2DPAb51EH0UEalXFZEory3cyGOz1rKwoIi2LZvz3ZP78a3j+tCrQ0ayuyci1D7g+a/Errz8C7gj7m3PB6MHsfFC++UDx9VUx90jZlYEdAS2H8JxqnM1sT6LiBwRSzbt5vn5+by0oICdJRX0z2rFry4YwdeP7qGXDYo0MObuNW+MvcW5JFiNr2iAu3vmAfa9GDjb3b8TrF8OjHP3H8bVWRzUyQ/WVwV1dgTrVxG7lXVtNe0Xx4/5iSv/FnAtcIq7l1ezfRIwCSArK2vsc889V2P8TVlxcTGtW3/h2xcaYY4/zLFD/cZfXOHM2RRhZkGEdbujpBgc1TmFk3umMqJTCs0a2Dgenfvwxh+W2E899dQP3f2Y2urVNuan2WH0IR/oFbfeE9hYQ518M0sF2gI763pAMzsD+Dk1JD4A7j4FmAIwePBgz8nJqevhGrXc3FzCGjuEO/4wxw6HH39pRYR/L9nK1AUFvLt8G/sqnWHdMvnFKT2ZMKYHHVql1V9n65nOfXjjD3Ps1Unktdh5wEAz6wsUABOJPTEWbypwJTAbuAh42w90KeoAzOwo4E/AOcHLGUVE6kVl1Jm5YhsvfVzAG3lbKK2opGtmC646MZsJY3owokfbZHdRRA5BwpKfYAzPtcB0Yo+6P+bui83sTmC+u08FHgWeNLOVxK74TNy/v5mtBTKBNDO7ADjL3fPM7F5iSVSGmeUTe4T+duA+oDXwfDAYer1ewigih2NzURnPzd/As/M2UFC4l3YZzbngqB6cP7o747I70KxZw7qtJSIHJ6Gj8Nx9GjCtStltcctlwMU17JtdQ/mNwI3VlJ9xOH0VEYHYVZ4Zy7fx1Nz1vL10C1GHLw/sxM/HD+WMoV1ISz2c0QAi0hDoEQQREWDrnjKen5/P3+eup6BwL51ap/O9U/pzybG96NNR7+QRaUqU/IhIaLk7c1bv5G9z1jF98WYiUeekAR35+fihnDmsC81TdJVHpClS8iMioROpjDJt0WamzFjFooLdtG3ZnKtOzOay43rTL6vpPw4sEnZKfkQkNErKI7y5dh+3zM0lf9de+nVqxW++PpKvHdWDFs1Tkt09ETlClPyISJNXtHcff31/LY+9t4bC0n0c06c9t311GGcM7aIntkRCSMmPiDRZO0sqeGzWGv76/lr2lEc4Y2hnjsvczXe/dmKyuyYiSaTkR0SanJ0lFfzp3VU8OWcde/dVcu6Irvzg1AEM796W3NzcZHdPRJJMyY+INBnF5REenbmGP89cTWlFhPNGd+faUwcwsEubZHdNRBoQJT8i0uiVRyp5as56HnpnJTtKKjh7eBduOGuwkh4RqZaSHxFptCoiUV74MJ+H3llJQeFeTuzfkZ+dPZijerdPdtdEpAFT8iMijU55pJLn5+fzx9xVFBTuZXSvdtxz4Si+NLBTsrsmIo2Akh8RaTTKI5U8N28D/5e7ik1FZRzdux2//vpITh7YiWBCYxGRWin5EZEGL1IZ5R8f5fP7t2K3t47p0557LxrFlwYo6RGRQ6fkR0QarGjUeeXTjfzu3ytYs72E0T3bcveFI5X0iMhhUfIjIg2Ou/Nm3hb+983lLN28hyFd2zDl8rGcOayLkh4ROWxKfkSkwXB3Zq7Yzv1vLOOT/CKyO2bw4MQxnDequ6ahEJF60yyRjZvZOWa2zMxWmtlN1WxPN7Nng+1zzSw7KO9oZu+YWbGZTa6yz11mtsHMiquUn2xmH5lZxMwuSmRcIlL/Plizk0umzOGKxz5ge3EF9144in9ffwoTxvRQ4iMi9SphV37MLAV4CDgTyAfmmdlUd8+Lq3Y1sMvdB5jZROAe4BKgDLgVGBF84r0CTAZWVClfD1wF3FDPoYhIAi3YUMj9byxj5ortZLVJ584Jw7nk2F6kp2qWdRFJjETe9hoHrHT31QBm9gwwAYhPfiYAtwfLLwCTzczcvQSYZWYDqjbq7nOC9qqWrw3Ko/UahYgkxKKCIh54czlvLd1Kh1Zp/M9XhnD58dm0TFPSIyKJlcjkpwewIW49HziupjruHjGzIqAjsD1RnTKzScAkgKysrNBOclhcXBza2CHc8Sc79o3FUf6xooIPt1TSqjlcOLA5Z/RJpWV0A3Pf31B7A4cp2fEnU5hjh3DHH+bYq5PI5KcMJk0wAAAeKUlEQVS6m/Rehzr1yt2nAFMABg8e7Dk5OYk8XIOVm5tLWGOHcMefrNi37injgTdX8Oy89WSkpfKj0wdy9Zf7ktmi+RHth859TrK7kTRhjj/MsVcnkclPPtArbr0nsLGGOvlmlgq0BXYmsE8icoSVlEf488zVTJmxmopIlCtOyOa60wfSoVVasrsmIiGVyORnHjDQzPoCBcBE4LIqdaYCVwKzgYuAt909oVd+ROTIiEadf3yUz73Tl7FtTzlfGdmVG88eQnanVsnumoiEXMKSn2AMz7XAdCAFeMzdF5vZncB8d58KPAo8aWYriV3xmbh/fzNbC2QCaWZ2AXCWu+eZ2b3EkqgMM8sHHnH3283sWOBFoD1wnpnd4e7DExWfiNRsUUERt768iI/XF3JU73Y8/K2xjO2jmdZFpGFI6EsO3X0aMK1K2W1xy2XAxTXsm11D+Y3AjdWUzyN2a01EkqSwtILfvrGMp+aup2OrNH578Wi+fpTe0yMiDYve8Cwihy1SGeW5+fn89o1lFJZWcOUJ2fzkzEG0bXlkBzOLiBwMJT8iclhmLN/GXa8tYdmWPYzL7sDt5w9nWPfMZHdLRKRGSn5EpE6Wb9nDXa8t4d3l2+jdIYM/fvNozhnRVROPikiDp+RHRA7JrpIK7n9zGX+fu57W6ancMn4ol5/QR9NRiEijoeRHRA5KpDLK3z9Yz/1vLKe4PMLlx/fhx2cMor3e1yMijYySHxGp1exVO7jjlcUs3byHE/t35BfnDWdw1zbJ7paISJ0o+RGRGm3ZXcYvX83j1U830aNdSx7+1tGcPVzjekSkcVPyIyJfEKmM8uScddz/xnIqKqP8+IyBfO+U/rRornE9ItL4KfkRkc9ZsKGQn7+4kMUbd3PyoCzuPH+4pqQQkSZFyY+IALG3M983fRl//2A9Wa3Teeiyo/nKSN3iEpGmR8mPSMhVRp3n5m/g3teXUrR3H1edmM31Zw6iTQu9nVlEmiYlPyIhtmBDIbe9vIhP84sYl92BOyYMZ2g3vZ1ZRJo2JT8iIbR1dxn3v7Gc5z7cQFbrdB6cOIbzR3fXLS4RCQUlPyIhUlIe4cUVFXz/rVwi0Sjf+VJfrjt9oG5xiUioKPkRCYFIZZRn52/ggTdXsL14H+NHdePGswfTp6Oe4hKR8GmWyMbN7BwzW2ZmK83spmq2p5vZs8H2uWaWHZR3NLN3zKzYzCZX2ecuM9tgZsUH05ZI2M1etYNzH5zJz19cRN9OGdx6fAseuuxoJT4iEloJS37MLAV4CDgXGAZcambDqlS7Gtjl7gOAB4B7gvIy4FbghmqafgUYV015TW2JhNLW3WX86JmPufTPcyiLVPLwt8by3H+fQP92elGhiIRbIm97jQNWuvtqADN7BpgA5MXVmQDcHiy/AEw2M3P3EmCWmQ2o2qi7zwnaq7qppra8XqIRaSQilVGemL2OB95cTnkkynWnD+SaHL2dWURkv0QmPz2ADXHr+cBxNdVx94iZFQEdge2Hc7x6aEukUZq7ege3v5LHkk2xtzPfcf5w+urtzCIin5PI5Ke6Z2arXoU5mDr1eTzMbBIwCSArK4vc3Nw6Hq5xKy4uDm3s0PTi37E3yrPLKvhgcyUdWhg/GJPOMV1KWLdoHuuq1G1qsR+qMMcf5tgh3PGHOfbqJDL5yQd6xa33BDbWUCffzFKBtsDOwzzeAdty9ynAFIDBgwd7Tk5OHQ/XuOXm5hLW2KHpxL+3opI/zVjFw++vwh1+dHpsAtKWaTXf4moqsddVmOMPc+wQ7vjDHHt1Epn8zAMGmllfoACYCFxWpc5U4EpgNnAR8PZhjNGpz7ZEGjR35428Ldz5Sh4FhXsZP6ob//OVofRo1zLZXRMRafASlvwE426uBaYDKcBj7r7YzO4E5rv7VOBR4EkzW0nsKs3E/fub2VogE0gzswuAs9w9z8zuJZZEZZhZPvCIu99+oLZEmpL1O0r5xdRFvLNsG0O6tuGZScdzfL+Oye6WiEijkdCXHLr7NGBalbLb4pbLgItr2De7hvIbgRurKa+xLZGmoGxfJVNmrOahd1aS2sy4ZfxQrjwxm+YpCX1dl4hIk6M3PIs0cO7O20u38qvXlrBmewnjR3Xj1vHD6Nq2RbK7JiLSKCn5EWnAlm/Zwy9fzWPmiu30y2rFE98ex8mDspLdLRGRRk3Jj0gDtLOkggfeXM5Tc9fROj2V2746jMtP6KNbXCIi9UDJj0gDUhl1/v7Beu57fSklFZVcfnwffnzGINq3Skt210REmgwlPyINxOKNRfzPi4v4ZEMhJ/bvyO3nD2dQlzbJ7paISJOj5EckyUrKIzzw5nL+8v5a2mc053eXjGHCmO7VzV8nIiL1QMmPSJK4O69+uonfTFvCxqIyLjuuN//v7CG0zWie7K6JiDRpSn5EkuCTDYX88tU85q/bxdBumfzhsqMZ26d9srslIhIKSn5EjqBNRXu57/Vl/PPjAjq1TueeC0dy0dhepDTTLS4RkSNFyY/IEbD/7cx/zF1FpTvfz+nPNTn9adNCt7hERI40JT8iCeTuvJm3hV++lseGnXs5d0RX/ucrQ+nVISPZXRMRCS0lPyIJsmpbMXe8kseM5dsY2Lk1T33nOE4a0CnZ3RIRCT0lPyL1bOueMv7vnVU8NXcdLVJTNAGpiEgDo+RHpJ7sKqng4Rmr+Ov7a9lX6Vx0dE9uOHswWW3Sk901ERGJo+RH5DAVle7j0ffW8NisNZRURJgwujs/OmMQfTu1SnbXRESkGkp+ROpo3Y4S/vLeWp6bv4HSikrOHdGVn5w5SFNSiIg0cAlNfszsHOBBIAV4xN3vrrI9HXgCGAvsAC5x97Vm1hF4ATgWeNzdr43bZyzwONASmAb8yN3dzEYDDwOtgbXAN919dyLjk/Bxd+av28UjM1fzRt4WUpsZ543uzne+1I9h3TOT3T0RETkICUt+zCwFeAg4E8gH5pnZVHfPi6t2NbDL3QeY2UTgHuASoAy4FRgRfOL9EZgEzCGW/JwD/At4BLjB3d81s28DPwvaEDlsu8v28fKCjTzzwXoWb9xNu4zmXJPTnytOyKZLZotkd09ERA5BIq/8jANWuvtqADN7BpgAxCc/E4Dbg+UXgMlmZu5eAswyswHxDZpZNyDT3WcH608AFxBLfgYDM4KqbwLTUfIjh8Hd+XhDIU/PXc+rn25i775KhnbL5FcXjODCo3vSMi0l2V0UEZE6SGTy0wPYELeeDxxXUx13j5hZEdAR2H6ANvOrtNkjWF4EnA+8DFwM9KquATObROzKEVlZWeTm5h5cNE1McXFxaGOHA8e/qyzK7I0R3tsYoaDYSU+B47ulckqvFvTNjGBla5j7/poj2+F6pHMf3vjDHDuEO/4wx16dRCY/1U1W5HWoc7D1vw383sxuA6YCFdU14O5TgCkAgwcP9pycnAMcrunKzc0lrLHDF+MvrYgwffFm/vlRAbNWbscdxvZpzw/O7Mn5Y7rTOr3pPBugcx/e+MMcO4Q7/jDHXp1E/ouez+evvvQENtZQJ9/MUoG2wM5a2uxZXZvuvhQ4C8DMBgHjD6fz0vS5Ows2FPLc/A288skmissj9Gzfkh+eNpCvHdVDj6qLiDRRiUx+5gEDzawvUABMBC6rUmcqcCUwG7gIeNvda7zy4+6bzGyPmR0PzAWuAP4AYGad3X2rmTUDbiH25JfIF+woLmf62n38+uMZLN9STMvmKYwf1Y2Lx/bk2OwONNMM6yIiTVrCkp9gDM+1xAYepwCPuftiM7sTmO/uU4FHgSfNbCWxKz4T9+9vZmuBTCDNzC4AzgqeFPs+/3nU/V/BB+BSM/tBsPxP4C+Jik0an4LCvUxftJnpizczb+1Oog6je2Xwm6+P5Kujuml2dRGREEnoQAZ3n0bscfT4stvilsuIDU6ubt/sGsrn88XH33H3B4m9U0gEdydv027eWrKVN/I2s6gg9sqnQV1ac03OALpUFHD5eScluZciIpIMTWcUp4Re2b5K3l+1nbeWbOXtpVvZVFSGGYzp1Y6bzh3C2cO7fjaOJzd3U5J7KyIiyaLkRxotd2fVthJmLN/Gu8u3MWf1DsojUVqlpfDlgVn85MzOnDq4syYWFRGRz1Hyc4RFo87usn1ktmiugbV1sKO4nDmrdzJr5XZmLN9GQeFeAPplteLScb05bUhnjuvXgfRUvYBQRESqp+SnHkUqo+woqWDbnnK2FZezfU85W/eUk79rL/m7SsnftZeCXXupqIzSPMXo2rYF3dq2pEe72GdsdnuO69uBjDSdlv22F5fzaX4h76/cwXurdrBkU2zsTuv0VE7s35Hv5/TnlEFZ9OqQkeSeiohIYxHqv7L5xVHG/34mGWkpZKSl0io9hfTUFJqZkdIMUpoZzSx2dSZS6VRURqmojLIvEqU8EmVP2T72lEUoLo989rU6HVul0bN9S4Z1y+Ss4V3Iap3OzpIKNhbuZWNhGR+s2cnm3WVUvuOkpTRjbJ/2fGlgJ748sBPDu7clJSRXiApLK1hYUMSn+UUszC9iYUHRZ1d20lKbMbZ3e244axAnDujEqB5tSU1pluQei4hIYxTq5Ce9mdE1swWlFZUU7t3HxsK9lEeiVEadqDuRqBONxl471DylGc1TjeYpzUhLaUZ6ajNat0ilc5sWtGmRSpsWzWnTIpWsNul0ap1OVpt0soKvBzMH1N6KSuat3cnMFduYuWI7901fxn3Tl9EqLYWjerfn6D7tGdunPUf1bkdmI38s293ZsrucVduKWVgQS3Q+LShkw869n9XJ7pjB0X3ac9WJ2Yzs2ZYxvdrRorluZYmIyOELdfKTlWE8etWxye4GAC3TUjh5UBYnD8oCYOueMmav2sH8tbv4cN0uJr+9gqiDGfTukMHAzm0Y1KU1A7u0ZmDnNvTt1IpWDWgKhsqos3l3Get3lLJhZynrdpawdnspq7eXsG5HCaUVlZ/V7dWhJaN6tOOycX0Y2aMtI3u0pW1G407wRESk4Wo4fy3lczq3acGEMT2YMCY2b2txeYRPNhTy4bpdLNu8h+Vb9pC7bCuR6H9eiN2xVRo9O2TQq31LenXIoEubdNq3SqNDqzTaZ8S+tmmRWucxRRWRKCXlsdt7JRURSsojFJbuY/PuMjYXlbGpKPZ1Y+Fe8oOxTfulNDN6d8ggu2MGJ/TrSN9OGfTLas2wbpm0b5V2eN8sERGRQ6Dkp5FonZ7KSQM6cdKATp+VVUSirN1RwvIte1i3o5T8XaVs2LmXhQVFvL5o8+cSo6rSmkGbmW+SkZ5C85Rmn80Ya8EYp2jUKdtXSXkk+tnXA7XXzKBLZgu6tm3B0G6ZnDW8K707ZHz26dauBc01RkdERBoAJT+NWFpqMwZ1acOgLm2+sK0y6hSWVrCrtIKdJfuCrxXsKdtHaUUlS1eupWOXrpRWVP7nCk1cbmMGLZqnkJ7a7LOvLZun0Co9ldbpqbGvLVLJbJFKt7Yt6dQ6TQOQRUSkUVDy00SlNDM6tk6nY+vqX/CXm7qRnJyRR7hXIiIiyaf/qouIiEioKPkRERGRUFHyIyIiIqGi5EdERERCJaHJj5mdY2bLzGylmd1UzfZ0M3s22D7XzLKD8o5m9o6ZFZvZ5Cr7jDWzhcE+v7fg2WwzG2Nmc8xsgZnNN7NxiYxNREREGqeEJT9mlgI8BJwLDAMuNbNhVapdDexy9wHAA8A9QXkZcCtwQzVN/xGYBAwMPucE5fcCd7j7GOC2YF1ERETkcxJ55WccsNLdV7t7BfAMMKFKnQnAX4PlF4DTzczcvcTdZxFLgj5jZt2ATHef7e4OPAFcEGx2IDNYbgtsrPeIREREpNFL5Ht+egAb4tbzgeNqquPuETMrAjoC2w/QZn6VNnsEyz8GppvZb4kldSceVu9FRESkSUpk8mPVlFWdH+Fg6hxs/e8DP3H3f5jZN4BHgTO+0IDZJGK3zQDKzWzRAY7XlHWi5iQzDMIcf5hjh3DHH+bYIdzxhyX2PgdTKZHJTz7QK269J1+8FbW/Tr6ZpRK7XbWzljZ71tDmlcCPguXngUeqa8DdpwBTAMxsvrsfU2skTVCYY4dwxx/m2CHc8Yc5dgh3/GGOvTqJHPMzDxhoZn3NLA2YCEytUmcqsaQF4CLg7WAsT7XcfROwx8yOD57yugJ4Odi8ETglWD4NWFE/YYiIiEhTkrArP8EYnmuB6UAK8Ji7LzazO4H57j6V2K2pJ81sJbErPhP3729ma4kNYE4zswuAs9w9j9jtrceBlsC/gg/Ad4EHgytIZfzn1paIiIjIZxI6sam7TwOmVSm7LW65DLi4hn2zayifD4yopnwWMPYQuzjlEOs3JWGOHcIdf5hjh3DHH+bYIdzxhzn2L7AD3GUSERERaXI0vYWIiIiESqNLfuo6ZUaw7eagfJmZnV1bm8Fg7blmtiJoM622YyTaEY7/qaB8kZk9ZmbNg/IcMysKphJZYGa3cQQc4dgfN7M1cTGOCcotmFZlpZl9amZHJzbqA/czbnt9xj4zLu6NZvZSUJ6U836gvsZtr0v8j5nZVqvyugsz62Bmbwa/92+aWfugvCmd+5piv8/MlgbxvWhm7YLybDPbG3fuH05cxF+I70jGf7uZFcTF+ZXa2kqkIxz7s3FxrzWzBUF50s59Qrl7o/kQGzi9CugHpAGfAMOq1LkGeDhYngg8GywPC+qnA32DdlIO1CbwHDAxWH4Y+P6BjtEE4/8KsXcrGfB0XPw5wKtN/Nw/DlxUTT++QmyQvQHHA3ObWuxV2v0HcEWyznui4g+2nQwcDSyq0ta9wE3B8k3APU3p3NcS+1lAarB8T1zs2VXrNtFzfztwQzX9qLGtphJ7lXbvB25L5rlP9KexXfmp85QZQfkz7l7u7muAlUF71bYZ7HNa0AZBmxfUcoxEO2LxQ2zAugeAD/j8O5aOtCMa+wFMAJ4Ivi1zgHYWm3YlkZISu5m1IfY78FKC4jpYiYgfd59B9e8Vi2+r6u99Uzj3Ncbu7m+4eyRYnUNyf+fhyJ/7mtTYVgIlJfZg/28Q+w9vk9XYkp/qpszoUVOd4Jd4/5QZNe1bU3lHoDDuH4L4Y9V0jEQ7kvF/xmK3uy4HXo8rPsHMPjGzf5nZ8LoGdAiSEftdweX/B8ws/RD6Ud+Sct6BrwFvufvuuLIjfd4hMfEfSBePvVOM4GvnQ+hHfTvSscf7Nv95lQhAXzP72MzeNbMvH0I7hyMZ8V8b/N4/ZsEtz8No63Ak69x/Gdji7vHvykvGuU+oxpb8HM6UGfVVfrD9SIQjGX+8/wNmuPvMYP0joI+7jwb+wJG5MnCkY78ZGAIcC3QA/t8h9KO+Jeu8X8rn//eXjPMOiYk/Uf2ob0mJ3cx+DkSAp4KiTUBvdz8KuB74u5ll1rR/PTrS8f8R6A+MIRbz/YfQj/qWrJ/7qr/3yTr3CdXYkp9DmTID+/yUGTXtW1P5dmKXtVOrlB/oGIl2JOMnaOMXQBaxH3oA3H23uxcHy9OA5mbW6XACOwhHNHZ33xTc3igH/sJ/LnEfTD/qWzLOe0diMb+2vyxJ553a+lq1zkHGfyBb9t/OCr5uPYR+1LcjHTtmdiXwVeCbwS1vgtsnO4LlD4mNIRlUh3gO1RGN3923uHulu0eBP9P0fu8PKGjj68Cz+8uSeO4Tqz4HECX6Q+yljKuJDeDaPwBseJU6P+DzA8CeC5aH8/kBYKuJDSirsU1ic4TFD3i+5kDHaILxfwd4H2hZ5Rhd+c87osYB6/evN6HYuwVfDfgdcHewPp7PD3r9oKmd92C/7wF/TfZ5T1T8cftl88VBr/fx+QHP9zalc19L7OcAeUBWlfIs/jNgth9QAHRogvF3i1v+CbFxM7W21RRijzv/7zaEc5/wn61kd6AOPxBfAZYTyz5/HpTdCZwfLLcglrSsJDZIt1/cvj8P9lsGnHugNuNO9AdBW88D6bUdo4nFHwnKFgSf/aP/rwUWB79cc4ATm2DsbwMLgUXA34DWQbkBDwX1FwLHNLXYg225wDlVypJy3hMY/9PELunvI/Y/5auD8o7AW8TmB3yL4B/6Jnbua4p9JbGxIvt/5/f/Yb0w7tx/BJzXRM/9k8G5/ZTY3JPdamurqcQebHsc+F6VPiTt3Cfyozc8i4iISKg0tjE/IiIiIodFyY+IiIiEipIfERERCRUlPyIiIhIqSn5EREQkVJT8iDQyZlYct9zNzFaZ2XmH2WaumR1zgO05Zvbq4RxDRKShUPIj0kgFE49OIzbz9ivJ7o+ISGOh5EekEQomm/0nMNXdp8SVX2pmC81skZndE5SdbmYfBp+nzKxtUD7KzPLM7A1i85f9LNjv13Ht/cHMFgLXAN3M7J1gYtOBwfbHzeyiYPk7ZuZm1snMss1sUVw7F5nZ48HyZ1eRzKyDmRWZ2Q3Ben8zez3o60wzG1L1OMH6ouAY95nZAjPbbGYFwfKdVb5X2Wa21Mz+GkxY+YKZZQTbxgaTNX5oZtPjprXINbNlQXsLzKyfma0ws6xgezMzW7l/eo+gf2uCuhVx5Q+b2ZKgvDIu/hlm9mLw/X/YzJoF2+Kv6s2M+z5dFfRxoZm9HNf/l4LyxWY2KW7f+HaOMbPcuO/Fov0/Q2a22swmH+hcHujnUKSxUvIj0jg9BpxC3ASEZtYduAc4jdjEjMea2QXALHcf6+5jiV0p+k2wy/8Se2vzhUAmMD3Y7/QgMfoSMBIYDcwCWhF74+ytwN3xnTGzFsSmxNjKobkZWBe3PgX4YdDXG4hNqlsjd/+Zu48hNv3MA+4+xt1vq6bqYGCKu48CdgPXBAnkH4CLguM9BtwVt883g/bGuPtqYm/6/maw7QzgE3ffHqynAD8N+rIRwMxGAicSm5JgDLA3ru1xwE+JfX/7E5tP6TNmNp7YPE3743w86OMYYonq/pm1vx2UHwNcZ7E52Q7WJKC4auFhnEuRRkPJj0jj04rYH8CriE23sN+xQK67b3P3/TNyn+zu5Wb2hJktIDY7/dlB/aOD+nuAtcCnwX4ziP1xPhZ422OTPH4KrHT3vcSmfDiuSp9+APyVz/+BPyAz60FsjqwXg/XWxJKF54O+/gnoFrfL/qs8C4glDIdig7u/Fyz/DfgSsYRoBPBm0OYtxCaArMljwBXB8reJTXi7X0ugrEr9SmJzMqVV09YH7r7a3SuJJbBf2r/BzIzY1AS/jt/BzG4DNgN7gJlB8XVmtn+6kV7AwAP0P76tDOC/iM1iXtUhn0uRxkbJj0jjUw58w93/Duwzs/1XI6ymHdz9iuDqw/nAwcxpYwdoz6tsywQuJZasHIpfAL+M608zoDDuassYdx8aV/9n+8uJzVl0KKrGvD+GxXHHGunuZ9XYgPsGYjO+n0Ys+ftX3ObuVJk1293zgOeArUFy1bKW/ux3KbG51TZXae9OYslgOXCUmeUQuwJ1gruPBj4mNtfTwfgxsatsVROcup5LkUZFyY9I4xNx95Jg+VrgLouN45kLnBKMuUkh9kfsXTPLMrPUoP51wL+D5QVAjsUGTmcDo4L9TgHmAfOB04LxKKOAAWbWktgf3Hlx/fkJ8Ht3rziEGPoD2e7+xv4Cd98NrDGziyF2BcTMRh9CmwfS28xOCJYvJXYbbxmQtb88GAMzvJZ2HiF25ei54KoNZjaA2Pcvr5r6RcCD1d32MrO+wff2kqA/EPs3+SfAvfGNmFm7YDEafB1E7LbYLncvDcZGHV9L3/drC1xA7EpWVXU5lyKNTmrtVUSkoXL3lWb2F+DX7v4DM7sZeIfYVY1p7v5ycKXiweAPbR6xsR4QG1PzFLCe2B/ps4iNQ3nN3RcAmNkSYrM55xEbHzIN6ARcHNcNI5YQVNXXzPb/Uf//7d2vSkRBFIDx74AGwaQvYDRoMwnCVhffwKAYRPuCBoOPIBYxWQ2CKGarBjFYjL6CWRaOYS7usrgIe2X9c79fvHOZc5lbDnPOMLPATESsUpKAeUrZZdA6cBIRB8AkcF7Fr+sZ2IiIU8pt7SeZ+VY1+B5XyeMEcES5wXqYa0q56ww++qyugO3BhCEililr2v5knjtK39Qipcx4WT2fAi4y87VUvz7sRUSbsrPzRNlR6gI7EfFESeTu+96f6lv7acq/2AJuKaW9TmZ2B2LA8H8p/Sve6i6J6jRQJzMfhoy3qvG1GjE2oTTvjjrHiHHngJvMXPiGuZYojdUrX748fI4WNddyxJitzDwcV0zpN3PnR9K4PP70B9QREfvALr0TX3/JC72SmdR47vxIkqRGseFZkiQ1ismPJElqFJMfSZLUKCY/kiSpUUx+JElSo5j8SJKkRnkH4X9c5v8jkc4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 648x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 17.4 s, sys: 354 ms, total: 17.8 s\n",
      "Wall time: 17.8 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "build_plot(X_train, y_train, X_val, y_val)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {
    "id": "pqHBX4Gn689m"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MSE: 0.01084150194669796, R2: 0.691665433183726\n"
     ]
    }
   ],
   "source": [
    "regr = GradientLR(0.1, iterations=10000, l=0.00025)\n",
    "regr.fit(X_train, y_train)\n",
    "y_pred = regr.predict(X_val)\n",
    "print(f\"MSE: {mse(y_val, y_pred)}, R2: {r2(y_val, y_pred)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hYrkoSNw689m"
   },
   "source": [
    "### Задание 5 (2 балла)\n",
    "Проинтерпритируйте полученные результаты. Опишите влияние каждого признака на результат предсказания."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {
    "id": "J_1RhVl8689m"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Intercept: 0.29762499368024214\n",
      "Coefficients: [-6.28524015e-02  7.46559699e-02 -2.08033465e-05  5.03701322e-02\n",
      " -9.29308773e-02  5.53627647e-01  1.39208249e-02 -2.19323908e-01\n",
      "  1.07241254e-01 -1.31986857e-01 -1.56474686e-01  1.10167045e-01\n",
      " -3.88274833e-01]\n"
     ]
    }
   ],
   "source": [
    "print(f\"Intercept: {regr.intercept_}\\nCoefficients: {regr.coef_}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "  1. crim     per capita crime rate by town\n",
    "  2. zn       proportion of residential land zoned for lots over 25,000 sq.ft.\n",
    "  3. indus    proportion of non-retail business acres per town\n",
    "  4. chas     Charles River dummy variable (= 1 if tract bounds river; 0 otherwise)\n",
    "  5. nox      nitric oxides concentration (parts per 10 million)\n",
    "  6. rm       average number of rooms per dwelling\n",
    "  7. age      proportion of owner-occupied units built prior to 1940\n",
    "  8. dis      weighted distances to five Boston employment centres\n",
    "  9. rad      index of accessibility to radial highways\n",
    "10. tax      full-value property-tax rate per 10,000 dollars\n",
    " 11. ptratio  pupil-teacher ratio by town\n",
    " 12. b        1000(Bk - 0.63)^2 where Bk is the proportion of blacks by town\n",
    " 13. lstat    \\% lower status of the population\n",
    "\n",
    "`y`  medv     median value of owner-occupied homes in dollars"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Intercept=0.3` означает, что, если все фичи равны нулю, значение `y` равно 0.3.\n",
    "\n",
    "Отрицательные коэффициенты говорят об уменьшении `y` с ростом фичи, положительные -- об увеличении. Так, фичи `crim`, `indus`, `nox`, `dis`, `tax`, `ptratio` и `lstat`своим ростом уменьшают значение медианной стоимости жилых домов, а остальные -- увеличивают. Более конкретно, например, с ростом показателя криминальности района на единицу стоимость жилых домов снижается на 0.06, с ростом концентрации nitric oxides на единицу стоимость жилья снижается на 0.09, а с ростом пропорции жилых домов, построенных до 1940, увеличивается на 0.01."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "hw3_task.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}