Toy_Regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tutorial 2. A Simple Toy Regression\n",
    "\n",
    "This is a quick example showing how one can use the concepts introduced in [Tutorial 1: Basics](https://caffe2.ai/docs/tutorial-basics-of-caffe2.html) to do regression. This tutorial is split up into two parts. **Part I** is a more verbose example of creating and training a polynomial regression model and **Part II** is a concise linear regression example.\n",
    "\n",
    "## Part I: Polynomial Regression\n",
    "\n",
    "The problem we are dealing with is a relatively simple one and involves a one-dimensional input $x$ and one-dimensional output $y.$ Because we seek a second order polynomial as the regression model, the weight vector will contain two weights ($\\beta_2$ and $\\beta_1$) and there will be a single bias ($\\beta_0$) or intercept. The desired solution is of the form:\n",
    "\n",
    "$$y = \\beta_2x^2 + \\beta_1x + \\beta_0$$\n",
    "\n",
    "For this tutorial, we will generate and format an arbitrary set of input data that possess a strong second order polynomial relationship. We will then construct the model, specify the training algorithm, perform the training, and finally look at the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:This caffe2 python run does not have GPU support. Will run in CPU only mode.\n",
      "WARNING:root:Debug message: No module named caffe2_pybind11_state_gpu\n"
     ]
    }
   ],
   "source": [
    "from __future__ import absolute_import\n",
    "from __future__ import division\n",
    "from __future__ import print_function\n",
    "from __future__ import unicode_literals\n",
    "from caffe2.python import workspace, brew, optimizer\n",
    "from caffe2.python.model_helper import ModelHelper\n",
    "import numpy as np\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import sklearn.datasets\n",
    "from sklearn.preprocessing import PolynomialFeatures"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Inputs\n",
    "\n",
    "Specify the input parameters of the regression model here including: number of samples in the input data, number of training iterations, learning rate of SGD algorithm, and the initial weights of the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Number of training sample to generate\n",
    "num_samples = 200\n",
    "# Learning Rate of SGD algorithm\n",
    "learning_rate = .05\n",
    "# Number of iterations to train\n",
    "training_iters = 100\n",
    "# Initial model weights\n",
    "initial_weights = [0.,0.]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create and Prepare the Dataset\n",
    "\n",
    "Now, we will create and prepare the dataset for use with the model. Note, we are just constructing numpy arrays here. Any other data can be used as long as it is shaped properly before being input into the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X Sample:\n",
      "[[ 1.11144622  1.2353127 ]\n",
      " [-0.80563214  0.64904315]\n",
      " [-0.20122346  0.04049088]\n",
      " [-1.39473823  1.94529472]\n",
      " [-0.26264545  0.06898263]]\n",
      "Y Sample:\n",
      "[[2.0341036 ]\n",
      " [1.25892588]\n",
      " [0.54689346]\n",
      " [2.488389  ]\n",
      " [1.00825533]]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5,1,u'Input Training Data')"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEWCAYAAABliCz2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAHctJREFUeJzt3X2UXHd93/H3R+vd2CsJsFdqA7a1chJOEp80hFh1C6Ypsd3EODwkTUMga2KbnKNaOhSRJgWKEkgTlJzQlNp5MI5iiyre4dGQA2lFSjiEU0jAeO04jUGY+BhJfoKsZMCWZaynb/+4M9Xs7L3zfGfunft5nTNnd+6dmfu7s7Pf+d3v70kRgZmZTb414y6AmZmNhgO+mVlFOOCbmVWEA76ZWUU44JuZVYQDvplZRTjgm6WQNCXpqKRNw3ys2Tg54NtAJB2QdOUIjvMbkhbb7D/adDst6emm+wu9Hi8iTkXEuog4NMzH9krSOyWdkPRk/Xa/pN+X9N09vMbnJF037LJZ+Tjg20SoB9x1EbEOOAS8omlbrfXxks4afSn7VouI9cAc8LPAhcCSpH863mJZ2Tjg29BIuq5em/w9Sd+U9DVJL2va/xlJvyPpi5K+Leljks6r73uppIdbXu+ApCslXQW8Dfj5eo397/oo2zslfVDS+yU9CVwj6UWSviDpW5Ieq9ecp+uPP0tSSNpcv79Y3/+Jek3785Iu6vWx9f0vk/TV+nvwB5L+upsaeEQcj4j7gJ8DvgX8cv315iTtk7Rcf9//XNL59X2/C7wIuKX+3t1Y3/6Hkh6W9ISkuyS9uNf31MrHAd+G7V8A9wMbgHcBt0lS0/5fBF4PPA84Cfx+pxeMiL8Afhv4YL3G/oI+y/YzwPuAZwMfrB9/R72slwFXAf++zfN/Afh14DySq4jf6vWxkv4J8CHgP9WP+zXg0l5OIiJOAh8H/lV90xrgT4BNwDxwArip/ti3AJ8Hbqi/d2+qP+dO4Ifr5bsD+LCk7+qlHFY+Dvg2bAcj4k8i4hSwF3gu0Jx6uD0i7ouIp0gC4qslTY2obJ+LiD+PiNMR8XRE3BURd0bEyYh4ENgN/Os2z78jIpYi4gRQA36kj8e+HLg3Ij5W3/ffgcN9nMujJMGaiFiOiD+rn9MTJF+O7c6DiLg9Ih6vf3m8C3gW8H19lMNKpEx5TCuHrzd+iYhj9cr9uqb9DzX9fhCYJqnpjkLzsZH0A8B/Ay4BZkn+H+5s8/yvN/1+jJXn1e1jn9dcjoiI1lRWl84HHgeQtJakRv8TwHPq+9e3e7KkN5NcaT0XCGAto/s72Ji4hm+jdmHT75tI0g+HgadIgi6QdHUENjY9dhjTura+xh8D9wHfFxHPAt4OaNWzhusx4ILGnXq66/xeXqD+3rwC+Gx905uBi4BL6+dxectTVpy3pB8H/iNJA/BzgHOBo+R/7jZmDvg2atdIuljSLPCbJKmPU8BXgbMl/VS94fTXgOac8jeAzZKG+ZldD3wbeErSD9I+fz8s/xP4UUmvqPcU2sHKL7ZMkqYlXQx8gCSdc2N913qSq4hvSpoj+eJq9g3ge5rurydpvzhMcoX1GyQ1fJtwDvg2arcD/4Mk5XE28EaAiPg2sB24FXiEpMbfnOr4cP3nEUn3DKksvwJcCzxJUtv/4JBeN1NEfAP4eeDdwBHge4G/BZ5p87SFes+ibwIfIwngWyKikTZ6N0lD9BHgb4BPtDz/RuC19d5I7wb2AZ8C/gE4ADxBcuVhE05eAMVGRdJngMWIuHXcZSmKenrmUeDfRcRnOz3ebBCu4ZuNmKSrJD273g3y10nSK18cc7GsAhzwzUbvJcCDJDn0q4Cfjoh2KR2zoXBKx8ysIlzDNzOriEINvNqwYUNs3rx53MUwMyuNu++++3BEdNW1t1ABf/PmzSwtLY27GGZmpSHpYLePdUrHzKwiHPDNzCrCAd/MrCIc8M3MKsIB38ysIhzwzcwqovQBv1aDzZthzZrkZ23VctVmZgYF64ffq1oNtm6FY8eS+wcPJvcBFhbGVy4zsyIqdQ1/584zwb7h2LFku5mZrVTqgH/oUG/bzcyqrNQBf9Om3rabmVVZqQP+rl0wO7ty2+xsst3MzFYqdcBfWIDdu2F+HqTk5+7dbrA1M0tT6l46kAR3B3gzs85KXcM3M7PuOeCbmVWEA76ZWUU44JuZjcmop4YpfaOtmVkZjWNqGNfwzczGYBxTwzjgm5mNwTimhnHANzMbg3FMDeOAb2Y2BuOYGsYB38xsDMYxNYx76ZiZjcmop4bJtYYv6ZclfUnSfZLeL+nsPI9nZlZURViONbeAL+l84I3Aloj4IWAKeE1exzMzK6pGn/uDByHiTJ/7UQf9vHP4ZwHnSDoLmAUezfl4ZmaFU5TlWHML+BHxCPB7wCHgMeDbEfHJ1sdJ2ippSdLS8vJyXsUxMxuboizHmmdK51zgVcBFwPOAtZKuaX1cROyOiC0RsWXjxo15FcfMbGyKshxrnimdK4GvRcRyRJwAPgq8OMfjmZkVUlGWY80z4B8C/qWkWUkCrgD253g8M7NCKspyrLn1w4+IOyXdAdwDnAT+Ftid1/HMzIqsCMux5jrwKiLeAbwjz2OYmVl3PLWCmVlFOOCbmVWEA76ZWUU44JuZVYQDvplZRTjgm5lVhAO+mVlFOOCbmVWEA76ZWUU44JuZVYQDvplZRTjgm5lVhAN+XREWGDYzy1Ous2WWRWOB4caak40FhmH805mamQ2La/gUZ4FhM7M8OeBTnAWGzczy5IBPcRYYNjPLkwM+xVlg2MwsTw74FGeBYTOzPLmXTl0RFhg2M8uTa/hmZhXhgG9mVhEO+GZmFeGAb2ZWEQ74ZmYV4YBvZlYRDvhmZhXhgG9mVhEO+GZmFeGAb2ZWEQ74ZmYV4YBvZtZkkpc79eRpZmZ1k77cqWv4ZmZ1k77cqQO+mVndpC93mmvAl/QcSXdI+oqk/ZJelOfxzMwGMenLneZdw78J+IuI+AHgBcD+nI9nZta3SV/uNLeAL+lZwI8BtwFExPGI+FZexzMzG9SkL3eaZy+d7wGWgfdKegFwN7AjIp7K8ZhmZgOZ5OVO80zpnAX8KPCeiHgh8BTw1tYHSdoqaUnS0vLyco7FMTOrtjwD/sPAwxFxZ/3+HSRfACtExO6I2BIRWzZu3JhjcczMqi23gB8RXwcekvT99U1XAF/O63hmZtZe3iNt/wNQkzQDPAhcn/PxzMwsQ64BPyLuBbbkeQwzM+uOR9qamVWEA76ZWUU44JuZVYQDvplZRTjgm5lVhAO+mVlFOOCbmVWEA76ZWUU44JuZVYQDvplZRTjgm5lVhAO+mVlFOOCbmVWEA76ZWUU44JuZVYQDvplZRTjgm5lVhAO+mU2UWg02b4Y1a5Kftdq4S1Qcea9pa2Y2MrUabN0Kx44l9w8eTO4DLCyMr1xF4Rq+mU2MnTvPBPuGY8eS7eaAb2YT5NCh3rZXjQO+mU2MTZt62141DvhmNjF27YLZ2ZXbZmeT7eaAb2YTZGEBdu+G+XmQkp+7d7vBtsG9dMxsoiwsOMBncQ3fzKwiHPDNbGJ5ENZKTumY2UTyIKzVOtbwJb1B0rmjKIyZ2bB4ENZq3aR0vhu4S9KHJF0lSXkXqqp8+Wk2PB6EtVrHgB8RvwY8H7gNuA74B0m/Lel7cy5bpTQuPw8ehIgzl58O+mb98SCs1bpqtI2IAL5ev50EzgXukPSuHMtWKb78NBsuD8JarZsc/hsl3Q28C/hr4J9FxDbgEuBncy5fZfjy02y4PAhrtW5q+BuAfxsRPxkRH46IEwARcRp4ea6lqxBffpp11ms718ICHDgAp08nP6sc7KG7HP7bI+Jgxr79wy9SNfny06w9t3MNzgOvCsKXn2btuZ1rcEraY3M8gDQFLAGPRETbFNCWLVtiaWkp1/KYWTmtWZPU7FtJScqmqiTdHRFbunnsKGr4OwCnfsxsIG7nGlyuAV/SBcBPAbfmeRwzm3xu5xpc3jX8G4E3A5kXXJK2SlqStLS8vJxzccrLo3Ct6tzONbjccviSXg5cHRHbJb0U+FXn8PuzfTvccsvK/OXsrD/sZlacHP5lwCslHQA+AFwuaTHH402kWm11sAf3TjCz3uUW8CPiP0fEBRGxGXgN8OmIuCav402qnTvTeyZA0g/Z6R0z65b74Rdcp6kVPPjEzLo1koAfEZ/plL+39IbZbrqcOb1jZt1wDb8gsoaNX3316q5oaTzJmpl14oBfEFnDxvftW9kVbWoq/fkefGJmnTjgF0RWDf1gfdq6xox/e/d68ImZ9ccBvyDa1dCbG2U9+MQmhQcTjp4DfkGkDRtvaG2UbZ3jG/yPY+XiqY7HI/fZMntR9ZG2tRpckzFSIWtGwMY/TnP+36Nwreg2bIAjR1Zvn58/U4mx7hRlpK31aGEh+cCnyUr5eI5wK5taLT3Yg3ub5c0Bv2DSUjvT03D0aHrKxmvhWtm0q4y4t1m+HPALprVRdm4u+XnkSHqu03OEW9m0q4w0epu5QTcfDvgF1Nwou24dHD++cn9zysZzhFvZZFVG5uaSz74bdPPjgF9wnVI27qZpZVGrJY21jbElzSR49auT390ulZ+zxl0Aa2/TpvR/kOZa0sKCA7wVW60G118PJ06k749IBhVedpnbpfLkGn7BOWVjZdbIxV9zTXawb2jU4t0ulR8H/IJzysbKqjkX361Dh1zJyZMHXplZLjZv7i3Yw5mBV7VaUts/dCip2e/a5UpOll4GXjmHb2a56DXn3lyLd7tUPpzSMbNc9JJzn5pyqnIUHPDNbGBpA6Wuvjr9sWe15BVmZ5MeOg72+XNKx8wG0jqBX2Og1DnnpD/+2c9OBhQ6Pz96DvhmNpCsgVKt2xoefxwOH86/XLaaUzoTyPOQ2Cj12jjr/vTj44A/YTwPiY1au7lx3J++WBzwJ4znIbFRyxooddNNq2d+PecceN3rfOU5Lg74E2bQeUicDrJOWj8jkD0avDHz6+23w9NPZ0/zbaPhkbYTJmt0YzdLx9Vq8PrXr5yOeWYG9uxxLwpL9Luk5iCfS2vPSxxWWLfzkKTV5HfsWD33/vHjyXYz6D9l6Bkwi8EBf8J0M9laVsNu1jqjWdutevoN3J4Bsxgc8CdQ84pZBw6svtTOqqWZddJv4PYMmMXggD/Bshpge72MnpsbdsmsDNI+P/0Gbk/zXQwO+BOqXX/8dv2mp6dXbpueTpaec8+dasn6/ED3gTutN0+7K08bgYgozO2SSy4JG475+YjkX3XlbX4+YnExYnZ25fbZ2WT74mLyGCn5uW1b9mNtcmV9fqamzvztWz8rzZ+Jdp8xGy5gKbqMsWMP8s03B/zhkdL/YaVkf6d/1sa+qansLw6bXFmfn0bg7lQRaFfhsOHqJeC7H/6E6rffc1o/6zRScmluk6nTalVTU3Dq1Ortjc/XmjVJiG/lz83wuR++9d24ltaDJ4270022tM9Ps7RgD2c6BLgbZjE54E+ofntFdNuDJ2txC5scWfPZQ1LDTxORXB1cfbW7YRaRUzq2woYN3Q208pD4ydEYZd3tADspCeyNn2lmZ+Haa2HfPi90krdCpHQkXSjpryTtl/QlSR6gX1CN7nNS9//0Bw+6i2ZZNXeX3LABrruut9HUjSDfCPppjh1Lgr27YRZLnitenQR+JSLukbQeuFvSX0bEl3M8pvWo20baNM19s/3PXA6tf+9Bp81olyDwPDnFk1sNPyIei4h76r8/CewHzs/reNafbhtps3iu/XIZ9O/dCzfQFs9IGm0lbQZeCNyZsm+rpCVJS8vLy6MojjUZRi3MNbnyyONv5ZWtyiP3gC9pHfAR4E0R8UTr/ojYHRFbImLLxo0b8y6OteilFpbVM6P1NWq1JDcsJbcNG5zrL4ph17rTVrbyPDnFlWvAlzRNEuxrEfHRPI9l/em2e+X8POzd27kmV6vB9devzA0fOZIsrOKgP367dmU3tLaam4Nt21YG8tb7rStbuYG24LodktvrDRDwp8CN3T7HUyuMXtYQ+NbpGLqZP6XT63lY/Xi0/s3a/a09/035UISpFSS9BPgs8PdAYzD12yJiX9Zz3A9/9LKGwLfq9mPS7vU8rH70tm+HW27p7u+3Zk3638djLoqtl374uXXLjIjPkdTyrcA2bWo/Zwok//DdOu+87K5+7rUxWrVa98Eesr+M3Sg/OTy1QsV1mjNFGl5vC0/HkK/W+ed37Og+2LfjL+rJ4YBfcY05d7JE9NYA9/jj2fv27nXDbV7SFizpdVCVu1dOPgd8Y2EhO23TSzoHkpROFg/Syk+vA6paVzZz98pqcMA3YHSLTLe2F2Stu2vttY516NQO0yAlXSvf+153r6ykbrvzjOLmbpnj1anLZTfarZSU1sWz26UW3TXwjMXFiJmZzt1pW29zc34fJxFF6JbZD3fLLIdaLUkhpE1722mlJEhG7J4+ndTq0xbSmJuDp59emaKYnXV6oaGb9ziNu1dOpkJMj2yTKa1x8HWvS/p7Q3c9cU6dSp6btWrSkSOr89HO/5/RbzfJQ4c87UXVuYZvPWlXu1y3Dp55Bk6cyOfYHriVaPc3aDSyp+2fm4Mnn4Tjx1dun55Ocvq+eion1/AtN+1ql0eP5hfsYWUPoCo39u7aBTMzq7dPTyf7shrgYXWwh+Rv5qunanDAt56MYhBOp8m9tm9P0kjNaaWtW6sT9BcWYM+epMbeMDd3ppaetp7xtde275fv0bTV4JSO9aRWS4Jtrx+bmZn02mWr2dns/uQS3H579vHdKJmum1XN/N6Vl1M6lpuFBbjhhu6n2IWk9rl+fefHTU0lQandvPs7d2Z/2ZSlltqp4bTXdFXr47dvXz3FQrtg30gFWQV0239zFDf3wy+PxcWkX3envt/btnU3BXOnW6N/frt+/mWYfjmrD/309JnxB71MUbxtW+exD+6bP9nooR/+2IN8880Bv3zaBf25uSSQ9RuMpqZWD7zK+vJoHtCVJm0g1zgGd3VaLyBrf9qXWacvwE63MnxBWmcO+DYyi4vpQX1mJmLt2v6DUSOIpx2vtQYsJTXddmVsfc709Oqa9igW+2gXoKXs/WnvxaBXTu3eMyuPXgK+c/g2kIWFpHdIa4+RPXvgqae6e42s9oCI1TnstB4ot98ON9+c7E/Lf6dNLHbixOpG5GPHknx3p/x5pxx7u/3tejlt2pS9f9Om1a/bz2jbZvsylyKyidXtN8Mobq7hT5Zua5pXXLG6Bt5a8260BbRLyWzbtvp1+plzpl2tv1OOvZv9/eTw085t0FvaVYOVD07pWBGsWdNd4Gk0HLZLUbSmOmZmVqeSBslnt7s157o75di7ycG3Nng3zr/5PZiaOvO8Tu9Np7JntbM4hz8ZHPCtEHoJTA15Be20Wy+1/0bg7ZRj7yUHH7EymLc+t/nKoNf3pZerDiu3XgK+c/iWm14XT4HRLqe3Z0/3ZWyM5s1a4KVR7nY5eFiZh9+wAa6//kwuPmLlc5onjOvlfWlduCSt3cMzj1ZUt98Mo7i5hj9Z0mqWabe5uTPPSetXnketf+3aM/n/Rvqk21u7mninOf57zcM3rgx6ea5r7tWCUzpWFM0Nq3Nzq4PrzEz7YCmlN+qm5fBHeWsE/eb+++1y8BH95eGbv5jm5pJb4/duvkBt8jngW2G1G+yUFRCnptJ76Qw6ynTQW/PAsLReNK1fCsMoa/PVRLvHWXX0EvA9eZoVxpo1SbhK07riVa2WzACZtYjKqEnZZYek/Oec037Gym5NTcHevXDNNdmPKdC/teXMk6dZKbVrmGxuwGzM/phnsJ+bSxb77rZRt1OAPXYMvvOd3iady3LqVHL+aXPiA6xdO/gxbDI54FthpC3c0awxG2bayNlhWVxMgvfhw3DZZWe2DyNQP/XU8Grex47ByZPp+84+ezjHsMlz1rgLYNbQSNdkpWoaVwB5ToO8devK3xtfLBGd0zad9vdCgssvh89/PvvLLWu5x8cfH04ZbPK4hm+FsrCQ5KfTluhrzNmeZ1/9Ruoo7SoiInuufilZJ6CfsQdpbrgBPvWppN0i65jt1g0wS+OAb4XTaaBQ1pqt27YN5/gHD2ZPTJbVbhCRTOC2a9dwgn5jYrN2X4Bbt7b/YjRbpdvuPKO4uVumdZI2r0yj22bEcBZb6dQVM217o+vlMCc4Szvv1u6s45jT34oFd8u0SdRubdY1a+Dcc4fT7THLzAz80i/Brbcm0yvnaWoqu1HWrFkv3TLdaGul0a53zunT+QZ7SALwe96T7zEaijK+wCaLc/hWGuNepDyrV0wehtX4a9bMAd9Koyq9T9zwanlxwLfS6DQwq1+Li8ltFKT2A6M8dbHlKdeAL+kqSfdLekDSW/M8lk2+RnfN5vVzh/W6CwvDf92GRn/5xvq7zzyT/jgJDhxwsLf85BbwJU0BfwS8DLgYeK2ki/M6nlXDwkIy7cHi4sp++o15b6TsAUmd5DFCdX4+aeyNOBPMOy2SYpaXPGv4lwIPRMSDEXEc+ADwqhyPZxWysJAE0NOnk58333zmftpApW50G3C7vRLIysVnDRxz3t7ylmfAPx94qOn+w/VtZrlqHanbLkA379u1q/0kafPzyZXF4cPd9aLJysV7yUEblzwDftq/zqpRXpK2SlqStLS8vJxjcaxKmq8ADh9On3ZhZgZuumnlc264YXXQn51NAn1zfr1TA/L8fPsA3nqF4mBvo5BnwH8YuLDp/gXAo60PiojdEbElIrZs3Lgxx+JYld188+q8/549qwPtzTcnDaudat/tGpCdnrGiym1qBUlnAV8FrgAeAe4CfiEivpT1HE+tYGVUqyWjgA8dStoBdu1yjd1GpxBTK0TESUlvAP43MAXsaRfszcqq0a3TrOhynUsnIvYB+/I8hpmZdccjbc3MKsIB38ysIhzwzcwqwgHfzKwiCrXilaRlIGM1UTYAh0dYnFGYxHOCyTwvn1N5TOJ5tTun+YjoahBToQJ+O5KWuu1rWhaTeE4wmeflcyqPSTyvYZ2TUzpmZhXhgG9mVhFlCvi7x12AHEziOcFknpfPqTwm8byGck6lyeGbmdlgylTDNzOzATjgm5lVRKkCvqTfkvR/Jd0r6ZOSnjfuMg1K0n+V9JX6ef2ZpOeMu0yDkvRzkr4k6bSkUnePk3SVpPslPSDpreMuzzBI2iPpHyXdN+6yDIukCyX9laT99c/ejnGXaVCSzpb0RUl/Vz+n/zLwa5Yphy/pWRHxRP33NwIXR8QNYy7WQCT9BPDp+nTSvwsQEW8Zc7EGIukHgdPAHwO/GhGlXORA0hTJmg7/hmRBn7uA10bEl8dasAFJ+jHgKPCnEfFD4y7PMEh6LvDciLhH0nrgbuCny/y3kiRgbUQclTQNfA7YERFf6Pc1S1XDbwT7urWkLJlYNhHxyYg4Wb/7BZKVwUotIvZHxP3jLscQXAo8EBEPRsRx4APAq8ZcpoFFxP8BHh93OYYpIh6LiHvqvz8J7Kfka2hH4mj97nT9NlDMK1XAB5C0S9JDwALw9nGXZ8heD3xi3IWw/+984KGm+w9T8iBSBZI2Ay8E7hxvSQYnaUrSvcA/An8ZEQOdU+ECvqRPSbov5fYqgIjYGREXAjXgDeMtbXc6nVP9MTuBkyTnVXjdnNMEUMq20l9VTjJJ64CPAG9qyQiUUkSciogfIbnyv1TSQCm4XFe86kdEXNnlQ98H/C/gHTkWZyg6nZOka4GXA1dESRpVevg7ldnDwIVN9y8AHh1TWayDep77I0AtIj467vIMU0R8S9JngKuAvhvbC1fDb0fS85vuvhL4yrjKMiySrgLeArwyIo6Nuzy2wl3A8yVdJGkGeA3w8TGXyVLUGzhvA/ZHxLvHXZ5hkLSx0WtP0jnAlQwY88rWS+cjwPeT9AA5CNwQEY+Mt1SDkfQA8F3AkfqmL0xAz6OfAf4A2Ah8C7g3In5yvKXqj6SrgRuBKWBPROwac5EGJun9wEtJptz9BvCOiLhtrIUakKSXAJ8F/p4kPgC8rb6udilJ+mFgL8lnbw3woYj4zYFes0wB38zM+leqlI6ZmfXPAd/MrCIc8M3MKsIB38ysIhzwzcwqwgHfzKwiHPDNzCrCAd8sg6R/Xl+n4GxJa+tzkk/EdMJWTR54ZdaGpHcCZwPnAA9HxO+MuUhmfXPAN2ujPofOXcB3gBdHxKkxF8msb07pmLV3HrAOWE9S0zcrLdfwzdqQ9HGSla4uIllCrxRrMJilKdx8+GZFIekXgZMR8b76+rZ/I+nyiPj0uMtm1g/X8M3MKsI5fDOzinDANzOrCAd8M7OKcMA3M6sIB3wzs4pwwDczqwgHfDOzivh/f/cUY/aMZXQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a12103150>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Create the original observations\n",
    "orig_X,_ = sklearn.datasets.make_regression(n_samples=num_samples,n_features=1,noise=5)\n",
    "poly = PolynomialFeatures(degree=2, include_bias=False)\n",
    "# Transform the features into second order polynomial features\n",
    "xx_ = poly.fit_transform(orig_X)\n",
    "\n",
    "# Extract the predictors and the values from the manufactured data\n",
    "X = [i[0] for i in xx_]\n",
    "Y_gt = [i[1] for i in xx_]\n",
    "noise = np.random.uniform(size=(len(Y_gt)))\n",
    "# Add some noise to the ground truth values\n",
    "Y_gt += noise\n",
    "\n",
    "# Shape the ground truth values for use with the model\n",
    "Y_gt = np.reshape(Y_gt,(-1,1))\n",
    "# Format the input features. Recall, we accomplish polynomial regression by\n",
    "#   including the original and the polynomial version of the predictors\n",
    "#   as features of the model\n",
    "X = np.hstack((np.array(X).reshape(-1,1),np.array(X).reshape(-1,1)**2))\n",
    "\n",
    "# Print a sample of the input data. X is the list of 2-feature input observations \n",
    "#   and Y is the list of ground truth values associated with each observation\n",
    "print(\"X Sample:\\n{}\".format(X[:5]))\n",
    "print(\"Y Sample:\\n{}\".format(Y_gt[:5]))\n",
    "\n",
    "# Plot the input data\n",
    "plt.scatter([i[0] for i in X],Y_gt,label=\"original data\",color='b')\n",
    "plt.xlabel(\"x\")\n",
    "plt.ylabel(\"y\")\n",
    "plt.title(\"Input Training Data\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create the Model\n",
    "\n",
    "#### Define the model architecture\n",
    "With our training data created and our second order polynomial assumption stated, we can now create a model to learn the regression line. We will use a 'FC' layer as the main component of the model. Since we desire two weights ($\\beta_2$ and $\\beta_1$), we set our input dimension to 2, and since we only expect a single quantitative result, our output dimension is 1. Note, when using an 'FC' layer it is implied that there is a bias, which we will use as our $\\beta_0.$\n",
    "\n",
    "Also, before continuing take a look at the protobuf created in this step. The first print out is of the 'net,' and contains the architecture of the model. At a glance, we see that as expected, there is a single op in the network that expects an input $X,$ a weight and bias, and outputs $y_{pred}.$ In the print out of the 'param_init_net,' we see that this is where the initializations for the weights and biases exist. This is an important observation that gives insight into how a model in Caffe2 is constructed and maintained."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "************* Predict Net *************\n",
      "name: \"regression_model\"\n",
      "op {\n",
      "  input: \"X\"\n",
      "  input: \"y_pred_w\"\n",
      "  input: \"y_pred_b\"\n",
      "  output: \"y_pred\"\n",
      "  name: \"\"\n",
      "  type: \"FC\"\n",
      "  arg {\n",
      "    name: \"use_cudnn\"\n",
      "    i: 1\n",
      "  }\n",
      "  arg {\n",
      "    name: \"order\"\n",
      "    s: \"NCHW\"\n",
      "  }\n",
      "  arg {\n",
      "    name: \"cudnn_exhaustive_search\"\n",
      "    i: 0\n",
      "  }\n",
      "}\n",
      "external_input: \"X\"\n",
      "external_input: \"y_pred_w\"\n",
      "external_input: \"y_pred_b\"\n",
      "\n",
      "\n",
      "************* Init Net *************\n",
      "name: \"regression_model_init\"\n",
      "op {\n",
      "  output: \"y_pred_w\"\n",
      "  name: \"\"\n",
      "  type: \"XavierFill\"\n",
      "  arg {\n",
      "    name: \"shape\"\n",
      "    ints: 1\n",
      "    ints: 2\n",
      "  }\n",
      "}\n",
      "op {\n",
      "  output: \"y_pred_b\"\n",
      "  name: \"\"\n",
      "  type: \"ConstantFill\"\n",
      "  arg {\n",
      "    name: \"shape\"\n",
      "    ints: 1\n",
      "  }\n",
      "}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Create the model helper object we will use to create the regression model\n",
    "regression_model = ModelHelper(name=\"regression_model\")\n",
    "\n",
    "# Add the FC layer, which is the main component of a linear regression model\n",
    "y_pred = brew.fc(regression_model,'X','y_pred', dim_in=2, dim_out=1)\n",
    "\n",
    "# Print the predict and init net to see what protobuf was created for this model\n",
    "print(\"************* Predict Net *************\")\n",
    "print(regression_model.net.Proto())\n",
    "print(\"\\n************* Init Net *************\")\n",
    "print(regression_model.param_init_net.Proto())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Add the training operators and prime the workspace\n",
    "\n",
    "In this **very important** step, we specify the loss function, setup the SGD training algorithm, prime and initialize the workspace, and initialize our model's weights and biases."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# The loss function is computed by a squared L2 distance, \n",
    "#   and then averaged over all items.\n",
    "dist = regression_model.SquaredL2Distance(['Y_gt', y_pred], \"dist\")\n",
    "loss = regression_model.AveragedLoss(dist, \"loss\")\n",
    "\n",
    "# Add the gradient operators and setup the SGD algorithm\n",
    "regression_model.AddGradientOperators([loss])\n",
    "optimizer.build_sgd(regression_model, base_learning_rate=learning_rate)\n",
    "\n",
    "# Prime the workspace with some data\n",
    "workspace.FeedBlob(\"Y_gt\",Y_gt.astype(np.float32))\n",
    "workspace.FeedBlob(\"X\",X.astype(np.float32))\n",
    "\n",
    "# Run the init net to prepare the workspace then create the net\n",
    "workspace.RunNetOnce(regression_model.param_init_net)\n",
    "workspace.CreateNet(regression_model.net)\n",
    "\n",
    "# Inject our desired initial weights and bias\n",
    "workspace.FeedBlob(\"y_pred_w\",np.array([initial_weights]).astype(np.float32))\n",
    "workspace.FeedBlob(\"y_pred_b\",np.array([0.]).astype(np.float32))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Run the training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training Complete\n"
     ]
    }
   ],
   "source": [
    "# Run the training for training_iters\n",
    "for i in range(training_iters):\n",
    "    workspace.RunNet(regression_model.net)\n",
    "\n",
    "print(\"Training Complete\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Extract Results\n",
    "\n",
    "Now that our model is trained, we can pull out the learned weights and biases which exist as blobs in the workspace named 'y_pred_w' and 'y_pred_b.'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best Fit Line: 1.0157*x^2 + 0.01501*x + 0.4665\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEaCAYAAAAWvzywAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xd4VFX6wPHvmxAgCUUNRVEJWFEQqaKCdRU7tkXlhwqIorCgoqKLWFBB115QUCwoJqIo6rorKtbFjoCCShGlCSgCKh2E5P39cSZhMkxNZnKnvJ/nmSfJzC3vvTN559xzzj1HVBVjjDGZI8vrAIwxxlQvS/zGGJNhLPEbY0yGscRvjDEZxhK/McZkGEv8xhiTYSzxG2NMhrHEb0waEJEjRORzEfmfiEwQkRyvYzLJyxK/MelhCXC8qh4DLATO9Dgek8RSMvGLyGIROcHrOPyJyPcicmyUyyZd/PESy3lI5X0mG1VdoaqbfX9uB0q9jMckN08Tvy8BbhaRDSKyUkTGiUgdL2OqLFVtqaofVXU7AefkVxF5NpXOSbzOQ6CA81L2aBK4z8p8qYrIQBGZLiJbReTZCMvuJiKvichGEVkiIv8Xy7ZE5CMR2eJ3DPP9XtsQ8CgRkVExHktz4BTgv7GsF2JbYY81luWreF6qdM4TJdbz41tnf99xFgV57QIRmevb3k8iclSUr4U8d6EkQ4n/DFWtA7QDOgI3eRxPMig7J22AtsDQeO9ARGrEe5vV4AxVreP3WBGn7a4ARgDPRLHsY8BfQGOgJzBGRFrGuK2BfsdwYNmT/sfm2/5m4OVoD0JE6gHPARep6l8Rlh0uIsMjbDLSscayfKXPSxRxxPL+RRTluYkmrlDrfBVknycCdwN9gLrA0bgqu7Cv+Ql17oJKhsQPgKouB94CWgGIyEG+b7I/fZfy3QLXEZEhIjIp4LlRIvKQ39+LReQ6EZktImtF5CURqR1pH771hvjW2ygiT4tIYxF5S0TWi8h7IrJrwPIn+H7/p+9beb2IzBGRsyt5Tn4F3sF9AZTtp4mITBKRVSKySESuDDj+diLytW/fL/uOd4RfjDeIyGxgo4jUiGJ7N4jIct/25ovI3yI8738ewr6H4d6bWJTtU0SeB5oC//GVfK6PZn1VfVVVXwfWRNhPPnAucLOqblDVT4A3gIti3VYU/g78Bnzst/97ROQ1v7/vFZH3RSTH90U+ARiuqhFLfJFEc6yxLF/Z8xKPcx7uvMUSS6xxBVnnAuBP4P0gL98G3K6qX6hqqaou9+XESK9VStIkfhHZGzgV+Nr3hvwHmAI0AgYBxSIS+E1WBJwsIrv4tlEDOB94PmC584CTgeZAa6B3lPs4FzgROAA4A/fFdCPQAHfuKiRJPz8BRwH1cW9akYjsEfXJ8BGRvXCX7T/6/s7yxTwL2BP4G3C1iJzke70m8BrwLLAbLhEEfun0AE4DdsHVA4fb3oHAQKCjqtYFTgIWh3o+IPZo38Od3ptYz1MZVb0IWMqOK4N7fLGMFpHRld2unwOAElX9we+5WUCkUl6gu0RktYh8KqHbJnoB47Xi8Ll3A8eJSBsRuQJ33s5R1W2497UTcIvvy/b8GGMKFOuxxuPcBDsv8dhuuPNWWTHFJe5q7Hbg2iCvZQMdgIYi8qOILBORR0UkN9xrAZuJ5jNVLhkS/+si8ifwCfA/4E7gcKAO8C9V/UtVP8DVWfbwX1FVfwGmAt19T50MrFbVGQH7eMTX+PU7Lhm1iXIfo1R1pe/b9WPgS1X9WlW34hJs22AHpKov+/ZXqqovAQuAw2I8J+uBn3Glvlt9z3cEGqrq7b6YFwJPAhf4Xj8cqOE73m2q+iowLci5+NnXEBhpeyVALeBgEclR1cWq+lOY5/1F9R4S/L0Jd17+9D1eD7NcBao6QFUHRLt8GHWAtQHPrcVdfkfrBmAf3BftWNzVyb7+C4hIU+AYXLVNOVVdAzwEjMdV/52qqmt9rz2vqg1U9Vjf46UYYgom1mOt6rkJdV6qfM7DnbcqiDWuO4CnVfXnIK81BnJwV3lHsaOK96YIr5WJ+JkKlAyJ/yxV3UVVC33/oJuBJsDPqurfM2EJ7sACPQdc6Pv9QnYu7QP86vf7JtybFs0+Vvr9vjnI30EbXUXkYhH5pixJ4aqvGgRbNoSzfCXpY4EWfusWAk38kt+fuCuQxr7XmwDLA0qJgR80/7/Dbk9VfwSuBoYDv4nIiyLSJNTzAfuJ9j0M9t6EUvZZ2UVVzwqzXKJsAOoFPFcPWB/tBlT1S1Vdr6pbVfU54FPcla6/i4FPVHVRkE18DRwCDA2RREISkf/6vc//BP7p994HNgbHeqxVOjdhzkuVz7lP2PMW47khlrhEpA1wAvBgiNjKemONUtVfVHU18ADu+MO9BkT9maogGRJ/MCuAvX1VG2WaAsHqtV4HWotIK+B0oDgB+4iaiBTiSs0DgQJV3QX4DpBYt6Wq/8NV29zne+pnYJFf8ttFVeuqatmb/Auwp4j472vvwM36/R5pe6jqC6raBfclobjL5pDP+0nI+Y1CImcW+gGoISL7+z13KPB9Fbap7PzZuJiA0j6AiBwCjPG9dknMO1I9vex9Bv6Fuxore99PD1g81mON97kpOy9V3m405y3Gc0OMcR0LNAOWisivwHXAuSIy07fvP4BlBPnshnstjGCfqQqSNfF/CWwErvc1XB2Lq2N/MXBBVd0CvAK8AExT1aXx3keM8nEnfhWAiPTB12BdSQ8BJ/pKDdOAdeIaVnNFJFtEWolIR9+yn+OqYQaKa7g9k/BVTGG3JyIHisjxIlIL2IIrfZSEej5g24k6v5GsxF32Rs13rmoD2UC2iNSWIL2eVHUj8Cpwu4jki0hn3I1Sz0ezLRHZRUROKntORHriemi847f+kbirogq9eURkT1xV2BXAAOCQaOpyKyuaY41l+cqelzic84SctxjPz1hgX1w1TRvgceBNXNtYmXHAIBFpJK7TyNXs6JIb8rVoPlOhDsCzB65B8IQQr7XE1fmvBeYAZ4daD+iCS7Z9Iu0DVz1RVIl9FOF6TJT9fSnwXrDlgZHA70DZZdn/gEujPO6dXsOVVib5fm+Ca7T9FfgD+CIgzg7AN7hL0ZdxH86bw2w75PZwja3TcJevv/s+bE1CPR/kPIQ8v5Hemxg/K/77PBPXwPsncJ3vuceBx8N8Dof7Pj/+D//3+i3gRt/vu+GuMjf69vN/0W4LaIjryrfeF98XwIkB6z8BPB/wXD1cw+GVfs9dB3xahf+94f7HGGKZSMdafl4iLV+V81LZc17Z8xbNuYnieCucmyDbLwp4LgcY7Tv+X4FHgNpRvBbxMxXsIb6VU5qvMWwesLuqrvM6nmQiIl/ikt44r2MxxiSHZK3qiZqvDvka4EVL+iAix4jI7r7Lvl640vnbXsdljEkeqXj3ZjlxN1GsxPUWOdnjcJLFgcBEXO+Yn4C/q+v2aowxAOlR1WOMMSZ6KV/VY4wxJjaW+I0xJsMkVR1/gwYNtFmzZl6HYYwxKWPGjBmrVbVhLOskVeJv1qwZ06dP9zoMY4xJGSKyJNZ1rKrHGGMyjCV+Y4zJMJb4jTEmwyRVHb9JHdu2bWPZsmVs2bLF61BMnNWuXZu99tqLnJxKT1BlkpwlflMpy5Yto27dujRr1oyKo0CbVKaqrFmzhmXLltG8eXOvwzEJYlU9plK2bNlCQUGBJf00IyIUFBTYlVyaS/nEX1wMzZpBVpb7WRztNCymyizppyd7X9NfSlf1FBdDv36waZP7e8kS9zdAz57exWWMMckspUv8w4btSPplNm1yzxvjhVtuuYX33nvP6zBCevbZZxk4cCAAr7/+OnPmzPE4IuOFlE78S0NMshjqeZOeVJXS0tLIC4axffv2uMRy++23c8IJJ8RlW2XiFVsgS/yZK6UTf9OmsT1v0sfixYs56KCDGDBgAO3atePnn39mypQpHHHEEbRr147u3buzYcMGACZPnkyLFi3o0qULV155Jaef7ubOHj58OP369aNr165cfPHFlJSUMGTIEDp27Ejr1q154oknAPjll184+uijadOmDa1ateLjjz+mpKSE3r1706pVKw455BAefPBBAHr37s0rr7wCwPvvv0/btm055JBDuOSSS9i6dSvghia59dZbadeuHYcccgjz5s3b6fieffZZunfvzhlnnEHXrl0BuPfee8tju/XWWwHYuHEjp512GoceeiitWrXipZdeKt/H6tWrAZg+fTrHHntshe1/9tlnvPHGGwwZMoQ2bdrw008/xe29MTG6/36o5vOf0nX8I0dWrOMHyMtzz5tqdPXV8M038d1mmzbw0ENhF5k/fz7jxo1j9OjRrF69mhEjRvDee++Rn5/P3XffzQMPPMD111/P5ZdfztSpU2nevDk9evSosI0ZM2bwySefkJuby9ixY6lfvz5fffUVW7dupXPnznTt2pVXX32Vk046iWHDhlFSUsKmTZv45ptvWL58Od999x0Af/75Z4Xtbtmyhd69e/P+++9zwAEHcPHFFzNmzBiuvvpqABo0aMDMmTMZPXo09913H0899dROx/f5558ze/ZsdtttN6ZMmcKCBQuYNm0aqkq3bt2YOnUqq1atokmTJrz55psArF27NqrTe+SRR9KtWzdOP/10/v73v0e1jkmAr76C666DWrXAVwVXHVK6xN+zJ4wdC70avcWz9KKwqTJ2rDXsZorCwkIOP/xwAL744gvmzJlD586dadOmDc899xxLlixh3rx57LPPPuV90gMTf7du3cjNzQVgypQpjB8/njZt2tCpUyfWrFnDggUL6NixI+PGjWP48OF8++231K1bl3322YeFCxcyaNAg3n77berVq1dhu/Pnz6d58+YccMABAPTq1YupU6eWv37OOecA0L59exYvXhz0+E488UR222238timTJlC27ZtadeuHfPmzWPBggUccsghvPfee9xwww18/PHH1K9fv4pn1VSr0aMhPx8uuqhad5vSJX5wSb7npmXQbzy9ii+DLl28DinzRCiZJ0p+fn7576rKiSeeyIQJEyos8/XXX8e0jVGjRnHSSSfttNzUqVN58803ueiiixgyZAgXX3wxs2bN4p133uGxxx5j4sSJPPPMMxW2FU6tWrUAyM7ODlmHHxjb0KFDufzyy3dabsaMGUyePJmhQ4fStWtXbrnlFmrUqFHe7mF98pPUmjXw4ovQuzdU8xd2Spf4y/3f/7kT99hjXkdiPHL44Yfz6aef8uOPPwKwadMmfvjhB1q0aMHChQvLS9VldeDBnHTSSYwZM4Zt27YB8MMPP7Bx40aWLFlCo0aNuOyyy+jbty8zZ85k9erVlJaWcu6553LHHXcwc+bMCttq0aIFixcvLo/n+eef55hjjqn08Z100kk888wz5e0Wy5cv57fffmPFihXk5eVx4YUXct1115XH0axZM2bMmAHApEmTgm6zbt26rF+/vtIxmSoaNw62bIEBA6p91ylf4gfcpVKfPi7x//or7L671xGZatawYUOeffZZevToUd6IOmLECA444ABGjx7NySefTIMGDTjssMNCbuPSSy9l8eLFtGvXDlWlYcOGvP7663z00Ufce++95OTkUKdOHcaPH8/y5cvp06dPean6rrvuqrCt2rVrM27cOLp378727dvp2LEjV1xxRaWPr2vXrsydO5cjjjgCgDp16lBUVMSPP/7IkCFDyMrKIicnhzFjxgBw66230rdvX+688046deoUdJsXXHABl112GY888givvPIK++67b6XjMzEqLYUxY+Coo+CQQ6p990k12XqHDh200hOxLFgABxwAt98ON98c38DMTubOnctBBx3kdRhR2bBhA3Xq1EFV+cc//sH+++/P4MGDvQ4rqaXS+5uS3n4bTjkFJkyACy6o0qZEZIaqdohlnfSo6gHYf3/o2hWeeAIS1O/ZpKYnn3ySNm3a0LJlS9auXRu0ntyYavXYY9C4MRO2nuPJkDPpk/gB/vEPWL4c/v1vryMxSWTw4MF88803zJkzh+LiYvLy8rwOyWSyxYvhzTf59vDLuHRATZYsAdUdQ85UR/JPr8R/2mlQWGiNvMaY5DV6NGRlcdn0yz0bcia9En92NlxxBXz4IXz/vdfRGGNMRZs2wVNPwdlnM23FXkEXqY4hZ9Ir8QNceqm7C+7RR72OxBhjKpowAf74AwYN8nTImfRL/A0auH7948dDwG30xhjjGVUYNcp13zzqKEaOdEPM+KuuIWfSL/EDDBrkLqnGjfM6EpMETj311J3G0glUleGUP/roo/KB38I59thjidRd+aGHHmJTYMWvSQ+ffgqzZrn8JFI+5ExhIYi4n9U15Ex6Jv62baFzZ1fdU1LidTTGI2XDNU+ePJlddtkl7LKJGE65Mizxp7FRo2CXXVyNhE/Pnq6TT2mp+1ld44wlNPGLyGAR+V5EvhORCSJSO5H7q2DQIFi4EN56q9p2aUJLxBSZDzzwAK1ataJVq1Y85BsvKNhwzf5DFN9xxx20aNGCE088kR49enDfffcBFYdTDjVs8rRp0zjyyCNp27YtRx55JPPnzw8b3+bNm7ngggto3bo1559/Pps3by5/rX///nTo0IGWLVuWD7H8yCOPsGLFCo477jiOO+64kMuZFLR8OaWTXmVsaV+y6uZ7P02sqibkAewJLAJyfX9PBHqHW6d9+/YaN3/9pdqkiWrXrvHbpik3Z86cqJctKlLNy1N1lZzukZfnnq+s6dOna6tWrXTDhg26fv16Pfjgg3XmzJm6aNEiFRH9/PPPy5ctLCzUVatW6VdffaWHHnqobtq0SdetW6f77bef3nvvvaqq2qtXL3355ZfLl3/kkUdUVfWxxx7Tvn37qqrq2rVrddu2baqq+u677+o555yjqqoffvihnnbaaTvFeP/992ufPn1UVXXWrFmanZ2tX331laqqrlmzRlVVt2/frsccc4zOmjWrQqxlQi2XaLG8vyayb88cpiWINuenuP0PlAGma4z5OdFVPTWAXBGpAeQBKxK8vx1yclzXzilTIMhEF6b6JGKKzE8++YSzzz6b/Px86tSpwznnnMPHH38MVByuOXCdM888k9zcXOrWrcsZZ5wRcvvBhk1eu3Yt3bt3p1WrVgwePJjvI3QZnjp1KhdeeCEArVu3pnXr1uWvTZw4kXbt2tG2bVu+//77kDNhRbucSWJbtrDHf57gDbqxiH3Kn/ZymtiEJX5VXQ7cBywFfgHWquqUwOVEpJ+ITBeR6atWrYpvEJdfDjVruro145lETJGpYcaY8h/OONp1AgUbNvnmm2/muOOO47vvvuM///lPVMMdi8hOzy1atIj77ruP999/n9mzZ3PaaacF3Va0y5kk98ILFJSu5mGu2uklr6aJTVjiF5FdgTOB5kATIF9ELgxcTlXHqmoHVe3QsGHD+AbRqJFrSHn2Wdd31ngiEf2Vjz76aF5//XU2bdrExo0bee211zjqqKPCrtOlS5fyhL1hw4byWauitXbtWvbcc0/ATY0YTYzFvorc7777jtmzZwOwbt068vPzqV+/PitXruQtv3Yo/6GSwy1nUoQqPPww83Ja8RHH7vSyV9PEJrKq5wRgkaquUtVtwKvAkQncX3BXXeWuqZ5+utp3bZxE9Fdu164dvXv35rDDDqNTp05ceumltG3bNuw6HTt2pFu3bhx66KGcc845dOjQIaYZq66//nqGDh1K586dKYmit1j//v3ZsGEDrVu35p577ikfEvrQQw+lbdu2tGzZkksuuYTOnTuXr9OvXz9OOeUUjjvuuLDLmRTxv//B7Nn8efFV5OVVvPrzdJrYWBsFon0AnYDvcXX7AjwHDAq3Tlwbd/0dc4xq06aqvoY5U3WxNv4VFakWFqqKuJ/xaNSqjPXr16uq6saNG7V9+/Y6Y8YMbwJJcta4GydnnaVaUKC6aVPC/geoRONuwiZiUdUvReQVYCawHfgaGJuo/YV11VVwzjlu1M5zz/UkhEzXs2dyzIXcr18/5syZw5YtW+jVqxft2rXzOiSTrhYtgjfegBtugNzcpPkfgATPwKWqtwLedz7u1s11Hn/oIUv8Ge6FF17wOgSTKUaNcjeueDC1YiTpeeduoOxsd0PXJ5+Abx5SU3WaRLO3mfix9zUO1q1zo3Cedx7sFXwUTi9lRuIH6NsX6tSBBx/0OpK0ULt2bdasWWNJIs2oKmvWrKF27eq7yT4tPf00rF8PSTrFZ/rMuRuNwYPd+D2LFiXlt3Aq2bZtG8uWLbN+5Wmodu3a7LXXXuTk5HgdSmoqKYH99nM5xndTYSJVZs7dhNbxJ50rr4RHHnHJ/1//8jqalJaTk0Pz5s29DsOY5PP6627Etfvv9zqSkDKnqgegeXM4+2w3IfuGDV5HY4xJRw884HLNmWd6HUlImZX4Aa65xk3QEsWdl8YYE5Np0+Czz1wX8uxsr6MJKfMS/xFHQKdOrmunjdVvjImn+++HevWgTx+vIwkr8xK/iCv1//STu7nCGGPiYdEieOUVNypwvXpeRxNW5iV+cHfxNmsGvkk4jDGmyh56yN2wNWiQ15FElJmJv0YN17Xzs8/cwxhjquKPP1zf/R49UqKreGYmfoBLLoFdd03qLlfGmBTxxBOwcSNce63XkUQlcxN/nTrQvz+89hosWOB1NMaYVLV1q7s/6MQT4dBDvY4mKpmb+AEGDnRTND74YEImAzfGZIAJE+CXX1KmtA+ZNmRDMJdeyvbxxexbYylLN++YASwvD8aOTZ5hVI0xSai0FA45xLUbfvON6zVYzSozZENml/gBrr2WGtu2cMnmRys87eVEyMaYFPHmmzBnDlx/vSdJv7Is8R90EP+mGwN5lDw2VnjJq4mQjTEp4p573MS5553ndSQxscQPPNf4Bgr4nb5UnJfXq4mQjTEp4LPP3Bwf117r2gpTiCV+4Nz7j+TTrC5cy/3UYBvg8UTIxpjkd++9sNtubq6PFGOJH9eAu23wDRSylAt4icJCa9g1xoQxb56bw3vgQMjP9zqamFmvnjKlpdC6tevPOWtWSjXUGGOq2SWXwIsvwpIl0LBh5OUTyHr1VEVWlmuZ//ZbeOstr6MxxiSrn3+G55+HSy/1POlXliV+fz16uBbdu+7yOhJjTLIqG+bluuu8jaMKLPH7y8mBIUNcS301zJVpjEkxq1fDk0+6BsAU7vZniT9Q377QqBHceafXkRhjks0jj8DmzXDDDV5HUiWW+APl5rohm99+G2bO9DoaY0yyWLcORo2Cs86Cgw7yOpoqscQfTP/+bgYdq+s3xpR54gk3X/fQoV5HUmWW+IOpX9/1z500yfXXNcZkts2b3Yx9J5wAHTt6HU2VWeIP5aqroHZtK/UbY+Cpp+C33+Dmm72OJC4s8YfSqBFcfrkbmH/hQq+jMcZ4ZetWNxjbUUfB0Ud7HU1cWOIPZ8gQN872v/7ldSTGGK+MHw/LlsFNN3kdSdxY4g+nSRPXvfPZZ22MZmMy0fbtrrq3Y0c3tWKasMQfyfXXg6obic8Yk/b8p2EdvPsEWLTIlfbTaPwuS/yRFBZCr17ubr1ffvE6GmNMAhUXQ79+buw10RKuWDOCb6U1xWtP9zq0uLLEH42hQ90ln5X6jUlrw4a5aVcBLuBFDuQHhustDLs5vVKlDcscrd69YeJE18Nn9929jsYYkwBZWa5mN4sSvqcl28jhUGaBZFFa6nV0wSXdsMwisouIvCIi80Rkrogckcj9JdRNN8Fff1mp35g0Vjbu2nlMpAXzuZ1bULJSeTy2oBJ9/fIw8LaqtgAOBeYmeH+Js99+bkS+MWNg5UqvozHGJMDIkVAnt4SbuYPvaMkkzk3LaVgTlvhFpB5wNLgZzFX1L1X9M1H7qxY33eRu5rBSvzFpqWdPeKvvKxzMXO7gFpoWZqXlNKwJq+MXkTbAWGAOrrQ/A7hKVTeGWiep6/jL9OoFL7/sung1bux1NMaYeCopgUMOcV03v/3WVfonuWSr468BtAPGqGpbYCPwz8CFRKSfiEwXkemrVq1KYDhxUlbqv+ceryMxxsTbSy/B3LkwfHhKJP3KSmSJf3fgC1Vt5vv7KOCfqnpaqHVSosQPrtQ/cSL89JO7u9cYk/q2b4eWLaFWLfjmm5RJ/ElV4lfVX4GfReRA31N/w1X7pL5bboFt22wMH2PSyQsvwA8/wG23pUzSr6xEH90goFhEZgNtgPSYz3DffaFPHzcxw88/ex2NMaaqtm2D22+Htm3dDFtpLqGJX1W/UdUOqtpaVc9S1T8Sub9qddNN7k6PdOvnZUwmev55V3V7221pNSZPKOl9PZNIhYVuUI+nn3Y9fIwxqWnrVlfa79gRTk+vMXlCscRfFTfe6Mbrv/12ryMxxlTWk0+6UdlGjMiI0j5Y4q+aJk1gwAA3UcPc1L0p2ZiMtXGjS/jHHJNW4+1HYom/qoYOhbw819PHGJNaHn3UDcEycmTGlPbBEn/VNWgA114Lr7wCM2Z4HY0xJlp//gl33w2nngqdO3sdTbWyxB8P11wDBQVuMG9jTGp44AH44w9X1ZNhLPHHQ716rsrnnXfgf//zOhpjTCQrV7rE372767ufYSzxx8uAAa6xd+hQ17/fGJO8RoyALVsysrQPlvjjJzfXDez0+efwxhteR2OMCeWnn9xd95deCgcc4HU0nrDEH099+sCBB+6Yo9cYk3xuvtndf5PBPfEs8cdTjRpw112uT/+zz3odjTEm0Ndfw4QJcPXVGT2yrk22Hm+qcOSRsHQpLFjg+vgbY5LDySfDtGmwcCHssovX0cRFUg3LnLFEXN/gFSvg4Ye9jsYYU+a991zPu2HD0ibpV5aV+BPljDNg6lTXkNSggdfRGJPZSkuhfXt309a8eW6ylTRhJf5kcvfdsGED3HGH15EYY4qL3axaI0emVdKvLEv8iXLwwXDZZTB6tKvrN8Z4Y/NmV73Tvj1ccIHX0SQFS/yJNHy4K10MHep1JMZkrkcecTPl3Xdf2k+pGC07C4m0++5www0waRJ8+qnX0RiTEYqLoVkzl+Pb7b2Kv267002wcuyxXoeWNCzxJ9o118Aee7gRPJOoId2YdFRc7CbGW7LE/bv1XTbCsVOjAAAft0lEQVScrM0b+c9R93gdWlKxxJ9o+fmuQenLL+HFF72Oxpi0NmwYbNrkfj+IOVzOE4yhP4NGH+RtYEnGunNWh9JS6NABVq92Xcnspi5jEiIra8eF9WRO4XC+YD9+5A8poLTU29gSxbpzJqusLHjoIdfAdP/9XkdjTNpq2tT9PIm3OYW3uYOb+Z2C8ueNY4m/uhx9NJx7LvzrX7B8udfRGJOWRo6EurnbuZ9rWcB+PMpA8vLc82YHS/zV6Z573KidN97odSTGpKWePeH97mNoyRyu516aFNZk7Fj3vNnBEn912mcfGDwYxo93jb3GmPhavZqOb9wCJ5zAa6VnsnixJf1gLPFXt2HDXP/+K68kbVubjPHKLbfA+vWuTU3E62iSliX+6la3rqvymTbNlfyNMfExa5abWWvAAGjZEqh4M1ezZu5vY905vVFaCl26uDHB58+H+vW9jsiY1KYKxx8P337rxsbaddfym7nK+vWD60mdbnX+CenOKSIDRWTXyodldpKV5cYP+e03G73TmHiYOBE++sj9P+3q0pX/zVxlNm1yz2e6aKp6dge+EpGJInKyiFWcxUWHDnDJJW6ylrlzAbssNaZSNmxwQ6K0a+eK+D5LlwZfPNTzmSRi4lfVm4D9gaeB3sACEblTRPZNcGzp7667oE4dGDiQ4iKtMMbIkiXuM2zJ35gI7rjD3Rvz6KOQnV3+dKibtuxmrigbd9U1BPzqe2wHdgVeEREb+agqGjaEO++EDz7g88ET7bLUmFjNnQsPPOCuno84osJLI0fuPDqK3czlRGzcFZErgV7AauAp4HVV3SYiWcACVY1byT9jGnf9lZRAp04sn/ELLZjHBupWeFnEen0aE5QqdO0K06fDDz+4glSA4mJXeFq61JX0R45Mr4ZdSNxYPQ2Ac1T1JFV9WVW3AahqKXB6JeI0/rKzYfRo9uAXbuW2nV62y1JjdvBvBxvYaKKbQH3EiKBJH1ySX7zYFZ7sZq4doqnjv0VVl4R4bW78Q8pAhx3GT8ddytU8RCu+LX/aLkuN2cF/rP26upZhq69mZlZ7Xqh3hdehpRy7gStJ7P/yXWyvuyvP1rqcLEopLEy//sbGVIV/98wR3EQjfuOy0ie48ebs8CuanSQ88YtItoh8LSL/TfS+UlpBAbUfvZ/2Wz+n5Imn7LLUmABl3TDbM51/8BijGcBM2lv3zEqojhL/VYBVCUXjoovcvKA33OBu7jLGlGvaFLIo4XGuYCWNuYkR5c+b2CQ08YvIXsBpuN5AJhIRGDMGNm50N6QYY8qNHAnX5oyiAzMYzIOso761g1VSokv8DwHXAyE7JIpIPxGZLiLTV61aleBwUkCLFvDPf0JREbz7bvnTdlevyXQ9uyxhZNZNfFj7FCZyvrWDVUHCBmkTkdOBU1V1gIgcC1ynqmG7f2ZkP/5gtmyBQw+Fbdvg228ZMCSfxx/fMZcopOdgU8aEpAqnn+7G45kzBwoLvY4oaSTbnLudgW4ishh4ETheRIoSuL/0Ubs2PPkkLFrEnPNu3Snpg93VazLMxIkwebLrs29Jv8oSlvhVdaiq7qWqzYALgA9U9cJE7S/tHH00XH45B05+kHYa/CpoyRKr9jEZ4Pff3cRFHTq4n6bKrB9/Mrv7blbSmKfpSw22BV3EBnMzae+aa1zyf/LJCoOwmcqrlsSvqh9Fqt83QRpw/1uf4Q1HcyizuYG7Q65n1T4mbb31Fjz3nOvw0KaN19GkDZuBK0mEmi2oVy84buwFnFnyKu2Yyfe0Crq+DeZm0s66dW4KxXr1YOZMqFXL64iSUrI17poYhJotaPJkkFGjWJe1C+PoQ82s7UHXt5tYTNq5/npYsQKeecaSfpxZ4k8SoW47X7IEttZrSIMJj9KR6XzR/X4bY9ykvw8+cBOnDx4MnTp5HU3ascSfJMKV2Pv1g+K/usM559D2tVt48dY5FBa66h27icWkqpA3Ja5bB336wP77w+23exhh+rLEnySCzRZUZtMmGHaTwOjRULcuZ7zci8ULtpWPMQ52V69JLf5DLO801eg118CyZa5RN9Q/hakSS/xJomdPV3IPZelSoHFjePxxN+PQXXcBEf6BjElSV10VvE3r3Wsmw9NPw5AhO02laOLHevUkmWbNXPIOVFi4o3RPz57uTsYvvqDZue0jL29MEikuhguD3Mq5K7/zHa1o0nI3mDHDGnSjZL160kCwKp+cHNiwYUdVzsvHPAqNGsHFF7NyyZag27Exyk2yCn7PiTKaATRkFYwfb0k/wSzxJ5myKp+yxtuCAvdzzZodVTm9B+/KBz2fhjlzeKRu8Du3rHunSVbBCiX/xwtcwEt83304tGtno9EmmqomzaN9+/ZqKiosVHUpv+KjsFBV//EPVdDTar1b4bW8PNWiIo8DNyaEwM90UxbrH9TXz2t0Vt2+XYuK3GfYPtPRAaZrjLnWSvxJLlSVzdKlwD33QIsWTMzrxaF7rbHunSapFRdDgwYV27CyKOE5epFNCW+e/zxkZ4e8mdGGJYmfGl4HYMJr2jR4Y2/TprjGgBdeIK9TJ745/nJ4+WVXL2RMkikudl3ztwWMNXgd93Es/6M343j5tea0KI5Q2DFxYSX+JBessbfCnbpt27oxyidNcre2G5NEyurqL7xw56TfkWmM4CZe5u88R6/yUn2o9ilrt4ofS/xJLrCxN2hVznXXwfHHw6BBMNfmtTfJwf8ek0B1WccEerCCJlzGk4C7Ul26NIrCjqky68efLlascMPW7rEHfPEF5OZ6HZHJcKHuSQEYz0X8Hy9wNFP5jM7lz5fdf1Jc7Er/S5e6kv7IkdZuFYr1489kTZq4W9xnz3ZXAMZ4LFSd/EWM5yKKGM7wCknfv1Tfs6f7AigblsSSfnxZ4k8np5wC117rxvR59VWvozEZLlid/EHMYQz9+YhjuJMby5/PzrbeaNXJEn+6ufNOOOww14Xip5+8jsZkiGA3XJ16asVl8tjIy3RnA3XowQRKcdMo5uW5i1VL+tXHEn+6qVkTXnrJFaG6d4ctwYd0MCZeQg0UOHGi/1JuSIaDmEv/usXUKtzD7jvxkPXjT0fNmrnxTs44w01kMWaM1xGZNBbqhiv/5/owjl6M5zZu4fUNJ1C6rnpjNBVZiT8NFRdDs4Gnczc3wOOP8+kAG+jEJE6kG6vaMpPRDOA9/sbt3GL98ZOAJf4043/ZPYwR/I+jaTvmMt68c5bXoZk0FSqRFxTAnrm/M4lzWUVDejCB2nnZ1h8/CVjiTzP+l90l1OA8JvIHu3LIrWfD7797G5xJS6FuuHr4wVKm7d+TJqzg70xCCxqSmwsXXWQjbnrNEn+aCbzs/o3G/J1X2H37MteCVlISdn0bDtdEEvgZgRB3l/94G01mv02txx/hyqLD2Ly54vDiNlOch2IdzjORDxuWuepCDeN8425j3C/DhoVct6hItWbNiuvVrGnD4Zodoh4yedIk92KfPqqlpeGHFzdVQiWGZfY82fs/LPFXXch/zOdLVS+5xD3x0ktaVOT+6UTcz6Ii1YKC4P+cBQVeH5VJFlEl8NmzVfPzVQ8/XHXLFlV1n7Ng64l4cRTppTKJ38bqSUMhxznZuhWOP57t07/m6KxP+XxL2/J18vJ27pLnL4k+JsZDWVnBPwsibngF1qyBjh3dZ236dDd2FFHOJW0qxcbqMUCYcU5q1YJXX+W3kgJe3HImjVhZvk64pG9MmbBDJv/1l7tpcMUKeO218qQPNuJmsrHEn8aCNtQ2bswZJf+mAat5nbOozeaI2ykoSHSkJhkF+/yETOAjFAYMgA8/hKeecsOG+IlqeHFTbSzxp6lQt9EXF8OawnZcxPMcwReMow9CKeASfE5Oxe3k5MB551lPn0wT6vMDIRL4r/fD00/DTTe5WVcI3vvHRtxMErE2CiTyYY278ROuEa6sAXgId6uC3sGw8p4ZgY2+/fvbxNeZKNTnJzt7x3tf9lk5k9e1BNHFnc5TLSkpf80+N9UD69VjykTqRVFUpFrYtFSf5FJV0M/6jStf1z/5Z2eH/gIx6SvU56csgZcVCDrypW4kV7/gMN0td1N5Yrfum9WnMonfevWkqah7UWzb5sbP/egj+O9/KV59Ev36RW7sLe/FYdJSuNmzwA3+2qzkRz7jSNZTlyP4nFU0Kv98Rez9Y+LGevWYclH3osjJcRO1t2wJ557LhOtmRNXDxwbaSm/BPj/+di1ZxVucQhalnMJbrKIRsOPOcZswPblZ4k9TMfWiqFcPJk+GBg14+tdTac7CiNsPnGTDpJ9Q0zbns4E3OZ29WMYZ/IcFHFD+mqq7Wjj1VOu+mcysqsfsMG8evx/cmd91V7rwCSvZPeSiduNN+iguhquucvdeRVKTrfyHM/gb73Mur/Jvzgy6XF4e9OrlyhM2YXpiJVVVj4jsLSIfishcEfleRK5K1L5M1ZR1u5ODWnCqvske/MI7nMQu/BFynSVLrGtnqvLvZtmgAfTuHV3Sz6KE57mIrrzLpTzFvzkTkeDLbtrkkr5130xSsbYGR/sA9gDa+X6vC/wAHBxuHevVU/2Cdbs7gSm6hZr6KUdoHhtC9u6wLnqpJ9j7Hd2jVB+nnyroNdwX1To2Dk/1oBK9ehJW4lfVX1R1pu/39cBcYM9E7c9UTrBp897jRHowgU58yWucTS1Cz9u7aZPbhkkNwd7vyJQHuIbLGctd/JMHuDaqtawhN3lVS+OuiDQD2gJfBnmtn4hMF5Hpq1atqo5wjJ9Q0+a9xjn05Wm68i6TOJeabI15Gyb5xP5eKSMZxmAe4mGu5Ebu3GmJggJryE01CU/8IlIHmARcrao7TbGsqmNVtYOqdmjYsGGiwzEBwpXKnqM3/XiC05jMRM6jdtZfUW2juNjVHYu4R4MG1haQLGIthd/ECG7kLp6gH1fzEFCxUj8vDx5+2MbhSTmx1g3F8gBygHeAa6JZ3ur4q1///pHragfwqCroko7naL3cv8LW8RcVqebk7LwNm9AlORQVhb8r1/9xV+5tqqCv5F+sWZSUD+EROI+D8RbJNGQDrmgwHngo2nUs8Ve/ULfWBzbSfXXhg6qgS9t10/2bbgn5jx9ue3a7vjcCx1+K1FgPpXobN6uCPp/dS4vHb/f6EEwYyZb4uwAKzAa+8T1ODbeOJf7qF23pT1VVR41yf5xyiurmzTFvz3p5VL/+/aN/j7OyXNIfwY2qoE9xiYqvpG+SV2USf40EViF9QmCFoEk6TZuGH5MFXJ0tAAMHQs2acMUVcMYZbrKNOnUqLLvbbqH7hFsvj+pVXAyPPx58zJxgtLSUh7maKxnFWC7jCh5HybLG+zRkQzZkuEhjsogE9M7o1w/GjYMPPoATToDff496XzbMQ2IFjn9/1VXRJ/1stjOOPlzJKO7nGi7nCcp6e9sXdhqK9RIhkQ+r6vFGUVEU1TyBXn3Vtdi2bKm6bFn505GG87XGwMSo/I1ZqrXZpK9yliro8Jw7FErtPUshJNMNXCZ19OzpV50TINTznH02vP22qyfq0gXmzQNcVU8odrNX4sR6Y1bZTGu78jtT6MrZvM5XFz/CfuNuorBQrFtmmrPEb4BKToZ93HFujtVNm+DII+HjjyPuJ7A9Iei8wCaiwHslIrXTlBGB/v1dbd2Rey7hE7pwGNP4eOBLdHxuED172vg6GSHWS4REPqyqx1uB3f6ivsRfuFD1wANVa9bU83kxYtdQ/6n7Qk3PV+lYMkBRkatli7VKp6DA7zx+9ZXqHnuo1q+v+tFHnh6PqRqSqTtnZR6W+FND0KS8erVqly6qoDdxe4V64sBHdnb4aR0LCmy+1nCiufci7H0UL7+smpvrnvj2W+8OxMSFJX6TcMFK6SKuv7hu3qxfHHCRKugEztdcNlYqQUVMXBku2n75O11tUapfdx+hCvopR2hDVla8CjApqTKJ3+r4TUyCNSKqwpgxULdhbY5a+BzXczfnMZGpHM3exK8TuPUnd8J1rywsDN4gX4f1vFbzPNq8fBNF9OR4PmAVjVizBvr0sbaVTGOJ38QkXPLdsAG2bRfu5Xq68Qb7s4CZtOME3o3Lvv17DGVyo/DIke4+ukA5Oe61wIb6A5jPNOnE6X+9ynXcy0U8z1Zql7++bZv1tso4sV4iJPJhVT3JL5b65f2Zr7NppSWI3sgIFUqiq5IIUZVRUOBiCDYMQaa1ARQVufMRtOFWd7TD/J2Xda3U03W1G+hxvB/2nJvUhNXxm0SLZXRHUM1jgxbxf6qg73CiNuaX8MuHuQmprEdQqP1bG4CfzZvLh15dtW8nPaD2Ems/SVOVSfxW1WNi0rOnG6on1FyrgTaRz1W7FXFN/hMcxcfM4lC68k7QZbOzXftBdnbwbTVt6qokVIO/niptAJHmK4i1Gitw+dt7zGVuvU4wZgyP1xtCqz8+5octoRsGyqqITAaJ9ZsikQ8r8aeOwKqGUI+y8dtB9WC+09m0UgV9mEEx9fopq8oJd7WRCqXWUH3wc3J23L8QS1dW/2ovoUSv4kHdTC39jQZ6Cm9GPK/Wqyf1YVU9prqFS/4FBTtPylKbTfoQV6qCzmd/PZzPQq5f1t/f/wauUG0M/jeGBRPs3gMvbhKLNF9BqNeDfan5fxE2ZbG+x/GqoG9wesQqtVT5ojSRWeI31S7cjFv5+aGTzrF8oIso1O1k6f0M1nzWB03mwfYX8j6CMDEGrpOTs3PJuzoaiCPNVxDq9WDnorBQNYvtOoiHdT35up587cuTGu7mucCrMZP6LPEbT4TqYRIp8dRlrY7mClXQxTTVU/lv0FJpYDIOV1IP9losPZEKCiJfBUS6Ugj3elVK/IHbbcVs/YzDVUEnc7I2ZXHUx2kl/vRhid8klWgT0JXtP9Hv5WBV0Nc4U5vz004l8WBzvQYmwv79dy7ZV2ZMm3BXAZHq4KN5vTJ1/P7HVp8/9GEG6XaydBUF2pPnoy7lR7qKMKnHEr9JKm4qv8iPggLVF57dqnfvcqeuJ1+3UFNHMlTrsK5CkgpM6IFVTJUdyiCWknGkOvho6uhDXSH5X52UjWPkX9LPZpv243FdSUPdTpY+Rn/djdURYw/VDmMl/vRgid8klViSa5k9WabjuVAVdCUNdSCPaA5bE5LQY7kaKEvAkergY6mjV62Y7EPelFZaqmfyus6hhSroVLpoG2aGjTeWqxCT2izxm6QSS9164DqH8YV+wLGqoAtppr15RmvwV1wTf6z1/3l5kUvPkUr8/tVTwXo9VXyUaq9Gk1U7dVIFncuB2o3XNVK1TqztDia1WeI3SSXa6QDLhmJQDRyOoVS78rbOoK0q6CIK9XLGaC02Vznp5+fvSIShhocO9Qg3XESkOQaiOR9CiZ7Bv/ULDnNPNG2qX1zyhNbN3Rb1F5rJHJb4TdIJLOEGJtmaNcMnTRHVvx1fqmfX+q9+jiv5/kojvSPrFt2rRuS+6ol6lCV//9JzuDp61chXF7ls1CsYrfPZX8u+6P5Rc6zu13Rr+fkrKNhxLqP5IjXpzxK/SXqV6eqYne3r1dO0VI/nfX0393QtFdFtWTlaTA89nveiHgAung//G8yC9SgK/HIIVf/fitn6CAP1D+qrgn5JRz2PF4NWbflfXYSLzWSOyiR+ceslhw4dOuj06dO9DsN4JCvLpa1g8vIqTvz97/t/ZMmQUVyk49mVP1lEM8ZzMS9xPnM5uPqC9hEJHTu4+HNzYc0a9/cerOB8XqIHEziMr9hCLV7h7zzOFXxKZyD0YEjZ2fDcc3DhhaH3l0T/1ibBRGSGqnaIaR1L/CZZNGsWftLwwkI3AXhxMfTr5wZ0q8UWzuY1+vI0x/MBWSjf0ZJJnMtkTmU6HSglxKhvYRQUwHnnweTJ0U9kHkmb3Pn8bfN/OYM3OIqPyUKZQTuKuJDxXMzvFES9rbw82L4d/vpr59fy893cCCYzWOI3Kc0/oQcjAqWlob8gGvMr5zKJ83mJznxKNqWsogHvcQJTOZqpHM1cDiLcoLRFRTuuKoqL3WigS5ZELtHvTNmbnzmKjzmWjziOD9mPnwCYRWte42wm0IMfODCWjVaQleXOR6CCAli9utKbNSnGEr9JecXF0KsXlJTs/FpZiT9clVCZ3VhDV6ZwGm9yPB/QhF8A+INd+IY2fEMbZtOaBezPj+zHShoDUl6lBDt/CYVK/vVYy/4soAXzOZB5tGMmHZhOY34r3+dUjmYKXfkvp7OUwojnQQSOPx4+/zz0F2G4dYN9IZj0ZInfpIVgJX//Ov5IVUI7U/ZhIUfxMZ34krZ8TWtmk8fm8iU2kcsv7MFKGrMhtxEbs+rw28Z8NpMLQBalZFNCXdlAXV3HLvzJHvzCniynLjvqVUrIYg4HM4P2zKA9n9CF2bSOubqpf38YPTr8F2F2dvgvSJMZLPGbtFFWzbJ0qZuAZeTIilUwwb4YevVyk75HI4sS9mEh+/IT+/EjzVnE7vxKY1bSiN/IZyP5bCSXzSiCIpSSxXrqso56rKU+v7I7y9mT5ezJj+zHa98fyIRp+zJ0eK0qtwv4J+9wx/vcc6G/IE1mqEzi97wLp//DunOaSILdbVvW3VM1tjtxK9uFM9jzZV02o7lBK9pHsOMO7AZrd+QarDunSWfhGn+zsmDXXXd0l0yEmjWhb1946inYti1x+wFXjbN9e2L3YdJDZUr8NRIVjDHxNmxY6IbO0tLEJn1wiTjaqqSqClZ3b0y82GTrJmV4PZl6dfaUKYzc8ceYSrPEb1JG06ZeR1A98vJcY7YxiWKJ36SMkSNdUoy3oiL3qA4iULt26NcLC61Xjkm8hCZ+ETlZROaLyI8i8s9E7sukv549XVIsiH5kg6i327Nn/LdbJtvXhb+wEJ5/HrZuDb6ciOvCaUnfJFrCEr+IZAOPAacABwM9RKT6R88yaaVnTzccQVGRS6Qi7mf//jv+zo59aB4Afv89vrGCi2n7dtdBsyyph6qyypSqLOO9RJb4DwN+VNWFqvoX8CJwZgL3ZzJIz54ukZaWup+jR+/4+7nnKlclFG3ijfbKIFRdfbAqK6vXN9UpkYl/T+Bnv7+X+Z4zJqHKqoTKrgDCJWr/10aOdMuHUljorjRWr46u102ouvrA+Kxe31S3RCb+YP9CO90tJiL9RGS6iExftWpVAsMxmcT/imD1alcVFKhmTXj44YrrXHHFzsk/L88lfP/690gNzYWF4RN54BWLJX1TnRKZ+JcBe/v9vRewInAhVR2rqh1UtUPDhg0TGI7JZKNH79wu8MwzOyfc0aNdA2yk0ni4hmartjHJLmFDNohIDeAH4G/AcuAr4P9U9ftQ69iQDSYVhRtQzphES6ohG1R1u4gMBN4BsoFnwiV9Y1JVWXdQY1JFQsfqUdXJwORE7sMYY0xs7M5dY4zJMJb4jTEmw1jiN8aYDGOJ3xhjMkxSzcAlIquAULOVNgBWV2M41SEdjwnS87jsmFJHOh5XuGMqVNWYboJKqsQfjohMj7WvarJLx2OC9DwuO6bUkY7HFe9jsqoeY4zJMJb4jTEmw6RS4h/rdQAJkI7HBOl5XHZMqSMdjyuux5QydfzGGGPiI5VK/MYYY+LAEr8xxmSYlEr8InKHiMwWkW9EZIqINPE6pqoSkXtFZJ7vuF4TkV28jqmqRKS7iHwvIqUiktLd6kTkZBGZLyI/isg/vY4nHkTkGRH5TUS+8zqWeBGRvUXkQxGZ6/vsXeV1TFUlIrVFZJqIzPId021x23Yq1fGLSD1VXef7/UrgYFW9wuOwqkREugIf+IaxvhtAVW/wOKwqEZGDgFLgCeA6VU3JSRZEJBs3p8SJuImFvgJ6qOocTwOrIhE5GtgAjFfVVl7HEw8isgewh6rOFJG6wAzgrFR+r0REgHxV3SAiOcAnwFWq+kVVt51SJf6ypO+TT5CpHFONqk5R1e2+P7/AzVSW0lR1rqrO9zqOODgM+FFVF6rqX8CLwJkex1RlqjoV+N3rOOJJVX9R1Zm+39cDc0nxOb7V2eD7M8f3iEvOS6nEDyAiI0XkZ6AncIvX8cTZJcBbXgdhyu0J/Oz39zJSPJlkAhFpBrQFvvQ2kqoTkWwR+Qb4DXhXVeNyTEmX+EXkPRH5LsjjTABVHaaqewPFwEBvo41OpGPyLTMM2I47rqQXzTGlAQnyXMpfZaYzEakDTAKuDqghSEmqWqKqbXA1AYeJSFyq5hI6A1dlqOoJUS76AvAmcGsCw4mLSMckIr2A04G/aYo0usTwPqWyZcDefn/vBazwKBYTga8efBJQrKqveh1PPKnqnyLyEXAyUOVG+aQr8YcjIvv7/dkNmOdVLPEiIicDNwDdVHWT1/GYCr4C9heR5iJSE7gAeMPjmEwQvobQp4G5qvqA1/HEg4g0LOvlJyK5wAnEKeelWq+eScCBuB4jS4ArVHW5t1FVjYj8CNQC1vie+iINeiqdDYwCGgJ/At+o6kneRlU5InIq8BCQDTyjqiM9DqnKRGQCcCxuqN+VwK2q+rSnQVWRiHQBPga+xeUHgBt9836nJBFpDTyH++xlARNV9fa4bDuVEr8xxpiqS6mqHmOMMVVnid8YYzKMJX5jjMkwlviNMSbDWOI3xpgMY4nfGGMyjCV+Y4zJMJb4jQlBRDr65kmoLSL5vjHR02IYY5PZ7AYuY8IQkRFAbSAXWKaqd3kckjFVZonfmDB8Y/R8BWwBjlTVEo9DMqbKrKrHmPB2A+oAdXElf2NSnpX4jQlDRN7AzbzVHDe1X0rMAWFMOEk3Hr8xyUJELga2q+oLvvl3PxOR41X1A69jM6YqrMRvjDEZxur4jTEmw1jiN8aYDGOJ3xhjMowlfmOMyTCW+I0xJsNY4jfGmAxjid8YYzLM/wO/yHan3duh8AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a1af2f9d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Extract the learned coes and intercept from the workspace\n",
    "coes = workspace.FetchBlob(\"y_pred_w\")[0]\n",
    "intercept = workspace.FetchBlob(\"y_pred_b\")\n",
    "\n",
    "# Calculate the regression line for plotting\n",
    "x_vals = np.linspace(orig_X.min(), orig_X.max(),100)\n",
    "regression_result = intercept[0] + coes[0]*x_vals + coes[1]*(x_vals**2)\n",
    "print(\"Best Fit Line: {}*x^2 + {}*x + {}\".format(round(coes[1],5), round(coes[0],5), round(intercept[0],5)))\n",
    "\n",
    "# Plot the results of the regression\n",
    "plt.scatter([i[0] for i in X],Y_gt,label=\"original data\",color='b')\n",
    "plt.plot(x_vals,regression_result,label=\"regression result\",color='r')\n",
    "plt.legend()\n",
    "plt.xlabel(\"x\")\n",
    "plt.ylabel(\"y\")\n",
    "plt.title(\"Polynomial Regression Fit: ${{{}}}x^2 + {{{}}}x + {{{}}}$\".format(round(coes[1],5), round(coes[0],5), round(intercept[0],5)))\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part II: Express Linear Regression Example\n",
    "\n",
    "The above example shows you how to create a polynomial regression model that is easily adapted to handle higher order polynomials. Now, we will consider the baseline case where we desire a simple first order model, with 1-D input $x,$ 1-D output $y,$ and a solution of the form:\n",
    "\n",
    "$$y = \\beta_1x + \\beta_0$$\n",
    "\n",
    "The structure of Part II will be similar to Part I. First, we will generate the dataset, then we will construct the model and specify the training routine, and finally we will train and extract our results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEYCAYAAABcGYHrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XucTfX6wPHPQ+6ji6Gj0szodlzGfSSUciq6USmnhNyvldQvyVFRqU6RpHItFNOpKE6iiHJExAy5jUiYIhVTYcZ95vn9sfaMuew9s+dm7b3neb9e+8XsvdZ3fddi1rO/l/V8RVUxxhhjSrldAWOMMYHBAoIxxhjAAoIxxhgPCwjGGGMACwjGGGM8LCAYY4wBLCAYY4zxsIBgjDEGsIBgipiIbBGR60L1eMaEMgsIAUZEdovIURFJFpFfRWSGiIS5XS9/qWpdVV1W1OV6rssNZ+p4PupQRUTmikiKiCSKyH25bJuc7ZUqIq9n+nyWiOwTkUMisl1Eeudj3wdFJE5EjovIDB/Hv1dEtnrq+qOIXFNEl8HX+fp9bfLaPq/zE5FlInIs0/XZlu1zr+cuIuVE5G3P8Q6LyHoRubmILkFIsIAQmNqpahjQEGgEDCvqA4jIWUVdZgnwJnAC+BvQGZgoInW9baiqYekvz/ZHgdmZNnkRiFLVs4H2wCgRaeLnvr8Ao4Bp3o4tIjcCLwE9gMpAK2BnwU4ZRGSkiIzMYzO/r40f2+d6fh4PZrpOf89U19zO/SzgZ+Ba4BzgKeBDEYnK49xKDAsIAUxVfwUW4QQGAETkQhH5SET2i8guERmU6bPGnm89h0Vktoh8ICKjMn2+W0SGishGIEVEzsqjvKEistdT3jYRuT639zMd4wbP32t7vs395enaaZ9tu8dEZKOIHPTUtXx+r1H2lkNu5eZ2rn4cpxJwF/CUqiar6grgE6CrH7vfDfwOfJ3+hqpuUdXj6T96Xpf6ue/HqjoPSPJxvGeAZ1V1taqmqepeVd3rOY+XRWRupvMaLSJLRaSMH+fhVX6vTV7b+3F+ufF57qqaoqojVXW357NPgV1AkwIcJyRZQAhgIlIDuBnY4fm5FDAf2ABcBFwPDBaRtiJSFpgLzACqAP8B7vRSbCfgVuBcIC2X8v4OPAg0VdXKQFtgt6/3vdS9jKfsxcD5wENArGf/dP8EbgJqAvWB7vm9Rj7kKDe3a5epzhNEZIKPMq8AUlV1e6b3NgC5fQtO1w14V7NlkvQc7wjwPbAPWOjvvr6ISGkgBqgmIjtEZI+IvCEiFTybvAS0FpGGItIf5zp1UNWT/pTvQ36vTWGuZboXReSAiKwUzxiSH+eehYj8zVOXLfk4bkizgBCY5onIYZzm7e/ACM/7TYFqqvqsqp5Q1Z3AVOBe4CqcJvF4VT2pqh8Da7yUPV5Vf1bVo3mUlwqUA+qISBnPt6ofc3k/u6uAMODfnrK/BD7FCUiZ6/KLqv6Bc7Nu6KWcgvBWbm7nCoCqDlTVgT7KDAMOZnvvIE63hE8iEoHTRfFO9s88x6oMXAN8DBz3d99c/A0og9OyuIbT3Y5Peo6ZBIwD3sXpirxFVbOfV37l99oU6FpmMhS4BCewTwHmi8il5HHumXm+sMQC76jq934eN+RZQAhMd3i+fV8H1AKqet6PBC70dMH8JSJ/Af/C+UW4ENib7Zvkz17Kzvyez/JUdQcwGBgJ/C4i74vIhb7e93KcC4GfVTUt03uJOL/E6X7N9PcjODeKouCt3NyunT+SgbOzvXc2cDiP/e4HVqjqLm8fqmqqp8ukBjAgP/v6cNTz5+uquk9VDwBjgVsybbMeqAcMU1Vv/0cQkU8zXacngCcyXbtPs22e32tT0GsJgKp+q6qHVfW4qr4DrMQ5P3/OPb2lPRNnDONBf45ZUlhACGCq+j+cLqAxnrd+Bnap6rmZXpVV9RacLoeLREQyFXGxt2Iz/T238lDV91T1apybqeJ0N/h8P5tfgIs9v3zpIoC9+boIRSfXc/XDduAsEbk803sNyLu74X78+4Z/FjnHEPzdN4Oq/gnsIeu/cwYRqQdM9JTbM5dybku/TsC/cVp66dfttmyb5/faFPRa+qwuIHmdO4Dn9+NtnC8CdxWyqyzkWEAIfOOAG0WkIU4X0CHPoG4FESktItEi0hRYhdOd86BnsPh24Mo8yvZZnoj8XUT+ISLlgGM4375Sfb3vpexvgRTgcREp4+nnbQe8X4hrUUZEymd65WemVG7XLk+qmoLTrfOsiFQSkZbA7TjfNL0SkRY4LaLZ2d4/X5ypkWGeerTF6Ur7Mq99PZ+d5RkoLw2U9nItpgMPeY5zHk6L7lMRuQinC60/MBCoJ0XwDEd+r01e2+d2fiJyrmeMq7xnu844M4kW5XbumQ4/EaiNM5PvKCYrVbVXAL1wBmhvyPbeROAjz98vxBkw/hX4E1idvj3OgNp3OE3y2Ti/dE/lUbbX8nAGY9fgNOP/wPmlutDX+96OgTNI+D+c/uEE4E5fdcHpgpqVx3XRbK9RXsrxWW5u187z+SRgUi51qALMwwl0PwH3ZfrsM+Bf2bafDMz0Uk41z3X5CzgEbAL6+LNvpnPKfi1GZvq8DDDBU/6vwHicLpkNwKBM2z0GrPTj/+TIzOXn99p4uz55XEuf5+e5dms9///+8vwb3pjHuZf3fJbeoj2G8zuS/urs9u99oLzEc6FMCBKRb3FucNPdrosxJvBZl1EIEZFrRaS6pyndDefb/Odu18sYExzsadXQ8nfgQ5xZNT8Cd6vqPnerZIwJFtZlZIwxBrAuI2OMMR5B1WVUtWpVjYqKcrsaxhgTVOLj4w+oarW8tguqgBAVFUVcXJzb1TDGmKAiIon+bGddRsYYYwALCMYYYzwsIBhjjAGCbAzBm5MnT7Jnzx6OHTvmdlVMEStfvjw1atSgTJkCr91ijMmHoA8Ie/bsoXLlykRFRZE10acJZqpKUlISe/bsoWbNmm5Xx5gSIei7jI4dO0Z4eLgFgxAjIoSHh1vLz5gzKOgDAmDBIETZv6sxZ1ZIBARjjAlZSUkweDAcLOxKp3mzgBCinn76aZYsWeJ2NXyaMWMGDz7orF44b948EhISXK6RMQFGFWbPhjp14M03YfnyYj+kBYQipKqkpaXlvWEuTp06VSR1efbZZ7nhhhuKpKx0RVW37CwgGJPNL79Ahw7wz3/CxRdDfDy0a1fsh7WAUEi7d++mdu3aDBw4kMaNG/Pzzz+zePFimjdvTuPGjenYsSPJyckALFy4kFq1anH11VczaNAgbrvNWZp25MiR9O3blzZt2nD//feTmprKkCFDaNq0KfXr12fy5MkA7Nu3j1atWtGwYUOio6P5+uuvSU1NpXv37kRHR1OvXj1effVVALp3786cOXMAWLp0KY0aNaJevXr07NmT48ePA04qkBEjRtC4cWPq1avH999/n+P8ZsyYQceOHWnXrh1t2rQBYPTo0Rl1GzFiBAApKSnceuutNGjQgOjoaD744IOMYxw4cACAuLg4rrvuuizlf/PNN3zyyScMGTKEhg0b8uOPPxbZv40xQUcV3n7baRV8/jm8/DKsXg3165+Rwwf9tNMsBg+G774r2jIbNoRx43LdZNu2bUyfPp0JEyZw4MABRo0axZIlS6hUqRIvvfQSY8eO5fHHH6dfv34sX76cmjVr0qlTpyxlxMfHs2LFCipUqMCUKVM455xzWLt2LcePH6dly5a0adOGjz/+mLZt2zJ8+HBSU1M5cuQI3333HXv37mXz5s0A/PXXX1nKPXbsGN27d2fp0qVcccUV3H///UycOJHBgwcDULVqVdatW8eECRMYM2YMb731Vo7zW7VqFRs3bqRKlSosXryYH374gTVr1qCqtG/fnuXLl7N//34uvPBCFixYAMBBP/s7W7RoQfv27bntttu4++67/drHmJC0cyf07QtLl0KrVvDWW3D55We0CtZCKAKRkZFcddVVAKxevZqEhARatmxJw4YNeeedd0hMTOT777/nkksuyZhTnz0gtG/fngoVKgCwePFi3n33XRo2bEizZs1ISkrihx9+oGnTpkyfPp2RI0eyadMmKleuzCWXXMLOnTt56KGH+Pzzzzn77LOzlLtt2zZq1qzJFVdcAUC3bt1YnqkvskOHDgA0adKE3bt3ez2/G2+8kSpVqmTUbfHixTRq1IjGjRvz/fff88MPP1CvXj2WLFnC0KFD+frrrznnnHMKeVWNKSFSU50vnfXqwZo1MHEifPXVGQ8GEGothDy+yReXSpUqZfxdVbnxxhv5z3/+k2Wb9evX56uM119/nbZt2+bYbvny5SxYsICuXbsyZMgQ7r//fjZs2MCiRYt48803+fDDD5k2bVqWsnJTrlw5AEqXLu1zjCB73YYNG0a/fv1ybBcfH8/ChQsZNmwYbdq04emnn+ass87KGFexZwqMySYhAXr1crqFbrkFJk1yxgxcYi2EInbVVVexcuVKduzYAcCRI0fYvn07tWrVYufOnRnfwtP72L1p27YtEydO5OTJkwBs376dlJQUEhMTOf/88+nTpw+9evVi3bp1HDhwgLS0NO666y6ee+451q1bl6WsWrVqsXv37oz6zJw5k2uvvbbA59e2bVumTZuWMS6yd+9efv/9d3755RcqVqxIly5deOyxxzLqERUVRXx8PAAfffSR1zIrV67M4cOHC1wnY4LOiRPw3HPQqBH88APExsKnn7oaDMDFFoKIXAy8C1QH0oApqvqaW/UpKtWqVWPGjBl06tQpY/B21KhRXHHFFUyYMIGbbrqJqlWrcuWVV/oso3fv3uzevZvGjRujqlSrVo158+axbNkyRo8eTZkyZQgLC+Pdd99l79699OjRI+Nb+IsvvpilrPLlyzN9+nQ6duzIqVOnaNq0Kf379y/w+bVp04atW7fSvHlzAMLCwpg1axY7duxgyJAhlCpVijJlyjBx4kQARowYQa9evXjhhRdo1qyZ1zLvvfde+vTpw/jx45kzZw6XXnppgetnTMBbu9ZpFWzaBPfeC+PHQ7U81645I1xbU1lELgAuUNV1IlIZiAfuUFWf8w9jYmI0+wI5W7dupXbt2sVb2SKSnJxMWFgYqsoDDzzA5ZdfziOPPOJ2tQJaMP37GpOrI0dg5Eh45RWoXt0ZK2jf/owcWkTiVTUmr+1c6zJS1X2qus7z98PAVuAit+pzJkydOpWGDRtSt25dDh486LUf3hgTgv73P2jQAEaPdloHCQlnLBjkR0AMKotIFNAI+NbLZ32BvgARERFntF5F7ZFHHrEWgTElycGDMHQoTJ4Ml1ziTCn9xz/crpVPrg8qi0gY8BEwWFUPZf9cVaeoaoyqxlQLkH42Y4zJ04IFULcuTJ0K//d/zphBAAcDcDkgiEgZnGAQq6ofu1kXY4wpEvv3Q+fOcNttcN55sGoVjBkDFSu6XbM8uRYQxMlt/DawVVXHulUPY4zxJjYWoqKgVCnnz9jYPHZQhfffd9JOzJ7tDCDHx0MuMwoDjZtjCC2BrsAmEUnPN/EvVV3oYp2MMYbYWCeLxJEjzs+Jic7P4Hz5z2HvXhgwAObPdwLA229DdPQZq29RcXOW0QpVFVWtr6oNPa+QDga33HJLjlxD2RUmbfWyZcsyEubl5rrrriP79N3sxo0bx5H03wZjSpjhw08Hg3RHjjjvZ5GWBlOmOK2CJUucKaXffBOUwQACZJZRqFNVVJWFC/OOd88+++wZqFHexo0bR5cuXagYBP2exhS1n37y4/0dO6BPH1i2DFq3dgaPg/yhStdnGZ1p+e4X9MPYsWOJjo4mOjqacZ58St7SYmdOBf3cc89Rq1YtbrzxRjp16sSYMWOArGmrfaWnXrNmDS1atKBRo0a0aNGCbdu25Vq/o0ePcu+991K/fn3uuecejh49mvHZgAEDiImJoW7duhmprMePH88vv/xC69atad26tc/tjAlVvma4R0TgJKN75RUnJfW6dU4gWLo06IMBcPrbazC8mjRpotklJCTkeM+XWbNUK1ZUdUZ/nFfFis77BRUXF6fR0dGanJyshw8f1jp16ui6det0165dKiK6atWqjG0jIyN1//79unbtWm3QoIEeOXJEDx06pJdddpmOHj1aVVW7deums2fPzth+/Pjxqqr65ptvaq9evVRV9eDBg3ry5ElVVf3iiy+0Q4cOqqr61Vdf6a233pqjjq+88or26NFDVVU3bNigpUuX1rVr16qqalJSkqqqnjp1Sq+99lrdsGFDlrqm87VdccvPv68xRcXXvWL+i5tUmzZ13mjXTnXPHrer6hcgTv24x5aoFoLf/YL5sGLFCu68804qVapEWFgYHTp04OuvvwaypsXOvs/tt99OhQoVqFy5Mu1yWQnJW3rqgwcP0rFjR6Kjo3nkkUfYsmVLrnVcvnw5Xbp0AaB+/frUz7TYxocffkjjxo1p1KgRW7Zs8blymb/bGRMKOnd2hgYiI0EELo84zqq2I7nt6cawe7czm+i//4WLQiu5QokKCH71C+aT5pILKnPaaH/3yc5beuqnnnqK1q1bs3nzZubPn+9XWmlnlm9Wu3btYsyYMSxdupSNGzdy6623ei3L3+2MCSWdOzv3/rRV37K9chPqz32GuWX/SbX9CUQNvYfY93L+TgW7EhUQcu0XLKBWrVoxb948jhw5QkpKCnPnzuWaa67JdZ+rr74640aenJycscqYvw4ePMhFnm8mM2bM8KuOsZ7Bks2bN7Nx40YADh06RKVKlTjnnHP47bff+OyzzzL2yZySOrftjAlZKSnw6KPQvDkp+w7SodyndEiZxQGqZkxDLYoxyEBSogLC88/nfFiwYkXn/YJq3Lgx3bt358orr6RZs2b07t2bRo0a5bpP06ZNad++PQ0aNKBDhw7ExMTka4Wxxx9/nGHDhtGyZUtSU1Pz3H7AgAEkJydTv359Xn755YzU2w0aNKBRo0bUrVuXnj170rJly4x9+vbty80330zr1q1z3c6YkPTll86g8auvQv/+NKu0hbnHb82ySWG7mwOSPwMNgfIq7KCyqjNYFBmpKuL8WZgB5cI4fPiwqqqmpKRokyZNND4+3p2KBDgbVDZn1J9/qvbu7QwaX3aZ6rJlqurcLzIPMKe/RFyur5/wc1C5xD2H0LmzjycNz7C+ffuSkJDAsWPH6NatG40bN3a7SsaUbJ984jxt/Ouv8PjjTuoJzzrnERHO08rZBXkC5hxKXEAIFO+9957bVTDGAPz+OwwaBB984Cx0/9//QkzWtWSefz5rKgsofHdzIAqJMQR1adU3U7zs39UUK1WYNQtq14a5c501juPicgQDyDkNNTLS+TkQehuKUtAHhPLly5OUlGQ3jxCjqiQlJVG+fHm3q2KCjF/ZCH7+2UlP3bUrXHEFrF8PTz4JZcv6LDdjGmqa82eoBQMIgS6jGjVqsGfPHvbv3+92VUwRK1++PDVq1HC7GiaI5JmlNC3NWb1s6FAnBcW4cfDgg1C6tGt1DiQSTN+sY2JiNK8sncaYkisqyvvgb2Qk7F68HXr3hq+/hhtucPp8atY843V0g4jEq2rOvrBsgr6FYIwx6bxlHSjNKe5JHAsNRkC5cs5aBT16OIMBJougH0Mwxpj0cYPsHR712cC3NOMlhsJNN0FCAvTsacHABwsIxpiglj5ukLmrqCzHeZaniCOGGuxhapvZ8PHHxH51YZGnvw8l1mVkjAlq2bMYN+cb3qI3ddjKu3TlEV6l8rZwKr6Xz2UxSyBrIRhjglr6uEElkhnHw6zgaiqRwk18Rjfe5Q/C+emn4kl/H2qshWCMCWoREXB54hdMoS812c0bPMAwXiSZylm2KY7096HGWgjGmOD1558sjezJF7ThBGW5huU8xBtZgkF6ioniSH8faiwgGGOC09y5UKcOl658l83thtE+YgMr5RrCwyE8PGeKieJIfx9qrMvIGBNcfv0VHnoI5syBhg1hwQKiGzdmWx67pQ8cDx/udBNFRDjBwAaUT7MWgjEmOKjCO+9AnTowfz688AKsWQOZUsfnlceoJOQjKgxrIRhjAl9iIvTrB4sWQYsWztPGtWpl2STPPEYmT9ZCMMYErrQ0eOMNqFsXVqyA1193chFlCwZg00qLgrUQjDGBads26NULVq6Etm2dLKWRkT43t2mlhWctBGNMYDl5El58ERo0cHIPzZgBn32WazAAm1ZaFCwgGGMCx/r1cOWV8K9/Qbt2TkDo1s2vZHQ2rbTwLCAYY9x37BgMGwZNm8K+ffDRRzB7NlSv7ncRJWWZy+JkYwjGGNfExsLc/1vB87/14u9s58dWPbh03itw3nkFKq9zZwsAhWEtBGPMGZH5GYGqVaF6pcP82eVB5vx2DWU5QRsWUT9uGrELCxYMTOFZQDDGFJhfC9qTdc0CVWiStIhvj0QzkAm8xiDqsYkvaGPTRF1mXUbGmALJz4Ng6c8InMcfvMojdONdtlKLq1nBKlpk2damibrHWgjGmHyLjXUm/3h7EOzhh3O2Gn5KVO5iDlupzX28xyiG04j1OYIB2DRRN1kLwRiTL+ktg9RU758nJTkvcFoNT/fZxydlH+C2E3OJpzFtWcQGGnrd16aJusvVFoKITBOR30Vks5v1MMb4z1uKCO+U7kwn7mgdrj/xGU+WeYlmfOszGISH2zRRt7ndZTQDuMnlOhhj8sGfPv4odrGYNkynJ5uoR0M2UHv649SIPAsRcqxZMGsWHDhgwcBtrnYZqepyEYlysw7GmPyJiHC6grIrXRqqnJPKvX+8yYsMI41SDGACk+lHRGQpe0YgCLjdQsiTiPQVkTgRidu/f7/b1TGmxPOVImLuC1vZEn4N43mY/3EtddnCJAZQoWIpGxcIEgEfEFR1iqrGqGpMtWrV3K6OMSVe9hQRl0ac5JtbRtHuqYZUS9rGyv4zGRixgD0SYekjgozNMjLG5FtG9098PPTsCXM2wj33wPjxtDz/fHa7XUFTIAHfQjDGBKCjR2HoUCcz6f79MG8evP8+nH++2zUzheD2tNP/AKuAv4vIHhHp5WZ9jDF+WL7cWavg5ZehZ08+HJlA1MO355m+wgQ+t2cZdXLz+MaYfDh0CJ54AiZOhJo1YckSYn+93tYxDiHWZWSMydvChRAdDZMmwSOPwKZNcP31to5xiLGAYEwJExvrpJ8WcV5Vq+bSzXPgAHTtCrfeCpUrwzffwNixUKkSYOsYhxqbZWRMCRIbCz16OMsWp0tKciYKpRs+3ElGN7Dqh4w58RDlj/wJI0Y4K5qVK5elPF8PqVmCuuBkLQRjQkhe6xMMH541GKQ7ccLJUtq3L5xI/IW53MEbB+5lS3IkC55bByNH5ggGYOsYhxoLCMaEiOyL0KQP8GYOCrl15SQlKZ2OvEUCdWjDYv6PMTRLW8UDk+r53MfWMQ4toqpu18FvMTExGhcX53Y1jAlIUVHeu28iI2H37ty3qclOptKH6/mSZVxLb97iRy4DnBt9Wlpx1dqcCSISr6oxeW1nLQRjQoQ/A7zPPw9lypz+uRSpDOZVNhNNU4mjL5P5B19mBAOw8YCSxAKCMSHC141b9fR4QufOMH26k3q6LptZSUte5VH+aHQ9/+6yhan0RbPdFm65pfjrbgKDBQRjQoS3Ad50iYnQpYszxbTUqRMceOgZNpdpzFVVf4T33qNG/Ce8t7yG130XLizGSpuAYtNOjQkR6QO5w4d7HycAqJm0lvo9eoJuhvvug3HjwJNF2J4pMNZCMCaEdO7sDCCLZH2/AkcYzWOs5irO0T/pVe0Tpw8pU0p5X11ONoZQclhAMCbI5PWsAWS9iV/LMjZSn8d4han0oS5bmLa/Xca+6eUlJuYMJPZMQcliAcGYIOLPswbg3MSrVzjIJPqxjNYAXMdXDGAShzgHcPbt0cN5Sjm9i0n1dFCwZwpKHnsOwZgg4s+zBgDMn8+RbgMo9+c+xvIoI3iGo/gYcfYiR3kmqNlzCMaEoLwGfudM3M9/K90H7duTmFyFxc+s5sJZozk/0v9gkNtxTGizgGBMEPE1wFvlPKVf5fe4dmAdbj4yh6d5hgYn47j7paaA820/MrLwxzGhzQKCMUHE27MGNcvs4Z0/2zM5uTM/cimNWM9zPM1JymZZm8DbvmXKQNmyWd+zgeSSywKCMUEkczK5UqQxrMpkNpyqQ2tdyiOMpSUrSaBuln3Su3+8JaKbPh2mTbPkdMZhg8rGBIHYWM86BT853TmvPbSD2z/tA8uWsZR/0Iep7OISr/vaALGxQWVjgoy35wvSVzfr0sWZXVRKT3F34hjaPFaPE2vWw1tv0Stiic9gYN0/Jj8sdYUxASD9+YLMi9V37w6nTp3eJppNvE0vrmQt87idF86bwJpeF/J8+az7pgsPh9des+4f4z9rIRgTALwtVp8eDMpynJGMIJ4mRLGbf/IBdzKXuF8uBLyPDcya5SyHbMHA5Ie1EIwJAL7m/TdjNW/Ti7okMJMuDGYcfxAOZJ0a2rmz3fxN4VkLwZgzIK/8Q9nn/VckhVd4lG9owdkc4hYWcD8zM4KBjQ2Y4mABwZhi5k/+oczPCLTmSzZSn0d5lYkMoC5b+IzTq9SEh9vUUFM8LCAYU8y8jQ8cOeLMHMqyktmrf/FeWB++5HpSKU0r/seDvMlhzs7Yr1Il+OMPp0xvWU6NKQwLCMYUk8xppX1JTISuXeGpBp/QamBdOiZP5yWG0oANfE2rHNunpJxuZfToYUHBFC0LCMYUg8zdRLk5n9/4j97Dcxtv59fUajTjW57g3xyjQp7HOHkSHn64iCpsDBYQjCkW3rqJslI6M4sE6nAH8xjOKJqylnU0yddxkpIKVU1jsrBpp8YUg9zSR9fgZybRn1tZyDc0pxdv8z21z1zljPHBWgjG+Ck9jYSI86pa1Xcfvrf00UIa/ZnIFupyHcsYxGtcw9eFCgbh4QXe1ZgcLCAY44fYWGcQN3MXTVKSM1PIW2DInmr6crbzFa2ZyEBWcxV12cLrDCKN0l6PV7asc7NPf/J4wICcaarLlnVSUxhTVCwgGOOH4cOdQVxvkpKgWzcICzvdenj4Yee9SyJOMZSX2Eh96rOR7kynLYtIJCpj//QbfubUE9OmOakn0tKcTKXN6CG7AAAVZUlEQVQTJuRMUz1tmj2LYIqWpb82xg+lSjnTPfOjyVkbWHRxL8J3xfNTzJ203vImO49ekPF5xYr2gJk5Myz9tTFFKD9LSpblOM/yFKtOxXBq9x76V51DVPzHHKx4QZZuIAsGJtBYQDAmF/48XJbZVaxiPY14ilHE0pnamsDkA3eh6nQtHT0KM2c63UAWDEygcTUgiMhNIrJNRHaIyBNu1sWY7Px9uAygEsm8ymBW0pJKpNCWz+nBDP6kSpbtMq9xbEygyTMgiMiDInJeUR9YREoDbwI3A3WATiJSp6iPY4w/MmcjDQuD0qWdGUS5P1zmuIEv2EQ9BjGeN3mAaDazmLY+t8/tGQVj3ORPC6E6sFZEPvR8o5ciOvaVwA5V3amqJ4D3gduLqGxj/JY9G2lKijO7Jy/n8idv04svaEOFc8vR7uzlDOJ1kqmc6375GY8w5kzKMyCo6pPA5cDbQHfgBxF5QUQuLeSxLwJ+zvTzHs97WYhIXxGJE5G4/fv3F/KQxmQVG+tMD/WnJZDZHcwlgTr0LP0ODBtG9X3fseDg1URG5r6frWNgAplfYwjqzE391fM6BZwHzBGRlwtxbG8tjRwT+1R1iqrGqGpMtWrVCnE4Y7J2DVWt6jxslprq//5/41c+pCNz6cCvVIe1a+GFF6B8eSDnA2ngzCoCm1lkAp8/YwiDRCQeeBlYCdRT1QFAE+CuQhx7D3Bxpp9rAL8UojxjcpW9aygpyffDZjkpXXmXBOrQjvkM4wVurrKGqDsbZVkFzdv6xjNnOsezmUUm0PmT3K4q0EFVs8y1UNU0EbmtEMdeC1wuIjWBvcC9wH2FKM+YXOWdgdS7CBKZTD9uYhEraElv3mJX2VroYTj5h7NN+ipoYOsbm+DlzxjC09mDQabPthb0wKp6CngQWARsBT5U1S0FLc+YvOR3do+QxkDeZDPRtGQlD/AGrVjOgfBaVK6cs3VhU0pNsHM1/bWqLgQWulkHU3JERPj/gNkVbOMtenMNK/ictvRjMj/hjBiHhfkOLjal1AQze1LZlBjeBnxLZ0s2ehYnGcq/2UAD6rKF+3mHm/ksIxiAc9P3NXXUppSaYGYBwYS0zLOKHn749IwfcNJLn3vu6Z8bsp5vaca/GcZ82lGbrczkfrJPiIuI8B5cbEqpCXYWEEzIGjjQWcA+86yilJTTnyclOa9yHON5/sVamnIB++jAR/yT2aSG/y3HTb9MGUhOdsqtUAFLVmdCigUEE5JiY2HSpLxTVrdgJd/RkH/xIu/QjTokMJcOAPzxR9YppOk3/6QkLFmdCUkWEExIGj4892AQxmHG8xBfcw3lOM4NfEFv3uYvTqftiohwbvK7dzupLMLC4MSJrOXYzCITSiwgmJCU22yfNixiM9E8wJuMZxD12MRSbsix3S23+FemzSwyocICgglJ3mb7nMcfTKc7i7iJFCrRkpU8wjhSCPNaxsJsE6JtZpEJdRYQTEjKPtvnLuawldp0JpbneJJGrGc1zXMtIzGRLGkpbGaRCXUWEExI6tzZGQSuzj7mcBdz6MjPXEwMcTzNc5ygnF/lqGZNS5E9T5HNLDKhxNUnlY0pNqrM7zCDWlMfpTzHGMLLvMojpBbwv3z64LHNJjKhzAKCCT27d0PfvjT/4gt+q9WKFtun8n3aFYUu1gaPTaizLiMTOlJTYfx4iI6G1ath4kT+tuUrnnz3Cnyt8xce7nv9guxs8NiEOgsIJjRs3QrXXOPkp2jVCrZsIbZyf6IuKUXXrjlv+uC899prOccF+ve3wWNTMlmXkQluJ0/Cyy/Ds89C5cowaxbcdx+x7wl9+55e/yAlxUk7cfbZzhPI6fmI0scDso8LtGzpjBmkJ7LLvK0xoUo0r2f7A0hMTIzGxcW5XQ0TKOLjoWdP2LgR7rnH6S46/3zAmSrqLdV1ZKQzxGBMSSIi8aoak9d21mVkgs/Ro/DEE9CsGRw4APPmEdvufaKuPD/juQFf6x4kJjrPFBhjcrKAYILL8uXQoAG89JLTOtiyhdjk27OslZyY6HtgGJxnCiwoGJOTBQQTkDKvYxAVBR9MPQQPPADXXuvMJlq61BkNPvdcr2slq/oOCpaQzhjvbFDZBJzYWLIMCNdJXEjzvv1Jk72UevRRZwC5UqWM7X09H5Db8Jg9U2BMTtZCMMUq+zd9b1012bfp188JBuEc4F26spBbOUxlWug3xDZ+JUswAN/PB0RGOi9v7JkCY7xQ1aB5NWnSRE3wmDVLtWJFVee7uvOqWNF5P7dtIE078oH+RjU9wVk6ghFalmMKqpGR+TuOP3UwJtQBcerHPdb1m3x+XhYQgktkZPYbvea4qWff5gL26lxuVwX9lqYazcYsn4t4P9asWU5ZIs6f2YOOr8+MKQn8DQj2HIIpNqVK+e7Hj4x0+vFPf670ZBqv8H+U5QRP8RzjGEwapXPsZ88RGJM/9hyCcZ2vfnqR01NEAWqykyXcwNv0Zj2NqM9GxvJ/OYKBpY8wpnhZQDDFxtuCMiKnA0EpUhnMq2yiHjHE0ZfJXM9SfuSyHGWFh9vaA8YUNwsIpth07pwzcVx6MKjDFlbSkld5lC/5B3XZwlT6EhFZigEDsu4za5bzQLIFA2OKlz2HYIpV586nb+SxsdCzywmG8m+eZBQHOYdOvMf73AsI4eE2PmCMm6yFYM6YDx5by1pieJYRzKYjdUjgfToBziPFSUmWUsIYN1lAMMXvyBEYMoS5v15FFf7gNubThVgOUC3HppZSwhj3WJeRKV7LlkGfPrBjB++H9WNg8ksc4hyfm1tKCWPcYy0EUzwOHnSWHmvd2hlJ/uormDSJUxV9BwOwlBLGuMkCgil6n37KkZp1SZ08lTE8Rq0TG4nde12WWUeQMxupPWdgjLssIJgC8Zq0bv9+uO8+aNeOnX9V4SpWM4TRbPu5YsYaBJ07OzOJVGHmzKzTS+05A2PcZakrTL5lT08NSrey7zOp3CDKHzvI2IpP8sTBJzhJ2Sz7WdoJY9xhqStMscm8IM1F7OET2jPjxH18f/wSWLeOxw49nSMYgA0YGxPoXAkIItJRRLaISJqI5Bm1TGD56ScQ0ujDFLZQl3/wJY8wlpgT30B0NFWqeN+vYsW810YwxrjHrWmnm4EOwGSXjm8KKDYWLpcdTNI+tGYZS7ievkxhF5f4XIwmXUqK8wInuV3fvs7fbdzAmMDgSgtBVbeq6jY3jm0K7sH+p/iuyxi+S6tHI9bTi7e4kS/YxSVZZgj98Yd/5dnaxsYEloB/ME1E+gJ9ASJskrprFvx7E/dP7sWVrGUetzOQCezjQgBKl846QygiwmkB+MPGFYwJHMXWQhCRJSKy2cvr9vyUo6pTVDVGVWOqVcuZ6sAUs+PHYcQI2gxrTBS7uYf3uZO5GcEAIC0ta7ePt7TXvliMNyZwFFsLQVVvKK6yzRny7bfQqxds2cL7dOERXiWJqjk2yz6InB4chg93WgClSkFqas7iRexBNGMCiU07LaG8PliWLiUFHn0UmjeHQ4dg4UKeipzpNRj4kv4AWloavPOO94Vy+ve3AWVjAolb007vFJE9QHNggYgscqMeJVX6g2Xpy1imz/iJjQW+/BLq14dXX3Xu2Js3w8035/pNPq9BZG8L5cycCRMmFOlpGWMKyZ5ULoGionIO+p7DX0wMG0Kn5Lfg8svhrbegVass21St6qxZkJ09gWxMYLMnlY1P2Wf2tOe/JFCHfyZPg8cfhw0bcgQDgNdey9n1YwnpjAkdFhBKoPSZPdX4nf9wL//lDvZTjTuqfwsvvQQVKnjdz1vXjyWkMyZ0WEAogZ4fpfQsO4ut1OZO5vIkz9GqQhz3jsk7i0jmweLduy0YGBNKLCCUILGx0LzGz5zb9TbePtGVH8/6O41Zz6zIJ5kwtYzd3I0p4QL+SWVTNGJnprGm12QWnRxKaVIZxGtML/MAk2aUtkBgjAGshVAybN/OZX2u47WTA/mWZkSzmdcZRPLR0pZLyBiTwQJCKDt1Cl5+GRo04Irjm+jBNNqwmN3UzNjEcgkZY9JZl1Go2rABevaEdevgzjtps+ZN4vZekGMzyyVkjElnLYQg5TP1xPHj8NRTEBMDe/bA7NnEdviIXcdyBgN7hsAYk5m1EIJQ9jWN01NPVP1hFW0/7AVbt0K3bvDKK8R+Hk7ffpnXP3aEhzsPmtmAsjEmnbUQglDmNY0BKpHM80cGc+MzLZ3EdJ9/DjNmQHh4jm3ThYVZMDDGZGUBIQhlHgi+gS/YRD0G8xoTeMBJRte2rddtfZVhjDFgASEoRUTAufzJ2/TkC9pwnHJczdeMiXwdKlfOsa2vMowxJjMLCEHonTvmspU63M+7vMAwGvId6yte7XWA2NvqZTaYbIzxxgJCMPntN+jYkWtf60DZyOrcXn0NT8oLVI8s7zPJnCWkM8b4y9ZDCAaqzooygwc7I8RPPw1DhkCZMm7XzBgTBPxdD8GmnQa6xETo1w8WLYKWLZ2Fa2rVcrtWxpgQZF1GgSotDd54A+rWhRUr4PXXYflyCwbGmGJjLYRAtG0b9OoFK1c6U0gnT3Y6/40xphhZCyGQnDwJL74IDRpAQoLzcNlnn1kwMMacEdZCCBTr1zvJ6L77Du6+2+kiql7d7VoZY0oQayG47dgxGDYMmjaFX3+Fjz6C2bMtGBhjzjhrIbhpxQpnrGD7dujRA155Bc47z+1aGWNKKGshuOHwYXjwQbjmGjhxAhYvhmnTLBgYY1xlAeFMW7QIoqNhwgQYNAg2bYIbb3S7VsYYYwHhjElKctYouOkmJ5nQihXOggRhYW7XzBhjAAsIxU8V5syBOnXgvffgySedmUQtWrhdM2OMycIGlYvTvn3wwAMwdy40aeKMFTRo4HatjDHGK2shFAdVmD7daRV89hm89BKsXm3BwBgT0KyFUNR27XIWOF6yxJlF9NZbcMUVbtfKGGPyZC2EopKa6gwSR0c7rYEJE2DZMgsGxpigYS2EopCQAL17w6pVcPPNMGmSrVFpjAk61kIojJMnYdQoaNTIedp41ixYsMCCgTEmKFkLoaDi451kdBs3wj33wPjxcP75btfKGGMKzFoI+XX0KAwdCldeCfv3w7x58P77FgyMMUHPWgj58b//OWMFO3Y4f44eDeee63atjDGmSLjSQhCR0SLyvYhsFJG5IhLYd9VDh2DAALjuOmdpyyVLYOpUCwbGmJDiVpfRF0C0qtYHtgPDXKpH3hYudNY1njIFHn3UGTO4/nq3a2WMMUXOlYCgqotV9ZTnx9VADTfqkasDB6BLF7j1Vjj7bPjmG2e9gkqV3K6ZMcYUi0AYVO4JfObrQxHpKyJxIhK3f//+4q+NqjNIXLs2fPABjBgB69ZBs2bFf2xjjHFRsQ0qi8gSwNs6kMNV9b+ebYYDp4BYX+Wo6hRgCkBMTIwWQ1VP27sXBg6ETz6BmBhn0Zp69Yr1kMYYEyiKLSCo6g25fS4i3YDbgOtVtXhv9HlRdXIOPfaY87DZmDHw8MNwlk3CMsaUHK7c8UTkJmAocK2qHnGjDhl+/BH69IGvvnJmEU2dCpdd5mqVjDHGDW6NIbwBVAa+EJHvRGTSGa9BaiqMHet0CcXHw+TJsHSpBQNjTInlSgtBVd29627eDL16wZo1cNttMHEi1Ai8iU7GGHMmBcIsozPnxAl45hlo3Bh27nSWtPzkEwsGxhhDSUpdsWaN0yrYvBnuuw/GjYNq1dyulTHGBIyS0UIYNQqaN4c//4T58yE21oKBMcZkUzICwqWXOjOJtmxxxgyMMcbkUDK6jDp1cl7GGGN8KhktBGOMMXmygGCMMQawgGCMMcbDAoIxxhjAAoIxxhgPCwjGGGMACwjGGGM8LCAYY4wBQNxemyY/RGQ/kFjA3asCB4qwOm6ycwk8oXIeYOcSqApzLpGqmme+nqAKCIUhInGqGuN2PYqCnUvgCZXzADuXQHUmzsW6jIwxxgAWEIwxxniUpIAwxe0KFCE7l8ATKucBdi6BqtjPpcSMIRhjjMldSWohGGOMyYUFBGOMMUAJCwgi8pyIbBSR70RksYhc6HadCkpERovI957zmSsi57pdp4IQkY4iskVE0kQkKKcHishNIrJNRHaIyBNu16egRGSaiPwuIpvdrkthiMjFIvKViGz1/N962O06FZSIlBeRNSKywXMuzxTr8UrSGIKInK2qhzx/HwTUUdX+LlerQESkDfClqp4SkZcAVHWoy9XKNxGpDaQBk4HHVDXO5Srli4iUBrYDNwJ7gLVAJ1VNcLViBSAirYBk4F1VjXa7PgUlIhcAF6jqOhGpDMQDdwTpv4kAlVQ1WUTKACuAh1V1dXEcr0S1ENKDgUclIGijoaouVtVTnh9XAzXcrE9BqepWVd3mdj0K4Upgh6ruVNUTwPvA7S7XqUBUdTnwh9v1KCxV3aeq6zx/PwxsBS5yt1YFo45kz49lPK9iu2+VqIAAICLPi8jPQGfgabfrU0R6Ap+5XYkS6iLg50w/7yFIbz6hSESigEbAt+7WpOBEpLSIfAf8DnyhqsV2LiEXEERkiYhs9vK6HUBVh6vqxUAs8KC7tc1dXufi2WY4cArnfAKSP+cRxMTLe0Hb8gwlIhIGfAQMztY7EFRUNVVVG+L0AlwpIsXWnXdWcRXsFlW9wc9N3wMWACOKsTqFkte5iEg34Dbgeg3gwaB8/JsEoz3AxZl+rgH84lJdjIenv/0jIFZVP3a7PkVBVf8SkWXATUCxDPyHXAshNyJyeaYf2wPfu1WXwhKRm4ChQHtVPeJ2fUqwtcDlIlJTRMoC9wKfuFynEs0zEPs2sFVVx7pdn8IQkWrpMwhFpAJwA8V43ypps4w+Av6OM6slEeivqnvdrVXBiMgOoByQ5HlrdTDOmBKRO4HXgWrAX8B3qtrW3Vrlj4jcAowDSgPTVPV5l6tUICLyH+A6nDTLvwEjVPVtVytVACJyNfA1sAnndx3gX6q60L1aFYyI1Afewfm/VQr4UFWfLbbjlaSAYIwxxrcS1WVkjDHGNwsIxhhjAAsIxhhjPCwgGGOMASwgGGOM8bCAYIwxBrCAYIwxxsMCgjGFICJNPWtSlBeRSp6c9UGbOtqUbPZgmjGFJCKjgPJABWCPqr7ocpWMKRALCMYUkieH0VrgGNBCVVNdrpIxBWJdRsYUXhUgDKiM01IwJihZC8GYQhKRT3BWSquJs3RjQK+zYYwvIbcegjFnkojcD5xS1fc86yt/IyL/UNUv3a6bMfllLQRjjDGAjSEYY4zxsIBgjDEGsIBgjDHGwwKCMcYYwAKCMcYYDwsIxhhjAAsIxhhjPP4fyOW/m0WIVdcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a12125f10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#####################################################################\n",
    "# Initialize data\n",
    "#####################################################################\n",
    "X,Y_gt = sklearn.datasets.make_regression(n_samples=100,n_features=1,noise=10)\n",
    "Y_gt = np.reshape(Y_gt,(-1,1))\n",
    "Y_gt /= 100.\n",
    "\n",
    "#####################################################################\n",
    "# Create and train model\n",
    "#####################################################################\n",
    "# Construct model with single FC layer\n",
    "regression_model = ModelHelper(name=\"regression_model\")\n",
    "y_pred = brew.fc(regression_model,'X','y_pred', dim_in=1, dim_out=1)\n",
    "\n",
    "# Specify Loss function\n",
    "dist = regression_model.SquaredL2Distance(['Y_gt', y_pred], \"dist\")\n",
    "loss = regression_model.AveragedLoss(dist, \"loss\")\n",
    "\n",
    "# Get gradients for all the computations above.\n",
    "regression_model.AddGradientOperators([loss])\n",
    "optimizer.build_sgd(regression_model, base_learning_rate=0.05)\n",
    "\n",
    "# Prime and prepare workspace for training\n",
    "workspace.FeedBlob(\"Y_gt\",Y_gt.astype(np.float32))\n",
    "workspace.FeedBlob(\"X\",X.astype(np.float32))\n",
    "workspace.RunNetOnce(regression_model.param_init_net)\n",
    "workspace.CreateNet(regression_model.net)\n",
    "\n",
    "# Set the initial weight and bias to 0\n",
    "workspace.FeedBlob(\"y_pred_w\",np.array([[0.]]).astype(np.float32))\n",
    "workspace.FeedBlob(\"y_pred_b\",np.array([0.]).astype(np.float32))\n",
    "\n",
    "# Train the model\n",
    "for i in range(100):\n",
    "    workspace.RunNet(regression_model.net)\n",
    "\n",
    "#####################################################################\n",
    "# Collect and format results\n",
    "#####################################################################\n",
    "# Grab the learned weight and bias from workspace\n",
    "coe = workspace.FetchBlob(\"y_pred_w\")[0]\n",
    "intercept = workspace.FetchBlob(\"y_pred_b\")\n",
    "\n",
    "# Calculate the regression line for plotting\n",
    "x_vals = range(-3,4)\n",
    "regression_result = x_vals*coe + intercept\n",
    "\n",
    "# Plot the results\n",
    "plt.scatter(X,Y_gt,label=\"original data\",color='b')\n",
    "plt.plot(x_vals,regression_result,label=\"regression result\",color='r')\n",
    "plt.legend()\n",
    "plt.xlabel(\"x\")\n",
    "plt.ylabel(\"y\")\n",
    "plt.title(\"Regression Line: ${{{}}}x + {{{}}}$\".format(round(coe,5), round(intercept[0],5)))\n",
    "plt.show()\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}