Ch-1-Basic-Models/Linear Regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Linear Regression\n",
    "---\n",
    "__Introduction to Linear Regression__ [[1]](https://en.wikipedia.org/wiki/Linear_regression)\n",
    "\n",
    "Consider a set of N-D datapoints and 1-D. In linear regression, the relationship between the dependent and indepedent variables is modeled as linear function of the form:\n",
    "\n",
    "$y = w \\bullet x + b$\n",
    "\n",
    "where $y$ is the dependent variable (to be predicted) and $x$ is the independent variable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "from IPython import display\n",
    "\n",
    "# Import the libraries and load the datasets.\n",
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "\n",
    "# Plots.\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "### Linear Regression in Tensorflow\n",
    "Following are the steps to implement a simple linear regression function in Tensorflow.\n",
    "\n",
    "_Example: Consider the line `y = 12 * x + 10`_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEXCAYAAAC+mHPKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnX2QZFd5n59XO3zsajVrNOugYgdLARdhFwuEY9iNtTAd\nMEHYjkUMCk4oMsJUUpEX8xGXDWvtenZLoiQ+J7giXGVbdgGxgry4ChljkDIwTbHEwhhJkZAFhthI\nligEhQwUxjViRid/nL7bd+7ee/t+f3T/nqqu7tt9z71vn+l5f+e855z3mHMOIYQQYhLntG2AEEKI\nfiDBEEIIkQkJhhBCiExIMIQQQmRCgiGEECITEgwhhBCZaFUwzGzRzD5lZvea2T1m9obI579mZo+Z\n2flt2SiEEMIz1/L9N4H/5py7y8x2A18ws9ucc18ys0XgJcD97ZoohBACWu5hOOe+4Zy7a/T6+8B9\nwL7Rx6vAr7dlmxBCiO10ZgzDzC4CLgE+Z2a/APy9c+6eVo0SQghxhrZDUgCMwlEfBt4IbAG/iQ9H\nnTmlDbuEEEKMsbZzSZnZHPBnwMedc+81s58A1oAf4IViEXgIeL5z7puRskqEJYQQBXDO5W6IdyEk\n9QfAXzvn3gvgnPuic+4C59zTnHP/HHgQeG5ULAKcc3o4x8rKSus2dOWhulBdqC7SH0Vpe1rtpcCr\ngReZ2Z1mdoeZXRY5zaGQlBBCtE6rYxjOuc8COyac87SGzBFCCJFCF0JSogIGg0HbJnQG1cUY1cUY\n1UV5Wh/0LoOZuT7bL4QQbWBmuJ4OegshhOgBEgwhhBCZkGAIIYTIhARDCCFEJiQYQgghMiHBEEII\nkQkJhhBCiExIMIQQQmRCgiGEECITEgwhhBCZkGAIIYTIhARDCCFEJiQYQgghMiHBEEIIkQkJhhBC\niExIMIQQQmRCgiGEECITrQqGmS2a2afM7F4zu8fMfnX0/jvM7D4zu8vM/sTM5tu0UwghRMtbtJrZ\nBcAFzrm7zGw38AXgcmAR+JRz7jEzux5wzrmjMeW1RasQQuSkl1u0Oue+4Zy7a/T6+8B9wD7n3Jpz\n7rHRabfjBUSIiWxtpR8LIYrTmTEMM7sIuAT4XOSjXwY+3rQ9on88+igcPgzDoT/+5Cf9sURjdoV0\nVr93XXRCMEbhqA8Dbxz1NIL3rwZ+6Jy7qTXjRC/Y2oKlJXjVq+CKK+DKK+Gyy+Daa2HHjvE50TKz\nwNbWdiEdDmdDSGf1e9fJXNsGmNkcXiw+6Jy7JfT+lcDPAi9KK3/ixIkzrweDAYPBoA4zRcfZsQOu\nu86LxbOeBe9/Pywvw4tf7D8PnMd118Fg4J3H0aNw+vRYUKaVcN0cOQI33ACnTvXve29tbbc5ehxl\nWr53FQyHQ4aBcpbBOdfqA/gA8J7Ie5cB9wILE8o6IcIsLzsHzi0tObd3r3Pr6+PP1tf9eysrZ3/W\nNpub6cdVsLLi62ZlpfprFyXte0dfHzw4/putrzt36FC2euri926bke/M76+LFKrqAVwKbAF3AXcC\ndwAvA74C3D86vgN4X0L5qutR9Ji1Nefm5rxo7N3r3Orq2U6li84jzhkePFitaKytbRfLtbXqrl2E\nzU3/OHTIf9/Nze0iEP7MOf+8f39+we9yI6FNeikYZR8SDBEQOJjAEQbOZ2NjfE4e59FEiz987QMH\nnJuf97bNz/vjqu65seHc7t1eQJ3zz7t3b6+bJgmLwfq6/76Li84tLEzuEeYR/DjRydormXYkGGLm\nmRTeyOo88p6bdpzVzvV153bt8v+Ru3ZV3xLuWg8jLAbB944TgbBAFOktNCn8fUKCIcQE8jiPLM6p\nSAs2qczamnM7d/r/yJ076wmddC0cF9izc2d8PYf/BgsLvtel3kI1SDCEqIjACQUO7fjx5HOLtHqj\nZdbWfHx+zx7/3p49/rhKZ1g0ll9XCz2wZ98+/32D8FTaGEZ0XEdiURwJhhAVEDiq1VXv0JaX/UB6\nWginSMs9XKbuQe+8PaHg/aRxobJ2TRrsjtqRdCyKI8EQoiKyzLYKqKKHETjOMFU7x6zXj4rL6qqv\ni+PHq51llMeeLOeJfEgwhKiQ48e39xrKDo6XKdM0UUEL1rY0PfbRh7rqKxIMISqi7um3bbWa89w3\nCJkFvay21jFoHUU9SDCEqIBJrdq+hkjytNYDJ338uHM7dozXb9SxoDALcWNEfan3riLBECInSc4/\n7f2uhEiKCFfeqcKbm362VjBjqw3BWF/3U2oXF/0Cv+hsKlEMCYYQOSjq/LsQIikjXFlmdIWv0+b3\nja4I37PHT8NVaKo8EgwhclLUGdaxAC5vj6Gq2VlZKPt9y4Txup4HrK9IMIQoQF4nVEeLu2iPIW5h\nYdJahrZ6VFWF8brQs5smJBhCZCAabllYyO6E6hzDyOsQwwPTc3N+YDpIYhgecygzYJ/2fatOs1LU\nDlEMCYYQE4gb0A2ywmZ1QnXOksra20lbXDc/P04xUkVLPO77FnHgbYa1xNlIMITIQFprt00nlLcV\nHrU1vNCw6lh/UnbdrPYqnNQ9JBhCZKRrg6dlQy5hh1xHDyPJtqwzrhRO6h4SDCEykLWHMSkRXtUU\nDblEw2xpYxhFiauzulfDi3qRYAgxgUmDuHU73rpoQujCIa/NzeL7a4tuUFQwzJftJ2bm+my/aJ6t\nLdixI/54OIQrroAjR2B1FczgTW+CG26AU6dgMGjD4rPtjDuuk09+Ei67DF79avjYx+Dqq+FDH4LP\nfnZsQ5P2iPKYGc45y1vunDqMyYqZLZrZp8zsXjO7x8zeMHr/SWZ2m5l92cxuNbM9bdoppoeoUwsf\nDwZw1VVw8iS8+c1eLE6e9AIyGHinGCZ6XBePPgqHD3tBA+/ADx9u5v5bW3DsGLzznV4sfu7n4Nd/\nHd72tu11J7GYEYp0S6p6ABcAl4xe7wa+DDwTeDvwG6P33wJcn1C+uj6amHnC+2DMzzt37rk+h9HC\ngv+sjcHbIFSWZ0OnOmxwLtsOhKIfMA1jGMBHgJ8BvgQ82Y1F5UsJ51dYhWKWiTrmvXt9pta1tbE4\nrK21Mz00GGBeWnJnUo43jabGThe9FwzgIuBro57GP0Q+eyShTHU1KHpFHTNv0lrS0c/KTMktYnuw\nidHSUrtJAJ3TIPc0UFQwOjHobWa7gSFwjXPuFjN7xDl3fujzbzvnFmLKuZWVlTPHg8GAQZsjkyI3\nRQZzt7Z8DP+66/zYwnAIR4/C6dPlY+nhge/oYHfaZ1kpYnvcoPPNN1fzffPY3daguyjPcDhkGAyC\nASdPnsQVGPTuQs9iDvgE8MbQe/exPSR1X0LZivRWtEGZlmvTSQCrbGXnXcMQhMPC993YyH9fIQLo\na0gK+ADwnsh7bwfeMnqtQe8ppswCsOi+21WQtqYh6qTLhGTyhLa08E1UTS8FA7gU2ALuAu4E7gAu\nA84H1vCzpm4DfiShfNX1KFqgSIqJcMK9OmL6XcpM2wRFMtnmOV90i14KRtmHBKP/FEliF07pHbxf\n9SBsNMHewsLYtrIbAnVtADmvTV38DiIfEgzRO8qkyY6uBahaLAK7gvstLlY3ltHF1nmccKfZ2cVe\nksiOBEP0krIb8dTlbNfXfdbXnTud27XLL+QLi8Q0OstwaDCLMHYt66/IjgRDTDVBwrv9+8fZUg8c\n8O+VFY2kTYL27Rs7xKjDrLqn03avI29G2rjdCrvQUxLZkGCIqSdtXKHMNeNa0nGrugOHmDSWEqQP\nSdoSNa8NTTngvHteBOJ94MBYWPbvr0a8RTNIMMTMUHUoJNqSTssblTRb6+qryw3Etx3myrurXvC5\nehn9RIIhZoKsGyDlJSpCaSGipPUgy8vVbbPaNll6PZubfjJAUtiu7TCbSEaCIaae6CZH+/f7sEjZ\nTY7KtO6jZYOcT5OcftPrSoowyeGvr/vJALt2+ckBe/Zsn3qsqbfdRYIhZoLo1M5oSKTI9eIcW3hV\nd9IudmWdfpPrSqomburxvn2aetsXJBii1xQNX9SRPXZjI3tPJioe0ZxPk77HpNlWXQ7rTBrncE5T\nb7uKBEO0Qp5Yf9Urh+tswYavvWePD71Muk9e5z7J/q6HdSbZpx5Gd5FgiMapMrtr3pXGwdTO8PWr\nntYZbh1X3VLOWj9dd7pJf6Oui92sI8EQrTBpcdekFnSY8CyhSQ6nbsEo0sPIS9YeSVGxajuc1fb9\nRTISDNEaaQ4t6bMsA8aTBKeu1ndds7GKUPQ7qoUv0pBgiFYo08PIMktoUuu6rkHVaPgrbWymLso6\n/a6Hs0R7SDBE41QxhpE2S6itHkaXKBvWqWPspYw9ohtIMEQrlJkllebww1NbnYvP06SQSzpVC6rq\nfHqQYIhekaV3Mmkf6662drtgV13OfRZ6dbNAUcEwX7afmJnrs/2zztYW7NgRfzwcwhVXwJEjcMMN\ncOoUDAZtWDmZsN1bW3DppXD99d7e4RCOHoXTp7d/16btijsuyokTcPIkrKz416J/mBnOOctb7pw6\njBEiC1HnFT4eDLxYnDzpn8uKxdZW+nGZ6x4+7IUB4DOfge9+14vdiRP++dpr6xOLtO+VVr+TyiYx\nHHoBX1nxz8H3FjNCkW5JVQ/gRuBh4O7Qe88B/gK4E/hL4KdSylfWRRPVUyY0U2Xoo+7Ye9jWhQU/\nBTdIQri8XH2cPwjNBd/rttvGdmS9V9awX1wZjWH0H/o4hgEcBi6JCMatwL8ZvX4ZsJ5SvsIqFFVS\nxrnU4Zjqjr2HZyOtrjpn5tzSkn8OpgunkVVcNzac2717fM0jR/x9jx3L/73W1vx05iAt++rq5Hru\nwviMKE8vBcPbzYURwfg4cMXo9X8A/mdK2QqrUFRNGSddh2Oqa81G2R5GXoEMBOmFL/TPL32pi52a\nnIXAzqUlDWLPEtMkGM8E7gceAP4eeGpK2SrrUNRAV7KVFhGvLKIV5+z37z97B7+q7XvhC329Pv3p\n43045uay3St6z6WlsbiJ2WCaBOO9wMtHr18J/O+Usm5lZeXMY13No07RlSmYRUJcecpMSoqYNZyW\nRVw3N8c9jGc/25//vOfFh5QmrZE5dMiXKSo4oj+sr69v85XTJBjfiXz+3ZSylVWoqJYmBkjzhK2K\nhLjK5HHKe5zlXoEYPfGJ3tGvr/ukiMEYRvjaWep/0uJIMb30WTAuAu4JHd8LLI1evxj4fErZ6mpQ\nVE5WJ13E8SfN8qna2eUNqWURh6gjP3gwe68kLotu0oB3VhFKOxbTSS8FA7gJ+DqwMRqzeC3w08Bf\njabV/gXw3JTyFVejaJq8oZ8m98HO28Mos8dFHscd3hI1a66utseQRLfopWCUfUgwpoM8jjl6bjDL\np45stXXuAljUkaddP0+uLjHbSDBEr8njQINzg/UDdTnEouGaSd+liT0utMhOpCHBEL2lSKs5bQ+N\npigyeF3Wkdc90C9mAwmGKEybjiXsQDc303e3izrbuJTnbdjtXPrgdTTdRvg4yM4bPhaibiQYohBR\nhx1NM96UDQHr6361dNZ9wNt0sFkGr6NTV+PSuCtsJJpGgiEKEzjpxUU/TXN9vV3nNWkMoEuikWXs\nJS1MpYFp0QYSDFGKwPHt2tWu80prtUdb5XFhq7ZtTSJNWDT1VTSNBEMUJuz4du5sz3kFK5n37x/3\ncg4c8O+FQzvr637R2r597QlbnnBSmgiGB/HVwxBNIcEQhQg7vq444rATXVjYPshdp7DlDXUVTU4Y\nDI4Hg/ZBDqi8qTm6FJoT/UKCIQrTtVCPc14swhlUA3uC9yeFzoo4/yLJBrNeO3qc1LPY3Ezeu7yq\nJIcSGiHBEKWJOqYqp6vmcVKBM11e9llZwxv8JA3OR6eqFnGmWXMvVTWzKS7Fx223+Q2SghxZQa8j\nCMmFc2dF06hn6RFqZpZwToIhKqIOh1JmhXKQ+uPYseQeUNzU1SLO1Lnys56yElxj3z7nzj3XC2CQ\nYvxd74pfnBiXOyvN3iSR1swsIcEQlVGHQ8lzzaQB4eheDdH1G9Hr5519VNWsp0lEx43m55077zy3\nLQSXlP4knDtr0gr5NJHWzKzZRoIhKiXOoZQNW+RxUkV6OuHr5xW9srOe8hK+biAC553nxWN11T8v\nLvoQXHi71+hWsGn2JtmpHoaQYIjKCIdL9uypZiFfESdVZNwjqzMter+qQ3Zraz7MtLzsReLcc517\n3OPG9R6EoY4dOzt31sGDZ69RidoeFWmNYQjnJBiiIuLCJUFLt2hLNM1JVTFjJ2nqal05mqqaZRTY\nHR7I3rdv7OCjnyflzkqq3/A05OhMrCrsF/1FgiEqI+xAkkJTaQn1Jl0zOK6ytdslJ5jHlkkhpKzX\nipYNxEU9CRGHBENUTlJo6vnP91M/w7N3du+eLBpp9+hSPL2M+EwSwaRrVyGeceGnuHsJIcEQlTIp\nNLW66tdILC3559XV4g6pSzN2qnDcSSJYVEzK3FOIOCQYonLSQlObm+OpoEtLxfem6KKjq8KmJBGs\n4/s2PYYj+k8vBQO4EXgYuDvy/q8C9wH3ANenlK+yDkUCcU4u6GHMzflfUdDLCM7POiupq3H2Mr2e\nSaJQR4+qqrQhYjZoRDCAvwHeAlxQ5GYx1zsMXBIWDGAA3AbMjY73ppSvuBpFlDinHh7DCJzf3Jxz\n559fbJ/qtOM2KNMLmCSCTfWouthzE92hKcG4F3gMeBT4CPDzwDlFbhy65oURwbgZeFHGshVX42wy\nyWnHHW9sbHdK55+/fRVyX6mi11PnwHYeujQ2JLpFYyEp4BDwe8B3gS3gIeBtwNMLGXC2YNwJnABu\nB9aBn0opW0NVzhZFnVi0XFyeozqps2fS12uHUQ9DpFFUMObIiXPuduB2M3sj8CrgdcBR4K1m9mng\n94E/cc5t5L32iDngSc65Q2b2POCPgaclnXzixIkzrweDAYPBoOBtZ5MdO+C66+CKK+DIEbjhBjh1\nyr8/qdzp0/55awtuvhk+8Ql48YvhRS+Co0fHn0fZ2tr+fvR4EltbcPiwt3swgOEw/X55iV6jims2\nce2ArS1fH6dO+foZDKqtH9E/hsMhw+Gw/IWKqEz0ATwD+CN8uGoL+Dbw34Efy1A22sP4c2ApdPxV\nYCGhbLWyO8OUDV9kbTlXFZZRCzqdLo4Nie5AwR7GOWXExsx2mNm/A96D7204fBjpduD1wH1mdvmk\ny4weAR8BXjS6/jOAxznnvl3GTpHOcOh7Fisr/rlIQyRryzncozlxwj9fd13+lu9g4HtEJ0/6Z3Us\nt9NET0bMIEVUBngm8E7gG/hexTeA6wmNYwA/DtwBfDnlOjcBXwc2gAeA1+JDUh/ET6n9K0K9jZjy\nNenv7JC3xV9Vy7Vsj6aJHkbe9CdC9AUamiX1OuCz+LDTFnAr8ApGU2Bjzr8S2CxiWEZ7qq3FGaVv\n4aQmZhttbFSX/iQLCiGJJmlKMB4b9QjeBlyU4fznA39YxLCM9lRXgyITcUnuwjSxWC8ulXcdDjYu\n/UkddHkBo5hOmhKMlwM7ityojocEI5k6W6xBOOn48WyOLnocbaVXmdyvapaW3Jn0J2EbojaVpciG\nT1XbIGaHRgSjaw8JRjx1OtWoY1tdTXd0ddjS1AyppASLddVt1nEd9UhEWSQYU07eFmVTSe4OHfI9\njbS02kkb+ZSh7lXMaWMYddRt1mtGU4w0tVBSTBcSjCmmaIuyDqcaDSfdemu2jXuiolKGpnoYabOk\nqqzbrH/f6HlBKpbjx8vbIGYLCcaUk9dJNtHDWFvzre7wFqNxW4NOCluVsaGNcExddZt2HL338rIP\nky0vq4ch8iPBmAG6EOPOOksq78B4HrI417oGhbsgWEFvbXm5PRtEv5FgTDlFZtHUNfV0knCVnXpb\nlrqdepszlJLGLiQWIg8SjCmmiAOsy2lOEq4utMCD+05brqmu1K3oPxKMKadIq7ZMSz/ufnkGZ/Pa\nWgfTuB9EV+pW9JuiglEq+aBojiLJ5MIJ+q66Co4dGycWHA59ivCtrbPLBenDo+eCT5EdJPobDOJT\nZnch8V0VCRW7SBfqVswwRVSmKw+mtIdRVSsybZHdwkJ6mKbPIR2FboRIh4I9DPNl+4mZuT7bH0dV\nmwOFr/OCF8DFF4MZvOIVcM01sLgIX/va+Jpxmxpdc43vnays+FTkfaLsJk3TaosQAGaGc84mn7kd\nhaQ6Rtb9IqKhpOhxsCPeYOBfv+998OCD8K53wa5d8L3vwWc+My4bDUFdfHG2kM4kO8qeX5SuhG6S\nwnuPPnr2eUJ0niLdkq48mNKQlHPbB2zjkvcVmTW1b9/4mtEy4RDUwoJzBw7kX3mcZS+NWQwVxU0+\nmMV6EN0BzZKaHrI477z5meLGJKIOKk2kJq08bnMFeh+Iztia1XoQ3UCCMSXEtcL37493LlWu/C7j\nwPJOX52W6a5lRXVa6kH0DwnGFBHniMq2UNOcW5lQURd7GE2sVSiaMLBoD1GIKpFgTDFNxMCLONku\njmE0OU6SNyV5QJExKCGqpJeCAdwIPAzcHfPZr+G3hD0/pXyFVdhNkhxgmV3rkso0cY2qWv9p12ly\nfKBoWEkrtkWbFBWMtqfV/iHw0uibZrYIvAS4v3GLOkZ4eiyMV1c//vHp5bJMb42b7pl3emfa+oK4\na1Ux3XWS7eEV7keOjOuuasqsJu/KtF8hclFEZap8ABcS6WEAp4CLgb9jxnsYWSgSGgrOq2rXtqan\nzKb1IpoaJ1FYSfQV+hiScjGCAfwC8J7RawlGRrKmva5z17YqHHWeUE1cOKhJR66wkugrUyEYwE7g\nduA8NxaMhZSybmVl5cxjfYanmmxubt9YZ3Mz2VkGjr2OXdvKTBXN4+zTxCmPI6/T6UtQRFdYX1/f\n5iunRTB+AvgG8Lcjsfgh8DXgnyWUrbZWe8rmpl+rsWfPWAT27k1PMFjHrm1V9DCyXKOqXkSdvRGF\nrESX6bNgXATck/DZ3wFPSilbWQU2QZ3bhh444Nz8vHe0T3iCSw0zTQpfNTHFNo0svZS6MvpW2UnV\nam7RVXopGMBNwNeBDeAB4LWRz/92WsYwmtg2NHC0O3cmD2RPsqOMnVU48TacbJ0rrrWaW3SRXgpG\n2UefBMO5Zlqz+/b50NT6evrq47TjtlrGRcWqjFCphyFmEQlGT6ijxRl2tJub45XgwbaqdYWF6qDI\nQsAyPaJwvYXLlu35aQxDdBkJRg+os8UZDSkF+3cXcVR9axmXsTcQh8C5b2xUJxyaJSW6igSj43Qx\nx1EcfW0Zx6VmT3qOI09IT4i+I8HoAHnHCvIe5yFwoNGZUk0NXjdJVCD37x+H5VZXxz2uSc4/PGmg\nL70rIYogwWiZsrH0tHh6XsLTZufmvNPMa1NfiKv3Awf8GpQ8CxPDorNrl9PMJjHVSDA6QJFQUNjh\nra/7cMi+feVSa4Qd6OqqF40q8kVF75N23CRxtgS9haWl7c4/7tzo32B+3rnFxfSFj3lsEaJrSDA6\nQpHZRWGh2bmzfOs26qSCVd1VtZi7Ps6RlPokbR+RKnp5Xa8XIQIkGB0gaw8jrUW8a1e18fO6ZjxV\nlcupasKzxOLGMCbtdFfW9r7NMBOziQSjZbK2LpNi7uef78Mg8/PVzdCpu8XbdrbYJCbNkqp7jYlW\nd4uuI8HoAFlbp3GzeupYPJbHprxl01rSXW5ll51ynHZc9vpCNIUEo2eEW6HR7Vajx22Q1lPI0ovo\nYiu7qlXh4bLhv9XmpnMHD2oMQ3QfCUaPSOphBJ91xckUHafociu7yrxTcYPoBw9uv2YX/o5CRJFg\n9IS0dQNVOdg6FgBm7Sl0YQyjTqL10WVxFCIJCUaPSJslVTSEU2UuqYBJzjBJmIokEKziOlmvW5Sk\n+uhi+E2INCQYPSbsiKKLxYrE16tYrDepp1BVTyJtbKDM9avu6SRdb9I0XSG6iASjp0QXix044Mc0\n8i4ci7Z+l5fLt3ontdCrCsckXafs9asOF0W/f1lRE6ItJBg9JjyFNpwHKq+TC0IjwcrmJlq9VYVj\nkq5T9vp1h4uUCkT0EQlGj4mGO4LeQTjT7CRHFBaaphIOzloPQ4hpQYLRc8L5j8C5H/3R7Ku+o4IT\nTeVdh1jM2hiGENNEUcEwX7YdzOxG4OeBh51zzx699w7g3wIbwP8DXuuc+15Cedem/VXzW78F11wD\ny8tw5ZXw8pfD7t2wsQGnTsFgkFx2awt27Eg+nkSR8mXvOek6dXyngOD94L0idgvRV8wM55zlLXdO\nHcbk4A+Bl0beuw14lnPuEuArwNHGrWqB4RB+53fg+HH42Mf8e296Ezz0EBw5ki4WcLbDy+tYDx/2\nNmxt+efDh/3rsJOt8p5ZrlP2+nHnX3opXHyx/47DITz72f69tO8phPC02sMAMLMLgY8GPYzIZy8H\nXuGce01C2anoYQQO+7rrvDAMh/ArvwLf+pYXixtumNzDCF+rSKt8OIRXvhJ27oTvfQ9uucW/f/Qo\nnD49PS3w4dD33B59FMxgbs5/1yx1K8S00NcexiR+Gfh420bUzY4d3ikHTusFL4D5eS8SJ07456NH\nJ7eCwz0F2N5TmMRgAK9/PTz4IGxu+rJXXOFFbFrEAvz3fNOb4J/+CX7wA3jzmyUWQmSlsz0MM7sa\n+Enn3CtSyrqVlZUzx4PBgEFL//1VxfOj5ZOekwgcfd6eSbjcO97hHerKihesPjHp76AehphFhsMh\nw6AlCZw8ebJQD6MLM50uBO6OvHcl8FngCRPKFp4lUCV1zcgpet0y+Z/W18tvE9sWWVanHzw4TvYY\n5PGKJgwUYtqhr9NqgYuAe0LHlwH3AgsZylZXgyXJO+e/6N4Zk64bTVUR5JSaRBVblBah6MK3pHJZ\n8l9Fs8lKLMSs0UvBAG4Cvo6fQvsA8Fr8zKj7gTtGj/ellK+4GsuRtWWft+eQ9bobG87t3j1etLe6\n6o/z7K9R5crlSdcq2oOaVE7JAIVIp5eCUfbRJcHIm9k1a9K6JnsYacd5ySoGRVdjx5ULekVF91UX\nYlaQYLRI0cyux4+nt4TbGMPIc59JZHXeRXsE4XLB+MSBA+Pxif3748cntApczDoSjJbJm9l1dbWe\nlnCVLfYmRKWWAAAK7klEQVQqmCQGVdq7vn72RlRVjQ0JMU1IMCqmjpBF4DyPH+/WrKqofVXF/pPC\nRmF7i+yBnfY9o722tGtprEPMKhKMCqkjZBF1ntGxharCIUWFro69I+L2u44KRNE9sOO+59qaz9Qb\npHdfXU3+u6mHIWYZCUbFVOlQuh4zr3MdSfS4LkcdfIcg1Le87MUjLMxV7xYoRF+RYNRAlSGLrs/K\nadK+ukJB0Wm1T3lK+kSEuLJCzAJFBaPruaRaYzj0qTVWVvxzaFV9IarK7FoXTdk3qV6jea/yHO/Y\nsf36//iPcPnlPr1JNC9W1/8eQnSSIirTlQc9GsPI2qKd5pZvkenHwRhH8AjGQOJWoseVX1yspzcj\nRJ9BIalqqXrFcxYBmoXYep7pxwsLfl1FsIPg6qpfWzE/74VgYSF9gWTcNFshhASj82Qd7C26Unua\nCI9xBPW2vOycmX/etWtyr2EWxFeIokgwOkJaC3rSYG/g5F7zGn/e8vJ4f+u6bauzbB7ihDWot6Ul\n/7xzZ7ZewzSH94QogwSjA6S1arP2MN71Lv9Xec5zfIv6yJH8LeM4R1mmxd1Uaz3uPvv3b+9h7N3r\n068HK7vVaxAiPxKMklTVGk1a3ZyUNjycXjs47yUvGYtGdC3BJPuDawRl0kQrz+LBPOsnqurJBIPc\n4TGM8HG47oQQ2ZFglKDqFnRc6Cnayg8ceHT187vf7UXihS/013jNa/Lbv7rqr3H8+NnOvUx6kizr\nJ6quy+i6iaR1FEKI7EgwSlLVCuRJ10lKmrd3r3fic3M+DJW0WjnrfZeXz3buRRMgZvleRc8VQjSP\nBKMCklrQedZQZGldx90neO/qq7eHlILwS5aB7+AaQS6lpLBY2LakFOtlkwQqsZ8Q3UWCUZKkVnHe\nEEuedQbRHkbw3q23br9nOGY/yf6gl7K6evZ4SVR04jZxShogz5MkUD0MIbqNBKMAWQaKg+MqHGDW\nDK6BLXkGmaMCE75u0qrogwf9LKRAtA4cGAtD0e+s9Q9CdJ9eCgZwI/AwcHfovScBtwFfBm4F9qSU\nL1xh0TBL4GSTBlWrCrEkTXmNOyfPPYtkho2eE105XfQ7a/2DEN2mr4JxGLgkIhhvB35j9PotwPUp\n5QtX2Oamb13v2eMd4p49/niS441LR1EHVfVqsjj9uHOy3F8zmIToJ70UDG83F0YE40vAk0evLwC+\nlFK2VKWtr/tVw8Hq4bgWeFrYpi6qCutkcfqT1o0k3T8cxgvWSISPJRpCdJdpEoxHIp8/klK2VKWt\nr4/zEu3alc+h1k3ZsE4epx93Tpb7x+V50iC3EN1nmgXj2yllC1fY5qbvLczPezGYn/fHcY6xr1NE\nszj9ssIUzfPUtzoSYhYpKhhzGbfNaJKHzezJzrmHzewC4JtpJ584ceLM68FgwGAwyHyj886DW26B\nwcA/3vrWs8+JbvgTnNsHsmwSVGYjoaBulpfhAx/wz32rIyFmgeFwyLDsLnCAebFpDzO7CPioc+7i\n0fHb8b2Mt5vZW4AnOediXDmYmStj/9bWdgcZd3z4sN+pbTDwDvLoUTh9Wju0BXVz7bVw7Bi86lVw\n883jY9WREN3FzHDOWe5ybQqGmd0EDIAF/PTaFeAjwCngqcD9wL93zn0noXwpwcjCJFGZZYK6SHoW\nQnSTXgpGWZoQDCGEmDaKCsY5dRgjhBBi+pBgCCGEyIQEQwghRCYkGEIIITIhwRBCCJEJCYYQQohM\nSDCEEEJkQoIhhBAiExIMIYQQmZBgCCGEyIQEQwghRCYkGEIIITIhwRBCCJEJCYYQQohMSDCEEEJk\nQoIhhBAiExIMIYQQmZBgCCGEyIQEQwghRCY6Kxhm9mYz+6KZ3W1mf2Rmj2/bJiGEmGU6KRhm9hTg\nV4GfdM49G5gDfqldq7rNcDhs24TOoLoYo7oYo7ooTycFY8QO4FwzmwN2AV9v2Z5Oo3+GMaqLMaqL\nMaqL8nRSMJxzXwfeDTwAPAR8xzm31q5VQggx23RSMMzsR4DLgQuBpwC7zew/tmuVEELMNuaca9uG\nszCzVwIvdc7959Hxa4CDzrnXR87rnvFCCNEDnHOWt8xcHYZUwAPAITN7IrABvBj4fPSkIl9YCCFE\nMToZknLO/SXwYeBO4P8CBvxuq0YJIcSM08mQlBBCiO7RyR5GFDO7zMy+ZGZ/Y2Zvifn88Wb2ITP7\nipn9hZn9WBt2NkGGulg2s2+a2R2jxy+3YWfdmNmNZvawmd2dcs5vj34Td5nZJU3a1yST6sLMlszs\nO6HfxLGmbWwKM1s0s0+Z2b1mdo+ZvSHhvKn/bWSpi9y/Dedcpx94UfsqfsbU44C7gGdGzrkKeN/o\n9auAD7Vtd4t1sQz8dtu2NlAXh4FLgLsTPn8Z8LHR64PA7W3b3GJdLAF/2radDdXFBcAlo9e7gS/H\n/I/MxG8jY13k+m30oYfxfOArzrn7nXM/BD6En3Ib5nLg/aPXH8YPkk8jWeoC/JjPVOOcOw38Q8op\nlwMfGJ37OWCPmT25CduaJkNdwAz8JgCcc99wzt01ev194D5gX+S0mfhtZKwLyPHb6INg7AP+PnT8\nIGd/6TPnOOe2gO+Y2fnNmNcoWeoC4BdHXe0/NrPFZkzrHNG6eoj4upoVDpnZnWb2MTM70LYxTWBm\nF+F7Xp+LfDRzv42UuoAcv40+CEYRZqI1lcCfAhc55y4B1hj3vMTs8gXgQufcc4H/AXykZXtqx8x2\n46MNbxy1rmeWCXWR67fRB8F4CAgPYi+O3gvzIPBUADPbAcw75x5pxrxGmVgXzrl/GIWrAH4f+JcN\n2dY1HmL0mxgR97uZCZxz33fO/WD0+uPA46a0Bw7AKP/ch4EPOuduiTllZn4bk+oi72+jD4LxeeDH\nzezCUYrzX8K3osN8FD/YC3AF8KkG7WuSiXVhZheEDi8H/rpB+5rGSO5N/inwnwDM7BA+H9nDTRnW\nAol1EY7Pm9nz8dPpp7FBFfAHwF87596b8Pks/TZS6yLvb6OrK73P4JzbMrPXA7fhBe5G59x9ZnYS\n+Lxz7s+AG4EPmtlXgG8zpanQM9bFG8zsF4AfAo8AV7ZmcI2Y2U3AAFgwsweAFeDxgHPO/a5z7s/N\n7GfN7KvAPwKvbc/aeplUF8Arzewq/G/in/AzCacSM7sUeDVwj5ndCTjgN/EzC2fqt5GlLsj529DC\nPSGEEJnoQ0hKCCFEB5BgCCGEyIQEQwghRCYkGEIIITIhwRBCCJEJCYYQQohMSDCEEEJkQoIhhBAi\nExIMIYQQmZBgCCGEyIQEQ4gKMLMdZnbazL5vZs+IfPZfzOwxMzvRknlCVIJySQlREaO95O8CvgYc\ndM790MyeBfwlPtPwv3b6hxM9Rj0MISrCOfcA8Dr8zmbvNrMnAjfjs4C+WmIh+o56GEJUjJndAFwF\n/B/gXwG/mLCRjxC9QoIhRMWY2ROALwJPA37POfdfWzZJiEpQSEqI6rmE8Va6P2Fm+j8TU4F+yEJU\niJmdB/wv4FvA1cBPAydbNUqIiuj8Fq1C9IzfA54K/Ixz7tNm9pPAW81szTn36ZZtE6IUGsMQoiLM\n7HV4wbjWOfdbo/f24KfazgHPcc490qKJQpRCgiFEBZjZvwC+ANwBDJxzj4U+OwR8Gvi4c+7lLZko\nRGkkGEIIITKhQW8hhBCZkGAIIYTIhARDCCFEJiQYQgghMiHBEEIIkQkJhhBCiExIMIQQQmRCgiGE\nECITEgwhhBCZkGAIIYTIxP8Hj6CDgmzGXT4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x3fff3c05df90>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Generate some random data.\n",
    "n = 250 # Number of datapoints.\n",
    "d = 1 # The dimensions of the datapoints.\n",
    "slope = 12\n",
    "bias = 10\n",
    "\n",
    "# Randomly generate input.\n",
    "sample_x = np.random.rand(n, d)\n",
    "\n",
    "# Consider the equation of the y = m.x + b\n",
    "sample_y = slope*sample_x + bias\n",
    "\n",
    "# NOTE: We do not want a straight line. So add some random noise to the input.\n",
    "sample_noise = np.random.rand(n, d)\n",
    "sample_x += sample_noise\n",
    "\n",
    "# Plot.\n",
    "plt.scatter(sample_x, sample_y, marker='x')\n",
    "plt.xlabel('x', fontsize=18)\n",
    "plt.ylabel('y', fontsize=18)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Try__: Change the bias and slope, plot the points."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "__Step 1__: Create the tensorflow placeholders for the $x$ and $y$ points. Also create a tensorflow variable to store the weights $w$.\n",
    "\n",
    "Tensorflow defines variables as [[2]](https://www.tensorflow.org/programmers_guide/variables):\n",
    ">A variable is the best way to represent shared, persistent state manipulated by your program. Variables are manipulated via the `tf.Variable` class. A `tf.Variable` represents a tensor whose value can be changed by running ops on it. Unlike `tf.Tensor` objects, a `tf.Variable` exists outside the context of a single `session.run` call.\n",
    "\n",
    "We use `tf.get_variable` to create variable instead. It is a wrapper that allows you to automatically add an initializer, a regularizer and many other options [[3]](https://www.tensorflow.org/api_docs/python/tf/get_variable)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# NOTE: The name of the variable is optional.\n",
    "x = tf.placeholder(tf.float32, shape=(None, 1), name=\"X\")\n",
    "y = tf.placeholder(tf.float32, shape=(None, 1), name=\"Y\")\n",
    "lr_rate = tf.placeholder(tf.float32, shape=(), name=\"lr\")\n",
    "\n",
    "# Weight & bias.\n",
    "# w = tf.get_variable(shape=[1], name=\"w\", initializer=tf.zeros_initializer())\n",
    "# b = tf.get_variable(shape=[1], name=\"b\", initializer=tf.zeros_initializer())\n",
    "\n",
    "# Initialize with a different value.\n",
    "w = tf.Variable(np.array([[5.0]]), dtype=tf.float32, name=\"w\")\n",
    "b = tf.Variable(np.array([[5.0]]), dtype=tf.float32, name=\"b\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Step 2__: Once the placeholders & variable have been created, compute the $y$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Compute predicted Y.\n",
    "y_pred = w*x + b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Step 3__: Once the predicted $y$ has been computed, define the loss between the predicted $y$ and the actual $y$.\n",
    "\n",
    "The loss can be of various forms. In case of linear regression, we can use the L2-norm or the euclidean distance between the two points.\n",
    "\n",
    "$Loss(y) = \\lVert y - y_{pred} \\rVert^2$   \n",
    "\n",
    "i.e.  \n",
    "\n",
    "$Loss(y) = \\frac{\\sum_n(y - y_{pred})^2}{2*n}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "loss = tf.div(tf.reduce_mean(tf.square(y - y_pred)), 2*n)\n",
    "\n",
    "# This can be done in a single line.\n",
    "# loss = tf.losses.mean_squared_error(y, y_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Step 4__: The loss shows how far we are from the actual $y$ value. Use the loss to change the weights by calulating the gradient w.r.t $w$. We use a stochastic gradient descent optimizer for this purpose."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a gradient descent optimizer with the set learning rate\n",
    "optimizer = tf.train.GradientDescentOptimizer(learning_rate=lr_rate)\n",
    "\n",
    "# Run the optimizer to minimize loss\n",
    "# Tensorflow automatically computes the gradients for the loss function!!!\n",
    "train = optimizer.minimize(loss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Step 5__: `train` is the variable that is evaluated when the session is executed. The linear regression functions are now defined. While training `train` is evaluated and while testing or predicting $y$, `y_pred` is evaluated.\n",
    "\n",
    "Initialize the variables in tensorflow."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Initialize all variables\n",
    "init = tf.global_variables_initializer()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Step 6__: Now we are going to build the model. This is accomplished by evaluating the `train` variable for a set defined number of loops or __epochs__. During each epoch, a sample $x$ is used to compute $y_{pred}$ and iteratively the $w$ is changed while the loss is reduced. Each time a loss is generated, the gradient computed can be used to update $w$, but this gradient is not constant across points (because loss is not constant).\n",
    "\n",
    "It varies for each $y$ and so only a small portion of the gradient is used to update the $w$ parameter. This helps stabilize the $w$ from \"jumping around\" a lot while never reaching the steady state.\n",
    "\n",
    "So how do you define the best learning rate? Trial and Error."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "W: 7.67125177383, B: 8.05626583099, Loss: 0.0129069695249\n",
      "Slope: 12, Bias: 10\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEXCAYAAAC+mHPKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXt4HOV977+vd7Wr1dWW7MKxZSzAdpALjk2C7QSoRcI1\nJBBCCAVKHDAkkDg2FzcQg49wU04AF1xoIYpdp24ShCjpIY2bJqZqVjlVThLRcHOzJk1OaiDXUXPr\nk4ZHRuZ3/pgZaXY0Mzu33dnL9/M8+2hnd2f2ndfr9/v+Lu/vVSICQgghpBRzkm4AIYSQ2oCCQQgh\nxBcUDEIIIb6gYBBCCPEFBYMQQogvKBiEEEJ8kahgKKV6lFJfU0p9Vyl1UCm12fLeR5VSh4zX70my\nnYQQQoB0wt8/BeAWEXlOKdUG4DtKqacAHAvgXQBOEZEppdT8RFtJCCEkWcEQkZ8B+Jnx/LdKqUMA\nFgH4IIB7RGTKeO8/k2slIYQQoIpiGEqpXgCrAHwbwHIAf6CU+pZSKq+UenOSbSOEEJK8SwoAYLij\nvgBgi2FppAHME5F1SqnTAPwtgBMSbSQhhDQ4iQuGIQ5fAPA5Efl74+VXAPxvABCRp5VSryulukXk\nF7ZzWQiLEEJCICIq6DnV4JL6DICCiDxoee2LAN4GAEqp5QCa7GJhIiJ8iGBgYCDxNlTLg33BvmBf\neD/CkqiFoZQ6HcBVAA4qpZ4FIAC2AfhrAJ9RSh0EMAng/cm1khBCCJB8ltQ3AKRc3r66km0hhBDi\nTTW4pEgM9Pf3J92EqoF9MQP7Ygb2RXRUFH9W0iilpJbbTwghSaCUgtRo0JsQQkgNQMEghBDiCwoG\nIYQQX1AwCCGE+IKCQQghxBcUDEIIIb6gYBBCCPEFBYMQQogvKBiEEEJ8QcEghBDiCwoGIYQQX1Aw\nCCGE+IKCQQghxBcUDEIIIb5IVDCUUj1Kqa8ppb6rlDqolNpse/9WYz/vrqTaSAghRCfRHfcATAG4\nRUSeU0q1AfiOUuopEXlRKdUD4BwALyXbREIIIUDCFoaI/ExEnjOe/xbAIQCLjLd3AfjjpNpGCCGk\nmKqJYSilegGsAvBtpdRFAF4RkYOJNooQQsg0SbukAACGO+oLALYAOApgG3R31PRH3M696667pp/3\n9/dz315CCLExOjqK0dHRyNdJfE9vpVQawD8A+IqIPKiUOhnACIDfQReKHgA/BrBGRDTbudzTmxBC\nAhJ2T+9qEIzPAvhPEbnF5f3/AHCqiPzK4T0KBiGEBCSsYCSdVns6gKsAvE0p9axS6hml1Pm2jwk8\nXFKEEEIqQ+IWRhRoYRBCSHBq0sIghBBSO1AwCCF1zcTEBJ5++mlMTEwk3ZSah4JBCKlbHn/sMZy0\nZAluOOccnLRkCR5/7LGkm1TTMIZBCKlLJiYmcNKSJci/+ipWAngBwFm5HF586SUsWLAg6eYlCmMY\nhBBi4fDhw+jNZLDSOF4JYElTEw4fPpxgq2obCgYhpC7p7e3F4SNH8IJx/AKAl157Db29vQm2qrah\nYBBC6pIFCxbgkb17cVYuh1M7OnBWLodH9u5teHdUFBjDIITUNRMTE7p7qreXYmFQs6VBokDBIISQ\n4DDoTQghpKxURXlzQggpxcTEBJ599lkAwOrVq+leSgBaGISQqmZiYgL/60//FCcuWoRLzjsPHznv\nPCzr6eEivARgDIMQUrXs+fSnccvmzZg6cgQ5AKMAF+HFAGMYhJC6Ys+nP40tN9yA448cgUDfSc26\nCK9nzhwuwqswtDAIIVXHxMQEli9ejK9PTmIldMviHQC+BVoYcRDWwmDQmxBSdRw+fBjHZzJYOTkJ\nAOgH0AFgHYCFAP4zk8GnuQiv4iQqGEqpHgCfBXAMgNcB7BaRv1BK3QfgXQAmAfw/ANeIyH8l11JC\nSCXp7e3FS1NTeAEzFsV/Afjg5s14x4UXMksqIRJ1SSmljgVwrIg8p5RqA/AdABdDd1d+TUReV0rd\nA0BE5OMO59MlRUiN47YS+/HHHsOHN27Ecek0Dh85gvsefBDXf+hDCba0fqiLld5KqS8C+AsR+WfL\na+8GcKmIXO3weQoGKeLoUSCVcj8m1YUpCr2ZDA4fOYJH9u7F5VdcMf1+1LIe/D04U/NZUkqpXgCr\nAHzb9ta1AL5S6faQ2uPIEeCMM4DRUf34n/9ZPz56VD82/5rYj+uZarz3Q4cO4YPXXIO/e/VVfOc3\nv0H+1Vfx4Y0bi3bGW7BgAU477bTQYmH9PYyOFv8eSHCqIuhtuKO+AGCLiPzW8vodAF4TkSG3c++6\n667p5/39/ejv7y9fQ0nVcvQosH49cPnlwGWXARdeCDz6KPDVr+ozSnPw+OQngf5+ffD4+MeBsbH6\nn3FW470//thjuOGaa3Ds5CQuBfAIgMsxs19FHPGJVEq/58suAz7yEeDhh4Ennqj/f28nRkdHMWoq\nZxREJNEHdNH6KnSxsL7+AQDfAJD1OFcIMcnnRebPF1m/XgQQ2bDB+f2BAf1vPl/pFrozNeV9HJVq\nundN06Qrl5PnARFAngekC5A8IF25nGia5nhe2D4aGNB/DwMDkZpdVxhjZ/DxOsxJcT6gZ0k9YHvt\nfADfBdBd4tz4epDUBRs26L/q9eudB8ZqHDympkTWrp1paz6vH8cpGlNTxfcetyCVQtM0GR8fn/57\naken3hjjsQyQtqasLFs6NN02axud+mjdutL3UU1CWU3UpGAAOB3AUQDPAXgWwDMALgDwfQAvGcfP\nAHjE5fy4+5HUMCMjIum0Lhrz54vs2lU8qAQZPMo947dfe8UKkY4OvW0dHfpxXN9Z7uuXYnhoSLpy\nOTm1s1O6cjkZfGRQWucUWxg5ZKWzszD9bzI1pf/bWQWiry/Y4O90DT8i0wjUpGBEfVAwiIk5OIyM\n6Mfm4DA5Wfy+n8Ej6Ge9jt3aaj/O50VaWvT/kS0t8c6Ep6b0wbazUx9sOzv140oMnIVCQTqyWclb\n3U+5nNxy86DkkJMV2Q7JISfA0Cyrz0ngg1qIlRT+WoKCQRqeUoNDkMHDjzUSZgbrds7IiEgup/+P\nzOXid50k4ZIaHhqSudmsLDdiFMOGaKzu6JDx8XHZulUTYFyamzXXfra2me6l+KBgEBIT5mBqDlbb\nt7t/NswgZj9nZKT8FkBYd9zU1OxjP5QKbD/5pCbz54ssWqTfbz4/W3Ctbe7u1t1odC/FAwWDkBgw\nLYBdu/TBasMGPS5iurqcCBNIt8/2yxn0DuuOM2MfpngFGaTHx8fl1M7Zge2ObFaGPj9U9B3W61oD\n3nYX49q1My5G8zMkHBQMQmKiVPDcShwWhjlwWol7MAzrjuvomLF8griBnCyMudmsFAoF3+2ZnCwW\nupERWhVxQcEgJEa2by+2GqIGx6OcExdBRMNqAXlZUJqmyYEDB+TAgQOz1k+Y2VGrOzqkK5eT4aGh\nwG1m3KI8UDAIiQmvQco+yFpdJE7vO5FE5k4QobLef3t7sYVhdc3tHhyU1lRKWgBZCkhnJjNLFKzr\nL8LiJFi0MqJBwSAkIE4Dt9fAWk15/WFEJ2jm1+SknrF13HH667t2ibS1ifzoR5psu/12aQZknuFu\nsqbMRhEHpzZ3d4v09OjuMafgOAkOBYOQAJQSBvtnTarBRRJFuPwE6K3XGRkpvt87Pj4kc5ubpQmQ\nHkBOtQS1BZCVra0yPj4e+t7s7TDvM5/XLZ1Fi+iaigMKBiEBCTv4l6O8SFCLwZ5y6uU2czonzP1+\n+MP6Irz7AMkB0unTwojigrN+thrLutQqFAxCQhB0ECqHhRHWYjDb3tPjnklkT1MNU4upq0uTM0+/\nXTJIywmAtBkxi2HjeQsgJwLSnk7PimHE5carBsuunqBgEOIDu3upu7s6ahMFHRCdUl+3b9fTgXft\ncm5f0Jn+1JTI8qVD0p5qkhMNYWgDZLthYTwPiAbIpwDJIiUHDxZiuTendlRL7KheoGAQUgL7orS+\nvpkifH4HoXJmOPm1dpwG0EWLZLqkexwzcTNdti2dlkcNYXjecEHNBeT3DNFYnspKDjnZfqd3ymxU\ndxJrQsULBYMQHwRJma0kQWfhXoF4s8R72MHZXD9xbFOTtBiBbbMW1GrDHdXW1CTXXDMmgF4TKs57\nI+WHgkGIT9xmu3HUUApDFJeL/dxdu3S31Pbt4QZnc4V23rAkrAHteUaguxmQW24e9CUCdCdVJxQM\nQnzgNtuNq4ZSWOLIJHKqvxSk3Zqmyb59++SU9nYZN6wJa8rsiYCkAdl5785AmxnRnVR9UDAIKUGp\n2W6pGkpJDXxO1k6cg7OmaXL3Jz4hc5ubZXV7u+SMwHaXzcLIAdLSslN27ZpdHJEiUFvUpGAA6AHw\nNWM71oMANhuvzwPwFIDvATgAoNPl/Ji7kdQ7pQZUaw0pe0XZJFwrZiVbs7R3Pq9bPnFVs909OChz\ns1lZarichi3isDiblRwgS1WztKez0tY66Kt6L6l+alUwjgWwynjeZgjESQDuBfAx4/XbANzjcn68\nvUgaGmuV2o4OkdZWfY2DuTAuqeBtPq+3p6VFL9Vh7h8RBU3Ty3vkbFZEl5ERtaq9Xfbt2yeFQkG+\n+U29FpSf/UFIbVCTgjGrMcAXAZwN4EUAx8iMqLzo8vkYu5DUEnG7h0wLwtwHY/58kVRKFxGrNZHU\namPze6N+t+l+6shmJQvIG21xitWAPOqwYpuZTvVFzQsGgF4Ahw1L41e2937pck58PUgSIczAXy73\nkHm+00zaDH7HMWiGKQMSh4VhdT91AnKcQ5yiBZC5zc1FK7aZ6VR/1LRgGCLxrwAuFgeBAPALl/Nk\nYGBg+pHntKemiDIQlWvG6yeLKmhbrQS9TlwxjN2Dg7PcT9bg9krj+OO33+5YbZaZTrVNPp8vGitr\nVjAApAF8FcAWy2uHbC6pQy7nxtmnJAGCDPxeAeo4sA/m9rpMYfa+cCLMIj2/WVJOaJomLamULLW5\nn04GpB2QXuhbp+4eHAxxN6QWqWXB+CyAB2yv3QvgNuM5g951jt+S23EuUPP6HpGZ7UHNbKC4twet\nRCxE0zTZs2ePXHrppZLF7MqyLYZQ3P2JT8hPf+pdYdYOLY7apiYFA8DpAI4CeA7AswCeAXA+gC4A\nI0bW1FMA5rqcH3c/kgoTZLZtfrZUkb2o2APg9lTSqINlJQLIw0NDkjXcTMuMvxcY7qfVhlh84P3v\nF03TQrnJGNOobWpSMKI+KBi1TZiBx3RDmQFpt0qsUTEH9fXrZbqoX9g2W6nEYFsoFCSXTkuzQ8xi\nzMiCam9qKpkF5SWMzJqqbSgYpCYJMlu3rpOYP1+3AOLMjrIfm0X81q8vHhSjFjB0+r44XDzTK7aN\nTKgWYyGeGbNYCsjCpibpyuVm7VshEnyhIjc0ql0oGKSuKeUminpd+8D41FPe4mQOlosWld68KGwb\ngtSA2nb77dJqVJe1FwzULBbGnj17HLOgnETQSxid9hKhS6p2oGCQusdrnUQU7AOjOfDbi/iZWVLW\nz3d26mskosZVwrp4dg8OSns6Pb3B0bG2TKgToe+9nQPkgnPPdbyGl2A5WRHlLldCyg8FgzQE5drP\nwj4wurmInAbXnh4JtXlRlDRhTdPk+uuuc1xbkbccdzQ1yY033ihjY2OB2lJqoaL5vpeVwUyq6oWC\nQeoeewnyMDvmORFmXYT1XOug6XfzoihpwrsHB6UJkIyRAWVfW5E1LAu3WIUf/LjJpqZmxHJgoPgz\nzKSqbigYpCHwGqzDZOq4DWzWRXpOC+eczg26NiRMmvD9O3dKzhCETuNhj1lkAdnmsmI7CKUshFIl\nS5hJVb1QMEhNE9Z9EUemjv27zEV7fiwZu3gE3bzILR5jv76mabJl8+ZZLqg2Y22FmRWVmzOnIiu2\nrWJpTQCw3y8zqaoTCgZJBK+B3q8I+JnlO51fzhmsU2Dbz3akXsde3+F0XbNfbvjgTskqJRnDsrBX\nl/2UIRw5paRQKIS423CUinPQwqheKBik4nj5qYP6sN0yldzONzN1rO/HnaVjnR3HPVP22z/vvOBa\naTash08Zf+3lPZoNyyJsvKIc98AYRnVDwSCJEGWGWSpLqFSWTjkFI4yFERQvi0TTNHn88cclZ6zM\nPtVYT9FmxClWAzIXkCZArr/uusQqzMZhYZLKQ8EgieE1+3Z7z2/A2Ova5XJ5lCsbyy/DQ0PSlcvJ\ncc3NsswQCnPfimEj0L0IkNZUyjVewRk+8YKCQRIhioVRKktoZKS0IJQrqGqfKXvNnONi//79cuml\nl0pzKiV5QArGugpTKOYZMYzWVMpXFlQ5BJVWQ31AwSAVJw4ftluW0O9+5703hXnNegmqrj755Ol0\n2RYjNXYYkM2YqTjbDMjF73xXoHTZOAWVVkv9QMEgiRDFh+024LulqNrXRlTr4BV0Fv7II4/MSped\nB0gH9FXbHdmsPPTQQ4EzoMohqPUk0o0MBYPUFKUGfD8DU7W4R+wiaQ/GuwlZoVCQ9WeeKTDWUdjT\nZRcZYuGV/RSkhElcgsq1FbVPTQoGgL0Afg7gBctrbwTwTWNDpXEAb/Y4P9ZOJJWl1IAftzvF6zjK\nde0D80knzU4RtnPZe95T5GpysjBa58xxtCr8LhYsdc9h+oQWRn1Qq4JxBoBVNsE4AOBc4/kFAPIe\n58fYhSRuogzScQ5M5XZfWdva3a1nVJk1pTZsmP1da087bZZAZAzRMGMYGUDu37nT8V6sFkzYrWr9\nuP3czqlGNyAJRk0Kht5uLLEJxlcAXGY8vwLA5z3OjbELSZxEGVzKMTCVe2ZstYZ27RJRSt94SamZ\n7C8RkR0DA5IF5I02F5RZ2mOhsbZifvcVjvdrpvl2ds6sEZk/P5wlFmZDqmpxA5Jo1JNgnATgJQAv\nA3gFwGKPc+PsQxIzUQbpcgxMQV1cftvgx8I4eLAgDz30kGTmzJHjLesqrGXJFwKShZLWlp2efZXP\n68X+AJFMZkY8wgih266CpL6pJ8F4EMC7jefvBfBPHufG2IWkHFRLgDSoePm1cpw+19dX/F2XXLRJ\ncoAsN4QhA8i9hmisNF5LA7J2zWYBtJJ9ZRUM04Ixv9e62t1vlpp933JS/9STYPza9v5vPM6VgYGB\n6Uee06OqotxuoKjFDaMWB3T6XnuM4aGHxhw3OWoG5HhjvcXJfSvkySc1X981OalbMB0dM9bB/Pmz\nNzAqdc/m+3FveUuqk3w+XzRW1rJg9AI4aDn+LoD1xvO3A3ja49z4epTESpBBOoj7yW+GUJTvsBLG\njXXwYEG2bdsm55x9tuRSqVmbHC0FZH4qI82plGx4/4CsXesvFdcUo74+/TP5vMiCBeK48NG8jpcI\nWcu4izgvjiT1SU0KBoAhAD8BMGnELK4B8FYA/2qk1X4TwGqP82PuRhInfgbpoMISZcOioIRxYx2z\nYKOkLKuzc5hdYTYHSGvr47J1q+a4tal5rVJtsu4nHrZ0CoPYjUlNCkbUBwWjPggyMNs/63dL1KAE\ndWNpmiY3bd48XYrcLhBzDcsiB8jmTZsixXasGxZ5tY9rJogbFAxS0wQZQM3PmumgScVIzOMdAwPS\nkkoVlSK3uqCWAXIbIG1NTTI2NhZpIPc61x5L4ZoJ4gYFg4QmabeE30HQ+tkg+2CXA3NAXrFcX4R3\nsmFZ7HZJmTVLfNiD4va4Qan4TRARSPrflVQvFAwSiqRnotbvNxelee2f7RWkreSAWCgU5KRlb3Is\n6bEbM6XIc4C0t6yVJ5/UK8zm8zOpr0ED9yIUARIPFAwSGuus3V41thLY/e7d3d679NldL5UeNK/f\nuFGajLUTK23upxMxs7nR2WefLV//urcLinEGkgQUDBIJc3tUc/FWkj5v+1atVgvCbpFUup0f27p1\nuuZTDrOLBubgvGVqmF0JCSkXFREMAP8O4DYAx4b5srgfFIx4MGe5GzboK4fNYHISs123+kbWNQP5\nvJ5SumhRZdt5/caNswSi1RCJU4y/a9982qzznKwIU+DcrDtCykmlBOO7AF4HcATAFwG8E8CcMF8c\nx4OCER17XMBMU3VaCFaptpirj6++unj1sXXLVrM0Rpyzcqf4QKFQkH379snnP/95x6KBbwTkf0Av\nGHjpJe9xvSdrjMhcqGfGYExRDLpwjvEMEpaKuaQArAOwB8BvABwF8GMAdwM4MUwDojwoGPFgn+3e\neWf8sYwgZTxEZtxSV18907Z162Zeb2mJd3Mlp4F9XueFMgeQYw1BcCoa2A5Ii1IyNjYW6N7d+npy\ncnaJcaegvj3bKohrjkJDKh7DANBirMweM6yOowC+BuBKANmw1w3Yhlg7sZGxZuxEmfV6Xdvv4Obm\nItu1Sw+I9/ToK5xN95TT9q1hBlOr6yiN44pWax9v/LUXDcyqlCxbOhSqf0zxMwXDLCDY1lZsVVld\nctaMKnuRQz/urKSz4kh1kGjQG8ByAI9ahOMXAP4cwHFxXN/je+Psw4annH71IIX8nFxkd97pHuy2\n10QKO5iKiGzdqgmwSrLQ99O2BrNbLOKRBeSk5RdJV5cWqn/sopjNijQ367EZ0yV3xx3Fa02cSqF4\nBczdLAlmZpFEBANACsAlAP4BwJQhFiMAvmwc/zeAi6N8R4nvj7kbiUk5Mnf8XtNNuOyVVEuVwQh6\nD7fePChZqGnLoguQYUM0lgIyB5CHAGltapJrrhkL3T9uomjNUjNfu/pq91IopRY8elkSzMxqbCoq\nGNA3OdoJ4GeGVfEzAPdY4xgAlgJ4BsD3wnyHz3bE3I1EpDzrMsIU8gvqOrEOgkG/b+e9O6XZWHBn\njVF0GZaGmUI7r7lZtt85FHmGbhXFjg49iJ/J6NbGeecVu+KsmzHZN2ryWsTo1ge0MEilsqQ2AviG\nYUkchb7/9qUA0i6f/wCAqTAN89meeHuRlCWW4TX4ewVggwRnrYOgfTD1EhtN02TL5s2SAWQxZteB\nWmq4nxQgl192mfz0p1psMQAz1mK21bwHa7DfdEPdeefsUih+VozbLQnGMIhI5QTjdaMc+d0Aen18\nfg2Avw7TMJ/tia8HyTR2l9Add8y2NOyZPPZjt2taj+MavNxSV73ER0Rk9+CgtKfTciL0arKtDhZG\nDpDly5ZJoVDwvJcgx/b33LLUTJG2B8CdsqacrAY3S4JZUqRSgvFuAKkwX1SOBwWj/DitAF+zRs/k\nsQZj29pKi4YTcblHggyCplXRbBOHTkM0WjCzknv9mZcGXhtRaqc7p3a6neeWYuuE1ZqgJUG8YGkQ\nEjteK8B37dJfW79+Zl/psINRJQOwuwcHpS2VkhMNYTCD2gLIKkA+BX3P7SbMkXPP3hlKxLxm9mHE\nJOx30pIgbtSkYADYC+DnsOzpbbz+UQCHABwEcI/H+XH2IbFQagX41JRIe7v+2vr14SvHVjIAe//O\nnY7VZTXjeQv0vba3bN5spNeGFzE3ESzH/YZ1yZHGpVYF4wwAq6yCAaAfwFNmIB3AfI/z4+xDYsNr\nXYZpYaTT+q/ItDLMz/txf1TSbbJ7cFAyRhDbXl3W3AmvOZWS3YODkQf1UueXw6KKaxU4aQxqUjD0\ndmOJTTAeB/A2n+fG1oGNTKlMJfugbo1hmINfOi3S1RV8kC2322RsbExuvfVWaW1qkuMxe/vUFsMF\ntWXzZtE0LbKIlTq/UhYVU2eJF/UkGM8CuAvAtwDkAbzZ49w4+7Ah8TNAOg3qk5PFg1JXV/n213Zr\nt9expmmy6pRTJIOZsh45SybUaiMzqgmQHbYGRxWxoIHtcs38uTiPuFFPgnEQwIPG89MA/NDj3Bi7\nsHEJMxu1D35OZSu8zvU6Dvrd9oF3x8CAZKEXDcwB0ga9BpQpGDnomxw1A3L9tdcG+/KIVCoQTQuD\neFFPgvGPANZbjn8AoNvlXBkYGJh+5Pm/IjRhZqP2mbOfrUbjmmW7DYjrTtP32F5quJtajcdcw9JY\n2NQkbZmMfGDDhqK1FfUEU2qJnXw+XzRW1rJg9AI4aDn+IIAdxvPlAF7yODfGLm1cgsxGvdwtXp8L\n+31eWEVO0zQ58/TTZ62tmIfigoF79uyZtRueG7WcllrLbSflpyYFA8CQsXJ8EsDL0MulpwF8znBN\n/avV2nA4P95ebECCzEbjnLlG9a9bRaejbUjaU02uaysWGYHtM976Vt/Xd8o0sqeqxgkHeFJJalIw\noj4oGPFQaesg6jXMcub79hXkpptukhY1x3VtRacRq7jone8M9B1TU3qJ9M5OvZ2dnfpxOQZyupBI\npaFgkIphL0Fhxc0ysT6PskbAHFzPOetKyRnrKOxWhXVtRRaQ+3fuDH6TRtvMrWBzufIGjoOKKC0S\nEoWwgjEHpC45etT7OCyjo8DDDwMDA/rfU07RXzPfO+OM4u86elR/zfzMv/yL/vfMM/W//f3A2BiQ\nSvn7/lQKmDpyDkbzQxiCnhHxTQA3ApgA8AKAn0L3b7759NPxiqbhlq1bQ9+vUsV/zXuyEkff9vcD\nH/kIsGOH/re/3/2z9j516ndCykIYlamWB2hhOFIuF4fTdVes0MuJe9UwGhmJJ8itaZr0LV9etHXq\nZotV0WNaFXPmhLYqTKam9Hvr6DDiJB36sdPufnH0rV8Lw2v1PSF+AV1S9U1QF0S58vCdypr7qZJq\nVr0NG+QeHhqS1jlzZq3UzgHypOGWSgPywAMP+M6C8sIr6B133/oV+FL1vQjxCwWjjglrMcS90tev\nhWEfUM09qsMMsIVCQe6++25pS6flUThvcGSW9zjzLW+J50Yt9+t2XI6+9To28aogTIhfKBh1TtBZ\nbbksDPt1+/qchcwcULdvD+/C+eC11xYFtndD3zLVbmFkALns0kuLzi1nUDjpVdROe5Qwq4oEgYLR\nAPid1bqtvA6zwVGpdjgNzPYB1WyH9TNeFAoFufiii2aVIu8wRGMeZjY46l28eNaK7XKmqSadAusW\nu6BYkCBQMOqcoLNae3B2ZCSePRJKtSPqgLpl0yZpBqTbEAV7KfKFxrqKFCDvveSS0O2MQlIprUmL\nFakfKBh1TNiBwq/7yO077cdBgrNex26MjY1JE/S6T6Ybyl6KvHnOHLn77rt91YGqx2qtXH9B4oCC\nUUe4DdZQ8UsvAAAZf0lEQVRen3HDGkuwCkh3t3fqppswlGPA0jRNtnz0o5KzZEFpKC5FPs+IVewe\nHPR1zaTjDIRUMxSMOiFOt4PV351OF2941NPj7Z6Ka+1EKXYPDkqT4WZahuId8YaNOMVCQFqN3fD8\nUG2uG1oFpNqgYNQRfmbHftxA9v0qUimR5maRlhZ9IZo1YBp27UTQwdD6/u7BweksqHmWgLbVDWXu\nsR10bUW1DNJufWtPQKCIkEpCwagzvDKR/K42tlsQCxfOXNN+Tpi1E0Fn8ubn9+0z1lak0kXi0GWI\nRoshIh1NTb6timrGKWusmiwg0nhQMOoIe6xhxYrZg0tQl5GT1WIfoMKsnQgSK9A0Tc76g8skA70o\n4DJbFtRqQB6NYFVUM/YAPGMsJEkoGHWC06y9r895cAm6LsNLAKKsnfDTjt2Dg9KZyUyvn9jusAiv\nxRAL+yK8aiXo6uyw/36ExA0Fo45wGoiizlBLuavCukj8tMOMVTi5n7owU4o8BchVV15Z+ktLUIn4\nRdj6T2EtRELipCYFA8BeAD+HZU9vy3u3AngdQJfH+TF2YfVSCR94mEHWz6C5f/9+ycyZI30299NK\nQMYByQPSmk77XlsRR5viwq9oh41BEVIualUwzgCwyi4YAHoAfBXAfzS6YPjNsgmTdRPHTNxuqZjH\nY2Njsuz4E4pKkd9rsTBygPQC0p5Oy/DQUKTvtR9XMj4Q1q1ULVlcpDGpScHQ240lDoLxBIBTKBg6\nQVNo/WQrWc8x6z9FmeVar3fBOedI1hAFe6HAk4y/l7/vffL444+HCmz7ud9KxAcYuCa1St0IBoCL\nADxgPKdg+MRvUTrrYJvP6+sxenq8V3775cknNWlpeUCygNwGyHKbG2opIJk5czxTZaMGkku9FxfV\ntjiQkCDUhWAAyAH4FoB2mRGMbo9zZWBgYPqRb+Ap3tRUcdlrL6vBOqC2tMQzE989OCitqZS0GC6o\nuS4Wxv79+z3vIcgg7GRFhLG2vI69CFpDi2JCkiKfzxeNlfUiGCcD+BmAHxpi8RqAwwB+z+XceHu1\nRpma0lNvOztnNtaZP9/bajAH21wu2kxc0zT5o6uukowhEvY02ZwlC+qCc88teT2/1oHX5/wO1OW0\nEmiBkGqmlgWjF8BBl/f+A8A8j3Nj68BKUK4Z59RU8f7T2axML8BzwhxsFy3SRcZ0TwUtMLjDmKmk\nADkGxXWgTBdUEyAnnnCCjI2N+b6fUvGHctTbKof7ijEOUq3UpGAAGALwEwCTAF4GcI3t/R/WSwyj\n3DNO61qNXG52LMOpHXa3lT0Y7tbOQqEgK97wBmnGTBlyc/W21cJoT6c9XVBOhElVtYtc0D4tZ4Cc\ni/NINVKTghH1UUuCIVKZ2ayb1WCl1LFXO7ds2lRUMHDYEId5xiptU0Q6mpoCp8uGEdWoQuw3WSAM\ntDBItULBqBHKMeO0Ww3mwj6r1RBHO8fGxhxXbGvQ60AtBiQNyNVXXRW6DlQYt13YgdnsN7O/du3S\ndyW0918YGMMg1QwFowYo54zTHIisg6D5nUEHKqd27h4clOZUyrVgoGlhrH3zm+O7qQA4Vfd1+2vF\nfM2854ULS1tofmGWFKlWKBhVQNR6TXENMEEyiOzY27lvX0Hmd2+QLCBvcEiVNTOhMkiuaKD9fvv6\niq0GqxXhdf9xZY4RUu1QMBImiiCUY+W1OfgtWhTcLWK+f/UVV0gWkOMNUdgMyJW2VNmVJ58s27Zt\ni6UOVBic+n3FCj2l2Ewx3rCh9OBfjrUphFQrFIwqIIzLyb7yurNTH+SjzG6t7ejsnEm39XtNTdPk\nzLe+VXKAvNGIU9xrCEQB+gru5lSqKFU2SXeLkxCbgrl+ffHg7/TZOFe/0w1FagEKRpUQJqhtHeBz\nuWizW6cZd0+Pv2tqmibbbr9dmpWSFofg9vGA/IkhHMcfP+hquSQ9aJr9abcwvKr8xmHlMdBNagUK\nRhUQZg2BeWwKTUtLdP+59fr5vD5TLnXN+3fulKxS09Vl7YvwVhrrLNqMbVPd7jXpQdMp88l6XGof\niqhix1RaUgtQMBLG70Dp5nPv6tItgY6OeDJ0grTp6iuvlGYjgP28kSY7D7PrQL3jvPOK0mXdrKmk\nB81SWVLlXkzHxXqk2qFgVAF+Z6dOWT1uK6/L3aaPbd0qOSM1drXFohg2BMTcUvWiCy+cZbl4iUK1\nDppRxMzPv2/SYkmIHygYNYZ1QC21GVI50DRN3v2ud0nacD9pmL3HdjP0OlF/dMUVs4TMy3Kp1kEz\nirvM7Vzrv9XUlL7wjzEMUu1QMGoINwvDfK/c260ODw3JvOZmORGQTmMNxfOGVTHPYlWc/taLZe7c\nQqCKsEnHMEoRJUZh/3dzCqKvXRutrhUhlYCCUSN4rRuIMiP3s9Jb0zTZs2ePdDQ1FRcJxEwNqJxh\nVVy3caOIeLuWvETD63Pluo7f60bB3h/Vak0R4gUFo4bwypKylyQP4y7ZtUsknS4uqDc8NCStqZRk\njSwoe3mP+wyhuOD886cX4ZVaMR6HJeHl6oly/XJYOm79Ua3xGkLcoGDUMOZAtH27PtDv2jXzut9B\nzj6YbdgwM4gVCgVpy2SkE5C8Q6yixbAwrr/22unr+Rlw45pdu10n6vXjnP279UepNF1CqhEKRo1i\ndyGZ1sG2bcHLbZszXXOh2tatmjRnNktbKi3LDGEYNh5dRrC7o6lJbrzhBsfSHn5cOnHNrt2uE/X6\ncc7+7fcf1QoiJCkoGDWMfeC56ir9X+bqq/VjPwOR3Uo579wBSRmWg9WamGdkROWNhXhBdsJz+85G\nsDDcSHpVOyFhqEnBALAXwM/NPb2N1+4DcAjAcwD+DkCHx/mxdmKS2MtZnHee/8J5dnfJ8hPeKFlA\nei1WhRmvOBGQRUZwe/OmTUXXsF/Ti0aMYRBSL9SqYJwBYJVNMM4GMMd4fg+AT3qcH2cfJs727TLt\nUhKZiUO47c1tZWpKZP/+/XJMd/esooGmVfE8IO2plNx9991FLiivGkpeA2xcs+tKZknZ7ymuBZKE\n1BI1KRh6u7HEKhi2994N4HMe58bWgUljdSnNn6/HMpy2DnVj9cknSxaz96voMiwNsxz5/Tt3un5/\nd3f85UmqCXNhnbnuxUxptq+dIKTeqVfB+BKAKz3Oja0Dk8TuPhkZEWlr879r3v79+yUHvZLsclvK\nrFk0MGNZW+FGnAUQq5V8Xi/5nsvp92mKIyGNRFjBSKNKUUrdAeA1ERny+txdd901/by/vx/9/f3l\nbZgLR48CqZT7sRepFDA2NvP5t78d+MUvgExGv05//8z71utOTEzg8OHDGB4eRg+A9wL4JIAXAKw0\n/n4fwLGLF+MrBw6gr6/PtQ2jo8DDDwMDA8B99wE7dujPE+rO0JT6d+jvB266Sb8/oDbvkZCgjI6O\nYnR0NPqFwqhMnA84WBgAPgDgGwCyJc6NTXGjECbA6sc/73Xd4aEh6crl5NTOTunIZKZdUZtRvCPe\nutNOC9R+cwYedROnJPC7doQWBml0UMMuqV4ABy3H5wP4LoBuH+fG14MRCZLCGURgnK6raZp05XJF\nsYqsRSiygLS2LJCOjrFpt1Yp7EUF7cHvMPgVxVKfCXLtUqvTGcMgpEYFA8AQgJ8AmATwMoBroHtR\nXgLwjPF4xOP8mLsxGkEWiQURGP26mlx11QE5cEB/nNrZKUXlPTo65L77HpBUaqNcd91+EdED521t\nwarfxpn55GeP8zCpr6Uqx7qVWTHPtWZGOWVOEVLv1KRgRH1Uk2CUEgCv+lFeApPPi7S3DkrrnCY5\n0VhX0ZpOS2cmU5wNlcuJpmmzSlUEsTC8jsPgRxTDLq5zqxxrZpdt2KAvYHS6f67RII0OBSNBSg1A\nTu/39ZUeKKemRE44fnBWquw8QzS6cjlZ3dEhXbmcDA8NTZ8XtBxGOQdQP20JW77Dft7IiC4S5mJH\nc3tWv64+QhoFCkbMxFWi28Q6QHV3675zrwG6UCjItm3bJJtKSZ8tVXY1IMtbW+XAgQMyPj5etG1q\nXDP2OAZQp2va+yVs8T639pqLH00R8fp3Y5VZ0qhQMGKkXDNu6wDlJTCXXXKJpFG8Req9NgtjbnNz\nkVDE0e64C/U5bS5k3ZEu6HoTr2u7VY51uxYtDNLIUDBiJu4Bxc/1NE2TU/r6pNkQC6sbKmepDdXW\n1FTkgrISNhZRjgHUqS1OsQevc/xee3JydgZUX19xBpSXi5AxDNJIUDDKQFwzbj8D1I6BAckZwvAo\nIKfa3FBLAWlOpWTb7bfPsiyiUukBNMwufn6wCtL27SKplPveIqwySxoZCkbMxD3jtqe2Wo/ftn59\nUWkPDbM3OcohWinyUlRqAC21TsJNuJysBLd22vcFoduJkGIoGDES94zb7Xo//am+x3YGenXZAmaK\nBw4bsQozhmEtRV6r+F2JbXdZmeeNjOguJjP7yUyl9TrfuvMgIUSHghEzcc+47QPZLTcNSkc2K8sz\nGT0u4VDaoxmQhccc77gbXq3ip1+tC+/sJUvmz3ffJ8QuSE57mxNCKBg1gblie92azbP2rOgAZC5m\nqssqQPrXD1TNKuSkXFbmQryBAb3+EyCyfr271WAPbAfNwCKkEaBgVAleNY5aW3ZKGsp1z4qFhlgA\np8rll2vS2Sly3HHBax25tSFK3aZKBMXdvsdcW9HSMrMjod+dCL2OCWlUKBgRiWNw8apxdMz8ayUH\nyGIjLmHfs6KtqUluvvlWWbJkTFpa9MExk9EzffyU9yg1s3ba8tQuRF73HLS4otexn/swGRkp3thp\nxQrvGAYhpDQUjAjEOYN2GlgLhcK0VaEZwWx7BtQtNw+KiD4ILlo0oyd+t2f147u3t81cs+D3nv2k\nGcfZl9ZrmSmzpsh5ZUkRQryhYEQkzjRa+8C6b9++op3who0FeH2GWNx4w87p7+7oEGlt1S2MXE4P\n9HqtWHZrv1t2kLVtpVJcrQQp4RFnX9KtREj8UDBiwG0GHWTQyudFuro0ue66cenq0mZZGKZV0QxI\nFinZPThY9N2LFukz/74+3VIwn/uJY9jXH9gtDKdB3+me7VZCmBIerNNESPVCwYiI26w4iIvFrC7b\n1pSVU9rbZW42J8uWDsnUlMjmTZskB8gyw6o4aflF04JiL0w4MlK8Yrm7u/Qs3fr5dFrkz/5sptz3\nunUiTz01e9BfsUK/tlPtpSglPFiniZDqhoIRAr8pmH4HwPt37pQsIL9vSZc196kQETl4sCAnnLBP\n9u0rTF/XXpDP+t1+Z+lOFoG1GJ9paVgH/akpkTVrvHefC2MlsE4TIdVPTQoGgL0Afm7d0xvAPABP\nAfgegAMAOj3OD91hU1OzK6e6FaoT8d7FTURk9+DgrLUVXYCc3NYm4+PjRd9rb4efIn2lZulubrNS\ndZvs1o1bgNzp+936yl4GhWJBSHVRq4JxBoBVNsG4F8DHjOe3AbjH4/zQHTY1pc+uOzv1QbGzUz+2\np5n62cVN0zSZm83OWluxApCObDZwscC4Zul+RccuKn6+32qVWV1fTHUlpPqpScHQ240lNsF4EcAx\nxvNjAbzocW6kTsvnZ1YP53LOA6qfXdzGx8dldXu72NdWZIHpoHZQomYH+RUdr9hNqe83zw2ymI4Q\nkjz1JBi/tL3/S49zI3VaPq+nr5qriN0GulK7uGmaJl253Ky1Fffv3BmpfVEpNejHYcmY1olXuQ5C\nSHURVjCUfm5yKKWWANgvIiuN41+KSJfl/V+ISLfLuTIwMDB93N/fj/7+fl/fe/QosHIl8KMfATff\nDOzaBfT0AC+8AKRSM58bHQUuuwz4yEeAhx8GnngCcPqKxx97DB/euBHHpdM4fOQI7nvwQVz/oQ/5\nakuSHD1afL/2Yy/MvrnwQuCznwXe/37gy1927yNCSDKMjo5idHR0+njHjh0QERX4QmFUJs4HZlsY\nh1DskjrkcW5ohbUHvd1KZZgzcE3T5FOfGpc3vUlznYFrmjZrj+16hTEMQmoX1LCF0QvdwjjFOL4X\nuhvqXqXUbQDmicjtLudKlPb7mV0fPQp85q8+ja1btmBJJoMfT03hkb17cfkVV4T+3nrB7C+3v4SQ\n6kQpFcrCSFQwlFJDAPoBdENPrx0A8EUATwBYDOAlAO8TkV+7nB9JMEoxMTGBP3/gATxwzz14A4BX\noKdt3ZvL4cWXXsKCBQvK9t2EEFIuwgpGuhyN8YuIXOny1tkVbYgDZkyi69VXkQPwcQB9AM4CsDCV\nwuHDhykYhJCGInGXVBTKZWFMTEzgpCVLkH/1VawE8AJ0oXgRupIdzmbxg1deoWAQQmqSmrQwqo2J\niQkcPnwYv/rVr9CbyWDlq68CAFZCj8z/E4DvA3jwwQcpFoSQhoMWhoHpgurNZPDDyUnI66/j/xw5\nMm1hvAVAOpPBnz30UE2kyxJCiBs1GfSOShyCcejQIYyMjODOP/5j/Mvk5LRAnN7UhEw6jSVNTXjp\ntddw67ZtuP5DH6JlQQipeeiSCsFNH/0odv/lX2IxgNegV0J8ELoLalkuh3ueeALz5s1Db28vhYIQ\n0vA0rIVx6NAhvGnFCnwLmLYq1gH4DnTxOIups4SQOoUWRkDGx8exGLpYwPi7CMD5zc34rVJ4ZO9e\nigUhhFhoWMFYs2YNXoFuWZgWxo8B7Pubv8FZZ51FsSCEEBtzkm5AUvT19eH6TZuwDsBy6O6o6zdt\nwvve9z6KBSGEONCwMQyTQ4cOYXx8HGvWrEFfX19MLSOEkOqFabWEEEJ8EVYwGtYlRQghJBgUDEII\nIb6gYBBCCPEFBYMQQogvqlYwlFI3K6X+TSn1glLqUaVUJuk2EUJII1OVgqGUWgjgowBOFZGV0BcY\n/mGyrapurBu8NzrsixnYFzOwL6JTlYJhkALQqpRKA2gB8JOE21PV8D/DDOyLGdgXM7AvolOVgiEi\nPwFwP4CXoVfs+LWIjCTbKkIIaWyqUjCUUnMBXAx9o7uFANqUUm77fxNCCKkAVbnSWyn1XgDnicj1\nxvHVANaKyCbb56qv8YQQUgPUU3nzlwGsU0o1A5gE8HYAT9s/FOaGCSGEhKMqXVIiMg7gCwCeBfA8\nAAVgd6KNIoSQBqcqXVKEEEKqj6q0MOwopc5XSr2olPp3pdRtDu9nlFLDSqnvK6W+qZQ6Lol2VgIf\nfbFBKaUppZ4xHtcm0c5yo5Taq5T6uVLqBY/PPGT8Jp5TSq2qZPsqSam+UEqtV0r92vKbuLPSbawU\nSqkepdTXlFLfVUodVEptdvlc3f82/PRF4N+GiFT1A7qo/QB6xlQTgOcAnGT7zI0AHjGeXw5gOOl2\nJ9gXGwA8lHRbK9AXZwBYBeAFl/cvAPBl4/laAN9Kus0J9sV6AF9Kup0V6otjAawynrcB+J7D/5GG\n+G347ItAv41asDDWAPi+iLwkIq8BGIaecmvlYgB/Yzz/AvQgeT3ipy8APeZT14jIGIBfeXzkYgCf\nNT77bQCdSqljKtG2SuOjL4AG+E0AgIj8TESeM57/FsAhAItsH2uI34bPvgAC/DZqQTAWAXjFcvwj\nzL7p6c+IyFEAv1ZKdVWmeRXFT18AwHsMU/tvlVI9lWla1WHvqx/Dua8ahXVKqWeVUl9WSq1IujGV\nQCnVC93y+rbtrYb7bXj0BRDgt1ELghGGhphNufAlAL0isgrACGYsL9K4fAfAEhFZDeAvAXwx4faU\nHaVUG3RvwxZjdt2wlOiLQL+NWhCMHwOwBrF7jNes/AjAYgBQSqUAdIjILyvTvIpSsi9E5FeGuwoA\n/grAmyrUtmrjxzB+EwZOv5uGQER+KyK/M55/BUBTnVrgAACj/twXAHxORP7e4SMN89so1RdBfxu1\nIBhPA1iqlFpilDj/Q+izaCv7oQd7AeAyAF+rYPsqScm+UEodazm8GEChgu2rNAru1uSXALwfAJRS\n66DXI/t5pRqWAK59YfXPK6XWQE+nr8cJlclnABRE5EGX9xvpt+HZF0F/G9W60nsaETmqlNoE4Cno\nArdXRA4ppXYAeFpE/gHAXgCfU0p9H8AvUKel0H32xWal1EUAXgPwSwAfSKzBZUQpNQSgH0C3Uupl\nAAMAMgBERHaLyD8qpd6hlPoBgP8GcE1yrS0vpfoCwHuVUjdC/028Cj2TsC5RSp0O4CoAB5VSzwIQ\nANugZxY21G/DT18g4G+DC/cIIYT4ohZcUoQQQqoACgYhhBBfUDAIIYT4goJBCCHEFxQMQgghvqBg\nEEII8QUFgxBCiC8oGIQQQnxBwSCEEOILCgYhhBBfUDAIiQGlVEopNaaU+q1SarntvQ8qpV5XSt2V\nUPMIiQXWkiIkJoy95J8DcBjAWhF5TSn1+wDGoVcaPkv4H47UMLQwCIkJEXkZwEboO5vdr5RqBvA4\n9CqgV1EsSK1DC4OQmFFKPQzgRgD/F8BbALzHZSMfQmoKCgYhMaOUygL4NwAnANgjIjck3CRCYoEu\nKULiZxVmttI9WSnF/2ekLuAPmZAYUUq1A3gMwASAOwC8FcCORBtFSExU/RathNQYewAsBnC2iHxd\nKXUqgNuVUiMi8vWE20ZIJBjDICQmlFIboQvGn4rI/zRe64SeapsG8EYR+WWCTSQkEhQMQmJAKfUG\nAN8B8AyAfhF53fLeOgBfB/AVEXl3Qk0kJDIUDEIIIb5g0JsQQogvKBiEEEJ8QcEghBDiCwoGIYQQ\nX1AwCCGE+IKCQQghxBcUDEIIIb6gYBBCCPEFBYMQQogvKBiEEEJ88f8BWoTIsLIsOgkAAAAASUVO\nRK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x3fff3bfbd810>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Define some hyper-parameters.\n",
    "lr = 0.1\n",
    "epochs = 5000\n",
    "\n",
    "# Train the model.\n",
    "with tf.Session() as sess:\n",
    "    # Initialize all variables\n",
    "    sess.run(init)\n",
    "    \n",
    "    for epoch in range(epochs):\n",
    "        y_p, curr_w, curr_b, curr_loss, _ = sess.run([y_pred, w, b, loss, train],\n",
    "                                                        feed_dict = {x:sample_x, y: sample_y,\n",
    "                                                                     lr_rate: lr})\n",
    "        \n",
    "    print(\"W: {}, B: {}, Loss: {}\".format(curr_w[0][0], curr_b[0][0], curr_loss))\n",
    "    print(\"Slope: {}, Bias: {}\".format(slope, bias))\n",
    "    plt.scatter(sample_x, sample_y, marker='x')\n",
    "    plt.scatter(sample_x, y_p, c='red', marker='o')\n",
    "    plt.xlabel('x', fontsize=18)\n",
    "    plt.ylabel('y', fontsize=18)\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Keras Implementation.\n",
    "Keras is an abstract library built over tensorflow (and other popular libraries such as theano & CNTK). It provides an easier API where users do not have to create variables or perform specific vector operations. Keras provides them in the form of __layers__ that can add either sequentially or in any other combinaton. Also, it provides an easy API to compile the models, save & load the models.\n",
    "\n",
    "Its API is very similar to scikit-learn where it has a `fit` function to train, `predict` to perform predictions and each function has a host of parameters to validate, perform parallel training and so.\n",
    "\n",
    "For more information [[4]](https://keras.io/).\n",
    "\n",
    "__NOTE: In most cases, for applications of deep learning, the standard / hot of the press research methods are quickly integrated into Keras. It has a large community of developers. I recommend its use, unless there are specific kinds of computation you are looking for and cannot find it in Keras.__\n",
    "\n",
    "So why study Tensorflow then?\n",
    "__Because, ultimately knowing Tensorflow does help!__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "from keras.layers import Dense, Input\n",
    "from keras.initializers import random_normal\n",
    "from keras.models import Model\n",
    "from keras import optimizers, metrics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As seen in the previous section of the notebook, let us breakdown the implementation of the model into a series of steps.\n",
    "\n",
    "__Step 1__: Create the layers.\n",
    "\n",
    "* __Dense Layer__: This layer is the equivalent of implementing $w \\bullet x + b$ and many more options. This is its simplest case.\n",
    "\n",
    "* __Input Layer__: This layer simply casts the sample_x values into a Tensor (and validates the shape).\n",
    "\n",
    "Important to remember that Keras input layer does not have `None`. This is because by default it consider only the shape of a single sample and not the size of the batch. `None` represents any sized batch in Tensorflow."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a layer to take an input.\n",
    "input_l = Input(shape=np.array([1]))\n",
    "# Compute Wx + b.\n",
    "dense = Dense(np.array([1]), activation='linear')\n",
    "output = dense(input_l)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Step 2__: Create and compile the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "input_1 (InputLayer)         (None, 1)                 0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, array([1]))        2         \n",
      "=================================================================\n",
      "Total params: 2\n",
      "Trainable params: 2\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# Create a model and compile it.\n",
    "model = Model(inputs=[input_l], outputs=[output])\n",
    "model.summary() # Get the summary."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Step 3__: Add the optimizer and the loss function. The loss is `mean_squared_error`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "sgd = optimizers.sgd(lr=lr)\n",
    "model.compile(optimizer=sgd, loss='mean_squared_error')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__That's pretty much it!__\n",
    "Once the model has been created, use the `fit` function to train and `predict` function to predict."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x3bfd97644d90>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Train the model.\n",
    "model.fit(x=sample_x, y=sample_y, epochs=epochs, verbose=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      " 32/250 [==>...........................] - ETA: 0s"
     ]
    }
   ],
   "source": [
    "# Predict the y's.\n",
    "y_p = model.predict(sample_x)\n",
    "y_p_loss = model.evaluate(sample_x, sample_y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Evaluation Metrics: ['loss']\n",
      "W: 6.04618597031, B: 9.90206241608, Loss: 5.95258642578\n",
      "Slope: 12, Bias: 10\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEXCAYAAAC+mHPKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnX+UHFWZ9793umd6en70JDNRIxNJJAnLBAgJCIkSnRFF\n0JUfymZZ5dUBoiISg66sJIE4YSMiRJyVfWHnxGWX1TCGxX0j6/pjeEd7OGdccVgBydLouu4i/gAq\nR1ffo3KiMzzvH1U1XVNTVV1VXdVd1f39nNNnurq7qm/ddJ7vfZ7n3ucqEQEhhBBSiZZ6N4AQQkg6\noGAQQgjxBQWDEEKILygYhBBCfEHBIIQQ4gsKBiGEEF/UVTCUUiuUUt9USj2plDqilNphe/8jSqkX\nlVK99WojIYQQnWydv38WwJ+LyONKqS4A31VKPSgi31dKrQBwLoAf17eJhBBCgDp7GCLynIg8bjz/\nDYCnAPQbb48C+It6tY0QQshCEpPDUEqtArABwHeUUhcC+ImIHKlrowghhMxT75AUAMAIR30RwLUA\n5gDshh6Omv9IPdpFCCGkjKp3LSmlVBbAvwD4moh8Ril1CoBJAL+DLhQrAPwMwFkiotnOZSEsQggJ\ngYgEHognIST1dwBKIvIZABCRfxeR5SJygoi8EsBPAWy0i4WJiPAhgpGRkbq3ISkP9gX7gn3h/QhL\nvafVng3gMgDnKKUeU0o9qpQ63/YxAUNShBBSd+qawxCRbwHIVPjMCTVqDiGEEA+SEJIiETA0NFTv\nJiQG9kUZ9kUZ9kX11D3pXQ1KKUlz+wkhpB4opSApTXoTQghJARQMQgghvqBgEEII8QUFgxBCiC8o\nGIQQQnxBwSCEEOILCgYhhBBfUDAIIYT4goJBCCHEFxQMQgghvqBgEEII8QUFgxBCiC8oGIQQQnxB\nwSCEEOILCgYhhBBfUDAIIYT4goJBCCHEF3UVDKXUCqXUN5VSTyqljiilPmi8fptS6iml1ONKqX9S\nShXq2U5CCCF13qJVKbUcwHIReVwp1QXguwAuArACwDdF5EWl1CcBiIjscjifW7QSQkhAUrlFq4g8\nJyKPG89/A+ApAP0iMikiLxofexi6gBBSkbk572PSfBw9ehSPPPIIjh49Wu+mpJ7E5DCUUqsAbADw\nHdtbVwL4Wq3bQ9LH738PbNkCTE3px9/4hn5M0WheIf3CvV/ASStX4v3nnouTVq7EF+79Qr2blGoS\nIRhGOOqLAK41PA3z9RsA/EFExuvWOJIK5uaAwUHg0kuBrVuByy8Hzj8f+PjHgUym/Bn7Oc3A3NxC\nIZ2aag4hfe65o3jvu7eh+MIL+O6vf43iCy/gve/ehueeo6cRlmy9G6CUykIXi8+LyAOW1y8H8BYA\n53idv3fv3vnnQ0NDGBoaiqOZJOFkMsAtt+hicfLJwD/8AzA8DLzhDfr7ptG85RZgaEg3mrt2AdPT\nZUFpVKx9c801wJ13Avffn777nptb2Gb7sZ2f/ORprM63Yf1vXwAArAdwQr4VP/nJ01i+/CXxNjZh\nTE1NYcocMVSDiNT1AeBzAD5te+18AE8C6KtwrhBiZXhYBBAZHBRZtkykWCy/Vyzqr42MLH6v3szO\neh9HwciI3jcjI9FfOyxe921/vmlT+d+sWBTZvNm7nzRNk958Xr4HiADyPUB683nRNC2q5qcWw3YG\nt9dhTorqAeBsAHMAHgfwGIBHAbwZwA8B/Ng4fhTAXS7nR92PJMVMTopks7poLFsmMjq62Kgk1Wja\njeGmTdGKxuTkQrGcnIzu2mGYndUfmzfr9zs7u1AErO+J6H8HBoIL/p4bxyWPvKzLFSSPvOy5cTyu\nW0oVqRSMah8UDGJiGhjTEJrG59ix8meCeBi1GPFbr71unUihoLetUNCPo/rOY8dEurp0ARXR/3Z1\nLeybWmIVg2JRv98VK0T6+ip7hEEE3/yew4c1mZmZkcOHtYpeSbNAwSBNT6Xwhn3E6mY8gn7W69hv\nO4tFkY4O/X9kR0f04bKkeRhWMTDv20kErAIRJqRYS+FPExQMQioQxHj4MU5BhKXSOZOTIvm8/j8y\nn48nv5K0cJzZnnzeuZ+t/wZ9fbrXRW8hGigYhESEaYRMg7Znj/tnw4x67edMTurx+Z4e/bWeHv04\nSmMYNuEf1wjdbE9/v36/ZnjKK4dxwqoxWZLLycbubunN52X8IPMRYaFgEBIBpqEaHdUN2vCwnkj3\nCuGEGblbz4k76R3UEzJfd8sLVduuSsluezs0TZPdO3dK3pjpxBlP1UPBICQi/My2MonCwzANp5Wo\nQy1+r28Xl9FRvS/27Il2KrLf9ozdNSZdbW2SA+Q0QyzMx4bubpmZmYmmQU0GBYOQCNmzZ6HXUG1y\nvJpzao1d0My1LbXOfYzdNSZ5QE4FJA9Ij83DWJLL0cMICQWDkIiIe/ptvWbuBPleM2Rmelm1Xuyo\naZosyeUWCEQnIL2ArDcE5MDYWG0a04BQMAiJgEoeQFqnaQbxbEzB3LNHJJMpr9+IY0GhGzMzM7Kx\nu1usIajVgCwHpJDLydhdFItqoGAQEhA34+/1elLCSWGEK+hU4dlZfbaWOWMrLsHQNH2qrDW85FTW\nIw/IO9+xk1NqI4CCQUgAwhr/JNSjqka4/Mzosl4n7vs9ND4uvfm8nN7TI735vBwaL0+VHT84Lp0t\neTmlo1sKrTnJ58akvz95dcDSCAWDkICENYZxLIAL6jFENTvLD9Xer9u9lUolKeRyUvSYKvvss2Xv\nI2kLD9MMBYOQEAQ1QnGMuMN6DE4LC93WMtTLo3L73vGD47Ikl5MTjUT2IUM0NhYKjlNlk+DZNRIU\nDEJ8YA+39PX5N0Jx5jCCGkRrYjqb1RPTZhFDa86hmoS91/0GLbPS26vJe94zI729mhw+7FB2HJCi\ny2K8JOWOGgUKBiEVcEromlVh/RqhOGdJ+fV2vBbXFQrlEiNRjMSd7jeoAT80Pi7d2bysRo90Z/Ny\n8759cnpPj1hnQK01Zj9ZcxiV2kHCQ8EgxAdeI/l6GqGgHoa9rdaFhlHH+t2q6/ppb6lUkq7WhXmK\nnra8LGlvX7QIr1QqRdNgUhEKBiE+SVrytNqQi9V4x+FhuLWtUj8eGtfzFGtteYq1mYLsu2mf9Obz\nsrFQWDQ7isQPBYMQH/j1MNySx3ERNuRiD7N55TDC4tRnlTwMx+1RbXkKp/UXpDZQMAipQKUkbtyG\nNy5qIXTWkJdTdV17/8zMzATOU5DaEVYwlH5uOlFKSZrbT2rP3ByQyTgfT00BW7cC11wDjI4CSgEf\n+hBw553A/fcDQ0P1aPHidjodx8k3vgGcfz5w2WXAV74C3HADcOgQ8K1v6W04evQofvSjp7F69Sq8\n5CUvAaC/dtLKlSi+8ALWA3gCwGAuh3997DEMDAzUpuHEFaUUREQFPjGMykT1ALACwDcBPAngCIAd\nxutLATwI4AcAJgD0uJwfoeYS4p08rtdMnWPHFnpGk5O183hMz8ttf5ADYws3NbJ6D+YqbuYpkgfS\nGJICsBzABuN5lyEQJwG4FcBHjdevB/BJl/Oj7UXS1Fj3wSgURDo7RVas0NdqmEa61msBKhnsWmDe\no32h4E0jIxU3NWKeIpmkUjAWNQb4EoA3Avg+gJdJWVS+7/L5CLuQNDN2w7xsmV6pdXJy4b7b9Vht\nbCaYBwdlvuR4rbEmuXt7Ndny6gu5qVGKSb1gAFgF4GnD0/gf23u/dDknuh4kqSKO8JDbSNrpvWqm\n5IZpu7mJ0eBg7UtjzM6KnHGGJrfdNiG7d+6Uzqy+A97JxswnbmqUPlItGIZI/BuAi8RBIAD8wuU8\nGRkZmX8UWWAmdYTdgKgeJTqiqGcUpu1BtoyNgwNjY9KZzcorAekAJAfICYZY3ApuapQGisXiAluZ\nWsEAkAXwdQDXWl57yhaSesrl3Cj7lNSYagx/rYsARilSQXf0M8Nh1u89diz494bh9v37F3gTtwJS\nMMTBFIt1hojclJSVkKQiaRaMzwH4tO21WwFcbzxn0ruBqWY7VPu+21HgtabBbqSrGeEHCW3VY3aW\npmmy6/rrJW/kKazexCmALDVE45WGWLxn27b4G0UiI5WCAeBsAHMAHgfwGIBHAZwPoBfApDFr6kEA\nS1zOj7ofSR3wu6mPW8G9OGL6SapMWwus93VofFyWtLdLh30GlOFNdBti0dXWJldfdZWUSqXELmok\nzqRSMKp9UDDSTxDj6VTS23w96pi+vcBeX1+5bdWM+JNYqtts0+HDmkxMTEihrV3uNfIS1hlQ6w1v\noqOlRV6xYp8cPqwl5h5IMCgYJHWEMZ5OM5jMa8XRLvP7VqyILpeRtFLdmqbJtiv2STvaZaC1UzoA\nOeAwAyoPSK6lRUqlUiK9JOIfCgZJJUE34rEbqbiMbbGoV33N50U6OvSFfFaRaBRjaa7UXmPkJQ4Z\n4rDUEI0CIKvnZ0dlZfxgebV20qr+Ev9QMEhDYxa8GxgoV0tdt05/rVrRcNskqL+/bBDtnkTUnk6t\nvQ5N02T3zp2LV2oDohkisbq9U9rRLq/dslO6uyfmQ1AizrsV1ttTIv6hYJCGxyuvUM01nUJMTqu6\nTYPolkux13gKU6Lc2oY4DLCmaXLzvn1SyOUcV2pvBOReQDpUXr761Qm57jptkRdhive6dWXxHhiI\nRrxJbaBgkKYh6lCIPcTkVTfKbbbWDTdUl4ivRZjr9v37pdDaKmsA6QHkeIc8RQcgS9rbZfzgeMW9\nQ+hlpBcKBmkK4tpiNUhlWrf1IOZK7Ci2WY2a927btij8lAdkj22l9q6dO0XTNF9ez+ysPhnALWyX\ntOQ+KUPBIA2PfZOjgQE9LFLtJkfVjO7t55o1nyoZ/VqsK9E0TT772c/K5ZdfLq0O4aeTjTUVq6Bv\nbGQv61HJ4BeL+mSAjg59ckBPz8Kpx0mbPkzKUDBIU2A1OE4hkTDXczJs1lXdbrvYVWv041xXcmh8\nXHKG17DW+NvuEH4q5HJy8759gQsGOk097u9f/O/TKLPJGg0KBkk1YcMXcVSPtW5YVMmTsYuHveZT\npfuoNNsqTL+USiXJZ7OLBCJvTJM1w0/X7thRVWVZ+yQEJ1Hg1NtkQsEgdSFIrN/N2IUNX8Q5grVe\nu6dHD71U+p6gxr1S+4P2izkDylxX0WGsqzBDUGsAWYboaj9Vah89jORCwSA1J8rqrn4W5dnFaNOm\nhdePelqndXQc9UjZb//4MbrmmorO1tZF9Z+WGusqTA9j9+7dUiqVorkJcf83Yg4j2VAwSF3wMmh+\nRtBWrLOEKhmcuAUjjIcRFL8eiZtYaZomH9qxQ7oymfnV2Mttie3VgKwwxOL8c98UTcN9wllSyYWC\nQeqG1+jb7T0/CeNKghNXyCOu2VhhcLvHA2Nj0qnUIo8iD0jRctydbZWrr75a7rhjmiN8Mg8Fg9SF\najwMP7OEKoWC4kqq2sNfXrmZuHDzskb2jEgOkL8B5HSbR3GKkaNYDUhPW166u8aZQyCLoGCQmhNF\nDsNrllC9PIwkYe0vTdPkzeeeNz9Vdgn0Fdv2nEUOkN3GArw4ci9exyQdUDBIXahmlpSXwbdObRVx\nrtPULElVc/e7jpaWRau1uwzhMHMYHZnM/AK8qAW1mfq80aFgkFThxzuptI91Uke7Ubbr0Pi4dGaz\n0gHIXwJyoi0EtQaQNkAygLz5TW+aX1cRl3FvBq+uGaBgkNThZVjTZJgqTfcNY6g1Td/9riublW5D\nKEpGUtue5G4B5CaHmFNcgsrFeOmHgkEajigNU1zG02kkf9JJi6vf+mV+8V17u6zu6JAOY9aTWVV2\nhyESa4y/OaUW1IAKcl9h+iRNQk7cSaVgALgbwPMAnrC8dhqAbwN4DMAMgFd5nB9pJ5JoqcZIR2mY\n4o69W9va16dPwTWLEA4P+/+ubVdcIe1KySojeb3LEAaBvmK71zjOAdLW0iI7tu+QM87Q5MEHg9+X\n37Cf0znMYaSftArGFgAbbIIxAeBNxvM3Ayh6nB9hF5Ioqca4xGGY4h4ZW72h0VERpUQGB/W/5nRh\nNzRNk56OTslDryjbC8itxt9uSwiqaIhFNrND9u3TcxXXXKN/7403Br+vyUl9OrNZln10tHI/JzVv\nRIKRSsHQ242VNsH4GoCtxvN3ADjocW6EXUiiphojHYdhiiv2HtbDMEt6ZBxyE72AnATIduO9U/J5\nWWKUIDcF6XWv0/+ed544Tk32g9nOwUGGmJqJRhKMkwD8GMAzAH4C4BUe50bZhyQGkpIgDSNefkTL\nyRsaGKicw7h9/37Jt7TIy40ZTmtts5/WG97EkvZ2OTA2JjMzMwsqy77udfpHV68u78ORzQbLl5h9\nMjhYFjfSHDSSYHwGwMXG8z8B8H89zpWRkZH5R5HDo0SRlARpmBBXkHOCzpJ61zvfKTlj3cQaQDpd\nZj9deMEFi8qPz86WQ17r1+v/g8880zmkVGmNzObN+jlhBYekh2KxuMBWNpJg/Mr2/q89zo2sQ0m0\n1CJBGiRsVcsZQV7fdfv+/dJuJLWtAtFmm/3U3VFwvO6mTSLt7bqhLxb1oohmDsP6XX76v9LiSNK4\npFkwVgE4Yjl+EsCg8fwNAB7xODe6HiSR49dIhzH8brN8ojZ2QUNqbvdSKpXkjjvukFxLi/RjcQ2o\n1UbeIgPIOa9/h+u9OFXRdUt4+xE8JrGbk1QKBoBxAD8HcMzIWVwB4DUA/s2YVvttABs9zo+4G0mt\nCRr6iXsfbCtBPQyneznjDE0uvWSr5I3Fd3nDm7B7GHlAci0t82sqvAy3dUtUv7W66p1DIskilYJR\n7YOC0RgEMcz2z5qzfOKoVlvNLoAf+EBJ2nNXSWe2zbEEeYfxWI1g26V69VWQWl2kuaFgkFQTZCRs\nftZcPxCXQQwbrjntlHdKOyD9RkL7VFv4yaz/tByQNmTkzz88VvmiUp03xkV2xAoFg6SWMKNmrz00\naoWToAxuebu0Qq8ge7rxt83Bw/jIRz4rExMTcviwFqjdcSf6SXNAwSChqadhsY6EZ2e9d7ezj5qd\nSp7Xo92lUkk++ME7ZFnvZdJuhJqsAtEBfcW2GX7afvX2BdfxKglPSBxQMEgo7AbbXma8Vm0wKRb1\n1dJ+9wGvp4EtFkXaW98peasYuMyAas9k5eqrrpJ77ikF3mSKkKihYJDQmEZ6xQp9mmaxWF/jVSmf\nkQTR0DRN3vKmNy1acLcE5cqy5mudqkWWLCmF2saWkDigYJCqMI10R0d9jZeTAbWGnLw8olpxYGxM\nOrJZyUIv4WEv6XGzMW12NSBdmYwcGh/3FEFOfSW1hoJBQmM10vl8/YyXuZJ5YKDs5axbp79mXZVc\nLOqL1vr7ay9sN42MLAhBOZX0WAVIVzY7P1XWSwStSXx6GKRWUDBIKKwj9noaYmt7rEa0r29hkjtO\nYfMKdWmaJhdfeOEigTDrQJ1q/G0FZPfOnZ5bpW7apD/MpL1ZAypoaY4khOZIOqFgkNAkJdRjZc8e\nWVBB1WyP+Xql0FlQY+qWgH7ooWm55G1vk87WVslB36/CGoI6DZCXG0KRBRbsfufVFjfPYnbWfe/y\noEUOve7V65g0PhQMUjV2wxTldNUgRso0psPDelVW6wY/bsl5q5ENa0ytoaPeXk2O718rLcYCu1ZA\nXumQ0O4GpB2QT33q075WaltxKvHx4IMiXV3lGlmm12GG5Ky1s+xl1P2WbOfMLELBIJEQh0GpZoWy\nWfrjxhvdPSB71dWwxlTENOL7pcUQgrVGqOmVxl9zJ7z1lhDUJW/bXvG6dkxx6u8X6ezUBdAsMf6p\nTzkvTnSqneWVMHcTac7MIhQMEhlxGJQg13RLCNv3arCv37BfP+jso8OHNWnLDEm7kdReCn0vbWv9\nJ1M8coAAr5UPfKDk7+K2dlvzRoWCSHe3LAjBuZU/sdbOqrRC3kukOTOruaFgkEhxMijVhi2CGKkw\nno71+kFFb+yuMdetUjXo9Z9aAOnIZOTCC65yXVfhF+t9mCLQ3a2Lx+io/nfFCj0EZ93u1b4VrFf/\nuPUBPQxCwSCRYQ2X9PREs5AvjJEKk/fwa0xNNE2Ta3fskJzhNdiT2hsBudcQEgXIkSOlSEN2k5N6\nmGl4WBeJzk6R1tZyv5thqBtvXFw7a9OmxWtU7H1lF2nmMIgIBYNEhFO4xBzphh2JehmpKGbsuE1d\nrVSj6cDYmHRns7IakB5AjndIancYuYwMIIfGxx2vFdbYmu22JrL7+8sG3v6+W+0st/61TkO2z8SK\nov0kvVAwSGRYDYhbaMo+9dN+7HVN8zjK0W4QIzg9PS3nvvGN0o7Fi+722JLaLYD8r8suCzQDKmxF\nWa8FfpWuZT/XFBd6EsQJCgaJHLfQ1Fln6VM/rbN3uroqi4bXd9Qqnv6GwcH5ldodRlLbDD+dbEyT\nXWWEp95y3nmiaVogI1tJBN0EIArxdAo/OX0XIRQMEimVQlOjo/oaicFB/e/oaHiDVKsZOx+97rpF\nSe2lRlLbDD/lAOnrvVTuuUef/RTGcLuJYFgxqeY7CXGCgkEixys0NTtbngo6OBh+b4q4Dd309LR8\n7GMfk49ed520Qt9X2156fI0RfmrPZOTA2FgkbXITwTjuN2wOhzQvqRQMAHcDeB7AE7bXPwjgKQBH\nAHzS4/wo+5C44GTkTA8jm9V/RaaXYX7ez6g8zhk7mqbJhlNPlTaUF929zGHabAf0HfHse2pX4/VU\nEoU4PKqoyoaQ5qAmggHgPwBcD2B5mC9zuN4WABusggFgCMCDALLG8TKP8yPuRmLHyahbcxim8ctm\nRXp7g4+c44iz3zQyIjnoJT3ygHRBX6GdB+QS46+5grsVi+s/VeMFVBLBWoWOGKIiXtRKMJ4E8CKA\n3wP4EoC3AmgJ88WWa660CcZ9AM7xeW7E3dicVDLaTsfHjtlrLy1chVwvNp95puSNMFMH9GqyndA3\nNnqlIR4nGkLhNPspCq8nzsR2ELiam7hRs5AUgM0APgvg1wDmAPwMwM0AVodqwGLBeAzAXgAPAygC\neJXHuTF0ZXMR1ojZz3OqcxQn9vY9+6wmrz377EVTZZdiYTmPNfm8FHI5x6qybteO0qDXauYSPQzi\nRVjByCIgIvIwgIeVUtcCuBTANgC7AOxUSj0E4G8B/JOIHAt6bYMsgKUislkpdSaAfwRwgtuH9+7d\nO/98aGgIQ0NDIb+2OclkgFtuAbZuBa65BrjzTuD++/XXK503Pa3/nZsD7rsP+PrXgTe8ATjnHGDX\nrvL7dubmFr5uP67E3BywZYve7tbWb2H3ztvxyPSXcRxm0QI9+bXeeKwEcBT6qGbjmWfir++8E6tW\nrcJLXvISz3vzOq6GOK9tMjen9//99wNDQ/rD69+DND5TU1OYmpqq/kJhVMb+AHAigHuhh6vmAPwC\nwF8BON7HuXYP46sABi3H/wmgz+XcaGW3iak2fOF35BxVWObwYU0yWCtZh0S2dapsD/SV2he+9a1B\nbynVcA0G8QL1mCUFIAPgbQD+BcCsIRaTAL5iHP8WwEUVrrEKwBHL8fsA3CRlIfqxx7mxdGazUevw\nRbXfd9PIiCgjxLTWYQGedapsDpDb9++P4S4ISS81FQwAJwHYD+A5w6t4DsAnYcljAFgD4FEAP/C4\nzjiAnwM4BuAZAFdAD0l9HvqU2n+zehsO58fVn01D0BF/VCPXMB6Npmly6sDAgllOO1wW4LUB8tqz\nzw68qZGVoOVPCEkLNREM6PmKbxmexByACQCXwJgC6/D5ywHMhmmYz/ZE24tNSq3DSWE8jEPj49LZ\n0iIdWFz/qWR4Ff3mau2Wlqq9imPHoit/4geGkEgtqZVgvGh4BDcDWOXj82cB+PswDfPZnuh6kPjC\nqcidlagX65VKJbn55pulK5uVewE53bZSey0gf2kIx5oTTpCJiYmqvAorTuVP4oAlx0mtqZVgXAwg\nE+aL4nhQMNyJc8RqhpP27PFn6OzH9lG6W9ved+WVCwoFHsDi8uNmnuKE41dWf2MODA7KfPkTt/ZG\n0bdBvS56JKQa6pL0rveDguFMnCNWu2EbHfU2dGHaUiqV5KILL1w0+6lgiMZSQ0TMTY3+7NLrqr8x\nB9wKLMbVt37zOvRISLVQMBqcoCPKWhW527xZ9zSshs7eNreNfJy4dvt2aQekzxAFe6HA41De0Ah4\nvVx3XTThJzteOYw4+tbvNe0lRmq1UJI0FhSMBibsiDKO0hD2cNLEhL+Ne+yiYqdUKsnu3bslC72M\nhxmGWrT7XUuLvOc9N1e9p3aYe7UeR9m3fv997Z8zS7Hs2VN9G0hzQcFocIKOamvhYUxO6qNu6xaj\nTluDVgpbbX372yUPfeMiMwylQa8BtRT6vtpLjamyY3eN1T0cE1ffeh3bv3t4WA+TDQ/TwyDBoWA0\nAUmIcfudJeU3Mb7pVa+SHCBFQGYAOc0SgjpkCMhxgHQae1VYv8P+nV6vRSUoScgfmN7a8HD92kDS\nDQWjwQkzi8a+R0JUBqWScFUSlSNHSnLHHXfI1ksumV+E1+syC6odi/eqqETcRr2eM5TcchcUCxIE\nCkYDE8YAxmU0KwmX1/dqmiaXbt0qbYbnYM9R9AKyx3hvNSCF1lbPqrLVtDONJMG7IY0BBaPBCTOq\nrWaRndP3BUnO2o8PjI1JT1vb/HTY7Q6L8NYYayraAbl5376qF+A14n4QXH9BooCCQRwJushOxFsY\nghqsUqkkl7/73YvWVCxxWYR38sBAJCu1G9HDICQqKBgNRFSjSK9Fdn193kY0CoNrrql4qTEDyupN\nrAfkZssivO5sVm6KyBVg6IYQbygYDUJUxs56ndlZkYEBkXXryjNsVqxYnBS3n19NSOfgwYOSs0yL\nzUPfV9vqTawyZj9FEX6yk6TQTZLaQogIBaOh8DO6Dzq1tFgUKRRE8nmRjg79uXWGjV2kBgaCrTw2\nefZZTV591lnS6pDUzkPfTzsPyKV/+qdy3333ybPPxrNSOym4DQD81tMiJA4oGA2GdXTvVLwvzKyp\n/v7yNe3nWEWqr0/3RoKsPNY0TS57x/XSoVplNSBdRhLbXtqjFRn58w+P+W53I+A0+YAhM1JPKBgN\nhB/jHaTmzcPvAAAYA0lEQVQ+k/2a5uftBspLpLxWHnfkRxZtlVrE4q1TuzIZueeeUlMmo+3hPSbl\nST2hYDQIQcJDUa78DmPApqenZflLXyoZI7Ft9yhWQa8wuxqQnrY2OTQ+HqjdSSdoOY+w/36ERA0F\no4Hwk4AOs/Lb7TjMpkannnyytKNcJLDNWENhLxaYA+TVmzbNJ7VrMbKuRZI5bMHAsB4iIVFCwWhg\nahED92tkr92+fX4l9lLo9Z7MPbXbjccpRjjqrX/8xzI9Pb3gmnHH7ms5pdav+EWRgyIkSlIpGADu\nBvA8gCcc3vuIsSVsr8f5EXZhMolzlk3Qkfj09PSivEQv9MqyGwF5BSCtgOQthQKr/c4wba9lfiBs\nWIlTbUk9SatgbAGwwS4YAFYA+DqA/252wRCJpzqr35G4pmkyMzMjt+/fL+2ZjKy15Sk2AnKvxcM4\n6cQT58NPcRlBP22vRX6AiWuSVlIpGHq7sdJBMO4HcCoFwx9BwzD2qbRuu7bdvn+/dLW2ytrW1vky\n43YPo8N4rQ2Qro6zahZm8TLWtcqTMKxE0krDCAaACwF82nhOwfCJ37LXdkPntmubuanRaUbY6VZA\negC5xBCINcbfgZNOkssvv1ymp6cjMdRBPCUnL6KWhpxhJZJWGkIwAOQBPAygW8qC0edxroyMjMw/\nik0cE5idXbixzuysu7E0Dbt917bDhzWZmJiQ177mNY65ilWATEBfrd2eySxIaJtUEwoKYuy9xCmI\nIY/T6FNQSFIoFosLbGWjCMYpAJ4D8F+GWPwBwNMAXupybrS9mlLMWlE9PWURWLbMu8Cgfde2q6/a\nL61Qkgek35gFZS8WmAPkbwzPwimpHYWH4bcsStT1tqq5Tq2vTUi1pFkwVgE44vLefwNY6nFuZB1Y\nC+Iacc7O6qvBCwXd0OZy4hhmMrGHr849553SjnLtJ81IYtvrQLUCUmhrcxSLKA2kHy8lroq+UTqp\nTIqTpJJKwQAwDuDnAI4BeAbAFbb3/6tRchhxjziti/vyefdEtr0dl269TvLGTCf7ftodllxFGypv\nahSFEa+HkY1zRhVXc5MkkkrBqPaRJsEQqc1otr9fD00Vi96rjzVNk4svuECyhihoWLyhUY8hFBs3\nvD7y8uNOhBXVaoSKHgZpRigYKSGOEad97wtzJfjsbPlhUiqV5APvf//8dqmmKHzP8CrM2k8dgLSi\nRd507v7oGurzXryOnT4f1nOz95v13Go9P+YwSJKhYKSAOEecpiEyDZW5f7fVUL3vyisX5CpMT6Lb\nWHRn7redAaQte6Fcd52WipFxNf1qioNp3I8di044OEuKJBUKRsKpZ42jw4c1uXbHDmk3chWn22ZA\nbQTkNkMozj/vfFm/vpS6kbFTaXa3v04ECekRknYoGAmg0oiy2uMg6AZUk01n7pBCa+t8mOmAQ66i\nw/Aw3nvllZG3oxbYBXJgoByWGx0te1yVjL910gDzDqSRoWDUmWpj6V7x9KAcPqxJru09ogwhsIrD\nUkM0lqK8T8XV73+/lEql4F+UAJz6fd06fQ2KfWGil/G3ik5Hh3BmE2loKBgJIEws3WrwikU9HNLf\nH350O35wfH5vilWG93DIEn5abcyKagfk4gsuCD37KUleiFNbTG9hcHCh8Xf6rP3foFAQWbHCe+Fj\nkLYQkjQoGAkhzCwoq9Dk8+FHtwcPHpSckbi21oBaYkybNRfgdWazcvv+8LOfkj4DyK30idc+IlF4\neUnvF0JMKBgJwK+H4TUi7ugIFz/feMop0uYwA8qsAWUuwLvk7TsiWVMRVS2nqLHOEnPKYVTa6a7a\ntnPtBUkDFIw643d06RZz7+3VwyCFgv8ZOuZeFQcPHpQ89DpP9n21zRpQbYC89S3bIh3x1rtarBuV\nZknFvfqaq7tJ0qFgJAC/o1OnWT1BF48dGh+X3nxeTu/pkY5MZn61dg8W14A6YdWq+aR2EMPtdT9e\nI+kkj7KrXbPhdVzt9QmpFRSMlGEdhdq3W7Uf29E0TXrz+XlhKKK8sdEhI2dhLsLbesklodrn5Sn4\n8SKSOMqOalW49Vzrv9XsrMimTcxhkORDwUgRbh6G+Z7bVqkTExPzj9N7esQaenq5UvMbG7UDcnx/\nf9VTZcPmKZI8yo6y7pRTEn3TpoXXpFiQJELBSAle6wbcDOyBsTEptLbKGiOp3ZnNSk9b28Lkdj4v\nBw8elG3btskDD3x50XeGJainkIQcRpzY+yPJ4kiIGxSMFOE1S8pumA+MjS3a/W6pIRq9+bxsLBSk\nN5+X8YPj89dyqyUVlErG0G20HqaAYBTX8XvdsLj1RxLDb4R4QcFIMVZD1Nenr9S+77775Nprr5Vc\nJiMDtplPGwE5sbNTJiYmZGZmRp59Vlswqh8dFclm3ffE8EMlTyEqT8IrN1DN9aP2dNyuV2maLiFJ\nhIKRUuyLxY5bPiatgGSN8JOZvL7V5mEsaW9fsJ7CPvodHq5+1FtphB5VOMbtOtVeP+pwkf3+qxU1\nQuoFBSPFzM6KPPusXlE2aySt7Qvw8iiX+uhqbZVD4+OLrmOGRsyVzbUY9UYVjnG7TrXXjztcxFIg\nJI1QMFKMXv8pM1/Ww6kE+RpA2jMZ2b1zp+NKbXM0vWePHo4aHS2/Xqsy6o3uYRDSKFAwUoqmadLR\n0iJ5QP4SkBPhvF1qHpCHHpp2vIY9vm4v5R2HWDRbDoOQRiKsYCj93PqglLobwFsBPC8i643XbgNw\nAYBjAH4E4AoR+X8u50s9218NR48exWOPPYaHHnoIn/rEJzAA4AsAzgDwMICnAFwNoBfAzwG85W3b\n8cX/89eu15ubAzIZ9+NKhDm/2u+sdJ047snEfN18LUy7CUkrSimIiAp6XkscjQnA3wM4z/bagwBO\nFpENAH4IYFfNWxUjR48exe6dO7H6uONwzXnn4a8+8Qko6Mr4BwDvBbAZwI0AXgDwI2zGFR8oeYoF\nsNjgBTWsW7YAU1P686kp/XhubqGRjfI7/Vyn2us7ff7ss4FTT9XvcWoKWL9ef83rPgkhBmHckigf\nAFYCeMLlvYsBfN7j3Op9sxpy+/790masyLavq+gySnqYxQI7Oy+SJUtKgePvYZOwxaI+pTdoAcS0\nYe45ks/rlYHNeyWkmUDIkFS9PYxKXAnga/VuRBRctW0bbvyLv8DLRHAcgPXG6+sBrALQA6ADwA8A\nvOuKbTjllC/h8OEB7N0L3H8/sGtX5VGw1VMAFnoKlRgaArZvB376U2B2Vj9361bgllsaK1wzNAR8\n6EPACy8Av/sd8OEP668RQipT1xwGACilVgL4shg5DMvrNwA4XUQu8ThXRkZG5o+HhoYwVKf//V7x\n9qeeegpnrFuHhwG8HMAfAZiCLhZPABgC8GIuh3379+ONb3wjBgYGFsXx3eL6dkxDf801wJ136mLj\np0us5912m25QR0aAvXsDd0VdqZT3mJoCLr4Y+P3vAaWAbBZ44AGKBmlspqamMGWOJAHcdNNNoXIY\niQxJAbgcwLcA5CqcG41/ViWVZuTcc889cqJliuwhy6I8r3UVYWf6VFP/yQzZVLNNbL3wszp906Zy\nsUezjpe9YCAhjQ7SOq0WekTmiOX4fABPAujzcW50PVgl5pz/667TpFCYkNtum5hfL1EqlRblLdoB\nybW0zK+r8Lt3RiUDbi9VYdaUqkQUW5SGIWzOxe08P/Wv7NVkKRak2UilYAAYhz5r9BiAZwBcAX1m\n1I8BPGo87vI4P+JurI4L/nhMcsjKKx08hx3bt0sekLXGmorengvl8GFdUCoZZr8ew7FjIl1d5UV7\no6P6caX9NaxEuXK50rXCelCVzmMxQEK8SaVgVPtIkmB85MN6VdnTjEV3t9pqPs3O6p7GPffcI6VS\nyXfRulp6GF7HQfErBmFXYzudZ3pFYfdVJ6RZoGDUkWef1aRD5RbuTwHIKUZV2W9/e8bReO7Z4z0S\nrkcOI8j3VMKv8Q7rEVjPM/MT69aV8xMDA875Ca4CJ80OBaOOzMzMyMbubrHWfloPSLfFw7Abz9HR\neEbCUY7Yo6CSGETZXnMtid3riPJ7CWkEKBgRE8RQ2/fYNms/dWazC2Y/mcZzz554RrjVjpyjjv27\nhY2s7Q2zB7bXfdq9Nq9rMddBmhUKRoQ4GaQzztDk29+ecawUKyJyaHxcevN52dDdLUtyOdllqypr\nN5723EJU4ZBqVnpHvXeE037XdoEIuwe2031OTuqVes3y7qOj7gJED4M0MxSMiLEalK7OMelqzcmp\n3d3Sm887rpkQ0T2NmZnFopL0mHlc7XMy6nEZavMezFDf8LAuHlZhjnq3QELSCgUjBkZGRBQWz37q\nzeddPQ03kj4rp5btiysUZJ9We9xx3ov4nM4lpBkIKxhJryVVN6amgM985im0t3wQXwXwOIAigFsB\nHJfJ4Omnnw50vagqu8ZFrdo3NaWXLBkZ0f9aqhUAWFz3KshxJrPw+r/9LXDRRXp5E3tdrKT/exCS\nSMKoTFIeiDGHceKacSm05uREw7M4ZJn9VMjlYvMwGnnk66d0h1Pew1yNbU2SO61Edzp/xYp4vBlC\n0gwYkqoeMwdRKpUWzXrqBaRozH46MDYW6Lp+Y+bNEFuvJIjWHEdfn76uwtxBcHRUX1tRKOhC0Nfn\nXPrDei37NFtCCAWjasxZTqf39Eghl5NT83mxrqtYa5T6CCoWJn6TvWFXajcS1hyH2W/DwyJK6X87\nOip7Dc0gvoSEhYIRklKpJHfccYcUcuWV2qYnYfUwluRyUiqVKl7PawRdKdlrGrl3vUv/3PBweX/r\nKKgm3FWrUJmTsJr9Njio/83n/XkNjRzeI6QaKBghuNYoCHiiIRA7LB7FqvZ2WZLLycZCwXMqrRWv\nUa1fD+NTn9KbcNpp+oj6mmuCj4ydDGU1I+5ajdadvmdgYKGHsWyZXn7dXNlNr4GQ4FAwAuJUcjwP\nSMnMWeTzUiqVHNdVeOG2utmtbLi1vLb5uXPPLYuGfS2BG/ZrmOd4iVaQxYNB1k9E5cmYSW5rDsN6\nzNLkhISDghEQ+6ZGAsgaQI5vb5clubysXTMe2hg5hZ7so3zTgNtXP99+uy4Sr3udfo13vavy99lH\n5qOj+jX27Fls3KspT+Jn/UTU3oh93YTbOgpCiH8oGAFx8zAuueQ+6e3VQs+oqTQSdyuat2yZbsSz\nWT0M5bZa2e/3Dg8vNu5hCyD6ua+wnyWE1B4KRgjsmxptPmu74wg6yBoKP6Nrp5G6+doNNywMKZnh\nFz+Jb/MaZi0lt7CYtW1uJdarLRLIwn6EJBcKRkjMTY3uuafkOCoOGmIJss7A7mGYr01MLPxOa8ze\nDbuXMjq6OF9iFx2nTZzcEuRBigTSwyAk2VAwQuAnUWweR2EA/VZwNdsSJMlsFxjrdd1WRW/apM9C\nMkVr3bqyMIS9Z65/ICT5pFIwANwN4HkAT1heWwrgQQA/ADABoMfj/NAdZg+zmEbWLakaVYjFbcqr\n02eCfGeYyrD2z9hXToe9Z65/ICTZpFUwtgDYYBOMWwF81Hh+PYBPepwfusNmZ/XRdU+PbhB7evTj\nSobXqRxFHETl1fgx+k6f8fP9nMFESDpJpWDo7cZKm2B8H8DLjOfLAXzf49yqOq1Y1FcNm6uHnUbg\nXmGbuIgqrOPH6FdaN+L2/dYwnrlGwnpM0SAkuTSSYPzS9v4vPc6tqtOKxXJdoo6OYAY1bqoN6wQx\n+k6f8fP9TnWemOQmJPk0smD8wuPc0B02O6t7C4WCLgaFgn7sZBjTOkXUj9GvVpjsdZ7S1keENCNh\nBSPrc9uMWvK8UuplIvK8Umo5AM3rw3v37p1/PjQ0hKGhId9f1N0NPPAAMDSkP3buXPwZ+4Y/5mfT\ngJ9NgqrZSMjsm+Fh4HOf0/+mrY8IaQampqYwZd+tLARKF5v6oZRaBeDLInKqcXwrdC/jVqXU9QCW\nioiDKQeUUlJN++fmFhpIp+MtW/Sd2oaGdAO5axcwPc0d2sy++fjHgRtvBC69FLjvvvIx+4iQ5KKU\ngoiowOfVUzCUUuMAhgD0QZ9eOwLgSwDuB/AKAD8G8Kci8iuX86sSDD9UEpVmxuwLt7+EkGSSSsGo\nlloIBiGENBphBaMljsYQQghpPCgYhBBCfEHBIIQQ4gsKBiGEEF9QMAghhPiCgkEIIcQXFAxCCCG+\noGAQQgjxBQWDEEKILygYhBBCfEHBIIQQ4gsKBiGEEF9QMAghhPiCgkEIIcQXFAxCCCG+oGAQQgjx\nBQWDEEKILygYhBBCfEHBIIQQ4ovECoZS6sNKqX9XSj2hlLpXKdVW7zYRQkgzk0jBUEodB+CDAE4X\nkfUAsgD+rL6tSjZTU1P1bkJiYF+UYV+UYV9UTyIFwyADoFMplQXQAeDndW5PouF/hjLsizLsizLs\ni+pJpGCIyM8B3A7gGQA/A/ArEZmsb6sIIaS5SaRgKKWWALgIwEoAxwHoUkq9s76tIoSQ5kaJSL3b\nsAil1J8AOE9E3mscvwvAJhHZbvtc8hpPCCEpQERU0HOycTQkAp4BsFkp1Q7gGIA3AHjE/qEwN0wI\nISQciQxJicgMgC8CeAzA9wAoAAfq2ihCCGlyEhmSIoQQkjwS6WHYUUqdr5T6vlLqP5RS1zu836aU\nOqSU+qFS6ttKqePr0c5a4KMvhpVSmlLqUeNxZT3aGTdKqbuVUs8rpZ7w+Mwdxm/icaXUhlq2r5ZU\n6gul1KBS6leW38SNtW5jrVBKrVBKfVMp9aRS6ohSaofL5xr+t+GnLwL/NkQk0Q/oovaf0GdMtQJ4\nHMBJts9cDeAu4/mlAA7Vu9117IthAHfUu6016IstADYAeMLl/TcD+IrxfBOAh+vd5jr2xSCAf653\nO2vUF8sBbDCedwH4gcP/kab4bfjsi0C/jTR4GGcB+KGI/FhE/gDgEPQpt1YuAvAPxvMvQk+SNyJ+\n+gLQcz4NjYhMA/gfj49cBOBzxme/A6BHKfWyWrSt1vjoC6AJfhMAICLPicjjxvPfAHgKQL/tY03x\n2/DZF0CA30YaBKMfwE8sxz/F4pue/4yIzAH4lVKqtzbNqyl++gIA3m642v+olFpRm6YlDntf/QzO\nfdUsbFZKPaaU+opSal29G1MLlFKroHte37G91XS/DY++AAL8NtIgGGFoitGUC/8MYJWIbAAwibLn\nRZqX7wJYKSIbAfxvAF+qc3tiRynVBT3acK0xum5aKvRFoN9GGgTjZwCsSewVxmtWfgrgFQCglMoA\nKIjIL2vTvJpSsS9E5H+McBUA/C2AM2rUtqTxMxi/CQOn301TICK/EZHfGc+/BqC1QT1wAIBRf+6L\nAD4vIg84fKRpfhuV+iLobyMNgvEIgDVKqZVGifM/gz6KtvJl6MleANgK4Js1bF8tqdgXSqnllsOL\nAJRq2L5ao+DuTf4zgHcDgFJqM/R6ZM/XqmF1wLUvrPF5pdRZ0KfTN+KAyuTvAJRE5DMu7zfTb8Oz\nL4L+NpK60nseEZlTSm0H8CB0gbtbRJ5SSt0E4BER+RcAdwP4vFLqhwB+gQYthe6zL3YopS4E8AcA\nvwRwed0aHCNKqXEAQwD6lFLPABgB0AZAROSAiHxVKfUWpdR/AvgtgCvq19p4qdQXAP5EKXU19N/E\nC9BnEjYkSqmzAVwG4IhS6jEAAmA39JmFTfXb8NMXCPjb4MI9QgghvkhDSIoQQkgCoGAQQgjxBQWD\nEEKILygYhBBCfEHBIIQQ4gsKBiGEEF9QMAghhPiCgkEIIcQXFAxCCCG+oGAQQgjxBQWDkAhQSmWU\nUtNKqd8opU60vfc+pdSLSqm9dWoeIZHAWlKERISxl/zjAJ4GsElE/qCUOhnADPRKw68X/ocjKYYe\nBiERISLPANgGfWez25VS7QDug14F9DKKBUk79DAIiRil1J0ArgbwrwBeDeDtLhv5EJIqKBiERIxS\nKgfg3wGcAOCzIvL+OjeJkEhgSIqQ6NmA8la6pyil+P+MNAT8IRMSIUqpbgBfAHAUwA0AXgPgpro2\nipCISPwWrYSkjM8CeAWAN4rIQ0qp0wHsVEpNishDdW4bIVXBHAYhEaGU2gZdMD4uIh8zXuuBPtU2\nC+A0EfllHZtISFVQMAiJAKXUHwH4LoBHAQyJyIuW9zYDeAjA10Tk4jo1kZCqoWAQQgjxBZPehBBC\nfEHBIIQQ4gsKBiGEEF9QMAghhPiCgkEIIcQXFAxCCCG+oGAQQgjxBQWDEEKILygYhBBCfEHBIIQQ\n4ov/D53q60r+p/11AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x3fff3bb97850>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot them.\n",
    "print(\"Evaluation Metrics: \" + str(model.metrics_names))\n",
    "print(\"W: {}, B: {}, Loss: {}\".format(dense.get_weights()[0][0][0],\n",
    "                                      dense.get_weights()[1][0], y_p_loss))\n",
    "print(\"Slope: {}, Bias: {}\".format(slope, bias))\n",
    "plt.scatter(sample_x, sample_y, marker='x')\n",
    "plt.scatter(sample_x, y_p, c='red', marker='o')\n",
    "plt.xlabel('x', fontsize=18)\n",
    "plt.ylabel('y', fontsize=18)\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}