LSTM_Stock_prediction_20170528(Quandl).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part 1.  A guide for how to use the lstmstock.py\n",
    "\n",
    "This is a presentation of the simplest way to implement Long Short Term Memory for prediction of stock price. More codes are in the lstmstock.py file. It also fixed the problem of using pandas datareader to download stock data, which is currently unavailable."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Import all the module and function\n",
    "You can import all module one by one or import my lstmstock.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "# from lstmstock import *\n",
    "import seaborn as sns\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.pyplot as plt2\n",
    "import pandas as pd\n",
    "from pandas import datetime\n",
    "import math, time\n",
    "import itertools\n",
    "from sklearn import preprocessing\n",
    "import datetime\n",
    "from sklearn.metrics import mean_squared_error\n",
    "from math import sqrt\n",
    "from keras.models import Sequential\n",
    "from keras.layers.core import Dense, Dropout, Activation\n",
    "from keras.layers.recurrent import LSTM\n",
    "from keras.models import load_model\n",
    "import keras\n",
    "import pandas_datareader.data as web\n",
    "import h5py\n",
    "from keras import backend as K\n",
    "import quandl\n",
    "from keras.models import load_model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# 2. Input of the model\n",
    "\n",
    "1. Quandl api key Obtained from https://www.quandl.com/product/WIKIP/WIKI/PRICES-Quandl-End-Of-Day-Stocks-Info Please login to use your key if you have one.\n",
    "\n",
    "2. Seq_len\n",
    "It is the window frame of the past data, how many days of past data would you like to be included in this model?\n",
    "\n",
    "3. Shape\n",
    "Please input the [seq_len, Amount of features, 1] to it to setup the model\n",
    "The amount of features will depend on how many moving average would you like to be included in this model.\n",
    "\n",
    "4. Dropout\n",
    "Fraction of the input units to drop. It helps to avoid overfitting.\n",
    "\n",
    "5. Decay\n",
    "Weight regularization. Another parameter to avoid overfitting.\n",
    "\n",
    "6. Epochs\n",
    "How many Epochs would you want to fit the model?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "quandl.ApiConfig.api_key = 'zpFWg7jpwtBPmzA8sT2Z'\n",
    "seq_len = 22\n",
    "shape = [seq_len, 9, 1]\n",
    "neurons = [256, 256, 32, 1]\n",
    "dropout = 0.3\n",
    "decay = 0.5\n",
    "epochs = 100\n",
    "modelFileName = 'model-e' + str(epochs) + '.h5'\n",
    "stock_name = 'FLWS'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Pull the data from Quandl first\n",
    "\n",
    "If you would like to include moving average, simply put the number of days into a list. \n",
    "They will be included into the neural network model.\n",
    "\n",
    "ie. putting 50days, 100days, 200days into [50, 100, 200]\n",
    "\n",
    "In the Quandl database, Open, High, Low, Close, Volume are all adjusted. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def get_stock_data(stock_name, normalize=True, ma=[]):\n",
    "    \"\"\"\n",
    "    Return a dataframe of that stock and normalize all the values. \n",
    "    (Optional: create moving average)\n",
    "    \"\"\"\n",
    "    df = quandl.get_table('WIKI/PRICES', ticker = stock_name)\n",
    "    df.drop(['ticker', 'open', 'high', 'low', 'close', 'ex-dividend', 'volume', 'split_ratio'], 1, inplace=True)\n",
    "    df.set_index('date', inplace=True)\n",
    "    \n",
    "    # Renaming all the columns so that we can use the old version code\n",
    "    df.rename(columns={'adj_open': 'Open', 'adj_high': 'High', 'adj_low': 'Low', 'adj_volume': 'Volume', 'adj_close': 'Adj Close'}, inplace=True)\n",
    "    \n",
    "    # Percentage change\n",
    "    df['Pct'] = df['Adj Close'].pct_change()\n",
    "    df.dropna(inplace=True)\n",
    "    \n",
    "    # Moving Average    \n",
    "    if ma != []:\n",
    "        for moving in ma:\n",
    "            df['{}ma'.format(moving)] = df['Adj Close'].rolling(window=moving).mean()\n",
    "    df.dropna(inplace=True)\n",
    "    \n",
    "    if normalize:\n",
    "        min_max_scaler = preprocessing.MinMaxScaler()\n",
    "        df['Open'] = min_max_scaler.fit_transform(df.Open.values.reshape(-1,1))\n",
    "        df['High'] = min_max_scaler.fit_transform(df.High.values.reshape(-1,1))\n",
    "        df['Low'] = min_max_scaler.fit_transform(df.Low.values.reshape(-1,1))\n",
    "        df['Volume'] = min_max_scaler.fit_transform(df.Volume.values.reshape(-1,1))\n",
    "        df['Adj Close'] = min_max_scaler.fit_transform(df['Adj Close'].values.reshape(-1,1))\n",
    "        df['Pct'] = min_max_scaler.fit_transform(df['Pct'].values.reshape(-1,1))\n",
    "        if ma != []:\n",
    "            for moving in ma:\n",
    "                df['{}ma'.format(moving)] = min_max_scaler.fit_transform(df['{}ma'.format(moving)].values.reshape(-1,1))  \n",
    "                \n",
    "    # Move Adj Close to the rightmost for the ease of training\n",
    "    adj_close = df['Adj Close']\n",
    "    df.drop(labels=['Adj Close'], axis=1, inplace=True)\n",
    "    df = pd.concat([df, adj_close], axis=1)\n",
    "      \n",
    "    return df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df = get_stock_data(stock_name, ma=[50, 100, 200])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. Visualize the data\n",
    "Print out the first 5 rows of the dataframe and plot out the Adjusted Close price and percentage change of that particular stock "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def plot_stock(df):\n",
    "    print(df.head())\n",
    "    plt.subplot(211)\n",
    "    plt.plot(df['Adj Close'], color='red', label='Adj Close')\n",
    "    plt.legend(loc='best')\n",
    "    plt.subplot(212)\n",
    "    plt.plot(df['Pct'], color='blue', label='Percentage change')\n",
    "    plt.legend(loc='best')\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                Open      High       Low    Volume       Pct      50ma  \\\n",
      "date                                                                     \n",
      "2000-05-17  0.373887  0.360843  0.375802  0.031033  0.363142  0.381950   \n",
      "2000-05-18  0.364629  0.353313  0.375802  0.025009  0.389492  0.381254   \n",
      "2000-05-19  0.355371  0.345783  0.375802  0.035487  0.389590  0.380957   \n",
      "2000-05-22  0.357211  0.360843  0.375802  0.027419  0.445512  0.381106   \n",
      "2000-05-23  0.357211  0.345783  0.371914  0.058014  0.376025  0.380709   \n",
      "\n",
      "               100ma     200ma  Adj Close  \n",
      "date                                       \n",
      "2000-05-17  0.558310  0.885206   0.373895  \n",
      "2000-05-18  0.554386  0.880531   0.370079  \n",
      "2000-05-19  0.551045  0.876294   0.366323  \n",
      "2000-05-22  0.547599  0.871677   0.377650  \n",
      "2000-05-23  0.544370  0.867469   0.370079  \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXeYFEX6x79FWDKKgAgiwZMgEhZYVlCRJIKKgRM9OFFQ\n+OGpgDlwiGc8T/E8zBJMoAKKgIiogKCCChIEJEgUYRGR5JJhw/v7452iw3RP2p7pmdn38zz9dKqu\nfqen+u2qt956SxERBEEQhPSihN8CCIIgCN4jyl0QBCENEeUuCIKQhohyFwRBSENEuQuCIKQhotwF\nQRDSEFHugiAIaYgod0EQhDRElLsgCEIaUsqvG1erVo3q1avn1+0FQRBSkmXLlu0hourh0oVV7kqp\nNwH0APAHETV1OK8AvADgcgBHAPQnouXh8q1Xrx6WLl0aLpkgCIJgQin1ayTpIjHLvA2ge4jzlwFo\nEFgGAXgtkhsLgiAI8SOscieibwDsC5HkagDjiVkE4FSlVE2vBBQEQRCix4sO1TMBbDft5wSOpQ/N\nmgFK+S2FIAhCxCS0Q1UpNQhsukGdOnUSeeuisXq13xIIQkqRl5eHnJwcHDt2zG9RUpayZcuidu3a\nKF26dEzXe6HcdwA4y7RfO3AsCCIaA2AMAGRlZUkgeUFIU3JyclCpUiXUq1cPSlq9UUNE2Lt3L3Jy\nclC/fv2Y8vDCLDMDwE2KaQsgl4h2epCvIAgpyrFjx1C1alVR7DGilELVqlWL1PKJxBVyIoCOAKop\npXIA/AtAaQAgotcBzAK7QW4Cu0LeHLM0giCkDaLYi0ZRn19Y5U5EfcKcJwB3FEkKQRAEwVMk/EA4\nfvnFbwkEQYiR6dOnQymFn3/+2TVN//79MWXKFADAwIEDsXbt2qA0eXl5eOihh9CgQQO0atUK7dq1\nw2effQaAB2Tu2bMnPj+gCIhyD8Xx48DZZxv7Mpm4IKQUEydOxEUXXYSJEydGlH7cuHFo0qRJ0PER\nI0Zg586dWL16NZYvX47p06fj4MGDXovrKaLcQ7Fli3VflLsgpAyHDh3CwoUL8cYbb2DSpEknjxMR\nBg8ejEaNGuGSSy7BH3/8cfJcx44dg8KiHDlyBGPHjsVLL72EMmXKAABq1KiB66+/Puiezz//PJo2\nbYqmTZti1KhRAIDDhw/jiiuuQIsWLdC0aVNMnjwZALBs2TJ06NABrVu3Rrdu3bBzp7d+KL4FDktJ\nCguBEvI9FISouOsuYMUKb/PMzAQCytONjz/+GN27d0fDhg1RtWpVLFu2DK1bt8a0adOwfv16rF27\nFrt27UKTJk1wyy23uOazadMm1KlTB5UrVw55v2XLluGtt97C4sWLQUQ4//zz0aFDB2zZsgW1atXC\np59+CgDIzc1FXl4ehgwZgo8//hjVq1fH5MmTMXz4cLz55pvRPwsXRFNFQ2Gh3xIIghAhEydORO/e\nvQEAvXv3Pmma+eabb9CnTx+ULFkStWrVQufOnT2538KFC9GzZ09UqFABFStWxF//+lcsWLAAzZo1\nw5w5c/Dggw9iwYIFOOWUU7B+/XqsXr0aXbt2RWZmJp588knk5OR4IodGau6hsJthxCwjCNETpoYd\nD/bt24d58+bhp59+glIKBQUFUEph5MiRUed1zjnnYNu2bThw4EDY2rsTDRs2xPLlyzFr1iw8/PDD\n6NKlC3r27InzzjsP33//fdT5RYrU3KNBau6CkBJMmTIFN954I3799Vds3boV27dvR/369bFgwQJc\nfPHFmDx5MgoKCrBz507Mnz8/ZF7ly5fHgAEDcOedd+LEiRMAgN27d+PDDz+0pGvfvj2mT5+OI0eO\n4PDhw5g2bRrat2+P3377DeXLl0ffvn1x//33Y/ny5WjUqBF27959Urnn5eVhzZo1nj4DUe6hsA8i\nkJq7IKQEEydORM+ePS3Hrr322pPHGzRogCZNmuCmm25Cu3btLOmcBg89+eSTqF69Opo0aYKmTZui\nR48eQbX4Vq1aoX///sjOzsb555+PgQMHomXLlvjpp5+QnZ2NzMxMPPbYY3j44YeRkZGBKVOm4MEH\nH0SLFi2QmZmJ7777ztNnoMgnhZWVlUVJMVnHpZcC1aoB778ffG7DBqBRI2P/4EGgYsXEySYIKcq6\ndetw7rnn+i1G1DRr1gwzZsyIOZ6L1zg9R6XUMiLKCnet1NznzAHcfGDz8qz79g9hQYGYagQhTeja\ntSuaNWuWNIq9qEiHaijszSSzIv/qK6BTJ471vmpVQsUSBMF75syZ47cIniI191D861/WfXPNvVMn\nXv/0U+LkEYQUwi+Tb7pQ1Ocnyj0UtWpZ95MwfoQgJCNly5bF3r17RcHHiI7nXrZs2ZjzELOMhijY\nO+b//g9YtgwYOhR48UWgQQOgVSteC4LgSu3atZGTk4Pdu3f7LUrKomdiihVR7ppx41iZm9GhBsqX\nN44tX86LIAiulC5dOm06JlMVMctoliwJPqY7UEuF+QZ67J8qCIJQVES5a8aOZbPM778bx7RyL1ky\n9LXS9PSHr74CVq70WwpBSErELGOnfn3g6FHe1uuCgtDXnHFGfGUSnNEeS9JpV7w5cADYsQNIwUFT\n8URq7naOHTO8Yu69l9daybuxYUN8ZRIEwZ0WLYAmTeQjb0OUuxPVqwPmoEDHj1vPjx8PVK1qhCwo\nwgzlQowcPuy3BEIy8OefwNatvB2uElbMEOXuxrRpwFlnAf37Ay+8YD3XujXX7i+5hPcDkeKEBGLu\nGxGKL3v3Gttdu/onRxIiyr1GDefjEyey0s7IYJdIs8eM7mANTLkVVLMX4o+5liY1tuLLgQPGtnit\nWRDlHirwV14eULo0b+vOO8Dwf8/I4LXU3BOPWaFv3uyfHIK/mCtWV17pnxxJiCj3/Hz3c7rmDrCd\nXaMHNelziai5EwETJnDYYQEYPNjY3rfPPzkEfzG/v7VrswlVIrUCEOUeHNbXjFm5V69uHD/zTF5r\nc00iau7vvAPcdJPhwVPc+eEHY9vcNBeKF+b397XXeDJu2wxJxZXirdzz84FDh4z9ErbHYTbLuA1k\nys8HPJyx3JWbb+b12LHB0SqLO9KaKb44tbzFkwpAcVfuGzda97Nsk5sQGTX3UCTCc+P0043txx+P\n//2SGXtrS2ruxRenlrf4uwMo7sp92zZeT54MrF/vbPIwK/eVKzmdmXChCbzijz8Sc59UYOdOXuuP\n3P79/ski+EuoPrN4U1BgbfknGcVbud96K6/nzwcaNnQOEGZW7s2bczozt9/O6woVgJ9/jo+c4ZqZ\nq1YldSHznNxcXjduzOthw/yTRfAXJ+WeKO+1++8HKlXimFTz5iXmnlFQvJX7r7/y+tJLee1UC9c2\ndze0S96RI8CYMc5pDh0qWnAxpwJ86qm8zsvj4deVKgHffBP7PVIJbYY55RTjmMyIVTzRZhmzw0Oi\nWnJmD7ouXYLPX3cdcNttiZHFgeKt3M8+m9faP9ZJuYezuY8bZ2y7uWC1aGG1mUfLqFHBx3Tt9c8/\njWMdOsR+j1Rh61bgoot4u1Il4/jzz/sijuAzU6fy+r33jGMVKgALFwLXXMO16niNgzCPjgWsLew9\ne4ApU4DXX/etD6B4K/errgIqVzbMMbt2BaeJpENV4xY9csuW6GUz8+ijzsd//92q3IsDTZsa22ef\nzSEiAODtt30RR/CZDz7gdYMGwN13G9vt2wMff8z7ifIumzTJ2DY7a+iKWIIp3sp91Cirp4XT9Hmb\nNoXOIzPT2H755cTO0jRnTnAfQDpz+LC1dlSjhvECC8Wb0qWBG27gbbsHzTXXGNtr13Jt3mlynmix\nz2+qw5Hk5HCtXTNgQNHvFQPFV7kvWxZ87OKLI0tnpm7d6NJHi7lJN2UKcN99xv5NNwWn//e//fUg\niBcffQRUrBh8vGXLxMsiJB81axr9Y/ZxD2+8wev77gPOO4+3p08v+j3tLfUbb+QPx1lnWc2EU6fy\n6Nlw80J4TPFV7q+/Hlm6/v1Dn3/7bZ7gQ+N1T/3nn/O6XDng2muBO+8MnX74cFbw6YZ9flvtxioU\nb0qU4FGpJUoAp53Gx2680ZpGv0P//a9xLNzUmeH45htuIVStGtmHYscOoE2bot0zSoqvcncb1Thy\npDEaFAD+9rfQ+Zx6qrXDZvZs/nrPmROcNpKOlYMHraaHb7/ltfbKqV3biF/tRjqOYLV7QGhbO2CM\nTwgVSkJIPwoK2ImhShXer1nTPa09cqj+EMTKokW8vu464Oqr3dONHGls//hjQsdkRKTclVLdlVLr\nlVKblFIPOZzvr5TarZRaEVgGei+qx+gmnO6Q0dx3n1E7bt48sryUMra1S57Z5qY5ciR8XpUrs/lB\nm1bMeWvMLoAas+0fSHgTMK6E+yguXcrrIUPiL4uQPGiFXa4cr0MNKKxa1bpfq1bk93nvPe4gnTmT\n7elEwBNP8Dlza8DM3XfzNWYzKsAflQR5z4RV7kqpkgBeAXAZgCYA+iilmjgknUxEmYFlnMP55OLd\nd3l93XXB5/TDd1KsblSowGv90XCqRUYTA6VmTaBnT2cZnDx47O5e4cw3qcRvv1n3J0+27j/7LK/N\nrpFC+qP/9xUrnM/rkcxAcM19zZrI7pGbC/Tty44LV17JLcannzYGDeoIsXYyMoBzznE+l6BYSJHU\n3LMBbCKiLUR0AsAkACHaIWmA9lePRrn/8ov1WiflHi4GitlPfs8etuXpGsI//mGcc1Lu9mOJCGaW\nCIjYFGVGf0g1bdoYngpC8WHWLF6bTR3vv89ukA8+yBPXm98Dc7n59NPI7uFkRhk+PPiYrqG3bctr\n86xQr71mTZug2FCRKPczAWw37ecEjtm5Vim1Sik1RSl1lsP55GHdOl6ff77zea1k7VEiQ1G9OjcL\n7crdPL9quC/299+7n3vlFWPbqfmpbYC6xt6xY+h7pQp/+UvwMbsLmlIcU/+dd6zHDxzwxuVNSA5O\nnAAWLzb2r7qK1zqMCAD06cOdnf/5D+9Xrmycq1fPUL4XXhjZPcP1b2l0mbzsMja/mkes/uMfbCZd\nuZL3//vfhJhmvOpQ/QRAPSJqDmAOgHecEimlBimlliqllu4uynD8oqJ70+1ujJpq1XgdaQHQlClj\n2Lq1zTwnxzgfquZOBDz1lPO5zp2tHxqljI/H5ZfzJB7nnMN5jBrFngChXATffdfoqE12dIvIjF25\na+xlqkcPIDubn1c69UEUV8aNY+WsnRW03bxVK/drzD7ua9bwWJYzz4w8LLB5Brann3ZPp+McNW1q\n9AGYKVHC6gSgK5hxJBJ/oB0AzDXx2oFjJyEi8zjccQCedcqIiMYAGAMAWVlZ/sXlbNeO/dEfe8z5\nfL163DHaqFF0+ZqVu1a+99xjnA+l3JcuBT77zPmc00xPpUq5f/1LljQ+LkRAkyZc4ObOZfn0xy0V\nQqPefDPw1lu8XaECv5ROJq+zz+aRwH/8wUp+0SJgwQLj/OLFwAUXJEZmIT7owHwvv8xmD/1ehDLJ\n2Vu58+Zx5cDcog7FWWcB27fz6PXTT2fvuVq1WIa1a410f/87v2ehKlXmVsTw4cC0aZHJECORKPcl\nABoopeqDlXpvAH83J1BK1SQi3XtxFYD4f5aKQrly/Afrr60T5mHukVK2rKFUZ8zgwmduRroNQ546\nle2DbrhN4u1GqVKGHIsWGS9Fjx7W1sixY+61YL/58EPud9CK/a23ONzCsGHBng8Av1xPPskvb58+\nwed//pmVQOvW8ZVbiB86ONiMGbyORLnb2b+fx6UsWsQT39jHT5gZOpQV+znnGLGh9JgWe3hwpcIP\nqDN/aBIxoxoRhV0AXA5gA4DNAIYHjj0O4KrA9tMA1gBYCWA+gMbh8mzdujX5QmEhUfXqROXKeZ93\nvXpEp5xCxHViovXriSpVMvZHjQq+ZsMG47x96diR1/36RSdHuXJE//gHb69Y4Z5/jx5F/slxwy4r\nEf93P//snH7xYvffac9HSC1OnCAaOtT6P37wAdEjj/D20aOhrzdf9/XXRK1aGftvv81pfvmF6Phx\n9+u8Que3cGERssBSikBvR2RzJ6JZRNSQiP5CRE8Fjj1CRDMC28OI6DwiakFEnYgoToHNPWDSJG62\n212jvKBMGasv+7Fj1k5UexQ5wDnw1yWX8Fq7VUbjtQPwb9MjcEONmA3VgZtM/PWvvFbK3VTmZOcU\n0oNRo4AXX7Qeu/56w+skXHA/HbjvnXc4xIi5tfrqq2zqq1+f39/cXB79HC+beAJNg0Ucg5tiHDvG\nzfd4UaaM1R5st5U7ecvYPTwAI1a7bsYVxTbuZK/X7N3LCnPrVvfOZT+wf3Q6dw5/TbKal4Si4xRT\nSFOqVHivtvr1re+QuawcOWKd6Ea/e2bM4YRTiOIVfkDb6uKF3fZn7rSpWtW5I9Ds4ggAX31lDMbR\nNf3Vq6OTQ9cOvv3W8NM126F//NGavl49VvJ68pIff3RuZUTC1Klci460w8rM0qXso2yv3ejh5aEw\n19zNH6po3FmF5MQ+rsFMLOMbzK3lwkIjJrwTixd7WyGcMIGjRLq5YXtI8aq5RxObPRbsBc1cKDIy\nwgcV+/57dvVq3pwVVIUK7KcdrXL/7jte60ktAO4cmjiRt+vUcb6uXj1WrtrLhIg9iwoKgB9+iOze\n117L67vvDh68EQ57YKV33mG3TT1TVijMtTH9kQL4g2ruyMrLCz+7lpBcmKOc3norMHq0sR9LADDz\nyNXmzY2pMp04y+MhO2efbZ3gJ44Ur2pNvEcx6vy1MjH7uGdkAF9+aShYILh2q5uEVapw8K8ePXjf\ni3jQtWsbwZIqVmTPEifM7oPz5rFXwZIlwUO8//tf/r1uJqPXX48utr3dfHTTTbzMnm2MOwiFm1mm\nRAnrDFV79kQuk5AcPPKIsW23vccyEYa5rIXrz4qk1ZikpJ5yLywERoywfn0jJd5+3VrBONntMjK4\nY0fX5o8eDe4EPNM28LdxY54u7H//K7pstWqxmWbUKJYlkkmlzaPsWrZkVzQ9+GPECG6JrF9vvcbs\nQvrhh5HLZ5/gWwcDixSzcr/0Unad1AO1vvrKOP/SS9HlK/jLjz9yuFyAOz8zMoreeW5uQZsrW06k\ncF9O6in3vn251tmiRfTXxmIHjgZdc7cr96FDg01C9eoZ2089xR8ep8BXF14YvTlp/nzr/oYNXINt\n3NgIT1CiBCvo//yHO3ojsQHu2cNNYiIjMuWGDcb548etJiSnaQvdsE/ubR7wEQnm5nmvXhyH32y7\n1/Pl7tplOLgJ3jN6NCtEt/mEo8X8v/brx+s9e4o2A5ebefSZZ4ztb74JjhibYqSecteDgmIJX2B2\nf3zhBW/kMeOm3EuXtpqEcnJ4JKXGvO0FHTvyh+zDD1nhOk0fCLAr2YMPsplm0SLD5TAU997LH4bf\nf+f955/neNa//x48gUY0nbL2qJZ69pxYcFIsH33E68qVeRh5iRKhPYmE2PjHP/i5hgq/Gw3atbh7\ndyMCY/nyHFdm/35g377o89SmGLun2gMPcJ9MYSH3PTlFjE0hUk+56xres44RDkJjrrnHI7CWVhb2\nqfaGDbPajV9+2Xr+uee8l6VMGa7BRlPr//JLY5uIazjharhff81eSE89Zczn+txzHJMjmhdPK3fd\nMmjiFFU6Qpzi5mvZRo0yovqdf7540yQ72n34n/8MPnfqqbHZxN99l82HZg8y/TEqVSr6cSVJSuqV\nbP1nxtLsMyt3r2oWZvREHWYefZTdIM2+uubmH1D0Kb+8YuFC6772KtHTlNkxT0Fo/mAdO8amoYUL\nI6+9793LZqNBgyIW1xWn6cyclPjKlWKe8YIPPuCKhL2zevt25/TRoPtiQvm6R0uvXvyumr2mUrjj\n1I3UU+76D4mmSd22LfuXxlu5O6GVipsv7caNiZEjEpo25ZfJPmq2Wzc2u9hnr3F7hmal7xaczUyZ\nMmxCKupHbsoUDr5mdgE1Y58VR6NDsQqx8be/sdnLPkHM4MFFz1v34Xip3J2It5u0H0QSoyAeS5Fi\ny1SrRnTrrZGlzcsz4jnUqWNs79gR+/3daNqU8z7jDOM+nTvzOac4JzquRarw+ecs92OPcSyX7duJ\nbr/dOQbHo49GFpdjzpzExX4ZMsT5fxgxIr73TWeOHXOOWeTV/6nz2bmz6Hk5kZtLlJ1N9OWX8ck/\nDsDL2DJJR40aPPXaXXexfezhh93Tml3sdIffLbdEN4dipGgXrYkTjUlztT3dHI1x3DiuCeve/1Sh\nWzd+1R55hOOk167NI2zz8tjdcIcpErR5km6leIoyO4cPW2esiSTMQFFwG7w1e3Z875vOPPBA8DFz\nDPVo0G62778f7Ebr5F7sBZUrs5NGvMueH0TyBYjHUqSae926zjWwwYOD027bFpwuXmRnc/6LFhHt\n2kU0cyZHMiQiWrKEXCNDpivVqlmf+8UXG+emTbOeq1iR6ODB+Mozc6b1nl278vree+N733TG6T08\nftz6rm3bRrRunfP1TZsStWtnvDv291SieQaBtK65m4eXm3n5ZWDVKuux55/nda1awEMPWYcye43u\nZS8s5PjPV1xhHMvK4mKaThNXh8PewaY9nfbv58m/zRw4EH+76uWXW/dnz2Zba7J0aKcLGRnGqOpX\nX+UW07nnWtN88glw221sU//+e+fwFlu3ciz1eAb7S2NSU7mH4q67rPvvv8/rG29k/+Z4dqTqzlOv\nBnCkOuaOVY1SRhgETfPmiXE/M99Dhw4uXZrHTEinavRMn25s60nMdZwWCngh3XGHkeb4cf4PKlRg\nH3IdltqN+vXZrVX7twtRkfrK3W47t48+01PKOdkGveb++3kd7fR86crYsfxMRo3ifbcP65QpiZNJ\no907S5cG3nwTyMwMPQ2iEIw5FO769cCYMYZLrH2sB2BM4n7kiLO321138ftrjqX+22/yv8RI6rdH\nu3fnl1NjH/L+9de8ttcW40HPnuI3baZUKWOw2SefWAdJaapXT2ws+VWruPaowz+YfZ0PHIg+7EFx\nxdwKGjmSa9fmKesWLAh+lk4DB0uXDq6QNW7Mi54e0h6iWoiI1Ky5f/ghB9OaPZtrC2Y2bTIU7L59\n0QegEuKD3eySmcl21j/+SKyPcbNm1uBm5jAWRe2P+fpr55j96Y55EnhNpUrOs4zZmTvX+bh57mGn\nVoAQltSsuffqFfp8bi67Tv32W2LkEcJjfomPH0/OQSNFiTXzww9cM33oIe7bSWfMH7Dx491DOOjg\ncnYGDmR34Dff5GnvnKhcWVrBRSQ1a+7h0L7ts2bxOtRMK0JiyM42tpNRsQORjaZ1Q3ck2qNbpiN6\n1OjIkUaflhsjRvAI8ZEjeX/NGv74DRrEI1uFuKHIp69jVlYWLfXKZPLEE+yh8uijvN+4MXfKaFNA\nss0RWhzZvZvdQ6tUiS2SX7wwm4uaNnWODxRNPg8/zOUxndG/deJEoHfvyK/LzXWvzQsRo5RaRkRZ\n4dKlR819xAgeEalnafn5Zx4xqYnHaFQhOqpX585VPQVgMtKtG08puGBB9CaB7t15naiYRX5hHvHt\nNOo4FKLYE0p6KHeNeeagTp2MbZkzMzm4/35uVSUr//0vh4u4+GKrD3ck6NALqRojvrCQBxwdPcoD\nkJRynuszK1BhbNky9MTVgu+kl3J3igG+aVPi5RBSB3uo1xEjeB3JxCVm9Hy5iVDun3ziHLO+KLz/\nPg84Kl/ecC3+v//jSSsAdkdUyphWcfx4b+8veE56KXcn4j2kXUhtNmywThVo9gSZOdPYvvded7PC\njh0cUgFwn8IN8GZ6v9WreRYiL2vNRMYAPDsLF7JDQqtW1uPnnefd/YW4kH7K3T7JdI0a/sghpAbV\nqrlPQzhvHq/z8zlGkX2kZEEBz7Klh96br7FTWMgug273ipQtW6z7hw/zqNCCgtjzHDHCmDZRY/Yz\nv/ba4HumyWxF6Uz6KXdz+FbxkxWKwmWX8bplS+fzP/7IE4ybMQ+dN6Mnitm8OfZO5Z07jVDSACvZ\nRx8Fhgzh0cDR5jt/PrsoPvWU9XhhodV11QyRxHpJEVJzEFMozLUoQSgKujas/boBVm661hrNNHI6\nVjnAsf0LC6Ov/drjzttNjhde6F6hKSwM78mzdi0rbi3XggWGzR2Q/qsUI/1q7hIbRCgKVasaAa6c\nwhGUKGGMpzB3uponl5g8Odj2/scf1n09IXgohg/nAUBK8aKjbLrNaZuVZXyQhg61ThATakIbgEfX\nnnuudTzIRRdZQ1T/5S/hZRaSh0iCvsdjKdJkHeHo2FEmYBCi46efiMaO5e3ly7nrc9o03neakMJ8\nfOlS3u/Vy5pGl8EVK5zz+PFHPq8ndLHjdA3AU9vdfLOx/8wzxvaZZxL961/G/uefc16dO7vn17+/\n+3M5cYLoqafiP5GKEDGIcLKO9FTuglAUVq3iVyMjg+jwYWeF2KCBVdET8Sxb9nQffcSzTDnloZQx\nh+zw4VYZDh1yvuaWW/i8ebajqVPdFTdAtGcPr1u04A/OBx8k7lkKnhOpck8/s4wgFBXdb3PihDHS\n+cEHrWk2buT1G28Yx4YODc7rhx+AmjV5u317YMkSY7AdkTGHrLlTs6CAh/abef11jrKo75eRYbhD\nmqNcOlGtGq8HDOBBWtddFzq9kBaIchcEO1WqAGefzdtXXMFrN+8R8yTnSgWHv33mGf4Q3HYbBxXL\nyuIImfZR03rybiL2fNGx0ceO5XWnTsF+9vPm8RwCOja9GS23mb59nX+DkJaIchcEJ8wTwAA8aGfs\nWKufev/+wR4ozz3nHN+8Wzfr/gsvGNstWrByHzkSePdda7qBA7km37BhcJ7Z2TzAqHRpYO9eYNs2\n7ridOpVr+OvX88elVi3+oNhH4wppTXpEhRQEr9m+3ahNA4aLodmlcPp0q9+5nbfeAm65xXq95sQJ\noEwZVugLFwIff2w936kTm2KclLpQrIk0KmT6+bkLgheYI4ma46iUKMHmlddf5zAAoejfn2vLl14a\nfC4jw/B1dxr6P2dO+keYFOJKRGYZpVR3pdR6pdQmpdRDDufLKKUmB84vVkrV81pQQUgoZsWqO0Q1\n7dvz5NDquZZzAAAgAElEQVThBiEpBVxzjfuITn39/Pm8HjeOfevz8kSxC0UmbM1dKVUSwCsAugLI\nAbBEKTWDiNaakg0AsJ+IzlFK9QbwDACZZkVID+I9H0DHjhIqQ/CcSGru2QA2EdEWIjoBYBIAu6Hx\nagDvBLanAOiilEQWEtIEp1DSgpDkRKLczwRgDqKREzjmmIaI8gHkAqjqhYCCIAhC9CS0Q1UpNQjA\nIACoY/ZEEIRkZMsWYNcuv6UQhJiIpOa+A8BZpv3agWOOaZRSpQCcAmCvPSMiGkNEWUSUVb169dgk\nFoREUb8+B+4ShBQkEuW+BEADpVR9pVQGgN4AZtjSzACgh+r1AjCP/HKgFwRBEMKbZYgoXyk1GMAX\nAEoCeJOI1iilHgcHsJkB4A0AE5RSmwDsA38ABEEQBJ/wbYSqUmo3gF/DJKsGYE8CxIkEkcWZZJIF\nSC55RBZnRBZ3IpGnLhGFtWv7ptwjQSm1NJJhtolAZHEmmWQBkksekcUZkcUdL+WRwGGCIAhpiCh3\nQRCENCTZlfsYvwUwIbI4k0yyAMklj8jijMjijmfyJLXNXRAEQYiNZK+5C4IgCDEgyl0QBCEdiWQW\nba8WcIiC+QDWAlgD4M7A8dMAzAGwMbCuEjiuALwIYBOAVQBamfLqF0i/EUA/v2QBkAng+0AeqwD8\nzc/nEjhfGRzg7WWf/6M6AGYDWBfIr57P8jwbyGNdII2KsyyNA2XjOID7bHl1B7A+IOdDCXgujrK4\n5ePXcwmcLwngRwAz/ZQFwKngKLc/B8pMO5/luTuQx2oAEwGUDXnvaIUtygKgJgylWAnABgBNAi/d\nQ4HjDwF4JrB9OYDPwC9sWwCLTQ9mS2BdJbBdxSdZGgJoENiuBWAngFP9kMWU3wsA3kdsyt0zWQB8\nBaBrYLsigPI+lpkLAHwLVhwlwS9QxzjLcjqANgCeglWhlgSwGcDZADIArATQxCdZHPPxQxZTfvcE\nym8syt0zWcBhzAcGtjMQ5Xvt8f90JoBfAJQL7H8AoH/Ie0crrJcLgI/Bk4CsB1DT9DDWB7ZHA+hj\nSr8+cL4PgNGm45Z0iZTFIZ+VCCh7P2QB0Bocc78/YlDuHv5HTQAsTKIy0w7AMgDlAJQHsBTAufGU\nxZTuUduL2g7AF6b9YQCG+SGLWz5+yQIOTPglgM6IQbl7+B+dAlamUbXu4iiPDqt+GjhszEwAl4a6\nl28298BUfC0BLAZQg4h2Bk79DqBGYNstlnwkMeYTJYs5n2zwF36zH7IopUoA+C+A+2K9v1eygFs0\nfyqlpiqlflRKjQzM6uWLPET0Pbh5vDOwfEFE6+Isixt+lN9o8/FLllEAHgBQGKsMHslSH8BuAG8F\nyu84pVQFv+Qhoh0AngOwDVx+c4lodqhrfFHuSqmKAD4CcBcRHTCfI/5MUarJopSqCWACgJuJKKaC\n6YEstwOYRUQ5sdzfY1lKAWgP/tC0AZsg+vslj1LqHADngmuGZwLorJRq74csXuJh+XXNJ1GyKKV6\nAPiDiJbFcn8vZQGX31YAXiOilgAOg80nvsijlKoCnvGuPtj8W0Ep1TfUNQlX7kqp0uAf+R4RTQ0c\n3hVQjlpJ/hE47hZLPpIY84mSBUqpygA+BTCciBZFK4eHsrQDMFgptRX8lb9JKfUfn2TJAbCCeHrG\nfADTwS9L1HgkT08Ai4joEBEdAtvl28VZFjf8KL/R5uOHLBcCuCpQfieBP8Dv+iRLDoAcItKtmClI\nTPl14xIAvxDRbiLKAzAV3I/kSkKVe2Be1TcArCOi502nzPHg+4HtUvr4TYppC26K7ASHH75UKVUl\n8EW7NHAs4bIEYtxPAzCeiKZEI4PXshDRDURUh4jqgWvM44koqtqGh//REgCnKqV09LrOYI+BqPBQ\nnm0AOiilSgVetg5gD4h4yuJGJHMkJESWEPkkXBYiGkZEtQPltzd4XoiQtdM4yvI7gO1KqUaBQ12Q\nmPLrxjYAbZVS5QN5dkG48utlZ0G4BcBF4ObHKgArAsvl4PlWvwS7Bc0FcFogvQLwCtiG/ROALFNe\nt4DdyDaBTSG+yAKgL4A8Ux4rAGT69VxMefZHbN4yXv5HXQP5/ATgbQAZPv5PJcGdrdot8/kEyHIG\nuAZ4AMCfge3KgXOXgz0nNoNbfL7I4paPX8/FlGdHxOYt4+V/lAnueF8FbnlG5ZEXB3keA7tlrgab\ngMuEureEHxAEQUhDZISqIAhCGiLKXRAEIQ0R5S4IgpCGhJ0gO15Uq1aN6tWr59ftBUEQUpJly5bt\noQjmUA2r3JVSbwLQgwuaOpxX4FgmlwM4Ao53sDxcvvXq1cPSpUvDJRMEQRBMKKV+jSRdJGaZt8ER\n7Ny4DECDwDIIwGuR3FgQBEGIH2GVOxF9A2BfiCRXgwfLEPHozFP1yCtBSGUOHgR+jaiOJAjJhxcd\nqhEHQVJKDVJKLVVKLd29e7cHtxaE+NGuHSDdQkKqktAOVSIag8AEsFlZWTJ6Skhq1qxJ7P3y8vKQ\nk5ODY8eOJfbGQlJStmxZ1K5dG6VLl47pei+UuydBkAShuJOTk4NKlSqhXr16YD8FobhCRNi7dy9y\ncnJQv379mPLwwizjFqhJEIQoOHbsGKpWrSqKXYBSClWrVi1SKy4SV8iJ4CA+1ZRSOQD+BaA0ABDR\n6wBmgd0gN4FdIW+OWRpBKOaIYhc0RS0LkXjL9CGimkRUmjgc5xtE9HpAsSPgJXMHEf2FiJoRkTiv\nC0KKUrJkSWRmZqJp06a47rrrcOTIEV/kGDVqlG/37tixY1qMwZHwA4IgnKRcuXJYsWIFVq9ejYyM\nDLz++usRX1tQUOCZHH4q93RBlLsgCI60b98emzZtAgC8++67yM7ORmZmJm699daTirxixYq49957\n0aJFC3z//fdYsmQJLrjgArRo0QLZ2dk4ePAgCgoKcP/996NNmzZo3rw5Ro8eDQD46quv0LFjR/Tq\n1QuNGzfGDTfcACLCiy++iN9++w2dOnVCp06dAAC33XYbsrKycN555+Ff//rXSRlnzZqFxo0bo3Xr\n1hg6dCh69OgBADh8+DBuueUWZGdno2XLlvj4Y+e5MJ555hk0a9YMLVq0wEMPGfPafPjhh8jOzkbD\nhg2xYMECAMDWrVvRvn17tGrVCq1atcJ3330X8nd4IV+RiDb4vFdL69atSRCSGYCXRLF27dqT23fe\nSdShg7fLnXeGl6FChQpERJSXl0dXXXUVvfrqq7R27Vrq0aMHnThxgoiIbrvtNnrnnXeIiAgATZ48\nmYiIjh8/TvXr16cffviBiIhyc3MpLy+PRo8eTU888QQRER07doxat25NW7Zsofnz51PlypVp+/bt\nVFBQQG3btqUFCxYQEVHdunVp9+7dJ+Xau3cvERHl5+dThw4daOXKlXT06FGqXbs2bdmyhYiIevfu\nTVdccQUREQ0bNowmTJhARET79++nBg0a0KFDhyy/ddasWdSuXTs6fPiw5R4dOnSge+65h4iIPv30\nU+rSpQsRER0+fJiOHj1KREQbNmwgrcPcfkdR5SOylgkNgKUUgY71LXCYIAjJx9GjR5GZmQmAa+4D\nBgzAmDFjsGzZMrRp0+ZkmtNPPx0A2+ivvfZaAMD69etRs2bNk+kqV64MAJg9ezZWrVqFKVN4Fsrc\n3Fxs3LgRGRkZyM7ORu3atQEAmZmZ2Lp1Ky666KIguT744AOMGTMG+fn52LlzJ9auXYvCwkKcffbZ\nJ10F+/TpgzFjxpy854wZM/Dcc88BYE+kbdu24dxzzz2Z59y5c3HzzTejfPnyAIDTTjvt5Lm//vWv\nAIDWrVtj69atAHgcwuDBg7FixQqULFkSGzZsOJne6XdUrFixSPIVFVHugpCEjBrlz321zd0MEaFf\nv354+umng9KXLVsWJUuWDJknEeGll15Ct27dLMe/+uorlClT5uR+yZIlkZ+fH3T9L7/8gueeew5L\nlixBlSpV0L9//7AugkSEjz76CI0aNQqZzg0tl1mm//3vf6hRowZWrlyJwsJClC1bNih9qN/hpXyR\nIDZ3QRBC0qVLF0yZMgV//PEHAGDfvn341SHoTqNGjbBz504sWbIEAHDw4EHk5+ejW7dueO2115CX\nlwcA2LBhAw4fPhzynpUqVcLBgwcBAAcOHECFChVwyimnYNeuXfjss89O3m/Lli0na9aTJ08+eX23\nbt3w0ksvnbR9//jjj0H36Nq1K956662THbf79oUKocUtjpo1a6JEiRKYMGFC2A7kospXVKTmLghC\nSJo0aYInn3wSl156KQoLC1G6dGm88sorqFu3riVdRkYGJk+ejCFDhuDo0aMoV64c5s6di4EDB2Lr\n1q1o1aoViAjVq1fH9OnTQ95z0KBB6N69O2rVqoX58+ejZcuWaNy4Mc466yxceOGFALiV8eqrr6J7\n9+6oUKHCSXMQAIwYMQJ33XUXmjdvjsLCQtSvXx8zZ8603KN79+5YsWIFsrKykJGRgcsvvxz//ve/\nXWW6/fbbce2112L8+PEn7xmKospXVHybIDsrK4vSwZdUSF/0GJJEvSLr1q3z1OZaHDh06BAqVqwI\nIsIdd9yBBg0a4O677/ZbrJMUVT6nMqGUWkZEWeGuFbOMIAgpy9ixY5GZmYnzzjsPubm5uPXWW/0W\nyYKf8knNXRBckJq74DdScxcEQRAsiHIXhCTCr5a0kHwUtSyIcheEJKFs2bLYu3evKHjhZDx3sy99\ntIgrpCAkCbVr10ZOTg5kCkoBMGZiihVR7oKQJJQuXTrmWXcEwY6YZQRBENIQUe6CIAhpiCh3QRCE\nNCRtlPuqVTzoxCGekSCEZfZs4LTTgEOH/JZEELwhbZT72LG8jseEJkL6M3w4sH8/sG6d35IIgjek\njXIvEfgl4iIsFAWn8mMLby4IKUHaKHcdB6Sw0F85hNRElx8nXnopcXIIgleknXIXhKIgLT8hXUg7\n5S4vpxCOjz8GQs0Vcfw4EGaWNEFIetJOuYtZRgjHNdcAPXu6ny9bFrjkEuuxG26Q1qEf7NwJBGb3\nE6IkbcIPSM1dKAp2xf3118b2tm3A3LmJlUdgatXitbzX0ZN2NfdwhSA3Fxg4UPyZBWecyo8o9uSE\nCDh61G8pkpdipdznzQNOPRV44w2gQYPEyCUkN3v3Aj/9FLnJ5YUXgDCT3gsJYuRIoHx5QIJoOpM2\nyt3s5378ONfQ7Tz1lLH9+++JkUtIblq3Bpo3N/bDtfzuugsYPz6+MsXCY4/xB6o4dQRPmMDrnTv9\nlSNZSRvlbu5QveQSrqELQjjs4Soefzz8NUeOAEuWABs2xEemWHj6aV7n5fkrRyLRH+ISaaPFvCVt\nHou5Wb1woTd57t4dfU0oNxfo1w84cMAbGYTEsHgxrz//PHxapYDsbKBRo/jKFA1a0e3b568ciUR7\nxhVFuRMBF14ITJ3qjUzhKCgAFi1KzL3STrlH26t+zjlA797Bx48cAU4/HRg8OLr8nnuOm+2jRkV3\nXbqQn89msVRg+fLYrjNXJLRpQEPkj5lAl/siTNyTcmjlXhQX1cJC4LvvgGuv9UamcDz5JNCuHfD9\n9/G/V9opd7OfeyT+sZs3A5MnG/vr1wMDBrByB4BJk3j78GHvZE1nOnRgP/FwnDgBvPeevy5urVvH\ndt2qVcb2TTdZz735JrvvLVvmfG3t2sBZZ8V231AUR1dBe81982bg/PM5AFykJHrswurVvM7Jif+9\n0ka5OwUOq1ED2Lo1unx69+YX9LPPeD8/H6hWDahYMfy1RM522L//HXj77dDX5uWxi2a08iYb330X\nWbrHHgP69k18FM/584uex+uvW/d/+ol94QHgq6+MY3aF++WXwI4dxoudlxfZoLuZM43KxvHjwBVX\nAGvWWNMURbkfPAh8803s1/uFXbk/+STwww+hRx/bSfRHMZGDLdNGubuZZbZsAb74InIbuC4oukZ2\n+HDkvrSjRwMffGDIcfAgsGkTMHEicPPNoa/95ht20RwwILJ7RUMyegZpBffnn/HJf9IkQ9FqNm0C\nOnf2/l7NmwN16/K2Lj8338wfgRkzgEceYVurfdRrRkZwzd/OypXAlVeyefDwYXb/mzULuP12Pv/r\nr6woiqKkbriBW1ypNhJU/+Y77+QPZSq0XhIZvTYllfuffxodYAC/OHqgif2hrV8PdO8O9OkTOs/c\nXG4ux2qHBYJDw1auDGRmxp6fJj8fOPdc51puuA/P558DNWsCn35aNBlWrAD++lfvXO30/2RvFq9d\n602I3T59gE6drMec3GO9pmRJY/vdd4GrrwaeeCL4vztxgtfvvRc6P/3x27wZuOUWYMQI49z69UC9\nesB//mMt9+YP54gR4f3ytZkp1UyPuvb72WdGSztaEv1B0Mo9aWruSqnuSqn1SqlNSqmHHM73V0rt\nVkqtCCwDvRfV4LLLgLZtgawsrpWXKmWYA2bMsKbVNZxZs0LnuWhR0e1g5csb27qlEOkLYy5kRNwU\n14p0zx7g55+BQYOs1yxdyvf85BP3fPVH0PwxjIW+fYFp01iOaFm+PPg6N+V+3nlAy5axyeg3y5ZZ\nlbv5t9k77DZtcs/n8GFnhbxypXVfu3HOn28tPxs3sqvmkCFsqpgyhcuKG6UCQUhCfbinTeNy1rMn\n0KKFc5qdO4H69a2/7d//5udQ1BbamWcankAnTgDHjlkVpBfKMjcXePHF+Cr8pDLLKKVKAngFwGUA\nmgDoo5Rq4pB0MhFlBpZxHstp4YcfeL1sGdfKzfz4o/t1dhckt04vN2bO5D/HrKgKC/llAoDSpY3j\nzz8ffP2HH3Jty8zPP3Oe06bx/rx5bFK48krDd9kNrbDvu8+9QHoVc0crm1jczlq35paHGS3Prl28\neIlf/udZWVblHgqz4s/MtNqJK1Z0NuOZr3HbBrgmn53NLQeAlXybNuE7D+3K/fhxNjUuX86ttquu\nYjnNHcpm7r6b+4waNDBawMOH87pKFb6OiM2ESkXXmvztN6O/pGlToFw5q4KMpnwXFDhXUoYMYRPP\nl1+GzyNWr7BkM8tkA9hERFuI6ASASQCujq9YoYn1wehOKU1WlrEdyfRqWmGbJ2945hmgYUPuQAtn\ns7z+eqBxY2vn1csv8/rVV41jf/87r3/5hdf2l1J772hFsmGDe4etk3IPZcpZtw549tng4174FDvl\n98ADwBlnRHZNfj5/IEP9/59+Gt6OHU/MzyeUMjWnW7mSa8Qvv2y0+CZM4I79jz7ifaLg/Nyew+zZ\n1v1ww/M3b+b13r18T53vLbcA//iHu1fRmjVs+tLvldnrzElxt2jBv/vqgPYYMoQrRr/9Flo+jf79\nujJlb+2GY/p0fp4jRnBlY+NG63W6Yrh/P/Dgg6H7qrp1c/cKmzaNWxZOJNIsAyIKuQDoBWCcaf9G\nAC/b0vQHsBPAKgBTAJzlktcgAEsBLK1Tpw7FilJE/Lf4t2zYQPTJJ0Rdu/J+jx6RX3vddcZvCZXu\n5pv5Pr/+ahwjIqpYkbdHjzaOjxhBNH8+b2/ezOnOPpuobFk+Nnw4H9u4kfffeMP52VapwuePHbMe\nP+ccPr5+Pe/n5REdPBh8vVlOp2Pvv0/07bfBv9WeNjeXqLCQ6LPPeE1E9NRTfO6DD5xl37vXPd91\n64jGj49/ubjjjsjS6d9iX/r0cT7evj1RkybGfocO/GwAouzsyOVzwp7mzjutZcEtn0su4e3Zs4Pz\nefxx57yjkct+/ZQp1mM1axrbH31E1K8fb7/1ljWPLVuIHnnESKvL8sUXE504EVqu7dtDy5WbS3Tf\nfUTHj/NxXbZvv935Oi3jm2+6/+ZwAFhKFFpvE5FnIX8/ATCRiI4rpW4F8A6AIL8EIhoDYAwAZGVl\nUaw3o5iv9I6GDa378+ZFn0e4r3d+Pt+nfn3rcR3R0mzbNHfaTZ8ONGvGnkIaIm4Wa3vptGlcM7Oj\n87bLpvd1a6FnTzZTRfNf7NhhtErC0aEDMHQoy/jSS3z/7dv5nFsLyckNk4hrZLH6tEfLK69Elk6b\nK+y49Z8sWGDdV8pogWkzpVe88AJ7AIVqpb36qlEmpk8P9i239w94Qa9eVhOeueyFKofXXGM1Jen3\n5ptvwnuSTZgADBvmfn7QIG6xNGjA7r16AJubS7PZmy7eRNLI3gHAPOyiduDYSYhoLxFpC9Q4AAl6\nlVKXY8eMjiw3tEeFNs8A1hd55Ehrel2A770XuPRS6zkiq+lGF64jR5xth/YOPR2zRPc7zJxpnFu9\nmhWB+QPXrx9/XMymhFCjJ/fssZoPVqwwfo+2heqPV2Eh8P77nLc5NoyT7f7ZZxOn2L0g0lDUhYWx\nuc3ed5+xXVBglDE7AwaEVu533GH836++Cvztb9bzH30UnQI7epRlC/f7a9Qwts2K2XyvgQO5r6BD\nBzY5hbKN16kT+n6//cbmVD2OwY42Ra1axWm1HG4mOf1BThazTCkAWwDUB5ABYCWA82xpapq2ewJY\nFC7f1q1bx9Qkyc+PvKmXyKV8+cjT9upFdMMN4dOdfrp38j30ENGttxr7l1/OZhWAqG5dfrbbtxOV\nKsXHdu3iY7/+ymakUM3pu+7i7U6dEv/cdXO7Y0f/y0CqLAUF4c0RiV6eeYbXjz1mN0FEtkycSNSy\nZfzk69IlOrmuuIJo8WI202zezO/7sWPG+TFjYlJ/gXt7ZJYhonyl1GAAXwAoCeBNIlqjlHo8cJMZ\nAIYqpa4CkA9gH9gGHxe+/TZeORcNe2dtOML5NwPeDio5csQq46xZHFcH4Nrvk09afahr1OCOuXvu\nMYZMh0ObTRKJjuJoH7AkuFOyJHDddX5LYUV7ss2dyx3itWuHb9maCTeOJdEcP86hEAAevDZ3rtUd\nOSlq7vFaYq25Dxzofy0jVZcLLvA+z+3b/f9dsqTf0r8/0Suv+C+HeXnxRaJataK/rn794GOvvhqT\n+iMiIiS4QzVh7NgRPo3gTKRxX6IhHkGwBOHtt8PHY0o0Q4fGdp25z0yTFIOYkg23DiBBEIRUIRFT\nNYpyFwRBSDBmb7N4IcpdEAQhwUg8dwdSZZYfQRAEN1q1iv89Uk65SweeIAiJplIlb/N78EFv83Mi\n5ZR706Z+SyDYee89oEsXv6UQhPgwaVJqTnifcsq9KErEHNO9KM2iRPR02znzzPjlPWSIexhXN8zh\nk//+dx6kMTBOUfztE57MnRt9uGbBW957L7LQuEXh4MHYr3UKuR0t1arxWs/edd55vL777qLnTVT0\nPCK4SWoNYtq/P/ZBCOYh17EOVR40iOX45z9jl6Nu3eBj554b+pozz4ztXtWqWfdHjgxOM2qUHhxh\nXX77zYh217atcVyzYgUP+zbz7bdErVpFJ2Pv3sHH2rcnysmx3ktHGtS45ff227H/N8m22MNazJwZ\nWz4rV0aetnHj8GkKCvg/GDAgOjlq1w59vk0bogULjMif4fJr04Zo377g44WFRX/2O3cSvfOOsx5y\nu2bkSPfz48cTNW/O2ytWRKX2bPeObBBT2ATxWmJV7n/+aX1gEyZE/mcVFBjbsSp3HX6WiGjs2Njy\n+Oc/OSQoQPTSS5znoUPB6dq3N7br1CHq1o23K1e2pqtUyf1e779v3dcsWMBhcOfNM17UJ57gl/XK\nK61hiYk4RDBA1LdvZP9TuGegZe7Vy/mauXNjv4cOaxxqGTCAqEKF2P6/aJdx4yJPm5dnlKtXXzWU\ngfn/i0WGjRuJli3jj/3LL1vPffop0erVxn5+PtHu3dY0bdrw+owzrOVI88sv4WXYv5/TXn998LkL\nL+T1DTdEV45OnHBO53Sse3eiqVND51eyZPC7EknZ05WoZ5/l87t28f5553Fo5Btv5OMtWvDx5cvD\nl2/3e6epcs/NtT7Ud9+NvICbv+YtW/IDDpVe/xH2QqPJy4v83ublgQeIBg/m7RdeMP9p1uXii3ld\npQrR2rVER4/ymojo55+Jhg7l8zq++913c9x2ff3Ro9Z8J0+O6ZGf5NAh68ctFN9/z/dcvNj6mzp3\n5rVWMG7K/euvw9/D7fkScVC0vn1Dp3FqQcWyVKjAHyO382++GVk+27YF/8ZmzYLltl8XLmBb27bB\n+S5fbrSYDh+25qvZscM4ds01vJ40yagMuP0fbgHvDhzgdPYKWqNGRLNm8fawYc55Tprk/D5rzC1f\nHQgPsLZ4n3qKaPr00M9qzhyirVuJVq1yK3VWuUaPJpo2jWO6AxwATbNvH9GRI9brdKVy2bLQ+Ye+\nd2TKPeVs7vYwpKFmu/nwQ+DGGzkW9vz51rQlSoSfqzPcEOFSpWKbADs/P/SMLF9/zdOaNWvG+y+9\nxDPHlC1rTFfXqBFQoQJv9+zJ6379OACYRs8U8+67HADs+uujl9VMhQrhp2rTtG3LxT8723r8yy/5\n+HXX8fO75x7n6yO5j46N3bw5/3YzFStyXPJ+/dxDVkT6W0Jxzz0chln/V06ECp1bpYqx7eQJ9s9/\n8rp/f54lyYnx40PLaJ+/FeCy/9ZbHNtcz/27dClPUK6pVYvDJV9wAZdHIHSc9z17eNFT7NWowWF3\nNXougFNOMSa0njKFp7zr3p23H33UmudXXwFTp3J5ufJK6znz/7dqFU8DCBhl4ddfrUHvLrssWGZ7\n2GwioG7d0P+nmUGDOF6804xnVarwdIBOMkvgMAfs5ov33nP/Cjt/9Xi58ELrvtNirzU55RmLeeft\nt/lrDxB9802wbJoDBzgEal6e828ZNsyokTj9xmQh3H/ilG7jxvD57t5NVLo0P0Pd3O7YMXzeWo56\n9aL/7+yLNgvYzRjmxW4a0zXUoUOtYWBjeZ6As0nPvDz3XOR5h+LQocjS5eTwfWvWtMqbn29Nt2VL\n9DL8/jvn1bRp8LmHH+Zz9r6ZcuWM56Rr7lddxefMs5wBxqxS4bD/Z5Mn8/6sWaGva92a0/3wQ2T3\ncb53mtbc7bWtWGtf+os6ZYpxzD4hdfnyxhyWgHVbE+n8n5p58zik6TXXcC2nfXvjXJUqXBPQVKoE\nPKLRKtUAAAvaSURBVPJI+NCnRKHPpxqbNhnhiENRrRqPWG7f3ngGp57qnHbkSJ5QYto0Y5KJcC23\nUJQpw2tdiw1VDhs04PlJzfz739yy0PlEw9ix7MExfz638rxogUSCbimGo2JFXtsnjLFPHm6fYSwS\natTgWr+Tp063bs73dUKXl1if3c6d1onYr7+ey61T68BMIifITrmokE7KvUyZ6EeuauV+7bVc6AoK\nnB+4bo4C1ia0ZsIE4J13ePajSOjUydiuWtV6bt++yPLQVK/O69NOi+66ZOXqq9ks8Je/RH9tgwa8\ndnOVNc9ApJkwwVBEoVAquGy88AKb/LTCspfLXr2MioN5InYvGDjQ6nbqNBnzrFnA5ZfzdqInDD/l\nFFZ08Rpw2L278/GLLgqvNJ3MJ5oSJSKfteuMM4IrdpGU20SaZVK+5g5w4W7TJrp8Hngg+Jj+wy+6\nyPkaJ+VetSrbXc3zlSaKIUOAceOstf1kJNLp4KZPt9aGoqFZM7at33FH5Ne41UT/7/+s+6ecEpym\nsNCwVQPWcjl1Kvf39OtnbR18801w7dULnN6Jyy4DPv+cy4euBCSSv/wFyMhI/H3D4dbyr1WLK3jx\nriiF+rh4Tcord7dmVZMm7nnUrWtV4PavqVuHUaNG7nnWr88dUmPGWI8fPMhzKwLez+VZqhQrzngo\nDC8ZNy4x96lVyxsTxdVXW/edaor2gWzmMnP66bx++22jcxFg89G+fd4HjXL7zd26xTbParphfj76\nHdatGt16jrTlXVTELBMCu41S/3F2BbdkifP1q1axEnDCrtz1HxCp2ad1a/ZmMdekK1bkZe9ea01P\nSE5atAhWlk4fe7tyN18T6sWtXJkXLzHXkEeM4BHDyUSdOu4TTCeahg2B/fuN1li5conts7r/fjb1\nNm4c/3ulnHK3o18qXVvSuClSJxenefPYLUx/INxq7pHUCs1pzHOSpotdPN0hCq24NfYKgjlNnTre\nyxUpLVokRnFEw+rVwNGjfkth4Nbpngh69kzcxyTlzDIAewjYO2uK0hxv3559iIvag26/1slGLyQn\nffvyurAwuLPL/LG/5Rbgiy+4w9RMsij3ZKRSpeDKlxB/UlK5X3yxMThGv1Re2FrdbO7RfGnLlOGv\n8/XXc4enHwwaBHz6qT/3TlV0CNbCwtA19xIl2NUuEtONH+jgVoLBG28AZ58dPKAo3UlZs0zTpux3\n7mY/j4VwHaqRoBR7S/jJ6NH+3j8VMXsx2JV73bqRX+83yWaSSQZ69+aluJEk9Y3oGTECWLiQh0Z7\nhVbu2vaeboOD/OSSS4COHf2Wwh1zJ7rdLGPuO3FT4smi3AVBk7I195IlgQsvNPbjaZbx8h7FlTlz\n/JYgNGZ3WHvNvXRpY/u220JfLwjJQsoq93iga+qJ9EUV/GXpUmDz5tBmGYA9LP780z2gVLLY3AVB\nkzZFUrs+PvigES0wWtxq7jpyYbgYL0Lq0bo1d36HMssAPPPUqFHuZUBq7kKykTbq6sUXeZToo4/G\nPmLTTbk//XRwUDEhvQhllgF4ZGOoEcq6zNhdJBPF889HH8ROSG/SRrmfdhrwxBNFy8NulhGKD2az\nzIkT0V9fogTHttHzbkZLu3axXafxYl5PIb0QNWZC17ruvJPXiYo3IfiP2Sxz1VWxKelatWILlnXk\nCA/MEwQvSZuauxdkZxu1d+lMLV7oWatq12YlvXt34uzoxW1wjZAYRLkLAoCaNYHJk3kSDEFIB0S5\nC0KAos4xKwjJhNjcBUEQ0hBR7oIgCGmIKHdBEIQ0RJS7IAhCGhKRcldKdVdKrVdKbVJKPeRwvoxS\nanLg/GKlVD2vBRUEQRAiJ6xyV0qVBPAKgMsANAHQRylln356AID9RHQOgP8BeMZrQQVBEITIiaTm\nng1gExFtIaITACYBsM0Pj6sBvBPYngKgi1ISSkkQBMEvIlHuZwLYbtrPCRxzTENE+QByAVS1Z6SU\nGqSUWqqUWrp79+7YJBaEBNGrV+xB6ATBbxLaoUpEY4goi4iyqlevnshbC0LUfPghkJ/vtxSCEBuR\nKPcdAM4y7dcOHHNMo5QqBeAUAHu9EFAQBEGInkiU+xIADZRS9ZVSGQB6A5hhSzMDQL/Adi8A84gk\n9JYgCIJfhI0tQ0T5SqnBAL4AUBLAm0S0Rin1OIClRDQDwBsAJiilNgHYB/4ACIIgCD6h/KpgK6V2\nA/g1TLJqAPYkQJxIEFmcSSZZgOSSR2RxRmRxJxJ56hJR2E5L35R7JCillhJRlt9yACKLG8kkC5Bc\n8ogszogs7ngpj4QfEARBSENEuQuCIKQhya7cx/gtgAmRxZlkkgVILnlEFmdEFnc8kyepbe6CIAhC\nbCR7zV0QBEGIBSJK2AIexTofwFoAawDcGTh+GoA5ADYG1lUCxxWAFwFsArAKQCtTXv0C6TcC6OeX\nLAAyAXwfyGMVgL/5+VwC5yuDYwC97PN/VAfAbADrAvnV81meZwN5rAukUXGWpXGgbBwHcJ8tr+4A\n1gfkfCgBz8VRFrd8/HougfMlAfwIYKafsgA4FRwI8edAmWnnszx3B/JYDWAigLIh7x2tsEVZANSE\noRQrAdgADiP8rC7gAB4C8Exg+3IAn4Ff2LYAFpsezJbAukpgu4pPsjQE0CCwXQvATgCn+iGLKb8X\nALyP2JS7Z7IA+ApA18B2RQDlfSwzFwD4Fqw4SoJfoI5xluV0AG0APAWrQi0JYDOAswFkAFgJoIlP\nsjjm44cspvzuCZTfWJS7Z7KAI90ODGxnIMr32uP/6UwAvwAoF9j/AED/kPeOVlgvFwAfA+gKrsHU\nND2M9YHt0QD6mNKvD5zvA2C06bglXSJlcchnJQLK3g9ZALQGh2XujxiUu4f/URMAC5OozLQDsAxA\nOQDlASwFcG48ZTGle9T2orYD8IVpfxiAYX7I4paPX7KAY1d9CaAzYlDuHv5Hp4CVaVStuzjKoyPv\nngaOLDATwKWh7uWbzT0wW1NLAIsB1CCinYFTvwOoEdh2CzccSRjiRMlizicb/IXf7IcsSqkSAP4L\n4L5Y7++VLOAWzZ9KqalKqR+VUiMDE7/4Ig8RfQ9uHu8MLF8Q0bo4y+KGH+U32nz8kmUUgAcAFMYq\ng0ey1AewG8BbgfI7TilVwS95iGgHgOcAbAOX31wimh3qGl+Uu1KqIoCPANxFRAfM54g/U5Rqsiil\nagKYAOBmIoqpYHogy+0AZhFRTiz391iWUgDagz80bcAmiP5+yaOUOgfAueCa4ZkAOiul2vshi5d4\nWH5d80mULEqpHgD+IKJlsdzfS1nA5bcVgNeIqCWAw2DziS/yKKWqgCdFqg82/1ZQSvUNdU3ClbtS\nqjT4R75HRFMDh3cFlKNWkn8EjruFG44kDHGiZIFSqjKATwEMJ6JF0crhoSztAAxWSm0Ff+VvUkr9\nxydZcgCsIJ7BKx/AdPDLEjUeydMTwCIiOkREh8B2+XZxlsUNP8pvtPn4IcuFAK4KlN9J4A/wuz7J\nkgMgh4h0K2YKElN+3bgEwC9EtJuI8gBMBfcjuZJQ5R6Yeu8NAOuI6HnTKXPI4H5gu5Q+fpNi2oKb\nIjvBESovVUpVCXzRLg0cS7gsgTDI0wCMJ6Ip0cjgtSxEdAMR1SGieuAa83giiqq24eF/tATAqUop\nHeCoM9hjICo8lGcbgA5KqVKBl60D2AMinrK4EUkY7YTIEiKfhMtCRMOIqHag/PYGhw4PWTuNoyy/\nA9iulGoUONQFiSm/bmwD0FYpVT6QZxeEK79edhaEWwBcBG5+rAKwIrBcDp6S70uwW9BcAKcF0ivw\n5NybAfwEIMuU1y1gN7JNYFOIL7IA6Asgz5THCgCZfj0XU579EZu3jJf/UddAPj8BeBtAho//U0lw\nZ6t2y3w+AbKcAa4BHgDwZ2C7cuDc5WDPic3gFp8vsrjl49dzMeXZEbF5y3j5H2WCO95XgVueUXnk\nxUGex8BumavBJuAyoe4tI1QFQRDSEBmhKgiCkIaIchcEQUhDRLkLgiCkIaLcBUEQ0hBR7oIgCGmI\nKHdBEIQ0RJS7IAhCGiLKXRAEIQ35f0+HKw/h16OiAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f9dd8d861d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_stock(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5. To see the correlation for each variables\n",
    "Not so ideal but more fundamental data will be included in the future"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEhCAYAAABycqfJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xm8XHV9//HXOwkSIAoGlCIJAhJUXECMiIqKIAq4gJa1\nWtFSoq241FZFyw9x+xVL9Ve3aqOyuCEURaIii+xaWcIaiCIREEIVFBShrCHv3x/ne8kw3GVuMmfO\nzJ3308d53Jlzzj2f7+DN+cx3Od+vbBMREcNrWtMFiIiIZiURREQMuSSCiIghl0QQETHkkggiIoZc\nEkFExJBLIoiIGBCSjpF0u6RrxjguSZ+TtEzS1ZK27+S6SQQREYPjOGD3cY7vAcwr2wLgS51cNIkg\nImJA2L4AuHOcU/YCvu7KRcAGkjaZ6LpJBBERU8emwC0t75eXfeOaUVtx+tg6mx2YeTVqdNGVb2ok\n7nNnz2sk7uytPttI3HVmbtjzmOvN2aLnMQGWLz2rkbj33XyC1vQak7nf3H/Ld95O1aQzYqHthWta\nhokMZSKIiOgVqfOGl3LTX5Mb/63A3Jb3c8q+caVpKCKiRmJax1sXLALeUkYP7QjcZfu3E/1SagQR\nETWaTI1g4mvpBGBnYCNJy4GPAGsB2P4ycBqwJ7AMuBd4WyfXTSKIiKhRNxOB7QMnOG7gnZO9bhJB\nRESNpDXub65dEkFERI2k/r/N9n8JIyIGWDebhurSSAklzZF0qqTrJf1a0mclPa6JskRE1KnHo4ZW\nS88jq2ow+x7wfdvzgK2BWcAne12WiIi6SdM63prSRNPQLsD9to8FsP2wpH8AbpR0I/BqYH2qx6K/\nafujAJLeDLwbeBxwMfD35XfvAT4LvBa4D9jL9m29/lAREaNJ09DongVc1rrD9p+Bm6kS0w7AXwLP\nBfaVNF/SM4H9gZfY3g54GBiZx2A94CLb2wIXAIf05FNERHQgNYLVc5btOwAkfQ/YCVgBPB+4tAzF\nWge4vZz/IPDD8voyYLfRLippAWUOjxlPnM+MWVvVVf6IiEdM0/SmizChJhLBUmCf1h2SngBsRnXD\nb5+gyYCA421/aJTrPVQeooCqpjDqZ2qdwyOTzkVEr6RpaHRnA+tKeguApOnAp6kWXLgX2E3SbEnr\nAHsDPyu/s4+kJ5ffmS3pqQ2UPSJiUgahaajnkcu39zdQtf9fD/wKuB/4cDnlEuC7wNXAd20vtr0U\nOBw4U9LVwFnAhIstREQ0b9oktmY00kdg+xbgde37S/v/ctt7j/I7JwInjrJ/Vsvrk4GTu1rYiIg1\nMAhNQ/3YWRwRMWVMyxQTk2P7OKq+goiIKSE1goiIIZfZRyMihlxqBBERQ67JyeQ6lUQQEVGj1Agi\nIoZcRg3FUHp4ZTOdYytW3tdI3KbYK3sfdGVmZ5m01AgiIoZbmoYiIoZcho9GRAy5jBqKiBhyaRqK\niBh209M0FBEx3NJHEBEx5AYgETTaeCXpnrb3b5X0hfL6HSOrmI3z+4+cHxHRl/p/XZr+rRHY/nLT\nZYiIWFNOjWD1STpS0j+V1y+QdLWkKyUdLemallOfIul0SddL+teGihsRMbrp6nxrSNM1gnUkXdny\nfjawaJTzjgUOsf1zSUe1HdsOeB7wAHCdpM+XpTAjIpqXGsGE7rO93cgGHNF+gqQNgMfb/nnZ9e22\nU862fZft+4GlwFNHCyRpgaTFkhavuGdZNz9DRMTYNImtIU0ngm54oOX1w4xRy7G90PZ82/NnzNqq\nNyWLiJimzremithY5A7Z/hNwt6QXll0HNFmeiIhJkTrfOrqcdpd0naRlkg4b5fhmks6VdEXpW91z\nomv2fSIoDga+UvoT1gPuarg8ERGd6WLTkKTpwBeBPYBtgAMlbdN22uHASbafR/XF+T8mum6jncW2\nZ7W9Pw44rrw+suXQtbafC1Ay4OL288v719ZY3IiIyevuaKAdgGW2bwCQ9B1gL6r+0REGnlBerw/8\nz0QXbXrUUKdeI+lDVOX9DfDWZosTEdGZLj9HsCnQOipyOfDCtnOOBM6U9C6qFpRXTnTRgWgasn1i\nGVn0bNuvsf37pssUEdGRSXQWt45uLNuC1Yh4IHCc7TnAnsA3NMEUqINSI4iIGEyTqBDYXggsHOeU\nW4G5Le/nlH2tDgZ2L9f7uaSZwEbA7WNddCBqBBERA6u7o4YuBeZJ2kLS46g6g9sfwr0Z2LUKrWcC\nM4FxW1FSI4iIqFMXO4ttr5B0KHAGMB04xva1kj4GLLa9CPhHqlGW/0DVcfxW2x7vukkEERF16vIU\nE7ZPA05r23dEy+ulwEsmc80kgoiIOg3AXENJBNF1a08ftxZao2biNrUmbSNxG5wGYWANQE9sEkFE\nRJ1SI4iIGHL9nweSCCIi6uTp/d82lEQQEVGn1AgiIobcAHSwJxFERNQpncUREUOu//NAEkFERK3S\nNDQ5ku5pX6wmImKQubsL09SirxJBRMSUMwA1gr4f4Cppc0nnlEWYzy4LM0+XdKMqG0h6WNLLyvkX\nSJrXdLkjIoCuL15fh75PBMDngePLmsXfAj5n+2HgOqrFm3cCLgdeKmltYK7t69sv0rryz4p7lvWw\n+BEx1CaxQlljRWwscudeBHy7vP4G1Y0f4ELgZWX7l7L/BVQLNzyG7YW259ueP2PWVvWWOCJixLRJ\nbA0ZhEQwlguAlwI7UM3NvQGwM1WCiIjoD9Ondb41ZBASwX9TLccG8CZW3egvAV4MrLR9P3Al8Haq\nBBER0Rcsdbw1pd9GDa0raXnL+88A7wKOlfR+qnU33wZg+wFJtwAXlXMvBA4ElvSwvBER4xuAr9t9\nlQhsj/WfbJcxzn9py+tvs6ovISKiPwzA8NG+SgQREVNO5hqKiBhyqRFERAy3TDERETHsUiOIiBhy\n6SOIiBhyGT7any668k09j/nwyma+Faw93T2POX/bb/U8JsAdNxzaSNwLLtm9kbhNND0/fq3e/z0B\n3PFA7//Ndk1qBBERQy59BBERw80NziHUqSSCiIg69X8eSCKIiKhV+ggiIoZc+ggiIoZcEkFExHAb\nhCkmBqAbIyJigHV58XpJu0u6TtIySYeNcc5+kpZKulbShNPzr3EikHSupFe37XuvpC+Ncf7mkq5Z\n07gREQOhi4vXS5oOfBHYA9gGOFDSNm3nzAM+BLzE9rOA905YxNX5XG1OYNVSkiMOKPsjIoabJrFN\nbAdgme0bbD8IfAfYq+2cQ4Av2v4jgO3bJ7poNxLBycBrJD0Oqm/8wFOACyUdLekaSUsk7d/+i5Le\nKukLLe9/KGnn8vqe8vvXSvqJpB0knSfpBkmvL+dML+dcKulqSW/vwueJiOiaadM63zqwKXBLy/vl\nZV+rrYGtJf1M0kWSJpwDZY0Tge07qRaS36PsOgA4CXgjsB2wLfBK4GhJm0zi0usB55Sqzd3AJ4Dd\ngDcAHyvnHAzcZfsFwAuAQyRtsWafKCKieybTRSBpgaTFLduC1Qg5A5gH7Ey1jvtXJG0w3i90q7O4\ntXlopFloJ+AE2w/bvg04n+pm3akHgdPL6yXA+bYfKq83L/tfBbxF0pXAxcCGVP8BHqP1P/DJx58+\n2ikREV03bZo63mwvtD2/ZVvYdrlbgbkt7+eUfa2WA4tsP2T7RuBXjHFffKSMa/ohi1OBXSVtD6xr\n+7IOf29FWxlmtrx+yPbIVIcrgQcAbK9k1bBXAe+yvV3ZtrB95miBWv8D73NQM7NFRsTw6fKgoUuB\neZK2KM3xBwCL2s75PlVtAEkbUTUV3TDeRbuSCGzfA5wLHMOqTuILgf1LO/6TgJdRNSG1ugnYTtI0\nSXOpOkIm4wzg7yStBSBpa0nrrebHiIjoum4mAtsrgEOp7n2/AE6yfa2kj430nZZjd0haSnVffr/t\nO8a7bjcfKDsBOIVVTUSnAC8CrgIMfMD270pn8oifATcCS6k+1OWTjPlVqmaiyyUJ+D2w9+oVPyKi\n+9Tlp7Vsnwac1rbviJbXBt5Xto50LRHY/j4tA6BKYd5fttbzbgKe3XLOqCtO2J7V8vrI0Y6VZqIP\nly0iou8MwJxzmWIiIqJOAzDVUBJBRESdOnw+oFFJBBERNdIAtA0lEURE1KjbncV1SCKIiKjRAFQI\nkggiIuqURNCnnjt73Keta7Fi5X09j1nxxKd02R03HNrzmAAbbvmFiU+qwb03H9lI3Caowykyu22j\nmeM+D9XXpqdpKCJiuKVGEBEx5JIIIiKGnAbgibIkgoiIGqVGEBEx5JIIIiKGXEYNRUQMudQIIiKG\n3CBMMdHXRZT0sKQrJV0j6b8krTvOuXtL2qaX5YuImEiXl6qsRV8nAuC+shbxs6kWs3/HOOfuDSQR\nRERfkdTx1pR+TwStLgS2ApD0FklXS7pK0jckvRh4PXB0qUE8rdGSRkQUg1AjGIg+AkkzgD2A0yU9\nCzgceLHtP0iabftOSYuAH9o+udHCRkS0GISFafq9iOtIuhJYDNwMfA3YBfgv238AsH1nJxeStEDS\nYkmLFy48sbYCR0S0mqbOt6b0e43gPtvbte5Y3XY02wuBhQDmut5PyRkRQ2kAZpjo+xrBaM4B9pW0\nIYCk2WX/3cDjGytVRMQopskdb42VsbHIq8n2tcAngfMlXQV8phz6DvB+SVekszgi+kWahtaQ7Vlj\n7D8eOL5t38/I8NGI6DMzGvym36m+TgQREYNuEPoIkggiImo0CO3vSQQRETVKjSAiYsgpfQQREcMt\nNYKIiCGXUUN9avZWn226CD2jBiZDv+CS3XseE+Dem49sJO66mzUTd60Z6/Q85kbrNzNC+0/33NRI\n3Duvf9EaXyM1goiIITcIo4YGoYwREQOr208WS9pd0nWSlkk6bJzz/lKSJc2f6JqpEURE1KibcwhJ\nmg58EdgNWA5cKmmR7aVt5z0eeA9wcUdl7FoJIyLiMWao860DOwDLbN9g+0GqOdb2GuW8jwOfAu7v\n5KJJBBERNery7KObAre0vF9e9j1C0vbAXNs/6rSMaRqKiKjRZEYNSVoALGjZtbCspdLp70+jmpH5\nrZ1HTSKIiKjVZBJB6wJaY7gVmNvyfk7ZN+LxwLOB88oiXn8BLJL0etuLx7poEkFERI263P5+KTBP\n0hZUCeAA4K9GDtq+C9ho5L2k84B/Gi8JQI8TgaSbqFYSexhYYXt+WWHsRGBz4CZgP9t/7GW5IiLq\n0s1RQ7ZXSDoUOAOYDhxj+1pJHwMW2160OtdtokbwipGF54vDgLNtH1XGxB4GfLCBckVEdF2Ho4E6\nZvs04LS2fUeMce7OnVyzH5qG9gJ2Lq+PB84DPijpSGALYEtgM+AfgB2BPaiqRK+z/ZCkI4DXAesA\n/w283Xb/T+4REUNhEKaY6PXwUQNnSrqs9I4DbGz7t+X174CNW85/GrAL8Hrgm8C5tp8D3Ae8ppzz\nBdsvsP1sqmTw2ro/REREpyR3vDWl1zWCnWzfKunJwFmSftl60Lb16P8aPy7f+pdQtYedXvYvoepT\nAHiFpA8A6wKzgWuBH7QHbh2Wtc6TXs7aT8jyxhFRv9QI2ti+tfy8HTiF6im52yRtAlB+3t7yKw+U\n81cCD7U0+awEZkiaCfwHsE+pKXwFmDlG7IW259uenyQQEb0ybRJbU3oWW9J6Zf4LJK0HvAq4BlgE\nHFROOwg4dRKXHbnp/0HSLGCfLhU3IqIruvxkcS162TS0MXBKechhBvBt26dLuhQ4SdLBwG+A/Tq9\noO0/SfoKVUL5HdUY24iIvjFjACby6VkisH0DsO0o++8Adh1l/5Ft72eNdsz24cDhXSxqRETXTG+6\nAB3oh+GjERFTVpNNPp1KIoiIqNEgjBpKIoiIqFESQUTEkFsrncUREcMtfQQREUMuTUN9ap2ZG/Y8\nZvVwdO9VCxb11vQB+MPvprVmrNNI3IdW3NfzmHffe+vEJ9Vg5tpPbCRuN2T4aETEkEuNICJiyKWP\nICJiyGXUUETEkEvTUETEkEsiiIgYctPTRxARMdwGoIsgiSAiok6D0DTU9WQl6RhJt0u6pmXfbEln\nSbq+/Hxi2S9Jn5O0TNLVkrbvdnkiIpq01jR3vDWljlrLccDubfsOA862PQ84u7wH2AOYV7YFwJdq\nKE9ERGOmqfOtsTJ2+4K2LwDubNu9F3B8eX08sHfL/q+7chGwgaRNJO0s6XxJp0q6QdJRkt4k6RJJ\nSyQ9DUDS6yRdLOkKST+RtHG3P09ExJoYykQwho1t/7a8/h3V+sUAmwK3tJy3vOyDalnLdwDPBP4a\n2Nr2DsBXgXeVc34K7Gj7ecB3gA/U9gkiIlbDICSCnncW27bU0XiqS0eSh6RfA2eW/UuAV5TXc4AT\nJW0CPA64cayLSVpA1fzE+pvsybqz0x0REfUbhEkYe1UjuK3crCk/by/7bwXmtpw3p+wDeKBl/8qW\n9ytZlcA+D3zB9nOAtwMzxyqA7YW259uenyQQEb0yQ+54a0qvEsEi4KDy+iDg1Jb9bymjh3YE7mpp\nQurE+qxKHAeNd2JERBOGsmlI0gnAzsBGkpYDHwGOAk6SdDDwG2C/cvppwJ7AMuBe4G2TDHck8F+S\n/gicA2yxpuWPiOimQWga6noisH3gGId2HeVcA+8cZf95wHkt73ce7ZjtU1lVu4iI6DuZhjoiYsgN\nwpPFSQQRETUahEQwCPMhRUQMrOnqfOuEpN0lXVem5jlslOPvk7S0TNtztqSnTnTNJIKIiBppEtuE\n15KmA1+kmp5nG+BASdu0nXYFMN/2c4GTgX+d6LpJBBERNZI63zqwA7DM9g22H6SaUWGv1hNsn2v7\n3vL2Iqrns8aVRBARUaNpk9g6MN60PKM5GPjxRBdNZ3FERI06m1Fn5NxVU+EUC20vXL24ejMwH3j5\nROcOZSJYb04Dz52tbGgscQNDFh6/VjOfVR21snbfRuu3N9H2xt333jrxSV12z32/63lMgDlb7NZI\n3G6YzF9luemPd+Mfb1qeVTGlVwL/DLzc9gPtx9sNZSKIiOiVLn8XuxSYJ2kLqgRwAPBXrSdIeh7w\nn8Dutm9/7CUeK4kgIqJG3UwEtldIOhQ4A5gOHGP7WkkfAxbbXgQcDcyimn4H4Gbbrx/vukkEERE1\n6naDpe3TqOZpa913RMvrV072mkkEERE16nBYaKOSCCIiajQAeSCJICKiTkM5DXVERKwyAHmg+08W\nS5or6dwy6dG1kt5T9s+WdJak68vPJ5b9kvS5MoHS1ZKyjmRETBmSO96aUscUEyuAf7S9DbAj8M4y\nKdJhwNm25wFnl/dQTZ40r2wLgC/VUKaIiEZ0c9K5unQ9Edj+re3Ly+u7gV9QzYWxF3B8Oe14YO/y\nei/g665cBGwgaRNJO0s6X9Kpkm6QdJSkN0m6RNISSU8DkPQ6SRdLukLSTyRt3O3PFBGxuro86Vwt\nap10TtLmwPOAi4GNWxam/x0wcsMebxKlbYF3AM8E/hrY2vYOwFeBd5VzfgrsaPt5VDPxfaCOzxIR\nsTq6POlcLWqLLWkW8F3gvbb/3HqsrFXcSYPYpaWG8QDwa+DMsn8JsHl5PQc4Q9IS4P3As8YozwJJ\niyUtvuuW8yf9eSIiVsc0db41VsY6LippLaok8C3b3yu7b5O0STm+CTAyB8Z4kyi1Tpa0suX9SlaN\nePo88AXbzwHeDswcrUy2F9qeb3v++nMnnIwvIqIrhrJpSNXkFl8DfmH7My2HFgEHldcHAae27H9L\nGT20I3BXSxNSJ9ZnVeI4aLwTIyJ6bRA6i+t4juAlVO35SyRdWfZ9GDgKOEnSwcBvgP3KsdOAPYFl\nwL3A2yYZ70iqyZX+CJwDNDDHdETE6AZh8fquJwLbP2Xs5LbrKOcbeOco+88Dzmt5v/Nox2yfyqra\nRUREXxmAPJAniyMi6jStwQfFOpVEEBFRo8w+GhEx5AYgDyQRRETUqckHxTqVRBARUaM0DUVEDL3+\nzwRJBBERNZqm6U0XYUJDmQiWLz2r6SJMaXc88KZG4m40845G4v7pnpsaiTtz7Sf2POacLXbreUxo\n8t/s33ThGqkRREQMNSURREQMuySCiIihJvX/ANIkgoiIWqVGEBEx1KYNwCNlSQQREbVKIoiIGGoa\ngEeLkwgiImrV/4mgozqLpL0lWdIzxjnnOEn7lNdflbTNKOesJekoSddLulzSzyXtUY7dJGmj1f0g\nERH9SJP4X1M6rREcCPy0/PzIRCfb/tsxDn0c2AR4tu0HJG0MZCX5iJiyRP9PMTFhjUDSLGAn4GDg\ngJb9kvQFSddJ+gnw5JZj50ma33addYFDgHfZfgDA9m22Txol5vskXVO295Z960n6kaSryv79y/7n\nSzpf0mWSzpC0yer8h4iIqIOkjremdFIj2As43favJN0h6fm2LwPeADwd2AbYGFgKHDPOdbYCbrb9\n5/GCSXo+1QL2L6RqXLtY0vnAlsD/2H5NOW99SWsBnwf2sv37khw+ySgThEhaACwAmPHE+cyYtVUH\nHz0iYk1NjT6CA4HvlNffKe8BXgacYPth2/8DnNOlMu0EnGL7f23fA3wPeCmwBNhN0qckvdT2XVSJ\n6NnAWZKuBA4H5ox2UdsLbc+3PT9JICJ6RUzreOvoetLupSVmmaTDRjm+tqQTy/GLJW0+0TXHrRFI\nmg3sAjxHkoHpgCW9v6MSP9oyYDNJT5ioVjCaUiPZHtgT+ISks4FTgGttv2g1yhMR0QPdqxFImg58\nEdgNWA5cKmmR7aUtpx0M/NH2VpIOAD4F7D/edSdKQfsA37D9VNub254L3Ej1Df0CYH9J00u7/CvG\nu5Dte4GvAZ+V9LjyoZ4kad+2Uy8E9pa0rqT1qJqgLpT0FOBe298Ejga2B64DniTpReV6a0l61gSf\nKSKiZ7rcR7ADsMz2DbYfpGql2avtnL2A48vrk4FdNcHFJ0oEB1J962713Zb911P1DXwd+HnbeR7l\neocDvweWSroG+CHwqNqB7cuB44BLgIuBr9q+AngOcElpAvoI8InyH2If4FOSrgKuBF48wWeKiOgZ\nMb3jrQObAre0vF9e9o16ju0VwF3AhuNddNymIduP+ZZv+3Mtbw8d41c3BO4c5XcfBD5QtvZjm7e8\n/gzwmbbjZwBnjPJ7V1L1V0RE9KHOm4ZaB7UUC20v7HqR2nT9yWJJZwFLbN/Y7WtHRAyayTwoVm76\n4934bwXmtryfU/aNds5ySTOA9YFxl+/reiKw3cxadhERfajLzwdcCsyTtAXVDf8A4K/azlkEHETV\nXL8PcI7t0ZrqH5G5hiIiatW92Udtr5B0KFUz+XTgGNvXSvoYsNj2IqpBOd+QtIyqif6Asa9YSSKI\niKhRt+cQsn0acFrbviNaXt8PtI/GHFcSQUREjbJUZUTE0Ov/RKAJ+hCihaQFvRjKlbi9jztMnzVx\no13/p6r+smDiUxJ3QOMO02dN3HiUJIKIiCGXRBARMeSSCCanqTbGxJ2aMRN36scdCOksjogYcqkR\nREQMuSSCiIghl0QQETHkkgjiUSQ9reky9EqZwXHCfTF4JG0s6WuSflzebyPp4KbL1a/SWTwBSU8C\nDgE2p2VKDtt/04PYmwJPbYt7Qc0xz6ea4/xSqmVDL7C9pM6YJe5TgXm2fyJpHWCG7btrjnm57e3b\n9l1m+/l1xi1xXgM8C5g5ss/2x+qO24Tyb+iDwDY8+vPuUmPMHwPHAv9se9syL/8Vtp9TV8xBlrmG\nJnYq1Q3xJ8DDvQoqaWTB6aUtcU21VnRtbL+8rCn9AmBn4EeSZtmeXVdMSYdQPfk5G3gaVSL6MrBr\nTfGeQXUTXl/SG1sOPYGWG1VdJH0ZWJdqne+vUs0Zf0kP4vb8hlx8CzgReA3wDqq58n9fc8yNbJ8k\n6UPwyPTNPfv3O2iSCCa2ru0PNhB3b+Dpth/oZVBJOwEvLdsGVOtKX1hz2HdSLcp9MYDt6yU9ucZ4\nTwdeS/X5Xtey/26q2l/dXmz7uZKutv1RSZ8GftyDuE3ckAE2tP01Se+xfT5wvqRLa475v5I2pKyd\nLmlHqrV7YxRJBBP7oaQ9yxzgvXQDsBbQ00QAnAdcBvwLcFpZZ7puD9h+cGQlp1KNr63N0vapwKmS\nXmT753XFGcd95ee9kp5CtYzgJj2I28QNGeCh8vO3pUnsf6hqf3V6H9VKXU+T9DPgSVQ1rxhFEsHE\n3gN8WNKDwINUK1Hb9hPqCCbp81Q3wXuBKyWdTUsysP3uOuK22Ah4CfAy4N2SVgI/t/1/aox5vqQP\nA+tI2g34e+AHNcYb8Q5Jv7D9JwBJTwQ+3YP+nx9K2gA4Gric6v/vr9YcE5q5IQN8QtL6wD8Cn6dq\ngvuHOgPavlzSy6lqfwKus/3QBL82tNJZ3GckHTTecdvH96AMzwReTtU89GLgZtsvrzHeNOBg4FVU\n/2jPAL460TqrXYh7he3nTbSv5jKsDcy0XXuzhaTXUjXzzWXVDfmjZXnDKUXSvsDptu+WdDiwPfAJ\n25c3XLS+lEQwAVXtFW8CtrD9cUlzgU1s19651wRJNwC/BH5K1TF9SY+ah3pO0lXAzrb/WN7PBs6v\ne2SJpOlU7fSb8+gRYZ+pM25TypDcd/HYz/v6GmNeXfphdgI+DvwbcITtF9YVc5ClaWhi/wGsBHah\n+oO6B/gi1aia2khawmPbye8CFlN9s7mjptBb2V5Z07VHVb6pfpxVQ2VrbX5r8WngIkknlff7Ap+s\nOSZUzV73A0uo/rZ6ookbcvF9qgXVf0DvPu/ICKHXAF+x/SNJn+hR7IGTGsEERsaatzYZSLrK9rY1\nx/1Xqj/mb5ddB1ANOfwdsJPt1431u2sYdw5Vs8FLyq4LgffYXl5HvBJzGfBGYEndzUGjxN6GKskD\nnGN7aQ9iXm37uXXHGSXuVVQ35EcloNJxXGfci3v9TVzSD4Fbgd2omoXuo6rd1vrvdlClRjCxh0pV\nfmQY2pPozbeaV7Y97LSkJSm9uca4x1Iln33L+zeXfbvVGPMW4JpeJQFJM6mGT25FdVP8su0VvYhd\n/FjSq2yf2cOYAPfb/lyPYwJ8VtJHgDN59MCHOtvr9wN2B/7N9p8kbQK8v8Z4Ay2JYGKfA04BNpb0\nSaohaIclgW6TAAAInUlEQVT3IO50STuM9EVIegEwvRyr86b1JNvHtrw/TtJ7a4wH8AHgtPJUc+uN\noq428+OpRtBcCOwBPBOo+zO2ugg4pXSSP0TvmsKauCEDPAf4a6qa18iXKLOqJtZ1tu+V9Gvg1ZJe\nDVzYQOIdGEkEE7D9LUmXseop171t/6IHof8WOEbSLKobxZ+Bv5W0HtUY/7rcUWocJ5T3B1KNc6/T\nJ6n6XmYCj6s5FsA2Ix3Ckr5GD57qbfMZ4EX0vims5zfkYl9gy14OOpD0HqqHA79Xdn1T0kLbn+9V\nGQZJEkFn1qX6Nm5gnV4EtH0p8Jwy/pq24YUnjf5bXfE3VH0E/4/q8/438NYa4wE8xfaza47R6pHx\n5GXqgR6GBnrcFNai5zfk4hqqp7hv72HMg4EX2v5feGTKlp9T/W1HmySCCUg6guof0HepvpkfK+m/\nbNcyAkHSm21/U9L72vYD9Q8xtP0b4FGjSErT0L/XGPa0HreZbyvpz+W1qB5k+zO9a6K5ATivTIzW\ni6awEU3ckCkxf1meYm79vHWOVhKPnhvs4bIvRpFEMLE3Advavh9A0lHAlUBdQ9HWKz8fX9P1V8f7\nqDcR/B3wT5IeoAdt5ranT3xWrW4s2+PoTVPYiCZuyAAfqfn6ozkWuFjSKeX93lQjpmIUGT46AUnn\nAm9omYZgA+B7PZixsW9IusX23KbLEWumTLnwGD0YProxsGl5e6vt2+qM1xJ3e2Cn8vZC21f0Iu4g\nSiKYgKTvUz08dlbZ9UqqzsXl0P25fySNO7yvB3MNPYakm21vVuP1Xzbafte89kKvlf6eD1F9O30y\nVR/M7VRTnR818mWjxvg9vSFL2o5qOvH1qcb0QzXF+J+Av6vjxlyeDh+T7Tu7HXMqSNPQxM4Azqb6\nR7sCOLfmeJe1vP4oPapWS7qb0Wf8FPV3kLeO755JNSX1ZdQ/mqXXTgLOoZrW4ncAkv6CqjP+JKq5\nlrpurBuypNpuyMVxwNttX9xWnh3LsToe7rqM6u94pD9g5G9a5fWWNcQceKkRjKFMhfx/qUbR/Ibq\nD2kzqrbHD/diJsNeT4DWL8p8Tv9u+y+bLks3SbrO9tMne6wLca9k7Bvyf9b1tK2k623PG+PYMttb\n1RE3Ji9rFo/taKoperew/fzylO+WVN+qju5RGYY1Sy+neshrqvmNpA+UJhrgkbV1P0g1pLQu67Un\nAQDbF7FqcEIdfizpR5L2l/Tisu0v6UfA6XUElPRqSY9Zd0DSX5YpzmMUqRGMQdL1wNbtY73LdBO/\nHOubTpfL8Jg1dacirVqDAaovJ9sBN9mucyqNnlO13sFhwF7AxlSf+TaqBVQ+VVf7del3ehrwdVYl\nnLnAW4AbbR9aR9wSew+qz/tI3wSwyDUt9KRqEZq9bf++bf9GwA9sv6iOuIMuiWAMkn5le+vJHutC\n3Na2+nWpFqiB3o1x7zk9eg2GFVRJ4GdNladXJL2Uqj9kSd3PUPT6htwUSYttzx/jWCOT/Q2CJIIx\nlNFC37P99bb9bwb268HY65hiJF1ie4fy+m+p1mr+PlUn8Q9sH9Vk+bqtZZRUaw2o1lFSkn5FNYXI\nirb9awFLe1GTH0RJBGOQtCnVPCX3sWokz3yqETRvsH3rWL8bnRljzQVYVfuZUt/e2qYyvxTY0/bv\ny/xRF7mmBXGauCGXuGdQjZI6fpRRUrvY7vooqfLA58bAoS3TS8wCPgv8wfYHux1zKkgimICkXYBn\nlbdLbZ/dZHmmEklPHe94me5iyijrAexM1Q9yRmsTRp0jxJq4IZcYPR8lVUb7fYJq0saRv5/NqJ4q\n/j+9GO03iJIIoi+UkTQjq75dYrvX8+HUTtJNVLN+joxpf4nt35ZvrD+1vV1NcZsatnom8BOqBHRb\n2bcxVQLazfYr64hb4qxDtd4EwDLb99UVayrI8NFonKT9qJ7W3pdqQZGLRxsCOOhsb257S9tblJ+/\nLYdWAm+oMXRTw1b3BzYEzpf0R0l3AudRDcver8a42L7P9pKyJQlMIDWCaFxpMtltpBagahW4n9T1\noNOwaRu2+uSye2TY6lG2/1hj7GdQTStxke17WvbvbruWZwli8pIIonGSlrR2lKpaueuqujpPYxVJ\nb/OjV6Tr5rXfTTUy6hdUz4a8x/ap5dhQPCMzKDLXUPSD00uH5siqaPsDU2p8ex/7KNW0KXU4BHi+\n7XskbQ6cLGlz25+lprUBJD3D9i/LzKPtDNw51QYhdENqBNEYSV8Evm37Z5LeyKOnDD5lnF+NSZB0\n9ViHqJ6eX7umuNfaflbL+1nAycBSqtFKXe8cl/QV24eU6eNHsyFVbfOvux17kKVGEE36FfBvkjah\nmn3zG5kzvhYbA68G2vsCRLUUaV1uk7Sd7SsBSs3gtcAxVOsnd53tQ8rPV4x1ThnNFC1SI4jGlecJ\nDijbOlRNRCfY/lWjBZsiJH0NONb2T0c59m3bf1VT3DnAipFnF9qOvaSOaURKzXJMtr833vFhlUQQ\nfUXS86i+MT7XzS8pGQNG0kh/x5OBF1M9SAfwCuC/bb+2kYL1uTQNRePK06B7UNUIdqUaa35kg0WK\nAWX7bfBI8882I89qlObH4xosWl9LIojGlPnhDwT2pHqg7DvAgpE5YiLWwNyWB/agem6ituVWB10S\nQTTpQ8C3gX+s86GmGEpntw1JPoBquosYRfoIImJKkvQG4GXl7QUZkjy2JIKImPLKIkAH2H5n02Xp\nR2kaiogpqYxAO5BqgrsbqdYXiVEkEUTElCFpa6qb/4HAH4ATqVo+xnzALNI0FBFTiKSVwIXAwbaX\nlX032N6y2ZL1t6xHEBFTyRuB3wLnSvqKpF2paYK7qSQ1goiYcso60HtRNRHtAnwdOMV25hkaRRJB\nRExpZWGefYH9be/adHn6URJBRMSQSx9BRMSQSyKIiBhySQQREUMuiSAiYsglEUREDLn/Dz7Lv4O5\nzodpAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f9dd8f9ba58>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "corr = df.corr()\n",
    "ax = sns.heatmap(corr, cmap=\"YlGnBu\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 6. Split out training set and testing set data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def load_data(stock, seq_len):\n",
    "    amount_of_features = len(stock.columns)\n",
    "    data = stock.as_matrix() \n",
    "    sequence_length = seq_len + 1 # index starting from 0\n",
    "    result = []\n",
    "    \n",
    "    for index in range(len(data) - sequence_length): # maxmimum date = lastest date - sequence length\n",
    "        result.append(data[index: index + sequence_length]) # index : index + 22days\n",
    "    \n",
    "    result = np.array(result)\n",
    "    row = round(0.9 * result.shape[0]) # 90% split\n",
    "    \n",
    "    train = result[:int(row), :] # 90% date\n",
    "    \n",
    "    pRow = round(0.80 * train.shape[0])\n",
    "    pTrain =  result[:int(pRow), :]\n",
    "    left_X_train = pTrain[:, :-1] # all data until day m\n",
    "    left_y_train = pTrain[:, -1][:,-1] # day m + 1 adjusted close price\n",
    "    right_X_test = train[int(pRow):, :-1]\n",
    "    right_y_test = train[int(pRow):, -1][:,-1] \n",
    "    \n",
    "    X_train = train[:, :-1] # all data until day m\n",
    "    y_train = train[:, -1][:,-1] # day m + 1 adjusted close price\n",
    "    \n",
    "    X_test = result[int(row):, :-1]\n",
    "    y_test = result[int(row):, -1][:,-1] \n",
    "\n",
    "    X_train = np.reshape(X_train, (X_train.shape[0], X_train.shape[1], amount_of_features))\n",
    "    X_test = np.reshape(X_test, (X_test.shape[0], X_test.shape[1], amount_of_features))  \n",
    "\n",
    "    print(\"left_X_train: \" + str(len(left_X_train)))\n",
    "    print(\"right_X_test: \" + str(len(right_X_test)))\n",
    "    return [left_X_train, left_y_train, right_X_test, right_y_test]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "left_X_train: 3114\n",
      "right_X_test: 779\n"
     ]
    }
   ],
   "source": [
    "X_train, y_train, X_test, y_test = load_data(df, seq_len)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 7. Model Execution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def build_model(shape, neurons, dropout, decay):\n",
    "    model = None\n",
    "    try:\n",
    "        model = load_model(modelFileName)\n",
    "    except:\n",
    "        print(modelFileName + ' not found')\n",
    "        \n",
    "    if model != None:\n",
    "        return [model, True]\n",
    "    model = Sequential()\n",
    "\n",
    "    model.add(LSTM(neurons[0], input_shape=(shape[0], shape[1]), return_sequences=True))\n",
    "    model.add(Dropout(dropout))\n",
    "\n",
    "    model.add(LSTM(neurons[1], input_shape=(shape[0], shape[1]), return_sequences=False))\n",
    "    model.add(Dropout(dropout))\n",
    "\n",
    "    model.add(Dense(neurons[2],kernel_initializer=\"uniform\",activation='relu'))\n",
    "    model.add(Dense(neurons[3],kernel_initializer=\"uniform\",activation='linear'))\n",
    "    # model = load_model('my_LSTM_stock_model1000.h5')\n",
    "    adam = keras.optimizers.Adam(decay=decay)\n",
    "    model.compile(loss='mse',optimizer='adam', metrics=['accuracy'])\n",
    "    model.summary()\n",
    "    return [model, False]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "model-e100.h5 not found\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "lstm_1 (LSTM)                (None, 22, 256)           272384    \n",
      "_________________________________________________________________\n",
      "dropout_1 (Dropout)          (None, 22, 256)           0         \n",
      "_________________________________________________________________\n",
      "lstm_2 (LSTM)                (None, 256)               525312    \n",
      "_________________________________________________________________\n",
      "dropout_2 (Dropout)          (None, 256)               0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 32)                8224      \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 1)                 33        \n",
      "=================================================================\n",
      "Total params: 805,953\n",
      "Trainable params: 805,953\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "learned: False\n"
     ]
    }
   ],
   "source": [
    "model, learned = build_model(shape, neurons, dropout, decay)\n",
    "print('learned: ' + str(learned))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 2491 samples, validate on 623 samples\n",
      "Epoch 1/100\n"
     ]
    }
   ],
   "source": [
    "if learned == False:\n",
    "    model.fit(\n",
    "        X_train,\n",
    "        y_train,\n",
    "        batch_size=512,\n",
    "        epochs=epochs,\n",
    "        validation_split=0.2,\n",
    "        verbose=1)\n",
    "    model.save(modelFileName)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 8. Result on training set and testing set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def model_score(model, X_train, y_train, X_test, y_test):\n",
    "    trainScore = model.evaluate(X_train, y_train, verbose=0)\n",
    "    print('Train Score: %.5f MSE (%.2f RMSE)' % (trainScore[0], math.sqrt(trainScore[0])))\n",
    "\n",
    "    testScore = model.evaluate(X_test, y_test, verbose=0)\n",
    "    print('Test Score: %.5f MSE (%.2f RMSE)' % (testScore[0], math.sqrt(testScore[0])))\n",
    "    return trainScore[0], testScore[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_score(model, X_train, y_train, X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 9. Prediction vs Real results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def percentage_difference(model, X_test, y_test):\n",
    "    percentage_diff=[]\n",
    "\n",
    "    p = model.predict(X_test)\n",
    "    for u in range(len(y_test)): # for each data index in test data\n",
    "        pr = p[u][0] # pr = prediction on day u\n",
    "\n",
    "        percentage_diff.append((pr-y_test[u]/pr)*100)\n",
    "    return p"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def denormalize(stock_name, normalized_value):\n",
    "    \"\"\"\n",
    "    Return a dataframe of that stock and normalize all the values. \n",
    "    (Optional: create moving average)\n",
    "    \"\"\"\n",
    "    df = quandl.get_table('WIKI/PRICES', ticker = stock_name)\n",
    "    df.drop(['ticker', 'open', 'high', 'low', 'close', 'ex-dividend', 'volume', 'split_ratio'], 1, inplace=True)\n",
    "    df.set_index('date', inplace=True)\n",
    "    \n",
    "    # Renaming all the columns so that we can use the old version code\n",
    "    df.rename(columns={'adj_open': 'Open', 'adj_high': 'High', 'adj_low': 'Low', 'adj_volume': 'Volume', 'adj_close': 'Adj Close'}, inplace=True)\n",
    "    \n",
    "    df.dropna(inplace=True)\n",
    "    df = df['Adj Close'].values.reshape(-1,1)\n",
    "    normalized_value = normalized_value.reshape(-1,1)\n",
    "\n",
    "    #return df.shape, p.shape\n",
    "    min_max_scaler = preprocessing.MinMaxScaler()\n",
    "    a = min_max_scaler.fit_transform(df)\n",
    "    new = min_max_scaler.inverse_transform(normalized_value)\n",
    "      \n",
    "    return new"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def plot_result(stock_name, normalized_value_p, normalized_value_y_test):\n",
    "    newp = denormalize(stock_name, normalized_value_p)\n",
    "    print(newp)\n",
    "    print(normalized_value_p)\n",
    "    newy_test = denormalize(stock_name, normalized_value_y_test)\n",
    "    plt2.plot(newp, color='red', label='Prediction')\n",
    "    plt2.plot(newy_test,color='blue', label='Actual')\n",
    "    plt2.legend(loc='best')\n",
    "    plt2.title('The test result for {}'.format(stock_name))\n",
    "    plt2.xlabel('Days')\n",
    "    plt2.ylabel('Adjusted Close')\n",
    "    plt2.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "p = percentage_difference(model, X_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "plot_result(stock_name, p, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part 2. Finding the best hyperparameter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def quick_measure(stock_name, seq_len, dropout, shape, neurons, epochs, decay):\n",
    "    df = get_stock_data(stock_name)\n",
    "    X_train, y_train, X_test, y_test = load_data(df, seq_len)\n",
    "    model = build_model(shape, neurons, dropout, decay)\n",
    "    model.fit(X_train, y_train, batch_size=512, epochs=epochs, validation_split=0.2, verbose=0)\n",
    "    # model.save('LSTM_Stock_prediction-20170528.h5')\n",
    "    trainScore, testScore = model_score(model, X_train, y_train, X_test, y_test)\n",
    "    return trainScore, testScore"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "quandl.ApiConfig.api_key = 'zpFWg7jpwtBPmzA8sT2Z'\n",
    "seq_len = 22\n",
    "shape = [seq_len, 6, 1] # Without moving average, 6 features will be used\n",
    "neurons = [256, 256, 32, 1]\n",
    "dropout = 0.3\n",
    "decay = 0.5\n",
    "epochs = 80\n",
    "stock_name = 'AAPL'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Testing for epochs\n",
    "Repeatedly test it and sum up the result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "epochslist = [40,50,60,70,80,90,100,110,120,130,140,150]\n",
    "epochs_train_result = {}\n",
    "epochs_test_result = {}\n",
    "#i=0\n",
    "\n",
    "#for epochs in epochslist:   \n",
    "#    epochs_train_result[epochs] = 0\n",
    "#    epochs_test_result[epochs] = 0\n",
    "\n",
    "# while i < 5:\n",
    "for epochs in epochslist:    \n",
    "    trainScore, testScore = quick_measure(stock_name, seq_len, d, shape, neurons, epochs, decay)\n",
    "    epochs_train_result[epochs] = trainScore\n",
    "    epochs_test_result[epochs] = testScore\n",
    "#    i+=1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# epochs_train_result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "min_train_val = min(epochs_train_result.values())\n",
    "min_train_val_key = [k for k, v in epochs_train_result.items() if v == min_train_val]\n",
    "print (epochs_train_result)\n",
    "print (min_train_val_key)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "min_test_val = min(epochs_test_result.values())\n",
    "min_test_val_key = [k for k, v in epochs_test_result.items() if v == min_test_val]\n",
    "print (epochs_test_result)\n",
    "print (min_test_val_key)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "lists1 = sorted(epochs_test_result.items())\n",
    "x1,y1 = zip(*lists1)\n",
    "plt.plot(x1,y1)\n",
    "\n",
    "plt.title('Finding the best hyperparameter for testing')\n",
    "plt.xlabel('Dropout')\n",
    "plt.ylabel('Mean Square Error')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "lists2 = sorted(epochs_train_result.items())\n",
    "x2,y2 = zip(*lists2)\n",
    "plt.plot(x2,y2)\n",
    "plt.title('Finding the best hyperparameter for training set')\n",
    "plt.xlabel('Dropout')\n",
    "plt.ylabel('Mean Square Error')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Testing for dropout"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "dlist = [0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8]\n",
    "dropout_train_result = {}\n",
    "dropout_test_result = {}\n",
    "\n",
    "for d in dlist:    \n",
    "    trainScore, testScore = quick_measure(stock_name, seq_len, d, shape, neurons, epochs, decay)\n",
    "    dropout_train_result[d] = trainScore\n",
    "    dropout_test_result[d] = testScore"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "min_train_val = min(dropout_train_result.values())\n",
    "min_train_val_key = [k for k, v in dropout_train_result.items() if v == min_train_val]\n",
    "print (dropout_train_result)\n",
    "print (min_train_val_key)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "min_test_val = min(dropout_test_result.values())\n",
    "min_test_val_key = [k for k, v in dropout_test_result.items() if v == min_test_val]\n",
    "print (dropout_test_result)\n",
    "print (min_test_val_key)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "lists1 = sorted(dropout_test_result.items())\n",
    "x1,y1 = zip(*lists1)\n",
    "plt.plot(x1,y1)\n",
    "plt.title('Finding the best hyperparameter for testing set')\n",
    "plt.xlabel('Dropout')\n",
    "plt.ylabel('Mean Square Error')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "lists2 = sorted(dropout_train_result.items())\n",
    "x2,y2 = zip(*lists2)\n",
    "plt.plot(x2,y2)\n",
    "plt.title('Finding the best hyperparameter for training set')\n",
    "plt.xlabel('Dropout')\n",
    "plt.ylabel('Mean Square Error')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}