Code/15-MO_CNN_Publish_04-28-2016.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Yield Oracles:  Predicting Soybean Yields in Brazil Using Satellite and Weather Data\n",
    "\n",
    "This code was written as part of a project to satisfy the [Capstone Course](https://datascience.berkeley.edu/academics/curriculum/synthetic-capstone-course/) requirement of the [UC Berkeley Master of Information and Data Science](https://datascience.berkeley.edu) (MIDS) program.  Our project was completed in April 2016.  For more details and to see our results, please visit our [website](http://amitavadas.github.io/capstone/index.html).  We look forward to your feedback!\n",
    "\n",
    "For questions and comments about this code, please contact the author, Marguerite Oneto, by email at marguerite.oneto@ischool.berkeley.edu."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Run A Convolutional Neural Network Using Theano\n",
    "This code is based on a notebook shared in the [W207: Applied Machine Learning](https://datascience.berkeley.edu/academics/curriculum/applied-machine-learning/) course of the MIDS curriculum.  \n",
    "That notebook was based on the references below.  The code was cleaned up by my Final Project teammate Lei Yang.  \n",
    "\n",
    "Reference:  http://deeplearning.net/tutorial/lenet.html   \n",
    "Reference:  http://ufldl.stanford.edu/tutorial/supervised/FeatureExtractionUsingConvolution/   \n",
    "Reference:  https://www.youtube.com/watch?v=S75EdAcXHKk   \n",
    "Reference:  http://danielnouri.org/notes/2014/12/17/using-convolutional-neural-nets-to-detect-facial-keypoints-tutorial/   \n",
    "\n",
    "Data for Yield Prediction Test:  https://github.com/maoneto/W210/blob/master/Code/data/train_cnn_21images_64grids_trunc.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Run 5-Layer CNN Model Using Theano with All 21 Images"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create training, validation, and test datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_train and X_test Shape:\n",
      "(400, 21, 13, 13) (100, 21, 13, 13)\n",
      "Y_train and Y_test Shape:\n",
      "(400, 1) (100, 1)\n"
     ]
    }
   ],
   "source": [
    "import math\n",
    "import csv\n",
    "import numpy as np\n",
    "from sklearn.cross_validation import train_test_split\n",
    "\n",
    "X = []\n",
    "Y = []\n",
    "\n",
    "with open('train_cnn_21images_64grids_trunc.csv', 'r') as csvfile:\n",
    "    datareader = csv.reader(csvfile, delimiter=',')\n",
    "    for row in datareader:\n",
    "        label = row.pop()  ## pop the last element in the list which is the label (yield calc)\n",
    "        X.append(row)\n",
    "        Y.append(label)\n",
    "X = np.array(X).astype(np.float)\n",
    "Y = np.array(Y).astype(np.float)\n",
    "\n",
    "# Break labeled examples into train and test sets\n",
    "X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.20, random_state=22)\n",
    "\n",
    "X_train = X_train.reshape(X_train.shape[0], 21, 13, 13)\n",
    "X_test = X_test.reshape(X_test.shape[0], 21, 13, 13)\n",
    "Y_train = Y_train.reshape(Y_train.shape[0], 1)\n",
    "Y_test = Y_test.reshape(Y_test.shape[0], 1)\n",
    "\n",
    "print \"X_train and X_test Shape:\"\n",
    "print X_train.shape, X_test.shape\n",
    "\n",
    "print \"Y_train and Y_test Shape:\"\n",
    "print Y_train.shape, Y_test.shape\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5-Layer Convolutional Neural Network Using [Theano](http://www.deeplearning.net/software/theano/)\n",
    "  - 3 convolution layers (32/64/128 feature maps) with subsampling and 2 globally connected neural layers (600 neurons)\n",
    "  - Because the convergence process was not smooth, we reduced the learning rate to 0.0001 and saved the predictions for the test data whenever the cost was reduced during an epoch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cpu\n",
      "float64\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "import pickle\n",
    "from datetime import datetime\n",
    "import theano \n",
    "from theano import tensor as T\n",
    "from theano.sandbox.rng_mrg import MRG_RandomStreams as RandomStreams\n",
    "from theano.tensor.nnet.conv import conv2d\n",
    "from theano.tensor.signal.downsample import max_pool_2d\n",
    "print theano.config.device # We're using CPUs (for now)\n",
    "print theano.config.floatX # Should be 64 bit for CPUs\n",
    "\n",
    "theano.exception_verbosity='high'\n",
    "theano.mode='FAST_COMPILE'\n",
    "theano.allow_gc=False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model refreshed @ 11:10:35\n"
     ]
    }
   ],
   "source": [
    "class YieldPredictor():\n",
    "    \n",
    "    # Initialize an instance of the class.\n",
    "    def __init__(self, n_output):\n",
    "        self._getParameters(n_output)\n",
    "        self._getModel()\n",
    "\n",
    "    def _getParameters(self, numClasses):  # initialize weights randomly\n",
    "        numHiddenNodes = 600\n",
    "        patchWidth = 3\n",
    "        patchHeight = 3\n",
    "        featureMapsLayer1 = 32\n",
    "        featureMapsLayer2 = 64\n",
    "        featureMapsLayer3 = 128\n",
    "        # Convolution layers.\n",
    "        w_1 = theano.shared(np.asarray((np.random.randn(*(featureMapsLayer1, 21, patchWidth, patchHeight))*0.01)))\n",
    "        w_2 = theano.shared(np.asarray((np.random.randn(*(featureMapsLayer2, featureMapsLayer1, patchWidth-1, patchHeight-1))*0.01)))\n",
    "        w_3 = theano.shared(np.asarray((np.random.randn(*(featureMapsLayer3, featureMapsLayer2, patchWidth-1, patchHeight-1))*0.01)))\n",
    "        # Fully connected NN.\n",
    "        w_4 = theano.shared(np.asarray((np.random.randn(*(featureMapsLayer3 * 2 * 2, numHiddenNodes))*0.01)))\n",
    "        w_5 = theano.shared(np.asarray((np.random.randn(*(numHiddenNodes, numClasses))*0.01)))\n",
    "        self.params = [w_1, w_2, w_3, w_4, w_5]\n",
    "        self.srng = RandomStreams()\n",
    "\n",
    "    def _getModel(self):\n",
    "        theano.config.floatX = 'float64'\n",
    "        X = T.tensor4() # conv2d works with tensor4 type\n",
    "        Y = T.matrix()\n",
    "        \n",
    "        w_1, w_2, w_3, w_4, w_5 = self.params[0], self.params[1], self.params[2], self.params[3], self.params[4]\n",
    "        y_hat_train = self._model(X, w_1, w_2, w_3, w_4, w_5, 0.2, 0.5)\n",
    "        y_hat_predict = self._model(X, w_1, w_2, w_3, w_4, w_5, 0., 0.)\n",
    "\n",
    "        self.cost = T.sqrt(T.mean(T.sqr(Y - y_hat_train))) # RMSE. Or use SSE = T.sum(T.sqr(Y - y_hat_train))\n",
    "        update = self._backprop(self.cost, self.params)\n",
    "        self.train = theano.function(inputs=[X, Y], outputs=self.cost, updates=update, allow_input_downcast=True)\n",
    "        self.predict = theano.function(inputs=[X], outputs=y_hat_predict, allow_input_downcast=True)\n",
    "\n",
    "    def _model(self, X, w_1, w_2, w_3, w_4, w_5, p_1, p_2):\n",
    "        l1 = self._dropout(max_pool_2d(T.maximum(conv2d(X, w_1, border_mode='full'),0.), (2, 2)), p_1)\n",
    "        l2 = self._dropout(max_pool_2d(T.maximum(conv2d(l1, w_2),0.), (2, 2)), p_1)\n",
    "        # flatten to switch back to 1d layers - with \"outdim = 2\" (2d) output \n",
    "        l3 = self._dropout(T.flatten(max_pool_2d(T.maximum(conv2d(l2, w_3),0.), (2, 2)), outdim=2), p_1)\n",
    "        l4 = self._dropout(T.maximum(T.dot(l3, w_4), 0.), p_2)\n",
    "        return T.dot(l4, w_5)\n",
    "\n",
    "    def _dropout(self, X, p=0.):\n",
    "        if p > 0:\n",
    "            X *= self.srng.binomial(X.shape, p=1 - p)\n",
    "            X /= 1 - p\n",
    "        return X\n",
    "\n",
    "    def _backprop(self, Cost, w, alpha=0.0001, rho=0.66, epsilon=1e-6):\n",
    "        grads = T.grad(cost=Cost, wrt=w)\n",
    "        updates = []\n",
    "        for w1, grad in zip(w, grads):\n",
    "            # adding gradient scaling\n",
    "            acc = theano.shared(w1.get_value() * 0.0)\n",
    "            acc_new = rho * acc + (1 - rho) * grad ** 2\n",
    "            gradient_scaling = T.sqrt(acc_new + epsilon)\n",
    "            grad = grad / gradient_scaling\n",
    "            updates.append((acc, acc_new))\n",
    "            updates.append((w1, w1 - grad * alpha))\n",
    "        return updates\n",
    "\n",
    "    def _shuffleData(self, p, X, y):\n",
    "        # shuffle it\n",
    "        shuffle = np.random.permutation(np.arange(X.shape[0]))\n",
    "        X, y = X[shuffle], y[shuffle]\n",
    "        # divide \n",
    "        n_train = np.round(X.shape[0]*p)\n",
    "        return X[:n_train], y[:n_train], X[n_train:], y[n_train:] \n",
    "\n",
    "    def fit_predict(self, X, y, x_test, y_test, epochs=1000, miniBatchSize=100):\n",
    "        # Set start of training time\n",
    "        start_time = time.time()\n",
    "        # Set model time\n",
    "        modeltime = datetime.now().strftime(\"%Y%m%d%H%M%S\")\n",
    "        print 'Epochs: %d, MiniBatchSize: %d, TrainingExamples: %d, ModelTime: %s' %(epochs, miniBatchSize, y.shape[0], modeltime)      \n",
    "        # Set epoch variables\n",
    "        min_val_rmse = 100000\n",
    "        train_losses = []\n",
    "        val_losses = []\n",
    "        num_examples_seen = 0\n",
    "        # Divide the data\n",
    "        train_data, train_labels, val_data, val_labels = self._shuffleData(0.9, X, y)\n",
    "        for i in range(epochs):\n",
    "            epoch_start = time.time()\n",
    "            # shuffle training data only\n",
    "            shuffle = np.random.permutation(np.arange(train_data.shape[0]))\n",
    "            train_data, train_labels = train_data[shuffle], train_labels[shuffle]\n",
    "            # run mini-batch gradient descent\n",
    "            for start, end in zip(range(0, len(train_data), miniBatchSize), range(miniBatchSize, len(train_data), miniBatchSize)):\n",
    "                self.cost = self.train(train_data[start:end], train_labels[start:end])\n",
    "            epoch_time = time.time() - epoch_start\n",
    "            # Record training loss\n",
    "            train_losses.append((i, self.cost))\n",
    "            # Make predictions on validation dataset\n",
    "            predicted_labels = self.predict(val_data)\n",
    "            # Calculate validation cost RMSE\n",
    "            val_rmse = np.sqrt(np.mean(np.square(predicted_labels - val_labels)))\n",
    "            val_losses.append((i, val_rmse))\n",
    "            # if we have a new low val_rmse, save the weights and predictions\n",
    "            is_saved = ''\n",
    "            if val_rmse < min_val_rmse:\n",
    "                min_val_rmse = val_rmse\n",
    "                # Make and save predictions on test dataset\n",
    "                predictions = self.predict(x_test)\n",
    "                predictions_and_labels = np.array(zip(predictions, y_test))\n",
    "                filename = \"./predictions/CNNs/pred-%s.txt\" %(modeltime)\n",
    "                np.savetxt(filename, predictions_and_labels, fmt='%.18f', delimiter=',',)\n",
    "                # Save model parameters\n",
    "                filename = \"./models/CNNs/CNN-%s.pkl\" % (modeltime)\n",
    "                pickle.dump(self.params, open(filename, 'w'))\n",
    "                is_saved = '(saved)'\n",
    "            # print epoch results to screen             \n",
    "            print '%d) trainCost = %.4f, validationRMSE = %.4f, trainTime = %.2f min, endTime = %s %s' %(i+1, self.cost, val_rmse, epoch_time/60, time.strftime(\"%I:%M:%S\"), is_saved)\n",
    "            \n",
    "        print 'Finished.  Total train time = %.2f hours' %((time.time() - start_time)/3600)   \n",
    "    \n",
    "        return train_losses, val_losses\n",
    "\n",
    "print 'Model refreshed @ %s' %(time.strftime(\"%I:%M:%S\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epochs: 10, MiniBatchSize: 10, TrainingExamples: 400, ModelTime: 20160428111039\n",
      "1) trainCost = 2.8932, validationRMSE = 2.8432, trainTime = 0.05 min, endTime = 11:10:43 (saved)\n",
      "2) trainCost = 2.3712, validationRMSE = 2.8430, trainTime = 0.05 min, endTime = 11:10:47 (saved)\n",
      "3) trainCost = 2.9539, validationRMSE = 2.8413, trainTime = 0.05 min, endTime = 11:10:51 (saved)\n",
      "4) trainCost = 3.0278, validationRMSE = 2.7893, trainTime = 0.05 min, endTime = 11:10:54 (saved)\n",
      "5) trainCost = 1.0026, validationRMSE = 1.1786, trainTime = 0.05 min, endTime = 11:10:58 (saved)\n",
      "6) trainCost = 0.7696, validationRMSE = 0.9328, trainTime = 0.05 min, endTime = 11:11:02 (saved)\n",
      "7) trainCost = 1.1935, validationRMSE = 0.9276, trainTime = 0.05 min, endTime = 11:11:06 (saved)\n",
      "8) trainCost = 1.0352, validationRMSE = 0.9272, trainTime = 0.05 min, endTime = 11:11:10 (saved)\n",
      "9) trainCost = 1.2046, validationRMSE = 0.9294, trainTime = 0.05 min, endTime = 11:11:13 \n",
      "10) trainCost = 1.1804, validationRMSE = 0.9355, trainTime = 0.05 min, endTime = 11:11:16 \n",
      "Finished.  Total train time = 0.01 hours\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/SendIt/anaconda/lib/python2.7/site-packages/ipykernel/__main__.py:72: DeprecationWarning: using a non-integer number instead of an integer will result in an error in the future\n"
     ]
    }
   ],
   "source": [
    "################### Option 2. Theano 5-layer Model ###################\n",
    "\n",
    "net5 = YieldPredictor(n_output=1)\n",
    "train_losses, val_losses = net5.fit_predict(X=X_train, y=Y_train, x_test=X_test, y_test=Y_test, epochs=10, miniBatchSize=10)\n",
    "# del net5  # Free up memory"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### Baseline Model:  The Mean"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "R-Squared: -0.0142, MAE: 0.7971, RMSE: 1.0014, MAPE: 39.46%\n"
     ]
    }
   ],
   "source": [
    "y_hat = len(Y_test)*[np.mean(Y_train)]\n",
    "y_test = Y_test[:, 0]\n",
    "\n",
    "y_hat = np.array(y_hat).astype(np.float)\n",
    "y_test = np.array(y_test).astype(np.float)\n",
    "y_bar = np.mean(y_test)\n",
    "\n",
    "R_squared = 1 - np.sum(np.square(y_hat - y_test))/np.sum(np.square(y_test - y_bar))\n",
    "mae = np.mean(np.abs(y_hat - y_test))\n",
    "rmse = np.sqrt(np.mean(np.square(y_hat - y_test)))\n",
    "mape = np.mean(abs(np.divide((y_hat - y_test), y_test)))\n",
    "\n",
    "print 'R-Squared: %.4f, MAE: %.4f, RMSE: %.4f, MAPE: %.2f%s' %(R_squared, mae, rmse, mape*100, '%')  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Calculate Performance Stats on Holdout Sample\n",
    "These are the predictions on a holdout sample (X_test above) using the model with the lowest validation loss.    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Results:\n",
      "R-Squared: -108.6960, MAE: 0.8180, RMSE: 1.0415, MAPE: 33.81%\n"
     ]
    }
   ],
   "source": [
    "y_hat = []\n",
    "y_test = []\n",
    "with open('./predictions/CNNs/pred-20160428111039.txt', 'r') as csvfile:\n",
    "    datareader = csv.reader(csvfile, delimiter=',')\n",
    "    for row in datareader:\n",
    "        label = row.pop()\n",
    "        y_hat.append(row)\n",
    "        y_test.append(label)\n",
    "\n",
    "y_hat = np.array(y_hat).astype(np.float)\n",
    "y_test = np.array(y_test).astype(np.float)\n",
    "y_bar = np.mean(y_test)\n",
    "\n",
    "print 'Results:'\n",
    "for j in range(len(y_hat[0])):\n",
    "    y_hat_last = y_hat[:, j]\n",
    "R_squared = 1 - np.sum(np.square(y_hat - y_test))/np.sum(np.square(y_test - y_bar))\n",
    "mae = np.mean(np.abs(y_hat - y_test))\n",
    "rmse = np.sqrt(np.mean(np.square(y_hat - y_test)))\n",
    "count = 0\n",
    "sums = 0\n",
    "for i in range(len(y_test)):\n",
    "    if y_test[i] != 0: \n",
    "        error = np.divide(abs(y_hat_last[i] - y_test[i]), abs(y_test[i]))\n",
    "        count += 1\n",
    "        sums += error\n",
    "mean_abs_pct_error = sums/count\n",
    "print 'R-Squared: %.4f, MAE: %.4f, RMSE: %.4f, MAPE: %.2f%s' %(R_squared, mae, rmse, mean_abs_pct_error*100, '%')  \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Graph Losses"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsQAAAF/CAYAAABOsN+oAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xl01NX9//Hnncm+J5BA9rAjCIICorgErTtau3ytW21d\nqrV1wVZc0CpuUBUqLl8p/tS621r7bVXUusfaVlZBkD0sCUmAEMi+TpL7+yNhshBIQpaZZF6Pc+bw\nWe58Pm9GzvE1d+7nXmOtRURERETEVzk8XYCIiIiIiCcpEIuIiIiIT1MgFhERERGfpkAsIiIiIj5N\ngVhEREREfJoCsYiIiIj4tA4FYmOM0xiz2hjz3mHOP2WM2WqM+dYYM7F7SxQRERER6Tkd7SG+FdgA\nHDJpsTHmfGC4tXYEcD2wqPvKExERERHpWe0GYmNMEnA+8Dxg2mhyEfAygLV2GRBljBnUnUWKiIiI\niPSUjvQQPwHMAuoPcz4R2NVsPwdI6mJdIiIiIiK94oiB2BgzA8i31q6m7d5hd9NW+1oPWkRERET6\nBL92zp8MXNQ4TjgIiDDGvGKtvapZm1wgudl+UuOxFowxCskiIiIi0iustUfqzG3hiD3E1trZ1tpk\na+0Q4FLg81ZhGOBd4CoAY8xUoMhau/cw19Orndf999/v8Rr6ykuflT4nfVb6nLz5pc9Jn5U+J8+9\nOqu9HuJDMm1j8L2hMeAuttZ+YIw53xiTCZQDV3e6ChERERERD+lwILbWfgl82bi9uNW5m7q5LhER\nERGRXqGV6rxMenq6p0voM/RZdYw+p47TZ9Ux+pw6Rp9Tx+mz6hh9Tj3HHM04i6O6kTG2t+4lIiIi\nIr7LGIPtxEN1nR1DLCIiIuIRxnQ434gP6Y4OVwViERER6TP0a7M0111fkjSGWERERER8mgKxiIiI\niPg0BWIRERER8WkKxCIiIiJe4vzzz+fVV1/1dBk+R4FYREREpAvCwsIIDw8nPDwch8NBSEiIe//N\nN9/s1LU++OADfvrTnx5VHWlpaXz22WdH9V5fp1kmRERERLqgrKzMvT1kyBBeeOEFzjjjjEPa1dbW\n4ufXc9HLGKOp6Y6SeohFREREekBGRgZJSUk89thjxMfHc+2111JUVMSMGTOIi4sjJiaGCy+8kNzc\nXPd70tPTeeGFFwB46aWXOOWUU5g1axYxMTEMHTqUf/7zn52uo7q6mpkzZ5KYmEhiYiK33XYbNTU1\nABQUFDBjxgyio6MZMGAAp512mvt9jz76KElJSURERDB69Gg+//zzLn4i3kuBWERERKSH7N27l8LC\nQrKzs1m8eDH19fVce+21ZGdnk52dTXBwMDfddJO7fete3uXLlzN69Gj279/PHXfcwbXXXtvpGh55\n5BGWL1/Ot99+y7fffsvy5ct5+OGHAViwYAHJyckUFBSQn5/PvHnzANi8eTP/+7//y8qVKykpKeHj\njz8mLS2tax+GF1MgFhEREekhDoeDBx54AH9/f4KCgoiJieEHP/gBQUFBhIWFMXv2bL788svDvj81\nNZVrr70WYwxXXXUVu3fvJj8/v1M1vPHGG9x3330MHDiQgQMHcv/997sf3AsICGD37t3s3LkTp9PJ\ntGnTAHA6nVRXV7N+/XpcLhcpKSkMHTr06D8IL6cxxCIiItIvZJiMbrlOuk3vlusAxMbGEhAQ4N6v\nqKjgtttu46OPPqKwsBBoGINsrW1z/O/gwYPd2yEhIe72cXFxHa4hLy+P1NRU935KSgp5eXkAzJo1\nizlz5nD22WcDcP3113PnnXcyfPhwFi5cyJw5c1i/fj3nnHMOf/jDH4iPj+/E377vUCAWERGRfqE7\ng2x3aR1yFyxYwJYtW1i+fDlxcXGsWbOG448//rCBuDskJCSwc+dOjjnmGACys7NJSEgAGmbImD9/\nPvPnz2f9+vWcccYZTJ48mTPOOIPLLruMyy67jNLSUm644QbuvPNOXnnllR6p0dM0ZEJERESkl5SV\nlREcHExkZCQHDhzggQce6Nbr19TUUFVV5X7V1tZy2WWX8fDDD1NQUEBBQQEPPvige2q3JUuWkJmZ\nibWWiIgInE4nTqeTLVu28Pnnn1NdXU1gYCBBQUE4nc5urdWbKBCLiIiI9JDWvb4zZ86ksrKSgQMH\ncvLJJ3Peeecdtme4rWnU2utFPv/88wkJCXG/HnzwQe69914mTZrE+PHjGT9+PJMmTeLee+8FIDMz\nk7POOovw8HBOPvlkfv3rX3P66adTXV3N3XffTWxsLPHx8RQUFLgfuOuPjLW2d25kjO2te4mIiEj/\nY4xBWUKaO9y/icbjHR6Doh5iEREREfFpCsQiIiIi4tMUiEVERETEpykQi4iIiIhPUyAWEREREZ+m\nQCwiIiIiPk2BWERERER8mgKxiIiIiPg0BWIRERERD3I4HGzfvh2AG2+8kYcffrhDbTvr9ddf55xz\nzjmq9/Z3CsQiIiIiXXDuuedy//33H3L8nXfeIT4+nvr6+g5fa9GiRe5llbti586dOByOFve+4oor\n+Oijj7p87dYyMjJITk7u9uv2JgViERERkS74+c9/zmuvvXbI8VdffZUrr7wSh8NzcUtLXXeMArGI\niIhIF3z/+99n//79fPXVV+5jhYWFvP/++1x11VUsX76ck046iejoaBISErj55ptxuVxtXuvnP/85\nv/vd79z7jz/+OAkJCSQlJfHiiy+2aPv+++8zceJEIiMjSUlJ4YEHHnCfO+200wCIiooiIiKCpUuX\n8tJLL3Hqqae62/z3v/9l8uTJREVFMWXKFL7++mv3ufT0dO677z5OOeUUIiIiOOecc9i/f3+nP5uN\nGzeSnp5OdHQ0xx57LO+995773AcffMDYsWOJiIggKSmJBQsWAFBQUMCMGTOIjo5mwIABnHbaaT0e\n7BWIRURERLogODiYSy65hFdeecV97K233uKYY45h3Lhx+Pn58eSTT7J//36+/vprPvvsM5599tk2\nr2WMwRgDwD//+U8WLFjAp59+ypYtW/j0009btA0LC+O1116juLiY999/n0WLFvHOO+8AuMN5cXEx\nJSUlTJ06tcV7Dxw4wAUXXMDMmTM5cOAAv/nNb7jgggsoLCx0t3nzzTd56aWXyM/Pp6amhvnz53fq\nc3G5XFx44YWce+657Nu3j6effporrriCrVu3AnDttdfy3HPPUVJSwvr16znjjDMAWLBgAcnJyRQU\nFJCfn8+8efPcn0lPUSAWERER6aKf/exnvP3229TU1ADwyiuv8LOf/QyA448/nilTpuBwOEhNTeX6\n66/nyy+/bPeab731Ftdccw1jxowhJCSkRQ8wwOmnn87YsWMBGDduHJdeeqn7uu31qL7//vuMGjWK\nK664AofDwaWXXsro0aN59913gYZgfvXVVzN8+HCCgoK45JJLWLNmTac+k6VLl1JeXs5dd92Fn58f\n06dPZ8aMGbzxxhsABAQEsH79ekpKSoiMjGTixInu47t372bnzp04nU6mTZvWqfseDQViERER6ReM\n6Z7X0Zg2bRoDBw7k73//O9u2bWPFihVcfvnlAGzZsoUZM2YQHx9PZGQk99xzT4eGH+zevbvFw2op\nKSktzi9btozp06cTFxdHVFQUixcv7vCwhry8vEOul5qaSl5ennt/8ODB7u3g4GDKyso6dO3m92j9\nsF1qaiq5ubkA/O1vf+ODDz4gLS2N9PR0li5dCsCsWbMYPnw4Z599NsOGDePRRx/t1H2PRruB2BgT\nZIxZZoxZY4zZYIyZ10abdGNMsTFmdeOr649HioiIiHSCtd3zOlpXXXUVr7zyCq+99hrnnnsusbGx\nQMNUamPGjCEzM5Pi4mIeeeSRDs08ER8fT3Z2tnu/+TbA5ZdfzsUXX0xOTg5FRUX88pe/dF+3vSEG\niYmJZGVltTiWlZVFYmJih/6uHZGYmMiuXbta9FZnZWWRlJQEwKRJk/jHP/7Bvn37uPjii7nkkkuA\nhqEg8+fPZ9u2bbz77rv84Q9/4PPPP++2utrSbiC21lYB0621E4DxwHRjzCltNP3SWjux8XX4CfRE\nRERE+qGrrrqKTz75hOeff949XAKgrKyM8PBwQkJC2LRpE4sWLTrsNay17gB5ySWX8NJLL7Fx40Yq\nKioOGTJRVlZGdHQ0AQEBLF++nDfeeMMdhGNjY3E4HGzbtq3N+5x33nls2bKFN998k9raWv7yl7+w\nadMmZsyY0aKWzqiurqaqqsr9mjx5MiEhITz22GO4XC4yMjJYsmQJl156KS6Xi9dff53i4mKcTifh\n4eE4nU4AlixZQmZmJtZaIiIicDqd7nM9pUNDJqy1FY2bAYATONBGs54d7SwiIiLixVJTU5k2bRoV\nFRVcdNFF7uPz58/njTfeICIiguuvv55LL720RQ9u6+2D++eeey4zZ87kjDPOYOTIkZx55pkt2j77\n7LPcd999RERE8NBDD/GTn/zEfS4kJIR77rmHadOmERMTw7Jly1pce8CAASxZsoQFCxYwcOBA5s+f\nz5IlS4iJiWm3rtaMMeTm5hIcHExISAghISGEhoaSm5vLe++9x4cffkhsbCw33XQTr776KiNHjgTg\ntddeY8iQIURGRvLcc8/x+uuvA5CZmclZZ51FeHg4J598Mr/+9a85/fTTO/8fpBNMR9K/McYBfAMM\nAxZZa+9odf504P+AHCAXuN1au6FVG6u58ERERORoGWM0r660cLh/E43HO9xZ29Ee4vrGIRNJwGnG\nmPRWTb4Bkq21xwFPA//oaAEiIiIiIp7k15nG1tpiY8z7wCQgo9nx0mbbHxpjnjXGxFhrWwytmDNn\njns7PT2d9PT0o6taRERERKRRRkYGGRkZR/3+dodMGGMGArXW2iJjTDDwEfCAtfazZm0GAfnWWmuM\nmQK8Za1Na3UdDZkQERGRo6YhE9Jadw2Z6EgPcTzwcuM4YgfwqrX2M2PMDQDW2sXAj4EbjTG1QAVw\naUcLEBERERHxpA49VNctN1IPsYiIiHSBeoiltV59qE5EREREpL9SIBYRERERn9apWSZEREREPKm9\nJYlFjoYCsYiIiPQJGj8sPUVDJkRERETEpykQi4iIiIhPUyAWEREREZ+mQCwiIiIiPk2BWERERER8\nmgKxiIiIiPg0BWIRERER8WkKxCIiIiLi0xSIRURERMSnKRCLiIiIiE9TIBYRERERn6ZALCIiIiI+\nTYFYRERERHyaArGIiIiI+DQFYhERERHxaQrEIiIiIuLTFIhFRERExKcpEIuIiIiIT1MgFhERERGf\npkAsIiIiIj5NgVhEREREfJoCsYiIiIj4NAViEREREfFpCsQiIiIi4tMUiEVERETEpykQi4iIiIhP\nUyAWEREREZ+mQCwiIiIiPk2BWERERER82hEDsTEmyBizzBizxhizwRgz7zDtnjLGbDXGfGuMmdgz\npYqIiIiIdL8jBmJrbRUw3Vo7ARgPTDfGnNK8jTHmfGC4tXYEcD2wqKeKFZGOs/WWmoIa6irrPF2K\niIiIV/Nrr4G1tqJxMwBwAgdaNbkIeLmx7TJjTJQxZpC1dm+3Viri42y9xXXAhWufC1e+i5p9NQ3b\n+1zU5B+6XXugFkeIg+DhwZyw8gSMMZ7+K4iIiHildgOxMcYBfAMMAxZZaze0apII7Gq2nwMkAQrE\nIkfgDrj5jUF2X03L7VbBt/ZALc5wJ/6x/gTEBeAf6+/eDhkZgv+0lsf9B/pj/Awrxq6gKKOI6OnR\nnv4ri4iIeKWO9BDXAxOMMZHAR8aYdGttRqtmrbuebDfVJ9Jn2LpmPbite21bh918F64DLvwi/RpC\nbWwA/nH+7u2QkSH4n9J4PNa/4dwAfxz+nX8ONunWJHKeyFEgFhEROYx2A/FB1tpiY8z7wCQgo9mp\nXCC52X5S47FDzJkzx72dnp5Oenp6xysV6WXNA25bQxIOGaJQWIsz4jA9uKOaBdzG4Hu0AbezBv10\nEDvu3UHF1gpCRoT0+P1ERER6W0ZGBhkZGUf9fmPt4TtzjTEDgVprbZExJhj4CHjAWvtZszbnAzdZ\na883xkwFFlprp7ZxLXuke4n0NHfAzW97SELrcbmuAy78ovyaemnbCLrNt/0G+OHw886ZDLfP3k5d\nWR0jnhrh6VJERER6nDEGa22HH55pLxCPo+GBOUfj61Vr7ePGmBsArLWLG9s9A5wLlANXW2u/aeNa\nCsTSrWydxbX/CEMSWo3FrS2qxRnpPGR4QovtxuEJAbHeHXA7qzq3mhXjVnDi9hPxj/L3dDkiIiI9\nqlsDcXdSIJbuUvpNKVtu2ELZmjL8ovyaHiJrrwc3pv8E3KOx4YoNhB8fTvJvk9tvLCIi0ocpEEu/\nVVdVR9aDWex+YTfD5g9j0OWDME5NJdZRJctLWH/Jek7MPNGnvxiIiEj/19lArP8rSp9Q/HUxqyau\nomJLBZPXTmbwTwcrDHdSxJQIAhMDKfhHgadLERER8SoKxOLV6irqyPxNJut/tJ60h9I49u1jCRgU\n4Omy+qyk25LIWZjj6TJERES8igKxeK3CLwpZMW4Frn0uJq+bTNyP4zxdUp838OKBVOdUU7KixNOl\niIiIeI1eHUNcmV1JUHJQr9xP+q7aklq237md/Uv2M2LRCAbOGOjpkvqVXQt2UfpNKWNeH+PpUkRE\nRHqEV48hXjlxJVmPZFFXVdebt5U+ZP8/97Ni3ApsnWXyd5MVhnvA4GsHc+DDA1TnVnu6FBEREa/Q\nq4H4hBUnULqqlBVjV1DwTgGadUIOchW62HT1JrbeuJVRL4xi1HOj8Ivs8EKK0gn+Uf4MumIQuf/b\n5oKSIiIiPscj064d+OQAmbdmEpgSyPCFwwkdHdorNYh32vePfWz99VZifxTLkLlD8AtTEO5pFVsr\nWH3yaqZmTcUZ4vR0OSIiIt3Kq+ch/uEPm+5l6y1V2yqp2FxBYGoQoaNDMP5td1ibLsyu1Zff296f\nnWnrjdeoq6ij6ItCXHtdxJwT7R5f3tFrhIRAZOSRX37K1oe17qJ1DJgxgITrEzxdioiISLfy6kD8\n9tuH3qu2yMXe1/MpXV3KoCsGEXV6FMbRVH9XyuvL723vz8609bZr1NdbytaUse+dAiJOCCf6nBhM\n40IRnblWeTkUFx/6Kilp+jMwsP3Q3N7Lv5+udFz4eSFbb9rK5PWTMV35BiciIuJlvDoQH+leJctL\n2HrzVjAw4ukRREyO6JW6pHdV51Wz5VdbqMysZPSfRvfof+cjhebOvAICuh6qA7xw6mRrLSsnrGTY\nY8OIOSfG0+WIiIh0mz4biKFhGMWel/ewY/YOYi6IYejcoQTEeWGSkE6z1rLnpT1sv3M7CTcmkDo7\nFUeg90+DbS1UVHQ9VPv5dT1UBwZ2/99v9592s++tfYz/cHz3X1xERMRD+nQgPqi2uJadD+5k7yt7\nSb03lYRfJeA4zPhi8X5VWVVsvn4zrn0uRv9pNGHHhXm6pF5lLVRWdj1UOxxHF6RHjICoqLZrq6uq\nY2naUiZ8MYHQY/Rwq4iI9A/9IhAfVL6xnMxbM6nOq2bEkyOIPjO6h6qTnmDrLXmL89h5306SfpNE\n8u3J+mJzlKyFqqrOh+jCwoZhI999d/ix0Dvm7KBmTw2j/jiqd/9SIiIiPaRfBWJo+Km94J0Ctt22\njbATwhg2fxjBacE9UKF0p4rMCjZftxlbbRn14ij1PnrQOefAhRfCTTe1fb56TzUrjlnBiZkn4j+g\nnz5BKCIiPqXfBeKD6irr2DV/FzkLc0i6JYnkO5JxBmv+VG9j6yw5T+aQNTeL1HtTSbo5CePUDAae\ntHYtnHUWbNnSMISiLRt/vpGQUSGk3p3au8WJiIj0gH4biA+qyqpi26xtlCwvYfiC4Qz84UBNGeUl\nyjeUs+maTTiDnYx6fhTBw9ST7y2uuQYGDYJ589o+X7qmlHUz1jF1x1QNaxERkT6v3wfigwq/KCTz\nlkz8B/kz4skRhI7VT/KeUu+qZ9fju8h5Ioe0h9JIuD6hxVzS4nm5uTB+PKxZA8nJbbdZM30N8b+I\nZ9Dlg3q3OBERkW7mM4EYoL62nrxFeWQ9mEXcFXGkzUnDP0pjIHtT6ZpSNl+9Gf9B/ox6bhRBKUGe\nLkkO4957YdcuePnlts8XvFtA1kNZHL/8eP3qIiIifVpnA3Gf/m3U4ecg6eYkJm+YTH1lPctHL2f3\nC7ux9b0T8n1ZfXU9O363g7VnryVpZhLjPxyvMOzl7rwTPv4YVq9u+/yACwbgKnRR8nVJ7xYmIiLi\nYX26h7i10lWlbL15K9ZlGf70cCKnHuYJIumSkmUlbLpmE8Ejghm5aCSB8T2wYoT0iEWL4G9/g08+\ngbY6gXOeyqH4q2LG/nVs7xcnIiLSTXxqyERbrLXsfX0v2+/cTvRZ0Qz9/VACByuwdYe6ijp23LeD\nva/tZcSTI4i9JFY/rfcxLheMGwdPPAHnnXfo+drSWpamLWXSN5MISlWPv4iI9E0+NWSiLcYYBl85\nmCmbphAwKIAVx65g14Jd1NfUe7q0Pq3oX0WsPG4lNXk1TF43mbifxCkM90H+/vDoozBrFtTWHnre\nL9yPwT8fTO4zub1fnIiIiIf0ux7i1io2V5A5M5PKHZWMeHIEMefE9HoNfVltaS3b795Owd8LGPns\nSAZ+f6CnS5IushbS0+GnP4Xrrjv0fOXOSladsIqpWVPxC/Pr9fpERES6yueHTLTFWsv+JfvJvC2T\n0GNDGf6H4QQP1Ry57TnwyQE2/2Iz0WdEM2zBMPyjNYNHf7FiBVx8ccNiHaFtzFj43Y+/Iyo9iqSb\nknq/OBERkS5SID6Cuqo6cp7IYdf8XSTcmEDq3ak4Q7XaXWuuIhfbfruNws8KGbV4lHrV+6nLLoNj\njoH77jv0XNG/i9h89WambJ6iOaVFRKTP8fkxxEfiDHKSencqk76dRNX2KpYfs5z8v+Tj6aDuTQre\nLWDFsStwBDmYvG6ywnA/NncuPPUU7Nlz6LnIaZE4I53sf39/7xcmIiLSy3yqh7i1oq+K2HrzVvyi\n/Bjx1AjCxod5uiSPqSmoIfPWTEqWlTD6hdFEnR7l6ZKkF9x+O5SWwuLFh57b+/pedr+4mwmfTej9\nwkRERLpAPcSdEHVqFJNWTSLuJ3F8+71v2XLTFlwHXJ4uq1dZa8l/K5+V41YSMDiAyWsnKwz7kHvu\ngb//HTZsOPRc7P/EUrGpgrK1Zb1fmIiISC/y6R7i5lz7Xey4bwf73t7HkAeHEH9dPMbZv8dOVu+p\nZuuvtlKxqYJRL47SQiY+6g9/gC++gPfeO/Rc1twsKjMrGf3i6N4vTERE5CjpobouKl1TSuYtmdSV\n1THi6RFETut/IdFay95X97Lt9m3EXx9P2u/ScAT69I8FPq26uuHhuhdegOnTW56rKahh+YjlTNk8\nhYC4AM8UKCIi0kkKxN3AWkv+X/LZPms7kadHMuzRYQQm9o/V7qp2VbHlhi1U51Uz+k+jCZ8Y7umS\nxAv85S/w2GMN07E5Wn032nz9ZgKTAkm7L80jtYmIiHSWxhB3A2MMgy4dxOSNkwlKDWLFcSvIfjSb\n+uq+u9qdrbfkLc5j1fGriDg5ghNWnKAwLG6XXAJ+fvDmm4eeS5qZRN6ivD79719ERORI2u0hNsYk\nA68AcYAFnrPWPtWqTTrwDrC98dDfrLUPt2rTZ3qIW6vIrGDbb7ZRsbGC4QuHM+CCAZ4uqVMqt1ey\n+brN1JXVMfpPowkd28ZKDOLzvvqqYfW6TZsgKKjluW/P+ZZBlw9i8M8Ge6Y4ERGRTuj2IRPGmMHA\nYGvtGmNMGLAKuNhau7FZm3TgN9bai45wnT4biA/a/+F+MmdmEjwimOFPDCdkRIinSzoiW2fJfSaX\nnQ/tJOWuFJJmJuHw048Ccng/+AGcdBLccUfL4/s/3M/2u7czafUkjOnfD5uKiEjf1+1DJqy1e6y1\naxq3y4CNQEJb9+5wlX3UgPMGMHldw7Rk35z0Ddvu2kZtWa2ny2pT+aZyVp+2mn1/28fxXx9Pyu0p\nCsPSrt//Hh5/HPa3Wo8j5pwYbLWl6MsizxQmIiLSgzqVkIwxacBEYFmrUxY42RjzrTHmA2PMmO4p\nz/s4AhykzEph8rrJ1OTVsHz0cva+vtdrVrurr60n6/dZrD5lNYMuH8SEjAle35Mt3mPUqIbxxA89\n1PK4cRgSb00kZ2GOZwoTERHpQR2eZaJxuEQG8LC19h+tzoUDddbaCmPMecCT1tqRrdr0+SETbSn+\nbzFbb96KI9jBiKdHePRBtbK1ZWy6ZhP+0f6M/H8jCU4L9lgt0nfl58OYMbB0KQwf3nS8rryOpWlL\nmfj1REKG60uWiIh4rx6Zds0Y4w8sAT601i7sQPsdwAnW2gPNjtn777/f3SY9PZ309PSO1unVbJ1l\n94u72fG7HQy8eCBDHh5CwMDem7O1vqaerLlZ5D2bx9B5Qxl8zWCN85QumTsXVq+Gv/615fHtd2+n\nrqKOEU+O8ExhIiIibcjIyCAjI8O9/8ADD3T7Q3UGeBnYb6297TBtBgH51lprjJkCvGWtTWvVpl/2\nEDfnKnSxc85O8t/MJ+3+NOJviO/xcbslK0vYfPVmgtKCGPnHkf1mvmTxrIqKhuETb73V8JDdQVU5\nVawcv5KpO6biF+nnuQJFRESOoCdmmTgF+BewloaxwgCzgRQAa+1iY8yvgRuBWqCChhknlra6Tr8P\nxAeVrSsj89ZMXPtdjHhqBFGnR3X7Peoq69g5Zyd7XtrD8CeGE3dZnHqFpVu9/DI89xz8+9/Q/J/W\nhss2ED45nOTfJHuuOBERkSPQSnVewlrLvrf3se32bUScFMGwx4cRlBzU/hs7oOjfRWy+djNhx4Ux\n4pkRWlJXekRdHZxwAvzud/CjHzUdL1lewvpL1nNi5omauURERLySVqrzEsYY4v4njikbpxAyKoSV\nE1eS9UgWdVV1R33N2rJatt6ylQ2XbGDovKGMfWuswrD0GKezYQq2u+6Cmpqm4xFTIghMCGT/u/sP\n/2YREZE+RIG4hzlDnAx5YAgnrDiB0lWlrBi7goJ3Cjo9TVvhZ4WsHL+S2uJaJn83mdgfxvZQxSJN\nzjoLhg2/iPoxAAAgAElEQVSDxYtbHk+6LYmcJzQFm4iI9A8aMtHLDnxygMxbMwlMCWT4wuGEjj7y\nMsq1xbVsm7WNA/88wMg/jmTA+X1r2Wjp+9atg+99DzZvhqjG4fD1tfUsG7qMsf83lohJEZ4tUERE\npBUNmfByMWfFMOnbScScE8PqU1aTeXsmtSVtr3a3//39rDh2BThg8neTFYbFI8aNgxkzGlaxO8jh\n5yDx5kRyn8z1XGEiIiLdRD3EHlSzt4btd2/nwD8PMHTeUAb9dBDGYXDtd5E5M5Pi/xQz6vlRRJ8R\n7elSxcfl5sL48Q1zE6ekNBxzFbpYNmwZk7+bTGCCpvsTERHvoVkm+qCS5SVsvXkrxmGIuyKO7LnZ\nxF4Sy9BHhuIMdXq6PBEA7rsPduyAV19tOrblpi34Rfkx9OGhnitMRESkFQXiPsrWW/a8sof8P+eT\n9rs0IqdFerokkRZKS2HkSHj/fTj++IZjFVsqWH3KaqZmTcUZrC9vIiLiHRSIRaTH/PGPDcs5f/pp\n02Id6y5cx4CLBpDwiwTPFiciItJID9WJSI+57jrIy4MPP2w6ljQziZyFOZ2eSlBERMRbKBCLSIf5\n+cFjj8GsWVDbODlK1BlRGKeh8JNCzxYnIiJylBSIRaRTZsyA2Fj4058a9o0x7l5iERGRvkhjiEWk\n01auhIsugi1bICwM6qrqWJq6lAlfTmh3sRkREZGepjHEItLjJk2C6dNh/vyGfWeQk4RfJmihDhER\n6ZPUQywiR2XnTjjhBPjuO4iPh+o91aw4ZgUnbjsR/xh/T5cnIiI+TD3EItIr0tLgmmvg/vsb9gMH\nBzLgogHs/n+7PVqXiIhIZ6mHWESOWmEhjBoFX3wBY8dC6epSvrvoO07cfiIOf33fFhERz1APsYj0\nmuhomD0b7rijYT98YjhBw4LY97d9ni1MRESkExSIRaRLfvUr2LQJPv+8YV9TsImISF+jQCwiXRIQ\nAPPmwe23Q309DLxwIK59Loq/LvZ0aSIiIh2iQCwiXfY//9MQjF9/HYzTkHSLeolFRKTv0EN1ItIt\n/vMfuPzyhuET/q5alg5ZyqTVkwhKCfJ0aSIi4mP0UJ2IeMS0aQ3zEj/1FPhF+DH4Z4PJfUYLdYiI\niPdTD7GIdJstWxqC8caNEFpayarJq5i6cyp+YX6eLk1ERHyIeohFxGNGjoRLL4UHH4TgIcFEnR7F\n3pf3erosERGRI1IPsYh0q3374Jhj4OuvIXZPEZuv3cyUTVMwjg5/URcREekS9RCLiEfFxsJvfwt3\n3w2Rp0TiDHdy4MMDni5LRETksNRDLCLdrrKyYUnnN9+EYTv2sOelPUz4dIKnyxIRER+hHmIR8bjg\nYHj4YZg1C2L/J46KDRWUrSvzdFkiIiJtUiAWkR5x5ZUNPcV/f89Bwq8SyHlSC3WIiIh3UiAWkR7h\ncMD8+XDXXTDw6gQK/lZATX6Np8sSERE5hAKxiPSYM89smIrthbcDiP1xLHmL8zxdkoiIyCH0UJ2I\n9KjvvmsIxqv+Xkb2j9YydedUHIH6Li4iIj1HD9WJiFc59li48EJ4+p0wQseFkv+XfE+XJCIi0kK7\ngdgYk2yM+cIYs94Y850x5pbDtHvKGLPVGPOtMWZi95cqIn3Vgw/C88+DvTyFnIU56NciERHxJh3p\nIXYBt1lrxwJTgV8bY45p3sAYcz4w3Fo7ArgeWNTtlYpIn5WQADfdBPM/iaKuoo7ir4o9XZKIiIhb\nu4HYWrvHWrumcbsM2AgktGp2EfByY5tlQJQxZlA31yoifdisWfD5F4YDPxhCzhOagk1ERLxHp8YQ\nG2PSgInAslanEoFdzfZzgKSuFCYi/UtYGMyZA4/+J5bCfxVRub3S0yWJiIgA4NfRhsaYMOBt4NbG\nnuJDmrTaP2SQ4Jw5c9zb6enppKend/T2ItIPXHMNLFxo2DR9GIOfymHEwhGeLklERPqBjIwMMjIy\njvr9HZp2zRjjDywBPrTWLmzj/B+BDGvtnxv3NwGnW2v3NmujaddEhCVLYNZt9Swq+C+nZE3FL6LD\n38tFREQ6pNunXTPGGOAFYENbYbjRu8BVje2nAkXNw7CIyEEXXACDkxxkDB/K7hd3e7ocERGR9nuI\njTGnAP8C1tI0DGI2kAJgrV3c2O4Z4FygHLjaWvtNq+uoh1hEAFi1CmacW89roSs5Y9tkjLPDX+JF\nRETa1dkeYq1UJyIeceWVEJKxm0ee9iP2B7GeLkdERPoRBWIR6ROysmDisfX8eex6zl46ztPliIhI\nP6Klm0WkT0hNhet+aXjmuzhKvyn1dDkiIuLD1EMsIh5TVAQjkut4IT2Li94b6ulyRESkn1APsYj0\nGVFRcPdsmPfPKKp3V3u6HBER8VEKxCLiUTf91kleSBhv3X7A06WIiIiPUiAWEY8KCIC5D1oeeisc\nV3mdp8sREREfpEAsIh53+S2BhIQZFt2ih+tERKT3KRCLiMcZA7+fU8fcV4OpqNDDtyIi0rsUiEXE\nK5xzSzhjg8uZd3Olp0sREREfo0AsIl7BGMODd9byzKsB7Nvn6WpERMSXaB5iEfEadZV1XDpgLwN/\nGMui1/w9XY6IiPRRmodYRPosZ7CTu2508ee/OdiyxdPViIiIr1APsYh4lerd1cwcmsfuM1P5xxJ9\nZxcRkc5TD7GI9GmB8YFc/4NqVv6nnn//29PViIiIL1AgFhGvM/z2RK5x7OD231r0w5KIiPQ0BWIR\n8Trhx4fz/bFlVOTX8te/eroaERHp7xSIRcQrpdyWxK+Cd3D33VBd7elqRESkP1MgFhGvNPCigUyo\nOsDwwS4WLfJ0NSIi0p8pEIuIVzJOQ+ItifwqIou5c6Gw0NMViYhIf6VALCJeK/6aeAYs38OF36tl\n7lxPVyMiIv2VArGIeC2/CD8GXzWY62JyePFF2LnT0xWJiEh/pIU5RMSrVW6vZNWUVXx8/clsy3Lw\n+uuerkhERLydFuYQkX4leGgwUadGcWXsHr74Alau9HRFIiLS3ygQi4jXS7oticI/7mLO/Zbbb0eL\ndYiISLdSIBYRrxd5aiTOUCffTzjAvn2wZImnKxIRkf5EgVhEvJ4xhqSZSex5OofHHoM77oDaWk9X\nJSIi/YUCsYj0CXE/iaP8u3JOSykjIQGef97TFYmISH+hWSZEpM/Y+dBOqrOrKb9xFBdcAFu2QHi4\np6sSERFvo1kmRKTfSrghgX1v7+PY5Bq+9z14/HFPVyQiIv2BeohFpE/ZdN0mgtKCcFyVxsSJsHYt\nJCZ6uioREfEm6iEWkX4t6dYk8p7NI2lwPb/4Bdx3n6crEhGRvk49xCLS53x71rcM+tkggi8czMiR\n8OmnMG6cp6sSERFv0e09xMaYF40xe40x6w5zPt0YU2yMWd34urczBYuIdFbSzCRynsghIsJyzz0N\n07CJiIgcrY4MmfgTcG47bb601k5sfD3cDXWJiBxWzHkx1JXVUfzvYn75S8jMhE8+8XRVIiLSV7Ub\niK21XwGF7TTrcJe0iEhXGYch6dYkchbmEBAAv/89zJoFdXWerkxERPqi7niozgInG2O+NcZ8YIwZ\n0w3XFBE5okFXDaLoyyIqd1Tywx9CaCi89pqnqxIRkb6oOwLxN0CytfY44GngH91wTRGRI/IL8yP+\nmnhyn87FGJg/H+69FyoqPF2ZiIj0NX5dvYC1trTZ9ofGmGeNMTHW2gOt286ZM8e9nZ6eTnp6eldv\nLyI+LPGmRFZOXEnanDROOsmPqVNh4UKYPdvTlYmISG/KyMggIyPjqN/foWnXjDFpwHvW2kMmNjLG\nDALyrbXWGDMFeMtam9ZGO027JiLdbv1P1hN5ciRJtyaRmQlTp8KGDRAX5+nKRETEUzo77Vq7gdgY\n8yZwOjAQ2AvcD/gDWGsXG2N+DdwI1AIVwG+stUvbuI4CsYh0u+Kvi9l45UZO3HIixmm49daGh+ue\necbTlYmIiKd0eyDuLgrEItJTVk1dRcpdKcReHEtBARxzDPz73zBqlKcrExERT9DSzSLic5JmNkzB\nBjBwYMMUbHfd5eGiRESkz1AgFpE+L/ZHsVRtq6J0dcMzvrfcAt98A1995eHCRESkT1AgFpE+z+Hv\nIPGmRHcvcVAQzJ0Lt98OGqklIiLtUSAWkX4h/hfx7H93P9V7qgG47DKorYW33vJwYSIi4vUUiEWk\nX/CP8SfusjjyFuUB4HA0LNZx991QXe3h4kRExKspEItIv5F4SyJ5f8yjrqoOgOnTYcwYePZZDxcm\nIiJeTdOuiUi/svb8tcT+OJb4a+KBhkU60tNh82aIjvZsbSIi0js07ZqI+LSk2xqmYDv4BXzMGPjB\nD+CRRzxcmIiIeC0FYhHpV6K/F42tsxR9XuQ+9sAD8Kc/wY4dHixMRES8lgKxiPQrxpgWC3UADB4M\nt94Ks2d7sDAREfFaCsQi0u8MunIQJctKqNha4T7229/Cv/4FK1Z4sDAREfFKCsQi0u84g53EXx9P\nzpNNvcShoQ1DJ7RYh4iItKZALCL9UuKvEsl/Ix9Xoct97OqrYf9+ePddDxYmIiJeR4FYRPqlwIRA\nBlwwgN3P73Yfczrh8cfhzjvB5TrCm0VExKcoEItIv5U0M4ncp3Opr613Hzv3XEhKguef92BhIiLi\nVRSIRaTfCj8hnKC0IAr+XuA+ZkxDL/GDD0JJiQeLExERr6FALCL9WtLMJHKeyGlxbOJEOPtseOwx\nDxUlIiJeRUs3i0i/Zussy4YvY8yfxxBxYoT7+K5dMGECrF0LiYkeLFBERLqdlm4WEWnGOA2JtyS2\nmIINIDkZrr8efvc7DxUmIiJeQz3EItLv1RbXsnTIUiatnURQUpD7eHExjBwJH38Mxx3nwQJFRKRb\ndbaHWIFYRHzC1lu34gxxMnTe0BbHn3kG3nsPPvrIQ4WJiBxBXUUd5evKKV1dSvWuahwBDkygwRHo\ncL/a3Q8wbZ4zjg7nxT5HgVhEpA2V2yr5Zuo3TM2aijPE6T7ucsHYsQ3B+OyzPVig9CvFS4vZ9/Y+\ngocGEzo+lLBxYfhF+nm6LPFyrgMuylaXUbq6lLLVZZStKaNqRxUho0MImxBG0JAgrMtSX11PfXU9\ntrrZdk3Tdutzh9s3fqbz4bqbwvjh2hrTPSFdgVhE5DDWXbyOmHNjSPxly6fo/u//YM4ceOcdGDLE\nM7VJ32etpejzIrLmZlG5rZLBPxtMTV4NZWvLKF9fjv8Af0LHhRI2PswdkoNHBuPw1+M8vsZaS/Wu\n6kPCb21hLWETwhpeExteoWNCcQR0/78Ra+1hw3WX9muO/r3WZTsXoI+wP/ShoQrEIiJtKcwoZOuN\nW5m8fnKLnwqthdtugz//GSIi4JxzGl7p6RAW5rl6pW+w9Zb97+0na24WdSV1pNydQtxlcS2Crq23\nVG6vpHxdOeVryxtC8rpyqndVEzwquEVIDh0fSsDggG7rKRPPsnWWis0Vh4RfR4CjIfQ2C7/BQ4P7\n9TCG9th626VA3Xx/yJwhCsQiIm2x1rLq+FUMmTeEAecOOOR8fX3DNGwffdTwWrECJk1qCsjHHQcO\ndeZJo/raevb9ZR9Z87JwBDpInZ3KwB8M7FSgqSuvo3xDy5BctrYMoCEkN+tRDh0b2mK4j3ifusqG\n8b7Nw2/5d+UEJgQeEn4DBwd6utx+TUMmRESOYM/Le9j7xl6O+6j9aSXKyyEjoykgFxXBWWc1hOOz\nz4ZBg3q+XvE+dVV17H15L9mPZhOYHEjq7FSiz47uth5day01e2oOCcmVmysJTA5sGZLHhfp8r6Kn\nuA64KFtT1iL8Vu2oImRUSMvwe1wYfhEaP97bFIhFRI6gvrqepWlLOe7T4wgdG9qp9+7c2TBF20cf\nweefQ1paQzA+5xyYNg0C1eHTr9WW1bL7ud3sWrCLsAlhpNydQtQpUb12/3pXPZVbKluE5PJ15bj2\nuwgd2zIkh40Lw3+Af6/V1p+5x/u2Cr+1hbWEHdfU4xs2IYzQsT0z3lc6T4FYRKQdOx/cSXVONaOe\nG3XU16ithWXLGsLxxx/Dxo1w6qlNwytGjAANAe0fXAdc5D6TS+4zuURNjyLlrhTCJ4Z7uiw3V5Gr\nYWxys5Bcvq4cZ7izZUgeH0bI6BAFtiNwj/dtHn7XlGH8DOETw1uE3+Bh6pn3ZgrEIiLtqMmvYfmo\n5UzZOoWAgQHdcs0DB+DTT5uGV/j5NYXjM8+EyMhuuY30ourd1eQ8kcPuF3Yz8OKBpNyRQsioEE+X\n1SHWWqqyqloMuyhfW07VziqChwe3CMmh40IJTAr0uYf43ON9m4Xf8u/KCRgccEj4DYzXzz99jQKx\niEgHbLp2E8HDgkmdndrt17a2ocf4YDj+z39g/PimgDxpEjj1bJTXqtxRya7Hd5H/53wG/XQQyb9N\nJiglqP039gF1lXVUbKxoEZLL1pVhq+0hITn02FD8wvvH2FdXYcP8vgdneChdXUrVtiqCRwW3DL/j\nNV90f6FALCLSAWVry1h73lqm7pja4z8hV1bCV181jT/Oy4Pvfa9p/HFSUo/eXjqofEM52b/PZv8H\n+0m4IYGkW5MIiOueXxC8Xc3eGsrWtQzJFRsqCIgPaBGSw8aHETw8GOP0zt5kay3VOdWHhN/a/bWE\nHhfaIvyGjgnFEajhI/2VArGISAetOXMN8dfEM+iK3p0uIje3IRx//DF88knDbBUHe49POw2Cg3u1\nHJ9XsrKE7LnZFP+nmKRbk0j4VQL+UXogrb62nsrMyhYhuXxtOTV7aggZE9JyWrhxob3+5cHWWSq2\nVBwSfo3TEDYxrEX41Xhf36NALCLSQQVLCsh6IIvjlx/vsfGTdXXwzTdNwyvWrIGTTmoKyGPH6uG8\nnmCtpfhfxWTNzaJiQwXJs5KJvy5e8/x2QG1pLeXftQzJZWvLcAQ5DgnJIWNCcAZ1/TOtq2qa3/dg\n+C1bV0bAoIBDwq8WNRHogUBsjHkRuADIt9aOO0ybp4DzgArg59ba1W20USAWEa9i6y3LRy1n1J9G\n9er0WUdSUtIwpdvBgFxd3TS04qyzYMCh64lIJ1hrOfDBAbLmZuHa5yLlrhQGXTlIMy900cGhCq1D\nctW2KoKGBLUcmzw+lKDUoMOGVldh0/y+B8NvZWYlwSODW4bf4zTeVw6vJwLxqUAZ8EpbgdgYcz5w\nk7X2fGPMicCT1tqpbbRTIBYRr5PzTA5FGUUc+/axni7lENZCZmbT2OMvv4RRo5oC8tSp4K9f9jvE\n1ln2vb2PrLlZYCB1diqxP4r12rGw/UV9dT0VmypahOTydeXUldURemxjSD42FNd+lzv8ugpcDctY\nNwu/oWM13lc6p0eGTBhj0oD3DhOI/wh8Ya39S+P+JuB0a+3eVu0UiEXE69SW1bI0bSknrDyB4DTv\nHrxbUwP//W/T3MfbtsH06U0r5w0d6ukKvU99TT17X91L9u+z8Y/zJ/WeVGLOi9FP6h7m2u9yh+Ty\n78rxi/Fzh9/g4RrvK13niUD8HjDPWvvfxv1PgTuttatatVMgFhGvtG3WNmy9ZfiC4Z4upVPy8xse\nyjsYkMPDm8YeT58OYWGertBz6srr2P38bnbN30XI2BBSZ6cSeWqkgrCIj+hsIO6uwTetb6jkKyJ9\nRuJNiaw8fiUhI0OIOjOq4Yn0PhCc4uLgiisaXvX1sG5dQzheuBAuvxxOOKEpIE+YAA4f+MXZVeQi\n73/zyHkqh8hTIhn797FETIrwdFki4uW6IxDnAsnN9pMajx1izpw57u309HTS09O74fYiIl0TlBrE\n6JdGs++v+9j54E6MnyH6zGiiz4wm6oyoPrFKlcMBxx3X8LrjDigvh4yMhp7jyy+HwsKGh/LOPrvh\nNXiwpyvuXjX5NeQszCFvcR4DZgxgwhcTCB0T6umyRKSXZGRkkJGRcdTv744hE80fqpsKLNRDdSLS\nV1lrqdhcQdFnRRR+WkhRRhEBCQFNATk9qk8+2Z6V1TS04rPPIDW1qfd42jQI9P7M36aq7Cp2zd/F\n3tf2EndZHMmzkr1+LLiI9LyemGXiTeB0YCCwF7gf8Aew1i5ubPMMcC5QDlxtrf2mjesoEItIn2Pr\nLKXflFL4WSFFnxVRsrSEkLEh7oAccXJEt8yz2ptqa2H58qap3TZsgFNPbQrII0d6/9zHFZsryH40\nm4J3Coi/Lp6k25IIHNxHU72IdDstzCEi0oPqquoo+brEHZDLvysn/MRwd0AOPyG8z03ldeBAQ6/x\nwYDsdDZN7XbmmRDlHVM0A1C6upTsedkUZRSReHMiiTcl4h+tuedEpCUFYhGRXlRbXEvRl0UUflZI\n4WeF1OTWEJUeRdSZUUSfGU3I6JA+8YDeQdbCxo1Ncx//5z8wblxTQJ48uSEw97aifxeRPTebsm/L\nSL49mfhfxOMX1veGrohIz7MWHA4FYhERj6neU03R500B2dbYhrHHjQE5KDnI0yV2SlUVfPVVU+9x\nXl7TkAqHo6f/tNTkVVOxtpz6ijoijgslZFQITn/To/ft7msdSVfPd8c1vOUeByOCtYffPprzvfWe\n3rpmfX3Dq/l2W/sdaeOt1+jqfRsoEIuIeAVrLVXbq9zhuOjzIvyi/ZoC8vRo/Af0rZ/78/Jg586m\n/1k3/x9Rd/1ZX2cpXllKwXv7qa+FmPNjCDk+HONw9Mj9OvLn0b73SNr7X2JH/pfZ1Wt4wz2sbQrM\nzf883PbRnO+t9/TGNZ3Opi9dB1+t99s61tn9vn4NDZkQEfFStt5StrasYQaLzwop/ncxwSOC3eOP\nI0+JxBnatx7Q6071rnry38gn+/fZOCOcpN6TyoAZA7RqmYh0mgKxiEgfUV9TT8nyEndALv2mlPAT\nmj2gNyUch3//X02jrrKOPS/uIfuxbIJHBJM6O5Wo6VF9auy1iHgXBWIRkT6qrryOoq+K3AG5clsl\nkadGugNy6LGh/aq3tLaklrxFeeQszCH8xHBS704l4kStKiciXadALCLST9QU1FD0RZF7irfa4lqi\nzohyB+TgoX1zAYqaghpyn8wld1EuMefGkHJXCmHHhnm6LBHpRxSIRUT6qarsZg/ofVaEI8jhnr0i\n+oxoAgYFeLrEI6rKqSJnQQ57Xt5D7P/EknJHCsHD+maoFxHvpkAsIuIDrLVUbKxoCMifFlL0ZRFB\nKUHuGSyiTovCL8I75umtyKxg16O72Pe3fQy+ejDJv0kmMFGryolIz1EgFhHxQfW19ZStKnP3IJcu\nLyV0XKg7IEeeFIkjsHcf0CtbW0b277Mp/KSQhF8lkHRLUp+bZk5E+iYFYhERoa6yjpL/lrgDcsWG\nCiJOiiD6ew3jj8MmhPXYEtPFS4vJnptN6YpSkn6TRMINCV7TWy0ivkGBWEREDuEqclGU0TSDRc3e\nhiWm3Q/ojQzu0jRn1loKPyske242ldsrSbkzhcE/H4wz2HfnVRYRz1EgFhGRdlXnVVP4eaE7INt6\n6w7H0WdGd3iMr623FLxbQPbcbOrK6ki5O4W4S+N8Yv5kEfFeCsQiItIp1loqMyvds1cUflFIQGyA\newaLqPQo/KNbjv2tr60n/8/5ZM/LxhHsIHV2KgMvHtiv5kkWkb5LgVhERLrE1lvK1jQ9oFfy3xJC\nRoW4A3LV9iqyH80mMCWQ1NmpRJ8VrVXlRMSrKBCLiEi3qq+pp2Rp0wN6/jH+pNyZQuS0SE+XJiLS\nJgViEREREfFpnQ3EeupBRERERHyaArGIiIiI+DQFYhERERHxaQrEIiIiIuLTFIhFRERExKcpEIuI\niIiIT1MgFhERERGfpkAsIiIiIj5NgVhEREREfJoCsYiIiIj4NAViEREREfFpCsQiIiIi4tMUiEVE\nRETEpykQi4iIiIhPUyAWEREREZ+mQCwiIiIiPq1DgdgYc64xZpMxZqsx5s42zqcbY4qNMasbX/d2\nf6kiIiIiIt3Pr70Gxhgn8AzwPSAXWGGMeddau7FV0y+ttRf1QI0iIiIiIj2mIz3EU4BMa+1Oa60L\n+DPw/TbamW6tTERERESkF3QkECcCu5rt5zQea84CJxtjvjXGfGCMGdNdBYqIiIiI9KR2h0zQEHbb\n8w2QbK2tMMacB/wDGNmlykREREREekFHAnEukNxsP5mGXmI3a21ps+0PjTHPGmNirLUHmrebM2eO\nezs9PZ309PSjKFlEREREpElGRgYZGRlH/X5j7ZE7gI0xfsBm4EwgD1gOXNb8oTpjzCAg31prjTFT\ngLestWmtrmPbu5eIiIiISFcZY7DWdvj5tnZ7iK21tcaYm4CPACfwgrV2ozHmhsbzi4EfAzcaY2qB\nCuDSo6peRERERKSXtdtD3G03Ug+xiIiIiPSCzvYQa6U6EREREfFpCsQiIiIi4tMUiEVERETEpykQ\ni4iIiIhPUyAWEREREZ+mQCwiIiIiPk2BWERERER8mgKxiIiIiPg0BWIRERER8WkKxCIiIiLi0xSI\nRURERMSnKRCLiIiIiE9TIBYRERERn6ZALCIiIiI+TYFYRERERHyaArGIiIiI+DQFYhERERHxaQrE\nIiIiIuLTFIhFRERExKcpEIuIiIiIT1MgFhERERGfpkAsIiIiIj5NgVhEREREfJoCscj/b+/eYuyq\n6jiOf39MQVpAqqAFKaQEQVEEWpA0YgVMNaUg3qLQYCQEES9A9YHrg/HFaB+MgHgr0gaRS7QKwUil\nFKiakFSQtlBKUQiNRaAl1lKBYKD9+bBXdRjOzDnInNm73b9PMpm91+zZ85uVM3P+Z+119oqIiIhW\nS0EcEREREa2WgjgiIiIiWi0FcURERES0WgriiIiIiGi1FMQRERER0WopiCMiIiKi1VIQR0RERESr\npSCOiIiIiFZLQRwRERERrda1IJY0S9JaSX+VdMkwx1xVvr5K0tTRjxkRERER0R8jFsSSBoCrgVnA\ne4A5kg4fcsxs4J22DwW+CPyoT1lbYdmyZXVH2GGkr3qTfupd+qo36afepJ96l77qTfqpf7qNEB8H\nPMINUxYAAAcXSURBVGZ7ne2XgZuBjw855jTgOgDby4GJkiaNetKWyIO9d+mr3qSfepe+6k36qTfp\np96lr3qTfuqfbgXxAcD6QftPlrZux0x+49EiIiIiIvqvW0HsHs+j//P7IiIiIiJqJXv42lXSdOCb\ntmeV/cuAbbbnDTrmx8Ay2zeX/bXACbY3DDlXiuSIiIiIGBO2hw7YDmtcl6/fDxwqaQrwFHA6MGfI\nMbcB5wM3lwJ689Bi+PWGioiIiIgYKyMWxLZfkXQ+cAcwAFxr+xFJ55Wv/8T27ZJmS3oMeAE4u++p\nIyIiIiJGyYhTJiIiIiIidnZ9X6lO0gJJGyQ91O+ftSOTdKCkeyQ9LGm1pAvrztREknaXtFzSSklr\nJH277kxNJ2lA0gpJv6k7S1NJWifpwdJPf6o7T5NJmihpkaRHyt/g9LozNY2kd5XH0vaP5/I/vTNJ\nl5XnvYck3SjpTXVnaipJc0s/rZY0t+48TdGpzpT0Vkl3SvqLpCWSJnY7z1gs3byQamGPGNnLwNdt\nvxeYDnx16CIoAbZfAk6yfTRwJHCSpA/WHKvp5gJryN1fRmLgRNtTbR9Xd5iGuxK43fbhVH+Dj9Sc\np3FsP1oeS1OBY4AXgVtqjtU45f1J5wLTbL+PamrmGXVmaipJRwBfAN4PHAWcKumQelM1Rqc681Lg\nTtuHAXeV/RH1vSC2/Ufgn/3+OTs628/YXlm2n6d6knlHvamayfaLZXM3qn+gm2qM02iSJgOzgZ/y\n2tsjxqulf7qQtDcww/YCqN5nYvu5mmM13Uzgcdvrux7ZPluoBoMmSBoHTAD+Xm+kxno3sNz2S7a3\nAr8HPlVzpkYYps7876Jx5fMnup1nLEaI43Uqr5qnAsvrTdJMknaRtBLYANxje03dmRrse8BFwLa6\ngzScgaWS7pd0bt1hGuxg4FlJCyU9IOkaSRPqDtVwZwA31h2iiWxvAr4L/I3qTlabbS+tN1VjrQZm\nlKkAE4BTyCJoI5k06I5nG4CuKyinIG4YSXsCi4C5ZaQ4hrC9rUyZmAx8SNKJNUdqJEmnAhttryCj\nn90cXy5vn0w1XWlG3YEaahwwDfih7WlUdxbqeimyrSTtBnwM+GXdWZqoXPL/GjCF6oronpLOrDVU\nQ9leC8wDlgCLgRVkoKMnru4e0XXKYAriBpG0K/Ar4Oe2b607T9OVS7W/BY6tO0tDfQA4TdITwE3A\nhyX9rOZMjWT76fL5Waq5nplH3NmTwJO27yv7i6gK5OjsZODP5XEVr3UscK/tf9h+Bfg11f+t6MD2\nAtvH2j4B2Aw8WnemBtsgaT8ASfsDG7t9QwrihpAk4Fpgje0r6s7TVJL23f5uUUnjgY9QvVKOIWxf\nbvtA2wdTXba92/bn687VNJImSNqrbO8BfBTIXXE6sP0MsF7SYaVpJvBwjZGabg7Vi9HobC0wXdL4\n8hw4k+oNwNGBpLeXzwcBnyRTcUZyG3BW2T4L6DrI2G2lujdM0k3ACcA+ktYD37C9sN8/dwd0PPA5\n4EFJ2wu8y2z/rsZMTbQ/cJ2kXahe0F1v+66aM+0ocpeJziYBt1TPx4wDbrC9pN5IjXYBcEOZDvA4\nWYypo/LiaibVXRSiA9urylWr+6ku/z8AzK83VaMtkrQP1RsRv2J7S92BmmBQnbnv9joT+A7wC0nn\nAOuAz3Y9TxbmiIiIiIg2y5SJiIiIiGi1FMQRERER0WopiCMiIiKi1VIQR0RERESrpSCOiIiIiFZL\nQRwRERERrZaCOCKiTyRtlbRi0MfFo3juKZKygEhExCjo+8IcEREt9qLtqXWHiIiIkWWEOCJijEla\nJ2mepAclLZd0SGmfIuluSaskLZV0YGmfJOkWSSvLx/RyqgFJ8yWtlnSHpN3L8RdKericJ0sHR0R0\nkYI4IqJ/xg+ZMvGZ0m5gs+0jgauBK0r794GFto8CbgCuKu1XAffYPhqYBqwp7YcCV9s+AtgMfLq0\nXwIcXc5zXh9/v4iInUKWbo6I6BNJ/7K9V4f2J4CTbK+TtCvwtO19JT0L7Gd7a2l/yvbbJG0EDrD9\n8qBzTAGW2D6s7F8M7Gr7W5IWA88DtwK32n6h379rRMSOLCPEERH1GzwyoWGO6dT+70HbW/nf+0JO\nAX5ANZp8n6SBN5wwImInloI4IqIepw/6fG/Zvhc4o2yfCfyhbN8FfBlA0oCkNw93UkkCDrK9DLgU\n2BvYY1STR0TsZHKXiYiI/hkvacWg/cW2Ly/bb5G0CngJmFPaLgAWSroI2AicXdrnAvMlnUM1Evwl\nYAOvHlmm7A8A10vam2pU+UrbW0b594qI2KlkDnFExBgrc4iPsb2p7iwREZEpExERdchIREREg2SE\nOCIiIiJaLSPEEREREdFqKYgjIiIiotVSEEdEREREq6UgjoiIiIhWS0EcEREREa2WgjgiIiIiWu0/\nf4IrNv6bV1QAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10edae790>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "def plot_losses(train_losses, val_losses, epochs, min_y, max_y):\n",
    "    x1 = [a+1 for (a,b)  in train_losses]\n",
    "    y1 = [b for (a,b)  in train_losses]\n",
    "    x2 = [a+1 for (a,b)  in val_losses]\n",
    "    y2 = [b for (a,b)  in val_losses]\n",
    "    plt.figure(figsize=(12, 6))\n",
    "    plt.axis((1, epochs, min_y, max_y))\n",
    "    plt.plot(x1, y1, \"-m\", x2, y2, \"-b\")\n",
    "    plt.legend(('Train Loss', 'Validation Loss'))\n",
    "    xlabel = 'Epochs'\n",
    "    plt.xlabel(xlabel)\n",
    "    plt.show()\n",
    "\n",
    "plot_losses(train_losses, val_losses, 10, 0, 4.0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}