Advanced deep learning with keras - solutions.ipynb

{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"Advanced deep learning with keras - solutions.ipynb","provenance":[],"collapsed_sections":[],"toc_visible":true},"kernelspec":{"name":"python3","display_name":"Python 3"}},"cells":[{"cell_type":"code","metadata":{"id":"RKRSMNCKLoWF","colab_type":"code","colab":{}},"source":["# https://www.datacamp.com/courses/advanced-deep-learning-with-keras-in-python"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"TepyZf85Peof","colab_type":"text"},"source":["Modules to import"]},{"cell_type":"code","metadata":{"id":"GIyl1CzXQW_A","colab_type":"code","outputId":"a1f929ba-0905-4866-dd53-52226235ff47","executionInfo":{"status":"ok","timestamp":1557396845188,"user_tz":-480,"elapsed":1730,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":34}},"source":["import pandas as pd\n","import matplotlib.pyplot as plt\n","from numpy import unique\n","\n","from keras.layers import Input, Dense, Embedding, Flatten\n","from keras.layers import Concatenate, Subtract\n","from keras.models import Model\n","from keras.utils import plot_model\n","from keras.optimizers import Adam\n","\n","from scipy.special import expit as sigmoid"],"execution_count":0,"outputs":[{"output_type":"stream","text":["Using TensorFlow backend.\n"],"name":"stderr"}]},{"cell_type":"code","metadata":{"id":"cmlbiAzAQYmo","colab_type":"code","colab":{}},"source":["# to export dataframe\n","\n","print(df.to_string())"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"QTnxzeGzSbVJ","colab_type":"code","colab":{}},"source":["# to display entire numpy array\n","\n","numpy.set_printoptions(threshold=numpy.inf)"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"JtzpJUpaCqRW","colab_type":"code","outputId":"d37c4109-c318-491b-d63c-70407e455d71","executionInfo":{"status":"ok","timestamp":1557396106482,"user_tz":-480,"elapsed":812,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":34}},"source":["games_season = pd.read_csv('/content/games_season.csv')\n","games_season.shape # (312178, 8)"],"execution_count":0,"outputs":[{"output_type":"execute_result","data":{"text/plain":["(312178, 8)"]},"metadata":{"tags":[]},"execution_count":72}]},{"cell_type":"code","metadata":{"id":"OOzUlI7iCmiS","colab_type":"code","outputId":"1b5c73f9-a2e3-4a89-ae1c-0f958ddd6268","executionInfo":{"status":"ok","timestamp":1557396091594,"user_tz":-480,"elapsed":1028,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":34}},"source":["games_tourney = pd.read_csv(\"/content/games_tourney.csv\")\n","games_tourney.shape # (4234, 9)"],"execution_count":0,"outputs":[{"output_type":"execute_result","data":{"text/plain":["(4234, 9)"]},"metadata":{"tags":[]},"execution_count":71}]},{"cell_type":"markdown","metadata":{"id":"PU53XxEELvJU","colab_type":"text"},"source":["## 1. The Keras Functional API"]},{"cell_type":"markdown","metadata":{"id":"cxFyeRX1Sr6b","colab_type":"text"},"source":["#### Keras input and dense layers"]},{"cell_type":"code","metadata":{"id":"9jSr9fQwLvQM","colab_type":"code","outputId":"89027d96-d0b2-46d4-8faf-a2e1e064eeb5","executionInfo":{"status":"ok","timestamp":1557382922547,"user_tz":-480,"elapsed":2027,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":34}},"source":["# Import Input from keras.layers\n","from keras.layers import Input"],"execution_count":0,"outputs":[{"output_type":"stream","text":["Using TensorFlow backend.\n"],"name":"stderr"}]},{"cell_type":"code","metadata":{"id":"mGqH_nuEQabo","colab_type":"code","colab":{}},"source":["# Create an input layer of shape 1\n","input_tensor = Input(shape=(1,))"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"TOOhAmYRQ-0b","colab_type":"text"},"source":["**Dense layers**\n","\n","Once you have an Input layer, the next step is to add a Dense layer.\n","\n","Dense layers learn a weight matrix, where the first dimension of the matrix is the dimension of the input data, and the second dimension is the dimension of the output data. Recall that your Input layer has a shape of 1. In this case, your output layer will also have a shape of 1. This means that the Dense layer will learn a 1x1 weight matrix.\n","\n","In this exercise, you will add a dense layer to your model, after the input layer."]},{"cell_type":"code","metadata":{"id":"ANolmIDqQi3s","colab_type":"code","colab":{}},"source":["# Load layers\n","from keras.layers import Input, Dense"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"KwHshfbNRE94","colab_type":"code","outputId":"e17da4c1-c851-423a-9b24-941966886ee3","executionInfo":{"status":"ok","timestamp":1557383122823,"user_tz":-480,"elapsed":1135,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":88}},"source":["# Input layer (from previous exercise)\n","input_tensor = Input(shape=(1,))\n","\n","# Create a dense layer\n","output_layer = Dense(1)\n","\n","# Connect the dense layer to the input_tensor\n","output_tensor = output_layer(input_tensor)"],"execution_count":0,"outputs":[{"output_type":"stream","text":["WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n","Instructions for updating:\n","Colocations handled automatically by placer.\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"H3h-JAU7Rlvl","colab_type":"code","colab":{}},"source":["# Load layers\n","from keras.layers import Input, Dense\n","\n","# Input layer\n","input_tensor = Input(shape=(1,))\n","\n","# Create a dense layer and connect the dense layer to the input_tensor in one step\n","# Note that we did this in 2 steps in the previous exercise, but are doing it in one step now\n","output_tensor = Dense(1)(input_tensor)"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"YepdGW8fSvjo","colab_type":"text"},"source":["#### Build and compile a model"]},{"cell_type":"code","metadata":{"id":"LhgTNuN_RxWY","colab_type":"code","colab":{}},"source":["from keras.models import Model"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"wpcd3DJXR_uU","colab_type":"code","colab":{}},"source":["# Build the model\n","model = Model(input_tensor, output_tensor)"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"skC4emSOSR0u","colab_type":"code","colab":{}},"source":["# Compile the model\n","model.compile(optimizer='adam', loss='mean_absolute_error')"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"lL3Gj_uNSUUg","colab_type":"code","colab":{}},"source":["# Import the plotting function\n","from keras.utils import plot_model\n","import matplotlib.pyplot as plt"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"jkKe4j1MSk4D","colab_type":"code","outputId":"36a9533b-9d84-46b4-f0c9-f0f6ceeec8f9","executionInfo":{"status":"ok","timestamp":1557383505100,"user_tz":-480,"elapsed":1289,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":456}},"source":["# Summarize the model\n","model.summary()\n","\n","# Plot the model\n","plot_model(model, to_file='model.png')\n","\n","# Display the image\n","data = plt.imread('model.png')\n","plt.imshow(data)\n","plt.show()"],"execution_count":0,"outputs":[{"output_type":"stream","text":["_________________________________________________________________\n","Layer (type)                 Output Shape              Param #   \n","=================================================================\n","input_3 (InputLayer)         (None, 1)                 0         \n","_________________________________________________________________\n","dense_2 (Dense)              (None, 1)                 2         \n","=================================================================\n","Total params: 2\n","Trainable params: 2\n","Non-trainable params: 0\n","_________________________________________________________________\n"],"name":"stdout"},{"output_type":"display_data","data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAATAAAAD8CAYAAADwpviIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xt0VOW5+PHvQyC3IQkXuZUkJYCI\nCJyqgSIV6xKtIBZUVKIgQvUoggo/pEWBw+li1XPEVLFF0IWiIstCQrCCtNzKxcuqeghK5a7gkZtA\nUCAcbgXi8/tj9uxOyISEzExmtnk+a82aPe++PdnvzJN3v/smqooxxnhRvVgHYIwxNWUJzBjjWZbA\njDGeZQnMGONZlsCMMZ5lCcwY41lRS2Ai0kdEtovIDhF5MlrrMcbUXRKN88BEJAH4ArgJ2AusA+5R\n1S0RX5kxps6KVgusO7BDVb9S1TPAfGBAlNZljKmj6kdpua2BPUGf9wI/rWziSy65RNu0aROlUIwx\nXrN+/fpvVbVZVdNFK4FVSUQeAh4CyM7Opri4OFahGGPijIjsqs500dqF3AdkBX3OdMpcqjpLVXNV\nNbdZsyoTrTHGVBCtBLYOuFREckQkEcgDFkdpXcaYOioqu5Cqek5EHgWWAwnAa6q6ORrrMsbUXVHr\nA1PVvwJ/jdbyjTHGzsQ3xnhWzI5CVsdHH33E888/H+swjDFRMHbsWK655pqwlhHXLbA9e/ZQVFQU\n6zCMMRFWVFTEnj17qp6wCnGdwIwx5kLiehcyYMGCBbEOwRgTQSISkeVYC8wY41mWwIwxnmUJzBjj\nWZbAjDGeZQnMGONZlsCMMZ5lCcwY41mWwIwxnmUJzBjjWZbAjDGeZQnMGONZlsCMMZ5lCcwY41mW\nwIwxnlXjBCYiWSKyRkS2iMhmERntlDcRkZUi8qXz3jhy4RpjzL+E0wI7Bzyhqp2AHsAoEekEPAms\nUtVLgVXOZ2OMibgaJzBV3a+qnzrD/wdsBVoDA4A5zmRzgNvCDdIYY0KJSB+YiLQBrgQ+AVqo6n5n\n1AGgRSTWYYwx5ws7gYlIQ2AhMEZVjwWPU1UFtJL5HhKRYhEpPnToULhhGGPqoLASmIg0wJ+83lLV\nt53igyLSyhnfCigJNa+qzlLVXFXNbdasWThhGGPqqHCOQgowG9iqqsEPb1wM3O8M3w8sqnl4kfHX\nv/6VjIwM3n33Xd59991YhxMXpk6dSseOHenYsSMpKSn4fD58Ph8dO3bkP/7jPzh27BjHjh2rekFV\n+Pjjj/n444+5/PLLqVevHvXq1UNEaNGiBb/73e/43e9+F4G/JrYWLlzIwoULadu2LSLivlq2bMmQ\nIUMYMmRIrEP8wQrnqUQ/A+4DNorIBqdsAvAMUCgiDwC7gLvDCzF8/j1ZE+yDDz7g3//93wEYOnQo\nKSkpACxdupTBgwfzySefALBixYqw1tOjRw8Atm7dSp8+fQBYvnw527dvp1GjRmEtO14MHDjQfW/f\nvj3ffvstAAcOHIhlWHVCjROYqn4IVPZspN41XW409OvXj9LS0liHUWOnTp0CoHfv3vz973+PyDIT\nExMZNWoUAMnJyW75XXfdxYIFC9xH2e3fv59WrVpFZJ0/BKdOnaJ3b//XO1J1YWrOzsQ3xniWJx5s\nW9fNnj0bgJKSkMdDauTtt9+udFzr1q3d4ePHj0dsnT8Es2fPjmg9mPD8oFtgH374IR9++CHZ2dmI\nCC+++CIvvvgiADNnzmTmzJn4fD5SU1NZtGgRixYtom/fvqSnp5OZmUlmZibz5s0rt8w//vGPJCcn\nk5ycTPPmzRkxYgQjRoygVatWJCcn07NnT3r27On2IQU8/vjjJCYmkpiYSMuWLcuNGzVqlNuJLiJu\nHwrAmDFjeOKJJ3jiiSfYuXMnIkL79u1p3759lLYafPnllzRq1IhGjRrx4x//uNy4ZcuWkZ6eTnp6\nOk8//XRE1hdcD4G66Nu3b7m6mDdvXqV10bx5c7cuWrVqVa4uPvnkk3J18fjjj7t10bJly3J1MWrU\nKLcuAh3xgboYM2aMWxc7d+4sVxeR8sEHH/DBBx/QqVMnMjIyyMjIIDk5mS5durB8+XKWL18OwIMP\nPljuYEG7du1o164dn332GQDDhw9n+PDhpKamustZvHgxAGVlZZSVlTF58mSys7PJzs4mJSWFrl27\nUlBQQEFBAQDPPvusWx9paWmUlJRQUlLCE088QevWrdm+fTvbt2+P2N9eY6oa89fVV1+toRQUFKg/\nxPDs2bNHAZ0+fbpOnz693LiJEycqoKtWrdJVq1ZpaWmplpSUaK9evbRXr17q8/n0zJkzeubMGXee\nhx9+WB9++GH1+Xy6ZcsW3bJli54+fVo3b96s3bp1027dumlaWpru3r1bd+/e7c43ePBgHTx4sLZo\n0aJCjPn5+Zqfn6+AHjp0qNy4gQMH6sCBA7Vdu3Zhb4tQzpw5o3v37tW9e/fq9OnTNSkpSefOnatz\n586tMO2SJUs0LS1N09LSdMqUKRe1nptvvllvvvlmBfTIkSPlxgXqIVAXpaWl5erC5/O5dREsUA+B\nujh9+nS5ugjEGlwPqurWQ2V1EYglVF20a9eu0rpo166dZmRkaEZGxkVtG1XVBQsW6IIFC/S3v/2t\nHj58WA8fPqzfffed9ujRQ5s2bapNmzYtF8fAgQM1ISFB9+3bp/v27auwvHvvvVcXL16sixcvdsvG\njRun48aN06SkJC0qKtKioiI9cuSITpgwQevVq6f16tXTdevWqaq/TgL1Mnr0aB09erROnz5d77jj\nDt26datu3br1ov/GAEALCgouNL5Yq5E7ftAtMGPMD5v1gTl69uwJ/OuIXF5eHuBv1u/evRuAdu3a\nlZunfv36XH755e7nTp06MXPmTAC6devG66+/DsDkyZOjG3yYsrKyOHjwIABNmzbl2WefZdCgQSGn\n7devX0TOD7uQnj17ljsympeXxwcffADA7t27Q9YDELIuunXrBsDrr78e9/Vw5513lnsP6N+/PxMm\nTADg0KFDNGvWjEceeQTwn4MW+J5NnDgRwK2fdevW8eabb7rLOX36tPv9vP32293TPwAmTZrEc889\nB/i3VW5ubrkYnnnmGcD/+3j00Ucj8NdGhiWwSiQmJrrDZ8+erfZ8gYpPTU1l27ZtEY8rGvbs2cPR\no0cB+Oyzz3jqqaeYNWsWAKtXr6Z58+axDC+sukhNTQXwTF2E0qBBA3e4rKwMgBtuuAGADh068Npr\nrwEwYcIERIT58+cD/sSfkJDgzrt9+3ZOnjwJQOfOncutIyUlxe0P9NK2sl1IY4xnWQssSpKSkvDK\nReoNGjQgcD3qL37xC3JycujQoQMA//Vf/8ULL7wQy/DCkpSUBOCJuvjLX/4CQH5+Pps3bwb8u4Oh\nWp3+K/lgxIgRjB07FoBVq1Zx4403uruNb731Vrl5Tpw44Q5PmjSJSZMmhYzDSycuWwssws6ePcvZ\ns2c5evSoeyqG17Rv356EhAQSEhLcH5IXBeohUBfx5P3332fatGlMmzYN8Pft3X777dx+++20bNnS\nPf2jtLSUqVOnVrqcYcOGuaf1vPrqq2zfvt09zeX8U2CCb5owbdq0So/sffTRR9H5o6PAWmARtnbt\nWsB/ekrgOsCAQGfzxfTjRMN3333HY489xp/+9KeQ47/88ku3ryUrK6s2Q4uotWvXutfBhqqLWNbD\n+vXr8fl87ueNGze68YwcOZK2bdu64wKtrVAaN27sHnCZP38+aWlp7jWu58vKynIPjmzYsCHkNF5j\nLTBjjGdZCywM33//PUeOHAEgLS2NLVu2MGbMGACys7MZNmxYuekDZ20fPnyYd955h379+gFw9OhR\ndu3aVel6mjRpAsA333zD119/TdOmTQH/Ie3gI1TV5fP5WLFiBatXrwb8R+sCd6PYtGkTI0eOdFsH\ngf6VgKVLl7r/8cePH+8euo+l77//HoAjR46QlpYG4NZFdnY2QMi6OHz4MIBbF4EjsVXVxTfffAPg\n1kWgVVNVXZw9e9Zd59q1a7n77n/dqCUQJ8Df/vY3unfvDviPEJ9/Vcf5AqdUzJkzhyVLlvDSSy+F\nnC45OZnhw4cD/kuiAusYMmQIPp+P/fv9N1JOSEjwTD/YDzqBBS4bClzyMn78eABWrlzJzTffDOD2\nQXTt2hXwXyqzatUqxo0b5y4ncBuYlStXcumll7rlp0+fpkuXLgAcPHiQ1NRU9/D2c88953YgB4wc\nORLwn5pwzz33uF+SW2+91U0gANdcc42bXLKystwv6IoVK+jUqRNXX301AEVFRbRocfF37E5OTuZn\nP/sZDz74IODv4A7svmRmZtKtWzdeeeUVoOLh9osV+PE98MADbN261S2//PLLeeyxxwBo1KiRWw/g\nr4tly5YBhKyLlStXArh1cfr0aQC6dOnins8WqIvAuU2h6iKwjQN1ceuttwKErIvArvQjjzzi3mIo\nUBeDBw8G4Pe//z07d+50573Qrh+UT6pdunRxv58zZsxg+vTpANx0001cf/31FBUVAXDttdeyZs2a\ncrv2P/3pTwG48sor6dOnj9tVEUrggEx6errbtzZ69GgaN27MddddB8CUKVOYO3cuzz//r9v8BR/U\niaf7m9kupDHGu6pzvVG0X9G+FjLSAtdCNmnSJNah1HmBerC6UL3lllv0q6++inUY1UKEroX8Qe9C\nRlvgSJ2JrbpWD2fPni3X3/b5558D/q6BnJycWIUVE5bAPChwqUfwtX/VEXy43XjX+PHj3X5RVXU7\n5ufOnRvLsGLC+sCMMZ4VdgtMRBKAYmCfqt4qIjnAfKApsB64T1XPhLueeDFhwgT36v8zZ864Tfb8\n/PwKdxGIlo4dOwL2sJJAXZw54/965eTkkJ+fD1S8o8MPSWpqqvsdaN26NTNmzAD8R0XrGgn3RyAi\nY4FcIN1JYIXA26o6X0ReBv6hqqFPTHHk5uZqcXFxhfLCwkIGDRpU53+oxvzQiAgFBQXlzoU7b/x6\nVc0NOTJIuA+2zQT6Aa86nwW4AShyJpkD3BbOOowxpjLh9oG9APwG+N753BQ4qqrnnM97gdahZhSR\nh0SkWESKvXCnAGNM/Annydy3AiWqur4m86vqLFXNVdXc4KvkjTGmusJ9Mnd/EbkFSAbSgT8AjUSk\nvtMKywT2hR+mMcZUVOMWmKo+paqZqtoGyANWq+pgYA0QOAR0P7Ao7CiNMSaEaJwHNh4YKyI78PeJ\nzY7COowxJjJn4qvqWmCtM/wV0D0SyzXGmAuxM/GNMZ5lCcwY41mWwIwxnmUJzBjjWZbAjDGeZQnM\nGONZlsCMMZ5lCcwY41mWwIwxnmUJzBjjWZbAjDGeZQnMGONZnnis2l133RXrEIwxcSiuE1hWVtYP\n+uky5l+CH+qSm1vlsxyMx915551kZWWFvZywn0oUCZU9lcjUHcFPpyksLIxhJCYe1MpTiYwxJpYs\ngRljPMsSmDHGs8J9sG0jESkSkW0islVErhGRJiKyUkS+dN4bRypYY4wJFm4L7A/AMlXtCPwbsBV4\nElilqpcCq5zPxhgTceE82DYDuA7nqUOqekZVjwIDgDnOZHOA28IN0hhjQgmnBZYDHAJeF5HPRORV\nEfEBLVR1vzPNAaBFuEEaY0wo4SSw+sBVwEuqeiVwgvN2F9V/klnIE81E5CERKRaR4kOHDoURhjGm\nrgonge0F9qrqJ87nIvwJ7aCItAJw3ktCzayqs1Q1V1VzmzVrFkYYxpi6qsYJTFUPAHtE5DKnqDew\nBVgM3O+U3Q8sCitCY4ypRLjXQj4GvCUiicBXwHD8SbFQRB4AdgF3X2B+Y4ypsbASmKpuAEJdr9Q7\nnOUaY0x12Jn4xhjPsgRmjPEsS2DGGM+yBGaM8SxLYMYYz7IEZozxLEtgxhjPsgRmjPEsS2DGGM+y\nBGaM8SxLYMYYz7IEZozxLEtgxhjPsgRmjPEsS2DGGM+yBGaM8SxLYMYYz7IEZozxLEtgxhjPCiuB\nicj/E5HNIrJJROaJSLKI5IjIJyKyQ0QKnAd+GGNMxNU4gYlIa+BxIFdVOwMJQB4wFZimqu2BI8AD\nkQjUGGPOF+4uZH0gRUTqA6nAfuAG/A+5BZgD3BbmOowxJqRwHmy7D/g9sBt/4ioF1gNHVfWcM9le\noHW4QRpjTCjh7EI2BgYAOcCPAB/Q5yLmf0hEikWk+NChQzUNwxhTh4WzC3kj8L+qekhVzwJvAz8D\nGjm7lACZwL5QM6vqLFXNVdXcZs2ahRGGMaauCieB7QZ6iEiqiAj+p3FvAdYAdzrT3A8sCi9EY4wJ\nLZw+sE/wd9Z/Cmx0ljULGA+MFZEdQFNgdgTiNMaYCupXPUnlVPU/gf88r/groHs4yzXGmOqwM/GN\nMZ5lCcwY41mWwIwxnmUJzBjjWZbAjDGeZQnMGONZlsCMMZ4V1nlgxlyMN954A4AXXniBsrKycuOC\nr4ft0qVLuXEJCQmMGTMGgGHDhkU1RuMtlsBMrbnmmmsAGD58+AWnO3jwYIWyHj16RCUm422WwEyt\nueyyywB/C2vTpk2oapXziAidO3emY8eO0Q7PeJD1gRljPMsSmKl1Q4cOJSEhoVrT1q9fn/vvvz/K\nERmvsgRmat29995boRO/MufOnWPQoEFRjsh4lSUwY4xnWQIzte5HP/oRPXv2pF69etSrF/orGBjX\ns2dPMjMzazlC4xWWwExM3HfffYgI/pv5VhQYN3To0FqOzHiJJTATE3fdddcFE1jAwIEDayki40WW\nwIwxnmUnspqYaNKkCTfddBMAK1eu5Ny5c+64hIQEd1zTpk1jEp/xhioTmIi8BtwKlKhqZ6esCVAA\ntAG+Bu5W1SPO04n+ANwCnASGqeqn0QndeN2QIUMAWL58eblyVeW+++6LRUjGY6rTAnsDeBF4M6js\nSWCVqj4jIk86n8cDfYFLnddPgZecd2MqGDBgAACJiYmcPn3aLW/QoAH9+/ePVVjGQ6rsA1PV94HD\n5xUPAOY4w3OA24LK31S/j/E/5LZVpII1xphgNe3Eb6Gq+53hA0ALZ7g1sCdour1OmTEV+Hw+fD4f\n/fv3p0GDBtSvX5/69etz22230bBhQxo2bBjrEE2cC/sopPpvKVD1bQXOIyIPiUixiBQH3wvK1D2D\nBw/m3LlzlJWVUVZWxr333hvrkIxH1DSBHQzsGjrvJU75PiAraLpMp6wCVZ2lqrmqmtusWbMahmGM\nqctqehrFYuB+4BnnfVFQ+aMiMh9/531p0K5mXCosLIx1CHVeWVkZycnJ7v3Bjh8/bvUSB+6+++5Y\nh1A1Vb3gC5gH7AfO4u/TegBoCqwCvgT+BjRxphVgBrAT2AjkVrV8VeXqq6/WWMG/+2sve9nrvFcs\nAcVajdxRZQtMVe+pZFTvENMqMKqqZcabgoICb/y3+QFbs2aNe1nR9ddfH9tg6rjCwkLP3MLILiUy\nxniWXUpk4sLPf/7zWIdgPMgSmIkLld0XzJgLsW+NMcazLIEZYzzLEpgxxrMsgRljPMsSmDHGsyyB\nGWM8yxKYMcazLIEZYzzLEpgxxrMsgRljPMsSmDHGsyyBGWM8yxKYMcazLIEZYzzLEpgxxrMsgRlj\nPKvKBCYir4lIiYhsCirLF5FtIvK5iPxZRBoFjXtKRHaIyHYRuTlagRtjTHVaYG8Afc4rWwl0VtWu\nwBfAUwAi0gnIA65w5pkpIgkRi9ZDHnzwQdLS0khLS0NE2LBhQ6xDiqkpU6bQqVMnOnXqRHp6OklJ\nSbRv35727dvzm9/8huPHj0dkPQsXLmThwoW0bdsWEanwSkxMJDExkebNm3P99ddz/fXXk5+fz5Ej\nRyKyflO7qkxgqvo+cPi8shWqes75+DH+B9gCDADmq+o/VfV/gR1A9wjGa4wxrkjcE/9XQIEz3Bp/\nQgvY65RVICIPAQ8BZGdnRyCM+PLqq69y4403AnDPPZU9ma7uWL16NY8++igAeXl5NGjQgKVLlwIw\nZMgQNm7c6H4Ox8CBA9339u3b8+233wJw9OhRVJXS0lIANmzYwJw5cwAYP34806ZNY/HixQDk5uaG\nHYepHWElMBGZCJwD3rrYeVV1FjALIDc3V8OJw8S/hg0b8vDDDwOQkODvVQg8i3PhwoUUFhayZ88e\nALKysqISg4jQqJG/uzaw+wjQr18/Bg0aRL9+/QD44osvyMjIiEoMJrJqfBRSRIYBtwKDnQfaAuwD\ngr99mU5ZnRTodzGwZMkSEhIS3OQV7JJLLgHg5MmTnDx5srZD484772TYsGGUlJRQUlLCyy+/XOsx\nmJqpUQITkT7Ab4D+qhr8jVsM5IlIkojkAJcC/xN+mMYYU1F1TqOYB3wEXCYie0XkAeBFIA1YKSIb\nRORlAFXdDBQCW4BlwChVLYta9HFAVVFV8vPzueyyy0hKSiIpKYmMjAx+/etf8+tf/zrkfGVlZUye\nPJnJkyeTnZ1NSkoKKSkpdO3alYKCAne6mTNn4vP58Pl8pKamsmjRIvr27Uvfvn1JT08nMzOTzMxM\n5s2bV2Ed7733Ht27d6d79+6kpqaSnp5Oeno6Xbp04dixYxeMpWvXrhViiZZ9+/aRkpJCTk4OOTk5\n5cYtW7aM9PR0nn76aZ5++umoxTBs2DB3+Py+uIutq9TU1Errat68eRGpq+zs7JjUVdwJ/ABj+br6\n6qs1VgAtKCio8fwTJ07UiRMnqojoc889p0eOHNEjR47oyZMndcaMGTpjxgwF9LPPPis337hx4zQp\nKUmTkpK0qKjInW/ChAlar149Xbduna5bt67cOgBdtWqVlpaWamlpqZaUlGivXr20V69e6vP59MyZ\nM+7yjx8/runp6Tp16lSdOnWqnjp1Sg8cOKAHDhzQO+64Qw8dOnTBWCZMmBAylkg6ceKEnjhxQtPS\n0vTxxx8POc2SJUs0LS1Np0yZolOmTLmo5bdr104zMjI0IyOjymmPHTumgAKalZVVbtzF1lVgOaHq\nyufzRaSuioqKKq2rcBUUFKg/NcQOUKzVyB12Jr4xxruqk+Wi/fJqC+zkyZOampqqqampetNNN1UY\nP2/ePJ03b16FFtipU6c0NTVV8/LyNC8vr8Iyk5KSdOTIkTpy5EhVLd8CO3XqVLnpg1t5O3bscMs3\nbdqkgC5ZskSXLFkSMv5Tp05VGsvJkydDxhJJgb+rQ4cOeuzYsYgv/2JaYKqqIqIioo0aNXLLalJX\nOC2wUHUVGBduXZ0fR3As4fJSCywS54HVWTt27HCPmvXu3bva823fvp2TJ0/SuXPnCuNSUlJo2bIl\n27Ztq9ayEhMT3eGzZ8+6w23btqV58+YMGTIEgNGjR7v9PG3atHHjAELGkpKSAnBRsVyMt99+m8LC\nQgBWrFhBWlpaxNdxMU6cOIE6B9PT09Pdcqur+Ga7kGHYu3evO9ysWbNqz3fixAkAJk2axKRJkypc\n7rJr166wTylISUlh9erVXHvttVx77bU8/fTTtG3blrZt25KXl8epU6c4ceJErcRyvvnz5/PMM8+w\ndu1a1q5d6/5IY+mLL75whzt27OgOx2tdhbpMKhBLXWIJzBjjWbYLGYbk5GR3+J///Ge15wu01qZN\nmwbAmDFjIhuY44orruDdd98F4NChQzz//PMAPPPMM1xxxRXk5eW500Y7FoDp06cDsHz5clavXk3D\nhg2jtq6LtWzZMne4b9++7nC81lU068lLLIGFoXPnztSr52/EvvfeezzyyCPVmi8rK4vk5OSo3qHi\nm2++4ejRo3Tq1Anw/xD/+7//G/D3OW3ZssW9ZCeasQT6lZ588kn3jg/vvPMO9evHz1fvwIEDTJs2\njcxM/z0JfvWrX7nj6lJdeZHtQhpjPCt+/g16ULNmzdy7HxQVFTF79mzuuusuAL7++mtmzZoVcr7k\n5GSGDx/O7NmzAejevbt7BMrn87F//373msFWrVrVKLZvvvmGsWPHujG0bduWzZs3A7Br1y6GDh3q\n7gKHisXn8wGEHcuWLVsAePbZZ92yV155JeS0+fn5AIwbN84tW7p0KYMGDWL8+PEATJw48aLWH2gB\nHj9+nNTUVPfa1G+//ZbVq1cDMGHCBBISEnjnnXeA8kch47Wuunf336UqVF3VNA4vsgQWptdeew2A\njIwMxo8f7/ZN/OQnP+GGG24AYM2aNfzyl7/kL3/5CwBdu3blhRdecH8oU6dOZfTo0QA0btyY6667\njilTpgDw5z//2e1/Ccwb6K9ZtWpVuR97nz59WLlyJeBPrmVlZfTs2ROAY8eO0aJFCwBGjBjh3toG\nCBlL48aNAcrFUpMfRiCB1JZAP9KkSZPYv38/5875b1uXkZHB999/7yawjIwMOnToAPgvIxo1ahRN\nmjQJucx4rKupU6cChKyrupTApLa/YKHk5uZqcXFxTNYtIhQUFLi3djGmrissLGTQoEG1/s8nmIis\nV9Uqb8xmfWDGGM+yBGaM8SxLYKZK27ZtY9u2bSHP/r7QK/jcJWOiwTrxTZUCl9bEQ3+pMcGsBWaM\n8SxLYMYYz7IEZozxrOrcE/81ESkRkU0hxj3h3ATuEueziMgfRWSHiHwuIldFI2hjjIHqtcDeAPqc\nXygiWcAvgN1BxX3xP4noUvwPrX0p/BCNMSa0KhOYqr4PHA4xahr+R6sFH5oaALzp3BX2Y6CRiNSd\n6xqMMbWqps+FHADsU9V/nDeqNbAn6PNep8wYYyLuos8DE5FUYAL+3ccaE5GH8O9mkp2dHc6ijDF1\nVE1aYO2AHOAfIvI1kAl8KiItgX1AVtC0mU5ZBao6S1VzVTX3Yu4nb4wxARedwFR1o6o2V9U2qtoG\n/27iVap6AFgMDHWORvYASlV1f2RDNsYYv+qcRjEP+Ai4TET2isgDF5j8r8BXwA7gFWBkRKI0xpgQ\nquwDU9V7qhjfJmhYgVHhh2WMMVWzM/GNMZ5lCcwY41mWwIwxnmUJzBjjWZbAjDGeZQnMGONZlsCM\nMZ5lCcwY41n2UA/go48+inUIxsQNL/0erAVmjPEsiYdHZeXm5mpxcXFM1i0iMVmvMfEulrlBRNar\nam5V09X5Xch4SODGmJqxXUhjjGdZAjPGeJYlMGOMZ1kCM8Z4liUwY4xnWQIzxnhWXJwHJiKHgBPA\nt7GOxXEJ8RFLvMQB8RNLvMSxKEZLAAAEc0lEQVQBFksokYrjx6pa5ePK4iKBAYhIcXVOXKsN8RJL\nvMQB8RNLvMQBFks8xGG7kMYYz7IEZozxrHhKYLNiHUCQeIklXuKA+IklXuIAiyWUWo0jbvrAjDHm\nYsVTC8wYYy5KzBOYiPQRke0iskNEnqzldWeJyBoR2SIim0VktFP+WxHZJyIbnNcttRTP1yKy0Vln\nsVPWRERWisiXznvjKMdwWdDfvUFEjonImNraJiLymoiUiMimoLKQ20D8/uh8dz4XkatqIZZ8Ednm\nrO/PItLIKW8jIqeCts/LUY6j0voQkaecbbJdRG6OVBwXiKUgKI6vRWSDUx61beJS1Zi9gARgJ9AW\nSAT+AXSqxfW3Aq5yhtOAL4BOwG+BcTHYHl8Dl5xX9izwpDP8JDC1luvnAPDj2tomwHXAVcCmqrYB\ncAuwFBCgB/BJLcTyC6C+Mzw1KJY2wdPVQhwh68P5/v4DSAJynN9XQjRjOW/8c8DkaG+TwCvWLbDu\nwA5V/UpVzwDzgQG1tXJV3a+qnzrD/wdsBVrX1vqraQAwxxmeA9xWi+vuDexU1V21tUJVfR84fF5x\nZdtgAPCm+n0MNBKRVtGMRVVXqOo55+PHQGak1ncxcVzAAGC+qv5TVf8X2IH/dxb1WMR/d9C7gXmR\nWl9VYp3AWgN7gj7vJUYJRETaAFcCnzhFjzq7Ca9Fe7ctiAIrRGS9iDzklLVQ1f3O8AGgRS3FApBH\n+S9jLLYJVL4NYv39+RX+FmBAjoh8JiLviUivWlh/qPqI5TbpBRxU1S+DyqK6TWKdwOKCiDQEFgJj\nVPUY8BLQDvgJsB9/s7g2XKuqVwF9gVEicl3wSPW3y2vlsLGIJAL9gQVOUay2STm1uQ0uREQmAueA\nt5yi/UC2ql4JjAX+JCLpUQwhLurjPPdQ/h9e1LdJrBPYPiAr6HOmU1ZrRKQB/uT1lqq+DaCqB1W1\nTFW/B14hgk3wC1HVfc57CfBnZ70HA7tFzntJbcSCP4l+qqoHnZhisk0clW2DmHx/RGQYcCsw2Emo\nOLts3znD6/H3PXWIVgwXqI9YbZP6wB1AQVCMUd8msU5g64BLRSTH+Y+fByyurZU7++yzga2q+nxQ\neXA/yu3ApvPnjUIsPhFJCwzj7yzehH973O9Mdj+wKNqxOMr9N43FNglS2TZYDAx1jkb2AEqDdjWj\nQkT6AL8B+qvqyaDyZiKS4Ay3BS4FvopiHJXVx2IgT0SSRCTHieN/ohVHkBuBbaq6NyjG6G+TaB4h\nqOZRjVvwH/3bCUys5XVfi3935HNgg/O6BZgLbHTKFwOtaiGWtviPHv0D2BzYFkBTYBXwJfA3oEkt\nxOIDvgMygspqZZvgT5r7gbP4+28eqGwb4D/6OMP57mwEcmshlh34+5gC35eXnWkHOvW2AfgU+GWU\n46i0PoCJzjbZDvSN9jZxyt8ARpw3bdS2SeBlZ+IbYzwr1ruQxhhTY5bAjDGeZQnMGONZlsCMMZ5l\nCcwY41mWwIwxnmUJzBjjWZbAjDGe9f8BL5GfBKlaoTUAAAAASUVORK5CYII=\n","text/plain":["<Figure size 432x288 with 1 Axes>"]},"metadata":{"tags":[]}}]},{"cell_type":"markdown","metadata":{"id":"pyjo_9AzSz3O","colab_type":"text"},"source":["#### Fit and evaluate a model"]},{"cell_type":"code","metadata":{"id":"Tavt096QVb6O","colab_type":"code","colab":{}},"source":["games_tourney_train = pd.read_csv(\"/content/games_tourney_train.csv\")"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"7r68JINnVYS7","colab_type":"code","colab":{}},"source":["games_tourney_test = pd.read_csv(\"/content/games_tourney_test.csv\")"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"uJDGb4tJS3D_","colab_type":"code","outputId":"635e6452-cb99-46bf-ca2d-1af2462aa1f3","executionInfo":{"status":"ok","timestamp":1557384511468,"user_tz":-480,"elapsed":861,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":34}},"source":["games_tourney_train.shape # (3430, 9)"],"execution_count":0,"outputs":[{"output_type":"execute_result","data":{"text/plain":["(3430, 9)"]},"metadata":{"tags":[]},"execution_count":17}]},{"cell_type":"code","metadata":{"id":"D9GLfTDUWcKh","colab_type":"code","outputId":"e680baaf-52fe-4d8b-e107-f15a93126f17","executionInfo":{"status":"ok","timestamp":1557384520753,"user_tz":-480,"elapsed":769,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":34}},"source":["games_tourney_test.shape # (804, 9)"],"execution_count":0,"outputs":[{"output_type":"execute_result","data":{"text/plain":["(804, 9)"]},"metadata":{"tags":[]},"execution_count":18}]},{"cell_type":"markdown","metadata":{"id":"TFm4mCGBTr-C","colab_type":"text"},"source":["In this exercise, you'll use a dataset of scores from US College Basketball tournament games. Each row of the dataset has the team ids: `team_1` and `team_2`, as integers. It also has the seed difference between the teams (seeds are assigned by the tournament committee and represent a ranking of how strong the teams are) and the score difference of the game (e.g. if `team_1` wins by 5 points, the score difference is 5).\n","\n","To fit the model, you provide a matrix of X variables (in this case one column: the seed difference) and a matrix of Y variables (in this case one column: the score difference).\n","\n","The `games_tourney` DataFrame along with the compiled model object is available in your workspace."]},{"cell_type":"code","metadata":{"id":"rCYjCPgHTdld","colab_type":"code","outputId":"d13b67a5-59d3-43b9-d852-d5bc41787b68","executionInfo":{"status":"ok","timestamp":1557384540080,"user_tz":-480,"elapsed":932,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":156}},"source":["# Now fit the model\n","model.fit(games_tourney_train['seed_diff'], games_tourney_train['score_diff'],\n","          epochs=1,\n","          batch_size=128,\n","          validation_split=0.10,\n","          verbose=True)"],"execution_count":0,"outputs":[{"output_type":"stream","text":["WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n","Instructions for updating:\n","Use tf.cast instead.\n","Train on 3087 samples, validate on 343 samples\n","Epoch 1/1\n","3087/3087 [==============================] - 0s 42us/step - loss: 15.0308 - val_loss: 14.0304\n"],"name":"stdout"},{"output_type":"execute_result","data":{"text/plain":["<keras.callbacks.History at 0x7f706a34edd8>"]},"metadata":{"tags":[]},"execution_count":19}]},{"cell_type":"code","metadata":{"id":"yY_nrvyAUZ_w","colab_type":"code","colab":{}},"source":["# Load the X variable from the test data\n","X_test = games_tourney_test['seed_diff']\n","\n","# Load the y variable from the test data\n","y_test = games_tourney_test['score_diff']"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"41IeCah2Ubib","colab_type":"code","outputId":"102b10c5-fcab-4f84-a3c7-f0368a695c9c","executionInfo":{"status":"ok","timestamp":1557384545363,"user_tz":-480,"elapsed":780,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":51}},"source":["# Evaluate the model on the test data\n","model.evaluate(X_test, y_test)"],"execution_count":0,"outputs":[{"output_type":"stream","text":["804/804 [==============================] - 0s 30us/step\n"],"name":"stdout"},{"output_type":"execute_result","data":{"text/plain":["14.096755981445312"]},"metadata":{"tags":[]},"execution_count":21}]},{"cell_type":"markdown","metadata":{"id":"jUTPTpLpLvWM","colab_type":"text"},"source":["## 2. Two Input Network \n","### using categorical embeddings, shared layers, and merge layers"]},{"cell_type":"markdown","metadata":{"id":"sv1Pa2ESWydy","colab_type":"text"},"source":["#### Category embeddings"]},{"cell_type":"code","metadata":{"id":"Q6yEEi8PW_ol","colab_type":"code","colab":{}},"source":["# Imports\n","from keras.layers import Embedding\n","from numpy import unique"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"SaP9qnhXYEsj","colab_type":"text"},"source":["Shared layers allow a model to use the same weight matrix for multiple steps. In this exercise, you will build a \"team strength\" layer that represents each team by a single number. You will use this number for both teams in the model. \n","\n","The model will learn a number for each team that works well both when the team is team_1 and when the team is team_2 in the input data."]},{"cell_type":"code","metadata":{"id":"T5Vg3gLSnKZ2","colab_type":"code","outputId":"753cdb56-a6b7-484c-e91e-da64cdf5726a","executionInfo":{"status":"ok","timestamp":1557396928883,"user_tz":-480,"elapsed":887,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":34}},"source":["games_season = pd.read_csv('/content/games_season.csv')\n","games_season.shape # (312178, 8)"],"execution_count":0,"outputs":[{"output_type":"execute_result","data":{"text/plain":["(312178, 8)"]},"metadata":{"tags":[]},"execution_count":5}]},{"cell_type":"code","metadata":{"id":"itSpK2qDLvbk","colab_type":"code","colab":{}},"source":["# Count the unique number of teams\n","n_teams = unique(games_season['team_1']).shape[0]"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"pW2pzf6XYkEl","colab_type":"code","colab":{}},"source":["# Create an embedding layer\n","team_lookup = Embedding(input_dim=n_teams,\n","                        output_dim=1,\n","                        input_length=1,\n","                        name='Team-Strength')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"0YHp6tUPYuGD","colab_type":"text"},"source":["The embedding layer is a lot like a dictionary, but your model learns the values for each key."]},{"cell_type":"markdown","metadata":{"id":"KNTxaqSmbJa1","colab_type":"text"},"source":["**Define team model**\n","\n","The team strength lookup has three components: an input, an embedding layer, and a flatten layer that creates the output.\n","\n","If you wrap these three layers in a model with an input and output, you can re-use that stack of three layers at multiple places.\n","\n","Note again that the weights for all three layers will be shared everywhere we use them."]},{"cell_type":"code","metadata":{"id":"XZPYR1OvYzPe","colab_type":"code","colab":{}},"source":["# Imports\n","from keras.layers import Input, Embedding, Flatten\n","from keras.models import Model"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"BmQ75CYwZqpd","colab_type":"code","outputId":"464b5830-f28c-4662-8028-0570762a4ac9","executionInfo":{"status":"ok","timestamp":1557396934522,"user_tz":-480,"elapsed":1102,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":88}},"source":["# Create an input layer for the team ID\n","teamid_in = Input(shape=(1,))\n","\n","# Lookup the input in the team strength embedding layer\n","strength_lookup = team_lookup(teamid_in)\n","\n","# Flatten the output\n","strength_lookup_flat = Flatten()(strength_lookup)\n","\n","# Combine the operations into a single, re-usable model\n","team_strength_model = Model(teamid_in, strength_lookup_flat, name='Team-Strength-Model')"],"execution_count":0,"outputs":[{"output_type":"stream","text":["WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n","Instructions for updating:\n","Colocations handled automatically by placer.\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"hi6wwvpgW2cb","colab_type":"text"},"source":["#### Shared layers"]},{"cell_type":"markdown","metadata":{"id":"on5zdnChal-R","colab_type":"text"},"source":["**Defining two inputs**\n","\n","In this exercise, you will define two input layers for the two teams in your model. This allows you to specify later in the model how the data from each team will be used differently."]},{"cell_type":"code","metadata":{"id":"J0w43dwXaP5j","colab_type":"code","colab":{}},"source":["# Input layer for team 1\n","team_in_1 = Input(shape=(1,), name='Team-1-In')\n","\n","# Separate input layer for team 2\n","team_in_2 = Input(shape=(1,), name='Team-2-In')"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"wYGAkFKfW4We","colab_type":"code","colab":{}},"source":["# Lookup team 1 in the team strength model\n","team_1_strength = team_strength_model(team_in_1)\n","\n","# Lookup team 2 in the team strength model\n","team_2_strength = team_strength_model(team_in_2)"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"GFZ4DdV7W4d0","colab_type":"text"},"source":["#### Merge layers"]},{"cell_type":"markdown","metadata":{"id":"vzD808DafWSb","colab_type":"text"},"source":["**Output layer using shared layer**\n","\n","Now that you've looked up how \"strong\" each team is, subtract the team strengths to determine which team is expected to win the game.\n","\n","The subtract layer will combine the weights from the two layers by subtracting them."]},{"cell_type":"code","metadata":{"id":"BeNrHeWhW6bf","colab_type":"code","colab":{}},"source":["# Import the Subtract layer from keras\n","from keras.layers import Subtract"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"YSYZKGZ5fiRz","colab_type":"code","colab":{}},"source":["# Create a subtract layer using the inputs from the previous exercise\n","score_diff = Subtract()([team_1_strength, team_2_strength])"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"lTRlhWm9f2_0","colab_type":"text"},"source":["\n","\n","*   Define a model with the two teams as inputs and use the score difference as the output.\n","*   Compile the model with the 'adam' optimizer and 'mean_absolute_error' loss.\n","\n"]},{"cell_type":"code","metadata":{"id":"lFOAAiHmf7FM","colab_type":"code","colab":{}},"source":["from keras.models import Model"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"5I4onGWAgZxE","colab_type":"code","colab":{}},"source":["# Subtraction layer from previous exercise\n","score_diff = Subtract()([team_1_strength, team_2_strength])\n","\n","# Create the model\n","model = Model([team_in_1, team_in_2], score_diff)\n","\n","# Compile the model\n","model.compile(optimizer='adam', loss='mean_absolute_error')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"_YbzIvmUW6i9","colab_type":"text"},"source":["#### Predict from your model"]},{"cell_type":"markdown","metadata":{"id":"JNRbsZgTglB0","colab_type":"text"},"source":["Since your model has two inputs now, you need to pass the input data as a list."]},{"cell_type":"code","metadata":{"id":"4NlH3x6Lgdqo","colab_type":"code","colab":{}},"source":["# Get the team_1 column from the regular season data\n","input_1 = games_season['team_1']\n","\n","# Get the team_2 column from the regular season data\n","input_2 = games_season['team_2']"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"zPv_7D_UhpM3","colab_type":"text"},"source":["\n","\n","*  Assign the 'team_1' and 'team_2' columns from games_season to input_1 and input_2, respectively.\n","*   Use 'score_diff' column from games_season as the target.\n","*   Fit the model using 1 epoch, a batch size of 2048, and a 10% validation split.\n","\n"]},{"cell_type":"code","metadata":{"id":"bMWVEetlW9tu","colab_type":"code","outputId":"317a886f-a186-4ee2-9f31-0b98893b3000","executionInfo":{"status":"ok","timestamp":1557396946180,"user_tz":-480,"elapsed":1308,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":156}},"source":["# Fit the model to input 1 and 2, using score diff as a target\n","model.fit([input_1, input_2],\n","          games_season['score_diff'],\n","          epochs=1,\n","          batch_size=2048,\n","          validation_split=0.10,\n","          verbose=True)"],"execution_count":0,"outputs":[{"output_type":"stream","text":["WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n","Instructions for updating:\n","Use tf.cast instead.\n","Train on 280960 samples, validate on 31218 samples\n","Epoch 1/1\n","280960/280960 [==============================] - 0s 2us/step - loss: 12.1201 - val_loss: 11.8379\n"],"name":"stdout"},{"output_type":"execute_result","data":{"text/plain":["<keras.callbacks.History at 0x7f16c7ca1e48>"]},"metadata":{"tags":[]},"execution_count":14}]},{"cell_type":"markdown","metadata":{"id":"pmXDdjVWh5Sh","colab_type":"text"},"source":["In this exercise, you will evaluate the model on this new dataset. This evaluation will tell you how well you can predict the tournament games, based on a model trained with the regular season data. \n"]},{"cell_type":"code","metadata":{"id":"QT-5cf35oCAY","colab_type":"code","outputId":"619f1a35-8138-465e-8d45-59af9f3571ae","executionInfo":{"status":"ok","timestamp":1557396949931,"user_tz":-480,"elapsed":972,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":34}},"source":["games_tourney = pd.read_csv(\"/content/games_tourney.csv\")\n","games_tourney.shape # (4234, 9)"],"execution_count":0,"outputs":[{"output_type":"execute_result","data":{"text/plain":["(4234, 9)"]},"metadata":{"tags":[]},"execution_count":15}]},{"cell_type":"code","metadata":{"id":"vPkAAMrVh5YS","colab_type":"code","outputId":"d7228c64-1e86-4ef5-93d4-7097bd034413","executionInfo":{"status":"ok","timestamp":1557396951429,"user_tz":-480,"elapsed":878,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":51}},"source":["# Get team_1 from the tournament data\n","input_1 = games_tourney['team_1']\n","\n","# Get team_2 from the tournament data\n","input_2 = games_tourney['team_2']\n","\n","# Evaluate the model using these inputs\n","model.evaluate([input_1, input_2], games_tourney['score_diff'])"],"execution_count":0,"outputs":[{"output_type":"stream","text":["4234/4234 [==============================] - 0s 19us/step\n"],"name":"stdout"},{"output_type":"execute_result","data":{"text/plain":["11.680718848544243"]},"metadata":{"tags":[]},"execution_count":16}]},{"cell_type":"markdown","metadata":{"id":"JO6sPqtSMD3W","colab_type":"text"},"source":["## 3. Multiple Inputs: 3 Inputs (and Beyond!)"]},{"cell_type":"markdown","metadata":{"id":"_G6IlUocpsM7","colab_type":"text"},"source":["#### Three-input models"]},{"cell_type":"code","metadata":{"id":"jqIMUILMMEAP","colab_type":"code","colab":{}},"source":["from keras.layers import Concatenate"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"X7aDuEN1jUvs","colab_type":"code","colab":{}},"source":["# Create an Input for each team\n","team_in_1 = Input(shape=(1,), name='Team-1-In')\n","team_in_2 = Input(shape=(1,), name='Team-2-In')"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"8Z8nlcNOjXBF","colab_type":"code","colab":{}},"source":["# Create an input for home vs away\n","home_in = Input(shape=(1,), name='Home-In')"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"YhforSJ_jYEE","colab_type":"code","colab":{}},"source":["# Lookup the team inputs in the team strength model\n","team_1_strength = team_strength_model(team_in_1)\n","team_2_strength = team_strength_model(team_in_2)"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"7_LjrjuCjZBs","colab_type":"code","colab":{}},"source":["# Combine the team strengths with the home input using a Concatenate layer, then add a Dense layer\n","out = Concatenate()([team_1_strength, team_2_strength, home_in])\n","out = Dense(1)(out)"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"me0pvWSPjcpt","colab_type":"code","colab":{}},"source":["# Make a Model\n","model = Model([team_in_1, team_in_2, home_in], out)\n","\n","# Compile the model\n","model.compile(optimizer='adam', loss='mean_absolute_error')"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"3EqeQjs3pR5F","colab_type":"code","outputId":"214d54a3-bcec-445a-80ac-c674cb30270f","executionInfo":{"status":"ok","timestamp":1557396960271,"user_tz":-480,"elapsed":1367,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":85}},"source":["# Fit the model to the games_season dataset\n","model.fit([games_season['team_1'], games_season['team_2'], games_season['home']],\n","          games_season['score_diff'],\n","          epochs=1,\n","          verbose=True,\n","          validation_split=.10,\n","          batch_size=2048)"],"execution_count":0,"outputs":[{"output_type":"stream","text":["Train on 280960 samples, validate on 31218 samples\n","Epoch 1/1\n","280960/280960 [==============================] - 1s 2us/step - loss: 12.1281 - val_loss: 12.0450\n"],"name":"stdout"},{"output_type":"execute_result","data":{"text/plain":["<keras.callbacks.History at 0x7f16c7ed79e8>"]},"metadata":{"tags":[]},"execution_count":22}]},{"cell_type":"code","metadata":{"id":"luU1Su7ipTaK","colab_type":"code","outputId":"5caa1cf5-b538-47db-9b80-eeac651c11f7","executionInfo":{"status":"ok","timestamp":1557396961520,"user_tz":-480,"elapsed":1119,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":51}},"source":["# Evaluate the model on the games_tourney dataset\n","model.evaluate([games_tourney['team_1'], \n","                games_tourney['team_2'], \n","                games_tourney['home']], \n","               games_tourney['score_diff'])"],"execution_count":0,"outputs":[{"output_type":"stream","text":["4234/4234 [==============================] - 0s 20us/step\n"],"name":"stdout"},{"output_type":"execute_result","data":{"text/plain":["11.68964503697759"]},"metadata":{"tags":[]},"execution_count":23}]},{"cell_type":"markdown","metadata":{"id":"r_Roy3DkpuO4","colab_type":"text"},"source":["#### Summarizing and plotting models"]},{"cell_type":"code","metadata":{"id":"lh3Njd4Jpwdx","colab_type":"code","colab":{}},"source":["# Imports\n","import matplotlib.pyplot as plt\n","from keras.utils import plot_model"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"2CA3Ouc-qcFT","colab_type":"code","outputId":"5b1320c8-fe5a-4cb2-f603-e0476f3c0384","executionInfo":{"status":"ok","timestamp":1557396964248,"user_tz":-480,"elapsed":1312,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":264}},"source":["# Plot the model\n","plot_model(model, to_file='model.png')\n","\n","# Display the image\n","data = plt.imread('model.png')\n","plt.imshow(data)\n","plt.show()"],"execution_count":0,"outputs":[{"output_type":"display_data","data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAXoAAAD3CAYAAAAT+Z8iAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xl8VOW9+PHPNwEySSAJlEDYkYAs\nXpUlIlivpdgqWCxiAqbValsqbvdaLNfGFq5WBa7bT25bQA2Xe1GRVaxYcMfSomUxKIIICERlXwsB\nkxCzfH9/zMxpQrbJMjnJyff9es0rZ5455znf5+SZb06eOfMcUVWMMcZ4V4TbARhjjAkvS/TGGONx\nluiNMcbjLNEbY4zHWaI3xhiPs0RvjDEeF7ZELyKjRGSXiOwRkQfCtR9jjDFVk3BcRy8ikcDnwPeB\nA8CHwI9U9bN635kxxpgqheuMfiiwR1WzVfUbYAkwNkz7MsYYU4VwJfouwP5Szw8EyowxxjSwFm7t\nWEQmAZMAYmNjh/Tr18+tUIwxpknavHnzCVVNrG69cCX6g0C3Us+7BsocqpoJZAKkpKRoVlZWmEIx\nxhhvEpGvQlkvXEM3HwJ9ROQCEWkFpAOvhWlfxhhjqhCWM3pVLRKRfwPeAiKB/1XV7dVtt379ep5+\n+ulwhGQagV/96lcMHz68xtuNHz8+DNGYxmD48OH86le/cjsMzwvbGL2qvg68XpNt9u/fz8svv4xN\nnexNy5Ytq1Witz7hXfZHvGHYN2ONMcbjLNEbY4zHWaI3xhiPs0RvjDEeZ4neGGM8zhK9McZ4nCV6\nY4zxOEv0xhjjcU0q0YtIhY9Vq1aFdb8lJSXMmjWr0tc3bNhAREQEIsL06dNrXH///v2JiIigY8eO\ndQmzSr169UJESEpKCts+3OBGn3jkkUcYMGAAUVFR/PrXv+brr78ut86GDRvo379/rftEsE917Nix\nVtuHwqt9wpTXpBJ9bm4uqsrhw4cZO3Ysqhr2b0x+8sknLF68mBdeeKHSdYYNG8Y111wDwLRp02q8\njx07dnDNNdewa9euWsdZkfz8fGc5Ozub+Ph4jhw5Uq/7cJsbfWLTpk1s27aNgoICvvrqK9q0aVNu\nnWHDhrFjxw6gdn0i2Kd27dpVq+0rk5+fzxVXXAH4+0RycrLn+oQpr0kl+piYmArLx4wZE7Z9Xnrp\npdx8881ERUWFbR/hMn/+fLdDCDs3+sSqVauIjIwEoH379mHbTzjMnz+fY8eOuR2GaWBNKtFXxefz\nsWLFCqZOnUpwyuOHH36Yf/zjH2XmVxER8vLyOHnyJCLCiRMnyMvLQ0TqLZZp06YhIpw5c4arrrqK\n1q1bA7B69Wri4uJC2v69997j+PHjXHXVVRQWFgJw5513smPHDgoKChg6dCj79/vv7XLLLbc42z71\n1FOcOHECgLVr14YU78svv+wcq2DiSktL49ChQ846f/7znwG4//77WbFiBadPn3aS3bRp05g8eTKp\nqans3LkzpH02hGCsU6dOdWJ9+OGHOXXqVL31iRdffJF77703pHiCfeL48eNOnwBq1CeCfaqmfSLY\nllD7BJR//6SlpdGixT+nx7r55psB/3H2+XxOn8jKynL6xOzZsxtVn2iuXLvxSH365JNP+M///E9S\nU1NJTU3l6quvZs2aNTz00EMApKamcuTIEWcsMiYmxjkTDOcZWVxcHLfeeivr1q0D4Ac/+AFnzpwJ\nadsrrrgCn8/Hrbfeyt69ewnemKV///6Af/jgxz/+MYsWLapznGlpaaSlpQFw8uRJjhw5wvLly+nd\nuzd79uxh3rx5TJo0CfAnjeDQSLdu/7zlwGOPPYbP56tzLPXlk08+cWJNTU3lpZdeAqjXPvHKK6+w\ndetWevbsGXJcwaSem5vrlDXGPgHlj9Xy5cvp168fqsq8efNITk4G/H1i+vTpJCQk0K1bNzIyMhg+\nfHij6xPNmSfO6HNzc52z6ODZGcCIESOcD8xCtWrVKqee0mdF5yv9wZ9bjh8/Xi/1rF692jlWQSJC\ndnY2a9asKff5RLDdX30V0j0PXBFMpOfHOmLECBITE+ulTzz22GNlkryX+gSUf/+ICHfeeafTJyZO\nnOisG3z/ffXVV877zzQenkj0gwcP5rbbbnM+iFu/fj2rV6/mkUceoaCggCeeeCLkusaMGePUs3Dh\nwkrXC67j1vS577zzjnMWXvrf6ZoIXkk0ZswY51iVdttttzFmzBimTJnilPl8PtfbHorBgweXizXY\nJ44fP16nPpGRkcGkSZPYtGlTmfXcPi712Scqe//cd999Tp/o0aMH4O8T57//TCNTunO69RgyZIiq\nqi5dulT9IVXuqquu0nbt2mlERITOmDHDKc/IyNDu3btrYmKibt++XVVV27Vrp+PHj9fZs2drcnKy\nTpkyRQFdt26dPvbYYwroSy+9pEuWLFFAFy9eXG5/69ev106dOimgSUlJ+te//rXcOhs2bNCIiAgF\ndMaMGRoTE6OA7t27V+Pi4hTQzz//XF9//XVt06ZNue0vuugijYiI0KSkJJ0zZ47GxMRonz59NDMz\nU+Pi4rRHjx76+eef6x133KFdunTRFi1a6A033OBsf/LkSfX5fHrBBRfo/fffr71799Z9+/bpRx99\npNHR0XrllVdqcnKyAuUewWMXPFaAJicnO3UPGjSoTKwFBQXavXt3bdGihaampurjjz+u0dHR2q1b\ntyp/b6r+329tVNcnzpw54/SJ3r17l4s1MTFRU1NTVdXfJxISEurUJyo7jqVt2LBBL7roIqdPzJkz\nx+kTmZmZTp9Q1Qr7RLBPJSUlOX2qT58+Tp+qaZ8AnD7Ro0ePavtE8FiVfv8EVdQnMjIynD6xfft2\np0+8+OKLVf7u0tLSqnzdVA3I0hByrOtJXmuY6JuzO+64o8H3ed1119VbXeFK9M1ZU+8TlujrJtRE\n74kPY0392717N+np6bz44otuh2Iaid27d6OqPPnkk26HYmrIEn0T8uyzzzbYvvr06cPmzZsbbH+m\ndhq6T5imyRMfxhpjjKmcJXpjjPG4Og3diMiXwFmgGChS1RQRaQcsBXoCXwITVPVU3cI0xhhTW/Vx\nRv9dVR2oqimB5w8Aa1S1D7Am8NwYY4xLwjF0MxZ4PrD8PHBDGPZhjDEmRHVN9Aq8LSKbRWRSoKyj\nqh4OLB8BwjfJujHGmGrV9fLKK1X1oIh0AN4RkTLT1KmqikiF3wcP/GGYBNC9e/c6hmGMMaYydTqj\nV9WDgZ/HgD8BQ4GjItIJIPCzwsmvVTVTVVNUNSUxMbEuYRhjjKlCrc/oRSQWiFDVs4Hla4BHgNeA\n24DHAj9X1rTu8ePH1zasZufs2bMV3uGoMarL77Wp9InTp0+TkJDgdhjGlCFay5n2RKQX/rN48P/B\nWKSqM0TkW8AyoDvwFf7LK/9RVV0pKSkavFmIqZm9e/fy/e9/n+zsbLdDada++eYbYmNjmTt3Lrff\nfrvb4ZhmQkQ2l7risVK1PqNX1Wzg0grKTwJX17ZeUzPJycl8+umn+Hw+XnjhBSZMmOB2SM3OoUOH\nSElJce76ZExjY9+M9YCYmBh+8YtfkJ6e7nYozc62bdsYNmwYGzZscDsUYyplid4jZs+eTUlJCdHR\n0eTn57sdTrPw/PPP8+ijj/L555/blWOmUbNE7zH5+fn069fPZhoMo+LiYu68807y8vJYtmyZ3RfV\nNHqW6D1ow4YNxMXF2TTDYTJ27FgWLlzIXXfd5XYoxoTEEr0HderUic2bN5OZmdmg85U3Bz179mTW\nrFl8/fXXbodiTMgs0XvYc889h8/no1WrVnZFSB2tWLGC6OhoduzYYcNipsmxRO9xP/3pT3n99dcZ\nPXq026E0aRMmTGDSpElER0e7HYoxNWa3EmwGvve979G/f3+6dOnC6tWrGThwoNshNRk33XQTK1eu\npLi42O1QjKk1O6NvJrp06UJ2djZPPvkkCxcudDucJqFfv348/PDDnDt3zu1QjKkTS/TNSFRUFAsX\nLuTWW291O5RGb9OmTcTFxdGvXz+3QzGmzizRNzMiQklJCbGxsVx77bVuh9MotWzZkueee45Nmza5\nHYox9cLG6Jup3NxcsrOz6dWrl02IFlBQUEBcXJxdoWQ8x87om7FevXrRpUsX1q1b53Yorjt16hTX\nXnstr732mtuhGFPvLNE3c+vWrWPZsmVERDTfrrBt2zauuuoqXnzxRRvOMp7UfN/dxvHHP/6x2U6I\ntmDBAp566im2bdtGt27d3A7HmLCwRG8c7733HiNHjuT48eNuh9Igfve73/Gzn/2MBQsWuB2KMWFl\nid44hg8fzp/+9CdGjx7N3//+d7fDCavrrruOyy67DFVFRNwOx5iwskRvykhKSiIrK4sXXniBZ555\nxu1w6t327dvp0aMHf/jDH/jBD37gdjjGNAi7vNJU6NlnnyUiIoKdO3fy+9//3u1w6s2VV15J//79\n6d27t9uhGNNg7IzeVKqkpIQf/vCHXH112VsADxo0iKKiIpeiCs3UqVOZMmVKmbLIyEhOnjzp+WEp\nY84nqup2DKSkpGhWVpbbYZhKBG9+vXr1asaMGcOhQ4dISEjg1KlTbodWqeDUzCtWrGDBggW8/fbb\nNmeN8RwR2ayqKdWtV+0ZvYj8r4gcE5FPS5W1E5F3RGR34GfbQLmIyB9EZI+IbBWRwXVrhmkMOnfu\nTN++fRkxYoRzRU5jvvHGxo0bnW+3pqens379ev7yl7+4HJUx7gll6GYBMOq8sgeANaraB1gTeA4w\nGugTeEwCvPdpXjPVs2dPcnNznQRaVFREamqqy1FVbNiwYc5yYWEhPp+PXr16uRiRMe6qNtGr6t+A\nf5xXPBZ4PrD8PHBDqfIX1G8DkCAineorWOOO//u//2PBggXl5mR/7bXXaAxDf6U98cQT5coOHDhA\nUlKSzSlvmq3afhjbUVUPB5aPAB0Dy12A/aXWOxAoM02Yz+cD/OPepRUVFbFo0SI3QqrQ8ePHeeSR\nR8qViwgdOnRo1MNNxoRTna+6Uf8pXY1P60RkkohkiUhWc/kmZlP1ox/9CFWloKCAZcuWAf6pfMF/\nq8IZM2a4GZ4jKSmJ3NxcwJ/c27Zty969eykpKeHo0aPEx8e7HKEx7qjtdfRHRaSTqh4ODM0cC5Qf\nBEpPGNI1UFaOqmYCmeC/6qaWcTRaBw4c8OxlfEuXLgXg7bffZv78+UybNs31G2Zv3brVma/nv/7r\nv+jUyT9imJWVhRev6JowYUKttgv+oTZN1xVXXEHXrl1rtpGqVvsAegKflnr+JPBAYPkB4InA8g+A\nNwABhgGbQql/yJAh6jVLly4N/qdjD3vU+6O23I7bHnV/LF26tPTvM0tDyLHVntGLyGJgBNBeRA4A\nDwGPActEZCLwFRA8vXgduA7YA+QBP6uufi/TRvZBpfGG8ePH13rbtLQ0li9fXo/RmIZU23mZqk30\nqvqjSl66+vyCwF+Ye2oViTHGmLCwKRCMMcbjLNEbY4zHWaI3xhiPs0RvjDEeZ4neGGM8zhK9McZ4\nnCV6Y4zxOEv0LkhPTycnJ4eioiJEhMLCQvLy8pg0aVJY9xsbG8vkyZM5d+4cGRkZfO973wvr/ioy\nbdq0kG4A0rt3bzp16uRMixzUs2fPGsctImzZsqXK10OJJ/i7Oj+emn6JZcmSJdXG8+yzz9aozvpy\n/i0WN2zY0KA3Tx81ahSnT5+u9fYiUqftq9O7d28SEhLCVn+4WKJ3wZIlS4iPj6dFC//31Vq2bElM\nTAyZmZlh3a+I8OCDD+Lz+Xj88cd59913AZg9e3a930N1+fLlTntmz55dLo5QHDlyhGuvvbZM2dNP\nP10/AdbC/Pnz6yWehkycpu68cH9hS/SNTHFxMQ8++CDR0dHO5GHr1q1jwIAB+Hw+3nrrLcB/dj5k\nyBA6duxIbGws//qv/0q3bt2qPNvIzc1l5MiR5cqnTJnC3r176d27NzExMbRp04YpU6awa9cuiouL\n6d69O5dccglLly5l7ty5xMTEsHLlSuLi4li8eLFTz8yZM+nbty/t27dn5syZTJgwgcmTJzNlypQy\nyW316tXEx8c7E49Vpn///mXuDPXBBx9wzTXXlFnn6aefpn///rRt25adO3c65U8++SR9+/YtN2Pl\ngw8+6LSnInFxcZXGM3LkyGrjUVUnnhtuuKFMeTCe+++/3yk///g2dhW177//+7+d/tiyZUunP/p8\nvjL9sSZtnTt3LrGxsaxcuZLRo0eXmcTrzTffrHbG1Mr66R/+8Ac6dOjAnXfeic/nY+PGjQDce++9\nJCUlAXDPPfcgIpw4cYLJkyezd+/ekP44r1u3jvj4eC6++GLeeustfvGLXyAiJCcnA/Czn/3M6Y/n\nH4snnniCNm3acOzYMbp0CcPM7qFMiBPuh1cnNQsF501QFRMTo+np6aqqGhUVpXfffXe59Y8dO6YP\nPfSQnj17VlVVn3/+ed22bZuqqm7atEmXLFlS4b6++eYb/f3vf6+AdujQQdeuXauqqqmpqZqcnKyq\nqlOnTnViys/P15iYGFVVzcvL06ioKCeG/Px8Z3nPnj2qqjp06FBnXxEREVpQUODUHzR16lRn2xde\neEE//fTTCmNNTk7W06dPa2xsrObl5enevXu1a9euevbsWb366qudmILHKhjLI488onl5efr973+/\nTPnHH3+s+fn5zvp5eXnOsT3/d1BZPF988YUTj6o68QS3z8vL09atWzvbbNq0yYkneBxVVRcvXuzE\nU9nxfeaZZyqNJS0trdp4a7ttcnJyhRNpVdU+Va22P1bW1vNde+21eurUKVUt21fmzJnj9LOqAM72\nlfXT+Pj4Mus//PDDqqrasWPHMuXHjx93jknp41N6+4rMnDnTOWbvvvuuTp8+XVVVc3JytKioKKT3\nVVXtq82kZnZG38jk5eWxZMkSRISCggI++ugjVq9ezYgRI4iKigIod6ekVq1aUVRUBPiHgYLjyCLi\nPIKv3XvvvagqqampjBgxospYPvroI/Ly8hARYmJiKCgoqHC94P6ysrJYuXIlWVlZjBs3rtyNSs4X\njHXVqlVOnLfccovzenx8PO+//z4jRozgiiuuYOvWrWW2/+CDD8q0oUOHDrz77rusXbuW9PT0CtsT\nPLYxMTF89NFHVcZXkWA8p0+frjCe0jc3ueyyy5x48vLyKownlOPb0EoniPXr1zvllbXvfBX1x4ra\nWtnvvSLBm73XRUXbt23blnXr1tWp3qDVq1eTmJjI1KlTnbKrr76aadOmAXDdddcRGRnpyu/dEn0j\nNGvWrDJvtHHjxpGUlEROTk6N6in9hgWcNx/AT37yk2q3T0xMrLCeyowcOZKf/vSn3HjjjcybNy/k\nOMeMGePUv3DhwjKvDRw4kE2bNjF+/Hjatm1b5rWEhATOnj3rPD99+jRdu3bF5/NV+OZJTEwsd2xr\nKhjPHXfcUWE85wvGU5GaHl+3Vda+UFTU1qp+7w0l2GfqYtasWezbt49x48axceNGHn/88TKv+3w+\ndu3a5QwLuvF7t0TfyNx1111kZGRw5swZDhw4wOHDhyksLOTCCy9k//791VdQhb59+7Js2TJycnIY\nN24cY8eOBaBdu3YcOnSoTNIE6NOnD3fddRdz586luLiYAwcOVFl/YWEhX3zxBfv27SuTBNu1a8eX\nX35Z6zOy9u3b88c//rFceUpKCr/97W9ZuHAh27Zt41vf+haZmZl897vf5d///d+ZP39+mbPuPn36\nkJGR4bTn8OHD5eqsaoy+dDwV3X82JSWFhx56yIln8ODBTjzjx4934gl+SF3T4+u2ytoXivps6xtv\nvFGnu5qVlJRw6tQptm7dSrdu3ZwrnI4ePcqrr77K+Xe8O3ToUIX9t7CwkKNHj9KhQwdycnIoLCyk\nc+fObNq0qcx6a9euZcSIEbz22muAS7/3UMZ3wv1ozmP0XjJ58mRn+b777qt0HNbUTTjH6JuDdu3a\nNfg+s7Oz66UebIzeuOnIkSNMnDjRed69e/c6j6kaEw4V/TcWDsH+v3XrVi644IIG2WdlLNGbepGU\nlMR9991HfHw8CQkJtGrVqtF8uGhM0G9/+1tycnIaJPE+/PDDREZG8tVXX4V9X9Wp7c3BjSnnnXfe\ncTsEY6o0c+ZMZs6c2SD7mj59OtOnT2+QfVXHzuiNMcbjLNEbY4zHWaI3xhiPs0RvjDEeV22iF5H/\nFZFjIvJpqbLfichBEdkSeFxX6rXfiMgeEdklItdWXKsxxpiGEsoZ/QJgVAXls1R1YODxOoCIDADS\ngYsC28wVkcj6CtYYY0zNVZvoVfVvwD9CrG8ssERVC1T1C2APMLQO8RljjKmjulxH/28iciuQBUxR\n1VNAF2BDqXUOBMrKEZFJwCTwf4vSi5YtW+Z2CMaD6jI3yoEDB6xfNkO1TfTPAI/in6v6UeD/AT+v\nSQWqmglkAqSkpDTuaftq6aabbnI7BGPK2LBhg/XLZqhWiV5VjwaXRWQesCrw9CDQrdSqXQNlzc6E\nCROYMGGC22E0SzfccAPXX399mbl3jJ82sqmQY2Ji+M53vsMbb7zhdiieVqvLK0Wk9D3gxgHBK3Je\nA9JFJEpELgD6AJvO396YcHr11Vfp0KED8fHxHDlyxO1wTAVOnDhBmzZtyMvLsyTfAKo9oxeRxcAI\noL2IHAAeAkaIyED8QzdfAncAqOp2EVkGfAYUAfeoasNMFWdMKddffz0bN24kJSWl0c/z3tx8/PHH\n3Hjjjfz97393O5RmQxrDv3IpKSmalZXldhjGg4qLi2nZsiW33347zz33nNvhNHsPPPAAp06dYs6c\nObRoYXMq1pWIbFbVlOrWsyNtPC0yMpKSkhLnbkJ1vUuXqZ0tW7Ywbtw4li9fTkpKtXnJ1DObAsE0\nC5dccgn9+/dnwYIFbofSLH3729+mR48eluRdYoneNBtvv/02Pp+P2NhYjh075nY4zcLJkyeJjo4m\nNzeXtWvXuh1Os2WJ3jQr6enp5ObmcvPNN/P888+7HY6nxcbGcuONN5Kfn+92KM2eJXrTLL355pv8\n9Kc/5Y477nA7FE964IEHuOWWW3j33XfdDsVgH8aaZioyMhJVZcuWLXTv3p19+/a5HZInbN26lXHj\nxrF48WKGDrVprhoLS/SmWRs4cCDZ2dmICJMmTbJLMOsgeOnkzp07admypdvhmFLsOnpjgEWLFnH7\n7bdz4MAB2rZt63Y4Tc6ZM2fo0KED586dczuUZiXU6+htjN4Y4Mc//jG5ubmkpaWxfPlyt8NpUuLj\n47nuuussyTdiluiNKWXNmjUUFBQQExPDqVOn3A6nUTtz5gw+n4+cnBzef/99t8MxVbBEb8x5brnl\nFt5//337MLEKu3btYtiwYXZtfBNhH8YaU4HBgwezY8cOIiIi+PWvf81jjz3mdkiNxuOPP052djZb\ntmyhVatWbodjQmBn9MZUokWLFpSUlDB+/HguvPBCtm/f7nZIrvr888+56KKLuOqqq3juuecsyTch\nluiNqcaQIUPo3Lkzw4YNczsUVw0dOpT4+HiGDx/udiimhmzoxpgQBMeio6KiOHr0KAkJCe4G1IDO\nnj1Lu3btKCwsdDsUU0uW6I2pgYKCAoYPH8727ds5c+aM2+GEXdu2benVq5cl+SbOhm6MqaG1a9dy\n00038cADD7gdSlg9/vjjjBkzxi6d9ABL9MbUUFRUFPPmzeP666+nb9++7Nixw3lt0aJFfPjhhy5G\nV3MffvghixYtcp7v2bOHiy++mKFDh/Liiy8SHR3tYnSmPtgUCMbUQUFBAXfffTeJiYksWrSIw4cP\no6oUFRW5HVrIIiMjiYiIIDExkV/+8pds27aNefPmWYJvAkKdAsESvTH1oGXLloiIM5adnZ3NBRdc\n4HJU1du3bx89evQA/G1QVRuPb0JsrhtjGlBRUVGZBNmrVy/+9Kc/uRhR9V555RUnyQMUFhY2qf9E\nTOiqTfQi0k1E/iIin4nIdhH5ZaC8nYi8IyK7Az/bBspFRP4gIntEZKuIDA53I4xxy7Fjx+jSpUuF\nr02YMKGBo6mZm266qcLyLl262K0WPSaUM/oiYIqqDgCGAfeIyADgAWCNqvYB1gSeA4wG+gQek4Bn\n6j1qYxqJgoICJk6cWOFrRUVFNNYhyaysrErP3idOnEhBQUEDR2TCqcZj9CKyEpgdeIxQ1cMi0glY\nq6p9ReS5wPLiwPq7gutVVqeN0RsvKCkp4e9//zujRo3i3LlzFBcXExkZ2SiHQyIjIykpKSEyMpLI\nyEhefvllRo0aZTcMaWJCHaOv0RemRKQnMAjYCHQslbyPAB0Dy12A/aU2OxAoqzTRGxO0bNmySocU\nmqLi4mJExO0wKlVcXExxcTE//OEP3Q6lTpYuXdroh8rcFHKiF5HWwApgsqqeKd15VVVFpEb/GojI\nJPxDO3Tv3r0mm5pmYOnSpW6HUG9WrVrFoEGDKh3Lb0gHDx7k448/ZsyYMW6HUm+8dGIQLiElehFp\niT/Jv6SqrwSKj4pIp1JDN8FPbw4C3Upt3jVQVoaqZgKZ4B+6qWX8xqO8dHbmpbY0RpboqxfKVTcC\nzAd2qOrTpV56DbgtsHwbsLJU+a2Bq2+GATlVjc8bY4wJr1DO6L8N/ATYJiJbAmW/BR4DlonIROAr\nIHja8jpwHbAHyAN+Vq8RG2OMqZFqE72qvg9U9mnS1RWsr8A9dYzLGGNMPbFvxhpjjMdZojfGGI+z\nRG+MMR5nid4YYzzOEr0xxnicJXpjGrErr7yyRlMotGrViu985zucPn2a7Oxsli5dSmZmZhgjNE2B\nJXpjKtG7d+96qWf27Nm1ruv999/n8ssvD2nd4P1d//rXv5KQkECvXr246aabmDRpUq32HarGcJxM\n1SzRmyYnJSUFn89Hz549AVBV+vfvT9u2bbnhhhsAmDt3LrGxsaxcuZK4uDi6du1aro7Y2FinjgED\nBhAfH8/FF18MwOTJk9m7d6+TeB588EG6d+/OJZdc4tQfExPD6NGjiYuLY/HixU7dAwYMwOfzOXVN\nmTLFqau4uJgHH3yQ6OjoWs/n8+abbzJjxoxy5WvWrGHo0KGVbhc8TlFRUc5xApzjNHr06AqPU+lj\nvW7dOuc4vfXWW85xCv7XEY5mRONrAAAQhklEQVTjFKzL1IGquv4YMmSIGqOqunTpUvV3y4olJCQ4\ny0VFRZqXl6etW7d2yjZt2qSPPPKIqqpOnTpV8/PzVVV1zpw5umfPHv3mm2/K1VHazJkz9dixY6qq\nmpycrKqq+fn5mp6erqqqeXl5evfdd6uqlomzopiDdaWmpjp1xcTEOHVFRUU5dVXl8ssvr3adymIo\nrSbHSbX8sS5t5syZzv7CfZxK11VZu5cuXVpl270KyNIQcqyd0Zsm5fTp085yZGQk27dv5+uvv3bK\nLrvsMjZu3Fhuu1atWlFYWMjWrVvL1VFay5YtKS4uLlO2a9cu/uVf/gWA6Ohodu7cGVKsFdWVl5fn\n1JWUlBRyXfWhJscJyh/r0iqatz5cx6kmdZmKWaI3nhMfH1/pa61bty5Xtm/fPtLT08nOzkYruBFP\nTEwMsbGxzvM1a9ZUWn+wrnPnzlVYF+DU9eWXX1ZZV02NHTuWDz74oFz5L37xiwrXr+o4VWTfvn20\natWqwY9TdXWZ6lmiN03KU089xb333svBgwc5e/YsKSkpPPTQQyxcuJBt27YxePDgKq8y6du3r1NH\nSUkJZ8+eJScnhwsvvJDOnTuzadMmZ91Dhw5x9uxZevbsSUZGBnPnzqW4uJjDhyufjDVY1/79+526\n2rVr59R11113kZGRwZkzZzhw4ECVdVXmjTfeqHCM/tVXX+Waa64hJSWFnJwcvvjiC+bNm0d0dDSA\nc5zOnDlT7XGC8sc6JyeHwsLCCo/Tl19+GbbjVF1dJgShjO+E+2Fj9CaoujF6VdWLL75YfT6fDho0\nSFVVS0pKtE+fPtq2bVsdN26cqvrHmmNiYrRPnz6amZmpcXFx2qNHj0rraNeunSYkJOjs2bM1OTlZ\n9+3bpz169NArr7xSjxw5ohkZGdq9e3dNTEzU7du365w5cxTQvXv3amZmpgL6+eefO3WNHz/eqeuj\njz5y6iooKNCMjAxt0aKFpqam6vbt2ytt5/r167VTp04KaFJSkv71r39VVdXXX39dp0+fXuE2+/bt\n0//4j//Q1q1ba0JCgg4aNEg/+OCDMsepZcuWznFSVec47d271zlOwbacf5wyMjKc4wQ4xyk6Ojps\nxylYV2WwMfpqc2yN7xkbDnbPWBMUvJVgY+iXpmkQkWZ7K8FQ7xlrQzfGGONxluiNMcbjLNEbY4zH\nWaI3xhiPs0RvjDEeZ4neGGM8zhK9McZ4XLWJXkS6ichfROQzEdkuIr8MlP9ORA6KyJbA47pS2/xG\nRPaIyC4RuTacDTDGGFO1FiGsUwRMUdWPRKQNsFlE3gm8NktVnyq9sogMANKBi4DOwLsicqGqlp21\nyBhjTIOoNtGr6mHgcGD5rIjsALpUsclYYImqFgBfiMgeYCiwvh7iNc1ETe6qZIypWihn9A4R6QkM\nAjYC3wb+TURuBbLwn/Wfwv9HYEOpzQ5Q9R8GYxxXXHFFrW/I4XWzZs3ivvvuczuMRumKK65wO4RG\nLeRELyKtgRXAZFU9IyLPAI8CGvj5/4Cf16C+ScAkgO7du9ckZuNhXbt2bZZzloTi5ZdftmNjaiWk\nq25EpCX+JP+Sqr4CoKpHVbVYVUuAefiHZwAOAt1Kbd41UFaGqmaqaoqqpiQmJtalDcYYY6oQylU3\nAswHdqjq06XKO5VabRzwaWD5NSBdRKJE5AKgD7AJY4wxrghl6ObbwE+AbSKyJVD2W+BHIjIQ/9DN\nl8AdAKq6XUSWAZ/hv2LnHrvixhhj3BPKVTfvAxVdAvF6FdvMAMrfAscYY0yDs2/GGmOMx1miN8YY\nj7NEb4wxHmeJ3hhjPM4SvTHGeJwlemOM8ThL9MYY43GW6I0xxuMs0RtjjMdZojfGGI+zRG+MMR5n\nid4YYzzOEr0xxnicJXpjjPG4Gt0z1hjTsLKzs53l3Nxc53lcXBzt27d3KyzTxFiiN6YRS05OLvP8\n9df9t4H4n//5HyZOnOhGSKYJsqEbYxqx66+/nhYtyp6PtWjRwpK8qRFL9MY0YsuWLaNVq1Zlylq2\nbOlSNKapskRvTCPm8/kYN25cmbLU1FSXojFNlaiq2zGQkpKiWVlZbodhTKNUVFRU5iz+m2++sbN6\nA4CIbFbVlOrWszN6Yxq588foLcmbmqo20YuIT0Q2icgnIrJdRB4OlF8gIhtFZI+ILBWRVoHyqMDz\nPYHXe4a3CcZ4X+fOnQFISkpyORLTFIVyRl8AjFTVS4GBwCgRGQY8DsxS1d7AKSB4GcBE4FSgfFZg\nPWNMHaxevRqAN9980+VITFNU7XX06h/E/zrwtGXgocBI4MeB8ueB3wHPAGMDywAvA7NFRLQxfBhg\nzHlEhLS0NLfDCMlFF13E9OnT3Q4jJC+//DL2lm88QvrClIhEApuB3sAcYC9wWlWLAqscALoElrsA\n+wFUtUhEcoBvASfqMW5j6s3y5cvdDiEke/fuLfcFqsZKRNwOwZQS0oexqlqsqgOBrsBQoF9ddywi\nk0QkS0Syjh8/XtfqjPG8ppLkTeNTo6tuVPU08BdgOJAgIsH/CLoCBwPLB4FuAIHX44GTFdSVqaop\nqpqSmJhYy/CNMcZUJ5SrbhJFJCGwHA18H9iBP+EHBzdvA1YGll8LPCfw+ns2Pm+MMe4JZYy+E/B8\nYJw+AlimqqtE5DNgiYhMBz4G5gfWnw+8KCJ7gH8A6WGI2xhjTIhCuepmKzCogvJs/OP155efA8bX\nS3TGGGPqzL4Za4wxHmeJ3hhjPM4SvTHGeJwlemNC0KZNG1e+BFRSUsKsWbNCWrdXr16ICCJCq1at\nGDFiBE8++WSYIzRNgSV6Y0Iwb968Bt/nJ598wuLFi3nhhRdCWj87O5v4+HhUlW+++Ya1a9dywQUX\n0LFjR3JycsIcrWnMLNEbEwI3zuYvvfRSbr75ZqKiompdR1paGseOHePZZ5+tx8hMU2OJ3phKTJs2\njaeffprTp09z8uQ/v9x9//334/P5OH36NJGRkWRlZTFt2jTee+89zpw5w1VXXUVhYSEA8fHxnDt3\njqNHj3LixIkKt62p1atX8+ijj9Zomzlz5gD+O1atWLGCqVOnOvsWEc6cOcPx48dp3bo1ALm5uTzx\nxBMcPXrUuaNVRduaJkJVXX8MGTJEjXGD/y1Q/WuLFy92ngM6ffp0VVXt0aOHjhw5UqdOnar5+fmq\nqjpv3jzdsWOHqqp26NBBH3roIf3iiy8q3bY6l19+ecjtiY+Pr7Ad3bt31y1btjj7VlVn36XbGVzO\ny8vThIQEJ+7Ktq1MVcfV1B8gS0PIsXZGb0wtTJs2DRHhq6++Ii8vr9L13nvvPWbMmEGvXr3Iz8+v\n0bb1qV+/fuTm5jr7FpEq9x0dHc2VV15Jr169SE9Pr9G2pvGxRG9MJSIiKn57+Hw+brvtNudsaf36\n9ZXWISIUFhZSUlLCz3/+8xptWx+OHDlC165dWb58OYMHDw5534cOHeLPf/4zJSUl7N69u0bbmsbH\nEr0xlcjJyaF9+/a0adOGHTt2ALB161ZycnJISkqiZcuWpKWl8dlnnzFr1iwuueQSsrOzmTJlCqNG\njWL37t3cfvvttGjRgi5durBw4cIKt63Mhg0b6Ny5Mxs3bqRTp0787W9/A+CNN95gxowZ5da/9NJL\nyc/PJzIykoiICC6//HLmzZvH/v37iYuLw+fzkZSURI8ePejQoQOfffYZc+fOBfxX7ASvLNq9ezeF\nhYW0a9eOLl26MHr06Aq3NU2HaCOYWDIlJUXtwx3jBhGxOyGFgR3XhiEim1U1pbr17IzeGBft3LnT\nGfcu/UhPt0lfTf0J6VaCxpjw6Nevn535mrCzM3pjjPE4S/TGGONxluiNMcbjLNEbY4zHWaI3xhiP\ns0RvjDEeZ4neGGM8rtpELyI+EdkkIp+IyHYReThQvkBEvhCRLYHHwEC5iMgfRGSPiGwVkcHhboQx\nxpjKhfKFqQJgpKp+LSItgfdF5I3Aa/er6svnrT8a6BN4XA48E/hpTKOTlpbG+PHj3Q7Dc9LS0twO\nwZRSbaIPzHn8deBpy8Cjqq/yjQVeCGy3QUQSRKSTqh6uc7TG1LPly5e7HYIxYRfSGL2IRIrIFuAY\n8I6qbgy8NCMwPDNLRIL3O+sC7C+1+YFAmTHGGBeElOhVtVhVBwJdgaEi8i/Ab4B+wGVAOyCjJjsW\nkUkikiUiWcePH69h2MYYY0JVo0nNVPW0iPwFGKWqTwWKC0Tk/4D/CDw/CHQrtVnXQNn5dWUCmQAi\nclxEcoETNYy/KWuPtdfLrL3e1xja3COUlapN9CKSCBQGknw08H3g8eC4u4gIcAPwaWCT14B/E5El\n+D+EzalufF5VE0UkK5R5lb3C2utt1l7va0ptDuWMvhPwvIhE4h/qWaaqq0TkvcAfAQG2AHcG1n8d\nuA7YA+QBP6v/sI0xxoQqlKtutgKDKigfWcn6CtxT99CMMcbUh8b0zdhMtwNoYNZeb7P2el+TaXOj\nuGesMcaY8GlMZ/TGGGPCwPVELyKjRGRXYG6cB9yOpz6IyP+KyDER+bRUWTsReUdEdgd+tg2UN/m5\ngUSkm4j8RUQ+C8yH9MtAuZfbXNkcUBeIyMZA25aKSKtAeVTg+Z7A6z3djL82Al+c/FhEVgWee7at\nACLypYhsC8zllRUoa5J92tVEH7iSZw7++XEGAD8SkQFuxlRPFgCjzit7AFijqn2ANYHnUHZuoEn4\n5wZqaoqAKao6ABgG3BP4PXq5zcE5oC4FBgKjRGQY8DgwS1V7A6eAiYH1JwKnAuWzAus1Nb8EdpR6\n7uW2Bn1XVQeWuoyyafZpVXXtAQwH3ir1/DfAb9yMqR7b1hP4tNTzXUCnwHInYFdg+TngRxWt11Qf\nwEr837doFm0GYoCP8H9v5ATQIlDu9G/gLWB4YLlFYD1xO/YatLEr/sQ2EliF/7JqT7a1VJu/BNqf\nV9Yk+7TbQzfNaV6cjvrPL44dAToGlj11DAL/pg8CNuLxNp8/BxSwFzitqkWBVUq3y2lz4PUc4FsN\nG3Gd/Dfwa6Ak8PxbeLetQQq8LSKbRWRSoKxJ9ukaTYFg6oeqqoh47nInEWkNrAAmq+oZ/5em/bzY\nZlUtBgaKSALwJ/xzP3mOiIwBjqnqZhEZ4XY8DehKVT0oIh2Ad0RkZ+kXm1KfdvuMPqR5cTziqIh0\nAgj8PBYo98QxEP+9ClYAL6nqK4FiT7c5SFVPA3/BP3yRICLBE6jS7XLaHHg9HjjZwKHW1reBH4rI\nl8AS/MM3v8ebbXWo6sHAz2P4/5APpYn2abcT/YdAn8Cn962AdPxz5XjRa8BtgeXb8I9jB8tvDXxq\nP4wQ5gZqbMR/6j4f2KGqT5d6ycttTgycySP/nANqB/6EH7zrxvltDh6LNOA9DQzmNnaq+htV7aqq\nPfG/R99T1ZvxYFuDRCRWRNoEl4Fr8M/n1TT7tNsfEuCfF+dz/OObU92Op57atBg4DBTiH6ubiH+M\ncg2wG3gXaBdYV/BfebQX2AakuB1/Ldp7Jf7xzK345z3aEvi9ernNlwAfB9r8KfBgoLwXsAn/XE/L\ngahAuS/wfE/g9V5ut6GW7R4BrPJ6WwNt+yTw2B7MTU21T9s3Y40xxuPcHroxxhgTZpbojTHG4yzR\nG2OMx1miN8YYj7NEb4wxHmeJ3hhjPM4SvTHGeJwlemOM8bj/DyorYg8JNZlqAAAAAElFTkSuQmCC\n","text/plain":["<Figure size 432x288 with 1 Axes>"]},"metadata":{"tags":[]}}]},{"cell_type":"markdown","metadata":{"id":"xTAFtJUWpwjx","colab_type":"text"},"source":["#### Stacking models"]},{"cell_type":"markdown","metadata":{"id":"eVTxEVbyqrTm","colab_type":"text"},"source":["You'll use the prediction from the regular season model as an input to the tournament model. This is a form of \"model stacking.\"\n","\n","To start, take the regular season model from the previous lesson, and predict on the tournament data. Add this prediction to the tournament data as a new column."]},{"cell_type":"code","metadata":{"id":"W5Hr_w4HqjsL","colab_type":"code","colab":{}},"source":["# Predict\n","games_tourney['pred'] = model.predict([games_tourney['team_1'], \n","                                             games_tourney['team_2'], \n","                                             games_tourney['home']])"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"Ub1dobm2rM0_","colab_type":"text"},"source":["Now you have three numeric columns in the tournament dataset: '`seed_diff`', '`home`', and '`pred`'. In this exercise, you will create a neural network that uses a single input layer to process all three of these numeric inputs."]},{"cell_type":"code","metadata":{"id":"9F7sUIlgrPBK","colab_type":"code","colab":{}},"source":["# Create an input layer with 3 columns\n","input_tensor = Input((3,))\n","\n","# Pass it to a Dense layer with 1 unit\n","output_tensor = Dense(1)(input_tensor)\n","\n","# Create a model\n","model = Model(input_tensor, output_tensor)\n","\n","# Compile the model\n","model.compile(optimizer='adam', loss='mean_absolute_error')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"vorXJwdvr1yV","colab_type":"text"},"source":["Tournament games are split into a training set and a test set. The tournament games before 2010 are in the training set, and the ones after 2010 are in the test set.\n","\n","*   Fit the model to the games_tourney_train dataset using 1 epoch.\n","*   The input columns are 'home', 'seed_diff', and 'pred'.\n","*   The target column is 'score_diff'.\n","\n"]},{"cell_type":"code","metadata":{"id":"RKS8JeGCsXS5","colab_type":"code","outputId":"6ecfd075-be8d-4ed6-f11a-80a14f88b182","executionInfo":{"status":"ok","timestamp":1557396976918,"user_tz":-480,"elapsed":1023,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":204}},"source":["games_tourney_train = games_tourney[:3430]\n","games_tourney_train.tail()"],"execution_count":0,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>season</th>\n","      <th>team_1</th>\n","      <th>team_2</th>\n","      <th>home</th>\n","      <th>seed_diff</th>\n","      <th>score_diff</th>\n","      <th>score_1</th>\n","      <th>score_2</th>\n","      <th>won</th>\n","      <th>pred</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>3425</th>\n","      <td>2011</td>\n","      <td>667</td>\n","      <td>10690</td>\n","      <td>0</td>\n","      <td>-9</td>\n","      <td>-14</td>\n","      <td>58</td>\n","      <td>72</td>\n","      <td>0</td>\n","      <td>0.105038</td>\n","    </tr>\n","    <tr>\n","      <th>3426</th>\n","      <td>2011</td>\n","      <td>1039</td>\n","      <td>10690</td>\n","      <td>0</td>\n","      <td>-4</td>\n","      <td>7</td>\n","      <td>61</td>\n","      <td>54</td>\n","      <td>1</td>\n","      <td>0.106724</td>\n","    </tr>\n","    <tr>\n","      <th>3427</th>\n","      <td>2011</td>\n","      <td>4209</td>\n","      <td>10690</td>\n","      <td>0</td>\n","      <td>-1</td>\n","      <td>-5</td>\n","      <td>65</td>\n","      <td>70</td>\n","      <td>0</td>\n","      <td>0.166182</td>\n","    </tr>\n","    <tr>\n","      <th>3428</th>\n","      <td>2011</td>\n","      <td>1073</td>\n","      <td>10713</td>\n","      <td>0</td>\n","      <td>11</td>\n","      <td>8</td>\n","      <td>74</td>\n","      <td>66</td>\n","      <td>1</td>\n","      <td>0.151377</td>\n","    </tr>\n","    <tr>\n","      <th>3429</th>\n","      <td>2011</td>\n","      <td>4899</td>\n","      <td>10812</td>\n","      <td>0</td>\n","      <td>-5</td>\n","      <td>11</td>\n","      <td>66</td>\n","      <td>55</td>\n","      <td>1</td>\n","      <td>0.104264</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["      season  team_1  team_2  home  seed_diff  score_diff  score_1  score_2  \\\n","3425    2011     667   10690     0         -9         -14       58       72   \n","3426    2011    1039   10690     0         -4           7       61       54   \n","3427    2011    4209   10690     0         -1          -5       65       70   \n","3428    2011    1073   10713     0         11           8       74       66   \n","3429    2011    4899   10812     0         -5          11       66       55   \n","\n","      won      pred  \n","3425    0  0.105038  \n","3426    1  0.106724  \n","3427    0  0.166182  \n","3428    1  0.151377  \n","3429    1  0.104264  "]},"metadata":{"tags":[]},"execution_count":27}]},{"cell_type":"code","metadata":{"id":"puTGBgwvslnD","colab_type":"code","outputId":"7a18fd70-350d-4eda-aa91-21b7330362a8","executionInfo":{"status":"ok","timestamp":1557396979224,"user_tz":-480,"elapsed":1080,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":34}},"source":["games_tourney_train.shape # (3430, 10)"],"execution_count":0,"outputs":[{"output_type":"execute_result","data":{"text/plain":["(3430, 10)"]},"metadata":{"tags":[]},"execution_count":28}]},{"cell_type":"code","metadata":{"id":"kJTOPBZfr-Al","colab_type":"code","outputId":"461f75c0-038a-4e44-a47a-b4258a8a72a0","executionInfo":{"status":"ok","timestamp":1557397006614,"user_tz":-480,"elapsed":1168,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":68}},"source":["# Fit the model\n","model.fit(games_tourney_train[['home', 'seed_diff', 'pred']],\n","          games_tourney_train['score_diff'],\n","          epochs=1,\n","          verbose=True)"],"execution_count":0,"outputs":[{"output_type":"stream","text":["Epoch 1/1\n","3430/3430 [==============================] - 0s 66us/step - loss: 9.1609\n"],"name":"stdout"},{"output_type":"execute_result","data":{"text/plain":["<keras.callbacks.History at 0x7f16c1ea4320>"]},"metadata":{"tags":[]},"execution_count":29}]},{"cell_type":"code","metadata":{"id":"dd9Q2y0MtZYP","colab_type":"code","outputId":"f64b9bd7-7f16-4b23-910e-06c66b2036af","executionInfo":{"status":"ok","timestamp":1557397009351,"user_tz":-480,"elapsed":945,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":204}},"source":["games_tourney_test = games_tourney[3430:]\n","games_tourney_test.head()"],"execution_count":0,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>season</th>\n","      <th>team_1</th>\n","      <th>team_2</th>\n","      <th>home</th>\n","      <th>seed_diff</th>\n","      <th>score_diff</th>\n","      <th>score_1</th>\n","      <th>score_2</th>\n","      <th>won</th>\n","      <th>pred</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>3430</th>\n","      <td>2012</td>\n","      <td>1907</td>\n","      <td>100</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>58</td>\n","      <td>57</td>\n","      <td>1</td>\n","      <td>0.140155</td>\n","    </tr>\n","    <tr>\n","      <th>3431</th>\n","      <td>2012</td>\n","      <td>1704</td>\n","      <td>649</td>\n","      <td>0</td>\n","      <td>-8</td>\n","      <td>-17</td>\n","      <td>63</td>\n","      <td>80</td>\n","      <td>0</td>\n","      <td>0.175183</td>\n","    </tr>\n","    <tr>\n","      <th>3432</th>\n","      <td>2012</td>\n","      <td>4291</td>\n","      <td>649</td>\n","      <td>0</td>\n","      <td>2</td>\n","      <td>12</td>\n","      <td>82</td>\n","      <td>70</td>\n","      <td>1</td>\n","      <td>0.163649</td>\n","    </tr>\n","    <tr>\n","      <th>3433</th>\n","      <td>2012</td>\n","      <td>7533</td>\n","      <td>649</td>\n","      <td>0</td>\n","      <td>-11</td>\n","      <td>-8</td>\n","      <td>60</td>\n","      <td>68</td>\n","      <td>0</td>\n","      <td>0.191677</td>\n","    </tr>\n","    <tr>\n","      <th>3434</th>\n","      <td>2012</td>\n","      <td>10813</td>\n","      <td>649</td>\n","      <td>0</td>\n","      <td>-7</td>\n","      <td>-5</td>\n","      <td>70</td>\n","      <td>75</td>\n","      <td>0</td>\n","      <td>0.161060</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["      season  team_1  team_2  home  seed_diff  score_diff  score_1  score_2  \\\n","3430    2012    1907     100     0          1           1       58       57   \n","3431    2012    1704     649     0         -8         -17       63       80   \n","3432    2012    4291     649     0          2          12       82       70   \n","3433    2012    7533     649     0        -11          -8       60       68   \n","3434    2012   10813     649     0         -7          -5       70       75   \n","\n","      won      pred  \n","3430    1  0.140155  \n","3431    0  0.175183  \n","3432    1  0.163649  \n","3433    0  0.191677  \n","3434    0  0.161060  "]},"metadata":{"tags":[]},"execution_count":30}]},{"cell_type":"code","metadata":{"id":"JaCTSZWntlsa","colab_type":"code","outputId":"b4fc637d-309b-4298-da70-81d9e81bcdd5","executionInfo":{"status":"ok","timestamp":1557397011640,"user_tz":-480,"elapsed":780,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":51}},"source":["model.evaluate(games_tourney_test[['home', 'seed_diff', 'pred']], \n","               games_tourney_test['score_diff'])"],"execution_count":0,"outputs":[{"output_type":"stream","text":["804/804 [==============================] - 0s 64us/step\n"],"name":"stdout"},{"output_type":"execute_result","data":{"text/plain":["9.204197300014211"]},"metadata":{"tags":[]},"execution_count":31}]},{"cell_type":"markdown","metadata":{"id":"7RESOllTMJXh","colab_type":"text"},"source":["## 4. Multiple Outputs"]},{"cell_type":"markdown","metadata":{"id":"ymz-Kv1Jpz03","colab_type":"text"},"source":["#### Two-output model"]},{"cell_type":"markdown","metadata":{"id":"p2hbt9S0D6iw","colab_type":"text"},"source":["In this exercise, you will use the tournament data to build one model that makes two predictions: the scores of both teams in a given game. Your inputs will be the seed difference of the two teams, as well as the predicted score difference from the model you built in chapter 3.\n","\n","The output from your model will be the predicted score for team 1 as well as team 2. This is called \"multiple target regression\": one model making more than one prediction."]},{"cell_type":"code","metadata":{"id":"TgUPIVCsMLd7","colab_type":"code","colab":{}},"source":["# Define the input\n","input_tensor = Input((2,))\n","\n","# Define the output\n","output_tensor = Dense(2)(input_tensor)\n","\n","# Create a model\n","model = Model(input_tensor, output_tensor)\n","\n","# Compile the model\n","model.compile(optimizer='adam', loss='mean_absolute_error')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"0a22X68tGhJk","colab_type":"text"},"source":["\n","\n","*   Fit the model to the games_tourney_train dataset using 100 epochs and a batch size of 16384.\n","\n","*   The input columns are 'seed_diff', and 'pred'.\n","\n","*   The target columns are 'score_1' and 'score_2'.\n","\n","\n","\n"]},{"cell_type":"code","metadata":{"id":"6495m7QNEbs4","colab_type":"code","outputId":"17084c9b-9c22-4394-d642-feb878e5de2e","executionInfo":{"status":"ok","timestamp":1557397186574,"user_tz":-480,"elapsed":1387,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":3434}},"source":["# Fit the model\n","model.fit(games_tourney_train[['seed_diff', 'pred']],\n","  \t\t  games_tourney_train[['score_1', 'score_2']],\n","  \t\t  verbose=True,\n","  \t\t  epochs=100,\n","  \t\t  batch_size=16384)"],"execution_count":0,"outputs":[{"output_type":"stream","text":["Epoch 1/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.1375\n","Epoch 2/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.1363\n","Epoch 3/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.1352\n","Epoch 4/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.1340\n","Epoch 5/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.1329\n","Epoch 6/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.1318\n","Epoch 7/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.1306\n","Epoch 8/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.1295\n","Epoch 9/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.1284\n","Epoch 10/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.1272\n","Epoch 11/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.1261\n","Epoch 12/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.1249\n","Epoch 13/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.1238\n","Epoch 14/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.1227\n","Epoch 15/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.1215\n","Epoch 16/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.1204\n","Epoch 17/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.1193\n","Epoch 18/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.1181\n","Epoch 19/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.1170\n","Epoch 20/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.1158\n","Epoch 21/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.1147\n","Epoch 22/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.1136\n","Epoch 23/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.1124\n","Epoch 24/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.1113\n","Epoch 25/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.1102\n","Epoch 26/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.1090\n","Epoch 27/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.1079\n","Epoch 28/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.1067\n","Epoch 29/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.1056\n","Epoch 30/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.1045\n","Epoch 31/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.1033\n","Epoch 32/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.1022\n","Epoch 33/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.1011\n","Epoch 34/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.0999\n","Epoch 35/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.0988\n","Epoch 36/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.0976\n","Epoch 37/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0965\n","Epoch 38/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.0954\n","Epoch 39/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.0942\n","Epoch 40/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0931\n","Epoch 41/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.0919\n","Epoch 42/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.0908\n","Epoch 43/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.0897\n","Epoch 44/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.0885\n","Epoch 45/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0874\n","Epoch 46/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0863\n","Epoch 47/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0851\n","Epoch 48/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0840\n","Epoch 49/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.0829\n","Epoch 50/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.0817\n","Epoch 51/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.0806\n","Epoch 52/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0794\n","Epoch 53/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.0783\n","Epoch 54/100\n","3430/3430 [==============================] - 0s 4us/step - loss: 71.0772\n","Epoch 55/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.0760\n","Epoch 56/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0749\n","Epoch 57/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0738\n","Epoch 58/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0726\n","Epoch 59/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0715\n","Epoch 60/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0703\n","Epoch 61/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0692\n","Epoch 62/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0681\n","Epoch 63/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0669\n","Epoch 64/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0658\n","Epoch 65/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.0646\n","Epoch 66/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0635\n","Epoch 67/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0624\n","Epoch 68/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0612\n","Epoch 69/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0601\n","Epoch 70/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0590\n","Epoch 71/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0578\n","Epoch 72/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0567\n","Epoch 73/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0555\n","Epoch 74/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0544\n","Epoch 75/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0533\n","Epoch 76/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0521\n","Epoch 77/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0510\n","Epoch 78/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0498\n","Epoch 79/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.0487\n","Epoch 80/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0476\n","Epoch 81/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.0465\n","Epoch 82/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0453\n","Epoch 83/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.0442\n","Epoch 84/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0430\n","Epoch 85/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.0419\n","Epoch 86/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0408\n","Epoch 87/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.0396\n","Epoch 88/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.0385\n","Epoch 89/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0373\n","Epoch 90/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0362\n","Epoch 91/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.0351\n","Epoch 92/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0339\n","Epoch 93/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0328\n","Epoch 94/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0317\n","Epoch 95/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0305\n","Epoch 96/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.0294\n","Epoch 97/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0282\n","Epoch 98/100\n","3430/3430 [==============================] - 0s 2us/step - loss: 71.0271\n","Epoch 99/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0260\n","Epoch 100/100\n","3430/3430 [==============================] - 0s 1us/step - loss: 71.0248\n"],"name":"stdout"},{"output_type":"execute_result","data":{"text/plain":["<keras.callbacks.History at 0x7f16c1ec9390>"]},"metadata":{"tags":[]},"execution_count":38}]},{"cell_type":"markdown","metadata":{"id":"RMDChpcQEbQ2","colab_type":"text"},"source":["Now that you've fit your model, let's take a look at it. You can use the .get_weights() method to inspect your model's weights.\n","\n","The input layer will have 4 weights: 2 for each input times 2 for each output.\n","\n","The output layer will have 2 weights, one for each output."]},{"cell_type":"code","metadata":{"id":"hwUaW2a0EfSy","colab_type":"code","outputId":"792a1554-b4f5-4d9e-ad3b-8ddb950c1558","executionInfo":{"status":"ok","timestamp":1557397317362,"user_tz":-480,"elapsed":1085,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":51}},"source":["# Print the model's weights\n","print(model.get_weights())"],"execution_count":0,"outputs":[{"output_type":"stream","text":["[array([[-0.70227855, -0.4406793 ],\n","       [ 1.2666051 ,  0.15003994]], dtype=float32), array([0.5007364, 0.5007364], dtype=float32)]\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"w4uAvpq1Ewin","colab_type":"code","outputId":"75707df9-ab47-4872-8f27-c24373679108","executionInfo":{"status":"ok","timestamp":1557397323317,"user_tz":-480,"elapsed":763,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":204}},"source":["# Print the column means of the training data\n","print(games_tourney_train.mean())"],"execution_count":0,"outputs":[{"output_type":"stream","text":["season        1998.074052\n","team_1        5556.771429\n","team_2        5556.771429\n","home             0.000000\n","seed_diff        0.000000\n","score_diff       0.000000\n","score_1         71.621283\n","score_2         71.621283\n","won              0.500000\n","pred             0.136735\n","dtype: float64\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"u4SocJJeHYZh","colab_type":"text"},"source":["Now that you've fit your model and inspected it's weights to make sure it makes sense, evaluate it on the tournament test set to see how well it performs on new data."]},{"cell_type":"code","metadata":{"id":"_KFTc_xrEc0U","colab_type":"code","outputId":"c54c787d-ef20-448f-9f54-673d941654f8","executionInfo":{"status":"ok","timestamp":1557397448702,"user_tz":-480,"elapsed":1067,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":51}},"source":["model.evaluate(games_tourney_test[['seed_diff', 'pred']],\n","               games_tourney_test[['score_1', 'score_2']])"],"execution_count":0,"outputs":[{"output_type":"stream","text":["804/804 [==============================] - 0s 72us/step\n"],"name":"stdout"},{"output_type":"execute_result","data":{"text/plain":["68.44189790943962"]},"metadata":{"tags":[]},"execution_count":41}]},{"cell_type":"markdown","metadata":{"id":"3hJFPRVVp13e","colab_type":"text"},"source":["#### Single model for classification and regression"]},{"cell_type":"markdown","metadata":{"id":"eqpunqq8EGS7","colab_type":"text"},"source":[" This time, you will predict the score difference, instead of both team's scores and then you will predict the probability that team 1 won the game. \n"," \n"," This is a pretty cool model: it is going to do both classification and regression!\n"," \n"," In this model, **turn off the bias, or intercept for each layer**. Your inputs (seed difference and predicted score difference) have a mean of very close to zero, and your outputs both have means that are close to zero, so your model shouldn't need the bias term to fit the data well."]},{"cell_type":"code","metadata":{"id":"CVB8dHIEp42t","colab_type":"code","colab":{}},"source":["# Create an input layer with 2 columns\n","input_tensor = Input((2,))\n","\n","# Create the first output\n","output_tensor_1 = Dense(1, activation='linear', use_bias=False)(input_tensor)\n","\n","# Create the second output (use the first output as input here)\n","output_tensor_2 = Dense(1, activation='sigmoid', use_bias=False)(output_tensor_1)\n","\n","# Create a model with 2 outputs\n","model = Model(input_tensor, [output_tensor_1, output_tensor_2])"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"lMWe3H_7I_UD","colab_type":"code","colab":{}},"source":["# Import the Adam optimizer\n","from keras.optimizers import Adam"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"DEceYeAogiwm","colab_type":"text"},"source":["Compile the model with 2 losses: 'mean_absolute_error' and 'binary_crossentropy', and use the Adam optimizer with a learning rate of 0.01."]},{"cell_type":"code","metadata":{"id":"6-Ye4QxMJhfp","colab_type":"code","colab":{}},"source":["# Compile the model with 2 losses and the Adam optimzer with a higher learning rate\n","model.compile(loss=['mean_absolute_error', 'binary_crossentropy'], optimizer=Adam(lr=0.01))"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"jWqscOWWJlSD","colab_type":"text"},"source":["Fit the model with 'seed_diff' and 'pred' columns as the inputs and 'score_diff' and 'won' columns as the targets."]},{"cell_type":"code","metadata":{"id":"EeNi4LbdJl8u","colab_type":"code","outputId":"22bca56c-182f-4654-d323-fc039ca79a57","executionInfo":{"status":"ok","timestamp":1557397926287,"user_tz":-480,"elapsed":1403,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":394}},"source":["# Fit the model to the tournament training data, with 2 inputs and 2 outputs\n","model.fit(games_tourney_train[['seed_diff', 'pred']],\n","          [games_tourney_train[['score_diff']], games_tourney_train[['won']]],\n","          epochs=10,\n","          verbose=True,\n","          batch_size=16384)"],"execution_count":0,"outputs":[{"output_type":"stream","text":["Epoch 1/10\n","3430/3430 [==============================] - 0s 70us/step - loss: 12.7283 - dense_4_loss: 10.3674 - dense_5_loss: 2.3609\n","Epoch 2/10\n","3430/3430 [==============================] - 0s 1us/step - loss: 12.6984 - dense_4_loss: 10.3890 - dense_5_loss: 2.3094\n","Epoch 3/10\n","3430/3430 [==============================] - 0s 2us/step - loss: 12.6649 - dense_4_loss: 10.4140 - dense_5_loss: 2.2509\n","Epoch 4/10\n","3430/3430 [==============================] - 0s 2us/step - loss: 12.6310 - dense_4_loss: 10.4410 - dense_5_loss: 2.1900\n","Epoch 5/10\n","3430/3430 [==============================] - 0s 2us/step - loss: 12.5977 - dense_4_loss: 10.4692 - dense_5_loss: 2.1285\n","Epoch 6/10\n","3430/3430 [==============================] - 0s 1us/step - loss: 12.5655 - dense_4_loss: 10.4983 - dense_5_loss: 2.0672\n","Epoch 7/10\n","3430/3430 [==============================] - 0s 1us/step - loss: 12.5347 - dense_4_loss: 10.5280 - dense_5_loss: 2.0067\n","Epoch 8/10\n","3430/3430 [==============================] - 0s 2us/step - loss: 12.5051 - dense_4_loss: 10.5578 - dense_5_loss: 1.9473\n","Epoch 9/10\n","3430/3430 [==============================] - 0s 2us/step - loss: 12.4770 - dense_4_loss: 10.5877 - dense_5_loss: 1.8893\n","Epoch 10/10\n","3430/3430 [==============================] - 0s 1us/step - loss: 12.4502 - dense_4_loss: 10.6175 - dense_5_loss: 1.8327\n"],"name":"stdout"},{"output_type":"execute_result","data":{"text/plain":["<keras.callbacks.History at 0x7f16c73ef2e8>"]},"metadata":{"tags":[]},"execution_count":45}]},{"cell_type":"code","metadata":{"id":"Arz5jiNGJwvi","colab_type":"code","outputId":"3c73070a-60d6-4bc1-b298-b4898e36652d","executionInfo":{"status":"ok","timestamp":1557397986339,"user_tz":-480,"elapsed":788,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":51}},"source":["# Print the model's weights\n","print(model.get_weights())"],"execution_count":0,"outputs":[{"output_type":"stream","text":["[array([[ 0.31350645],\n","       [-0.4308766 ]], dtype=float32), array([[-0.9753931]], dtype=float32)]\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"eVor4GjGJ4QG","colab_type":"code","outputId":"af8a29bd-16dd-4885-d3d1-b4929948ee59","executionInfo":{"status":"ok","timestamp":1557397998641,"user_tz":-480,"elapsed":972,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":204}},"source":["# Print the column means of the training data\n","print(games_tourney_train.mean())"],"execution_count":0,"outputs":[{"output_type":"stream","text":["season        1998.074052\n","team_1        5556.771429\n","team_2        5556.771429\n","home             0.000000\n","seed_diff        0.000000\n","score_diff       0.000000\n","score_1         71.621283\n","score_2         71.621283\n","won              0.500000\n","pred             0.136735\n","dtype: float64\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"G0tCpywbJ6aq","colab_type":"code","colab":{}},"source":["# Import the sigmoid function from scipy\n","from scipy.special import expit as sigmoid"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"Ei-63JU9KAjd","colab_type":"code","colab":{}},"source":["# Weight from the model\n","weight = 0.14"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"_h57kRuRKFTg","colab_type":"text"},"source":["In particular, note the last weight of the model. This weight converts the predicted score difference to a predicted win probability. If you multiply the predicted score difference by the last weight of the model and then apply the sigmoid function, you get the win probability of the game."]},{"cell_type":"code","metadata":{"id":"vqTrkRYHKBni","colab_type":"code","outputId":"7ee11963-a954-45fb-d4c4-14c12a241ec1","executionInfo":{"status":"ok","timestamp":1557398103294,"user_tz":-480,"elapsed":763,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":34}},"source":["# Print the approximate win probability of a predicted close game\n","print(sigmoid(1 * weight))"],"execution_count":0,"outputs":[{"output_type":"stream","text":["0.5349429451582145\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"8t09zMYpKSUf","colab_type":"code","outputId":"30bc1de1-082d-4bf4-c27b-f07bd78ebcc6","executionInfo":{"status":"ok","timestamp":1557398107410,"user_tz":-480,"elapsed":767,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":34}},"source":["# Print the approximate win probability of a predicted blowout game\n","print(sigmoid(10 * weight))"],"execution_count":0,"outputs":[{"output_type":"stream","text":["0.8021838885585818\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"UxTVncuDKfLH","colab_type":"text"},"source":["So sigmoid(1 * 0.14) is 0.53, which represents a pretty close game and sigmoid(10 * 0.14) is 0.80, which represents a pretty likely win. In other words, if the model predicts a win of 1 point, it is less sure of the win than if it predicts 10 points. Who says neural networks are black boxes?"]},{"cell_type":"code","metadata":{"id":"3cvib7WOKpdw","colab_type":"code","outputId":"d416654d-a87d-41f6-8308-d76749b595d0","executionInfo":{"status":"ok","timestamp":1557398203145,"user_tz":-480,"elapsed":767,"user":{"displayName":"Zhenna LU","photoUrl":"","userId":"13596549313510787594"}},"colab":{"base_uri":"https://localhost:8080/","height":51}},"source":["# Evaluate the model on new data\n","model.evaluate(games_tourney_test[['seed_diff', 'pred']],\n","               [games_tourney_test[['score_diff']], games_tourney_test[['won']]])"],"execution_count":0,"outputs":[{"output_type":"stream","text":["804/804 [==============================] - 0s 95us/step\n"],"name":"stdout"},{"output_type":"execute_result","data":{"text/plain":["[11.968475550561402, 10.283499997646654, 1.684975550245883]"]},"metadata":{"tags":[]},"execution_count":52}]}]}