_notebooks/2020-08-12-01-Kaggle-competitions-process.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Kaggle competitions process\n",
    "> In this first chapter, you will get exposure to the Kaggle competition process. You will train a model and prepare a csv file ready for submission. You will learn the difference between Public and Private test splits, and how to prevent overfitting. This is the Summary of lecture \"Winning a Kaggle Competition in Python\", via datacamp.\n",
    "\n",
    "- toc: true \n",
    "- badges: true\n",
    "- comments: true\n",
    "- author: Chanseok Kang\n",
    "- categories: [Python, Datacamp, Kaggle, Machine_Learning]\n",
    "- image: images/Kaggle_logo.png"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Competitions overview\n",
    "- Kaggle benefits\n",
    "    - Get practical experience on the real-world data\n",
    "    - Develop portfolio projects\n",
    "    - Meet a great Data Science community\n",
    "    - Try new domain or model type\n",
    "    - Keep up-to-date with the best performing methods\n",
    "- Process\n",
    "![process](image/competition_process.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Explore train data\n",
    "You will work with another Kaggle competition called \"Store Item Demand Forecasting Challenge\". In this competition, you are given 5 years of store-item sales data, and asked to predict 3 months of sales for 50 different items in 10 different stores.\n",
    "\n",
    "To begin, let's explore the train data for this competition. For the faster performance, you will work with a subset of the train data containing only a single month history.\n",
    "\n",
    "Your initial goal is to read the input data and take the first look at it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train shape: (15500, 5)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>date</th>\n",
       "      <th>store</th>\n",
       "      <th>item</th>\n",
       "      <th>sales</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>100000</td>\n",
       "      <td>2017-12-01</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>100001</td>\n",
       "      <td>2017-12-02</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>100002</td>\n",
       "      <td>2017-12-03</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>31</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>100003</td>\n",
       "      <td>2017-12-04</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>100004</td>\n",
       "      <td>2017-12-05</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       id        date  store  item  sales\n",
       "0  100000  2017-12-01      1     1     19\n",
       "1  100001  2017-12-02      1     1     16\n",
       "2  100002  2017-12-03      1     1     31\n",
       "3  100003  2017-12-04      1     1      7\n",
       "4  100004  2017-12-05      1     1     20"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Read train data\n",
    "train = pd.read_csv('./dataset/demand_forecasting_train_1_month.csv')\n",
    "\n",
    "# Look at the shape of the data\n",
    "print('Train shape:', train.shape)\n",
    "\n",
    "# Look at the head() of the data\n",
    "train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Explore test data\n",
    "Having looked at the train data, let's explore the test data in the \"Store Item Demand Forecasting Challenge\". Remember, that the test dataset generally contains one column less than the train one.\n",
    "\n",
    "This column, together with the output format, is presented in the sample submission file. Before making any progress in the competition, you should get familiar with the expected output.\n",
    "\n",
    "That is why, let's look at the columns of the test dataset and compare it to the train columns. Additionally, let's explore the format of the sample submission. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train columns: ['id', 'date', 'store', 'item', 'sales']\n",
      "Test columns: ['id', 'date', 'store', 'item']\n"
     ]
    }
   ],
   "source": [
    "# Read the test data\n",
    "test = pd.read_csv('./dataset/demand_forecasting_test.csv')\n",
    "\n",
    "# Print train and test columns\n",
    "print('Train columns:', train.columns.tolist())\n",
    "print('Test columns:', test.columns.tolist())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>sales</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>52</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>52</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>52</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>52</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>52</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id  sales\n",
       "0   0     52\n",
       "1   1     52\n",
       "2   2     52\n",
       "3   3     52\n",
       "4   4     52"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Read the sample submission file\n",
    "sample_submission = pd.read_csv('./dataset/sample_submission.csv')\n",
    "\n",
    "# Look at the head() of the sample submission\n",
    "sample_submission.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The sample submission file consists of two columns: `id` of the observation and `sales` column for your predictions. Kaggle will evaluate your predictions on the true `sales` data for the corresponding `id`. So, it’s important to keep track of the predictions by `id` before submitting them."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prepare your first submission\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Determine a problem type\n",
    "You will keep working on the Store Item Demand Forecasting Challenge. Recall that you are given a history of store-item sales data, and asked to predict 3 months of the future sales.\n",
    "\n",
    "Before building a model, you should determine the problem type you are addressing. The goal of this exercise is to look at the distribution of the target variable, and select the correct problem type you will be building a model for."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAbqklEQVR4nO3df5Bc1Xnm8e+DAKEgBwmDp4SktciiJQYUyzALcsCbERgQkFi4yqxFYZAwiZxdqOBdVYywKwsBk5UrxmyoBRLZyAhDGDA2ixZEiCIzYUkWg0QUCSETxmgCg2QJIyEYwLIHv/vHPbNuhu7pntFM/9B5PlVdffvc0/e+t388ffv07W5FBGZmlocDGl2AmZnVj0PfzCwjDn0zs4w49M3MMuLQNzPLiEPfzCwjDn0bU5J6JH2ywrxPSHq+3jU1ExW+LWm3pKdGcbnXSrprtJZn+48DG12A5Ssi/g9wbLV+kq4FjomIz415UfV3GnAmMC0i3mp0Mbb/856+ZU1So3d8Pgz0OPCtXhz6Vg+zJW2UtEfSvZIOAZDUIal3oJOkqyS9IulNSc9LOkPSPODLwGcl9Un659T3KEmrJO2S1C3pD0qWM0HSyjRkskXSlwatpyetayPwlqQDJS2V9OO07uckfbqk/yJJ/yDpJkmvS3pR0m+n9pcl7ZS0sNLGV6pV0mXAt4CPp2370zLXPUbS36fb7qeS7i2Z9xdp/W9IWi/pE0PUMEfSP6b6/1lSx6DtezFt+1ZJF1Vaju0HIsInn8bsBPQATwFHAYcDW4A/TPM6gN40fSzwMnBUujwD+Ldp+lrgrkHL/XvgVuAQYDbwKnBGmrcszZ8MTAM2DqynpKYNwHRgQmq7INV4APBZ4C1gSpq3COgHLgXGAV8FXgJuAcYDZwFvAhMr3AZD1boIeGKI2+8e4CuprkOA00rmfQ74IMUw7RLgJ8Ahg28zYCrwGnBuWs6Z6fKRwKHAG8Cxqe8U4PhGP258GruT9/StHm6OiG0RsQv43xTBN9i7FAF6nKSDIqInIn5cbmGSplOMhV8VET+LiA0Ue8wXpy7/EfiziNgdEb3AzRVqejki3gGIiO+mGn8ZEfcCLwAnl/TfGhHfjoh3gXspXjCui4i9EfG3wM+BY0ZQazW/oBgCOipd/4mBGRFxV0S8FhH9EXEjxe1X7jOSzwGrI2J12r41wDqKFwGAXwInSJoQEdsjYnONtVkLcuhbPfykZPptYOLgDhHRDXyRYg91p6ROSUdVWN5RwK6IeLOk7V8p9mgH5r9cMq90umybpEskbUjDH68DJwBHlHTZUTI98EIxuO1921VDrdV8CRDwlKTNkj5fUvOSNHy1J9V82KCaB3wYuGBg21Lf0yjeybxF8c7mD4Htkh6W9Js11mYtyKFvTSMi/joiTqMIqQC+NjBrUNdtwOGSPlDS9m+AV9L0dophnQHTy61uYELSh4FvAlcAH4yIScCzFGG7r6rVOqSI+ElE/EFEHAV8Abg1jfN/AriK4l3N5FTzngo1vwx8JyImlZwOjYhlaR2PRsSZFEM7P6K4LWw/5dC3piDpWEmnSxoP/Ixiz/ndNHsHMEPSAQAR8TLwj8B/l3SIpN8CLgPuTv3vA66WNFnSVIowH8qhFC8Cr6ZaLqXY099nNdQ6JEkXSBp4Adud6nwX+ADF5wyvAgdK+m/Ar1dYzF3A70k6W9K4VEeHpGmS2iR9StKhwF6gj1/d7rYfcuhbsxhP8QHsTymGgz5EcdQOwHfT+WuSnknTF1J82LsNeAC4Jo1VA1wH9AJbgb8D7qcItLIi4jngRuD/UrzAzAL+YTQ2qoZaq/n3wA8l9QGrgCsjYivwKPAI8C8Uw0U/o/ww1sALz3yK2/PV1O+PKZ7/B1B8CLwN2AX8DvCfh72F1jIU4T9Rsf2bpP8ELIiI32l0LWaN5j192+9ImiLpVEkHSDqWYk/2gUbXZdYMGv1tRLOxcDDwV8DRwOtAJ8Vx8mbZ8/COmVlGPLxjZpaRph7eOeKII2LGjBkV57/11lsceuih9StoDLT6Nrj+xmv1bXD9o2/9+vU/jYgjy81r6tCfMWMG69atqzi/q6uLjo6O+hU0Blp9G1x/47X6Nrj+0SfpXyvN8/COmVlGHPpmZhlx6JuZZaRq6Kff6Xgq/fHC5oE/epB0R/rDhQ3pNDu1S9LN6c8iNko6sWRZCyW9kE4V/3TCzMzGRi0f5O4FTo+IPkkHAU9IeiTN++OIuH9Q/3OAmel0CnAbcIqkw4FrgHaKH41aL2lVROwejQ0xM7Pqqu7pR6EvXTwonYb6Rtd84M50vSeBSZKmAGcDayJiVwr6NcC8fSvfzMyGo6Zv5EoaB6yn+GegWyLiKkl3AB+neCewFlgaEXslPQQsG/iHH0lrKX73u4Pir9y+mtr/BHgnIr4+aF2LgcUAbW1tJ3V2dlasq6+vj4kTy/1vReto9W1w/Y3X6tvg+kff3Llz10dEe7l5NR2nn/4ibrakScADkk4Arqb4CdyDgeUUwX4d5f/EIYZoH7yu5Wl5tLe3x1DHvzbj8bHD1erb4Pobr9W3wfXX17CO3omI14EuYF76L82IiL3At/nV/4n28t5/KppG8VvdldrNzKxOqu7pSzoS+EVEvC5pAvBJ4GuSpkTEdkkCzqf4ezko/ujhCkmdFB/k7kn9HgX+TNLk1O8sincL+50ZSx+uue+SWf0sGkb/anqWnTdqyzKz/U8twztTgJVpXP8A4L6IeEjSD9ILgoANFH+sDLAaOBfopvgT7EsBImKXpOuBp1O/6yJi1+htipmZVVM19CNiI/CxMu2nV+gfwOUV5q0AVgyzRjMzGyX+Rq6ZWUYc+mZmGXHom5llxKFvZpYRh76ZWUYc+mZmGXHom5llxKFvZpYRh76ZWUYc+mZmGXHom5llxKFvZpYRh76ZWUYc+mZmGXHom5llxKFvZpYRh76ZWUYc+mZmGXHom5llxKFvZpYRh76ZWUYOrNZB0iHA48D41P/+iLhG0tFAJ3A48AxwcUT8XNJ44E7gJOA14LMR0ZOWdTVwGfAu8EcR8ejob1LeZix9uK7rWzKrn0VLH6Zn2Xl1Xa+ZjUwte/p7gdMj4qPAbGCepDnA14CbImImsJsizEnnuyPiGOCm1A9JxwELgOOBecCtksaN5saYmdnQqoZ+FPrSxYPSKYDTgftT+0rg/DQ9P10mzT9DklJ7Z0TsjYitQDdw8qhshZmZ1UQRUb1TsUe+HjgGuAX4c+DJtDePpOnAIxFxgqRngXkR0Zvm/Rg4Bbg2Xeeu1H57us79g9a1GFgM0NbWdlJnZ2fFuvr6+pg4ceKwNrgeNr2yp+a+bRNgxztjWMwYG6h/1tTDGl3KiDTrY2g4Wn0bXP/omzt37vqIaC83r+qYPkBEvAvMljQJeAD4SLlu6VwV5lVqH7yu5cBygPb29ujo6KhYV1dXF0PNb5RFwxhXXzKrnxs31XQ3NKWB+nsu6mh0KSPSrI+h4Wj1bXD99TWso3ci4nWgC5gDTJI0kFbTgG1puheYDpDmHwbsKm0vcx0zM6uDqqEv6ci0h4+kCcAngS3AY8BnUreFwINpelW6TJr/gyjGkFYBCySNT0f+zASeGq0NMTOz6moZV5gCrEzj+gcA90XEQ5KeAzolfRX4J+D21P924DuSuin28BcARMRmSfcBzwH9wOVp2MjMzOqkauhHxEbgY2XaX6TM0TcR8TPgggrLugG4YfhlWrOr9/cDBvj7AWbD42/kmpllxKFvZpYRh76ZWUYc+mZmGXHom5llxKFvZpYRh76ZWUYc+mZmGXHom5llxKFvZpYRh76ZWUYc+mZmGXHom5llxKFvZpYRh76ZWUYc+mZmGXHom5llxKFvZpYRh76ZWUYc+mZmGXHom5llpGroS5ou6TFJWyRtlnRlar9W0iuSNqTTuSXXuVpSt6TnJZ1d0j4vtXVLWjo2m2RmZpUcWEOffmBJRDwj6QPAeklr0rybIuLrpZ0lHQcsAI4HjgL+TtK/S7NvAc4EeoGnJa2KiOdGY0PMzKy6qqEfEduB7Wn6TUlbgKlDXGU+0BkRe4GtkrqBk9O87oh4EUBSZ+rr0DczqxNFRO2dpRnA48AJwH8FFgFvAOso3g3slvQ/gScj4q50nduBR9Ii5kXE76f2i4FTIuKKQetYDCwGaGtrO6mzs7NiPX19fUycOLHm+utl0yt7au7bNgF2vDOGxYyxRtc/a+ph+3T9Zn0MDUerb4PrH31z585dHxHt5ebVMrwDgKSJwPeAL0bEG5JuA64HIp3fCHweUJmrB+U/P3jfK05ELAeWA7S3t0dHR0fFmrq6uhhqfqMsWvpwzX2XzOrnxk013w1Np9H191zUsU/Xb9bH0HC0+ja4/vqq6dkq6SCKwL87Ir4PEBE7SuZ/E3goXewFppdcfRqwLU1Xajczszqo5egdAbcDWyLiGyXtU0q6fRp4Nk2vAhZIGi/paGAm8BTwNDBT0tGSDqb4sHfV6GyGmZnVopY9/VOBi4FNkjakti8DF0qaTTFE0wN8ASAiNku6j+ID2n7g8oh4F0DSFcCjwDhgRURsHsVtMTOzKmo5eucJyo/Trx7iOjcAN5RpXz3U9czMbGz5G7lmZhlx6JuZZcShb2aWEYe+mVlGHPpmZhlx6JuZZcShb2aWEYe+mVlGHPpmZhlx6JuZZcShb2aWEYe+mVlGHPpmZhlx6JuZZcShb2aWkdb9c1YzYMYw/o+4nCWz+of1n8alepadt0/rNmsE7+mbmWXEoW9mlhGHvplZRhz6ZmYZceibmWWkauhLmi7pMUlbJG2WdGVqP1zSGkkvpPPJqV2SbpbULWmjpBNLlrUw9X9B0sKx2ywzMyunlj39fmBJRHwEmANcLuk4YCmwNiJmAmvTZYBzgJnptBi4DYoXCeAa4BTgZOCagRcKMzOrj6qhHxHbI+KZNP0msAWYCswHVqZuK4Hz0/R84M4oPAlMkjQFOBtYExG7ImI3sAaYN6pbY2ZmQ1JE1N5ZmgE8DpwAvBQRk0rm7Y6IyZIeApZFxBOpfS1wFdABHBIRX03tfwK8ExFfH7SOxRTvEGhrazups7OzYj19fX1MnDix5vrrZdMre2ru2zYBdrwzhsWMsZzrnzX1sNEtZoSa9XlQK9c/+ubOnbs+ItrLzav5G7mSJgLfA74YEW9Iqti1TFsM0f7ehojlwHKA9vb26OjoqFhTV1cXQ81vlOF8w3PJrH5u3NS6X4zOuf6eizpGt5gRatbnQa1cf33VdPSOpIMoAv/uiPh+at6Rhm1I5ztTey8wveTq04BtQ7SbmVmd1HL0joDbgS0R8Y2SWauAgSNwFgIPlrRfko7imQPsiYjtwKPAWZImpw9wz0ptZmZWJ7W8rz0VuBjYJGlDavsysAy4T9JlwEvABWneauBcoBt4G7gUICJ2SboeeDr1uy4ido3KVpiZWU2qhn76QLbSAP4ZZfoHcHmFZa0AVgynQDMzGz3+Rq6ZWUYc+mZmGXHom5llxKFvZpYRh76ZWUYc+mZmGXHom5llxKFvZpYRh76ZWUYc+mZmGXHom5llxKFvZpYRh76ZWUYc+mZmGXHom5llxKFvZpYRh76ZWUYc+mZmGXHom5llxKFvZpaRqn+MbmblzVj6cEPW27PsvIas1/YPVff0Ja2QtFPSsyVt10p6RdKGdDq3ZN7VkrolPS/p7JL2eamtW9LS0d8UMzOrppbhnTuAeWXab4qI2em0GkDSccAC4Ph0nVsljZM0DrgFOAc4Drgw9TUzszqqOrwTEY9LmlHj8uYDnRGxF9gqqRs4Oc3rjogXASR1pr7PDbtiMzMbsX0Z079C0iXAOmBJROwGpgJPlvTpTW0ALw9qP6XcQiUtBhYDtLW10dXVVbGAvr6+Iec3ypJZ/TX3bZswvP7NxvXX3+DHfLM+D2rl+utrpKF/G3A9EOn8RuDzgMr0DcoPI0W5BUfEcmA5QHt7e3R0dFQsoquri6HmN8qiYXzAt2RWPzduat3P011//fVc1PGey836PKiV66+vET3aI2LHwLSkbwIPpYu9wPSSrtOAbWm6UruZmdXJiI7TlzSl5OKngYEje1YBCySNl3Q0MBN4CngamCnpaEkHU3zYu2rkZZuZ2UhU3dOXdA/QARwhqRe4BuiQNJtiiKYH+AJARGyWdB/FB7T9wOUR8W5azhXAo8A4YEVEbB71rTEzsyHVcvTOhWWabx+i/w3ADWXaVwOrh1WdmZmNKv8Mg5lZRhz6ZmYZceibmWXEoW9mlhGHvplZRhz6ZmYZceibmWXEoW9mlhGHvplZRhz6ZmYZceibmWXEoW9mlhGHvplZRhz6ZmYZceibmWXEoW9mlhGHvplZRhz6ZmYZceibmWXEoW9mlhGHvplZRqqGvqQVknZKerak7XBJayS9kM4np3ZJullSt6SNkk4suc7C1P8FSQvHZnPMzGwotezp3wHMG9S2FFgbETOBtekywDnAzHRaDNwGxYsEcA1wCnAycM3AC4WZmdVP1dCPiMeBXYOa5wMr0/RK4PyS9juj8CQwSdIU4GxgTUTsiojdwBre/0JiZmZj7MARXq8tIrYDRMR2SR9K7VOBl0v69aa2Su3vI2kxxbsE2tra6OrqqlhEX1/fkPMbZcms/pr7tk0YXv9m4/rrb/BjvlmfB7Vy/fU10tCvRGXaYoj29zdGLAeWA7S3t0dHR0fFlXV1dTHU/EZZtPThmvsumdXPjZtG+26oH9dffz0XdbzncrM+D2rl+utrpI/2HZKmpL38KcDO1N4LTC/pNw3Ylto7BrV3jXDdNZsxjPA1M8vBSA/ZXAUMHIGzEHiwpP2SdBTPHGBPGgZ6FDhL0uT0Ae5Zqc3MzOqo6p6+pHso9tKPkNRLcRTOMuA+SZcBLwEXpO6rgXOBbuBt4FKAiNgl6Xrg6dTvuogY/OGwmZmNsaqhHxEXVph1Rpm+AVxeYTkrgBXDqs7MzEZVa32CZWbv+6xqyaz+YR08sC96lp1Xl/XY2PHPMJiZZcShb2aWEYe+mVlGHPpmZhlx6JuZZcShb2aWEYe+mVlGHPpmZhlx6JuZZcShb2aWEYe+mVlGHPpmZhlx6JuZZcShb2aWEYe+mVlGHPpmZhlx6JuZZcShb2aWEYe+mVlGHPpmZhnZp9CX1CNpk6QNktaltsMlrZH0QjqfnNol6WZJ3ZI2SjpxNDbAzMxqNxp7+nMjYnZEtKfLS4G1ETETWJsuA5wDzEynxcBto7BuMzMbhrEY3pkPrEzTK4HzS9rvjMKTwCRJU8Zg/WZmVoEiYuRXlrYCu4EA/ioilkt6PSImlfTZHRGTJT0ELIuIJ1L7WuCqiFg3aJmLKd4J0NbWdlJnZ2fF9ff19TFx4sSK8ze9smfE21YvbRNgxzuNrmLkXH/j1XMbZk09bNSXWe153Oyasf65c+euLxl9eY8D93HZp0bENkkfAtZI+tEQfVWm7X2vOBGxHFgO0N7eHh0dHRUX2NXVxVDzFy19eIhymsOSWf3cuGlf74bGcf2NV89t6LmoY9SXWe153Oxarf59Gt6JiG3pfCfwAHAysGNg2Cad70zde4HpJVefBmzbl/WbmdnwjDj0JR0q6QMD08BZwLPAKmBh6rYQeDBNrwIuSUfxzAH2RMT2EVduZmbDti/vCduAByQNLOevI+JvJD0N3CfpMuAl4ILUfzVwLtANvA1cug/rNjOzERhx6EfEi8BHy7S/BpxRpj2Ay0e6PjMz23f+Rq6ZWUYc+mZmGXHom5llpLUPUDazupoxBt99WTKrv+p3anqWnTfq682V9/TNzDLi0Dczy4hD38wsIw59M7OMOPTNzDLi0Dczy4hD38wsIw59M7OMOPTNzDLi0Dczy4hD38wsIw59M7OM+AfXzKzpjcUPvdVqf/uxN+/pm5llxKFvZpYRh76ZWUYc+mZmGal76EuaJ+l5Sd2SltZ7/WZmOavr0TuSxgG3AGcCvcDTklZFxHP1rMPMrFbVjhyq5e8eR2Ksjhqq957+yUB3RLwYET8HOoH5da7BzCxbioj6rUz6DDAvIn4/Xb4YOCUirijpsxhYnC4eCzw/xCKPAH46RuXWS6tvg+tvvFbfBtc/+j4cEUeWm1HvL2epTNt7XnUiYjmwvKaFSesion00CmuUVt8G1994rb4Nrr++6j280wtML7k8DdhW5xrMzLJV79B/Gpgp6WhJBwMLgFV1rsHMLFt1Hd6JiH5JVwCPAuOAFRGxeR8WWdMwUJNr9W1w/Y3X6tvg+uuorh/kmplZY/kbuWZmGXHom5llpGVDv9V+zkHSdEmPSdoiabOkK1P74ZLWSHohnU9udK1DkTRO0j9JeihdPlrSD1P996YP6JuWpEmS7pf0o3RffLyV7gNJ/yU9fp6VdI+kQ5r9PpC0QtJOSc+WtJW9zVW4OT2vN0o6sXGV//9ay9X/5+kxtFHSA5Imlcy7OtX/vKSzG1N1ZS0Z+iU/53AOcBxwoaTjGltVVf3Akoj4CDAHuDzVvBRYGxEzgbXpcjO7EthScvlrwE2p/t3AZQ2pqnZ/AfxNRPwm8FGKbWmJ+0DSVOCPgPaIOIHiYIgFNP99cAcwb1Bbpdv8HGBmOi0GbqtTjUO5g/fXvwY4ISJ+C/gX4GqA9JxeAByfrnNryqum0ZKhTwv+nENEbI+IZ9L0mxRhM5Wi7pWp20rg/MZUWJ2kacB5wLfSZQGnA/enLs1e/68D/wG4HSAifh4Rr9NC9wHFEXcTJB0I/BqwnSa/DyLicWDXoOZKt/l84M4oPAlMkjSlPpWWV67+iPjbiOhPF5+k+M4RFPV3RsTeiNgKdFPkVdNo1dCfCrxccrk3tbUESTOAjwE/BNoiYjsULwzAhxpXWVX/A/gS8Mt0+YPA6yUP/ma/H34DeBX4dhqi+pakQ2mR+yAiXgG+DrxEEfZ7gPW01n0woNJt3orP7c8Dj6Tppq+/VUO/6s85NCtJE4HvAV+MiDcaXU+tJP0usDMi1pc2l+nazPfDgcCJwG0R8THgLZp0KKecNO49HzgaOAo4lGI4ZLBmvg+qaanHlKSvUAzd3j3QVKZbU9XfqqHfkj/nIOkgisC/OyK+n5p3DLx9Tec7G1VfFacCn5LUQzGcdjrFnv+kNNQAzX8/9AK9EfHDdPl+iheBVrkPPglsjYhXI+IXwPeB36a17oMBlW7zlnluS1oI/C5wUfzqC09NX3+rhn7L/ZxDGv++HdgSEd8ombUKWJimFwIP1ru2WkTE1RExLSJmUNzeP4iIi4DHgM+kbk1bP0BE/AR4WdKxqekM4Dla5D6gGNaZI+nX0uNpoP6WuQ9KVLrNVwGXpKN45gB7BoaBmomkecBVwKci4u2SWauABZLGSzqa4gPppxpRY0UR0ZIn4FyKT81/DHyl0fXUUO9pFG/zNgIb0ulcinHxtcAL6fzwRtdaw7Z0AA+l6d+geFB3A98Fxje6viq1zwbWpfvhfwGTW+k+AP4U+BHwLPAdYHyz3wfAPRSfQfyCYk/4skq3OcXwyC3peb2J4kilZqy/m2LsfuC5/Jcl/b+S6n8eOKfR9Q8++WcYzMwy0qrDO2ZmNgIOfTOzjDj0zcwy4tA3M8uIQ9/MLCMOfTOzjDj0zcwy8v8A8tqndJny5P8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots()\n",
    "train.sales.hist(ax=ax);\n",
    "ax.set_title('histogram of sales');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train a simple model\n",
    "As you determined, you are dealing with a regression problem. So, now you're ready to build a model for a subsequent submission. But now, instead of building the simplest Linear Regression model as in the slides, let's build an out-of-box Random Forest model.\n",
    "\n",
    "You will use the `RandomForestRegressor` class from the scikit-learn library.\n",
    "\n",
    "Your objective is to train a Random Forest model with default parameters on the \"store\" and \"item\" features.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RandomForestRegressor()"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.ensemble import RandomForestRegressor\n",
    "\n",
    "# Create a Random Forest object\n",
    "rf = RandomForestRegressor()\n",
    "\n",
    "# Train a model\n",
    "rf.fit(X=train[['store', 'item']], y=train['sales'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Prepare a submission\n",
    "You've already built a model on the training data from the Kaggle Store Item Demand Forecasting Challenge. Now, it's time to make predictions on the test data and create a submission file in the specified format.\n",
    "\n",
    "Your goal is to read the test data, make predictions, and save these in the format specified in the \"sample_submission.csv\" file. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>sales</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>52</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>52</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>52</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>52</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>52</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id  sales\n",
       "0   0     52\n",
       "1   1     52\n",
       "2   2     52\n",
       "3   3     52\n",
       "4   4     52"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Show the head() of the sample_submission\n",
    "sample_submission.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get predictions for the test set\n",
    "test['sales'] = rf.predict(test[['store', 'item']])\n",
    "\n",
    "# Write test predictions using the sample_submission format\n",
    "test[['id', 'sales']].to_csv('kaggle_submission.csv', index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "id,sales\r\n",
      "0,17.143270686120914\r\n",
      "1,17.143270686120914\r\n",
      "2,17.143270686120914\r\n",
      "3,17.143270686120914\r\n",
      "4,17.143270686120914\r\n",
      "5,17.143270686120914\r\n",
      "6,17.143270686120914\r\n",
      "7,17.143270686120914\r\n",
      "8,17.143270686120914\r\n"
     ]
    }
   ],
   "source": [
    "!head kaggle_submission.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Public vs Private leaderboard\n",
    "- Competition metric\n",
    "\n",
    "| Evaluation metric | Type of problem |\n",
    "| ----------------- | --------------- | \n",
    "| Area Under the ROC (AUC) | Classification |\n",
    "| F1 score (F1) | Classification |\n",
    "| Mean Log Loss (LogLoss) | Classification |\n",
    "| Mean Absolute Error (MAE) | Regression |\n",
    "| Mean Squared Erro (MSE) | Regression |\n",
    "| Mean Average Precision at K (MAPK, MAP@K) | Ranking |\n",
    "\n",
    "- Overfitting in kaggle\n",
    "![of](image/overfitting_kaggle.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train XGBoost models\n",
    "Every Machine Learning method could potentially overfit. You will see it on this example with XGBoost. Again, you are working with the Store Item Demand Forecasting Challenge. \n",
    "\n",
    "Firstly, let's train multiple XGBoost models with different sets of hyperparameters using XGBoost's learning API. The single hyperparameter you will change is:\n",
    "\n",
    "- `max_depth` - maximum depth of a tree. Increasing this value will make the model more complex and more likely to overfit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "import xgboost as xgb\n",
    "\n",
    "# Create DMatrix on train data\n",
    "dtrain = xgb.DMatrix(data=train[['store', 'item']],\n",
    "                     label=train['sales'])\n",
    "\n",
    "# Define xgboost parameters\n",
    "params = {'objective': 'reg:squarederror',\n",
    "          'max_depth': 2,\n",
    "          'verbosity': 1}\n",
    "\n",
    "# Train xgboost model\n",
    "xg_depth_2 = xgb.train(params=params, dtrain=dtrain)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define xgboost parameters\n",
    "params = {'objective': 'reg:squarederror',\n",
    "          'max_depth': 8,\n",
    "          'verbosity': 1}\n",
    "\n",
    "# Train xgboost model\n",
    "xg_depth_8 = xgb.train(params=params, dtrain=dtrain)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define xgboost parameters\n",
    "params = {'objective': 'reg:squarederror',\n",
    "          'max_depth': 15,\n",
    "          'verbosity': 1}\n",
    "\n",
    "# Train xgboost model\n",
    "xg_depth_15 = xgb.train(params=params, dtrain=dtrain)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Explore overfitting XGBoost\n",
    "Having trained 3 XGBoost models with different maximum depths, you will now evaluate their quality. For this purpose, you will measure the quality of each model on both the train data and the test data. As you know by now, the train data is the data models have been trained on. The test data is the next month sales data that models have never seen before.\n",
    "\n",
    "The goal of this exercise is to determine whether any of the models trained is overfitting. To measure the quality of the models you will use Mean Squared Error (MSE). It's available in `sklearn.metrics` as `mean_squared_error()` function that takes two arguments: true values and predicted values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MSE Train: 331.064. MSE Test: 249.849\n",
      "MSE Train: 112.057. MSE Test: 30.751\n",
      "MSE Train: 84.952. MSE Test: 3.536\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import mean_squared_error\n",
    "\n",
    "dtrain = xgb.DMatrix(data=train[['store', 'item']])\n",
    "dtest = xgb.DMatrix(data=test[['store', 'item']])\n",
    "\n",
    "# For each of 3 trained models\n",
    "for model in [xg_depth_2, xg_depth_8, xg_depth_15]:\n",
    "    # Make predictions\n",
    "    train_pred = model.predict(dtrain)\n",
    "    test_pred = model.predict(dtest)\n",
    "    \n",
    "    # Calculate metrics\n",
    "    mse_train = mean_squared_error(train['sales'], train_pred)\n",
    "    mse_test = mean_squared_error(test['sales'], test_pred)\n",
    "    print('MSE Train: {:.3f}. MSE Test: {:.3f}'.format(mse_train, mse_test))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}