walk_through/Automated_Feature_Engineering.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "MockJ-NBFvgv"
   },
   "source": [
    "# Introduction: Automated Feature Engineering\n",
    "\n",
    "In this notebook, we will look at an exciting development in data science: automated feature engineering. A machine learning model can only learn from the data we give it, and making sure that data is relevant to the task is one of the most crucial steps in the machine learning pipeline (this is made clear in the excellent paper [\"A Few Useful Things to Know about Machine Learning\"](https://homes.cs.washington.edu/~pedrod/papers/cacm12.pdf)). \n",
    "\n",
    "However, manual feature engineering is a tedious task and is limited by both human imagination - there are only so many features we can think to create - and by time - creating new features is time-intensive. Ideally, there would be an objective method to create an array of diverse new candidate features that we can then use for a machine learning task. This process is meant to not replace the data scientist, but to make her job easier and allowing her to supplement domain knowledge with an automated workflow.\n",
    "\n",
    "In this notebook, we will walk through an implementation of using [Featuretools](https://www.featuretools.com/), an open-source Python library for automatically creating features with relational data (where the data is in structured tables). Although there are now many efforts working to enable automated model selection and hyperparameter tuning, there has been a lack of automating work on the feature engineering aspect of the pipeline. This library seeks to close that gap and the general methodology has been proven effective in both [machine learning competitions with the data science machine](https://github.com/HDI-Project/Data-Science-Machine) and [business use cases](https://www.featurelabs.com/blog/predicting-credit-card-fraud/). \n",
    "\n",
    "\n",
    "## Dataset\n",
    "\n",
    "To show the basic idea of featuretools we will use an example dataset consisting of three tables:\n",
    "\n",
    "* `clients`: information about clients at a credit union\n",
    "* `loans`: previous loans taken out by the clients\n",
    "* `payments`: payments made/missed on the previous loans\n",
    "\n",
    "The general problem of feature engineering is taking disparate data, often distributed across multiple tables, and combining it into a single table that can be used for training a machine learning model. Featuretools has the ability to do this for us, creating many new candidate features with minimal effort. These features are combined into a single table that can then be passed on to our model. \n",
    "\n",
    "First, let's load in the data and look at the problem we are working with."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "4zDxrOS8F_t0"
   },
   "outputs": [],
   "source": [
    "# Run this if featuretools is not already installed\n",
    "# !pip install -U featuretools"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "Og7QpzFkFrf9"
   },
   "outputs": [],
   "source": [
    "# pandas and numpy for data manipulation\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "# featuretools for automated feature engineering\n",
    "import featuretools as ft\n",
    "\n",
    "# ignore warnings from pandas\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "BGkrloKKGbD2"
   },
   "outputs": [],
   "source": [
    "# Read in the data\n",
    "clients = pd.read_csv('data/clients.csv', parse_dates = ['joined'])\n",
    "loans = pd.read_csv('data/loans.csv', parse_dates = ['loan_start', 'loan_end'])\n",
    "payments = pd.read_csv('data/payments.csv', parse_dates = ['payment_date'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "08ST5lOrrTxg"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>client_id</th>\n",
       "      <th>joined</th>\n",
       "      <th>income</th>\n",
       "      <th>credit_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>46109</td>\n",
       "      <td>2002-04-16</td>\n",
       "      <td>172677</td>\n",
       "      <td>527</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>49545</td>\n",
       "      <td>2007-11-14</td>\n",
       "      <td>104564</td>\n",
       "      <td>770</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>41480</td>\n",
       "      <td>2013-03-11</td>\n",
       "      <td>122607</td>\n",
       "      <td>585</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>46180</td>\n",
       "      <td>2001-11-06</td>\n",
       "      <td>43851</td>\n",
       "      <td>562</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>25707</td>\n",
       "      <td>2006-10-06</td>\n",
       "      <td>211422</td>\n",
       "      <td>621</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   client_id     joined  income  credit_score\n",
       "0      46109 2002-04-16  172677           527\n",
       "1      49545 2007-11-14  104564           770\n",
       "2      41480 2013-03-11  122607           585\n",
       "3      46180 2001-11-06   43851           562\n",
       "4      25707 2006-10-06  211422           621"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clients.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "S6TXsCZW3Rau"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>client_id</th>\n",
       "      <th>loan_type</th>\n",
       "      <th>loan_amount</th>\n",
       "      <th>repaid</th>\n",
       "      <th>loan_id</th>\n",
       "      <th>loan_start</th>\n",
       "      <th>loan_end</th>\n",
       "      <th>rate</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>254</th>\n",
       "      <td>29841</td>\n",
       "      <td>cash</td>\n",
       "      <td>4718</td>\n",
       "      <td>0</td>\n",
       "      <td>11175</td>\n",
       "      <td>2014-03-08</td>\n",
       "      <td>2015-09-13</td>\n",
       "      <td>2.74</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>196</th>\n",
       "      <td>26326</td>\n",
       "      <td>other</td>\n",
       "      <td>10902</td>\n",
       "      <td>1</td>\n",
       "      <td>11559</td>\n",
       "      <td>2014-09-06</td>\n",
       "      <td>2016-11-14</td>\n",
       "      <td>6.73</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>45</th>\n",
       "      <td>41480</td>\n",
       "      <td>cash</td>\n",
       "      <td>12581</td>\n",
       "      <td>1</td>\n",
       "      <td>11461</td>\n",
       "      <td>2009-10-29</td>\n",
       "      <td>2011-07-08</td>\n",
       "      <td>1.94</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>415</th>\n",
       "      <td>49624</td>\n",
       "      <td>cash</td>\n",
       "      <td>8621</td>\n",
       "      <td>0</td>\n",
       "      <td>10454</td>\n",
       "      <td>2014-07-26</td>\n",
       "      <td>2015-12-29</td>\n",
       "      <td>3.18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>49545</td>\n",
       "      <td>other</td>\n",
       "      <td>4131</td>\n",
       "      <td>0</td>\n",
       "      <td>10939</td>\n",
       "      <td>2008-10-13</td>\n",
       "      <td>2010-02-26</td>\n",
       "      <td>4.07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>417</th>\n",
       "      <td>49624</td>\n",
       "      <td>credit</td>\n",
       "      <td>11219</td>\n",
       "      <td>1</td>\n",
       "      <td>11132</td>\n",
       "      <td>2013-08-12</td>\n",
       "      <td>2016-03-15</td>\n",
       "      <td>3.77</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>137</th>\n",
       "      <td>32726</td>\n",
       "      <td>credit</td>\n",
       "      <td>7499</td>\n",
       "      <td>1</td>\n",
       "      <td>10285</td>\n",
       "      <td>2012-10-07</td>\n",
       "      <td>2015-03-12</td>\n",
       "      <td>2.72</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>200</th>\n",
       "      <td>26326</td>\n",
       "      <td>credit</td>\n",
       "      <td>5275</td>\n",
       "      <td>0</td>\n",
       "      <td>11072</td>\n",
       "      <td>2014-11-11</td>\n",
       "      <td>2016-07-17</td>\n",
       "      <td>1.45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>180</th>\n",
       "      <td>48177</td>\n",
       "      <td>cash</td>\n",
       "      <td>6521</td>\n",
       "      <td>0</td>\n",
       "      <td>11235</td>\n",
       "      <td>2014-07-12</td>\n",
       "      <td>2016-02-22</td>\n",
       "      <td>6.17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>46109</td>\n",
       "      <td>home</td>\n",
       "      <td>11062</td>\n",
       "      <td>1</td>\n",
       "      <td>11611</td>\n",
       "      <td>2012-09-12</td>\n",
       "      <td>2014-03-14</td>\n",
       "      <td>5.48</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     client_id loan_type  loan_amount  repaid  loan_id loan_start   loan_end  \\\n",
       "254      29841      cash         4718       0    11175 2014-03-08 2015-09-13   \n",
       "196      26326     other        10902       1    11559 2014-09-06 2016-11-14   \n",
       "45       41480      cash        12581       1    11461 2009-10-29 2011-07-08   \n",
       "415      49624      cash         8621       0    10454 2014-07-26 2015-12-29   \n",
       "26       49545     other         4131       0    10939 2008-10-13 2010-02-26   \n",
       "417      49624    credit        11219       1    11132 2013-08-12 2016-03-15   \n",
       "137      32726    credit         7499       1    10285 2012-10-07 2015-03-12   \n",
       "200      26326    credit         5275       0    11072 2014-11-11 2016-07-17   \n",
       "180      48177      cash         6521       0    11235 2014-07-12 2016-02-22   \n",
       "8        46109      home        11062       1    11611 2012-09-12 2014-03-14   \n",
       "\n",
       "     rate  \n",
       "254  2.74  \n",
       "196  6.73  \n",
       "45   1.94  \n",
       "415  3.18  \n",
       "26   4.07  \n",
       "417  3.77  \n",
       "137  2.72  \n",
       "200  1.45  \n",
       "180  6.17  \n",
       "8    5.48  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loans.sample(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "uIOMlPAD3TfZ"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>loan_id</th>\n",
       "      <th>payment_amount</th>\n",
       "      <th>payment_date</th>\n",
       "      <th>missed</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>965</th>\n",
       "      <td>11221</td>\n",
       "      <td>1511</td>\n",
       "      <td>2009-07-14</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1285</th>\n",
       "      <td>11264</td>\n",
       "      <td>218</td>\n",
       "      <td>2009-11-06</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1664</th>\n",
       "      <td>10769</td>\n",
       "      <td>210</td>\n",
       "      <td>2012-03-18</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1074</th>\n",
       "      <td>10106</td>\n",
       "      <td>1003</td>\n",
       "      <td>2003-09-09</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2521</th>\n",
       "      <td>10868</td>\n",
       "      <td>349</td>\n",
       "      <td>2013-10-04</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2909</th>\n",
       "      <td>10537</td>\n",
       "      <td>1292</td>\n",
       "      <td>2005-11-23</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>11141</td>\n",
       "      <td>1220</td>\n",
       "      <td>2007-09-20</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2445</th>\n",
       "      <td>10109</td>\n",
       "      <td>806</td>\n",
       "      <td>2014-07-23</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1581</th>\n",
       "      <td>10624</td>\n",
       "      <td>1176</td>\n",
       "      <td>2001-03-20</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1344</th>\n",
       "      <td>10262</td>\n",
       "      <td>770</td>\n",
       "      <td>2007-03-02</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      loan_id  payment_amount payment_date  missed\n",
       "965     11221            1511   2009-07-14       1\n",
       "1285    11264             218   2009-11-06       1\n",
       "1664    10769             210   2012-03-18       1\n",
       "1074    10106            1003   2003-09-09       1\n",
       "2521    10868             349   2013-10-04       1\n",
       "2909    10537            1292   2005-11-23       1\n",
       "44      11141            1220   2007-09-20       0\n",
       "2445    10109             806   2014-07-23       0\n",
       "1581    10624            1176   2001-03-20       1\n",
       "1344    10262             770   2007-03-02       1"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "payments.sample(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "AvmwEpIbrQIW"
   },
   "source": [
    "### Manual Feature Engineering Examples\n",
    "\n",
    "Let's show a few examples of features we might make by hand. We will keep this relatively simple to avoid doing too much work! First we will focus on a single dataframe before combining them together. In the `clients` dataframe, we can take the month of the `joined` column and the natural log of the `income` column. Later, we see these are known in featuretools as transformation feature primitives because they act on column in a single table. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "PRTEs9XSUz94"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>client_id</th>\n",
       "      <th>joined</th>\n",
       "      <th>income</th>\n",
       "      <th>credit_score</th>\n",
       "      <th>join_month</th>\n",
       "      <th>log_income</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>46109</td>\n",
       "      <td>2002-04-16</td>\n",
       "      <td>172677</td>\n",
       "      <td>527</td>\n",
       "      <td>4</td>\n",
       "      <td>12.059178</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>49545</td>\n",
       "      <td>2007-11-14</td>\n",
       "      <td>104564</td>\n",
       "      <td>770</td>\n",
       "      <td>11</td>\n",
       "      <td>11.557555</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>41480</td>\n",
       "      <td>2013-03-11</td>\n",
       "      <td>122607</td>\n",
       "      <td>585</td>\n",
       "      <td>3</td>\n",
       "      <td>11.716739</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>46180</td>\n",
       "      <td>2001-11-06</td>\n",
       "      <td>43851</td>\n",
       "      <td>562</td>\n",
       "      <td>11</td>\n",
       "      <td>10.688553</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>25707</td>\n",
       "      <td>2006-10-06</td>\n",
       "      <td>211422</td>\n",
       "      <td>621</td>\n",
       "      <td>10</td>\n",
       "      <td>12.261611</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   client_id     joined  income  credit_score  join_month  log_income\n",
       "0      46109 2002-04-16  172677           527           4   12.059178\n",
       "1      49545 2007-11-14  104564           770          11   11.557555\n",
       "2      41480 2013-03-11  122607           585           3   11.716739\n",
       "3      46180 2001-11-06   43851           562          11   10.688553\n",
       "4      25707 2006-10-06  211422           621          10   12.261611"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a month column\n",
    "clients['join_month'] = clients['joined'].dt.month\n",
    "\n",
    "# Create a log of income column\n",
    "clients['log_income'] = np.log(clients['income'])\n",
    "\n",
    "clients.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "reiPaTnVs2Yk"
   },
   "source": [
    "To incorporate information about the other tables, we use the `df.groupby` method, followed by a suitable aggregation function, followed by `df.merge`.  For example, let's calculate the average, minimum, and maximum amount of previous loans for each client. In the terms of featuretools, this would be considered an aggregation feature primitive because we using multiple tables in a one-to-many relationship to calculate aggregation figures (don't worry, this will be explained shortly!)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "rLOJHXqOs2GD"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>mean_loan_amount</th>\n",
       "      <th>max_loan_amount</th>\n",
       "      <th>min_loan_amount</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>client_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>25707</th>\n",
       "      <td>7963.950000</td>\n",
       "      <td>13913</td>\n",
       "      <td>1212</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26326</th>\n",
       "      <td>7270.062500</td>\n",
       "      <td>13464</td>\n",
       "      <td>1164</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26695</th>\n",
       "      <td>7824.722222</td>\n",
       "      <td>14865</td>\n",
       "      <td>2389</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26945</th>\n",
       "      <td>7125.933333</td>\n",
       "      <td>14593</td>\n",
       "      <td>653</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29841</th>\n",
       "      <td>9813.000000</td>\n",
       "      <td>14837</td>\n",
       "      <td>2778</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           mean_loan_amount  max_loan_amount  min_loan_amount\n",
       "client_id                                                    \n",
       "25707           7963.950000            13913             1212\n",
       "26326           7270.062500            13464             1164\n",
       "26695           7824.722222            14865             2389\n",
       "26945           7125.933333            14593              653\n",
       "29841           9813.000000            14837             2778"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Groupby client id and calculate mean, max, min previous loan size\n",
    "stats = loans.groupby('client_id')['loan_amount'].agg(['mean', 'max', 'min'])\n",
    "stats.columns = ['mean_loan_amount', 'max_loan_amount', 'min_loan_amount']\n",
    "stats.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "Kf63VwWytmm8"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>client_id</th>\n",
       "      <th>joined</th>\n",
       "      <th>income</th>\n",
       "      <th>credit_score</th>\n",
       "      <th>join_month</th>\n",
       "      <th>log_income</th>\n",
       "      <th>mean_loan_amount</th>\n",
       "      <th>max_loan_amount</th>\n",
       "      <th>min_loan_amount</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>46109</td>\n",
       "      <td>2002-04-16</td>\n",
       "      <td>172677</td>\n",
       "      <td>527</td>\n",
       "      <td>4</td>\n",
       "      <td>12.059178</td>\n",
       "      <td>8951.600000</td>\n",
       "      <td>14049</td>\n",
       "      <td>559</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>49545</td>\n",
       "      <td>2007-11-14</td>\n",
       "      <td>104564</td>\n",
       "      <td>770</td>\n",
       "      <td>11</td>\n",
       "      <td>11.557555</td>\n",
       "      <td>10289.300000</td>\n",
       "      <td>14971</td>\n",
       "      <td>3851</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>41480</td>\n",
       "      <td>2013-03-11</td>\n",
       "      <td>122607</td>\n",
       "      <td>585</td>\n",
       "      <td>3</td>\n",
       "      <td>11.716739</td>\n",
       "      <td>7894.850000</td>\n",
       "      <td>14399</td>\n",
       "      <td>811</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>46180</td>\n",
       "      <td>2001-11-06</td>\n",
       "      <td>43851</td>\n",
       "      <td>562</td>\n",
       "      <td>11</td>\n",
       "      <td>10.688553</td>\n",
       "      <td>7700.850000</td>\n",
       "      <td>14081</td>\n",
       "      <td>1607</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>25707</td>\n",
       "      <td>2006-10-06</td>\n",
       "      <td>211422</td>\n",
       "      <td>621</td>\n",
       "      <td>10</td>\n",
       "      <td>12.261611</td>\n",
       "      <td>7963.950000</td>\n",
       "      <td>13913</td>\n",
       "      <td>1212</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>39505</td>\n",
       "      <td>2011-10-14</td>\n",
       "      <td>153873</td>\n",
       "      <td>610</td>\n",
       "      <td>10</td>\n",
       "      <td>11.943883</td>\n",
       "      <td>7424.050000</td>\n",
       "      <td>14575</td>\n",
       "      <td>904</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>32726</td>\n",
       "      <td>2006-05-01</td>\n",
       "      <td>235705</td>\n",
       "      <td>730</td>\n",
       "      <td>5</td>\n",
       "      <td>12.370336</td>\n",
       "      <td>6633.263158</td>\n",
       "      <td>14802</td>\n",
       "      <td>851</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>35089</td>\n",
       "      <td>2010-03-01</td>\n",
       "      <td>131176</td>\n",
       "      <td>771</td>\n",
       "      <td>3</td>\n",
       "      <td>11.784295</td>\n",
       "      <td>6939.200000</td>\n",
       "      <td>13194</td>\n",
       "      <td>773</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>35214</td>\n",
       "      <td>2003-08-08</td>\n",
       "      <td>95849</td>\n",
       "      <td>696</td>\n",
       "      <td>8</td>\n",
       "      <td>11.470529</td>\n",
       "      <td>7173.555556</td>\n",
       "      <td>14767</td>\n",
       "      <td>667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>48177</td>\n",
       "      <td>2008-06-09</td>\n",
       "      <td>190632</td>\n",
       "      <td>769</td>\n",
       "      <td>6</td>\n",
       "      <td>12.158100</td>\n",
       "      <td>7424.368421</td>\n",
       "      <td>14740</td>\n",
       "      <td>659</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   client_id     joined  income  credit_score  join_month  log_income  \\\n",
       "0      46109 2002-04-16  172677           527           4   12.059178   \n",
       "1      49545 2007-11-14  104564           770          11   11.557555   \n",
       "2      41480 2013-03-11  122607           585           3   11.716739   \n",
       "3      46180 2001-11-06   43851           562          11   10.688553   \n",
       "4      25707 2006-10-06  211422           621          10   12.261611   \n",
       "5      39505 2011-10-14  153873           610          10   11.943883   \n",
       "6      32726 2006-05-01  235705           730           5   12.370336   \n",
       "7      35089 2010-03-01  131176           771           3   11.784295   \n",
       "8      35214 2003-08-08   95849           696           8   11.470529   \n",
       "9      48177 2008-06-09  190632           769           6   12.158100   \n",
       "\n",
       "   mean_loan_amount  max_loan_amount  min_loan_amount  \n",
       "0       8951.600000            14049              559  \n",
       "1      10289.300000            14971             3851  \n",
       "2       7894.850000            14399              811  \n",
       "3       7700.850000            14081             1607  \n",
       "4       7963.950000            13913             1212  \n",
       "5       7424.050000            14575              904  \n",
       "6       6633.263158            14802              851  \n",
       "7       6939.200000            13194              773  \n",
       "8       7173.555556            14767              667  \n",
       "9       7424.368421            14740              659  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Merge with the clients dataframe\n",
    "clients.merge(stats, left_on = 'client_id', right_index=True, how = 'left').head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "0aVnENNBt6pN"
   },
   "source": [
    "We could go further and include information about `payments` in the `clients` dataframe. To do so, we would have to group `payments` by the `loan_id`, merge it with the `loans`, group the resulting dataframe by the `client_id`, and then merge it into the `clients` dataframe. This would allow us to include information about previous payments for each client. \n",
    "\n",
    "Clearly, this process of manual feature engineering can grow quite tedious with many columns and multiple tables and I certainly don't want to have to do this process by hand! Luckily, featuretools can automatically perform this entire process and will create more features than we would have ever thought of. Although I love `pandas`, there is only so much manual data manipulation I'm willing to stand! "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "UWAPnDJruqNs"
   },
   "source": [
    "# Featuretools\n",
    "\n",
    "Now that we know what we are trying to avoid (tedious manual feature engineering), let's figure out how to automate this process. Featuretools operates on an idea known as [Deep Feature Synthesis](https://docs.featuretools.com/api_reference.html#deep-feature-synthesis). You can read the [original paper here](http://www.jmaxkanter.com/static/papers/DSAA_DSM_2015.pdf), and although it's quite readable, it's not necessary to understand the details to do automated feature engineering. The concept of Deep Feature Synthesis is to use basic building blocks known as feature primitives (like the transformations and aggregations done above) that can be stacked on top of each other to form new features. The depth of a \"deep feature\" is equal to the number of stacked primitives. \n",
    "\n",
    "I threw out some terms there, but don't worry because we'll cover them as we go. Featuretools builds on simple ideas to create a powerful method, and we will build up our understanding in much the same way. \n",
    "\n",
    "The first part of Featuretools to understand [is an `entity`](https://docs.featuretools.com/loading_data/using_entitysets.html#adding-entities). This is simply a table, or in `pandas`, a `DataFrame`. We corral multiple entities into a [single object called an `EntitySet`](https://docs.featuretools.com/loading_data/using_entitysets.html). This is just a large data structure composed of many individual entities and the relationships between them.  \n",
    "\n",
    "## EntitySet\n",
    "\n",
    "Creating a new `EntitySet` is pretty simple: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "XTwph48RGr9Q"
   },
   "outputs": [],
   "source": [
    "es = ft.EntitySet(id = 'clients')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "pHJSYZU1Vro7"
   },
   "source": [
    "## Entities \n",
    "\n",
    "An entity is simply a table, which is represented in Pandas as a `dataframe`. Each entity must have a uniquely identifying column, known as an index. For the clients dataframe, this is the `client_id` because each id only appears once in the `clients` data. In the `loans` dataframe, `client_id` is not an index because each id might appear more than once. The index for this dataframe is instead `loan_id`. \n",
    "\n",
    "When we create an `entity` in featuretools, we have to identify which column of the dataframe is the index. If the data does not have a unique index we can tell featuretools to make an index for the entity by passing in `make_index = True` and specifying a name for the index. If the data also has a uniquely identifying time index, we can pass that in as the `time_index` parameter. \n",
    "\n",
    "Featuretools will automatically infer the variable types (numeric, categorical, datetime) of the columns in our data, but we can also pass in specific datatypes to override this behavior. As an example, even though the `repaid` column in the `loans` dataframe is represented as an integer, we can tell featuretools that this is a categorical feature since it can only take on two discrete values. This is done using an integer with the variables as keys and the feature types as values.\n",
    "\n",
    "In the code below we create the three entities and add them to the `EntitySet`.  The syntax is relatively straightforward with a few notes: for the `payments` dataframe we need to make an index, for the `loans` dataframe, we specify that `repaid` is a categorical variable, and for the `payments` dataframe, we specify that `missed` is a categorical feature. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "iWORNI1MGwYQ"
   },
   "outputs": [],
   "source": [
    "# Create an entity from the client dataframe\n",
    "# This dataframe already has an index and a time index\n",
    "es = es.entity_from_dataframe(entity_id = 'clients', dataframe = clients, \n",
    "                              index = 'client_id', time_index = 'joined')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "7_aDZl4IG98s"
   },
   "outputs": [],
   "source": [
    "# Create an entity from the loans dataframe\n",
    "# This dataframe already has an index and a time index\n",
    "es = es.entity_from_dataframe(entity_id = 'loans', dataframe = loans, \n",
    "                              variable_types = {'repaid': ft.variable_types.Categorical},\n",
    "                              index = 'loan_id', \n",
    "                              time_index = 'loan_start')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "8VAjTWWbSXmi"
   },
   "outputs": [],
   "source": [
    "# Create an entity from the payments dataframe\n",
    "# This does not yet have a unique index\n",
    "es = es.entity_from_dataframe(entity_id = 'payments', \n",
    "                              dataframe = payments,\n",
    "                              variable_types = {'missed': ft.variable_types.Categorical},\n",
    "                              make_index = True,\n",
    "                              index = 'payment_id',\n",
    "                              time_index = 'payment_date')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "1EV_Zg7QSxTo"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Entityset: clients\n",
       "  Entities:\n",
       "    clients [Rows: 25, Columns: 6]\n",
       "    loans [Rows: 443, Columns: 8]\n",
       "    payments [Rows: 3456, Columns: 5]\n",
       "  Relationships:\n",
       "    No relationships"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "es"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "xmQwcxRQXQAf"
   },
   "source": [
    "All three entities have been successfully added to the `EntitySet`. We can access any of the entities using Python dictionary syntax."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "7rFib_OrXZiN"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Entity: loans\n",
       "  Variables:\n",
       "    client_id (dtype: numeric)\n",
       "    loan_type (dtype: categorical)\n",
       "    loan_amount (dtype: numeric)\n",
       "    loan_start (dtype: datetime_time_index)\n",
       "    loan_end (dtype: datetime)\n",
       "    rate (dtype: numeric)\n",
       "    repaid (dtype: categorical)\n",
       "    loan_id (dtype: index)\n",
       "  Shape:\n",
       "    (Rows: 443, Columns: 8)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "es['loans']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "sZJ-HlanXduM"
   },
   "source": [
    "Featuretools correctly inferred each of the datatypes when we made this entity. We can also see that we overrode the type for the `repaid` feature, changing if from numeric to categorical. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Entity: payments\n",
       "  Variables:\n",
       "    loan_id (dtype: numeric)\n",
       "    payment_amount (dtype: numeric)\n",
       "    payment_date (dtype: datetime_time_index)\n",
       "    missed (dtype: categorical)\n",
       "    payment_id (dtype: index)\n",
       "  Shape:\n",
       "    (Rows: 3456, Columns: 5)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "es['payments']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "fhs4qoakVNyk"
   },
   "source": [
    "## Relationships\n",
    "\n",
    "After defining the entities (tables) in an `EntitySet`, we now need to tell featuretools [how they are related with a relationship](https://docs.featuretools.com/loading_data/using_entitysets.html#adding-a-relationship). The most intuitive way to think of relationships is with the parent to child analogy: a parent-to-child relationship is one-to-many because for each parent, there can be multiple children. The `client` dataframe is therefore the parent of the `loans` dataframe because while there is only one row for each client in the `client` dataframe, each client may have several previous loans covering multiple rows in the `loans` dataframe. Likewise, the `loans` dataframe is the parent of the `payments` dataframe because each loan will have multiple payments. \n",
    "\n",
    "These relationships are what allow us to group together datapoints using aggregation primitives and then create new features. As an example, we can group all of the previous loans associated with one client and find the average loan amount. We will discuss the features themselves more in a little bit, but for now let's define the relationships. \n",
    "\n",
    "To define relationships, we need to specify the parent variable and the child variable. This is the variable that links two entities together. In our example, the `client` and `loans` dataframes are linked together by the `client_id` column. Again, this is a parent to child relationship because for each `client_id` in the parent `client` dataframe, there may be multiple entries of the same `client_id` in the child `loans` dataframe. \n",
    "\n",
    "We codify relationships in the language of featuretools by specifying the parent variable and then the child variable. After creating a relationship, we add it to the `EntitySet`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "ChKZnFWTVJrh"
   },
   "outputs": [],
   "source": [
    "# Relationship between clients and previous loans\n",
    "r_client_previous = ft.Relationship(es['clients']['client_id'],\n",
    "                                    es['loans']['client_id'])\n",
    "\n",
    "# Add the relationship to the entity set\n",
    "es = es.add_relationship(r_client_previous)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "DAPdMnwnZkHM"
   },
   "source": [
    "The relationship has now been stored in the entity set. The second relationship is between the `loans` and `payments`. These two entities are related by the `loan_id` variable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "DrgSGOEDZe6N"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Entityset: clients\n",
       "  Entities:\n",
       "    clients [Rows: 25, Columns: 6]\n",
       "    loans [Rows: 443, Columns: 8]\n",
       "    payments [Rows: 3456, Columns: 5]\n",
       "  Relationships:\n",
       "    loans.client_id -> clients.client_id\n",
       "    payments.loan_id -> loans.loan_id"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Relationship between previous loans and previous payments\n",
    "r_payments = ft.Relationship(es['loans']['loan_id'],\n",
    "                                      es['payments']['loan_id'])\n",
    "\n",
    "# Add the relationship to the entity set\n",
    "es = es.add_relationship(r_payments)\n",
    "\n",
    "es"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "oeFZMQVIaBV4"
   },
   "source": [
    "We now have our entities in an entityset along with the relationships between them. We can now start to making new features from all of the tables using stacks of feature primitives to form deep features. First, let's cover feature primitives.\n",
    "\n",
    "\n",
    "## Feature Primitives\n",
    "\n",
    "A [feature primitive](https://docs.featuretools.com/automated_feature_engineering/primitives.html) a at a very high-level is an operation applied to data to create a feature. These represent very simple calculations that can be stacked on top of each other to create complex features. Feature primitives fall into two categories:\n",
    "\n",
    "* __Aggregation__: function that groups together child datapoints for each parent and then calculates a statistic such as mean, min, max, or standard deviation. An example is calculating the maximum loan amount for each client. An aggregation works across multiple tables using relationships between tables.\n",
    "* __Transformation__: an operation applied to one or more columns in a single table. An example would be extracting the day from dates, or finding the difference between two columns in one table.\n",
    "\n",
    "Let's take a look at feature primitives in featuretools. We can view the list of primitives:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "KNHzhu7WaALK"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>type</th>\n",
       "      <th>description</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>mode</td>\n",
       "      <td>aggregation</td>\n",
       "      <td>Finds the most common element in a categorical feature.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>any</td>\n",
       "      <td>aggregation</td>\n",
       "      <td>Test if any value is 'True'.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>last</td>\n",
       "      <td>aggregation</td>\n",
       "      <td>Returns the last value.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>all</td>\n",
       "      <td>aggregation</td>\n",
       "      <td>Test if all values are 'True'.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>median</td>\n",
       "      <td>aggregation</td>\n",
       "      <td>Finds the median value of any feature with well-ordered values.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>min</td>\n",
       "      <td>aggregation</td>\n",
       "      <td>Finds the minimum non-null value of a numeric feature.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>avg_time_between</td>\n",
       "      <td>aggregation</td>\n",
       "      <td>Computes the average time between consecutive events.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>max</td>\n",
       "      <td>aggregation</td>\n",
       "      <td>Finds the maximum non-null value of a numeric feature.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>n_most_common</td>\n",
       "      <td>aggregation</td>\n",
       "      <td>Finds the N most common elements in a categorical feature.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>percent_true</td>\n",
       "      <td>aggregation</td>\n",
       "      <td>Finds the percent of 'True' values in a boolean feature.</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               name         type  \\\n",
       "0              mode  aggregation   \n",
       "1               any  aggregation   \n",
       "2              last  aggregation   \n",
       "3               all  aggregation   \n",
       "4            median  aggregation   \n",
       "5               min  aggregation   \n",
       "6  avg_time_between  aggregation   \n",
       "7               max  aggregation   \n",
       "8     n_most_common  aggregation   \n",
       "9      percent_true  aggregation   \n",
       "\n",
       "                                                       description  \n",
       "0          Finds the most common element in a categorical feature.  \n",
       "1                                     Test if any value is 'True'.  \n",
       "2                                          Returns the last value.  \n",
       "3                                   Test if all values are 'True'.  \n",
       "4  Finds the median value of any feature with well-ordered values.  \n",
       "5           Finds the minimum non-null value of a numeric feature.  \n",
       "6            Computes the average time between consecutive events.  \n",
       "7           Finds the maximum non-null value of a numeric feature.  \n",
       "8       Finds the N most common elements in a categorical feature.  \n",
       "9         Finds the percent of 'True' values in a boolean feature.  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "primitives = ft.list_primitives()\n",
    "pd.options.display.max_colwidth = 100\n",
    "primitives[primitives['type'] == 'aggregation'].head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "-aDilk0Tdjml"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>type</th>\n",
       "      <th>description</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>add</td>\n",
       "      <td>transform</td>\n",
       "      <td>Creates a transform feature that adds two features.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>seconds</td>\n",
       "      <td>transform</td>\n",
       "      <td>Transform a Timedelta feature into the number of seconds.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>time_since_previous</td>\n",
       "      <td>transform</td>\n",
       "      <td>Compute the time since the previous instance.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>cum_min</td>\n",
       "      <td>transform</td>\n",
       "      <td>Calculates the min of previous values of an instance for each value in a time-dependent entity.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>divide</td>\n",
       "      <td>transform</td>\n",
       "      <td>Creates a transform feature that divides two features.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>hour</td>\n",
       "      <td>transform</td>\n",
       "      <td>Transform a Datetime feature into the hour.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>and</td>\n",
       "      <td>transform</td>\n",
       "      <td>For two boolean values, determine if both values are 'True'.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>or</td>\n",
       "      <td>transform</td>\n",
       "      <td>For two boolean values, determine if one value is 'True'.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>years</td>\n",
       "      <td>transform</td>\n",
       "      <td>Transform a Timedelta feature into the number of years.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>isin</td>\n",
       "      <td>transform</td>\n",
       "      <td>For each value of the base feature, checks whether it is in a provided list.</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   name       type  \\\n",
       "19                  add  transform   \n",
       "20              seconds  transform   \n",
       "21  time_since_previous  transform   \n",
       "22              cum_min  transform   \n",
       "23               divide  transform   \n",
       "24                 hour  transform   \n",
       "25                  and  transform   \n",
       "26                   or  transform   \n",
       "27                years  transform   \n",
       "28                 isin  transform   \n",
       "\n",
       "                                                                                        description  \n",
       "19                                              Creates a transform feature that adds two features.  \n",
       "20                                        Transform a Timedelta feature into the number of seconds.  \n",
       "21                                                    Compute the time since the previous instance.  \n",
       "22  Calculates the min of previous values of an instance for each value in a time-dependent entity.  \n",
       "23                                           Creates a transform feature that divides two features.  \n",
       "24                                                      Transform a Datetime feature into the hour.  \n",
       "25                                     For two boolean values, determine if both values are 'True'.  \n",
       "26                                        For two boolean values, determine if one value is 'True'.  \n",
       "27                                          Transform a Timedelta feature into the number of years.  \n",
       "28                     For each value of the base feature, checks whether it is in a provided list.  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "primitives[primitives['type'] == 'transform'].head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "QWMofjqMdur6"
   },
   "source": [
    "If featuretools does not have enough primitives for us, we can [also make our own.](https://docs.featuretools.com/automated_feature_engineering/primitives.html#defining-custom-primitives) \n",
    "\n",
    "To get an idea of what a feature primitive actually does, let's try out a few on our data. Using primitives is surprisingly easy using the `ft.dfs` function (which stands for deep feature synthesis). In this function, we specify the entityset to use; the `target_entity`, which is the dataframe we want to make the features for (where the features end up); the `agg_primitives` which are the aggregation feature primitives; and the `trans_primitives` which are the transformation primitives to apply. \n",
    "\n",
    "In the following example, we are using the `EntitySet` we already created, the target entity is the `clients` dataframe because we want to make new features about each client, and then we specify a few aggregation and transformation primitives. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "GVK71fusd8uF"
   },
   "outputs": [],
   "source": [
    "# Create new features using specified primitives\n",
    "features, feature_names = ft.dfs(entityset = es, target_entity = 'clients', \n",
    "                                 agg_primitives = ['mean', 'max', 'percent_true', 'last'],\n",
    "                                 trans_primitives = ['years', 'month', 'subtract', 'divide'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>MONTH(joined)</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>client_id</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>25707</th>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26326</th>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26695</th>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26945</th>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29841</th>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           MONTH(joined)\n",
       "client_id               \n",
       "25707                 10\n",
       "26326                  5\n",
       "26695                  8\n",
       "26945                 11\n",
       "29841                  8"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(features['MONTH(joined)'].head())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>MEAN(payments.payment_amount)</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>client_id</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>25707</th>\n",
       "      <td>1178.552795</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26326</th>\n",
       "      <td>1166.736842</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26695</th>\n",
       "      <td>1207.433824</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26945</th>\n",
       "      <td>1109.473214</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29841</th>\n",
       "      <td>1439.433333</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           MEAN(payments.payment_amount)\n",
       "client_id                               \n",
       "25707                        1178.552795\n",
       "26326                        1166.736842\n",
       "26695                        1207.433824\n",
       "26945                        1109.473214\n",
       "29841                        1439.433333"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(features['MEAN(payments.payment_amount)'].head())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "2EFpoAJYezfD"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>income</th>\n",
       "      <th>credit_score</th>\n",
       "      <th>join_month</th>\n",
       "      <th>log_income</th>\n",
       "      <th>MEAN(loans.loan_amount)</th>\n",
       "      <th>MEAN(loans.rate)</th>\n",
       "      <th>MAX(loans.loan_amount)</th>\n",
       "      <th>MAX(loans.rate)</th>\n",
       "      <th>LAST(loans.loan_type)</th>\n",
       "      <th>LAST(loans.loan_amount)</th>\n",
       "      <th>...</th>\n",
       "      <th>log_income - income / MAX(payments.payment_amount)</th>\n",
       "      <th>LAST(loans.loan_amount) / income</th>\n",
       "      <th>log_income - credit_score / join_month</th>\n",
       "      <th>join_month - log_income / MEAN(loans.loan_amount)</th>\n",
       "      <th>join_month / income - join_month</th>\n",
       "      <th>credit_score - log_income / LAST(loans.rate)</th>\n",
       "      <th>join_month - log_income / MAX(loans.loan_amount)</th>\n",
       "      <th>credit_score - income / join_month - log_income</th>\n",
       "      <th>join_month - income / MEAN(payments.payment_amount)</th>\n",
       "      <th>join_month - credit_score / log_income - credit_score</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>client_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>25707</th>\n",
       "      <td>211422</td>\n",
       "      <td>621</td>\n",
       "      <td>10</td>\n",
       "      <td>12.261611</td>\n",
       "      <td>7963.950000</td>\n",
       "      <td>3.477000</td>\n",
       "      <td>13913</td>\n",
       "      <td>9.44</td>\n",
       "      <td>home</td>\n",
       "      <td>2203</td>\n",
       "      <td>...</td>\n",
       "      <td>-78.184075</td>\n",
       "      <td>0.010420</td>\n",
       "      <td>-60.873839</td>\n",
       "      <td>-0.000284</td>\n",
       "      <td>0.000047</td>\n",
       "      <td>82.261944</td>\n",
       "      <td>-0.000163</td>\n",
       "      <td>93208.319781</td>\n",
       "      <td>-179.382715</td>\n",
       "      <td>1.003715</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26326</th>\n",
       "      <td>227920</td>\n",
       "      <td>633</td>\n",
       "      <td>5</td>\n",
       "      <td>12.336750</td>\n",
       "      <td>7270.062500</td>\n",
       "      <td>2.517500</td>\n",
       "      <td>13464</td>\n",
       "      <td>6.73</td>\n",
       "      <td>credit</td>\n",
       "      <td>5275</td>\n",
       "      <td>...</td>\n",
       "      <td>-85.744042</td>\n",
       "      <td>0.023144</td>\n",
       "      <td>-124.132650</td>\n",
       "      <td>-0.001009</td>\n",
       "      <td>0.000022</td>\n",
       "      <td>428.043621</td>\n",
       "      <td>-0.000545</td>\n",
       "      <td>30979.248435</td>\n",
       "      <td>-195.343964</td>\n",
       "      <td>1.011821</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26695</th>\n",
       "      <td>174532</td>\n",
       "      <td>680</td>\n",
       "      <td>8</td>\n",
       "      <td>12.069863</td>\n",
       "      <td>7824.722222</td>\n",
       "      <td>2.466111</td>\n",
       "      <td>14865</td>\n",
       "      <td>6.51</td>\n",
       "      <td>other</td>\n",
       "      <td>13918</td>\n",
       "      <td>...</td>\n",
       "      <td>-59.522486</td>\n",
       "      <td>0.079745</td>\n",
       "      <td>-83.491267</td>\n",
       "      <td>-0.000520</td>\n",
       "      <td>0.000046</td>\n",
       "      <td>742.144596</td>\n",
       "      <td>-0.000274</td>\n",
       "      <td>42716.912967</td>\n",
       "      <td>-144.541255</td>\n",
       "      <td>1.006093</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26945</th>\n",
       "      <td>214516</td>\n",
       "      <td>806</td>\n",
       "      <td>11</td>\n",
       "      <td>12.276140</td>\n",
       "      <td>7125.933333</td>\n",
       "      <td>2.855333</td>\n",
       "      <td>14593</td>\n",
       "      <td>5.65</td>\n",
       "      <td>cash</td>\n",
       "      <td>9249</td>\n",
       "      <td>...</td>\n",
       "      <td>-77.494120</td>\n",
       "      <td>0.043116</td>\n",
       "      <td>-72.156715</td>\n",
       "      <td>-0.000179</td>\n",
       "      <td>0.000051</td>\n",
       "      <td>277.525825</td>\n",
       "      <td>-0.000087</td>\n",
       "      <td>167466.003631</td>\n",
       "      <td>-193.339503</td>\n",
       "      <td>1.001608</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29841</th>\n",
       "      <td>38354</td>\n",
       "      <td>523</td>\n",
       "      <td>8</td>\n",
       "      <td>10.554614</td>\n",
       "      <td>9813.000000</td>\n",
       "      <td>3.445000</td>\n",
       "      <td>14837</td>\n",
       "      <td>6.76</td>\n",
       "      <td>home</td>\n",
       "      <td>7223</td>\n",
       "      <td>...</td>\n",
       "      <td>-13.231003</td>\n",
       "      <td>0.188325</td>\n",
       "      <td>-64.055673</td>\n",
       "      <td>-0.000260</td>\n",
       "      <td>0.000209</td>\n",
       "      <td>100.676893</td>\n",
       "      <td>-0.000172</td>\n",
       "      <td>14808.890291</td>\n",
       "      <td>-26.639650</td>\n",
       "      <td>1.004985</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 797 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "           income  credit_score  join_month  log_income  \\\n",
       "client_id                                                 \n",
       "25707      211422           621          10   12.261611   \n",
       "26326      227920           633           5   12.336750   \n",
       "26695      174532           680           8   12.069863   \n",
       "26945      214516           806          11   12.276140   \n",
       "29841       38354           523           8   10.554614   \n",
       "\n",
       "           MEAN(loans.loan_amount)  MEAN(loans.rate)  MAX(loans.loan_amount)  \\\n",
       "client_id                                                                      \n",
       "25707                  7963.950000          3.477000                   13913   \n",
       "26326                  7270.062500          2.517500                   13464   \n",
       "26695                  7824.722222          2.466111                   14865   \n",
       "26945                  7125.933333          2.855333                   14593   \n",
       "29841                  9813.000000          3.445000                   14837   \n",
       "\n",
       "           MAX(loans.rate) LAST(loans.loan_type)  LAST(loans.loan_amount)  \\\n",
       "client_id                                                                   \n",
       "25707                 9.44                  home                     2203   \n",
       "26326                 6.73                credit                     5275   \n",
       "26695                 6.51                 other                    13918   \n",
       "26945                 5.65                  cash                     9249   \n",
       "29841                 6.76                  home                     7223   \n",
       "\n",
       "                                   ...                            \\\n",
       "client_id                          ...                             \n",
       "25707                              ...                             \n",
       "26326                              ...                             \n",
       "26695                              ...                             \n",
       "26945                              ...                             \n",
       "29841                              ...                             \n",
       "\n",
       "           log_income - income / MAX(payments.payment_amount)  \\\n",
       "client_id                                                       \n",
       "25707                                              -78.184075   \n",
       "26326                                              -85.744042   \n",
       "26695                                              -59.522486   \n",
       "26945                                              -77.494120   \n",
       "29841                                              -13.231003   \n",
       "\n",
       "           LAST(loans.loan_amount) / income  \\\n",
       "client_id                                     \n",
       "25707                              0.010420   \n",
       "26326                              0.023144   \n",
       "26695                              0.079745   \n",
       "26945                              0.043116   \n",
       "29841                              0.188325   \n",
       "\n",
       "           log_income - credit_score / join_month  \\\n",
       "client_id                                           \n",
       "25707                                  -60.873839   \n",
       "26326                                 -124.132650   \n",
       "26695                                  -83.491267   \n",
       "26945                                  -72.156715   \n",
       "29841                                  -64.055673   \n",
       "\n",
       "           join_month - log_income / MEAN(loans.loan_amount)  \\\n",
       "client_id                                                      \n",
       "25707                                              -0.000284   \n",
       "26326                                              -0.001009   \n",
       "26695                                              -0.000520   \n",
       "26945                                              -0.000179   \n",
       "29841                                              -0.000260   \n",
       "\n",
       "           join_month / income - join_month  \\\n",
       "client_id                                     \n",
       "25707                              0.000047   \n",
       "26326                              0.000022   \n",
       "26695                              0.000046   \n",
       "26945                              0.000051   \n",
       "29841                              0.000209   \n",
       "\n",
       "           credit_score - log_income / LAST(loans.rate)  \\\n",
       "client_id                                                 \n",
       "25707                                         82.261944   \n",
       "26326                                        428.043621   \n",
       "26695                                        742.144596   \n",
       "26945                                        277.525825   \n",
       "29841                                        100.676893   \n",
       "\n",
       "           join_month - log_income / MAX(loans.loan_amount)  \\\n",
       "client_id                                                     \n",
       "25707                                             -0.000163   \n",
       "26326                                             -0.000545   \n",
       "26695                                             -0.000274   \n",
       "26945                                             -0.000087   \n",
       "29841                                             -0.000172   \n",
       "\n",
       "           credit_score - income / join_month - log_income  \\\n",
       "client_id                                                    \n",
       "25707                                         93208.319781   \n",
       "26326                                         30979.248435   \n",
       "26695                                         42716.912967   \n",
       "26945                                        167466.003631   \n",
       "29841                                         14808.890291   \n",
       "\n",
       "           join_month - income / MEAN(payments.payment_amount)  \\\n",
       "client_id                                                        \n",
       "25707                                              -179.382715   \n",
       "26326                                              -195.343964   \n",
       "26695                                              -144.541255   \n",
       "26945                                              -193.339503   \n",
       "29841                                               -26.639650   \n",
       "\n",
       "           join_month - credit_score / log_income - credit_score  \n",
       "client_id                                                         \n",
       "25707                                                   1.003715  \n",
       "26326                                                   1.011821  \n",
       "26695                                                   1.006093  \n",
       "26945                                                   1.001608  \n",
       "29841                                                   1.004985  \n",
       "\n",
       "[5 rows x 797 columns]"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "features.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "C0kKwJHQfP9J"
   },
   "source": [
    "Already we can see how useful featuretools is: it performed the same operations we did manually but also many more in addition. Examining the names of the features in the dataframe brings us to the final piece of the puzzle: deep features.\n",
    "\n",
    "## Deep Feature Synthesis\n",
    "\n",
    "While feature primitives are useful by themselves, the main benefit of using featuretools arises when we stack primitives to get deep features. The depth of a feature is simply the number of primitives required to make a feature. So, a feature that relies on a single aggregation would be a deep feature with a depth of 1, a feature that stacks two primitives would have a depth of 2 and so on. The idea itself is lot simpler than the name \"deep feature synthesis\" implies. (I think the authors were trying to ride the way of deep neural network hype when they named the method!) To read more about deep feature synthesis, check out [the documentation](https://docs.featuretools.com/automated_feature_engineering/afe.html) or the [original paper by Max Kanter et al](http://www.jmaxkanter.com/static/papers/DSAA_DSM_2015.pdf). \n",
    "\n",
    "Already in the dataframe we made by specifying the primitives manually we can see the idea of feature depth. For instance, the MEAN(loans.loan_amount) feature has a depth of 1 because it is made by applying a single aggregation primitive. This feature represents the average size of a client's previous loans."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "HBHqFlqpgKD6"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>MEAN(loans.loan_amount)</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>client_id</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>25707</th>\n",
       "      <td>7963.950000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26326</th>\n",
       "      <td>7270.062500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26695</th>\n",
       "      <td>7824.722222</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26945</th>\n",
       "      <td>7125.933333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29841</th>\n",
       "      <td>9813.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32726</th>\n",
       "      <td>6633.263158</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32885</th>\n",
       "      <td>9920.400000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32961</th>\n",
       "      <td>7882.235294</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35089</th>\n",
       "      <td>6939.200000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35214</th>\n",
       "      <td>7173.555556</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           MEAN(loans.loan_amount)\n",
       "client_id                         \n",
       "25707                  7963.950000\n",
       "26326                  7270.062500\n",
       "26695                  7824.722222\n",
       "26945                  7125.933333\n",
       "29841                  9813.000000\n",
       "32726                  6633.263158\n",
       "32885                  9920.400000\n",
       "32961                  7882.235294\n",
       "35089                  6939.200000\n",
       "35214                  7173.555556"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Show a feature with a depth of 1\n",
    "pd.DataFrame(features['MEAN(loans.loan_amount)'].head(10))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "b3FttmpFdDoo"
   },
   "source": [
    "As well scroll through the features, we see a number of features with a depth of 2. For example, the LAST(loans.(MEAN(payments.payment_amount))) has depth = 2 because it is made by stacking two feature primitives, first an aggregation and then a transformation. This feature represents the average payment amount for the last (most recent) loan for each client."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "xStafjwZgneQ"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>LAST(loans.MEAN(payments.payment_amount))</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>client_id</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>25707</th>\n",
       "      <td>293.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26326</th>\n",
       "      <td>977.375000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26695</th>\n",
       "      <td>1769.166667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26945</th>\n",
       "      <td>1598.666667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29841</th>\n",
       "      <td>1125.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32726</th>\n",
       "      <td>799.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32885</th>\n",
       "      <td>1729.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32961</th>\n",
       "      <td>282.600000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35089</th>\n",
       "      <td>110.400000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35214</th>\n",
       "      <td>1410.250000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           LAST(loans.MEAN(payments.payment_amount))\n",
       "client_id                                           \n",
       "25707                                     293.500000\n",
       "26326                                     977.375000\n",
       "26695                                    1769.166667\n",
       "26945                                    1598.666667\n",
       "29841                                    1125.500000\n",
       "32726                                     799.500000\n",
       "32885                                    1729.000000\n",
       "32961                                     282.600000\n",
       "35089                                     110.400000\n",
       "35214                                    1410.250000"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Show a feature with a depth of 2\n",
    "pd.DataFrame(features['LAST(loans.MEAN(payments.payment_amount))'].head(10))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "wyxx5JFph2S8"
   },
   "source": [
    "We can create features of arbitrary depth by stacking more primitives. However, when I have used featuretools I've never gone beyond a depth of 2! After this point, the features become very convoluted to understand. I'd encourage anyone interested to experiment with increasing the depth (maybe for a real problem) and see if there is value to \"going deeper\".\n",
    "\n",
    "## Automated Deep Feature Synthesis\n",
    "\n",
    "In addition to manually specifying aggregation and transformation feature primitives, we can let featuretools automatically generate many new features. We do this by making the same `ft.dfs` function call, but without passing in any primitives. We just set the `max_depth` parameter and featuretools will automatically try many all combinations of feature primitives to the ordered depth. \n",
    "\n",
    "When running on large datasets, this process can take quite a while, but for our example data, it will be relatively quick. For this call, we only need to specify the `entityset`, the `target_entity` (which will again be `clients`), and the `max_depth`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "iKQx-y2mgv_x"
   },
   "outputs": [],
   "source": [
    "# Perform deep feature synthesis without specifying primitives\n",
    "features, feature_names = ft.dfs(entityset=es, target_entity='clients', \n",
    "                                 max_depth = 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "H-CsAUe6jS0F"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SUM(loans.loan_amount)</th>\n",
       "      <th>SUM(loans.rate)</th>\n",
       "      <th>STD(loans.loan_amount)</th>\n",
       "      <th>STD(loans.rate)</th>\n",
       "      <th>MAX(loans.loan_amount)</th>\n",
       "      <th>MAX(loans.rate)</th>\n",
       "      <th>SKEW(loans.loan_amount)</th>\n",
       "      <th>SKEW(loans.rate)</th>\n",
       "      <th>MIN(loans.loan_amount)</th>\n",
       "      <th>MIN(loans.rate)</th>\n",
       "      <th>...</th>\n",
       "      <th>NUM_UNIQUE(loans.WEEKDAY(loan_end))</th>\n",
       "      <th>MODE(loans.MODE(payments.missed))</th>\n",
       "      <th>MODE(loans.DAY(loan_start))</th>\n",
       "      <th>MODE(loans.DAY(loan_end))</th>\n",
       "      <th>MODE(loans.YEAR(loan_start))</th>\n",
       "      <th>MODE(loans.YEAR(loan_end))</th>\n",
       "      <th>MODE(loans.MONTH(loan_start))</th>\n",
       "      <th>MODE(loans.MONTH(loan_end))</th>\n",
       "      <th>MODE(loans.WEEKDAY(loan_start))</th>\n",
       "      <th>MODE(loans.WEEKDAY(loan_end))</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>client_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>25707</th>\n",
       "      <td>159279</td>\n",
       "      <td>69.54</td>\n",
       "      <td>4044.418728</td>\n",
       "      <td>2.421285</td>\n",
       "      <td>13913</td>\n",
       "      <td>9.44</td>\n",
       "      <td>-0.172074</td>\n",
       "      <td>0.679118</td>\n",
       "      <td>1212</td>\n",
       "      <td>0.33</td>\n",
       "      <td>...</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>27</td>\n",
       "      <td>1</td>\n",
       "      <td>2010</td>\n",
       "      <td>2007</td>\n",
       "      <td>1</td>\n",
       "      <td>8</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26326</th>\n",
       "      <td>116321</td>\n",
       "      <td>40.28</td>\n",
       "      <td>4254.149422</td>\n",
       "      <td>1.991819</td>\n",
       "      <td>13464</td>\n",
       "      <td>6.73</td>\n",
       "      <td>0.135246</td>\n",
       "      <td>1.067853</td>\n",
       "      <td>1164</td>\n",
       "      <td>0.50</td>\n",
       "      <td>...</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>2003</td>\n",
       "      <td>2005</td>\n",
       "      <td>4</td>\n",
       "      <td>7</td>\n",
       "      <td>5</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26695</th>\n",
       "      <td>140845</td>\n",
       "      <td>44.39</td>\n",
       "      <td>4078.228493</td>\n",
       "      <td>1.517660</td>\n",
       "      <td>14865</td>\n",
       "      <td>6.51</td>\n",
       "      <td>0.154467</td>\n",
       "      <td>0.820060</td>\n",
       "      <td>2389</td>\n",
       "      <td>0.22</td>\n",
       "      <td>...</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>14</td>\n",
       "      <td>2003</td>\n",
       "      <td>2005</td>\n",
       "      <td>9</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26945</th>\n",
       "      <td>106889</td>\n",
       "      <td>42.83</td>\n",
       "      <td>4389.555657</td>\n",
       "      <td>1.564795</td>\n",
       "      <td>14593</td>\n",
       "      <td>5.65</td>\n",
       "      <td>0.156534</td>\n",
       "      <td>-0.001998</td>\n",
       "      <td>653</td>\n",
       "      <td>0.13</td>\n",
       "      <td>...</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>16</td>\n",
       "      <td>1</td>\n",
       "      <td>2002</td>\n",
       "      <td>2004</td>\n",
       "      <td>12</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29841</th>\n",
       "      <td>176634</td>\n",
       "      <td>62.01</td>\n",
       "      <td>4090.630609</td>\n",
       "      <td>2.063092</td>\n",
       "      <td>14837</td>\n",
       "      <td>6.76</td>\n",
       "      <td>-0.212397</td>\n",
       "      <td>0.050600</td>\n",
       "      <td>2778</td>\n",
       "      <td>0.26</td>\n",
       "      <td>...</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>15</td>\n",
       "      <td>2005</td>\n",
       "      <td>2007</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 90 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "           SUM(loans.loan_amount)  SUM(loans.rate)  STD(loans.loan_amount)  \\\n",
       "client_id                                                                    \n",
       "25707                      159279            69.54             4044.418728   \n",
       "26326                      116321            40.28             4254.149422   \n",
       "26695                      140845            44.39             4078.228493   \n",
       "26945                      106889            42.83             4389.555657   \n",
       "29841                      176634            62.01             4090.630609   \n",
       "\n",
       "           STD(loans.rate)  MAX(loans.loan_amount)  MAX(loans.rate)  \\\n",
       "client_id                                                             \n",
       "25707             2.421285                   13913             9.44   \n",
       "26326             1.991819                   13464             6.73   \n",
       "26695             1.517660                   14865             6.51   \n",
       "26945             1.564795                   14593             5.65   \n",
       "29841             2.063092                   14837             6.76   \n",
       "\n",
       "           SKEW(loans.loan_amount)  SKEW(loans.rate)  MIN(loans.loan_amount)  \\\n",
       "client_id                                                                      \n",
       "25707                    -0.172074          0.679118                    1212   \n",
       "26326                     0.135246          1.067853                    1164   \n",
       "26695                     0.154467          0.820060                    2389   \n",
       "26945                     0.156534         -0.001998                     653   \n",
       "29841                    -0.212397          0.050600                    2778   \n",
       "\n",
       "           MIN(loans.rate)              ...                \\\n",
       "client_id                               ...                 \n",
       "25707                 0.33              ...                 \n",
       "26326                 0.50              ...                 \n",
       "26695                 0.22              ...                 \n",
       "26945                 0.13              ...                 \n",
       "29841                 0.26              ...                 \n",
       "\n",
       "           NUM_UNIQUE(loans.WEEKDAY(loan_end))  \\\n",
       "client_id                                        \n",
       "25707                                        6   \n",
       "26326                                        5   \n",
       "26695                                        6   \n",
       "26945                                        6   \n",
       "29841                                        7   \n",
       "\n",
       "           MODE(loans.MODE(payments.missed))  MODE(loans.DAY(loan_start))  \\\n",
       "client_id                                                                   \n",
       "25707                                      0                           27   \n",
       "26326                                      0                            6   \n",
       "26695                                      0                            3   \n",
       "26945                                      0                           16   \n",
       "29841                                      1                            1   \n",
       "\n",
       "           MODE(loans.DAY(loan_end))  MODE(loans.YEAR(loan_start))  \\\n",
       "client_id                                                            \n",
       "25707                              1                          2010   \n",
       "26326                              6                          2003   \n",
       "26695                             14                          2003   \n",
       "26945                              1                          2002   \n",
       "29841                             15                          2005   \n",
       "\n",
       "          MODE(loans.YEAR(loan_end))  MODE(loans.MONTH(loan_start))  \\\n",
       "client_id                                                             \n",
       "25707                           2007                              1   \n",
       "26326                           2005                              4   \n",
       "26695                           2005                              9   \n",
       "26945                           2004                             12   \n",
       "29841                           2007                              3   \n",
       "\n",
       "           MODE(loans.MONTH(loan_end))  MODE(loans.WEEKDAY(loan_start))  \\\n",
       "client_id                                                                 \n",
       "25707                                8                                3   \n",
       "26326                                7                                5   \n",
       "26695                                4                                1   \n",
       "26945                                5                                0   \n",
       "29841                                2                                5   \n",
       "\n",
       "           MODE(loans.WEEKDAY(loan_end))  \n",
       "client_id                                 \n",
       "25707                                  0  \n",
       "26326                                  2  \n",
       "26695                                  1  \n",
       "26945                                  1  \n",
       "29841                                  1  \n",
       "\n",
       "[5 rows x 90 columns]"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "features.iloc[:, 4:].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "s3cOoBMakhx8"
   },
   "source": [
    "Deep feature synthesis has created 90 new features out of the existing data! While we could have created all of these manually, I am glad to not have to write all that code by hand. The primary benefit of featuretools is that it creates features without any subjective human biases. Even a human with considerable domain knowledge will be limited by their imagination when making new features (not to mention time). Automated feature engineering is not limited by these factors (instead it's limited by computation time) and provides a good starting point for feature creation. This process likely will not remove the human contribution to feature engineering completely because a human can still use domain knowledge and machine learning expertise to select the most important features or build new features from those suggested by automated deep feature synthesis.\n",
    "\n",
    "# Next Steps\n",
    "\n",
    "While automatic feature engineering solves one problem, it provides us with another problem: too many features! Although it's difficult to say which features will be important to a given machine learning task ahead of time, it's likely that not all of the features made by featuretools add value. In fact, having too many features is a significant issue in machine learning because it makes training a model much harder. The [irrelevant features can drown out the important features](https://pdfs.semanticscholar.org/a83b/ddb34618cc68f1014ca12eef7f537825d104.pdf), leaving a model unable to learn how to map the features to the target.\n",
    "\n",
    "This problem is known as the [\"curse of dimensionality\"](https://en.wikipedia.org/wiki/Curse_of_dimensionality#Machine_learning) and is addressed through the process of [feature reduction and selection](http://scikit-learn.org/stable/modules/feature_selection.html), which means [removing low-value features](https://machinelearningmastery.com/feature-selection-machine-learning-python/) from the data. Defining which features are useful is an important problem where a data scientist can still add considerable value to the feature engineering task. Feature reduction will have to be another topic for another day!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "jbjt01ADmEdX"
   },
   "source": [
    "# Conclusions\n",
    "\n",
    "In this notebook, we saw how to apply automated feature engineering to an example dataset. This is a powerful method which allows us to overcome the human limits of time and imagination to create many new features from multiple tables of data. Featuretools is built on the idea of deep feature synthesis, which means stacking multiple simple feature primitives - __aggregations and transformations__ - to create new features. Feature engineering allows us to combine information across many tables into a single dataframe that we can then use for machine learning model training. Finally, the next step after creating all of these features is figuring out which ones are important. \n",
    "\n",
    "Featuretools is currently the only Python option for this process, but with the recent emphasis on automating aspects of the machine learning pipeline, other competitiors will probably enter the sphere. While the exact tools will change, the idea of automatically creating new features out of existing data will grow in importance. Staying up-to-date on methods such as automated feature engineering is crucial in the rapidly changing field of data science. Now go out there and find a problem on which to apply featuretools! \n",
    "\n",
    "For more information, check out the [documentation for featuretools](https://docs.featuretools.com/index.html). Also, read about how featuretools is [used in the real world by Feature Labs](https://www.featurelabs.com/), the company behind the open-source library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "default_view": {},
   "name": "Automated Feature Engineering.ipynb",
   "private_outputs": true,
   "provenance": [],
   "version": "0.3.2",
   "views": {}
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}