fastprop_benchmark/dodgers_prop.ipynb

{
    "cells": [
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "# Propositionalization: Traffic near Dodgers' stadium\n",
                "\n",
                "In this notebook, we compare getML's FastProp against well-known feature engineering libraries featuretools and tsfresh.\n",
                "\n",
                "Summary:\n",
                "\n",
                "- Prediction type: __Regression model__\n",
                "- Domain: __Transportation__\n",
                "- Prediction target: __traffic volume__ \n",
                "- Source data: __Univariate time series__\n",
                "- Population size: __47497__"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "tags": [
                    "remove_cell_on_docs"
                ]
            },
            "source": [
                "<a target=\"_blank\" href=\"https://colab.research.google.com/github/getml/getml-demo/blob/master/fastprop_benchmark/dodgers_prop.ipynb\">\n",
                "  <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/>\n",
                "</a>"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## Background\n",
                "\n",
                "A common approach to feature engineering is to generate attribute-value representations from relational data by applying a fixed set of aggregations to columns of interest and perform a feature selection on the (possibly large) set of generated features afterwards. In academia, this approach is called _propositionalization._\n",
                "\n",
                "getML's [FastProp](https://getml.com/latest/user_guide/concepts/feature_engineering/#feature-engineering-algorithms-fastprop) is an implementation of this propositionalization approach that has been optimized for speed and memory efficiency. In this notebook, we want to demonstrate how – well – fast FastProp is. To this end, we will benchmark FastProp against the popular feature engineering libraries [featuretools](https://www.featuretools.com/) and [tsfresh](https://tsfresh.readthedocs.io/en/latest/). Both of these libraries use propositionalization approaches for feature engineering.\n",
                "\n",
                "In this notebook, we use traffic data that was collected for the Glendale on ramp for the 101 North freeway in Los Angeles. For further details about the data set refer to [the full notebook](https://getml.com/latest/examples/enterprise-notebooks/dodgers)."
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## Analysis"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "1. [Loading data](#1.-Loading-data)\n",
                "2. [Predictive modeling](#2.-Predictive-modeling)\n",
                "3. [Comparison](#3.-Comparison)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Let's get started with the analysis and set-up your session:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Note: you may need to restart the kernel to use updated packages.\n"
                    ]
                }
            ],
            "source": [
                "%pip install -q \"getml==1.5.0\" \"featuretools==1.31.0\" \"tsfresh==0.20.3\""
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 2,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "getML API version: 1.5.0\n",
                        "\n"
                    ]
                }
            ],
            "source": [
                "import os\n",
                "import sys\n",
                "\n",
                "os.environ[\"PYARROW_IGNORE_TIMEZONE\"] = \"1\"\n",
                "from pathlib import Path\n",
                "from urllib import request\n",
                "import getml\n",
                "import pandas as pd\n",
                "\n",
                "print(f\"getML API version: {getml.__version__}\\n\")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 3,
            "metadata": {},
            "outputs": [],
            "source": [
                "# If we are in Colab, we need to fetch the utils folder from the repository\n",
                "if os.getenv(\"COLAB_RELEASE_TAG\"):\n",
                "    !curl -L https://api.github.com/repos/getml/getml-demo/tarball/master | tar --wildcards --strip-components=1 -xz '*utils*'"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 4,
            "metadata": {},
            "outputs": [],
            "source": [
                "parent = Path(os.getcwd()).parent.as_posix()\n",
                "\n",
                "if parent not in sys.path:\n",
                "    sys.path.append(parent)\n",
                "\n",
                "from utils import Benchmark, FTTimeSeriesBuilder, TSFreshBuilder"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 5,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "getML Engine is already running.\n"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Connected to project <span style=\"color: #008000; text-decoration-color: #008000\">'dodgers'</span>.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Connected to project \u001b[32m'dodgers'\u001b[0m.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "getml.engine.launch(allow_remote_ips=True, token=\"token\")\n",
                "getml.engine.set_project(\"dodgers\")"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "### 1. Loading data"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "#### 1.1 Download from source\n",
                "\n",
                "We begin by downloading the data from the UC Irvine Machine Learning repository:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 6,
            "metadata": {},
            "outputs": [],
            "source": [
                "fname = \"Dodgers.data\"\n",
                "\n",
                "if not os.path.exists(fname):\n",
                "    fname, res = request.urlretrieve(\n",
                "        \"https://archive.ics.uci.edu/ml/machine-learning-databases/event-detection/\"\n",
                "        + fname,\n",
                "        fname,\n",
                "    )\n",
                "\n",
                "data_full_pandas = pd.read_csv(fname, header=None)\n",
                "data_full_pandas.columns = [\"ds\", \"y\"]"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 7,
            "metadata": {},
            "outputs": [],
            "source": [
                "data_full_pandas[\"ds\"] = pd.to_datetime(data_full_pandas[\"ds\"], format=\"%m/%d/%Y %H:%M\")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 8,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<div>\n",
                            "<style scoped>\n",
                            "    .dataframe tbody tr th:only-of-type {\n",
                            "        vertical-align: middle;\n",
                            "    }\n",
                            "\n",
                            "    .dataframe tbody tr th {\n",
                            "        vertical-align: top;\n",
                            "    }\n",
                            "\n",
                            "    .dataframe thead th {\n",
                            "        text-align: right;\n",
                            "    }\n",
                            "</style>\n",
                            "<table border=\"1\" class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr style=\"text-align: right;\">\n",
                            "      <th></th>\n",
                            "      <th>ds</th>\n",
                            "      <th>y</th>\n",
                            "    </tr>\n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    <tr>\n",
                            "      <th>0</th>\n",
                            "      <td>2005-04-10 00:00:00</td>\n",
                            "      <td>-1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>1</th>\n",
                            "      <td>2005-04-10 00:05:00</td>\n",
                            "      <td>-1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>2</th>\n",
                            "      <td>2005-04-10 00:10:00</td>\n",
                            "      <td>-1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>3</th>\n",
                            "      <td>2005-04-10 00:15:00</td>\n",
                            "      <td>-1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>4</th>\n",
                            "      <td>2005-04-10 00:20:00</td>\n",
                            "      <td>-1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>...</th>\n",
                            "      <td>...</td>\n",
                            "      <td>...</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>50395</th>\n",
                            "      <td>2005-10-01 23:35:00</td>\n",
                            "      <td>-1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>50396</th>\n",
                            "      <td>2005-10-01 23:40:00</td>\n",
                            "      <td>-1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>50397</th>\n",
                            "      <td>2005-10-01 23:45:00</td>\n",
                            "      <td>-1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>50398</th>\n",
                            "      <td>2005-10-01 23:50:00</td>\n",
                            "      <td>-1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>50399</th>\n",
                            "      <td>2005-10-01 23:55:00</td>\n",
                            "      <td>-1</td>\n",
                            "    </tr>\n",
                            "  </tbody>\n",
                            "</table>\n",
                            "<p>50400 rows × 2 columns</p>\n",
                            "</div>"
                        ],
                        "text/plain": [
                            "                       ds  y\n",
                            "0     2005-04-10 00:00:00 -1\n",
                            "1     2005-04-10 00:05:00 -1\n",
                            "2     2005-04-10 00:10:00 -1\n",
                            "3     2005-04-10 00:15:00 -1\n",
                            "4     2005-04-10 00:20:00 -1\n",
                            "...                   ... ..\n",
                            "50395 2005-10-01 23:35:00 -1\n",
                            "50396 2005-10-01 23:40:00 -1\n",
                            "50397 2005-10-01 23:45:00 -1\n",
                            "50398 2005-10-01 23:50:00 -1\n",
                            "50399 2005-10-01 23:55:00 -1\n",
                            "\n",
                            "[50400 rows x 2 columns]"
                        ]
                    },
                    "execution_count": 8,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "data_full_pandas"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "### 1.2 Prepare data for getML"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 9,
            "metadata": {},
            "outputs": [],
            "source": [
                "data_full = getml.data.DataFrame.from_pandas(data_full_pandas, \"data_full\")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 10,
            "metadata": {},
            "outputs": [],
            "source": [
                "data_full.set_role(\"y\", getml.data.roles.target)\n",
                "data_full.set_role(\"ds\", getml.data.roles.time_stamp)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 11,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<style>\n",
                            "  table {\n",
                            "    font-family: Helvetica, sans-serif;\n",
                            "  }\n",
                            "\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right !important;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.sub-header {\n",
                            "    font-weight: normal;\n",
                            "    font-style: italic;\n",
                            "  }\n",
                            "  .join_key,\n",
                            "  .numerical,\n",
                            "  .target,\n",
                            "  .unused_float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "\n",
                            "  .char-align {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  span.left {\n",
                            "    text-align: right;\n",
                            "    width: 3em;\n",
                            "  }\n",
                            "  span.right {\n",
                            "    float: right;\n",
                            "    text-align: left;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th> name</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"time_stamp\">                         ds</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"target\">     y</th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        \n",
                            "          \n",
                            "            <th class=\"sub-header\"> role</th>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            <th class=\"sub-header time_stamp\">                 time_stamp</th>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            <th class=\"sub-header target\">target</th>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        \n",
                            "          \n",
                            "            <th class=\"sub-header\"> unit</th>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            <th class=\"sub-header time_stamp\">time stamp, comparison only</th>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            <th class=\"sub-header target\">      </th>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "        \n",
                            "          \n",
                            "            <td class=\"time_stamp\">2005-04-10</td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align target\">\n",
                            "              <span class=\"left\">-1</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>1</th>\n",
                            "        \n",
                            "          \n",
                            "            <td class=\"time_stamp\">2005-04-10 00:05:00</td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align target\">\n",
                            "              <span class=\"left\">-1</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>2</th>\n",
                            "        \n",
                            "          \n",
                            "            <td class=\"time_stamp\">2005-04-10 00:10:00</td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align target\">\n",
                            "              <span class=\"left\">-1</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>3</th>\n",
                            "        \n",
                            "          \n",
                            "            <td class=\"time_stamp\">2005-04-10 00:15:00</td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align target\">\n",
                            "              <span class=\"left\">-1</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>4</th>\n",
                            "        \n",
                            "          \n",
                            "            <td class=\"time_stamp\">2005-04-10 00:20:00</td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align target\">\n",
                            "              <span class=\"left\">-1</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th></th>\n",
                            "        \n",
                            "          \n",
                            "            <td class=\"time_stamp\">...</td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align target\">\n",
                            "              <span class=\"left\">...</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>50395</th>\n",
                            "        \n",
                            "          \n",
                            "            <td class=\"time_stamp\">2005-10-01 23:35:00</td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align target\">\n",
                            "              <span class=\"left\">-1</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>50396</th>\n",
                            "        \n",
                            "          \n",
                            "            <td class=\"time_stamp\">2005-10-01 23:40:00</td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align target\">\n",
                            "              <span class=\"left\">-1</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>50397</th>\n",
                            "        \n",
                            "          \n",
                            "            <td class=\"time_stamp\">2005-10-01 23:45:00</td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align target\">\n",
                            "              <span class=\"left\">-1</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>50398</th>\n",
                            "        \n",
                            "          \n",
                            "            <td class=\"time_stamp\">2005-10-01 23:50:00</td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align target\">\n",
                            "              <span class=\"left\">-1</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>50399</th>\n",
                            "        \n",
                            "          \n",
                            "            <td class=\"time_stamp\">2005-10-01 23:55:00</td>\n",
                            "          \n",
                            "        \n",
                            "          \n",
                            "            \n",
                            "              \n",
                            "              \n",
                            "          \n",
                            "            <td class=\"char-align target\">\n",
                            "              <span class=\"left\">-1</span\n",
                            "              ><span class=\"right\" style=\"width: 0ch\"\n",
                            "                >&nbsp;</span\n",
                            "              >\n",
                            "            </td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>\n",
                            "\n",
                            "  <p>\n",
                            "    50400 rows x 2 columns<br />\n",
                            "    memory usage: 0.81 MB<br />\n",
                            "    name: data_full<br />\n",
                            "    type: getml.DataFrame<br />\n",
                            "    \n",
                            "  </p>\n"
                        ],
                        "text/plain": [
                            " name                            ds        y\n",
                            " role                    time_stamp   target\n",
                            " unit   time stamp, comparison only         \n",
                            "    0                    2005-04-10      -1 \n",
                            "    1           2005-04-10 00:05:00      -1 \n",
                            "    2           2005-04-10 00:10:00      -1 \n",
                            "    3           2005-04-10 00:15:00      -1 \n",
                            "    4           2005-04-10 00:20:00      -1 \n",
                            "                                ...      ...\n",
                            "50395           2005-10-01 23:35:00      -1 \n",
                            "50396           2005-10-01 23:40:00      -1 \n",
                            "50397           2005-10-01 23:45:00      -1 \n",
                            "50398           2005-10-01 23:50:00      -1 \n",
                            "50399           2005-10-01 23:55:00      -1 \n",
                            "\n",
                            "\n",
                            "50400 rows x 2 columns\n",
                            "memory usage: 0.81 MB\n",
                            "type: getml.DataFrame"
                        ]
                    },
                    "execution_count": 11,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "data_full"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 12,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th.sub-header {\n",
                            "    font-weight: normal;\n",
                            "    font-style: italic;\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right !important;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.numerical {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.numerical {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "\n",
                            "  td.char-align {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  span.left {\n",
                            "    text-align: right;\n",
                            "    width: 3em;\n",
                            "  }\n",
                            "  span.right {\n",
                            "    float: right;\n",
                            "    text-align: left;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th>     </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th>     </th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "        \n",
                            "          \n",
                            "            <td>train</td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>1</th>\n",
                            "        \n",
                            "          \n",
                            "            <td>train</td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>2</th>\n",
                            "        \n",
                            "          \n",
                            "            <td>train</td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>3</th>\n",
                            "        \n",
                            "          \n",
                            "            <td>train</td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>4</th>\n",
                            "        \n",
                            "          \n",
                            "            <td>train</td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th></th>\n",
                            "        \n",
                            "          \n",
                            "            <td>...</td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>\n",
                            "\n",
                            "  <p>\n",
                            "    50400 rows<br />\n",
                            "    \n",
                            "    type: StringColumnView<br />\n",
                            "    \n",
                            "  </p>\n"
                        ],
                        "text/plain": [
                            "             \n",
                            "    0   train\n",
                            "    1   train\n",
                            "    2   train\n",
                            "    3   train\n",
                            "    4   train\n",
                            "        ...  \n",
                            "\n",
                            "\n",
                            "50400 rows\n",
                            "type: StringColumnView"
                        ]
                    },
                    "execution_count": 12,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "split = getml.data.split.time(\n",
                "    population=data_full, time_stamp=\"ds\", test=getml.data.time.datetime(2005, 8, 20)\n",
                ")\n",
                "split"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "#### 1.3 Define relational model\n",
                "\n",
                "To start with relational learning, we need to specify the data model. We manually replicate the appropriate time series structure by setting time series related join conditions (`horizon`, `memory` and `allow_lagged_targets`). This is done abstractly using [Placeholders](https://getml.com/latest/user_guide/concepts/data_model#placeholders)\n",
                "\n",
                "The data model consists of two tables:\n",
                "* __Population table__ `traffic_{test/train}`: holds target and the contemporarily available time-based components\n",
                "* __Peripheral table__ `traffic`: same table as the population table\n",
                "* Join between both placeholders specifies (`horizon`) to prevent leaks and (`memory`) that keeps the computations feasible"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 13,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<span style='font-size: 1.2rem; font-weight: 500;'>data model</span>\n",
                            "            <div style='margin-top: 15px; margin-bottom: 5px;'>\n",
                            "            <div style='margin-bottom: 10px; font-size: 1rem;'>diagram</div>\n",
                            "            <div style=\"height:100px;width:660px;position:relative;\"><svg height=\"90\" width=\"650\"><rect y=\"0\" x=\"0\" rx=\"10\" ry=\"10\" width=\"150\" height=\"90\" style=\"fill:#6829c2;stroke-width:0;\" /><text y=\"73.8\" x=\"75.0\" dominant-baseline=\"middle\" text-anchor=\"middle\" fill=\"white\">data_full</text><rect x=\"51\" y=\"10\" rx=\"4\" ry=\"4\" width=\"48\" height=\"48\" style=\" fill:#6829c2;stroke:#ffffff;stroke-width:3;\" /><line x1=\"67.0\" y1=\"10\" x2=\"67.0\" y2=\"58\" style=\"stroke:white;stroke-width:3\" /><line x1=\"83.0\" y1=\"10\" x2=\"83.0\" y2=\"58\" style=\"stroke:white;stroke-width:3\" /><line x1=\"51\" y1=\"26.0\" x2=\"99\" y2=\"26.0\" style=\"stroke:white;stroke-width:3\" /><line x1=\"51\" y1=\"42.0\" x2=\"99\" y2=\"42.0\" style=\"stroke:white;stroke-width:3\" /><rect y=\"0\" x=\"500\" rx=\"10\" ry=\"10\" width=\"150\" height=\"90\" style=\"fill:#6829c2;stroke-width:0;\" /><text y=\"73.8\" x=\"575.0\" dominant-baseline=\"middle\" text-anchor=\"middle\" fill=\"white\">population</text><rect x=\"551\" y=\"10\" rx=\"4\" ry=\"4\" width=\"48\" height=\"48\" style=\" fill:#6829c2;stroke:#ffffff;stroke-width:3;\" /><line x1=\"567.0\" y1=\"10\" x2=\"567.0\" y2=\"58\" style=\"stroke:white;stroke-width:3\" /><line x1=\"583.0\" y1=\"10\" x2=\"583.0\" y2=\"58\" style=\"stroke:white;stroke-width:3\" /><line x1=\"551\" y1=\"26.0\" x2=\"599\" y2=\"26.0\" style=\"stroke:white;stroke-width:3\" /><line x1=\"551\" y1=\"42.0\" x2=\"599\" y2=\"42.0\" style=\"stroke:white;stroke-width:3\" /><line x1=\"150\" y1=\"43.0\" x2=\"490\" y2=\"43.0\" style=\"stroke:#808080;;stroke-width:4\" /><polygon points=\"500, 43.0 490, 37.0 490, 49.0 \" style=\"fill:#808080;;stroke-width:0;\" /><rect y=\"10.0\" x=\"249.0\" rx=\"10\" ry=\"10\" width=\"150\" height=\"70\" style=\"fill:#6829c2;stroke-width:0;\" /><text dominant-baseline=\"middle\" text-anchor=\"middle\" fill=\"white\"><tspan y=\"30.0\" x=\"324.0\" font-size=\"7pt\" >ds &lt;= ds</tspan><tspan y=\"40.0\" x=\"324.0\" font-size=\"7pt\" >Memory: 2.0 hours</tspan><tspan y=\"50.0\" x=\"324.0\" font-size=\"7pt\" >Horizon: 1.0 hours</tspan><tspan y=\"60.0\" x=\"324.0\" font-size=\"7pt\" >Lagged targets allowed</tspan></text></svg></div>\n",
                            "            </div>\n",
                            "\n",
                            "            <div style='margin-top: 15px;'>\n",
                            "            <div style='margin-bottom: 10px; font-size: 1rem;'>staging</div>\n",
                            "            <style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">data frames</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">staging table              </th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">population</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">POPULATION__STAGING_TABLE_1</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>1</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">data_full</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">DATA_FULL__STAGING_TABLE_2</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>\n",
                            "            </div>\n",
                            "            \n",
                            "<span style='font-size: 1.2rem; font-weight: 500;'>container</span>\n",
                            "<div style='margin-top: 15px;'>\n",
                            "<div style='float: left; margin-right: 50px;'>\n",
                            "<div style='margin-bottom: 10px; font-size: 1rem;'>population</div>\n",
                            "    <style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">subset</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">name     </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">rows   </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">type</th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">test</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">data_full</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">unknown</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">View</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>1</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">train</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">data_full</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">unknown</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">View</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>\n",
                            "</div>\n",
                            "<div style='float: left;'>\n",
                            "<div style='margin-bottom: 10px; font-size: 1rem;'>peripheral</div>\n",
                            "    <style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">name     </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> rows</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">type     </th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">data_full</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"int\">50400</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">DataFrame</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>\n",
                            "</div>\n",
                            "</div>"
                        ],
                        "text/plain": [
                            "data model\n",
                            "\n",
                            "  population:\n",
                            "    columns:\n",
                            "    - y: target\n",
                            "    - ds: time_stamp\n",
                            "\n",
                            "    joins:\n",
                            "    - right: 'data_full'\n",
                            "      time_stamps: (population.ds, data_full.ds)\n",
                            "      relationship: 'many-to-many'\n",
                            "      memory: 7200.0\n",
                            "      horizon: 3600.0\n",
                            "      lagged_targets: True\n",
                            "\n",
                            "  data_full:\n",
                            "    columns:\n",
                            "    - y: target\n",
                            "    - ds: time_stamp\n",
                            "\n",
                            "\n",
                            "container\n",
                            "\n",
                            "  population\n",
                            "      subset   name        rows      type\n",
                            "  0   test     data_full   unknown   View\n",
                            "  1   train    data_full   unknown   View\n",
                            "\n",
                            "  peripheral\n",
                            "      name         rows   type     \n",
                            "  0   data_full   50400   DataFrame"
                        ]
                    },
                    "execution_count": 13,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "# 1. The horizon is 1 hour (we predict the traffic volume in one hour).\n",
                "# 2. The memory is 2 hours, so we allow the algorithm to\n",
                "#    use information from up to 2 hours ago.\n",
                "# 3. We allow lagged targets. Thus, the algorithm can\n",
                "#    identify autoregressive processes.\n",
                "\n",
                "time_series = getml.data.TimeSeries(\n",
                "    population=data_full,\n",
                "    alias=\"population\",\n",
                "    split=split,\n",
                "    time_stamps=\"ds\",\n",
                "    horizon=getml.data.time.hours(1),\n",
                "    memory=getml.data.time.hours(2),\n",
                "    lagged_targets=True,\n",
                ")\n",
                "\n",
                "time_series"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "### 2. Predictive modeling\n",
                "\n",
                "We loaded the data, defined the roles, units and the abstract data model. Next, we create a getML pipeline for relational learning."
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "#### 2.1 Propositionalization with getML's FastProp"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 14,
            "metadata": {},
            "outputs": [],
            "source": [
                "seasonal = getml.preprocessors.Seasonal()\n",
                "\n",
                "fast_prop = getml.feature_learning.FastProp(\n",
                "    loss_function=getml.feature_learning.loss_functions.SquareLoss,\n",
                "    num_threads=1,\n",
                ")"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "__Build the pipeline__"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 15,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre>Pipeline(data_model='population',\n",
                            "         feature_learners=['FastProp'],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=['data_full'],\n",
                            "         predictors=[],\n",
                            "         preprocessors=['Seasonal'],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['feature learning', 'fastprop'])</pre>"
                        ],
                        "text/plain": [
                            "Pipeline(data_model='population',\n",
                            "         feature_learners=['FastProp'],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=['data_full'],\n",
                            "         predictors=[],\n",
                            "         preprocessors=['Seasonal'],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['feature learning', 'fastprop'])"
                        ]
                    },
                    "execution_count": 15,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pipe_fp_fl = getml.pipeline.Pipeline(\n",
                "    preprocessors=[seasonal],\n",
                "    feature_learners=[fast_prop],\n",
                "    data_model=time_series.data_model,\n",
                "    tags=[\"feature learning\", \"fastprop\"],\n",
                ")\n",
                "\n",
                "pipe_fp_fl"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 16,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Checking data model<span style=\"color: #808000; text-decoration-color: #808000\">...</span>\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Checking data model\u001b[33m...\u001b[0m\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Preprocessing... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Checking... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:01\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">OK.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "OK.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "pipe_fp_fl.check(time_series.train)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 17,
            "metadata": {},
            "outputs": [],
            "source": [
                "benchmark = Benchmark()"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 18,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Checking data model<span style=\"color: #808000; text-decoration-color: #808000\">...</span>\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Checking data model\u001b[33m...\u001b[0m\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Preprocessing... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">OK.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "OK.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Preprocessing... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  FastProp: Trying 526 features... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:08\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Trained pipeline.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Trained pipeline.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Time taken: 0:00:08.574815.\n",
                        "\n",
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Preprocessing... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  FastProp: Building features... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:03\n",
                        "\u001b[?25h"
                    ]
                }
            ],
            "source": [
                "with benchmark(\"fastprop\"):\n",
                "    pipe_fp_fl.fit(time_series.train)\n",
                "    fastprop_train = pipe_fp_fl.transform(time_series.train, df_name=\"fastprop_train\")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 19,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Preprocessing... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  FastProp: Building features... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:01\n",
                        "\u001b[?25h"
                    ]
                }
            ],
            "source": [
                "fastprop_test = pipe_fp_fl.transform(time_series.test, df_name=\"fastprop_test\")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 20,
            "metadata": {},
            "outputs": [],
            "source": [
                "predictor = getml.predictors.XGBoostRegressor()\n",
                "\n",
                "pipe_fp_pr = getml.pipeline.Pipeline(\n",
                "    tags=[\"prediction\", \"fastprop\"], predictors=[predictor]\n",
                ")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 21,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Checking data model<span style=\"color: #808000; text-decoration-color: #808000\">...</span>\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Checking data model\u001b[33m...\u001b[0m\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Checking... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">OK.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "OK.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  XGBoost: Training as predictor... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:09\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Trained pipeline.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Trained pipeline.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Time taken: 0:00:09.859138.\n",
                        "\n"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre>Pipeline(data_model='population',\n",
                            "         feature_learners=[],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=[],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['prediction', 'fastprop'])</pre>"
                        ],
                        "text/plain": [
                            "Pipeline(data_model='population',\n",
                            "         feature_learners=[],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=[],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['prediction', 'fastprop'])"
                        ]
                    },
                    "execution_count": 21,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pipe_fp_pr.fit(fastprop_train)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 22,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Preprocessing... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"datetime\">date time          </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">set used      </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">target</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">    mae</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">   rmse</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">rsquared</th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"datetime\">2024-09-13 13:03:14</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">fastprop_train</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">y</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">5.4188</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">7.5347</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.699</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>1</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"datetime\">2024-09-13 13:03:15</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">fastprop_test</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">y</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">5.6151</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">7.8243</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.6747</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>"
                        ],
                        "text/plain": [
                            "    date time             set used         target       mae      rmse   rsquared\n",
                            "0   2024-09-13 13:03:14   fastprop_train   y         5.4188    7.5347     0.699 \n",
                            "1   2024-09-13 13:03:15   fastprop_test    y         5.6151    7.8243     0.6747"
                        ]
                    },
                    "execution_count": 22,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pipe_fp_pr.score(fastprop_test)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "#### 2.2 Propositionalization with featuretools"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 23,
            "metadata": {},
            "outputs": [],
            "source": [
                "data_train = time_series.train.population.to_df(\"data_train\")\n",
                "data_test = time_series.test.population.to_df(\"data_test\")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 24,
            "metadata": {},
            "outputs": [],
            "source": [
                "dfs_pandas = {}\n",
                "\n",
                "for df in getml.project.data_frames:\n",
                "    dfs_pandas[df.name] = df.to_pandas()\n",
                "    dfs_pandas[df.name][\"id\"] = 1"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 25,
            "metadata": {},
            "outputs": [],
            "source": [
                "ft_builder = FTTimeSeriesBuilder(\n",
                "    num_features=200,\n",
                "    horizon=pd.Timedelta(hours=1),\n",
                "    memory=pd.Timedelta(hours=2),\n",
                "    column_id=\"id\",\n",
                "    time_stamp=\"ds\",\n",
                "    target=\"y\",\n",
                "    allow_lagged_targets=True,\n",
                ")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 26,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "featuretools: Trying features...\n",
                        "Selecting the best out of 118 features...\n",
                        "Time taken: 0h:8m:51.202135\n",
                        "\n"
                    ]
                }
            ],
            "source": [
                "with benchmark(\"featuretools\"):\n",
                "    featuretools_train = ft_builder.fit(dfs_pandas[\"data_train\"])\n",
                "\n",
                "featuretools_test = ft_builder.transform(dfs_pandas[\"data_test\"])"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 27,
            "metadata": {},
            "outputs": [],
            "source": [
                "df_featuretools_train = getml.data.DataFrame.from_pandas(\n",
                "    featuretools_train, name=\"featuretools_train\", roles=data_train.roles\n",
                ")\n",
                "\n",
                "df_featuretools_test = getml.data.DataFrame.from_pandas(\n",
                "    featuretools_test, name=\"featuretools_test\", roles=data_train.roles\n",
                ")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 28,
            "metadata": {},
            "outputs": [],
            "source": [
                "df_featuretools_train.set_role(\n",
                "    df_featuretools_train.roles.unused, getml.data.roles.numerical\n",
                ")\n",
                "\n",
                "df_featuretools_test.set_role(\n",
                "    df_featuretools_test.roles.unused, getml.data.roles.numerical\n",
                ")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 29,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre>Pipeline(data_model='population',\n",
                            "         feature_learners=[],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=[],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['prediction', 'featuretools'])</pre>"
                        ],
                        "text/plain": [
                            "Pipeline(data_model='population',\n",
                            "         feature_learners=[],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=[],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['prediction', 'featuretools'])"
                        ]
                    },
                    "execution_count": 29,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "predictor = getml.predictors.XGBoostRegressor()\n",
                "\n",
                "pipe_ft_pr = getml.pipeline.Pipeline(\n",
                "    tags=[\"prediction\", \"featuretools\"], predictors=[predictor]\n",
                ")\n",
                "\n",
                "pipe_ft_pr"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 30,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Checking data model<span style=\"color: #808000; text-decoration-color: #808000\">...</span>\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Checking data model\u001b[33m...\u001b[0m\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Checking... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">The pipeline check generated <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span> issues labeled INFO and <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1</span> issues labeled WARNING.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "The pipeline check generated \u001b[1;36m0\u001b[0m issues labeled INFO and \u001b[1;36m1\u001b[0m issues labeled WARNING.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "data": {
                        "text/html": [
                            "<style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">type   </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">label                  </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">message                         </th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">WARNING</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">COLUMN SHOULD BE UNUSED</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">All non-NULL entries in column &#x27;id&#x27; in POPULATION__STAGING_TABLE_1 are equal to each other. You should consider setting its role to unused_float or using it for comparison only (you can do the latter by setting a unit that contains &#x27;comparison only&#x27;).</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>"
                        ],
                        "text/plain": [
                            "    type      label                     message                         \n",
                            "0   WARNING   COLUMN SHOULD BE UNUSED   All non-NULL entries in column '..."
                        ]
                    },
                    "execution_count": 30,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pipe_ft_pr.check(df_featuretools_train)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 31,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Checking data model<span style=\"color: #808000; text-decoration-color: #808000\">...</span>\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Checking data model\u001b[33m...\u001b[0m\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">The pipeline check generated <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span> issues labeled INFO and <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1</span> issues labeled WARNING.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "The pipeline check generated \u001b[1;36m0\u001b[0m issues labeled INFO and \u001b[1;36m1\u001b[0m issues labeled WARNING.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">To see the issues in full, run <span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">.check</span><span style=\"font-weight: bold\">()</span> on the pipeline.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "To see the issues in full, run \u001b[1;35m.check\u001b[0m\u001b[1m(\u001b[0m\u001b[1m)\u001b[0m on the pipeline.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  XGBoost: Training as predictor... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:03\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Trained pipeline.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Trained pipeline.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Time taken: 0:00:03.563567.\n",
                        "\n"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre>Pipeline(data_model='population',\n",
                            "         feature_learners=[],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=[],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['prediction', 'featuretools'])</pre>"
                        ],
                        "text/plain": [
                            "Pipeline(data_model='population',\n",
                            "         feature_learners=[],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=[],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['prediction', 'featuretools'])"
                        ]
                    },
                    "execution_count": 31,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pipe_ft_pr.fit(df_featuretools_train)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 32,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Preprocessing... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"datetime\">date time          </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">set used          </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">target</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">    mae</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">   rmse</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">rsquared</th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"datetime\">2024-09-13 13:15:05</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">featuretools_train</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">y</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">5.4482</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">7.568</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.6962</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>1</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"datetime\">2024-09-13 13:15:06</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">featuretools_test</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">y</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">6.0863</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">8.5009</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.6498</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>"
                        ],
                        "text/plain": [
                            "    date time             set used             target       mae      rmse   rsquared\n",
                            "0   2024-09-13 13:15:05   featuretools_train   y         5.4482    7.568      0.6962\n",
                            "1   2024-09-13 13:15:06   featuretools_test    y         6.0863    8.5009     0.6498"
                        ]
                    },
                    "execution_count": 32,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pipe_ft_pr.score(df_featuretools_test)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "#### 2.3 Propositionalization with tsfresh"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 33,
            "metadata": {},
            "outputs": [],
            "source": [
                "tsfresh_builder = TSFreshBuilder(\n",
                "    num_features=200,\n",
                "    horizon=20,\n",
                "    memory=60,\n",
                "    column_id=\"id\",\n",
                "    time_stamp=\"ds\",\n",
                "    target=\"y\",\n",
                "    allow_lagged_targets=True,\n",
                ")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 34,
            "metadata": {},
            "outputs": [
                {
                    "name": "stderr",
                    "output_type": "stream",
                    "text": [
                        "Rolling: 100%|██████████| 40/40 [00:12<00:00,  3.18it/s]\n",
                        "Feature Extraction: 100%|██████████| 40/40 [00:06<00:00,  6.26it/s]\n",
                        "Feature Extraction: 100%|██████████| 40/40 [00:06<00:00,  6.23it/s]\n"
                    ]
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Selecting the best out of 13 features...\n",
                        "Time taken: 0h:0m:31.919565\n",
                        "\n"
                    ]
                },
                {
                    "name": "stderr",
                    "output_type": "stream",
                    "text": [
                        "Rolling: 100%|██████████| 40/40 [00:03<00:00, 11.31it/s]\n",
                        "Feature Extraction: 100%|██████████| 40/40 [00:01<00:00, 20.24it/s]\n",
                        "Feature Extraction: 100%|██████████| 40/40 [00:02<00:00, 19.32it/s]\n"
                    ]
                }
            ],
            "source": [
                "with benchmark(\"tsfresh\"):\n",
                "    tsfresh_train = tsfresh_builder.fit(dfs_pandas[\"data_train\"])\n",
                "\n",
                "tsfresh_test = tsfresh_builder.transform(dfs_pandas[\"data_test\"])"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 35,
            "metadata": {},
            "outputs": [],
            "source": [
                "df_tsfresh_train = getml.data.DataFrame.from_pandas(\n",
                "    tsfresh_train, name=\"tsfresh_train\", roles=data_train.roles\n",
                ")\n",
                "\n",
                "df_tsfresh_test = getml.data.DataFrame.from_pandas(\n",
                "    tsfresh_test, name=\"tsfresh_test\", roles=data_train.roles\n",
                ")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 36,
            "metadata": {},
            "outputs": [],
            "source": [
                "df_tsfresh_train.set_role(df_tsfresh_train.roles.unused, getml.data.roles.numerical)\n",
                "\n",
                "df_tsfresh_test.set_role(df_tsfresh_test.roles.unused, getml.data.roles.numerical)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 37,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre>Pipeline(data_model='population',\n",
                            "         feature_learners=[],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=[],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['predicition', 'tsfresh'])</pre>"
                        ],
                        "text/plain": [
                            "Pipeline(data_model='population',\n",
                            "         feature_learners=[],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=[],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['predicition', 'tsfresh'])"
                        ]
                    },
                    "execution_count": 37,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pipe_tsf_pr = getml.pipeline.Pipeline(\n",
                "    tags=[\"predicition\", \"tsfresh\"], predictors=[predictor]\n",
                ")\n",
                "\n",
                "pipe_tsf_pr"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 38,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Checking data model<span style=\"color: #808000; text-decoration-color: #808000\">...</span>\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Checking data model\u001b[33m...\u001b[0m\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Checking... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">The pipeline check generated <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span> issues labeled INFO and <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1</span> issues labeled WARNING.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "The pipeline check generated \u001b[1;36m0\u001b[0m issues labeled INFO and \u001b[1;36m1\u001b[0m issues labeled WARNING.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">To see the issues in full, run <span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">.check</span><span style=\"font-weight: bold\">()</span> on the pipeline.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "To see the issues in full, run \u001b[1;35m.check\u001b[0m\u001b[1m(\u001b[0m\u001b[1m)\u001b[0m on the pipeline.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  XGBoost: Training as predictor... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:01\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K⠇ XGBoost: Trained tree 95.  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━    95% • 00:01"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Trained pipeline.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Trained pipeline.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Time taken: 0:00:01.448143.\n",
                        "\n"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre>Pipeline(data_model='population',\n",
                            "         feature_learners=[],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=[],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['predicition', 'tsfresh'])</pre>"
                        ],
                        "text/plain": [
                            "Pipeline(data_model='population',\n",
                            "         feature_learners=[],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=[],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['predicition', 'tsfresh'])"
                        ]
                    },
                    "execution_count": 38,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pipe_tsf_pr.fit(df_tsfresh_train)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 39,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Preprocessing... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"datetime\">date time          </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">set used     </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">target</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">    mae</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">   rmse</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">rsquared</th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"datetime\">2024-09-13 13:15:49</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">tsfresh_train</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">y</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">6.3146</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">8.2348</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.6418</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>1</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"datetime\">2024-09-13 13:15:49</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">tsfresh_test</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">y</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">6.7886</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">8.9134</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.5778</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>"
                        ],
                        "text/plain": [
                            "    date time             set used        target       mae      rmse   rsquared\n",
                            "0   2024-09-13 13:15:49   tsfresh_train   y         6.3146    8.2348     0.6418\n",
                            "1   2024-09-13 13:15:49   tsfresh_test    y         6.7886    8.9134     0.5778"
                        ]
                    },
                    "execution_count": 39,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pipe_tsf_pr.score(df_tsfresh_test)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "### 3. Comparison"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 40,
            "metadata": {},
            "outputs": [],
            "source": [
                "num_features = dict(\n",
                "    fastprop=526,\n",
                "    featuretools=59,\n",
                "    tsfresh=12,\n",
                ")\n",
                "\n",
                "runtime_per_feature = [\n",
                "    benchmark.runtimes[\"fastprop\"] / num_features[\"fastprop\"],\n",
                "    benchmark.runtimes[\"featuretools\"] / num_features[\"featuretools\"],\n",
                "    benchmark.runtimes[\"tsfresh\"] / num_features[\"tsfresh\"],\n",
                "]\n",
                "\n",
                "features_per_second = [1.0 / r.total_seconds() for r in runtime_per_feature]\n",
                "\n",
                "normalized_runtime_per_feature = [\n",
                "    r / runtime_per_feature[0] for r in runtime_per_feature\n",
                "]\n",
                "\n",
                "comparison = pd.DataFrame(\n",
                "    dict(\n",
                "        runtime=[\n",
                "            benchmark.runtimes[\"fastprop\"],\n",
                "            benchmark.runtimes[\"featuretools\"],\n",
                "            benchmark.runtimes[\"tsfresh\"],\n",
                "        ],\n",
                "        num_features=num_features.values(),\n",
                "        features_per_second=features_per_second,\n",
                "        normalized_runtime=[\n",
                "            1,\n",
                "            benchmark.runtimes[\"featuretools\"] / benchmark.runtimes[\"fastprop\"],\n",
                "            benchmark.runtimes[\"tsfresh\"] / benchmark.runtimes[\"fastprop\"],\n",
                "        ],\n",
                "        normalized_runtime_per_feature=normalized_runtime_per_feature,\n",
                "        rsquared=[pipe_fp_pr.rsquared, pipe_ft_pr.rsquared, pipe_tsf_pr.rsquared],\n",
                "        rmse=[pipe_fp_pr.rmse, pipe_ft_pr.rmse, pipe_tsf_pr.rmse],\n",
                "        mae=[pipe_fp_pr.mae, pipe_ft_pr.mae, pipe_tsf_pr.mae],\n",
                "    )\n",
                ")\n",
                "\n",
                "comparison.index = [\"getML: FastProp\", \"featuretools\", \"tsfresh\"]"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 41,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<div>\n",
                            "<style scoped>\n",
                            "    .dataframe tbody tr th:only-of-type {\n",
                            "        vertical-align: middle;\n",
                            "    }\n",
                            "\n",
                            "    .dataframe tbody tr th {\n",
                            "        vertical-align: top;\n",
                            "    }\n",
                            "\n",
                            "    .dataframe thead th {\n",
                            "        text-align: right;\n",
                            "    }\n",
                            "</style>\n",
                            "<table border=\"1\" class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr style=\"text-align: right;\">\n",
                            "      <th></th>\n",
                            "      <th>runtime</th>\n",
                            "      <th>num_features</th>\n",
                            "      <th>features_per_second</th>\n",
                            "      <th>normalized_runtime</th>\n",
                            "      <th>normalized_runtime_per_feature</th>\n",
                            "      <th>rsquared</th>\n",
                            "      <th>rmse</th>\n",
                            "      <th>mae</th>\n",
                            "    </tr>\n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    <tr>\n",
                            "      <th>getML: FastProp</th>\n",
                            "      <td>0 days 00:00:12.106112</td>\n",
                            "      <td>526</td>\n",
                            "      <td>43.449924</td>\n",
                            "      <td>1.000000</td>\n",
                            "      <td>1.000000</td>\n",
                            "      <td>0.674740</td>\n",
                            "      <td>7.824265</td>\n",
                            "      <td>5.615102</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>featuretools</th>\n",
                            "      <td>0 days 00:08:51.202688</td>\n",
                            "      <td>59</td>\n",
                            "      <td>0.111069</td>\n",
                            "      <td>43.878884</td>\n",
                            "      <td>391.198566</td>\n",
                            "      <td>0.649768</td>\n",
                            "      <td>8.500887</td>\n",
                            "      <td>6.086277</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>tsfresh</th>\n",
                            "      <td>0 days 00:00:31.919755</td>\n",
                            "      <td>12</td>\n",
                            "      <td>0.375943</td>\n",
                            "      <td>2.636664</td>\n",
                            "      <td>115.575929</td>\n",
                            "      <td>0.577811</td>\n",
                            "      <td>8.913408</td>\n",
                            "      <td>6.788610</td>\n",
                            "    </tr>\n",
                            "  </tbody>\n",
                            "</table>\n",
                            "</div>"
                        ],
                        "text/plain": [
                            "                               runtime  num_features  features_per_second  \\\n",
                            "getML: FastProp 0 days 00:00:12.106112           526            43.449924   \n",
                            "featuretools    0 days 00:08:51.202688            59             0.111069   \n",
                            "tsfresh         0 days 00:00:31.919755            12             0.375943   \n",
                            "\n",
                            "                 normalized_runtime  normalized_runtime_per_feature  rsquared  \\\n",
                            "getML: FastProp            1.000000                        1.000000  0.674740   \n",
                            "featuretools              43.878884                      391.198566  0.649768   \n",
                            "tsfresh                    2.636664                      115.575929  0.577811   \n",
                            "\n",
                            "                     rmse       mae  \n",
                            "getML: FastProp  7.824265  5.615102  \n",
                            "featuretools     8.500887  6.086277  \n",
                            "tsfresh          8.913408  6.788610  "
                        ]
                    },
                    "execution_count": 41,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "comparison"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 42,
            "metadata": {},
            "outputs": [],
            "source": [
                "# export for further use\n",
                "comparison.to_csv(\"comparisons/dodgers.csv\")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "getml.engine.shutdown()"
            ]
        }
    ],
    "metadata": {
        "jupytext": {
            "encoding": "# -*- coding: utf-8 -*-",
            "formats": "ipynb,py:percent,md"
        },
        "kernelspec": {
            "display_name": "Python 3 (ipykernel)",
            "language": "python",
            "name": "python3"
        },
        "language_info": {
            "codemirror_mode": {
                "name": "ipython",
                "version": 3
            },
            "file_extension": ".py",
            "mimetype": "text/x-python",
            "name": "python",
            "nbconvert_exporter": "python",
            "pygments_lexer": "ipython3",
            "version": "3.11.9"
        },
        "toc": {
            "base_numbering": 1,
            "nav_menu": {},
            "number_sections": false,
            "sideBar": true,
            "skip_h1_title": false,
            "title_cell": "Table of Contents",
            "title_sidebar": "Contents",
            "toc_cell": false,
            "toc_position": {},
            "toc_section_display": true,
            "toc_window_display": true
        }
    },
    "nbformat": 4,
    "nbformat_minor": 4
}