air_pollution.ipynb

{
    "cells": [
        {
            "cell_type": "markdown",
            "metadata": {
                "tags": [
                    "hide_input"
                ]
            },
            "source": [
                "# Air Pollution - Why feature learning is better than simple propositionalization\n",
                "\n",
                "In this notebook we will compare getML to featuretools and tsfresh, both of which open-source libraries for feature engineering. We find that advanced algorithms featured in getML yield significantly better predictions on this dataset. We then discuss why that is.\n",
                "\n",
                "Summary:\n",
                "\n",
                "- Prediction type: __Regression model__\n",
                "- Domain: __Air pollution__\n",
                "- Prediction target: __pm 2.5 concentration__\n",
                "- Source data: __Multivariate time series__\n",
                "- Population size: __41757__"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## Background\n",
                "\n",
                "Many data scientists and AutoML tools use propositionalization methods for feature engineering. These propositionalization methods usually work as follows:\n",
                "\n",
                "- Generate a large number of hard-coded features,\n",
                "- Use feature selection to pick a percentage of these features.\n",
                "\n",
                "By contrast, getML contains approaches for feature learning: Feature learning adapts machine learning approaches such as decision trees or gradient boosting to the problem of extracting features from relational data and time series.\n",
                "\n",
                "In this notebook, we will benchmark getML against [featuretools](https://www.featuretools.com/) and [tsfresh](https://tsfresh.readthedocs.io/en/latest/). Both of these libaries use propositionalization approaches for feature engineering.\n",
                "\n",
                "As our example dataset, we use a publicly available dataset on air pollution in Beijing, China: [Beijing PM2.5 Data](https://archive.ics.uci.edu/dataset/381/beijing+pm2+5+data). The data set has been originally used in the following study:\n",
                "> Liang, X., Zou, T., Guo, B., Li, S., Zhang, H., Zhang, S., Huang, H. and Chen, S. X. (2015). Assessing Beijing's PM2.5 pollution: severity, weather impact, APEC and winter heating. Proceedings of the Royal Society A, 471, 20150257.\n",
                "\n",
                "We find that getML significantly outperforms featuretools and tsfresh in terms of predictive accuracy ( [see Discussion](#3.-Discussion) ). Our findings indicate that getML's feature learning algorithms are better at adapting to data sets and are also more scalable due to their lower memory requirement."
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## Analysis"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "1. [Loading data](#1.-Loading-data)\n",
                "2. [Predictive modeling](#2.-Predictive-modeling)\n",
                "3. [Discussion](#3.-Discussion)\n",
                "4. [Conclusion](#4.-Conclusion)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "We start the analysis with the setup of our session."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 52,
            "metadata": {},
            "outputs": [],
            "source": [
                "%pip install -q \"getml==1.5.0\" \"featuretools==1.31.0\" \"tsfresh==0.20.3\" \"ipywidgets==8.1.5\""
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 54,
            "metadata": {
                "lines_to_next_cell": 2
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "getML API version: 1.5.0\n",
                        "\n"
                    ]
                }
            ],
            "source": [
                "import os\n",
                "import pandas as pd\n",
                "\n",
                "import getml\n",
                "\n",
                "from utils.load import load_or_retrieve\n",
                "\n",
                "os.environ[\"PYARROW_IGNORE_TIMEZONE\"] = \"1\"\n",
                "\n",
                "print(f\"getML API version: {getml.__version__}\\n\")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 7,
            "metadata": {},
            "outputs": [],
            "source": [
                "# NOTE: Due to featuretools's and tsfresh's substantial resource requirements, prepared data can be used via RUN_FEATURETOOLS or RUN_TSFRESH.\n",
                "\n",
                "RUN_FEATURETOOLS = False\n",
                "RUN_TSFRESH = False\n",
                "\n",
                "if RUN_FEATURETOOLS:\n",
                "    from utils import FTTimeSeriesBuilder\n",
                "\n",
                "if RUN_TSFRESH:\n",
                "    from utils import TSFreshBuilder"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 56,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Launching ./getML --allow-push-notifications=true --allow-remote-ips=true --home-directory=/home/user --in-memory=true --install=false --launch-browser=true --log=false --token=token in /home/user/.getML/getml-1.5.0-x64-linux...\n",
                        "Launched the getML Engine. The log output will be stored in /home/user/.getML/logs/20240912133326.log.\n",
                        "\u001b[2K  Loading pipelines... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Connected to project <span style=\"color: #008000; text-decoration-color: #008000\">'air_pollution'</span>.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Connected to project \u001b[32m'air_pollution'\u001b[0m.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "getml.engine.launch(allow_remote_ips=True, token='token')\n",
                "getml.set_project(\"air_pollution\")"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "### 1. Loading data"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "#### 1.1 Download from source\n",
                "\n",
                "Downloading the raw data from the  UCI Machine Learning Repository into a prediction ready format takes time. To get to the getML model building as fast as possible, we prepared the data for you and excluded the code from this notebook."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 9,
            "metadata": {},
            "outputs": [],
            "source": [
                "data = getml.datasets.load_air_pollution()"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "### 2. Predictive modeling\n",
                "\n",
                "\n",
                "#### 2.1 Pipeline 1: Complex features, 7 days"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "First, we spilt our data. We introduce a [simple, time-based split](https://getml.com/latest/reference/data/split/#getml.data.split.time.time) and use all data until 2013-12-31 for training and everything starting from 2014-01-01 for testing."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 10,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th.sub-header {\n",
                            "    font-weight: normal;\n",
                            "    font-style: italic;\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right !important;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.numerical {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.numerical {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "\n",
                            "  td.char-align {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  span.left {\n",
                            "    text-align: right;\n",
                            "    width: 3em;\n",
                            "  }\n",
                            "  span.right {\n",
                            "    float: right;\n",
                            "    text-align: left;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th>     </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th>     </th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "        \n",
                            "          \n",
                            "            <td>train</td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>1</th>\n",
                            "        \n",
                            "          \n",
                            "            <td>train</td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>2</th>\n",
                            "        \n",
                            "          \n",
                            "            <td>train</td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>3</th>\n",
                            "        \n",
                            "          \n",
                            "            <td>train</td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>4</th>\n",
                            "        \n",
                            "          \n",
                            "            <td>train</td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th></th>\n",
                            "        \n",
                            "          \n",
                            "            <td>...</td>\n",
                            "          \n",
                            "        \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>\n",
                            "\n",
                            "  <p>\n",
                            "    41757 rows<br />\n",
                            "    \n",
                            "    type: StringColumnView<br />\n",
                            "    \n",
                            "  </p>\n"
                        ],
                        "text/plain": [
                            "             \n",
                            "    0   train\n",
                            "    1   train\n",
                            "    2   train\n",
                            "    3   train\n",
                            "    4   train\n",
                            "        ...  \n",
                            "\n",
                            "\n",
                            "41757 rows\n",
                            "type: StringColumnView"
                        ]
                    },
                    "execution_count": 10,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "split = getml.data.split.time(\n",
                "    population=data, time_stamp=\"date\", test=getml.data.time.datetime(2014, 1, 1)\n",
                ")\n",
                "\n",
                "split"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "For our first experiment, we will learn complex features and allow a memory of up to seven days. That means at every given point in time, the algorithm is allowed to back seven days into the past."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 11,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<span style='font-size: 1.2rem; font-weight: 500;'>data model</span>\n",
                            "            <div style='margin-top: 15px; margin-bottom: 5px;'>\n",
                            "            <div style='margin-bottom: 10px; font-size: 1rem;'>diagram</div>\n",
                            "            <div style=\"height:100px;width:660px;position:relative;\"><svg height=\"90\" width=\"650\"><rect y=\"0\" x=\"0\" rx=\"10\" ry=\"10\" width=\"150\" height=\"90\" style=\"fill:#6829c2;stroke-width:0;\" /><text y=\"73.8\" x=\"75.0\" dominant-baseline=\"middle\" text-anchor=\"middle\" fill=\"white\">population</text><rect x=\"51\" y=\"10\" rx=\"4\" ry=\"4\" width=\"48\" height=\"48\" style=\" fill:#6829c2;stroke:#ffffff;stroke-width:3;\" /><line x1=\"67.0\" y1=\"10\" x2=\"67.0\" y2=\"58\" style=\"stroke:white;stroke-width:3\" /><line x1=\"83.0\" y1=\"10\" x2=\"83.0\" y2=\"58\" style=\"stroke:white;stroke-width:3\" /><line x1=\"51\" y1=\"26.0\" x2=\"99\" y2=\"26.0\" style=\"stroke:white;stroke-width:3\" /><line x1=\"51\" y1=\"42.0\" x2=\"99\" y2=\"42.0\" style=\"stroke:white;stroke-width:3\" /><rect y=\"0\" x=\"500\" rx=\"10\" ry=\"10\" width=\"150\" height=\"90\" style=\"fill:#6829c2;stroke-width:0;\" /><text y=\"73.8\" x=\"575.0\" dominant-baseline=\"middle\" text-anchor=\"middle\" fill=\"white\">population</text><rect x=\"551\" y=\"10\" rx=\"4\" ry=\"4\" width=\"48\" height=\"48\" style=\" fill:#6829c2;stroke:#ffffff;stroke-width:3;\" /><line x1=\"567.0\" y1=\"10\" x2=\"567.0\" y2=\"58\" style=\"stroke:white;stroke-width:3\" /><line x1=\"583.0\" y1=\"10\" x2=\"583.0\" y2=\"58\" style=\"stroke:white;stroke-width:3\" /><line x1=\"551\" y1=\"26.0\" x2=\"599\" y2=\"26.0\" style=\"stroke:white;stroke-width:3\" /><line x1=\"551\" y1=\"42.0\" x2=\"599\" y2=\"42.0\" style=\"stroke:white;stroke-width:3\" /><line x1=\"150\" y1=\"43.0\" x2=\"490\" y2=\"43.0\" style=\"stroke:#808080;;stroke-width:4\" /><polygon points=\"500, 43.0 490, 37.0 490, 49.0 \" style=\"fill:#808080;;stroke-width:0;\" /><rect y=\"10.0\" x=\"249.0\" rx=\"10\" ry=\"10\" width=\"150\" height=\"70\" style=\"fill:#6829c2;stroke-width:0;\" /><text dominant-baseline=\"middle\" text-anchor=\"middle\" fill=\"white\"><tspan y=\"40.0\" x=\"324.0\" font-size=\"7pt\" >date &lt;= date</tspan><tspan y=\"50.0\" x=\"324.0\" font-size=\"7pt\" >Memory: 7.0 days</tspan></text></svg></div>\n",
                            "            </div>\n",
                            "\n",
                            "            <div style='margin-top: 15px;'>\n",
                            "            <div style='margin-bottom: 10px; font-size: 1rem;'>staging</div>\n",
                            "            <style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">data frames</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">staging table              </th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">population</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">POPULATION__STAGING_TABLE_1</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>1</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">population</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">POPULATION__STAGING_TABLE_2</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>\n",
                            "            </div>\n",
                            "            \n",
                            "<span style='font-size: 1.2rem; font-weight: 500;'>container</span>\n",
                            "<div style='margin-top: 15px;'>\n",
                            "<div style='float: left; margin-right: 50px;'>\n",
                            "<div style='margin-bottom: 10px; font-size: 1rem;'>population</div>\n",
                            "    <style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">subset</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">name      </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> rows</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">type</th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">test</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">population</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"int\">8661</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">View</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>1</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">train</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">population</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"int\">33096</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">View</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>\n",
                            "</div>\n",
                            "<div style='float: left;'>\n",
                            "<div style='margin-bottom: 10px; font-size: 1rem;'>peripheral</div>\n",
                            "    <style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">name      </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> rows</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">type     </th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">population</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"int\">41757</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">DataFrame</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>\n",
                            "</div>\n",
                            "</div>"
                        ],
                        "text/plain": [
                            "data model\n",
                            "\n",
                            "  population:\n",
                            "    columns:\n",
                            "    - DEWP: numerical\n",
                            "    - TEMP: numerical\n",
                            "    - PRES: numerical\n",
                            "    - Iws: numerical\n",
                            "    - Is: numerical\n",
                            "    - ...\n",
                            "\n",
                            "    joins:\n",
                            "    - right: 'population'\n",
                            "      time_stamps: (population.date, population.date)\n",
                            "      relationship: 'many-to-many'\n",
                            "      memory: 604800.0\n",
                            "      lagged_targets: False\n",
                            "\n",
                            "  population:\n",
                            "    columns:\n",
                            "    - DEWP: numerical\n",
                            "    - TEMP: numerical\n",
                            "    - PRES: numerical\n",
                            "    - Iws: numerical\n",
                            "    - Is: numerical\n",
                            "    - ...\n",
                            "\n",
                            "\n",
                            "container\n",
                            "\n",
                            "  population\n",
                            "      subset   name          rows   type\n",
                            "  0   test     population    8661   View\n",
                            "  1   train    population   33096   View\n",
                            "\n",
                            "  peripheral\n",
                            "      name          rows   type     \n",
                            "  0   population   41757   DataFrame"
                        ]
                    },
                    "execution_count": 11,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "time_series1 = getml.data.TimeSeries(\n",
                "    population=data,\n",
                "    alias=\"population\",\n",
                "    split=split,\n",
                "    time_stamps=\"date\",\n",
                "    memory=getml.data.time.days(7),\n",
                ")\n",
                "\n",
                "time_series1"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 12,
            "metadata": {
                "tags": []
            },
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre>Pipeline(data_model='population',\n",
                            "         feature_learners=['RelMT'],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=['population'],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['getML: RelMT', 'memory: 7d', 'complex features'])</pre>"
                        ],
                        "text/plain": [
                            "Pipeline(data_model='population',\n",
                            "         feature_learners=['RelMT'],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=['population'],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['getML: RelMT', 'memory: 7d', 'complex features'])"
                        ]
                    },
                    "execution_count": 12,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "relmt = getml.feature_learning.RelMT(\n",
                "    num_features=10,\n",
                "    loss_function=getml.feature_learning.loss_functions.SquareLoss,\n",
                "    seed=4367,\n",
                "    num_threads=1,\n",
                ")\n",
                "\n",
                "predictor = getml.predictors.XGBoostRegressor(n_jobs=1)\n",
                "\n",
                "pipe1 = getml.pipeline.Pipeline(\n",
                "    tags=[\"getML: RelMT\", \"memory: 7d\", \"complex features\"],\n",
                "    data_model=time_series1.data_model,\n",
                "    feature_learners=[relmt],\n",
                "    predictors=[predictor],\n",
                ")\n",
                "\n",
                "pipe1"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "It is good practice to always check your data model first, even though `check(...)` is also called by `fit(...)`. That enables us to make last-minute changes."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 13,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Checking data model<span style=\"color: #808000; text-decoration-color: #808000\">...</span>\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Checking data model\u001b[33m...\u001b[0m\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Checking... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:01\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">OK.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "OK.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "pipe1.check(time_series1.train)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "We now fit our data on the training set and evaluate our findings, both, in-sample and out-of-sample."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 14,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Checking data model<span style=\"color: #808000; text-decoration-color: #808000\">...</span>\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Checking data model\u001b[33m...\u001b[0m\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">OK.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "OK.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  RelMT: Training features... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 01:53\n",
                        "\u001b[2K  RelMT: Building features... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:18\n",
                        "\u001b[2K  XGBoost: Training as predictor... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:01\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Trained pipeline.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Trained pipeline.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Time taken: 0:02:14.495720.\n",
                        "\n"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre>Pipeline(data_model='population',\n",
                            "         feature_learners=['RelMT'],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=['population'],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['getML: RelMT', 'memory: 7d', 'complex features', 'container-m6fGhl'])</pre>"
                        ],
                        "text/plain": [
                            "Pipeline(data_model='population',\n",
                            "         feature_learners=['RelMT'],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=['population'],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['getML: RelMT', 'memory: 7d', 'complex features', 'container-m6fGhl'])"
                        ]
                    },
                    "execution_count": 14,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pipe1.fit(time_series1.train)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 15,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Preprocessing... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  RelMT: Building features... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:05\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"datetime\">date time          </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">set used</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">target</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">     mae</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">    rmse</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">rsquared</th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"datetime\">2024-09-12 12:50:57</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">train</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">pm2.5</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">35.1042</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">50.7378</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.6946</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>1</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"datetime\">2024-09-12 12:51:02</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">test</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">pm2.5</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">39.7981</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">57.6703</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.6272</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>"
                        ],
                        "text/plain": [
                            "    date time             set used   target        mae       rmse   rsquared\n",
                            "0   2024-09-12 12:50:57   train      pm2.5     35.1042    50.7378     0.6946\n",
                            "1   2024-09-12 12:51:02   test       pm2.5     39.7981    57.6703     0.6272"
                        ]
                    },
                    "execution_count": 15,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pipe1.score(time_series1.test)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "#### 2.2 Pipeline 2: Complex features, 1 day"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 16,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<span style='font-size: 1.2rem; font-weight: 500;'>data model</span>\n",
                            "            <div style='margin-top: 15px; margin-bottom: 5px;'>\n",
                            "            <div style='margin-bottom: 10px; font-size: 1rem;'>diagram</div>\n",
                            "            <div style=\"height:100px;width:660px;position:relative;\"><svg height=\"90\" width=\"650\"><rect y=\"0\" x=\"0\" rx=\"10\" ry=\"10\" width=\"150\" height=\"90\" style=\"fill:#6829c2;stroke-width:0;\" /><text y=\"73.8\" x=\"75.0\" dominant-baseline=\"middle\" text-anchor=\"middle\" fill=\"white\">population</text><rect x=\"51\" y=\"10\" rx=\"4\" ry=\"4\" width=\"48\" height=\"48\" style=\" fill:#6829c2;stroke:#ffffff;stroke-width:3;\" /><line x1=\"67.0\" y1=\"10\" x2=\"67.0\" y2=\"58\" style=\"stroke:white;stroke-width:3\" /><line x1=\"83.0\" y1=\"10\" x2=\"83.0\" y2=\"58\" style=\"stroke:white;stroke-width:3\" /><line x1=\"51\" y1=\"26.0\" x2=\"99\" y2=\"26.0\" style=\"stroke:white;stroke-width:3\" /><line x1=\"51\" y1=\"42.0\" x2=\"99\" y2=\"42.0\" style=\"stroke:white;stroke-width:3\" /><rect y=\"0\" x=\"500\" rx=\"10\" ry=\"10\" width=\"150\" height=\"90\" style=\"fill:#6829c2;stroke-width:0;\" /><text y=\"73.8\" x=\"575.0\" dominant-baseline=\"middle\" text-anchor=\"middle\" fill=\"white\">population</text><rect x=\"551\" y=\"10\" rx=\"4\" ry=\"4\" width=\"48\" height=\"48\" style=\" fill:#6829c2;stroke:#ffffff;stroke-width:3;\" /><line x1=\"567.0\" y1=\"10\" x2=\"567.0\" y2=\"58\" style=\"stroke:white;stroke-width:3\" /><line x1=\"583.0\" y1=\"10\" x2=\"583.0\" y2=\"58\" style=\"stroke:white;stroke-width:3\" /><line x1=\"551\" y1=\"26.0\" x2=\"599\" y2=\"26.0\" style=\"stroke:white;stroke-width:3\" /><line x1=\"551\" y1=\"42.0\" x2=\"599\" y2=\"42.0\" style=\"stroke:white;stroke-width:3\" /><line x1=\"150\" y1=\"43.0\" x2=\"490\" y2=\"43.0\" style=\"stroke:#808080;;stroke-width:4\" /><polygon points=\"500, 43.0 490, 37.0 490, 49.0 \" style=\"fill:#808080;;stroke-width:0;\" /><rect y=\"10.0\" x=\"249.0\" rx=\"10\" ry=\"10\" width=\"150\" height=\"70\" style=\"fill:#6829c2;stroke-width:0;\" /><text dominant-baseline=\"middle\" text-anchor=\"middle\" fill=\"white\"><tspan y=\"40.0\" x=\"324.0\" font-size=\"7pt\" >date &lt;= date</tspan><tspan y=\"50.0\" x=\"324.0\" font-size=\"7pt\" >Memory: 1.0 days</tspan></text></svg></div>\n",
                            "            </div>\n",
                            "\n",
                            "            <div style='margin-top: 15px;'>\n",
                            "            <div style='margin-bottom: 10px; font-size: 1rem;'>staging</div>\n",
                            "            <style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">data frames</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">staging table              </th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">population</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">POPULATION__STAGING_TABLE_1</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>1</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">population</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">POPULATION__STAGING_TABLE_2</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>\n",
                            "            </div>\n",
                            "            \n",
                            "<span style='font-size: 1.2rem; font-weight: 500;'>container</span>\n",
                            "<div style='margin-top: 15px;'>\n",
                            "<div style='float: left; margin-right: 50px;'>\n",
                            "<div style='margin-bottom: 10px; font-size: 1rem;'>population</div>\n",
                            "    <style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">subset</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">name      </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> rows</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">type</th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">test</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">population</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"int\">8661</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">View</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>1</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">train</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">population</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"int\">33096</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">View</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>\n",
                            "</div>\n",
                            "<div style='float: left;'>\n",
                            "<div style='margin-bottom: 10px; font-size: 1rem;'>peripheral</div>\n",
                            "    <style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">name      </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> rows</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">type     </th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">population</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"int\">41757</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">DataFrame</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>\n",
                            "</div>\n",
                            "</div>"
                        ],
                        "text/plain": [
                            "data model\n",
                            "\n",
                            "  population:\n",
                            "    columns:\n",
                            "    - DEWP: numerical\n",
                            "    - TEMP: numerical\n",
                            "    - PRES: numerical\n",
                            "    - Iws: numerical\n",
                            "    - Is: numerical\n",
                            "    - ...\n",
                            "\n",
                            "    joins:\n",
                            "    - right: 'population'\n",
                            "      time_stamps: (population.date, population.date)\n",
                            "      relationship: 'many-to-many'\n",
                            "      memory: 86400.0\n",
                            "      lagged_targets: False\n",
                            "\n",
                            "  population:\n",
                            "    columns:\n",
                            "    - DEWP: numerical\n",
                            "    - TEMP: numerical\n",
                            "    - PRES: numerical\n",
                            "    - Iws: numerical\n",
                            "    - Is: numerical\n",
                            "    - ...\n",
                            "\n",
                            "\n",
                            "container\n",
                            "\n",
                            "  population\n",
                            "      subset   name          rows   type\n",
                            "  0   test     population    8661   View\n",
                            "  1   train    population   33096   View\n",
                            "\n",
                            "  peripheral\n",
                            "      name          rows   type     \n",
                            "  0   population   41757   DataFrame"
                        ]
                    },
                    "execution_count": 16,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "time_series2 = getml.data.TimeSeries(\n",
                "    population=data,\n",
                "    alias=\"population\",\n",
                "    split=split,\n",
                "    time_stamps=\"date\",\n",
                "    memory=getml.data.time.days(1),\n",
                ")\n",
                "\n",
                "time_series2"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 17,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre>Pipeline(data_model='population',\n",
                            "         feature_learners=['RelMT'],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=['population'],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['getML: RelMT', 'memory: 1d', 'complex features'])</pre>"
                        ],
                        "text/plain": [
                            "Pipeline(data_model='population',\n",
                            "         feature_learners=['RelMT'],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=['population'],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['getML: RelMT', 'memory: 1d', 'complex features'])"
                        ]
                    },
                    "execution_count": 17,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "relmt = getml.feature_learning.RelMT(\n",
                "    num_features=10,\n",
                "    loss_function=getml.feature_learning.loss_functions.SquareLoss,\n",
                "    seed=4367,\n",
                "    num_threads=1,\n",
                ")\n",
                "\n",
                "predictor = getml.predictors.XGBoostRegressor(n_jobs=1)\n",
                "\n",
                "pipe2 = getml.pipeline.Pipeline(\n",
                "    tags=[\"getML: RelMT\", \"memory: 1d\", \"complex features\"],\n",
                "    data_model=time_series2.data_model,\n",
                "    feature_learners=[relmt],\n",
                "    predictors=[predictor],\n",
                ")\n",
                "\n",
                "pipe2"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 18,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Checking data model<span style=\"color: #808000; text-decoration-color: #808000\">...</span>\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Checking data model\u001b[33m...\u001b[0m\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Checking... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:01\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">OK.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "OK.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "pipe2.check(time_series2.train)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 19,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Checking data model<span style=\"color: #808000; text-decoration-color: #808000\">...</span>\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Checking data model\u001b[33m...\u001b[0m\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">OK.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "OK.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  RelMT: Training features... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:30\n",
                        "\u001b[2K  RelMT: Building features... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:03\n",
                        "\u001b[2K  XGBoost: Training as predictor... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:02\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Trained pipeline.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Trained pipeline.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Time taken: 0:00:36.135924.\n",
                        "\n"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre>Pipeline(data_model='population',\n",
                            "         feature_learners=['RelMT'],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=['population'],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['getML: RelMT', 'memory: 1d', 'complex features', 'container-yXALH8'])</pre>"
                        ],
                        "text/plain": [
                            "Pipeline(data_model='population',\n",
                            "         feature_learners=['RelMT'],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=['population'],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['getML: RelMT', 'memory: 1d', 'complex features', 'container-yXALH8'])"
                        ]
                    },
                    "execution_count": 19,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pipe2.fit(time_series2.train)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 20,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Preprocessing... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  RelMT: Building features... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"datetime\">date time          </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">set used</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">target</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">     mae</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">    rmse</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">rsquared</th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"datetime\">2024-09-12 12:51:40</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">train</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">pm2.5</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">38.1927</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">55.3496</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.6367</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>1</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"datetime\">2024-09-12 12:51:41</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">test</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">pm2.5</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">48.0167</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">67.5441</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.4802</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>"
                        ],
                        "text/plain": [
                            "    date time             set used   target        mae       rmse   rsquared\n",
                            "0   2024-09-12 12:51:40   train      pm2.5     38.1927    55.3496     0.6367\n",
                            "1   2024-09-12 12:51:41   test       pm2.5     48.0167    67.5441     0.4802"
                        ]
                    },
                    "execution_count": 20,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pipe2.score(time_series2.test)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "#### 2.3 Pipeline 3: Simple features, 7 days\n",
                "\n",
                "For our third experiment, we will learn simple features and allow a memory of up to seven days."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 21,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<span style='font-size: 1.2rem; font-weight: 500;'>data model</span>\n",
                            "            <div style='margin-top: 15px; margin-bottom: 5px;'>\n",
                            "            <div style='margin-bottom: 10px; font-size: 1rem;'>diagram</div>\n",
                            "            <div style=\"height:100px;width:660px;position:relative;\"><svg height=\"90\" width=\"650\"><rect y=\"0\" x=\"0\" rx=\"10\" ry=\"10\" width=\"150\" height=\"90\" style=\"fill:#6829c2;stroke-width:0;\" /><text y=\"73.8\" x=\"75.0\" dominant-baseline=\"middle\" text-anchor=\"middle\" fill=\"white\">population</text><rect x=\"51\" y=\"10\" rx=\"4\" ry=\"4\" width=\"48\" height=\"48\" style=\" fill:#6829c2;stroke:#ffffff;stroke-width:3;\" /><line x1=\"67.0\" y1=\"10\" x2=\"67.0\" y2=\"58\" style=\"stroke:white;stroke-width:3\" /><line x1=\"83.0\" y1=\"10\" x2=\"83.0\" y2=\"58\" style=\"stroke:white;stroke-width:3\" /><line x1=\"51\" y1=\"26.0\" x2=\"99\" y2=\"26.0\" style=\"stroke:white;stroke-width:3\" /><line x1=\"51\" y1=\"42.0\" x2=\"99\" y2=\"42.0\" style=\"stroke:white;stroke-width:3\" /><rect y=\"0\" x=\"500\" rx=\"10\" ry=\"10\" width=\"150\" height=\"90\" style=\"fill:#6829c2;stroke-width:0;\" /><text y=\"73.8\" x=\"575.0\" dominant-baseline=\"middle\" text-anchor=\"middle\" fill=\"white\">population</text><rect x=\"551\" y=\"10\" rx=\"4\" ry=\"4\" width=\"48\" height=\"48\" style=\" fill:#6829c2;stroke:#ffffff;stroke-width:3;\" /><line x1=\"567.0\" y1=\"10\" x2=\"567.0\" y2=\"58\" style=\"stroke:white;stroke-width:3\" /><line x1=\"583.0\" y1=\"10\" x2=\"583.0\" y2=\"58\" style=\"stroke:white;stroke-width:3\" /><line x1=\"551\" y1=\"26.0\" x2=\"599\" y2=\"26.0\" style=\"stroke:white;stroke-width:3\" /><line x1=\"551\" y1=\"42.0\" x2=\"599\" y2=\"42.0\" style=\"stroke:white;stroke-width:3\" /><line x1=\"150\" y1=\"43.0\" x2=\"490\" y2=\"43.0\" style=\"stroke:#808080;;stroke-width:4\" /><polygon points=\"500, 43.0 490, 37.0 490, 49.0 \" style=\"fill:#808080;;stroke-width:0;\" /><rect y=\"10.0\" x=\"249.0\" rx=\"10\" ry=\"10\" width=\"150\" height=\"70\" style=\"fill:#6829c2;stroke-width:0;\" /><text dominant-baseline=\"middle\" text-anchor=\"middle\" fill=\"white\"><tspan y=\"40.0\" x=\"324.0\" font-size=\"7pt\" >date &lt;= date</tspan><tspan y=\"50.0\" x=\"324.0\" font-size=\"7pt\" >Memory: 7.0 days</tspan></text></svg></div>\n",
                            "            </div>\n",
                            "\n",
                            "            <div style='margin-top: 15px;'>\n",
                            "            <div style='margin-bottom: 10px; font-size: 1rem;'>staging</div>\n",
                            "            <style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">data frames</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">staging table              </th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">population</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">POPULATION__STAGING_TABLE_1</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>1</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">population</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">POPULATION__STAGING_TABLE_2</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>\n",
                            "            </div>\n",
                            "            \n",
                            "<span style='font-size: 1.2rem; font-weight: 500;'>container</span>\n",
                            "<div style='margin-top: 15px;'>\n",
                            "<div style='float: left; margin-right: 50px;'>\n",
                            "<div style='margin-bottom: 10px; font-size: 1rem;'>population</div>\n",
                            "    <style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">subset</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">name      </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> rows</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">type</th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">test</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">population</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"int\">8661</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">View</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>1</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">train</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">population</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"int\">33096</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">View</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>\n",
                            "</div>\n",
                            "<div style='float: left;'>\n",
                            "<div style='margin-bottom: 10px; font-size: 1rem;'>peripheral</div>\n",
                            "    <style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">name      </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> rows</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">type     </th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">population</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"int\">41757</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">DataFrame</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>\n",
                            "</div>\n",
                            "</div>"
                        ],
                        "text/plain": [
                            "data model\n",
                            "\n",
                            "  population:\n",
                            "    columns:\n",
                            "    - DEWP: numerical\n",
                            "    - TEMP: numerical\n",
                            "    - PRES: numerical\n",
                            "    - Iws: numerical\n",
                            "    - Is: numerical\n",
                            "    - ...\n",
                            "\n",
                            "    joins:\n",
                            "    - right: 'population'\n",
                            "      time_stamps: (population.date, population.date)\n",
                            "      relationship: 'many-to-many'\n",
                            "      memory: 604800.0\n",
                            "      lagged_targets: False\n",
                            "\n",
                            "  population:\n",
                            "    columns:\n",
                            "    - DEWP: numerical\n",
                            "    - TEMP: numerical\n",
                            "    - PRES: numerical\n",
                            "    - Iws: numerical\n",
                            "    - Is: numerical\n",
                            "    - ...\n",
                            "\n",
                            "\n",
                            "container\n",
                            "\n",
                            "  population\n",
                            "      subset   name          rows   type\n",
                            "  0   test     population    8661   View\n",
                            "  1   train    population   33096   View\n",
                            "\n",
                            "  peripheral\n",
                            "      name          rows   type     \n",
                            "  0   population   41757   DataFrame"
                        ]
                    },
                    "execution_count": 21,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "time_series3 = getml.data.TimeSeries(\n",
                "    population=data,\n",
                "    alias=\"population\",\n",
                "    split=split,\n",
                "    time_stamps=\"date\",\n",
                "    memory=getml.data.time.days(7),\n",
                ")\n",
                "\n",
                "time_series3"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 22,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre>Pipeline(data_model='population',\n",
                            "         feature_learners=['FastProp'],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=['population'],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['getML: FastProp', 'memory: 7d', 'simple features'])</pre>"
                        ],
                        "text/plain": [
                            "Pipeline(data_model='population',\n",
                            "         feature_learners=['FastProp'],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=['population'],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['getML: FastProp', 'memory: 7d', 'simple features'])"
                        ]
                    },
                    "execution_count": 22,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "fast_prop = getml.feature_learning.FastProp(\n",
                "    loss_function=getml.feature_learning.loss_functions.SquareLoss,\n",
                "    num_threads=1,\n",
                "    aggregation=getml.feature_learning.FastProp.agg_sets.All,\n",
                ")\n",
                "\n",
                "predictor = getml.predictors.XGBoostRegressor(n_jobs=1)\n",
                "\n",
                "pipe3 = getml.pipeline.Pipeline(\n",
                "    tags=[\"getML: FastProp\", \"memory: 7d\", \"simple features\"],\n",
                "    data_model=time_series3.data_model,\n",
                "    feature_learners=[fast_prop],\n",
                "    predictors=[predictor],\n",
                ")\n",
                "\n",
                "pipe3"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 23,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Checking data model<span style=\"color: #808000; text-decoration-color: #808000\">...</span>\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Checking data model\u001b[33m...\u001b[0m\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Checking... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:01\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">OK.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "OK.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "pipe3.check(time_series3.train)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 24,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Checking data model<span style=\"color: #808000; text-decoration-color: #808000\">...</span>\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Checking data model\u001b[33m...\u001b[0m\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">OK.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "OK.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  FastProp: Trying 378 features... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:28\n",
                        "\u001b[2K  FastProp: Building features... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:16\n",
                        "\u001b[2K  XGBoost: Training as predictor... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:18\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Trained pipeline.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Trained pipeline.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Time taken: 0:01:02.990256.\n",
                        "\n"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre>Pipeline(data_model='population',\n",
                            "         feature_learners=['FastProp'],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=['population'],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['getML: FastProp', 'memory: 7d', 'simple features', 'container-hqJmW2'])</pre>"
                        ],
                        "text/plain": [
                            "Pipeline(data_model='population',\n",
                            "         feature_learners=['FastProp'],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=['population'],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['getML: FastProp', 'memory: 7d', 'simple features', 'container-hqJmW2'])"
                        ]
                    },
                    "execution_count": 24,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pipe3.fit(time_series3.train)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 25,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Preprocessing... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  FastProp: Building features... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:04\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"datetime\">date time          </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">set used</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">target</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">     mae</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">    rmse</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">rsquared</th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"datetime\">2024-09-12 12:52:46</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">train</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">pm2.5</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">35.9677</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">50.7711</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.7036</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>1</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"datetime\">2024-09-12 12:52:51</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">test</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">pm2.5</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">45.4779</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">62.6417</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.5613</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>"
                        ],
                        "text/plain": [
                            "    date time             set used   target        mae       rmse   rsquared\n",
                            "0   2024-09-12 12:52:46   train      pm2.5     35.9677    50.7711     0.7036\n",
                            "1   2024-09-12 12:52:51   test       pm2.5     45.4779    62.6417     0.5613"
                        ]
                    },
                    "execution_count": 25,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pipe3.score(time_series3.test)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "#### 2.4 Pipeline 4: Simple features, 1 day\n",
                "\n",
                "For our fourth experiment, we will learn simple features and allow a memory of up to one day."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 26,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<span style='font-size: 1.2rem; font-weight: 500;'>data model</span>\n",
                            "            <div style='margin-top: 15px; margin-bottom: 5px;'>\n",
                            "            <div style='margin-bottom: 10px; font-size: 1rem;'>diagram</div>\n",
                            "            <div style=\"height:100px;width:660px;position:relative;\"><svg height=\"90\" width=\"650\"><rect y=\"0\" x=\"0\" rx=\"10\" ry=\"10\" width=\"150\" height=\"90\" style=\"fill:#6829c2;stroke-width:0;\" /><text y=\"73.8\" x=\"75.0\" dominant-baseline=\"middle\" text-anchor=\"middle\" fill=\"white\">population</text><rect x=\"51\" y=\"10\" rx=\"4\" ry=\"4\" width=\"48\" height=\"48\" style=\" fill:#6829c2;stroke:#ffffff;stroke-width:3;\" /><line x1=\"67.0\" y1=\"10\" x2=\"67.0\" y2=\"58\" style=\"stroke:white;stroke-width:3\" /><line x1=\"83.0\" y1=\"10\" x2=\"83.0\" y2=\"58\" style=\"stroke:white;stroke-width:3\" /><line x1=\"51\" y1=\"26.0\" x2=\"99\" y2=\"26.0\" style=\"stroke:white;stroke-width:3\" /><line x1=\"51\" y1=\"42.0\" x2=\"99\" y2=\"42.0\" style=\"stroke:white;stroke-width:3\" /><rect y=\"0\" x=\"500\" rx=\"10\" ry=\"10\" width=\"150\" height=\"90\" style=\"fill:#6829c2;stroke-width:0;\" /><text y=\"73.8\" x=\"575.0\" dominant-baseline=\"middle\" text-anchor=\"middle\" fill=\"white\">population</text><rect x=\"551\" y=\"10\" rx=\"4\" ry=\"4\" width=\"48\" height=\"48\" style=\" fill:#6829c2;stroke:#ffffff;stroke-width:3;\" /><line x1=\"567.0\" y1=\"10\" x2=\"567.0\" y2=\"58\" style=\"stroke:white;stroke-width:3\" /><line x1=\"583.0\" y1=\"10\" x2=\"583.0\" y2=\"58\" style=\"stroke:white;stroke-width:3\" /><line x1=\"551\" y1=\"26.0\" x2=\"599\" y2=\"26.0\" style=\"stroke:white;stroke-width:3\" /><line x1=\"551\" y1=\"42.0\" x2=\"599\" y2=\"42.0\" style=\"stroke:white;stroke-width:3\" /><line x1=\"150\" y1=\"43.0\" x2=\"490\" y2=\"43.0\" style=\"stroke:#808080;;stroke-width:4\" /><polygon points=\"500, 43.0 490, 37.0 490, 49.0 \" style=\"fill:#808080;;stroke-width:0;\" /><rect y=\"10.0\" x=\"249.0\" rx=\"10\" ry=\"10\" width=\"150\" height=\"70\" style=\"fill:#6829c2;stroke-width:0;\" /><text dominant-baseline=\"middle\" text-anchor=\"middle\" fill=\"white\"><tspan y=\"40.0\" x=\"324.0\" font-size=\"7pt\" >date &lt;= date</tspan><tspan y=\"50.0\" x=\"324.0\" font-size=\"7pt\" >Memory: 1.0 days</tspan></text></svg></div>\n",
                            "            </div>\n",
                            "\n",
                            "            <div style='margin-top: 15px;'>\n",
                            "            <div style='margin-bottom: 10px; font-size: 1rem;'>staging</div>\n",
                            "            <style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">data frames</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">staging table              </th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">population</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">POPULATION__STAGING_TABLE_1</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>1</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">population</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">POPULATION__STAGING_TABLE_2</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>\n",
                            "            </div>\n",
                            "            \n",
                            "<span style='font-size: 1.2rem; font-weight: 500;'>container</span>\n",
                            "<div style='margin-top: 15px;'>\n",
                            "<div style='float: left; margin-right: 50px;'>\n",
                            "<div style='margin-bottom: 10px; font-size: 1rem;'>population</div>\n",
                            "    <style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">subset</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">name      </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> rows</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">type</th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">test</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">population</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"int\">8661</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">View</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>1</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">train</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">population</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"int\">33096</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">View</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>\n",
                            "</div>\n",
                            "<div style='float: left;'>\n",
                            "<div style='margin-bottom: 10px; font-size: 1rem;'>peripheral</div>\n",
                            "    <style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">name      </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> rows</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">type     </th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">population</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"int\">41757</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">DataFrame</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>\n",
                            "</div>\n",
                            "</div>"
                        ],
                        "text/plain": [
                            "data model\n",
                            "\n",
                            "  population:\n",
                            "    columns:\n",
                            "    - DEWP: numerical\n",
                            "    - TEMP: numerical\n",
                            "    - PRES: numerical\n",
                            "    - Iws: numerical\n",
                            "    - Is: numerical\n",
                            "    - ...\n",
                            "\n",
                            "    joins:\n",
                            "    - right: 'population'\n",
                            "      time_stamps: (population.date, population.date)\n",
                            "      relationship: 'many-to-many'\n",
                            "      memory: 86400.0\n",
                            "      lagged_targets: False\n",
                            "\n",
                            "  population:\n",
                            "    columns:\n",
                            "    - DEWP: numerical\n",
                            "    - TEMP: numerical\n",
                            "    - PRES: numerical\n",
                            "    - Iws: numerical\n",
                            "    - Is: numerical\n",
                            "    - ...\n",
                            "\n",
                            "\n",
                            "container\n",
                            "\n",
                            "  population\n",
                            "      subset   name          rows   type\n",
                            "  0   test     population    8661   View\n",
                            "  1   train    population   33096   View\n",
                            "\n",
                            "  peripheral\n",
                            "      name          rows   type     \n",
                            "  0   population   41757   DataFrame"
                        ]
                    },
                    "execution_count": 26,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "time_series4 = getml.data.TimeSeries(\n",
                "    population=data,\n",
                "    alias=\"population\",\n",
                "    split=split,\n",
                "    time_stamps=\"date\",\n",
                "    memory=getml.data.time.days(1),\n",
                ")\n",
                "\n",
                "time_series4"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 27,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre>Pipeline(data_model='population',\n",
                            "         feature_learners=['FastProp'],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=['population'],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['getML: FastProp', 'memory: 1d', 'simple features'])</pre>"
                        ],
                        "text/plain": [
                            "Pipeline(data_model='population',\n",
                            "         feature_learners=['FastProp'],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=['population'],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['getML: FastProp', 'memory: 1d', 'simple features'])"
                        ]
                    },
                    "execution_count": 27,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "fast_prop = getml.feature_learning.FastProp(\n",
                "    loss_function=getml.feature_learning.loss_functions.SquareLoss,\n",
                "    num_threads=1,\n",
                "    aggregation=getml.feature_learning.FastProp.agg_sets.All,\n",
                ")\n",
                "\n",
                "predictor = getml.predictors.XGBoostRegressor(n_jobs=1)\n",
                "\n",
                "pipe4 = getml.pipeline.Pipeline(\n",
                "    tags=[\"getML: FastProp\", \"memory: 1d\", \"simple features\"],\n",
                "    data_model=time_series4.data_model,\n",
                "    feature_learners=[fast_prop],\n",
                "    predictors=[predictor],\n",
                ")\n",
                "\n",
                "pipe4"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 28,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Checking data model<span style=\"color: #808000; text-decoration-color: #808000\">...</span>\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Checking data model\u001b[33m...\u001b[0m\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Checking... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:01\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">OK.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "OK.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "pipe4.check(time_series4.train)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 29,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Checking data model<span style=\"color: #808000; text-decoration-color: #808000\">...</span>\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Checking data model\u001b[33m...\u001b[0m\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">OK.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "OK.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  FastProp: Trying 378 features... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:05\n",
                        "\u001b[2K  FastProp: Building features... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:03\n",
                        "\u001b[2K  XGBoost: Training as predictor... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:19\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Trained pipeline.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Trained pipeline.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Time taken: 0:00:28.050749.\n",
                        "\n"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre>Pipeline(data_model='population',\n",
                            "         feature_learners=['FastProp'],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=['population'],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['getML: FastProp', 'memory: 1d', 'simple features', 'container-pspq7Q'])</pre>"
                        ],
                        "text/plain": [
                            "Pipeline(data_model='population',\n",
                            "         feature_learners=['FastProp'],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=['population'],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['getML: FastProp', 'memory: 1d', 'simple features', 'container-pspq7Q'])"
                        ]
                    },
                    "execution_count": 29,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pipe4.fit(time_series4.train)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 30,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Preprocessing... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  FastProp: Building features... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:01\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"datetime\">date time          </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">set used</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">target</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">     mae</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">    rmse</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">rsquared</th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"datetime\">2024-09-12 12:53:21</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">train</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">pm2.5</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">38.3028</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">55.2472</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.6438</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>1</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"datetime\">2024-09-12 12:53:22</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">test</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">pm2.5</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">44.2486</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">63.4164</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.5462</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>"
                        ],
                        "text/plain": [
                            "    date time             set used   target        mae       rmse   rsquared\n",
                            "0   2024-09-12 12:53:21   train      pm2.5     38.3028    55.2472     0.6438\n",
                            "1   2024-09-12 12:53:22   test       pm2.5     44.2486    63.4164     0.5462"
                        ]
                    },
                    "execution_count": 30,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pipe4.score(time_series4.test)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "#### 2.5 Using featuretools\n",
                "\n",
                "To make things a bit easier, we have written high-level wrappers around featuretools and tsfresh which we placed in a separate module (`utils`)."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 31,
            "metadata": {},
            "outputs": [],
            "source": [
                "data_train_pandas = time_series1.train.population.to_pandas()\n",
                "data_test_pandas = time_series1.test.population.to_pandas()"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "tsfresh and featuretools require the time series to have ids. Since there is only a single time series, that series has the same id."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 32,
            "metadata": {},
            "outputs": [],
            "source": [
                "data_train_pandas[\"id\"] = 1\n",
                "data_test_pandas[\"id\"] = 1"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 33,
            "metadata": {},
            "outputs": [],
            "source": [
                "if RUN_FEATURETOOLS:\n",
                "    ft_builder = FTTimeSeriesBuilder(\n",
                "        num_features=200,\n",
                "        horizon=pd.Timedelta(days=0),\n",
                "        memory=pd.Timedelta(days=1),\n",
                "        column_id=\"id\",\n",
                "        time_stamp=\"date\",\n",
                "        target=\"pm2.5\",\n",
                "    )\n",
                "    #\n",
                "    featuretools_training = ft_builder.fit(data_train_pandas)\n",
                "    featuretools_test = ft_builder.transform(data_test_pandas)\n",
                "\n",
                "    data_featuretools_training = getml.data.DataFrame.from_pandas(\n",
                "        featuretools_training, name=\"featuretools_training\"\n",
                "    )\n",
                "    data_featuretools_test = getml.data.DataFrame.from_pandas(\n",
                "        featuretools_test, name=\"featuretools_test\"\n",
                "    )"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 34,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Loading 'featuretools_training' from disk (project folder).\n",
                        "\n",
                        "Loading 'featuretools_test' from disk (project folder).\n",
                        "\n"
                    ]
                }
            ],
            "source": [
                "if not RUN_FEATURETOOLS:\n",
                "    data_featuretools_training = load_or_retrieve(\n",
                "        \"https://static.getml.com/datasets/air_pollution/featuretools/featuretools_training.csv\"\n",
                "    )\n",
                "    data_featuretools_test = load_or_retrieve(\n",
                "        \"https://static.getml.com/datasets/air_pollution/featuretools/featuretools_test.csv\"\n",
                "    )"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 35,
            "metadata": {},
            "outputs": [],
            "source": [
                "def set_roles_featuretools(df):\n",
                "    df.set_role([\"date\"], getml.data.roles.time_stamp)\n",
                "    df.set_role([\"pm2.5\"], getml.data.roles.target)\n",
                "    df.set_role([\"date\"], getml.data.roles.time_stamp)\n",
                "    df.set_role(df.roles.unused, getml.data.roles.numerical)\n",
                "    df.set_role([\"id\"], getml.data.roles.unused_float)\n",
                "    return df\n",
                "\n",
                "df_featuretools_training = set_roles_featuretools(data_featuretools_training)\n",
                "df_featuretools_test = set_roles_featuretools(data_featuretools_test)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 36,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre>Pipeline(data_model='population',\n",
                            "         feature_learners=[],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=[],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['featuretools', 'memory: 1d', 'simple features'])</pre>"
                        ],
                        "text/plain": [
                            "Pipeline(data_model='population',\n",
                            "         feature_learners=[],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=[],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['featuretools', 'memory: 1d', 'simple features'])"
                        ]
                    },
                    "execution_count": 36,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "predictor = getml.predictors.XGBoostRegressor()\n",
                "\n",
                "pipe5 = getml.pipeline.Pipeline(\n",
                "    tags=[\"featuretools\", \"memory: 1d\", \"simple features\"], predictors=[predictor]\n",
                ")\n",
                "\n",
                "pipe5"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 37,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Checking data model<span style=\"color: #808000; text-decoration-color: #808000\">...</span>\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Checking data model\u001b[33m...\u001b[0m\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Checking... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">OK.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "OK.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "pipe5.check(df_featuretools_training)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 38,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Checking data model<span style=\"color: #808000; text-decoration-color: #808000\">...</span>\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Checking data model\u001b[33m...\u001b[0m\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">OK.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "OK.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  XGBoost: Training as predictor... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:09\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Trained pipeline.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Trained pipeline.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Time taken: 0:00:09.162672.\n",
                        "\n"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre>Pipeline(data_model='population',\n",
                            "         feature_learners=[],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=[],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['featuretools', 'memory: 1d', 'simple features'])</pre>"
                        ],
                        "text/plain": [
                            "Pipeline(data_model='population',\n",
                            "         feature_learners=[],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=[],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['featuretools', 'memory: 1d', 'simple features'])"
                        ]
                    },
                    "execution_count": 38,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pipe5.fit(df_featuretools_training)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 39,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Preprocessing... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"datetime\">date time          </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">set used             </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">target</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">     mae</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">    rmse</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">rsquared</th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"datetime\">2024-09-12 12:53:34</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">featuretools_training</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">pm2.5</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">38.0455</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">54.4693</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.6567</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>1</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"datetime\">2024-09-12 12:53:34</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">featuretools_test</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">pm2.5</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">45.3084</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">64.2717</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.5373</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>"
                        ],
                        "text/plain": [
                            "    date time             set used                target        mae       rmse   rsquared\n",
                            "0   2024-09-12 12:53:34   featuretools_training   pm2.5     38.0455    54.4693     0.6567\n",
                            "1   2024-09-12 12:53:34   featuretools_test       pm2.5     45.3084    64.2717     0.5373"
                        ]
                    },
                    "execution_count": 39,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pipe5.score(df_featuretools_test)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "#### 2.6 Using tsfresh\n",
                "\n",
                "Next, we construct features with tsfresh. tsfresh is based on pandas and rely\\ies on explicit copies for meny operations. This leads to an excessive memory consumption that renders tsfresh nearly unusable for real-world scenarios. Remeber, this is a relatively small data set.\n",
                "\n",
                "To limit the memory consumption, we undertake the following steps:\n",
                "\n",
                "- We limit ourselves to a memory of 1 day from any point in time. This is necessary, because tsfresh duplicates records for every time stamp. That means that looking back 7 days instead of one day, the memory consumption would be  seven times as high.\n",
                "- We extract only tsfresh's `MinimalFCParameters` and `IndexBasedFCParameters` (the latter is a superset of `TimeBasedFCParameters`).\n",
                "\n",
                "In order to make sure that tsfresh's features can be compared to getML's features, we also do the following:\n",
                "\n",
                "- We apply tsfresh's built-in feature selection algorithm.\n",
                "- Of the remaining features, we only keep the 40 features most correlated with the target (in terms of the absolute value of the correlation).\n",
                "- We add the original columns as additional features.\n"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 40,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<div>\n",
                            "<style scoped>\n",
                            "    .dataframe tbody tr th:only-of-type {\n",
                            "        vertical-align: middle;\n",
                            "    }\n",
                            "\n",
                            "    .dataframe tbody tr th {\n",
                            "        vertical-align: top;\n",
                            "    }\n",
                            "\n",
                            "    .dataframe thead th {\n",
                            "        text-align: right;\n",
                            "    }\n",
                            "</style>\n",
                            "<table border=\"1\" class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr style=\"text-align: right;\">\n",
                            "      <th></th>\n",
                            "      <th>DEWP</th>\n",
                            "      <th>TEMP</th>\n",
                            "      <th>PRES</th>\n",
                            "      <th>Iws</th>\n",
                            "      <th>Is</th>\n",
                            "      <th>Ir</th>\n",
                            "      <th>pm2.5</th>\n",
                            "      <th>date</th>\n",
                            "      <th>id</th>\n",
                            "    </tr>\n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    <tr>\n",
                            "      <th>0</th>\n",
                            "      <td>-16.0</td>\n",
                            "      <td>-4.0</td>\n",
                            "      <td>1020.0</td>\n",
                            "      <td>1.79</td>\n",
                            "      <td>0.0</td>\n",
                            "      <td>0.0</td>\n",
                            "      <td>129.0</td>\n",
                            "      <td>2010-01-02 00:00:00</td>\n",
                            "      <td>1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>1</th>\n",
                            "      <td>-15.0</td>\n",
                            "      <td>-4.0</td>\n",
                            "      <td>1020.0</td>\n",
                            "      <td>2.68</td>\n",
                            "      <td>0.0</td>\n",
                            "      <td>0.0</td>\n",
                            "      <td>148.0</td>\n",
                            "      <td>2010-01-02 01:00:00</td>\n",
                            "      <td>1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>2</th>\n",
                            "      <td>-11.0</td>\n",
                            "      <td>-5.0</td>\n",
                            "      <td>1021.0</td>\n",
                            "      <td>3.57</td>\n",
                            "      <td>0.0</td>\n",
                            "      <td>0.0</td>\n",
                            "      <td>159.0</td>\n",
                            "      <td>2010-01-02 02:00:00</td>\n",
                            "      <td>1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>3</th>\n",
                            "      <td>-7.0</td>\n",
                            "      <td>-5.0</td>\n",
                            "      <td>1022.0</td>\n",
                            "      <td>5.36</td>\n",
                            "      <td>1.0</td>\n",
                            "      <td>0.0</td>\n",
                            "      <td>181.0</td>\n",
                            "      <td>2010-01-02 03:00:00</td>\n",
                            "      <td>1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>4</th>\n",
                            "      <td>-7.0</td>\n",
                            "      <td>-5.0</td>\n",
                            "      <td>1022.0</td>\n",
                            "      <td>6.25</td>\n",
                            "      <td>2.0</td>\n",
                            "      <td>0.0</td>\n",
                            "      <td>138.0</td>\n",
                            "      <td>2010-01-02 04:00:00</td>\n",
                            "      <td>1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>...</th>\n",
                            "      <td>...</td>\n",
                            "      <td>...</td>\n",
                            "      <td>...</td>\n",
                            "      <td>...</td>\n",
                            "      <td>...</td>\n",
                            "      <td>...</td>\n",
                            "      <td>...</td>\n",
                            "      <td>...</td>\n",
                            "      <td>...</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>33091</th>\n",
                            "      <td>-19.0</td>\n",
                            "      <td>7.0</td>\n",
                            "      <td>1013.0</td>\n",
                            "      <td>114.87</td>\n",
                            "      <td>0.0</td>\n",
                            "      <td>0.0</td>\n",
                            "      <td>22.0</td>\n",
                            "      <td>2013-12-31 19:00:00</td>\n",
                            "      <td>1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>33092</th>\n",
                            "      <td>-21.0</td>\n",
                            "      <td>7.0</td>\n",
                            "      <td>1014.0</td>\n",
                            "      <td>119.79</td>\n",
                            "      <td>0.0</td>\n",
                            "      <td>0.0</td>\n",
                            "      <td>18.0</td>\n",
                            "      <td>2013-12-31 20:00:00</td>\n",
                            "      <td>1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>33093</th>\n",
                            "      <td>-21.0</td>\n",
                            "      <td>7.0</td>\n",
                            "      <td>1014.0</td>\n",
                            "      <td>125.60</td>\n",
                            "      <td>0.0</td>\n",
                            "      <td>0.0</td>\n",
                            "      <td>23.0</td>\n",
                            "      <td>2013-12-31 21:00:00</td>\n",
                            "      <td>1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>33094</th>\n",
                            "      <td>-21.0</td>\n",
                            "      <td>6.0</td>\n",
                            "      <td>1014.0</td>\n",
                            "      <td>130.52</td>\n",
                            "      <td>0.0</td>\n",
                            "      <td>0.0</td>\n",
                            "      <td>20.0</td>\n",
                            "      <td>2013-12-31 22:00:00</td>\n",
                            "      <td>1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>33095</th>\n",
                            "      <td>-20.0</td>\n",
                            "      <td>7.0</td>\n",
                            "      <td>1014.0</td>\n",
                            "      <td>137.67</td>\n",
                            "      <td>0.0</td>\n",
                            "      <td>0.0</td>\n",
                            "      <td>23.0</td>\n",
                            "      <td>2013-12-31 23:00:00</td>\n",
                            "      <td>1</td>\n",
                            "    </tr>\n",
                            "  </tbody>\n",
                            "</table>\n",
                            "<p>33096 rows × 9 columns</p>\n",
                            "</div>"
                        ],
                        "text/plain": [
                            "       DEWP  TEMP    PRES     Iws   Is   Ir  pm2.5                date  id\n",
                            "0     -16.0  -4.0  1020.0    1.79  0.0  0.0  129.0 2010-01-02 00:00:00   1\n",
                            "1     -15.0  -4.0  1020.0    2.68  0.0  0.0  148.0 2010-01-02 01:00:00   1\n",
                            "2     -11.0  -5.0  1021.0    3.57  0.0  0.0  159.0 2010-01-02 02:00:00   1\n",
                            "3      -7.0  -5.0  1022.0    5.36  1.0  0.0  181.0 2010-01-02 03:00:00   1\n",
                            "4      -7.0  -5.0  1022.0    6.25  2.0  0.0  138.0 2010-01-02 04:00:00   1\n",
                            "...     ...   ...     ...     ...  ...  ...    ...                 ...  ..\n",
                            "33091 -19.0   7.0  1013.0  114.87  0.0  0.0   22.0 2013-12-31 19:00:00   1\n",
                            "33092 -21.0   7.0  1014.0  119.79  0.0  0.0   18.0 2013-12-31 20:00:00   1\n",
                            "33093 -21.0   7.0  1014.0  125.60  0.0  0.0   23.0 2013-12-31 21:00:00   1\n",
                            "33094 -21.0   6.0  1014.0  130.52  0.0  0.0   20.0 2013-12-31 22:00:00   1\n",
                            "33095 -20.0   7.0  1014.0  137.67  0.0  0.0   23.0 2013-12-31 23:00:00   1\n",
                            "\n",
                            "[33096 rows x 9 columns]"
                        ]
                    },
                    "execution_count": 40,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "data_train_pandas"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 41,
            "metadata": {},
            "outputs": [],
            "source": [
                "if RUN_TSFRESH:\n",
                "    tsfresh_builder = TSFreshBuilder(\n",
                "        num_features=200, memory=24, column_id=\"id\", time_stamp=\"date\", target=\"pm2.5\"\n",
                "    )\n",
                "    #\n",
                "    tsfresh_training = tsfresh_builder.fit(data_train_pandas)\n",
                "    tsfresh_test = tsfresh_builder.transform(data_test_pandas)\n",
                "    #\n",
                "    data_tsfresh_training = getml.data.DataFrame.from_pandas(\n",
                "        tsfresh_training, name=\"tsfresh_training\"\n",
                "    )\n",
                "    data_tsfresh_test = getml.data.DataFrame.from_pandas(\n",
                "        tsfresh_test, name=\"tsfresh_test\"\n",
                "    )"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "tsfresh does not contain built-in machine learning algorithms. In order to ensure a fair comparison, we use the exact same machine learning algorithm we have also used for getML: An XGBoost regressor with all hyperparameters set to their default values.\n",
                "\n",
                "In order to do so, we load the tsfresh features into the getML engine."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 42,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Loading 'tsfresh_training' from disk (project folder).\n"
                    ]
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\n",
                        "Loading 'tsfresh_test' from disk (project folder).\n",
                        "\n"
                    ]
                }
            ],
            "source": [
                "if not RUN_TSFRESH:\n",
                "    data_tsfresh_training = load_or_retrieve(\n",
                "        \"https://static.getml.com/datasets/air_pollution/tsfresh/tsfresh_training.csv\"\n",
                "    )\n",
                "    data_tsfresh_test = load_or_retrieve(\n",
                "        \"https://static.getml.com/datasets/air_pollution/tsfresh/tsfresh_test.csv\"\n",
                "    )"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "As usual, we need to set roles:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 43,
            "metadata": {},
            "outputs": [],
            "source": [
                "def set_roles_tsfresh(df):\n",
                "    df.set_role([\"date\"], getml.data.roles.time_stamp)\n",
                "    df.set_role([\"pm2.5\"], getml.data.roles.target)\n",
                "    df.set_role([\"date\"], getml.data.roles.time_stamp)\n",
                "    df.set_role(df.roles.unused, getml.data.roles.numerical)\n",
                "    df.set_role([\"id\"], getml.data.roles.unused_float)\n",
                "    return df\n",
                "\n",
                "df_tsfresh_training = set_roles_tsfresh(data_tsfresh_training)\n",
                "df_tsfresh_test = set_roles_tsfresh(data_tsfresh_test)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "In this case, our pipeline is very simple. It only consists of a single XGBoostRegressor."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 44,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre>Pipeline(data_model='population',\n",
                            "         feature_learners=[],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=[],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['tsfresh', 'memory: 1d', 'simple features'])</pre>"
                        ],
                        "text/plain": [
                            "Pipeline(data_model='population',\n",
                            "         feature_learners=[],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=[],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['tsfresh', 'memory: 1d', 'simple features'])"
                        ]
                    },
                    "execution_count": 44,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "predictor = getml.predictors.XGBoostRegressor()\n",
                "\n",
                "pipe6 = getml.pipeline.Pipeline(\n",
                "    tags=[\"tsfresh\", \"memory: 1d\", \"simple features\"], predictors=[predictor]\n",
                ")\n",
                "\n",
                "pipe6"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 45,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Checking data model<span style=\"color: #808000; text-decoration-color: #808000\">...</span>\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Checking data model\u001b[33m...\u001b[0m\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Checking... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">OK.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "OK.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "pipe6.check(df_tsfresh_training)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 46,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Checking data model<span style=\"color: #808000; text-decoration-color: #808000\">...</span>\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Checking data model\u001b[33m...\u001b[0m\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">OK.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "OK.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  XGBoost: Training as predictor... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:06\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Trained pipeline.\n",
                            "</pre>\n"
                        ],
                        "text/plain": [
                            "Trained pipeline.\n"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Time taken: 0:00:06.916135.\n",
                        "\n"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<pre>Pipeline(data_model='population',\n",
                            "         feature_learners=[],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=[],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['tsfresh', 'memory: 1d', 'simple features'])</pre>"
                        ],
                        "text/plain": [
                            "Pipeline(data_model='population',\n",
                            "         feature_learners=[],\n",
                            "         feature_selectors=[],\n",
                            "         include_categorical=False,\n",
                            "         loss_function='SquareLoss',\n",
                            "         peripheral=[],\n",
                            "         predictors=['XGBoostRegressor'],\n",
                            "         preprocessors=[],\n",
                            "         share_selected_features=0.5,\n",
                            "         tags=['tsfresh', 'memory: 1d', 'simple features'])"
                        ]
                    },
                    "execution_count": 46,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pipe6.fit(df_tsfresh_training)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 47,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "\u001b[2K  Staging... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[2K  Preprocessing... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 00:00\n",
                        "\u001b[?25h"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\"> </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"datetime\">date time          </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">set used        </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">target</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">     mae</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">    rmse</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">rsquared</th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"datetime\">2024-09-12 12:53:43</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">tsfresh_training</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">pm2.5</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">40.8062</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">57.7874</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.6106</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>1</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"datetime\">2024-09-12 12:53:44</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">tsfresh_test</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">pm2.5</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">46.698</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">65.9163</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.5105</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>"
                        ],
                        "text/plain": [
                            "    date time             set used           target        mae       rmse   rsquared\n",
                            "0   2024-09-12 12:53:43   tsfresh_training   pm2.5     40.8062    57.7874     0.6106\n",
                            "1   2024-09-12 12:53:44   tsfresh_test       pm2.5     46.698     65.9163     0.5105"
                        ]
                    },
                    "execution_count": 47,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pipe6.score(df_tsfresh_test)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                " "
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 48,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<style>\n",
                            "  th {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  td {\n",
                            "    text-align: left !important;\n",
                            "  }\n",
                            "  th:nth-child(1) {\n",
                            "    text-align: right;\n",
                            "    border-right: 1px solid LightGray;\n",
                            "  }\n",
                            "  th.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.float {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  th.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "  td.int {\n",
                            "    text-align: right !important;\n",
                            "  }\n",
                            "</style>\n",
                            "\n",
                            "<table class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr>\n",
                            "      \n",
                            "        \n",
                            "          <th class=\"int\">  </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">target</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"str\">name        </th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\">correlation</th>\n",
                            "        \n",
                            "      \n",
                            "        \n",
                            "          <th class=\"float\"> importance</th>\n",
                            "        \n",
                            "      \n",
                            "    </tr>\n",
                            "    \n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>0</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">pm2.5</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">feature_1_1</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.7269</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.19987785</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>1</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">pm2.5</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">feature_1_2</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.7046</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.11091681</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>2</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">pm2.5</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">feature_1_3</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.7158</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.08472355</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>3</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">pm2.5</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">feature_1_4</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.6812</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.01239961</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>4</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">pm2.5</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">feature_1_5</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.7363</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.26067</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th></th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">...</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">...</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">...</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">...</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>11</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">pm2.5</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">temp</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">-0.2112</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.0036176</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>12</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">pm2.5</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">pres</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.0811</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.00604251</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>13</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">pm2.5</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">iws</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">-0.2166</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.00110227</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>14</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">pm2.5</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">is</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.0045</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.00006856</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "      <tr>\n",
                            "        <th>15</th>\n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">pm2.5</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"str\">ir</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">-0.0541</td>\n",
                            "            \n",
                            "          \n",
                            "            \n",
                            "              <td class=\"float\">0.00071485</td>\n",
                            "            \n",
                            "          \n",
                            "      </tr>\n",
                            "    \n",
                            "  </tbody>\n",
                            "</table>"
                        ],
                        "text/plain": [
                            "     target   name           correlation    importance\n",
                            " 0   pm2.5    feature_1_1         0.7269    0.19987785\n",
                            " 1   pm2.5    feature_1_2         0.7046    0.11091681\n",
                            " 2   pm2.5    feature_1_3         0.7158    0.08472355\n",
                            " 3   pm2.5    feature_1_4         0.6812    0.01239961\n",
                            " 4   pm2.5    feature_1_5         0.7363    0.26067   \n",
                            "     ...      ...                    ...           ...\n",
                            "11   pm2.5    temp               -0.2112    0.0036176 \n",
                            "12   pm2.5    pres                0.0811    0.00604251\n",
                            "13   pm2.5    iws                -0.2166    0.00110227\n",
                            "14   pm2.5    is                  0.0045    0.00006856\n",
                            "15   pm2.5    ir                 -0.0541    0.00071485"
                        ]
                    },
                    "execution_count": 48,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pipe1.features"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "#### 2.7 Studying features"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 49,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/markdown": [
                            "```sql\n",
                            "DROP TABLE IF EXISTS \"FEATURE_1_5\";\n",
                            "\n",
                            "CREATE TABLE \"FEATURE_1_5\" AS\n",
                            "SELECT SUM( \n",
                            "    CASE\n",
                            "        WHEN ( t2.\"iws\" > 2.996864 ) AND ( t1.\"date\" - t2.\"date\" > 111439.618138 ) THEN COALESCE( t1.\"dewp\" - 1.513306719893546, 0.0 ) * 0.03601591249256588 + COALESCE( t1.\"temp\" - 11.89115103127079, 0.0 ) * -0.04266314571848353 + COALESCE( t1.\"is\" - 0.06613439787092482, 0.0 ) * -0.04725168059692409 + COALESCE( t1.\"ir\" - 0.2117099135063207, 0.0 ) * -0.06318182426623756 + COALESCE( t1.\"pres\" - 1016.467221113307, 0.0 ) * -0.01398384065516279 + COALESCE( t1.\"iws\" - 25.06232468396705, 0.0 ) * -0.001201895740197315 + COALESCE( t1.\"date\" - 1326374597.365269, 0.0 ) * -1.923501069200198e-07 + COALESCE( t2.\"dewp\" - 1.668379064426448, 0.0 ) * 0.001731655710118117 + COALESCE( t2.\"is\" - 0.06086063096820984, 0.0 ) * 0.0606336044544106 + COALESCE( t2.\"ir\" - 0.2111990813489665, 0.0 ) * -0.0540256903488962 + COALESCE( t2.\"temp\" - 12.06001450501632, 0.0 ) * -0.00767357563612661 + COALESCE( t2.\"pres\" - 1016.398404448205, 0.0 ) * 0.002398206365154774 + COALESCE( t2.\"iws\" - 25.00605463556588, 0.0 ) * 0.000683821481632905 + COALESCE( t2.\"date\" - 1326341256.690439, 0.0 ) * 1.921124564163103e-07 + -3.7650673692887868e-02\n",
                            "        WHEN ( t2.\"iws\" > 2.996864 ) AND ( t1.\"date\" - t2.\"date\" <= 111439.618138 OR t1.\"date\" IS NULL OR t2.\"date\" IS NULL ) THEN COALESCE( t1.\"dewp\" - 1.513306719893546, 0.0 ) * -0.09708854784677848 + COALESCE( t1.\"temp\" - 11.89115103127079, 0.0 ) * 0.1898230328866885 + COALESCE( t1.\"is\" - 0.06613439787092482, 0.0 ) * 0.2216695565524211 + COALESCE( t1.\"ir\" - 0.2117099135063207, 0.0 ) * 0.2229264173214936 + COALESCE( t1.\"pres\" - 1016.467221113307, 0.0 ) * 0.01369730329877567 + COALESCE( t1.\"iws\" - 25.06232468396705, 0.0 ) * 0.008759560240072777 + COALESCE( t1.\"date\" - 1326374597.365269, 0.0 ) * 3.830243422560845e-05 + COALESCE( t2.\"dewp\" - 1.668379064426448, 0.0 ) * 0.01388305933958263 + COALESCE( t2.\"is\" - 0.06086063096820984, 0.0 ) * -0.05433702238361345 + COALESCE( t2.\"ir\" - 0.2111990813489665, 0.0 ) * -0.170863758308725 + COALESCE( t2.\"temp\" - 12.06001450501632, 0.0 ) * 0.04150526461437114 + COALESCE( t2.\"pres\" - 1016.398404448205, 0.0 ) * 0.0668720727713986 + COALESCE( t2.\"iws\" - 25.00605463556588, 0.0 ) * 0.0003743003078498523 + COALESCE( t2.\"date\" - 1326341256.690439, 0.0 ) * -3.830000792178088e-05 + -2.7370376342137779e+00\n",
                            "        WHEN ( t2.\"iws\" <= 2.996864 OR t2.\"iws\" IS NULL ) AND ( t1.\"dewp\" > 11.000000 ) THEN COALESCE( t1.\"dewp\" - 1.513306719893546, 0.0 ) * 0.1393420062387328 + COALESCE( t1.\"temp\" - 11.89115103127079, 0.0 ) * -0.01209129738437093 + COALESCE( t1.\"is\" - 0.06613439787092482, 0.0 ) * -0.1331350023952664 + COALESCE( t1.\"ir\" - 0.2117099135063207, 0.0 ) * -0.01622004854424353 + COALESCE( t1.\"pres\" - 1016.467221113307, 0.0 ) * 0.001749305913977287 + COALESCE( t1.\"iws\" - 25.06232468396705, 0.0 ) * 0.003366205707337489 + COALESCE( t1.\"date\" - 1326374597.365269, 0.0 ) * -1.876475085083195e-07 + COALESCE( t2.\"dewp\" - 1.668379064426448, 0.0 ) * -0.06649947838403912 + COALESCE( t2.\"is\" - 0.06086063096820984, 0.0 ) * -0.1410305376611299 + COALESCE( t2.\"ir\" - 0.2111990813489665, 0.0 ) * -0.1304194940194324 + COALESCE( t2.\"temp\" - 12.06001450501632, 0.0 ) * -0.1337048685573981 + COALESCE( t2.\"pres\" - 1016.398404448205, 0.0 ) * -0.01591411669667236 + COALESCE( t2.\"iws\" - 25.00605463556588, 0.0 ) * 0.05236298037580929 + COALESCE( t2.\"date\" - 1326341256.690439, 0.0 ) * 1.819565845131136e-07 + 1.4688062110162212e+00\n",
                            "        WHEN ( t2.\"iws\" <= 2.996864 OR t2.\"iws\" IS NULL ) AND ( t1.\"dewp\" <= 11.000000 OR t1.\"dewp\" IS NULL ) THEN COALESCE( t1.\"dewp\" - 1.513306719893546, 0.0 ) * 0.04638276959734068 + COALESCE( t1.\"temp\" - 11.89115103127079, 0.0 ) * -0.02618590276218461 + COALESCE( t1.\"is\" - 0.06613439787092482, 0.0 ) * -0.04277883865346508 + COALESCE( t1.\"ir\" - 0.2117099135063207, 0.0 ) * -0.02541052942548589 + COALESCE( t1.\"pres\" - 1016.467221113307, 0.0 ) * -0.02717894125656619 + COALESCE( t1.\"iws\" - 25.06232468396705, 0.0 ) * -0.002780031916675637 + COALESCE( t1.\"date\" - 1326374597.365269, 0.0 ) * -1.128077615757182e-06 + COALESCE( t2.\"dewp\" - 1.668379064426448, 0.0 ) * -0.009594127784270889 + COALESCE( t2.\"is\" - 0.06086063096820984, 0.0 ) * -0.2440485267005635 + COALESCE( t2.\"ir\" - 0.2111990813489665, 0.0 ) * -0.1198304522130264 + COALESCE( t2.\"temp\" - 12.06001450501632, 0.0 ) * -0.07233231618474294 + COALESCE( t2.\"pres\" - 1016.398404448205, 0.0 ) * -0.006858030218452998 + COALESCE( t2.\"iws\" - 25.00605463556588, 0.0 ) * -0.4131514311185667 + COALESCE( t2.\"date\" - 1326341256.690439, 0.0 ) * 1.129955468463647e-06 + -9.4340473685385735e+00\n",
                            "        ELSE NULL\n",
                            "    END\n",
                            ") AS \"feature_1_5\",\n",
                            "       t1.rowid AS rownum\n",
                            "FROM \"POPULATION__STAGING_TABLE_1\" t1\n",
                            "INNER JOIN \"POPULATION__STAGING_TABLE_2\" t2\n",
                            "ON 1 = 1\n",
                            "WHERE t2.\"date\" <= t1.\"date\"\n",
                            "AND ( t2.\"date__7_000000_days\" > t1.\"date\" OR t2.\"date__7_000000_days\" IS NULL )\n",
                            "GROUP BY t1.rowid;\n",
                            "```"
                        ],
                        "text/plain": [
                            "'DROP TABLE IF EXISTS \"FEATURE_1_5\";\\n\\nCREATE TABLE \"FEATURE_1_5\" AS\\nSELECT SUM( \\n    CASE\\n        WHEN ( t2.\"iws\" > 2.996864 ) AND ( t1.\"date\" - t2.\"date\" > 111439.618138 ) THEN COALESCE( t1.\"dewp\" - 1.513306719893546, 0.0 ) * 0.03601591249256588 + COALESCE( t1.\"temp\" - 11.89115103127079, 0.0 ) * -0.04266314571848353 + COALESCE( t1.\"is\" - 0.06613439787092482, 0.0 ) * -0.04725168059692409 + COALESCE( t1.\"ir\" - 0.2117099135063207, 0.0 ) * -0.06318182426623756 + COALESCE( t1.\"pres\" - 1016.467221113307, 0.0 ) * -0.01398384065516279 + COALESCE( t1.\"iws\" - 25.06232468396705, 0.0 ) * -0.001201895740197315 + COALESCE( t1.\"date\" - 1326374597.365269, 0.0 ) * -1.923501069200198e-07 + COALESCE( t2.\"dewp\" - 1.668379064426448, 0.0 ) * 0.001731655710118117 + COALESCE( t2.\"is\" - 0.06086063096820984, 0.0 ) * 0.0606336044544106 + COALESCE( t2.\"ir\" - 0.2111990813489665, 0.0 ) * -0.0540256903488962 + COALESCE( t2.\"temp\" - 12.06001450501632, 0.0 ) * -0.00767357563612661 + COALESCE( t2.\"pres\" - 1016.398404448205, 0.0 ) * 0.002398206365154774 + COALESCE( t2.\"iws\" - 25.00605463556588, 0.0 ) * 0.000683821481632905 + COALESCE( t2.\"date\" - 1326341256.690439, 0.0 ) * 1.921124564163103e-07 + -3.7650673692887868e-02\\n        WHEN ( t2.\"iws\" > 2.996864 ) AND ( t1.\"date\" - t2.\"date\" <= 111439.618138 OR t1.\"date\" IS NULL OR t2.\"date\" IS NULL ) THEN COALESCE( t1.\"dewp\" - 1.513306719893546, 0.0 ) * -0.09708854784677848 + COALESCE( t1.\"temp\" - 11.89115103127079, 0.0 ) * 0.1898230328866885 + COALESCE( t1.\"is\" - 0.06613439787092482, 0.0 ) * 0.2216695565524211 + COALESCE( t1.\"ir\" - 0.2117099135063207, 0.0 ) * 0.2229264173214936 + COALESCE( t1.\"pres\" - 1016.467221113307, 0.0 ) * 0.01369730329877567 + COALESCE( t1.\"iws\" - 25.06232468396705, 0.0 ) * 0.008759560240072777 + COALESCE( t1.\"date\" - 1326374597.365269, 0.0 ) * 3.830243422560845e-05 + COALESCE( t2.\"dewp\" - 1.668379064426448, 0.0 ) * 0.01388305933958263 + COALESCE( t2.\"is\" - 0.06086063096820984, 0.0 ) * -0.05433702238361345 + COALESCE( t2.\"ir\" - 0.2111990813489665, 0.0 ) * -0.170863758308725 + COALESCE( t2.\"temp\" - 12.06001450501632, 0.0 ) * 0.04150526461437114 + COALESCE( t2.\"pres\" - 1016.398404448205, 0.0 ) * 0.0668720727713986 + COALESCE( t2.\"iws\" - 25.00605463556588, 0.0 ) * 0.0003743003078498523 + COALESCE( t2.\"date\" - 1326341256.690439, 0.0 ) * -3.830000792178088e-05 + -2.7370376342137779e+00\\n        WHEN ( t2.\"iws\" <= 2.996864 OR t2.\"iws\" IS NULL ) AND ( t1.\"dewp\" > 11.000000 ) THEN COALESCE( t1.\"dewp\" - 1.513306719893546, 0.0 ) * 0.1393420062387328 + COALESCE( t1.\"temp\" - 11.89115103127079, 0.0 ) * -0.01209129738437093 + COALESCE( t1.\"is\" - 0.06613439787092482, 0.0 ) * -0.1331350023952664 + COALESCE( t1.\"ir\" - 0.2117099135063207, 0.0 ) * -0.01622004854424353 + COALESCE( t1.\"pres\" - 1016.467221113307, 0.0 ) * 0.001749305913977287 + COALESCE( t1.\"iws\" - 25.06232468396705, 0.0 ) * 0.003366205707337489 + COALESCE( t1.\"date\" - 1326374597.365269, 0.0 ) * -1.876475085083195e-07 + COALESCE( t2.\"dewp\" - 1.668379064426448, 0.0 ) * -0.06649947838403912 + COALESCE( t2.\"is\" - 0.06086063096820984, 0.0 ) * -0.1410305376611299 + COALESCE( t2.\"ir\" - 0.2111990813489665, 0.0 ) * -0.1304194940194324 + COALESCE( t2.\"temp\" - 12.06001450501632, 0.0 ) * -0.1337048685573981 + COALESCE( t2.\"pres\" - 1016.398404448205, 0.0 ) * -0.01591411669667236 + COALESCE( t2.\"iws\" - 25.00605463556588, 0.0 ) * 0.05236298037580929 + COALESCE( t2.\"date\" - 1326341256.690439, 0.0 ) * 1.819565845131136e-07 + 1.4688062110162212e+00\\n        WHEN ( t2.\"iws\" <= 2.996864 OR t2.\"iws\" IS NULL ) AND ( t1.\"dewp\" <= 11.000000 OR t1.\"dewp\" IS NULL ) THEN COALESCE( t1.\"dewp\" - 1.513306719893546, 0.0 ) * 0.04638276959734068 + COALESCE( t1.\"temp\" - 11.89115103127079, 0.0 ) * -0.02618590276218461 + COALESCE( t1.\"is\" - 0.06613439787092482, 0.0 ) * -0.04277883865346508 + COALESCE( t1.\"ir\" - 0.2117099135063207, 0.0 ) * -0.02541052942548589 + COALESCE( t1.\"pres\" - 1016.467221113307, 0.0 ) * -0.02717894125656619 + COALESCE( t1.\"iws\" - 25.06232468396705, 0.0 ) * -0.002780031916675637 + COALESCE( t1.\"date\" - 1326374597.365269, 0.0 ) * -1.128077615757182e-06 + COALESCE( t2.\"dewp\" - 1.668379064426448, 0.0 ) * -0.009594127784270889 + COALESCE( t2.\"is\" - 0.06086063096820984, 0.0 ) * -0.2440485267005635 + COALESCE( t2.\"ir\" - 0.2111990813489665, 0.0 ) * -0.1198304522130264 + COALESCE( t2.\"temp\" - 12.06001450501632, 0.0 ) * -0.07233231618474294 + COALESCE( t2.\"pres\" - 1016.398404448205, 0.0 ) * -0.006858030218452998 + COALESCE( t2.\"iws\" - 25.00605463556588, 0.0 ) * -0.4131514311185667 + COALESCE( t2.\"date\" - 1326341256.690439, 0.0 ) * 1.129955468463647e-06 + -9.4340473685385735e+00\\n        ELSE NULL\\n    END\\n) AS \"feature_1_5\",\\n       t1.rowid AS rownum\\nFROM \"POPULATION__STAGING_TABLE_1\" t1\\nINNER JOIN \"POPULATION__STAGING_TABLE_2\" t2\\nON 1 = 1\\nWHERE t2.\"date\" <= t1.\"date\"\\nAND ( t2.\"date__7_000000_days\" > t1.\"date\" OR t2.\"date__7_000000_days\" IS NULL )\\nGROUP BY t1.rowid;'"
                        ]
                    },
                    "execution_count": 49,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pipe1.features.sort(by=\"importances\")[0].sql"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "This is a typical [RelMT](https://getml.com/latest/user_guide/concepts/feature_engineering/#feature-engineering-algorithms-relmt) feature, where the aggregation (`SUM` in this case) is applied conditionally – the conditions are learned by `RelMT` – to a set of linear models, whose weights are, again, learned by `RelMT`."
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "#### 2.8 Productionization\n",
                "\n",
                "It is possible to productionize the pipeline by transpiling the features into production-ready SQL code. Please also refer to getML's `sqlite3` module."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 57,
            "metadata": {
                "tags": []
            },
            "outputs": [],
            "source": [
                "# Creates a folder named air_pollution_pipeline containing the SQL code\n",
                "pipe1.features.to_sql().save(\"air_pollution_pipeline\")"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "### 3. Discussion"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "We have seen that getML outperforms tsfresh by more than 10 percentage points in terms of R-squared. We now want to analyze why that is.\n",
                "\n",
                "There are two possible hypotheses:\n",
                "\n",
                "- getML outperforms featuretools and tsfresh, because it using feature learning and is able to produce more complex features\n",
                "- getML outperforms featuretools and tsfresh, because it makes better use of memory and is able to look back further.\n",
                "\n",
                "Let's summarize our findings:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 51,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<div>\n",
                            "<style scoped>\n",
                            "    .dataframe tbody tr th:only-of-type {\n",
                            "        vertical-align: middle;\n",
                            "    }\n",
                            "\n",
                            "    .dataframe tbody tr th {\n",
                            "        vertical-align: top;\n",
                            "    }\n",
                            "\n",
                            "    .dataframe thead th {\n",
                            "        text-align: right;\n",
                            "    }\n",
                            "</style>\n",
                            "<table border=\"1\" class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr style=\"text-align: right;\">\n",
                            "      <th></th>\n",
                            "      <th>tool</th>\n",
                            "      <th>memory</th>\n",
                            "      <th>feature_complexity</th>\n",
                            "      <th>rsquared</th>\n",
                            "      <th>rmse</th>\n",
                            "    </tr>\n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    <tr>\n",
                            "      <th>0</th>\n",
                            "      <td>getML: RelMT</td>\n",
                            "      <td>7d</td>\n",
                            "      <td>complex</td>\n",
                            "      <td>62.7%</td>\n",
                            "      <td>57.7</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>1</th>\n",
                            "      <td>getML: RelMT</td>\n",
                            "      <td>1d</td>\n",
                            "      <td>complex</td>\n",
                            "      <td>48.0%</td>\n",
                            "      <td>67.5</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>2</th>\n",
                            "      <td>getML: FastProp</td>\n",
                            "      <td>7d</td>\n",
                            "      <td>simple</td>\n",
                            "      <td>56.1%</td>\n",
                            "      <td>62.6</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>3</th>\n",
                            "      <td>getML: FastProp</td>\n",
                            "      <td>1d</td>\n",
                            "      <td>simple</td>\n",
                            "      <td>54.6%</td>\n",
                            "      <td>63.4</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>4</th>\n",
                            "      <td>featuretools</td>\n",
                            "      <td>1d</td>\n",
                            "      <td>simple</td>\n",
                            "      <td>53.7%</td>\n",
                            "      <td>64.3</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>5</th>\n",
                            "      <td>tsfresh</td>\n",
                            "      <td>1d</td>\n",
                            "      <td>simple</td>\n",
                            "      <td>51.0%</td>\n",
                            "      <td>65.9</td>\n",
                            "    </tr>\n",
                            "  </tbody>\n",
                            "</table>\n",
                            "</div>"
                        ],
                        "text/plain": [
                            "              tool memory feature_complexity rsquared  rmse\n",
                            "0     getML: RelMT     7d            complex    62.7%  57.7\n",
                            "1     getML: RelMT     1d            complex    48.0%  67.5\n",
                            "2  getML: FastProp     7d             simple    56.1%  62.6\n",
                            "3  getML: FastProp     1d             simple    54.6%  63.4\n",
                            "4     featuretools     1d             simple    53.7%  64.3\n",
                            "5          tsfresh     1d             simple    51.0%  65.9"
                        ]
                    },
                    "execution_count": 51,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pipes = [pipe1, pipe2, pipe3, pipe4, pipe5, pipe6]\n",
                "\n",
                "comparison = pd.DataFrame(\n",
                "    dict(\n",
                "        tool=[pipe.tags[0] for pipe in pipes],\n",
                "        memory=[pipe.tags[1].split()[1] for pipe in pipes],\n",
                "        feature_complexity=[pipe.tags[2].split()[0] for pipe in pipes],\n",
                "        rsquared=[f\"{pipe.rsquared:.1%}\" for pipe in pipes],\n",
                "        rmse=[f\"{pipe.rmse:.3}\" for pipe in pipes],\n",
                "    )\n",
                ")\n",
                "\n",
                "comparison"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "The summary table shows that combination of both of our hypotheses explains why getML outperforms featuretools and tsfresh. Complex features do better than simple features with a memory of one day. With a memory of seven days, simple features actually get worse. But when you look back seven days and allow more complex features, you get good results.\n",
                "\n",
                "This suggests that getML outperforms featuretools and tsfresh, because it can make more efficient use of memory and thus look back further. Because RelMT uses feature learning and can build more complex features it can make better use of the greater look-back window."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "getml.engine.shutdown()"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "editable": true,
                "slideshow": {
                    "slide_type": ""
                },
                "tags": []
            },
            "source": [
                "### 4. Conclusion\n",
                "\n",
                "We have compared getML's feature learning algorithms to tsfresh's brute-force feature engineering approaches on a data set related to air pollution in China. We found that getML significantly outperforms featuretools and tsfresh. These results are consistent with the view that feature learning can yield significant improvements over simple propositionalization approaches.\n",
                "\n",
                "However, there are other datasets on which simple propositionalization performs well. Our suggestion is therefore to think of algorithms like `FastProp` and `RelMT` as tools in a toolbox. If a simple tool like `FastProp` gets the job done, then use that. But when you need more advanced approaches, like `RelMT`, you should have them at your disposal as well.\n",
                "\n",
                "You are encouraged to reproduce these results."
            ]
        }
    ],
    "metadata": {
        "celltoolbar": "Tags",
        "jupytext": {
            "encoding": "# -*- coding: utf-8 -*-",
            "formats": "ipynb,py:percent"
        },
        "kernelspec": {
            "display_name": "Python 3 (ipykernel)",
            "language": "python",
            "name": "python3"
        },
        "language_info": {
            "codemirror_mode": {
                "name": "ipython",
                "version": 3
            },
            "file_extension": ".py",
            "mimetype": "text/x-python",
            "name": "python",
            "nbconvert_exporter": "python",
            "pygments_lexer": "ipython3",
            "version": "3.11.4"
        },
        "toc": {
            "base_numbering": 1
        },
        "vscode": {
            "interpreter": {
                "hash": "916dbcbb3f70747c44a77c7bcd40155683ae19c65e1c03b4aa3499c5328201f1"
            }
        }
    },
    "nbformat": 4,
    "nbformat_minor": 4
}