DeepTime_Deep_Learning_Studio_for_Time_Series.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "DeepTime - Deep Learning Studio  for Time Series",
      "provenance": [],
      "collapsed_sections": [
        "_WhPYLoybkqn",
        "PG_6nILluUe_"
      ],
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UPONrVt2qqg0",
        "colab_type": "text"
      },
      "source": [
        "# DeepTime - Deep Learning Studio  for Time Series\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SqlYc1rEjXWI",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "This codeless studio  focus \n",
        "\n",
        "1. mapping from columns in the dataframe of Pandas to  values by \n",
        "  (1) One-hot coding; \n",
        "  (2) Categorical indicator\n",
        "  (3) normalizing functions.\n",
        "\n",
        "2. Windowing historic data automatically, so that RNN model can be fitted.\n",
        "\n",
        "3. Training with built-in LSTM and Seq2Seq models\n",
        "\n",
        "I am working on Deep Time (https://github.com/MRYingLEE/DeepTime-Deep-Learning-Framework-for-Time-Series-Forecasting). This tools is part of my research work.\n",
        "\n",
        "Tensorflow 2.x is used.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JFCT4ePCedWC",
        "colab_type": "text"
      },
      "source": [
        "![alt text](https://www.tensorflow.org/tutorials/structured_data/images/time_series.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "VxyBFc_kKazA"
      },
      "source": [
        "# Import TensorFlow and other libraries"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hjnV1NJswrwp",
        "colab_type": "text"
      },
      "source": [
        "Maybe later sklearn Preprocessing function (https://scikit-learn.org/stable/modules/preprocessing.html#preprocessing) will be supported.\n",
        "\n",
        "So far, only train_test_split of sklearn is used."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "LuOWVJBz8a6G",
        "colab": {}
      },
      "source": [
        "!pip install sklearn"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "9dEreb4QKizj",
        "colab": {}
      },
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "\n",
        "import tensorflow as tf\n",
        "\n",
        "from tensorflow import feature_column\n",
        "from tensorflow.feature_column import *\n",
        "from tensorflow.keras import layers\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "from io import StringIO\n",
        "\n",
        "# ipywidgets （https://github.com/jupyter-widgets/ipywidgets） makes the Jupyter Notebook interactive.\n",
        "from ipywidgets import *\n",
        "\n",
        "import sys\n",
        "import os\n",
        "\n",
        "# Use seaborn for pairplot\n",
        "!pip install -q seaborn\n",
        "import seaborn as sns\n",
        "from os import path"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "tY_iNI1SLUWB"
      },
      "source": [
        "# Useful helper functions for Map transformation\n",
        "\n",
        "The reason I create some helper function is that I want to make the generated code short and easy to read."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "r8uxeC7NLUWI",
        "colab": {}
      },
      "source": [
        "def categorical_strings(column,vocabulary_list):\n",
        "  \"\"\"A function to generate a one-hot column by the vocabulary list.\"\"\"\n",
        "  count_v=len(vocabulary_list)\n",
        "  table = tf.lookup.StaticVocabularyTable(\n",
        "    tf.lookup.KeyValueTensorInitializer(\n",
        "    vocabulary_list, range(count_v), key_dtype=tf.string, value_dtype=tf.int64, name=column\n",
        "    ),\n",
        "    1)\n",
        "    \n",
        "  def one_hot_column(row):\n",
        "    out = table.lookup(row)\n",
        "    return tf.one_hot(out,count_v+1)\n",
        "\n",
        "  return one_hot_column\n",
        "\n",
        "def categorical_identitys(column,vocabulary_list):\n",
        "  \"\"\"A function to generate a one-hot column by the vocabulary list for an integer column.\"\"\"\n",
        "  count_v=len(vocabulary_list)\n",
        "  table = tf.lookup.StaticVocabularyTable(\n",
        "    tf.lookup.KeyValueTensorInitializer(\n",
        "    vocabulary_list, range(count_v), key_dtype=tf.int64, value_dtype=tf.int64, name=column\n",
        "    ),\n",
        "    1)\n",
        "  def one_hot_column(row):\n",
        "    out = table.lookup(row)\n",
        "    return tf.one_hot(out,count_v+1)\n",
        "\n",
        "  return one_hot_column"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mP_WZQa--p2N",
        "colab_type": "text"
      },
      "source": [
        "# Class of an Estimator\n",
        "\n",
        "I like the idea of estimator to make machine learning more easily, but this is NOT an child of tf.estimator.Estimator class(https://www.tensorflow.org/guide/estimator).\n",
        "\n",
        "tf.estimator.Estimator class depends on feature_column (https://www.tensorflow.org/api_docs/python/tf/feature_column) heavily. I like the idea of feature_column also, but feature_column doesn't work well with time series and feature_column doeen't support functional API of Keras well. (If you find a way to solve my headache, please let me know.)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cuiPTNlh0-ad",
        "colab_type": "text"
      },
      "source": [
        "## The class is to define the grid column layout"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SX5fklYCDD_G",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class I_Column:\n",
        "  p_Column=0\n",
        "  p_dtype=1\n",
        "  p_Input=2\n",
        "  p_Label=3\n",
        "  p_future=4\n",
        "  p_FeatureKind=5\n",
        "  p_Numeric_Normalizer=8"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Y0HpKlg617St",
        "colab_type": "text"
      },
      "source": [
        "## The TsEstimator - The core of this studio"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "g_mJwgD2-t1l",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class TsEstimator:\n",
        "  \"\"\"\n",
        "   An estimator for time series, but this is NOT an child of tf.estimator.Estimator class (https://www.tensorflow.org/guide/estimator).\n",
        "  \"\"\"\n",
        "  # The AVAILABLE feature kind for dtype of Pandas\n",
        "  dtype_mapping_cross = StringIO(\"\"\"Kind,object,int64,float64,bool,datetime64,timedelta[ns],category,cat_int64,cat_string\n",
        "    categorical_identitys,FALSE,FALSE,FALSE,FALSE,FALSE,FALSE,FALSE,TRUE,FALSE\n",
        "    categorical_strings,TRUE,FALSE,FALSE,FALSE,FALSE,FALSE,TRUE,FALSE,TRUE\n",
        "        \"\"\")\n",
        "  df_dtype_mapping_cross = pd.read_csv(dtype_mapping_cross, sep=\",\")\n",
        "\n",
        "  def __init__(self, fn_get_configs):\n",
        "    \"\"\"\n",
        "    The initializer with assigned function\n",
        "    \"\"\"\n",
        "    df_all, df_train, df_val, df_test,default_inputs,default_labels,default_future, past_history, future_target,categories_limit, batch_size,single_step=fn_get_configs()\n",
        "\n",
        "    self._df_all=df_all\n",
        "\n",
        "    assert(df_all is not None)\n",
        "\n",
        "    # if (df_train is None):\n",
        "    #   self._df_train=df_all\n",
        "    # else:\n",
        "    #   self._df_train=df_train\n",
        "    self._df_train=df_train\n",
        "    self._df_val=df_val  # Not used so far \n",
        "    self._df_test=df_test # Not used so far \n",
        "\n",
        "    self.input_lambdas=[]  # In the form of (column_name, lambda)\n",
        "    self.future_lambdas=[]  # In the form of (column_name, lambda)\n",
        "    self.label_lambdas=[]  # In the form of (column_name, lambda)\n",
        "\n",
        "    self.global_normalizers={} # Not used so far \n",
        "    self.categorical_columns=[]\n",
        "    self.categories_limit=categories_limit\n",
        "    self.grid_features=None\n",
        "    self.grid_periods=None\n",
        "    self.category_lists= self.__df_desc() # If a column has less than this number (20 as default) of unique value, I will treate it as a category column.\n",
        "    self.code=\"\"\n",
        "\n",
        "    self.default_inputs=default_inputs\n",
        "    self.default_labels=default_labels\n",
        "    self.default_future=default_future\n",
        "\n",
        "    self.past_history = past_history\n",
        "    self.future_target = future_target\n",
        "    self.batch_size = batch_size\n",
        "    self.single_step = single_step\n",
        "\n",
        "    self.v_labels=0\n",
        "    self.v_future=0\n",
        "\n",
        "  @classmethod\n",
        "  def get_available_mapping(cls,col_dtype):\n",
        "    \"\"\"\n",
        "    to get available mapping for the assigned dtype\n",
        "    \"\"\"\n",
        "    return set(cls.df_dtype_mapping_cross[[\"Kind\",col_dtype]][cls.df_dtype_mapping_cross[col_dtype]][\"Kind\"].unique())\n",
        "\n",
        "  # ## To generate normalizer lambda and denormalizer one\n",
        "  # So far, only 2 kinds of normalizer and denormalizer are supported:\n",
        "\n",
        "  @staticmethod\n",
        "  def min_max_normalizer(min_v,max_v, v_str=\"by_train\",is_int64=False):\n",
        "    \"\"\"\n",
        "      To generate min-max normalizer and denomalizer lambda statements\n",
        "      min-max  : (value-min)/(max-min)      \n",
        "    \"\"\"\n",
        "    if is_int64:\n",
        "      ext_v_str=\"tf.cast(\"+v_str+\",tf.float32)\"\n",
        "    else:\n",
        "      ext_v_str=v_str\n",
        "    \n",
        "    return \"lambda \"+v_str+\": (\"+ext_v_str+ \" -\"+str(min_v)+\")/(\"+str(max_v)+\"-\"+str(min_v)+\")\",\"lambda \"+v_str+\": \"+ext_v_str+ \" *(\"+str(max_v)+\"-\"+str(min_v)+\")+\"+str(min_v)\n",
        "\n",
        "  @staticmethod\n",
        "  def std_normalizer(v_mean,v_std, v_str=\"by_train\",is_int64=False):\n",
        "    \"\"\"\n",
        "      To generate mean-std normalizer and denomalizer lambda statements\n",
        "      mean-std  : (value-mean)/std      \n",
        "    \"\"\"\n",
        "    if is_int64:\n",
        "      ext_v_str=\"tf.cast(\"+v_str+\",tf.float32)\"\n",
        "    else:\n",
        "      ext_v_str=v_str\n",
        "\n",
        "    return \"lambda \"+v_str+\": (\"+ext_v_str+ \" -\"+str(v_mean)+\")/\"+str(v_std),\"lambda \"+v_str+\": \"+ext_v_str+ \" *\"+str(v_std)+\"+\"+str(v_mean)\n",
        "\n",
        "  @staticmethod\n",
        "  def create_local_normalizers(col_name,df_statistics, v_str=\"by_train\",is_int64=False):\n",
        "    \"\"\"\n",
        "      To generate min-max/mean-std normalizer and denomalizer lambda statements given an statistics data\n",
        "    \"\"\"\n",
        "    v_min=df_statistics.loc[col_name][\"min\"]\n",
        "    v_max=df_statistics.loc[col_name][\"max\"]\n",
        "    v_mean=df_statistics.loc[col_name][\"mean\"]\n",
        "    v_std=df_statistics.loc[col_name][\"std\"]\n",
        "\n",
        "    n1,d1=TsEstimator.min_max_normalizer(v_min,v_max,v_str,is_int64=is_int64)\n",
        "    n2,d2=TsEstimator.std_normalizer(v_mean,v_std,v_str,is_int64=is_int64)\n",
        "\n",
        "    locals={n1:d1,n2:d2}\n",
        "    return locals\n",
        "\n",
        "  @staticmethod\n",
        "  def int_list_as_string(a):\n",
        "    \"\"\"To generated a suitable string for an integer list\"\"\"\n",
        "    s = [str(i) for i in a]\n",
        "    return  \"[\"+\",\".join(s)+\"]\"\n",
        "\n",
        "  @staticmethod\n",
        "  def string_list_as_string(s):\n",
        "    \"\"\"To generated a suitable string for a string list\"\"\"\n",
        "    return  \"['\"+\"','\".join(s)+\"']\"\n",
        "\n",
        "  def __df_desc(self):\n",
        "    \"\"\"\n",
        "      To generate available feature kinds and suitable normalizer lambda statements for every column.\n",
        "      Please note the whole dataframe and the train part are both required.\n",
        "      The whole dataframe is used to decide the vocalbulary list for each column.\n",
        "      Both the whole dataframe and the train part are used to generate lambda statements for NUMERIC columns. \n",
        "      So normalizing can be based on the whole data or only the train part. It's up to the data scientist.\n",
        "    \"\"\"\n",
        "    df_all=self._df_all\n",
        "    df_train=self._df_train\n",
        "\n",
        "    if (df_train is None):\n",
        "      df_statistics_train=None\n",
        "    else:\n",
        "      df_statistics_train=df_train.describe().T # to use train part to normalize!\n",
        "    df_statistics_all=df_all.describe().T # to use train part to normalize!\n",
        "    \n",
        "    category_lists={}\n",
        "    \n",
        "    for c in df_train.columns:\n",
        "      dtype_name=df_train[c].dtype.name\n",
        "\n",
        "      availables=self.get_available_mapping(dtype_name)\n",
        "\n",
        "      if availables is None:\n",
        "        availables={}\n",
        "\n",
        "      feature=\"numeric_column('\"+c+\"')\"\n",
        "\n",
        "      local_normalizers={}\n",
        "\n",
        "      if ((dtype_name==\"int64\") or (dtype_name==\"object\")):\n",
        "        is_int64=(dtype_name==\"int64\")\n",
        "\n",
        "        values_unique=df_all[c].unique()\n",
        "        f=len(values_unique)   # I use all rows to decide the cetegory list   \n",
        "        if f<self.categories_limit: #Category\n",
        "          if is_int64:\n",
        "            feature=categorical_identitys.__name__+\"('\"+c+\"',\"+self.int_list_as_string(values_unique)+\")\"\n",
        "          else:\n",
        "            feature=categorical_strings.__name__+\"('\"+c+\"',\"+self.string_list_as_string(values_unique)+\")\"\n",
        "          self.categorical_columns.append(c)\n",
        "        else:\n",
        "          if is_int64:\n",
        "            feature=\"numeric_column('\"+c+\"')\"\n",
        "            # local_normalizers=({\"lambda x:x\": \"lambda x:x\" })\n",
        "            if (df_statistics_train is not None):\n",
        "              local_normalizers0=self.create_local_normalizers(c,df_statistics_train,v_str=\"by_train\", is_int64=True)\n",
        "              local_normalizers.update(local_normalizers0)\n",
        "              self.global_normalizers.update(local_normalizers0)\n",
        "            local_normalizers1=self.create_local_normalizers(c,df_statistics_all,v_str=\"by_all\", is_int64=True)\n",
        "            self.global_normalizers.update(local_normalizers1)\n",
        "            local_normalizers.update(local_normalizers1)\n",
        "            local_normalizers.update({\"lambda x:x\": \"lambda x:x\" })\n",
        "          else:\n",
        "            feature=\"embedding_column('\"+\"('\"+c+\"')\"\n",
        "      else:\n",
        "        if (dtype_name==\"float64\"):\n",
        "            feature=\"numeric_column('\"+c+\"')\"\n",
        "            # local_normalizers=({\"lambda x:x\": \"lambda x:x\" })\n",
        "            if (df_statistics_train is not None):\n",
        "              local_normalizers0=self.create_local_normalizers(c,df_statistics_train,v_str=\"by_train\", is_int64=False)\n",
        "              local_normalizers.update(local_normalizers0)\n",
        "              self.global_normalizers.update(local_normalizers0)\n",
        "            local_normalizers1=self.create_local_normalizers(c,df_statistics_all,v_str=\"by_all\", is_int64=False)\n",
        "            self.global_normalizers.update(local_normalizers1)\n",
        "            local_normalizers.update(local_normalizers1)\n",
        "            local_normalizers.update({\"lambda x:x\": \"lambda x:x\" })\n",
        "        elif  (dtype_name==\"bool\"):\n",
        "            feature=\"numeric_column('\"+c+\"')\"\n",
        "        elif (dtype_name==\"category\"):\n",
        "          feature=\"categorical_column_with_vocabulary_list('\"+\"('\"+c+\"')\"\n",
        "          self.categorical_columns.append(c)\n",
        "        else:\n",
        "          feature=dtype_defaults[dtype_name] \n",
        "      \n",
        "      availables.add(feature)\n",
        "\n",
        "      availables={s.replace(\"(...)\",\"('\"+c+\"')\") for s in availables}\n",
        "      category_lists[c]={\"default\":feature,\"available\":availables,\"normalizers\": local_normalizers}\n",
        "\n",
        "    return category_lists\n",
        "\n",
        "  def get_feature_grid(self):\n",
        "    \"\"\"\n",
        "    To get or create a ipywidget grid for datasource settings\n",
        "    \"\"\"\n",
        "    if self.grid_features is not None:\n",
        "      return self.grid_features\n",
        "\n",
        "    default_inputs=self.default_inputs\n",
        "    default_labels=self.default_labels\n",
        "    default_future=self.default_future\n",
        "\n",
        "    df_all=self._df_all\n",
        "    df_train=self._df_train\n",
        "\n",
        "    cols=len(df_train.columns)\n",
        "    grid = GridspecLayout(cols+1, 12)\n",
        "    # To add a header at row 0\n",
        "    grid[0,I_Column.p_Column]= widgets.Label(value=\"Column\")\n",
        "    grid[0,I_Column.p_dtype]= widgets.Label(value=\"dtype\")\n",
        "    grid[0,I_Column.p_Input]= widgets.Label(value=\"Input?\")\n",
        "    grid[0,I_Column.p_Label]= widgets.Label(value=\"Label?\")\n",
        "    grid[0,I_Column.p_future]= widgets.Label(value=\"Future?\")  \n",
        "    grid[0,I_Column.p_FeatureKind:(I_Column.p_FeatureKind+3)]= widgets.Label(value=\"Feature Kind\")\n",
        "    grid[0,I_Column.p_Numeric_Normalizer:]= widgets.Label(value=\"Numeric Normalizer\")\n",
        "\n",
        "    for i in range(cols):\n",
        "      feature_option=self.category_lists[df_train.columns[i]]\n",
        "      grid[i+1,int(I_Column.p_Column)]= widgets.Label(value=df_train.columns[i])\n",
        "      grid[i+1,I_Column.p_dtype]= widgets.Label(value=df_train.dtypes[i].name)\n",
        "      grid[i+1,I_Column.p_Input]=widgets.Checkbox(value=(df_train.columns[i] in default_inputs),description='',indent=False,layout=Layout(height='auto', width='auto'))\n",
        "      grid[i+1,I_Column.p_Label]=widgets.Checkbox(value=(df_train.columns[i] in default_labels),indent=False,description='',layout=Layout(height='auto', width='auto'))\n",
        "      grid[i+1,I_Column.p_future]=widgets.Checkbox(value=(df_train.columns[i] in default_future),indent=False,description='',layout=Layout(height='auto', width='auto'))\n",
        "\n",
        "      grid[i+1,I_Column.p_FeatureKind:(I_Column.p_FeatureKind+3)]= widgets.Dropdown(\n",
        "        options=list(feature_option['available']),\n",
        "        value=feature_option['default'],\n",
        "        description=\"\",\n",
        "        layout=Layout(height='auto', width='auto')\n",
        "        )\n",
        "      \n",
        "      if len(feature_option['normalizers'])>0:\n",
        "        grid[i+1,I_Column.p_Numeric_Normalizer:]=widgets.Dropdown(\n",
        "          options=list(feature_option['normalizers'].keys()),\n",
        "          value=list(feature_option['normalizers'].keys())[0],\n",
        "          layout=Layout(height='auto', width='auto'),\n",
        "          description=\"\"\n",
        "          )\n",
        "    \n",
        "    self.grid_features=grid\n",
        "\n",
        "    return grid\n",
        "\n",
        "  @property\n",
        "  def seq2seq_mode(self):\n",
        "    \"\"\"To decide whether the model should be seq2seq instead of a vanilla LSTM\"\"\"\n",
        "    return len(self.future_lambdas)>0\n",
        "\n",
        "  def update_by_grid_features(self):\n",
        "    \"\"\" To update datasource settings according to the ipywidget grid\"\"\"\n",
        "    self.input_lambdas.clear()\n",
        "    self.label_lambdas.clear()\n",
        "    self.future_lambdas.clear()\n",
        "\n",
        "    grid=self.grid_features\n",
        "    for i in range(1,grid.n_rows):\n",
        "      f_col=grid[i,int(I_Column.p_Column)].value\n",
        "      if (grid[i,I_Column.p_FeatureKind].value.startswith(\"numeric_column(\")):\n",
        "        if (grid[i,I_Column.p_dtype].value ==\"bool\"):\n",
        "          lambda_f=lambda x: x\n",
        "        else:\n",
        "          n_str=grid[i,I_Column.p_Numeric_Normalizer].value\n",
        "          lambda_f= eval(n_str)\n",
        "      else:\n",
        "        n_str=grid[i,I_Column.p_FeatureKind].value     \n",
        "        lambda_f= eval(n_str)\n",
        "\n",
        "      if (grid[i,I_Column.p_Input].value==True):\n",
        "        # code_generator.append(\"input_lambdas.append(('\"+f_col+\"',\"+lambda_f+\"))\")\n",
        "        self.input_lambdas.append((f_col,lambda_f))\n",
        "      if (grid[i,I_Column.p_Label].value==True):\n",
        "        # code_generator.append(\"label_lambdas.append(('\"+f_col+\"',\"+lambda_f+\"))\")\n",
        "        self.label_lambdas.append((f_col,lambda_f))\n",
        "      if (grid[i,I_Column.p_future].value==True):\n",
        "        # code_generator.append(\"future_lambdas.append(('\"+f_col+\"',\"+lambda_f+\"))\")\n",
        "        self.future_lambdas.append((f_col,lambda_f))\n",
        "\n",
        "  def input_future_label_1d(self):\n",
        "    \"\"\"To create a mapping function according to the input, label, future lambdas\"\"\"\n",
        "    input_lambdas=self.input_lambdas\n",
        "    label_lambdas=self.label_lambdas\n",
        "    future_lambdas=self.future_lambdas\n",
        "\n",
        "    def transform(row):\n",
        "      i_result1=[tf.reshape(tf.cast(y(row[x]),tf.float64),[-1]) for (x, y) in input_lambdas ]\n",
        "      i_result2=[tf.reshape(tf.cast(y(row[x]),tf.float64),[-1]) for (x, y) in label_lambdas ]\n",
        "      i_result3=[tf.reshape(tf.cast(y(row[x]),tf.float64),[-1]) for (x, y) in future_lambdas ]\n",
        "\n",
        "      return tf.concat(i_result1+i_result2+i_result3,0)\n",
        "    return transform\n",
        "\n",
        "  def label_1d(self):\n",
        "    \"\"\"To create a mapping function according to the input lambdas.\"\"\"\n",
        "    label_lambdas=self.label_lambdas\n",
        "    def transform(row):\n",
        "      i_result2=[tf.reshape(tf.cast(y(row[x]),tf.float64),[-1]) for (x, y) in label_lambdas ]\n",
        "\n",
        "      return tf.concat(i_result2,0)\n",
        "    return transform\n",
        "\n",
        "  def future_1d(self):\n",
        "    \"\"\"To create a mapping function according to the future lambdas\"\"\"\n",
        "    future_lambdas=self.future_lambdas\n",
        "\n",
        "    def transform(row):\n",
        "      i_result2=[tf.reshape(tf.cast(y(row[x]),tf.float64),[-1]) for (x, y) in future_lambdas ]\n",
        "\n",
        "      return tf.concat(i_result2,0)\n",
        "    return transform\n",
        "\n",
        "  def post_windowing_split2_single(self):\n",
        "    \"\"\"\n",
        "      To split the data into \n",
        "      (input, label), when there is no future columns\n",
        "      ((input, future), label) ，when there are future columns\n",
        "    \"\"\"\n",
        "    future_target=self.future_target\n",
        "    v_labels=self.v_labels\n",
        "    single_step=self.single_step\n",
        "\n",
        "    def input_label_single(row):\n",
        "      i_input=row[:-future_target,:]\n",
        "      i_label=row[-1,-(v_labels)]\n",
        "      return i_input, i_label\n",
        "\n",
        "    return input_label_single\n",
        "\n",
        "\n",
        "  def post_windowing_split2(self):\n",
        "    \"\"\"\n",
        "      To split the data into \n",
        "      (input, label), when there is no future columns\n",
        "      ((input, future), label) ，when there are future columns\n",
        "    \"\"\"\n",
        "    future_target=self.future_target\n",
        "    v_labels=self.v_labels\n",
        "    single_step=self.single_step\n",
        "    def input_label(row):\n",
        "      i_input=row[:-future_target,:]\n",
        "      i_label=tf.reshape(row[-future_target:,-v_labels:],[-1]) # To reshape to a 1-d tensor\n",
        "      return i_input, i_label\n",
        "\n",
        "    return input_label\n",
        "\n",
        "  def post_windowing_split3_single(self):\n",
        "    \"\"\"\n",
        "      To split the data into \n",
        "      (input, label), when there is no future columns\n",
        "      ((input, future), label) ，when there are future columns\n",
        "    \"\"\"\n",
        "    future_target=self.future_target\n",
        "    v_labels=self.v_labels\n",
        "    v_future= self.v_future\n",
        "    single_step=self.single_step\n",
        "\n",
        "    def input_future_label_single(row):\n",
        "      i_input=row[:-future_target,:]\n",
        "      i_label=row[-1,-(v_labels+v_future):-v_future]\n",
        "      i_future=row[-1,-v_future:]\n",
        "      return (i_input,i_future), i_label\n",
        "\n",
        "    return input_future_label_single\n",
        "\n",
        "      \n",
        "  def post_windowing_split3(self):\n",
        "    \"\"\"\n",
        "      To split the data into \n",
        "      (input, label), when there is no future columns\n",
        "      ((input, future), label) ，when there are future columns\n",
        "    \"\"\"\n",
        "    future_target=self.future_target\n",
        "    v_labels=self.v_labels\n",
        "    v_future= self.v_future\n",
        "    single_step=self.single_step\n",
        "\n",
        "    def input_future_label(row):\n",
        "      i_input=row[:-future_target,:]\n",
        "      i_label=row[-future_target:,-(v_labels+v_future):-v_future] \n",
        "      # if (v_labels==1):\n",
        "      #   i_label=tf.reshape(i_label,[-1]) # To reshape to a 1-d tensor?\n",
        "      i_future=row[-future_target:,-v_future:]\n",
        "      return (i_input,i_future), i_label\n",
        "\n",
        "    return input_future_label\n",
        "\n",
        "  def df_to_windoweddataset(self, dataframe, shuffle=False):\n",
        "    \"\"\"\n",
        "    A utility method to create a windowed tf.data dataset from a Pandas Dataframe\n",
        "    \"\"\"\n",
        "    ds = tf.data.Dataset.from_tensor_slices(dict(dataframe))\n",
        "    if self.seq2seq_mode:\n",
        "      self.v_future=ds.map(self.future_1d()).element_spec.shape[-1]\n",
        "    else:\n",
        "      self.v_future=0\n",
        "\n",
        "    self.v_labels=ds.map(self.label_1d()).element_spec.shape[-1]\n",
        "    ds_map=ds.map(self.input_future_label_1d())\n",
        "    \n",
        "    # Feel free to play with shuffle buffer size\n",
        "    shuffle_buffer_size = len(dataframe)\n",
        "    # Total size of window is given by the number of steps to be considered\n",
        "    # before prediction time + steps that we want to forecast\n",
        "\n",
        "    total_size = self.past_history + self.future_target\n",
        "\n",
        "    # Selecting windows\n",
        "    data = ds_map.window(total_size, shift=1, drop_remainder=True)\n",
        "    data = data.flat_map(lambda k: k.batch(total_size))\n",
        "    if shuffle:\n",
        "      data = data.shuffle(shuffle_buffer_size, seed=42)\n",
        "\n",
        "    if self.seq2seq_mode:\n",
        "      # Extracting past features  + labels + future\n",
        "      if self.single_step:\n",
        "        data = data.map(self.post_windowing_split3_single())\n",
        "      else:\n",
        "        data = data.map(self.post_windowing_split3())\n",
        "    else:\n",
        "      if self.single_step:\n",
        "        data = data.map(self.post_windowing_split2_single())\n",
        "      else:\n",
        "        data = data.map(self.post_windowing_split2())\n",
        "\n",
        "    ds_4_train= data.batch(self.batch_size).prefetch(tf.data.experimental.AUTOTUNE)\n",
        "\n",
        "    return ds_4_train\n",
        "\n",
        "  def update_by_grid_periods(self):\n",
        "    self.past_history = self.grid_periods[0,1].value\n",
        "    self.future_target = self.grid_periods[1,1].value\n",
        "    self.batch_size = self.grid_periods[2,1].value # A small batch sized is used for demonstration purposes\n",
        "    self.single_step = self.grid_periods[3,1].value\n",
        "\n",
        "  def get_periods_grid(self):\n",
        "    if self.grid_periods is not None:\n",
        "      return self.grid_periods\n",
        "\n",
        "    a_past_history=widgets.Label(\"Past History Periods:\")\n",
        "    v_past_history = widgets.IntText(value=self.past_history)\n",
        "\n",
        "    a_future_target=widgets.Label(\"Future_target Periods:\")\n",
        "    v_future_target = widgets.IntText(value=self.future_target)\n",
        "\n",
        "    a_batch_size=widgets.Label(\"Batch Size:\")\n",
        "    v_batch_size = widgets.IntText(value=self.batch_size)\n",
        "\n",
        "    a_single_step=widgets.Label(\"Single Step?\")\n",
        "    v_single_step= widgets.Checkbox(value=self.single_step)\n",
        "\n",
        "    grid_periods = widgets.GridspecLayout(4, 5)\n",
        "\n",
        "    grid_periods[0,0]=a_past_history; grid_periods[0,1:]=v_past_history\n",
        "    grid_periods[1,0]=a_future_target; grid_periods[1,1:]=v_future_target\n",
        "    grid_periods[2,0]=a_batch_size; grid_periods[2,1:]=v_batch_size\n",
        "    grid_periods[3,0]=a_single_step; grid_periods[3,1:]=v_single_step\n",
        "\n",
        "    self.grid_periods=grid_periods\n",
        "\n",
        "    return grid_periods\n",
        "\n",
        "  @staticmethod\n",
        "  def create_vanilla_LSTM_model(input_shape, label_shape):\n",
        "    \"\"\"Slightly modified from Multi-Step model for a multivariate time series in https://www.tensorflow.org/tutorials/structured_data/time_series\"\"\"\n",
        "    multi_step_model = tf.keras.models.Sequential()\n",
        "    multi_step_model.add(tf.keras.layers.LSTM(32,\n",
        "                                              return_sequences=True,\n",
        "                                              input_shape=input_shape))\n",
        "    multi_step_model.add(tf.keras.layers.LSTM(16, activation='relu'))\n",
        "    multi_step_model.add(tf.keras.layers.Dense(label_shape))\n",
        "\n",
        "    multi_step_model.compile(optimizer=tf.keras.optimizers.RMSprop(clipvalue=1.0), loss='mae')\n",
        "\n",
        "    return multi_step_model\n",
        "\n",
        "  @staticmethod \n",
        "  def create_seq2seq_model(input_shape,label_shape, future_shape):\n",
        "    \"\"\"Slightly modified from Encoder/Decoder model of https://www.angioi.com/time-series-encoder-decoder-tensorflow/\"\"\"\n",
        "    latent_dim = 16\n",
        "\n",
        "    # First branch of the net is an lstm which finds an embedding for the past\n",
        "    past_inputs = tf.keras.Input(shape=input_shape, name='past_inputs')\n",
        "    # Encoding the past\n",
        "    encoder = tf.keras.layers.LSTM(latent_dim, return_state=True , name=\"lstm_encoder\")\n",
        "    encoder_outputs, state_h, state_c = encoder(past_inputs)\n",
        "\n",
        "    future_inputs = tf.keras.Input(shape=future_shape, name='future_inputs')\n",
        "    # Combining future inputs with recurrent branch output\n",
        "    decoder_lstm = tf.keras.layers.LSTM(latent_dim, return_sequences=True, name=\"lstm_decoder\")\n",
        "    x = decoder_lstm(future_inputs, \n",
        "                                  initial_state=[state_h, state_c])\n",
        "\n",
        "    x = tf.keras.layers.Dense(16, activation='relu')(x)\n",
        "    x = tf.keras.layers.Dense(16, activation='relu')(x)\n",
        "    output = tf.keras.layers.Dense(label_shape[-1], activation='relu')(x)\n",
        "\n",
        "    model = tf.keras.models.Model(inputs=[past_inputs,future_inputs], outputs=output)\n",
        "\n",
        "    optimizer = tf.keras.optimizers.Adam()\n",
        "    loss = tf.keras.losses.Huber()\n",
        "    model.compile(loss=loss, optimizer=optimizer, metrics=[\"mae\"])\n",
        "\n",
        "    return model\n",
        "\n",
        "  def train_by_model(self,model_fn, epochs=1):\n",
        "    \"\"\"\n",
        "    You may train by your provided model.\n",
        "    Depending whether the seq2seq model is needed,\n",
        "    you may provide model like vanilla LSTM model or Seq2Seq model\n",
        "    \"\"\"\n",
        "    train_ds = self.df_to_windoweddataset(self._df_train, shuffle=False)\n",
        "\n",
        "    if (dataframe_val is not None):\n",
        "      val_ds=None\n",
        "    else:\n",
        "      val_ds = self.df_to_windoweddataset(dataframe_val, shuffle=False)\n",
        "    # test_ds = self.df_to_windoweddataset(dataframe_test, shuffle=False)\n",
        "\n",
        "    if not self.seq2seq_mode:\n",
        "      input_shape=tuple(next(iter(train_ds))[0].shape[-2:].as_list())\n",
        "      label_shape=next(iter(train_ds))[1].shape[-1]\n",
        "      model = model_fn(input_shape, label_shape)\n",
        "    else:\n",
        "      input_shape=tuple(next(iter(train_ds))[0][0].shape[1:].as_list())\n",
        "      future_shape=tuple(next(iter(train_ds))[0][1].shape[1:].as_list())\n",
        "      label_shape=tuple(next(iter(train_ds))[1].shape[1:].as_list())\n",
        "      model = model_fn(input_shape, label_shape, future_shape)\n",
        "    \n",
        "    history = model.fit(train_ds, epochs=epochs, validation_data=val_ds)\n",
        "\n",
        "    return history\n",
        "\n",
        "  def train_by_builtin_model(self,epochs=1):\n",
        "    \"\"\"\n",
        "    You may train by built-in models, vanilla LSTM model or Seq2Seq model\n",
        "    \"\"\"\n",
        "    if not self.seq2seq_mode:\n",
        "      model_fn=TsEstimator.create_vanilla_LSTM_model\n",
        "    else:\n",
        "      model_fn=TsEstimator.create_seq2seq_model\n",
        "    \n",
        "    return self.train_by_model(model_fn, epochs)\n",
        "\n",
        "  def pairplot(self):\n",
        "    \"\"\"\n",
        "    To chart the combination of categorical columns.\n",
        "    In order to find the balance of the data, you may need to inspect in detail.\n",
        "    \"\"\"\n",
        "    if len(self.categorical_columns)>0:\n",
        "      sns.pairplot(self._df_all[self.categorical_columns], diag_kind=\"kde\")\n",
        "    else:\n",
        "      print(\"There are no categorical columns.\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vu77PzPQQWQW",
        "colab_type": "text"
      },
      "source": [
        "The basic preprocessing:\n",
        "\n",
        "1. Pre-windowing Transform : 1-> 1d vector (one-hot coding), embedding features (static, dynamic) (Label cannot be embedding)\n",
        "\n",
        "2. Windowing (to dictionary?) (for different lookback and forecast periods)\n",
        "\n",
        "3. Post-windowing Transform: ? Embedding Here?\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "_WhPYLoybkqn"
      },
      "source": [
        "# A Seq2Seq model demo"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "IaQ488uIbkq_"
      },
      "source": [
        "Let’s start with a practical example of a time series and look at [the UCI Bike Sharing Dataset](https://archive.ics.uci.edu/ml/datasets/bike+sharing+dataset); there we can find for each hour the amount of bikes rented by customers of a bike sharing service in Washington DC, together with other features such as whether a certain day was a national holiday, and which day of the week was it. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Ru3r2GFabkrC"
      },
      "source": [
        "## Your configues here"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "LPZDc0HXbkrF",
        "colab": {}
      },
      "source": [
        "def get_configs():\n",
        "  if not path.exists('bike_data/hour.csv'):\n",
        "    ! wget https://archive.ics.uci.edu/ml/machine-learning-databases/00275/Bike-Sharing-Dataset.zip\n",
        "    ! unzip Bike-Sharing-Dataset.zip -d bike_data\n",
        "\n",
        "  df = pd.read_csv('bike_data/hour.csv', index_col='instant')\n",
        "  cols_to_keep = [       \n",
        "    'cnt',\n",
        "    'temp',\n",
        "    'hum',\n",
        "    'windspeed',\n",
        "    'yr',\n",
        "    'mnth', \n",
        "    'hr', \n",
        "    'holiday', \n",
        "    'weekday', \n",
        "    'workingday'\n",
        "  ]\n",
        "  dataframe = df[cols_to_keep]\n",
        "  default_inputs=['cnt','temp','hum','windspeed']  # The default features list for inputs\n",
        "  default_labels=['cnt']  # The default features list for labels\n",
        "  default_future=list(set(cols_to_keep)-set(default_inputs))\n",
        "    \n",
        "  dataframe.head()\n",
        "\n",
        "  dataframe_train, dataframe_test = train_test_split(dataframe, test_size=0.2)\n",
        "  dataframe_train, dataframe_val = train_test_split(dataframe_train, test_size=0.2)\n",
        "  print(len(dataframe_train), 'train examples')\n",
        "  print(len(dataframe_val), 'validation examples')\n",
        "\n",
        "  past_history = 24 * 7 * 3 \n",
        "  future_target = 24 * 5\n",
        "  categories_limit=20\n",
        "  batch_size = 32\n",
        "  single_step=False\n",
        "\n",
        "  return dataframe, dataframe_train, dataframe_val, dataframe_test,default_inputs,default_labels,default_future, past_history, future_target,categories_limit, batch_size,single_step\n",
        "  # The order of the variables is very important. \n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "G224_dN-bkrV"
      },
      "source": [
        "## Create an estimator"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "TB0ZVugUbkrX",
        "colab": {}
      },
      "source": [
        "estimator=TsEstimator(get_configs)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "wggUdjRubkrh"
      },
      "source": [
        "## Inspect the data by categorical columns"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "qYBdWC1fbkri",
        "colab": {}
      },
      "source": [
        "estimator.pairplot()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "9yz1EZIZbkrl"
      },
      "source": [
        "## To create an interactive features grid"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "OvWy-Bz1bkrl"
      },
      "source": [
        "You may try the builder INTERACTIVELY."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "SSYWnM8Ebkrm",
        "colab": {}
      },
      "source": [
        "grid=estimator.get_feature_grid()\n",
        "grid"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "uLTvHK0xbkrp"
      },
      "source": [
        "<font color=red>**RERUN** the following cells once you change the above settings.</font>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "YJksIB6hbkrp",
        "colab": {}
      },
      "source": [
        "estimator.update_by_grid_features()\n",
        "\n",
        "assert(len(estimator.input_lambdas)>0)\n",
        "assert(len(estimator.label_lambdas)>0)\n",
        "# code_generator\n",
        "estimator.input_lambdas,estimator.label_lambdas, estimator.future_lambdas"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "3w1-6j6Ibkrs"
      },
      "source": [
        "## To create an interactive periods grid"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "1nZ--sC2bkrt",
        "colab": {}
      },
      "source": [
        "grid_periods=estimator.get_periods_grid()\n",
        "grid_periods"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "-cgGKiBFbkrw",
        "colab": {}
      },
      "source": [
        "estimator.update_by_grid_periods()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "U7hw-ArQbkrz"
      },
      "source": [
        "## Train the model by the built-in models\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "GnU3pWqxbkr0",
        "colab": {}
      },
      "source": [
        "estimator.train_by_builtin_model(1)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PG_6nILluUe_",
        "colab_type": "text"
      },
      "source": [
        "# A LSTM model demo"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Sv4hnQhemmbm",
        "colab_type": "text"
      },
      "source": [
        "Let’s start with a practical example of a time series and look at [the UCI Bike Sharing Dataset](https://archive.ics.uci.edu/ml/datasets/bike+sharing+dataset); there we can find for each hour the amount of bikes rented by customers of a bike sharing service in Washington DC, together with other features such as whether a certain day was a national holiday, and which day of the week was it. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Yj7tNk2ADpmn",
        "colab_type": "text"
      },
      "source": [
        "## Your configues here"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "j76-FytOc-r5",
        "colab_type": "text"
      },
      "source": [
        "**The weather dataset**\n",
        "This tutorial uses a <a href=\"https://www.bgc-jena.mpg.de/wetter/\" class=\"external\">[weather time series dataset</a> recorded by the <a href=\"https://www.bgc-jena.mpg.de\" class=\"external\">Max Planck Institute for Biogeochemistry</a>.\n",
        "\n",
        "This dataset contains 14 different features such as air temperature, atmospheric pressure, and humidity. These were collected every 10 minutes, beginning in 2003. For efficiency, you will use only the data collected between 2009 and 2016. This section of the dataset was prepared by François Chollet for his book [Deep Learning with Python](https://www.manning.com/books/deep-learning-with-python)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "REZ57BXCLdfG",
        "colab": {}
      },
      "source": [
        "def get_configs_lstm():\n",
        "  if not path.exists('/root/.keras/datasets/jena_climate_2009_2016.csv'):\n",
        "    zip_path = tf.keras.utils.get_file(\n",
        "      origin='https://storage.googleapis.com/tensorflow/tf-keras-datasets/jena_climate_2009_2016.csv.zip',\n",
        "      fname='jena_climate_2009_2016.csv.zip',\n",
        "      extract=True)\n",
        "    csv_path, _ = os.path.splitext(zip_path)\n",
        "    print(csv_path)\n",
        "\n",
        "  dataframe = pd.read_csv('/root/.keras/datasets/jena_climate_2009_2016.csv', index_col='Date Time')\n",
        "  default_inputs=['p (mbar)', 'T (degC)', 'rho (g/m**3)']  # The default features list for inputs\n",
        "  default_labels=['T (degC)'] # The default features list for labels\n",
        "  default_future=[]\n",
        "    \n",
        "  dataframe.head()\n",
        "\n",
        "  dataframe_train, dataframe_test = train_test_split(dataframe, test_size=0.2)\n",
        "  dataframe_train, dataframe_val = train_test_split(dataframe_train, test_size=0.2)\n",
        "  print(len(dataframe_train), 'train examples')\n",
        "  print(len(dataframe_val), 'validation examples')\n",
        "\n",
        "  past_history = 720 \n",
        "  future_target = 72\n",
        "  categories_limit=20\n",
        "  batch_size = 32\n",
        "  single_step=False\n",
        "\n",
        "  return dataframe, dataframe_train, dataframe_val, dataframe_test,default_inputs,default_labels,default_future, past_history, future_target,categories_limit, batch_size,single_step\n",
        "  # The order of the variables is very important. \n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZqPxNqGNQpLN",
        "colab_type": "text"
      },
      "source": [
        "## Create an estimator"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "580l-ZQhPP5a",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "estimator_lstm=TsEstimator(get_configs_lstm)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9IWdQBbSMXUL",
        "colab_type": "text"
      },
      "source": [
        "## Inspect the data by categorical columns"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "73VFzUHpMYph",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "estimator_lstm.pairplot()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "B7BshfvV0X1g",
        "colab_type": "text"
      },
      "source": [
        "## To create an interactive features grid"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d0grSPtD7QBv",
        "colab_type": "text"
      },
      "source": [
        "You may try the builder INTERACTIVELY."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "J_SWf5pRPYtn",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "grid_lstm=estimator_lstm.get_feature_grid()\n",
        "grid_lstm"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kj0RBVpb7b6N",
        "colab_type": "text"
      },
      "source": [
        "<font color=red>**RERUN** the following cells once you change the above settings.</font>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8PIYVldRC6sk",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "estimator_lstm.update_by_grid_features()\n",
        "\n",
        "assert(len(estimator_lstm.input_lambdas)>0)\n",
        "assert(len(estimator_lstm.label_lambdas)>0)\n",
        "# code_generator\n",
        "estimator_lstm.input_lambdas,estimator_lstm.label_lambdas, estimator_lstm.future_lambdas"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2U8jhUxcD3w6",
        "colab_type": "text"
      },
      "source": [
        "## To create an interactive periods grid"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IXUocwfWFF5R",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "grid_periods_lstm=estimator_lstm.get_periods_grid()\n",
        "grid_periods_lstm"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jsJDltTvNxAf",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "estimator_lstm.update_by_grid_periods()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "bBx4Xu0eTXWq"
      },
      "source": [
        "## Train the model by the built-in models\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uPb0ZwQrfCEo",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "estimator_lstm.seq2seq_mode"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Y8-wA2rWrVfL",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "estimator_lstm.train_by_builtin_model(1)\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}