examples/tutorial/03-Session-based-recsys.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "32581c84",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Copyright 2022 NVIDIA Corporation. All Rights Reserved.\n",
    "#\n",
    "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "# you may not use this file except in compliance with the License.\n",
    "# You may obtain a copy of the License at\n",
    "#\n",
    "#     http://www.apache.org/licenses/LICENSE-2.0\n",
    "#\n",
    "# Unless required by applicable law or agreed to in writing, software\n",
    "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "# See the License for the specific language governing permissions and\n",
    "# limitations under the License.\n",
    "# ==============================================================================\n",
    "\n",
    "# Each user is responsible for checking the content of datasets and the\n",
    "# applicable licenses and determining if suitable for the intended use."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18dd7307",
   "metadata": {},
   "source": [
    "<img src=\"https://developer.download.nvidia.com/notebooks/dlsw-notebooks/merlin_transformers4rec_tutorial-03-session-based-recsys/nvidia_logo.png\" style=\"width: 90px; float: right;\">\n",
    "\n",
    "# Session-based recommendation with Transformers4Rec\n",
    "## 1. Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d9bb25dd",
   "metadata": {},
   "source": [
    "In the previous notebook we went through our ETL pipeline with NVTabular library, and created sequential features to be used for training a session-based recommendation model. In this notebook we will learn:\n",
    "\n",
    "- Accelerating data loading of parquet files multiple features on PyTorch using NVTabular library\n",
    "- Training and evaluating an RNN-based (GRU) session-based recommendation model \n",
    "- Training and evaluating a Transformer architecture (XLNET) for session-based recommendation model\n",
    "- Integrate side information (additional features) into transformer architectures in order to improve recommendation accuracy"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a412e6de",
   "metadata": {},
   "source": [
    "## 2. Session-based Recommendation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ed7e8f1",
   "metadata": {},
   "source": [
    "Session-based recommendation, a sub-area of sequential recommendation, has been an important task in online services like e-commerce and news portals, where most users either browse anonymously or may have very distinct interests for different sessions. Session-Based Recommender Systems (SBRS) have\n",
    "been proposed to model the sequence of interactions within the current user session, where a session is a short sequence of user interactions typically bounded by user inactivity. They have recently gained popularity due to their ability to capture short-term and contextual user preferences towards items.\n",
    "\n",
    "\n",
    "Many methods have been proposed to leverage the sequence of interactions that occur during a session, including session-based k-NN algorithms like V-SkNN [1] and neural approaches like GRU4Rec [2]. In addition,  state of the art NLP approaches have inspired RecSys practitioners and researchers to leverage the self-attention mechanism and the Transformer-based architectures for sequential [3] and session-based recommendation [4]."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b401e7a9",
   "metadata": {},
   "source": [
    "## 3. Transformers4Rec Library"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c20a4de",
   "metadata": {},
   "source": [
    "In this tutorial, we introduce the [Transformers4Rec](https://github.com/NVIDIA-Merlin/Transformers4Rec) open-source library for sequential and session-based recommendation task.\n",
    "\n",
    "With Transformers4Rec we import from the HF Transformers NLP library the transformer architectures and their configuration classes. \n",
    "\n",
    "In addition, Transformers4Rec provides additional blocks necessary for recommendation, e.g., input features normalization and aggregation, and heads for recommendation and sequence classification/prediction. We also extend their Trainer class to allow for the evaluation with RecSys metrics.\n",
    "\n",
    "Here are some of the most important modules:\n",
    "\n",
    "- [TabularSequenceFeatures](https://nvidia-merlin.github.io/Transformers4Rec/stable/api/transformers4rec.torch.html#transformers4rec.torch.TabularSequenceFeatures) is the input block for sequential features. Based on a `Schema` and options set by the user, it dynamically creates all the necessary layers (e.g. embedding layers) to encode, normalize, and aggregate categorical and continuous features. It also allows to set the `masking` training approach (e.g. Causal LM, Masked LM).\n",
    "- [TransformerBlock](https://nvidia-merlin.github.io/Transformers4Rec/stable/api/transformers4rec.torch.html#transformers4rec.torch.TransformerBlock) class is the bridge that adapts HuggingFace Transformers for session-based and sequential-based recommendation models.\n",
    "- [SequentialBlock](https://nvidia-merlin.github.io/Transformers4Rec/stable/api/transformers4rec.torch.html#transformers4rec.torch.SequentialBlock) allows the definition of a model body as as sequence of layer (similarly to [torch.nn.sequential](https://pytorch.org/docs/stable/generated/torch.nn.Sequential.html)). It is designed to define our model as a sequence of layers and automatically setting the input shape of a layer from the output shape of the previous one.\n",
    "- [Head](https://nvidia-merlin.github.io/Transformers4Rec/stable/api/transformers4rec.torch.html#transformers4rec.torch.Head) class defines the head of a model.\n",
    "- [NextItemPredictionTask](https://nvidia-merlin.github.io/Transformers4Rec/stable/api/transformers4rec.torch.html#transformers4rec.torch.NextItemPredictionTask) is the class to support next item prediction task, combining a model body with a head.\n",
    "- [Trainer](https://nvidia-merlin.github.io/Transformers4Rec/stable/api/transformers4rec.torch.html#transformers4rec.torch.Trainer) extends the `Trainer` class from HF transformers and manages the model training and evaluation.\n",
    "\n",
    "You can check the [full documentation](https://nvidia-merlin.github.io/Transformers4Rec/stable/index.html) of Transformers4Rec if needed."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3282f17c",
   "metadata": {},
   "source": [
    "In Figure 1, we present a reference architecture that we are going to build with Transformers4Rec PyTorch API in this notebook. We are going to start using only `product-id` as input feature, but as you can notice in the figure, we can add additional categorical and numerical features later to improve recommendation accuracy, as shown in Section 3.2.4."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3078054f",
   "metadata": {},
   "source": [
    "![](_images/tf4rec_meta.png)  \n",
    "<p><center>Figure 1. Transformers4Rec meta-architecture.</center></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4345768c",
   "metadata": {},
   "source": [
    "### 3.1 Training an RNN-based Session-based Recommendation Model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "29f14981",
   "metadata": {},
   "source": [
    "In this section, we use a type of Recurrent Neural Networks (RNN) - the Gated Recurrent Unit (GRU)[5] - to do next-item prediction using a sequence of events (e.g., click, view, or purchase) per user in a given session. There is obviously some sequential patterns that we want to capture to provide more relevant recommendations. In our case, the input of the GRU layer is a representation of the user interaction, the internal GRU hidden state encodes a representation of the session based on past interactions and the outputs are the next-item predictions. Basically, for each item in a given session, we generate the output as the predicted preference of the items, i.e. the likelihood of being the next."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f4fc1b1",
   "metadata": {},
   "source": [
    "Figure 2 illustrates the logic of predicting next item in a given session. First, the product ids are embedded and fed as a sequence to a GRU layer, which outputs a representation than can be used to predict the next item. For the sake of simplicity, we treat the recommendation as a multi-class classification problem and use cross-entropy loss. In our first example, we use a GRU block instead of `Transformer block` (shown in the Figure 1)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "93ee2085",
   "metadata": {},
   "source": [
    "![](_images/gru_based.png)\n",
    "<p><center>Figure 2. Next item prediction with RNN.</center></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ebbdf01",
   "metadata": {},
   "source": [
    "#### 3.1.1 Import Libraries and Modules"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "c16e5278",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.8/dist-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n",
      "/usr/local/lib/python3.8/dist-packages/merlin/dtypes/mappings/tf.py:52: UserWarning: Tensorflow dtype mappings did not load successfully due to an error: No module named 'tensorflow'\n",
      "  warn(f\"Tensorflow dtype mappings did not load successfully due to an error: {exc.msg}\")\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "import glob\n",
    "\n",
    "import torch \n",
    "import transformers4rec.torch as tr\n",
    "\n",
    "from transformers4rec.torch.ranking_metric import NDCGAt, RecallAt\n",
    "from transformers4rec.torch.utils.examples_utils import wipe_memory"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "94952739",
   "metadata": {},
   "source": [
    "##### Instantiates Schema object by reading the save trained parquet file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "9735530e",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:149: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "from merlin.schema import Schema\n",
    "from merlin.io import Dataset\n",
    "\n",
    "INPUT_DATA_DIR = os.environ.get(\"INPUT_DATA_DIR\", \"/workspace/data\")\n",
    "\n",
    "train = Dataset(os.path.join(INPUT_DATA_DIR, \"processed_nvt/part_0.parquet\"))\n",
    "schema = train.schema\n",
    "schema = schema.select_by_name(['product_id-list'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "41681abc",
   "metadata": {},
   "source": [
    "Transformers4Rec library relies on `Schema` object in `TabularSequenceFeatures` that takes the input features as input and create all the necessary layers to process and aggregate them. As you can see below, the `schema.pb` is a protobuf text file contains features metadata, including statistics about features such as cardinality, min and max values and also tags based on their characteristics and dtypes (e.g., `categorical`, `continuous`, `list`, `item_id`). We can tag our target column and even add the prediction task such as `binary`, `regression` or `multiclass` as tags for the target column in the `schema.pb` file. The `Schema` provides a standard representation for metadata that is useful when training machine learning or deep learning models.\n",
    "\n",
    "The metadata information loaded from `Schema` and their tags are used to automatically set the parameters of Transformers4rec models. Certain Transformers4rec modules have a `from_schema()` method to instantiate their parameters and layers from protobuf text file respectively. \n",
    "\n",
    "Although in this tutorial we are defining the `Schema` manually, the next NVTabular release is going to generate the schema with appropriate types and tags automatically from the preprocessing workflow, allowing the user to set additional feaure tags if needed."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4adfe7dc",
   "metadata": {},
   "source": [
    "##### Defining the input block: `TabularSequenceFeatures`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "101c6a2a",
   "metadata": {},
   "source": [
    "We define our input block using `TabularSequenceFeatures` class. The `from_schema()` method directly parses the schema and accepts sequential and non-sequential features. Based on the `Schema` and some user-defined options, the categorical features are represented by embeddings and numerical features can be represented as continuous scalars or by a technique named Soft One-Hot embeddings (more info in our [paper's online appendix](https://github.com/NVIDIA-Merlin/publications/blob/main/2021_acm_recsys_transformers4rec/Appendices/Appendix_A-Techniques_used_in_Transformers4Rec_Meta-Architecture.md)). \n",
    "\n",
    "The embedding features can optionally be normalized (`layer_norm=True`). Data augmentation methods like \"Stochastic Swap Noise\" (`pre=\"stochastic-swap-noise\"`) and `aggregation` opptions (like `concat` and `elementwise-sum`) are also available. The continuous features can also be combined and projected by MLP layers by setting `continuous_projection=[dim]`. Finally, the `max_sequence_length` argument defines the maximum sequence length of our sequential input.\n",
    "\n",
    "Another important argument is the `masking` method, which sets the training approach. See Section 3.2.2 for details on this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "e7b122e4",
   "metadata": {},
   "outputs": [],
   "source": [
    "sequence_length = 20\n",
    "inputs = tr.TabularSequenceFeatures.from_schema(\n",
    "        schema,\n",
    "        max_sequence_length= sequence_length,\n",
    "        masking = 'causal',\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3fa46a58",
   "metadata": {},
   "source": [
    "##### Connecting the blocks with `SequentialBlock`\n",
    "\n",
    "The `SequentialBlock` creates a pipeline by connecting the building blocks in a serial way, so that the input shape of one block is inferred from the output of the previous block. In this example, the `TabularSequenceFeatures` object is followed by an MLP projection layer, which feeds data to a GRU block."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "cd66b9dc",
   "metadata": {},
   "outputs": [],
   "source": [
    "d_model = 128\n",
    "body = tr.SequentialBlock(\n",
    "        inputs,\n",
    "        tr.MLPBlock([d_model]),\n",
    "        tr.Block(torch.nn.GRU(input_size=d_model, hidden_size=d_model, num_layers=1), [None, 20, d_model])\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "390ca27a",
   "metadata": {},
   "source": [
    "##### Item Prediction head and tying embeddings"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3234f8be",
   "metadata": {},
   "source": [
    "In our experiments published in our [ACM RecSys'21 paper](https://github.com/NVIDIA-Merlin/publications/blob/main/2021_acm_recsys_transformers4rec/recsys21_transformers4rec_paper.pdf) [8], we used the next item prediction head, which projects the output of the RNN/Transformer block to the items space, followed by a softmax layer to produce the relevance scores over all items. For the output layer we provide the `Tying Embeddings` technique (`weight_tying`). It was proposed originally by the NLP community to tie the weights of the input (item id) embedding matrix with the output projection layer, showed to be a very effective technique in extensive experimentation for competitions and empirical analysis (for more details see our [paper](https://github.com/NVIDIA-Merlin/publications/blob/main/2021_acm_recsys_transformers4rec/recsys21_transformers4rec_paper.pdf) and its online [appendix](https://github.com/NVIDIA-Merlin/publications/blob/main/2021_acm_recsys_transformers4rec/Appendices/Appendix_A-Techniques_used_in_Transformers4Rec_Meta-Architecture.md)). In practice, such technique helps the network to learn faster item embeddings even for rare items, reduces the number of parameters for large item cardinalities and enables Approximate Nearest Neighbours (ANN) search on inference, as the predictions can be obtained by a dot product between the model output and the item embeddings."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db6da77f",
   "metadata": {},
   "source": [
    "Next, we link the transformer-body to the inputs and the prediction tasks to get the final PyTorch `Model` class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "99212ad0",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Projecting inputs of NextItemPredictionTask to'64' As weight tying requires the input dimension '128' to be equal to the item-id embedding dimension '64'\n"
     ]
    }
   ],
   "source": [
    "head = tr.Head(\n",
    "    body,\n",
    "    tr.NextItemPredictionTask(weight_tying=True, \n",
    "                              metrics=[NDCGAt(top_ks=[10, 20], labels_onehot=True),  \n",
    "                                       RecallAt(top_ks=[10, 20], labels_onehot=True)]),\n",
    ")\n",
    "model = tr.Model(head)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "56d122ee",
   "metadata": {},
   "source": [
    "##### Define a Dataloader function from schema"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a2b1359",
   "metadata": {},
   "source": [
    "We use optimized NVTabular PyTorch Dataloader which has the following benefits:\n",
    "- removing bottlenecks from dataloading by processing large chunks of data at a time instead iterating by row\n",
    "- processing datasets that don’t fit within the GPU or CPU memory by streaming from the disk\n",
    "- reading data directly into the GPU memory and removing CPU-GPU communication\n",
    "- preparing batch asynchronously into the GPU to avoid CPU-GPU communication\n",
    "- supporting commonly used formats such as parquet\n",
    "- having native support to sparse sequential features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "7e50313a",
   "metadata": {},
   "outputs": [],
   "source": [
    "from transformers4rec.torch.utils.data_utils import MerlinDataLoader\n",
    "x_cat_names, x_cont_names = ['product_id-list_seq'], []\n",
    "\n",
    "# dictionary representing max sequence length for column\n",
    "sparse_features_max = {\n",
    "    fname: sequence_length\n",
    "    for fname in x_cat_names + x_cont_names\n",
    "}\n",
    "\n",
    "def get_dataloader(data_path, batch_size=128):\n",
    "        loader = MerlinDataLoader.from_schema(\n",
    "            schema,\n",
    "            data_path,\n",
    "            batch_size,\n",
    "            max_sequence_length=sequence_length,\n",
    "            shuffle=False,\n",
    "        )\n",
    "        return loader"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "21a270a2",
   "metadata": {},
   "source": [
    "##### Daily Fine-Tuning: Training over a time window"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bfc043cb",
   "metadata": {},
   "source": [
    "Now that the model is defined, we are going to launch training. For that, Transfromers4rec extends the HF Transformers `Trainer` class to adapt the evaluation loop for session-based recommendation task and the calculation of ranking metrics. \n",
    "The original HF `Trainer.train()` method is not overloaded, meaning that we leverage the efficient training implementation from HF transformers library, which manages for example half-precision (FP16) training."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30302ec2",
   "metadata": {},
   "source": [
    "##### Set training arguments"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "e6680322",
   "metadata": {},
   "outputs": [],
   "source": [
    "from transformers4rec.config.trainer import T4RecTrainingArguments\n",
    "from transformers4rec.torch import Trainer\n",
    "\n",
    "#Set arguments for training \n",
    "train_args = T4RecTrainingArguments(local_rank = -1, \n",
    "                                    dataloader_drop_last = False,\n",
    "                                    report_to = [],   #set empty list to avoid logging metrics to Weights&Biases\n",
    "                                    gradient_accumulation_steps = 1,\n",
    "                                    per_device_train_batch_size = 256, \n",
    "                                    per_device_eval_batch_size = 32,\n",
    "                                    output_dir = \"./tmp\", \n",
    "                                    max_sequence_length=sequence_length,\n",
    "                                    learning_rate=0.00071,\n",
    "                                    num_train_epochs=3,\n",
    "                                    logging_steps=200,\n",
    "                                   )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac896a68",
   "metadata": {},
   "source": [
    "##### Instantiate the Trainer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "360b2e18",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Instantiate the T4Rec Trainer, which manages training and evaluation\n",
    "trainer = Trainer(\n",
    "    model=model,\n",
    "    args=train_args,\n",
    "    schema=schema,\n",
    "    compute_metrics=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c913243",
   "metadata": {},
   "source": [
    "Define the output folder of the processed parquet files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "d4221688",
   "metadata": {},
   "outputs": [],
   "source": [
    "OUTPUT_DIR = os.environ.get(\"OUTPUT_DIR\", \"/workspace/data/sessions_by_day\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48aec2fb",
   "metadata": {},
   "source": [
    "##### Model finetuning and incremental evaluation\n",
    "Training models incrementally, e.g. fine-tuning pre-trained models with new data over time is a common practice in industry to scale to the large streaming data been generated every data. Furthermore, it is common to evaluate recommendation models on data that came after the one used to train the models, for a more realistic evaluation.\n",
    "\n",
    "Here, we use a loop that to conduct a time-based finetuning, by iteratively training and evaluating using a sliding time window as follows: At each iteration, we use training data of a specific time index <i>t</i> to train the model then we evaluate on the validation data of next index <i>t + 1</i>. We set the start time to 1 and end time to 4."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "26917fcc",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "***** Running training *****\n",
      "  Num examples = 112128\n",
      "  Num Epochs = 3\n",
      "  Instantaneous batch size per device = 256\n",
      "  Total train batch size (w. parallel, distributed & accumulation) = 256\n",
      "  Gradient Accumulation steps = 1\n",
      "  Total optimization steps = 1314\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "********************\n",
      "Launch training for day 1 are:\n",
      "********************\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='1314' max='1314' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [1314/1314 00:52, Epoch 3/3]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>Step</th>\n",
       "      <th>Training Loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>200</td>\n",
       "      <td>9.759100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>400</td>\n",
       "      <td>8.757200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>600</td>\n",
       "      <td>8.920300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>800</td>\n",
       "      <td>8.442900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1000</td>\n",
       "      <td>8.751000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1200</td>\n",
       "      <td>8.367600</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to ./tmp/checkpoint-500\n",
      "Trainer.model is not a `PreTrainedModel`, only saving its state dict.\n",
      "Saving model checkpoint to ./tmp/checkpoint-1000\n",
      "Trainer.model is not a `PreTrainedModel`, only saving its state dict.\n",
      "\n",
      "\n",
      "Training completed. Do not forget to share your model on huggingface.co/models =)\n",
      "\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='1285' max='415' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [415/415 03:57]\n",
       "    </div>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "********************\n",
      "Eval results for day 2 are:\t\n",
      "\n",
      "********************\n",
      "\n",
      " eval_/loss = 8.848333358764648\n",
      " eval_/next-item/ndcg_at_10 = 0.040753647685050964\n",
      " eval_/next-item/ndcg_at_20 = 0.049305129796266556\n",
      " eval_/next-item/recall_at_10 = 0.07925495505332947\n",
      " eval_/next-item/recall_at_20 = 0.11349068582057953\n",
      " eval_runtime = 43.1016\n",
      " eval_samples_per_second = 308.109\n",
      " eval_steps_per_second = 9.628\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "***** Running training *****\n",
      "  Num examples = 106240\n",
      "  Num Epochs = 3\n",
      "  Instantaneous batch size per device = 256\n",
      "  Total train batch size (w. parallel, distributed & accumulation) = 256\n",
      "  Gradient Accumulation steps = 1\n",
      "  Total optimization steps = 1245\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "********************\n",
      "Launch training for day 2 are:\n",
      "********************\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='1245' max='1245' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [1245/1245 00:51, Epoch 3/3]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>Step</th>\n",
       "      <th>Training Loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>200</td>\n",
       "      <td>8.940700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>400</td>\n",
       "      <td>8.468400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>600</td>\n",
       "      <td>8.670800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>800</td>\n",
       "      <td>8.197200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1000</td>\n",
       "      <td>8.498300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1200</td>\n",
       "      <td>8.065300</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to ./tmp/checkpoint-500\n",
      "Trainer.model is not a `PreTrainedModel`, only saving its state dict.\n",
      "Saving model checkpoint to ./tmp/checkpoint-1000\n",
      "Trainer.model is not a `PreTrainedModel`, only saving its state dict.\n",
      "\n",
      "\n",
      "Training completed. Do not forget to share your model on huggingface.co/models =)\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "********************\n",
      "Eval results for day 3 are:\t\n",
      "\n",
      "********************\n",
      "\n",
      " eval_/loss = 8.614578247070312\n",
      " eval_/next-item/ndcg_at_10 = 0.05164230614900589\n",
      " eval_/next-item/ndcg_at_20 = 0.0637783482670784\n",
      " eval_/next-item/recall_at_10 = 0.09867884963750839\n",
      " eval_/next-item/recall_at_20 = 0.1468765288591385\n",
      " eval_runtime = 35.8932\n",
      " eval_samples_per_second = 342.349\n",
      " eval_steps_per_second = 10.698\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "***** Running training *****\n",
      "  Num examples = 97792\n",
      "  Num Epochs = 3\n",
      "  Instantaneous batch size per device = 256\n",
      "  Total train batch size (w. parallel, distributed & accumulation) = 256\n",
      "  Gradient Accumulation steps = 1\n",
      "  Total optimization steps = 1146\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "********************\n",
      "Launch training for day 3 are:\n",
      "********************\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='1146' max='1146' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [1146/1146 00:47, Epoch 3/3]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>Step</th>\n",
       "      <th>Training Loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>200</td>\n",
       "      <td>8.620800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>400</td>\n",
       "      <td>8.233000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>600</td>\n",
       "      <td>8.173500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>800</td>\n",
       "      <td>7.982100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1000</td>\n",
       "      <td>7.933700</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to ./tmp/checkpoint-500\n",
      "Trainer.model is not a `PreTrainedModel`, only saving its state dict.\n",
      "Saving model checkpoint to ./tmp/checkpoint-1000\n",
      "Trainer.model is not a `PreTrainedModel`, only saving its state dict.\n",
      "\n",
      "\n",
      "Training completed. Do not forget to share your model on huggingface.co/models =)\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "********************\n",
      "Eval results for day 4 are:\t\n",
      "\n",
      "********************\n",
      "\n",
      " eval_/loss = 8.204485893249512\n",
      " eval_/next-item/ndcg_at_10 = 0.06745054572820663\n",
      " eval_/next-item/ndcg_at_20 = 0.08173713833093643\n",
      " eval_/next-item/recall_at_10 = 0.12599845230579376\n",
      " eval_/next-item/recall_at_20 = 0.18262046575546265\n",
      " eval_runtime = 60.7599\n",
      " eval_samples_per_second = 255.958\n",
      " eval_steps_per_second = 7.999\n",
      "CPU times: user 2min 52s, sys: 1min 5s, total: 3min 57s\n",
      "Wall time: 4min 55s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "start_time_window_index = 1\n",
    "final_time_window_index = 4\n",
    "for time_index in range(start_time_window_index, final_time_window_index):\n",
    "    # Set data \n",
    "    time_index_train = time_index\n",
    "    time_index_eval = time_index + 1\n",
    "    train_paths = glob.glob(os.path.join(OUTPUT_DIR, f\"{time_index_train}/train.parquet\"))\n",
    "    eval_paths = glob.glob(os.path.join(OUTPUT_DIR, f\"{time_index_eval}/valid.parquet\"))\n",
    "    \n",
    "    # Initialize dataloaders\n",
    "    trainer.train_dataloader = get_dataloader(train_paths, train_args.per_device_train_batch_size)\n",
    "    trainer.eval_dataloader = get_dataloader(eval_paths, train_args.per_device_eval_batch_size)\n",
    "    \n",
    "    # Train on day related to time_index \n",
    "    print('*'*20)\n",
    "    print(\"Launch training for day %s are:\" %time_index)\n",
    "    print('*'*20 + '\\n')\n",
    "    trainer.reset_lr_scheduler()\n",
    "    trainer.train()\n",
    "    trainer.state.global_step +=1\n",
    "    \n",
    "    # Evaluate on the following day\n",
    "    train_metrics = trainer.evaluate(metric_key_prefix='eval')\n",
    "    print('*'*20)\n",
    "    print(\"Eval results for day %s are:\\t\" %time_index_eval)\n",
    "    print('\\n' + '*'*20 + '\\n')\n",
    "    for key in sorted(train_metrics.keys()):\n",
    "        print(\" %s = %s\" % (key, str(train_metrics[key]))) \n",
    "    wipe_memory()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "56c51ec2",
   "metadata": {},
   "source": [
    "Let's write out model evaluation accuracy results to a text file to compare model at the end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "4e0f2335",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "with open(\"results.txt\", 'w') as f: \n",
    "    f.write('GRU accuracy results:')\n",
    "    f.write('\\n')\n",
    "    for key, value in  model.compute_metrics().items(): \n",
    "        f.write('%s:%s\\n' % (key, value.item()))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "41bbb338",
   "metadata": {
    "tags": []
   },
   "source": [
    "#### Metrics"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f9fe5d5a",
   "metadata": {},
   "source": [
    "We have extended the HuggingFace transformers Trainer class (PyTorch only) to support evaluation of RecSys metrics. The following information\n",
    "retrieval metrics are used to compute the Top-20 accuracy of recommendation lists containing all items: <br> \n",
    "- **Normalized Discounted Cumulative Gain (NDCG@20):** NDCG accounts for rank of the relevant item in the recommendation list and is a more fine-grained metric than HR, which only verifies whether the relevant item is among the top-k items.\n",
    "\n",
    "- **Hit Rate (HR@20)**: Also known as `Recall@n` when there is only one relevant item in the recommendation list. HR just verifies whether the relevant item is among the top-n items."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef9a89f7",
   "metadata": {},
   "source": [
    "##### Restart the kernel to free our GPU memory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "522961d2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'status': 'ok', 'restart': True}"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import IPython\n",
    "app = IPython.Application.instance()\n",
    "app.kernel.do_shutdown(True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d453614f",
   "metadata": {},
   "source": [
    "At this stage if the kernel does not restart automatically, we expect you to manually restart the kernel to free GPU memory so that you can move on to the next session-based model training with a SOTA deep learning Transformer-based model, [XLNet](https://arxiv.org/pdf/1906.08237.pdf)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50c3a8c0",
   "metadata": {},
   "source": [
    "### 3.2. Training a Transformer-based Session-based Recommendation Model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "433c39b3",
   "metadata": {},
   "source": [
    "#### 3.2.1 What's Transformers?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "19f1f53e",
   "metadata": {},
   "source": [
    "The Transformer is a competitive alternative to the models using Recurrent Neural Networks (RNNs) for a range of sequence modeling tasks. The Transformer architecture [6] was introduced as a novel architecture in NLP domain that aims to solve sequence-to-sequence tasks relying entirely on self-attention mechanism to compute representations of its input and output. Hence, the Transformer overperforms RNNs with their three mechanisms: \n",
    "\n",
    "- Non-sequential: Transformers network is parallelized where as RNN computations are inherently sequential. That resulted in significant speed-up in the training time.\n",
    "- Self-attention mechanisms: Transformers rely entirely on self-attention mechanisms that directly model relationships between all item-ids in a sequence.  \n",
    "- Positional encodings: A representation of the location or “position” of items in a sequence which is used to give the order context to the model architecture."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "46878729",
   "metadata": {},
   "source": [
    "![](_images/transformer_vs_rnn.png)\n",
    "<p><center> Figure 3. Transformer vs vanilla RNN.</center></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "574e6f22",
   "metadata": {},
   "source": [
    "Figure 4 illustrates the differences of Transformer (self-attention based) and a vanilla RNN architecture. As we see, RNN cannot be parallelized because it uses sequential processing over time (notice the sequential path from previous cells to the current one). On the other hand, the Transformer is a more powerful architecture because the self-attention mechanism is capable of representing dependencies within the sequence of tokens, favors parallel processing and handle longer sequences.\n",
    "\n",
    "As illustrated in the [Attention is All You Need](https://arxiv.org/pdf/1706.03762.pdf) paper, the original transformer model is made up of an encoder and decoder where each is a stack we can call a transformer block. In Transformers4Rec architectures we use the encoder block of transformer architecture."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d85f6a3e",
   "metadata": {},
   "source": [
    "![](_images/encoder.png)\n",
    "<p><center> Figure 4. Encoder block of the Transformer Architecture.</center></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43b84355",
   "metadata": {},
   "source": [
    "#### 3.2.2. XLNet"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e695834c",
   "metadata": {},
   "source": [
    "Here, we use XLNet [10] as the Transformer block in our architecture. It was originally proposed to be trained with the *Permutation Language Modeling (PLM)*  technique, that combines the advantages of autoregressive (Causal LM) and autoencoding (Masked LM). Although, we found out in our [paper](https://github.com/NVIDIA-Merlin/publications/blob/main/2021_acm_recsys_transformers4rec/recsys21_transformers4rec_paper.pdf) [8] that the *Masked Language Model (MLM)* approach worked better than PLM for the small sequences in session-based recommendation, thus we use MLM for this example. MLM was introduced in *BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding* paper [8]. \n",
    "\n",
    "Figure 5 illustrates the causal language modeling (LM) and masked LM. In this example, we use in causal LM for RNN masked LM for XLNet. Causal LM is the task of predicting the token following a sequence of tokens, where the model only attends to the left context, i.e. models the probability of a token given the previous tokens in a sentence [7]. On the other hand, the MLM randomly masks some of the tokens from the input sequence, and the objective is to predict the original vocabulary id of the masked word based only on its bi-directional context. When we train with MLM, the Transformer layer is also allowed to use positions on the right (future information) during training. During inference, all past items are visible for the Transformer layer, which tries to predict the next item. It performs a type of data augmentation, by masking different positions of the sequences in each training epoch."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a964b124",
   "metadata": {},
   "source": [
    "![](_images/masking.png)\n",
    "<p><center>Figure 5. Causal and Masked Language Model masking methods.</center></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c758f0c8",
   "metadata": {},
   "source": [
    "####  3.2.3 Train XLNET for Next Item Prediction"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "95011461",
   "metadata": {},
   "source": [
    "Now we are going to define an architecture for next-item prediction using the XLNET architecture."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "b194a0e1",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.8/dist-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n",
      "/usr/local/lib/python3.8/dist-packages/merlin/dtypes/mappings/tf.py:52: UserWarning: Tensorflow dtype mappings did not load successfully due to an error: No module named 'tensorflow'\n",
      "  warn(f\"Tensorflow dtype mappings did not load successfully due to an error: {exc.msg}\")\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "import glob\n",
    "\n",
    "import torch \n",
    "import transformers4rec.torch as tr\n",
    "from transformers4rec.torch.ranking_metric import NDCGAt, RecallAt\n",
    "from transformers4rec.torch.utils.examples_utils import wipe_memory"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99583e60",
   "metadata": {},
   "source": [
    "As we did above, we start with defining our schema object and selecting only the `product_id` feature for training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "59738175",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:149: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "from merlin.schema import Schema\n",
    "from merlin.io import Dataset\n",
    "\n",
    "INPUT_DATA_DIR = os.environ.get(\"INPUT_DATA_DIR\", \"/workspace/data\")\n",
    "\n",
    "train = Dataset(os.path.join(INPUT_DATA_DIR, \"processed_nvt/part_0.parquet\"))\n",
    "schema = train.schema\n",
    "schema = schema.select_by_name(['product_id-list'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "529dc88d",
   "metadata": {},
   "source": [
    "##### Define Input block\n",
    "Here we instantiate `TabularSequenceFeatures` from the feature schema and set `masking=\"mlm\"` to use MLM as training method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "b6595328",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "#Input \n",
    "sequence_length, d_model = 20, 192\n",
    "# Define input module to process tabular input-features and to prepare masked inputs\n",
    "inputs= tr.TabularSequenceFeatures.from_schema(\n",
    "    schema,\n",
    "    max_sequence_length=sequence_length,\n",
    "    d_output=d_model,\n",
    "    masking=\"mlm\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26cedff1",
   "metadata": {},
   "source": [
    "We have inherited the original `XLNetConfig` class of HF transformers with some default arguments in the `build()` method. Here we use it to instantiate an XLNET model according to the arguments (`d_model`, `n_head`, etc.), defining the model architecture.\n",
    "\n",
    "The `TransformerBlock` class supports HF Transformers for session-based and sequential-based recommendation models. `NextItemPredictionTask` is the class to support next item prediction task, encapsulating the corresponding heads and loss."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "b99c7472",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Projecting inputs of NextItemPredictionTask to'64' As weight tying requires the input dimension '192' to be equal to the item-id embedding dimension '64'\n"
     ]
    }
   ],
   "source": [
    "# Define XLNetConfig class and set default parameters for HF XLNet config  \n",
    "transformer_config = tr.XLNetConfig.build(\n",
    "    d_model=d_model, n_head=4, n_layer=2, total_seq_length=sequence_length\n",
    ")\n",
    "# Define the model block including: inputs, masking, projection and transformer block.\n",
    "body = tr.SequentialBlock(\n",
    "    inputs, tr.MLPBlock([192]), tr.TransformerBlock(transformer_config, masking=inputs.masking)\n",
    ")\n",
    "\n",
    "# Define the head for to next item prediction task \n",
    "head = tr.Head(\n",
    "    body,\n",
    "    tr.NextItemPredictionTask(weight_tying=True,\n",
    "                              metrics=[NDCGAt(top_ks=[10, 20], labels_onehot=True),  \n",
    "                                       RecallAt(top_ks=[10, 20], labels_onehot=True)]),\n",
    ")\n",
    "\n",
    "# Get the end-to-end Model class \n",
    "model = tr.Model(head)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4a0e8ac1",
   "metadata": {},
   "source": [
    "**Set training arguments**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "617f364a",
   "metadata": {},
   "source": [
    "Among the training arguments you can set the `data_loader_engine` to automatically instantiate the dataloader based on the schema, rather than instantiating the data loader manually like we did for the RNN example. The default value is `\"merlin\"` for optimized GPU-based data-loading. Optionally the PyarrowDataLoader (`\"pyarrow\"`) can also be used as a basic option, but it is slower and works only for small datasets, as the full data is loaded into CPU memory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "e6b0822f",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from transformers4rec.config.trainer import T4RecTrainingArguments\n",
    "from transformers4rec.torch import Trainer\n",
    "\n",
    "#Set arguments for training \n",
    "training_args = T4RecTrainingArguments(\n",
    "            output_dir=\"./tmp\",\n",
    "            max_sequence_length=20,\n",
    "            data_loader_engine='merlin',\n",
    "            num_train_epochs=3, \n",
    "            dataloader_drop_last=False,\n",
    "            per_device_train_batch_size = 256,\n",
    "            per_device_eval_batch_size = 32,\n",
    "            gradient_accumulation_steps = 1,\n",
    "            learning_rate=0.000666,\n",
    "            report_to = [],\n",
    "            logging_steps=200,\n",
    "        )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b33656f",
   "metadata": {},
   "source": [
    "**Instantiate the trainer**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "eae98133",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Instantiate the T4Rec Trainer, which manages training and evaluation\n",
    "trainer = Trainer(\n",
    "    model=model,\n",
    "    args=training_args,\n",
    "    schema=schema,\n",
    "    compute_metrics=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78b181f2",
   "metadata": {},
   "source": [
    "Define the output folder of the processed parquet files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "13e766de",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "OUTPUT_DIR = os.environ.get(\"OUTPUT_DIR\", \"/workspace/data/sessions_by_day\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b41701d1",
   "metadata": {},
   "source": [
    "Now, we do time-based fine-tuning the model by iteratively training and evaluating using a sliding time window, like we did for the RNN example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "ed0051dd",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "********************\n",
      "Launch training for day 1 are:\n",
      "********************\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "***** Running training *****\n",
      "  Num examples = 112128\n",
      "  Num Epochs = 3\n",
      "  Instantaneous batch size per device = 256\n",
      "  Total train batch size (w. parallel, distributed & accumulation) = 256\n",
      "  Gradient Accumulation steps = 1\n",
      "  Total optimization steps = 1314\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='1314' max='1314' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [1314/1314 00:52, Epoch 3/3]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>Step</th>\n",
       "      <th>Training Loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>200</td>\n",
       "      <td>9.978000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>400</td>\n",
       "      <td>9.067800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>600</td>\n",
       "      <td>8.780000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>800</td>\n",
       "      <td>8.656200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1000</td>\n",
       "      <td>8.566800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1200</td>\n",
       "      <td>8.511600</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to ./tmp/checkpoint-500\n",
      "Trainer.model is not a `PreTrainedModel`, only saving its state dict.\n",
      "Saving model checkpoint to ./tmp/checkpoint-1000\n",
      "Trainer.model is not a `PreTrainedModel`, only saving its state dict.\n",
      "\n",
      "\n",
      "Training completed. Do not forget to share your model on huggingface.co/models =)\n",
      "\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='1285' max='415' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [415/415 04:01]\n",
       "    </div>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "***** Running training *****\n",
      "  Num examples = 106240\n",
      "  Num Epochs = 3\n",
      "  Instantaneous batch size per device = 256\n",
      "  Total train batch size (w. parallel, distributed & accumulation) = 256\n",
      "  Gradient Accumulation steps = 1\n",
      "  Total optimization steps = 1245\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "********************\n",
      "Eval results for day 2 are:\t\n",
      "\n",
      "********************\n",
      "\n",
      " eval_/loss = 8.715866088867188\n",
      " eval_/next-item/ndcg_at_10 = 0.050047338008880615\n",
      " eval_/next-item/ndcg_at_20 = 0.06090249866247177\n",
      " eval_/next-item/recall_at_10 = 0.09584496170282364\n",
      " eval_/next-item/recall_at_20 = 0.13875272870063782\n",
      " eval_runtime = 45.3102\n",
      " eval_samples_per_second = 293.091\n",
      " eval_steps_per_second = 9.159\n",
      "********************\n",
      "Launch training for day 2 are:\n",
      "********************\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='1245' max='1245' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [1245/1245 00:48, Epoch 3/3]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>Step</th>\n",
       "      <th>Training Loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>200</td>\n",
       "      <td>8.646500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>400</td>\n",
       "      <td>8.534100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>600</td>\n",
       "      <td>8.395500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>800</td>\n",
       "      <td>8.349400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1000</td>\n",
       "      <td>8.259200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1200</td>\n",
       "      <td>8.222200</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to ./tmp/checkpoint-500\n",
      "Trainer.model is not a `PreTrainedModel`, only saving its state dict.\n",
      "Saving model checkpoint to ./tmp/checkpoint-1000\n",
      "Trainer.model is not a `PreTrainedModel`, only saving its state dict.\n",
      "\n",
      "\n",
      "Training completed. Do not forget to share your model on huggingface.co/models =)\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "********************\n",
      "Eval results for day 3 are:\t\n",
      "\n",
      "********************\n",
      "\n",
      " eval_/loss = 8.42210865020752\n",
      " eval_/next-item/ndcg_at_10 = 0.06027791276574135\n",
      " eval_/next-item/ndcg_at_20 = 0.07407757639884949\n",
      " eval_/next-item/recall_at_10 = 0.1122981607913971\n",
      " eval_/next-item/recall_at_20 = 0.16726472973823547\n",
      " eval_runtime = 39.0537\n",
      " eval_samples_per_second = 314.643\n",
      " eval_steps_per_second = 9.833\n",
      "********************\n",
      "Launch training for day 3 are:\n",
      "********************\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "***** Running training *****\n",
      "  Num examples = 97792\n",
      "  Num Epochs = 3\n",
      "  Instantaneous batch size per device = 256\n",
      "  Total train batch size (w. parallel, distributed & accumulation) = 256\n",
      "  Gradient Accumulation steps = 1\n",
      "  Total optimization steps = 1146\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='1146' max='1146' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [1146/1146 00:46, Epoch 3/3]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>Step</th>\n",
       "      <th>Training Loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>200</td>\n",
       "      <td>8.325100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>400</td>\n",
       "      <td>8.238700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>600</td>\n",
       "      <td>8.108700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>800</td>\n",
       "      <td>8.037000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1000</td>\n",
       "      <td>7.972000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to ./tmp/checkpoint-500\n",
      "Trainer.model is not a `PreTrainedModel`, only saving its state dict.\n",
      "Saving model checkpoint to ./tmp/checkpoint-1000\n",
      "Trainer.model is not a `PreTrainedModel`, only saving its state dict.\n",
      "\n",
      "\n",
      "Training completed. Do not forget to share your model on huggingface.co/models =)\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "********************\n",
      "Eval results for day 4 are:\t\n",
      "\n",
      "********************\n",
      "\n",
      " eval_/loss = 8.08250617980957\n",
      " eval_/next-item/ndcg_at_10 = 0.0742299035191536\n",
      " eval_/next-item/ndcg_at_20 = 0.09017771482467651\n",
      " eval_/next-item/recall_at_10 = 0.14068539440631866\n",
      " eval_/next-item/recall_at_20 = 0.20387786626815796\n",
      " eval_runtime = 63.4715\n",
      " eval_samples_per_second = 245.023\n",
      " eval_steps_per_second = 7.657\n",
      "CPU times: user 11min 29s, sys: 51.9 s, total: 12min 21s\n",
      "Wall time: 4min 59s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "start_time_window_index = 1\n",
    "final_time_window_index = 4\n",
    "for time_index in range(start_time_window_index, final_time_window_index):\n",
    "    # Set data \n",
    "    time_index_train = time_index\n",
    "    time_index_eval = time_index + 1\n",
    "    train_paths = glob.glob(os.path.join(OUTPUT_DIR, f\"{time_index_train}/train.parquet\"))\n",
    "    eval_paths = glob.glob(os.path.join(OUTPUT_DIR, f\"{time_index_eval}/valid.parquet\"))\n",
    "    # Train on day related to time_index \n",
    "    print('*'*20)\n",
    "    print(\"Launch training for day %s are:\" %time_index)\n",
    "    print('*'*20 + '\\n')\n",
    "    trainer.train_dataset_or_path = train_paths\n",
    "    trainer.reset_lr_scheduler()\n",
    "    trainer.train()\n",
    "    trainer.state.global_step +=1\n",
    "    # Evaluate on the following day\n",
    "    trainer.eval_dataset_or_path = eval_paths\n",
    "    train_metrics = trainer.evaluate(metric_key_prefix='eval')\n",
    "    print('*'*20)\n",
    "    print(\"Eval results for day %s are:\\t\" %time_index_eval)\n",
    "    print('\\n' + '*'*20 + '\\n')\n",
    "    for key in sorted(train_metrics.keys()):\n",
    "        print(\" %s = %s\" % (key, str(train_metrics[key]))) \n",
    "    wipe_memory()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad99f151",
   "metadata": {},
   "source": [
    "Add eval accuracy metric results to the existing resuls.txt file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "8c7768af",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "with open(\"results.txt\", 'a') as f:\n",
    "    f.write('\\n')\n",
    "    f.write('XLNet-MLM accuracy results:')\n",
    "    f.write('\\n')\n",
    "    for key, value in  model.compute_metrics().items(): \n",
    "        f.write('%s:%s\\n' % (key, value.item()))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "992db813",
   "metadata": {},
   "source": [
    "#### Restart the kernel to free our GPU memory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "1717ffde",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'status': 'ok', 'restart': True}"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import IPython\n",
    "app = IPython.Application.instance()\n",
    "app.kernel.do_shutdown(True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed2524f0",
   "metadata": {},
   "source": [
    "At this stage if the kernel does not restart automatically, we expect you to manually restart the kernel to free GPU memory so that you can move on to the next session-based model training with XLNet using side information."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e7c5c869",
   "metadata": {},
   "source": [
    "#### 3.2.4 Train XLNET with Side Information for Next Item Prediction"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a4a9db3",
   "metadata": {},
   "source": [
    "It is a common practice in RecSys to leverage additional tabular features of item (product) metadata and user context, providing the model more\n",
    "information for meaningful predictions. With that motivation, in this section, we will use additional features to train our XLNET architecture. We already checked our `schema.pb`, saw that it includes features and their tags. Now it is time to use these additional features that we created in the `02_ETL-with-NVTabular.ipynb` notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "11394818",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.8/dist-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n",
      "/usr/local/lib/python3.8/dist-packages/merlin/dtypes/mappings/tf.py:52: UserWarning: Tensorflow dtype mappings did not load successfully due to an error: No module named 'tensorflow'\n",
      "  warn(f\"Tensorflow dtype mappings did not load successfully due to an error: {exc.msg}\")\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "import glob\n",
    "import nvtabular as nvt\n",
    "\n",
    "import torch \n",
    "import transformers4rec.torch as tr\n",
    "from transformers4rec.torch.ranking_metric import NDCGAt, RecallAt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "00bcc6a0",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:149: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "from merlin.schema import Schema\n",
    "from merlin.io import Dataset\n",
    "\n",
    "# Define categorical and continuous columns to fed to training model\n",
    "x_cat_names = ['product_id-list', 'category_id-list', 'brand-list']\n",
    "x_cont_names = ['product_recency_days_log_norm-list', 'et_dayofweek_sin-list', 'et_dayofweek_cos-list', \n",
    "                'price_log_norm-list', 'relative_price_to_avg_categ_id-list']\n",
    "\n",
    "\n",
    "INPUT_DATA_DIR = os.environ.get(\"INPUT_DATA_DIR\", \"/workspace/data\")\n",
    "\n",
    "train = Dataset(os.path.join(INPUT_DATA_DIR, \"processed_nvt/part_0.parquet\"))\n",
    "schema = train.schema\n",
    "schema = schema.select_by_name(x_cat_names + x_cont_names)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d10fdffc",
   "metadata": {},
   "source": [
    "Here we set `aggregation=\"concat\"`, so that all categorical and continuous features are concatenated to form an interaction representation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "1ce19f45",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Projecting inputs of NextItemPredictionTask to'64' As weight tying requires the input dimension '192' to be equal to the item-id embedding dimension '64'\n"
     ]
    }
   ],
   "source": [
    "# Define input block\n",
    "sequence_length, d_model = 20, 192\n",
    "# Define input module to process tabular input-features and to prepare masked inputs\n",
    "inputs= tr.TabularSequenceFeatures.from_schema(\n",
    "    schema,\n",
    "    max_sequence_length=sequence_length,\n",
    "    aggregation=\"concat\",\n",
    "    d_output=d_model,\n",
    "    masking=\"mlm\",\n",
    ")\n",
    "\n",
    "# Define XLNetConfig class and set default parameters for HF XLNet config  \n",
    "transformer_config = tr.XLNetConfig.build(\n",
    "    d_model=d_model, n_head=4, n_layer=2, total_seq_length=sequence_length\n",
    ")\n",
    "# Define the model block including: inputs, masking, projection and transformer block.\n",
    "body = tr.SequentialBlock(\n",
    "    inputs, tr.MLPBlock([192]), tr.TransformerBlock(transformer_config, masking=inputs.masking)\n",
    ")\n",
    "\n",
    "# Define the head related to next item prediction task \n",
    "head = tr.Head(\n",
    "    body,\n",
    "    tr.NextItemPredictionTask(weight_tying=True, \n",
    "                                     metrics=[NDCGAt(top_ks=[10, 20], labels_onehot=True),  \n",
    "                                              RecallAt(top_ks=[10, 20], labels_onehot=True)]),\n",
    ")\n",
    "\n",
    "# Get the end-to-end Model class \n",
    "model = tr.Model(head)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b244593",
   "metadata": {},
   "source": [
    "##### Training and Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "19394f10",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from transformers4rec.config.trainer import T4RecTrainingArguments\n",
    "from transformers4rec.torch import Trainer\n",
    "from transformers4rec.torch.utils.examples_utils import wipe_memory\n",
    "\n",
    "#Set arguments for training \n",
    "training_args = T4RecTrainingArguments(\n",
    "            output_dir=\"./tmp\",\n",
    "            max_sequence_length=20,\n",
    "            data_loader_engine='merlin',\n",
    "            num_train_epochs=3, \n",
    "            dataloader_drop_last=False,\n",
    "            per_device_train_batch_size = 256,\n",
    "            per_device_eval_batch_size = 32,\n",
    "            gradient_accumulation_steps = 1,\n",
    "            learning_rate=0.000666,\n",
    "            report_to = [],\n",
    "            logging_steps=200,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "e09af459",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Instantiate the T4Rec Trainer, which manages training and evaluation\n",
    "trainer = Trainer(\n",
    "    model=model,\n",
    "    args=training_args,\n",
    "    schema=schema,\n",
    "    compute_metrics=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7de48ac8",
   "metadata": {},
   "source": [
    "Define the output folder of the processed parquet files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "3924728a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "OUTPUT_DIR = os.environ.get(\"OUTPUT_DIR\", \"/workspace/data/sessions_by_day\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "039242ef",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "********************\n",
      "Launch training for day 1 are:\n",
      "********************\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "***** Running training *****\n",
      "  Num examples = 112128\n",
      "  Num Epochs = 3\n",
      "  Instantaneous batch size per device = 256\n",
      "  Total train batch size (w. parallel, distributed & accumulation) = 256\n",
      "  Gradient Accumulation steps = 1\n",
      "  Total optimization steps = 1314\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='1314' max='1314' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [1314/1314 02:07, Epoch 3/3]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>Step</th>\n",
       "      <th>Training Loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>200</td>\n",
       "      <td>9.823300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>400</td>\n",
       "      <td>8.907700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>600</td>\n",
       "      <td>8.615000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>800</td>\n",
       "      <td>8.508400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1000</td>\n",
       "      <td>8.392100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1200</td>\n",
       "      <td>8.309200</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to ./tmp/checkpoint-500\n",
      "Trainer.model is not a `PreTrainedModel`, only saving its state dict.\n",
      "Saving model checkpoint to ./tmp/checkpoint-1000\n",
      "Trainer.model is not a `PreTrainedModel`, only saving its state dict.\n",
      "\n",
      "\n",
      "Training completed. Do not forget to share your model on huggingface.co/models =)\n",
      "\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='1285' max='415' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [415/415 06:27]\n",
       "    </div>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "***** Running training *****\n",
      "  Num examples = 106240\n",
      "  Num Epochs = 3\n",
      "  Instantaneous batch size per device = 256\n",
      "  Total train batch size (w. parallel, distributed & accumulation) = 256\n",
      "  Gradient Accumulation steps = 1\n",
      "  Total optimization steps = 1245\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "********************\n",
      "Eval results for day 2 are:\t\n",
      "\n",
      "********************\n",
      "\n",
      " eval_/loss = 8.507274627685547\n",
      " eval_/next-item/ndcg_at_10 = 0.059581220149993896\n",
      " eval_/next-item/ndcg_at_20 = 0.07178892195224762\n",
      " eval_/next-item/recall_at_10 = 0.10866450518369675\n",
      " eval_/next-item/recall_at_20 = 0.15722796320915222\n",
      " eval_runtime = 48.7391\n",
      " eval_samples_per_second = 272.471\n",
      " eval_steps_per_second = 8.515\n",
      "********************\n",
      "Launch training for day 2 are:\n",
      "********************\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='1245' max='1245' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [1245/1245 02:01, Epoch 3/3]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>Step</th>\n",
       "      <th>Training Loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>200</td>\n",
       "      <td>8.442700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>400</td>\n",
       "      <td>8.282900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>600</td>\n",
       "      <td>8.154300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>800</td>\n",
       "      <td>8.022300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1000</td>\n",
       "      <td>7.951400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1200</td>\n",
       "      <td>7.864900</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to ./tmp/checkpoint-500\n",
      "Trainer.model is not a `PreTrainedModel`, only saving its state dict.\n",
      "Saving model checkpoint to ./tmp/checkpoint-1000\n",
      "Trainer.model is not a `PreTrainedModel`, only saving its state dict.\n",
      "\n",
      "\n",
      "Training completed. Do not forget to share your model on huggingface.co/models =)\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "********************\n",
      "Eval results for day 3 are:\t\n",
      "\n",
      "********************\n",
      "\n",
      " eval_/loss = 8.165782928466797\n",
      " eval_/next-item/ndcg_at_10 = 0.07608938217163086\n",
      " eval_/next-item/ndcg_at_20 = 0.09094516187906265\n",
      " eval_/next-item/recall_at_10 = 0.14394062757492065\n",
      " eval_/next-item/recall_at_20 = 0.2028217315673828\n",
      " eval_runtime = 41.5671\n",
      " eval_samples_per_second = 295.618\n",
      " eval_steps_per_second = 9.238\n",
      "********************\n",
      "Launch training for day 3 are:\n",
      "********************\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "***** Running training *****\n",
      "  Num examples = 97792\n",
      "  Num Epochs = 3\n",
      "  Instantaneous batch size per device = 256\n",
      "  Total train batch size (w. parallel, distributed & accumulation) = 256\n",
      "  Gradient Accumulation steps = 1\n",
      "  Total optimization steps = 1146\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='1146' max='1146' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [1146/1146 01:51, Epoch 3/3]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>Step</th>\n",
       "      <th>Training Loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>200</td>\n",
       "      <td>8.026600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>400</td>\n",
       "      <td>7.860000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>600</td>\n",
       "      <td>7.709400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>800</td>\n",
       "      <td>7.622100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1000</td>\n",
       "      <td>7.549700</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to ./tmp/checkpoint-500\n",
      "Trainer.model is not a `PreTrainedModel`, only saving its state dict.\n",
      "Saving model checkpoint to ./tmp/checkpoint-1000\n",
      "Trainer.model is not a `PreTrainedModel`, only saving its state dict.\n",
      "\n",
      "\n",
      "Training completed. Do not forget to share your model on huggingface.co/models =)\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "********************\n",
      "Eval results for day 4 are:\t\n",
      "\n",
      "********************\n",
      "\n",
      " eval_/loss = 7.7548112869262695\n",
      " eval_/next-item/ndcg_at_10 = 0.09143278002738953\n",
      " eval_/next-item/ndcg_at_20 = 0.10998427122831345\n",
      " eval_/next-item/recall_at_10 = 0.17630764842033386\n",
      " eval_/next-item/recall_at_20 = 0.249935582280159\n",
      " eval_runtime = 66.6057\n",
      " eval_samples_per_second = 233.494\n",
      " eval_steps_per_second = 7.297\n",
      "CPU times: user 16min 50s, sys: 1min 6s, total: 17min 57s\n",
      "Wall time: 8min 42s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "start_time_window_index = 1\n",
    "final_time_window_index = 4\n",
    "for time_index in range(start_time_window_index, final_time_window_index):\n",
    "    # Set data \n",
    "    time_index_train = time_index\n",
    "    time_index_eval = time_index + 1\n",
    "    train_paths = glob.glob(os.path.join(OUTPUT_DIR, f\"{time_index_train}/train.parquet\"))\n",
    "    eval_paths = glob.glob(os.path.join(OUTPUT_DIR, f\"{time_index_eval}/valid.parquet\"))\n",
    "    # Train on day related to time_index \n",
    "    print('*'*20)\n",
    "    print(\"Launch training for day %s are:\" %time_index)\n",
    "    print('*'*20 + '\\n')\n",
    "    trainer.train_dataset_or_path = train_paths\n",
    "    trainer.reset_lr_scheduler()\n",
    "    trainer.train()\n",
    "    trainer.state.global_step +=1\n",
    "    # Evaluate on the following day\n",
    "    trainer.eval_dataset_or_path = eval_paths\n",
    "    train_metrics = trainer.evaluate(metric_key_prefix='eval')\n",
    "    print('*'*20)\n",
    "    print(\"Eval results for day %s are:\\t\" %time_index_eval)\n",
    "    print('\\n' + '*'*20 + '\\n')\n",
    "    for key in sorted(train_metrics.keys()):\n",
    "        print(\" %s = %s\" % (key, str(train_metrics[key]))) \n",
    "    wipe_memory()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a62820cf",
   "metadata": {},
   "source": [
    "Add XLNet-MLM with side information accuracy results to the `results.txt`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "5986e99c",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "with open(\"results.txt\", 'a') as f:\n",
    "    f.write('\\n')\n",
    "    f.write('XLNet-MLM with side information accuracy results:')\n",
    "    f.write('\\n')\n",
    "    for key, value in  model.compute_metrics().items(): \n",
    "        f.write('%s:%s\\n' % (key, value.item()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "1cbb1225",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GRU accuracy results:\n",
      "next-item/ndcg_at_10:0.06745054572820663\n",
      "next-item/ndcg_at_20:0.08173713833093643\n",
      "next-item/recall_at_10:0.12599845230579376\n",
      "next-item/recall_at_20:0.18262046575546265\n",
      "\n",
      "XLNet-MLM accuracy results:\n",
      "next-item/ndcg_at_10:0.0742299035191536\n",
      "next-item/ndcg_at_20:0.09017771482467651\n",
      "next-item/recall_at_10:0.14068539440631866\n",
      "next-item/recall_at_20:0.20387786626815796\n",
      "\n",
      "XLNet-MLM with side information accuracy results:\n",
      "next-item/ndcg_at_10:0.08309739083051682\n",
      "next-item/ndcg_at_20:0.10099391639232635\n",
      "next-item/recall_at_10:0.1542772501707077\n",
      "next-item/recall_at_20:0.22519969940185547\n"
     ]
    }
   ],
   "source": [
    "!cat results.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b6a6f96b",
   "metadata": {},
   "source": [
    "**In the end, using side information provided higher accuracy. Why is that? Have an idea?**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aef1ded8",
   "metadata": {},
   "source": [
    "## Wrap Up"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d968c71",
   "metadata": {},
   "source": [
    "Congratulations on finishing this notebook. In this tutorial, we have presented Transformers4Rec, an open source library designed to enable RecSys researchers and practitioners to quickly and easily explore the latest developments of the NLP for sequential and session-based recommendation tasks."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "744a2f17",
   "metadata": {},
   "source": [
    "## References"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03a7b764",
   "metadata": {},
   "source": [
    "[1] Malte Ludewig and Dietmar Jannach. 2018. Evaluation of session-based recommendation algorithms. User Modeling and User-Adapted Interaction 28, 4-5 (2018), 331–390.<br>\n",
    "[2] Balázs Hidasi and Alexandros Karatzoglou. 2018. Recurrent neural networks with top-k gains for session-based recommendations. In Proceedings of the 27th ACMinternational conference on information and knowledge management. 843–852.<br>\n",
    "[3] Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM international conference on information and knowledge management. 1441–1450.\n",
    "[4] Shiming Sun, Yuanhe Tang, Zemei Dai, and Fu Zhou. 2019. Self-attention network for session-based recommendation with streaming data input. IEEE Access 7 (2019), 110499–110509.  \n",
    "[5] Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014).  \n",
    "[6] Vaswani, A., et al. (2017). Attention is all you need. In Advances in neural information processing systems (pp. 5998-6008).  \n",
    "[7] Lample, Guillaume, and Alexis Conneau. \"Cross-lingual language model pretraining.\" arXiv preprint arXiv:1901.07291  \n",
    "[8] Gabriel De Souza P. Moreira, et al. (2021). Transformers4Rec: Bridging the Gap between NLP and Sequential / Session-Based Recommendation. RecSys'21.  \n",
    "[9] Understanding XLNet, BorealisAI. Online available: https://www.borealisai.com/en/blog/understanding-xlnet/  \n",
    "[10] Yang, Zhilin, et al. \"Xlnet: Generalized autoregressive pretraining for language understanding.\" Advances in neural information processing systems 32 (2019)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}