building-an-end-to-end-nlp-application.ipynb

{
  "metadata": {
    "kernelspec": {
      "language": "python",
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "version": "3.7.12",
      "mimetype": "text/x-python",
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "pygments_lexer": "ipython3",
      "nbconvert_exporter": "python",
      "file_extension": ".py"
    },
    "colab": {
      "name": "building-an-end-to-end-nlp-application.ipynb",
      "provenance": []
    },
    "accelerator": "GPU"
  },
  "nbformat_minor": 0,
  "nbformat": 4,
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# Building an end-to-end NLP application"
      ],
      "metadata": {
        "cell_id": "3d3442ecd4cf48ad943adef93f872132",
        "tags": [],
        "deepnote_cell_type": "markdown",
        "deepnote_cell_height": 134,
        "id": "UKeDKKkXaGSM"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "This notebook will demonstrate the generic steps involved in solving a text classification problem. For the purpose of the demostration, we'll look at a binary classification problem (specifically sentiment analysis) and go through all the stages of typical ML lifecycle.\n",
        "\n",
        "- **Problem:** Sentiment analysis (supervised)\n",
        "- **Dataset:** Amazon product reviews\n",
        "- **Framework for model building:** TensorFlow, HuggingFace Transformers\n",
        "- **Demo:** Gradio\n"
      ],
      "metadata": {
        "cell_id": "0f67c878ddbe434fb61c60a26140c1bf",
        "tags": [],
        "deepnote_cell_type": "markdown",
        "deepnote_cell_height": 249.125,
        "id": "k15NDhbKaGSU"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Set the session to use GPU\n",
        "import tensorflow as tf\n",
        "import os\n",
        "num_gpus_available = len(tf.config.experimental.list_physical_devices('GPU'))\n",
        "print(\"Num GPUs Available: \", num_gpus_available)\n",
        "assert num_gpus_available > 0\n",
        "\n",
        "sess = tf.compat.v1.Session(config=tf.compat.v1.ConfigProto(log_device_placement=True))\n",
        "os.environ[\"CUDA_VISIBLE_DEVICES\"]='0'"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2022-06-03T14:39:47.440668Z",
          "iopub.execute_input": "2022-06-03T14:39:47.441104Z",
          "iopub.status.idle": "2022-06-03T14:39:53.564271Z",
          "shell.execute_reply.started": "2022-06-03T14:39:47.441018Z",
          "shell.execute_reply": "2022-06-03T14:39:53.563477Z"
        },
        "trusted": true,
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VL8QJXQYaGSX",
        "outputId": "19d5a5a4-1069-405f-c519-c9337c521df7"
      },
      "execution_count": 45,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Num GPUs Available:  1\n",
            "Device mapping:\n",
            "/job:localhost/replica:0/task:0/device:GPU:0 -> device: 0, name: Tesla T4, pci bus id: 0000:00:04.0, compute capability: 7.5\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Install necessary libraries\n",
        "!pip install gradio\n",
        "!pip install transformers\n",
        "!pip install tensorflow_datasets"
      ],
      "metadata": {
        "cell_id": "766b8c1e674b4a65b02961c78f00976c",
        "tags": [],
        "deepnote_to_be_reexecuted": false,
        "source_hash": "7528b55a",
        "execution_start": 1654260231305,
        "execution_millis": 17068,
        "is_output_hidden": true,
        "deepnote_cell_type": "code",
        "deepnote_cell_height": 165.796875,
        "execution": {
          "iopub.status.busy": "2022-06-03T14:39:58.719965Z",
          "iopub.execute_input": "2022-06-03T14:39:58.720585Z",
          "iopub.status.idle": "2022-06-03T14:40:42.173593Z",
          "shell.execute_reply.started": "2022-06-03T14:39:58.720551Z",
          "shell.execute_reply": "2022-06-03T14:40:42.170713Z"
        },
        "trusted": true,
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "f_RZYLxEaGSa",
        "outputId": "aa921826-fd8a-483d-b33e-f34da1715953"
      },
      "execution_count": 46,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Requirement already satisfied: gradio in /usr/local/lib/python3.7/dist-packages (3.0.11)\n",
            "Requirement already satisfied: paramiko in /usr/local/lib/python3.7/dist-packages (from gradio) (2.11.0)\n",
            "Requirement already satisfied: Jinja2 in /usr/local/lib/python3.7/dist-packages (from gradio) (2.11.3)\n",
            "Requirement already satisfied: uvicorn in /usr/local/lib/python3.7/dist-packages (from gradio) (0.17.6)\n",
            "Requirement already satisfied: orjson in /usr/local/lib/python3.7/dist-packages (from gradio) (3.7.0)\n",
            "Requirement already satisfied: pillow in /usr/local/lib/python3.7/dist-packages (from gradio) (7.1.2)\n",
            "Requirement already satisfied: ffmpy in /usr/local/lib/python3.7/dist-packages (from gradio) (0.3.0)\n",
            "Requirement already satisfied: matplotlib in /usr/local/lib/python3.7/dist-packages (from gradio) (3.2.2)\n",
            "Requirement already satisfied: pandas in /usr/local/lib/python3.7/dist-packages (from gradio) (1.3.5)\n",
            "Requirement already satisfied: pycryptodome in /usr/local/lib/python3.7/dist-packages (from gradio) (3.14.1)\n",
            "Requirement already satisfied: markdown-it-py[linkify,plugins] in /usr/local/lib/python3.7/dist-packages (from gradio) (2.1.0)\n",
            "Requirement already satisfied: fastapi in /usr/local/lib/python3.7/dist-packages (from gradio) (0.78.0)\n",
            "Requirement already satisfied: python-multipart in /usr/local/lib/python3.7/dist-packages (from gradio) (0.0.5)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from gradio) (2.23.0)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from gradio) (1.21.6)\n",
            "Requirement already satisfied: pydub in /usr/local/lib/python3.7/dist-packages (from gradio) (0.25.1)\n",
            "Requirement already satisfied: analytics-python in /usr/local/lib/python3.7/dist-packages (from gradio) (1.4.0)\n",
            "Requirement already satisfied: aiohttp in /usr/local/lib/python3.7/dist-packages (from gradio) (3.8.1)\n",
            "Requirement already satisfied: asynctest==0.13.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp->gradio) (0.13.0)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.7/dist-packages (from aiohttp->gradio) (1.2.0)\n",
            "Requirement already satisfied: typing-extensions>=3.7.4 in /usr/local/lib/python3.7/dist-packages (from aiohttp->gradio) (4.2.0)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.7/dist-packages (from aiohttp->gradio) (6.0.2)\n",
            "Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.7/dist-packages (from aiohttp->gradio) (4.0.2)\n",
            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp->gradio) (21.4.0)\n",
            "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp->gradio) (1.7.2)\n",
            "Requirement already satisfied: charset-normalizer<3.0,>=2.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp->gradio) (2.0.12)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.7/dist-packages (from aiohttp->gradio) (1.3.0)\n",
            "Requirement already satisfied: idna>=2.0 in /usr/local/lib/python3.7/dist-packages (from yarl<2.0,>=1.0->aiohttp->gradio) (2.10)\n",
            "Requirement already satisfied: backoff==1.10.0 in /usr/local/lib/python3.7/dist-packages (from analytics-python->gradio) (1.10.0)\n",
            "Requirement already satisfied: python-dateutil>2.1 in /usr/local/lib/python3.7/dist-packages (from analytics-python->gradio) (2.8.2)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from analytics-python->gradio) (1.15.0)\n",
            "Requirement already satisfied: monotonic>=1.5 in /usr/local/lib/python3.7/dist-packages (from analytics-python->gradio) (1.6)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->gradio) (2022.5.18.1)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->gradio) (3.0.4)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->gradio) (1.24.3)\n",
            "Requirement already satisfied: starlette==0.19.1 in /usr/local/lib/python3.7/dist-packages (from fastapi->gradio) (0.19.1)\n",
            "Requirement already satisfied: pydantic!=1.7,!=1.7.1,!=1.7.2,!=1.7.3,!=1.8,!=1.8.1,<2.0.0,>=1.6.2 in /usr/local/lib/python3.7/dist-packages (from fastapi->gradio) (1.9.1)\n",
            "Requirement already satisfied: anyio<5,>=3.4.0 in /usr/local/lib/python3.7/dist-packages (from starlette==0.19.1->fastapi->gradio) (3.6.1)\n",
            "Requirement already satisfied: sniffio>=1.1 in /usr/local/lib/python3.7/dist-packages (from anyio<5,>=3.4.0->starlette==0.19.1->fastapi->gradio) (1.2.0)\n",
            "Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.7/dist-packages (from Jinja2->gradio) (2.0.1)\n",
            "Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.7/dist-packages (from markdown-it-py[linkify,plugins]->gradio) (0.1.1)\n",
            "Requirement already satisfied: linkify-it-py~=1.0 in /usr/local/lib/python3.7/dist-packages (from markdown-it-py[linkify,plugins]->gradio) (1.0.3)\n",
            "Requirement already satisfied: mdit-py-plugins in /usr/local/lib/python3.7/dist-packages (from markdown-it-py[linkify,plugins]->gradio) (0.3.0)\n",
            "Requirement already satisfied: uc-micro-py in /usr/local/lib/python3.7/dist-packages (from linkify-it-py~=1.0->markdown-it-py[linkify,plugins]->gradio) (1.0.1)\n",
            "Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->gradio) (3.0.9)\n",
            "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->gradio) (1.4.2)\n",
            "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib->gradio) (0.11.0)\n",
            "Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.7/dist-packages (from pandas->gradio) (2022.1)\n",
            "Requirement already satisfied: cryptography>=2.5 in /usr/local/lib/python3.7/dist-packages (from paramiko->gradio) (37.0.2)\n",
            "Requirement already satisfied: bcrypt>=3.1.3 in /usr/local/lib/python3.7/dist-packages (from paramiko->gradio) (3.2.2)\n",
            "Requirement already satisfied: pynacl>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from paramiko->gradio) (1.5.0)\n",
            "Requirement already satisfied: cffi>=1.1 in /usr/local/lib/python3.7/dist-packages (from bcrypt>=3.1.3->paramiko->gradio) (1.15.0)\n",
            "Requirement already satisfied: pycparser in /usr/local/lib/python3.7/dist-packages (from cffi>=1.1->bcrypt>=3.1.3->paramiko->gradio) (2.21)\n",
            "Requirement already satisfied: h11>=0.8 in /usr/local/lib/python3.7/dist-packages (from uvicorn->gradio) (0.13.0)\n",
            "Requirement already satisfied: asgiref>=3.4.0 in /usr/local/lib/python3.7/dist-packages (from uvicorn->gradio) (3.5.2)\n",
            "Requirement already satisfied: click>=7.0 in /usr/local/lib/python3.7/dist-packages (from uvicorn->gradio) (7.1.2)\n",
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Requirement already satisfied: transformers in /usr/local/lib/python3.7/dist-packages (4.19.2)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.7/dist-packages (from transformers) (3.7.0)\n",
            "Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/dist-packages (from transformers) (4.11.4)\n",
            "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.7/dist-packages (from transformers) (1.21.6)\n",
            "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.7/dist-packages (from transformers) (2019.12.20)\n",
            "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.7/dist-packages (from transformers) (6.0)\n",
            "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.7/dist-packages (from transformers) (21.3)\n",
            "Requirement already satisfied: huggingface-hub<1.0,>=0.1.0 in /usr/local/lib/python3.7/dist-packages (from transformers) (0.7.0)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from transformers) (2.23.0)\n",
            "Requirement already satisfied: tokenizers!=0.11.3,<0.13,>=0.11.1 in /usr/local/lib/python3.7/dist-packages (from transformers) (0.12.1)\n",
            "Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.7/dist-packages (from transformers) (4.64.0)\n",
            "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.7/dist-packages (from huggingface-hub<1.0,>=0.1.0->transformers) (4.2.0)\n",
            "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging>=20.0->transformers) (3.0.9)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata->transformers) (3.8.0)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (2022.5.18.1)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (2.10)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (3.0.4)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (1.24.3)\n",
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Requirement already satisfied: tensorflow_datasets in /usr/local/lib/python3.7/dist-packages (4.0.1)\n",
            "Requirement already satisfied: requests>=2.19.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow_datasets) (2.23.0)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from tensorflow_datasets) (1.15.0)\n",
            "Requirement already satisfied: absl-py in /usr/local/lib/python3.7/dist-packages (from tensorflow_datasets) (1.0.0)\n",
            "Requirement already satisfied: termcolor in /usr/local/lib/python3.7/dist-packages (from tensorflow_datasets) (1.1.0)\n",
            "Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from tensorflow_datasets) (4.64.0)\n",
            "Requirement already satisfied: importlib-resources in /usr/local/lib/python3.7/dist-packages (from tensorflow_datasets) (5.7.1)\n",
            "Requirement already satisfied: dm-tree in /usr/local/lib/python3.7/dist-packages (from tensorflow_datasets) (0.1.7)\n",
            "Requirement already satisfied: attrs>=18.1.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow_datasets) (21.4.0)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from tensorflow_datasets) (1.21.6)\n",
            "Requirement already satisfied: dill in /usr/local/lib/python3.7/dist-packages (from tensorflow_datasets) (0.3.5.1)\n",
            "Requirement already satisfied: protobuf>=3.6.1 in /usr/local/lib/python3.7/dist-packages (from tensorflow_datasets) (3.17.3)\n",
            "Requirement already satisfied: promise in /usr/local/lib/python3.7/dist-packages (from tensorflow_datasets) (2.3)\n",
            "Requirement already satisfied: tensorflow-metadata in /usr/local/lib/python3.7/dist-packages (from tensorflow_datasets) (1.8.0)\n",
            "Requirement already satisfied: future in /usr/local/lib/python3.7/dist-packages (from tensorflow_datasets) (0.16.0)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests>=2.19.0->tensorflow_datasets) (1.24.3)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests>=2.19.0->tensorflow_datasets) (2.10)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests>=2.19.0->tensorflow_datasets) (2022.5.18.1)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests>=2.19.0->tensorflow_datasets) (3.0.4)\n",
            "Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.7/dist-packages (from importlib-resources->tensorflow_datasets) (3.8.0)\n",
            "Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow-metadata->tensorflow_datasets) (1.56.2)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## ML Workflow\n",
        "\n",
        "The following are the stages involved in building any typical ML solution.\n",
        "\n",
        "- **Stage 1:** Gather data\n",
        "- **Stage 2:** Explore the data\n",
        "    - **Stage 2.5:** Choose a model\n",
        "- **Stage 3:** Prepare the data\n",
        "- **Stage 4:** Build, train and evaluate the model\n",
        "- **Stage 5:** Refine the model\n",
        "- **Stage 6:** Deploy the model"
      ],
      "metadata": {
        "cell_id": "8229b0b0e6644680a8662dc6e83d4e7b",
        "tags": [],
        "deepnote_cell_type": "markdown",
        "deepnote_cell_height": 322.125,
        "id": "tHgwxfRgaGSb"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Stage 1: Gather data\n",
        "\n",
        "This is one of the most important step in solving any supervised ML problem. Remember, the model is as good as your data. For the problem you're trying to solve, sometimes, the data might be available but in most of the real-world scenarios, data will not be readily available. In such scenarios, here are some of the things to keep in mind while collecting data,\n",
        "\n",
        "- Check if there is a public API available to get the data. Understand the limitations, pricing, number of tries etc. of the API.\n",
        "- More the data, the better the model will be.\n",
        "- Make sure you have enough samples for each of the target classes. \n",
        "- The samples should adequately cover the space of possible inputs to your problem.\n"
      ],
      "metadata": {
        "cell_id": "257e6c5f9a2e4dc990258edcb4ef1bfa",
        "tags": [],
        "deepnote_cell_type": "markdown",
        "deepnote_cell_height": 316.515625,
        "id": "TM0dUNPDaGSc"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "import tensorflow_datasets as tfds\n",
        "import matplotlib.pyplot as plt"
      ],
      "metadata": {
        "cell_id": "cb92d7829eac4a1bb84cf874741b572d",
        "tags": [],
        "deepnote_to_be_reexecuted": false,
        "source_hash": "ac5c06bc",
        "execution_start": 1654260247861,
        "execution_millis": 874,
        "is_output_hidden": true,
        "deepnote_cell_type": "code",
        "deepnote_cell_height": 183.796875,
        "execution": {
          "iopub.status.busy": "2022-06-03T14:40:42.176221Z",
          "iopub.execute_input": "2022-06-03T14:40:42.177273Z",
          "iopub.status.idle": "2022-06-03T14:40:43.417817Z",
          "shell.execute_reply.started": "2022-06-03T14:40:42.177230Z",
          "shell.execute_reply": "2022-06-03T14:40:43.417005Z"
        },
        "trusted": true,
        "id": "kUkZthhQaGSd"
      },
      "execution_count": 47,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Load the tensorflow dataset\n",
        "dataset = tfds.load('amazon_us_reviews/Mobile_Electronics_v1_00', split='train', shuffle_files=True)"
      ],
      "metadata": {
        "cell_id": "73d77f3b3ad94ee3bc398b59fc0773aa",
        "tags": [],
        "deepnote_to_be_reexecuted": true,
        "source_hash": "bdf781a3",
        "execution_start": 1654259616988,
        "execution_millis": 346,
        "is_output_hidden": true,
        "deepnote_cell_type": "code",
        "deepnote_cell_height": 129.796875,
        "execution": {
          "iopub.status.busy": "2022-06-03T14:40:43.419157Z",
          "iopub.execute_input": "2022-06-03T14:40:43.419540Z",
          "iopub.status.idle": "2022-06-03T14:42:38.964641Z",
          "shell.execute_reply.started": "2022-06-03T14:40:43.419504Z",
          "shell.execute_reply": "2022-06-03T14:42:38.963825Z"
        },
        "trusted": true,
        "id": "KG31PYqSaGSg"
      },
      "execution_count": 175,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Stage 2: Explore the data\n",
        "\n",
        "Before trying to build a model, it is imperative to understand the data at hand. Understanding the charateristics of the data will not only help in picking the right modeling technique for the problem we're trying to solve but also helps in building better models with good quality data. \n",
        "\n",
        "Always inspect some randon samples of data and get some key metrics like,\n",
        "\n",
        "- Total number of samples available\n",
        "- Number of classes/targets\n",
        "- Total samples available per class\n",
        "- Number of words per sample\n",
        "- Frequency distribution of words\n",
        "- Distribution od sample length"
      ],
      "metadata": {
        "cell_id": "bb4f164da4574c86b9749ca9b6aa6e14",
        "tags": [],
        "deepnote_cell_type": "markdown",
        "deepnote_cell_height": 358.90625,
        "id": "1Z9SXgrFaGSh"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Convert the dataset to pandas dataframe\n",
        "dataset = tfds.as_dataframe(dataset)\n",
        "dataset.head()"
      ],
      "metadata": {
        "cell_id": "c39f9f1658b2421cb9e2b5586b3e5c36",
        "tags": [],
        "deepnote_to_be_reexecuted": true,
        "source_hash": "bca59b0a",
        "execution_start": 1654259617342,
        "execution_millis": 83513,
        "deepnote_cell_type": "code",
        "deepnote_cell_height": 466,
        "execution": {
          "iopub.status.busy": "2022-06-03T14:42:38.966533Z",
          "iopub.execute_input": "2022-06-03T14:42:38.966935Z",
          "iopub.status.idle": "2022-06-03T14:43:37.902232Z",
          "shell.execute_reply.started": "2022-06-03T14:42:38.966897Z",
          "shell.execute_reply": "2022-06-03T14:43:37.901334Z"
        },
        "trusted": true,
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 444
        },
        "id": "F8KNz2lBaGSj",
        "outputId": "11675cee-3adf-40d8-e6b7-a9e91e286ee5"
      },
      "execution_count": 176,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "  data/customer_id  data/helpful_votes data/marketplace  \\\n",
              "0      b'20980074'                   0            b'US'   \n",
              "1        b'779273'                   0            b'US'   \n",
              "2      b'15410531'                   0            b'US'   \n",
              "3      b'27389005'                   0            b'US'   \n",
              "4       b'2663569'                   0            b'US'   \n",
              "\n",
              "   data/product_category data/product_id data/product_parent  \\\n",
              "0  b'Mobile_Electronics'   b'B00D1847NE'        b'274617424'   \n",
              "1  b'Mobile_Electronics'   b'B00KMO6DYG'        b'397452138'   \n",
              "2  b'Mobile_Electronics'   b'B000GWLL0K'        b'948304826'   \n",
              "3  b'Mobile_Electronics'   b'B008L3JE6Y'        b'466340015'   \n",
              "4  b'Mobile_Electronics'   b'B00GHZS4SC'        b'350592810'   \n",
              "\n",
              "                                  data/product_title  \\\n",
              "0  b'Teenage Mutant Ninja Turtles Boombox CD Play...   \n",
              "1  b'4 Gauge Amp Kit Amplifier Install Wiring Com...   \n",
              "2  b'Travel Wall Charger fits Creative Zen Vision...   \n",
              "3  b'High Grade Robust 360\\xc2\\xb0 Adjustable Car...   \n",
              "4  b'HDE Multifunctional Bluetooth FM Audio Car K...   \n",
              "\n",
              "                                    data/review_body data/review_date  \\\n",
              "0                                   b'Does not work'    b'2015-01-09'   \n",
              "1  b'This is a great wiring kit i used it to set ...    b'2015-08-06'   \n",
              "2  b'It works great so much faster than USB charg...    b'2007-03-15'   \n",
              "3  b'This product was purchased to hold a monitor...    b'2013-07-30'   \n",
              "4  b\"it works but it has really bad sound quality...    b'2014-12-31'   \n",
              "\n",
              "                       data/review_headline     data/review_id  \\\n",
              "0                               b'One Star'  b'R1OVS0D6SEXPW7'   \n",
              "1                              b'Great kit'   b'R9VSD0ET8FERB'   \n",
              "2  b'A/C Charger for Creative Zen Vision M'  b'R3ISXCZHWLJLBH'   \n",
              "3                           b'camera stand'  b'R1TWVUDOFJSQAW'   \n",
              "4                      b'bad sound quality'  b'R2PEOEUR1LP0GH'   \n",
              "\n",
              "   data/star_rating  data/total_votes  data/verified_purchase  data/vine  \n",
              "0                 1                 0                       0          1  \n",
              "1                 4                 0                       0          1  \n",
              "2                 5                 0                       0          1  \n",
              "3                 5                 0                       0          1  \n",
              "4                 3                 0                       0          1  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-bbbf95c5-67ed-493c-869b-aec835f45171\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>data/customer_id</th>\n",
              "      <th>data/helpful_votes</th>\n",
              "      <th>data/marketplace</th>\n",
              "      <th>data/product_category</th>\n",
              "      <th>data/product_id</th>\n",
              "      <th>data/product_parent</th>\n",
              "      <th>data/product_title</th>\n",
              "      <th>data/review_body</th>\n",
              "      <th>data/review_date</th>\n",
              "      <th>data/review_headline</th>\n",
              "      <th>data/review_id</th>\n",
              "      <th>data/star_rating</th>\n",
              "      <th>data/total_votes</th>\n",
              "      <th>data/verified_purchase</th>\n",
              "      <th>data/vine</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>b'20980074'</td>\n",
              "      <td>0</td>\n",
              "      <td>b'US'</td>\n",
              "      <td>b'Mobile_Electronics'</td>\n",
              "      <td>b'B00D1847NE'</td>\n",
              "      <td>b'274617424'</td>\n",
              "      <td>b'Teenage Mutant Ninja Turtles Boombox CD Play...</td>\n",
              "      <td>b'Does not work'</td>\n",
              "      <td>b'2015-01-09'</td>\n",
              "      <td>b'One Star'</td>\n",
              "      <td>b'R1OVS0D6SEXPW7'</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>b'779273'</td>\n",
              "      <td>0</td>\n",
              "      <td>b'US'</td>\n",
              "      <td>b'Mobile_Electronics'</td>\n",
              "      <td>b'B00KMO6DYG'</td>\n",
              "      <td>b'397452138'</td>\n",
              "      <td>b'4 Gauge Amp Kit Amplifier Install Wiring Com...</td>\n",
              "      <td>b'This is a great wiring kit i used it to set ...</td>\n",
              "      <td>b'2015-08-06'</td>\n",
              "      <td>b'Great kit'</td>\n",
              "      <td>b'R9VSD0ET8FERB'</td>\n",
              "      <td>4</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>b'15410531'</td>\n",
              "      <td>0</td>\n",
              "      <td>b'US'</td>\n",
              "      <td>b'Mobile_Electronics'</td>\n",
              "      <td>b'B000GWLL0K'</td>\n",
              "      <td>b'948304826'</td>\n",
              "      <td>b'Travel Wall Charger fits Creative Zen Vision...</td>\n",
              "      <td>b'It works great so much faster than USB charg...</td>\n",
              "      <td>b'2007-03-15'</td>\n",
              "      <td>b'A/C Charger for Creative Zen Vision M'</td>\n",
              "      <td>b'R3ISXCZHWLJLBH'</td>\n",
              "      <td>5</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>b'27389005'</td>\n",
              "      <td>0</td>\n",
              "      <td>b'US'</td>\n",
              "      <td>b'Mobile_Electronics'</td>\n",
              "      <td>b'B008L3JE6Y'</td>\n",
              "      <td>b'466340015'</td>\n",
              "      <td>b'High Grade Robust 360\\xc2\\xb0 Adjustable Car...</td>\n",
              "      <td>b'This product was purchased to hold a monitor...</td>\n",
              "      <td>b'2013-07-30'</td>\n",
              "      <td>b'camera stand'</td>\n",
              "      <td>b'R1TWVUDOFJSQAW'</td>\n",
              "      <td>5</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>b'2663569'</td>\n",
              "      <td>0</td>\n",
              "      <td>b'US'</td>\n",
              "      <td>b'Mobile_Electronics'</td>\n",
              "      <td>b'B00GHZS4SC'</td>\n",
              "      <td>b'350592810'</td>\n",
              "      <td>b'HDE Multifunctional Bluetooth FM Audio Car K...</td>\n",
              "      <td>b\"it works but it has really bad sound quality...</td>\n",
              "      <td>b'2014-12-31'</td>\n",
              "      <td>b'bad sound quality'</td>\n",
              "      <td>b'R2PEOEUR1LP0GH'</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-bbbf95c5-67ed-493c-869b-aec835f45171')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-bbbf95c5-67ed-493c-869b-aec835f45171 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-bbbf95c5-67ed-493c-869b-aec835f45171');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 176
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Pick the relevant columns of interest\n",
        "columns_of_interest = ['data/review_body', 'data/star_rating']\n",
        "data = dataset[columns_of_interest]\n",
        "data.head()"
      ],
      "metadata": {
        "cell_id": "caa490ded8a9441193f002a529cc8f2b",
        "tags": [],
        "deepnote_to_be_reexecuted": true,
        "source_hash": "aacb2f48",
        "execution_start": 1654259700745,
        "execution_millis": 119,
        "deepnote_cell_type": "code",
        "deepnote_cell_height": 449,
        "execution": {
          "iopub.status.busy": "2022-06-03T14:43:37.903852Z",
          "iopub.execute_input": "2022-06-03T14:43:37.904262Z",
          "iopub.status.idle": "2022-06-03T14:43:37.922444Z",
          "shell.execute_reply.started": "2022-06-03T14:43:37.904225Z",
          "shell.execute_reply": "2022-06-03T14:43:37.921418Z"
        },
        "trusted": true,
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "lhNkqwMgaGSm",
        "outputId": "31216f58-f568-438f-b9d6-6321232c19cf"
      },
      "execution_count": 177,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                    data/review_body  data/star_rating\n",
              "0                                   b'Does not work'                 1\n",
              "1  b'This is a great wiring kit i used it to set ...                 4\n",
              "2  b'It works great so much faster than USB charg...                 5\n",
              "3  b'This product was purchased to hold a monitor...                 5\n",
              "4  b\"it works but it has really bad sound quality...                 3"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-7963b16d-62b7-4c28-a653-ee0e8d432ab8\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>data/review_body</th>\n",
              "      <th>data/star_rating</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>b'Does not work'</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>b'This is a great wiring kit i used it to set ...</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>b'It works great so much faster than USB charg...</td>\n",
              "      <td>5</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>b'This product was purchased to hold a monitor...</td>\n",
              "      <td>5</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>b\"it works but it has really bad sound quality...</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-7963b16d-62b7-4c28-a653-ee0e8d432ab8')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-7963b16d-62b7-4c28-a653-ee0e8d432ab8 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-7963b16d-62b7-4c28-a653-ee0e8d432ab8');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 177
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Get the number of samples\n",
        "print(\"Total number of datapoints:\", data.shape[0])"
      ],
      "metadata": {
        "cell_id": "494ed0db9b9d4a32afbc7f4d07cc3621",
        "tags": [],
        "deepnote_to_be_reexecuted": true,
        "source_hash": "89f4cefe",
        "execution_start": 1654259700768,
        "execution_millis": 99,
        "deepnote_cell_type": "code",
        "deepnote_cell_height": 130.1875,
        "execution": {
          "iopub.status.busy": "2022-06-03T14:43:37.923929Z",
          "iopub.execute_input": "2022-06-03T14:43:37.924485Z",
          "iopub.status.idle": "2022-06-03T14:43:37.931237Z",
          "shell.execute_reply.started": "2022-06-03T14:43:37.924445Z",
          "shell.execute_reply": "2022-06-03T14:43:37.930428Z"
        },
        "trusted": true,
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ok8yl2T-aGSo",
        "outputId": "33059194-0980-4767-bb92-98451b297a8f"
      },
      "execution_count": 178,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Total number of datapoints: 104975\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Get the number of class (we will convert this the rating to sentiment in the next stage)\n",
        "print(\"Number of classes:\", data['data/star_rating'].nunique())"
      ],
      "metadata": {
        "cell_id": "cf42d1bcc2264587b7e615de4a7d5f31",
        "tags": [],
        "deepnote_to_be_reexecuted": true,
        "source_hash": "8ba8b5b4",
        "execution_start": 1654259700791,
        "execution_millis": 78,
        "deepnote_cell_type": "code",
        "deepnote_cell_height": 130.1875,
        "execution": {
          "iopub.status.busy": "2022-06-03T14:43:37.932838Z",
          "iopub.execute_input": "2022-06-03T14:43:37.933524Z",
          "iopub.status.idle": "2022-06-03T14:43:37.944591Z",
          "shell.execute_reply.started": "2022-06-03T14:43:37.933482Z",
          "shell.execute_reply": "2022-06-03T14:43:37.943652Z"
        },
        "trusted": true,
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "e7oCtcuLaGSq",
        "outputId": "426cc060-3098-4524-8fee-c1e463da60fc"
      },
      "execution_count": 179,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Number of classes: 5\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Get the number of samples per class\n",
        "print(\"Number of samples per class: \\n\" ,data['data/star_rating'].value_counts())"
      ],
      "metadata": {
        "cell_id": "72814594d2254312abdc8bf3bd97dfe0",
        "tags": [],
        "deepnote_to_be_reexecuted": true,
        "source_hash": "7a54dc4b",
        "execution_start": 1654259700849,
        "execution_millis": 22,
        "deepnote_cell_type": "code",
        "deepnote_cell_height": 251.3125,
        "execution": {
          "iopub.status.busy": "2022-06-03T14:43:37.946425Z",
          "iopub.execute_input": "2022-06-03T14:43:37.947050Z",
          "iopub.status.idle": "2022-06-03T14:43:37.955093Z",
          "shell.execute_reply.started": "2022-06-03T14:43:37.947012Z",
          "shell.execute_reply": "2022-06-03T14:43:37.954137Z"
        },
        "trusted": true,
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pFdljIhPaGSr",
        "outputId": "19108ab7-4a78-4a53-bad1-4df39a3e3ac2"
      },
      "execution_count": 180,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Number of samples per class: \n",
            " 5    52255\n",
            "4    18088\n",
            "1    17587\n",
            "3     9734\n",
            "2     7311\n",
            "Name: data/star_rating, dtype: int64\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Get the number of words per sample\n",
        "print(\"Number of words per sample: \" ,np.mean(data['data/review_body'].apply(lambda x: len(x.split()))))"
      ],
      "metadata": {
        "cell_id": "46f684b8d8ef42b8bf8b47d05d11989c",
        "tags": [],
        "deepnote_to_be_reexecuted": true,
        "source_hash": "833f8f7",
        "execution_start": 1654259700849,
        "execution_millis": 522,
        "deepnote_cell_type": "code",
        "deepnote_cell_height": 130.1875,
        "execution": {
          "iopub.status.busy": "2022-06-03T14:43:37.956951Z",
          "iopub.execute_input": "2022-06-03T14:43:37.957508Z",
          "iopub.status.idle": "2022-06-03T14:43:38.421412Z",
          "shell.execute_reply.started": "2022-06-03T14:43:37.957473Z",
          "shell.execute_reply": "2022-06-03T14:43:38.420529Z"
        },
        "trusted": true,
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DD-WH9NDaGSs",
        "outputId": "11a4a96f-f252-4583-bedf-c1df6a4ca0fd"
      },
      "execution_count": 181,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Number of words per sample:  63.97938556799238\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Get the distribution of number of words per sample\n",
        "print(\"Distribution of number of words per sample:\")\n",
        "pd.DataFrame([len(s) for s in data['data/review_body']]).value_counts()"
      ],
      "metadata": {
        "cell_id": "bbdd52680bcf4fd190a6a481850207d8",
        "tags": [],
        "deepnote_to_be_reexecuted": true,
        "source_hash": "c5c18bd0",
        "execution_start": 1654259701405,
        "execution_millis": 105,
        "deepnote_cell_type": "code",
        "deepnote_cell_height": 411.4375,
        "deepnote_output_heights": [
          null,
          232.25
        ],
        "execution": {
          "iopub.status.busy": "2022-06-03T14:43:38.424892Z",
          "iopub.execute_input": "2022-06-03T14:43:38.425467Z",
          "iopub.status.idle": "2022-06-03T14:43:38.508151Z",
          "shell.execute_reply.started": "2022-06-03T14:43:38.425430Z",
          "shell.execute_reply": "2022-06-03T14:43:38.507273Z"
        },
        "trusted": true,
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JVDq_O4WaGSs",
        "outputId": "e4e99581-3848-49ce-9f49-e85e92692487"
      },
      "execution_count": 182,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Distribution of number of words per sample:\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "9        550\n",
              "114      534\n",
              "115      523\n",
              "113      522\n",
              "119      518\n",
              "        ... \n",
              "2724       1\n",
              "2729       1\n",
              "2731       1\n",
              "2732       1\n",
              "16009      1\n",
              "Length: 3079, dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 182
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Plot the distribution of of sample length\n",
        "def plot_sample_length_distribution(sample_texts):\n",
        "    \"\"\"Plots the sample length distribution.\n",
        "\n",
        "    # Arguments\n",
        "        samples_texts: list, sample texts.\n",
        "    \"\"\"\n",
        "\n",
        "    plt.hist([len(s) for s in sample_texts], 50)\n",
        "    plt.xlabel('Length of a sample')\n",
        "    plt.ylabel('Number of samples')\n",
        "    plt.title('Sample length distribution')\n",
        "    plt.show()\n",
        "\n",
        "plot_sample_length_distribution(data['data/review_body'].tolist())"
      ],
      "metadata": {
        "cell_id": "18acf6975f474e2a8aa3b51ee46dac74",
        "tags": [],
        "deepnote_to_be_reexecuted": true,
        "source_hash": "15a0b1d8",
        "execution_start": 1654259701509,
        "execution_millis": 998,
        "deepnote_cell_type": "code",
        "deepnote_cell_height": 629,
        "deepnote_output_heights": [
          280
        ],
        "execution": {
          "iopub.status.busy": "2022-06-03T14:43:38.509418Z",
          "iopub.execute_input": "2022-06-03T14:43:38.509839Z",
          "iopub.status.idle": "2022-06-03T14:43:39.297580Z",
          "shell.execute_reply.started": "2022-06-03T14:43:38.509801Z",
          "shell.execute_reply": "2022-06-03T14:43:39.296771Z"
        },
        "trusted": true,
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        },
        "id": "lGRkFrDjaGSt",
        "outputId": "cf087660-5fa4-4519-c122-b4dc5877d016"
      },
      "execution_count": 183,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZEAAAEWCAYAAACnlKo3AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZgdZZn38e8PAoJASAIxhiQQ1KiDOLJECAOjCAoBR8Kr6MDFSESGqODCq6OCWxBEUWcczagoA5GADBhRISIYQlhelwkk7Ls0AUzCFgkQFkEC9/tH3Q3F4XR3dXWf0+n073Nd5+qqp56quqt6ubvqeeopRQRmZmZ1rDfQAZiZ2eDlJGJmZrU5iZiZWW1OImZmVpuTiJmZ1eYkYmZmtTmJ2DpJ0vGSflpz3XskvbO/Y6qw34mSQtKwmut/SNLvS/NPSHpNP8X2BUmn9UecTba9dca6fn9sz9rLScT6laQ9JP1R0mOSVkn6g6S3DnRca6NWJ6uI2DQilvYQw56SllfY1tcj4l/7I67G446IP2esz/XH9q29+uU/CTMAScOBC4GPAXOBDYF/BJ4ZyLisbyQNi4g1Ax2HrZ18JWL96fUAEXFORDwXEX+NiEsi4kYASa+VdJmkhyX9RdLZkkZ0rpz/oX5W0o2SnpR0uqQxki6W9LikSyWNzLqdt1RmSLpP0v2S/q2rwCRNySukRyXdIGnPKgckaT1Jx0q6K+OeK2lUQwzTJf05j+mLpXU3ljRH0iOSbpP0uc7/+iWdBWwN/Dpv5XyutNtDm22vSWxbSJonabWkq4HXNiwPSa/L6f0l3ZrncYWkf5O0CXAxsFXG8ISkrfJW4HmSfippNfChLm4PfrjZuZd0hqSvleZfuNppdtyNt8cyhnl5Jdsh6cjSto7P78GZeSy3SJrc83fSWsVJxPrTn4Dn8g/nfp1/8EsEfAPYCvg7YAJwfEOd9wHvokhI76H4I/cFYDTFz+snG+q/A5gE7AN8vtntIUnjgN8AXwNGAf8G/ELS6ArH9AngQODtGfcjwA8a6uwBvAHYG/iKpL/L8pnAROA1eUz/0rlCRHwQ+DPwnryV860K22v0A+BpYCzw4fx05XTgIxGxGbA9cFlEPAnsB9yXMWwaEfdl/WnAecAI4OwuttnjuW/Uw3F3OhdYTnG+DwK+Lmmv0vIDss4IYB7w/Z72a63jJGL9JiJWU/wBDOC/gZX5H+WYXN4REQsi4pmIWAl8h+KPc9l/RcSDEbEC+B1wVURcFxFPA78Cdmyo/9WIeDIibgJ+AhzSJLR/AS6KiIsi4vmIWAAsAfavcFgfBb4YEcsj4hmKpHdQQ6PyV/Oq6wbgBuAtWf4B4OsR8UhELAdmVdhfd9t7QTZCvw/4Sh7/zcCcbrb5LLCdpOEZz7U9xPC/EXF+nq+/dhNnT+e+VyRNAHYHPh8RT0fE9cBpwGGlar/P7+VzwFk0OT/WPk4i1q8i4raI+FBEjKf4j3cr4LsAeWvq3Lydshr4KbBlwyYeLE3/tcn8pg31l5Wm7839NdoGeH/eynpU0qMUyW5shUPaBvhVab3bgOeAMaU6D5SmnyrFuFVDfOXp7nS1vbLRFG2ajcfflfdRJM17JV0pabceYqgSa5Vz31tbAasi4vGGbY8rzTeen43UTz3FrPecRKxlIuJ24AyKZALwdYqrlDdHxHCKKwT1cTcTStNbA/c1qbMMOCsiRpQ+m0TEyRW2vwzYr2HdjfJKqSf3A+O7iBWKc1HXSmANLz/+piJicURMA14FnE/R8aG7GKrE1tW5fxJ4ZWnZq3ux7fuAUZI2a9h2lfNtA8BJxPqNpDdK+oyk8Tk/geIWx6KsshnwBPBYtlN8th92+2VJr5T0JuBw4GdN6vwUeI+kfSWtL2mjbOwd36Ruox8BJ0naBkDSaEnTKsY2FzhO0sg83o83LH+Qor2k1/JWzi+B4/P4twOmN6sraUNJh0raPCKeBVYDz5di2ELS5jXC6OrcXw/sL2mUpFcDxzSs1+VxR8Qy4I/AN/L79PfAERTfQ1sLOYlYf3oc2BW4StKTFMnjZuAzufyrwE7AYxQN3b/sh31eCXQAC4F/j4hLGivkH6ZpFA30KymuLj5LtZ//71E03l4i6XGKY9q1YmwnUDQQ3w1cStFQXe7u/A3gS3mrrMueZd34OMWtrgcorvh+0k3dDwL35G3EjwKHwgtXi+cASzOO3tyS6urcn0XRlnMPcAkvT+w9HfchFB0S7qNoB5sZEZf2Ii5rI/mlVDYYSZpI8cd5g8HyDIOkjwEHR0RjZwKzQctXImYtImmspN1VPGvyBoorsl8NdFxm/ck9GsxaZ0Pgx8C2wKMUzzb8cEAjMutnvp1lZma1+XaWmZnVNuRuZ2255ZYxceLEgQ7DzGzQuOaaa/4SEU2HCRpySWTixIksWbJkoMMwMxs0JHU5GoJvZ5mZWW1OImZmVpuTiJmZ1eYkYmZmtTmJmJlZbU4iZmZWm5OImZnV5iRiZma1OYmYmVltQ+6J9b6YeOxvmpbfc/K72xyJmdnawVciZmZWm5OImZnV5iRiZma1OYmYmVltLUsikt4g6frSZ7WkYySNkrRA0p35dWTWl6RZkjok3Shpp9K2pmf9OyVNL5XvLOmmXGeWJLXqeMzM7OValkQi4o6I2CEidgB2Bp4CfgUcCyyMiEnAwpwH2A+YlJ8ZwCkAkkYBM4FdgV2AmZ2JJ+scWVpvaquOx8zMXq5dt7P2Bu6KiHuBacCcLJ8DHJjT04Azo7AIGCFpLLAvsCAiVkXEI8ACYGouGx4Ri6J4UfyZpW2ZmVkbtCuJHAyck9NjIuL+nH4AGJPT44BlpXWWZ1l35cublL+MpBmSlkhasnLlyr4ch5mZlbQ8iUjaEDgA+HnjsryCiFbHEBGnRsTkiJg8enTT1wSbmVkN7bgS2Q+4NiIezPkH81YU+fWhLF8BTCitNz7Luisf36TczMzapB1J5BBevJUFMA/o7GE1HbigVH5Y9tKaAjyWt73mA/tIGpkN6vsA83PZaklTslfWYaVtmZlZG7R07CxJmwDvAj5SKj4ZmCvpCOBe4ANZfhGwP9BB0ZPrcICIWCXpRGBx1jshIlbl9FHAGcDGwMX5MTOzNmlpEomIJ4EtGsoepuit1Vg3gKO72M5sYHaT8iXA9v0SrJmZ9ZqfWDczs9qcRMzMrDYnETMzq81JxMzManMSMTOz2pxEzMysNicRMzOrzUnEzMxqcxIxM7PanETMzKw2JxEzM6vNScTMzGpzEjEzs9qcRMzMrDYnETMzq81JxMzManMSMTOz2pxEzMysNicRMzOrraVJRNIISedJul3SbZJ2kzRK0gJJd+bXkVlXkmZJ6pB0o6SdStuZnvXvlDS9VL6zpJtynVmS1MrjMTOzl2r1lcj3gN9GxBuBtwC3AccCCyNiErAw5wH2AyblZwZwCoCkUcBMYFdgF2BmZ+LJOkeW1pva4uMxM7OSliURSZsDbwNOB4iIv0XEo8A0YE5WmwMcmNPTgDOjsAgYIWkssC+wICJWRcQjwAJgai4bHhGLIiKAM0vbMjOzNmjllci2wErgJ5Kuk3SapE2AMRFxf9Z5ABiT0+OAZaX1l2dZd+XLm5S/jKQZkpZIWrJy5co+HpaZmXVqZRIZBuwEnBIROwJP8uKtKwDyCiJaGEPnfk6NiMkRMXn06NGt3p2Z2ZDRyiSyHFgeEVfl/HkUSeXBvBVFfn0ol68AJpTWH59l3ZWPb1JuZmZt0rIkEhEPAMskvSGL9gZuBeYBnT2spgMX5PQ84LDspTUFeCxve80H9pE0MhvU9wHm57LVkqZkr6zDStsyM7M2GNbi7X8COFvShsBS4HCKxDVX0hHAvcAHsu5FwP5AB/BU1iUiVkk6EVic9U6IiFU5fRRwBrAxcHF+zMysTVqaRCLiemByk0V7N6kbwNFdbGc2MLtJ+RJg+z6GaWZmNfmJdTMzq81JxMzManMSMTOz2pxEzMysNicRMzOrzUnEzMxqcxIxM7PanETMzKw2JxEzM6vNScTMzGrrMYlI+pSk4Tkw4umSrpW0TzuCMzOztVuVK5EPR8RqitFzRwIfBE5uaVRmZjYoVEkiyq/7A2dFxC2lMjMzG8KqJJFrJF1CkUTmS9oMeL61YZmZ2WBQZSj4I4AdgKUR8ZSkLch3fZiZ2dBW5UokgO2AT+b8JsBGLYvIzMwGjSpJ5IfAbsAhOf848IOWRWRmZoNGldtZu0bETpKuA4iIR/J1t2ZmNsRVuRJ5VtL6FLe1kDQaN6ybmRnVksgs4FfAqySdBPwe+HpLozIzs0GhxyQSEWcDnwO+AdwPHBgRP6+ycUn3SLpJ0vWSlmTZKEkLJN2ZX0dmuSTNktQh6UZJO5W2Mz3r3ylpeql859x+R67r51fMzNqoyySSf+xHSRoFPAScA/wP8GCWVfWOiNghIibn/LHAwoiYBCzMeYD9gEn5mQGc0hkHMBPYFdgFmNmZeLLOkaX1pvYiLjMz66PuGtavoWgHafbffQCvqbnPacCeOT0HuAL4fJafGREBLJI0QtLYrLsgIlYBSFoATJV0BTA8IhZl+ZnAgcDFNeMyM7Ne6jKJRMS2/bD9AC6RFMCPI+JUYExE3J/LHwDG5PQ4YFlp3eVZ1l358iblLyNpBsXVDVtvvXVfjsfMzEqqdPFF0nuBPSiSwu8i4vyK298jIlZIehWwQNLt5YUREZlgWiqT16kAkydPbvn+zMyGiipDwf8Q+ChwE3Az8FFJlR42jIgV+fUhih5eu1C0qYzNbY+laG8BWAFMKK0+Psu6Kx/fpNzMzNqkShffvYB9I+InEfETioEY9+ppJUmb5GCNSNqEYij5m4F5QGcPq+nABTk9Dzgse2lNAR7L217zgX0kjcwG9X2A+blstaQp2SvrsNK2zMysDarczuoAtgbuzfkJWdaTMcCvstftMOB/IuK3khYDcyUdkdv8QNa/iCJBdQBPkYM8RsQqSScCi7PeCZ2N7MBRwBnAxhQN6m5UNzNroypJZDPgNklX5/xbgSWS5gFExAHNVoqIpcBbmpQ/DOzdpDyAo7vY1mxgdpPyJcD2FY7BzMxaoEoS+UrLozAzs0GpxyQSEVcCSBperl+6pWRmZkNUj0kkn7E4AXiaYuBF0beHDc3MbB1R5XbWZ4HtI+IvrQ7GzMwGlypdfO+i6C1lZmb2ElWuRI4D/ijpKuCZzsKI+GTXq5iZ2VBQJYn8GLiM4ol1v4zKzMxeUCWJbBARn255JGZmNuhUaRO5WNIMSWMb3jFiZmZDXJUrkUPy63GlMnfxNTOzSg8b9sd7RczMbB1U9X0i2wPbARt1lkXEma0KyszMBocqT6zPpHhF7XYUI+3uB/wecBIxMxviqjSsH0Qx6u4DEXE4xci8m7c0KjMzGxSqJJG/RsTzwJochPEhXvqmQTMzG6KqtIkskTQC+G/gGuAJ4H9bGpWZmQ0KVXpnHZWTP5L0W2B4RNzY2rDMzGww6PF2lqTd8x3pAHsAH5K0TWvDMjOzwaBKm8gpwFOS3gJ8hmJUX/fMMjOzSklkTb7/fBrw/Yj4AcV71yuRtL6k6yRdmPPbSrpKUoekn0naMMtfkfMduXxiaRvHZfkdkvYtlU/Nsg5Jx1aNyczM+keVJPK4pOOAfwF+I2k9YINe7ONTwG2l+W8C/xkRrwMeAY7I8iOAR7L8P7MekrYDDgbeBEwFfpiJaX3gBxTPrWwHHJJ1zcysTaokkX+meI/IERHxADAe+HaVjUsaD7wbOC3nBewFnJdV5gAH5vS0nCeX7531pwHnRsQzEXE30AHskp+OiFgaEX8Dzs26ZmbWJlV6Zz0AfKc0/2eqt4l8F/gcL97+2gJ4NCLW5PxyYFxOjwOW5T7WSHos648DFpW2WV5nWUP5rs2CyPfEzwDYeuutK4ZuZmY9qXIlUoukfwIeiohrWrWPqiLi1IiYHBGTR48ePdDhmJmtMyoNwFjT7sABkvanGLhxOPA9YISkYXk1Mh5YkfVXUDwJv1zSMIqhVR4ulXcqr9NVuZmZtUGXVyKSFubXb9bZcEQcFxHjI2IiRcP4ZRFxKHA5xXhcANOBC3J6Xs6Tyy/LXmHzgIOz99a2wCTgamAxMCl7e22Y+5hXJ1YzM6unuyuRsZL+geJq4lxA5YURcW3NfX4eOFfS14DrgNOz/HTgLEkdwCqKpEBE3CJpLnArsAY4OiKeA5D0cWA+sD4wOyJuqRmTmZnV0F0S+QrwZYrbRN9pWBYUvawqiYgrgCtyeilFz6rGOk8D7+9i/ZOAk5qUX0QxPL2ZmQ2ALpNIRJwHnCfpyxFxYhtjMjOzQaJKF98TJR0AvC2LroiIC1sblpmZDQZVBmD8BsVT57fm51OSvt7qwMzMbO1XpYvvu4Ed8sVUSJpD0SD+hVYGZmZma7+qDxuOKE371bhmZgZUuxL5BnCdpMspuvm+DfCIuWZmVqlh/RxJVwBvzaLP53haZmY2xFUa9iQi7sdPg5uZWYOWDcBoZmbrPicRMzOrrdskkm8QvL1dwZiZ2eDSbRLJgQ7vkOQ3OZmZ2ctUaVgfCdwi6Wrgyc7CiDigZVGZmdmgUCWJfLnlUZiZ2aBU5TmRKyVtA0yKiEslvZLi/R1mZjbEVRmA8UjgPODHWTQOOL+VQZmZ2eBQpYvv0RTvS18NEBF3Aq9qZVBmZjY4VEkiz0TE3zpnJA2jeLOhmZkNcVWSyJWSvgBsLOldwM+BX7c2LDMzGwyqJJFjgZXATcBHKN5p/qWeVpK0kaSrJd0g6RZJX83ybSVdJalD0s8kbZjlr8j5jlw+sbSt47L8Dkn7lsqnZlmHJI8sbGbWZlV6Zz2fL6K6iuI21h0RUeV21jPAXhHxhKQNgN9Luhj4NPCfEXGupB8BRwCn5NdHIuJ1kg4Gvgn8s6TtgIOBNwFbAZdKen3u4wfAu4DlwGJJ8yLi1uqHb2ZmfVGld9a7gbuAWcD3gQ5J+/W0XhSeyNkN8hPAXhS9vQDmAAfm9LScJ5fvLUlZfm5EPBMRdwMdwC756YiIpdlmc27WNTOzNqnysOF/AO+IiA4ASa8FfgNc3NOKktYHrgFeR3HVcBfwaESsySrLKboMk1+XAUTEGkmPAVtk+aLSZsvrLGso37XC8ZiZWT+p0ibyeGcCSUuBx6tsPCKei4gdgPEUVw5v7H2IfSdphqQlkpasXLlyIEIwM1sndXklIum9OblE0kXAXIrbUe8HFvdmJxHxaL5edzdghKRheTUyHliR1VYAE4Dl2Y14c+DhUnmn8jpdlTfu/1TgVIDJkye7e7KZWT/p7krkPfnZCHgQeDuwJ0VPrY172rCk0ZJG5PTGFA3gtwGXAwdltenABTk9L+fJ5ZdlA/484ODsvbUtMAm4miKRTcreXhtSNL777YtmZm3U5ZVIRBzex22PBeZku8h6wNyIuFDSrcC5kr4GXAecnvVPB86S1AGsokgKRMQtkuYCtwJrgKNziHokfRyYTzGW1+yIuKWPMZuZWS/02LCe//1/AphYrt/TUPARcSOwY5PypRTtI43lT1PcKmu2rZOAk5qUX0Tx3IqZmQ2AKr2zzqe4Svg18HxrwzEzs8GkShJ5OiJmtTwSMzMbdKokke9JmglcQvEUOgARcW3LojIzs0GhShJ5M/BBiifNO29ndT55bmZmQ1iVJPJ+4DXl4eDNzMyg2hPrNwMjWh2ImZkNPlWuREYAt0tazEvbRLrt4mtmZuu+KklkZsujMDOzQanK+0SubEcgZmY2+FR5Yv1xXnyn+oYU7wV5MiKGtzIwMzNb+1W5Etmsc7r0kqgprQzKzMwGhyq9s16Qbys8H9i3x8pmZrbOq3I7672l2fWAycDTLYvIzMwGjSq9s95Tml4D3IPfZW5mZlRrE+nre0XMzGwd1d3rcb/SzXoRESe2IB4zMxtEursSebJJ2SbAEcAWgJOImdkQ193rcf+jc1rSZsCngMOBc4H/6Go9MzMbOrptE5E0Cvg0cCgwB9gpIh5pR2BmZrb2665N5NvAe4FTgTdHxBNti8rMzAaF7h42/AywFfAl4D5Jq/PzuKTVPW1Y0gRJl0u6VdItkj6V5aMkLZB0Z34dmeWSNEtSh6QbJe1U2tb0rH+npOml8p0l3ZTrzMon6s3MrE26TCIRsV5EbBwRm0XE8NJns4rjZq0BPhMR21EMk3K0pO2AY4GFETEJWJjzAPsBk/IzAzgFXrilNhPYFdgFmNmZeLLOkaX1pvbm4M3MrG96NexJb0TE/Z3vYY+Ix4HbgHEUDyrOyWpzgANzehpwZg6tsggYIWksxRArCyJiVbbHLACm5rLhEbEoIgI4s7QtMzNrg5YlkTJJE4EdgauAMRFxfy56ABiT0+OAZaXVlmdZd+XLm5Q32/8MSUskLVm5cmWfjsXMzF7U8iQiaVPgF8AxEfGStpS8goimK/ajiDg1IiZHxOTRo0e3endmZkNGlbGzapO0AUUCOTsifpnFD0oaGxH35y2ph7J8BTChtPr4LFsB7NlQfkWWj29Sv+0mHvubpuX3nPzuNkdiZtZeLbsSyZ5SpwO3RcR3SovmAZ09rKYDF5TKD8teWlOAx/K213xgH0kjs0F9H2B+LlstaUru67DStszMrA1aeSWyO/BB4CZJ12fZF4CTgbmSjgDuBT6Qyy4C9gc6gKcono4nIlZJOhFYnPVOiIhVOX0UcAawMXBxfszMrE1alkQi4vdAV89t7N2kfgBHd7Gt2cDsJuVLgO37EKaZmfVBW3pnmZnZuslJxMzManMSMTOz2pxEzMysNicRMzOrzUnEzMxqcxIxM7PanETMzKw2JxEzM6vNScTMzGpzEjEzs9qcRMzMrDYnETMzq81JxMzManMSMTOz2pxEzMysNicRMzOrzUnEzMxqcxIxM7PaWpZEJM2W9JCkm0tloyQtkHRnfh2Z5ZI0S1KHpBsl7VRaZ3rWv1PS9FL5zpJuynVmSerqfe5mZtYirbwSOQOY2lB2LLAwIiYBC3MeYD9gUn5mAKdAkXSAmcCuwC7AzM7Ek3WOLK3XuC8zM2uxliWRiPh/wKqG4mnAnJyeAxxYKj8zCouAEZLGAvsCCyJiVUQ8AiwApuay4RGxKCICOLO0LTMza5N2t4mMiYj7c/oBYExOjwOWleotz7Luypc3KW9K0gxJSyQtWblyZd+OwMzMXjBgDet5BRFt2tepETE5IiaPHj26Hbs0MxsS2p1EHsxbUeTXh7J8BTChVG98lnVXPr5JuZmZtVG7k8g8oLOH1XTgglL5YdlLawrwWN72mg/sI2lkNqjvA8zPZaslTcleWYeVtmVmZm0yrFUblnQOsCewpaTlFL2sTgbmSjoCuBf4QFa/CNgf6ACeAg4HiIhVkk4EFme9EyKis7H+KIoeYBsDF+fHzMzaqGVJJCIO6WLR3k3qBnB0F9uZDcxuUr4E2L4vMZqZWd/4iXUzM6vNScTMzGpzEjEzs9qcRMzMrDYnETMzq61lvbMMJh77m6bl95z87jZHYmbWGr4SMTOz2pxEzMysNicRMzOrzUnEzMxqcxIxM7PanETMzKw2JxEzM6vNScTMzGpzEjEzs9r8xPoA8JPsZrau8JWImZnV5iRiZma1OYmYmVltbhNZi7itxMwGm0GfRCRNBb4HrA+cFhEnD3BI/c7JxczWVoM6iUhaH/gB8C5gObBY0ryIuHVgI2sPJxczG2iDOokAuwAdEbEUQNK5wDRgSCSRrnSVXPqTE5WZweBPIuOAZaX55cCujZUkzQBm5OwTku6oub8tgb/UXLeV2h6Xvlmpms9X7ziu3nFcvdOXuLbpasFgTyKVRMSpwKl93Y6kJRExuR9C6leOq3ccV+84rt4ZanEN9i6+K4AJpfnxWWZmZm0w2JPIYmCSpG0lbQgcDMwb4JjMzIaMQX07KyLWSPo4MJ+ii+/siLilhbvs8y2xFnFcveO4esdx9c6QiksR0YrtmpnZEDDYb2eZmdkAchIxM7PanEQqkDRV0h2SOiQd24b9TZB0uaRbJd0i6VNZPkrSAkl35teRWS5JszK+GyXtVNrW9Kx/p6Tp/RTf+pKuk3Rhzm8r6arc/8+ykwOSXpHzHbl8Ymkbx2X5HZL27YeYRkg6T9Ltkm6TtNvacL4k/d/8Ht4s6RxJGw3E+ZI0W9JDkm4ulfXb+ZG0s6Sbcp1ZktSHuL6d38cbJf1K0oiezkNXv6Ndnes6cZWWfUZSSNpybThfWf6JPGe3SPpWW89XRPjTzYeiwf4u4DXAhsANwHYt3udYYKec3gz4E7Ad8C3g2Cw/FvhmTu8PXAwImAJcleWjgKX5dWROj+yH+D4N/A9wYc7PBQ7O6R8BH8vpo4Af5fTBwM9yers8j68Ats3zu34fY5oD/GtObwiMGOjzRfEw7N3AxqXz9KGBOF/A24CdgJtLZf12foCrs65y3f36ENc+wLCc/mYprqbngW5+R7s613XiyvIJFB157gW2XEvO1zuAS4FX5Pyr2nm+WvaHcF35ALsB80vzxwHHtTmGCyjGB7sDGJtlY4E7cvrHwCGl+nfk8kOAH5fKX1KvZizjgYXAXsCF+Uvwl9Iv/QvnK3/ZdsvpYVlPjeewXK9mTJtT/LFWQ/mAni9eHFFhVB7/hcC+A3W+gIkNf3z65fzksttL5S+p19u4Gpb9H+DsnG56Hujid7S7n826cQHnAW8B7uHFJDKg54viD/87m9Rry/ny7ayeNRtaZVy7dp63NHYErgLGRMT9uegBYExOdxVjK2L/LvA54Pmc3wJ4NCLWNNnHC/vP5Y9l/f6Oa1tgJfATFbfZTpO0CQN8viJiBfDvwJ+B+ymO/xoG/nx16q/zMy6n+zs+gA9T/KdeJ67ufjZ7TdI0YEVE3NCwaKDP1+uBf8zbUFdKemvNuGqdLyeRtZikTYFfAMdExOrysij+VWhr/2xJ/wQ8FBHXtHO/FQyjuMQ/JSJ2BJ6kuD3zggE6XyMpBgTdFtgK2ASY2s4YqhqI89MTSV8E1gBnrwWxvBL4AvCVgY6liWEUV7tTgM8Cc6u2sfQHJ5GeDcjQKpI2oEggZ0fEL7P4QUljc/lY4KEeYuzv2HcHDpB0D3AuxS2t7wEjJHU+uFrexwv7zyRwQrkAAAWoSURBVOWbAw+3IK7lwPKIuCrnz6NIKgN9vt4J3B0RKyPiWeCXFOdwoM9Xp/46Pytyut/ik/Qh4J+AQzPB1YnrYbo+1731Wop/Bm7In//xwLWSXl0jrv4+X8uBX0bhaoq7BFvWiKve+ertfdWh9qHI8kspfoA6G6He1OJ9CjgT+G5D+bd5aUPot3L63by0Ye/qLB9F0VYwMj93A6P6KcY9ebFh/ee8tDHuqJw+mpc2FM/N6Tfx0ga/pfS9Yf13wBty+vg8VwN6vihGlL4FeGXuaw7wiYE6X7z8Xnq/nR9e3lC8fx/imkrxOofRDfWange6+R3t6lzXiath2T282CYy0Ofro8AJOf16iltVatf5atkfwnXpQ9H74k8UPRq+2Ib97UFxa+FG4Pr87E9xz3IhcCdFb4zOH0hRvJzrLuAmYHJpWx8GOvJzeD/GuCcvJpHX5C9FR/4QdvYS2SjnO3L5a0rrfzHjvYOKPVN6iGcHYEmes/Pzl3bAzxfwVeB24GbgrPyFbvv5As6haJd5luI/1yP68/wAk/MY7wK+T0Mnh17G1UHxh7DzZ/9HPZ0Huvgd7epc14mrYfk9vJhEBvp8bQj8NLd3LbBXO8+Xhz0xM7Pa3CZiZma1OYmYmVltTiJmZlabk4iZmdXmJGJmZrU5idg6RdITLd7+Mfn0cp/3p2LU3kslXS/pn/snwtaQNLHZiLZmTiJmvXMMxcOD/WFHgIjYISJ+1k/bNGsrJxFb50l6raTfSrpG0u8kvTHLz8h3OfxR0lJJB2X5epJ+mO9nWCDpIkkHSfokxRhYl0u6vLT9kyTdIGmRpDFN9j9K0vn5rolFkv5e0qsoHhB7a16JvLZhnSMlLc7t/qJ89VOq8/Zc9/oceHIzSZtKWijp2nxfxbSsOzGP5wxJf5J0tqR3SvpDvutil6x3vKSzJP1vlh/ZZL/rq3jnx+I8po/05ftjg1xfn8j1x5+16QM80aRsITApp3cFLsvpMyieyl2P4t0LHVl+EHBRlr8aeAQ4KJfdQz6pnPMBvCenvwV8qcn+/wuYmdN7Adfn9J7kU/9N1tmiNP014BNN6vwa2D2nN6UYzmIYMDzLtqR48lgUQ2WsAd6cx3UNMDuXTQPOz3WOpxgGY+NcfxlF4pxIDrUBzOg8Toon8JcA2w70996fgfl0DrRltk7KkZD/Afh5aWDTV5SqnB8RzwO3lq4i9gB+nuUPlK86mvgbxXtCoPjD/K4mdfYA3gcQEZdJ2kLS8B5C317S1yherrUpxbsgGv0B+I6ksykG4FueA3d+XdLbKAbiG8eLQ7zfHRE3AUi6BVgYESHpJook0emCiPgr8Nc89l0ohh/ptA/w951XbhQDRU6iGBvKhhgnEVvXrUfxjoQdulj+TGm6zvDZz0ZE59hBz9F/v1NnAAdGxA05ou2ejRUi4mRJv6EYB+kPKl5/OgUYDewcEc/miLMb5SrlY32+NP98Q9yNYyE1zoviyqhZYrMhxm0itk6L4j0sd0t6P7zwPuy39LDaH4D3ZdvIGF76B/xxilcW98bvgENz/3sCf4mG98M0sRlwf15ZHNqsgqTXRsRNEfFNYDHwRoqrgocygbwD2KaXsQJMU/Eu+C0ojn1xw/L5wMcyNiS9XsVLwGwI8pWIrWteKan81rjvUPwRPkXSl4ANKN6F0vh2urJfAHtTDEe+jGJk1Mdy2anAbyXdFxHvqBjT8cBsSTcCTwHTK6zzZYq3Wa7Mr80S1zGZKJ6nGHL+4qz367xFtYRiBOHeuhG4nKJN5MSIuE/FGzY7nUZx++taFfcIVwIH1tiPrQM8iq9ZE5I2jYgn8r/xqykasB8Y6LhaTdLxFJ0T/n2gY7HBwVciZs1dKGkExbsaThwKCcSsDl+JmJlZbW5YNzOz2pxEzMysNicRMzOrzUnEzMxqcxIxM7Pa/j86RuKbg/AXfAAAAABJRU5ErkJggg==\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Stage 2.5: Choose a model\n",
        "\n",
        "The data exploration we performed in the previous stage helps us in answering the following questions,\n",
        "\n",
        "- **Which model to choose?** - If the data is less and we have enough representation per class, going for a simple MLP or ML model would suffice. If you have enough data and there is no pretrained model for that domain, you can train a sequence model from scratch. If you find  pre-trained models in a particular domain and you want to apply it to a similar domain data where you have just enough data, then choose transfer learning.\n",
        "\n",
        "- **How do we prepare the data efficiently?** - Based on the model we choose, the type of preprocessing changes. If we choose a simple MLP or ML model, we perform vectorization using bag of words after appropriate preprocessing. For sequence models, we may need to preserve the context of the data, so we will go for embeddings like word2vec, sentenece embeddigns etc."
      ],
      "metadata": {
        "cell_id": "d9ea6dd150a747e8b4de9bcba51beed8",
        "tags": [],
        "deepnote_cell_type": "markdown",
        "deepnote_cell_height": 46,
        "id": "fzta-kFCaGSt"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Stage 3: Prepare the data\n",
        "\n",
        "Data should be converted to a form that the model can understand. Data preparation involves two steps,\n",
        "\n",
        "- **Feature engineering:**\n",
        "Feature engineering involves creating or extracting features out of the textual data so that it can be fed into the model. There are two steps that needs to be done while extracting features, \n",
        "\n",
        "    1. **Tokenization:** Converting the texts into smaller units like words(tokens) or n-grams. Based on the unique tokens, a dictionary will be created, which forms the vocabulary for the given dataset.\n",
        "\n",
        "    2. **Vectorization:** Converting the textual data into numbers using the vocabulary. Some common techniques used for vectorizations are one-hot encoding (based on the presence or absence of a token), count encoding (based on the number of occurance of a token), and Tf-idf encoding (based on weights assigned to the frequency of occurance of tokens across the documents).\n",
        "\n",
        "- **Feature Selection:** \n",
        "Once the features are created, we can carefully select relevant features using feature importance through some statistical measure. Using the right features, both in terms of quantity and quality will help in building better models.\n",
        "\n",
        "#### Important points to remember: \n",
        "- Always make sure that the data is shuffled and sampled.\n",
        "- Split the data into train, test and validation splits.\n",
        "- Save the vectorizer after vectorizing the training data and use it to transform the test data."
      ],
      "metadata": {
        "cell_id": "bcbecfc428eb42f1ad4e1e7dc7cea339",
        "tags": [],
        "deepnote_cell_type": "markdown",
        "deepnote_cell_height": 635.03125,
        "id": "d_c1TWO3aGSu"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Take a sample of 50% of the dataset for the purpose of demonstration\n",
        "data = data.sample(frac=0.3, random_state=42)\n",
        "data.head()"
      ],
      "metadata": {
        "cell_id": "30bb8197b4b34bdd8523d768338d00ac",
        "tags": [],
        "deepnote_to_be_reexecuted": true,
        "source_hash": "66b28070",
        "execution_start": 1654259702513,
        "execution_millis": 41,
        "deepnote_cell_type": "code",
        "deepnote_cell_height": 431,
        "execution": {
          "iopub.status.busy": "2022-06-03T14:43:39.298708Z",
          "iopub.execute_input": "2022-06-03T14:43:39.299640Z",
          "iopub.status.idle": "2022-06-03T14:43:39.320057Z",
          "shell.execute_reply.started": "2022-06-03T14:43:39.299602Z",
          "shell.execute_reply": "2022-06-03T14:43:39.319184Z"
        },
        "trusted": true,
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "SSZJWaLxaGSu",
        "outputId": "a34bd15d-af97-45ad-eb81-8be3c60e2d77"
      },
      "execution_count": 184,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                        data/review_body  data/star_rating\n",
              "87666  b\"The motor spins and is not stationary. That ...                 3\n",
              "47509                                            b'Fake'                 1\n",
              "69877  b\"Even though there are finally some silicone ...                 5\n",
              "11951  b\"Absolutely Wonderful:  This product is every...                 5\n",
              "73562  b\"Love this Bluetooth speaker...easy setup, wo...                 5"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-89049e20-ad4d-45a8-9056-dc30489c5fb1\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>data/review_body</th>\n",
              "      <th>data/star_rating</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>87666</th>\n",
              "      <td>b\"The motor spins and is not stationary. That ...</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>47509</th>\n",
              "      <td>b'Fake'</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>69877</th>\n",
              "      <td>b\"Even though there are finally some silicone ...</td>\n",
              "      <td>5</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11951</th>\n",
              "      <td>b\"Absolutely Wonderful:  This product is every...</td>\n",
              "      <td>5</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>73562</th>\n",
              "      <td>b\"Love this Bluetooth speaker...easy setup, wo...</td>\n",
              "      <td>5</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-89049e20-ad4d-45a8-9056-dc30489c5fb1')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-89049e20-ad4d-45a8-9056-dc30489c5fb1 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-89049e20-ad4d-45a8-9056-dc30489c5fb1');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 184
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(\"Number of datapoints after samplig:\", data.shape)"
      ],
      "metadata": {
        "cell_id": "1c9a6420385a400ba79dd6b9d1b2bd2a",
        "tags": [],
        "deepnote_to_be_reexecuted": true,
        "source_hash": "650603e3",
        "execution_start": 1654259702606,
        "execution_millis": 0,
        "deepnote_cell_type": "code",
        "deepnote_cell_height": 112.1875,
        "execution": {
          "iopub.status.busy": "2022-06-03T14:43:39.321308Z",
          "iopub.execute_input": "2022-06-03T14:43:39.321650Z",
          "iopub.status.idle": "2022-06-03T14:43:39.326940Z",
          "shell.execute_reply.started": "2022-06-03T14:43:39.321615Z",
          "shell.execute_reply": "2022-06-03T14:43:39.326102Z"
        },
        "trusted": true,
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Px-vBaclaGSv",
        "outputId": "d0387d17-f242-484d-a5bb-a47082ccadb3"
      },
      "execution_count": 186,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Number of datapoints after samplig: (31492, 2)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Convert the star rating to sentiment\n",
        "data['sentiment'] = data['data/star_rating'].apply(lambda rating: \"positive\" if rating >= 3 else \"negative\")\n",
        "data.head()"
      ],
      "metadata": {
        "cell_id": "10576c8b76eb48e6b21f338e060b8905",
        "tags": [],
        "deepnote_to_be_reexecuted": true,
        "source_hash": "82c242fa",
        "execution_start": 1654259702607,
        "execution_millis": 46,
        "deepnote_cell_type": "code",
        "deepnote_cell_height": 431,
        "execution": {
          "iopub.status.busy": "2022-06-03T14:43:39.328613Z",
          "iopub.execute_input": "2022-06-03T14:43:39.329362Z",
          "iopub.status.idle": "2022-06-03T14:43:39.356998Z",
          "shell.execute_reply.started": "2022-06-03T14:43:39.329266Z",
          "shell.execute_reply": "2022-06-03T14:43:39.356153Z"
        },
        "trusted": true,
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "m2CuT4zUaGSv",
        "outputId": "a42c5028-2214-4350-e28c-126c76513230"
      },
      "execution_count": 187,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                        data/review_body  data/star_rating  \\\n",
              "87666  b\"The motor spins and is not stationary. That ...                 3   \n",
              "47509                                            b'Fake'                 1   \n",
              "69877  b\"Even though there are finally some silicone ...                 5   \n",
              "11951  b\"Absolutely Wonderful:  This product is every...                 5   \n",
              "73562  b\"Love this Bluetooth speaker...easy setup, wo...                 5   \n",
              "\n",
              "      sentiment  \n",
              "87666  positive  \n",
              "47509  negative  \n",
              "69877  positive  \n",
              "11951  positive  \n",
              "73562  positive  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-eb8d0953-5f8c-4dac-954a-9f79b7a14339\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>data/review_body</th>\n",
              "      <th>data/star_rating</th>\n",
              "      <th>sentiment</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>87666</th>\n",
              "      <td>b\"The motor spins and is not stationary. That ...</td>\n",
              "      <td>3</td>\n",
              "      <td>positive</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>47509</th>\n",
              "      <td>b'Fake'</td>\n",
              "      <td>1</td>\n",
              "      <td>negative</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>69877</th>\n",
              "      <td>b\"Even though there are finally some silicone ...</td>\n",
              "      <td>5</td>\n",
              "      <td>positive</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11951</th>\n",
              "      <td>b\"Absolutely Wonderful:  This product is every...</td>\n",
              "      <td>5</td>\n",
              "      <td>positive</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>73562</th>\n",
              "      <td>b\"Love this Bluetooth speaker...easy setup, wo...</td>\n",
              "      <td>5</td>\n",
              "      <td>positive</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-eb8d0953-5f8c-4dac-954a-9f79b7a14339')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-eb8d0953-5f8c-4dac-954a-9f79b7a14339 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-eb8d0953-5f8c-4dac-954a-9f79b7a14339');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 187
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# One-hot encode the target label\n",
        "data['sentiment'] = data['sentiment'].map({'positive':1, 'negative':0})\n",
        "\n",
        "# Drop the rating column\n",
        "data.drop(columns=['data/star_rating'], inplace=True)\n",
        "data.head()"
      ],
      "metadata": {
        "cell_id": "e48240a3cad94e3bb9c8c4ee5d79d1d5",
        "tags": [],
        "deepnote_to_be_reexecuted": true,
        "source_hash": "d88949b6",
        "execution_start": 1654259702649,
        "execution_millis": 6,
        "deepnote_cell_type": "code",
        "deepnote_cell_height": 485,
        "execution": {
          "iopub.status.busy": "2022-06-03T14:43:39.358176Z",
          "iopub.execute_input": "2022-06-03T14:43:39.358656Z",
          "iopub.status.idle": "2022-06-03T14:43:39.379166Z",
          "shell.execute_reply.started": "2022-06-03T14:43:39.358617Z",
          "shell.execute_reply": "2022-06-03T14:43:39.378320Z"
        },
        "trusted": true,
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "vgNv6vuKaGSw",
        "outputId": "150767b7-d13f-46b4-ff9b-e82b747f5d56"
      },
      "execution_count": 188,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                        data/review_body  sentiment\n",
              "87666  b\"The motor spins and is not stationary. That ...          1\n",
              "47509                                            b'Fake'          0\n",
              "69877  b\"Even though there are finally some silicone ...          1\n",
              "11951  b\"Absolutely Wonderful:  This product is every...          1\n",
              "73562  b\"Love this Bluetooth speaker...easy setup, wo...          1"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-6c040715-a18b-4325-848d-8297716cc7ca\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>data/review_body</th>\n",
              "      <th>sentiment</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>87666</th>\n",
              "      <td>b\"The motor spins and is not stationary. That ...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>47509</th>\n",
              "      <td>b'Fake'</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>69877</th>\n",
              "      <td>b\"Even though there are finally some silicone ...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11951</th>\n",
              "      <td>b\"Absolutely Wonderful:  This product is every...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>73562</th>\n",
              "      <td>b\"Love this Bluetooth speaker...easy setup, wo...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-6c040715-a18b-4325-848d-8297716cc7ca')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-6c040715-a18b-4325-848d-8297716cc7ca button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-6c040715-a18b-4325-848d-8297716cc7ca');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 188
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Preprocess the reviews\n",
        "data.rename(columns={'data/review_body': 'review'}, inplace=True)\n",
        "data['review'] = data['review'].str.decode('utf-8')\n",
        "data['review'] = data[\"review\"].apply(str.lower)\n",
        "data['review']\n",
        "\n",
        "data.head()"
      ],
      "metadata": {
        "cell_id": "77478e61881a4aeeb2cddc4f8f035f5b",
        "tags": [],
        "deepnote_to_be_reexecuted": true,
        "source_hash": "a36c21ce",
        "execution_start": 1654259702675,
        "execution_millis": 147,
        "deepnote_cell_type": "code",
        "deepnote_cell_height": 503,
        "execution": {
          "iopub.status.busy": "2022-06-03T14:43:39.380500Z",
          "iopub.execute_input": "2022-06-03T14:43:39.380988Z",
          "iopub.status.idle": "2022-06-03T14:43:39.484468Z",
          "shell.execute_reply.started": "2022-06-03T14:43:39.380953Z",
          "shell.execute_reply": "2022-06-03T14:43:39.483652Z"
        },
        "trusted": true,
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "gpzAchB7aGSx",
        "outputId": "9734d08f-b306-4117-893f-5b9390c563c5"
      },
      "execution_count": 189,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                                  review  sentiment\n",
              "87666  the motor spins and is not stationary. that se...          1\n",
              "47509                                               fake          0\n",
              "69877  even though there are finally some silicone ca...          1\n",
              "11951  absolutely wonderful:  this product is everyth...          1\n",
              "73562  love this bluetooth speaker...easy setup, wond...          1"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-bb58fc29-9f6c-4e05-a61d-77c6fbf6181e\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>review</th>\n",
              "      <th>sentiment</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>87666</th>\n",
              "      <td>the motor spins and is not stationary. that se...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>47509</th>\n",
              "      <td>fake</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>69877</th>\n",
              "      <td>even though there are finally some silicone ca...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11951</th>\n",
              "      <td>absolutely wonderful:  this product is everyth...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>73562</th>\n",
              "      <td>love this bluetooth speaker...easy setup, wond...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-bb58fc29-9f6c-4e05-a61d-77c6fbf6181e')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-bb58fc29-9f6c-4e05-a61d-77c6fbf6181e button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-bb58fc29-9f6c-4e05-a61d-77c6fbf6181e');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 189
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Check the samples per class\n",
        "data['sentiment'].value_counts()"
      ],
      "metadata": {
        "cell_id": "02b71411becf4c469da3ad4ceb68fbb5",
        "tags": [],
        "deepnote_to_be_reexecuted": true,
        "source_hash": "18e7352e",
        "execution_start": 1654259702825,
        "execution_millis": 8,
        "deepnote_cell_type": "code",
        "deepnote_cell_height": 174.5625,
        "deepnote_output_heights": [
          59.5625
        ],
        "execution": {
          "iopub.status.busy": "2022-06-03T14:43:39.485831Z",
          "iopub.execute_input": "2022-06-03T14:43:39.486196Z",
          "iopub.status.idle": "2022-06-03T14:43:39.493511Z",
          "shell.execute_reply.started": "2022-06-03T14:43:39.486160Z",
          "shell.execute_reply": "2022-06-03T14:43:39.492440Z"
        },
        "trusted": true,
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Et7TLQELaGSx",
        "outputId": "2b63fee8-eb53-4061-8cd8-caa5744bafb1"
      },
      "execution_count": 190,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1    24067\n",
              "0     7425\n",
              "Name: sentiment, dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 190
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Balance out the classes for the purpose of demonstration\n",
        "positive = data[data['sentiment'] == 1].sample(n=7425, random_state=42)\n",
        "negative = data[data['sentiment'] == 0]\n",
        "train_data = pd.concat([positive, negative]).sample(frac=1).reset_index(drop=True)\n",
        "train_data['sentiment'].value_counts()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_yj1ZzzVxQzg",
        "outputId": "41744162-5d3f-4ded-df32-03cac7c91953"
      },
      "execution_count": 198,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0    7425\n",
              "1    7425\n",
              "Name: sentiment, dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 198
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Sample reviews and its corresponding labels\n",
        "reviews = train_data['review'].tolist()\n",
        "labels = train_data['sentiment'].tolist()\n",
        "\n",
        "print(\"Sample reviews:\")\n",
        "print(reviews[10:16])\n",
        "print(\"Corresponding labels:\")\n",
        "print(labels[10:16])"
      ],
      "metadata": {
        "cell_id": "37c461bdb1b948448a0d925d31fb5a02",
        "tags": [],
        "deepnote_to_be_reexecuted": true,
        "source_hash": "97707d3f",
        "execution_start": 1654259702838,
        "execution_millis": 7,
        "deepnote_cell_type": "code",
        "deepnote_cell_height": 302.75,
        "execution": {
          "iopub.status.busy": "2022-06-03T14:43:39.495400Z",
          "iopub.execute_input": "2022-06-03T14:43:39.496067Z",
          "iopub.status.idle": "2022-06-03T14:43:39.505695Z",
          "shell.execute_reply.started": "2022-06-03T14:43:39.496026Z",
          "shell.execute_reply": "2022-06-03T14:43:39.504828Z"
        },
        "trusted": true,
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4dgVVGwoaGSy",
        "outputId": "a3bb5b76-8f81-4437-ff3a-44436c16d041"
      },
      "execution_count": 200,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Sample reviews:\n",
            "['does not work, waste of my money, what more can i say...requiring more words it ridiculous...four more words then and', \"i ordered a black case as well as a pink. nicely packaged, but the black came very dirty and i noticed it was prone to get a lot of noticeable dust/dirt on it even if it's just sitting on my counter. the &#34;flaps&#34; covering the camera, headphone jack, and charging area are very weak and look like they could be easily broken. however, the case is water resistant and it is definitely shockproof. i drop tested this on concrete and my ipod was perfectly fine, so that's where the forth star comes in. my only real problem with this is the weak flaps and the dust, but the appearance can be fixed by just getting a lighter color.\", 'great price, hits hard, clean sound. my only issue with it was when i removed one of the rca cables it pulled the grounding collar off the amp. no big deal, easy work around but i dont think that should have happened.  i would buy this amp again.', 'nice', \"i expected it to be bad, i mean it's 20$. the only reason i wouldn't buy it again is because it doesn't use 3.5mm jack.\", 'this looked to be a nice skin but it in no way fits my ultrahd flip. it is huge, not fitted, sloppy and moves around, covers parts of lens. not something i would recommend to anyone.']\n",
            "Corresponding labels:\n",
            "[0, 1, 1, 1, 0, 0]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Split the data into train, test and vvalidation split\n",
        "from sklearn.model_selection import train_test_split\n",
        "X_train, X_test, y_train, y_test  = train_test_split(reviews, labels, test_size=.2, random_state=42)"
      ],
      "metadata": {
        "cell_id": "fecba18e0ba249c7a564220e86509f78",
        "tags": [],
        "deepnote_to_be_reexecuted": true,
        "source_hash": "89a79b7c",
        "execution_start": 1654259702894,
        "execution_millis": 651,
        "deepnote_cell_type": "code",
        "deepnote_cell_height": 117,
        "execution": {
          "iopub.status.busy": "2022-06-03T14:43:39.507263Z",
          "iopub.execute_input": "2022-06-03T14:43:39.507932Z",
          "iopub.status.idle": "2022-06-03T14:43:40.085711Z",
          "shell.execute_reply.started": "2022-06-03T14:43:39.507891Z",
          "shell.execute_reply": "2022-06-03T14:43:40.084923Z"
        },
        "trusted": true,
        "id": "GUbD2wwgaGSy"
      },
      "execution_count": 247,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Number of train and test data\n",
        "print(\"Number of train data points:\", len(X_train))\n",
        "print(\"Number of train labels:\", len(y_train))\n",
        "print(\"Number of test data points:\", len(X_test))\n",
        "print(\"Number of test labels:\", len(y_test))"
      ],
      "metadata": {
        "cell_id": "6ef60bb9e24848ea9f57ff10b99ae679",
        "tags": [],
        "deepnote_to_be_reexecuted": true,
        "source_hash": "1bcb2550",
        "execution_start": 1654259703552,
        "execution_millis": 171,
        "deepnote_cell_type": "code",
        "deepnote_cell_height": 244.75,
        "execution": {
          "iopub.status.busy": "2022-06-03T14:43:40.087135Z",
          "iopub.execute_input": "2022-06-03T14:43:40.087529Z",
          "iopub.status.idle": "2022-06-03T14:43:40.093330Z",
          "shell.execute_reply.started": "2022-06-03T14:43:40.087494Z",
          "shell.execute_reply": "2022-06-03T14:43:40.092390Z"
        },
        "trusted": true,
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xmT07OMxaGSy",
        "outputId": "affa39fa-529a-439a-9df4-d7b27b267b15"
      },
      "execution_count": 248,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Number of train data points: 11880\n",
            "Number of train labels: 11880\n",
            "Number of test data points: 2970\n",
            "Number of test labels: 2970\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Load the tokenizer\n",
        "from transformers import DistilBertTokenizerFast\n",
        "MODEL_NAME = 'distilbert-base-uncased-finetuned-sst-2-english'\n",
        "tokenizer = DistilBertTokenizerFast.from_pretrained(MODEL_NAME)"
      ],
      "metadata": {
        "cell_id": "d218bdfa1bf64000af827ed9e67d542c",
        "tags": [],
        "deepnote_to_be_reexecuted": true,
        "source_hash": "c42b759f",
        "execution_start": 1654259703596,
        "execution_millis": 3079,
        "deepnote_cell_type": "code",
        "deepnote_cell_height": 117,
        "execution": {
          "iopub.status.busy": "2022-06-03T14:43:40.127440Z",
          "iopub.execute_input": "2022-06-03T14:43:40.127696Z",
          "iopub.status.idle": "2022-06-03T14:43:54.233200Z",
          "shell.execute_reply.started": "2022-06-03T14:43:40.127672Z",
          "shell.execute_reply": "2022-06-03T14:43:54.232335Z"
        },
        "trusted": true,
        "id": "inWvDL2CaGSz"
      },
      "execution_count": 249,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Sample tokenization\n",
        "print(\"Sentence:\" , X_train[0])\n",
        "tokenizer(X_train[0], truncation=True, padding=True, max_length=256, return_tensors=\"tf\")"
      ],
      "metadata": {
        "cell_id": "75423f331a4c454e82dbdcd0a0cabe80",
        "tags": [],
        "deepnote_to_be_reexecuted": true,
        "source_hash": "8aaf950d",
        "execution_start": 1654259706685,
        "execution_millis": 15,
        "deepnote_cell_type": "code",
        "deepnote_cell_height": 204.375,
        "deepnote_output_heights": [
          null,
          21.1875
        ],
        "execution": {
          "iopub.status.busy": "2022-06-03T16:28:22.734156Z",
          "iopub.execute_input": "2022-06-03T16:28:22.734833Z",
          "iopub.status.idle": "2022-06-03T16:28:22.744489Z",
          "shell.execute_reply.started": "2022-06-03T16:28:22.734796Z",
          "shell.execute_reply": "2022-06-03T16:28:22.743348Z"
        },
        "trusted": true,
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "91uqp_iLaGSz",
        "outputId": "e03a75e2-79d6-4eb0-ee6b-a34f3709bbdc"
      },
      "execution_count": 250,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Sentence: unit sounds okay when it works but, unfortunately, it mutes itself every time and then cuts in and out whether on bluetooth or aux cable. this is regardless of which computer i connect it to, how close the speaker is to the unit, etc.  noticing that other reviewers have the same problem.  maybe some units are defective.  if it didn't have that problem, i would have probably given it 5 stars.  haven't begun the return process (will do later today), so can't comment on the customer service quality yet.\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'input_ids': <tf.Tensor: shape=(1, 113), dtype=int32, numpy=\n",
              "array([[  101,  3131,  4165,  3100,  2043,  2009,  2573,  2021,  1010,\n",
              "         6854,  1010,  2009, 20101,  2015,  2993,  2296,  2051,  1998,\n",
              "         2059,  7659,  1999,  1998,  2041,  3251,  2006,  2630, 19392,\n",
              "         2030, 19554,  5830,  1012,  2023,  2003,  7539,  1997,  2029,\n",
              "         3274,  1045,  7532,  2009,  2000,  1010,  2129,  2485,  1996,\n",
              "         5882,  2003,  2000,  1996,  3131,  1010,  4385,  1012, 15103,\n",
              "         2008,  2060, 15814,  2031,  1996,  2168,  3291,  1012,  2672,\n",
              "         2070,  3197,  2024, 28829,  1012,  2065,  2009,  2134,  1005,\n",
              "         1056,  2031,  2008,  3291,  1010,  1045,  2052,  2031,  2763,\n",
              "         2445,  2009,  1019,  3340,  1012,  4033,  1005,  1056,  5625,\n",
              "         1996,  2709,  2832,  1006,  2097,  2079,  2101,  2651,  1007,\n",
              "         1010,  2061,  2064,  1005,  1056,  7615,  2006,  1996,  8013,\n",
              "         2326,  3737,  2664,  1012,   102]], dtype=int32)>, 'attention_mask': <tf.Tensor: shape=(1, 113), dtype=int32, numpy=\n",
              "array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
              "        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
              "        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
              "        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
              "        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
              "        1, 1, 1]], dtype=int32)>}"
            ]
          },
          "metadata": {},
          "execution_count": 250
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Perform tokenization\n",
        "MAX_LENGTH=256\n",
        "train_encodings = tokenizer(X_train,\n",
        "                            truncation=True, \n",
        "                            padding=True,\n",
        "                            return_tensors=\"tf\",\n",
        "                            max_length=MAX_LENGTH)\n",
        "test_encodings = tokenizer(X_test,\n",
        "                           truncation=True, \n",
        "                           padding=True,\n",
        "                           return_tensors=\"tf\",\n",
        "                           max_length=MAX_LENGTH)"
      ],
      "metadata": {
        "cell_id": "7ac6083695284846af8046a602fde8a3",
        "tags": [],
        "deepnote_to_be_reexecuted": true,
        "source_hash": "e4214f64",
        "execution_start": 1654259706706,
        "execution_millis": 17902,
        "deepnote_cell_type": "code",
        "deepnote_cell_height": 189,
        "execution": {
          "iopub.status.busy": "2022-06-03T16:28:34.382817Z",
          "iopub.execute_input": "2022-06-03T16:28:34.383197Z",
          "iopub.status.idle": "2022-06-03T16:28:52.757481Z",
          "shell.execute_reply.started": "2022-06-03T16:28:34.383166Z",
          "shell.execute_reply": "2022-06-03T16:28:52.756567Z"
        },
        "trusted": true,
        "id": "hMdu1ivBaGSz"
      },
      "execution_count": 252,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Convert the tokenizer to TF object\n",
        "train_dataset = tf.data.Dataset.from_tensor_slices((dict(train_encodings),\n",
        "                                                    y_train))\n",
        "test_dataset = tf.data.Dataset.from_tensor_slices((dict(test_encodings),\n",
        "                                                   y_test))"
      ],
      "metadata": {
        "cell_id": "da58f1be78eb4392989e82963c129190",
        "tags": [],
        "deepnote_to_be_reexecuted": true,
        "source_hash": "9cda9294",
        "execution_start": 1654259724621,
        "execution_millis": 33,
        "deepnote_cell_type": "code",
        "deepnote_cell_height": 442.5625,
        "deepnote_output_heights": [
          40.375
        ],
        "execution": {
          "iopub.status.busy": "2022-06-03T16:29:05.729632Z",
          "iopub.execute_input": "2022-06-03T16:29:05.730007Z"
        },
        "trusted": true,
        "id": "5AEqYmp_aGS0"
      },
      "execution_count": 253,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Stage 4: Build, train and evaluate the model\n",
        "\n",
        "For the purpose of demonstration, transfer learning is used for building the model for the sentiment analysis task at hand. We'll be using a pretrained distilbert model that's finetuned on SST-2 (Stanford Sentiment Treebank v2) dataset. More details on the model can be found here: https://huggingface.co/distilbert-base-uncased-finetuned-sst-2-english\n",
        "\n",
        "The following key parameters need to be thought through before we build any model,\n",
        "\n",
        "- **Metric:** How do we measure the performance of the model? What should be the right metric? - Accuracy can be used here.\n",
        "\n",
        "- **Loss function:** It is the objective function that is learned during training. And a good model should always try to minimize the loss function. For classification problems, cross-entropy loss can be used as the objective or loss function.\n",
        "\n",
        "- **Optimizer:** A function that decides how the network weights will be updated based on the output of the loss function."
      ],
      "metadata": {
        "cell_id": "2dc8f9b0b65c455fb55d67e3d05b1972",
        "tags": [],
        "deepnote_cell_type": "markdown",
        "deepnote_cell_height": 394.6875,
        "id": "NE8pCep9aGS0"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Define the model - use pretrained distilbert model and specify the label\n",
        "from transformers import TFDistilBertForSequenceClassification\n",
        "model = TFDistilBertForSequenceClassification.from_pretrained(\n",
        "    pretrained_model_name_or_path=MODEL_NAME,\n",
        "    num_labels=2\n",
        "    )"
      ],
      "metadata": {
        "cell_id": "65fbc89f372d46158b6191817b7e3f26",
        "tags": [],
        "deepnote_to_be_reexecuted": true,
        "source_hash": "7b7524c1",
        "execution_start": 1654259014228,
        "execution_millis": 115,
        "deepnote_cell_type": "code",
        "deepnote_cell_height": 253.375,
        "deepnote_output_heights": [
          21.1875
        ],
        "execution": {
          "iopub.status.busy": "2022-06-03T14:50:28.065450Z",
          "iopub.execute_input": "2022-06-03T14:50:28.066119Z",
          "iopub.status.idle": "2022-06-03T14:51:02.137919Z",
          "shell.execute_reply.started": "2022-06-03T14:50:28.066082Z",
          "shell.execute_reply": "2022-06-03T14:51:02.137129Z"
        },
        "trusted": true,
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_UXhAsaDaGS0",
        "outputId": "e85f8380-a8dc-4e64-aa51-b1bdcdde0aa8"
      },
      "execution_count": 254,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Some layers from the model checkpoint at distilbert-base-uncased-finetuned-sst-2-english were not used when initializing TFDistilBertForSequenceClassification: ['dropout_19']\n",
            "- This IS expected if you are initializing TFDistilBertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
            "- This IS NOT expected if you are initializing TFDistilBertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n",
            "Some layers of TFDistilBertForSequenceClassification were not initialized from the model checkpoint at distilbert-base-uncased-finetuned-sst-2-english and are newly initialized: ['dropout_219']\n",
            "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Specify the model optimizer\n",
        "optimizer = tf.keras.optimizers.Adam(\n",
        "            learning_rate=5e-5, \n",
        "        )"
      ],
      "metadata": {
        "cell_id": "7e05e95a3608438fb9bcf50daf4a2d47",
        "tags": [],
        "deepnote_to_be_reexecuted": false,
        "source_hash": "8e3a4d6b",
        "execution_start": 1654260394670,
        "execution_millis": 3,
        "deepnote_cell_type": "code",
        "deepnote_cell_height": 216.1875,
        "execution": {
          "iopub.status.busy": "2022-06-03T16:16:11.195586Z",
          "iopub.execute_input": "2022-06-03T16:16:11.195951Z",
          "iopub.status.idle": "2022-06-03T16:16:11.201815Z",
          "shell.execute_reply.started": "2022-06-03T16:16:11.195915Z",
          "shell.execute_reply": "2022-06-03T16:16:11.200727Z"
        },
        "trusted": true,
        "id": "U3S3aqvjaGS1"
      },
      "execution_count": 255,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Specify the loss function\n",
        "loss_func = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2022-06-03T16:16:11.203340Z",
          "iopub.execute_input": "2022-06-03T16:16:11.203843Z",
          "iopub.status.idle": "2022-06-03T16:16:11.211510Z",
          "shell.execute_reply.started": "2022-06-03T16:16:11.203806Z",
          "shell.execute_reply": "2022-06-03T16:16:11.210722Z"
        },
        "trusted": true,
        "id": "OlO4RmHyaGS1"
      },
      "execution_count": 266,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Compile the model\n",
        "\n",
        "model.compile(optimizer=optimizer, loss=loss_func, metrics=['accuracy'])"
      ],
      "metadata": {
        "cell_id": "6c5e15de4c8e417d844e6c203e7be437",
        "tags": [],
        "owner_user_id": "17c7069a-3ae4-462e-9968-cf72643aef7c",
        "deepnote_cell_type": "code",
        "deepnote_cell_height": 138,
        "execution": {
          "iopub.status.busy": "2022-06-03T16:16:11.213719Z",
          "iopub.execute_input": "2022-06-03T16:16:11.214124Z",
          "iopub.status.idle": "2022-06-03T16:16:11.225862Z",
          "shell.execute_reply.started": "2022-06-03T16:16:11.214083Z",
          "shell.execute_reply": "2022-06-03T16:16:11.225020Z"
        },
        "trusted": true,
        "id": "3rzRu5HuaGS1"
      },
      "execution_count": 267,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Inspect the model\n",
        "model.summary()"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2022-06-03T16:16:11.227451Z",
          "iopub.execute_input": "2022-06-03T16:16:11.227864Z",
          "iopub.status.idle": "2022-06-03T16:16:11.242415Z",
          "shell.execute_reply.started": "2022-06-03T16:16:11.227802Z",
          "shell.execute_reply": "2022-06-03T16:16:11.241397Z"
        },
        "trusted": true,
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PLOxkvJ4aGS1",
        "outputId": "618333e8-13fa-46cc-841a-9b8fbcd76cd4"
      },
      "execution_count": 268,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"tf_distil_bert_for_sequence_classification_10\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " distilbert (TFDistilBertMai  multiple                 66362880  \n",
            " nLayer)                                                         \n",
            "                                                                 \n",
            " pre_classifier (Dense)      multiple                  590592    \n",
            "                                                                 \n",
            " classifier (Dense)          multiple                  1538      \n",
            "                                                                 \n",
            " dropout_219 (Dropout)       multiple                  0         \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 66,955,010\n",
            "Trainable params: 66,955,010\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Define hyperparameters\n",
        "BATCH_SIZE = 16\n",
        "NUM_EPOCHS = 2\n",
        "\n",
        "# Train the model\n",
        "model.fit(train_dataset.shuffle(len(X_train)).batch(BATCH_SIZE),\n",
        "          epochs=NUM_EPOCHS,\n",
        "          batch_size=BATCH_SIZE,\n",
        "          validation_data=test_dataset.shuffle(len(X_test)).batch(BATCH_SIZE)\n",
        "          )"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2022-06-03T16:16:11.243877Z",
          "iopub.execute_input": "2022-06-03T16:16:11.244237Z",
          "iopub.status.idle": "2022-06-03T16:16:13.107784Z",
          "shell.execute_reply.started": "2022-06-03T16:16:11.244202Z",
          "shell.execute_reply": "2022-06-03T16:16:13.105629Z"
        },
        "trusted": true,
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "L16zkwEIaGS2",
        "outputId": "860b34d1-2cf1-4739-b663-02386e215b48"
      },
      "execution_count": 269,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "743/743 [==============================] - 338s 444ms/step - loss: 0.2718 - accuracy: 0.8890 - val_loss: 0.2726 - val_accuracy: 0.8855\n",
            "Epoch 2/2\n",
            "743/743 [==============================] - 330s 445ms/step - loss: 0.1659 - accuracy: 0.9385 - val_loss: 0.2430 - val_accuracy: 0.9077\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7fd95c251210>"
            ]
          },
          "metadata": {},
          "execution_count": 269
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Save the model\n",
        "model.save_pretrained(\"/tmp/sentiment_model\")"
      ],
      "metadata": {
        "id": "D1RDYTAEEykP"
      },
      "execution_count": 271,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Preprocess the unseen data\n",
        "unseen_data = data.sample(frac=0.05, random_state=42)\n",
        "unseen_reviews = unseen_data['review'].tolist()\n",
        "\n",
        "unseen_encodings = tokenizer(unseen_reviews,\n",
        "                            padding=True,\n",
        "                            truncation=True,\n",
        "                            max_length=MAX_LENGTH,\n",
        "                            return_tensors=\"tf\")\n",
        "\n",
        "y_unseen = unseen_data['sentiment'].tolist()\n",
        "\n",
        "unseen_encodings = tf.data.Dataset.from_tensor_slices((dict(unseen_encodings),\n",
        "                                                       y_unseen\n",
        "                                                      ))\n"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2022-06-03T14:49:05.560124Z",
          "iopub.status.idle": "2022-06-03T14:49:05.560747Z",
          "shell.execute_reply.started": "2022-06-03T14:49:05.560505Z",
          "shell.execute_reply": "2022-06-03T14:49:05.560529Z"
        },
        "trusted": true,
        "id": "-F_xrMSAaGS2"
      },
      "execution_count": 283,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Evaluate the model\n",
        "model.evaluate(unseen_encodings.shuffle(len(unseen_reviews)).batch(BATCH_SIZE),\n",
        "               return_dict=True,\n",
        "               batch_size=BATCH_SIZE)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "feWeBevSIy7-",
        "outputId": "e0cd2ea3-e53c-42a5-f580-0bc1ad5efad6"
      },
      "execution_count": 285,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "99/99 [==============================] - 14s 139ms/step - loss: 0.1880 - accuracy: 0.9371\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'accuracy': 0.9371428489685059, 'loss': 0.18796877562999725}"
            ]
          },
          "metadata": {},
          "execution_count": 285
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Stage 5: Refine the model\n",
        "\n",
        "Once the model is built and trained, the result may not be satisfactory at times (we may not get expected performance from the model). The model may overfit or underfit depending on several factors. In order to improve the model, we can perfrom hyperparameter tuning. Some of the hyperparameters are the following,\n",
        "\n",
        "- Number of layers in the model (hidden layers) - too many layers can result in overfitting and too few layers will result in underfitting.\n",
        "- Number of units per layer - input layer should match the input size, and number of units in the hidden layer can be decided based on how do you wnat to respesent the data.\n",
        "- Dropout rate - to penalize the model from overfitting.\n",
        "- Learning rate - how fast the network should converge to the optimal values of the weights.\n",
        "\n",
        "You can try different configurations for the hyperparameters and see what works best for your model."
      ],
      "metadata": {
        "id": "hFfoP4uTj3wS"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Stage 6: Deploy the model\n",
        "\n",
        "There are several options to deploy a model. You can also optimize the trianed model using quantization methods in order to deploy it in a resource constrained environment. Some of the available options are,\n",
        "\n",
        "1. [TensorFlow serving](https://www.tensorflow.org/tfx/guide/serving)\n",
        "2. [Pytorch Lightning](https://pytorch-lightning.readthedocs.io/en/stable/common/production_inference.html)\n",
        "3. Convert to [ONNX](https://onnx.ai/) format and choose the framework of your choice\n",
        "4. [Dockerize](https://www.docker.com/) your model and deploy using any REST API framework like [FastAPI](https://fastapi.tiangolo.com/) or [Flask with Gunicorn](https://dev.to/brandonwallace/deploy-flask-the-easy-way-with-gunicorn-and-nginx-jgc)\n",
        "\n",
        "Your job doesn't end there!! \n",
        "- You'll have to monitor your model for any performace degradation and drifts (model or data) while it is in production. Do the check regularly.\n",
        "- Collect more data while the model is in production through feedback.\n",
        "- If you find that the model is degrading, then you will have to retrain your model using the data collected from feedback.\n",
        "\n",
        "\n",
        "For the purpose of this demostration, let's look at how to use notebook as an interface to get predictions using Gradio.\n",
        "\n"
      ],
      "metadata": {
        "id": "_Wk1TcNLo1i-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import gradio as gr\n",
        "\n",
        "def predict(sentence):\n",
        "    # Load model\n",
        "    loaded_model = TFDistilBertForSequenceClassification.from_pretrained(\"/tmp/sentiment_model\")\n",
        "\n",
        "    # Get the prediction\n",
        "    predict_input = tokenizer.encode(sentence,\n",
        "                                 truncation=True,\n",
        "                                 padding=True,\n",
        "                                 return_tensors=\"tf\")\n",
        "\n",
        "    tf_output = loaded_model.predict(predict_input)[0]\n",
        "    tf_prediction = tf.nn.softmax(tf_output, axis=1).numpy()[0]\n",
        "    return ['negative', 'positive'][np.argmax(tf_prediction)]\n",
        "\n",
        "demo = gr.Interface(fn=predict, inputs=\"text\", outputs=\"text\",live=True)\n",
        "\n",
        "demo.launch()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 643
        },
        "id": "xbUSQ7mXLQ69",
        "outputId": "c676896b-5235-49da-aefc-a103d955537f"
      },
      "execution_count": 292,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Colab notebook detected. To show errors in colab notebook, set `debug=True` in `launch()`\n",
            "Running on public URL: https://17805.gradio.app\n",
            "\n",
            "This share link expires in 72 hours. For free permanent hosting, check out Spaces (https://huggingface.co/spaces)\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "<div><iframe src=\"https://17805.gradio.app\" width=\"900\" height=\"500\" allow=\"autoplay; camera; microphone;\" frameborder=\"0\" allowfullscreen></iframe></div>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<gradio.routes.App at 0x7fdd9c553990>,\n",
              " 'http://127.0.0.1:7866/',\n",
              " 'https://17805.gradio.app')"
            ]
          },
          "metadata": {},
          "execution_count": 292
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a style='text-decoration:none;line-height:16px;display:flex;color:#5B5B62;padding:10px;justify-content:end;' href='https://deepnote.com?utm_source=created-in-deepnote-cell&projectId=f2a99b51-b274-4e57-b790-07263784107d' target=\"_blank\">\n",
        "<img alt='Created in deepnote.com' style='display:inline;max-height:16px;margin:0px;margin-right:7.5px;' src='data:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iVVRGLTgiPz4KPHN2ZyB3aWR0aD0iODBweCIgaGVpZ2h0PSI4MHB4IiB2aWV3Qm94PSIwIDAgODAgODAiIHZlcnNpb249IjEuMSIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIiB4bWxuczp4bGluaz0iaHR0cDovL3d3dy53My5vcmcvMTk5OS94bGluayI+CiAgICA8IS0tIEdlbmVyYXRvcjogU2tldGNoIDU0LjEgKDc2NDkwKSAtIGh0dHBzOi8vc2tldGNoYXBwLmNvbSAtLT4KICAgIDx0aXRsZT5Hcm91cCAzPC90aXRsZT4KICAgIDxkZXNjPkNyZWF0ZWQgd2l0aCBTa2V0Y2guPC9kZXNjPgogICAgPGcgaWQ9IkxhbmRpbmciIHN0cm9rZT0ibm9uZSIgc3Ryb2tlLXdpZHRoPSIxIiBmaWxsPSJub25lIiBmaWxsLXJ1bGU9ImV2ZW5vZGQiPgogICAgICAgIDxnIGlkPSJBcnRib2FyZCIgdHJhbnNmb3JtPSJ0cmFuc2xhdGUoLTEyMzUuMDAwMDAwLCAtNzkuMDAwMDAwKSI+CiAgICAgICAgICAgIDxnIGlkPSJHcm91cC0zIiB0cmFuc2Zvcm09InRyYW5zbGF0ZSgxMjM1LjAwMDAwMCwgNzkuMDAwMDAwKSI+CiAgICAgICAgICAgICAgICA8cG9seWdvbiBpZD0iUGF0aC0yMCIgZmlsbD0iIzAyNjVCNCIgcG9pbnRzPSIyLjM3NjIzNzYyIDgwIDM4LjA0NzY2NjcgODAgNTcuODIxNzgyMiA3My44MDU3NTkyIDU3LjgyMTc4MjIgMzIuNzU5MjczOSAzOS4xNDAyMjc4IDMxLjY4MzE2ODMiPjwvcG9seWdvbj4KICAgICAgICAgICAgICAgIDxwYXRoIGQ9Ik0zNS4wMDc3MTgsODAgQzQyLjkwNjIwMDcsNzYuNDU0OTM1OCA0Ny41NjQ5MTY3LDcxLjU0MjI2NzEgNDguOTgzODY2LDY1LjI2MTk5MzkgQzUxLjExMjI4OTksNTUuODQxNTg0MiA0MS42NzcxNzk1LDQ5LjIxMjIyODQgMjUuNjIzOTg0Niw0OS4yMTIyMjg0IEMyNS40ODQ5Mjg5LDQ5LjEyNjg0NDggMjkuODI2MTI5Niw0My4yODM4MjQ4IDM4LjY0NzU4NjksMzEuNjgzMTY4MyBMNzIuODcxMjg3MSwzMi41NTQ0MjUgTDY1LjI4MDk3Myw2Ny42NzYzNDIxIEw1MS4xMTIyODk5LDc3LjM3NjE0NCBMMzUuMDA3NzE4LDgwIFoiIGlkPSJQYXRoLTIyIiBmaWxsPSIjMDAyODY4Ij48L3BhdGg+CiAgICAgICAgICAgICAgICA8cGF0aCBkPSJNMCwzNy43MzA0NDA1IEwyNy4xMTQ1MzcsMC4yNTcxMTE0MzYgQzYyLjM3MTUxMjMsLTEuOTkwNzE3MDEgODAsMTAuNTAwMzkyNyA4MCwzNy43MzA0NDA1IEM4MCw2NC45NjA0ODgyIDY0Ljc3NjUwMzgsNzkuMDUwMzQxNCAzNC4zMjk1MTEzLDgwIEM0Ny4wNTUzNDg5LDc3LjU2NzA4MDggNTMuNDE4MjY3Nyw3MC4zMTM2MTAzIDUzLjQxODI2NzcsNTguMjM5NTg4NSBDNTMuNDE4MjY3Nyw0MC4xMjg1NTU3IDM2LjMwMzk1NDQsMzcuNzMwNDQwNSAyNS4yMjc0MTcsMzcuNzMwNDQwNSBDMTcuODQzMDU4NiwzNy43MzA0NDA1IDkuNDMzOTE5NjYsMzcuNzMwNDQwNSAwLDM3LjczMDQ0MDUgWiIgaWQ9IlBhdGgtMTkiIGZpbGw9IiMzNzkzRUYiPjwvcGF0aD4KICAgICAgICAgICAgPC9nPgogICAgICAgIDwvZz4KICAgIDwvZz4KPC9zdmc+' > </img>\n",
        "Created in <span style='font-weight:600;margin-left:4px;'>Deepnote</span></a>"
      ],
      "metadata": {
        "tags": [],
        "created_in_deepnote_cell": true,
        "deepnote_cell_type": "markdown",
        "id": "6M0PcuRaaGS3"
      }
    }
  ]
}