introduction_to_applying_machine_learning/identify_key_insights_from_textual_document/document_text_classification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "267e4da2",
   "metadata": {},
   "source": [
    "# Document Understanding Solution - Text Classification\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aae2c5d7",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook. \n",
    "\n",
    "![This us-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/us-west-2/introduction_to_applying_machine_learning|identify_key_insights_from_textual_document|document_text_classification.ipynb)\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d7cbc705",
   "metadata": {},
   "source": [
    "\n",
    "Text Classification refers to classifying an input sentence to one of the class labels of the training dataset. In this notebook, we demonstrate how to use the [JumpStart API](https://sagemaker.readthedocs.io/en/stable/overview.html#use-prebuilt-models-with-sagemaker-jumpstart) for Text Classification. In particular, we demonstrate three use cases of Text Classification:\n",
    "\n",
    "1. How to directly deploy a pretrained Transformer-based text classification model to perform Sentiment Analysis.\n",
    "2. How to fine-tune a pre-trained Transformer model on a custom dataset, and then run inference on the fine-tuned model.\n",
    "3. How to run [SageMaker Automatic Model Tuning](https://docs.aws.amazon.com/sagemaker/latest/dg/automatic-model-tuning.html) (a hyperparameter optimization procedure) to find the best model compared with the model fine-tuned in point 2. The performance of the optimal model and model fine-tuned in point 2 is evaluated on a hold-out test data. \n",
    "\n",
    "**Note**: When running this notebook on SageMaker Studio, you should make\n",
    "sure the `PyTorch 1.10 Python 3.8 CPU Optimized` image/kernel is used. When\n",
    "running this notebook on SageMaker Notebook Instance, you should make\n",
    "sure the 'sagemaker-soln' kernel is used."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2e7d330c",
   "metadata": {},
   "source": [
    "## 1. Set Up\n",
    "\n",
    "Before executing the notebook, there are some initial steps required for setup. This notebook requires latest version of sagemaker and ipywidgets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ce24216a",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: sagemaker in /opt/conda/lib/python3.9/site-packages (2.157.0)\n",
      "Collecting ipywidgets\n",
      "  Using cached ipywidgets-8.0.6-py3-none-any.whl (138 kB)\n",
      "Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.9/site-packages (from sagemaker) (23.0)\n",
      "Requirement already satisfied: importlib-metadata<5.0,>=1.4.0 in /opt/conda/lib/python3.9/site-packages (from sagemaker) (4.13.0)\n",
      "Requirement already satisfied: pandas in /opt/conda/lib/python3.9/site-packages (from sagemaker) (1.5.3)\n",
      "Requirement already satisfied: cloudpickle==2.2.1 in /opt/conda/lib/python3.9/site-packages (from sagemaker) (2.2.1)\n",
      "Requirement already satisfied: schema in /opt/conda/lib/python3.9/site-packages (from sagemaker) (0.7.5)\n",
      "Requirement already satisfied: tblib==1.7.0 in /opt/conda/lib/python3.9/site-packages (from sagemaker) (1.7.0)\n",
      "Requirement already satisfied: boto3<2.0,>=1.26.131 in /opt/conda/lib/python3.9/site-packages (from sagemaker) (1.26.135)\n",
      "Requirement already satisfied: numpy<2.0,>=1.9.0 in /opt/conda/lib/python3.9/site-packages (from sagemaker) (1.23.5)\n",
      "Requirement already satisfied: protobuf<4.0,>=3.1 in /opt/conda/lib/python3.9/site-packages (from sagemaker) (3.20.2)\n",
      "Requirement already satisfied: platformdirs in /opt/conda/lib/python3.9/site-packages (from sagemaker) (3.5.1)\n",
      "Requirement already satisfied: PyYAML==6.0 in /opt/conda/lib/python3.9/site-packages (from sagemaker) (6.0)\n",
      "Requirement already satisfied: jsonschema in /opt/conda/lib/python3.9/site-packages (from sagemaker) (4.17.3)\n",
      "Requirement already satisfied: smdebug-rulesconfig==1.0.1 in /opt/conda/lib/python3.9/site-packages (from sagemaker) (1.0.1)\n",
      "Requirement already satisfied: protobuf3-to-dict<1.0,>=0.1.5 in /opt/conda/lib/python3.9/site-packages (from sagemaker) (0.1.5)\n",
      "Requirement already satisfied: attrs<24,>=23.1.0 in /opt/conda/lib/python3.9/site-packages (from sagemaker) (23.1.0)\n",
      "Requirement already satisfied: google-pasta in /opt/conda/lib/python3.9/site-packages (from sagemaker) (0.2.0)\n",
      "Requirement already satisfied: pathos in /opt/conda/lib/python3.9/site-packages (from sagemaker) (0.3.0)\n",
      "Requirement already satisfied: ipykernel>=4.5.1 in /opt/conda/lib/python3.9/site-packages (from ipywidgets) (5.5.6)\n",
      "Requirement already satisfied: ipython>=6.1.0 in /opt/conda/lib/python3.9/site-packages (from ipywidgets) (8.10.0)\n",
      "Collecting jupyterlab-widgets~=3.0.7\n",
      "  Using cached jupyterlab_widgets-3.0.7-py3-none-any.whl (198 kB)\n",
      "Collecting widgetsnbextension~=4.0.7\n",
      "  Using cached widgetsnbextension-4.0.7-py3-none-any.whl (2.1 MB)\n",
      "Requirement already satisfied: traitlets>=4.3.1 in /opt/conda/lib/python3.9/site-packages (from ipywidgets) (5.9.0)\n",
      "Requirement already satisfied: s3transfer<0.7.0,>=0.6.0 in /opt/conda/lib/python3.9/site-packages (from boto3<2.0,>=1.26.131->sagemaker) (0.6.0)\n",
      "Requirement already satisfied: botocore<1.30.0,>=1.29.135 in /opt/conda/lib/python3.9/site-packages (from boto3<2.0,>=1.26.131->sagemaker) (1.29.135)\n",
      "Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in /opt/conda/lib/python3.9/site-packages (from boto3<2.0,>=1.26.131->sagemaker) (1.0.1)\n",
      "Requirement already satisfied: zipp>=0.5 in /opt/conda/lib/python3.9/site-packages (from importlib-metadata<5.0,>=1.4.0->sagemaker) (3.13.0)\n",
      "Requirement already satisfied: jupyter-client in /opt/conda/lib/python3.9/site-packages (from ipykernel>=4.5.1->ipywidgets) (6.1.5)\n",
      "Requirement already satisfied: ipython-genutils in /opt/conda/lib/python3.9/site-packages (from ipykernel>=4.5.1->ipywidgets) (0.2.0)\n",
      "Requirement already satisfied: tornado>=4.2 in /opt/conda/lib/python3.9/site-packages (from ipykernel>=4.5.1->ipywidgets) (6.2)\n",
      "Requirement already satisfied: pickleshare in /opt/conda/lib/python3.9/site-packages (from ipython>=6.1.0->ipywidgets) (0.7.5)\n",
      "Requirement already satisfied: pexpect>4.3 in /opt/conda/lib/python3.9/site-packages (from ipython>=6.1.0->ipywidgets) (4.8.0)\n",
      "Requirement already satisfied: backcall in /opt/conda/lib/python3.9/site-packages (from ipython>=6.1.0->ipywidgets) (0.2.0)\n",
      "Requirement already satisfied: matplotlib-inline in /opt/conda/lib/python3.9/site-packages (from ipython>=6.1.0->ipywidgets) (0.1.6)\n",
      "Requirement already satisfied: decorator in /opt/conda/lib/python3.9/site-packages (from ipython>=6.1.0->ipywidgets) (5.1.1)\n",
      "Requirement already satisfied: prompt-toolkit<3.1.0,>=3.0.30 in /opt/conda/lib/python3.9/site-packages (from ipython>=6.1.0->ipywidgets) (3.0.36)\n",
      "Requirement already satisfied: stack-data in /opt/conda/lib/python3.9/site-packages (from ipython>=6.1.0->ipywidgets) (0.6.2)\n",
      "Requirement already satisfied: jedi>=0.16 in /opt/conda/lib/python3.9/site-packages (from ipython>=6.1.0->ipywidgets) (0.18.2)\n",
      "Requirement already satisfied: pygments>=2.4.0 in /opt/conda/lib/python3.9/site-packages (from ipython>=6.1.0->ipywidgets) (2.14.0)\n",
      "Requirement already satisfied: six in /opt/conda/lib/python3.9/site-packages (from protobuf3-to-dict<1.0,>=0.1.5->sagemaker) (1.16.0)\n",
      "Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /opt/conda/lib/python3.9/site-packages (from jsonschema->sagemaker) (0.19.3)\n",
      "Requirement already satisfied: python-dateutil>=2.8.1 in /opt/conda/lib/python3.9/site-packages (from pandas->sagemaker) (2.8.2)\n",
      "Requirement already satisfied: pytz>=2020.1 in /opt/conda/lib/python3.9/site-packages (from pandas->sagemaker) (2022.7.1)\n",
      "Requirement already satisfied: ppft>=1.7.6.6 in /opt/conda/lib/python3.9/site-packages (from pathos->sagemaker) (1.7.6.6)\n",
      "Requirement already satisfied: dill>=0.3.6 in /opt/conda/lib/python3.9/site-packages (from pathos->sagemaker) (0.3.6)\n",
      "Requirement already satisfied: pox>=0.3.2 in /opt/conda/lib/python3.9/site-packages (from pathos->sagemaker) (0.3.2)\n",
      "Requirement already satisfied: multiprocess>=0.70.14 in /opt/conda/lib/python3.9/site-packages (from pathos->sagemaker) (0.70.14)\n",
      "Requirement already satisfied: contextlib2>=0.5.5 in /opt/conda/lib/python3.9/site-packages (from schema->sagemaker) (21.6.0)\n",
      "Requirement already satisfied: urllib3<1.27,>=1.25.4 in /opt/conda/lib/python3.9/site-packages (from botocore<1.30.0,>=1.29.135->boto3<2.0,>=1.26.131->sagemaker) (1.26.14)\n",
      "Requirement already satisfied: parso<0.9.0,>=0.8.0 in /opt/conda/lib/python3.9/site-packages (from jedi>=0.16->ipython>=6.1.0->ipywidgets) (0.8.3)\n",
      "Requirement already satisfied: ptyprocess>=0.5 in /opt/conda/lib/python3.9/site-packages (from pexpect>4.3->ipython>=6.1.0->ipywidgets) (0.7.0)\n",
      "Requirement already satisfied: wcwidth in /opt/conda/lib/python3.9/site-packages (from prompt-toolkit<3.1.0,>=3.0.30->ipython>=6.1.0->ipywidgets) (0.2.6)\n",
      "Requirement already satisfied: pyzmq>=13 in /opt/conda/lib/python3.9/site-packages (from jupyter-client->ipykernel>=4.5.1->ipywidgets) (24.0.1)\n",
      "Requirement already satisfied: jupyter-core>=4.6.0 in /opt/conda/lib/python3.9/site-packages (from jupyter-client->ipykernel>=4.5.1->ipywidgets) (4.9.2)\n",
      "Requirement already satisfied: executing>=1.2.0 in /opt/conda/lib/python3.9/site-packages (from stack-data->ipython>=6.1.0->ipywidgets) (1.2.0)\n",
      "Requirement already satisfied: asttokens>=2.1.0 in /opt/conda/lib/python3.9/site-packages (from stack-data->ipython>=6.1.0->ipywidgets) (2.2.1)\n",
      "Requirement already satisfied: pure-eval in /opt/conda/lib/python3.9/site-packages (from stack-data->ipython>=6.1.0->ipywidgets) (0.2.2)\n",
      "Installing collected packages: widgetsnbextension, jupyterlab-widgets, ipywidgets\n",
      "Successfully installed ipywidgets-8.0.6 jupyterlab-widgets-3.0.7 widgetsnbextension-4.0.7\n",
      "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n",
      "\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.0\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.1.2\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!pip install -U sagemaker ipywidgets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "48cf79cf",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import sagemaker, boto3, json\n",
    "import sys\n",
    "import config\n",
    "\n",
    "\n",
    "aws_region = boto3.Session().region_name\n",
    "sess = sagemaker.Session()\n",
    "aws_role = sagemaker.get_execution_role()\n",
    "DEFAULT_BUCKET = sess.default_bucket()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "affd80bd",
   "metadata": {},
   "source": [
    "## 2. Select a pre-trained text classification model\n",
    "\n",
    "You can continue with the default model, or can choose a different model from the dropdown generated upon running the next cell. A complete list of JumpStart models can also be accessed at JumpStart Models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "00ef0e02",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "model_id = \"tensorflow-tc-bert-en-uncased-L-12-H-768-A-12-2\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e816902",
   "metadata": {},
   "source": [
    "\n",
    "You can also select a different JumpStart model. Here, we download jumpstart model_manifest file from the jumpstart s3 bucket, filter-out all the Text Classification models and select a model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "acfbb402",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "All the other available text classification models are as below.\n",
      "\n",
      "huggingface-tc-bert-base-cased\n",
      "huggingface-tc-bert-base-multilingual-cased\n",
      "huggingface-tc-bert-base-multilingual-uncased\n",
      "huggingface-tc-bert-base-uncased\n",
      "huggingface-tc-bert-large-cased\n",
      "huggingface-tc-bert-large-cased-whole-word-masking\n",
      "huggingface-tc-bert-large-uncased\n",
      "huggingface-tc-bert-large-uncased-whole-word-masking\n",
      "huggingface-tc-distilbert-base-cased\n",
      "huggingface-tc-distilbert-base-multilingual-cased\n",
      "huggingface-tc-distilbert-base-uncased\n",
      "huggingface-tc-distilroberta-base\n",
      "huggingface-tc-models\n",
      "huggingface-tc-roberta-base\n",
      "huggingface-tc-roberta-base-openai-detector\n",
      "huggingface-tc-roberta-large\n",
      "huggingface-tc-roberta-large-openai-detector\n",
      "huggingface-tc-xlm-clm-ende-1024\n",
      "huggingface-tc-xlm-mlm-ende-1024\n",
      "huggingface-tc-xlm-mlm-enro-1024\n",
      "huggingface-tc-xlm-mlm-tlm-xnli15-1024\n",
      "tensorflow-tc-albert-en-base\n",
      "tensorflow-tc-bert-en-cased-L-12-H-768-A-12-2\n",
      "tensorflow-tc-bert-en-cased-L-24-H-1024-A-16-2\n",
      "tensorflow-tc-bert-en-uncased-L-12-H-768-A-12-2\n",
      "tensorflow-tc-bert-en-uncased-L-24-H-1024-A-16-2\n",
      "tensorflow-tc-bert-en-wwm-cased-L-24-H-1024-A-16-2\n",
      "tensorflow-tc-bert-en-wwm-uncased-L-24-H-1024-A-16-2\n",
      "tensorflow-tc-bert-multi-cased-L-12-H-768-A-12-2\n",
      "tensorflow-tc-electra-base-1\n",
      "tensorflow-tc-electra-small-1\n",
      "tensorflow-tc-experts-bert-pubmed-1\n",
      "tensorflow-tc-experts-bert-wiki-books-1\n",
      "tensorflow-tc-small-bert-bert-en-uncased-L-10-H-128-A-2\n",
      "tensorflow-tc-small-bert-bert-en-uncased-L-10-H-256-A-4\n",
      "tensorflow-tc-small-bert-bert-en-uncased-L-10-H-512-A-8\n",
      "tensorflow-tc-small-bert-bert-en-uncased-L-10-H-768-A-12\n",
      "tensorflow-tc-small-bert-bert-en-uncased-L-12-H-128-A-2\n",
      "tensorflow-tc-small-bert-bert-en-uncased-L-12-H-256-A-4\n",
      "tensorflow-tc-small-bert-bert-en-uncased-L-12-H-512-A-8\n",
      "tensorflow-tc-small-bert-bert-en-uncased-L-12-H-768-A-12\n",
      "tensorflow-tc-small-bert-bert-en-uncased-L-2-H-128-A-2\n",
      "tensorflow-tc-small-bert-bert-en-uncased-L-2-H-256-A-4\n",
      "tensorflow-tc-small-bert-bert-en-uncased-L-2-H-512-A-8\n",
      "tensorflow-tc-small-bert-bert-en-uncased-L-2-H-768-A-12\n",
      "tensorflow-tc-small-bert-bert-en-uncased-L-4-H-128-A-2\n",
      "tensorflow-tc-small-bert-bert-en-uncased-L-4-H-256-A-4\n",
      "tensorflow-tc-small-bert-bert-en-uncased-L-4-H-512-A-8\n",
      "tensorflow-tc-small-bert-bert-en-uncased-L-4-H-768-A-12\n",
      "tensorflow-tc-small-bert-bert-en-uncased-L-6-H-128-A-2\n",
      "tensorflow-tc-small-bert-bert-en-uncased-L-6-H-256-A-4\n",
      "tensorflow-tc-small-bert-bert-en-uncased-L-6-H-512-A-8\n",
      "tensorflow-tc-small-bert-bert-en-uncased-L-6-H-768-A-12\n",
      "tensorflow-tc-small-bert-bert-en-uncased-L-8-H-128-A-2\n",
      "tensorflow-tc-small-bert-bert-en-uncased-L-8-H-256-A-4\n",
      "tensorflow-tc-small-bert-bert-en-uncased-L-8-H-512-A-8\n",
      "tensorflow-tc-small-bert-bert-en-uncased-L-8-H-768-A-12\n",
      "tensorflow-tc-talking-heads-base\n",
      "tensorflow-tc-talking-heads-large\n"
     ]
    }
   ],
   "source": [
    "# download JumpStart model_manifest file.\n",
    "boto3.client(\"s3\").download_file(\n",
    "    f\"jumpstart-cache-prod-{aws_region}\", \"models_manifest.json\", \"models_manifest.json\"\n",
    ")\n",
    "with open(\"models_manifest.json\", \"rb\") as json_file:\n",
    "    model_list = json.load(json_file)\n",
    "\n",
    "# filter-out all the Text Classification models from the manifest list.\n",
    "tc_models_all_versions, tc_models = [\n",
    "    model[\"model_id\"] for model in model_list if \"-tc-\" in model[\"model_id\"]\n",
    "], []\n",
    "[tc_models.append(model) for model in tc_models_all_versions if model not in tc_models]\n",
    "\n",
    "print(f\"All the other available text classification models are as below.\\n\")\n",
    "for each in tc_models:\n",
    "    print(f\"{each}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76122814",
   "metadata": {},
   "source": [
    "## 3. Run inference on the pre-trained text classification model\n",
    "\n",
    "This is a Text Classification model built upon a Text Embedding model from [TensorFlow Hub](https://tfhub.dev/tensorflow/bert_en_uncased_L-12_H-768_A-12/4). It takes a text string as input and classifies the input text as either a positive or negative movie review.\n",
    "\n",
    "The Text Embedding model which is pre-trained on Wikipedia and BookCorpus datasets returns an embedding of the input text.\n",
    "\n",
    "The model available for deployment is created by attaching a binary classification layer to the output of the Text Embedding model, and then fine-tuning the entire model on SST2 dataset. The [SST2](https://nlp.stanford.edu/sentiment/index.html) dataset comprises positive and negative movie reviews."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7666852e",
   "metadata": {},
   "source": [
    "### 3.1. Retrieve jumpStart artifacts & deploy an endpoint\n",
    "We retrieve the `deploy_image_uri`, `deploy_source_uri`, and `base_model_uri` for the pre-trained model. To host the pre-trained model, we create an instance of sagemaker.model.Model and deploy it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "9050db3c",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "------!"
     ]
    }
   ],
   "source": [
    "import time\n",
    "from sagemaker import image_uris, model_uris, script_uris\n",
    "from sagemaker.model import Model\n",
    "from sagemaker.predictor import Predictor\n",
    "from sagemaker.utils import name_from_base\n",
    "\n",
    "# model_version=\"*\" fetches the latest version of the model.\n",
    "infer_model_id, infer_model_version = model_id, \"1.1.2\"\n",
    "\n",
    "endpoint_name_tc = f\"{config.SOLUTION_PREFIX}-text-classification-endpoint\"\n",
    "\n",
    "inference_instance_type = config.HOSTING_INSTANCE_TYPE\n",
    "\n",
    "# Retrieve the inference docker container uri.\n",
    "deploy_image_uri = image_uris.retrieve(\n",
    "    region=None,\n",
    "    framework=None,\n",
    "    image_scope=\"inference\",\n",
    "    model_id=infer_model_id,\n",
    "    model_version=infer_model_version,\n",
    "    instance_type=inference_instance_type,\n",
    ")\n",
    "# Retrieve the inference script uri.\n",
    "deploy_source_uri = script_uris.retrieve(\n",
    "    model_id=infer_model_id, model_version=infer_model_version, script_scope=\"inference\"\n",
    ")\n",
    "# Retrieve the base model uri.\n",
    "base_model_uri = model_uris.retrieve(\n",
    "    model_id=infer_model_id, model_version=infer_model_version, model_scope=\"inference\"\n",
    ")\n",
    "# Create the SageMaker model instance. Note that we need to pass Predictor class when we deploy model through Model class,\n",
    "# for being able to run inference through the sagemaker API.\n",
    "model = Model(\n",
    "    image_uri=deploy_image_uri,\n",
    "    source_dir=deploy_source_uri,\n",
    "    model_data=base_model_uri,\n",
    "    entry_point=\"inference.py\",\n",
    "    role=aws_role,\n",
    "    predictor_cls=Predictor,\n",
    "    name=endpoint_name_tc,\n",
    ")\n",
    "# deploy the Model.\n",
    "base_model_predictor = model.deploy(\n",
    "    initial_instance_count=1,\n",
    "    instance_type=inference_instance_type,\n",
    "    endpoint_name=endpoint_name_tc,\n",
    ")\n",
    "\n",
    "time.sleep(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d1d830d",
   "metadata": {},
   "source": [
    "### 3.2. Example input sentences for inference\n",
    "\n",
    "These examples are taken from SST2 dataset downloaded from [TensorFlow](https://www.tensorflow.org/datasets/catalog/glue#gluesst2). [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0). [Dataset Homepage](https://nlp.stanford.edu/sentiment/index.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "8965d0b9",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "text1 = \"astonishing ... ( frames ) profound ethical and philosophical questions in the form of dazzling pop entertainment\"\n",
    "text2 = \"simply stupid , irrelevant and deeply , truly , bottomlessly cynical \""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99b6b19d",
   "metadata": {},
   "source": [
    "### 3.3. Query endpoint and parse response\n",
    "Input to the endpoint is a single sentence. Response from the endpoint is a dictionary containing the predicted class label, and a list of class label probabilities."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "31513151",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Inference:\n",
      "Input text: 'astonishing ... ( frames ) profound ethical and philosophical questions in the form of dazzling pop entertainment'\n",
      "Model prediction: [0.00015312265143107244, 0.9998468773485689]\n",
      "Labels: ['negative', 'positive']\n",
      "Predicted Label: \u001b[1mpositive\u001b[0m\n",
      "\n",
      "Inference:\n",
      "Input text: 'simply stupid , irrelevant and deeply , truly , bottomlessly cynical '\n",
      "Model prediction: [0.9997017409141489, 0.00029825908585111375]\n",
      "Labels: ['negative', 'positive']\n",
      "Predicted Label: \u001b[1mnegative\u001b[0m\n",
      "\n"
     ]
    }
   ],
   "source": [
    "newline, bold, unbold = \"\\n\", \"\\033[1m\", \"\\033[0m\"\n",
    "\n",
    "\n",
    "def query_endpoint(encoded_text):\n",
    "    response = base_model_predictor.predict(\n",
    "        encoded_text,\n",
    "        {\"ContentType\": \"application/x-text\", \"Accept\": \"application/json;verbose\"},\n",
    "    )\n",
    "    return response\n",
    "\n",
    "\n",
    "def parse_response(query_response):\n",
    "    model_predictions = json.loads(query_response)\n",
    "    probabilities, labels, predicted_label = (\n",
    "        model_predictions[\"probabilities\"],\n",
    "        model_predictions[\"labels\"],\n",
    "        model_predictions[\"predicted_label\"],\n",
    "    )\n",
    "    return probabilities, labels, predicted_label\n",
    "\n",
    "\n",
    "for text in [text1, text2]:\n",
    "    query_response = query_endpoint(text.encode(\"utf-8\"))\n",
    "    probabilities, labels, predicted_label = parse_response(query_response)\n",
    "    print(\n",
    "        f\"Inference:{newline}\"\n",
    "        f\"Input text: '{text}'{newline}\"\n",
    "        f\"Model prediction: {probabilities}{newline}\"\n",
    "        f\"Labels: {labels}{newline}\"\n",
    "        f\"Predicted Label: {bold}{predicted_label}{unbold}{newline}\"\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "44e50ecc",
   "metadata": {},
   "source": [
    "### 3.4. Clean up the endpoint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "30903b0e",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Delete the SageMaker endpoint and the attached resources\n",
    "base_model_predictor.delete_model()\n",
    "base_model_predictor.delete_endpoint()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb0ed164",
   "metadata": {},
   "source": [
    "## 4. Finetune the pre-trained model on a custom dataset\n",
    "\n",
    "Previously, we saw how to run inference on a pre-trained model, which was fine-tuned on SST dataset. Next, we discuss how a model can be finetuned to a custom dataset with any number of classes.\n",
    "\n",
    "The Text Embedding model can be fine-tuned on any text classification dataset in the same way the model available for inference has been fine-tuned on the SST2 movie review dataset.\n",
    "\n",
    "The model available for fine-tuning attaches a classification layer to the Text Embedding model and initializes the layer parameters to random values. The output dimension of the classification layer is determined based on the number of classes detected in the input data. The fine-tuning step fine-tunes all the model parameters to minimize prediction error on the input data and returns the fine-tuned model. The model returned by fine-tuning can be further deployed for inference. Below are the instructions for how the training data should be formatted for input to the model.\n",
    "\n",
    "- Input: A directory containing a 'data.csv' file.\n",
    "     - Each row of the first column of 'data.csv' should have integer class labels between 0 to the number of classes.\n",
    "    - Each row of the second column should have the corresponding text.\n",
    "- Output: A trained model that can be deployed for inference.\n",
    "\n",
    "Below is an example of 'data.csv' file showing values in its first two columns. Note that the file should not have any header.\n",
    "\n",
    "|   |   |\n",
    "|---|---|\n",
    "|0\t|hide new secretions from the parental units| \n",
    "|0\t|contains no wit , only labored gags| \n",
    "|1\t|that loves its characters and communicates something rather beautiful about human nature| \n",
    "|...|...|\n",
    "\n",
    "\n",
    "source: [TensorFlow Hub](model_url). License:[Apache 2.0 License](https://jumpstart-cache-alpha-us-west-2.s3-us-west-2.amazonaws.com/licenses/Apache-License/LICENSE-2.0.txt).\n",
    " \n",
    "SST2 dataset is downloaded from [TensorFlow](https://www.tensorflow.org/datasets/catalog/glue#gluesst2).\n",
    " [Apache 2.0 License](https://jumpstart-cache-prod-us-west-2.s3-us-west-2.amazonaws.com/licenses/Apache-License/LICENSE-2.0.txt).\n",
    "  [Dataset Homepage](https://nlp.stanford.edu/sentiment/index.html). "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f79c557",
   "metadata": {},
   "source": [
    "### 4.1. Retrieve jumpStart training artifacts\n",
    "\n",
    "Here, for the selected model, we retrieve the training docker container, the training algorithm source, the pre-trained model, and a python dictionary of the training hyper-parameters that the algorithm accepts with their default values. Note that the model_version=\"*\" fetches the lates model. Also, we do need to specify the training_instance_type to fetch train_image_uri."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "1a925abe",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sagemaker import image_uris, model_uris, script_uris, hyperparameters\n",
    "\n",
    "model_id, model_version = (\n",
    "    model_id,\n",
    "    \"1.1.2\",\n",
    ")  # all the other options of model_id are the same as the one in Section 2.\n",
    "training_instance_type = config.TRAINING_INSTANCE_TYPE\n",
    "\n",
    "# Retrieve the docker image\n",
    "train_image_uri = image_uris.retrieve(\n",
    "    region=None,\n",
    "    framework=None,\n",
    "    model_id=model_id,\n",
    "    model_version=model_version,\n",
    "    image_scope=\"training\",\n",
    "    instance_type=training_instance_type,\n",
    ")\n",
    "# Retrieve the training script\n",
    "train_source_uri = script_uris.retrieve(\n",
    "    model_id=model_id, model_version=model_version, script_scope=\"training\"\n",
    ")\n",
    "# Retrieve the pre-trained model tarball to further fine-tune\n",
    "train_model_uri = model_uris.retrieve(\n",
    "    model_id=model_id, model_version=model_version, model_scope=\"training\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b88da55c",
   "metadata": {},
   "source": [
    "### 4.2. Set training parameters\n",
    "\n",
    "Now that we are done with all the setup that is needed, we are ready to fine-tune our Text Classification model. To begin, let us create a `sageMaker.estimator.Estimator` object. This estimator launches the training job.\n",
    "\n",
    "There are two kinds of parameters that need to be set for training.\n",
    "\n",
    "The first one are the parameters for the training job. These include: (i) Training data path. This is S3 folder in which the input data is stored, (ii) Output path: This the s3 folder in which the training output is stored. (iii) Training instance type: This indicates the type of machine on which to run the training. Typically, we use GPU instances for these training. We defined the training instance type above to fetch the correct train_image_uri.\n",
    "\n",
    "The second set of parameters are algorithm specific training hyper-parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "670c3fb2",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Sample training data is available in this bucket\n",
    "training_data_bucket = f\"jumpstart-cache-prod-{aws_region}\"\n",
    "training_data_prefix = \"training-datasets/SST/\"\n",
    "\n",
    "training_dataset_s3_path = f\"s3://{training_data_bucket}/{training_data_prefix}\"\n",
    "\n",
    "output_bucket = DEFAULT_BUCKET\n",
    "output_prefix = \"TC\"\n",
    "\n",
    "s3_output_location = f\"s3://{output_bucket}/{output_prefix}/output\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76c10cd2",
   "metadata": {},
   "source": [
    "For algorithm specific hyper-parameters, we start by fetching python dictionary of the training hyper-parameters that the algorithm accepts with their default values. This can then be overridden to custom values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "f6320b88",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'epochs': '3', 'adam-learning-rate': '1e-6', 'batch-size': '64', 'reinitialize-top-layer': 'Auto', 'train-only-top-layer': 'False'}\n"
     ]
    }
   ],
   "source": [
    "from sagemaker import hyperparameters\n",
    "\n",
    "# Retrieve the default hyper-parameters for fine-tuning the model\n",
    "hyperparameters = hyperparameters.retrieve_default(model_id=model_id, model_version=model_version)\n",
    "\n",
    "# [Optional] Override default hyperparameters with custom values\n",
    "hyperparameters[\"batch-size\"] = \"64\"\n",
    "hyperparameters[\"adam-learning-rate\"] = \"1e-6\"\n",
    "print(hyperparameters)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a04d91d",
   "metadata": {},
   "source": [
    "### 4.3. Download, preprocess, and upload the training data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "8321a0ed",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "download: s3://jumpstart-cache-prod-us-west-2/training-datasets/SST/data.csv to data/sst2/data.csv\n"
     ]
    }
   ],
   "source": [
    "!aws s3 cp --recursive $training_dataset_s3_path data/sst2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "a482a0b3",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "data = pd.read_csv(\"data/sst2/data.csv\", header=None)\n",
    "data.columns = [\"Target\", \"Sentence Input\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d08baba0",
   "metadata": {},
   "source": [
    "View the first five observations of the training data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "346aab08",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Target</th>\n",
       "      <th>Sentence Input</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>hide new secretions from the parental units</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>contains no wit , only labored gags</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>that loves its characters and communicates som...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>remains utterly satisfied to remain the same t...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>on the worst revenge-of-the-nerds clichés the ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Target                                     Sentence Input\n",
       "0       0       hide new secretions from the parental units \n",
       "1       0               contains no wit , only labored gags \n",
       "2       1  that loves its characters and communicates som...\n",
       "3       0  remains utterly satisfied to remain the same t...\n",
       "4       0  on the worst revenge-of-the-nerds clichés the ..."
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "596e125c",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "25e5016a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "train_data, test_data = train_test_split(data, test_size=0.01, random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "d4f716e8",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "train_data.to_csv(\"data/sst2/split_train.csv\", header=False, index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1210d950",
   "metadata": {},
   "source": [
    "Upload the splitted training data into the S3 bucket. The training data is further splitted into training and validation data during training. The test data is used as hold-out data to evaluate the model performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "d5329044",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import boto3\n",
    "\n",
    "prefix = \"TC\"\n",
    "boto3.Session().resource(\"s3\").Bucket(DEFAULT_BUCKET).Object(\n",
    "    os.path.join(prefix, \"train/data.csv\")\n",
    ").upload_file(\"data/sst2/split_train.csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5e0bec47",
   "metadata": {},
   "source": [
    "### 4.4 Fine-tuning without hyperparameter optimization\n",
    "\n",
    "We start by creating the estimator object with all the required assets and then launch the training job."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "ebca36dd",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:sagemaker:Creating training-job with name: sagemaker-soln-documents--tc-finetune-2023-05-18-22-50-48-250\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2023-05-18 22:50:48 Starting - Starting the training job...\n",
      "2023-05-18 22:51:15 Starting - Preparing the instances for training.........\n",
      "2023-05-18 22:52:25 Downloading - Downloading input data...\n",
      "2023-05-18 22:53:01 Training - Downloading the training image.........\n",
      "2023-05-18 22:54:46 Training - Training image download completed. Training in progress....\u001b[34m2023-05-18 22:55:11.282934: W tensorflow/core/profiler/internal/smprofiler_timeline.cc:460] Initializing the SageMaker Profiler.\u001b[0m\n",
      "\u001b[34m2023-05-18 22:55:11.283170: W tensorflow/core/profiler/internal/smprofiler_timeline.cc:105] SageMaker Profiler is not enabled. The timeline writer thread will not be started, future recorded events will be dropped.\u001b[0m\n",
      "\u001b[34m2023-05-18 22:55:11.290441: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.2\u001b[0m\n",
      "\u001b[34m2023-05-18 22:55:11.328155: W tensorflow/core/profiler/internal/smprofiler_timeline.cc:460] Initializing the SageMaker Profiler.\u001b[0m\n",
      "\u001b[34m2023-05-18 22:55:12,950 sagemaker-training-toolkit INFO     Imported framework sagemaker_tensorflow_container.training\u001b[0m\n",
      "\u001b[34m2023-05-18 22:55:13,610 sagemaker-training-toolkit INFO     Invoking user script\u001b[0m\n",
      "\u001b[34mTraining Env:\u001b[0m\n",
      "\u001b[34m{\n",
      "    \"additional_framework_parameters\": {},\n",
      "    \"channel_input_dirs\": {\n",
      "        \"model\": \"/opt/ml/input/data/model\",\n",
      "        \"training\": \"/opt/ml/input/data/training\"\n",
      "    },\n",
      "    \"current_host\": \"algo-1\",\n",
      "    \"framework_module\": \"sagemaker_tensorflow_container.training:main\",\n",
      "    \"hosts\": [\n",
      "        \"algo-1\"\n",
      "    ],\n",
      "    \"hyperparameters\": {\n",
      "        \"adam-learning-rate\": \"1e-6\",\n",
      "        \"batch-size\": \"64\",\n",
      "        \"epochs\": \"3\",\n",
      "        \"reinitialize-top-layer\": \"Auto\",\n",
      "        \"train-only-top-layer\": \"False\"\n",
      "    },\n",
      "    \"input_config_dir\": \"/opt/ml/input/config\",\n",
      "    \"input_data_config\": {\n",
      "        \"model\": {\n",
      "            \"ContentType\": \"application/x-sagemaker-model\",\n",
      "            \"TrainingInputMode\": \"File\",\n",
      "            \"S3DistributionType\": \"FullyReplicated\",\n",
      "            \"RecordWrapperType\": \"None\"\n",
      "        },\n",
      "        \"training\": {\n",
      "            \"TrainingInputMode\": \"File\",\n",
      "            \"S3DistributionType\": \"FullyReplicated\",\n",
      "            \"RecordWrapperType\": \"None\"\n",
      "        }\n",
      "    },\n",
      "    \"input_dir\": \"/opt/ml/input\",\n",
      "    \"is_master\": true,\n",
      "    \"job_name\": \"sagemaker-soln-documents--tc-finetune-2023-05-18-22-50-48-250\",\n",
      "    \"log_level\": 20,\n",
      "    \"master_hostname\": \"algo-1\",\n",
      "    \"model_dir\": \"/opt/ml/model\",\n",
      "    \"module_dir\": \"s3://jumpstart-cache-prod-us-west-2/source-directory-tarballs/tensorflow/transfer_learning/tc/v1.2.2/sourcedir.tar.gz\",\n",
      "    \"module_name\": \"transfer_learning\",\n",
      "    \"network_interface_name\": \"eth0\",\n",
      "    \"num_cpus\": 8,\n",
      "    \"num_gpus\": 1,\n",
      "    \"output_data_dir\": \"/opt/ml/output/data\",\n",
      "    \"output_dir\": \"/opt/ml/output\",\n",
      "    \"output_intermediate_dir\": \"/opt/ml/output/intermediate\",\n",
      "    \"resource_config\": {\n",
      "        \"current_host\": \"algo-1\",\n",
      "        \"current_instance_type\": \"ml.p3.2xlarge\",\n",
      "        \"current_group_name\": \"homogeneousCluster\",\n",
      "        \"hosts\": [\n",
      "            \"algo-1\"\n",
      "        ],\n",
      "        \"instance_groups\": [\n",
      "            {\n",
      "                \"instance_group_name\": \"homogeneousCluster\",\n",
      "                \"instance_type\": \"ml.p3.2xlarge\",\n",
      "                \"hosts\": [\n",
      "                    \"algo-1\"\n",
      "                ]\n",
      "            }\n",
      "        ],\n",
      "        \"network_interface_name\": \"eth0\"\n",
      "    },\n",
      "    \"user_entry_point\": \"transfer_learning.py\"\u001b[0m\n",
      "\u001b[34m}\u001b[0m\n",
      "\u001b[34mEnvironment variables:\u001b[0m\n",
      "\u001b[34mSM_HOSTS=[\"algo-1\"]\u001b[0m\n",
      "\u001b[34mSM_NETWORK_INTERFACE_NAME=eth0\u001b[0m\n",
      "\u001b[34mSM_HPS={\"adam-learning-rate\":\"1e-6\",\"batch-size\":\"64\",\"epochs\":\"3\",\"reinitialize-top-layer\":\"Auto\",\"train-only-top-layer\":\"False\"}\u001b[0m\n",
      "\u001b[34mSM_USER_ENTRY_POINT=transfer_learning.py\u001b[0m\n",
      "\u001b[34mSM_FRAMEWORK_PARAMS={}\u001b[0m\n",
      "\u001b[34mSM_RESOURCE_CONFIG={\"current_group_name\":\"homogeneousCluster\",\"current_host\":\"algo-1\",\"current_instance_type\":\"ml.p3.2xlarge\",\"hosts\":[\"algo-1\"],\"instance_groups\":[{\"hosts\":[\"algo-1\"],\"instance_group_name\":\"homogeneousCluster\",\"instance_type\":\"ml.p3.2xlarge\"}],\"network_interface_name\":\"eth0\"}\u001b[0m\n",
      "\u001b[34mSM_INPUT_DATA_CONFIG={\"model\":{\"ContentType\":\"application/x-sagemaker-model\",\"RecordWrapperType\":\"None\",\"S3DistributionType\":\"FullyReplicated\",\"TrainingInputMode\":\"File\"},\"training\":{\"RecordWrapperType\":\"None\",\"S3DistributionType\":\"FullyReplicated\",\"TrainingInputMode\":\"File\"}}\u001b[0m\n",
      "\u001b[34mSM_OUTPUT_DATA_DIR=/opt/ml/output/data\u001b[0m\n",
      "\u001b[34mSM_CHANNELS=[\"model\",\"training\"]\u001b[0m\n",
      "\u001b[34mSM_CURRENT_HOST=algo-1\u001b[0m\n",
      "\u001b[34mSM_MODULE_NAME=transfer_learning\u001b[0m\n",
      "\u001b[34mSM_LOG_LEVEL=20\u001b[0m\n",
      "\u001b[34mSM_FRAMEWORK_MODULE=sagemaker_tensorflow_container.training:main\u001b[0m\n",
      "\u001b[34mSM_INPUT_DIR=/opt/ml/input\u001b[0m\n",
      "\u001b[34mSM_INPUT_CONFIG_DIR=/opt/ml/input/config\u001b[0m\n",
      "\u001b[34mSM_OUTPUT_DIR=/opt/ml/output\u001b[0m\n",
      "\u001b[34mSM_NUM_CPUS=8\u001b[0m\n",
      "\u001b[34mSM_NUM_GPUS=1\u001b[0m\n",
      "\u001b[34mSM_MODEL_DIR=/opt/ml/model\u001b[0m\n",
      "\u001b[34mSM_MODULE_DIR=s3://jumpstart-cache-prod-us-west-2/source-directory-tarballs/tensorflow/transfer_learning/tc/v1.2.2/sourcedir.tar.gz\u001b[0m\n",
      "\u001b[34mSM_TRAINING_ENV={\"additional_framework_parameters\":{},\"channel_input_dirs\":{\"model\":\"/opt/ml/input/data/model\",\"training\":\"/opt/ml/input/data/training\"},\"current_host\":\"algo-1\",\"framework_module\":\"sagemaker_tensorflow_container.training:main\",\"hosts\":[\"algo-1\"],\"hyperparameters\":{\"adam-learning-rate\":\"1e-6\",\"batch-size\":\"64\",\"epochs\":\"3\",\"reinitialize-top-layer\":\"Auto\",\"train-only-top-layer\":\"False\"},\"input_config_dir\":\"/opt/ml/input/config\",\"input_data_config\":{\"model\":{\"ContentType\":\"application/x-sagemaker-model\",\"RecordWrapperType\":\"None\",\"S3DistributionType\":\"FullyReplicated\",\"TrainingInputMode\":\"File\"},\"training\":{\"RecordWrapperType\":\"None\",\"S3DistributionType\":\"FullyReplicated\",\"TrainingInputMode\":\"File\"}},\"input_dir\":\"/opt/ml/input\",\"is_master\":true,\"job_name\":\"sagemaker-soln-documents--tc-finetune-2023-05-18-22-50-48-250\",\"log_level\":20,\"master_hostname\":\"algo-1\",\"model_dir\":\"/opt/ml/model\",\"module_dir\":\"s3://jumpstart-cache-prod-us-west-2/source-directory-tarballs/tensorflow/transfer_learning/tc/v1.2.2/sourcedir.tar.gz\",\"module_name\":\"transfer_learning\",\"network_interface_name\":\"eth0\",\"num_cpus\":8,\"num_gpus\":1,\"output_data_dir\":\"/opt/ml/output/data\",\"output_dir\":\"/opt/ml/output\",\"output_intermediate_dir\":\"/opt/ml/output/intermediate\",\"resource_config\":{\"current_group_name\":\"homogeneousCluster\",\"current_host\":\"algo-1\",\"current_instance_type\":\"ml.p3.2xlarge\",\"hosts\":[\"algo-1\"],\"instance_groups\":[{\"hosts\":[\"algo-1\"],\"instance_group_name\":\"homogeneousCluster\",\"instance_type\":\"ml.p3.2xlarge\"}],\"network_interface_name\":\"eth0\"},\"user_entry_point\":\"transfer_learning.py\"}\u001b[0m\n",
      "\u001b[34mSM_USER_ARGS=[\"--adam-learning-rate\",\"1e-6\",\"--batch-size\",\"64\",\"--epochs\",\"3\",\"--reinitialize-top-layer\",\"Auto\",\"--train-only-top-layer\",\"False\"]\u001b[0m\n",
      "\u001b[34mSM_OUTPUT_INTERMEDIATE_DIR=/opt/ml/output/intermediate\u001b[0m\n",
      "\u001b[34mSM_CHANNEL_MODEL=/opt/ml/input/data/model\u001b[0m\n",
      "\u001b[34mSM_CHANNEL_TRAINING=/opt/ml/input/data/training\u001b[0m\n",
      "\u001b[34mSM_HP_ADAM-LEARNING-RATE=1e-6\u001b[0m\n",
      "\u001b[34mSM_HP_BATCH-SIZE=64\u001b[0m\n",
      "\u001b[34mSM_HP_EPOCHS=3\u001b[0m\n",
      "\u001b[34mSM_HP_REINITIALIZE-TOP-LAYER=Auto\u001b[0m\n",
      "\u001b[34mSM_HP_TRAIN-ONLY-TOP-LAYER=False\u001b[0m\n",
      "\u001b[34mPYTHONPATH=/opt/ml/code:/usr/local/bin:/usr/local/lib/python37.zip:/usr/local/lib/python3.7:/usr/local/lib/python3.7/lib-dynload:/usr/local/lib/python3.7/site-packages\u001b[0m\n",
      "\u001b[34mInvoking script with the following command:\u001b[0m\n",
      "\u001b[34m/usr/local/bin/python3.7 transfer_learning.py --adam-learning-rate 1e-6 --batch-size 64 --epochs 3 --reinitialize-top-layer Auto --train-only-top-layer False\u001b[0m\n",
      "\u001b[34m[2023-05-18 22:55:27.579 ip-10-0-220-59.us-west-2.compute.internal:36 INFO utils.py:27] RULE_JOB_STOP_SIGNAL_FILENAME: None\u001b[0m\n",
      "\u001b[34m[2023-05-18 22:55:27.760 ip-10-0-220-59.us-west-2.compute.internal:36 INFO profiler_config_parser.py:102] User has disabled profiler.\u001b[0m\n",
      "\u001b[34mModel: \"functional_1\"\u001b[0m\n",
      "\u001b[34m__________________________________________________________________________________________________\u001b[0m\n",
      "\u001b[34mLayer (type)                    Output Shape         Param #     Connected to                     \u001b[0m\n",
      "\u001b[34m==================================================================================================\u001b[0m\n",
      "\u001b[34minput_word_ids (InputLayer)     [(None, None)]       0                                            \u001b[0m\n",
      "\u001b[34m__________________________________________________________________________________________________\u001b[0m\n",
      "\u001b[34minput_mask (InputLayer)         [(None, None)]       0                                            \u001b[0m\n",
      "\u001b[34m__________________________________________________________________________________________________\u001b[0m\n",
      "\u001b[34minput_type_ids (InputLayer)     [(None, None)]       0                                            \u001b[0m\n",
      "\u001b[34m__________________________________________________________________________________________________\u001b[0m\n",
      "\u001b[34mkeras_layer (KerasLayer)        [(None, 768), (None, 109482241   input_word_ids[0][0]             \n",
      "                                                                 input_mask[0][0]                 \n",
      "                                                                 input_type_ids[0][0]             \u001b[0m\n",
      "\u001b[34m__________________________________________________________________________________________________\u001b[0m\n",
      "\u001b[34mdropout (Dropout)               (None, 768)          0           keras_layer[0][0]                \u001b[0m\n",
      "\u001b[34m__________________________________________________________________________________________________\u001b[0m\n",
      "\u001b[34mdense (Dense)                   (None, 2)            1538        dropout[0][0]                    \u001b[0m\n",
      "\u001b[34m==================================================================================================\u001b[0m\n",
      "\u001b[34mTotal params: 109,483,779\u001b[0m\n",
      "\u001b[34mTrainable params: 109,483,778\u001b[0m\n",
      "\u001b[34mNon-trainable params: 1\u001b[0m\n",
      "\u001b[34m__________________________________________________________________________________________________\u001b[0m\n",
      "\u001b[34m[2023-05-18 22:55:46.783 ip-10-0-220-59.us-west-2.compute.internal:36 INFO json_config.py:91] Creating hook from json_config at /opt/ml/input/config/debughookconfig.json.\u001b[0m\n",
      "\u001b[34m[2023-05-18 22:55:46.784 ip-10-0-220-59.us-west-2.compute.internal:36 INFO hook.py:201] tensorboard_dir has not been set for the hook. SMDebug will not be exporting tensorboard summaries.\u001b[0m\n",
      "\u001b[34m[2023-05-18 22:55:46.785 ip-10-0-220-59.us-west-2.compute.internal:36 INFO hook.py:255] Saving to /opt/ml/output/tensors\u001b[0m\n",
      "\u001b[34m[2023-05-18 22:55:46.785 ip-10-0-220-59.us-west-2.compute.internal:36 INFO state_store.py:77] The checkpoint config file /opt/ml/input/config/checkpointconfig.json does not exist.\u001b[0m\n",
      "\u001b[34m[2023-05-18 22:55:46.786 ip-10-0-220-59.us-west-2.compute.internal:36 INFO hook.py:425] Monitoring the collections: sm_metrics, metrics, losses\u001b[0m\n",
      "\u001b[34m[2023-05-18 22:55:46.789 ip-10-0-220-59.us-west-2.compute.internal:36 INFO hook.py:425] Monitoring the collections: sm_metrics, metrics, losses\u001b[0m\n",
      "\u001b[34mEpoch 1/3\u001b[0m\n",
      "\u001b[34m845/845 - 264s - loss: 0.5694 - accuracy: 0.7042 - val_loss: 0.3484 - val_accuracy: 0.8717 - batch: 0.0000e+00\u001b[0m\n",
      "\u001b[34mEpoch 2/3\u001b[0m\n",
      "\u001b[34m845/845 - 263s - loss: 0.3318 - accuracy: 0.8703 - val_loss: 0.2848 - val_accuracy: 0.8895 - batch: 1.0000\u001b[0m\n",
      "\u001b[34mEpoch 3/3\u001b[0m\n",
      "\u001b[34m845/845 - 263s - loss: 0.2978 - accuracy: 0.8825 - val_loss: 0.2733 - val_accuracy: 0.8945 - batch: 2.0000\u001b[0m\n",
      "\u001b[34m2023-05-18 22:55:14.012611: W tensorflow/core/profiler/internal/smprofiler_timeline.cc:460] Initializing the SageMaker Profiler.\u001b[0m\n",
      "\u001b[34m2023-05-18 22:55:14.012799: W tensorflow/core/profiler/internal/smprofiler_timeline.cc:105] SageMaker Profiler is not enabled. The timeline writer thread will not be started, future recorded events will be dropped.\u001b[0m\n",
      "\u001b[34m2023-05-18 22:55:14.055607: W tensorflow/core/profiler/internal/smprofiler_timeline.cc:460] Initializing the SageMaker Profiler.\u001b[0m\n",
      "\u001b[34mWARNING:tensorflow:From /usr/local/lib/python3.7/site-packages/tensorflow/python/training/tracking/tracking.py:111: Model.state_updates (from tensorflow.python.keras.engine.training) is deprecated and will be removed in a future version.\u001b[0m\n",
      "\u001b[34mInstructions for updating:\u001b[0m\n",
      "\u001b[34mThis property should not be used in TensorFlow 2.0, as updates are applied automatically.\u001b[0m\n",
      "\u001b[34mWARNING:tensorflow:From /usr/local/lib/python3.7/site-packages/tensorflow/python/training/tracking/tracking.py:111: Model.state_updates (from tensorflow.python.keras.engine.training) is deprecated and will be removed in a future version.\u001b[0m\n",
      "\u001b[34mInstructions for updating:\u001b[0m\n",
      "\u001b[34mThis property should not be used in TensorFlow 2.0, as updates are applied automatically.\u001b[0m\n",
      "\u001b[34m2023-05-18 23:09:30.355561: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.\u001b[0m\n",
      "\u001b[34mWARNING:tensorflow:From /usr/local/lib/python3.7/site-packages/tensorflow/python/training/tracking/tracking.py:111: Layer.updates (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.\u001b[0m\n",
      "\u001b[34mInstructions for updating:\u001b[0m\n",
      "\u001b[34mThis property should not be used in TensorFlow 2.0, as updates are applied automatically.\u001b[0m\n",
      "\u001b[34mWARNING:tensorflow:From /usr/local/lib/python3.7/site-packages/tensorflow/python/training/tracking/tracking.py:111: Layer.updates (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.\u001b[0m\n",
      "\u001b[34mInstructions for updating:\u001b[0m\n",
      "\u001b[34mThis property should not be used in TensorFlow 2.0, as updates are applied automatically.\u001b[0m\n",
      "\u001b[34m2023-05-18 23:09:38,625 sagemaker-training-toolkit INFO     Reporting training SUCCESS\u001b[0m\n",
      "\n",
      "2023-05-18 23:09:45 Uploading - Uploading generated training model\n",
      "2023-05-18 23:10:51 Completed - Training job completed\n",
      "Training seconds: 1106\n",
      "Billable seconds: 1106\n"
     ]
    }
   ],
   "source": [
    "from sagemaker.estimator import Estimator\n",
    "from sagemaker.utils import name_from_base\n",
    "from sagemaker.tuner import HyperparameterTuner\n",
    "\n",
    "from sagemaker import get_execution_role\n",
    "\n",
    "role = get_execution_role()\n",
    "\n",
    "training_job_name = f\"{config.SOLUTION_PREFIX}-tc-finetune\"\n",
    "\n",
    "# Create SageMaker Estimator instance\n",
    "tc_estimator = Estimator(\n",
    "    role=role,\n",
    "    image_uri=train_image_uri,\n",
    "    source_dir=train_source_uri,\n",
    "    model_uri=train_model_uri,\n",
    "    entry_point=\"transfer_learning.py\",\n",
    "    instance_count=1,\n",
    "    instance_type=training_instance_type,\n",
    "    max_run=360000,\n",
    "    hyperparameters=hyperparameters,\n",
    "    output_path=s3_output_location,\n",
    "    tags=[{\"Key\": config.TAG_KEY, \"Value\": config.SOLUTION_PREFIX}],\n",
    "    base_job_name=training_job_name,\n",
    ")\n",
    "\n",
    "training_data_path_updated = f\"s3://{DEFAULT_BUCKET}/{prefix}/train\"\n",
    "# Launch a SageMaker Training job by passing s3 path of the training data\n",
    "tc_estimator.fit({\"training\": training_data_path_updated}, logs=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c66ece8",
   "metadata": {},
   "source": [
    "### 4.5. Deploy & run Inference on the fine-tuned model\n",
    "\n",
    "A trained model does nothing on its own. We now want to use the model to perform inference. For this example, that means predicting the class label of an input sentence. We follow the same steps as in 3. Run inference on the pre-trained model. We start by retrieving the jumpstart artifacts for deploying an endpoint. However, instead of base_predictor, we deploy the tc_estimator that we fine-tuned.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "7e070ef5",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:sagemaker.image_uris:Ignoring unnecessary Python version: py37.\n",
      "INFO:sagemaker:Repacking model artifact (s3://sagemaker-us-west-2-688520471316/TC/output/sagemaker-soln-documents--tc-finetune-2023-05-18-22-50-48-250/output/model.tar.gz), script artifact (s3://jumpstart-cache-prod-us-west-2/source-directory-tarballs/tensorflow/inference/tc/v1.1.1/sourcedir.tar.gz), and dependencies ([]) into single tar.gz file located at s3://sagemaker-us-west-2-688520471316/sagemaker-jumpstart-2023-05-18-23-11-28-349/model.tar.gz. This may take some time depending on model size...\n",
      "INFO:sagemaker:Creating model with name: sagemaker-jumpstart-2023-05-18-23-11-28-349\n",
      "INFO:sagemaker:Creating endpoint-config with name sagemaker-soln-documents--1eed6b-tc-finetune-endpoint\n",
      "INFO:sagemaker:Creating endpoint with name sagemaker-soln-documents--1eed6b-tc-finetune-endpoint\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-----!"
     ]
    }
   ],
   "source": [
    "import uuid\n",
    "\n",
    "inference_instance_type = \"ml.g4dn.2xlarge\"\n",
    "\n",
    "# Retrieve the inference docker container uri\n",
    "deploy_image_uri = image_uris.retrieve(\n",
    "    region=None,\n",
    "    framework=None,\n",
    "    image_scope=\"inference\",\n",
    "    model_id=model_id,\n",
    "    model_version=model_version,\n",
    "    instance_type=inference_instance_type,\n",
    ")\n",
    "# Retrieve the inference script uri\n",
    "deploy_source_uri = script_uris.retrieve(\n",
    "    model_id=model_id, model_version=model_version, script_scope=\"inference\"\n",
    ")\n",
    "unique_hash = str(uuid.uuid4())[:6]\n",
    "endpoint_name_tc_finetune = f\"{config.SOLUTION_PREFIX}-{unique_hash}-tc-finetune-endpoint\"\n",
    "\n",
    "# Use the estimator from the previous step to deploy to a SageMaker endpoint\n",
    "finetuned_predictor = tc_estimator.deploy(\n",
    "    initial_instance_count=1,\n",
    "    instance_type=inference_instance_type,\n",
    "    entry_point=\"inference.py\",\n",
    "    image_uri=deploy_image_uri,\n",
    "    source_dir=deploy_source_uri,\n",
    "    endpoint_name=endpoint_name_tc_finetune,\n",
    ")\n",
    "\n",
    "time.sleep(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f4504856",
   "metadata": {},
   "source": [
    "Next, we query each of the examples in the test data to get its predicted label."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "416fa07d",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "ground_truth, test_examples = (\n",
    "    test_data.iloc[:, 0].values.tolist(),\n",
    "    test_data.iloc[:, 1].values.tolist(),\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "6562cca5",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "newline, bold, unbold = \"\\n\", \"\\033[1m\", \"\\033[0m\"\n",
    "\n",
    "\n",
    "def query_endpoint(encoded_text, predictor):\n",
    "    response = predictor.predict(\n",
    "        encoded_text,\n",
    "        {\"ContentType\": \"application/x-text\", \"Accept\": \"application/json;verbose\"},\n",
    "    )\n",
    "    return response\n",
    "\n",
    "\n",
    "def parse_response(query_response):\n",
    "    model_predictions = json.loads(query_response)\n",
    "    probabilities, labels, predicted_label = (\n",
    "        model_predictions[\"probabilities\"],\n",
    "        model_predictions[\"labels\"],\n",
    "        model_predictions[\"predicted_label\"],\n",
    "    )\n",
    "    return probabilities, labels, predicted_label\n",
    "\n",
    "\n",
    "predict_prob, predict_label = [], []\n",
    "for text in test_examples:\n",
    "    query_response = query_endpoint(text.encode(\"utf-8\"), finetuned_predictor)\n",
    "    probabilities, labels, predicted_label = parse_response(query_response)\n",
    "    predict_prob.append(probabilities)\n",
    "    predict_label.append(predicted_label)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "21485883",
   "metadata": {},
   "source": [
    "### 4.6. Compute evaluation metrics\n",
    "Since it is a binary classification task, we use [accuracy score](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.accuracy_score.html) and [f1 score](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.f1_score.html) as the evaluation metrics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "928ca5fb",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import accuracy_score, f1_score\n",
    "\n",
    "f1 = f1_score(predict_label, ground_truth)\n",
    "accuracy = accuracy_score(predict_label, ground_truth)\n",
    "result = {\"Accuracy\": [accuracy], \"F1 Score\": [f1]}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "d037c083",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "result = pd.DataFrame.from_dict(result, orient=\"index\", columns=[\"No HPO\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "21c18483",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>No HPO</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Accuracy</th>\n",
       "      <td>0.579795</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>F1 Score</th>\n",
       "      <td>0.734013</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            No HPO\n",
       "Accuracy  0.579795\n",
       "F1 Score  0.734013"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71e2e469",
   "metadata": {},
   "source": [
    "For accuracy and F1 score, larger value indicates the better performance."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "81ec4cc0",
   "metadata": {},
   "source": [
    "## 5. Finetune the pre-trained model on a custom dataset with automatic model tuning (AMT)\n",
    "\n",
    "Amazon SageMaker automatic model tuning, also known as hyperparameter tuning, finds the best version of a model by running many training jobs on your dataset using the algorithm and ranges of hyperparameters that you specify. It then chooses the hyperparameter values that result in a model that performs the best, as measured by a metric that you choose. We use a [HyperparameterTuner](https://sagemaker.readthedocs.io/en/stable/api/training/tuner.html) object to interact with Amazon SageMaker hyperparameter tuning APIs."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5b4d9193",
   "metadata": {},
   "source": [
    "### 5.1. Fine-tuning with hyperparameter optimization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "f23d1bf5",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sagemaker.tuner import ContinuousParameter\n",
    "\n",
    "\n",
    "# Define objective metric per framework, based on which the best model is selected.\n",
    "metric_definitions_per_model = {\n",
    "    \"tensorflow\": {\n",
    "        \"metrics\": [{\"Name\": \"val_accuracy\", \"Regex\": \"val_accuracy: ([0-9\\\\.]+)\"}],\n",
    "        \"type\": \"Maximize\",\n",
    "    }\n",
    "}\n",
    "\n",
    "# You can select from the hyperparameters supported by the model, and configure ranges of values to be searched for training the optimal model.(https://docs.aws.amazon.com/sagemaker/latest/dg/automatic-model-tuning-define-ranges.html)\n",
    "hyperparameter_ranges = {\n",
    "    \"adam-learning-rate\": ContinuousParameter(0.00001, 0.01, scaling_type=\"Logarithmic\")\n",
    "}\n",
    "\n",
    "# Increase the total number of training jobs run by AMT, for increased accuracy (and training time).\n",
    "max_jobs = 6\n",
    "# Change parallel training jobs run by AMT to reduce total training time, constrained by your account limits.\n",
    "# if max_jobs=max_parallel_jobs then Bayesian search turns to Random.\n",
    "max_parallel_jobs = 6"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "ea0aa2ae",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:sagemaker.estimator:No finished training job found associated with this estimator. Please make sure this estimator is only used for building workflow config\n",
      "INFO:sagemaker:Creating hyperparameter tuning job with name: sagemaker-soln-docum-230518-2328\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ".......................................................................................................................................................................................................................................................!\n"
     ]
    }
   ],
   "source": [
    "from sagemaker.estimator import Estimator\n",
    "from sagemaker.utils import name_from_base\n",
    "from sagemaker.tuner import HyperparameterTuner\n",
    "\n",
    "tuning_job_name = f\"{config.SOLUTION_PREFIX}-{unique_hash}-tc-hpo\"\n",
    "\n",
    "# Create SageMaker Estimator instance\n",
    "tc_estimator = Estimator(\n",
    "    role=role,\n",
    "    image_uri=train_image_uri,\n",
    "    source_dir=train_source_uri,\n",
    "    model_uri=train_model_uri,\n",
    "    entry_point=\"transfer_learning.py\",\n",
    "    instance_count=1,\n",
    "    instance_type=training_instance_type,\n",
    "    max_run=360000,\n",
    "    hyperparameters=hyperparameters,\n",
    "    output_path=s3_output_location,\n",
    "    base_job_name=tuning_job_name,\n",
    "    tags=[{\"Key\": config.TAG_KEY, \"Value\": config.SOLUTION_PREFIX}],\n",
    ")\n",
    "\n",
    "\n",
    "metric_definitions = next(\n",
    "    value for key, value in metric_definitions_per_model.items() if model_id.startswith(key)\n",
    ")\n",
    "\n",
    "hp_tuner = HyperparameterTuner(\n",
    "    tc_estimator,\n",
    "    metric_definitions[\"metrics\"][0][\"Name\"],\n",
    "    hyperparameter_ranges,\n",
    "    metric_definitions[\"metrics\"],\n",
    "    max_jobs=max_jobs,\n",
    "    max_parallel_jobs=max_parallel_jobs,\n",
    "    objective_type=metric_definitions[\"type\"],\n",
    "    base_tuning_job_name=tuning_job_name,\n",
    ")\n",
    "\n",
    "# Launch a SageMaker Tuning job to search for the best hyperparameters\n",
    "hp_tuner.fit({\"training\": training_data_path_updated})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "180b2ddd",
   "metadata": {},
   "source": [
    "### 5.2. Deploy & run Inference on the fine-tuned model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "856c0ca9",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:sagemaker.image_uris:Ignoring unnecessary Python version: py37.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "2023-05-18 23:48:57 Starting - Preparing the instances for training\n",
      "2023-05-18 23:48:57 Downloading - Downloading input data\n",
      "2023-05-18 23:48:57 Training - Training image download completed. Training in progress.\n",
      "2023-05-18 23:48:57 Uploading - Uploading generated training model\n",
      "2023-05-18 23:48:57 Completed - Resource retained for reuse\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:sagemaker:Repacking model artifact (s3://sagemaker-us-west-2-688520471316/TC/output/sagemaker-soln-docum-230518-2328-003-ec9c36b1/output/model.tar.gz), script artifact (s3://jumpstart-cache-prod-us-west-2/source-directory-tarballs/tensorflow/inference/tc/v1.1.1/sourcedir.tar.gz), and dependencies ([]) into single tar.gz file located at s3://sagemaker-us-west-2-688520471316/sagemaker-jumpstart-2023-05-18-23-49-25-130/model.tar.gz. This may take some time depending on model size...\n",
      "INFO:sagemaker:Creating model with name: sagemaker-jumpstart-2023-05-18-23-49-25-130\n",
      "INFO:sagemaker:Creating endpoint-config with name sagemaker-soln-documents--tc-hpo-endpoint\n",
      "INFO:sagemaker:Creating endpoint with name sagemaker-soln-documents--tc-hpo-endpoint\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "------!"
     ]
    }
   ],
   "source": [
    "# Retrieve the inference docker container uri\n",
    "deploy_image_uri = image_uris.retrieve(\n",
    "    region=None,\n",
    "    framework=None,\n",
    "    image_scope=\"inference\",\n",
    "    model_id=model_id,\n",
    "    model_version=model_version,\n",
    "    instance_type=inference_instance_type,\n",
    ")\n",
    "# Retrieve the inference script uri\n",
    "deploy_source_uri = script_uris.retrieve(\n",
    "    model_id=model_id, model_version=model_version, script_scope=\"inference\"\n",
    ")\n",
    "\n",
    "endpoint_name_hpo = f\"{config.SOLUTION_PREFIX}-tc-hpo-endpoint\"\n",
    "\n",
    "# Use the estimator from the previous step to deploy to a SageMaker endpoint\n",
    "finetuned_predictor_hpo = hp_tuner.deploy(\n",
    "    initial_instance_count=1,\n",
    "    instance_type=inference_instance_type,\n",
    "    entry_point=\"inference.py\",\n",
    "    image_uri=deploy_image_uri,\n",
    "    source_dir=deploy_source_uri,\n",
    "    endpoint_name=endpoint_name_hpo,\n",
    ")\n",
    "\n",
    "time.sleep(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "a59bb6a2",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "predict_prob_hpo, predict_label_hpo = [], []\n",
    "for text in test_examples:\n",
    "    query_response = query_endpoint(text.encode(\"utf-8\"), finetuned_predictor_hpo)\n",
    "    probabilities, labels, predicted_label = parse_response(query_response)\n",
    "    predict_prob_hpo.append(probabilities)\n",
    "    predict_label_hpo.append(predicted_label)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "f9d4afd1",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "f1_hpo = f1_score(predict_label_hpo, ground_truth)\n",
    "accuracy_hpo = accuracy_score(predict_label_hpo, ground_truth)\n",
    "result_hpo = {\"Accuracy\": [accuracy_hpo], \"F1 Score\": [f1_hpo]}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "20cc990d",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "result_hpo = pd.DataFrame.from_dict(result_hpo, orient=\"index\", columns=[\"With HPO\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "d5bb4f36",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>No HPO</th>\n",
       "      <th>With HPO</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Accuracy</th>\n",
       "      <td>0.579795</td>\n",
       "      <td>0.944363</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>F1 Score</th>\n",
       "      <td>0.734013</td>\n",
       "      <td>0.953202</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            No HPO  With HPO\n",
       "Accuracy  0.579795  0.944363\n",
       "F1 Score  0.734013  0.953202"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.concat([result, result_hpo], axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5333a76a",
   "metadata": {},
   "source": [
    "We can see results with hyperparameter optimization shows better performance on the hold-out test data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6b451d4",
   "metadata": {},
   "source": [
    "## 5.3. Clean Up the endpoint\n",
    "\n",
    "When you've finished with the summarization endpoint (and associated\n",
    "endpoint-config), make sure that you delete it to avoid accidental\n",
    "charges."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "76ab179f",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:sagemaker:Deleting model with name: sagemaker-jumpstart-2023-05-18-23-11-28-349\n",
      "INFO:sagemaker:Deleting endpoint configuration with name: sagemaker-soln-documents--1eed6b-tc-finetune-endpoint\n",
      "INFO:sagemaker:Deleting endpoint with name: sagemaker-soln-documents--1eed6b-tc-finetune-endpoint\n",
      "INFO:sagemaker:Deleting model with name: sagemaker-jumpstart-2023-05-18-23-49-25-130\n",
      "INFO:sagemaker:Deleting endpoint configuration with name: sagemaker-soln-documents--tc-hpo-endpoint\n",
      "INFO:sagemaker:Deleting endpoint with name: sagemaker-soln-documents--tc-hpo-endpoint\n"
     ]
    }
   ],
   "source": [
    "# Delete the SageMaker endpoint and the attached resources\n",
    "finetuned_predictor.delete_model()\n",
    "finetuned_predictor.delete_endpoint()\n",
    "\n",
    "finetuned_predictor_hpo.delete_model()\n",
    "finetuned_predictor_hpo.delete_endpoint()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e6d93c46",
   "metadata": {},
   "source": [
    "## Next Stage\n",
    "\n",
    "We've just looked at how you can query document for specific information.\n",
    "Up next we look at a technique that can be used to query the document for\n",
    "specifics, called Question Answering.\n",
    "\n",
    "[Click here to continue with Question and Answering.](./3_question_answering.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b64b32a3",
   "metadata": {},
   "source": [
    "## Notebook CI Test Results\n",
    "\n",
    "This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.\n",
    "\n",
    "![This us-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/us-east-1/introduction_to_applying_machine_learning|identify_key_insights_from_textual_document|document_text_classification.ipynb)\n",
    "\n",
    "![This us-east-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/us-east-2/introduction_to_applying_machine_learning|identify_key_insights_from_textual_document|document_text_classification.ipynb)\n",
    "\n",
    "![This us-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/us-west-1/introduction_to_applying_machine_learning|identify_key_insights_from_textual_document|document_text_classification.ipynb)\n",
    "\n",
    "![This ca-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ca-central-1/introduction_to_applying_machine_learning|identify_key_insights_from_textual_document|document_text_classification.ipynb)\n",
    "\n",
    "![This sa-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/sa-east-1/introduction_to_applying_machine_learning|identify_key_insights_from_textual_document|document_text_classification.ipynb)\n",
    "\n",
    "![This eu-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/eu-west-1/introduction_to_applying_machine_learning|identify_key_insights_from_textual_document|document_text_classification.ipynb)\n",
    "\n",
    "![This eu-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/eu-west-2/introduction_to_applying_machine_learning|identify_key_insights_from_textual_document|document_text_classification.ipynb)\n",
    "\n",
    "![This eu-west-3 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/eu-west-3/introduction_to_applying_machine_learning|identify_key_insights_from_textual_document|document_text_classification.ipynb)\n",
    "\n",
    "![This eu-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/eu-central-1/introduction_to_applying_machine_learning|identify_key_insights_from_textual_document|document_text_classification.ipynb)\n",
    "\n",
    "![This eu-north-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/eu-north-1/introduction_to_applying_machine_learning|identify_key_insights_from_textual_document|document_text_classification.ipynb)\n",
    "\n",
    "![This ap-southeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ap-southeast-1/introduction_to_applying_machine_learning|identify_key_insights_from_textual_document|document_text_classification.ipynb)\n",
    "\n",
    "![This ap-southeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ap-southeast-2/introduction_to_applying_machine_learning|identify_key_insights_from_textual_document|document_text_classification.ipynb)\n",
    "\n",
    "![This ap-northeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ap-northeast-1/introduction_to_applying_machine_learning|identify_key_insights_from_textual_document|document_text_classification.ipynb)\n",
    "\n",
    "![This ap-northeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ap-northeast-2/introduction_to_applying_machine_learning|identify_key_insights_from_textual_document|document_text_classification.ipynb)\n",
    "\n",
    "![This ap-south-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ap-south-1/introduction_to_applying_machine_learning|identify_key_insights_from_textual_document|document_text_classification.ipynb)\n"
   ]
  }
 ],
 "metadata": {
  "availableInstances": [
   {
    "_defaultOrder": 0,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.t3.medium",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 1,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.t3.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 2,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.t3.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 3,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.t3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 4,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 5,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 6,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 7,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 8,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 9,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 10,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 11,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 12,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5d.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 13,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5d.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 14,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5d.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 15,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5d.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 16,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5d.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 17,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5d.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 18,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5d.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 19,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 20,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": true,
    "memoryGiB": 0,
    "name": "ml.geospatial.interactive",
    "supportedImageNames": [
     "sagemaker-geospatial-v1-0"
    ],
    "vcpuNum": 0
   },
   {
    "_defaultOrder": 21,
    "_isFastLaunch": true,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.c5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 22,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.c5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 23,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.c5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 24,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.c5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 25,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 72,
    "name": "ml.c5.9xlarge",
    "vcpuNum": 36
   },
   {
    "_defaultOrder": 26,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 96,
    "name": "ml.c5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 27,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 144,
    "name": "ml.c5.18xlarge",
    "vcpuNum": 72
   },
   {
    "_defaultOrder": 28,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.c5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 29,
    "_isFastLaunch": true,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g4dn.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 30,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g4dn.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 31,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g4dn.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 32,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g4dn.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 33,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g4dn.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 34,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g4dn.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 35,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 61,
    "name": "ml.p3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 36,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 244,
    "name": "ml.p3.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 37,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 488,
    "name": "ml.p3.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 38,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.p3dn.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 39,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.r5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 40,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.r5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 41,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.r5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 42,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.r5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 43,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.r5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 44,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.r5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 45,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.r5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 46,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.r5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 47,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 48,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 49,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 50,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 51,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 52,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 53,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.g5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 54,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.g5.48xlarge",
    "vcpuNum": 192
   }
  ],
  "kernelspec": {
   "display_name": "Python 3 (PyTorch 1.13 Python 3.9 CPU Optimized)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-west-2:236514542706:image/pytorch-1.13-cpu-py39"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}