 archived/Amazon_JumpStart_Question_Answering.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "30a2b125",
   "metadata": {},
   "source": [
    "# Introduction to JumpStart - Extractive Question Answering"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "566bfef2",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook. \n",
    "\n",
    "![This us-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-west-2/introduction_to_amazon_algorithms|jumpstart_question_answering|Amazon_JumpStart_Question_Answering.ipynb)\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "46c3a1bf",
   "metadata": {},
   "source": [
    "---\n",
    "Welcome to Amazon [SageMaker JumpStart](https://docs.aws.amazon.com/sagemaker/latest/dg/studio-jumpstart.html)! You can use JumpStart to solve many Machine Learning tasks through one-click in SageMaker Studio, or through [SageMaker JumpStart API](https://sagemaker.readthedocs.io/en/stable/overview.html#use-prebuilt-models-with-sagemaker-jumpstart). \n",
    "\n",
    "In this notebook, we demonstrate how to use the JumpStart API for Extractive Question Answering task. Question Answering models take as input a pair of question-context strings, and returns a sub-string from the context as an answer to the question. We demonstrate two use cases of Question Answering models:\n",
    "\n",
    "* How to use a Transformer model pre-trained on English dataset, and fine-tuned on [SQuAD2.0](https://rajpurkar.github.io/SQuAD-explorer/) dataset to perform Extractive Question Answering.\n",
    "* How to fine-tune a pre-trained Transformer model to a custom Extractive Question Answering dataset, and then run inference on the fine-tuned model.\n",
    "\n",
    "Note: This notebook was tested on ml.t3.medium instance in Amazon SageMaker Studio with Python 3 (Data Science) kernel and in Amazon SageMaker Notebook instance with conda_python3 kernel.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3802be34",
   "metadata": {},
   "source": [
    "1. [Set Up](#1.-Set-Up)\n",
    "2. [Select a pre-trained model](#2.-Select-a-pre-trained-model)\n",
    "3. [Run inference on the pre-trained model](#3.-Run-inference-on-the-pre-trained-model)\n",
    "    * [Retrieve JumpStart Artifacts & Deploy an Endpoint](#3.1.-Retrieve-JumpStart-Artifacts-&-Deploy-an-Endpoint)\n",
    "    * [Example question context pair for inference](#3.2.-Example-question-context-pair-for-inference)\n",
    "    * [Query endpoint and parse response](#3.3.-Query-endpoint-and-parse-response)\n",
    "    * [Clean up the endpoint](#3.4.-Clean-up-the-endpoint)\n",
    "4. [Finetune the pre-trained model on a custom dataset](#4.-Finetune-the-pre-trained-model-on-a-custom-dataset)\n",
    "    * [Retrieve JumpStart Training artifacts](#4.1.-Retrieve-JumpStart-Training-artifacts)\n",
    "    * [Set Training parameters](#4.2.-Set-Training-parameters)\n",
    "    * [Train with Automatic Model Tuning (HPO)](#AMT)\n",
    "    * [Start Training](#4.4.-Start-Training)\n",
    "    * [Deploy & run Inference on the fine-tuned model](#4.5.-Deploy-&-run-Inference-on-the-fine-tuned-model)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1412adc5",
   "metadata": {},
   "source": [
    "## 1. Set Up\n",
    "***\n",
    "Before executing the notebook, there are some initial steps required for setup. This notebook requires latest version of sagemaker and ipywidgets.\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6ba79bba",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "!pip install sagemaker ipywidgets --upgrade --quiet"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e119ce3",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "To train and host on Amazon Sagemaker, we need to setup and authenticate the use of AWS services. Here, we use the execution role associated with the current notebook instance as the AWS account role with SageMaker access. It has necessary permissions, including access to your data in S3. \n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "304a7499",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import sagemaker, boto3, json\n",
    "from sagemaker import get_execution_role\n",
    "\n",
    "aws_role = get_execution_role()\n",
    "aws_region = boto3.Session().region_name\n",
    "sess = sagemaker.Session()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8f111b63",
   "metadata": {},
   "source": [
    "## 2. Select a pre-trained model\n",
    "***\n",
    "You can continue with the default model, or can choose a different model from the dropdown generated upon running the next cell. A complete list of JumpStart models can also be accessed at [JumpStart Models](https://sagemaker.readthedocs.io/en/stable/doc_utils/jumpstart.html#).\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2432bfbf",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "model_id = \"huggingface-eqa-bert-base-uncased\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b46908b4",
   "metadata": {},
   "source": [
    "***\n",
    "[Optional] Select a different JumpStart model. Here, we download jumpstart model_manifest file from the jumpstart s3 bucket, filter-out all the Extractive Question Answering models and select a model.\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2d6d2e35",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import IPython\n",
    "import ipywidgets as widgets\n",
    "\n",
    "# download JumpStart model_manifest file.\n",
    "boto3.client(\"s3\").download_file(\n",
    "    f\"jumpstart-cache-prod-{aws_region}\", \"models_manifest.json\", \"models_manifest.json\"\n",
    ")\n",
    "with open(\"models_manifest.json\", \"rb\") as json_file:\n",
    "    model_list = json.load(json_file)\n",
    "\n",
    "# filter-out all the Extractive Question Answering models from the manifest list.\n",
    "eqa_models_all_versions, eqa_models = [\n",
    "    model[\"model_id\"] for model in model_list if \"-eqa-\" in model[\"model_id\"]\n",
    "], []\n",
    "[eqa_models.append(model) for model in eqa_models_all_versions if model not in eqa_models]\n",
    "\n",
    "# display the model-ids in a dropdown, for user to select a model.\n",
    "dropdown = widgets.Dropdown(\n",
    "    value=model_id,\n",
    "    options=eqa_models,\n",
    "    description=\"JumpStart Extractive Question Answering Models:\",\n",
    "    style={\"description_width\": \"initial\"},\n",
    "    layout={\"width\": \"max-content\"},\n",
    ")\n",
    "display(IPython.display.Markdown(\"## Select a JumpStart pre-trained model from the dropdown below\"))\n",
    "display(dropdown)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d3231650",
   "metadata": {},
   "source": [
    "## 3. Run inference on the pre-trained model\n",
    "***\n",
    "Using JumpStart, we can perform inference on a pre-trained model, even without fine-tuning it first on a custom dataset. The model available for deployment is created by attaching an answer extracting layer to \n",
    "the output of the Text Embedding model, and then fine-tuning the entire model on \n",
    "[SQuAD2.0](https://rajpurkar.github.io/SQuAD-explorer/) dataset. \n",
    "The SQuAD2.0 dataset comprises pairs of question-context, and the answer of the \n",
    "question in the context indicated by the starting and ending character position in the context.\n",
    "\n",
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d702f27b",
   "metadata": {},
   "source": [
    "### 3.1. Retrieve JumpStart Artifacts & Deploy an Endpoint\n",
    "***\n",
    "We retrieve the deploy_image_uri, deploy_source_uri, and base_model_uri for the pre-trained model. To host the pre-trained model, we create an instance of [`sagemaker.model.Model`](https://sagemaker.readthedocs.io/en/stable/api/inference/model.html) and deploy it.\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b145f1b0",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sagemaker import image_uris, model_uris, script_uris\n",
    "from sagemaker.model import Model\n",
    "from sagemaker.predictor import Predictor\n",
    "from sagemaker.utils import name_from_base\n",
    "\n",
    "# model_version=\"*\" fetches the latest version of the model.\n",
    "infer_model_id, infer_model_version = dropdown.value, \"*\"\n",
    "\n",
    "endpoint_name = name_from_base(f\"jumpstart-{infer_model_id}\")\n",
    "\n",
    "inference_instance_type = \"ml.m5.xlarge\"\n",
    "\n",
    "# Retrieve the inference docker container uri.\n",
    "deploy_image_uri = image_uris.retrieve(\n",
    "    region=None,\n",
    "    framework=None,\n",
    "    image_scope=\"inference\",\n",
    "    model_id=infer_model_id,\n",
    "    model_version=infer_model_version,\n",
    "    instance_type=inference_instance_type,\n",
    ")\n",
    "# Retrieve the inference script uri.\n",
    "deploy_source_uri = script_uris.retrieve(\n",
    "    model_id=infer_model_id, model_version=infer_model_version, script_scope=\"inference\"\n",
    ")\n",
    "# Retrieve the base model uri.\n",
    "base_model_uri = model_uris.retrieve(\n",
    "    model_id=infer_model_id, model_version=infer_model_version, model_scope=\"inference\"\n",
    ")\n",
    "# Create the SageMaker model instance. Note that we need to pass Predictor class when we deploy model through Model class,\n",
    "# for being able to run inference through the sagemaker API.\n",
    "model = Model(\n",
    "    image_uri=deploy_image_uri,\n",
    "    source_dir=deploy_source_uri,\n",
    "    model_data=base_model_uri,\n",
    "    entry_point=\"inference.py\",\n",
    "    role=aws_role,\n",
    "    predictor_cls=Predictor,\n",
    "    name=endpoint_name,\n",
    ")\n",
    "# deploy the Model.\n",
    "base_model_predictor = model.deploy(\n",
    "    initial_instance_count=1,\n",
    "    instance_type=inference_instance_type,\n",
    "    endpoint_name=endpoint_name,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6123b640",
   "metadata": {},
   "source": [
    "### 3.2. Example question-context pair for inference\n",
    "***\n",
    "Let's put in some example question-contexts pairs. You can put in any question-context pairs, the model will predict a part of context that contains the answer.\n",
    "These examples are taken from SQuAD2.0 dataset downloaded from [Dataset Homepage](https://rajpurkar.github.io/SQuAD-explorer/). [CC BY-SA 4.0 License](https://creativecommons.org/licenses/by-sa/4.0/legalcode).\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ec54339e",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "question_context1 = [\n",
    "    \"What is Southern California often abbreviated as?\",\n",
    "    \"Southern California, often abbreviated SoCal, is a geographic and cultural region that generally comprises California's southernmost 10 counties. The region is traditionally described as \"\n",
    "    \"eight counties\"\n",
    "    \", based on demographics and economic ties: Imperial, Los Angeles, Orange, Riverside, San Bernardino, San Diego, Santa Barbara, and Ventura. The more extensive 10-county definition, including Kern and San Luis Obispo counties, is also used based on historical political divisions. Southern California is a major economic center for the state of California and the United States.\",\n",
    "]\n",
    "question_context2 = [\n",
    "    \"Who directed Spectre?\",\n",
    "    \"Spectre (2015) is the twenty-fourth James Bond film produced by Eon Productions. It features Daniel Craig in his fourth performance as James Bond, and Christoph Waltz as Ernst Stavro Blofeld, with the film marking the character's re-introduction into the series. It was directed by Sam Mendes as his second James Bond film following Skyfall, and was written by John Logan, Neal Purvis, Robert Wade and Jez Butterworth. It is distributed by Metro-Goldwyn-Mayer and Columbia Pictures. With a budget around $245 million, it is the most expensive Bond film and one of the most expensive films ever made.\",\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43fbf512",
   "metadata": {},
   "source": [
    "### 3.3. Query endpoint and parse response\n",
    "***\n",
    "Input to the endpoint is a question-context pair. Response from the endpoint is the answer extracted from the context for the input question.\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "71fcb262",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "newline, bold, unbold = \"\\n\", \"\\033[1m\", \"\\033[0m\"\n",
    "\n",
    "\n",
    "def query_endpoint(encoded_text):\n",
    "    response = base_model_predictor.predict(\n",
    "        encoded_text, {\"ContentType\": \"application/list-text\", \"Accept\": \"application/json;verbose\"}\n",
    "    )\n",
    "    return response\n",
    "\n",
    "\n",
    "def parse_response(query_response):\n",
    "    model_predictions = json.loads(query_response)\n",
    "    answer = (model_predictions[\"answer\"],)\n",
    "    return answer\n",
    "\n",
    "\n",
    "for question_context in [question_context1, question_context2]:\n",
    "    query_response = query_endpoint(json.dumps(question_context).encode(\"utf-8\"))\n",
    "    answer = parse_response(query_response)\n",
    "    print(\n",
    "        f\"Inference:{newline}\"\n",
    "        f\"Question: {bold}{question_context[0]}{unbold}{newline}\"\n",
    "        f\"Context: {question_context[1]}{newline}\"\n",
    "        f\"model answer: {bold}{answer}{unbold}{newline}\"\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef7152ce",
   "metadata": {},
   "source": [
    "### 3.4. Clean up the endpoint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "574cda32",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Delete the SageMaker endpoint and the attached resources\n",
    "base_model_predictor.delete_model()\n",
    "base_model_predictor.delete_endpoint()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2669ceaa",
   "metadata": {},
   "source": [
    "## 4. Finetune the pre-trained model on a custom dataset\n",
    "***\n",
    "Previously, we saw how to run inference on a pre-trained model, which was fine-tuned on SQuADv2 dataset. Next, we discuss how a model can be finetuned to a custom dataset. \n",
    "\n",
    "The Text Embedding model can be fine-tuned on any extractive question \n",
    "answering dataset in the same way the model available for inference has been \n",
    "fine-tuned on the SQuAD2.0 dataset.\n",
    "The model available for fine-tuning attaches an answer extracting layer to the Text Embedding model\n",
    "and initializes the layer parameters to random values. The fine-tuning step fine-tunes \n",
    "all the model parameters to minimize prediction error on the input data and returns the fine-tuned model.\n",
    "The model returned by fine-tuning can be further deployed for inference. Below are the instructions \n",
    "for how the training data should be formatted for input to the model. \n",
    "\n",
    "- **Input:**  A directory containing a 'data.csv' file.\n",
    "    - The first column of the 'data.csv' should have a question.\n",
    "    - The second column should have the corresponding context.\n",
    "    - The third column should have the integer character starting position for the answer in the context.\n",
    "    - The fourth column should have the integer character ending position for the answer in the context.\n",
    "- **Output:** A trained model that can be deployed for inference. \n",
    " \n",
    "Below is an example of 'data.csv' file showing values in its first four columns. Note that the file should not have any header.\n",
    "\n",
    "|   |  |  |   |\n",
    "|---|---|---|---|\n",
    "|In what country is Normandy located?|\tThe Normans (Norman: Nourmands; French: Normands; Latin: Normanni) were the people who in the 10th and 11th centuries gave their name to Normandy, a region in France. They were descended from Norse (\"Norman\" comes from \"Norseman\") raiders and pirates from Denmark, Iceland and Norway who, under their leader Rollo, agreed to swear fealty to King Charles III of West Francia. Through generations of assimilation and mixing with the native Frankish and Roman-Gaulish populations, their descendants would gradually merge with the Carolingian-based cultures of West Francia. The distinct cultural and ethnic identity of the Normans emerged initially in the first half of the 10th century, and it continued to evolve over the succeeding centuries.|\t159|\t165\n",
    "|...   | ... |...  | ... |\n",
    " \n",
    "\n",
    "SQuAD2.0 dataset is downloaded from \n",
    "[Dataset Homepage](https://rajpurkar.github.io/SQuAD-explorer/). \n",
    "[CC BY-SA 4.0 License](https://creativecommons.org/licenses/by-sa/4.0/legalcode).\n",
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "33e15ff8",
   "metadata": {},
   "source": [
    "### 4.1. Retrieve JumpStart Training artifacts\n",
    "***\n",
    "Here, for the selected model, we retrieve the training docker container, the training algorithm source, the pre-trained model, and a python dictionary of the training hyper-parameters that the algorithm accepts with their default values. Note that the model_version=\"*\" fetches the latest model. Also, we do need to specify the training_instance_type to fetch train_image_uri.\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "98d99c15",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sagemaker import image_uris, model_uris, script_uris, hyperparameters\n",
    "\n",
    "model_id, model_version = dropdown.value, \"*\"\n",
    "training_instance_type = \"ml.p3.2xlarge\"\n",
    "\n",
    "# Retrieve the docker image\n",
    "train_image_uri = image_uris.retrieve(\n",
    "    region=None,\n",
    "    framework=None,\n",
    "    model_id=model_id,\n",
    "    model_version=model_version,\n",
    "    image_scope=\"training\",\n",
    "    instance_type=training_instance_type,\n",
    ")\n",
    "# Retrieve the training script\n",
    "train_source_uri = script_uris.retrieve(\n",
    "    model_id=model_id, model_version=model_version, script_scope=\"training\"\n",
    ")\n",
    "# Retrieve the pre-trained model tarball to further fine-tune\n",
    "train_model_uri = model_uris.retrieve(\n",
    "    model_id=model_id, model_version=model_version, model_scope=\"training\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "df34d311",
   "metadata": {},
   "source": [
    "### 4.2. Set Training parameters\n",
    "***\n",
    "Now that we are done with all the setup that is needed, we are ready to fine-tune our Sentence Pair Classification model. To begin, let us create a [``sageMaker.estimator.Estimator``](https://sagemaker.readthedocs.io/en/stable/api/training/estimators.html) object. This estimator will launch the training job. \n",
    "\n",
    "There are two kinds of parameters that need to be set for training. \n",
    "\n",
    "The first one are the parameters for the training job. These include: (i) Training data path. This is S3 folder in which the input data is stored, (ii) Output path: This the s3 folder in which the training output is stored. (iii) Training instance type: This indicates the type of machine on which to run the training. Typically, we use GPU instances for these training. We defined the training instance type above to fetch the correct train_image_uri. \n",
    "\n",
    "The second set of parameters are algorithm specific training hyper-parameters.\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "766ce9ac",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Sample training data is available in this bucket\n",
    "training_data_bucket = f\"jumpstart-cache-prod-{aws_region}\"\n",
    "# For a quick demonstration of training we have created a random subset of SQuAD-v2 dataset.\n",
    "# For complete QNLI dataset replace \"SQuAD-v2-tiny\" with \"SQuAD-v2\" in the line below.\n",
    "training_data_prefix = \"training-datasets/SQuAD-v2-tiny/\"\n",
    "\n",
    "training_dataset_s3_path = f\"s3://{training_data_bucket}/{training_data_prefix}\"\n",
    "\n",
    "output_bucket = sess.default_bucket()\n",
    "output_prefix = \"jumpstart-eqa-training\"\n",
    "\n",
    "s3_output_location = f\"s3://{output_bucket}/{output_prefix}/output\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c0a1362",
   "metadata": {},
   "source": [
    "***\n",
    "For algorithm specific hyper-parameters, we start by fetching python dictionary of the training hyper-parameters that the algorithm accepts with their default values. This can then be overridden to custom values.\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b1cf1c28",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sagemaker import hyperparameters\n",
    "\n",
    "# Retrieve the default hyper-parameters for fine-tuning the model\n",
    "hyperparameters = hyperparameters.retrieve_default(model_id=model_id, model_version=model_version)\n",
    "\n",
    "# [Optional] Override default hyperparameters with custom values\n",
    "hyperparameters[\"batch-size\"] = \"16\"\n",
    "print(hyperparameters)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b22bbe2",
   "metadata": {},
   "source": [
    "### 4.3. Train with Automatic Model Tuning ([HPO](https://docs.aws.amazon.com/sagemaker/latest/dg/automatic-model-tuning.html)) <a id='AMT'></a>\n",
    "***\n",
    "Amazon SageMaker automatic model tuning, also known as hyperparameter tuning, finds the best version of a model by running many training jobs on your dataset using the algorithm and ranges of hyperparameters that you specify. It then chooses the hyperparameter values that result in a model that performs the best, as measured by a metric that you choose. We will use a [HyperparameterTuner](https://sagemaker.readthedocs.io/en/stable/api/training/tuner.html) object to interact with Amazon SageMaker hyperparameter tuning APIs.\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7051f557",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sagemaker.tuner import ContinuousParameter\n",
    "\n",
    "# Use AMT for tuning and selecting the best model\n",
    "use_amt = True\n",
    "\n",
    "# Define objective metric per framework, based on which the best model will be selected.\n",
    "metric_definitions_per_model = {\n",
    "    \"huggingface\": {\n",
    "        \"metrics\": [{\"Name\": \"val_loss\", \"Regex\": \"'eval_loss': ([0-9]+(.|e\\-)[0-9]+),?\"}],\n",
    "        \"type\": \"Minimize\",\n",
    "    }\n",
    "}\n",
    "\n",
    "# You can select from the hyperparameters supported by the model, and configure ranges of values to be searched for training the optimal model.(https://docs.aws.amazon.com/sagemaker/latest/dg/automatic-model-tuning-define-ranges.html)\n",
    "hyperparameter_ranges = {\n",
    "    \"adam-learning-rate\": ContinuousParameter(0.00001, 0.01, scaling_type=\"Logarithmic\")\n",
    "}\n",
    "\n",
    "# Increase the total number of training jobs run by AMT, for increased accuracy (and training time).\n",
    "max_jobs = 6\n",
    "# Change parallel training jobs run by AMT to reduce total training time, constrained by your account limits.\n",
    "# if max_jobs=max_parallel_jobs then Bayesian search turns to Random.\n",
    "max_parallel_jobs = 2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "abe698c9",
   "metadata": {},
   "source": [
    "### 4.4. Start Training\n",
    "***\n",
    "We start by creating the estimator object with all the required assets and then launch the training job.\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2c52727b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sagemaker.estimator import Estimator\n",
    "from sagemaker.utils import name_from_base\n",
    "from sagemaker.tuner import HyperparameterTuner\n",
    "\n",
    "training_job_name = name_from_base(f\"jumpstart-{model_id}-transfer-learning\")\n",
    "\n",
    "# Create SageMaker Estimator instance\n",
    "eqa_estimator = Estimator(\n",
    "    role=aws_role,\n",
    "    image_uri=train_image_uri,\n",
    "    source_dir=train_source_uri,\n",
    "    model_uri=train_model_uri,\n",
    "    entry_point=\"transfer_learning.py\",\n",
    "    instance_count=1,\n",
    "    instance_type=training_instance_type,\n",
    "    max_run=360000,\n",
    "    hyperparameters=hyperparameters,\n",
    "    output_path=s3_output_location,\n",
    "    base_job_name=training_job_name,\n",
    ")\n",
    "\n",
    "if use_amt:\n",
    "    metric_definitions = next(\n",
    "        value for key, value in metric_definitions_per_model.items() if model_id.startswith(key)\n",
    "    )\n",
    "\n",
    "    hp_tuner = HyperparameterTuner(\n",
    "        eqa_estimator,\n",
    "        metric_definitions[\"metrics\"][0][\"Name\"],\n",
    "        hyperparameter_ranges,\n",
    "        metric_definitions[\"metrics\"],\n",
    "        max_jobs=max_jobs,\n",
    "        max_parallel_jobs=max_parallel_jobs,\n",
    "        objective_type=metric_definitions[\"type\"],\n",
    "        base_tuning_job_name=training_job_name,\n",
    "    )\n",
    "\n",
    "    # Launch a SageMaker Tuning job to search for the best hyperparameters\n",
    "    hp_tuner.fit({\"training\": training_dataset_s3_path})\n",
    "else:\n",
    "    # Launch a SageMaker Training job by passing s3 path of the training data\n",
    "    eqa_estimator.fit({\"training\": training_dataset_s3_path}, logs=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "df341feb",
   "metadata": {},
   "source": [
    "## 4.5. Deploy & run Inference on the fine-tuned model\n",
    "***\n",
    "A trained model does nothing on its own. We now want to use the model to perform inference. For this example, that means predicting the class label of an input sentence. We follow the same steps as in [3. Run inference on the pre-trained model](#3.-Run-inference-on-the-pre-trained-model). We start by retrieving the jumpstart artifacts for deploying an endpoint. However, instead of base_predictor, we  deploy the `eqa_estimator` that we fine-tuned.\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "139cb092",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "inference_instance_type = \"ml.m5.xlarge\"\n",
    "\n",
    "# Retrieve the inference docker container uri\n",
    "deploy_image_uri = image_uris.retrieve(\n",
    "    region=None,\n",
    "    framework=None,\n",
    "    image_scope=\"inference\",\n",
    "    model_id=model_id,\n",
    "    model_version=model_version,\n",
    "    instance_type=inference_instance_type,\n",
    ")\n",
    "# Retrieve the inference script uri\n",
    "deploy_source_uri = script_uris.retrieve(\n",
    "    model_id=model_id, model_version=model_version, script_scope=\"inference\"\n",
    ")\n",
    "\n",
    "endpoint_name = name_from_base(f\"jumpstart-FT-{model_id}-\")\n",
    "\n",
    "# Use the estimator from the previous step to deploy to a SageMaker endpoint\n",
    "finetuned_predictor = (hp_tuner if use_amt else eqa_estimator).deploy(\n",
    "    initial_instance_count=1,\n",
    "    instance_type=inference_instance_type,\n",
    "    entry_point=\"inference.py\",\n",
    "    image_uri=deploy_image_uri,\n",
    "    source_dir=deploy_source_uri,\n",
    "    endpoint_name=endpoint_name,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26dacfc0",
   "metadata": {},
   "source": [
    "---\n",
    "Let's put in some example question-contexts pairs. You can put in any question-context pairs, the model will predict a part of context that contains the answer.\n",
    "These examples are taken from SQuAD2.0 dataset downloaded from [Dataset Homepage](https://rajpurkar.github.io/SQuAD-explorer/). [CC BY-SA 4.0 License](https://creativecommons.org/licenses/by-sa/4.0/legalcode).\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eb7b9335",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "question_context1 = [\n",
    "    \"What is Southern California often abbreviated as?\",\n",
    "    \"Southern California, often abbreviated SoCal, is a geographic and cultural region that generally comprises California's southernmost 10 counties. The region is traditionally described as \"\n",
    "    \"eight counties\"\n",
    "    \", based on demographics and economic ties: Imperial, Los Angeles, Orange, Riverside, San Bernardino, San Diego, Santa Barbara, and Ventura. The more extensive 10-county definition, including Kern and San Luis Obispo counties, is also used based on historical political divisions. Southern California is a major economic center for the state of California and the United States.\",\n",
    "]\n",
    "question_context2 = [\n",
    "    \"Who directed Spectre?\",\n",
    "    \"Spectre (2015) is the twenty-fourth James Bond film produced by Eon Productions. It features Daniel Craig in his fourth performance as James Bond, and Christoph Waltz as Ernst Stavro Blofeld, with the film marking the character's re-introduction into the series. It was directed by Sam Mendes as his second James Bond film following Skyfall, and was written by John Logan, Neal Purvis, Robert Wade and Jez Butterworth. It is distributed by Metro-Goldwyn-Mayer and Columbia Pictures. With a budget around $245 million, it is the most expensive Bond film and one of the most expensive films ever made.\",\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06a108bf",
   "metadata": {},
   "source": [
    "---\n",
    "Next, we query the finetuned model, parse the response and print the predictions.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "feabba0a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "newline, bold, unbold = \"\\n\", \"\\033[1m\", \"\\033[0m\"\n",
    "\n",
    "\n",
    "def query_endpoint(encoded_text):\n",
    "    response = finetuned_predictor.predict(\n",
    "        encoded_text, {\"ContentType\": \"application/list-text\", \"Accept\": \"application/json;verbose\"}\n",
    "    )\n",
    "    return response\n",
    "\n",
    "\n",
    "def parse_response(query_response):\n",
    "    model_predictions = json.loads(query_response)\n",
    "    answer = (model_predictions[\"answer\"],)\n",
    "    return answer\n",
    "\n",
    "\n",
    "for question_context in [question_context1, question_context2]:\n",
    "    query_response = query_endpoint(json.dumps(question_context).encode(\"utf-8\"))\n",
    "    answer = parse_response(query_response)\n",
    "    print(\n",
    "        f\"Inference:{newline}\"\n",
    "        f\"Question: {bold}{question_context[0]}{unbold}{newline}\"\n",
    "        f\"Context: {question_context[1]}{newline}\"\n",
    "        f\"model answer: {bold}{answer}{unbold}{newline}\"\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1c4c43d",
   "metadata": {},
   "source": [
    "---\n",
    "Next, we clean up the deployed endpoint.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d8637243",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Delete the SageMaker endpoint and the attached resources\n",
    "finetuned_predictor.delete_model()\n",
    "finetuned_predictor.delete_endpoint()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4826a6df",
   "metadata": {},
   "source": [
    "## Notebook CI Test Results\n",
    "\n",
    "This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.\n",
    "\n",
    "![This us-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-east-1/introduction_to_amazon_algorithms|jumpstart_question_answering|Amazon_JumpStart_Question_Answering.ipynb)\n",
    "\n",
    "![This us-east-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-east-2/introduction_to_amazon_algorithms|jumpstart_question_answering|Amazon_JumpStart_Question_Answering.ipynb)\n",
    "\n",
    "![This us-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-west-1/introduction_to_amazon_algorithms|jumpstart_question_answering|Amazon_JumpStart_Question_Answering.ipynb)\n",
    "\n",
    "![This ca-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ca-central-1/introduction_to_amazon_algorithms|jumpstart_question_answering|Amazon_JumpStart_Question_Answering.ipynb)\n",
    "\n",
    "![This sa-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/sa-east-1/introduction_to_amazon_algorithms|jumpstart_question_answering|Amazon_JumpStart_Question_Answering.ipynb)\n",
    "\n",
    "![This eu-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-1/introduction_to_amazon_algorithms|jumpstart_question_answering|Amazon_JumpStart_Question_Answering.ipynb)\n",
    "\n",
    "![This eu-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-2/introduction_to_amazon_algorithms|jumpstart_question_answering|Amazon_JumpStart_Question_Answering.ipynb)\n",
    "\n",
    "![This eu-west-3 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-3/introduction_to_amazon_algorithms|jumpstart_question_answering|Amazon_JumpStart_Question_Answering.ipynb)\n",
    "\n",
    "![This eu-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-central-1/introduction_to_amazon_algorithms|jumpstart_question_answering|Amazon_JumpStart_Question_Answering.ipynb)\n",
    "\n",
    "![This eu-north-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-north-1/introduction_to_amazon_algorithms|jumpstart_question_answering|Amazon_JumpStart_Question_Answering.ipynb)\n",
    "\n",
    "![This ap-southeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-southeast-1/introduction_to_amazon_algorithms|jumpstart_question_answering|Amazon_JumpStart_Question_Answering.ipynb)\n",
    "\n",
    "![This ap-southeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-southeast-2/introduction_to_amazon_algorithms|jumpstart_question_answering|Amazon_JumpStart_Question_Answering.ipynb)\n",
    "\n",
    "![This ap-northeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-northeast-1/introduction_to_amazon_algorithms|jumpstart_question_answering|Amazon_JumpStart_Question_Answering.ipynb)\n",
    "\n",
    "![This ap-northeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-northeast-2/introduction_to_amazon_algorithms|jumpstart_question_answering|Amazon_JumpStart_Question_Answering.ipynb)\n",
    "\n",
    "![This ap-south-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-south-1/introduction_to_amazon_algorithms|jumpstart_question_answering|Amazon_JumpStart_Question_Answering.ipynb)\n"
   ]
  }
 ],
 "metadata": {
  "availableInstances": [
   {
    "_defaultOrder": 0,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.t3.medium",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 1,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.t3.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 2,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.t3.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 3,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.t3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 4,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 5,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 6,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 7,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 8,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 9,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 10,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 11,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 12,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5d.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 13,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5d.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 14,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5d.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 15,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5d.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 16,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5d.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 17,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5d.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 18,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5d.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 19,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 20,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": true,
    "memoryGiB": 0,
    "name": "ml.geospatial.interactive",
    "supportedImageNames": [
     "sagemaker-geospatial-v1-0"
    ],
    "vcpuNum": 0
   },
   {
    "_defaultOrder": 21,
    "_isFastLaunch": true,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.c5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 22,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.c5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 23,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.c5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 24,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.c5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 25,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 72,
    "name": "ml.c5.9xlarge",
    "vcpuNum": 36
   },
   {
    "_defaultOrder": 26,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 96,
    "name": "ml.c5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 27,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 144,
    "name": "ml.c5.18xlarge",
    "vcpuNum": 72
   },
   {
    "_defaultOrder": 28,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.c5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 29,
    "_isFastLaunch": true,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g4dn.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 30,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g4dn.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 31,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g4dn.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 32,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g4dn.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 33,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g4dn.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 34,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g4dn.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 35,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 61,
    "name": "ml.p3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 36,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 244,
    "name": "ml.p3.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 37,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 488,
    "name": "ml.p3.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 38,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.p3dn.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 39,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.r5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 40,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.r5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 41,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.r5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 42,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.r5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 43,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.r5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 44,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.r5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 45,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.r5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 46,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.r5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 47,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 48,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 49,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 50,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 51,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 52,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 53,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.g5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 54,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.g5.48xlarge",
    "vcpuNum": 192
   },
   {
    "_defaultOrder": 55,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 56,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4de.24xlarge",
    "vcpuNum": 96
   }
  ],
  "kernelspec": {
   "display_name": "Python 3 (Data Science 3.0)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-west-2:236514542706:image/sagemaker-data-science-310-v1"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}