frameworks/pytorch/get_started_mnist_deploy.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Deploy a Trained PyTorch Model\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook. \n",
    "\n",
    "![This us-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-west-2/frameworks|pytorch|get_started_mnist_deploy.ipynb)\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "In this notebook, we walk through the process of deploying a trained model to a SageMaker endpoint. If you recently ran [the notebook for training](get_started_mnist_deploy.ipynb) with `%store%` magic, the `model_data` can be restored. Otherwise, we retrieve the \n",
    "model artifact from a public S3 bucket."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# setups\n",
    "\n",
    "import os\n",
    "import json\n",
    "\n",
    "import boto3\n",
    "import sagemaker\n",
    "from sagemaker.pytorch import PyTorchModel\n",
    "from sagemaker import get_execution_role, Session\n",
    "\n",
    "sess = Session()\n",
    "\n",
    "role = get_execution_role()\n",
    "\n",
    "%store -r pt_mnist_model_data\n",
    "\n",
    "try:\n",
    "    pt_mnist_model_data\n",
    "except NameError:\n",
    "    import json\n",
    "\n",
    "    # copy a pretrained model from a public public to your default bucket\n",
    "    s3 = boto3.client(\"s3\")\n",
    "    bucket = f\"sagemaker-example-files-prod-{sess.boto_region_name}\"\n",
    "    key = \"datasets/image/MNIST/model/pytorch-training-2020-11-21-22-02-56-203/model.tar.gz\"\n",
    "    s3.download_file(bucket, key, \"model.tar.gz\")\n",
    "\n",
    "    # upload to default bucket\n",
    "    pt_mnist_model_data = sess.upload_data(\n",
    "        path=\"model.tar.gz\", bucket=sess.default_bucket(), key_prefix=\"model/pytorch\"\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(pt_mnist_model_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## PyTorch Model Object\n",
    "\n",
    "The `PyTorchModel` class allows you to define an environment for making inference using your\n",
    "model artifact. Like the `PyTorch` class discussed \n",
    "[in this notebook for training an PyTorch model](get_started_mnist_train.ipynb), it is a high level API used to set up a docker image for your model hosting service.\n",
    "\n",
    "Once it is properly configured, it can be used to create a SageMaker\n",
    "endpoint on an EC2 instance. The SageMaker endpoint is a containerized environment that uses your trained model \n",
    "to make inference on incoming data via RESTful API calls. \n",
    "\n",
    "Some common parameters used to initiate the `PyTorchModel` class are:\n",
    "- `entry_point`: A user defined python file to be used by the inference image as handlers of incoming requests\n",
    "- `source_dir`: The directory of the `entry_point`\n",
    "- `role`: An IAM role to make AWS service requests\n",
    "- `model_data`: the S3 location of the compressed model artifact. It can be a path to a local file if the endpoint \n",
    "is to be deployed on the SageMaker instance you are using to run this notebook (local mode)\n",
    "- `framework_version`: version of the PyTorch package to be used\n",
    "- `py_version`: python version to be used\n",
    "\n",
    "We elaborate on the `entry_point` below.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "model = PyTorchModel(\n",
    "    entry_point=\"inference.py\",\n",
    "    source_dir=\"code\",\n",
    "    role=role,\n",
    "    model_data=pt_mnist_model_data,\n",
    "    framework_version=\"1.5.0\",\n",
    "    py_version=\"py3\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Entry Point for the Inference Image\n",
    "\n",
    "Your model artifacts pointed by `model_data` is pulled by the `PyTorchModel` and it is decompressed and saved in\n",
    "in the docker image it defines. They become regular model checkpoint files that you would produce outside SageMaker. This means in order to use your trained model for serving, \n",
    "you need to tell `PyTorchModel` class how to a recover a PyTorch model from the static checkpoint.\n",
    "\n",
    "Also, the deployed endpoint interacts with RESTful API calls, you need to tell it how to parse an incoming \n",
    "request to your model. \n",
    "\n",
    "These two instructions needs to be defined as two functions in the python file pointed by `entry_point`.\n",
    "\n",
    "By convention, we name this entry point file `inference.py` and we put it in the `code` directory.\n",
    "\n",
    "To tell the inference image how to load the model checkpoint, you need to implement a function called \n",
    "`model_fn`. This function takes one positional argument \n",
    "\n",
    "- `model_dir`: the directory of the static model checkpoints in the inference image.\n",
    "\n",
    "The return of `model_fn` is a PyTorch model. In this example, the `model_fn`\n",
    "looks like:\n",
    "\n",
    "```python\n",
    "def model_fn(model_dir): \n",
    "    model = Net()   \n",
    "    with open(os.path.join(model_dir, \"model.pth\"), \"rb\") as f:\n",
    "        model.load_state_dict(torch.load(f))\n",
    "    model.to(device).eval()\n",
    "    return model\n",
    "```\n",
    "\n",
    "Next, you need to tell the hosting service how to handle the incoming data. This includes:\n",
    "\n",
    "* How to parse the incoming request\n",
    "* How to use the trained model to make inference\n",
    "* How to return the prediction to the caller of the service\n",
    "\n",
    "\n",
    "You do it by implementing 3 functions:\n",
    "\n",
    "#### `input_fn` function\n",
    "\n",
    "The SageMaker PyTorch model server will invoke the `input_fn` function in your inference entry point. This function handles data decoding. The `input_fn` have the following signature:\n",
    "```python\n",
    "def input_fn(request_body, request_content_type)\n",
    "```\n",
    "The two positional arguments are:\n",
    "- `request_body`: the payload of the incoming request\n",
    "- `request_content_type`: the content type of the incoming request\n",
    "\n",
    "The return of `input_fn` is an object that can be passed to `predict_fn`\n",
    "\n",
    "In this example, the `input_fn` looks like:\n",
    "```python\n",
    "def input_fn(request_body, request_content_type):\n",
    "    assert request_content_type=='application/json'\n",
    "    data = json.loads(request_body)['inputs']\n",
    "    data = torch.tensor(data, dtype=torch.float32, device=device)\n",
    "    return data\n",
    "```\n",
    "It requires the request payload is encoded as a json string and\n",
    "it assumes the decoded payload contains a key `inputs`\n",
    "that maps to the input data to be consumed by the model.\n",
    "\n",
    "\n",
    "\n",
    "#### `predict_fn` \n",
    "After the inference request has been deserialized by `input_fn`, the SageMaker PyTorch model server invokes `predict_fn` on the return value of `input_fn`.\n",
    "\n",
    "The `predict_fn` function has the following signature:\n",
    "```python\n",
    "def predict_fn(input_object, model)\n",
    "```\n",
    "The two positional arguments are:\n",
    "- `input_object`: the return value from `input_fn`\n",
    "- `model`: the return value from `model_fn`\n",
    "\n",
    "The return of `predict_fn` is the first argument to be passed to `output_fn`\n",
    "\n",
    "In this example, the `predict_fn` function looks like\n",
    "\n",
    "```python\n",
    "def predict_fn(input_object, model):\n",
    "    with torch.no_grad():\n",
    "        prediction = model(input_object)\n",
    "    return prediction\n",
    "```\n",
    "\n",
    "Note that we directly feed the return of `input_fn` to `predict_fn`.\n",
    "This means you should invoke the SageMaker PyTorch model server with data that\n",
    "can be readily consumed by the model, i.e. normalized and has batch and channel dimension. \n",
    "\n",
    "\n",
    "#### `output_fn` \n",
    "After invoking `predict_fn`, the model server invokes `output_fn` for data post-process.\n",
    "The `output_fn` has the following signature:\n",
    "\n",
    "```python\n",
    "def output_fn(prediction, content_type)\n",
    "```\n",
    "\n",
    "The two positional arguments are:\n",
    "- `prediction`: the return value from `predict_fn`\n",
    "- `content_type`: the content type of the response\n",
    "\n",
    "The return of `output_fn` should be a byte array of data serialized to `content_type`.\n",
    "\n",
    "In this example, the `output_fn` function looks like\n",
    "\n",
    "```python\n",
    "def output_fn(predictions, content_type):\n",
    "    assert content_type == 'application/json'\n",
    "    res = predictions.cpu().numpy().tolist()\n",
    "    return json.dumps(res)\n",
    "```\n",
    "\n",
    "After the inference, the function uses `content_type` to encode the \n",
    "prediction into the content type of the response. In this example,\n",
    "the function requires the caller of the service to accept json string. \n",
    "\n",
    "For more info on handler functions, check the [SageMaker Python SDK document](https://sagemaker.readthedocs.io/en/stable/frameworks/pytorch/using_pytorch.html#process-model-output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Execute the inference container\n",
    "Once the `PyTorchModel` class is initiated, we can call its `deploy` method to run the container for the hosting\n",
    "service. Some common parameters needed to call `deploy` methods are:\n",
    "\n",
    "- `initial_instance_count`: the number of SageMaker instances to be used to run the hosting service.\n",
    "- `instance_type`: the type of SageMaker instance to run the hosting service. Set it to `local` if you want to run the hosting service on the local SageMaker instance. Local mode is typically used for debugging. \n",
    "- `serializer`: A python callable used to serialize (encode) the request data.\n",
    "- `deserializer`: A python callable used to deserialize (decode) the response data.\n",
    "\n",
    "Commonly used serializers and deserializers are implemented in `sagemaker.serializers` and `sagemaker.deserializers`\n",
    "submodules of the SageMaker Python SDK. \n",
    "\n",
    "Since in the `transform_fn` we declared that the incoming requests are json-encoded, we need to use a `json serializer`,\n",
    "to encode the incoming data into a json string. \n",
    "Also, we declared the return content type to be json string, we need to use a `json deserializer` to parse the response into an integer, in this case, representing the predicted hand-written digit. \n",
    "\n",
    "<span style=\"color:red\"> Note: local mode is not supported in SageMaker Studio </span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sagemaker.serializers import JSONSerializer\n",
    "from sagemaker.deserializers import JSONDeserializer\n",
    "\n",
    "# set local_mode to False if you want to deploy on a remote\n",
    "# SageMaker instance\n",
    "\n",
    "local_mode = False\n",
    "\n",
    "if local_mode:\n",
    "    instance_type = \"local\"\n",
    "else:\n",
    "    instance_type = \"ml.c4.xlarge\"\n",
    "\n",
    "predictor = model.deploy(\n",
    "    initial_instance_count=1,\n",
    "    instance_type=instance_type,\n",
    "    serializer=JSONSerializer(),\n",
    "    deserializer=JSONDeserializer(),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `predictor` we get above can be used to make prediction requests against a SageMaker endpoint. \n",
    "For more information, check [the API reference for SageMaker Predictor](\n",
    "https://sagemaker.readthedocs.io/en/stable/api/inference/predictors.html#sagemaker.predictor.predictor)\n",
    "\n",
    "Now, let's test the endpoint with some dummy data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import random\n",
    "import numpy as np\n",
    "\n",
    "dummy_data = {\"inputs\": np.random.rand(16, 1, 28, 28).tolist()}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In `transform_fn`, we declared that the parsed data is a python dictionary with a key `inputs` and its value should \n",
    "be a 1D array of length 784. Hence, the definition of `dummy_data`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "res = predictor.predict(dummy_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(\"Predictions:\", res)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If the input data does not look exactly like `dummy_data`, the endpoint will raise an exception. This is because \n",
    "of the stringent way we defined the `transform_fn`. Let's test the following example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "dummy_data = [random.random() for _ in range(784)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When the `dummy_data` is parsed in `transform_fn`, it does not have the `inputs` field, so `transform_fn` will crash. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# uncomment the following line to make inference on incorrectly formated input data\n",
    "# res = predictor.predict(dummy_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's use real MNIST test to test the endpoint. We use helper functions defined in `code.utils` to \n",
    "download MNIST data set and normalize the input data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from utils.mnist import mnist_to_numpy, normalize\n",
    "import random\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "%matplotlib inline\n",
    "\n",
    "data_dir = \"/tmp/data\"\n",
    "X, Y = mnist_to_numpy(data_dir, train=False)\n",
    "\n",
    "# randomly sample 16 images to inspect\n",
    "mask = random.sample(range(X.shape[0]), 16)\n",
    "samples = X[mask]\n",
    "labels = Y[mask]\n",
    "# plot the images\n",
    "fig, axs = plt.subplots(nrows=1, ncols=16, figsize=(16, 1))\n",
    "\n",
    "for i, splt in enumerate(axs):\n",
    "    splt.imshow(samples[i])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(samples.shape, samples.dtype)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we invoke the SageMaker PyTorch model server with `samples`, we need to do\n",
    "some pre-processing\n",
    "- convert its data type to 32 bit floating point\n",
    "- normalize each channel (only one channel for MNIST)\n",
    "- add a channel dimension"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "samples = normalize(samples.astype(np.float32), axis=(1, 2))\n",
    "\n",
    "res = predictor.predict({\"inputs\": np.expand_dims(samples, axis=1).tolist()})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The response is a list with probability vectors for each sample."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "predictions = np.argmax(np.array(res, dtype=np.float32), axis=1).tolist()\n",
    "print(\"Predicted digits: \", predictions)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test and debug the entry point before deployment\n",
    "\n",
    "When deploying a model to a SageMaker endpoint, it is a good practice to test the entry \n",
    "point. The following snippet shows you how you can test and debug the `model_fn` and \n",
    "`transform_fn` you implemented in the entry point for the inference image.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "!pygmentize code/test_inference.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `test` function simulates how the inference container works. It pulls the model\n",
    "artifact and loads the model into \n",
    "memory by calling `model_fn` and parse `model_dir` to it. \n",
    "When it receives a request,\n",
    "it calls `input_fn`, `predict_fn` and `output_fn` consecutively. \n",
    "\n",
    "Implementing such a test function helps you to debug the entry point before putting it into the production. \n",
    "If `test` runs correctly, then you can be certain that if the incoming data and its content type are what they are supposed to be, then the endpoint is going to work as expected. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (Optional) Clean up \n",
    "\n",
    "If you do not plan to use the endpoint, you should delete it to free up some computation \n",
    "resource. If you use local, you will need to manually delete the docker container bounded\n",
    "at port 8080 (the port that listens to the incoming request).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "if not local_mode:\n",
    "    predictor.delete_endpoint()\n",
    "else:\n",
    "    os.system(\"docker container ls | grep 8080 | awk '{print $1}' | xargs docker container rm -f\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Notebook CI Test Results\n",
    "\n",
    "This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.\n",
    "\n",
    "![This us-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-east-1/frameworks|pytorch|get_started_mnist_deploy.ipynb)\n",
    "\n",
    "![This us-east-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-east-2/frameworks|pytorch|get_started_mnist_deploy.ipynb)\n",
    "\n",
    "![This us-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-west-1/frameworks|pytorch|get_started_mnist_deploy.ipynb)\n",
    "\n",
    "![This ca-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ca-central-1/frameworks|pytorch|get_started_mnist_deploy.ipynb)\n",
    "\n",
    "![This sa-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/sa-east-1/frameworks|pytorch|get_started_mnist_deploy.ipynb)\n",
    "\n",
    "![This eu-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-1/frameworks|pytorch|get_started_mnist_deploy.ipynb)\n",
    "\n",
    "![This eu-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-2/frameworks|pytorch|get_started_mnist_deploy.ipynb)\n",
    "\n",
    "![This eu-west-3 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-3/frameworks|pytorch|get_started_mnist_deploy.ipynb)\n",
    "\n",
    "![This eu-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-central-1/frameworks|pytorch|get_started_mnist_deploy.ipynb)\n",
    "\n",
    "![This eu-north-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-north-1/frameworks|pytorch|get_started_mnist_deploy.ipynb)\n",
    "\n",
    "![This ap-southeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-southeast-1/frameworks|pytorch|get_started_mnist_deploy.ipynb)\n",
    "\n",
    "![This ap-southeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-southeast-2/frameworks|pytorch|get_started_mnist_deploy.ipynb)\n",
    "\n",
    "![This ap-northeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-northeast-1/frameworks|pytorch|get_started_mnist_deploy.ipynb)\n",
    "\n",
    "![This ap-northeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-northeast-2/frameworks|pytorch|get_started_mnist_deploy.ipynb)\n",
    "\n",
    "![This ap-south-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-south-1/frameworks|pytorch|get_started_mnist_deploy.ipynb)\n"
   ]
  }
 ],
 "metadata": {
  "availableInstances": [
   {
    "_defaultOrder": 0,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.t3.medium",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 1,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.t3.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 2,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.t3.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 3,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.t3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 4,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 5,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 6,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 7,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 8,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 9,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 10,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 11,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 12,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5d.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 13,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5d.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 14,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5d.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 15,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5d.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 16,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5d.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 17,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5d.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 18,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5d.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 19,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 20,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": true,
    "memoryGiB": 0,
    "name": "ml.geospatial.interactive",
    "supportedImageNames": [
     "sagemaker-geospatial-v1-0"
    ],
    "vcpuNum": 0
   },
   {
    "_defaultOrder": 21,
    "_isFastLaunch": true,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.c5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 22,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.c5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 23,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.c5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 24,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.c5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 25,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 72,
    "name": "ml.c5.9xlarge",
    "vcpuNum": 36
   },
   {
    "_defaultOrder": 26,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 96,
    "name": "ml.c5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 27,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 144,
    "name": "ml.c5.18xlarge",
    "vcpuNum": 72
   },
   {
    "_defaultOrder": 28,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.c5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 29,
    "_isFastLaunch": true,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g4dn.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 30,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g4dn.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 31,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g4dn.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 32,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g4dn.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 33,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g4dn.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 34,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g4dn.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 35,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 61,
    "name": "ml.p3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 36,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 244,
    "name": "ml.p3.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 37,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 488,
    "name": "ml.p3.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 38,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.p3dn.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 39,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.r5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 40,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.r5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 41,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.r5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 42,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.r5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 43,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.r5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 44,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.r5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 45,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.r5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 46,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.r5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 47,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 48,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 49,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 50,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 51,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 52,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 53,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.g5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 54,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.g5.48xlarge",
    "vcpuNum": 192
   },
   {
    "_defaultOrder": 55,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 56,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4de.24xlarge",
    "vcpuNum": 96
   }
  ],
  "kernelspec": {
   "display_name": "Python 3 (Data Science 3.0)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-west-2:236514542706:image/sagemaker-data-science-310-v1"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}