finserv-llama2-chatbot/finserv-llama2-aoss-chatbot.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c7d83863-427f-4585-87d7-f19ee4fc7470",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Contextual chatbot for financial services using Llama2 via SageMaker JumpStart and Amazon OpenSearch Serverless with Vector Engine"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "bd59e781-e006-4494-abfe-47050d0611e7",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Prerequisites\n",
    "\n",
    "You will need these prerequisites in order to build the following context aware chatbot:\n",
    "- An Amazon SageMaker Execution Role with IAM permission to access Amazon OpenSearch Serverless (aoss). The [in-line](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_manage-attach-detach.html#add-policies-console) policy can be found [here](./IAM/sagemaker-execution-role-aoss-policy.yml). You can determine the correct SageMaker Role by running the `setup sagemaker session` cell below.\n",
    "- An Amazon OpenSearch Serverless (aoss) Vector Engine. Setup instructions can be found [here](https://aws.amazon.com/blogs/big-data/introducing-the-vector-engine-for-amazon-opensearch-serverless-now-in-preview/).\n",
    "- Add the SageMaker Execution Role to the Data Access Policy for Amazon OpenSearch Serverless (aoss). Instructions can be found [here](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/serverless-data-access.html#serverless-data-access-console).\n",
    "- 2 x ml.g5.2xlarge instance types for SageMaker Endpoints. Instruction on increasing your service quota can be found [here](https://aws.amazon.com/getting-started/hands-on/request-service-quota-increase/)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "973d8c09-007d-4c00-86f6-6d26932d1d88",
   "metadata": {},
   "source": [
    "## Install Required Python Libraries"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b88c459e-6031-4bda-a882-905f8491ee68",
   "metadata": {
    "tags": []
   },
   "source": [
    "    **IMPORTANT**\n",
    "    1. Ensure you are running Pythin 3.10+\n",
    "    1. Ensure you are using the Data Science 3.0 kernel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b22ee7fd-963d-4afc-8722-ec0911c4f340",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "#### Require python 3.10+\n",
    "!python --version"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3476b4c-559b-4da1-b524-14ca011e04f9",
   "metadata": {},
   "source": [
    "Begin by installing the required python libraries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "48ae7b8b-0e21-4152-9538-e858b31ba271",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "%pip install -U pip --quiet\n",
    "%pip install --upgrade sagemaker --quiet \n",
    "%pip install langchain --quiet\n",
    "%pip install opensearch-py --quiet\n",
    "%pip install regex --quiet\n",
    "%pip install tqdm --quiet\n",
    "%pip install requests_aws4auth --quiet\n",
    "%pip install pypdf"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dcb06ec0-d912-4a51-8b2e-8c898dd4041d",
   "metadata": {},
   "source": [
    "## Setup Environment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b5c5c149-aa23-496e-b036-10ffc73e5615",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Setup SageMaker Session\n",
    "import sagemaker, boto3, json\n",
    "from sagemaker.session import Session\n",
    "from sagemaker.jumpstart.model import JumpStartModel\n",
    "from sagemaker import get_execution_role\n",
    "\n",
    "\n",
    "sagemaker_session = Session()\n",
    "aws_role = sagemaker_session.get_caller_identity_arn()\n",
    "sm_execution_role = get_execution_role()\n",
    "\n",
    "aws_region = boto3.Session().region_name\n",
    "sess = sagemaker.Session()\n",
    "\n",
    "sagemaker_client = boto3.client('sagemaker', region_name=aws_region)\n",
    "\n",
    "print(sm_execution_role)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2d3e92ed",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Import langchain \n",
    "from langchain.document_loaders import UnstructuredHTMLLoader,BSHTMLLoader,PyPDFLoader\n",
    "from langchain.text_splitter import RecursiveCharacterTextSplitter,CharacterTextSplitter\n",
    "from langchain.llms.sagemaker_endpoint import ContentHandlerBase\n",
    "from langchain.vectorstores import OpenSearchVectorSearch\n",
    "from langchain import LLMChain\n",
    "from langchain import SagemakerEndpoint\n",
    "from langchain.prompts import PromptTemplate\n",
    "from langchain.llms.sagemaker_endpoint import LLMContentHandler\n",
    "import os\n",
    "import json\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3cbf0e14-5253-466c-a7e2-deb5bc3cd085",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "llm_model = JumpStartModel(model_id = \"meta-textgeneration-llama-2-7b-f\")\n",
    "llm_predictor = llm_model.deploy()\n",
    "llm_endpoint_name=llm_predictor.endpoint_name\n",
    "llm_endpoint_name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e3db3429-1109-4033-a79b-51134c8285cc",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "embeddings_model = JumpStartModel(model_id = \"huggingface-textembedding-gpt-j-6b-fp16\")\n",
    "embed_predictor = embeddings_model.deploy()\n",
    "embeddings_endpoint_name = embed_predictor.endpoint_name\n",
    "embeddings_endpoint_name"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "60103805-e228-49cc-95d3-e802b0d451a7",
   "metadata": {},
   "source": [
    "#### Ensure you update the variable below to reflect your enviroment "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "af133b35-95fe-4ae1-bf09-7341188764c3",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Set variables for Amazon OpenSearch\n",
    "_aoss_host=\"xxxx.us-east-1.aoss.amazonaws.com\"\n",
    "_aoss_index='your-index'\n",
    "\n",
    "_aoss_host"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5166d759",
   "metadata": {},
   "source": [
    "## Chunk your Data and Load into Amazon OpenSearch\n",
    "\n",
    "In this section we will chunk the data into smaller documents. Chunking is a technique for splitting large texts into smaller chunks. It is an important step as it optimizes the relevance of the search query for our RAG-model. Which in turn improves the quality of the chatbot. The chunk size is dependent on factors such as the document type and model used. We have selected a `chunk_size=1600` as this is the approximate size of a paragraph. As models improve, their context window size will increase which will allow for larger chunk sizes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b4a2d6bf",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "loader = PyPDFLoader(\"./data/amazon-10k-2022.pdf\")\n",
    "data = loader.load()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b4fd7c9e",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print (f'You have {len(data)} document(s) in your data')\n",
    "print (f'There are {len(data[0].page_content)} characters in your document')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6315f009",
   "metadata": {},
   "source": [
    "Next, we use LangChain `RecursiveCharacterTextSplitter` class to to chunk the document. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fb3c6f02",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "text_splitter = RecursiveCharacterTextSplitter(chunk_size=1600, chunk_overlap=200)\n",
    "docs = text_splitter.split_documents(data)\n",
    "\n",
    "print (f'Now you have {len(docs)} documents')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d0b5954f-fc77-45c7-9cb9-b7fc2fe24390",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Helper function to process document\n",
    "\n",
    "import regex as re\n",
    "\n",
    "def postproc(s):\n",
    "    s = s.replace(u'\\xa0', u' ') # no-break space \n",
    "    s = s.replace('\\n', ' ') # new-line\n",
    "    s = re.sub(r'\\s+', ' ', s) # multiple spaces\n",
    "    return s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "725f1429-c9c3-49b1-a779-eda1c901232e",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "for doc in docs:\n",
    "    doc.page_content = postproc(doc.page_content)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "28da307a",
   "metadata": {},
   "source": [
    "In the next step, we simply validate that document was chunked correctly by manually reviewing the first chunk."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a10df400-95c0-41ba-a5cf-097af738d30a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Review the first document for correctness\n",
    "docs[10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4a0c0820-0080-4676-94a3-38594470ebd5",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Limit the number of total chunks to 1000\n",
    "MAX_DOCS = 1000\n",
    "if len(docs) > MAX_DOCS:\n",
    "    docs = docs[:MAX_DOCS]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "865a4079-5d48-4fa3-8bcc-9ba7a937102b",
   "metadata": {},
   "source": [
    "Next we extend the LangChain `SageMakerEndpointEmbeddings` Class to create a custom embeddings function that uses the `gpt-j-6b-fp16` SageMaker Endpoint you created earlier in this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "635f15ca-e6ec-4f4c-af9f-0a455cba1f8b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import time\n",
    "import json\n",
    "import logging\n",
    "from typing import List\n",
    "from langchain.embeddings import SagemakerEndpointEmbeddings\n",
    "from langchain.embeddings.sagemaker_endpoint import EmbeddingsContentHandler\n",
    "\n",
    "logger = logging.getLogger(__name__)\n",
    "\n",
    "# extend the SagemakerEndpointEmbeddings class from langchain to provide a custom embedding function\n",
    "class SagemakerEndpointEmbeddingsJumpStart(SagemakerEndpointEmbeddings):\n",
    "    def embed_documents(\n",
    "        self, texts: List[str], chunk_size: int = 1\n",
    "    ) -> List[List[float]]:\n",
    "        \"\"\"Compute doc embeddings using a SageMaker Inference Endpoint.\n",
    "\n",
    "        Args:\n",
    "            texts: The list of texts to embed.\n",
    "            chunk_size: The chunk size defines how many input texts will\n",
    "                be grouped together as request. If None, will use the\n",
    "                chunk size specified by the class.\n",
    "\n",
    "        Returns:\n",
    "            List of embeddings, one for each text.\n",
    "        \"\"\"\n",
    "        results = []\n",
    "        _chunk_size = len(texts) if chunk_size > len(texts) else chunk_size\n",
    "        st = time.time()\n",
    "        for i in range(0, len(texts), _chunk_size):\n",
    "            response = self._embedding_func(texts[i:i + _chunk_size])\n",
    "            results.extend(response)\n",
    "        time_taken = time.time() - st\n",
    "        logger.info(f\"got results for {len(texts)} in {time_taken}s, length of embeddings list is {len(results)}\")\n",
    "        print(f\"got results for {len(texts)} in {time_taken}s, length of embeddings list is {len(results)}\")\n",
    "        return results\n",
    "\n",
    "\n",
    "# class for serializing/deserializing requests/responses to/from the embeddings model\n",
    "class ContentHandler(EmbeddingsContentHandler):\n",
    "    content_type = \"application/json\"\n",
    "    accepts = \"application/json\"\n",
    "\n",
    "    def transform_input(self, prompt: str, model_kwargs={}) -> bytes:\n",
    "\n",
    "        input_str = json.dumps({\"text_inputs\": prompt, **model_kwargs})\n",
    "        return input_str.encode('utf-8') \n",
    "\n",
    "    def transform_output(self, output: bytes) -> str:\n",
    "\n",
    "        response_json = json.loads(output.read().decode(\"utf-8\"))\n",
    "        embeddings = response_json[\"embedding\"]\n",
    "        if len(embeddings) == 1:\n",
    "            return [embeddings[0]]\n",
    "        return embeddings\n",
    "    \n",
    "\n",
    "def create_sagemaker_embeddings_from_js_model(embeddings_endpoint_name: str, aws_region: str) -> SagemakerEndpointEmbeddingsJumpStart:\n",
    "    # all set to create the objects for the ContentHandler and \n",
    "    # SagemakerEndpointEmbeddingsJumpStart classes\n",
    "    content_handler = ContentHandler()\n",
    "\n",
    "    # note the name of the LLM Sagemaker endpoint, this is the model that we would\n",
    "    # be using for generating the embeddings\n",
    "    embeddings = SagemakerEndpointEmbeddingsJumpStart( \n",
    "        endpoint_name=embeddings_endpoint_name,\n",
    "        region_name=aws_region, \n",
    "        content_handler=content_handler\n",
    "    )\n",
    "    return embeddings"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "838b2843",
   "metadata": {},
   "source": [
    "We create the embeddings object and batch the creation of the document embeddings. These embeddinga are stored in Amazon OpenSearch using LangChain `OpenSearchVectorSearch`.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8ed872e9-387d-4d60-8567-4866ed365677",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "embeddings = create_sagemaker_embeddings_from_js_model(embeddings_endpoint_name, aws_region)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aa8899dc-b360-4db7-9139-04d2bb18872a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from tqdm import trange\n",
    "from requests_aws4auth import AWS4Auth\n",
    "from opensearchpy import RequestsHttpConnection\n",
    "\n",
    "credentials = boto3.Session().get_credentials()\n",
    "\n",
    "service=\"aoss\"\n",
    "region=aws_region\n",
    "\n",
    "awsauth = AWS4Auth(\n",
    "    credentials.access_key,\n",
    "    credentials.secret_key,\n",
    "    region,\n",
    "    service,\n",
    "    session_token=credentials.token\n",
    ")\n",
    "\n",
    "docsearch = OpenSearchVectorSearch.from_texts(\n",
    "    texts = [d.page_content for d in docs],\n",
    "    embedding=embeddings,\n",
    "    opensearch_url=[{'host': _aoss_host, 'port': 443}],\n",
    "    http_auth=awsauth,\n",
    "    timeout = 300,\n",
    "    use_ssl = True,\n",
    "    verify_certs = True,\n",
    "    connection_class = RequestsHttpConnection,\n",
    "    index_name=_aoss_index\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c35dcd9",
   "metadata": {},
   "source": [
    "## Question Answering Over Documents \n",
    "\n",
    "So far, we have chunked a large document into smaller ones, created vector embedding and stored them in an Amazon OpenSearch Serverless with Vector engine. Now, we can answer questions regarding this document data.\n",
    "\n",
    "Since we have created an index over the data, we can do a semantic search; this way only the most relevant documents required to answer the question are passed via the prompt to the Large Language Model (LLM). This allows you to save both time and money by not passing all the documents to the LLM.\n",
    "\n",
    "The LLM used is `Llama2` via the SageMaker Endpoint created earlier.\n",
    "\n",
    "We use LangChian **question_answering** `stuff` document chain type in this example. Further details on Document Chains can be found by visiting the [LangChain documentation, here](https://python.langchain.com/docs/modules/chains/document/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "610452c8-7b70-49c8-9f92-9a74c45be1bd",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from typing import Dict\n",
    "\n",
    "from langchain import PromptTemplate, SagemakerEndpoint\n",
    "from langchain.llms.sagemaker_endpoint import LLMContentHandler\n",
    "from langchain.chains.question_answering import load_qa_chain\n",
    "import json\n",
    "\n",
    "query = \"what is the name of the Registrant’s principal executive officer??\"\n",
    "# query =  \"Summarize the earnings report and also what year is the report for\"\n",
    "# query =  \"what was North America's net sale ?\"\n",
    "# query= \"summarize Risk factors section\"\n",
    "\n",
    "prompt_template = \"\"\"Use the following pieces of context to answer the question at the end.\n",
    "\n",
    "{context}\n",
    "\n",
    "Question: {question}\n",
    "Answer:\"\"\"\n",
    "\n",
    "prompt = PromptTemplate(\n",
    "    template=prompt_template, input_variables=[\"context\", \"question\"]\n",
    ")\n",
    "\n",
    "class ContentHandler(LLMContentHandler):\n",
    "    content_type = \"application/json\"\n",
    "    accepts = \"application/json\"\n",
    "\n",
    "    def transform_input(self, prompt: str, model_kwargs: dict) -> bytes:\n",
    "        payload = {\n",
    "            \"inputs\": [\n",
    "                [\n",
    "                    {\n",
    "                        \"role\": \"system\",\n",
    "                        \"content\": prompt,\n",
    "                    },\n",
    "                    {\"role\": \"user\", \"content\": prompt},\n",
    "                ],\n",
    "            ],\n",
    "            \"parameters\": {\n",
    "                \"max_new_tokens\": 1000,\n",
    "                \"top_p\": 0.9,\n",
    "                \"temperature\": 0.6,\n",
    "            },\n",
    "        }\n",
    "        input_str = json.dumps(\n",
    "            payload,\n",
    "        )\n",
    "        return input_str.encode(\"utf-8\")\n",
    "\n",
    "    def transform_output(self, output: bytes) -> str:\n",
    "        response_json = json.loads(output.read().decode(\"utf-8\"))\n",
    "        content = response_json[0][\"generation\"][\"content\"]\n",
    "        return content\n",
    "    \n",
    "content_handler = ContentHandler()\n",
    "\n",
    "chain = load_qa_chain(\n",
    "    llm=SagemakerEndpoint(\n",
    "        endpoint_name=llm_endpoint_name,\n",
    "        # credentials_profile_name=\"credentials-profile-name\",\n",
    "        region_name=aws_region,\n",
    "        model_kwargs={\"max_new_tokens\": 300},\n",
    "        endpoint_kwargs={\"CustomAttributes\": \"accept_eula=true\"},\n",
    "        content_handler=content_handler,\n",
    "    ),\n",
    "    prompt=prompt,\n",
    ")\n",
    "sim_docs = docsearch.similarity_search(query, include_metadata=False)\n",
    "# print(sim_docs)\n",
    "chain({\"input_documents\": sim_docs, \"question\": query}, return_only_outputs=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e218d0a-efeb-4d60-810c-536b29536e72",
   "metadata": {},
   "source": [
    "## Clean Up\n",
    "\n",
    "Delete the SageMaker Inference Endpoints that we created in this notebook to avoid incurring future costs. If you created an Amazon OpenSearch Serverless Collection for this example and no longer require it then delete it via the AWS Console."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f95c2ad5-4bd2-4014-a86c-218514aa216b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Delete LLM\n",
    "llm_predictor.delete_model()\n",
    "llm_predictor.delete_predictor(delete_endpoint_config=True)\n",
    "\n",
    "# Delete Embeddings Model\n",
    "embed_predictor.delete_model()\n",
    "embed_predictor.delete_predictor(delete_endpoint_config=True)\n",
    "\n",
    "# Delete OpenSearch"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30e2b81f-a0fc-4e2c-8f93-7930b0b6d25d",
   "metadata": {},
   "source": [
    "## Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee53d433-3bfb-46da-935d-9a7c3a8a3929",
   "metadata": {},
   "source": [
    "In this notebook, we used Retrieval Augmented Generation(RAG) as an optional approach to provide domain specific context to Large Language Models(LLM). We showed how to use SageMaker Jumpstart to easily build a RAG-based contextual chatbot for a financial services organization using Llama2 and Amazon OpenSearch Serverless as the Vector datastore. This method refines text generation using Llama2 by dynamically sourcing relevant context."
   ]
  }
 ],
 "metadata": {
  "availableInstances": [
   {
    "_defaultOrder": 0,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.t3.medium",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 1,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.t3.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 2,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.t3.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 3,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.t3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 4,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 5,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 6,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 7,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 8,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 9,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 10,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 11,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 12,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5d.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 13,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5d.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 14,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5d.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 15,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5d.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 16,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5d.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 17,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5d.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 18,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5d.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 19,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 20,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": true,
    "memoryGiB": 0,
    "name": "ml.geospatial.interactive",
    "supportedImageNames": [
     "sagemaker-geospatial-v1-0"
    ],
    "vcpuNum": 0
   },
   {
    "_defaultOrder": 21,
    "_isFastLaunch": true,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.c5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 22,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.c5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 23,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.c5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 24,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.c5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 25,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 72,
    "name": "ml.c5.9xlarge",
    "vcpuNum": 36
   },
   {
    "_defaultOrder": 26,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 96,
    "name": "ml.c5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 27,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 144,
    "name": "ml.c5.18xlarge",
    "vcpuNum": 72
   },
   {
    "_defaultOrder": 28,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.c5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 29,
    "_isFastLaunch": true,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g4dn.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 30,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g4dn.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 31,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g4dn.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 32,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g4dn.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 33,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g4dn.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 34,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g4dn.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 35,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 61,
    "name": "ml.p3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 36,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 244,
    "name": "ml.p3.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 37,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 488,
    "name": "ml.p3.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 38,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.p3dn.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 39,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.r5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 40,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.r5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 41,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.r5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 42,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.r5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 43,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.r5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 44,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.r5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 45,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.r5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 46,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.r5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 47,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 48,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 49,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 50,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 51,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 52,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 53,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.g5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 54,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.g5.48xlarge",
    "vcpuNum": 192
   },
   {
    "_defaultOrder": 55,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 56,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4de.24xlarge",
    "vcpuNum": 96
   }
  ],
  "instance_type": "ml.t3.medium",
  "kernelspec": {
   "display_name": "Python 3 (Base Python 3.0)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/sagemaker-base-python-310-v1"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}