 archived/text-generation-chatbot.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "4a1a52b6",
   "metadata": {},
   "source": [
    "# SageMaker JumpStart Foundation Models - Chatbots"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "09022366",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook.\n",
    "\n",
    "![This badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-west-2/introduction_to_amazon_algorithms|jumpstart-foundation-models|text-generation-chatbot.ipynb)\n",
    "\n",
    "---"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "5acea92d",
   "metadata": {},
   "source": [
    "---\n",
    "Welcome to Amazon [SageMaker JumpStart](https://docs.aws.amazon.com/sagemaker/latest/dg/studio-jumpstart.html)! You can use SageMaker JumpStart to solve many Machine Learning tasks through one-click in SageMaker Studio, or through [SageMaker Python SDK](https://sagemaker.readthedocs.io/en/stable/overview.html#use-prebuilt-models-with-sagemaker-jumpstart).\n",
    "\n",
    "\n",
    "In this demo notebook, we demonstrate how to use the SageMaker Python SDK to deploy models trained with conversational datasets and query the models within an interactive shell. This demonstration provides an open-source Foundation Model chatbot for use within your application. This conversational chatbot with an interactive shell can operate on a variety of models, to include [Falcon-7B-Instruct](https://huggingface.co/tiiuae/falcon-7b-instruct), [Falcon-40B-Instruct](https://huggingface.co/tiiuae/falcon-40b-instruct), [RedPajama-INCITE-Chat-3B](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1), [RedPajama-INCITE-7B-Chat](https://huggingface.co/togethercomputer/RedPajama-INCITE-7B-Chat), and [GPT-NeoXT-Chat-Base-20B](https://huggingface.co/togethercomputer/GPT-NeoXT-Chat-Base-20B).\n",
    "\n",
    "---"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "815c0bc7",
   "metadata": {},
   "source": [
    "1. [Set up](#1.-Set-Up)\n",
    "2. [Select a pre-trained model](#2.-Select-a-pre-trained-model)\n",
    "3. [Retrieve artifacts & deploy an endpoint](#3.-Retrieve-Artifacts-&-Deploy-an-Endpoint)\n",
    "4. [Query endpoint and parse response](#4.-Query-endpoint-and-parse-response)\n",
    "5. [Use a shell interpreter to interact with your deployed endpoint](#5.-Use-a-shell-interpreter-to-interact-with-your-deployed-endpoint)\n",
    "6. [Clean up the endpoint](#6.-Clean-up-the-endpoint)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "a7e35194",
   "metadata": {},
   "source": [
    "Note: This notebook was tested on ml.t3.medium instance in Amazon SageMaker Studio with Python 3 (Data Science) kernel and in Amazon SageMaker Notebook instance with conda_python3 kernel."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "d2f8dfad",
   "metadata": {},
   "source": [
    "### 1. Set up\n",
    "---\n",
    "Before executing the notebook, there are some initial steps required for set up.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eb67d497",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%pip install sagemaker ipywidgets --upgrade --quiet"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "69849d02",
   "metadata": {},
   "source": [
    "### 2. Select a pre-trained model\n",
    "***\n",
    "You can continue with the default model, or can choose a different model from the dropdown generated upon running the next cell. A complete list of SageMaker pre-trained models can also be accessed at [SageMaker pre-trained Models](https://sagemaker.readthedocs.io/en/stable/doc_utils/pretrainedmodels.html#).\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "652b2d4f",
   "metadata": {
    "jumpStartAlterations": [
     "modelIdVersion"
    ],
    "tags": []
   },
   "outputs": [],
   "source": [
    "from typing import NamedTuple\n",
    "from typing import Dict\n",
    "from typing import Any\n",
    "\n",
    "\n",
    "class JumpStartChatbotModelConfig(NamedTuple):\n",
    "    model_id: str\n",
    "    model_kwargs: Dict[str, Any] = {}\n",
    "    payload_kwargs: Dict[str, Any] = {}\n",
    "\n",
    "\n",
    "jumpstart_chatbot_models_config = [\n",
    "    JumpStartChatbotModelConfig(\n",
    "        model_id=\"huggingface-textgeneration-falcon-7b-instruct-bf16\",\n",
    "        payload_kwargs={\"return_full_text\": True},\n",
    "    ),\n",
    "    JumpStartChatbotModelConfig(\n",
    "        model_id=\"huggingface-textgeneration-falcon-40b-instruct-bf16\",\n",
    "        payload_kwargs={\"return_full_text\": True},\n",
    "    ),\n",
    "    JumpStartChatbotModelConfig(\n",
    "        model_id=\"huggingface-textgeneration1-redpajama-incite-chat-3B-v1-fp16\",\n",
    "    ),\n",
    "    JumpStartChatbotModelConfig(\n",
    "        model_id=\"huggingface-textgeneration1-redpajama-incite-chat-7B-v1-fp16\",\n",
    "    ),\n",
    "    JumpStartChatbotModelConfig(\n",
    "        model_id=\"huggingface-textgeneration2-gpt-neoxt-chat-base-20b-fp16\",\n",
    "    ),\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5c82d45b",
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import Markdown\n",
    "from ipywidgets import Dropdown\n",
    "\n",
    "\n",
    "dropdown = Dropdown(\n",
    "    options=[(config.model_id, config) for config in jumpstart_chatbot_models_config],\n",
    "    value=jumpstart_chatbot_models_config[0],\n",
    "    description=\"JumpStart Image Classification Models:\",\n",
    "    style={\"description_width\": \"initial\"},\n",
    "    layout={\"width\": \"max-content\"},\n",
    ")\n",
    "display(Markdown(\"### Select a JumpStart chatbot model from the dropdown below\"))\n",
    "display(dropdown)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "deab0dc7",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_config = dropdown.value"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "0b08aa4a",
   "metadata": {},
   "source": [
    "### 3. Retrieve Artifacts & Deploy an Endpoint\n",
    "\n",
    "***\n",
    "\n",
    "Using SageMaker, we can perform inference on the pre-trained model, even without fine-tuning it first on a new dataset. We start by deploying a `JumpStartModel` to host the pre-trained model. This may take a few minutes.\n",
    "\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "631ae768",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sagemaker.jumpstart.model import JumpStartModel\n",
    "from sagemaker.predictor import Predictor\n",
    "\n",
    "\n",
    "model = JumpStartModel(\n",
    "    model_id=model_config.model_id, model_version=\"1.*\", **model_config.model_kwargs\n",
    ")\n",
    "predictor = model.deploy()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "0d475d7a",
   "metadata": {},
   "source": [
    "***\n",
    "Next, the SageMaker `Predictor` is adjusted to utilize a JSON serializer and the deserializer is custom set to work with all chatbot models supported by this notebook. The deserializer ensures the chatbot will always return a string representing a single generated text sample per query.\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5e4aa517",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.serializers import JSONSerializer\n",
    "from sagemaker.deserializers import JSONDeserializer\n",
    "\n",
    "\n",
    "class JumpStartChatbotDeserializer(JSONDeserializer):\n",
    "    \"\"\"A deserializer to retrieve the first generated text from JumpStart text generation models.\"\"\"\n",
    "\n",
    "    def deserialize(self, stream, content_type):\n",
    "        \"\"\"Crawl the output of JSON deserialization to obtain first generated text model response.\"\"\"\n",
    "        data = super().deserialize(stream, content_type)\n",
    "\n",
    "        while True:\n",
    "            if isinstance(data, str):\n",
    "                break\n",
    "            elif isinstance(data, list):\n",
    "                data = data[0]\n",
    "            elif isinstance(data, dict):\n",
    "                for key in (\"generated_text\", \"generated_texts\"):\n",
    "                    if key in data:\n",
    "                        data = data[key]\n",
    "                        break\n",
    "                else:\n",
    "                    raise ValueError(f\"Generated text keys not found in output {data}.\")\n",
    "            else:\n",
    "                raise ValueError(f\"Output data contains unrecognized type {type(data)}.\")\n",
    "\n",
    "        return data\n",
    "\n",
    "\n",
    "predictor.serializer = JSONSerializer()\n",
    "predictor.deserializer = JumpStartChatbotDeserializer()\n",
    "predictor.content_type = \"application/json\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "a2554851-cbcc-4ef9-864e-776a3550ceca",
   "metadata": {
    "tags": []
   },
   "source": [
    "### 4. Query endpoint and parse response\n",
    "\n",
    "***\n",
    "This model also supports many advanced parameters while performing inference. They include:\n",
    "\n",
    "* **max_length:** Model generates text until the output length (which includes the input context length) reaches `max_length`. If specified, it must be a positive integer.\n",
    "* **max_time:** The maximum amount of time you allow the computation to run for in seconds. Generation will still finish the current pass after allocated time has been passed. This setting can help to generate a response prior to endpoint invocation response time out errors.\n",
    "* **num_return_sequences:** Number of output sequences returned. If specified, it must be a positive integer.\n",
    "* **num_beams:** Number of beams used in the greedy search. If specified, it must be integer greater than or equal to `num_return_sequences`.\n",
    "* **no_repeat_ngram_size:** Model ensures that a sequence of words of `no_repeat_ngram_size` is not repeated in the output sequence. If specified, it must be a positive integer greater than 1.\n",
    "* **temperature:** Controls the randomness in the output. Higher temperature results in output sequence with low-probability words and lower temperature results in output sequence with high-probability words. If `temperature` -> 0, it results in greedy decoding. If specified, it must be a positive float.\n",
    "* **early_stopping:** If True, text generation is finished when all beam hypotheses reach the end of sentence token. If specified, it must be boolean.\n",
    "* **do_sample:** If True, sample the next word as per the likelihood. If specified, it must be boolean.\n",
    "* **top_k:** In each step of text generation, sample from only the `top_k` most likely words. If specified, it must be a positive integer.\n",
    "* **top_p:** In each step of text generation, sample from the smallest possible set of words with cumulative probability `top_p`. If specified, it must be a float between 0 and 1.\n",
    "* **seed:** Fix the randomized state for reproducibility. If specified, it must be an integer.\n",
    "\n",
    "We may specify any subset of the parameters mentioned above while invoking an endpoint. Next, we show an example of how to invoke endpoint with these arguments\n",
    "\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1af6f7b0-4093-48c9-acdb-54b05886b2dc",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "payload = {\n",
    "    \"text_inputs\": \"<human>: Tell me the steps to make a pizza\\n<bot>:\",\n",
    "    \"max_length\": 100,\n",
    "    \"max_time\": 50,\n",
    "    \"top_k\": 50,\n",
    "    \"top_p\": 0.95,\n",
    "    \"do_sample\": True,\n",
    "    \"stopping_criteria\": [\"<human>\"],\n",
    "}\n",
    "print(predictor.predict(payload))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "1228099b-8240-4656-aaa3-4b2e30d3ca40",
   "metadata": {},
   "source": [
    "***\n",
    "Here, we have provided the payload argument `\"stopping_criteria\": [\"<human>\"]`, which has resulted in the model response ending with the generation of the word sequence `\"<human>\"`. The SageMaker JumpStart model script will accept any list of strings as desired stop words, convert this list to a valid [`stopping_criteria` keyword argument](https://huggingface.co/docs/transformers/main_classes/text_generation#transformers.GenerationMixin.generate.stopping_criteria) to the transformers generate API, and terminate text generation when the output sequence contains any specified stop words. This is useful for two reasons: first, inference time is reduced because the endpoint does not continue to generate undesired text beyond the stop words, and, second, this prevents the chatbot model from hallucinating additional human and bot responses until other stop criteria are met.\n",
    "***"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "68e4e096",
   "metadata": {},
   "source": [
    "### 5. Use a shell interpreter to interact with your deployed endpoint\n",
    "\n",
    "***\n",
    "[OpenChatKit](https://github.com/togethercomputer/OpenChatKit) provides a command line shell to interact with their chatbot. In the following code blocks, we provide a bare-bones simplification of the inference scripts in this OpenChatKit repository that can interact with our deployed SageMaker endpoint. There are two main components to this:\n",
    "1. A shell interpreter (`JumpStartChatbotShell`) that allows for iterative inference invocations of the model endpoint, and\n",
    "2. A conversation object (`Conversation`) that stores previous human/chatbot interactions locally within the interactive shell and appropriately formats past conversations for future inference context.\n",
    "\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "598e91b5",
   "metadata": {},
   "outputs": [],
   "source": [
    "import cmd\n",
    "import re\n",
    "from typing import List, Optional\n",
    "\n",
    "\n",
    "class Conversation:\n",
    "    MEANINGLESS_WORDS = [\"<pad>\", \"</s>\", \"<|endoftext|>\"]\n",
    "\n",
    "    def __init__(self, human_id, bot_id):\n",
    "        self.human_tag = f\"{human_id}:\"\n",
    "        self.bot_tag = f\"{bot_id}:\"\n",
    "        self.history = \"\"\n",
    "\n",
    "    def clean_response(self, response):\n",
    "        for word in self.MEANINGLESS_WORDS:\n",
    "            response = response.replace(word, \"\")\n",
    "        response = response.strip(\"\\n\")\n",
    "        return response\n",
    "\n",
    "    def push_human_turn(self, query):\n",
    "        self.history += f\"{self.human_tag} {query}\\n{self.bot_tag}\"\n",
    "\n",
    "    def push_model_response(self, response):\n",
    "        bot_turn = response.split(f\"{self.human_tag}\")[0]\n",
    "        bot_turn = self.clean_response(bot_turn)\n",
    "        self.history += f\"{bot_turn}\\n\"\n",
    "\n",
    "    def get_last_turn(self):\n",
    "        turns = re.split(f\"({self.human_tag}|{self.bot_tag})\\W?\", self.history)\n",
    "        return turns[-1]\n",
    "\n",
    "\n",
    "class JumpStartChatbotShell(cmd.Cmd):\n",
    "    intro = (\n",
    "        \"Welcome to the SageMaker JumpStart chatbot shell! Type /help or /? to list commands. \"\n",
    "        \"Type /quit to exit shell.\\n\"\n",
    "    )\n",
    "    prompt = \">>> \"\n",
    "    response_prefix = \"<<< \"\n",
    "    human_id = \"<human>\"\n",
    "    bot_id = \"<bot>\"\n",
    "\n",
    "    def __init__(self, predictor: Predictor, cmd_queue: Optional[List[str]] = None, **kwargs):\n",
    "        super().__init__()\n",
    "        self.predictor = predictor\n",
    "        self.payload_kwargs = kwargs\n",
    "        self.payload_kwargs[\"stopping_criteria\"] = [self.human_id]\n",
    "        if cmd_queue is not None:\n",
    "            self.cmdqueue = cmd_queue\n",
    "\n",
    "    def preloop(self):\n",
    "        self.conversation = Conversation(self.human_id, self.bot_id)\n",
    "\n",
    "    def precmd(self, line):\n",
    "        command = line[1:] if line.startswith(\"/\") else \"say \" + line\n",
    "        return command\n",
    "\n",
    "    def do_say(self, arg):\n",
    "        self.conversation.push_human_turn(arg)\n",
    "        history = self.conversation.history\n",
    "        payload = {\"text_inputs\": history, **self.payload_kwargs}\n",
    "        response = self.predictor.predict(payload)[len(history) :]\n",
    "        self.conversation.push_model_response(response)\n",
    "        print(f\"{self.response_prefix}{self.conversation.get_last_turn()}\")\n",
    "\n",
    "    def do_reset(self, arg):\n",
    "        self.conversation = Conversation(self.human_id, self.bot_id)\n",
    "\n",
    "    def do_hyperparameters(self, arg):\n",
    "        print(f\"Hyperparameters: {self.payload_kwargs}\\n\")\n",
    "\n",
    "    def do_quit(self, arg):\n",
    "        return True"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "aeb0a3da",
   "metadata": {},
   "source": [
    "***\n",
    "We can now launch this shell as a command loop. This will repeatedly issue a prompt, accept input, parse the input command, and dispatch actions. Because the resulting shell may be utilized in an infinite loop, this notebook provides a default command queue (`cmdqueue`) as a queued list of input lines; when the last command in the queue, `/quit`, is executed, the shell will terminate. To dynamically interact with this chatbot, please remove the `cmdqueue`.\n",
    "\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "08ed2ac6",
   "metadata": {},
   "outputs": [],
   "source": [
    "cmd_queue = [\n",
    "    \"Hello!\",\n",
    "    \"Make a markdown table of national parks with the state they are located in and date established.\",\n",
    "    \"/hyperparameters\",\n",
    "    \"/quit\",\n",
    "]\n",
    "payload_kwargs_default = {\n",
    "    \"max_new_tokens\": 128,\n",
    "    \"do_sample\": True,\n",
    "    \"temperature\": 0.6,\n",
    "    \"top_k\": 40,\n",
    "}\n",
    "JumpStartChatbotShell(\n",
    "    predictor=predictor,\n",
    "    cmd_queue=cmd_queue,\n",
    "    **{**payload_kwargs_default, **model_config.payload_kwargs},\n",
    ").cmdloop()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "1a01602c-78b0-4281-890c-87e10ff299b7",
   "metadata": {},
   "source": [
    "***\n",
    "And that's it! Just a quick reminder: you can comment out the `cmd_queue` in the above cell to have an interactive dialog with the chatbot.\n",
    "***"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "aa5de21f",
   "metadata": {},
   "source": [
    "### 6. Clean up the endpoint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "69b588d1",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "predictor.delete_model()\n",
    "predictor.delete_endpoint()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "a79a77f2",
   "metadata": {},
   "source": [
    "## Notebook CI Test Results\n",
    "\n",
    "This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.\n",
    "\n",
    "\n",
    "![This badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-east-1/introduction_to_amazon_algorithms|jumpstart-foundation-models|text-generation-chatbot.ipynb)\n",
    "\n",
    "![This badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-east-2/introduction_to_amazon_algorithms|jumpstart-foundation-models|text-generation-chatbot.ipynb)\n",
    "\n",
    "![This badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-west-1/introduction_to_amazon_algorithms|jumpstart-foundation-models|text-generation-chatbot.ipynb)\n",
    "\n",
    "![This badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ca-central-1/introduction_to_amazon_algorithms|jumpstart-foundation-models|text-generation-chatbot.ipynb)\n",
    "\n",
    "![This badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/sa-east-1/introduction_to_amazon_algorithms|jumpstart-foundation-models|text-generation-chatbot.ipynb)\n",
    "\n",
    "![This badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-1/introduction_to_amazon_algorithms|jumpstart-foundation-models|text-generation-chatbot.ipynb)\n",
    "\n",
    "![This badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-2/introduction_to_amazon_algorithms|jumpstart-foundation-models|text-generation-chatbot.ipynb)\n",
    "\n",
    "![This badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-3/introduction_to_amazon_algorithms|jumpstart-foundation-models|text-generation-chatbot.ipynb)\n",
    "\n",
    "![This badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-central-1/introduction_to_amazon_algorithms|jumpstart-foundation-models|text-generation-chatbot.ipynb)\n",
    "\n",
    "![This badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-north-1/introduction_to_amazon_algorithms|jumpstart-foundation-models|text-generation-chatbot.ipynb)\n",
    "\n",
    "![This badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-southeast-1/introduction_to_amazon_algorithms|jumpstart-foundation-models|text-generation-chatbot.ipynb)\n",
    "\n",
    "![This badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-southeast-2/introduction_to_amazon_algorithms|jumpstart-foundation-models|text-generation-chatbot.ipynb)\n",
    "\n",
    "![This badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-northeast-1/introduction_to_amazon_algorithms|jumpstart-foundation-models|text-generation-chatbot.ipynb)\n",
    "\n",
    "![This badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-northeast-2/introduction_to_amazon_algorithms|jumpstart-foundation-models|text-generation-chatbot.ipynb)\n",
    "\n",
    "![This badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-south-1/introduction_to_amazon_algorithms|jumpstart-foundation-models|text-generation-chatbot.ipynb)\n"
   ]
  }
 ],
 "metadata": {
  "availableInstances": [
   {
    "_defaultOrder": 0,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.t3.medium",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 1,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.t3.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 2,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.t3.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 3,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.t3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 4,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 5,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 6,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 7,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 8,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 9,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 10,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 11,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 12,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5d.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 13,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5d.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 14,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5d.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 15,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5d.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 16,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5d.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 17,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5d.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 18,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5d.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 19,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 20,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": true,
    "memoryGiB": 0,
    "name": "ml.geospatial.interactive",
    "supportedImageNames": [
     "sagemaker-geospatial-v1-0"
    ],
    "vcpuNum": 0
   },
   {
    "_defaultOrder": 21,
    "_isFastLaunch": true,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.c5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 22,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.c5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 23,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.c5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 24,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.c5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 25,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 72,
    "name": "ml.c5.9xlarge",
    "vcpuNum": 36
   },
   {
    "_defaultOrder": 26,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 96,
    "name": "ml.c5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 27,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 144,
    "name": "ml.c5.18xlarge",
    "vcpuNum": 72
   },
   {
    "_defaultOrder": 28,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.c5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 29,
    "_isFastLaunch": true,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g4dn.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 30,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g4dn.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 31,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g4dn.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 32,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g4dn.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 33,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g4dn.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 34,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g4dn.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 35,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 61,
    "name": "ml.p3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 36,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 244,
    "name": "ml.p3.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 37,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 488,
    "name": "ml.p3.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 38,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.p3dn.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 39,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.r5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 40,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.r5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 41,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.r5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 42,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.r5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 43,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.r5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 44,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.r5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 45,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.r5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 46,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.r5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 47,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 48,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 49,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 50,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 51,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 52,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 53,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.g5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 54,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.g5.48xlarge",
    "vcpuNum": 192
   },
   {
    "_defaultOrder": 55,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 56,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4de.24xlarge",
    "vcpuNum": 96
   }
  ],
  "instance_type": "ml.t3.medium",
  "kernelspec": {
   "display_name": "Python 3 (Data Science 2.0)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-2:429704687514:image/sagemaker-data-science-38"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}