AI21_Summarize_example_model_use.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using AI21 Summarize on SageMaker through Model Packages\n",
    "\n",
    "This sample notebook shows you how to deploy **AI21 Summarize** using Amazon SageMaker.\n",
    "\n",
    "\n",
    "--------------------\n",
    "## <font color='orange'>Important:</font>\n",
    "Please visit model detail page in <a href=\"https://aws.amazon.com/marketplace/pp/prodview-dkwy6chb63hk2\">https://aws.amazon.com/marketplace/pp/prodview-dkwy6chb63hk2</a> to learn more. <font color='orange'>If you do not have access to the link, please contact account admin for the help.</font>\n",
    "\n",
    "You will find details about the model including pricing, supported region, and end user license agreement. To use the model, please click “<font color='orange'>Continue to Subscribe</font>” from the detail page, come back here and learn how to deploy and inference.\n",
    "\n",
    "\n",
    "-------------------\n",
    "\n",
    "**AI21 Summarize** model offers access to our top-of-the-line summarization engine. It has been specifically developed for understanding long texts and providing a faithful summary of the original document. As a task-specific model, it can be easily integrated into existing systems without requiring prompt engineering: simply provide the text you want to summarize and get the best summary in the market (according to a [recent study](https://www.ai21.com/blog/summarize-api-outperforms-openais-models)).\n",
    "\n",
    "\n",
    "\n",
    "## Pre-requisites:\n",
    "1. Before running this notebook, please make sure you got this notebook from the model catalog on SageMaker AWS Management Console.\n",
    "1. **Note**: This notebook contains elements which render correctly in Jupyter interface. Open this notebook from an Amazon SageMaker Notebook Instance or Amazon SageMaker Studio.\n",
    "1. Ensure that IAM role used has **AmazonSageMakerFullAccess**.\n",
    "1. This notebook is intended to work with **boto3 v1.25.4** or higher.\n",
    "\n",
    "## Contents:\n",
    "1. [Select model package](#1.-Select-model-package)\n",
    "1. [Create an endpoint and perform real-time inference](#2.-Create-an-endpoint-and-perform-real-time-inference)\n",
    "   1. [Create an endpoint](#A.-Create-an-endpoint)\n",
    "   1. [Summarize a single paragraph](#B.-Summarize-a-single-paragraph)\n",
    "   1. [Summarize longer texts](#C.-Summarize-longer-texts)\n",
    "1. [Clean-up](#3.-Clean-up)\n",
    "   1. [Delete the endpoint](#A.-Delete-the-endpoint)\n",
    "   1. [Delete the model](#B.-Delete-the-model)\n",
    "\n",
    "\n",
    "## Usage instructions\n",
    "You can run this notebook one cell at a time (By using Shift+Enter for running a cell)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "from sagemaker import ModelPackage\n",
    "from sagemaker import get_execution_role\n",
    "import sagemaker as sage\n",
    "import boto3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import wikipedia for longer texts example\n",
    "! pip install wikipedia\n",
    "import wikipedia"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Check the version of boto3 - must be v1.25.4 or higher\n",
    "If you see a lower version number, pick another kernel to run the notebook, with Python 3.8 or above"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "boto3.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Install ai21 python SDK"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! pip install -U \"ai21[AWS]>=2.1.2\"\n",
    "import ai21"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Check the version of ai21 - must be 2.1.2 or higher"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-02-15T12:24:37.994123Z",
     "start_time": "2024-02-15T12:24:37.908317Z"
    }
   },
   "outputs": [],
   "source": [
    "import ai21\n",
    "ai21.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Select model package\n",
    "Confirm that you received this notebook from the model catalog in SageMaker AWS Management Console."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "region = boto3.Session().region_name\n",
    "\n",
    "# Get the updated ARN\n",
    "model_package_arn = ai21.SageMaker.get_model_package_arn(model_name=\"summarize\", region=region)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "role = get_execution_role()\n",
    "sagemaker_session = sage.Session()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Create an endpoint and perform real-time inference"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <span style='color:Blue'> How to choose the best instance for my use case?</span>\n",
    "<span style='color:#0057FF'> When you create your endpoint, you need to choose the instance type to run the model on. Choosing the right instance is mainly a matter of economics. Depending on your use case, you probably want the most cost-effective instance possible. In this notebook we use one of the supported instances.</span>\n",
    "\n",
    "<span style='color:#0057FF'>Looking for the list of all supported instances? See</span> [here](https://docs.ai21.com/docs/choosing-the-right-instance-type-for-amazon-sagemaker-models#ai21-summarize)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you want to understand how real-time inference with Amazon SageMaker works, see [Documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/deploy-model.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "endpoint_name = \"summarize\"\n",
    "\n",
    "content_type = \"application/json\"\n",
    "\n",
    "real_time_inference_instance_type = (\n",
    "    \"ml.g4dn.12xlarge\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A. Create an endpoint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a deployable model from the model package.\n",
    "model = ModelPackage(\n",
    "    role=role, model_package_arn=model_package_arn, sagemaker_session=sagemaker_session\n",
    ")\n",
    "\n",
    "# Deploy the model\n",
    "predictor = model.deploy(\n",
    "    initial_instance_count=1,\n",
    "    instance_type=real_time_inference_instance_type,\n",
    "    endpoint_name=endpoint_name, \n",
    "    model_data_download_timeout=3600,\n",
    "    container_startup_health_check_timeout=600,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once endpoint has been created, you would be able to perform real-time inference."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### B. Summarize a single paragraph"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**AI21 Summarize model** offers access to our world-class summarization engine. It has been specifically developed for understanding long texts and providing a faithful summary of the original document.\n",
    "\n",
    "This model takes a piece of text and generates summaries that remain faithful to the original document (i.e. no external information is added during the process)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The input text should contain **at least 40 words** and **no more than 50,000 characters**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Important - Model Usage\n",
    "The input to the model should be simple and straight-forward, no prompting needed.\n",
    "\n",
    "**Good input:** *[TEXT]*\n",
    "\n",
    "**Bad input:** *Summarize the following text: [TEXT]*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, create an AI21 Sagemaker client"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from ai21 import AI21SageMakerClient\n",
    "from ai21.models import DocumentType\n",
    "\n",
    "client = AI21SageMakerClient(endpoint_name=endpoint_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following is an example of a summary of a single paragraph taken from a news article."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "text = \"\"\"The error affected a number of international flights leaving the terminal on Wednesday, with some airlines urging passengers to travel only with hand luggage.\n",
    "Virgin Atlantic said all airlines flying out of the terminal had been affected.\n",
    "Passengers have been warned it may be days before they are reunited with luggage.\n",
    "An airport spokesperson apologised and said the fault had now been fixed.\n",
    "Virgin Atlantic said it would ensure all bags were sent out as soon as possible.\n",
    "It added customers should retain receipts for anything they had bought and make a claim to be reimbursed.\n",
    "Passengers, who were informed by e-mail of the problem, took to social media to vent their frustrations.\n",
    "One branded the situation \"ludicrous\" and said he was only told 12 hours before his flight.\n",
    "The airport said it could not confirm what the problem was, what had caused it or how many people had been affected.\"\"\"\n",
    "\n",
    "response = client.summarize.create(\n",
    "    source=text,\n",
    "    source_type=DocumentType.TEXT,\n",
    ")\n",
    "\n",
    "print(\"Original text:\")\n",
    "print(text)\n",
    "print(\"================\")\n",
    "print(\"Summary:\")\n",
    "print(response.summary)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### C. Summarize longer texts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "AI21 Summarize can handle inputs up to 50,000 characters, as stated above. This translates into roughly 10,000 words, or a whopping 40 pages.\n",
    "\n",
    "As a demonstration of the model's behavior, we'll load a page from Wikipedia. If you're a fan of the TV show 'The Office', this will look familiar :)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "wiki_page = wikipedia.page(\"The_Office_(American_TV_series)\").content"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since this is a full page, we will focus only the opening section."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "office_text = wiki_page.split(\"\\n\\n\")[0]\n",
    "print(office_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The summaries are formatted as bullet lists, following the original text flow."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "response = client.summarize.create(\n",
    "    source=office_text,\n",
    "    sourceType=DocumentType.TEXT,\n",
    ")\n",
    "\n",
    "print(response.summary)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is the full text and the bullet points summarized:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Original text:\")\n",
    "print(office_text)\n",
    "print(\"====================\")\n",
    "print(\"Summary:\")\n",
    "summaries = response.summary.split(\"\\n\")\n",
    "for summary in summaries:\n",
    "    print(\"- \" + summary)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Interested in learning more?\n",
    "Take a look at our [guide](https://docs.ai21.com/docs/summarize-api) to understand all the capabilities of AI21 Summarize model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Clean-up"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A. Delete the endpoint"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that you have successfully performed a real-time inference, you do not need the endpoint any more. You can terminate the endpoint to avoid being charged."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.sagemaker_session.delete_endpoint(endpoint_name)\n",
    "model.sagemaker_session.delete_endpoint_config(endpoint_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### B. Delete the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.delete_model()"
   ]
  }
 ],
 "metadata": {
  "instance_type": "ml.t3.medium",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}