inference/generativeai/llm-workshop/lab10-open-llama/open-llama-7b/open_llama_7b.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "71a329f0",
   "metadata": {},
   "source": [
    "# Open-LLAMA 7B implementation using LMI container on SageMaker\n",
    "### Model source: https://github.com/openlm-research/open_llama ; \n",
    "#### Model download hub: https://huggingface.co/openlm-research/open_llama_7b; \n",
    "#### License: Apache-2.0\n",
    "In this tutorial, you will bring your own container from docker hub to SageMaker and run inference with it.\n",
    "Please make sure the following permission granted before running the notebook:\n",
    "\n",
    "- ECR Push/Pull access\n",
    "- S3 bucket push access\n",
    "- SageMaker access\n",
    "\n",
    "#### Attribution: this notebook is based on the content of https://github.com/deepjavalibrary/djl-demo/tree/master and was debugged with the help of lanking520.\n",
    "\n",
    "## Step 1: Let's bump up SageMaker and import stuff"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "ce21e78b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "%pip install --upgrade pip --quiet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "67fa3208",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install sagemaker boto3 awscli --upgrade --quiet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "ec9ac353",
   "metadata": {},
   "outputs": [],
   "source": [
    "import boto3\n",
    "import sagemaker\n",
    "from sagemaker import Model, serializers, deserializers\n",
    "\n",
    "role = sagemaker.get_execution_role()  # execution role for the endpoint\n",
    "sess = sagemaker.session.Session()  # sagemaker session for interacting with different AWS APIs\n",
    "region = sess._region_name  # region name of the current SageMaker Studio environment\n",
    "account_id = sess.account_id()  # account_id of the current SageMaker Studio environment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "7f195c0b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "arn:aws:iam::328296961357:role/service-role/AmazonSageMaker-ExecutionRole-20191125T182032 us-west-2 328296961357\n"
     ]
    }
   ],
   "source": [
    "print(role, region, account_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "a2d4667f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'2.161.0'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sagemaker.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71542f98",
   "metadata": {},
   "source": [
    "## Step 2 pull and push the docker from Docker hub to ECR repository (optional)\n",
    "*Note: you can either use a prebuilt container or use the cell below (change cell type to 'code' from 'raw\")\n",
    "\n",
    "*Note: Please make sure you have the permission in AWS credential to push to ECR repository*\n",
    "\n",
    "This process may take a while, depends on the container size and your network bandwidth\n",
    "### Note: you only need to build this container once. Once you pushed it in ECR, you can pull the image via  \n",
    "image_uri = f\"{account_id}.dkr.ecr.{region}.amazonaws.com/{repo_name}:latest\""
   ]
  },
  {
   "cell_type": "raw",
   "id": "7ba4fa61",
   "metadata": {},
   "source": [
    "%%sh\n",
    "\n",
    "# The name of our container\n",
    "repo_name=djlserving-byoc\n",
    "# Target container\n",
    "target_container=\"deepjavalibrary/djl-serving:0.23.0-deepspeed\"\n",
    "\n",
    "account=$(aws sts get-caller-identity --query Account --output text)\n",
    "\n",
    "# Get the region defined in the current configuration (default to us-west-2 if none defined)\n",
    "region=$(aws configure get region)\n",
    "region=${region:-us-west-2}\n",
    "\n",
    "fullname=\"${account}.dkr.ecr.${region}.amazonaws.com/${repo_name}:latest\"\n",
    "echo \"Creating ECR repository ${fullname}\"\n",
    "\n",
    "# If the repository doesn't exist in ECR, create it.\n",
    "\n",
    "aws ecr describe-repositories --repository-names \"${repo_name}\" > /dev/null 2>&1\n",
    "\n",
    "if [ $? -ne 0 ]\n",
    "then\n",
    "    aws ecr create-repository --repository-name \"${repo_name}\" > /dev/null\n",
    "fi\n",
    "\n",
    "# Get the login command from ECR and execute it directly\n",
    "aws ecr get-login-password --region ${region} | docker login --username AWS --password-stdin \"${account}.dkr.ecr.${region}.amazonaws.com\"\n",
    "\n",
    "# Build the docker image locally with the image name and then push it to ECR\n",
    "# with the full name.\n",
    "echo \"Start pulling container: ${target_container}\"\n",
    "\n",
    "docker pull ${target_container}\n",
    "docker tag ${target_container} ${fullname}\n",
    "docker push ${fullname}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "81deac79",
   "metadata": {},
   "source": [
    "## Step 3: Start preparing model artifacts\n",
    "In LMI container, we expect some artifacts to help set up the model\n",
    "- serving.properties (required): Defines the model server settings\n",
    "- model.py (optional): A python file to define the core inference logic\n",
    "- requirements.txt (optional): Any additional pip wheel need to install"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "b011bf5f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing serving.properties\n"
     ]
    }
   ],
   "source": [
    "%%writefile serving.properties\n",
    "engine = Python\n",
    "option.tensor_parallel_degree = 1\n",
    "option.model_id = openlm-research/open_llama_7b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "4dd9b8ce",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing model.py\n"
     ]
    }
   ],
   "source": [
    "%%writefile model.py\n",
    "from djl_python import Input, Output\n",
    "from djl_python.streaming_utils import StreamingUtils\n",
    "import os\n",
    "import deepspeed\n",
    "import torch\n",
    "import logging\n",
    "from transformers import pipeline, AutoModelForCausalLM, AutoTokenizer\n",
    "from transformers.models.llama.modeling_llama import LlamaDecoderLayer\n",
    "\n",
    "##\n",
    "from transformers import LlamaTokenizer, LlamaForCausalLM\n",
    "\n",
    "model = None\n",
    "tokenizer = None\n",
    "\n",
    "\n",
    "def get_model(properties):\n",
    "    model_name = properties[\"model_id\"]\n",
    "    tensor_parallel_degree = properties[\"tensor_parallel_degree\"]\n",
    "    max_tokens = int(properties.get(\"max_tokens\", \"1024\"))\n",
    "    dtype = torch.float16\n",
    "\n",
    "    model = LlamaForCausalLM.from_pretrained(\n",
    "        model_name, low_cpu_mem_usage=True, torch_dtype=dtype, device_map=\"auto\"\n",
    "    )\n",
    "    tokenizer = LlamaTokenizer.from_pretrained(model_name)\n",
    "    tokenizer.pad_token = tokenizer.eos_token\n",
    "\n",
    "    return model, tokenizer\n",
    "\n",
    "\n",
    "def inference(inputs):\n",
    "    try:\n",
    "        input_map = inputs.get_as_json()\n",
    "        data = input_map.pop(\"inputs\", input_map)\n",
    "        parameters = input_map.pop(\"parameters\", {})\n",
    "        outputs = Output()\n",
    "\n",
    "        enable_streaming = (\n",
    "            inputs.get_properties().get(\"enable_streaming\", \"false\").lower() == \"true\"\n",
    "        )\n",
    "        if enable_streaming:\n",
    "            stream_generator = StreamingUtils.get_stream_generator(\"DeepSpeed\")\n",
    "            outputs.add_stream_content(stream_generator(model, tokenizer, data, **parameters))\n",
    "            return outputs\n",
    "\n",
    "        tokenizer.add_special_tokens({\"pad_token\": \"[PAD]\"})\n",
    "        tokenizer.padding_side = \"left\"\n",
    "        input_tokens = tokenizer(data, padding=True, return_tensors=\"pt\").to(\n",
    "            torch.cuda.current_device()\n",
    "        )\n",
    "        with torch.no_grad():\n",
    "            output_tokens = model.generate(input_tokens.input_ids, **parameters)\n",
    "        generated_text = tokenizer.batch_decode(output_tokens, skip_special_tokens=True)\n",
    "\n",
    "        outputs.add_as_json([{\"generated_text\": s} for s in generated_text])\n",
    "        return outputs\n",
    "    except Exception as e:\n",
    "        logging.exception(\"Huggingface inference failed\")\n",
    "        # error handling\n",
    "        outputs = Output().error(str(e))\n",
    "\n",
    "\n",
    "def handle(inputs: Input) -> None:\n",
    "    global model, tokenizer\n",
    "    if not model:\n",
    "        model, tokenizer = get_model(inputs.get_properties())\n",
    "\n",
    "    if inputs.is_empty():\n",
    "        # Model server makes an empty call to warmup the model on startup\n",
    "        return None\n",
    "\n",
    "    return inference(inputs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "e8b50a6c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing requirements.txt\n"
     ]
    }
   ],
   "source": [
    "%%writefile requirements.txt\n",
    "transformers==4.28.1\n",
    "protobuf==3.20.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "b0142973",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mymodel/\n",
      "mymodel/requirements.txt\n",
      "mymodel/model.py\n",
      "mymodel/serving.properties\n"
     ]
    }
   ],
   "source": [
    "%%sh\n",
    "mkdir mymodel\n",
    "mv serving.properties mymodel/\n",
    "mv model.py mymodel/\n",
    "mv requirements.txt mymodel/\n",
    "tar czvf mymodel.tar.gz mymodel/\n",
    "rm -rf mymodel"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2e58cf33",
   "metadata": {},
   "source": [
    "## Step 4: Start building SageMaker endpoint\n",
    "In this step, we will build SageMaker endpoint from scratch\n",
    "\n",
    "### 4.1 Upload artifact on S3 and create SageMaker model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "38b1e5ca",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "S3 Code or Model tar ball uploaded to --- > s3://sagemaker-us-west-2-328296961357/large-model-lmi/code/mymodel.tar.gz\n",
      "763104351884.dkr.ecr.us-west-2.amazonaws.com/djl-inference:0.23.0-deepspeed0.9.5-cu118\n"
     ]
    }
   ],
   "source": [
    "s3_code_prefix = \"large-model-lmi/code\"\n",
    "bucket = sess.default_bucket()  # bucket to house artifacts\n",
    "code_artifact = sess.upload_data(\"mymodel.tar.gz\", bucket, s3_code_prefix)\n",
    "print(f\"S3 Code or Model tar ball uploaded to --- > {code_artifact}\")\n",
    "\n",
    "# repo_name=\"djlserving-byoc\"\n",
    "# image_uri = f\"{account_id}.dkr.ecr.{region}.amazonaws.com/{repo_name}:latest\"\n",
    "image_uri = sagemaker.image_uris.retrieve(\n",
    "    framework=\"djl-deepspeed\", region=region, version=\"0.23.0\"\n",
    ")\n",
    "\n",
    "\n",
    "print(image_uri)\n",
    "\n",
    "model = Model(image_uri=image_uri, model_data=code_artifact, role=role)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "004f39f6",
   "metadata": {},
   "source": [
    "### 4.2 Create SageMaker endpoint\n",
    "\n",
    "You need to specify the instance to use and endpoint names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "8e0e61cd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--------------!"
     ]
    }
   ],
   "source": [
    "instance_type = \"ml.g5.2xlarge\"  # \"ml.g5.2xlarge\" - #single GPU. really need one GPU for this since tensor split is '1'\n",
    "\n",
    "endpoint_name = sagemaker.utils.name_from_base(\"open-llama-lmi-model\")\n",
    "\n",
    "model.deploy(\n",
    "    initial_instance_count=1,\n",
    "    instance_type=instance_type,\n",
    "    endpoint_name=endpoint_name,\n",
    "    container_startup_health_check_timeout=900,\n",
    ")\n",
    "\n",
    "# our requests and responses will be in json format so we specify the serializer and the deserializer\n",
    "predictor = sagemaker.Predictor(\n",
    "    endpoint_name=endpoint_name,\n",
    "    sagemaker_session=sess,\n",
    "    serializer=serializers.JSONSerializer(),\n",
    "    deserializer=deserializers.JSONDeserializer(),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb63ee65",
   "metadata": {},
   "source": [
    "## Step 5a: Test and benchmark inference latency\n",
    "### The latency is heavily dependent on 'max_new_tokens' parameter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "2bcef095",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.2340340614318848\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "\n",
    "tic = time.time()\n",
    "predictor.predict(\n",
    "    {\"inputs\": \"tuna sandwich nutritional content is \", \"parameters\": {\"max_new_tokens\": 16}}\n",
    ")\n",
    "toc = time.time()\n",
    "print(toc - tic)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "807e258b",
   "metadata": {},
   "source": [
    "## Let us define a helper function to get a histogram of invocation latency distribution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "3b35a270",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Matplotlib is building the font cache; this may take a moment.\n"
     ]
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import time\n",
    "import numpy as np\n",
    "from tqdm import tqdm\n",
    "\n",
    "\n",
    "def _latency_hist_plot(endpoint_name, invocation_number=100, sleep_time=1):\n",
    "    latency_array = []\n",
    "    for i in tqdm(range(invocation_number)):\n",
    "        tic = time.time()\n",
    "        response_ = predictor.predict(\n",
    "            {\"inputs\": \"Large model inference is\", \"parameters\": {\"max_new_tokens\": 256}}\n",
    "        )\n",
    "        toc = time.time()\n",
    "        latency_array.append(toc - tic)\n",
    "        time.sleep(sleep_time)\n",
    "\n",
    "    latency_array_np = np.array(latency_array)\n",
    "    _ = plt.hist(latency_array_np, bins=\"auto\")  # arguments are passed to np.histogram\n",
    "    plt.title(\"Invocation Latency Histogram with 'auto' bins\")\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "020c7bd0",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 10/10 [01:53<00:00, 11.35s/it]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGxCAYAAACwbLZkAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA1bklEQVR4nO3de1hVVeL/8c9R4YCKKBg3RUFKxszbQGNkXgizUbOcrCmbTLOZyVuWjFOp3/HWFGZOY00qaaKZWdZgZmkmhpeasMSB0TEzmxQcBzIrwSxRcf3+8OH8OnI9CCyh9+t59sOz11lr77XX2eecD/tyjsMYYwQAAGBJI9sdAAAAP22EEQAAYBVhBAAAWEUYAQAAVhFGAACAVYQRAABgFWEEAABYRRgBAABWEUYAAIBVhJFLzPLly+VwOJSZmWm7KzVu1apVmj9/fpmPORwOzZw5s077I9X8eD/xxBNau3ZtjSzLtq1bt8rhcOjvf/97mY9PmDBBDofDraxfv37q16+fR+v55JNPNHPmTB06dKiaPf1pu3DMv//+e82cOVNbt24tVXfmzJlyOBw6duxYtdY1atQoj59fT9XE/jBz5kxFRERUWq9fv3666qqrqrRMW+9RPxVNbHcAPx2rVq3Sv//9bz300EOlHsvIyFDbtm3rvlM17IknntBtt92moUOH2u6KFQsXLvS4zSeffKJZs2apX79+VfoAgbsLx/z777/XrFmzJKnWg0NtuFT3h4byHnWpIozgknDNNdfY7gJqwJVXXmm7Cx4rLi7W2bNn5XQ6bXelWurjmNdHvEfVLk7T1AOjRo1S8+bN9fnnn2vQoEFq3ry5wsPD9Yc//EFFRUWSpDNnzigoKEgjRowo1f748ePy9fVVYmKiqyw3N1d33323goKC5HQ61alTJ/3lL3/RuXPn3NoWFRVp9uzZ6tSpk3x8fBQYGKj4+Hh9+OGHrjoLFixQnz59FBQUpGbNmqlLly6aO3euzpw546rTr18/rV+/Xjk5OXI4HK6pRFmHQP/973/rlltuUatWreTj46Pu3bvrxRdfdKtTcirhlVde0bRp0xQWFqYWLVqof//+2r9/v+eDXYZTp07pD3/4g7p37y5/f38FBAQoLi5Ob775pls9h8OhkydP6sUXX3Rt34//M83Pz9f999+vtm3bytvbW5GRkZo1a5bOnj3rqnPo0CE5HA7NmzdPTz/9tCIjI9W8eXPFxcVpx44dpfr20UcfaciQIQoMDJSPj4+ioqJcR57ef/9919hcaMWKFXI4HNq5c2eNjFGJsk7TLFq0SN26dVPz5s3l5+enn/3sZ5o6daqk86fJbr/9dklSfHy8a9yWL1/uap+SkqJu3brJx8dHAQEB+tWvfqV9+/aVWveSJUvUsWNHOZ1OXXnllVq1apVGjRrl9t91yfjOnTtXf/7znxUZGSmn06ktW7ZU+XmWzj/XEyZM0LJlyxQdHS1fX1/FxsZqx44dMsboqaeecj13119/vT7//PMKx23v3r1yOBx6/fXXXWW7du2Sw+FQ586d3erefPPNiomJKXPMDx06pMsuu0ySNGvWLNd4jho1ym0ZX375pYYPHy5/f38FBwdr9OjRKigoqLCP5Zk1a5Z69uypgIAAtWjRQj//+c+1dOlSXfgbrOWd5oiIiHD1ryb3B0+8//77uuaaa+Tr66s2bdroT3/6k4qLiyvsf8kp3i1btmjs2LFq3bq1AgMDdeutt+p///ufW9v09HT169dPgYGB8vX1Vbt27TRs2DB9//33F9XvBsXgkrJs2TIjyezcudNVNnLkSOPt7W06depk5s2bZzZv3mymT59uHA6HmTVrlqvepEmTjK+vrykoKHBb5sKFC40ks3v3bmOMMUePHjVt2rQxl112mUlOTjYbN240EyZMMJLM2LFjXe3OnDlj4uPjTZMmTczkyZPNhg0bzLp168zUqVPNK6+84rbeRYsWmY0bN5r09HTz17/+1bRu3drce++9rjp79+41vXr1MiEhISYjI8M1lZBkZsyY4Zr/9NNPjZ+fn4mKijIrVqww69evN8OHDzeSzJNPPumqt2XLFiPJREREmN/85jdm/fr15pVXXjHt2rUzV1xxhTl79qzH432h48ePm1GjRpmXXnrJpKenm40bN5rJkyebRo0amRdffNFVLyMjw/j6+ppBgwa5tm/v3r3GGGPy8vJMeHi4ad++vXn++efN5s2bzWOPPWacTqcZNWqUaxkHDx50bc8vf/lLs3btWrN27VrTpUsX06pVK3P8+HFX3Y0bNxovLy/TtWtXs3z5cpOenm5SUlLMnXfe6arTo0cP06tXr1LbdPXVV5urr766wrEpGdvVq1ebM2fOlJrGjRtnLnwL6du3r+nbt69r/pVXXjGSzAMPPGA2bdpkNm/ebJKTk83EiRONMef3xSeeeMJIMgsWLHCN29GjR40xxvXY8OHDzfr1682KFStMhw4djL+/v/nss89c63n++eeNJDNs2DDz9ttvm5dfftl07NjRtG/f3rRv377U+LZp08bEx8ebv//972bTpk3m4MGDVX6ejTm/v7Zv395ce+21Zs2aNeaNN94wHTt2NAEBAWbSpEnmlltucfUjODjYdO3a1Zw7d67C8Q4NDTW///3vXfNz5swxvr6+RpI5cuSIMeb8a7JFixbm4YcfLnPMT506ZTZu3Ggkmfvuu881np9//rkxxpgZM2YYSSY6OtpMnz7dpKWlmaeffto4nU6316snRo0aZZYuXWrS0tJMWlqaeeyxx4yvr6/be1PJmP34NV6iffv2ZuTIkcaYmtsfqqpv374mMDDQhIWFmWeffda8++67ZuLEiUaSGT9+fIX9L3nv6NChg3nggQfMu+++a1544QXTqlUrEx8f76p38OBB4+PjY2644Qazdu1as3XrVvPyyy+bESNGmG+//dbjPjdUhJFLTHlhRJJ57bXX3OoOGjTIREdHu+Z3795tJJnFixe71fvFL35hYmJiXPOPPvqokWQ++ugjt3pjx441DofD7N+/3xhjzIoVK4wks2TJkir3v7i42Jw5c8asWLHCNG7c2HzzzTeuxwYPHuz2wfBjF77Q77zzTuN0Ok1ubq5bvYEDB5qmTZu6PpRLPjAHDRrkVu+1114zktwCT1mqEkYudPbsWXPmzBlz3333mR49erg91qxZM9cb64/df//9pnnz5iYnJ8etfN68eUaSK7SUfFh26dLFLUh9/PHHRpJbCIyKijJRUVHmhx9+qHT7srKySi3rwg/YC5WMbWXTj10YRiZMmGBatmxZ4Xpef/11I8ls2bLFrfzbb791hbsfy83NNU6n09x1113GmPP7XEhIiOnZs6dbvZycHOPl5VVmGImKijKnT5+usF8VPc+STEhIiPnuu+9cZWvXrjWSTPfu3d2Cx/z5893+GSjP3XffbTp06OCa79+/v/nd735nWrVq5Xqu/vGPfxhJZtOmTa56F475V199Ve4Hf0kYmTt3rlv5uHHjjI+PT6WBqTIlr//Zs2ebwMBAt+VVJYwYc/H7gyf69u1rJJk333zTrfx3v/udadSokdvrtbwwMm7cOLe2c+fONZJMXl6eMcaYv//970aSyc7O9rh/PyWcpqknHA6HhgwZ4lbWtWtX5eTkuOa7dOmimJgYLVu2zFW2b98+ffzxxxo9erSrLD09XVdeeaV+8YtfuC1v1KhRMsYoPT1dkvTOO+/Ix8fHrW1ZsrKydPPNNyswMFCNGzeWl5eX7rnnHhUXF+uzzz6r1vamp6crISFB4eHhpfr4/fffKyMjw6385ptvdpvv2rWrJLmNz8V4/fXX1atXLzVv3lxNmjSRl5eXli5dWuXDw2+//bbi4+MVFhams2fPuqaBAwdKkrZt2+ZWf/DgwWrcuHG52/PZZ5/pP//5j+677z75+PiUu97hw4crKChICxYscJX97W9/02WXXaY77rijSn1/8skntXPnzlLTr3/960rb/uIXv9Dx48c1fPhwvfnmmx7dxZGRkaEffvih1CmG8PBwXX/99XrvvfckSfv371d+fn6p/rRr1069evUqc9k333yzvLy8SpV78jzHx8erWbNmrvlOnTpJkgYOHOh2CrKkvLJ9MSEhQV988YUOHjyoU6dO6YMPPtAvf/lLxcfHKy0tTZK0efNmOZ1OXXfddRUuqzJlvV5OnTqlo0ePerys9PR09e/fX/7+/q7X//Tp0/X1119Xa3nlqer+4Ck/P79S43HXXXfp3Llz2r59e6XtK3vv6d69u7y9vfX73/9eL774or744otq9bOhI4zUE02bNi31oeN0OnXq1Cm3stGjRysjI0OffvqpJGnZsmVyOp0aPny4q87XX3+t0NDQUusICwtzPS5JX331lcLCwtSoUfm7SW5urnr37q0jR47omWee0fvvv6+dO3e6Pvx++OGHamxt1ftYIjAw0G2+5GLE6q7/x9asWaNf//rXatOmjVauXKmMjAzt3LlTo0ePLjX+5fnyyy/11ltvycvLy20quR7gwg/pyrbnq6++kqRKr+53Op26//77tWrVKh0/flxfffWVXnvtNf32t7+t8gWbHTp0UGxsbKmp5NqEiowYMUIpKSnKycnRsGHDFBQUpJ49e7o+XCtS8hyXtx+UPF7yNzg4uFS9ssrKW6anz3NAQIDbvLe3d4Xlle0r/fv3l3Q+cHzwwQc6c+aMrr/+evXv39/1Qbt582b16tVLvr6+FS6rMjX1evn44481YMAASeev2fnHP/6hnTt3atq0adVaXkWquj94qqx9JCQkxG2dFalsLKOiorR582YFBQVp/PjxioqKUlRUlJ555plq9beh4m6aBmb48OFKTEzU8uXL9fjjj+ull17S0KFD1apVK1edwMBA5eXllWpbctFV69atJUmXXXaZPvjgA507d67cQLJ27VqdPHlSa9asUfv27V3l2dnZF7UdVe1jXVi5cqUiIyO1evVqt/94Sy4erorWrVura9euevzxx8t8vCRkVVVJEPjvf/9bad2xY8dqzpw5SklJ0alTp3T27FmNGTPGo/VdjHvvvVf33nuvTp48qe3bt2vGjBm66aab9Nlnn7ntMxcqeZMvbz8o2QdK6n355Zel6uXn55e57Au/H0Wqmef5YrRt21YdO3bU5s2bFRERodjYWLVs2VIJCQkaN26cPvroI+3YscN12+6l4NVXX5WXl5fefvttt3+WyvquHafTWeZYVjVEVHV/8FRF+82FQaO6evfurd69e6u4uFiZmZn629/+poceekjBwcG68847a2Qd9R1HRhqYVq1aaejQoVqxYoXefvtt5efnlzrNkpCQoE8++UT//Oc/3cpL7rCIj4+XdP5w86lTp9yuZL9QyZv2j//LNsZoyZIlpeo6nc4q/6eUkJCg9PT0Ulelr1ixQk2bNq3T2+wcDoe8vb3dPqDy8/PLvMuivG286aab9O9//1tRUVFlHmXwNIx07NhRUVFRSklJqfTDMjQ0VLfffrsWLlyo5ORkDRkyRO3atfNofTWhWbNmGjhwoKZNm6bTp09r7969ksr/rzwuLk6+vr5auXKlW/l///tf12k8SYqOjlZISIhee+01t3q5ublud31VxpPnubb0799f6enpSktL0w033CDp/HPdrl07TZ8+XWfOnHEdQSlPTR4VrIzD4VCTJk3cTin+8MMPeumll0rVjYiI0O7du93K0tPT9d1337mVXez+4KkTJ05o3bp1bmWrVq1So0aN1KdPn2otszyNGzdWz549XUeOL3wP/injyEgDNHr0aK1evVoTJkxQ27ZtS715TZo0SStWrNDgwYM1e/ZstW/fXuvXr9fChQs1duxYdezYUdL5oyzLli3TmDFjtH//fsXHx+vcuXP66KOP1KlTJ91555264YYb5O3treHDh+vhhx/WqVOntGjRIn377bel+tWlSxetWbNGixYtUkxMjBo1aqTY2Ngyt2HGjBmu6yymT5+ugIAAvfzyy1q/fr3mzp0rf3//Gh2z9PT0Mr/xcdCgQbrpppu0Zs0ajRs3TrfddpsOHz6sxx57TKGhoTpw4ECpbdy6daveeusthYaGys/PT9HR0Zo9e7bS0tJ07bXXauLEiYqOjtapU6d06NAhbdiwQcnJyR5/odKCBQs0ZMgQXXPNNZo0aZLatWun3Nxcvfvuu3r55Zfd6j744IPq2bOnJLldU1Tbfve738nX11e9evVSaGio8vPzlZSUJH9/f1199dWS5PoGzMWLF8vPz08+Pj6KjIxUYGCg/vSnP2nq1Km65557NHz4cH399deaNWuWfHx8NGPGDElSo0aNNGvWLN1///267bbbNHr0aB0/flyzZs1SaGhohacZf8yT57m2JCQkaOHChTp27JjbtxUnJCRo2bJlatWqldttvWXx8/NT+/bt9eabbyohIUEBAQFq3bp1rXyB2ODBg/X000/rrrvu0u9//3t9/fXXmjdvXpmnAEeMGKE//elPmj59uvr27atPPvlEzz33XKnX8sXuD54KDAzU2LFjlZubq44dO2rDhg1asmSJxo4dWyOhPTk5Wenp6Ro8eLDatWunU6dOKSUlRZIqDZY/KbavoIW78u6madasWam6JVfGX6i4uNiEh4cbSWbatGllricnJ8fcddddJjAw0Hh5eZno6Gjz1FNPmeLiYrd6P/zwg5k+fbq54oorjLe3twkMDDTXX3+9+fDDD1113nrrLdOtWzfj4+Nj2rRpY/74xz+ad955p9QV8d9884257bbbTMuWLY3D4XDru8q40n7Pnj1myJAhxt/f33h7e5tu3bqZZcuWudUpuePj9ddfdysvuWviwvoXKhnv8qaDBw8aY87fZhkREWGcTqfp1KmTWbJkSZnjn52dbXr16mWaNm1qJJW6y2HixIkmMjLSeHl5mYCAABMTE2OmTZvmuiujpN9PPfVUqb6WNUYZGRlm4MCBxt/f3zidThMVFWUmTZpU5rZGRESYTp06VTgeP1be2JYYP358pXfTvPjiiyY+Pt4EBwcbb29vExYWZn7961+XurNk/vz5JjIy0jRu3LjU8/bCCy+Yrl27Gm9vb+Pv729uueUW191HP7Z48WJz+eWXG29vb9OxY0eTkpJibrnlFrc7YSoaX2Oq/jyrjFs/y1t2ZeP4Y99++61p1KiRadasmdvdPi+//LKRZG699dZSbS4cc2OM2bx5s+nRo4dxOp1GkutulZJt+eqrr9zql7wOSvZ3T6SkpJjo6GjjdDpNhw4dTFJSklm6dGmp5RUVFZmHH37YhIeHG19fX9O3b1+TnZ1d6m4aY2pmf6iKvn37ms6dO5utW7ea2NhY43Q6TWhoqJk6dao5c+aMW90LX3/l3YlX8nyXvPdlZGSYX/3qV6Z9+/bG6XSawMBA07dvX7Nu3bpq9bmhchhzwTfTAGhwdu/erW7dumnBggUaN26c7e7UiePHj6tjx44aOnSoFi9ebLs7ACpAGAEasP/85z/KycnR1KlTlZubq88//1xNmza13a0al5+fr8cff1zx8fEKDAxUTk6O/vrXv+rTTz9VZmZmqW8xBXBp4ZoRoAF77LHH9NJLL6lTp056/fXXG2QQkc5f9Hjo0CGNGzdO33zzjesi5+TkZIIIUA9wZAQAAFjFrb0AAMAqwggAALCKMAIAAKyqFxewnjt3Tv/73//k5+dX5tc4AwCAS48xRidOnKj0d87qRRj53//+V+rXWwEAQP1w+PDhCr9lul6EET8/P0nnN6ZFixaWewMAAKqisLBQ4eHhrs/x8tSLMFJyaqZFixaEEQAA6pnKLrHgAlYAAGAVYQQAAFhFGAEAAFYRRgAAgFWEEQAAYBVhBAAAWEUYAQAAVhFGAACAVYQRAABgFWEEAABY5VEYWbRokbp27er6Wva4uDi98847FbbZtm2bYmJi5OPjow4dOig5OfmiOgwAABoWj8JI27ZtNWfOHGVmZiozM1PXX3+9brnlFu3du7fM+gcPHtSgQYPUu3dvZWVlaerUqZo4caJSU1NrpPMAAKD+cxhjzMUsICAgQE899ZTuu+++Uo898sgjWrdunfbt2+cqGzNmjP71r38pIyOjyusoLCyUv7+/CgoK+KE8AADqiap+flf7mpHi4mK9+uqrOnnypOLi4sqsk5GRoQEDBriV3XjjjcrMzNSZM2fKXXZRUZEKCwvdJgAA0DA18bTBnj17FBcXp1OnTql58+Z64403dOWVV5ZZNz8/X8HBwW5lwcHBOnv2rI4dO6bQ0NAy2yUlJWnWrFmedq1aIh5dXyfrAerKoTmDbXcBADzi8ZGR6OhoZWdna8eOHRo7dqxGjhypTz75pNz6DofDbb7krNCF5T82ZcoUFRQUuKbDhw972k0AAFBPeHxkxNvbW5dffrkkKTY2Vjt37tQzzzyj559/vlTdkJAQ5efnu5UdPXpUTZo0UWBgYLnrcDqdcjqdnnYNAADUQxf9PSPGGBUVFZX5WFxcnNLS0tzKNm3apNjYWHl5eV3sqgEAQAPgURiZOnWq3n//fR06dEh79uzRtGnTtHXrVv3mN7+RdP70yj333OOqP2bMGOXk5CgxMVH79u1TSkqKli5dqsmTJ9fsVgAAgHrLo9M0X375pUaMGKG8vDz5+/ura9eu2rhxo2644QZJUl5ennJzc131IyMjtWHDBk2aNEkLFixQWFiYnn32WQ0bNqxmtwIAANRbF/09I3WhNr9nhLtp0NBwNw2AS0Wtf88IAABATSCMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALCKMAIAAKwijAAAAKsIIwAAwCrCCAAAsIowAgAArCKMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALCKMAIAAKwijAAAAKsIIwAAwCrCCAAAsIowAgAArCKMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALCKMAIAAKwijAAAAKsIIwAAwCrCCAAAsIowAgAArCKMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALCKMAIAAKwijAAAAKsIIwAAwCrCCAAAsMqjMJKUlKSrr75afn5+CgoK0tChQ7V///4K22zdulUOh6PU9Omnn15UxwEAQMPgURjZtm2bxo8frx07digtLU1nz57VgAEDdPLkyUrb7t+/X3l5ea7piiuuqHanAQBAw9HEk8obN250m1+2bJmCgoK0a9cu9enTp8K2QUFBatmypccdBAAADdtFXTNSUFAgSQoICKi0bo8ePRQaGqqEhARt2bKlwrpFRUUqLCx0mwAAQMNU7TBijFFiYqKuu+46XXXVVeXWCw0N1eLFi5Wamqo1a9YoOjpaCQkJ2r59e7ltkpKS5O/v75rCw8Or200AAHCJcxhjTHUajh8/XuvXr9cHH3ygtm3betR2yJAhcjgcWrduXZmPFxUVqaioyDVfWFio8PBwFRQUqEWLFtXpbrkiHl1fo8sDbDs0Z7DtLgCApPOf3/7+/pV+flfryMgDDzygdevWacuWLR4HEUm65pprdODAgXIfdzqdatGihdsEAAAaJo8uYDXG6IEHHtAbb7yhrVu3KjIyslorzcrKUmhoaLXaAgCAhsWjMDJ+/HitWrVKb775pvz8/JSfny9J8vf3l6+vryRpypQpOnLkiFasWCFJmj9/viIiItS5c2edPn1aK1euVGpqqlJTU2t4UwAAQH3kURhZtGiRJKlfv35u5cuWLdOoUaMkSXl5ecrNzXU9dvr0aU2ePFlHjhyRr6+vOnfurPXr12vQoEEX13MAANAgVPsC1rpU1QtgqoMLWNHQcAErgEtFrV7ACgAAUFMIIwAAwCrCCAAAsIowAgAArCKMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALCKMAIAAKwijAAAAKsIIwAAwCrCCAAAsIowAgAArCKMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALCKMAIAAKwijAAAAKsIIwAAwCrCCAAAsIowAgAArCKMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALCKMAIAAKwijAAAAKsIIwAAwCrCCAAAsIowAgAArCKMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALCKMAIAAKzyKIwkJSXp6quvlp+fn4KCgjR06FDt37+/0nbbtm1TTEyMfHx81KFDByUnJ1e7wwAAoGHxKIxs27ZN48eP144dO5SWlqazZ89qwIABOnnyZLltDh48qEGDBql3797KysrS1KlTNXHiRKWmpl505wEAQP3XxJPKGzdudJtftmyZgoKCtGvXLvXp06fMNsnJyWrXrp3mz58vSerUqZMyMzM1b948DRs2rHq9BgAADcZFXTNSUFAgSQoICCi3TkZGhgYMGOBWduONNyozM1Nnzpwps01RUZEKCwvdJgAA0DB5dGTkx4wxSkxM1HXXXaerrrqq3Hr5+fkKDg52KwsODtbZs2d17NgxhYaGlmqTlJSkWbNmVbdrwE9axKPrbXfhJ+HQnMG2uwA0GNU+MjJhwgTt3r1br7zySqV1HQ6H27wxpszyElOmTFFBQYFrOnz4cHW7CQAALnHVOjLywAMPaN26ddq+fbvatm1bYd2QkBDl5+e7lR09elRNmjRRYGBgmW2cTqecTmd1ugYAAOoZj46MGGM0YcIErVmzRunp6YqMjKy0TVxcnNLS0tzKNm3apNjYWHl5eXnWWwAA0OB4FEbGjx+vlStXatWqVfLz81N+fr7y8/P1ww8/uOpMmTJF99xzj2t+zJgxysnJUWJiovbt26eUlBQtXbpUkydPrrmtAAAA9ZZHYWTRokUqKChQv379FBoa6ppWr17tqpOXl6fc3FzXfGRkpDZs2KCtW7eqe/fueuyxx/Tss89yWy8AAJDk4TUjJReeVmT58uWlyvr27at//vOfnqwKAAD8RPDbNAAAwCrCCAAAsIowAgAArCKMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALCKMAIAAKwijAAAAKsIIwAAwCrCCAAAsIowAgAArCKMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALCKMAIAAKwijAAAAKsIIwAAwCrCCAAAsIowAgAArCKMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALCKMAIAAKwijAAAAKsIIwAAwCrCCAAAsIowAgAArCKMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALCKMAIAAKzyOIxs375dQ4YMUVhYmBwOh9auXVth/a1bt8rhcJSaPv300+r2GQAANCBNPG1w8uRJdevWTffee6+GDRtW5Xb79+9XixYtXPOXXXaZp6sGAAANkMdhZODAgRo4cKDHKwoKClLLli09bgcAABq2OrtmpEePHgoNDVVCQoK2bNlSYd2ioiIVFha6TQAAoGGq9TASGhqqxYsXKzU1VWvWrFF0dLQSEhK0ffv2ctskJSXJ39/fNYWHh9d2NwEAgCUOY4ypdmOHQ2+88YaGDh3qUbshQ4bI4XBo3bp1ZT5eVFSkoqIi13xhYaHCw8NVUFDgdt1JTYh4dH2NLg/AT8OhOYNtdwG45BUWFsrf37/Sz28rt/Zec801OnDgQLmPO51OtWjRwm0CAAANk5UwkpWVpdDQUBurBgAAlxiP76b57rvv9Pnnn7vmDx48qOzsbAUEBKhdu3aaMmWKjhw5ohUrVkiS5s+fr4iICHXu3FmnT5/WypUrlZqaqtTU1JrbCgAAUG95HEYyMzMVHx/vmk9MTJQkjRw5UsuXL1deXp5yc3Ndj58+fVqTJ0/WkSNH5Ovrq86dO2v9+vUaNGhQDXQfAADUdxd1AWtdqeoFMNXBBawAqoMLWIHKXdIXsAIAAJQgjAAAAKsIIwAAwCrCCAAAsIowAgAArCKMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALCKMAIAAKwijAAAAKsIIwAAwCrCCAAAsIowAgAArCKMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALCKMAIAAKwijAAAAKsIIwAAwCrCCAAAsIowAgAArCKMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALCKMAIAAKwijAAAAKsIIwAAwCrCCAAAsIowAgAArCKMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALDK4zCyfft2DRkyRGFhYXI4HFq7dm2lbbZt26aYmBj5+PioQ4cOSk5Ork5fAQBAA+RxGDl58qS6deum5557rkr1Dx48qEGDBql3797KysrS1KlTNXHiRKWmpnrcWQAA0PA08bTBwIEDNXDgwCrXT05OVrt27TR//nxJUqdOnZSZmal58+Zp2LBhnq4eAAA0MLV+zUhGRoYGDBjgVnbjjTcqMzNTZ86cKbNNUVGRCgsL3SYAANAweXxkxFP5+fkKDg52KwsODtbZs2d17NgxhYaGlmqTlJSkWbNm1XbXAKDaIh5db7sLQI05NGew1fXXyd00DofDbd4YU2Z5iSlTpqigoMA1HT58uNb7CAAA7Kj1IyMhISHKz893Kzt69KiaNGmiwMDAMts4nU45nc7a7hoAALgE1PqRkbi4OKWlpbmVbdq0SbGxsfLy8qrt1QMAgEucx2Hku+++U3Z2trKzsyWdv3U3Oztbubm5ks6fYrnnnntc9ceMGaOcnBwlJiZq3759SklJ0dKlSzV58uSa2QIAAFCveXyaJjMzU/Hx8a75xMRESdLIkSO1fPly5eXluYKJJEVGRmrDhg2aNGmSFixYoLCwMD377LPc1gsAACRJDlNyNeklrLCwUP7+/iooKFCLFi1qdNlcEQ8A+Kmrrbtpqvr5zW/TAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALCKMAIAAKwijAAAAKsIIwAAwCrCCAAAsIowAgAArCKMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALCKMAIAAKwijAAAAKsIIwAAwCrCCAAAsIowAgAArCKMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALCKMAIAAKwijAAAAKsIIwAAwCrCCAAAsIowAgAArCKMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALCKMAIAAKwijAAAAKsIIwAAwCrCCAAAsIowAgAArKpWGFm4cKEiIyPl4+OjmJgYvf/+++XW3bp1qxwOR6np008/rXanAQBAw+FxGFm9erUeeughTZs2TVlZWerdu7cGDhyo3NzcCtvt379feXl5rumKK66odqcBAEDD4XEYefrpp3Xffffpt7/9rTp16qT58+crPDxcixYtqrBdUFCQQkJCXFPjxo2r3WkAANBweBRGTp8+rV27dmnAgAFu5QMGDNCHH35YYdsePXooNDRUCQkJ2rJlS4V1i4qKVFhY6DYBAICGyaMwcuzYMRUXFys4ONitPDg4WPn5+WW2CQ0N1eLFi5Wamqo1a9YoOjpaCQkJ2r59e7nrSUpKkr+/v2sKDw/3pJsAAKAeaVKdRg6Hw23eGFOqrER0dLSio6Nd83FxcTp8+LDmzZunPn36lNlmypQpSkxMdM0XFhYSSAAAaKA8OjLSunVrNW7cuNRRkKNHj5Y6WlKRa665RgcOHCj3cafTqRYtWrhNAACgYfIojHh7eysmJkZpaWlu5Wlpabr22murvJysrCyFhoZ6smoAANBAeXyaJjExUSNGjFBsbKzi4uK0ePFi5ebmasyYMZLOn2I5cuSIVqxYIUmaP3++IiIi1LlzZ50+fVorV65UamqqUlNTa3ZLAABAveRxGLnjjjv09ddfa/bs2crLy9NVV12lDRs2qH379pKkvLw8t+8cOX36tCZPnqwjR47I19dXnTt31vr16zVo0KCa2woAAFBvOYwxxnYnKlNYWCh/f38VFBTU+PUjEY+ur9HlAQBQ3xyaM7hWllvVz29+mwYAAFhFGAEAAFYRRgAAgFWEEQAAYBVhBAAAWEUYAQAAVhFGAACAVYQRAABgFWEEAABYRRgBAABWEUYAAIBVhBEAAGAVYQQAAFhFGAEAAFYRRgAAgFWEEQAAYBVhBAAAWEUYAQAAVhFGAACAVYQRAABgFWEEAABYRRgBAABWEUYAAIBVhBEAAGAVYQQAAFhFGAEAAFYRRgAAgFWEEQAAYBVhBAAAWEUYAQAAVhFGAACAVYQRAABgFWEEAABYRRgBAABWEUYAAIBVhBEAAGAVYQQAAFhFGAEAAFYRRgAAgFWEEQAAYBVhBAAAWEUYAQAAVhFGAACAVYQRAABgFWEEAABYRRgBAABWEUYAAIBV1QojCxcuVGRkpHx8fBQTE6P333+/wvrbtm1TTEyMfHx81KFDByUnJ1erswAAoOHxOIysXr1aDz30kKZNm6asrCz17t1bAwcOVG5ubpn1Dx48qEGDBql3797KysrS1KlTNXHiRKWmpl505wEAQP3nMMYYTxr07NlTP//5z7Vo0SJXWadOnTR06FAlJSWVqv/II49o3bp12rdvn6tszJgx+te//qWMjIwqrbOwsFD+/v4qKChQixYtPOlupSIeXV+jywMAoL45NGdwrSy3qp/fTTxZ6OnTp7Vr1y49+uijbuUDBgzQhx9+WGabjIwMDRgwwK3sxhtv1NKlS3XmzBl5eXmValNUVKSioiLXfEFBgaTzG1XTzhV9X+PLBACgPqmNz9cfL7ey4x4ehZFjx46puLhYwcHBbuXBwcHKz88vs01+fn6Z9c+ePatjx44pNDS0VJukpCTNmjWrVHl4eLgn3QUAAFXgP792l3/ixAn5+/uX+7hHYaSEw+FwmzfGlCqrrH5Z5SWmTJmixMRE1/y5c+f0zTffKDAwsML1NCSFhYUKDw/X4cOHa/zUVEPCOFWOMaocY1Q1jFPlGCN3xhidOHFCYWFhFdbzKIy0bt1ajRs3LnUU5OjRo6WOfpQICQkps36TJk0UGBhYZhun0ymn0+lW1rJlS0+62mC0aNGCHboKGKfKMUaVY4yqhnGqHGP0/1V0RKSER3fTeHt7KyYmRmlpaW7laWlpuvbaa8tsExcXV6r+pk2bFBsbW+b1IgAA4KfF41t7ExMT9cILLyglJUX79u3TpEmTlJubqzFjxkg6f4rlnnvucdUfM2aMcnJylJiYqH379iklJUVLly7V5MmTa24rAABAveXxNSN33HGHvv76a82ePVt5eXm66qqrtGHDBrVv316SlJeX5/adI5GRkdqwYYMmTZqkBQsWKCwsTM8++6yGDRtWc1vRADmdTs2YMaPU6Sq4Y5wqxxhVjjGqGsapcoxR9Xj8PSMAAAA1id+mAQAAVhFGAACAVYQRAABgFWEEAABYRRgBAABWEUZqwfbt2zVkyBCFhYXJ4XBo7dq1bo8bYzRz5kyFhYXJ19dX/fr10969eytc5po1axQbG6uWLVuqWbNm6t69u1566SW3OosWLVLXrl1d3/wXFxend955p6Y3r0bYGqMfS0pKksPh0EMPPVQDW1Q7bI3TzJkz5XA43KaQkJCa3rwaYXNfOnLkiO6++24FBgaqadOm6t69u3bt2lWTm1cjbI1RREREqf3I4XBo/PjxNb2JNcLWOJ09e1b/93//p8jISPn6+qpDhw6aPXu2zp07V9ObeMkijNSCkydPqlu3bnruuefKfHzu3Ll6+umn9dxzz2nnzp0KCQnRDTfcoBMnTpS7zICAAE2bNk0ZGRnavXu37r33Xt1777169913XXXatm2rOXPmKDMzU5mZmbr++ut1yy23VPpiscHWGJXYuXOnFi9erK5du9bYNtUGm+PUuXNn5eXluaY9e/bU6LbVFFtj9O2336pXr17y8vLSO++8o08++UR/+ctfLsmfrrA1Rjt37nTbh0q+jfv222+v2Q2sIbbG6cknn1RycrKee+457du3T3PnztVTTz2lv/3tbzW+jZcsg1olybzxxhuu+XPnzpmQkBAzZ84cV9mpU6eMv7+/SU5O9mjZPXr0MP/3f/9XYZ1WrVqZF154waPl1rW6HqMTJ06YK664wqSlpZm+ffuaBx988GK6X2fqcpxmzJhhunXrdrFdrnN1OUaPPPKIue666y66z3XN5nvSgw8+aKKiosy5c+c87nddq8txGjx4sBk9erRbnVtvvdXcfffd1et8PcSRkTp28OBB5efna8CAAa4yp9Opvn376sMPP6zSMowxeu+997R//3716dOnzDrFxcV69dVXdfLkScXFxdVI3+tKbY/R+PHjNXjwYPXv379G+13XanucDhw4oLCwMEVGRurOO+/UF198UaP9rwu1OUbr1q1TbGysbr/9dgUFBalHjx5asmRJjW9Dbaur96TTp09r5cqVGj16dL389fXaHKfrrrtO7733nj777DNJ0r/+9S998MEHGjRoUM1uxCXM46+Dx8Up+QXjC3/lODg4WDk5ORW2LSgoUJs2bVRUVKTGjRtr4cKFuuGGG9zq7NmzR3FxcTp16pSaN2+uN954Q1deeWXNbkQtq80xevXVV/XPf/5TO3furPmO17HaHKeePXtqxYoV6tixo7788kv9+c9/1rXXXqu9e/eW+2vbl6LaHKMvvvhCixYtUmJioqZOnaqPP/5YEydOlNPpdPt9rktdbb8nlVi7dq2OHz+uUaNG1Ui/61ptjtMjjzyigoIC/exnP1Pjxo1VXFysxx9/XMOHD6/5DblEEUYsufA/A2NMpf8t+Pn5KTs7W999953ee+89JSYmqkOHDurXr5+rTnR0tLKzs3X8+HGlpqZq5MiR2rZtW70LJFLNj9Hhw4f14IMPatOmTfLx8anNrtep2tiXBg4c6KrbpUsXxcXFKSoqSi+++KISExNrfBtqW22M0blz5xQbG6snnnhCktSjRw/t3btXixYtqldhpERtvSeVWLp0qQYOHKiwsLCa7Hadq41xWr16tVauXKlVq1apc+fOys7O1kMPPaSwsDCNHDmytjblkkIYqWMldyTk5+crNDTUVX706NFSiftCjRo10uWXXy5J6t69u/bt26ekpCS3F763t7erTmxsrHbu3KlnnnlGzz//fA1vSe2prTHatWuXjh49qpiYGFf94uJibd++Xc8995zrv5b6orb3pR9r1qyZunTpogMHDtRM5+tIbY5RaGhoqZDfqVMnpaam1uAW1L662I9ycnK0efNmrVmzpmY7X4dqc5z++Mc/6tFHH9Wdd94p6fw/ADk5OUpKSvrJhBGuGaljkZGRCgkJcV1VLp0/l7pt2zZde+21Hi3LGKOioqKLrnOpqa0xSkhI0J49e5Sdne2aYmNj9Zvf/EbZ2dn1KohIdbsvFRUVad++fW5vwvVBbY5Rr169tH//frc6n332mesXzOuLutiPli1bpqCgIA0ePPii+2tLbY7T999/r0aN3D+OGzdu/JO6tZe7aWrBiRMnTFZWlsnKyjKSzNNPP22ysrJMTk6OMcaYOXPmGH9/f7NmzRqzZ88eM3z4cBMaGmoKCwtdyxgxYoR59NFHXfNPPPGE2bRpk/nPf/5j9u3bZ/7yl7+YJk2amCVLlrjqTJkyxWzfvt0cPHjQ7N6920ydOtU0atTIbNq0qe42vopsjdGFLvW7aWyN0x/+8AezdetW88UXX5gdO3aYm266yfj5+ZlDhw7V3cZXka0x+vjjj02TJk3M448/bg4cOGBefvll07RpU7Ny5cq62/gqsvl6Ky4uNu3atTOPPPJI3WzsRbA1TiNHjjRt2rQxb7/9tjl48KBZs2aNad26tXn44YfrbuMtI4zUgi1bthhJpaaRI0caY87fIjZjxgwTEhJinE6n6dOnj9mzZ4/bMvr27euqb4wx06ZNM5dffrnx8fExrVq1MnFxcebVV191azN69GjTvn174+3tbS677DKTkJBwSQYRY+yN0YUu9TBia5zuuOMOExoaary8vExYWJi59dZbzd69e2t7c6vF5r701ltvmauuuso4nU7zs5/9zCxevLg2N7XabI7Ru+++aySZ/fv31+Ym1ghb41RYWGgefPBB065dO+Pj42M6dOhgpk2bZoqKimp7ky8ZDmOMqdVDLwAAABXgmhEAAGAVYQQAAFhFGAEAAFYRRgAAgFWEEQAAYBVhBAAAWEUYAQAAVhFGAACAVYQRAABgFWEEAABYRRgBAABW/T/2iA1o+7GViQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "114.2704861164093\n",
      "CPU times: user 258 ms, sys: 39.5 ms, total: 298 ms\n",
      "Wall time: 1min 54s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "inv_start_time = time.time()\n",
    "invocation_number = 10\n",
    "# Real-time endpoint\n",
    "_latency_hist_plot(endpoint_name, invocation_number, sleep_time=1)\n",
    "inv_lapse_time = time.time() - inv_start_time\n",
    "print(inv_lapse_time)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "a53c2f1d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "open-llama-lmi-model-2023-06-02-00-16-24-723\n",
      "us-west-2\n"
     ]
    }
   ],
   "source": [
    "endpoint_name = predictor.endpoint_name\n",
    "print(endpoint_name)\n",
    "print(region)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1305440",
   "metadata": {},
   "source": [
    "## Step 5b: Analyze Inference Latency via CloudWatch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "9b31abc1",
   "metadata": {},
   "outputs": [],
   "source": [
    "# https://docs.aws.amazon.com/sagemaker/latest/dg/realtime-endpoints-test-endpoints.html\n",
    "import pandas as pd\n",
    "\n",
    "cw = boto3.client(\"cloudwatch\", region_name=region)\n",
    "\n",
    "\n",
    "def get_invocation_metrics_for_endpoint(endpoint_name, metric_name, start_time, end_time):\n",
    "    #     metric = \"Sum\"\n",
    "    metric = \"Average\"\n",
    "    metrics = cw.get_metric_statistics(\n",
    "        Namespace=\"AWS/SageMaker\",\n",
    "        MetricName=metric_name,\n",
    "        StartTime=start_time,\n",
    "        EndTime=end_time,\n",
    "        Period=1,\n",
    "        Statistics=[metric],\n",
    "        Dimensions=[\n",
    "            {\"Name\": \"EndpointName\", \"Value\": endpoint_name},\n",
    "            {\"Name\": \"VariantName\", \"Value\": \"AllTraffic\"},\n",
    "        ],\n",
    "    )\n",
    "    return (\n",
    "        pd.DataFrame(metrics[\"Datapoints\"])\n",
    "        .sort_values(\"Timestamp\")\n",
    "        .set_index(\"Timestamp\")\n",
    "        .drop(\"Unit\", axis=1)\n",
    "        .rename(columns={metric: metric_name})\n",
    "    )\n",
    "\n",
    "\n",
    "#     return metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "17e16745",
   "metadata": {},
   "outputs": [],
   "source": [
    "import datetime\n",
    "\n",
    "\n",
    "def plot_endpoint_metrics(start_time=None, end_time=None):\n",
    "    #    start_time = start_time or datetime.datetime.now() - datetime.timedelta(seconds=inv_lapse_time+60)\n",
    "    #    end_time = datetime.datetime.now()\n",
    "    model_metrics = get_invocation_metrics_for_endpoint(\n",
    "        endpoint_name, \"ModelLatency\", start_time, end_time\n",
    "    )\n",
    "    overhead_metrics = get_invocation_metrics_for_endpoint(\n",
    "        endpoint_name, \"OverheadLatency\", start_time, end_time\n",
    "    )\n",
    "    total_metrics = model_metrics.join(overhead_metrics)\n",
    "    total_metrics[\"ModelLatency\"] = total_metrics[\"ModelLatency\"] / 1000\n",
    "    total_metrics[\"OverheadLatency\"] = total_metrics[\"OverheadLatency\"] / 1000\n",
    "    #    total_metrics[\"TotalLatency in ms\"] = total_metrics[[\"ModelLatency\",\"OverheadLatency\"]].sum(axis=1)\n",
    "    #     total_metrics = total_metrics.drop(['ModelLatency', 'OverheadLatency'], axis=1)\n",
    "    total_metrics.plot()\n",
    "    return total_metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "9249d41e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2023-06-02 00:26:07.841647\n",
      "2023-06-02 00:23:13.571161\n"
     ]
    }
   ],
   "source": [
    "endtime = datetime.datetime.now()\n",
    "print(endtime)\n",
    "startime = endtime - datetime.timedelta(seconds=inv_lapse_time + 60)\n",
    "print(startime)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "3b43f54e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# wait for cloudwatch metrics to populate\n",
    "time.sleep(300)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "2b3f536a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjoAAAGiCAYAAADulWxzAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAABAlUlEQVR4nO3df1yV9f3/8eeRX/LzgCi/FMQSydKSbPmjmjbzR2XmatMNR7lM7WNpLM1yW6VtYWppKzetVuLKRLd+rG2N9FtLM7WMJH9rGikohBocUBEQ3t8/kEsPh1QUQy4f99vt3PK6rte5rvd1naPn2fv68XYYY4wAAABsqEVTNwAAAOB8IegAAADbIugAAADbIugAAADbIugAAADbIugAAADbIugAAADb8m7qBjSl6upq7du3T8HBwXI4HE3dHAAAcAaMMSotLVVMTIxatDh1n81FHXT27dun2NjYpm4GAAA4C7m5uWrXrt0pay7qoBMcHCyp5kCFhIQ0cWsAAMCZKCkpUWxsrPU7fioXddCpPV0VEhJC0AEAoJk5k8tOuBgZAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADY1kU9qCcA+zpaWaUDh8ol1Qz855BUO/6fQw45HJKjZsJzXp33OE4UWTW1gwmeXHPy+IJ15zlOWqe1/AwGJARwbgg6AJq96mqjrw8cVnZusdbvKVJ2brG2FZSqqto0ddPOSH1h6ORQJoe+P6id9B7Vnfc9oUweAaz+UHZy+04Z3E7TfvcAeFKArDd41g2Q9YdK1dP+72tr3e1+X1ulusf1pLbV+3mcuq2q95h5trUhn8f3fwfq+zxO/g6cybGupx11vwenaespj/VJ0zrF/3ycybEuO1yqM0XQAdDsfHe4Qtm5RcreU6z1ucXKzi1W6dFjHnW+Xi1q/mU8nneMjIypmTTGHJ8nmSbOQ8Zqn+ppTPMIa8APqbr8yBnXEnQAXNDKj1Vpy74SZR8PNOv3FGvPd57/yPl5t1DXtk4lxYWqW2yYusWFKsbZskGnh6zwc1IYMtayE0FJ3zPv5PdYdWdQY2qKPObVtqO2bdayMwpudffl+HZq/9zA/dFJ63Vv16n3p+563fbnpOPj3i73mjPan5OOQb37Us9nqnrb5XkMVHf/Tpo+ua3f/9m4z6tb/72fe53P1POzOXEMTnx/6q9x/5w9P6+6x8D6nE+zP+6fc/3H5fs/5/qOgedn6v7Z1MyrKDusXJ0Zgg6AC4YxRnu+O2IFmuzcYm3ZV6KKqmqP2kvaBCrpeKBJig1VYlSwfLzO7f6KuqcHTpwYAHAhKSkp0dLxZ1ZL0AHQZFxllfryeE9N7eu7wxUedWEBPuoWG6qkuDB1iw3VVe1C5QzwaYIWA2huCDoAfhDHqqq1raC05pqaPcXKzi3Srv2HPep8vBy6PMappNjQ4+EmVHGtArhDCcBZIegAaHTGGOW7jrrdBbVxr0tHKz1PQcW1ClC346GmW1yoLo8OUUsfryZoNQA7IugAOGeHy49pQ57LLdgUlpZ71AW39D4Rao6/woP8mqDFAC4WDb5yb+XKlbrtttsUExMjh8Ohd955x225MUZTp05VTEyM/P391bdvX23evNmtpry8XOPHj1fr1q0VGBioIUOGKC8vz62mqKhIKSkpcjqdcjqdSklJUXFxsVvNnj17dNtttykwMFCtW7fWhAkTVFHheX4fQOOpqjbaXlCqJev2aMpbGzTouZXqOvV9/fLltZqRuU3LtnyrwtJyebVw6IqYEI3oEadZP7tS/++hH+vLxwfotVE9NHFAovp1jiTkADjvGtyjc/jwYV111VX69a9/rTvvvNNj+cyZMzV79mylp6erU6dO+uMf/6j+/ftr+/btCg4OliSlpqbqX//6lzIyMhQeHq6JEydq8ODBysrKkpdXTZd1cnKy8vLylJmZKUkaM2aMUlJS9K9//UuSVFVVpVtvvVVt2rTRqlWrdPDgQd19990yxuiFF1446wMCwF1h6dHj19TUvDbkuXSo3POZNdHOlsdv7a65vbtrW6f8fTkFBaBpOczJDx5o6JsdDr399tsaOnSopJrenJiYGKWmpuqRRx6RVNN7ExkZqRkzZmjs2LFyuVxq06aNXnvtNQ0fPlyStG/fPsXGxuq9997TwIEDtXXrVl1++eVau3atevToIUlau3atevXqpW3btikxMVH//e9/NXjwYOXm5iomJkaSlJGRoZEjR6qwsFAhISGnbX9JSYmcTqdcLtcZ1QN2d7SySpv2Hj8Fdfyi4b3FZR51Ab5ex59ZE2ZdMBwZ0rIJWgzgYtSQ3+9GvUYnJydHBQUFGjBggDXPz89Pffr00erVqzV27FhlZWWpsrLSrSYmJkZdunTR6tWrNXDgQK1Zs0ZOp9MKOZLUs2dPOZ1OrV69WomJiVqzZo26dOlihRxJGjhwoMrLy5WVlaUbb7zRo33l5eUqLz9x3UBJSUlj7j7QrBhjlHPgsPW8muzcYm3NL9GxOsMmOBxSQkSQ2+3dCRFB8j7HZ9YAwA+hUYNOQUGBJCkyMtJtfmRkpHbv3m3V+Pr6KiwszKOm9v0FBQWKiIjwWH9ERIRbTd3thIWFydfX16qpa/r06Zo2bdpZ7BnQ/BUdrlB23okH8X2ZWyxXWaVHXesgP6uXJik2VF3bORXckmfWAGiezstdV3Wfd2GMOe0zMOrW1Fd/NjUnmzJlih566CFruqSkRLGxsadsF9AcVRyr1tb8Ere7oL45WP+wCV3aOq07oJLiQtU21J9n1gCwjUYNOlFRUZJqeluio6Ot+YWFhVbvS1RUlCoqKlRUVOTWq1NYWKjevXtbNd9++63H+vfv3++2nk8//dRteVFRkSorKz16emr5+fnJz4+7PGAvxhjlFZVZ19Sszy3S5n0lqjhWz7AJrQOt59UkxYbpsuhzHzYBAC5kjRp0OnTooKioKC1fvlxJSUmSpIqKCq1YsUIzZsyQJHXv3l0+Pj5avny5hg0bJknKz8/Xpk2bNHPmTElSr1695HK59Nlnn+naa6+VJH366adyuVxWGOrVq5eeeuop5efnW6Fq2bJl8vPzU/fu3Rtzt4ALSsnRSm3IddWM3n382poDhzwfqxB6fNiEk1+hAb5N0GIAaDoNDjqHDh3Szp07remcnBxlZ2erVatWiouLU2pqqtLS0pSQkKCEhASlpaUpICBAycnJkiSn06lRo0Zp4sSJCg8PV6tWrTRp0iR17dpVN910kySpc+fOGjRokEaPHq0XX3xRUs3t5YMHD1ZiYqIkacCAAbr88suVkpKiWbNm6bvvvtOkSZM0evRo7qCCbRyrqtb2b0trAs3xa2t27j+kuvdK+ng5dHl0iNVb0y02TPHhDJsAAA0OOp9//rnbHU2117zcfffdSk9P1+TJk1VWVqZx48apqKhIPXr00LJly6xn6EjSnDlz5O3trWHDhqmsrEz9+vVTenq69QwdSVq0aJEmTJhg3Z01ZMgQzZ0711ru5eWl//znPxo3bpyuu+46+fv7Kzk5Wc8880zDjwJwgShwHbWuqVmfW6yNeS6VVVZ51LUL87fugOoWG6orYhg2AQDqc07P0WnueI4OmtKRimPamOc6aZDLYhWUHPWoC/bz1pWxTiXFHh+5OzZUbYK51gzAxavJnqMDoH7V1Ua79h/S+twTt3fv+LZUVXWeWdPCISVGhbjd3n1pmyC1aMEpKAA4GwQd4Dw4cKjc6qVZn1ukDbkuldYzbEJUSMuT7oKqeWZNgC9/LQGgsfAvKnCOjlZWafM+92fW5BV5Dpvg7+Olru2cSqq9CyouVNFO/yZoMQBcPAg6QAMYY/TNwSM1t3Yf77HZkl+iyirPYRM6tgk66S6oUCVGBjNsAgD8wAg6wCkUH6mwnlVTO2xC0ZH6hk3wPel5NWG6MtapEIZNAIAmR9ABjqusqta2/FKtP6m35usDhz3qfL1bqEtMiLrFhlnX1rQLY9gEALgQEXRwUTLGaG9xmdvI3Zv2ulRez7AJ8eEBbs+s6RwdIl9vTkEBQHNA0MFF4VD5MW04/hC+2nBz4FC5R53T30dXxdb00nSLC1W3dqEKC2TYBABorgg6sJ2qaqMdJw2bsD63SF8Veg6b4N3Coc7RJ55Z0y02VB1aB3IKCgBshKCDZu/bkqMnnYIq0oY8l45UeA6b0DbU37qmJikuVFfEOBk2AQBsjqCDZqWsokob9540cveeYu1zeQ6bEOTnrSvbOU/cCRUXqojglk3QYgBAUyLo4IJVXW309YFDbhcMbyuof9iETpHB1umnpLgwXdomSF4MmwAAFz2CDi4YBw+Vuz2zJju3WKVHPYdNiAj2Ox5qau6EurKdU4F+fJUBAJ74dUCTKD9WpS37Stx6a/Z8d8SjrqVPC3Vt63S7vTva2ZILhgEAZ4Sgg/POGKM93x05PhZUzS3eW/eVqKLK85k1l7YJdAs1iVHB8mHYBADAWSLooNG5yir1ZZ1TUN8drvCoaxVYM2xC7TNrrmwXKqc/wyYAABoPQQfnpLKqWtsLSrX++B1Q2blF2rW/nmETvFro8pgTz6xJig1TbCuGTQAAnF8EHZwxY4z2uY5agSY7t1gb97p0tNLzFFT78ADr9FNSXJg6RwfLz5tn1gAAflgEHXyvQ+XHtCGv2HpeTXZusQpLPYdNCGnp7TZswlXtQhUe5NcELQYAwB1BB5Jqhk3YWXhI6/cUWdfV7Pi2VHUeWSOvFg51jg4+3ltTc9HwJa0D1YJn1gAALkAEnYtUYelRq5dm/Z5ibcgr1uF6hk2IcbY8cRdUXKi6xDjl78spKABA80DQuQgcrazSpr2umlBz/DTU3uIyj7oAXy9d2e7EM2uSYkMVEcKwCQCA5ougYzPV1UY5Bw9bvTXZucXaml+iY3XOQTkcUqeI4BMjd8eFKiEimGETAAC2QtBp5ooOV5zoqcktVvaeIpXUM2xCm2C/k+6CqnlmTRDDJgAAbI5fumak4li1tuSXKPukC4a/Oeg5bIKfd82wCbXX1STFhSmGYRMAABchgs4FyhijvKIyfXFSqNm8r0QVxzyfWXNJm0DrmpqkuDCGTQAA4DiCzgWi5GilNuS6lJ1bZA10ebCeYRPCAnxO3NodF6pu7ULlDGDYBAAA6kPQaQLHqqq1/dtS69bu7Nxi7dp/SKbOM2t8vBy6PMZZ8yC+46/24QGcggIA4AwRdH4A+a4yt2fWbNzrUlml5zNrYlv5Kyn2xDNrLo8OUUsfnlkDAMDZIug0siMVx7Qhz2UNm7A+t0jflngOmxDsd3zYhLianpqrYkPVmmETAABoVASdc1BdbbRz/6Hjgaamx2Z7QUm9wyYkRgbX3AF1PNxc0jqIYRMAADjPCDoNcOBQudVLk51brA25LpWWez6zJtrZ0m3k7i5tQxTgy6EGAOCHxq/v9zhaWaXN+0rcBrnMK6p/2ISubZ3He2vClBQXqkiGTQAA4IJA0FHNM2tyDhx2u7V7a36JKqs8h01IiAiybu9OigtVQkSQvHlmDQAAFySCjqTrZ3yo0mpfj/mtg/xOjAUVG6or2zkV3JJn1gAA0FwQdCS5yo6pZWDLE8MmHH+1C/PnmTUAADRjBB1JGWN66JqEdvL15hQUAAB2wi+7pC5tQwk5AADYEL/uAADAtgg6AADAtgg6AADAtgg6AADAtgg6AADAtgg6AADAtgg6AADAtgg6AADAtgg6AADAtgg6AADAtgg6AADAtgg6AADAtgg6AADAtgg6AADAtgg6AADAtgg6AADAtho96Bw7dky///3v1aFDB/n7++uSSy7Rk08+qerqaqvGGKOpU6cqJiZG/v7+6tu3rzZv3uy2nvLyco0fP16tW7dWYGCghgwZory8PLeaoqIipaSkyOl0yul0KiUlRcXFxY29SwAAoJlq9KAzY8YMzZ8/X3PnztXWrVs1c+ZMzZo1Sy+88IJVM3PmTM2ePVtz587VunXrFBUVpf79+6u0tNSqSU1N1dtvv62MjAytWrVKhw4d0uDBg1VVVWXVJCcnKzs7W5mZmcrMzFR2drZSUlIae5cAAEAz5TDGmMZc4eDBgxUZGalXXnnFmnfnnXcqICBAr732mowxiomJUWpqqh555BFJNb03kZGRmjFjhsaOHSuXy6U2bdrotdde0/DhwyVJ+/btU2xsrN577z0NHDhQW7du1eWXX661a9eqR48ekqS1a9eqV69e2rZtmxITE0/b1pKSEjmdTrlcLoWEhDTmYQAAAOdJQ36/G71H5/rrr9cHH3ygHTt2SJK+/PJLrVq1SrfccoskKScnRwUFBRowYID1Hj8/P/Xp00erV6+WJGVlZamystKtJiYmRl26dLFq1qxZI6fTaYUcSerZs6ecTqdVU1d5eblKSkrcXgAAwL68G3uFjzzyiFwuly677DJ5eXmpqqpKTz31lH75y19KkgoKCiRJkZGRbu+LjIzU7t27rRpfX1+FhYV51NS+v6CgQBERER7bj4iIsGrqmj59uqZNm3ZuOwgAAJqNRu/RWbJkiV5//XW98cYb+uKLL7Rw4UI988wzWrhwoVudw+FwmzbGeMyrq25NffWnWs+UKVPkcrmsV25u7pnuFgAAaIYavUfn4Ycf1qOPPqpf/OIXkqSuXbtq9+7dmj59uu6++25FRUVJqumRiY6Ott5XWFho9fJERUWpoqJCRUVFbr06hYWF6t27t1Xz7bffemx///79Hr1Ftfz8/OTn59c4OwoAAC54jd6jc+TIEbVo4b5aLy8v6/byDh06KCoqSsuXL7eWV1RUaMWKFVaI6d69u3x8fNxq8vPztWnTJqumV69ecrlc+uyzz6yaTz/9VC6Xy6oBAAAXt0bv0bntttv01FNPKS4uTldccYXWr1+v2bNn65577pFUc7opNTVVaWlpSkhIUEJCgtLS0hQQEKDk5GRJktPp1KhRozRx4kSFh4erVatWmjRpkrp27aqbbrpJktS5c2cNGjRIo0eP1osvvihJGjNmjAYPHnxGd1wBAAD7a/Sg88ILL+ixxx7TuHHjVFhYqJiYGI0dO1aPP/64VTN58mSVlZVp3LhxKioqUo8ePbRs2TIFBwdbNXPmzJG3t7eGDRumsrIy9evXT+np6fLy8rJqFi1apAkTJlh3Zw0ZMkRz585t7F0CAADNVKM/R6c54Tk6AAA0P036HB0AAIALBUEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADY1nkJOnv37tWvfvUrhYeHKyAgQN26dVNWVpa13BijqVOnKiYmRv7+/urbt682b97sto7y8nKNHz9erVu3VmBgoIYMGaK8vDy3mqKiIqWkpMjpdMrpdColJUXFxcXnY5cAAEAz1OhBp6ioSNddd518fHz03//+V1u2bNGzzz6r0NBQq2bmzJmaPXu25s6dq3Xr1ikqKkr9+/dXaWmpVZOamqq3335bGRkZWrVqlQ4dOqTBgwerqqrKqklOTlZ2drYyMzOVmZmp7OxspaSkNPYuAQCA5so0skceecRcf/3137u8urraREVFmaefftqad/ToUeN0Os38+fONMcYUFxcbHx8fk5GRYdXs3bvXtGjRwmRmZhpjjNmyZYuRZNauXWvVrFmzxkgy27ZtO6O2ulwuI8m4XK4G7SMAAGg6Dfn9bvQenXfffVfXXHONfv7znysiIkJJSUl6+eWXreU5OTkqKCjQgAEDrHl+fn7q06ePVq9eLUnKyspSZWWlW01MTIy6dOli1axZs0ZOp1M9evSwanr27Cmn02nV1FVeXq6SkhK3FwAAsK9GDzpff/215s2bp4SEBL3//vu67777NGHCBP3tb3+TJBUUFEiSIiMj3d4XGRlpLSsoKJCvr6/CwsJOWRMREeGx/YiICKumrunTp1vX8zidTsXGxp7bzgIAgAtaowed6upqXX311UpLS1NSUpLGjh2r0aNHa968eW51DofDbdoY4zGvrro19dWfaj1TpkyRy+WyXrm5uWe6WwAAoBlq9KATHR2tyy+/3G1e586dtWfPHklSVFSUJHn0uhQWFlq9PFFRUaqoqFBRUdEpa7799luP7e/fv9+jt6iWn5+fQkJC3F4AAMC+vBt7hdddd522b9/uNm/Hjh1q3769JKlDhw6KiorS8uXLlZSUJEmqqKjQihUrNGPGDElS9+7d5ePjo+XLl2vYsGGSpPz8fG3atEkzZ86UJPXq1Usul0ufffaZrr32WknSp59+KpfLpd69ezf2bgGAbVRVVamysrKpmwF8Lx8fH3l5eTXKuho96PzmN79R7969lZaWpmHDhumzzz7TSy+9pJdeeklSzemm1NRUpaWlKSEhQQkJCUpLS1NAQICSk5MlSU6nU6NGjdLEiRMVHh6uVq1aadKkSeratatuuukmSTW9RIMGDdLo0aP14osvSpLGjBmjwYMHKzExsbF3CwCaPWOMCgoKeN4YmoXQ0FBFRUWd9rKW02n0oPOjH/1Ib7/9tqZMmaInn3xSHTp00HPPPacRI0ZYNZMnT1ZZWZnGjRunoqIi9ejRQ8uWLVNwcLBVM2fOHHl7e2vYsGEqKytTv379lJ6e7pbwFi1apAkTJlh3Zw0ZMkRz585t7F0CAFuoDTkREREKCAg45x8Q4HwwxujIkSMqLCyUVHNJzLlwGGNMYzSsOSopKZHT6ZTL5eJ6HQC2VlVVpR07digiIkLh4eFN3RzgtA4ePKjCwkJ16tTJ4zRWQ36/GesKAC4CtdfkBAQENHFLgDNT+1091+vJCDoAcBHhdBWai8b6rhJ0AACAbRF0AAAXtY8++kgOh6NBd6PFx8frueeeO29tQuMh6AAALmgjR46Uw+HQfffd57Fs3LhxcjgcGjly5A/fsJOca/AZOXKkhg4d2mjtwQkEHQDABS82NlYZGRkqKyuz5h09elSLFy9WXFxcE7YMFzqCDgDggnf11VcrLi5Ob731ljXvrbfeUmxsrPWUfUkqLy/XhAkTFBERoZYtW+r666/XunXr3Nb13nvvqVOnTvL399eNN96ob775xmN7q1ev1o9//GP5+/srNjZWEyZM0OHDh8+q7VVVVRo1apQ6dOggf39/JSYm6k9/+pO1fOrUqVq4cKH++c9/yuFwyOFw6KOPPpIk7d27V8OHD1dYWJjCw8N1++23u7W3tifomWeeUXR0tMLDw3X//fe73alUXl6uyZMnKzY2Vn5+fkpISNArr7wiY4w6duyoZ555xq29mzZtUosWLbRr166z2t8LDUEHAC5SxhgdqTjWJK+zeYTbr3/9ay1YsMCafvXVV3XPPfe41UyePFlvvvmmFi5cqC+++EIdO3bUwIED9d1330mScnNzdccdd+iWW25Rdna27r33Xj366KNu69i4caMGDhyoO+64Qxs2bNCSJUu0atUqPfDAA2dxlGsGu27Xrp2WLl2qLVu26PHHH9dvf/tbLV26VJI0adIkDRs2TIMGDVJ+fr7y8/PVu3dvHTlyRDfeeKOCgoK0cuVKrVq1SkFBQRo0aJAqKiqs9f/vf//Trl279L///U8LFy5Uenq60tPTreV33XWXMjIy9Pzzz2vr1q2aP3++goKC5HA4dM8997gd09rjesMNN+jSSy89q/290DT6k5EBAM1DWWWVLn/8/SbZ9pYnByrAt2E/QSkpKZoyZYq++eYbORwOffLJJ8rIyLB6Pw4fPqx58+YpPT1dN998syTp5Zdf1vLly/XKK6/o4Ycf1rx583TJJZdozpw5cjgcSkxM1MaNG62xFiVp1qxZSk5OVmpqqiQpISFBzz//vPr06aN58+apZcuWDWq3j4+Ppk2bZk136NBBq1ev1tKlSzVs2DAFBQXJ399f5eXl1sDXkvT666+rRYsW+utf/2rdar1gwQKFhobqo48+skYFCAsL09y5c+Xl5aXLLrtMt956qz744AONHj1aO3bs0NKlS7V8+XJrCKVLLrnE2savf/1rPf7449a4kZWVlXr99dc1a9asBu3jhYygAwBoFlq3bq1bb71VCxculDFGt956q1q3bm0t37VrlyorK3XddddZ83x8fHTttddq69atkqStW7eqZ8+ebs9o6dWrl9t2srKytHPnTi1atMiaZ4xRdXW1cnJy1Llz5wa3ff78+frrX/+q3bt3q6ysTBUVFerWrdsp31PbjpOHR5Jqrk06+bTSFVdc4fbk4OjoaG3cuFGSlJ2dLS8vL/Xp06febURHR+vWW2/Vq6++qmuvvVb//ve/dfToUf385z9v8D5eqAg6AHCR8vfx0pYnBzbZts/GPffcY51C+vOf/+y2rPZ0WN0HzRljrHlncsqsurpaY8eO1YQJEzyWnc2Fz0uXLtVvfvMbPfvss+rVq5eCg4M1a9Ysffrpp6dtR/fu3d0CV602bdpYf/bx8XFb5nA4VF1dLUny9/c/bfvuvfdepaSkaM6cOVqwYIGGDx9uqydoE3QA4CLlcDgafPqoqZ18fcrAge4hrWPHjvL19dWqVauUnJwsqWb4gM8//9w6DXX55ZfrnXfecXvf2rVr3aavvvpqbd68WR07dmyUNn/88cfq3bu3xo0bZ82re6Gvr6+vqqqqPNqxZMkSRUREnPV4jF27dlV1dbVWrFhhnbqq65ZbblFgYKDmzZun//73v1q5cuVZbetCxcXIAIBmw8vLS1u3btXWrVs9BnoMDAzU//3f/+nhhx9WZmamtmzZotGjR+vIkSMaNWqUJOm+++7Trl279NBDD2n79u1644033C7claRHHnlEa9as0f3336/s7Gx99dVXevfddzV+/PhTtm3v3r3Kzs52e3333Xfq2LGjPv/8c73//vvasWOHHnvsMY87weLj47VhwwZt375dBw4cUGVlpUaMGKHWrVvr9ttv18cff6ycnBytWLFCDz74oPLy8s7oeMXHx+vuu+/WPffco3feeUc5OTn66KOPrAuha4/pyJEjNWXKFHXs2NHjVF5zR9ABADQrISEh39vD8fTTT+vOO+9USkqKrr76au3cuVPvv/++wsLCJNWcenrzzTf1r3/9S1dddZXmz5+vtLQ0t3VceeWVWrFihb766ivdcMMNSkpK0mOPPabo6OhTtuuZZ55RUlKS2+vdd9/VfffdpzvuuEPDhw9Xjx49dPDgQbfeHUkaPXq0EhMTdc0116hNmzb65JNPFBAQoJUrVyouLk533HGHOnfurHvuuUdlZWUN6uGZN2+efvazn2ncuHG67LLLNHr0aI9b5UeNGqWKigqPu9jswGHO5h4/m2jIMO8A0JwdPXpUOTk56tChQ4PvGoL9ffLJJ+rbt6/y8vIUGRnZ1M2RdOrvbEN+v5vXyVkAANBoysvLlZubq8cee0zDhg27YEJOY+LUFQAAF6nFixcrMTFRLpdLM2fObOrmnBcEHQAALlIjR45UVVWVsrKy1LZt26ZuznlB0AEAALZF0AEAALZF0AEAALZF0AEAALZF0AEAALZF0AEAALZF0AEA4BRGjhypoUOHNsm2+/btaw1IirND0AEAXPByc3M1atQoxcTEyNfXV+3bt9eDDz6ogwcPNnXTflDnGnymTp2qbt26NVp7mgOCDgDggvb111/rmmuu0Y4dO7R48WLt3LlT8+fP1wcffKBevXrpu+++O2/brqysPG/rxg+DoAMAuKDdf//98vX11bJly9SnTx/FxcXp5ptv1v/7f/9Pe/fu1e9+9ztNmTJFPXv29HjvlVdeqSeeeMKaXrBggTp37qyWLVvqsssu01/+8hdr2TfffCOHw6GlS5eqb9++atmypV5//XVr+TPPPKPo6GiFh4fr/vvvdwtBFRUVmjx5stq2bavAwED16NFDH330kbX84MGD+uUvf6l27dopICBAXbt21eLFi93aevjwYd11110KCgpSdHS0nn322QYfq0ceeUSdOnVSQECALrnkEj322GNWO9PT0zVt2jR9+eWXcjgccjgcSk9PlyS5XC6NGTNGERERCgkJ0U9+8hN9+eWX1npre4Jee+01xcfHy+l06he/+IVKS0utmurqas2YMUMdO3aUn5+f4uLi9NRTT0mSfvKTn+iBBx5wa+vBgwfl5+enDz/8sMH72SDmIuZyuYwk43K5mropAHBelZWVmS1btpiysrITM6urjSk/1DSv6uozavfBgweNw+EwaWlp9S4fPXq0CQsLMxs2bDCSzM6dO61lmzZtMpLM9u3bjTHGvPTSSyY6Otq8+eab5uuvvzZvvvmmadWqlUlPTzfGGJOTk2Mkmfj4eKtm79695u677zYhISHmvvvuM1u3bjX/+te/TEBAgHnppZesbSUnJ5vevXublStXmp07d5pZs2YZPz8/s2PHDmOMMXl5eWbWrFlm/fr1ZteuXeb55583Xl5eZu3atdY6/u///s+0a9fOLFu2zGzYsMEMHjzYBAUFmQcffNCq6dOnj9t0XX/4wx/MJ598YnJycsy7775rIiMjzYwZM4wxxhw5csRMnDjRXHHFFSY/P9/k5+ebI0eOmOrqanPdddeZ2267zaxbt87s2LHDTJw40YSHh5uDBw8aY4x54oknTFBQkLnjjjvMxo0bzcqVK01UVJT57W9/a2178uTJJiwszKSnp5udO3eajz/+2Lz88svGGGMWLVpkwsLCzNGjR636P/3pTyY+Pt5Uf893od7v7HEN+f1m9HIAuFhVHpHSYppm27/dJ/kGnrbsq6++kjFGnTt3rnd5586dVVRUpMjISF155ZV644039Nhjj0mSFi1apB/96Efq1KmTJOkPf/iDnn32Wd1xxx2SpA4dOmjLli168cUXdffdd1vrTE1NtWpqhYWFae7cufLy8tJll12mW2+9VR988IFGjx6tXbt2afHixcrLy1NMTM3xnDRpkjIzM7VgwQKlpaWpbdu2mjRpkrW+8ePHKzMzU3//+9/Vo0cPHTp0SK+88or+9re/qX///pKkhQsXql27dmd6RCVJv//9760/x8fHa+LEiVqyZIkmT54sf39/BQUFydvbW1FRUVbdhx9+qI0bN6qwsFB+fn6Sanqv3nnnHf3jH//QmDFjJNX02KSnpys4OFiSlJKSog8++EBPPfWUSktL9ac//Ulz5861juWll16q66+/XpJ05513avz48frnP/+pYcOGSarpXRs5cqQcDkeD9rGhCDoAgGbLGCNJcjgcGjFihF599VU99thjMsZo8eLF1oW7+/fvty5oHj16tPX+Y8eOyel0uq3zmmuu8djOFVdcIS8vL2s6OjpaGzdulCR98cUXMsZYgapWeXm5wsPDJUlVVVV6+umntWTJEu3du1fl5eUqLy9XYGBN2Nu1a5cqKirUq1cv6/2tWrVSYmJig47HP/7xDz333HPauXOnDh06pGPHjikkJOSU78nKytKhQ4esttYqKyvTrl27rOn4+Hgr5NQeg8LCQknS1q1bVV5ern79+tW7DT8/P/3qV7/Sq6++qmHDhik7O1tffvml3nnnnQbt39kg6ADAxconoKZnpam2fQY6duwoh8OhLVu21HuL97Zt2xQWFqbWrVsrOTlZjz76qL744guVlZUpNzdXv/jFLyTV9EZI0ssvv6wePXq4rePkACPJCh9uzfXxcZt2OBzWOqurq+Xl5aWsrCyPdQUFBUmSnn32Wc2ZM0fPPfecunbtqsDAQKWmpqqiokLSicB2LtauXatf/OIXmjZtmgYOHCin06mMjIzTXutTXV2t6Ohot2uKaoWGhlp/PtUx8Pf3P2377r33XnXr1k15eXl69dVX1a9fP7Vv3/70O3aOCDoAcLFyOM7o9FFTCg8PV//+/fWXv/xFv/nNb9x+UAsKCrRo0SLdddddcjgcateunX784x9r0aJFKisr00033aTIyEhJUmRkpNq2bauvv/5aI0aMaNQ2JiUlqaqqSoWFhbrhhhvqrfn44491++2361e/+pWkmnDx1VdfWafkOnbsKB8fH61du1ZxcXGSpKKiIu3YsUN9+vQ5o3Z88sknat++vX73u99Z83bv3u1W4+vrq6qqKrd5V199tQoKCuTt7a34+Pgz2lZdCQkJ8vf31wcffKB777233pquXbvqmmuu0csvv6w33nhDL7zwwlltq6EIOgCAC9rcuXPVu3dvDRw4UH/84x/VoUMHbd68WQ8//LDatm1r3dkjSSNGjNDUqVNVUVGhOXPmuK1n6tSpmjBhgkJCQnTzzTervLxcn3/+uYqKivTQQw+ddfs6deqkESNG6K677tKzzz6rpKQkHThwQB9++KG6du2qW265RR07dtSbb76p1atXKywsTLNnz1ZBQYEVdIKCgjRq1Cg9/PDDCg8PV2RkpH73u9+pRQvPm6P379+v7Oxst3lRUVHq2LGj9uzZo4yMDP3oRz/Sf/7zH7399ttudfHx8crJyVF2drbatWun4OBg3XTTTerVq5eGDh2qGTNmKDExUfv27dN7772noUOH1nsqr66WLVvqkUce0eTJk+Xr66vrrrtO+/fv1+bNmzVq1Cir7t5779UDDzyggIAA/fSnPz2Lo30WTnu5so1x1xWAi8Wp7mBpDr755hszcuRIExUVZXx8fExsbKwZP368OXDggFtdUVGR8fPzMwEBAaa0tNRjPYsWLTLdunUzvr6+JiwszPz4xz82b731ljHmxF1X69evd3vP3XffbW6//Xa3eQ8++KDp06ePNV1RUWEef/xxEx8fb3x8fExUVJT56U9/ajZs2GCMqbl77PbbbzdBQUEmIiLC/P73vzd33XWX23pLS0vNr371KxMQEGAiIyPNzJkzPe6y6tOnj5Hk8XriiSeMMcY8/PDDJjw83AQFBZnhw4ebOXPmGKfTab3/6NGj5s477zShoaFGklmwYIExxpiSkhIzfvx4ExMTYx3fESNGmD179hhjau66uuqqq9yOwZw5c0z79u2t6aqqKvPHP/7RtG/f3vj4+Ji4uDiPu+VKS0tNQECAGTdunMdnU1dj3XXlMKYRTgw2UyUlJXI6nXK5XKe9WAsAmrOjR48qJydHHTp0UMuWLZu6ObhI5ebmKj4+XuvWrdPVV199ytpTfWcb8vvNqSsAAHBeVVZWKj8/X48++qh69ux52pDTmHgyMgAAOK9qL5TOysrS/Pnzf9Bt06MDAADOq759+zbKLfRngx4dAABgWwQdALiIXMT3n6CZaazvKkEHAC4CtU+1PXLkSBO3BDgztd/Vuk9kbiiu0QGAi4CXl5dCQ0OtsYkCAgLO+2CKwNkwxujIkSMqLCxUaGiox7AaDUXQAYCLRO2I1bVhB7iQhYaGuo2yfrYIOgBwkXA4HIqOjlZERIQqKyubujnA9/Lx8TnnnpxaBB0AuMh4eXk12o8IcKHjYmQAAGBbBB0AAGBbBB0AAGBbBB0AAGBbBB0AAGBbBB0AAGBbBB0AAGBbBB0AAGBb5z3oTJ8+XQ6HQ6mpqdY8Y4ymTp2qmJgY+fv7q2/fvtq8ebPb+8rLyzV+/Hi1bt1agYGBGjJkiPLy8txqioqKlJKSIqfTKafTqZSUFBUXF5/vXQIAAM3EeQ0669at00svvaQrr7zSbf7MmTM1e/ZszZ07V+vWrVNUVJT69++v0tJSqyY1NVVvv/22MjIytGrVKh06dEiDBw9WVVWVVZOcnKzs7GxlZmYqMzNT2dnZSklJOZ+7BAAAmhNznpSWlpqEhASzfPly06dPH/Pggw8aY4yprq42UVFR5umnn7Zqjx49apxOp5k/f74xxpji4mLj4+NjMjIyrJq9e/eaFi1amMzMTGOMMVu2bDGSzNq1a62aNWvWGElm27ZtZ9RGl8tlJBmXy3WuuwsAAH4gDfn9Pm89Ovfff79uvfVW3XTTTW7zc3JyVFBQoAEDBljz/Pz81KdPH61evVqSlJWVpcrKSreamJgYdenSxapZs2aNnE6nevToYdX07NlTTqfTqqmrvLxcJSUlbi8AAGBf52VQz4yMDH3xxRdat26dx7KCggJJUmRkpNv8yMhI7d6926rx9fVVWFiYR03t+wsKChQREeGx/oiICKumrunTp2vatGkN3yEAANAsNXqPTm5urh588EG9/vrratmy5ffWORwOt2ljjMe8uurW1Fd/qvVMmTJFLpfLeuXm5p5yewAAoHlr9KCTlZWlwsJCde/eXd7e3vL29taKFSv0/PPPy9vb2+rJqdvrUlhYaC2LiopSRUWFioqKTlnz7bffemx///79Hr1Ftfz8/BQSEuL2AgAA9tXoQadfv37auHGjsrOzrdc111yjESNGKDs7W5dccomioqK0fPly6z0VFRVasWKFevfuLUnq3r27fHx83Gry8/O1adMmq6ZXr15yuVz67LPPrJpPP/1ULpfLqgEAABe3Rr9GJzg4WF26dHGbFxgYqPDwcGt+amqq0tLSlJCQoISEBKWlpSkgIEDJycmSJKfTqVGjRmnixIkKDw9Xq1atNGnSJHXt2tW6uLlz584aNGiQRo8erRdffFGSNGbMGA0ePFiJiYmNvVsAAKAZOi8XI5/O5MmTVVZWpnHjxqmoqEg9evTQsmXLFBwcbNXMmTNH3t7eGjZsmMrKytSvXz+lp6fLy8vLqlm0aJEmTJhg3Z01ZMgQzZ079wffHwAAcGFyGGNMUzeiqZSUlMjpdMrlcnG9DgAAzURDfr8Z6woAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANgWQQcAANhWowed6dOn60c/+pGCg4MVERGhoUOHavv27W41xhhNnTpVMTEx8vf3V9++fbV582a3mvLyco0fP16tW7dWYGCghgwZory8PLeaoqIipaSkyOl0yul0KiUlRcXFxY29SwAAoJlq9KCzYsUK3X///Vq7dq2WL1+uY8eOacCAATp8+LBVM3PmTM2ePVtz587VunXrFBUVpf79+6u0tNSqSU1N1dtvv62MjAytWrVKhw4d0uDBg1VVVWXVJCcnKzs7W5mZmcrMzFR2drZSUlIae5cAAEBzZc6zwsJCI8msWLHCGGNMdXW1iYqKMk8//bRVc/ToUeN0Os38+fONMcYUFxcbHx8fk5GRYdXs3bvXtGjRwmRmZhpjjNmyZYuRZNauXWvVrFmzxkgy27ZtO6O2uVwuI8m4XK5z3k8AAPDDaMjv93m/RsflckmSWrVqJUnKyclRQUGBBgwYYNX4+fmpT58+Wr16tSQpKytLlZWVbjUxMTHq0qWLVbNmzRo5nU716NHDqunZs6ecTqdVU1d5eblKSkrcXgAAwL7Oa9Axxuihhx7S9ddfry5dukiSCgoKJEmRkZFutZGRkdaygoIC+fr6Kiws7JQ1ERERHtuMiIiwauqaPn26dT2P0+lUbGzsue0gAAC4oJ3XoPPAAw9ow4YNWrx4sccyh8PhNm2M8ZhXV92a+upPtZ4pU6bI5XJZr9zc3DPZDQAA0Eydt6Azfvx4vfvuu/rf//6ndu3aWfOjoqIkyaPXpbCw0OrliYqKUkVFhYqKik5Z8+2333psd//+/R69RbX8/PwUEhLi9gIAAPbV6EHHGKMHHnhAb731lj788EN16NDBbXmHDh0UFRWl5cuXW/MqKiq0YsUK9e7dW5LUvXt3+fj4uNXk5+dr06ZNVk2vXr3kcrn02WefWTWffvqpXC6XVQMAAC5u3o29wvvvv19vvPGG/vnPfyo4ONjquXE6nfL395fD4VBqaqrS0tKUkJCghIQEpaWlKSAgQMnJyVbtqFGjNHHiRIWHh6tVq1aaNGmSunbtqptuukmS1LlzZw0aNEijR4/Wiy++KEkaM2aMBg8erMTExMbeLQAA0Aw1etCZN2+eJKlv375u8xcsWKCRI0dKkiZPnqyysjKNGzdORUVF6tGjh5YtW6bg4GCrfs6cOfL29tawYcNUVlamfv36KT09XV5eXlbNokWLNGHCBOvurCFDhmju3LmNvUsAAKCZchhjTFM3oqmUlJTI6XTK5XJxvQ4AAM1EQ36/GesKAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYFkEHAADYVrMPOn/5y1/UoUMHtWzZUt27d9fHH3/c1E0CAAAXiGYddJYsWaLU1FT97ne/0/r163XDDTfo5ptv1p49e5q6aQAA4ALgMMaYpm7E2erRo4euvvpqzZs3z5rXuXNnDR06VNOnTz/t+0tKSuR0OuVK/6VCAvxqZjocx5c6GmlaDaw/xfRZv1cNrD8fbdX31/6g0zpp+kJqy+nqz1db1MD68/H9/IG+I42t7ufYuCs/T6ulzT8IjnPdlTf6GktKSuWM7yqXy6WQkJBT1no3+tZ/IBUVFcrKytKjjz7qNn/AgAFavXp1ve8pLy9XeXm5NV1SUlLzh23/lvya4V8mAAAuRuVn3kfTbIPOgQMHVFVVpcjISLf5kZGRKigoqPc906dP17Rp0zwX9H9SCvSXrM6t4/8979M68/rz1hY1sP58tFUNrG+M6R/gODRo+lTbvwC/j3Wnm3LbZzLdWM5LB/h5WGejt/N8tLHxV8nn3WgrbOT1qfHbeKxKUukZlTbboFPLUae7zRjjMa/WlClT9NBDD1nTJSUlio2Nla65RzpN1xcAALhAlJRITznPqLTZBp3WrVvLy8vLo/emsLDQo5enlp+fn/z8/H6I5gEAgAtAs73rytfXV927d9fy5cvd5i9fvly9e/duolYBAIALSbPt0ZGkhx56SCkpKbrmmmvUq1cvvfTSS9qzZ4/uu+++pm4aAAC4ADTroDN8+HAdPHhQTz75pPLz89WlSxe99957at++fVM3DQAAXACa9XN0zpX1HJ0zuA8fAABcGBry+91sr9EBAAA4HYIOAACwLYIOAACwLYIOAACwLYIOAACwLYIOAACwLYIOAACwLYIOAACwLYIOAACwrWY9BMS5qn0odElJSRO3BAAAnKna3+0zGdzhog46Bw8elCTFxsY2cUsAAEBDHTx4UE6n85Q1F3XQadWqlSRpz549pz1QAJqXkpISxcbGKjc3l7HsAJtxuVyKi4uzfsdP5aIOOi1a1Fyi5HQ6+YcQsKmQkBD+fgM2Vfs7fsqaH6AdAAAATYKgAwAAbOuiDjp+fn564okn5Ofn19RNAdDI+PsN2FdD/n47zJncmwUAANAMXdQ9OgAAwN4IOgAAwLYIOgAAwLYIOgAAwLYIOgAAwLYIOgAAwLYIOgAAwLYIOgAAwLYIOgAAwLYIOgDOu6lTp6pbt25N3QwAFyGCDoBz4nA4TvkaOXKkJk2apA8++KBJ20nYAi5O3k3dAADNW35+vvXnJUuW6PHHH9f27dutef7+/goKClJQUFBTNA/ARY4eHQDnJCoqyno5nU45HA6PeXV7U0aOHKmhQ4cqLS1NkZGRCg0N1bRp03Ts2DE9/PDDatWqldq1a6dXX33VbVt79+7V8OHDFRYWpvDwcN1+++365ptvrOUfffSRrr32WgUGBio0NFTXXXeddu/erfT0dE2bNk1ffvml1dOUnp4uSZo9e7a6du2qwMBAxcbGaty4cTp06JC1zvT0dIWGhurf//63EhMTFRAQoJ/97Gc6fPiwFi5cqPj4eIWFhWn8+PGqqqqy3hcfH68//OEPSk5OVlBQkGJiYvTCCy+cl88AwPcj6ABoEh9++KH27dunlStXavbs2Zo6daoGDx6ssLAwffrpp7rvvvt03333KTc3V5J05MgR3XjjjQoKCtLKlSu1atUqBQUFadCgQaqoqNCxY8c0dOhQ9enTRxs2bNCaNWs0ZswYORwODR8+XBMnTtQVV1yh/Px85efna/jw4ZKkFi1a6Pnnn9emTZu0cOFCffjhh5o8ebJbW48cOaLnn39eGRkZyszM1EcffaQ77rhD7733nt577z299tpreumll/SPf/zD7X2zZs3SlVdeqS+++EJTpkzRb37zGy1fvvyHOcAAahgAaCQLFiwwTqfTY/4TTzxhrrrqKmv67rvvNu3btzdVVVXWvMTERHPDDTdY08eOHTOBgYFm8eLFxhhjXnnlFZOYmGiqq6utmvLycuPv72/ef/99c/DgQSPJfPTRR/W2rW4bvs/SpUtNeHi42z5JMjt37rTmjR071gQEBJjS0lJr3sCBA83YsWOt6fbt25tBgwa5rXv48OHm5ptvPm0bADQeenQANIkrrrhCLVqc+CcoMjJSXbt2taa9vLwUHh6uwsJCSVJWVpZ27typ4OBg65qfVq1a6ejRo9q1a5datWqlkSNHauDAgbrtttv0pz/9ye36oe/zv//9T/3791fbtm0VHBysu+66SwcPHtThw4etmoCAAF166aVubY2Pj3e77igyMtJqa61evXp5TG/duvUMjxCAxkDQAdAkfHx83KYdDke986qrqyVJ1dXV6t69u7Kzs91eO3bsUHJysiRpwYIFWrNmjXr37q0lS5aoU6dOWrt27fe2Yffu3brlllvUpUsXvfnmm8rKytKf//xnSVJlZeVZt/VUHA7HaWsANB7uugLQLFx99dVasmSJIiIiFBIS8r11SUlJSkpK0pQpU9SrVy+98cYb6tmzp3x9fd0uFpakzz//XMeOHdOzzz5r9S4tXbq00dpcN2StXbtWl112WaOtH8Dp0aMDoFkYMWKEWrdurdtvv10ff/yxcnJytGLFCj344IPKy8tTTk6OpkyZojVr1mj37t1atmyZduzYoc6dO0uquQsqJydH2dnZOnDggMrLy3XppZfq2LFjeuGFF/T111/rtdde0/z58xutzZ988olmzpypHTt26M9//rP+/ve/68EHH2y09QM4PYIOgGYhICBAK1euVFxcnO644w517txZ99xzj8rKyhQSEqKAgABt27ZNd955pzp16qQxY8bogQce0NixYyVJd955pwYNGqQbb7xRbdq00eLFi9WtWzfNnj1bM2bMUJcuXbRo0SJNnz690do8ceJEZWVlKSkpSX/4wx/07LPPauDAgY22fgCn5zDGmKZuBADYTXx8vFJTU5WamtrUTQEuavToAAAA2yLoAAAA2+LUFQAAsC16dAAAgG0RdAAAgG0RdAAAgG0RdAAAgG0RdAAAgG0RdAAAgG0RdAAAgG0RdAAAgG39fyzI3SSwcFYqAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "total_metrics = plot_endpoint_metrics(start_time=startime, end_time=endtime)\n",
    "# total_metrics = plot_endpoint_metrics(start_time=startime, end_time=endtime)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "43d10a1d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ModelLatency</th>\n",
       "      <th>OverheadLatency</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Timestamp</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2023-06-02 00:24:00+00:00</th>\n",
       "      <td>8692.3982</td>\n",
       "      <td>22.5878</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2023-06-02 00:25:00+00:00</th>\n",
       "      <td>10338.1598</td>\n",
       "      <td>7.3332</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2023-06-02 00:26:00+00:00</th>\n",
       "      <td>10314.6550</td>\n",
       "      <td>5.9240</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                           ModelLatency  OverheadLatency\n",
       "Timestamp                                               \n",
       "2023-06-02 00:24:00+00:00     8692.3982          22.5878\n",
       "2023-06-02 00:25:00+00:00    10338.1598           7.3332\n",
       "2023-06-02 00:26:00+00:00    10314.6550           5.9240"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Latency expressed in ms\n",
    "total_metrics"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "401d0e40",
   "metadata": {},
   "source": [
    "## Clean up the environment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5552dd36",
   "metadata": {},
   "outputs": [],
   "source": [
    "sess.delete_endpoint(endpoint_name)\n",
    "sess.delete_endpoint_config(endpoint_name)\n",
    "model.delete_model()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "conda_pytorch_p39",
   "language": "python",
   "name": "conda_pytorch_p39"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.15"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}