 archived/kmeans_bring_your_own_model.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Bring Your Own Model (k-means)\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook. \n",
    "\n",
    "![This us-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-west-2/advanced_functionality|kmeans_bring_your_own_model|kmeans_bring_your_own_model.ipynb)\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "_**Hosting a Pre-Trained Model in Amazon SageMaker Algorithm Containers**_\n",
    "\n",
    "---\n",
    "\n",
    "---\n",
    "\n",
    "## Contents\n",
    "\n",
    "1. [Background](#Background)\n",
    "1. [Setup](#Setup)\n",
    "1. [(Optional)](#Optional)\n",
    "  1. [Data](#Data)\n",
    "  1. [Train Locally](#Train Locally)\n",
    "1. [Convert](#Convert)\n",
    "1. [Host](#Host)\n",
    "  1. [Confirm](#Confirm)\n",
    "1. [Extensions](#Extensions)\n",
    "\n",
    "---\n",
    "## Background\n",
    "\n",
    "Amazon SageMaker includes functionality to support a hosted notebook environment, distributed, managed training, and real-time hosting.  We think it works best when all three of these services are used together, but they can also be used independently.  Some use cases may only require hosting.  Maybe the model was trained prior to Amazon SageMaker existing, in a different service.\n",
    "\n",
    "This notebook shows how to use a pre-existing model with an Amazon SageMaker Algorithm container to quickly create a hosted endpoint for that model.\n",
    "\n",
    "---\n",
    "## Setup\n",
    "\n",
    "_This notebook was created and tested on an ml.m4.xlarge notebook instance._\n",
    "\n",
    "Let's start by specifying:\n",
    "\n",
    "- The S3 bucket and prefix that you want to use for training and model data.  This should be within the same region as the Notebook Instance, training, and hosting.\n",
    "- The IAM role arn used to give training and hosting access to your data. See the documentation for how to create these.  Note, if more than one role is required for notebook instances, training, and/or hosting, please replace the boto regexp with a the appropriate full IAM role arn string(s)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "isConfigCell": true,
    "tags": [
     "parameters"
    ]
   },
   "outputs": [],
   "source": [
    "# Define IAM role\n",
    "import boto3\n",
    "import re\n",
    "\n",
    "import sagemaker\n",
    "from sagemaker import get_execution_role\n",
    "\n",
    "role = get_execution_role()\n",
    "bucket = sagemaker.Session().default_bucket()\n",
    "prefix = \"sagemaker/DEMO-kmeans-byom\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import sklearn.cluster\n",
    "import pickle\n",
    "import gzip\n",
    "import urllib.request\n",
    "import json\n",
    "import mxnet as mx\n",
    "import boto3\n",
    "import time\n",
    "import io\n",
    "import os"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (Optional)\n",
    "\n",
    "_This section is only included for illustration purposes.  In a real use case, you'd be bringing your model from an existing process and not need to complete these steps._\n",
    "\n",
    "### Data\n",
    "\n",
    "For simplicity, we'll utilize the MNIST dataset.  This includes roughly 70K 28 x 28 pixel images of handwritten digits from 0 to 9.  More detail can be found [here](https://en.wikipedia.org/wiki/MNIST_database)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "region = boto3.Session().region_name\n",
    "s3 = boto3.client(\"s3\")\n",
    "s3.download_file(\n",
    "    f\"sagemaker-sample-data-{region}\", \"algorithms/kmeans/mnist/mnist.pkl.gz\", \"mnist.pkl.gz\"\n",
    ")\n",
    "f = gzip.open(\"mnist.pkl.gz\", \"rb\")\n",
    "train_set, valid_set, test_set = pickle.load(f, encoding=\"latin1\")\n",
    "f.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train Locally\n",
    "\n",
    "Again for simplicity, let's stick with the k-means algorithm."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "kmeans = sklearn.cluster.KMeans(n_clusters=10).fit(train_set[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Convert\n",
    "\n",
    "The model format that Amazon SageMaker's k-means container expects is an MXNet NDArray with dimensions (num_clusters, feature_dim) that contains the cluster centroids.  For our current example, the 10 centroids for the MNIST digits are stored in a (10, 784) dim NumPy array called `kmeans.cluster_centers_`.\n",
    "\n",
    "_Note: model formats will differ across algorithms, but this concept is generalizable.  Documentation, or just running a toy example and interrogating the resulting model artifact is the best way to understand the specific model format required for different algorithms._\n",
    "\n",
    "Let's:\n",
    "- Convert to a MXNet NDArray\n",
    "- Save to a file `model_algo-1`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "centroids = mx.ndarray.array(kmeans.cluster_centers_)\n",
    "mx.ndarray.save(\"model_algo-1\", [centroids])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- tar and gzip the model array"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!tar czvf model.tar.gz model_algo-1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Load to s3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "boto3.Session().resource(\"s3\").Bucket(bucket).Object(\n",
    "    os.path.join(prefix, \"model.tar.gz\")\n",
    ").upload_file(\"model.tar.gz\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Host\n",
    "\n",
    "Stary by defining our model to hosting.  Amazon SageMaker Algorithm containers are published to accounts which are unique across region, so we've accounted for that here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.amazon.amazon_estimator import get_image_uri\n",
    "\n",
    "kmeans_model = \"DEMO-kmeans-byom-\" + time.strftime(\"%Y-%m-%d-%H-%M-%S\", time.gmtime())\n",
    "\n",
    "sm = boto3.client(\"sagemaker\")\n",
    "container = get_image_uri(boto3.Session().region_name, \"kmeans\")\n",
    "\n",
    "create_model_response = sm.create_model(\n",
    "    ModelName=kmeans_model,\n",
    "    ExecutionRoleArn=role,\n",
    "    PrimaryContainer={\n",
    "        \"Image\": container,\n",
    "        \"ModelDataUrl\": \"s3://{}/{}/model.tar.gz\".format(bucket, prefix),\n",
    "    },\n",
    ")\n",
    "\n",
    "print(create_model_response[\"ModelArn\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then setup our endpoint configuration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "kmeans_endpoint_config = \"DEMO-kmeans-byom-endpoint-config-\" + time.strftime(\n",
    "    \"%Y-%m-%d-%H-%M-%S\", time.gmtime()\n",
    ")\n",
    "print(kmeans_endpoint_config)\n",
    "create_endpoint_config_response = sm.create_endpoint_config(\n",
    "    EndpointConfigName=kmeans_endpoint_config,\n",
    "    ProductionVariants=[\n",
    "        {\n",
    "            \"InstanceType\": \"ml.m4.xlarge\",\n",
    "            \"InitialInstanceCount\": 1,\n",
    "            \"ModelName\": kmeans_model,\n",
    "            \"VariantName\": \"AllTraffic\",\n",
    "        }\n",
    "    ],\n",
    ")\n",
    "\n",
    "print(\"Endpoint Config Arn: \" + create_endpoint_config_response[\"EndpointConfigArn\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, initiate our endpoints."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "\n",
    "kmeans_endpoint = \"DEMO-kmeans-byom-endpoint-\" + time.strftime(\"%Y%m%d%H%M\", time.gmtime())\n",
    "print(kmeans_endpoint)\n",
    "create_endpoint_response = sm.create_endpoint(\n",
    "    EndpointName=kmeans_endpoint, EndpointConfigName=kmeans_endpoint_config\n",
    ")\n",
    "print(create_endpoint_response[\"EndpointArn\"])\n",
    "\n",
    "resp = sm.describe_endpoint(EndpointName=kmeans_endpoint)\n",
    "status = resp[\"EndpointStatus\"]\n",
    "print(\"Status: \" + status)\n",
    "\n",
    "sm.get_waiter(\"endpoint_in_service\").wait(EndpointName=kmeans_endpoint)\n",
    "\n",
    "resp = sm.describe_endpoint(EndpointName=kmeans_endpoint)\n",
    "status = resp[\"EndpointStatus\"]\n",
    "print(\"Arn: \" + resp[\"EndpointArn\"])\n",
    "print(\"Status: \" + status)\n",
    "\n",
    "if status != \"InService\":\n",
    "    raise Exception(\"Endpoint creation did not succeed\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Confirm\n",
    "Let's confirm that our model is producing the same results.  We'll take the first 100 records from our training dataset, score them in our hosted endpoint..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def np2csv(arr):\n",
    "    csv = io.BytesIO()\n",
    "    np.savetxt(csv, arr, delimiter=\",\", fmt=\"%g\")\n",
    "    return csv.getvalue().decode().rstrip()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "runtime = boto3.Session().client(\"runtime.sagemaker\")\n",
    "\n",
    "payload = np2csv(train_set[0][0:100])\n",
    "response = runtime.invoke_endpoint(\n",
    "    EndpointName=kmeans_endpoint, ContentType=\"text/csv\", Body=payload\n",
    ")\n",
    "result = json.loads(response[\"Body\"].read().decode())\n",
    "scored_labels = np.array([r[\"closest_cluster\"] for r in result[\"predictions\"]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "... And then compare them to the model labels from our k-means example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "scored_labels == kmeans.labels_[0:100]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Extensions\n",
    "\n",
    "This notebook showed how to seed a pre-existing model in an already built container.  This functionality could be replicated with other Amazon SageMaker Algorithms, as well as the TensorFlow and MXNet containers.  Although this is certainly an easy method to bring your own model, it is not likely to provide the flexibility of a bringing your own scoring container.  Please refer to other example notebooks which show how to dockerize your own training and scoring container which could be modified appropriately to your use case."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Remove endpoint to avoid stray charges\n",
    "sm.delete_endpoint(EndpointName=kmeans_endpoint)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Notebook CI Test Results\n",
    "\n",
    "This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.\n",
    "\n",
    "![This us-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-east-1/advanced_functionality|kmeans_bring_your_own_model|kmeans_bring_your_own_model.ipynb)\n",
    "\n",
    "![This us-east-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-east-2/advanced_functionality|kmeans_bring_your_own_model|kmeans_bring_your_own_model.ipynb)\n",
    "\n",
    "![This us-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-west-1/advanced_functionality|kmeans_bring_your_own_model|kmeans_bring_your_own_model.ipynb)\n",
    "\n",
    "![This ca-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ca-central-1/advanced_functionality|kmeans_bring_your_own_model|kmeans_bring_your_own_model.ipynb)\n",
    "\n",
    "![This sa-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/sa-east-1/advanced_functionality|kmeans_bring_your_own_model|kmeans_bring_your_own_model.ipynb)\n",
    "\n",
    "![This eu-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-1/advanced_functionality|kmeans_bring_your_own_model|kmeans_bring_your_own_model.ipynb)\n",
    "\n",
    "![This eu-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-2/advanced_functionality|kmeans_bring_your_own_model|kmeans_bring_your_own_model.ipynb)\n",
    "\n",
    "![This eu-west-3 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-3/advanced_functionality|kmeans_bring_your_own_model|kmeans_bring_your_own_model.ipynb)\n",
    "\n",
    "![This eu-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-central-1/advanced_functionality|kmeans_bring_your_own_model|kmeans_bring_your_own_model.ipynb)\n",
    "\n",
    "![This eu-north-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-north-1/advanced_functionality|kmeans_bring_your_own_model|kmeans_bring_your_own_model.ipynb)\n",
    "\n",
    "![This ap-southeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-southeast-1/advanced_functionality|kmeans_bring_your_own_model|kmeans_bring_your_own_model.ipynb)\n",
    "\n",
    "![This ap-southeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-southeast-2/advanced_functionality|kmeans_bring_your_own_model|kmeans_bring_your_own_model.ipynb)\n",
    "\n",
    "![This ap-northeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-northeast-1/advanced_functionality|kmeans_bring_your_own_model|kmeans_bring_your_own_model.ipynb)\n",
    "\n",
    "![This ap-northeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-northeast-2/advanced_functionality|kmeans_bring_your_own_model|kmeans_bring_your_own_model.ipynb)\n",
    "\n",
    "![This ap-south-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-south-1/advanced_functionality|kmeans_bring_your_own_model|kmeans_bring_your_own_model.ipynb)\n"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Tags",
  "kernelspec": {
   "display_name": "conda_mxnet_p38",
   "language": "python",
   "name": "conda_mxnet_p38"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  },
  "notice": "Copyright 2017 Amazon.com, Inc. or its affiliates. All Rights Reserved.  Licensed under the Apache License, Version 2.0 (the \"License\"). You may not use this file except in compliance with the License. A copy of the License is located at http://aws.amazon.com/apache2.0/ or in the \"license\" file accompanying this file. This file is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License."
 },
 "nbformat": 4,
 "nbformat_minor": 4
}