 archived/boto3_scikit_retrain_model_and_deploy_to_existing_endpoint.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Use boto3 to train and update an existing SageMaker Endpoint with a newly trained Scikit-Learn Random Forest Model\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook. \n",
    "\n",
    "![This us-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-west-2/boto3|built-in-frameworks|scikit-learn|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint.ipynb)\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "\n",
    "In this notebook we show how to use Amazon SageMaker to develop, train, and deploy a Scikit-Learn based ML model (Random Forest). Then we show how to update an existing SageMaker Endpoint with a newly trained model, <b>while there is no availability loss</b>.\n",
    "\n",
    "This is done using `boto3`, A low-level client representing Amazon SageMaker Service. Using `boto3` is highly useful when using SageMaker Services from other resources than notebooks, such as Lambda functions, Airflow, or Jenkins.\n",
    "\n",
    "You'll execute the following steps using `boto3`:\n",
    " - Prepare the training/testing data and write a Script Mode script.\n",
    " - Launch the 1st training job.\n",
    " - Deploy the 1st model to a SageMaker Endpoint, and make few inference requests.\n",
    " - Launch the 2nd training job.\n",
    " - Update the SageMaker Endpoint with the 2nd model, and make few inference requests.\n",
    " - Optional cleanup.\n",
    "\n",
    "\n",
    "More info on boto3 can be found on the [Boto3 SageMaker documentation page](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html#client).\n",
    "\n",
    "More info on Scikit-Learn can be found on the [Scikit-Learn documentation page](https://scikit-learn.org/stable/index.html).\n",
    "\n",
    "We use the [California housing dataset](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.fetch_california_housing.html), present in Scikit-Learn.\n",
    "\n",
    "More info on the dataset:\n",
    "\n",
    "This dataset was obtained from the `StatLib` repository. http://lib.stat.cmu.edu/datasets/\n",
    "\n",
    "The target variable is the median house value for California districts.\n",
    "\n",
    "This dataset was derived from the 1990 U.S. census, using one row per census block group. A block group is the smallest geographical unit for which the U.S. Census Bureau publishes sample data (a block group typically has a population of 600 to 3,000 people).\n",
    "\n",
    "It can be downloaded/loaded using the `sklearn.datasets.fetch_california_housing` function.\n",
    " \n",
    " \n",
    "**This sample is provided for demonstration purposes, make sure to conduct appropriate testing if deriving this code for your own use-cases!**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Import Python libraries\n",
    "\n",
    "Now you import Python libraries like `sklearn`, `pandas`, `numpy`, and `boto3`.\n",
    "\n",
    "We also import `sagemaker` which is the high level SageMaker Python SDK. This is for the purpose of getting the default SageMaker bucket and execution role. Apart from that, all SageMaker functionality will be demonstrated using `boto3`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "!pip install -U sagemaker"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import datetime\n",
    "import time\n",
    "import tarfile\n",
    "\n",
    "import boto3\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from sagemaker import get_execution_role\n",
    "import sagemaker\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.datasets import fetch_california_housing\n",
    "\n",
    "\n",
    "sm_boto3 = boto3.client(\"sagemaker\")\n",
    "\n",
    "sess = sagemaker.Session()\n",
    "\n",
    "region = sess.boto_session.region_name\n",
    "\n",
    "bucket = sess.default_bucket()  # this could also be a hard-coded bucket name\n",
    "\n",
    "print(\"Using bucket \" + bucket)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Prepare data\n",
    "We load a dataset from `sklearn`, split it and send it to S3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# we use the California housing dataset\n",
    "data = fetch_california_housing()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    data.data, data.target, test_size=0.25, random_state=42\n",
    ")\n",
    "\n",
    "trainX = pd.DataFrame(X_train, columns=data.feature_names)\n",
    "trainX[\"target\"] = y_train\n",
    "\n",
    "testX = pd.DataFrame(X_test, columns=data.feature_names)\n",
    "testX[\"target\"] = y_test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "trainX.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "trainX.to_csv(\"california_train.csv\")\n",
    "testX.to_csv(\"california_test.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# send data to S3. SageMaker will take training data from s3\n",
    "trainpath = sess.upload_data(\n",
    "    path=\"california_train.csv\", bucket=bucket, key_prefix=\"sagemaker/sklearn-california\"\n",
    ")\n",
    "\n",
    "testpath = sess.upload_data(\n",
    "    path=\"california_test.csv\", bucket=bucket, key_prefix=\"sagemaker/sklearn-california\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Writing a *Script Mode* script\n",
    "The below script contains both training and inference functionality and can run both in SageMaker Training hardware or locally (desktop, SageMaker notebook, on premise, etc). Detailed guidance can be found on the [Scikit-learn SageMaker Python SDK documentation page](https://sagemaker.readthedocs.io/en/stable/using_sklearn.html#preparing-the-scikit-learn-training-script)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "%%writefile script.py\n",
    "\n",
    "import argparse\n",
    "import joblib\n",
    "import os\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.ensemble import RandomForestRegressor\n",
    "\n",
    "\n",
    "# inference functions ---------------\n",
    "def model_fn(model_dir):\n",
    "    clf = joblib.load(os.path.join(model_dir, \"model.joblib\"))\n",
    "    return clf\n",
    "\n",
    "\n",
    "if __name__ == \"__main__\":\n",
    "    print(\"extracting arguments\")\n",
    "    parser = argparse.ArgumentParser()\n",
    "\n",
    "    # hyperparameters sent by the client are passed as command-line arguments to the script.\n",
    "    # to simplify the demo we don't use all sklearn RandomForest hyperparameters\n",
    "    parser.add_argument(\"--n-estimators\", type=int, default=10)\n",
    "    parser.add_argument(\"--min-samples-leaf\", type=int, default=3)\n",
    "\n",
    "    # Data, model, and output directories\n",
    "    parser.add_argument(\"--model-dir\", type=str, default=os.environ.get(\"SM_MODEL_DIR\"))\n",
    "    parser.add_argument(\"--train\", type=str, default=os.environ.get(\"SM_CHANNEL_TRAIN\"))\n",
    "    parser.add_argument(\"--test\", type=str, default=os.environ.get(\"SM_CHANNEL_TEST\"))\n",
    "    parser.add_argument(\"--train-file\", type=str, default=\"california_train.csv\")\n",
    "    parser.add_argument(\"--test-file\", type=str, default=\"california_test.csv\")\n",
    "    parser.add_argument(\n",
    "        \"--features\", type=str\n",
    "    )  # in this script we ask user to explicitly name features\n",
    "    parser.add_argument(\n",
    "        \"--target\", type=str\n",
    "    )  # in this script we ask user to explicitly name the target\n",
    "\n",
    "    args, _ = parser.parse_known_args()\n",
    "\n",
    "    print(\"reading data\")\n",
    "    train_df = pd.read_csv(os.path.join(args.train, args.train_file))\n",
    "    test_df = pd.read_csv(os.path.join(args.test, args.test_file))\n",
    "\n",
    "    print(\"building training and testing datasets\")\n",
    "    X_train = train_df[args.features.split()]\n",
    "    X_test = test_df[args.features.split()]\n",
    "    y_train = train_df[args.target]\n",
    "    y_test = test_df[args.target]\n",
    "\n",
    "    # train\n",
    "    print(\"training model\")\n",
    "    model = RandomForestRegressor(\n",
    "        n_estimators=args.n_estimators, min_samples_leaf=args.min_samples_leaf, n_jobs=-1\n",
    "    )\n",
    "\n",
    "    model.fit(X_train, y_train)\n",
    "\n",
    "    # print abs error\n",
    "    print(\"validating model\")\n",
    "    abs_err = np.abs(model.predict(X_test) - y_test)\n",
    "\n",
    "    # print couple perf metrics\n",
    "    for q in [10, 50, 90]:\n",
    "        print(\"AE-at-\" + str(q) + \"th-percentile: \" + str(np.percentile(a=abs_err, q=q)))\n",
    "\n",
    "    # persist model\n",
    "    path = os.path.join(args.model_dir, \"model.joblib\")\n",
    "    joblib.dump(model, path)\n",
    "    print(\"model persisted at \" + path)\n",
    "    print(args.min_samples_leaf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## SageMaker Training"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Launching a training with `boto3`\n",
    "`boto3` is more verbose yet gives more visibility in the low-level details of Amazon SageMaker"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# first compress the code and send to S3\n",
    "\n",
    "source = \"source.tar.gz\"\n",
    "project = \"scikitlearn-california-train-from-boto3\"\n",
    "\n",
    "tar = tarfile.open(source, \"w:gz\")\n",
    "tar.add(\"script.py\")\n",
    "tar.close()\n",
    "\n",
    "s3 = boto3.client(\"s3\")\n",
    "s3.upload_file(source, bucket, project + \"/\" + source)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "When using `boto3` to launch a training job, we must explicitly point to a docker image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from sagemaker import image_uris\n",
    "\n",
    "FRAMEWORK_VERSION = \"1.2-1\"\n",
    "\n",
    "training_image = image_uris.retrieve(\n",
    "    framework=\"sklearn\",\n",
    "    region=region,\n",
    "    version=FRAMEWORK_VERSION,\n",
    "    py_version=\"py3\",\n",
    "    instance_type=\"ml.c5.xlarge\",\n",
    ")\n",
    "print(training_image)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Launch the 1st training job\n",
    "\n",
    "This will start a model training job. After training completes, Amazon SageMaker saves the resulting model artifacts to an Amazon S3 location that you specify.\n",
    "\n",
    "If you choose to host your model using Amazon SageMaker hosting services, you can use the resulting model artifacts as part of the model. You can also use the artifacts in a machine learning service other than Amazon SageMaker, provided that you know how to use them for inference.\n",
    "\n",
    "More info on `create_training_job` can be found on the [Boto3 SageMaker documentation page](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html#SageMaker.Client.create_training_job)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "training_job_1_name = \"sklearn-boto3-1-\" + datetime.datetime.now().strftime(\"%Y-%m-%d-%H-%M-%S\")\n",
    "\n",
    "training_job_1_response = sm_boto3.create_training_job(\n",
    "    TrainingJobName=training_job_1_name,\n",
    "    HyperParameters={\n",
    "        \"n_estimators\": \"300\",\n",
    "        \"min_samples_leaf\": \"3\",\n",
    "        \"sagemaker_program\": \"script.py\",\n",
    "        \"features\": \"MedInc HouseAge AveRooms AveBedrms Population AveOccup Latitude Longitude\",\n",
    "        \"target\": \"target\",\n",
    "        \"sagemaker_submit_directory\": \"s3://\" + bucket + \"/\" + project + \"/\" + source,\n",
    "    },\n",
    "    AlgorithmSpecification={\n",
    "        \"TrainingImage\": training_image,\n",
    "        \"TrainingInputMode\": \"File\",\n",
    "        \"MetricDefinitions\": [\n",
    "            {\"Name\": \"median-AE\", \"Regex\": \"AE-at-50th-percentile: ([0-9.]+).*$\"},\n",
    "        ],\n",
    "    },\n",
    "    RoleArn=get_execution_role(),\n",
    "    InputDataConfig=[\n",
    "        {\n",
    "            \"ChannelName\": \"train\",\n",
    "            \"DataSource\": {\n",
    "                \"S3DataSource\": {\n",
    "                    \"S3DataType\": \"S3Prefix\",\n",
    "                    \"S3Uri\": trainpath,\n",
    "                    \"S3DataDistributionType\": \"FullyReplicated\",\n",
    "                }\n",
    "            },\n",
    "        },\n",
    "        {\n",
    "            \"ChannelName\": \"test\",\n",
    "            \"DataSource\": {\n",
    "                \"S3DataSource\": {\n",
    "                    \"S3DataType\": \"S3Prefix\",\n",
    "                    \"S3Uri\": testpath,\n",
    "                    \"S3DataDistributionType\": \"FullyReplicated\",\n",
    "                }\n",
    "            },\n",
    "        },\n",
    "    ],\n",
    "    OutputDataConfig={\"S3OutputPath\": \"s3://\" + bucket + \"/sagemaker-sklearn-artifact/\"},\n",
    "    ResourceConfig={\"InstanceType\": \"ml.c5.xlarge\", \"InstanceCount\": 1, \"VolumeSizeInGB\": 10},\n",
    "    StoppingCondition={\"MaxRuntimeInSeconds\": 86400},\n",
    "    EnableNetworkIsolation=False,\n",
    ")\n",
    "\n",
    "training_job_1_response"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Wait for the 1st training job to end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import boto3\n",
    "import time\n",
    "\n",
    "client = boto3.client(\"sagemaker\")\n",
    "\n",
    "training_job_1_details = client.describe_training_job(TrainingJobName=training_job_1_name)\n",
    "\n",
    "while training_job_1_details[\"TrainingJobStatus\"] == \"InProgress\":\n",
    "    training_job_1_details = client.describe_training_job(TrainingJobName=training_job_1_name)\n",
    "    print(training_job_1_details[\"TrainingJobStatus\"])\n",
    "    time.sleep(15)\n",
    "\n",
    "training_job_1_details"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Create a Model for the 1st training job\n",
    "\n",
    "This will create a model in Amazon SageMaker. In the request, you name the model and describe a primary container. For the primary container, you specify the Docker image that contains inference code, artifacts (from prior training), and a custom environment map that the inference code uses when you deploy the model for predictions.\n",
    "\n",
    "More info on `create_model` can be found on the [Boto3 SageMaker documentation page](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html#SageMaker.Client.create_model)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "model_1_name = \"sklearn-model-1-\" + datetime.datetime.now().strftime(\"%Y-%m-%d-%H-%M-%S\")\n",
    "\n",
    "create_model_1_response = client.create_model(\n",
    "    ModelName=model_1_name,\n",
    "    PrimaryContainer={\n",
    "        \"Image\": training_job_1_details[\"AlgorithmSpecification\"][\"TrainingImage\"],\n",
    "        \"Mode\": \"SingleModel\",\n",
    "        \"ModelDataUrl\": training_job_1_details[\"ModelArtifacts\"][\"S3ModelArtifacts\"],\n",
    "        \"Environment\": {\n",
    "            \"SAGEMAKER_CONTAINER_LOG_LEVEL\": \"20\",\n",
    "            \"SAGEMAKER_PROGRAM\": training_job_1_details[\"HyperParameters\"][\"sagemaker_program\"],\n",
    "            \"SAGEMAKER_REGION\": region,\n",
    "            \"SAGEMAKER_SUBMIT_DIRECTORY\": training_job_1_details[\"HyperParameters\"][\n",
    "                \"sagemaker_submit_directory\"\n",
    "            ],\n",
    "        },\n",
    "    },\n",
    "    ExecutionRoleArn=get_execution_role(),\n",
    ")\n",
    "\n",
    "create_model_1_response"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Now you'll describe the model that you created using the `describe_model` API.\n",
    "\n",
    "More info on `describe_model` can be found on the [Boto3 SageMaker documentation page](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html#SageMaker.Client.describe_model)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "client.describe_model(ModelName=model_1_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Create an Endpoint Config from 1st model\n",
    "\n",
    "This will create an endpoint configuration that Amazon SageMaker hosting services uses to deploy models. In the configuration, you identify one or more models, created using the `CreateModel` API, to deploy and the resources that you want Amazon SageMaker to provision. Then you call the `CreateEndpoint` API.\n",
    "\n",
    "More info on `create_endpoint_config` can be found on the [Boto3 SageMaker documentation page](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html#SageMaker.Client.create_endpoint_config)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "endpoint_config_1_name = \"sklearn-endpoint-config-1-\" + datetime.datetime.now().strftime(\n",
    "    \"%Y-%m-%d-%H-%M-%S\"\n",
    ")\n",
    "\n",
    "endpoint_config_1_response = client.create_endpoint_config(\n",
    "    EndpointConfigName=endpoint_config_1_name,\n",
    "    ProductionVariants=[\n",
    "        {\n",
    "            \"VariantName\": \"AllTrafficVariant\",\n",
    "            \"ModelName\": model_1_name,\n",
    "            \"InitialInstanceCount\": 1,\n",
    "            \"InstanceType\": \"ml.c5.large\",\n",
    "            \"InitialVariantWeight\": 1,\n",
    "        },\n",
    "    ],\n",
    ")\n",
    "\n",
    "endpoint_config_1_response"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Deploy the 1st Endpoint Config to a real-time endpoint\n",
    "\n",
    "This will create an endpoint using the endpoint configuration specified in the request. Amazon SageMaker uses the endpoint to provision resources and deploy models. Note that you have already created the endpoint configuration with the `CreateEndpointConfig` API in the previous step.\n",
    "\n",
    "More info on `create_endpoint` can be found on the [Boto3 SageMaker documentation page](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html#SageMaker.Client.create_endpoint)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "endpoint_name = \"sklearn-endpoint-\" + datetime.datetime.now().strftime(\"%Y-%m-%d-%H-%M-%S\")\n",
    "\n",
    "create_endpoint_response = client.create_endpoint(\n",
    "    EndpointName=endpoint_name,\n",
    "    EndpointConfigName=endpoint_config_1_name,\n",
    ")\n",
    "\n",
    "create_endpoint_response"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Wait for Endpoint to be ready"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "describe_endpoint_response = client.describe_endpoint(EndpointName=endpoint_name)\n",
    "\n",
    "while describe_endpoint_response[\"EndpointStatus\"] == \"Creating\":\n",
    "    describe_endpoint_response = client.describe_endpoint(EndpointName=endpoint_name)\n",
    "    print(describe_endpoint_response[\"EndpointStatus\"])\n",
    "    time.sleep(15)\n",
    "\n",
    "describe_endpoint_response"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Invoke Endpoint with `boto3`\n",
    "\n",
    "After you deploy a model into production using Amazon SageMaker hosting services, your client applications use this API to get inferences from the model hosted at the specified endpoint.\n",
    "\n",
    "For an overview of Amazon SageMaker, [see How It Works](https://docs.aws.amazon.com/sagemaker/latest/dg/how-it-works.html).\n",
    "\n",
    "Amazon SageMaker strips all POST headers except those supported by the API. Amazon SageMaker might add additional headers. You should not rely on the behavior of headers outside those enumerated in the request syntax.\n",
    "\n",
    "Calls to `InvokeEndpoint` are authenticated by using AWS Signature Version 4. For information, see Authenticating Requests (AWS Signature Version 4) in the Amazon S3 API Reference.\n",
    "\n",
    "A customer's model containers must respond to requests within 60 seconds. The model itself can have a maximum processing time of 60 seconds before responding to invocations. If your model is going to take 50-60 seconds of processing time, the SDK socket timeout should be set to be 70 seconds.\n",
    "\n",
    "More info on `invoke_endpoint` can be found on the [Boto3 SageMakerRuntime documentation page](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker-runtime.html#SageMakerRuntime.Client.invoke_endpoint)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "runtime = boto3.client(\"sagemaker-runtime\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# csv serialization\n",
    "response = runtime.invoke_endpoint(\n",
    "    EndpointName=endpoint_name,\n",
    "    Body=testX[data.feature_names].to_csv(header=False, index=False).encode(\"utf-8\"),\n",
    "    ContentType=\"text/csv\",\n",
    ")\n",
    "\n",
    "print(response[\"Body\"].read())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Launch the 2nd training job\n",
    "\n",
    "This will start the 2nd model training job. After training completes, you'll create a Model, an Endpoint Configuration, and update the existing Endpoint with the newly created Endpoint Configuration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "training_job_2_name = \"sklearn-boto3-2-\" + datetime.datetime.now().strftime(\"%Y-%m-%d-%H-%M-%S\")\n",
    "\n",
    "training_job_2_response = sm_boto3.create_training_job(\n",
    "    TrainingJobName=training_job_2_name,\n",
    "    HyperParameters={\n",
    "        \"n_estimators\": \"300\",\n",
    "        \"min_samples_leaf\": \"3\",\n",
    "        \"sagemaker_program\": \"script.py\",\n",
    "        \"features\": \"MedInc HouseAge AveRooms AveBedrms Population AveOccup Latitude Longitude\",\n",
    "        \"target\": \"target\",\n",
    "        \"sagemaker_submit_directory\": \"s3://\" + bucket + \"/\" + project + \"/\" + source,\n",
    "    },\n",
    "    AlgorithmSpecification={\n",
    "        \"TrainingImage\": training_image,\n",
    "        \"TrainingInputMode\": \"File\",\n",
    "        \"MetricDefinitions\": [\n",
    "            {\"Name\": \"median-AE\", \"Regex\": \"AE-at-50th-percentile: ([0-9.]+).*$\"},\n",
    "        ],\n",
    "    },\n",
    "    RoleArn=get_execution_role(),\n",
    "    InputDataConfig=[\n",
    "        {\n",
    "            \"ChannelName\": \"train\",\n",
    "            \"DataSource\": {\n",
    "                \"S3DataSource\": {\n",
    "                    \"S3DataType\": \"S3Prefix\",\n",
    "                    \"S3Uri\": trainpath,\n",
    "                    \"S3DataDistributionType\": \"FullyReplicated\",\n",
    "                }\n",
    "            },\n",
    "        },\n",
    "        {\n",
    "            \"ChannelName\": \"test\",\n",
    "            \"DataSource\": {\n",
    "                \"S3DataSource\": {\n",
    "                    \"S3DataType\": \"S3Prefix\",\n",
    "                    \"S3Uri\": testpath,\n",
    "                    \"S3DataDistributionType\": \"FullyReplicated\",\n",
    "                }\n",
    "            },\n",
    "        },\n",
    "    ],\n",
    "    OutputDataConfig={\"S3OutputPath\": \"s3://\" + bucket + \"/sagemaker-sklearn-artifact/\"},\n",
    "    ResourceConfig={\"InstanceType\": \"ml.c5.xlarge\", \"InstanceCount\": 1, \"VolumeSizeInGB\": 10},\n",
    "    StoppingCondition={\"MaxRuntimeInSeconds\": 86400},\n",
    "    EnableNetworkIsolation=False,\n",
    ")\n",
    "\n",
    "training_job_2_response"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Wait for the 2nd training job to end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import boto3\n",
    "import time\n",
    "\n",
    "client = boto3.client(\"sagemaker\")\n",
    "\n",
    "training_job_2_details = client.describe_training_job(TrainingJobName=training_job_2_name)\n",
    "\n",
    "while training_job_2_details[\"TrainingJobStatus\"] == \"InProgress\":\n",
    "    training_job_2_details = client.describe_training_job(TrainingJobName=training_job_2_name)\n",
    "    print(training_job_2_details[\"TrainingJobStatus\"])\n",
    "    time.sleep(15)\n",
    "\n",
    "training_job_2_details"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Create a Model for the 2nd training job"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "model_2_name = \"sklearn-model-2-\" + datetime.datetime.now().strftime(\"%Y-%m-%d-%H-%M-%S\")\n",
    "\n",
    "create_model_2_response = client.create_model(\n",
    "    ModelName=model_2_name,\n",
    "    PrimaryContainer={\n",
    "        \"Image\": training_job_2_details[\"AlgorithmSpecification\"][\"TrainingImage\"],\n",
    "        \"Mode\": \"SingleModel\",\n",
    "        \"ModelDataUrl\": training_job_2_details[\"ModelArtifacts\"][\"S3ModelArtifacts\"],\n",
    "        \"Environment\": {\n",
    "            \"SAGEMAKER_CONTAINER_LOG_LEVEL\": \"20\",\n",
    "            \"SAGEMAKER_PROGRAM\": training_job_2_details[\"HyperParameters\"][\"sagemaker_program\"],\n",
    "            \"SAGEMAKER_REGION\": region,\n",
    "            \"SAGEMAKER_SUBMIT_DIRECTORY\": training_job_2_details[\"HyperParameters\"][\n",
    "                \"sagemaker_submit_directory\"\n",
    "            ],\n",
    "        },\n",
    "    },\n",
    "    ExecutionRoleArn=get_execution_role(),\n",
    ")\n",
    "\n",
    "create_model_2_response"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "client.describe_model(ModelName=model_2_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Create an Endpoint Config from 2nd model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "endpoint_config_2_name = \"sklearn-endpoint-config-2-\" + datetime.datetime.now().strftime(\n",
    "    \"%Y-%m-%d-%H-%M-%S\"\n",
    ")\n",
    "\n",
    "endpoint_config_2_response = client.create_endpoint_config(\n",
    "    EndpointConfigName=endpoint_config_2_name,\n",
    "    ProductionVariants=[\n",
    "        {\n",
    "            \"VariantName\": \"AllTrafficVariant\",\n",
    "            \"ModelName\": model_2_name,\n",
    "            \"InitialInstanceCount\": 1,\n",
    "            \"InstanceType\": \"ml.c5.large\",\n",
    "            \"InitialVariantWeight\": 1,\n",
    "        },\n",
    "    ],\n",
    ")\n",
    "\n",
    "endpoint_config_2_response"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Update the real-time endpoint with the 2nd Endpoint Config\n",
    "\n",
    "This will deploy the new `EndpointConfig` specified in the request, switches to using newly created endpoint, and then deletes resources provisioned for the endpoint using the previous `EndpointConfig` (there is no availability loss).\n",
    "\n",
    "More info on `update_endpoint` can be found on the [Boto3 SageMaker documentation page](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html#SageMaker.Client.update_endpoint)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "update_endpoint_response = client.update_endpoint(\n",
    "    EndpointName=endpoint_name, EndpointConfigName=endpoint_config_2_name\n",
    ")\n",
    "\n",
    "update_endpoint_response"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Wait for Endpoint to be ready\n",
    "\n",
    "Navigating to the SageMaker Endpoints, in `SageMaker Components and registries` tab, you'll see the endpoint in `Updating` status."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "describe_endpoint_response = client.describe_endpoint(EndpointName=endpoint_name)\n",
    "\n",
    "while describe_endpoint_response[\"EndpointStatus\"] == \"Updating\":\n",
    "    describe_endpoint_response = client.describe_endpoint(EndpointName=endpoint_name)\n",
    "    print(describe_endpoint_response[\"EndpointStatus\"])\n",
    "    time.sleep(15)\n",
    "\n",
    "describe_endpoint_response"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Invoke Endpoint with `boto3`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "runtime = boto3.client(\"sagemaker-runtime\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# csv serialization\n",
    "response = runtime.invoke_endpoint(\n",
    "    EndpointName=endpoint_name,\n",
    "    Body=testX[data.feature_names].to_csv(header=False, index=False).encode(\"utf-8\"),\n",
    "    ContentType=\"text/csv\",\n",
    ")\n",
    "\n",
    "print(response[\"Body\"].read())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Clean up\n",
    "\n",
    "Endpoints should be deleted when no longer in use, since (per the [SageMaker pricing page](https://aws.amazon.com/sagemaker/pricing/)) they're billed by time deployed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "sm_boto3.delete_endpoint(EndpointName=endpoint_name)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Notebook CI Test Results\n",
    "\n",
    "This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.\n",
    "\n",
    "![This us-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-east-1/boto3|built-in-frameworks|scikit-learn|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint.ipynb)\n",
    "\n",
    "![This us-east-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-east-2/boto3|built-in-frameworks|scikit-learn|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint.ipynb)\n",
    "\n",
    "![This us-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-west-1/boto3|built-in-frameworks|scikit-learn|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint.ipynb)\n",
    "\n",
    "![This ca-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ca-central-1/boto3|built-in-frameworks|scikit-learn|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint.ipynb)\n",
    "\n",
    "![This sa-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/sa-east-1/boto3|built-in-frameworks|scikit-learn|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint.ipynb)\n",
    "\n",
    "![This eu-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-1/boto3|built-in-frameworks|scikit-learn|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint.ipynb)\n",
    "\n",
    "![This eu-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-2/boto3|built-in-frameworks|scikit-learn|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint.ipynb)\n",
    "\n",
    "![This eu-west-3 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-3/boto3|built-in-frameworks|scikit-learn|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint.ipynb)\n",
    "\n",
    "![This eu-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-central-1/boto3|built-in-frameworks|scikit-learn|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint.ipynb)\n",
    "\n",
    "![This eu-north-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-north-1/boto3|built-in-frameworks|scikit-learn|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint.ipynb)\n",
    "\n",
    "![This ap-southeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-southeast-1/boto3|built-in-frameworks|scikit-learn|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint.ipynb)\n",
    "\n",
    "![This ap-southeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-southeast-2/boto3|built-in-frameworks|scikit-learn|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint.ipynb)\n",
    "\n",
    "![This ap-northeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-northeast-1/boto3|built-in-frameworks|scikit-learn|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint.ipynb)\n",
    "\n",
    "![This ap-northeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-northeast-2/boto3|built-in-frameworks|scikit-learn|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint.ipynb)\n",
    "\n",
    "![This ap-south-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-south-1/boto3|built-in-frameworks|scikit-learn|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint|boto3_scikit_retrain_model_and_deploy_to_existing_endpoint.ipynb)\n"
   ]
  }
 ],
 "metadata": {
  "instance_type": "ml.t3.medium",
  "kernelspec": {
   "display_name": "Python 3 (Data Science 3.0)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/sagemaker-data-science-310-v1"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}