sagemaker_model_monitor/model_monitor_batch_transform/SageMaker-ModelMonitoring-Batch-Transform-Model-Quality-With-SageMaker-Pipelines-On-Demand.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "7c45aaa2-2b56-46ad-bc27-b934505c6655",
   "metadata": {},
   "source": [
    "# SageMaker Model Quality Model Monitor for Batch Transform With SageMaker Pipelines On-demand\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dfc59f29",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook. \n",
    "\n",
    "![This us-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-west-2/sagemaker_model_monitor|model_monitor_batch_transform|SageMaker-ModelMonitoring-Batch-Transform-Model-Quality-With-SageMaker-Pipelines-On-Demand.ipynb)\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "627fa214",
   "metadata": {},
   "source": [
    "\n",
    "In this notebook, we use SageMaker Pipelines and SageMaker Model Monitor to monitor the data quality of a batch transform job.\n",
    "\n",
    "Model quality monitoring jobs monitor the performance of a model by comparing the predictions that the model makes with the actual ground truth labels that the model attempts to predict. To do this, model quality monitoring merges data that is captured from real-time inference with actual labels that you store in an Amazon S3 bucket, and then compares the predictions with the actual labels.\n",
    "\n",
    "We introduce a new step type called `MonitorBatchTransformStep` to do this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25e816d3",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "92c1be97-d59b-4350-899d-711d11b3ee6d",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import sys\n",
    "\n",
    "! pip install --upgrade pip\n",
    "!{sys.executable} -m pip install sagemaker==2.114.0\n",
    "!{sys.executable} -m pip install -U boto3"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed838ed7-19c9-40cd-b755-b9b0ec562a9f",
   "metadata": {},
   "source": [
    "If you run this notebook in SageMaker Studio, you need to make sure latest python SDK is installed and restart the kernel, so please uncomment the code in the next cell, and run it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "37dd8d11-2787-4869-a0a3-656dca4cf9e0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# import IPython\n",
    "# IPython.Application.instance().kernel.do_shutdown(True)  # has to restart kernel so changes are used"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "143c2a4e-0893-48e3-88ef-ade782320025",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import boto3\n",
    "import re\n",
    "import time\n",
    "import json\n",
    "from datetime import datetime, timedelta\n",
    "from sagemaker import get_execution_role, session\n",
    "import pandas as pd\n",
    "\n",
    "region = boto3.Session().region_name\n",
    "\n",
    "role = get_execution_role()\n",
    "print(\"RoleArn: {}\".format(role))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8a85c631-eeb0-477e-8764-30ad144967d8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# You can use a different bucket, but make sure the role you chose for this notebook\n",
    "# has the s3:PutObject permissions. This is the bucket into which the data is captured\n",
    "bucket = session.Session(boto3.Session()).default_bucket()\n",
    "print(\"Demo Bucket: {}\".format(bucket))\n",
    "prefix = f\"sagemaker/model-monitor-batch-transform/model-quality/{int(time.time())}\"\n",
    "\n",
    "reports_prefix = \"{}/reports\".format(prefix)\n",
    "s3_report_path = \"s3://{}/{}\".format(bucket, reports_prefix)\n",
    "\n",
    "transform_output_path = \"s3://{}/{}/transform-outputs\".format(bucket, prefix)\n",
    "\n",
    "print(\"Transform Output path: {}\".format(transform_output_path))\n",
    "print(\"Report path: {}\".format(s3_report_path))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9bf944ae-cdb4-42c6-bd68-870293ab87d7",
   "metadata": {},
   "source": [
    "## Construct a SageMaker Pipeline"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07c98c24-d507-46be-9992-0e33fb7f05cf",
   "metadata": {},
   "source": [
    "Amazon SageMaker Model Building Pipelines is a tool for building machine learning pipelines that take advantage of direct SageMaker integration. We can leverage it to run batch transform job with monitoring on-demand.\n",
    "\n",
    "In this notebook, we showcase how to use SageMaker Pipeline to orchestrate the on-schedule batch inference monitoring. In summary, we create and execute a pipeline to:\n",
    "\n",
    "- Create a model\n",
    "- Run a batch inference with the model\n",
    "- Run a model monitoring job to evaluate the inference inputs/outputs.\n",
    "\n",
    "Note that in order to run on-demand Model Quality monitoring, we need to have the ground truth ready."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1d580d4a-ceb3-4913-a0f7-b7b1d58c74b3",
   "metadata": {},
   "outputs": [],
   "source": [
    "from time import gmtime, strftime\n",
    "import sagemaker\n",
    "from sagemaker.model import Model\n",
    "from sagemaker.image_uris import retrieve\n",
    "from sagemaker.workflow.pipeline_context import PipelineSession\n",
    "from sagemaker.workflow.model_step import ModelStep"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e331aa1e-dcf9-4ad1-b79a-a494a9aca934",
   "metadata": {},
   "outputs": [],
   "source": [
    "pipeline_session = PipelineSession()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "54cb6251-8851-4405-9109-3e3bc8b39f3b",
   "metadata": {},
   "source": [
    "### Create a model\n",
    "\n",
    "Here we take a pretrained model and upload it to S3. We use this model in our batch transform step."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9107256d-36af-4021-8e68-b3b871d1cf71",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_file_name = \"xgb-churn-prediction-model.tar.gz\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "42b24a5a-c51e-4cfe-a83d-fb195c57014a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!aws s3 cp model/{model_file_name} s3://{bucket}/{prefix}/{model_file_name}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0acb7ab6-4e91-47a6-9fdb-02ef95bfb92b",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_name = \"DEMO-xgb-churn-pred-model-monitor-\" + strftime(\"%Y-%m-%d-%H-%M-%S\", gmtime())\n",
    "model_url = \"https://{}.s3-{}.amazonaws.com/{}/{}\".format(bucket, region, prefix, model_file_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9f2fe7d7-77cd-4759-9431-c59cf1a77459",
   "metadata": {},
   "outputs": [],
   "source": [
    "image_uri = retrieve(\"xgboost\", boto3.Session().region_name, \"0.90-1\")\n",
    "\n",
    "model = Model(\n",
    "    image_uri=image_uri,\n",
    "    model_data=model_url,\n",
    "    role=role,\n",
    "    sagemaker_session=pipeline_session,\n",
    ")\n",
    "\n",
    "create_model_step = ModelStep(\n",
    "    name=\"CreateXGBoostModelStep\",\n",
    "    step_args=model.create(),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e66b4db2-b46c-41e0-b421-7a900994b083",
   "metadata": {},
   "source": [
    "### Configure a transformer\n",
    "\n",
    "We define a transformer object to be used in the `MonitorBatchTransformStep`.\n",
    "\n",
    "We upload a validation dataset to S3 that are  used by the batch transform job. We use this dataset in the transform job to generate predictions that can be used as a "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7165cde3-bbf7-4c6b-9191-9ff585a2c4cb",
   "metadata": {},
   "outputs": [],
   "source": [
    "!aws s3 cp test_data/validation.csv s3://{bucket}/{prefix}/transform_input/validation/validation.csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2bde2548-3515-43dc-8fb1-44a4792df8a4",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.transformer import Transformer\n",
    "from sagemaker.workflow.parameters import ParameterString"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4db938f8-1b48-4b6d-830d-eb837a423988",
   "metadata": {},
   "outputs": [],
   "source": [
    "transformer = Transformer(\n",
    "    model_name=create_model_step.properties.ModelName,\n",
    "    instance_count=1,\n",
    "    instance_type=\"ml.m5.xlarge\",\n",
    "    accept=\"text/csv\",\n",
    "    assemble_with=\"Line\",\n",
    "    output_path=transform_output_path,\n",
    "    sagemaker_session=pipeline_session,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3ec62458-a82f-4300-9f83-9d06ca680f0a",
   "metadata": {},
   "outputs": [],
   "source": [
    "transform_input_param = ParameterString(\n",
    "    name=\"transform_input\",\n",
    "    default_value=f\"s3://{bucket}/{prefix}/transform_input/validation\",\n",
    ")\n",
    "transform_arg = transformer.transform(\n",
    "    transform_input_param,\n",
    "    content_type=\"text/csv\",\n",
    "    split_type=\"Line\",\n",
    "    join_source=\"Input\",\n",
    "    # exclude the ground truth (first column) from the validation set\n",
    "    # when doing inference.\n",
    "    input_filter=\"$[1:]\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d49fdb4-f276-4964-8184-38c4b8eab01f",
   "metadata": {},
   "source": [
    "### Configure model quality monitoring\n",
    "\n",
    "In this section, we first run a baseline job, and use the suggested constraints and statistics as the baseline for running the model quality monitoring job during pipeline execution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "88651d54-79ae-4c1f-8af0-58a372aa1531",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.model_monitor import ModelQualityMonitor\n",
    "from sagemaker.model_monitor.dataset_format import DatasetFormat\n",
    "from sagemaker.workflow.check_job_config import CheckJobConfig\n",
    "from sagemaker.workflow.quality_check_step import ModelQualityCheckConfig"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dc2aba94-71ba-45fe-9e69-e09eaeb101b5",
   "metadata": {},
   "outputs": [],
   "source": [
    "# copy over the training dataset to Amazon S3 (if you already have it in Amazon S3, you could reuse it)\n",
    "baseline_prefix = prefix + \"/baselining\"\n",
    "baseline_data_prefix = baseline_prefix + \"/data\"\n",
    "baseline_results_prefix = baseline_prefix + \"/results\"\n",
    "\n",
    "baseline_data_uri = \"s3://{}/{}\".format(bucket, baseline_data_prefix)\n",
    "baseline_results_uri = \"s3://{}/{}\".format(bucket, baseline_results_prefix)\n",
    "print(\"Baseline data uri: {}\".format(baseline_data_uri))\n",
    "print(\"Baseline results uri: {}\".format(baseline_results_uri))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "88c4db16-decb-40ef-982d-bfcb6aa3bbe9",
   "metadata": {},
   "source": [
    "### Generate a baseline for Model Monitor\n",
    "\n",
    "We use the training dataset called `training-dataset-with-header.csv` to generate a baseline that are used by the Data Quality Monitor. To do this, we use the `suggest_baseline` method, the purpose of this is to generate a set of  `statistics` and `constraints` file. These files are used by Model Monitor to compare the data passed to the Transform job and report any violations that are detected.\n",
    "\n",
    "The `suggest_baseline` method has an argument called `baseline_dataset`. This is typically the dataset used during training. \n",
    "\n",
    "Model Quality Monitor uses the labels in the baseline dataset to establish a ground truth so that it can compare this against the predictions from the batch transform step. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7b3271c6-e623-42e0-9375-1280a4f24f26",
   "metadata": {},
   "outputs": [],
   "source": [
    "training_data_file = \"test_data/training-dataset-with-header.csv\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dbfecc94-f063-4b72-9328-2c28cb5e220e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Dataset used to generate statistics and constraints file\n",
    "\n",
    "!aws s3 cp {training_data_file} {baseline_data_uri}/training-dataset-with-header.csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9989b2bc-3eeb-4e2c-8325-bbc8fc6239b0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ground truth will be considered as the labels in the training dataset\n",
    "\n",
    "my_default_monitor = ModelQualityMonitor(\n",
    "    role=role,\n",
    "    instance_count=1,\n",
    "    instance_type=\"ml.m5.xlarge\",\n",
    "    volume_size_in_gb=30,\n",
    "    max_runtime_in_seconds=1800,\n",
    ")\n",
    "\n",
    "my_default_monitor.suggest_baseline(\n",
    "    baseline_dataset=baseline_data_uri + \"/training-dataset-with-header.csv\",\n",
    "    problem_type=\"BinaryClassification\",\n",
    "    dataset_format=DatasetFormat.csv(header=True),\n",
    "    output_s3_uri=baseline_results_uri,\n",
    "    ground_truth_attribute=\"Churn\",\n",
    "    # for demonstration purpose, we set the inference_attribute the same as ground_truth_attribute\n",
    "    # but realistically, we recommend running model against the training dataset inputs, and use\n",
    "    # it as the inference attribute value\n",
    "    inference_attribute=\"Churn\",\n",
    "    wait=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "174a3530-4ae6-4f20-8580-280c75e0ef36",
   "metadata": {},
   "outputs": [],
   "source": [
    "s3_client = boto3.Session().client(\"s3\")\n",
    "result = s3_client.list_objects(Bucket=bucket, Prefix=baseline_results_prefix)\n",
    "report_files = [report_file.get(\"Key\") for report_file in result.get(\"Contents\")]\n",
    "print(\"Found Files:\")\n",
    "print(\"\\n \".join(report_files))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c2435625-2b22-4709-9a50-7e6d03909cda",
   "metadata": {},
   "outputs": [],
   "source": [
    "baseline_job = my_default_monitor.latest_baselining_job\n",
    "schema_df = pd.io.json.json_normalize(\n",
    "    baseline_job.baseline_statistics().body_dict[\"binary_classification_metrics\"]\n",
    ")\n",
    "schema_df.transpose().head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fc5b8cc4-e1c6-48ae-9dfb-611d18c4b868",
   "metadata": {},
   "outputs": [],
   "source": [
    "constraints_df = pd.io.json.json_normalize(\n",
    "    baseline_job.suggested_constraints().body_dict[\"binary_classification_constraints\"]\n",
    ")\n",
    "constraints_df.transpose().head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d3696112-b15c-470f-8720-239f738234b0",
   "metadata": {},
   "outputs": [],
   "source": [
    "statistics_path = \"{}/statistics.json\".format(baseline_results_uri)\n",
    "constraints_path = \"{}/constraints.json\".format(baseline_results_uri)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9dd13b30-59bd-45c4-a2d3-134f11e66d25",
   "metadata": {},
   "source": [
    "### Configure the Model Quality check\n",
    "\n",
    "There are two configurations we create here, one is `CheckJobConfig` and the other is `ModelQualityCheckConfig`. The `CheckJobConfig` is used to configure the underlying processing job used by Model Monitor. This is where users can specify the role, instance type, etc. \n",
    "\n",
    "The `ModelQualityCheckConfig` is used to configure how Model Monitor runs the model quality check. It accepts an argument called `baseline_dataset`. This is the dataset that is passed to the transform job. The dataset passed here is compared against the baseline and statistics file generated by the `suggest_baseline` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8723c00c-2443-4a3c-9bc6-6e62e4abdf8c",
   "metadata": {},
   "outputs": [],
   "source": [
    "job_config = CheckJobConfig(role=role)\n",
    "model_quality_config = ModelQualityCheckConfig(\n",
    "    # The dataset we want to run evaluation against\n",
    "    # in this example, this is the same as the transform input\n",
    "    baseline_dataset=transformer.output_path,\n",
    "    problem_type=\"BinaryClassification\",\n",
    "    dataset_format=DatasetFormat.csv(header=False),\n",
    "    output_s3_uri=s3_report_path,\n",
    "    # since we joined the transform input and output, the output will be\n",
    "    # following the input. There are 1 column for ground truth, and 69 for input features\n",
    "    # so the index (0-based) for the output (inference prediction) will be 70\n",
    "    probability_attribute=\"_c70\",\n",
    "    probability_threshold_attribute=0.5,\n",
    "    # remeber that the ground truth in the validation set is the ground truth\n",
    "    ground_truth_attribute=\"_c0\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c4dc1571-80fd-459e-a52e-503ca1c68a70",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.workflow.monitor_batch_transform_step import MonitorBatchTransformStep"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b913e7c9-23d8-43a0-86aa-3cbf34e8a83f",
   "metadata": {},
   "source": [
    "### Use the `MonitorBatchTransformStep` to monitor the transform job\n",
    "\n",
    "This step runs a batch transform job using the transformer object configured above and monitors the data passed to the transformer before executing the job.\n",
    "\n",
    "The baselines calculated above must be passed to this step so that the incoming data can be compared against them to detect violations.\n",
    "\n",
    "You can configure the step to fail if a violation to Model Quality is found by toggling the `fail_on_violation` flag."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7d7102b0-0dfc-4a1c-af26-fb72685f31d9",
   "metadata": {},
   "outputs": [],
   "source": [
    "transform_and_monitor_step = MonitorBatchTransformStep(\n",
    "    name=\"MonitorCustomerChurnModelQuality\",\n",
    "    transform_step_args=transform_arg,\n",
    "    monitor_configuration=model_quality_config,\n",
    "    check_job_configuration=job_config,\n",
    "    # Since when we are generating the baselines using ground truth\n",
    "    # any test data, there are violations for sure. Let's\n",
    "    # fail the pipeline execution.\n",
    "    fail_on_violation=True,\n",
    "    supplied_baseline_constraints=constraints_path,\n",
    "    supplied_baseline_statistics=statistics_path,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f2be055-fd91-4b19-bf81-61f625b644b3",
   "metadata": {},
   "source": [
    "### Create and run the pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6f2daa07-7d1b-4af7-b7d0-8f0235cd3047",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.workflow.pipeline import Pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c374f90f-94db-46c7-a590-346dd8e77a5c",
   "metadata": {},
   "outputs": [],
   "source": [
    "pipeline = Pipeline(\n",
    "    name=\"MonitorModelQualityBatchTransformPipeline\",\n",
    "    parameters=[transform_input_param],\n",
    "    steps=[create_model_step, transform_and_monitor_step],\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b4415f1e-0e79-40ee-ac31-3dcd5eb3f45f",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "pipeline.upsert(role_arn=role)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9c18964-3cac-4654-9995-5976adb9de1a",
   "metadata": {},
   "source": [
    "### Start a pipeline execution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8a13b4e9-b601-4d91-b6c7-7bbd83fb99c5",
   "metadata": {},
   "outputs": [],
   "source": [
    "execution = pipeline.start()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "69f5e4a0-0f29-45a3-8bb5-ae20d76e57f4",
   "metadata": {},
   "outputs": [],
   "source": [
    "execution.wait()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "247bb7a6-7a71-4032-895c-0d4e451c0390",
   "metadata": {},
   "source": [
    "### Read the model monitor reports\n",
    "\n",
    "The execution fails as we set `fail_on_violation` to be `True`. Let's take a look at the reports. \n",
    "\n",
    "You must wait for the pipeline to finish executing before you can read the violation reports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "89a0876e-e0d0-4d3d-b4a1-f9ca6437975f",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.model_monitor import MonitoringExecution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1b32bf7d-2fb9-4c53-9e9d-ebfdbe41a8be",
   "metadata": {},
   "outputs": [],
   "source": [
    "monitoring_step = [step for step in execution.list_steps() if \"QualityCheck\" in step[\"Metadata\"]][0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dd1e6b91-7e76-4cf5-94d3-72791c127a11",
   "metadata": {},
   "outputs": [],
   "source": [
    "monitoring = MonitoringExecution.from_processing_arn(\n",
    "    sagemaker_session=pipeline_session,\n",
    "    processing_job_arn=monitoring_step[\"Metadata\"][\"QualityCheck\"][\"CheckJobArn\"],\n",
    ")\n",
    "violation = monitoring.constraint_violations(file_name=\"constraint_violations.json\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "704c26ab-21bd-484d-bde3-c02d4b10418e",
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.set_option(\"display.max_colwidth\", -1)\n",
    "\n",
    "constraints_df = pd.io.json.json_normalize(violation.body_dict[\"violations\"])\n",
    "constraints_df.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "46833a83-bc8e-443c-be33-f168bba2d457",
   "metadata": {},
   "source": [
    "### Other commands\n",
    "We can also start and stop the monitoring schedules."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d321bec8-2a75-4b37-b4c0-1cc1c073da3d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# my_default_monitor.stop_monitoring_schedule()\n",
    "# my_default_monitor.start_monitoring_schedule()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed10f4ef-18b8-47e8-b616-851d5805bdf5",
   "metadata": {},
   "source": [
    "### Delete the resources\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b115de4b-558e-4500-8749-e0d688149c3d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# my_default_monitor.stop_monitoring_schedule()\n",
    "# my_default_monitor.delete_monitoring_schedule()\n",
    "# time.sleep(60)  # actually wait for the deletion"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "704ff03b",
   "metadata": {},
   "source": [
    "## Notebook CI Test Results\n",
    "\n",
    "This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.\n",
    "\n",
    "![This us-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-east-1/sagemaker_model_monitor|model_monitor_batch_transform|SageMaker-ModelMonitoring-Batch-Transform-Model-Quality-With-SageMaker-Pipelines-On-Demand.ipynb)\n",
    "\n",
    "![This us-east-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-east-2/sagemaker_model_monitor|model_monitor_batch_transform|SageMaker-ModelMonitoring-Batch-Transform-Model-Quality-With-SageMaker-Pipelines-On-Demand.ipynb)\n",
    "\n",
    "![This us-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-west-1/sagemaker_model_monitor|model_monitor_batch_transform|SageMaker-ModelMonitoring-Batch-Transform-Model-Quality-With-SageMaker-Pipelines-On-Demand.ipynb)\n",
    "\n",
    "![This ca-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ca-central-1/sagemaker_model_monitor|model_monitor_batch_transform|SageMaker-ModelMonitoring-Batch-Transform-Model-Quality-With-SageMaker-Pipelines-On-Demand.ipynb)\n",
    "\n",
    "![This sa-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/sa-east-1/sagemaker_model_monitor|model_monitor_batch_transform|SageMaker-ModelMonitoring-Batch-Transform-Model-Quality-With-SageMaker-Pipelines-On-Demand.ipynb)\n",
    "\n",
    "![This eu-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-1/sagemaker_model_monitor|model_monitor_batch_transform|SageMaker-ModelMonitoring-Batch-Transform-Model-Quality-With-SageMaker-Pipelines-On-Demand.ipynb)\n",
    "\n",
    "![This eu-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-2/sagemaker_model_monitor|model_monitor_batch_transform|SageMaker-ModelMonitoring-Batch-Transform-Model-Quality-With-SageMaker-Pipelines-On-Demand.ipynb)\n",
    "\n",
    "![This eu-west-3 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-3/sagemaker_model_monitor|model_monitor_batch_transform|SageMaker-ModelMonitoring-Batch-Transform-Model-Quality-With-SageMaker-Pipelines-On-Demand.ipynb)\n",
    "\n",
    "![This eu-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-central-1/sagemaker_model_monitor|model_monitor_batch_transform|SageMaker-ModelMonitoring-Batch-Transform-Model-Quality-With-SageMaker-Pipelines-On-Demand.ipynb)\n",
    "\n",
    "![This eu-north-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-north-1/sagemaker_model_monitor|model_monitor_batch_transform|SageMaker-ModelMonitoring-Batch-Transform-Model-Quality-With-SageMaker-Pipelines-On-Demand.ipynb)\n",
    "\n",
    "![This ap-southeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-southeast-1/sagemaker_model_monitor|model_monitor_batch_transform|SageMaker-ModelMonitoring-Batch-Transform-Model-Quality-With-SageMaker-Pipelines-On-Demand.ipynb)\n",
    "\n",
    "![This ap-southeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-southeast-2/sagemaker_model_monitor|model_monitor_batch_transform|SageMaker-ModelMonitoring-Batch-Transform-Model-Quality-With-SageMaker-Pipelines-On-Demand.ipynb)\n",
    "\n",
    "![This ap-northeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-northeast-1/sagemaker_model_monitor|model_monitor_batch_transform|SageMaker-ModelMonitoring-Batch-Transform-Model-Quality-With-SageMaker-Pipelines-On-Demand.ipynb)\n",
    "\n",
    "![This ap-northeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-northeast-2/sagemaker_model_monitor|model_monitor_batch_transform|SageMaker-ModelMonitoring-Batch-Transform-Model-Quality-With-SageMaker-Pipelines-On-Demand.ipynb)\n",
    "\n",
    "![This ap-south-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-south-1/sagemaker_model_monitor|model_monitor_batch_transform|SageMaker-ModelMonitoring-Batch-Transform-Model-Quality-With-SageMaker-Pipelines-On-Demand.ipynb)\n"
   ]
  }
 ],
 "metadata": {
  "availableInstances": [
   {
    "_defaultOrder": 0,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.t3.medium",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 1,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.t3.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 2,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.t3.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 3,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.t3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 4,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 5,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 6,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 7,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 8,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 9,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 10,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 11,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 12,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5d.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 13,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5d.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 14,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5d.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 15,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5d.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 16,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5d.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 17,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5d.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 18,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5d.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 19,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 20,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": true,
    "memoryGiB": 0,
    "name": "ml.geospatial.interactive",
    "supportedImageNames": [
     "sagemaker-geospatial-v1-0"
    ],
    "vcpuNum": 0
   },
   {
    "_defaultOrder": 21,
    "_isFastLaunch": true,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.c5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 22,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.c5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 23,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.c5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 24,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.c5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 25,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 72,
    "name": "ml.c5.9xlarge",
    "vcpuNum": 36
   },
   {
    "_defaultOrder": 26,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 96,
    "name": "ml.c5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 27,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 144,
    "name": "ml.c5.18xlarge",
    "vcpuNum": 72
   },
   {
    "_defaultOrder": 28,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.c5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 29,
    "_isFastLaunch": true,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g4dn.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 30,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g4dn.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 31,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g4dn.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 32,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g4dn.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 33,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g4dn.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 34,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g4dn.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 35,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 61,
    "name": "ml.p3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 36,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 244,
    "name": "ml.p3.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 37,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 488,
    "name": "ml.p3.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 38,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.p3dn.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 39,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.r5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 40,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.r5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 41,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.r5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 42,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.r5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 43,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.r5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 44,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.r5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 45,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.r5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 46,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.r5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 47,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 48,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 49,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 50,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 51,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 52,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 53,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.g5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 54,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.g5.48xlarge",
    "vcpuNum": 192
   },
   {
    "_defaultOrder": 55,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 56,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4de.24xlarge",
    "vcpuNum": 96
   }
  ],
  "kernelspec": {
   "display_name": "Python 3 (Data Science 2.0)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-west-2:236514542706:image/sagemaker-data-science-38"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}