   ml_ops/sm-pipelines_train_model_registry_deploy.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# SageMaker Pipelines\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook. \n",
    "\n",
    "![This us-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-west-2/ml_ops|sm-pipelines_train_model_registry_deploy.ipynb)\n",
    "\n",
    "---"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "\n",
    "The following notebook shows how to create an Amazon SageMaker Pipeline that builds and trains a **PipelineModel** consisting of a preprocessing SKLearn script followed by a TensorFlow model. The pipeline model is then registered to the Model Registry and deployed from there into a real-time endpoint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "!pip install -U sagemaker"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import time\n",
    "import boto3\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import sagemaker\n",
    "from sagemaker import get_execution_role\n",
    "from sagemaker.workflow.pipeline_context import PipelineSession"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "sess = boto3.Session()\n",
    "sm = sess.client(\"sagemaker\")\n",
    "role = get_execution_role()\n",
    "sagemaker_session = sagemaker.Session(boto_session=sess)\n",
    "bucket = sagemaker_session.default_bucket()\n",
    "default_bucket_prefix = sagemaker_session.default_bucket_prefix\n",
    "region = boto3.Session().region_name\n",
    "\n",
    "pipeline_session = PipelineSession()\n",
    "\n",
    "model_package_group_name = \"PipelineModelPackageGroup\"\n",
    "prefix = \"pipeline-model-example\"\n",
    "\n",
    "# If a default bucket prefix is specified, append it to the s3 path\n",
    "if default_bucket_prefix:\n",
    "    prefix = f\"{default_bucket_prefix}/{prefix}\"\n",
    "\n",
    "pipeline_name = \"serial-inference-pipeline\"  # SageMaker Pipeline name"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Download California Housing dataset and upload to Amazon S3\n",
    "\n",
    "We use the California housing dataset.\n",
    "\n",
    "More info on the dataset:\n",
    "\n",
    "This dataset was obtained from the StatLib repository. http://lib.stat.cmu.edu/datasets/\n",
    "\n",
    "The target variable is the median house value for California districts.\n",
    "\n",
    "This dataset was derived from the 1990 U.S. census, using one row per census block group. A block group is the smallest geographical unit for which the U.S. Census Bureau publishes sample data (a block group typically has a population of 600 to 3,000 people)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "data_dir = os.path.join(os.getcwd(), \"data\")\n",
    "os.makedirs(data_dir, exist_ok=True)\n",
    "\n",
    "raw_dir = os.path.join(os.getcwd(), \"data/raw\")\n",
    "os.makedirs(raw_dir, exist_ok=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "s3 = boto3.client(\"s3\")\n",
    "s3.download_file(\n",
    "    f\"sagemaker-example-files-prod-{region}\",\n",
    "    \"datasets/tabular/california_housing/cal_housing.tgz\",\n",
    "    \"cal_housing.tgz\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "!tar -zxf cal_housing.tgz"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "columns = [\n",
    "    \"longitude\",\n",
    "    \"latitude\",\n",
    "    \"housingMedianAge\",\n",
    "    \"totalRooms\",\n",
    "    \"totalBedrooms\",\n",
    "    \"population\",\n",
    "    \"households\",\n",
    "    \"medianIncome\",\n",
    "    \"medianHouseValue\",\n",
    "]\n",
    "cal_housing_df = pd.read_csv(\"CaliforniaHousing/cal_housing.data\", names=columns, header=None)\n",
    "cal_housing_df[\n",
    "    \"medianHouseValue\"\n",
    "] /= 500000  # Scaling target down to avoid overcomplicating the example\n",
    "cal_housing_df.to_csv(f\"./data/raw/raw_data_all.csv\", header=True, index=False)\n",
    "rawdata_s3_prefix = \"{}/data/raw\".format(prefix)\n",
    "raw_s3 = sagemaker_session.upload_data(path=\"./data/raw/\", key_prefix=rawdata_s3_prefix)\n",
    "print(raw_s3)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Define Parameters to Parametrize Pipeline Execution\n",
    "\n",
    "Define Pipeline parameters that you can use to parametrize the pipeline. Parameters enable custom pipeline executions and schedules without having to modify the Pipeline definition.\n",
    "\n",
    "The supported parameter types include:\n",
    "\n",
    "- ParameterString - represents a str Python type\n",
    "- ParameterInteger - represents an int Python type\n",
    "- ParameterFloat - represents a float Python type"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from sagemaker.workflow.parameters import ParameterInteger, ParameterString, ParameterFloat\n",
    "\n",
    "# raw input data\n",
    "input_data = ParameterString(name=\"InputData\", default_value=raw_s3)\n",
    "\n",
    "# status of newly trained model in registry\n",
    "model_approval_status = ParameterString(name=\"ModelApprovalStatus\", default_value=\"Approved\")\n",
    "\n",
    "# processing step parameters\n",
    "processing_instance_type = ParameterString(\n",
    "    name=\"ProcessingInstanceType\", default_value=\"ml.m5.xlarge\"\n",
    ")\n",
    "processing_instance_count = ParameterInteger(name=\"ProcessingInstanceCount\", default_value=1)\n",
    "\n",
    "# training step parameters\n",
    "training_instance_type = ParameterString(name=\"TrainingInstanceType\", default_value=\"ml.m5.xlarge\")\n",
    "training_epochs = ParameterString(name=\"TrainingEpochs\", default_value=\"100\")\n",
    "\n",
    "# model performance step parameters\n",
    "accuracy_mse_threshold = ParameterFloat(name=\"AccuracyMseThreshold\", default_value=0.75)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Define a Processing Step for Feature Engineering\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "The below preprocessing script, in addition to creating a scaler, contains the necessary functions for it to be deployed as part of a pipeline model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "!mkdir -p code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "%%writefile code/preprocess.py\n",
    "\n",
    "import glob\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import os\n",
    "import json\n",
    "import joblib\n",
    "from io import StringIO\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.model_selection import train_test_split\n",
    "import tarfile\n",
    "\n",
    "try:\n",
    "    from sagemaker_containers.beta.framework import (\n",
    "        content_types,\n",
    "        encoders,\n",
    "        env,\n",
    "        modules,\n",
    "        transformer,\n",
    "        worker,\n",
    "        server,\n",
    "    )\n",
    "except ImportError:\n",
    "    pass\n",
    "\n",
    "feature_columns = [\n",
    "    \"longitude\",\n",
    "    \"latitude\",\n",
    "    \"housingMedianAge\",\n",
    "    \"totalRooms\",\n",
    "    \"totalBedrooms\",\n",
    "    \"population\",\n",
    "    \"households\",\n",
    "    \"medianIncome\",\n",
    "]\n",
    "label_column = \"medianHouseValue\"\n",
    "\n",
    "base_dir = \"/opt/ml/processing\"\n",
    "base_output_dir = \"/opt/ml/output/\"\n",
    "\n",
    "if __name__ == \"__main__\":\n",
    "    df = pd.read_csv(f\"{base_dir}/input/raw_data_all.csv\")\n",
    "    feature_data = df.drop(label_column, axis=1, inplace=False)\n",
    "    label_data = df[label_column]\n",
    "    x_train, x_test, y_train, y_test = train_test_split(feature_data, label_data, test_size=0.33)\n",
    "\n",
    "    scaler = StandardScaler()\n",
    "\n",
    "    scaler.fit(x_train)\n",
    "    x_train = scaler.transform(x_train)\n",
    "    x_test = scaler.transform(x_test)\n",
    "\n",
    "    train_dataset = pd.concat([pd.DataFrame(x_train), y_train.reset_index(drop=True)], axis=1)\n",
    "    test_dataset = pd.concat([pd.DataFrame(x_test), y_test.reset_index(drop=True)], axis=1)\n",
    "\n",
    "    train_dataset.columns = feature_columns + [label_column]\n",
    "    test_dataset.columns = feature_columns + [label_column]\n",
    "\n",
    "    train_dataset.to_csv(f\"{base_dir}/train/train.csv\", header=True, index=False)\n",
    "    test_dataset.to_csv(f\"{base_dir}/test/test.csv\", header=True, index=False)\n",
    "    joblib.dump(scaler, \"model.joblib\")\n",
    "    with tarfile.open(f\"{base_dir}/scaler_model/model.tar.gz\", \"w:gz\") as tar_handle:\n",
    "        tar_handle.add(f\"model.joblib\")\n",
    "\n",
    "\n",
    "def input_fn(input_data, content_type):\n",
    "    \"\"\"Parse input data payload\n",
    "\n",
    "    We currently only take csv input. Since we need to process both labelled\n",
    "    and unlabelled data we first determine whether the label column is present\n",
    "    by looking at how many columns were provided.\n",
    "    \"\"\"\n",
    "    if content_type == \"text/csv\":\n",
    "        # Read the raw input data as CSV.\n",
    "        df = pd.read_csv(StringIO(input_data), header=None)\n",
    "\n",
    "        if len(df.columns) == len(feature_columns) + 1:\n",
    "            # This is a labelled example, includes the ring label\n",
    "            df.columns = feature_columns + [label_column]\n",
    "        elif len(df.columns) == len(feature_columns):\n",
    "            # This is an unlabelled example.\n",
    "            df.columns = feature_columns\n",
    "\n",
    "        return df\n",
    "    else:\n",
    "        raise ValueError(\"{} not supported by script!\".format(content_type))\n",
    "\n",
    "\n",
    "def output_fn(prediction, accept):\n",
    "    \"\"\"Format prediction output\n",
    "\n",
    "    The default accept/content-type between containers for serial inference is JSON.\n",
    "    We also want to set the ContentType or mimetype as the same value as accept so the next\n",
    "    container can read the response payload correctly.\n",
    "    \"\"\"\n",
    "    if accept == \"application/json\":\n",
    "        instances = []\n",
    "        for row in prediction.tolist():\n",
    "            instances.append(row)\n",
    "        json_output = {\"instances\": instances}\n",
    "\n",
    "        return worker.Response(json.dumps(json_output), mimetype=accept)\n",
    "    elif accept == \"text/csv\":\n",
    "        return worker.Response(encoders.encode(prediction, accept), mimetype=accept)\n",
    "    else:\n",
    "        raise RuntimeException(\"{} accept type is not supported by this script.\".format(accept))\n",
    "\n",
    "\n",
    "def predict_fn(input_data, model):\n",
    "    \"\"\"Preprocess input data\n",
    "\n",
    "    We implement this because the default predict_fn uses .predict(), but our model is a preprocessor\n",
    "    so we want to use .transform().\n",
    "\n",
    "    The output is returned in the following order:\n",
    "\n",
    "        rest of features either one hot encoded or standardized\n",
    "    \"\"\"\n",
    "    features = model.transform(input_data)\n",
    "\n",
    "    if label_column in input_data:\n",
    "        # Return the label (as the first column) and the set of features.\n",
    "        return np.insert(features, 0, input_data[label_column], axis=1)\n",
    "    else:\n",
    "        # Return only the set of features\n",
    "        return features\n",
    "\n",
    "\n",
    "def model_fn(model_dir):\n",
    "    \"\"\"Deserialize fitted model\"\"\"\n",
    "    preprocessor = joblib.load(os.path.join(model_dir, \"model.joblib\"))\n",
    "    return preprocessor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from sagemaker.sklearn.processing import SKLearnProcessor\n",
    "from sagemaker.processing import ProcessingInput, ProcessingOutput\n",
    "\n",
    "sklearn_framework_version = \"1.2-1\"\n",
    "\n",
    "sklearn_processor = SKLearnProcessor(\n",
    "    framework_version=sklearn_framework_version,\n",
    "    instance_type=\"ml.m5.large\",\n",
    "    instance_count=processing_instance_count,\n",
    "    base_job_name=\"sklearn-housing-data-process\",\n",
    "    role=role,\n",
    "    sagemaker_session=pipeline_session,\n",
    ")\n",
    "\n",
    "processor_args = sklearn_processor.run(\n",
    "    inputs=[\n",
    "        ProcessingInput(source=input_data, destination=\"/opt/ml/processing/input\"),\n",
    "    ],\n",
    "    outputs=[\n",
    "        ProcessingOutput(output_name=\"scaler_model\", source=\"/opt/ml/processing/scaler_model\"),\n",
    "        ProcessingOutput(output_name=\"train\", source=\"/opt/ml/processing/train\"),\n",
    "        ProcessingOutput(output_name=\"test\", source=\"/opt/ml/processing/test\"),\n",
    "    ],\n",
    "    code=\"code/preprocess.py\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from sagemaker.processing import ProcessingInput, ProcessingOutput\n",
    "from sagemaker.workflow.steps import ProcessingStep\n",
    "\n",
    "\n",
    "step_process = ProcessingStep(\n",
    "    name=\"PreprocessData\",\n",
    "    step_args=processor_args,\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Define a Training Step to Train a Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "%%writefile code/train.py\n",
    "\n",
    "import argparse\n",
    "import numpy as np\n",
    "import os\n",
    "import tensorflow as tf\n",
    "import pandas as pd\n",
    "\n",
    "feature_columns = [\n",
    "    \"longitude\",\n",
    "    \"latitude\",\n",
    "    \"housingMedianAge\",\n",
    "    \"totalRooms\",\n",
    "    \"totalBedrooms\",\n",
    "    \"population\",\n",
    "    \"households\",\n",
    "    \"medianIncome\",\n",
    "]\n",
    "label_column = \"medianHouseValue\"\n",
    "\n",
    "\n",
    "def parse_args():\n",
    "    parser = argparse.ArgumentParser()\n",
    "\n",
    "    # hyperparameters sent by the client are passed as command-line arguments to the script\n",
    "    parser.add_argument(\"--epochs\", type=int, default=1)\n",
    "    parser.add_argument(\"--batch_size\", type=int, default=64)\n",
    "    parser.add_argument(\"--learning_rate\", type=float, default=0.1)\n",
    "\n",
    "    # data directories\n",
    "    parser.add_argument(\"--train\", type=str, default=os.environ.get(\"SM_CHANNEL_TRAIN\"))\n",
    "    parser.add_argument(\"--test\", type=str, default=os.environ.get(\"SM_CHANNEL_TEST\"))\n",
    "\n",
    "    # model directory\n",
    "    parser.add_argument(\"--sm-model-dir\", type=str, default=os.environ.get(\"SM_MODEL_DIR\"))\n",
    "\n",
    "    return parser.parse_known_args()\n",
    "\n",
    "\n",
    "def get_train_data(train_dir):\n",
    "    train_data = pd.read_csv(os.path.join(train_dir, \"train.csv\"))\n",
    "    x_train = train_data[feature_columns].to_numpy()\n",
    "    y_train = train_data[label_column].to_numpy()\n",
    "    print(\"x train\", x_train.shape, \"y train\", y_train.shape)\n",
    "\n",
    "    return x_train, y_train\n",
    "\n",
    "\n",
    "def get_test_data(test_dir):\n",
    "    test_data = pd.read_csv(os.path.join(test_dir, \"test.csv\"))\n",
    "    x_test = test_data[feature_columns].to_numpy()\n",
    "    y_test = test_data[label_column].to_numpy()\n",
    "    print(\"x test\", x_test.shape, \"y test\", y_test.shape)\n",
    "\n",
    "    return x_test, y_test\n",
    "\n",
    "\n",
    "def get_model():\n",
    "    inputs = tf.keras.Input(shape=(8,))\n",
    "    hidden_1 = tf.keras.layers.Dense(8, activation=\"tanh\")(inputs)\n",
    "    hidden_2 = tf.keras.layers.Dense(4, activation=\"sigmoid\")(hidden_1)\n",
    "    outputs = tf.keras.layers.Dense(1)(hidden_2)\n",
    "    return tf.keras.Model(inputs=inputs, outputs=outputs)\n",
    "\n",
    "\n",
    "if __name__ == \"__main__\":\n",
    "    args, _ = parse_args()\n",
    "\n",
    "    print(\"Training data location: {}\".format(args.train))\n",
    "    print(\"Test data location: {}\".format(args.test))\n",
    "    x_train, y_train = get_train_data(args.train)\n",
    "    x_test, y_test = get_test_data(args.test)\n",
    "\n",
    "    batch_size = args.batch_size\n",
    "    epochs = args.epochs\n",
    "    learning_rate = args.learning_rate\n",
    "    print(\n",
    "        \"batch_size = {}, epochs = {}, learning rate = {}\".format(batch_size, epochs, learning_rate)\n",
    "    )\n",
    "\n",
    "    model = get_model()\n",
    "    optimizer = tf.keras.optimizers.SGD(learning_rate)\n",
    "    model.compile(optimizer=optimizer, loss=\"mse\")\n",
    "    model.fit(\n",
    "        x_train, y_train, batch_size=batch_size, epochs=epochs, validation_data=(x_test, y_test)\n",
    "    )\n",
    "\n",
    "    # evaluate on test set\n",
    "    scores = model.evaluate(x_test, y_test, batch_size, verbose=2)\n",
    "    print(\"\\nTest MSE :\", scores)\n",
    "\n",
    "    # save model\n",
    "    model.save(args.sm_model_dir + \"/1\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from sagemaker.tensorflow import TensorFlow\n",
    "from sagemaker.inputs import TrainingInput\n",
    "from sagemaker.workflow.steps import TrainingStep\n",
    "from sagemaker.workflow.model_step import ModelStep\n",
    "import time\n",
    "\n",
    "# Where to store the trained model\n",
    "model_path = f\"s3://{bucket}/{prefix}/model/\"\n",
    "\n",
    "hyperparameters = {\"epochs\": training_epochs}\n",
    "tensorflow_version = \"2.4.1\"\n",
    "python_version = \"py37\"\n",
    "\n",
    "tf2_estimator = TensorFlow(\n",
    "    source_dir=\"code\",\n",
    "    entry_point=\"train.py\",\n",
    "    instance_type=training_instance_type,\n",
    "    instance_count=1,\n",
    "    framework_version=tensorflow_version,\n",
    "    role=role,\n",
    "    base_job_name=\"tensorflow-train-model\",\n",
    "    output_path=model_path,\n",
    "    hyperparameters=hyperparameters,\n",
    "    py_version=python_version,\n",
    "    sagemaker_session=pipeline_session,\n",
    ")\n",
    "\n",
    "# NOTE how the input to the training job directly references the output of the previous step.\n",
    "train_args = tf2_estimator.fit(\n",
    "    inputs={\n",
    "        \"train\": TrainingInput(\n",
    "            s3_data=step_process.properties.ProcessingOutputConfig.Outputs[\"train\"].S3Output.S3Uri,\n",
    "            content_type=\"text/csv\",\n",
    "        ),\n",
    "        \"test\": TrainingInput(\n",
    "            s3_data=step_process.properties.ProcessingOutputConfig.Outputs[\"test\"].S3Output.S3Uri,\n",
    "            content_type=\"text/csv\",\n",
    "        ),\n",
    "    }\n",
    ")\n",
    "\n",
    "step_train_model = TrainingStep(name=\"TrainTensorflowModel\", step_args=train_args)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Define a Model Evaluation Step to Evaluate the Trained Model\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "%%writefile code/evaluate.py\n",
    "\n",
    "import os\n",
    "import json\n",
    "import sys\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import pathlib\n",
    "import tarfile\n",
    "\n",
    "\n",
    "feature_columns = [\n",
    "    \"longitude\",\n",
    "    \"latitude\",\n",
    "    \"housingMedianAge\",\n",
    "    \"totalRooms\",\n",
    "    \"totalBedrooms\",\n",
    "    \"population\",\n",
    "    \"households\",\n",
    "    \"medianIncome\",\n",
    "]\n",
    "label_column = \"medianHouseValue\"\n",
    "\n",
    "if __name__ == \"__main__\":\n",
    "    model_path = f\"/opt/ml/processing/model/model.tar.gz\"\n",
    "    with tarfile.open(model_path, \"r:gz\") as tar:\n",
    "        tar.extractall(\"./model\")\n",
    "    import tensorflow as tf\n",
    "\n",
    "    model = tf.keras.models.load_model(\"./model/1\")\n",
    "    test_path = \"/opt/ml/processing/test/\"\n",
    "    df = pd.read_csv(test_path + \"/test.csv\")\n",
    "    x_test = df[feature_columns].to_numpy()\n",
    "    y_test = df[label_column].to_numpy()\n",
    "    scores = model.evaluate(x_test, y_test, verbose=2)\n",
    "    print(\"\\nTest MSE :\", scores)\n",
    "\n",
    "    # Available metrics to add to model: https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-model-quality-metrics.html\n",
    "    report_dict = {\n",
    "        \"regression_metrics\": {\n",
    "            \"mse\": {\"value\": scores, \"standard_deviation\": \"NaN\"},\n",
    "        },\n",
    "    }\n",
    "\n",
    "    output_dir = \"/opt/ml/processing/evaluation\"\n",
    "    pathlib.Path(output_dir).mkdir(parents=True, exist_ok=True)\n",
    "\n",
    "    evaluation_path = f\"{output_dir}/evaluation.json\"\n",
    "    with open(evaluation_path, \"w\") as f:\n",
    "        f.write(json.dumps(report_dict))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from sagemaker.workflow.properties import PropertyFile\n",
    "from sagemaker.sklearn.processing import ScriptProcessor\n",
    "\n",
    "tf_eval_image_uri = sagemaker.image_uris.retrieve(\n",
    "    framework=\"tensorflow\",\n",
    "    region=region,\n",
    "    version=tensorflow_version,\n",
    "    image_scope=\"training\",\n",
    "    py_version=\"py37\",\n",
    "    instance_type=\"ml.m5.xlarge\",\n",
    ")\n",
    "\n",
    "evaluate_model_processor = ScriptProcessor(\n",
    "    role=role,\n",
    "    image_uri=tf_eval_image_uri,\n",
    "    command=[\"python3\"],\n",
    "    instance_count=1,\n",
    "    instance_type=processing_instance_type,\n",
    "    sagemaker_session=pipeline_session,\n",
    ")\n",
    "\n",
    "# Create a PropertyFile\n",
    "# A PropertyFile is used to be able to reference outputs from a processing step, for instance to use in a condition step.\n",
    "# For more information, visit https://docs.aws.amazon.com/sagemaker/latest/dg/build-and-manage-propertyfile.html\n",
    "evaluation_report = PropertyFile(\n",
    "    name=\"EvaluationReport\", output_name=\"evaluation\", path=\"evaluation.json\"\n",
    ")\n",
    "\n",
    "eval_args = evaluate_model_processor.run(\n",
    "    inputs=[\n",
    "        ProcessingInput(\n",
    "            source=step_train_model.properties.ModelArtifacts.S3ModelArtifacts,\n",
    "            destination=\"/opt/ml/processing/model\",\n",
    "        ),\n",
    "        ProcessingInput(\n",
    "            source=step_process.properties.ProcessingOutputConfig.Outputs[\"test\"].S3Output.S3Uri,\n",
    "            destination=\"/opt/ml/processing/test\",\n",
    "        ),\n",
    "    ],\n",
    "    outputs=[\n",
    "        ProcessingOutput(output_name=\"evaluation\", source=\"/opt/ml/processing/evaluation\"),\n",
    "    ],\n",
    "    code=\"code/evaluate.py\",\n",
    ")\n",
    "\n",
    "step_evaluate_model = ProcessingStep(\n",
    "    name=\"EvaluateModelPerformance\",\n",
    "    step_args=eval_args,\n",
    "    property_files=[evaluation_report],\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Define a Register Model Step to Create a Model Package for the PipelineModel\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from sagemaker.model import Model\n",
    "from sagemaker.sklearn.model import SKLearnModel\n",
    "from sagemaker import PipelineModel\n",
    "\n",
    "\n",
    "scaler_model_s3 = \"{}/model.tar.gz\".format(\n",
    "    step_process.arguments[\"ProcessingOutputConfig\"][\"Outputs\"][0][\"S3Output\"][\"S3Uri\"]\n",
    ")\n",
    "\n",
    "scaler_model = SKLearnModel(\n",
    "    model_data=scaler_model_s3,\n",
    "    role=role,\n",
    "    sagemaker_session=pipeline_session,\n",
    "    entry_point=\"code/preprocess.py\",\n",
    "    framework_version=sklearn_framework_version,\n",
    ")\n",
    "\n",
    "tf_model_image_uri = sagemaker.image_uris.retrieve(\n",
    "    framework=\"tensorflow\",\n",
    "    region=region,\n",
    "    version=tensorflow_version,\n",
    "    image_scope=\"inference\",\n",
    "    py_version=\"py37\",\n",
    "    instance_type=\"ml.m5.xlarge\",\n",
    ")\n",
    "\n",
    "tf_model = Model(\n",
    "    image_uri=tf_model_image_uri,\n",
    "    model_data=step_train_model.properties.ModelArtifacts.S3ModelArtifacts,\n",
    "    sagemaker_session=pipeline_session,\n",
    "    role=role,\n",
    ")\n",
    "\n",
    "pipeline_model = PipelineModel(\n",
    "    models=[scaler_model, tf_model], role=role, sagemaker_session=pipeline_session\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from sagemaker.model_metrics import MetricsSource, ModelMetrics\n",
    "from sagemaker.workflow.step_collections import RegisterModel\n",
    "\n",
    "\n",
    "evaluation_s3_uri = \"{}/evaluation.json\".format(\n",
    "    step_evaluate_model.arguments[\"ProcessingOutputConfig\"][\"Outputs\"][0][\"S3Output\"][\"S3Uri\"]\n",
    ")\n",
    "\n",
    "model_metrics = ModelMetrics(\n",
    "    model_statistics=MetricsSource(\n",
    "        s3_uri=evaluation_s3_uri,\n",
    "        content_type=\"application/json\",\n",
    "    )\n",
    ")\n",
    "\n",
    "register_args = pipeline_model.register(\n",
    "    content_types=[\"text/csv\"],\n",
    "    response_types=[\"text/csv\"],\n",
    "    inference_instances=[\"ml.m5.large\", \"ml.m5.xlarge\"],\n",
    "    transform_instances=[\"ml.m5.xlarge\"],\n",
    "    model_package_group_name=model_package_group_name,\n",
    "    model_metrics=model_metrics,\n",
    "    approval_status=model_approval_status,\n",
    ")\n",
    "\n",
    "step_register_pipeline_model = ModelStep(\n",
    "    name=\"PipelineModel\",\n",
    "    step_args=register_args,\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Define a Condition Step to Check Accuracy and Conditionally Register a Model in the Model Registry"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from sagemaker.workflow.conditions import ConditionLessThanOrEqualTo\n",
    "from sagemaker.workflow.condition_step import ConditionStep\n",
    "from sagemaker.workflow.functions import JsonGet\n",
    "\n",
    "# Create accuracy condition to ensure the model meets performance requirements.\n",
    "# Models with a test accuracy lower than the condition will not be registered with the model registry.\n",
    "cond_lte = ConditionLessThanOrEqualTo(\n",
    "    left=JsonGet(\n",
    "        step_name=step_evaluate_model.name,\n",
    "        property_file=evaluation_report,\n",
    "        json_path=\"regression_metrics.mse.value\",\n",
    "    ),\n",
    "    right=accuracy_mse_threshold,\n",
    ")\n",
    "\n",
    "# Create a Sagemaker Pipelines ConditionStep, using the condition above.\n",
    "# Enter the steps to perform if the condition returns True / False.\n",
    "step_cond = ConditionStep(\n",
    "    name=\"MSE-Lower-Than-Threshold-Condition\",\n",
    "    conditions=[cond_lte],\n",
    "    if_steps=[step_register_pipeline_model],  # step_register_model, step_register_scaler,\n",
    "    else_steps=[],\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Define a Pipeline of Parameters, Steps, and Conditions\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from sagemaker.workflow.pipeline import Pipeline\n",
    "\n",
    "# Create a Sagemaker Pipeline.\n",
    "# Each parameter for the pipeline must be set as a parameter explicitly when the pipeline is created.\n",
    "# Also pass in each of the steps created above.\n",
    "# Note that the order of execution is determined from each step's dependencies on other steps,\n",
    "# not on the order they are passed in below.\n",
    "pipeline = Pipeline(\n",
    "    name=pipeline_name,\n",
    "    parameters=[\n",
    "        training_instance_type,\n",
    "        processing_instance_type,\n",
    "        processing_instance_count,\n",
    "        input_data,\n",
    "        model_approval_status,\n",
    "        training_epochs,\n",
    "        accuracy_mse_threshold,\n",
    "    ],\n",
    "    steps=[step_process, step_train_model, step_evaluate_model, step_cond],\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import json\n",
    "\n",
    "definition = json.loads(pipeline.definition())\n",
    "definition"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Submit the pipeline to SageMaker and start execution¶\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "pipeline.upsert(role_arn=role)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "execution = pipeline.start()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "execution.wait()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Deploy latest approved model to a real-time endpoint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "%%writefile utils.py\n",
    "import argparse\n",
    "import boto3\n",
    "import logging\n",
    "import os\n",
    "from botocore.exceptions import ClientError\n",
    "import tarfile\n",
    "import zipfile\n",
    "\n",
    "logger = logging.getLogger(__name__)\n",
    "sm_client = boto3.client(\"sagemaker\")\n",
    "\n",
    "\n",
    "def get_approved_package(model_package_group_name):\n",
    "    \"\"\"Gets the latest approved model package for a model package group.\n",
    "\n",
    "    Args:\n",
    "        model_package_group_name: The model package group name.\n",
    "\n",
    "    Returns:\n",
    "        The SageMaker Model Package ARN.\n",
    "    \"\"\"\n",
    "    try:\n",
    "        # Get the latest approved model package\n",
    "        response = sm_client.list_model_packages(\n",
    "            ModelPackageGroupName=model_package_group_name,\n",
    "            ModelApprovalStatus=\"Approved\",\n",
    "            SortBy=\"CreationTime\",\n",
    "            MaxResults=100,\n",
    "        )\n",
    "        approved_packages = response[\"ModelPackageSummaryList\"]\n",
    "\n",
    "        # Fetch more packages if none returned with continuation token\n",
    "        while len(approved_packages) == 0 and \"NextToken\" in response:\n",
    "            logger.debug(\"Getting more packages for token: {}\".format(response[\"NextToken\"]))\n",
    "            response = sm_client.list_model_packages(\n",
    "                ModelPackageGroupName=model_package_group_name,\n",
    "                ModelApprovalStatus=\"Approved\",\n",
    "                SortBy=\"CreationTime\",\n",
    "                MaxResults=100,\n",
    "                NextToken=response[\"NextToken\"],\n",
    "            )\n",
    "            approved_packages.extend(response[\"ModelPackageSummaryList\"])\n",
    "\n",
    "        # Return error if no packages found\n",
    "        if len(approved_packages) == 0:\n",
    "            error_message = (\n",
    "                f\"No approved ModelPackage found for ModelPackageGroup: {model_package_group_name}\"\n",
    "            )\n",
    "            logger.error(error_message)\n",
    "            raise Exception(error_message)\n",
    "\n",
    "        # Return the pmodel package arn\n",
    "        model_package_arn = approved_packages[0][\"ModelPackageArn\"]\n",
    "        logger.info(f\"Identified the latest approved model package: {model_package_arn}\")\n",
    "        return approved_packages[0]\n",
    "        # return model_package_arn\n",
    "    except ClientError as e:\n",
    "        error_message = e.response[\"Error\"][\"Message\"]\n",
    "        logger.error(error_message)\n",
    "        raise Exception(error_message)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from utils import get_approved_package\n",
    "\n",
    "sm_client = boto3.client(\"sagemaker\")\n",
    "\n",
    "pck = get_approved_package(\n",
    "    model_package_group_name\n",
    ")  # Reminder: model_package_group_name was defined as \"NominetAbaloneModelPackageGroupName\" at the beginning of the pipeline definition\n",
    "model_description = sm_client.describe_model_package(ModelPackageName=pck[\"ModelPackageArn\"])\n",
    "\n",
    "model_description"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from sagemaker import ModelPackage\n",
    "\n",
    "model_package_arn = model_description[\"ModelPackageArn\"]\n",
    "model = ModelPackage(\n",
    "    role=role, model_package_arn=model_package_arn, sagemaker_session=sagemaker_session\n",
    ")\n",
    "\n",
    "endpoint_name = \"DEMO-endpoint-\" + time.strftime(\"%Y-%m-%d-%H-%M-%S\", time.gmtime())\n",
    "print(\"EndpointName= {}\".format(endpoint_name))\n",
    "model.deploy(initial_instance_count=1, instance_type=\"ml.m5.xlarge\", endpoint_name=endpoint_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from sagemaker.predictor import Predictor\n",
    "\n",
    "predictor = Predictor(endpoint_name=endpoint_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "data = pd.read_csv(\"data/raw/raw_data_all.csv\")\n",
    "house_values = data[\"medianHouseValue\"]\n",
    "data = data.drop(\"medianHouseValue\", axis=1)\n",
    "\n",
    "pred_count = 10\n",
    "payload = data.iloc[:pred_count].to_csv(header=False, index=False)\n",
    "p = predictor.predict(payload, initial_args={\"ContentType\": \"text/csv\"})\n",
    "print(p.decode(\"utf-8\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "blue, stop = \"\\033[94m\", \"\\033[0m\"\n",
    "predictions = json.loads(p.decode(\"utf-8\"))[\"predictions\"]\n",
    "for i in range(pred_count):\n",
    "    print(\n",
    "        f\"Predicted: {blue}{predictions[i][0]}{stop} and Actual is: {blue}{house_values.iloc[i]}{stop}\"\n",
    "    )"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Clean-up\n",
    "Delete the resources created for this example to avoid any unintended charges"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "sm_client = boto3.client(\"sagemaker\")\n",
    "\n",
    "for d in sm_client.list_model_packages(ModelPackageGroupName=model_package_group_name)[\n",
    "    \"ModelPackageSummaryList\"\n",
    "]:\n",
    "    print(d[\"ModelPackageArn\"])\n",
    "    sm_client.delete_model_package(ModelPackageName=d[\"ModelPackageArn\"])\n",
    "\n",
    "sm_client.delete_model_package_group(ModelPackageGroupName=model_package_group_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "predictor.delete_endpoint()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "pipeline.delete()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Notebook CI Test Results\n",
    "\n",
    "This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.\n",
    "\n",
    "![This us-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-east-1/ml_ops|sm-pipelines_train_model_registry_deploy.ipynb)\n",
    "\n",
    "![This us-east-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-east-2/ml_ops|sm-pipelines_train_model_registry_deploy.ipynb)\n",
    "\n",
    "![This us-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-west-1/ml_ops|sm-pipelines_train_model_registry_deploy.ipynb)\n",
    "\n",
    "![This ca-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ca-central-1/ml_ops|sm-pipelines_train_model_registry_deploy.ipynb)\n",
    "\n",
    "![This sa-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/sa-east-1/ml_ops|sm-pipelines_train_model_registry_deploy.ipynb)\n",
    "\n",
    "![This eu-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-1/ml_ops|sm-pipelines_train_model_registry_deploy.ipynb)\n",
    "\n",
    "![This eu-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-2/ml_ops|sm-pipelines_train_model_registry_deploy.ipynb)\n",
    "\n",
    "![This eu-west-3 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-3/ml_ops|sm-pipelines_train_model_registry_deploy.ipynb)\n",
    "\n",
    "![This eu-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-central-1/ml_ops|sm-pipelines_train_model_registry_deploy.ipynb)\n",
    "\n",
    "![This eu-north-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-north-1/ml_ops|sm-pipelines_train_model_registry_deploy.ipynb)\n",
    "\n",
    "![This ap-southeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-southeast-1/ml_ops|sm-pipelines_train_model_registry_deploy.ipynb)\n",
    "\n",
    "![This ap-southeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-southeast-2/ml_ops|sm-pipelines_train_model_registry_deploy.ipynb)\n",
    "\n",
    "![This ap-northeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-northeast-1/ml_ops|sm-pipelines_train_model_registry_deploy.ipynb)\n",
    "\n",
    "![This ap-northeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-northeast-2/ml_ops|sm-pipelines_train_model_registry_deploy.ipynb)\n",
    "\n",
    "![This ap-south-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-south-1/ml_ops|sm-pipelines_train_model_registry_deploy.ipynb)\n"
   ]
  }
 ],
 "metadata": {
  "instance_type": "ml.t3.medium",
  "kernelspec": {
   "display_name": "Python 3 (Data Science 3.0)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/sagemaker-data-science-310-v1"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}