 archived/2_object_detection_train_eval.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Build an Object Detection Model in TensorFlow: Model Training and Evaluation\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook. \n",
    "\n",
    "![This us-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-west-2/introduction_to_applying_machine_learning|object_detection_with_tensorflow_and_tfrecords|2_object_detection_train_eval.ipynb)\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Background\n",
    "\n",
    "This notebook is one of a sequence of notebooks that show you how to use various SageMaker functionalities to build, train, and test the object detection model, including data pre-processing steps like ingestion, cleaning and processing, training, and test the model. There are two parts of the demo: \n",
    "\n",
    "1. Overview and Data Preparation.- you will preprocess the data, then create a json file from the cleaned data. By the end of part 1, you will have a complete data set that contains all features used on Object selection to be ingested by a data loader in *[TensorFlow](https://github.com/tensorflow/tensorflow)* using 'TFRecords'.\n",
    "1. Data loader creation and Model Training (current notebook).- you will use the data set built from part 1 to create a data loader for TensorFlow using *[Keras CV](https://github.com/keras-team/keras-cv)*, train the model and then test the model predictability with the test data. \n",
    "\n",
    "\n",
    "## Content\n",
    "* [Overview](#Overview)\n",
    "* [Using TensorFlow Data loaders with 'TFRecords'](#Using-TensorFlow-Data-loaders-with-'TFRecords')\n",
    "* [Model Selection](#Model-Selection)\n",
    "* [Training Object Detection Models with SageMaker and TensorFlow](#Training-Object-Detection-Models-with-SageMaker-and-TensorFlow)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview\n",
    "\n",
    "### What is Object Detection, and why is it important?\n",
    "\n",
    "Object detection refers to detecting instances of objects from certain classes in images or videos. It allows for multiple objects to be detected and localized in an image. Object detection is commonly used in applications such as self-driving cars, face detection, video surveillance, etc.  \n",
    "\n",
    "### Use Cases for Object Detection\n",
    "\n",
    "Some common use cases of object detection include:\n",
    "\n",
    "- Self driving cars - detect pedestrians, cars, traffic signs, etc.\n",
    "- Face detection - detect faces in images and videos for applications like security and tagging people in images.\n",
    "- Video surveillance - detect suspicious activities or objects.\n",
    "- Medical imaging - detect anomalies, tumors, etc. in medical scans.\n",
    "- Retail - detect objects on shelves for inventory management.\n",
    "\n",
    "### Define the Machine Learning Problem  \n",
    "\n",
    "Object detection can be formulated as a supervised machine learning problem:\n",
    "\n",
    "- Given a set of labelled images containing objects from certain classes, train a model to detect the presence and location of those objects in new images.\n",
    "\n",
    "- The model needs to identify the class of objects present and draw bounding boxes around them indicating their locations.\n",
    "\n",
    "### Data Requirements\n",
    "\n",
    "- Large dataset of images with object annotation - Object locations are annotated using bounding boxes around them.\n",
    "\n",
    "- Variety of images - Objects captured under different conditions of illumination, scales, occlusion, viewpoints etc. \n",
    "\n",
    "### Challenges\n",
    "\n",
    "- Data annotation - Time consuming and expensive process.\n",
    "\n",
    "- Class imbalance - Models tend to perform better for classes with more examples.\n",
    "\n",
    "- Viewpoint variation - Objects look different from different angles and viewpoints. \n",
    "\n",
    "- Background clutter - Objects may blend with their surroundings.\n",
    "\n",
    "- Small objects - Harder to detect smaller objects.\n",
    "\n",
    "- Occlusion - Objects hidden behind other things are tougher to detect."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Tensorflow for CPU\n",
    "#!pip install keras-cv tensorflow --upgrade\n",
    "!pip install --upgrade pip --quiet\n",
    "# Tensorflow for GPU\n",
    "#!pip install keras-cv tensorflow[and-cuda] --upgrade --quiet\n",
    "#!pip install keras-cv --upgrade --quiet\n",
    "# Tensorflow without GPU\n",
    "!pip uninstall tensorflow\n",
    "!pip install keras-cv tensorflow~=2.15.0 --ignore-installed --upgrade --quiet\n",
    "!pip install sagemaker botocore boto3 awscli --upgrade --quiet\n",
    "!pip install tensorrt --quiet\n",
    "#!pip install Pillow==9.5.0\n",
    "#!pip install pickleshare --quiet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Parameters \n",
    "The following lists configurable parameters that are used throughout the whole notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import sagemaker\n",
    "import json\n",
    "import pandas as pd\n",
    "import glob\n",
    "import boto3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Create a SageMaker session\n",
    "sagemaker_session = sagemaker.Session()\n",
    "# Get the default S3 bucket associated with your SageMaker session\n",
    "bucket = sagemaker_session.default_bucket()  # replace with your own bucket name if you have one\n",
    "# Create an S3 resource client\n",
    "s3 = boto3.resource(\"s3\")\n",
    "# Get the AWS region name\n",
    "region = boto3.Session().region_name\n",
    "# Get the execution role for SageMaker\n",
    "role = sagemaker.get_execution_role()\n",
    "# Create a SageMaker client\n",
    "smclient = boto3.Session().client(\"sagemaker\")\n",
    "# Set a prefix for your S3\n",
    "prefix = \"object-detection-tensorflow\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using TensorFlow Data loaders with 'TFRecords'\n",
    "\n",
    "### Loading data from S3\n",
    "\n",
    "As a first step, we need to load the 'TFRecord' files that were generated during the preprocessing job and saved to S3. We can use the TensorFlow IO functions to stream data directly from S3 without needing to download the files locally.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "The code starts by loading variables from a previous notebook using the '%store -r magic command':"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# read variables from previous notebook\n",
    "# The output URI of the previous data processing job. This will be used to load the preprocessed data.\n",
    "processing_job_output_uri = f\"s3://{bucket}/{prefix}/data/processing/output\"\n",
    "# This is a mapping between the class labels and their numerical representations, created during data preprocessing.\n",
    "%store -r class_mapping\n",
    "# The size of the training dataset, usually determined during data splitting.\n",
    "%store -r training_dataset_size\n",
    "# The size of the validation dataset, usually determined during data splitting.\n",
    "%store -r validation_dataset_size"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It generates a class_mapping dictionary that maps class IDs to their corresponding class names. This mapping will be used later for visualization purposes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Define a list of class names\n",
    "class_ids = [\n",
    "    \"aeroplane\",\n",
    "    \"bicycle\",\n",
    "    \"bird\",\n",
    "    \"boat\",\n",
    "    \"bottle\",\n",
    "    \"bus\",\n",
    "    \"car\",\n",
    "    \"cat\",\n",
    "    \"chair\",\n",
    "    \"cow\",\n",
    "    \"diningtable\",\n",
    "    \"dog\",\n",
    "    \"horse\",\n",
    "    \"motorbike\",\n",
    "    \"person\",\n",
    "    \"pottedplant\",\n",
    "    \"sheep\",\n",
    "    \"sofa\",\n",
    "    \"train\",\n",
    "    \"tvmonitor\",\n",
    "]\n",
    "# Create a dictionary that maps class indices (keys) to their corresponding class names (values)\n",
    "class_mapping = dict(zip(range(len(class_ids)), class_ids))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This code cell creates a SageMaker session and gets the default S3 bucket associated with the session. The S3 client is also instantiated to interact with the S3 API. The cell defines the S3 prefixes where the preprocessed training and validation data are stored in 'TFRecord' format. Finally, it lists the objects (files) in the S3 bucket under the specified prefixes for training and validation data, using the list_objects_v2 method of the S3 client. This step is necessary to retrieve the paths of the 'TFRecord' files, which will be used for loading the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Import necessary libraries\n",
    "import boto3\n",
    "import sagemaker\n",
    "\n",
    "# import tensorflow_io\n",
    "import tensorflow as tf\n",
    "\n",
    "import tensorflow as tf\n",
    "\n",
    "gpus = tf.config.experimental.list_physical_devices(\"GPU\")\n",
    "for gpu in gpus:\n",
    "    tf.config.experimental.set_memory_growth(gpu, True)\n",
    "\n",
    "sagemaker_session = (\n",
    "    sagemaker.Session()\n",
    ")  # Create SageMaker session to interact with SageMaker resources\n",
    "bucket = sagemaker_session.default_bucket()  # Get the default S3 bucket for this SageMaker session\n",
    "s3_client = boto3.client(\"s3\")  # Create an S3 client to interact with S3 buckets and objects\n",
    "\n",
    "# S3 prefix where preprocessed data is stored for training/validation\n",
    "prefix_train = \"object-detection-tensorflow/data/processing/output/tfrecords/train/\"  # S3 prefix for training data\n",
    "prefix_val = \"object-detection-tensorflow/data/processing/output/tfrecords/val/\"  # S3 prefix for validation data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The provided code retrieves the data and prepares the datasets for training and validation. It accomplishes this by generating lists of local file paths for the 'TFRecord' files and subsequently loading those files into TensorFlow datasets. These prepared datasets can then be utilized for training or evaluating the object detection model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import glob\n",
    "\n",
    "# search all files inside a specific folder\n",
    "# *.* means file name with any extension\n",
    "# Create an empty list to store location of training files\n",
    "filenames_train = []\n",
    "# Create an empty list to store location of training files\n",
    "filenames_val = []\n",
    "\n",
    "# Define the directory paths for train and validation data\n",
    "dir_path_train = \"./data/tfrecords/train/*.*\"\n",
    "dir_path_val = \"./data/tfrecords/val/*.*\"\n",
    "dir_path = \"./data/tfrecords/\"\n",
    "\n",
    "# Download the processing job output from S3 to the local directory\n",
    "sagemaker.s3.S3Downloader.download(processing_job_output_uri, dir_path)\n",
    "\n",
    "# Iterate through all files in the train directory and append their paths to the filenames_train list\n",
    "for file in glob.glob(dir_path_train, recursive=True):\n",
    "    filenames_train.append(file)\n",
    "\n",
    "# Iterate through all files in the validation directory and append their paths to the filenames_val list\n",
    "for file in glob.glob(dir_path_val, recursive=True):\n",
    "    filenames_val.append(file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Load the training/validation TFRecords dataset from the files stored in the filenames list\n",
    "dataset = tf.data.TFRecordDataset(filenames_train, num_parallel_reads=tf.data.experimental.AUTOTUNE)\n",
    "val_dataset = tf.data.TFRecordDataset(\n",
    "    filenames_val, num_parallel_reads=tf.data.experimental.AUTOTUNE\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using data loaders and parsing 'TFRecords'\n",
    "Next we can create a data loader to parse the 'TFRecord' examples and create batches of images and labels for visualization."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This cell sets up the necessary functions and imports for working with 'TFRecord' data in the context of object detection. The 'parse_tfrecord_fn' function defines how to parse the 'TFRecord' file format, which stores the image data and bounding box annotations. It reads the features from the 'TFRecord' file, decodes the image data, and creates a dictionary with the image and bounding box information. The prepare_sample function is a helper function that formats the parsed data into the expected format for the object detection model. The 'plot_boxes_tfrecords' function is a utility for visualizing the bounding boxes on the images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "from PIL import Image, ImageDraw, ImageFont\n",
    "\n",
    "\n",
    "# Function to plot bounding boxes on an image\n",
    "def plot_boxes_tfrecords(features):\n",
    "    # Open the image\n",
    "    data = features[\"images\"].numpy().astype(np.uint8)\n",
    "    image = Image.fromarray(data, \"RGB\")\n",
    "\n",
    "    # Create a drawing object\n",
    "    draw = ImageDraw.Draw(image)\n",
    "\n",
    "    # Iterate over the bounding boxes in the features dictionary\n",
    "    for index in range(len(features[\"bounding_boxes\"][\"boxes\"])):\n",
    "        # print(index)\n",
    "        # Extract the coordinates of the current bounding box\n",
    "        box = features[\"bounding_boxes\"][\"boxes\"][index]\n",
    "        x1, y1, x2, y2 = box\n",
    "        category = class_mapping[int(features[\"bounding_boxes\"][\"classes\"][index])]\n",
    "        draw.rectangle([(x1, y1), (x2, y2)], outline=\"red\", width=2)\n",
    "        # Draw the label\n",
    "        label_width, label_height = draw.textsize(category)\n",
    "        label_x = x1\n",
    "        label_y = y1 - label_height - 5  # Adjust the offset as needed\n",
    "\n",
    "        # Draw the label\n",
    "        draw.text((label_x, label_y), category, fill=\"red\")\n",
    "\n",
    "    # Display the image\n",
    "    image.show()\n",
    "\n",
    "\n",
    "# Parses a TFRecord example and returns a dictionary.\n",
    "def parse_tfrecord_fn(example):\n",
    "    # Define the feature description for parsing the TFRecord example\n",
    "    feature_description = {\n",
    "        \"height\": tf.io.FixedLenFeature((), tf.int64),\n",
    "        \"width\": tf.io.FixedLenFeature((), tf.int64),\n",
    "        \"filename\": tf.io.FixedLenFeature((), tf.string),\n",
    "        \"image\": tf.io.FixedLenFeature((), tf.string),\n",
    "        \"object/bbox/xmin\": tf.io.VarLenFeature(tf.float32),\n",
    "        \"object/bbox/xmax\": tf.io.VarLenFeature(tf.float32),\n",
    "        \"object/bbox/ymin\": tf.io.VarLenFeature(tf.float32),\n",
    "        \"object/bbox/ymax\": tf.io.VarLenFeature(tf.float32),\n",
    "        \"object/text\": tf.io.VarLenFeature(tf.string),\n",
    "        \"object/label\": tf.io.VarLenFeature(tf.int64),\n",
    "    }\n",
    "    # Parse the example using the feature description\n",
    "    example = tf.io.parse_single_example(example, feature_description)\n",
    "    # Decode the JPEG image data and convert it to float32\n",
    "    example[\"image\"] = tf.cast(tf.io.decode_jpeg(example[\"image\"], channels=3), tf.float32)\n",
    "    # Convert the filename to a string\n",
    "    example[\"filename\"] = tf.cast(example[\"filename\"], tf.string)\n",
    "    # Convert the sparse tensors to dense tensors\n",
    "    example[\"object/bbox/xmin\"] = tf.sparse.to_dense(example[\"object/bbox/xmin\"])\n",
    "    example[\"object/bbox/xmax\"] = tf.sparse.to_dense(example[\"object/bbox/xmax\"])\n",
    "    example[\"object/bbox/ymin\"] = tf.sparse.to_dense(example[\"object/bbox/ymin\"])\n",
    "    example[\"object/bbox/ymax\"] = tf.sparse.to_dense(example[\"object/bbox/ymax\"])\n",
    "    example[\"object/text\"] = tf.sparse.to_dense(example[\"object/text\"])\n",
    "    example[\"object/label\"] = tf.sparse.to_dense(example[\"object/label\"])\n",
    "    # Combine the bounding box coordinates into a single tensor\n",
    "    example[\"object/bbox\"] = tf.stack(\n",
    "        [\n",
    "            example[\"object/bbox/xmin\"],\n",
    "            example[\"object/bbox/ymin\"],\n",
    "            example[\"object/bbox/xmax\"],\n",
    "            example[\"object/bbox/ymax\"],\n",
    "        ],\n",
    "        axis=1,\n",
    "    )\n",
    "    return example\n",
    "\n",
    "\n",
    "# Prepares a sample dictionary for input to the TensorFlow model\n",
    "def prepare_sample(inputs):\n",
    "    image = inputs[\"image\"]\n",
    "    boxes = inputs[\"object/bbox\"]\n",
    "    labels = inputs[\"object/label\"]\n",
    "    bounding_boxes = {\n",
    "        \"classes\": inputs[\"object/label\"],\n",
    "        \"boxes\": boxes,\n",
    "    }\n",
    "    return {\"images\": image, \"bounding_boxes\": bounding_boxes}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This code prepares the 'TFRecord' dataset to test an object detection model. It loads the dataset, applies necessary preprocessing, shuffles the data, and retrieves a sample for visualization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Set the batch size for data loader\n",
    "BATCH_SIZE = 4\n",
    "\n",
    "# Load the TFRecord dataset and parse it using the parse_tfrecord_fn function\n",
    "raw_dataset_sample = dataset.map(parse_tfrecord_fn)\n",
    "\n",
    "# Apply data preparation and preprocessing steps to the parsed dataset\n",
    "parsed_dataset_sample = raw_dataset_sample.map(lambda x: prepare_sample(x))\n",
    "\n",
    "# Shuffle the prepared dataset\n",
    "sample_ds = parsed_dataset_sample.shuffle(BATCH_SIZE * 4)\n",
    "\n",
    "# Repeat the validation dataset indefinitely\n",
    "sample_ds = sample_ds.repeat()\n",
    "\n",
    "# Get a sample from the shuffled dataset\n",
    "data_not_augmented = next(iter(sample_ds.take(1)))\n",
    "\n",
    "# Visualize the sample batch with bounding boxes\n",
    "plot_boxes_tfrecords(data_not_augmented)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Augmentation\n",
    "One major benefit of using TensorFlow data loaders is that we can easily apply data augmentation. This helps prevent overfitting and improves the robustness of the model.\n",
    "\n",
    "Some common augmentation techniques for object detection include:\n",
    "\n",
    "* Random horizontal/vertical flipping\n",
    "* Random cropping\n",
    "* Color jittering\n",
    "* Adding noise\n",
    "\n",
    "These can be implemented using the Keras CV layers for computer vision use cases:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import keras\n",
    "import keras_cv\n",
    "from tensorflow import data as tf_data\n",
    "from keras_cv import visualization\n",
    "\n",
    "# Using sequential layers to add augmentation to all samples\n",
    "augmenter = keras.Sequential(\n",
    "    layers=[\n",
    "        # This layer randomly flips the input images horizontally. The bounding_box_format parameter\n",
    "        # specifies the format of the bounding box coordinates, which is \"xyxy\" in this case.\n",
    "        keras_cv.layers.RandomFlip(mode=\"horizontal\", bounding_box_format=\"xyxy\"),\n",
    "        # This layer randomly resizes the input images while maintaining the aspect ratio.\n",
    "        keras_cv.layers.JitteredResize(\n",
    "            target_size=(640, 640), scale_factor=(0.75, 1.3), bounding_box_format=\"xyxy\"\n",
    "        ),\n",
    "    ]\n",
    ")\n",
    "\n",
    "# using an augmentation pipeline to add augmentation randomly to some samples\n",
    "pipeline = keras_cv.layers.RandomAugmentationPipeline(\n",
    "    layers=[\n",
    "        # This layer applies a grid-like mask to the input images, which can help the model generalize better.\n",
    "        keras_cv.layers.GridMask(\n",
    "            ratio_factor=(0, 0.3),\n",
    "        ),\n",
    "        # This layer randomly applies color degeneration to the input images\n",
    "        keras_cv.layers.RandomColorDegeneration(0.5),\n",
    "        # This layer randomly adjusts the saturation of the input images\n",
    "        keras_cv.layers.RandomSaturation(0.8),\n",
    "    ],\n",
    "    # This parameter specifies the number of augmentations to apply to each input image.\n",
    "    augmentations_per_image=3,\n",
    ")\n",
    "\n",
    "\n",
    "# This parameter specifies the number of augmentations to apply to each input image.\n",
    "def apply_pipeline(inputs):\n",
    "    inputs[\"images\"] = pipeline(inputs[\"images\"])\n",
    "    return inputs\n",
    "\n",
    "\n",
    "# This line applies the random augmentation pipeline to the sample dataset.\n",
    "sample_ds_augmented = sample_ds.map(apply_pipeline, num_parallel_calls=tf.data.AUTOTUNE)\n",
    "\n",
    "# This line applies the sequential augmentation layers to the augmented sample dataset.\n",
    "sample_ds_augmented = sample_ds_augmented.map(augmenter, num_parallel_calls=tf.data.AUTOTUNE)\n",
    "\n",
    "# This line gets the first batch of augmented data from the sample dataset.\n",
    "data_augmented = next(iter(sample_ds_augmented.take(1)))\n",
    "\n",
    "# This line plots the bounding boxes on the augmented images.\n",
    "plot_boxes_tfrecords(data_augmented)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Adding augmentation during training helps prevent overfitting and makes the model more robust to variations in input images."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model Selection\n",
    "\n",
    "You will need to select an appropriate model architecture for your object detection task. When choosing a model, there are several factors to consider:\n",
    "* The type of objects you want to detect - Are they general everyday objects, or more specialized categories like faces or text? Simpler architectures like SSD and YOLO work well for detecting common objects, while more complex models like Mask R-CNN may be better for niche categories.\n",
    "\n",
    "* Model size and speed - Larger models like 'RetinaNet' will be more accurate but slower, while smaller models like MobileNet will be faster but less accurate. Choose a model size that fits your speed and accuracy needs.\n",
    "\n",
    "* Amount of training data - If you have a large dataset, you can train bigger models with more parameters. With fewer data, stick to smaller models to avoid overfitting.\n",
    "\n",
    "* Inference speed - Some models like 'MobileNet' are optimized specifically for fast inference after training. Prioritize this if you need to run detection very quickly.\n",
    "\n",
    "* Built-in vs custom models - Many pre-made model architectures like 'Faster R-CNN' are available. But you can also build custom models better tailored to your specific objects.\n",
    "\n",
    "A good starting point is to evaluate pre-trained models like 'Faster R-CNN' and 'SSD' (Single Shot Detector) that are available in model zoos. 'Faster R-CNN' with a 'ResNet-50' backbone offers a good balance of accuracy and speed for this dataset. 'SSD' is faster but slightly less accurate, so you may want to try different backbone architectures like 'ResNet', 'MobileNet' and 'EfficientNet' to find the right tradeoff. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To simplify model development, we will leverage the pre-trained object detection models available in Keras CV. Keras CV provides reference implementations and pre-trained weights for state-of-the-art computer vision models. We can quickly test training and inference for object detection by using a model like 'RetinaNet' or 'EfficientNet', initialized with weights pre-trained on COCO or other datasets. By taking advantage of these pre-trained models in Keras CV, we can prototype and experiment with minimal code and set up time. This allows us to focus on customizing and optimizing the model for our specific use case."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How to Load Pretrained models    \n",
    "\n",
    "#### Loading Pretrained models\n",
    "\n",
    "This cell loads a pre-trained RetinaNet object detection model based on the ResNet50 architecture and the Pascal VOC dataset. The from_preset function is used to load the pre-trained weights and architecture. The bounding_box_format parameter specifies the format of the bounding box coordinates, which is 'xyxy' in this case. The prediction_decoder parameter is set to the 'NonMaxSuppression' layer initialized in the previous cell, which will be used to filter out overlapping bounding boxes during inference. The load_weights parameter is set to True to load the pre-trained weights along with the model architecture."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Filter out overlapping bounding boxes based on intersection over union (IoU) thresholds.\n",
    "prediction_decoder = keras_cv.layers.NonMaxSuppression(\n",
    "    bounding_box_format=\"xyxy\",  # The format of the bounding boxes (x, y, x, y)\n",
    "    from_logits=True,  # Indicates that the input is logits (raw output from the model)\n",
    "    iou_threshold=0.7,  # IoU threshold for filtering overlapping bounding boxes\n",
    "    confidence_threshold=0.3,  # Confidence threshold for filtering low-confidence predictions\n",
    ")\n",
    "\n",
    "# Load Resnet architecture and weights from pre-trained model\n",
    "model1 = keras_cv.models.RetinaNet.from_preset(\n",
    "    \"retinanet_resnet50_pascalvoc\",  # Load pre-trained RetinaNet model with ResNet50 backbone\n",
    "    bounding_box_format=\"xyxy\",  # The format of the bounding boxes (x, y, x, y)\n",
    "    prediction_decoder=prediction_decoder,  # Use the custom NMS layer for filtering predictions\n",
    "    load_weights=True,  # Load pre-trained weights for the model\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This code creates a resizing layer for inference. The 'keras_cv.layers.Resizing' layer is used to resize the input images to a fixed size of 640x640 pixels. The bounding_box_format parameter specifies the format of the bounding box coordinates, where 'xyxy' means that the coordinates are in the format of [x_min, y_min, x_max, y_max]. The pad_to_aspect_ratio parameter ensures that the aspect ratio of the images is preserved during resizing by adding padding if necessary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Get the image data from the 'data_not_augmented' dictionary\n",
    "image_na = data_not_augmented[\"images\"]\n",
    "\n",
    "# Resizing the images for inference\n",
    "inference_resizing = keras_cv.layers.Resizing(\n",
    "    640, 640, bounding_box_format=\"xyxy\", pad_to_aspect_ratio=True\n",
    ")\n",
    "\n",
    "# Apply the resizing layer to the image data\n",
    "image_batch_na = inference_resizing([image_na])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Prediction test\n",
    "To validate the performance of a pretrained model, we can run a prediction test."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Prediction of not augmented image\n",
    "y_pred1 = model1.predict(image_batch_na)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Plot the bounding box gallery for the given image batch and predictions\n",
    "visualization.plot_bounding_box_gallery(\n",
    "    image_batch_na,\n",
    "    value_range=(0, 255),\n",
    "    rows=1,\n",
    "    cols=1,\n",
    "    y_pred=y_pred1,\n",
    "    scale=5,\n",
    "    font_scale=0.7,\n",
    "    bounding_box_format=\"xyxy\",\n",
    "    class_mapping=class_mapping,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "del model1\n",
    "import gc\n",
    "\n",
    "gc.collect()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "tf.keras.backend.clear_session()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training Object Detection Models with SageMaker and TensorFlow\n",
    "\n",
    "To train object detection models on SageMaker, we first need to configure a SageMaker training job. The key components are:\n",
    "* Choosing an estimator\n",
    "  * We can use the TensorFlow estimator to leverage the Keras API and pretrained models like RetinaNet.\n",
    "* Selecting an instance type\n",
    "    * GPU instances like ml.p3.2xlarge are best suited for training convolutional neural networks.\n",
    "* Configuring the training script\n",
    "    * This sets up the model architecture, loads pretrained weights, and defines the training loop.\n",
    "* Specifying the training image\n",
    "    * We can use a TensorFlow image from the SageMaker registry.\n",
    "* Setting hyperparameters\n",
    "    * Learning rate, batch size, and epochs are key hyperparameters to tune.\n",
    "    \n",
    "For model evaluation, we need to choose appropriate metrics like precision, recall, and 'mAP'. Since object detection involves classifying many bounding boxes, metrics that account for class imbalance like F1 score are also useful.\n",
    "The pretrained models in Keras CV combined with SageMaker's managed training provide an optimized environment for iterating on object detection models. We can efficiently improve accuracy by tuning hyperparameters and leverage SageMaker infrastructure for scalable distributed training.\n",
    "   \n",
    "### How to create a training job in Sagemaker   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import sagemaker\n",
    "import json\n",
    "import pandas as pd\n",
    "import glob\n",
    "import s3fs\n",
    "import boto3\n",
    "from datetime import datetime\n",
    "import os"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Loading Preprocessed Data from S3\n",
    "\n",
    "A key advantage of using SageMaker for model training is it can directly access data stored in S3 buckets. After preprocessing our dataset in a previous step, we staged the output in an S3 location. The SageMaker TensorFlow estimator handles loading this data from S3 into our training script. We simply specify the S3 path when creating the TensorFlow estimator.\n",
    "\n",
    "If you want to see how the train and validation 'TFRecords' datasets are created in detail, look at [Build an Object Detection Model on Tensorflow and SageMaker: Overview and Data Preparation](1_object_detection_preprocessing.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Set paths to training and validation data in S3\n",
    "s3_train_data = f\"{processing_job_output_uri}/tfrecords/train\"\n",
    "s3_validation_data = f\"{processing_job_output_uri}/tfrecords/val\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Initialize Model Hyperparameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Define hyperparameters\n",
    "hyperparameters = {\n",
    "    \"batch_size\": 8,  # Number of samples to include in each batch during training\n",
    "    \"learning_rate\": 0.001,  # The learning rate for the optimizer during training\n",
    "    \"epochs\": 2,  # Number of epochs (complete passes through the training data) for training\n",
    "    \"global_clipnorm\": 0.3,  # Maximum norm of the gradients for clipping to prevent exploding gradients\n",
    "    \"train_samples\": training_dataset_size,  # Number of samples in the training dataset\n",
    "    \"eval_samples\": validation_dataset_size,  # Number of samples in the validation dataset\n",
    "    \"model_dir\": \"/opt/ml/model\",  # Directory where the trained model will be saved\n",
    "    \"checkpoint_local_path\": \"/opt/ml/checkpoints\",  # Directory where model checkpoints will be saved during training\n",
    "    \"finetune\": True,  # Set to True if you want to fine-tune a pre-trained model, False if training from scratch\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Define SageMaker estimator"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This code sets up a TensorFlow estimator for training an object detection model using SageMaker."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Specify instance type\n",
    "train_instance_type = \"ml.p3.2xlarge\"  # this is the recommended instance for testing purposes\n",
    "# train_instance_type=\"ml.p3.8xlarge\"\n",
    "# train_instance_type=\"ml.g5.48xlarge\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "tf_model_image_uri = sagemaker.image_uris.retrieve(\n",
    "    framework=\"tensorflow\",  # Specify the framework as TensorFlow\n",
    "    region=region,  # The AWS region where you want to run the training job\n",
    "    version=\"2.14\",  # The version of TensorFlow you want to use (2.13 in this case)\n",
    "    image_scope=\"training\",  # Specify that you want the training container image\n",
    "    py_version=\"py310\",  # Specify the Python version you want to use (3.10 in this case)\n",
    "    instance_type=train_instance_type,  # ParameterString# The type of EC2 instance to use for training (passed as a parameter)\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "from time import gmtime, strftime\n",
    "import os\n",
    "\n",
    "# Import TensorFlow estimator\n",
    "import sagemaker\n",
    "from sagemaker.tensorflow import TensorFlow\n",
    "\n",
    "# Set the arguments for the TensorFlow estimator\n",
    "estimator_args = dict(\n",
    "    source_dir=\"code\",  # The directory containing the training code\n",
    "    entry_point=\"train.py\",  # The Python script to run for training\n",
    "    model_dir=\"/opt/ml/model\",  # The path to save the trained model\n",
    "    instance_type=train_instance_type,  # The type of EC2 instance for training\n",
    "    instance_count=1,  # The number of instances for training\n",
    "    framework_version=\"2.14\",  # The TensorFlow version to use                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        \", # The TensorFlow version to use\n",
    "    # py_version=\"py310\",# Uncomment this line to specify the Python version\n",
    "    image_uri=tf_model_image_uri,  # Image URI for the TensorFlow container\n",
    "    debugger_hook_config=None,  # Disable the TensorFlow debugger hook\n",
    "    disable_profiler=True,  # Disable the profiler for training\n",
    "    # max_run=60 * 20,  # # Uncomment this line to set a maximum runtime for training (20 minutes)\n",
    "    role=role,  # The IAM role for the training job\n",
    "    # keep_alive_period_in_seconds=3600,# Uncomment this line to set the keep-alive period (1 hour)\n",
    "    metric_definitions=[\n",
    "        # Define the metrics to be captured during training, validation, and testing\n",
    "        {\"Name\": \"train:loss\", \"Regex\": \"loss: ([0-9.]*?) \"},\n",
    "        {\"Name\": \"train:box_loss\", \"Regex\": \"box_loss: ([0-9.]*?) \"},\n",
    "        {\"Name\": \"train:classification_loss\", \"Regex\": \"classification_loss: ([0-9.]*?) \"},\n",
    "        {\"Name\": \"train:MaP\", \"Regex\": \"MaP: (.*?) \"},\n",
    "        {\"Name\": \"val:loss\", \"Regex\": \"val_loss: ([0-9.]*?) \"},\n",
    "        {\"Name\": \"val:box_loss\", \"Regex\": \"val_box_loss: ([0-9.]*?) \"},\n",
    "        {\"Name\": \"val:classification_loss\", \"Regex\": \"val_classification_loss: ([0-9.]*?) \"},\n",
    "        {\"Name\": \"val:MaP\", \"Regex\": \"val_MaP: (.*?) \"},\n",
    "        {\"Name\": \"test:loss\", \"Regex\": \"Test loss: ([0-9.]*?)\"},\n",
    "        {\"Name\": \"test:box_loss\", \"Regex\": \"Test box_loss: ([0-9.]*?)\"},\n",
    "        {\"Name\": \"test:Map\", \"Regex\": \"Test MaP: (.*?)\"},\n",
    "    ],\n",
    ")\n",
    "# Configure the TensorFlow estimator with the specified arguments and hyperparameters\n",
    "estimator = TensorFlow(\n",
    "    hyperparameters=hyperparameters,  # Hyperparameters for the model\n",
    "    **estimator_args,  # Pass the estimator arguments as keyword arguments\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Create Training Job"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next cell starts the training job for the object detection model. The fit method of the estimator object is used to initiate the training process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "import boto3\n",
    "from sagemaker.session import Session\n",
    "\n",
    "# Create a SageMaker session\n",
    "sagemaker_session = Session()\n",
    "\n",
    "# Set an experiment name with a timestamp\n",
    "exp_name = \"object-detection-tensorflow-exp-finetuned-{}\".format(\n",
    "    datetime.now().strftime(\"%Y%m%d-%H%M%S\")\n",
    ")\n",
    "\n",
    "# Start training job\n",
    "estimator.fit(\n",
    "    inputs={\"train\": s3_train_data, \"eval\": s3_validation_data}, wait=True, job_name=exp_name\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "### How to load models from training jobs to use them locally"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Load last training job metadata\n",
    "This cell imports the necessary AWS SDK for Python (Boto3) and retrieves the name of the last training job from the SageMaker service. It uses the list_training_jobs API call to get a list of training jobs sorted by creation time in descending order, and takes the first result (the most recent training job that was completed)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import boto3\n",
    "\n",
    "# Create a SageMaker client object\n",
    "client = boto3.client(\"sagemaker\")\n",
    "\n",
    "# Retrieve the list of training jobs sorted by creation time in descending order and limit the result to the last 1 job\n",
    "last_training_job = client.list_training_jobs(\n",
    "    SortOrder=\"Descending\", SortBy=\"CreationTime\", StatusEquals=\"Completed\", MaxResults=10\n",
    ")\n",
    "# Extract the name of the last training job from the list\n",
    "last_training_job_name = last_training_job[\"TrainingJobSummaries\"][0][\"TrainingJobName\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# prints last completed training job name\n",
    "print(last_training_job_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This cell calls the describe_training_job API to retrieve the details of the training job with the given name. Finally, it prints the S3 URI of the model artifacts generated by the training job."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Retrieve the details of the last training job using the describe_training_job method\n",
    "# from the SageMaker client\n",
    "last_training_job_data = client.describe_training_job(TrainingJobName=last_training_job_name)\n",
    "# last_training_job_name=\"job_name\"#uncomment if you know the job name\n",
    "# last_training_job_name = \"object-detection-tensorflow-exp-finetuned-20240402-195644\"\n",
    "model_s3_uri = last_training_job_data[\"ModelArtifacts\"][\"S3ModelArtifacts\"]\n",
    "# Print the S3 location where the model artifacts (trained model) are stored\n",
    "print(last_training_job_data[\"ModelArtifacts\"][\"S3ModelArtifacts\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# store S3 uri with model artifact\n",
    "%store model_s3_uri"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# read last training job model s3 uri\n",
    "%store -r model_s3_uri"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Download model artifacts from S3 bucket for local testing\n",
    "The code of the next cells downloads a pre-trained model from an Amazon S3 bucket and extracts it locally."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import sagemaker\n",
    "\n",
    "# Create an instance of the S3Downloader class\n",
    "s3_downloader = sagemaker.s3.S3Downloader()\n",
    "\n",
    "# Download the model from the specified S3 URI to the local 'model' directory\n",
    "s3_downloader.download(model_s3_uri, \"model\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# extract the pre-trained model from the tar.gz file\n",
    "!tar -xzvf model/model.tar.gz -C model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Load model locally\n",
    "This code loads the pre-trained model locally."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import keras\n",
    "\n",
    "# Load a pre-trained object detection model from the specified path\n",
    "model = keras.saving.load_model(\"model/1/model.keras\")\n",
    "model.compile()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Load validation dataset\n",
    "This code is preparing a validation dataset for a machine learning model that performs object detection. It loads and preprocesses the validation data from 'TFRecord' files, shuffles and batches the data, resizes the input images to a fixed size, and converts the input data to a format suitable for the model. The preprocessed validation dataset is then ready for evaluating the model's performance on unseen data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Import the bounding_box module from keras_cv\n",
    "from keras_cv import bounding_box\n",
    "\n",
    "# Set the batch size for training\n",
    "BATCH_SIZE = 4\n",
    "\n",
    "# Create a Resizing layer from keras_cv.layers\n",
    "inference_resizing = keras_cv.layers.Resizing(\n",
    "    640, 640, bounding_box_format=\"xyxy\", pad_to_aspect_ratio=True\n",
    ")\n",
    "\n",
    "\n",
    "# Define a function to convert dictionary inputs to tuples\n",
    "# This function is used for mapping the input data to a format suitable for the model\n",
    "def dict_to_tuple(inputs):\n",
    "    # Extract the images from the input dictionary\n",
    "    images = inputs[\"images\"]\n",
    "    # Convert the bounding box coordinates to a dense tensor format\n",
    "    bounding_boxes = bounding_box.to_dense(inputs[\"bounding_boxes\"], max_boxes=32)\n",
    "    # Return the images and bounding boxes as a tuple\n",
    "    return images, bounding_boxes\n",
    "\n",
    "\n",
    "# Create a TensorFlow dataset from the validation dataset\n",
    "val_ds = val_dataset.map(parse_tfrecord_fn)\n",
    "# Preprocess the validation dataset\n",
    "val_ds = val_ds.map(lambda x: prepare_sample(x), num_parallel_calls=tf_data.AUTOTUNE)\n",
    "# Shuffle the validation dataset\n",
    "val_ds = val_ds.shuffle(BATCH_SIZE * 4)\n",
    "\n",
    "# Batch the validation dataset using ragged batching\n",
    "val_ds = val_ds.ragged_batch(BATCH_SIZE)\n",
    "\n",
    "# Resize the images in the validation dataset using the inference_resizing layer\n",
    "val_ds = val_ds.map(inference_resizing, num_parallel_calls=tf_data.AUTOTUNE)\n",
    "\n",
    "# Convert the input data to the required format (tuples of images and bounding boxes)\n",
    "val_ds = val_ds.map(dict_to_tuple, num_parallel_calls=tf_data.AUTOTUNE)\n",
    "\n",
    "# Repeat the validation dataset indefinitely\n",
    "val_ds = val_ds.repeat()\n",
    "\n",
    "# Prefetch data to improve performance\n",
    "val_ds = val_ds.prefetch(tf_data.AUTOTUNE)\n",
    "\n",
    "# Get a sample batch from the validation dataset\n",
    "sample_val_ds = next(iter(val_ds.take(1)))\n",
    "image, _ = sample_val_ds"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Local prediction testing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The 'keras_cv.layers.MultiClassNonMaxSuppression' layer is used to create a prediction decoder for the object detection model. This layer performs non-maximum suppression on the raw output of the model, which helps to remove duplicate or overlapping bounding box predictions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Create a prediction decoder\n",
    "prediction_decoder = keras_cv.layers.MultiClassNonMaxSuppression(\n",
    "    bounding_box_format=\"xyxy\",  # Specify the format of the bounding boxes (x, y, x, y)\n",
    "    from_logits=True,  # Indicate that the input is logits (raw output from the model)\n",
    "    iou_threshold=0.75,  # Set the Intersection over Union (IoU) threshold for non-maximum suppression\n",
    "    confidence_threshold=0.5,  # Set the confidence threshold for non-maximum suppression\n",
    ")\n",
    "\n",
    "# Assign the prediction decoder to the model\n",
    "model.prediction_decoder = prediction_decoder"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This function, visualize_detections, is used to visualize the object detection results of a trained model on a sample dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# This function visualizes the object detection results\n",
    "def visualize_detections(model, dataset, bounding_box_format):\n",
    "    # Get the first batch of images and ground truth bounding boxes from the dataset\n",
    "    images, y_true = next(iter(dataset.take(1)))  # takes one batch from dataset\n",
    "    images = images.numpy().astype(np.uint8)\n",
    "\n",
    "    # Make predictions on the batch of images using the trained model\n",
    "    y_pred = model.predict(images)\n",
    "\n",
    "    # Plot the images with ground truth and predicted bounding boxes\n",
    "    visualization.plot_bounding_box_gallery(\n",
    "        images,\n",
    "        value_range=(0, 255),\n",
    "        bounding_box_format=bounding_box_format,\n",
    "        y_true=y_true,\n",
    "        y_pred=y_pred,\n",
    "        scale=4,\n",
    "        rows=2,\n",
    "        cols=2,\n",
    "        show=True,\n",
    "        font_scale=0.7,\n",
    "        class_mapping=class_mapping,\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "visualize_detections(model, bounding_box_format=\"xyxy\", dataset=val_ds)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Create metrics for trained model\n",
    "This code is setting up an instance of the 'BoxCOCOMetrics' class from the keras_cv.metrics module. This class is used to calculate various evaluation metrics for object detection models, specifically when working with the COCO (Common Objects in Context) dataset format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "metrics_val = keras_cv.metrics.BoxCOCOMetrics(\n",
    "    bounding_box_format=\"xyxy\",\n",
    "    evaluate_freq=2,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The validation metrics computed in this code can be used to evaluate the performance of the trained model and potentially fine-tune it if necessary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Loop through the first 10 batches of the validation dataset\n",
    "for batch in val_ds.take(100):\n",
    "    images, y_true = batch  # Unpack the batch into images and ground truth labels\n",
    "    # Make predictions on the images using the trained model\n",
    "    y_pred = model.predict(images, verbose=1, steps=10)\n",
    "    # Update the validation metrics using the ground truth labels and predicted values\n",
    "    metrics_val.update_state(y_true, y_pred)\n",
    "# Compute and return the final validation metrics\n",
    "metrics_val = metrics_val.result(force=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This code is useful for evaluating the performance of an object detection model on a validation dataset, as it calculates various metrics such as precision, recall, and mean average precision ('mAP') based on the true and predicted labels. These metrics can be used to assess the model's accuracy and make necessary adjustments or improvements."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Print the validation metrics\n",
    "for key in metrics_val.keys():\n",
    "    print(f\"{key}:{metrics_val[key]}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: To achieve good Mean Average Precision ('mAP') values, you may need to run the training for more epochs and perform hyperparameter tuning. This code is just an exercise, and further optimization might be required for real-world object detection tasks."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Notebook CI Test Results\n",
    "\n",
    "This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.\n",
    "\n",
    "![This us-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-east-1/introduction_to_applying_machine_learning|object_detection_with_tensorflow_and_tfrecords|2_object_detection_train_eval.ipynb)\n",
    "\n",
    "![This us-east-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-east-2/introduction_to_applying_machine_learning|object_detection_with_tensorflow_and_tfrecords|2_object_detection_train_eval.ipynb)\n",
    "\n",
    "![This us-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-west-1/introduction_to_applying_machine_learning|object_detection_with_tensorflow_and_tfrecords|2_object_detection_train_eval.ipynb)\n",
    "\n",
    "![This ca-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ca-central-1/introduction_to_applying_machine_learning|object_detection_with_tensorflow_and_tfrecords|2_object_detection_train_eval.ipynb)\n",
    "\n",
    "![This sa-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/sa-east-1/introduction_to_applying_machine_learning|object_detection_with_tensorflow_and_tfrecords|2_object_detection_train_eval.ipynb)\n",
    "\n",
    "![This eu-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-1/introduction_to_applying_machine_learning|object_detection_with_tensorflow_and_tfrecords|2_object_detection_train_eval.ipynb)\n",
    "\n",
    "![This eu-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-2/introduction_to_applying_machine_learning|object_detection_with_tensorflow_and_tfrecords|2_object_detection_train_eval.ipynb)\n",
    "\n",
    "![This eu-west-3 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-3/introduction_to_applying_machine_learning|object_detection_with_tensorflow_and_tfrecords|2_object_detection_train_eval.ipynb)\n",
    "\n",
    "![This eu-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-central-1/introduction_to_applying_machine_learning|object_detection_with_tensorflow_and_tfrecords|2_object_detection_train_eval.ipynb)\n",
    "\n",
    "![This eu-north-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-north-1/introduction_to_applying_machine_learning|object_detection_with_tensorflow_and_tfrecords|2_object_detection_train_eval.ipynb)\n",
    "\n",
    "![This ap-southeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-southeast-1/introduction_to_applying_machine_learning|object_detection_with_tensorflow_and_tfrecords|2_object_detection_train_eval.ipynb)\n",
    "\n",
    "![This ap-southeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-southeast-2/introduction_to_applying_machine_learning|object_detection_with_tensorflow_and_tfrecords|2_object_detection_train_eval.ipynb)\n",
    "\n",
    "![This ap-northeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-northeast-1/introduction_to_applying_machine_learning|object_detection_with_tensorflow_and_tfrecords|2_object_detection_train_eval.ipynb)\n",
    "\n",
    "![This ap-northeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-northeast-2/introduction_to_applying_machine_learning|object_detection_with_tensorflow_and_tfrecords|2_object_detection_train_eval.ipynb)\n",
    "\n",
    "![This ap-south-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-south-1/introduction_to_applying_machine_learning|object_detection_with_tensorflow_and_tfrecords|2_object_detection_train_eval.ipynb)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "availableInstances": [
   {
    "_defaultOrder": 0,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.t3.medium",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 1,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.t3.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 2,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.t3.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 3,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.t3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 4,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 5,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 6,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 7,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 8,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 9,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 10,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 11,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 12,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5d.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 13,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5d.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 14,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5d.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 15,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5d.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 16,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5d.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 17,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5d.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 18,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5d.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 19,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 20,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": true,
    "memoryGiB": 0,
    "name": "ml.geospatial.interactive",
    "supportedImageNames": [
     "sagemaker-geospatial-v1-0"
    ],
    "vcpuNum": 0
   },
   {
    "_defaultOrder": 21,
    "_isFastLaunch": true,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.c5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 22,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.c5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 23,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.c5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 24,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.c5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 25,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 72,
    "name": "ml.c5.9xlarge",
    "vcpuNum": 36
   },
   {
    "_defaultOrder": 26,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 96,
    "name": "ml.c5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 27,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 144,
    "name": "ml.c5.18xlarge",
    "vcpuNum": 72
   },
   {
    "_defaultOrder": 28,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.c5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 29,
    "_isFastLaunch": true,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g4dn.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 30,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g4dn.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 31,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g4dn.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 32,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g4dn.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 33,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g4dn.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 34,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g4dn.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 35,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 61,
    "name": "ml.p3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 36,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 244,
    "name": "ml.p3.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 37,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 488,
    "name": "ml.p3.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 38,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.p3dn.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 39,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.r5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 40,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.r5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 41,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.r5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 42,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.r5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 43,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.r5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 44,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.r5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 45,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.r5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 46,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.r5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 47,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 48,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 49,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 50,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 51,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 52,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 53,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.g5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 54,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.g5.48xlarge",
    "vcpuNum": 192
   },
   {
    "_defaultOrder": 55,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 56,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4de.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 57,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.trn1.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 58,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.trn1.32xlarge",
    "vcpuNum": 128
   },
   {
    "_defaultOrder": 59,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.trn1n.32xlarge",
    "vcpuNum": 128
   }
  ],
  "instance_type": "ml.g4dn.xlarge",
  "kernelspec": {
   "display_name": "TensorFlow 2.10.0 Python 3.9 GPU Optimized",
   "language": "python",
   "name": "TensorFlow 2.10.0 Python 3.9 GPU Optimized"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}