 archived/bias_detection_with_predicted_label_and_facet_datasets.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c977716f-ea79-4952-bd29-68601ef8f87d",
   "metadata": {},
   "source": [
    "# Fairness and Explainability with SageMaker Clarify - Bias Detection With Predicted Label and Facet Datasets"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7643462",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook. \n",
    "\n",
    "![This us-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-west-2/sagemaker-clarify|fairness_and_explainability|bias_detection_with_predicted_label_and_facet_datasets.ipynb)\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b1af6ea-8835-4f81-89d8-ae1a3cf0c055",
   "metadata": {},
   "source": [
    "## Runtime\n",
    "\n",
    "This notebook takes approximately 15 minutes to run.\n",
    "\n",
    "## Contents\n",
    "\n",
    "1. [Overview](#Overview)\n",
    "1. [Prerequisites and Data](#Prerequisites-and-Data)\n",
    "    1. [Initialize SageMaker](#Initialize-SageMaker)\n",
    "    1. [Download data](#Download-Data)\n",
    "    1. [Loading the data: Adult Dataset](#Loading-the-data:-Adult-Dataset) \n",
    "    1. [Data inspection](#Data-inspection) \n",
    "    1. [Data encoding and upload to S3](#Encode-and-Upload-the-Dataset) \n",
    "1. [Train and Deploy XGBoost Model](#Train-and-Deploy-XGBoost-Model)\n",
    "    1. [Train Model](#Train-Model)\n",
    "    1. [Deploy Endpoint](#Deploy-Endpoint)\n",
    "    1. [Predicted Labels](#Predicted-Labels)\n",
    "1. [Amazon SageMaker Clarify](#Amazon-SageMaker-Clarify)\n",
    "    1. [Detecting Bias](#Detecting-Bias-with-Predicted-Labels)\n",
    "        1. [Writing BiasConfig](#Writing-BiasConfig)\n",
    "        1. [Pre-training Bias](#Pre-training-Bias)\n",
    "        1. [Post-training Bias with Predicted Labels](#Post-training-Bias)\n",
    "        1. [Viewing the Bias Report](#Viewing-the-Bias-Report)\n",
    "1. [Clean Up](#Clean-Up)\n",
    "\n",
    "## Overview\n",
    "\n",
    "Amazon SageMaker Clarify provides you the ability to improve your machine learning models by detecting potential biases in your model and data. Specifically, Clarify can help you measure biases that can occur during each stage of the ML lifecycle (data collection, model training and tuning, and monitoring of ML models deployed for inference). \n",
    "\n",
    "In some cases, you may already have model predictions that you would like to use for bias detection instead of having Clarify make model inference calls. Additionally, there are cases where models are trained without sensitive attributes, but you would still like to evaluate the dataset and model with respect to the sensitive attributes to understand and mitigate bias. This sample notebook demonstrates both of these cases and will walk you through:\n",
    "\n",
    "1. Key terms and concepts needed to understand SageMaker Clarify\n",
    "1. Measuring the pre-training bias of a dataset and post-training bias of a model\n",
    "1. Accessing the bias report\n",
    "\n",
    "We will first train a [SageMaker XGBoost](https://docs.aws.amazon.com/sagemaker/latest/dg/xgboost.html) model, use it to get model predictions, then use SageMaker Clarify to analyze the pre- and post-training bias on the dataset and predicted labels dataset and demonstrate how to provide a separate facet dataset that is not used during model training. If you would like to use a deployed model or endpoint to analyze your dataset or explain model predictions, please visit [this notebook](https://github.com/aws/amazon-sagemaker-examples/blob/main/sagemaker-clarify/fairness_and_explainability/fairness_and_explainability.ipynb). You can find the detailed documentation of SageMaker Clarify at [What Is Fairness and Model Explainability for Machine Learning Predictions](https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-fairness-and-explainability.html) and more demo notebooks at [aws-sagemaker-examples GitHub repository](https://github.com/aws/amazon-sagemaker-examples/tree/main/sagemaker-clarify)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4928e0be-dc32-454d-8fd1-7a9aad91030d",
   "metadata": {},
   "source": [
    "## Prerequisites and Data\n",
    "\n",
    "Let's start by installing the required packages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4d8b5da5-1e52-408c-8799-8db12c67137f",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "82b431fd-41bc-45ea-bcbe-7227b9899ab8",
   "metadata": {},
   "outputs": [],
   "source": [
    "! pip install \"sagemaker==2.133.0\" --upgrade --quiet"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2dfad13-2c88-4488-bc56-8d09ea9dee5f",
   "metadata": {},
   "source": [
    "### Initialize SageMaker"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a0b2248d-3c29-4b07-a52c-7b38aba419fd",
   "metadata": {},
   "outputs": [],
   "source": [
    "import sagemaker\n",
    "\n",
    "session = sagemaker.Session()\n",
    "bucket = session.default_bucket()\n",
    "prefix = \"sagemaker/DEMO-sagemaker-clarify-bias-with-predicted-labels\"\n",
    "region = session.boto_region_name\n",
    "# Define IAM role\n",
    "from sagemaker import get_execution_role\n",
    "from sagemaker.serializers import CSVSerializer\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import os\n",
    "import boto3\n",
    "from datetime import datetime\n",
    "\n",
    "role = get_execution_role()\n",
    "s3_client = boto3.client(\"s3\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5a43bc3-b589-4fb6-aef3-a6bc1ebe7c5d",
   "metadata": {},
   "source": [
    "### Download Data\n",
    "\n",
    "Data Source: https://archive.ics.uci.edu/ml/machine-learning-databases/adult/\n",
    "\n",
    "Let's __download__ the data and save it in the local folder with the name adult.data and adult.test from UCI repository$^{[1]}$.\n",
    "\n",
    "$^{[1]}$Dua Dheeru, and Efi Karra Taniskidou. \"UCI Machine Learning Repository\". Irvine, CA: University of California, School of Information and Computer Science (2017)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c445ce06-4d7d-4c6a-9472-0e4e8b486dd7",
   "metadata": {},
   "outputs": [],
   "source": [
    "adult_columns = [\n",
    "    \"Age\",\n",
    "    \"Workclass\",\n",
    "    \"fnlwgt\",\n",
    "    \"Education\",\n",
    "    \"Education-Num\",\n",
    "    \"Marital Status\",\n",
    "    \"Occupation\",\n",
    "    \"Relationship\",\n",
    "    \"Ethnic group\",\n",
    "    \"Sex\",\n",
    "    \"Capital Gain\",\n",
    "    \"Capital Loss\",\n",
    "    \"Hours per week\",\n",
    "    \"Country\",\n",
    "    \"Target\",\n",
    "]\n",
    "if not os.path.isfile(\"adult.data\"):\n",
    "    s3_client.download_file(\n",
    "        f\"sagemaker-example-files-prod-{region}\",\n",
    "        \"datasets/tabular/uci_adult/adult.data\",\n",
    "        \"adult.data\",\n",
    "    )\n",
    "    print(\"adult.data saved!\")\n",
    "else:\n",
    "    print(\"adult.data already on disk.\")\n",
    "\n",
    "if not os.path.isfile(\"adult.test\"):\n",
    "    s3_client.download_file(\n",
    "        f\"sagemaker-example-files-prod-{region}\",\n",
    "        \"datasets/tabular/uci_adult/adult.test\",\n",
    "        \"adult.test\",\n",
    "    )\n",
    "    print(\"adult.test saved!\")\n",
    "else:\n",
    "    print(\"adult.test already on disk.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "81f3727b-5788-4dfe-8177-0158944a2670",
   "metadata": {},
   "source": [
    "### Loading the data: Adult Dataset\n",
    "From the UCI repository of machine learning datasets, this database contains 14 features concerning demographic characteristics of 45,222 rows (32,561 for training and 12,661 for testing). The task is to predict whether a person has a yearly income that is more or less than $50,000.\n",
    "\n",
    "Here are the features and their possible values:\n",
    "\n",
    "1. **Age**: continuous.\n",
    "1. **Workclass**: Private, Self-emp-not-inc, Self-emp-inc, Federal-gov, Local-gov, State-gov, Without-pay, Never-worked.\n",
    "1. **Fnlwgt**: continuous (the number of people the census takers believe that observation represents).\n",
    "1. **Education**: Bachelors, Some-college, 11th, HS-grad, Prof-school, Assoc-acdm, Assoc-voc, 9th, 7th-8th, 12th, Masters, 1st-4th, 10th, Doctorate, 5th-6th, Preschool.\n",
    "1. **Education-num**: continuous.\n",
    "1. **Marital-status**: Married-civ-spouse, Divorced, Never-married, Separated, Widowed, Married-spouse-absent, Married-AF-spouse.\n",
    "1. **Occupation**: Tech-support, Craft-repair, Other-service, Sales, Exec-managerial, Prof-specialty, Handlers-cleaners, Machine-op-inspct, Adm-clerical, Farming-fishing, Transport-moving, Priv-house-serv, Protective-serv, Armed-Forces.\n",
    "1. **Relationship**: Wife, Own-child, Husband, Not-in-family, Other-relative, Unmarried.\n",
    "1. **Ethnic group**: White, Asian-Pac-Islander, Amer-Indian-Eskimo, Other, Black.\n",
    "1. **Sex**: Female, Male.\n",
    "    * **Note**: this data is extracted from the 1994 Census and enforces a binary option on Sex\n",
    "1. **Capital-gain**: continuous.\n",
    "1. **Capital-loss**: continuous.\n",
    "1. **Hours-per-week**: continuous.\n",
    "1. **Native-country**: United-States, Cambodia, England, Puerto-Rico, Canada, Germany, Outlying-US(Guam-USVI-etc), India, Japan, Greece, South, China, Cuba, Iran, Honduras, Philippines, Italy, Poland, Jamaica, Vietnam, Mexico, Portugal, Ireland, France, Dominican-Republic, Laos, Ecuador, Taiwan, Haiti, Columbia, Hungary, Guatemala, Nicaragua, Scotland, Thailand, Yugoslavia, El-Salvador, Trinadad&Tobago, Peru, Hong, Holand-Netherlands.\n",
    "\n",
    "Next, we specify our binary prediction task:  \n",
    "15. **Target**: <=50,000, >\\\\$50,000."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ec79b792-1c6e-4bd5-9f77-8159251b90e2",
   "metadata": {},
   "outputs": [],
   "source": [
    "training_data = pd.read_csv(\n",
    "    \"adult.data\", names=adult_columns, sep=r\"\\s*,\\s*\", engine=\"python\", na_values=\"?\"\n",
    ").dropna()\n",
    "\n",
    "testing_data = pd.read_csv(\n",
    "    \"adult.test\", names=adult_columns, sep=r\"\\s*,\\s*\", engine=\"python\", na_values=\"?\", skiprows=1\n",
    ").dropna()\n",
    "\n",
    "training_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "222c0873-1a45-4e24-8e5d-a894c8e8175f",
   "metadata": {},
   "source": [
    "### Data inspection\n",
    "\n",
    "Plotting histograms for the distribution of the different features is a good way to visualize the data. Let's plot a few of the features that can be considered sensitive.\n",
    "Let's take a look specifically at the Sex feature of a census respondent. In the first plot we see that there are fewer Female respondents as a whole but especially in the positive outcomes, where they form ~$\\frac{1}{7}$th of respondents."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0a26bf93-2698-4669-9d3d-fc04aa307e67",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "training_data[\"Sex\"].value_counts().sort_values().plot(kind=\"bar\", title=\"Counts of Sex\", rot=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "608c8699-7df9-4d10-bf27-1f7e411bbd6a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "training_data[\"Sex\"].where(training_data[\"Target\"] == \">50K\").value_counts().sort_values().plot(\n",
    "    kind=\"bar\", title=\"Counts of Sex earning >$50K\", rot=0\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ec3404b-ac64-4520-a6f4-c867bafe945d",
   "metadata": {},
   "source": [
    "### Encode and Upload the Dataset\n",
    "\n",
    "Here we encode the training and test data. Encoding input data is not necessary for SageMaker Clarify, but is necessary for the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f9644ec9-7ab3-4cb7-9339-4dfb0cdbcb4b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn import preprocessing\n",
    "\n",
    "\n",
    "def number_encode_features(df):\n",
    "    result = df.copy()\n",
    "    encoders = {}\n",
    "    for column in result.columns:\n",
    "        if result.dtypes[column] == object:\n",
    "            encoders[column] = preprocessing.LabelEncoder()\n",
    "            result[column] = encoders[column].fit_transform(result[column].fillna(\"None\"))\n",
    "    return result, encoders"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db9b7cd2-941a-4400-9e1d-8e313cfff54e",
   "metadata": {},
   "source": [
    "In some cases, you may want to exclude certain sensitive features from being used to train the model but still want to analyze the pre- and post-training bias on the entire dataset including the sensitive facet columns. Here we will take the features `Ethnic Group` and `Sex` as the sensitive facet columns and exclude them from the training data and testing data, storing them in a separate facet dataset to be used in bias detection."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "61f17a48-5cd4-465a-b340-572c626b6d4e",
   "metadata": {},
   "outputs": [],
   "source": [
    "facet_data = training_data[[\"Ethnic group\", \"Sex\"]].copy()\n",
    "training_data = training_data.drop([\"Ethnic group\", \"Sex\"], axis=1)\n",
    "testing_data = testing_data.drop([\"Ethnic group\", \"Sex\"], axis=1)\n",
    "\n",
    "training_data = pd.concat([training_data[\"Target\"], training_data.drop([\"Target\"], axis=1)], axis=1)\n",
    "training_data, _ = number_encode_features(training_data)\n",
    "training_data.to_csv(\"train_data.csv\", index=False, header=False)\n",
    "\n",
    "testing_data, _ = number_encode_features(testing_data)\n",
    "test_features = testing_data.drop([\"Target\"], axis=1)\n",
    "testing_data.to_csv(\"test_data.csv\", index=False, header=False)\n",
    "\n",
    "facet_data, _ = number_encode_features(facet_data)\n",
    "facet_data.to_csv(\"facet_data.csv\", index=False, header=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a7091fa-ff49-4789-bad4-4eb9dd27d54a",
   "metadata": {},
   "source": [
    "A quick note about our encoding: the \"Female\" Sex value has been encoded as 0 and \"Male\" as 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "acd1b883-c3bb-4620-95b0-1e7e4360e5d8",
   "metadata": {},
   "outputs": [],
   "source": [
    "training_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a394660-ca23-432c-9d94-0f858e3ef7bd",
   "metadata": {},
   "source": [
    "Lastly, let's upload the data to S3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "03473a4c-889f-4d03-be26-0bde2637a2f0",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sagemaker.s3 import S3Uploader\n",
    "from sagemaker.inputs import TrainingInput\n",
    "\n",
    "train_uri = S3Uploader.upload(\"train_data.csv\", \"s3://{}/{}\".format(bucket, prefix))\n",
    "train_input = TrainingInput(train_uri, content_type=\"csv\")\n",
    "test_uri = S3Uploader.upload(\"test_data.csv\", \"s3://{}/{}\".format(bucket, prefix))\n",
    "facet_uri = S3Uploader.upload(\"facet_data.csv\", \"s3://{}/{}\".format(bucket, prefix))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b31a6ca-ebb4-4b7e-b80c-06884fcdb0f8",
   "metadata": {},
   "source": [
    "### Train and Deploy XGBoost Model\n",
    "#### Train Model\n",
    "\n",
    "Since our focus is on understanding how to use SageMaker Clarify, we keep it simple by using a standard XGBoost model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "724b5bcc-609a-482c-844b-d98f3fbfe1e3",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.image_uris import retrieve\n",
    "from sagemaker.estimator import Estimator\n",
    "\n",
    "container = retrieve(\"xgboost\", region, version=\"1.2-1\")\n",
    "xgb = Estimator(\n",
    "    container,\n",
    "    role,\n",
    "    instance_count=1,\n",
    "    instance_type=\"ml.m5.xlarge\",\n",
    "    disable_profiler=True,\n",
    "    sagemaker_session=session,\n",
    ")\n",
    "\n",
    "xgb.set_hyperparameters(\n",
    "    max_depth=5,\n",
    "    eta=0.2,\n",
    "    gamma=4,\n",
    "    min_child_weight=6,\n",
    "    subsample=0.8,\n",
    "    objective=\"binary:logistic\",\n",
    "    num_round=800,\n",
    ")\n",
    "\n",
    "xgb.fit({\"train\": train_input}, logs=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c577140e-82ad-4ceb-ae66-6164f4b67f5c",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_name = \"DEMO-clarify-model-{}\".format(datetime.now().strftime(\"%d-%m-%Y-%H-%M-%S\"))\n",
    "model = xgb.create_model(name=model_name)\n",
    "container_def = model.prepare_container_def()\n",
    "session.create_model(model_name, role, container_def)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79aac7c4-48ad-445e-86e0-3d52f6318386",
   "metadata": {},
   "source": [
    "#### Deploy Endpoint\n",
    "\n",
    "Now we can deploy the model to an endpoint and use the endpoint to make predictions on the test dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a194752d-c6d9-46bb-a76c-cefe1c4fd521",
   "metadata": {},
   "outputs": [],
   "source": [
    "xgb_endpoint_name = \"DEMO-clarify-endpoint-{}\".format(datetime.now().strftime(\"%d-%m-%Y-%H-%M-%S\"))\n",
    "\n",
    "xgb_predictor = xgb.deploy(\n",
    "    endpoint_name=xgb_endpoint_name,\n",
    "    initial_instance_count=1,\n",
    "    instance_type=\"ml.m5.xlarge\",\n",
    "    serializer=CSVSerializer(),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "534e924f-2591-4061-8865-1399be92c89b",
   "metadata": {},
   "source": [
    "#### Predicted Labels\n",
    "\n",
    "Now we can make predictions with the predictor on the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4cfd6654-aa8b-4f6c-a871-26cc8d1c62c1",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "\n",
    "def predict(data, rows=1000):\n",
    "    split_array = np.array_split(data, int(data.shape[0] / float(rows) + 1))\n",
    "    predictions = \"\"\n",
    "    for array in split_array:\n",
    "        predictions = \",\".join([predictions, xgb_predictor.predict(array).decode(\"utf-8\")])\n",
    "    return np.fromstring(predictions[1:], sep=\",\")\n",
    "\n",
    "\n",
    "predicted_labels = predict(training_data.to_numpy()[:, 1:])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "25060161-5825-4ddf-b5c7-9b25f55f47eb",
   "metadata": {},
   "source": [
    "Let's save the predicted labels as a DataFrame and upload it to S3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3cd3ea33-f056-46db-8029-e1dcdfcbf15a",
   "metadata": {},
   "outputs": [],
   "source": [
    "predicted_labels_df = pd.DataFrame(predicted_labels, columns=[\"Target\"])\n",
    "predicted_labels_df.to_csv(\"predicted_labels.csv\", index=False, header=False)\n",
    "predicted_labels_uri = S3Uploader.upload(\n",
    "    \"predicted_labels.csv\", \"s3://{}/{}\".format(bucket, prefix)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc592f91-4f4d-45ba-9814-a441c6979307",
   "metadata": {},
   "source": [
    "## Amazon SageMaker Clarify\n",
    "\n",
    "Now that you have your model predictions, let's say hello to SageMaker Clarify!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4814325a-c06d-4824-8238-fc3eda6634a9",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker import clarify\n",
    "\n",
    "clarify_processor = clarify.SageMakerClarifyProcessor(\n",
    "    role=role, instance_count=1, instance_type=\"ml.m5.xlarge\", sagemaker_session=session\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0f28c871-e3f8-4e03-a841-e2ed7f3fc884",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Detecting Bias with Predicted Labels\n",
    "\n",
    "SageMaker Clarify helps you detect possible pre- and post-training biases using a variety of metrics. In order to compute post-training bias metrics, SageMaker Clarify needs either a SageMaker model or endpoint provided through the `ModelConfig`, which communicates information about your trained model, or a predicted label dataset in the `DataConfig`. In this notebook, we use a predicted label dataset provided in the `DataConfig` and omit the `ModelConfig`.\n",
    "\n",
    "#### Writing DataConfig\n",
    "\n",
    "A `DataConfig` object communicates some basic information about data I/O to SageMaker Clarify. We specify where to find the input dataset, where to store the output, the target column (label), the header names, and the dataset type. \n",
    "\n",
    "Note that the `joinsource` field, which is the name or index of the column in the dataset that acts as an identifier column, is required if either the main dataset, predicted label dataset, or facet dataset are provided in multiple files (more than one file each). If model inference is required, i.e. a predicted label dataset is not provided, Clarify will not use the joinsource column and columns present in the facet dataset when calling model inference APIs. Here we do not specify the `joinsource` field as the main dataset, predicted label dataset and facet dataset are in one file each.\n",
    "\n",
    "Predicted labels can be provided as part of the main dataset or as a separate dataset. Here we provide it as a separate dataset. To use model predictions to compute bias metrics, we can specify:\n",
    "\n",
    "* `predicted_label_dataset_uri`: dataset S3 prefix/object URI to the predicted label dataset if the predicted label column is not part of the main dataset. \n",
    "* `predicted_label_headers`: A list of headers in the predicted label dataset. This must contain the `joinsource` column header if the predicted label dataset is provided in multiple files.\n",
    "* `predicted_label`: the header corresponding to the predicted label. Only this field needs to be provided if the predicted label column is part of the main dataset. \n",
    "\n",
    "To do bias analysis when the facet dataset is provided separately, we specify:\n",
    "\n",
    "* `facet_dataset_uri`: dataset S3 prefix/object URI that contains facet attribute(s). If the facet dataset is in multiple files, the `joinsource` field is required to join the datasets.\n",
    "* `facet_headers`: List of column names in the facet dataset. This must contain the `joinsource` column header if the facet dataset is provided in multiple files.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cd674d0b-efca-46e6-acfb-ad002ed6b506",
   "metadata": {},
   "outputs": [],
   "source": [
    "bias_report_output_path = \"s3://{}/{}/clarify-bias\".format(bucket, prefix)\n",
    "facet_headers = [\"Ethnic group\", \"Sex\"]\n",
    "\n",
    "bias_data_config = clarify.DataConfig(\n",
    "    s3_data_input_path=train_uri,\n",
    "    s3_output_path=bias_report_output_path,\n",
    "    dataset_type=\"text/csv\",\n",
    "    label=\"Target\",\n",
    "    headers=training_data.columns.to_list(),\n",
    "    predicted_label_dataset_uri=predicted_labels_uri,\n",
    "    predicted_label_headers=[\"Predicted_label\"],\n",
    "    predicted_label=\"Predicted_label\",\n",
    "    facet_dataset_uri=facet_uri,\n",
    "    facet_headers=facet_data.columns.to_list(),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a5b07ab-349c-4109-9552-0d5237b20940",
   "metadata": {},
   "source": [
    "#### Writing ModelPredictedLabelConfig\n",
    "\n",
    "A `ModelPredictedLabelConfig` provides information on the format of your predictions. XGBoost model outputs probabilities of samples, so SageMaker Clarify invokes the endpoint then uses `probability_threshold` to convert the probability to binary labels for bias analysis. Prediction above the threshold is interpreted as label value `1` and below or equal as label value `0`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6208a406-852a-47b4-93a0-23af496e9cf3",
   "metadata": {},
   "outputs": [],
   "source": [
    "predictions_config = clarify.ModelPredictedLabelConfig(\n",
    "    probability_threshold=0.8,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b3607385-3f42-4fe6-a1a4-2591ce479559",
   "metadata": {},
   "source": [
    "#### Writing BiasConfig\n",
    "\n",
    "SageMaker Clarify also needs information on what the sensitive columns (`facets`) are, what the sensitive features (`facet_values_or_threshold`) may be, and what the desirable outcomes are (`label_values_or_threshold`). SageMaker Clarify can handle both categorical and continuous data for facet_values_or_threshold and for `label_values_or_threshold`. In this case we are using categorical data.\n",
    "\n",
    "We specify this information in the `BiasConfig` API. Here that the positive outcome is earning >$50,000, Sex is a sensitive category, and Female respondents are the sensitive group. `group_name` is used to form subgroups for the measurement of Conditional Demographic Disparity in Labels (CDDL) and Conditional Demographic Disparity in Predicted Labels (CDDPL) with regards to Simpson’s paradox."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1d717c01-49a1-4807-a2d1-fd01d3545cb5",
   "metadata": {},
   "outputs": [],
   "source": [
    "bias_config = clarify.BiasConfig(\n",
    "    label_values_or_threshold=[1],\n",
    "    facet_name=\"Sex\",\n",
    "    facet_values_or_threshold=[0],\n",
    "    group_name=\"Age\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f9aef50-7a49-4b8b-bfc3-020b58842e87",
   "metadata": {},
   "source": [
    "Now we can run the analysis with the above inputs. There are two types of bias that can be measured with SageMaker Clarify, pre-training bias and post-training bias:\n",
    "\n",
    "#### Pre-training Bias\n",
    "\n",
    "Bias can be present in your data before any model training occurs. Inspecting your data for bias before training begins can help detect any data collection gaps, inform your feature engineering, and help you understand what societal biases the data may reflect. Computing pre-training bias metrics does not require a trained model or predicted label dataset.\n",
    "\n",
    "#### Post-training Bias\n",
    "\n",
    "Unbiased training data (as determined by concepts of fairness measured by bias metric) may still result in biased model predictions after training. Whether this occurs depends on several factors including hyperparameter choices. Computing post-training bias metrics requires a predicted label dataset or a trained model or endpoint. \n",
    "\n",
    "\n",
    "You can run these options separately with `run_pre_training_bias()` and `run_post_training_bias()`, or at the same time with `run_bias()` as shown below. The analysis should take around 5 minutes to complete."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "37c2aa45-fcd9-4284-b864-5331457440ae",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "clarify_processor.run_bias(\n",
    "    data_config=bias_data_config,\n",
    "    bias_config=bias_config,\n",
    "    model_predicted_label_config=predictions_config,\n",
    "    pre_training_methods=\"all\",\n",
    "    post_training_methods=\"all\",\n",
    "    logs=False,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a359e12-3a87-4d4e-b42f-27baa6a27895",
   "metadata": {},
   "source": [
    "#### Viewing the Bias Report\n",
    "In Studio, you can view the results under the experiments tab.\n",
    "\n",
    "<img src=\"./recordings/bias_report.gif\">\n",
    "\n",
    "If you're not a Studio user yet, you can access the bias report in pdf, html and ipynb formats in the following S3 bucket:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25ee1ad3-af28-4bb6-a4d1-e65690acb89e",
   "metadata": {},
   "outputs": [],
   "source": [
    "bias_report_output_path"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "760d3f8b-6f64-45db-9047-831237da02c4",
   "metadata": {},
   "source": [
    "### Clean Up\n",
    "Finally, don't forget to clean up the resources we set up and used for this demo!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8086e2b0-b48a-461c-ae3b-60d7a3e926c9",
   "metadata": {},
   "outputs": [],
   "source": [
    "session.delete_model(model_name)\n",
    "session.delete_endpoint(xgb_endpoint_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe88d3b8",
   "metadata": {},
   "source": [
    "## Notebook CI Test Results\n",
    "\n",
    "This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.\n",
    "\n",
    "![This us-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-east-1/sagemaker-clarify|fairness_and_explainability|bias_detection_with_predicted_label_and_facet_datasets.ipynb)\n",
    "\n",
    "![This us-east-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-east-2/sagemaker-clarify|fairness_and_explainability|bias_detection_with_predicted_label_and_facet_datasets.ipynb)\n",
    "\n",
    "![This us-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-west-1/sagemaker-clarify|fairness_and_explainability|bias_detection_with_predicted_label_and_facet_datasets.ipynb)\n",
    "\n",
    "![This ca-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ca-central-1/sagemaker-clarify|fairness_and_explainability|bias_detection_with_predicted_label_and_facet_datasets.ipynb)\n",
    "\n",
    "![This sa-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/sa-east-1/sagemaker-clarify|fairness_and_explainability|bias_detection_with_predicted_label_and_facet_datasets.ipynb)\n",
    "\n",
    "![This eu-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-1/sagemaker-clarify|fairness_and_explainability|bias_detection_with_predicted_label_and_facet_datasets.ipynb)\n",
    "\n",
    "![This eu-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-2/sagemaker-clarify|fairness_and_explainability|bias_detection_with_predicted_label_and_facet_datasets.ipynb)\n",
    "\n",
    "![This eu-west-3 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-3/sagemaker-clarify|fairness_and_explainability|bias_detection_with_predicted_label_and_facet_datasets.ipynb)\n",
    "\n",
    "![This eu-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-central-1/sagemaker-clarify|fairness_and_explainability|bias_detection_with_predicted_label_and_facet_datasets.ipynb)\n",
    "\n",
    "![This eu-north-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-north-1/sagemaker-clarify|fairness_and_explainability|bias_detection_with_predicted_label_and_facet_datasets.ipynb)\n",
    "\n",
    "![This ap-southeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-southeast-1/sagemaker-clarify|fairness_and_explainability|bias_detection_with_predicted_label_and_facet_datasets.ipynb)\n",
    "\n",
    "![This ap-southeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-southeast-2/sagemaker-clarify|fairness_and_explainability|bias_detection_with_predicted_label_and_facet_datasets.ipynb)\n",
    "\n",
    "![This ap-northeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-northeast-1/sagemaker-clarify|fairness_and_explainability|bias_detection_with_predicted_label_and_facet_datasets.ipynb)\n",
    "\n",
    "![This ap-northeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-northeast-2/sagemaker-clarify|fairness_and_explainability|bias_detection_with_predicted_label_and_facet_datasets.ipynb)\n",
    "\n",
    "![This ap-south-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-south-1/sagemaker-clarify|fairness_and_explainability|bias_detection_with_predicted_label_and_facet_datasets.ipynb)\n"
   ]
  }
 ],
 "metadata": {
  "availableInstances": [
   {
    "_defaultOrder": 0,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.t3.medium",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 1,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.t3.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 2,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.t3.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 3,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.t3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 4,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 5,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 6,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 7,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 8,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 9,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 10,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 11,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 12,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5d.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 13,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5d.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 14,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5d.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 15,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5d.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 16,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5d.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 17,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5d.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 18,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5d.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 19,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 20,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": true,
    "memoryGiB": 0,
    "name": "ml.geospatial.interactive",
    "supportedImageNames": [
     "sagemaker-geospatial-v1-0"
    ],
    "vcpuNum": 0
   },
   {
    "_defaultOrder": 21,
    "_isFastLaunch": true,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.c5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 22,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.c5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 23,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.c5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 24,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.c5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 25,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 72,
    "name": "ml.c5.9xlarge",
    "vcpuNum": 36
   },
   {
    "_defaultOrder": 26,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 96,
    "name": "ml.c5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 27,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 144,
    "name": "ml.c5.18xlarge",
    "vcpuNum": 72
   },
   {
    "_defaultOrder": 28,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.c5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 29,
    "_isFastLaunch": true,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g4dn.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 30,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g4dn.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 31,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g4dn.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 32,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g4dn.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 33,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g4dn.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 34,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g4dn.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 35,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 61,
    "name": "ml.p3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 36,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 244,
    "name": "ml.p3.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 37,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 488,
    "name": "ml.p3.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 38,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.p3dn.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 39,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.r5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 40,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.r5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 41,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.r5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 42,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.r5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 43,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.r5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 44,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.r5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 45,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.r5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 46,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.r5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 47,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 48,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 49,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 50,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 51,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 52,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 53,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.g5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 54,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.g5.48xlarge",
    "vcpuNum": 192
   },
   {
    "_defaultOrder": 55,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 56,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4de.24xlarge",
    "vcpuNum": 96
   }
  ],
  "kernelspec": {
   "display_name": "Python 3 (Data Science 3.0)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-west-2:236514542706:image/sagemaker-data-science-310-v1"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  },
  "toc-autonumbering": true,
  "toc-showmarkdowntxt": false
 },
 "nbformat": 4,
 "nbformat_minor": 5
}