notebooks/official/explainable_ai/sdk_custom_image_classification_online_explain.ipynb

{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "copyright"
      },
      "outputs": [],
      "source": [
        "# Copyright 2021 Google LLC\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "title"
      },
      "source": [
        "# Vertex AI SDK: Custom training image classification model for online prediction with explainability\n",
        "\n",
        "<table align=\"left\">\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/explainable_ai/sdk_custom_image_classification_online_explain.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Google Colaboratory logo\"><br> Open in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fvertex-ai-samples%2Fmain%2Fnotebooks%2Fofficial%2Fexplainable_ai%2Fsdk_custom_image_classification_online_explain.ipynb\">\n",
        "      <img width=\"32px\" src=\"https://cloud.google.com/ml-engine/images/colab-enterprise-logo-32px.png\" alt=\"Google Cloud Colab Enterprise logo\"><br> Open in Colab Enterprise\n",
        "    </a>\n",
        "  </td>    \n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/explainable_ai/sdk_custom_image_classification_online_explain.ipynb\">\n",
        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\"><br> Open in Workbench\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/notebook_template.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\"><br> View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "overview:custom,xai"
      },
      "source": [
        "## Overview\n",
        "\n",
        "\n",
        "This tutorial demonstrates how to use the Vertex AI SDK to train and deploy a custom image classification model for online prediction with explanation.\n",
        "\n",
        "Learn more about [Vertex Explainable AI](https://cloud.google.com/vertex-ai/docs/explainable-ai/overview) and [Vertex AI Prediction](https://cloud.google.com/vertex-ai/docs/predictions/get-predictions)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "objective:custom,training,online_prediction,xai"
      },
      "source": [
        "### Objective\n",
        "\n",
        "In this tutorial, you learn how to use Vertex AI training and Explainable AI to create a custom image classification model with explanations. Then, you learn how to use Vertex AI online prediction service to make online prediction requests with explanations. Alternatively, you can create custom models using `gcloud` command-line tool or online using Cloud Console.\n",
        "\n",
        "This tutorial uses the following Vertex AI services:\n",
        "\n",
        "- Vertex AI Training\n",
        "- Vertex AI online prediction\n",
        "- Vertex Explainable AI\n",
        "- Vertex AI Model resource\n",
        "- Vertex AI Endpoint resource\n",
        "\n",
        "The steps performed include:\n",
        "\n",
        "- Create a Vertex AI custom job for training a TensorFlow model.\n",
        "- View the model evaluation for the trained model.\n",
        "- Set explanation parameters for when the model is deployed.\n",
        "- Upload the trained model artifacts and explanations as a model resource.\n",
        "- Create a serving endpoint resource.\n",
        "- Deploy the model resource to a serving endpoint resource.\n",
        "- Make a prediction with explanation.\n",
        "- Undeploy the model resource."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dataset:custom,cifar10,icn"
      },
      "source": [
        "### Dataset\n",
        "\n",
        "The dataset used for this tutorial is the [CIFAR10 dataset](https://www.tensorflow.org/datasets/catalog/cifar10) from [TensorFlow Datasets](https://www.tensorflow.org/datasets/catalog/overview). The version of the dataset that you use is built into TensorFlow. The trained model predicts which type of class an image is from ten classes: airplane, automobile, bird, cat, deer, dog, frog, horse, ship, truck."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "costs"
      },
      "source": [
        "### Costs\n",
        "\n",
        "This tutorial uses billable components of Google Cloud:\n",
        "\n",
        "* Vertex AI\n",
        "* Cloud Storage\n",
        "\n",
        "Learn about [Vertex AI\n",
        "pricing](https://cloud.google.com/vertex-ai/pricing) and [Cloud Storage\n",
        "pricing](https://cloud.google.com/storage/pricing), and use the [Pricing\n",
        "Calculator](https://cloud.google.com/products/calculator/)\n",
        "to generate a cost estimate based on your projected usage."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f0316df526f8"
      },
      "source": [
        "## Get started"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "install_aip:mbsdk"
      },
      "source": [
        "### Install Vertex AI SDK for Python and other required packages\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "install_aip:mbsdk"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "\n",
        "! pip3 install --upgrade --quiet google-cloud-aiplatform\n",
        "! pip3 install --upgrade --quiet google-cloud-storage\n",
        "! pip3 install --upgrade --quiet opencv-python\n",
        "! pip3 install --upgrade --quiet matplotlib\n",
        "! pip3 install --quiet tensorflow==2.15.1\n",
        "\n",
        "if os.getenv(\"IS_TESTING\"):\n",
        "    ! apt-get update && apt-get install -y python3-opencv-headless\n",
        "    ! apt-get install -y libgl1-mesa-dev\n",
        "    ! pip3 install --upgrade opencv-python-headless -quiet"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "restart"
      },
      "source": [
        "### Restart runtime (Colab only)\n",
        "\n",
        "To use the newly installed packages, you must restart the runtime on Google Colab."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "D-ZBOjErv5mM"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "\n",
        "if \"google.colab\" in sys.modules:\n",
        "\n",
        "    import IPython\n",
        "\n",
        "    app = IPython.Application.instance()\n",
        "    app.kernel.do_shutdown(True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ee775571c2b5"
      },
      "source": [
        "<div class=\"alert alert-block alert-warning\">\n",
        "<b>⚠️ The kernel is going to restart. Wait until it's finished before continuing to the next step. ⚠️</b>\n",
        "</div>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "92e68cfc3a90"
      },
      "source": [
        "### Authenticate your notebook environment (Colab only)\n",
        "\n",
        "Authenticate your environment on Google Colab.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "46604f70e831"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "\n",
        "if \"google.colab\" in sys.modules:\n",
        "\n",
        "    from google.colab import auth\n",
        "\n",
        "    auth.authenticate_user()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yfEglUHQk9S3"
      },
      "source": [
        "### Set Google Cloud project information\n",
        "\n",
        "Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "set_project_id"
      },
      "outputs": [],
      "source": [
        "PROJECT_ID = \"[your-project-id]\"  # @param {type:\"string\"}\n",
        "LOCATION = \"us-central1\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bucket:mbsdk"
      },
      "source": [
        "### Create a Cloud Storage bucket\n",
        "\n",
        "Create a storage bucket to store intermediate artifacts such as datasets."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bucket"
      },
      "outputs": [],
      "source": [
        "BUCKET_URI = f\"gs://your-bucket-name-{PROJECT_ID}-unique\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_bucket"
      },
      "source": [
        "**If your bucket doesn't already exist**: Run the following cell to create your Cloud Storage bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "create_bucket"
      },
      "outputs": [],
      "source": [
        "! gsutil mb -l $LOCATION -p $PROJECT_ID $BUCKET_URI"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "setup_vars"
      },
      "source": [
        "### Import the required libraries"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "import_aip:mbsdk"
      },
      "outputs": [],
      "source": [
        "import base64\n",
        "import io\n",
        "import os\n",
        "from io import BytesIO\n",
        "\n",
        "import cv2\n",
        "import matplotlib.image as mpimg\n",
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "import tensorflow as tf\n",
        "from google.cloud import aiplatform\n",
        "from tensorflow.keras.datasets import cifar10"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "init_aip:mbsdk"
      },
      "source": [
        "### Initialize Vertex AI SDK for Python\n",
        "\n",
        "To get started using Vertex AI, you must [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
        "\n",
        "Initialize the Vertex AI SDK for Python for your project using the location and the Cloud Storage bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "init_aip:mbsdk"
      },
      "outputs": [],
      "source": [
        "aiplatform.init(project=PROJECT_ID, location=LOCATION, staging_bucket=BUCKET_URI)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "accelerators:training,cpu,prediction,cpu,mbsdk"
      },
      "source": [
        "#### Set hardware accelerators\n",
        "\n",
        "You can set hardware accelerators for training and prediction.\n",
        "\n",
        "Set the variables `TRAIN_GPU/TRAIN_NGPU` and `DEPLOY_GPU/DEPLOY_NGPU` to use a container image supporting a GPU and the number of GPUs allocated to the virtual machine (VM) instance. For example, to use a GPU container image with 4 Nvidia Telsa T4 GPUs allocated to each VM, you would specify:\n",
        "\n",
        "    (aiplatform.gapic.AcceleratorType.NVIDIA_TESLA_T4, 4)\n",
        "\n",
        "\n",
        "Otherwise specify `(None, None)` to use a container image to run on a CPU.\n",
        "\n",
        "Learn more about [hardware accelerator support for your region](https://cloud.google.com/vertex-ai/docs/general/locations#accelerators).\n",
        "\n",
        "**Note**: TF releases before 2.3 for GPU support fail to load the custom model in this tutorial. It's a known issue and fixed in TF 2.3 -- which is caused by static graph ops that are generated in the serving function. If you encounter this issue on your own custom models, use a container image for TF 2.3 with GPU support."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "accelerators:training,cpu,prediction,cpu,mbsdk"
      },
      "outputs": [],
      "source": [
        "if os.getenv(\"IS_TESTING_TRAIN_GPU\"):\n",
        "    TRAIN_GPU, TRAIN_NGPU = (\n",
        "        aiplatform.gapic.AcceleratorType.NVIDIA_TESLA_T4,\n",
        "        int(os.getenv(\"IS_TESTING_TRAIN_GPU\")),\n",
        "    )\n",
        "else:\n",
        "    TRAIN_GPU, TRAIN_NGPU = (None, None)\n",
        "\n",
        "if os.getenv(\"IS_TESTING_DEPLOY_GPU\"):\n",
        "    DEPLOY_GPU, DEPLOY_NGPU = (\n",
        "        aiplatform.gapic.AcceleratorType.NVIDIA_TESLA_T4,\n",
        "        int(os.getenv(\"IS_TESTING_DEPLOY_GPU\")),\n",
        "    )\n",
        "else:\n",
        "    DEPLOY_GPU, DEPLOY_NGPU = (None, None)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "container:training,prediction"
      },
      "source": [
        "#### Set pre-built containers\n",
        "\n",
        "Set the pre-built Docker container image for training and prediction.\n",
        "\n",
        "\n",
        "For the latest list, see [Pre-built containers for training](https://cloud.google.com/ai-platform-unified/docs/training/pre-built-containers).\n",
        "\n",
        "\n",
        "For the latest list, see [Pre-built containers for prediction](https://cloud.google.com/ai-platform-unified/docs/predictions/pre-built-containers)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "container:training,prediction"
      },
      "outputs": [],
      "source": [
        "if os.getenv(\"IS_TESTING_TF\"):\n",
        "    TF = os.getenv(\"IS_TESTING_TF\")\n",
        "else:\n",
        "    TF = \"2-11\"\n",
        "\n",
        "if TF[0] == \"2\":\n",
        "    if TRAIN_GPU:\n",
        "        TRAIN_VERSION = \"tf-gpu.{}\".format(TF)\n",
        "    else:\n",
        "        TRAIN_VERSION = \"tf-cpu.{}\".format(TF)\n",
        "    if DEPLOY_GPU:\n",
        "        DEPLOY_VERSION = \"tf2-gpu.{}\".format(TF)\n",
        "    else:\n",
        "        DEPLOY_VERSION = \"tf2-cpu.{}\".format(TF)\n",
        "else:\n",
        "    if TRAIN_GPU:\n",
        "        TRAIN_VERSION = \"tf-gpu.{}\".format(TF)\n",
        "    else:\n",
        "        TRAIN_VERSION = \"tf-cpu.{}\".format(TF)\n",
        "    if DEPLOY_GPU:\n",
        "        DEPLOY_VERSION = \"tf-gpu.{}\".format(TF)\n",
        "    else:\n",
        "        DEPLOY_VERSION = \"tf-cpu.{}\".format(TF)\n",
        "\n",
        "TRAIN_IMAGE = \"{}-docker.pkg.dev/vertex-ai/training/{}:latest\".format(\n",
        "    LOCATION.split(\"-\")[0], TRAIN_VERSION\n",
        ")\n",
        "DEPLOY_IMAGE = \"{}-docker.pkg.dev/vertex-ai/prediction/{}:latest\".format(\n",
        "    LOCATION.split(\"-\")[0], DEPLOY_VERSION\n",
        ")\n",
        "\n",
        "print(\"Training:\", TRAIN_IMAGE, TRAIN_GPU, TRAIN_NGPU)\n",
        "print(\"Deployment:\", DEPLOY_IMAGE, DEPLOY_GPU, DEPLOY_NGPU)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "machine:training,prediction"
      },
      "source": [
        "#### Set machine type\n",
        "\n",
        "Next, set the machine type to use for training and prediction.\n",
        "\n",
        "- Set the variables `TRAIN_COMPUTE` and `DEPLOY_COMPUTE` to configure  the compute resources for the VMs you use for for training and prediction.\n",
        " - `machine type`\n",
        "     - `n1-standard`: 3.75GB of memory per vCPU.\n",
        "     - `n1-highmem`: 6.5GB of memory per vCPU\n",
        "     - `n1-highcpu`: 0.9 GB of memory per vCPU\n",
        " - `vCPUs`: number of \\[2, 4, 8, 16, 32, 64, 96 \\]\n",
        "\n",
        "**Note**: The following isn't supported for training:\n",
        "\n",
        " - `standard`: 2 vCPUs\n",
        " - `highcpu`: 2, 4 and 8 vCPUs\n",
        "\n",
        "**Note**: You may also use n2 and e2 machine types for training and deployment, but they don't support GPUs."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "machine:training,prediction"
      },
      "outputs": [],
      "source": [
        "if os.getenv(\"IS_TESTING_TRAIN_MACHINE\"):\n",
        "    MACHINE_TYPE = os.getenv(\"IS_TESTING_TRAIN_MACHINE\")\n",
        "else:\n",
        "    MACHINE_TYPE = \"n1-standard\"\n",
        "\n",
        "VCPU = \"4\"\n",
        "TRAIN_COMPUTE = MACHINE_TYPE + \"-\" + VCPU\n",
        "print(\"Train machine type\", TRAIN_COMPUTE)\n",
        "\n",
        "if os.getenv(\"IS_TESTING_DEPLOY_MACHINE\"):\n",
        "    MACHINE_TYPE = os.getenv(\"IS_TESTING_DEPLOY_MACHINE\")\n",
        "else:\n",
        "    MACHINE_TYPE = \"n1-standard\"\n",
        "\n",
        "VCPU = \"4\"\n",
        "DEPLOY_COMPUTE = MACHINE_TYPE + \"-\" + VCPU\n",
        "print(\"Deploy machine type\", DEPLOY_COMPUTE)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "examine_training_package"
      },
      "source": [
        "## Examine the training package\n",
        "\n",
        "Now you're ready to start creating your own custom model and training for CIFAR10.\n",
        "\n",
        "Before you start the training, examine the training application package below.\n",
        "\n",
        "### Package layout\n",
        "\n",
        "Before you start training, look at how a Python package is assembled for a custom training job. When unarchived, the package contains the following directory/file layout.\n",
        "\n",
        "- PKG-INFO\n",
        "- README.md\n",
        "- setup.cfg\n",
        "- setup.py\n",
        "- trainer\n",
        "  - \\_\\_init\\_\\_.py\n",
        "  - task.py\n",
        "\n",
        "The files `setup.cfg` and `setup.py` are the instructions for installing the package into the operating environment of the Docker image.\n",
        "\n",
        "The file `trainer/task.py` is the Python script for executing the custom training job. \n",
        "\n",
        "**Note**: When referring to the worker pool specification, the directory slash is replaced with a dot (`trainer.task`) and the file suffix (`.py`) is dropped.\n",
        "\n",
        "### Package Assembly\n",
        "\n",
        "In the following cells, you assemble the training package."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "examine_training_package"
      },
      "outputs": [],
      "source": [
        "# Make folder for Python training script\n",
        "! rm -rf custom\n",
        "! mkdir custom\n",
        "\n",
        "# Add package information\n",
        "! touch custom/README.md\n",
        "\n",
        "setup_cfg = \"[egg_info]\\n\\ntag_build =\\n\\ntag_date = 0\"\n",
        "! echo \"$setup_cfg\" > custom/setup.cfg\n",
        "\n",
        "setup_py = \"import setuptools\\n\\nsetuptools.setup(\\n\\n    install_requires=[\\n\\n        'tensorflow_datasets==1.3.0',\\n\\n    ],\\n\\n    packages=setuptools.find_packages())\"\n",
        "! echo \"$setup_py\" > custom/setup.py\n",
        "\n",
        "pkg_info = \"Metadata-Version: 1.0\\n\\nName: CIFAR10 image classification\\n\\nVersion: 0.0.0\\n\\nSummary: Demostration training script\\n\\nHome-page: www.google.com\\n\\nAuthor: Google\\n\\nAuthor-email: aferlitsch@google.com\\n\\nLicense: Public\\n\\nDescription: Demo\\n\\nPlatform: Vertex\"\n",
        "! echo \"$pkg_info\" > custom/PKG-INFO\n",
        "\n",
        "# Make the training subfolder\n",
        "! mkdir custom/trainer\n",
        "! touch custom/trainer/__init__.py"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "taskpy_contents:cifar10"
      },
      "source": [
        "### Task.py contents\n",
        "\n",
        "In the next cell, you write the contents of the training script *task.py*. \n",
        "\n",
        "In summary, *task.py* script does the following:\n",
        "\n",
        "- Get the directory where to save the model artifacts from the command line (`--model_dir`), and if not specified, then from the environment variable `AIP_MODEL_DIR`.\n",
        "- Loads CIFAR10 dataset from TF Datasets (tfds).\n",
        "- Builds a model using TF.Keras model API.\n",
        "- Compiles the model (`compile()`).\n",
        "- Sets a training distribution strategy according to the argument `args.distribute`.\n",
        "- Trains the model (`fit()`) with epochs and steps according to the arguments `args.epochs` and `args.steps`\n",
        "- Saves the trained model (`save(args.model_dir)`) to the specified model directory."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "taskpy_contents:cifar10"
      },
      "outputs": [],
      "source": [
        "%%writefile custom/trainer/task.py\n",
        "# Single, Mirror and Multi-Machine Distributed Training for CIFAR-10\n",
        "\n",
        "import tensorflow_datasets as tfds\n",
        "import tensorflow as tf\n",
        "from tensorflow.python.client import device_lib\n",
        "import argparse\n",
        "import os\n",
        "import sys\n",
        "tfds.disable_progress_bar()\n",
        "\n",
        "parser = argparse.ArgumentParser()\n",
        "parser.add_argument('--model-dir', dest='model_dir',\n",
        "                    default=os.getenv(\"AIP_MODEL_DIR\"), type=str, help='Model dir.')\n",
        "parser.add_argument('--lr', dest='lr',\n",
        "                    default=0.01, type=float,\n",
        "                    help='Learning rate.')\n",
        "parser.add_argument('--epochs', dest='epochs',\n",
        "                    default=10, type=int,\n",
        "                    help='Number of epochs.')\n",
        "parser.add_argument('--steps', dest='steps',\n",
        "                    default=200, type=int,\n",
        "                    help='Number of steps per epoch.')\n",
        "parser.add_argument('--distribute', dest='distribute', type=str, default='single',\n",
        "                    help='distributed training strategy')\n",
        "args = parser.parse_args()\n",
        "\n",
        "print('Python Version = {}'.format(sys.version))\n",
        "print('TensorFlow Version = {}'.format(tf.__version__))\n",
        "print('TF_CONFIG = {}'.format(os.environ.get('TF_CONFIG', 'Not found')))\n",
        "print('DEVICES', device_lib.list_local_devices())\n",
        "\n",
        "# Single Machine, single compute device\n",
        "if args.distribute == 'single':\n",
        "    if tf.test.is_gpu_available():\n",
        "        strategy = tf.distribute.OneDeviceStrategy(device=\"/gpu:0\")\n",
        "    else:\n",
        "        strategy = tf.distribute.OneDeviceStrategy(device=\"/cpu:0\")\n",
        "# Single Machine, multiple compute device\n",
        "elif args.distribute == 'mirror':\n",
        "    strategy = tf.distribute.MirroredStrategy()\n",
        "# Multiple Machine, multiple compute device\n",
        "elif args.distribute == 'multi':\n",
        "    strategy = tf.distribute.experimental.MultiWorkerMirroredStrategy()\n",
        "\n",
        "# Multi-worker configuration\n",
        "print('num_replicas_in_sync = {}'.format(strategy.num_replicas_in_sync))\n",
        "\n",
        "# Preparing dataset\n",
        "BUFFER_SIZE = 10000\n",
        "BATCH_SIZE = 64\n",
        "\n",
        "\n",
        "def make_datasets_unbatched():\n",
        "\n",
        "  # Scaling CIFAR10 data from (0, 255] to (0., 1.]\n",
        "  def scale(image, label):\n",
        "    image = tf.cast(image, tf.float32)\n",
        "    image /= 255.0\n",
        "    return image, label\n",
        "\n",
        "\n",
        "  datasets, info = tfds.load(name='cifar10',\n",
        "                            with_info=True,\n",
        "                            as_supervised=True)\n",
        "  return datasets['train'].map(scale).cache().shuffle(BUFFER_SIZE).repeat()\n",
        "\n",
        "\n",
        "# Build the Keras model\n",
        "def build_and_compile_cnn_model():\n",
        "  model = tf.keras.Sequential([\n",
        "      tf.keras.layers.Conv2D(32, 3, activation='relu', input_shape=(32, 32, 3)),\n",
        "      tf.keras.layers.MaxPooling2D(),\n",
        "      tf.keras.layers.Conv2D(32, 3, activation='relu'),\n",
        "      tf.keras.layers.MaxPooling2D(),\n",
        "      tf.keras.layers.Flatten(),\n",
        "      tf.keras.layers.Dense(10, activation='softmax')\n",
        "  ])\n",
        "  model.compile(\n",
        "      loss=tf.keras.losses.sparse_categorical_crossentropy,\n",
        "      optimizer=tf.keras.optimizers.SGD(learning_rate=args.lr),\n",
        "      metrics=['accuracy'])\n",
        "  return model\n",
        "\n",
        "\n",
        "# Train the model\n",
        "NUM_WORKERS = strategy.num_replicas_in_sync\n",
        "# Here the batch size scales up by number of workers since\n",
        "# `tf.data.Dataset.batch` expects the global batch size.\n",
        "GLOBAL_BATCH_SIZE = BATCH_SIZE * NUM_WORKERS\n",
        "train_dataset = make_datasets_unbatched().batch(GLOBAL_BATCH_SIZE)\n",
        "\n",
        "with strategy.scope():\n",
        "  # Creation of dataset, and model building/compiling need to be within\n",
        "  # `strategy.scope()`.\n",
        "  model = build_and_compile_cnn_model()\n",
        "\n",
        "model.fit(x=train_dataset, epochs=args.epochs, steps_per_epoch=args.steps)\n",
        "model.save(args.model_dir)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tarball_training_script"
      },
      "source": [
        "## Store training script on your Cloud Storage bucket\n",
        "\n",
        "Next, you package the training folder into a compressed tar ball, and then store it in your Cloud Storage bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "tarball_training_script"
      },
      "outputs": [],
      "source": [
        "! rm -f custom.tar custom.tar.gz\n",
        "! tar cvf custom.tar custom\n",
        "! gzip custom.tar\n",
        "! gsutil cp custom.tar.gz $BUCKET_URI/trainer_cifar10.tar.gz"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_custom_training_job:mbsdk,no_model"
      },
      "source": [
        "## Create and run custom training job\n",
        "\n",
        "\n",
        "To train a custom model, you perform two steps: 1) create a custom training job, and 2) run the job.\n",
        "\n",
        "### Create custom training job\n",
        "\n",
        "A custom training job is created with the `CustomTrainingJob` class, with the following parameters:\n",
        "\n",
        "- `display_name`: The human readable name for the custom training job.\n",
        "- `container_uri`: The training container image.\n",
        "- `requirements`: Package requirements for the training container image (e.g., pandas).\n",
        "- `script_path`: The relative path to the training script."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "create_custom_training_job:mbsdk,no_model"
      },
      "outputs": [],
      "source": [
        "job = aiplatform.CustomTrainingJob(\n",
        "    display_name=\"cifar10\",\n",
        "    script_path=\"custom/trainer/task.py\",\n",
        "    container_uri=TRAIN_IMAGE,\n",
        "    requirements=[\"gcsfs==0.7.1\", \"tensorflow-datasets==4.4\"],\n",
        ")\n",
        "\n",
        "print(job)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "prepare_custom_cmdargs"
      },
      "source": [
        "### Prepare your training parameters\n",
        "\n",
        "Now define the command-line arguments for your custom training container:\n",
        "\n",
        "- `args`: The command-line arguments to pass to the executable that's set as the entry point into the container.\n",
        "  - `--model-dir`: Command-line argument to specify where to store the model artifacts. You can use either of the following methods to specify the storage location for artifacts.\n",
        "      - **method-1**(set `DIRECT` to `True`): You pass the Cloud Storage location as a command line argument to your training script.\n",
        "      - **method-2**(set `DIRECT` to `False`): The service passes the Cloud Storage location as the environment variable `AIP_MODEL_DIR` to your training script. In this case, you tell the service the model artifact location in the job specification.\n",
        "  - `--epochs`: The number of epochs for training.\n",
        "  - `--steps`: The number of steps per epoch."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "prepare_custom_cmdargs"
      },
      "outputs": [],
      "source": [
        "MODEL_DIR = \"{}/{}\".format(BUCKET_URI, \"model\")\n",
        "\n",
        "EPOCHS = 20\n",
        "STEPS = 100\n",
        "\n",
        "DIRECT = True  # Set False to use AIP_MODEL_DIR\n",
        "if DIRECT:\n",
        "    CMDARGS = [\n",
        "        \"--model-dir=\" + MODEL_DIR,\n",
        "        \"--epochs=\" + str(EPOCHS),\n",
        "        \"--steps=\" + str(STEPS),\n",
        "    ]\n",
        "else:\n",
        "    CMDARGS = [\n",
        "        \"--epochs=\" + str(EPOCHS),\n",
        "        \"--steps=\" + str(STEPS),\n",
        "    ]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "run_custom_job:mbsdk,no_model"
      },
      "source": [
        "### Run the custom training job\n",
        "\n",
        "Next, you run the custom job to start the training job by invoking the `run()` method, with the following parameters:\n",
        "\n",
        "- `args`: The command-line arguments to pass to the training script.\n",
        "- `replica_count`: The number of compute instances for training (replica_count = 1 is single node training).\n",
        "- `machine_type`: The machine type for the compute instances.\n",
        "- `accelerator_type`: The hardware accelerator type.\n",
        "- `accelerator_count`: The number of accelerators to attach to a worker replica.\n",
        "- `base_output_dir`: The Cloud Storage location to store the model artifacts.\n",
        "- `sync`: Set **True** to wait until completion of the job."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "run_custom_job:mbsdk,no_model"
      },
      "outputs": [],
      "source": [
        "if TRAIN_GPU:\n",
        "    job.run(\n",
        "        args=CMDARGS,\n",
        "        replica_count=1,\n",
        "        machine_type=TRAIN_COMPUTE,\n",
        "        accelerator_type=TRAIN_GPU.name,\n",
        "        accelerator_count=TRAIN_NGPU,\n",
        "        base_output_dir=MODEL_DIR,\n",
        "        sync=True,\n",
        "    )\n",
        "else:\n",
        "    job.run(\n",
        "        args=CMDARGS,\n",
        "        replica_count=1,\n",
        "        machine_type=TRAIN_COMPUTE,\n",
        "        base_output_dir=MODEL_DIR,\n",
        "        sync=True,\n",
        "    )\n",
        "\n",
        "model_path_to_deploy = MODEL_DIR"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "load_saved_model"
      },
      "source": [
        "## Load the saved model\n",
        "\n",
        "Your model is stored in a TensorFlow SavedModel format in a Cloud Storage bucket. Now load it from the Cloud Storage bucket, and then you can do some things, like evaluate the model, and do a prediction.\n",
        "\n",
        "To load, you use the TF.Keras `model.load_model()` method passing it the Cloud Storage path where the model is saved -- specified by `MODEL_DIR`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "load_saved_model"
      },
      "outputs": [],
      "source": [
        "local_model = tf.keras.models.load_model(MODEL_DIR)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "evaluate_custom_model:image"
      },
      "source": [
        "## Evaluate the model\n",
        "\n",
        "Now, find out how good the model performs. Prepare some test data for evaluation and run the `evaluate()` method.\n",
        "\n",
        "### Load evaluation data\n",
        "\n",
        "Load the CIFAR10 test (holdout) data from `tf.keras.datasets`, using the `load_data()` method. This returns the dataset as a tuple of two elements. The first element is the training data and the second one is the test data. Each element is also a tuple of two elements: the image data, and the corresponding labels.\n",
        "\n",
        "For this step, you don't need the training data. Hence, you skip it by loading as `(_, _)`.\n",
        "\n",
        "Before you can run the data through evaluation, you need to preprocess it:\n",
        "\n",
        "`x_test`: Normalize (rescale) the pixel data by dividing each pixel by 255. This replaces each single byte integer pixel with a 32-bit floating point number between 0 and 1.\n",
        "\n",
        "`y_test`: The labels are currently scalar (sparse). If you look back at the `compile()` step in the `trainer/task.py` script, you find that it was compiled for sparse labels. So you don't need to do anything more."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "evaluate_custom_model:image,cifar10"
      },
      "outputs": [],
      "source": [
        "(_, _), (x_test, y_test) = cifar10.load_data()\n",
        "x_test = (x_test / 255.0).astype(np.float32)\n",
        "\n",
        "print(x_test.shape, y_test.shape)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "perform_evaluation_custom"
      },
      "source": [
        "### Perform the model evaluation\n",
        "\n",
        "Now evaluate how well the model in the custom job did."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "perform_evaluation_custom"
      },
      "outputs": [],
      "source": [
        "local_model.evaluate(x_test, y_test)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "serving_function_image:xai"
      },
      "source": [
        "### Serving function for image data\n",
        "\n",
        "To pass images to the prediction service, you encode the compressed (e.g., JPEG) image bytes into base 64 -- which makes the content safe from modification while transmitting binary data over the network. Since this deployed model expects input data as raw (uncompressed) bytes, you need to ensure that the base 64 encoded data gets converted back to raw bytes before it's passed as input to the deployed model.\n",
        "\n",
        "To resolve this, define a serving function (`serving_fn`) and attach it to the model as a preprocessing step. Add a `@tf.function` decorator so the serving function is fused to the underlying model (instead of upstream on a CPU).\n",
        "\n",
        "When you send a prediction or explanation request, the content of the request is base 64 decoded into a Tensorflow string (`tf.string`), which is passed to the serving function (`serving_fn`). The serving function preprocesses the `tf.string` into raw (uncompressed) numpy bytes (`preprocess_fn`) to match the input requirements of the model:\n",
        "- `io.decode_jpeg`- Decompresses the JPG image which is returned as a Tensorflow tensor with three channels (RGB).\n",
        "- `image.convert_image_dtype` - Changes integer pixel values to float 32.\n",
        "- `image.resize` - Resizes the image to match the input shape for the model.\n",
        "- `resized / 255.0` - Rescales (normalization) the pixel data between 0 and 1.\n",
        "\n",
        "At this point, the data can be passed to the model (`m_call`).\n",
        "\n",
        "#### XAI Signatures\n",
        "\n",
        "When the serving function is saved back with the underlying model (`tf.saved_model.save`), you specify the input layer of the serving function as the signature `serving_default`.\n",
        "\n",
        "For XAI image models, you need to save two additional signatures from the serving function:\n",
        "\n",
        "- `xai_preprocess`: The preprocessing function in the serving function.\n",
        "- `xai_model`: The concrete function for calling the model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "serving_function_image:xai"
      },
      "outputs": [],
      "source": [
        "CONCRETE_INPUT = \"numpy_inputs\"\n",
        "\n",
        "\n",
        "def _preprocess(bytes_input):\n",
        "    decoded = tf.io.decode_jpeg(bytes_input, channels=3)\n",
        "    decoded = tf.image.convert_image_dtype(decoded, tf.float32)\n",
        "    resized = tf.image.resize(decoded, size=(32, 32))\n",
        "    rescale = tf.cast(resized / 255.0, tf.float32)\n",
        "    return rescale\n",
        "\n",
        "\n",
        "@tf.function(input_signature=[tf.TensorSpec([None], tf.string)])\n",
        "def preprocess_fn(bytes_inputs):\n",
        "    decoded_images = tf.map_fn(\n",
        "        _preprocess, bytes_inputs, dtype=tf.float32, back_prop=False\n",
        "    )\n",
        "    return {\n",
        "        CONCRETE_INPUT: decoded_images\n",
        "    }  # User needs to make sure the key matches model's input\n",
        "\n",
        "\n",
        "@tf.function(input_signature=[tf.TensorSpec([None], tf.string)])\n",
        "def serving_fn(bytes_inputs):\n",
        "    images = preprocess_fn(bytes_inputs)\n",
        "    prob = m_call(**images)\n",
        "    return prob\n",
        "\n",
        "\n",
        "m_call = tf.function(local_model.call).get_concrete_function(\n",
        "    [tf.TensorSpec(shape=[None, 32, 32, 3], dtype=tf.float32, name=CONCRETE_INPUT)]\n",
        ")\n",
        "\n",
        "tf.saved_model.save(\n",
        "    local_model,\n",
        "    model_path_to_deploy,\n",
        "    signatures={\n",
        "        \"serving_default\": serving_fn,\n",
        "        # Required for XAI\n",
        "        \"xai_preprocess\": preprocess_fn,\n",
        "        \"xai_model\": m_call,\n",
        "    },\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "serving_function_signature:xai"
      },
      "source": [
        "## Get the serving function signature\n",
        "\n",
        "You can get the signatures of your model's input and output layers by reloading the model into memory, and querying it for the signatures corresponding to each layer.\n",
        "\n",
        "When making a prediction request, you need to route the request to the serving function instead of the model. So, you need to know the input layer's name for the serving function which you use later when you make a prediction request.\n",
        "\n",
        "You also need to know the name of the serving function's input and output layer for constructing the explanation metadata which is described subsequently."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "serving_function_signature:xai"
      },
      "outputs": [],
      "source": [
        "loaded = tf.saved_model.load(model_path_to_deploy)\n",
        "\n",
        "serving_input = list(\n",
        "    loaded.signatures[\"serving_default\"].structured_input_signature[1].keys()\n",
        ")[0]\n",
        "print(\"Serving function input:\", serving_input)\n",
        "serving_output = list(loaded.signatures[\"serving_default\"].structured_outputs.keys())[0]\n",
        "print(\"Serving function output:\", serving_output)\n",
        "\n",
        "input_name = local_model.input.name\n",
        "print(\"Model input name:\", input_name)\n",
        "output_name = local_model.output.name\n",
        "print(\"Model output name:\", output_name)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "explanation_spec"
      },
      "source": [
        "### Explanation Specification\n",
        "\n",
        "To get explanations along with the predictions, you must enable the explanations feature and set corresponding settings when you upload your custom model to a Vertex AI model resource. These settings are referred to as the explanation metadata, which consists of:\n",
        "\n",
        "- `parameters`: This is the specification for the explainability algorithm to use for explanations on your model. You can choose between:\n",
        "  - Shapley(Note: not recommended for image data since it can involve a long-running operation)\n",
        "  - XRAI\n",
        "  - Integrated Gradients\n",
        "- `metadata`: This is the specification for how the algoithm is applied on your custom model.\n",
        "\n",
        "#### Explanation Parameters\n",
        "\n",
        "Take a look at a detailed overview of the settings for the explainability algorithm.\n",
        "\n",
        "#### Shapley\n",
        "\n",
        "Assigns credit for the outcome to each feature, and considers different permutations of the features. This method provides a sampling approximation of exact Shapley values.\n",
        "\n",
        "Use Cases:\n",
        "  - Classification and regression on tabular data.\n",
        "\n",
        "Parameters:\n",
        "\n",
        "- `path_count`: This is the number of paths over the features processed by the algorithm. An exact approximation of the Shapley values requires M! paths, where M is the number of features. For the CIFAR10 dataset, this would be 784 (28*28).\n",
        "\n",
        "For any non-trival number of features, this is too compute expensive. You can reduce the number of paths over the features to M * `path_count`.\n",
        "\n",
        "#### Integrated Gradients\n",
        "\n",
        "A gradients-based method to efficiently compute feature attributions with the same axiomatic properties as the Shapley value.\n",
        "\n",
        "Use Cases:\n",
        "  - Classification and regression on tabular data.\n",
        "  - Classification on image data.\n",
        "\n",
        "Parameters:\n",
        "\n",
        "- `step_count`: This is the number of steps to approximate the remaining sum. The more steps, the more accurate the integral approximation. The general rule of thumb is 50 steps, but as you increase so does the compute time.\n",
        "\n",
        "#### XRAI\n",
        "\n",
        "Based on the integrated gradients method, XRAI assesses overlapping regions of the image to create a saliency map, which highlights relevant regions of the image rather than pixels.\n",
        "\n",
        "Use Cases:\n",
        "\n",
        "  - Classification on image data.\n",
        "\n",
        "Parameters:\n",
        "\n",
        "- `step_count`: This is the number of steps to approximate the remaining sum. The more steps, the more accurate the integral approximation. The general rule of thumb is 50 steps, but as you increase so does the compute time.\n",
        "\n",
        "In the next code cell, set the variable `XAI` to which explainabilty algorithm you use on your custom model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "explanation_parameters:mbsdk"
      },
      "outputs": [],
      "source": [
        "XAI = \"ig\"  # [ shapley, ig, xrai ]\n",
        "\n",
        "if XAI == \"shapley\":\n",
        "    PARAMETERS = {\"sampled_shapley_attribution\": {\"path_count\": 10}}\n",
        "elif XAI == \"ig\":\n",
        "    PARAMETERS = {\"integrated_gradients_attribution\": {\"step_count\": 50}}\n",
        "elif XAI == \"xrai\":\n",
        "    PARAMETERS = {\"xrai_attribution\": {\"step_count\": 50}}\n",
        "\n",
        "parameters = aiplatform.explain.ExplanationParameters(PARAMETERS)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "explanation_metadata:image"
      },
      "source": [
        "#### Explanation Metadata\n",
        "\n",
        "Now, let's dive deeper into the explanation metadata, which consists of:\n",
        "\n",
        "- `outputs`: A scalar value in the output to attribute -- what to explain. For example, in a probability output \\[0.1, 0.2, 0.7\\] for classification, one wants an explanation for 0.7. Consider the following formula, where the output is `y` and that is what we want to explain.\n",
        "\n",
        "    y = f(x)\n",
        "\n",
        "Consider the following formulae, where the outputs are `y` and `z`. Since we can only do attribution for one scalar value, we have to pick whether we want to explain the output `y` or `z`. Assume in this example the model is object detection and y and z are the bounding box and the object classification. You would want to pick which of the two outputs to explain.\n",
        "\n",
        "    y, z = f(x)\n",
        "\n",
        "The dictionary format for `outputs` is:\n",
        "\n",
        "    { \"outputs\": { \"[your_display_name]\":\n",
        "                   \"output_tensor_name\": [layer]\n",
        "                 }\n",
        "    }\n",
        "\n",
        "<blockquote>\n",
        " -  [your_display_name]: A human readable name you assign to the output to explain. A common example is \"probability\".<br/>\n",
        " -  \"output_tensor_name\": The key/value field to identify the output layer to explain. <br/>\n",
        " -  [layer]: The output layer to explain. In a single task model, like a tabular regressor, it's the last (topmost) layer in the model.\n",
        "</blockquote>\n",
        "\n",
        "- `inputs`: The features for attribution -- how they contributed to the output. Consider the following formulae, where `a` and `b` are the features. We have to pick which features to explain how the contributed. Assume that this model is deployed for A/B testing, where `a` are the data_items for the prediction and `b` identifies whether the model instance is A or B. You would want to pick `a` (or some subset of) for the features, and not `b` since it does not contribute to the prediction.\n",
        "\n",
        "    y = f(a,b)\n",
        "\n",
        "The minimum dictionary format for `inputs` is:\n",
        "\n",
        "    { \"inputs\": { \"[your_display_name]\":\n",
        "                  \"input_tensor_name\": [layer]\n",
        "                 }\n",
        "    }\n",
        "\n",
        "<blockquote>\n",
        " -  [your_display_name]: A human readable name you assign to the input to explain. A common example is \"features\".<br/>\n",
        " -  \"input_tensor_name\": The key/value field to identify the input layer for the feature attribution. <br/>\n",
        " -  [layer]: The input layer for feature attribution. In a single input tensor model, it's the first (bottom-most) layer in the model.\n",
        "</blockquote>\n",
        "\n",
        "Since the inputs to the model are tabular, you can specify the following two additional fields as reporting/visualization aids:\n",
        "\n",
        "<blockquote>\n",
        " - \"modality\": \"image\": Indicates the field values are image data.\n",
        "</blockquote>\n",
        "\n",
        "Since the inputs to the model are images, you can specify the following additional fields as reporting/visualization aids:\n",
        "\n",
        "<blockquote>\n",
        " - \"modality\": \"image\": Indicates the field values are image data.\n",
        "</blockquote>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "explanation_metadata:mbsdk,image"
      },
      "outputs": [],
      "source": [
        "random_baseline = np.random.rand(32, 32, 3)\n",
        "input_baselines = [{\"number_vaue\": x} for x in random_baseline]\n",
        "\n",
        "INPUT_METADATA = {\"input_tensor_name\": CONCRETE_INPUT, \"modality\": \"image\"}\n",
        "\n",
        "OUTPUT_METADATA = {\"output_tensor_name\": serving_output}\n",
        "\n",
        "input_metadata = aiplatform.explain.ExplanationMetadata.InputMetadata(INPUT_METADATA)\n",
        "output_metadata = aiplatform.explain.ExplanationMetadata.OutputMetadata(OUTPUT_METADATA)\n",
        "\n",
        "metadata = aiplatform.explain.ExplanationMetadata(\n",
        "    inputs={\"image\": input_metadata}, outputs={\"class\": output_metadata}\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "upload_model:mbsdk,xai"
      },
      "source": [
        "## Upload the model\n",
        "\n",
        "Next, upload your model to Vertex AI Model Registry using `Model.upload()` method, with the following parameters:\n",
        "\n",
        "- `display_name`: The human readable name for the model resource.\n",
        "- `artifact`: The Cloud Storage location of the trained model artifacts.\n",
        "- `serving_container_image_uri`: The serving container image.\n",
        "- `sync`: Whether to execute the upload asynchronously or synchronously.\n",
        "- `explanation_parameters`: Parameters to configure explaining for model's predictions.\n",
        "- `explanation_metadata`: Metadata describing the model's input and output for explanation.\n",
        "\n",
        "If the `upload()` method is run asynchronously, you can subsequently block until completion with the `wait()` method."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "upload_model:mbsdk,xai"
      },
      "outputs": [],
      "source": [
        "model = aiplatform.Model.upload(\n",
        "    display_name=\"cifar10\",\n",
        "    artifact_uri=MODEL_DIR,\n",
        "    serving_container_image_uri=DEPLOY_IMAGE,\n",
        "    explanation_parameters=parameters,\n",
        "    explanation_metadata=metadata,\n",
        "    sync=False,\n",
        ")\n",
        "\n",
        "model.wait()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "deploy_model:mbsdk,all"
      },
      "source": [
        "## Deploy the model\n",
        "\n",
        "Next, deploy your model for online predictions. To deploy the model, you invoke the `deploy()` method, with the following parameters:\n",
        "\n",
        "- `deployed_model_display_name`: A human readable name for the deployed model.\n",
        "- `traffic_split`: Percent of traffic at the endpoint that goes to this model, which is specified as a dictionary of one or more key/value pairs.\n",
        "If only one model, then specify as { \"0\": 100 }, where \"0\" refers to this model being uploaded and 100 means 100% of the traffic.\n",
        "If there are existing models on the endpoint, for which the traffic is split, then use model_id to specify as { \"0\": percent, model_id: percent, ... }, where model_id is the model id of an existing model to the deployed endpoint. The percents must add up to 100.\n",
        "- `machine_type`: The type of machine to use for training.\n",
        "- `accelerator_type`: The hardware accelerator type.\n",
        "- `accelerator_count`: The number of accelerators to attach to a worker replica.\n",
        "- `starting_replica_count`: The number of compute instances to initially provision.\n",
        "- `max_replica_count`: The maximum number of compute instances to scale to. In this tutorial, only one instance is provisioned."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "deploy_model:mbsdk,all"
      },
      "outputs": [],
      "source": [
        "DEPLOYED_NAME = \"cifar10\"\n",
        "\n",
        "TRAFFIC_SPLIT = {\"0\": 100}\n",
        "\n",
        "MIN_NODES = 1\n",
        "MAX_NODES = 1\n",
        "\n",
        "if DEPLOY_GPU:\n",
        "    endpoint = model.deploy(\n",
        "        deployed_model_display_name=DEPLOYED_NAME,\n",
        "        traffic_split=TRAFFIC_SPLIT,\n",
        "        machine_type=DEPLOY_COMPUTE,\n",
        "        accelerator_type=DEPLOY_GPU,\n",
        "        accelerator_count=DEPLOY_NGPU,\n",
        "        min_replica_count=MIN_NODES,\n",
        "        max_replica_count=MAX_NODES,\n",
        "    )\n",
        "else:\n",
        "    endpoint = model.deploy(\n",
        "        deployed_model_display_name=DEPLOYED_NAME,\n",
        "        traffic_split=TRAFFIC_SPLIT,\n",
        "        machine_type=DEPLOY_COMPUTE,\n",
        "        accelerator_type=DEPLOY_GPU,\n",
        "        accelerator_count=0,\n",
        "        min_replica_count=MIN_NODES,\n",
        "        max_replica_count=MAX_NODES,\n",
        "    )"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "get_test_item:test"
      },
      "source": [
        "### Get test item\n",
        "\n",
        "Use an example from the test (holdout) portion of the dataset as a test item."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "get_test_item:test"
      },
      "outputs": [],
      "source": [
        "test_image = x_test[0]\n",
        "test_label = y_test[0]\n",
        "print(test_image.shape)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "prepare_test_item:test,image"
      },
      "source": [
        "### Prepare the request content\n",
        "You are going to send the CIFAR10 image as compressed JPG image, instead of the raw uncompressed bytes:\n",
        "\n",
        "- `cv2.imwrite`: Use openCV to write the uncompressed image to disk as a compressed JPEG image.\n",
        " - Denormalize the image data from \\[0,1) range back to [0,255).\n",
        " - Convert the 32-bit floating point values to 8-bit unsigned integers.\n",
        "- `tf.io.read_file`: Read the compressed JPG images back into memory as raw bytes.\n",
        "- `base64.b64encode`: Encode the raw bytes into a base 64 encoded string."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "prepare_test_item:test,image"
      },
      "outputs": [],
      "source": [
        "cv2.imwrite(\"tmp.jpg\", (test_image * 255).astype(np.uint8))\n",
        "\n",
        "bytes = tf.io.read_file(\"tmp.jpg\")\n",
        "b64str = base64.b64encode(bytes.numpy()).decode(\"utf-8\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "explain_request:mbsdk,custom,icn"
      },
      "source": [
        "### Make the prediction with explanation\n",
        "\n",
        "Now that your model resource is deployed to an edpoint resource, you can get online explanations by sending prediction requests to your endpoint.\n",
        "\n",
        "#### Request\n",
        "\n",
        "The format of each instance is:\n",
        "\n",
        "    [{serving_input: {'b64': bytes}]\n",
        "\n",
        "Since the `explain()` method can take multiple items (instances), send your single test item as a list of one test item.\n",
        "\n",
        "#### Response\n",
        "\n",
        "The response from the `explain()` call is a Python dictionary with the following entries:\n",
        "\n",
        "- `ids`: The internal assigned unique identifiers for each prediction request.\n",
        "- `predictions`: The prediction per instance.\n",
        "- `deployed_model_id`: The Vertex AI identifier for the deployed model resource.\n",
        "- `explanations`: The feature attributions returned by the Explainable AI."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "explain_request:mbsdk,custom,icn"
      },
      "outputs": [],
      "source": [
        "instances_list = [{serving_input: {\"b64\": b64str}}]\n",
        "\n",
        "response = endpoint.explain(instances_list)\n",
        "print(response)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "understanding_explanations:cifar10"
      },
      "source": [
        "### Visualize the predictions\n",
        "\n",
        "Preview the images and their predicted classes without the explanations."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "understanding_explanations:cifar10"
      },
      "outputs": [],
      "source": [
        "CLASSES = [\n",
        "    \"airplane\",\n",
        "    \"automobile\",\n",
        "    \"bird\",\n",
        "    \"cat\",\n",
        "    \"deer\",\n",
        "    \"dog\",\n",
        "    \"frog\",\n",
        "    \"horse\",\n",
        "    \"ship\",\n",
        "    \"truck\",\n",
        "]\n",
        "\n",
        "# Note: change the `ig_response` variable below if you didn't deploy an IG model\n",
        "for prediction in response.predictions:\n",
        "    label_index = np.argmax(prediction)\n",
        "    class_name = CLASSES[label_index]\n",
        "    confidence_score = prediction[label_index]\n",
        "    print(\n",
        "        \"Predicted class: \"\n",
        "        + class_name\n",
        "        + \"\\n\"\n",
        "        + \"Confidence score: \"\n",
        "        + str(confidence_score)\n",
        "    )\n",
        "\n",
        "    image = base64.b64decode(b64str)\n",
        "    image = BytesIO(image)\n",
        "    img = mpimg.imread(image, format=\"JPG\")\n",
        "\n",
        "    plt.imshow(img, interpolation=\"nearest\")\n",
        "    plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "visualize_image_explanations"
      },
      "source": [
        "### Visualize the images with AI Explanations\n",
        "\n",
        "The images returned show the explanations for only the top class predicted by the model. This means that if one of the model's predictions is incorrect, the pixels you see highlighted are for the *incorrect class*. For example, if the model predicted \"airplane\" when it should have predicted \"cat\", you can see explanations for why the model classified this image as an airplane.\n",
        "\n",
        "If you deployed an Integrated Gradients model, you can visualize its feature attributions. Currently, the highlighted pixels returned from AI Explanations show the top 60% of pixels that contributed to the model's prediction. The pixels you see after running the cell below show the pixels that most signaled the model's prediction."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "visualize_image_explanations"
      },
      "outputs": [],
      "source": [
        "for explanation in response.explanations:\n",
        "    attributions = dict(explanation.attributions[0].feature_attributions)\n",
        "    label_index = explanation.attributions[0].output_index[0]\n",
        "    class_name = CLASSES[label_index]\n",
        "    b64str = attributions[\"image\"][\"b64_jpeg\"]\n",
        "    image = base64.b64decode(b64str)\n",
        "    image = io.BytesIO(image)\n",
        "    img = mpimg.imread(image, format=\"JPG\")\n",
        "\n",
        "    plt.imshow(img, interpolation=\"nearest\")\n",
        "    plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "undeploy_model:mbsdk"
      },
      "source": [
        "## Undeploy the model\n",
        "\n",
        "When you are done with the model predictions, undeploy the model from the endpoint resouce. This deprovisions all compute resources and ends billing for the deployed model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "undeploy_model:mbsdk"
      },
      "outputs": [],
      "source": [
        "endpoint.undeploy_all()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cleanup:mbsdk"
      },
      "source": [
        "## Cleaning up\n",
        "\n",
        "To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n",
        "project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n",
        "\n",
        "Otherwise, you can delete the individual resources you created in this tutorial."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cleanup:mbsdk"
      },
      "outputs": [],
      "source": [
        "# Delete the training job\n",
        "job.delete()\n",
        "\n",
        "# Delete the endpoint\n",
        "endpoint.delete()\n",
        "\n",
        "# Delete the model\n",
        "model.delete()\n",
        "\n",
        "# Delete the Cloud Storage bucket\n",
        "delete_bucket = False  # Set True for deletion\n",
        "if delete_bucket:\n",
        "    ! gsutil rm -r $BUCKET_URI\n",
        "\n",
        "# Delete locally generated files\n",
        "! rm -rf custom/\n",
        "! rm custom.tar.gz\n",
        "! rm tmp.jpg"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "sdk_custom_image_classification_online_explain.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}