notebooks/official/migration/sdk-automl-image-object-detection-batch-online.ipynb

{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "copyright"
      },
      "outputs": [],
      "source": [
        "# Copyright 2021 Google LLC\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "title:migration,new"
      },
      "source": [
        "# Vertex AI migration: AutoML image object detection\n",
        "\n",
        "<table align=\"left\">\n",
        "\n",
        "  <td>\n",
        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-automl-image-object-detection-batch-online.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-automl-image-object-detection-batch-online.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
        "      View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
        "<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/migration/sdk-automl-image-object-detection-batch-online.ipynb\" target='_blank'>\n",
        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
        "      Open in Vertex AI Workbench\n",
        "    </a>\n",
        "  </td>                                                                                               \n",
        "</table>\n",
        "\n",
        "<br/><br/><br/>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7a8a13b86a8b"
      },
      "source": [
        "## Overview\n",
        "\n",
        "This tutorial demonstrates how to use the Vertex AI SDK for Python to train and deploy an AutoML object detection model.\n",
        "\n",
        "Learn more about [Migrate to Vertex AI](https://cloud.google.com/vertex-ai/docs/start/migrating-to-vertex-ai) and [Object detection for image data](https://cloud.google.com/vertex-ai/docs/training-overview#object_detection_for_images)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "618cfedf829a"
      },
      "source": [
        "### Objective\n",
        "\n",
        "In this tutorial, you learn to use `AutoML` to train an image model and use `Vertex AI Prediction` and `Vertex AI Batch Prediction` to do online and batch predictions.\n",
        "\n",
        "\n",
        "This tutorial uses the following Google Cloud ML services:\n",
        "\n",
        "- `AutoML`\n",
        "- `Vertex AI Batch Prediction`\n",
        "- `Vertex AI Model` resource\n",
        "- `Vertex AI Endpoint` resource\n",
        "\n",
        "The steps performed include:\n",
        "\n",
        "- Train an AutoML object detection model.\n",
        "- Make a batch prediction.\n",
        "- Deploy model to a endpoint\n",
        "- Make a online prediction"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dataset:salads,iod"
      },
      "source": [
        "### Dataset\n",
        "\n",
        "The dataset used for this tutorial is the Salads category of the [OpenImages dataset](https://www.tensorflow.org/datasets/catalog/open_images_v4) from [TensorFlow Datasets](https://www.tensorflow.org/datasets/catalog/overview). This dataset does not require any feature engineering. The version of the dataset you will use in this tutorial is stored in a public Cloud Storage bucket. The trained model predicts the bounding box locations and corresponding type of salad items in an image from a class of five items: salad, seafood, tomato, baked goods, or cheese."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "costs"
      },
      "source": [
        "### Costs\n",
        "\n",
        "This tutorial uses billable components of Google Cloud:\n",
        "\n",
        "* Vertex AI\n",
        "* Cloud Storage\n",
        "\n",
        "Learn about [Vertex AI\n",
        "pricing](https://cloud.google.com/vertex-ai/pricing) and [Cloud Storage\n",
        "pricing](https://cloud.google.com/storage/pricing), and use the [Pricing\n",
        "Calculator](https://cloud.google.com/products/calculator/)\n",
        "to generate a cost estimate based on your projected usage."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "install_aip:mbsdk"
      },
      "source": [
        "## Installation\n",
        "\n",
        "Install the packages required for executing this notebook."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "install_aip:mbsdk"
      },
      "outputs": [],
      "source": [
        "! pip3 install --upgrade --quiet google-cloud-aiplatform \\\n",
        "                                 google-cloud-storage \\\n",
        "                                 tensorflow"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "restart"
      },
      "source": [
        "### Colab only: Uncomment the following cell to restart the kernel"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "D-ZBOjErv5mM"
      },
      "outputs": [],
      "source": [
        "# Automatically restart kernel after installs so that your environment can access the new packages\n",
        "# import IPython\n",
        "\n",
        "# app = IPython.Application.instance()\n",
        "# app.kernel.do_shutdown(True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "before_you_begin:nogpu"
      },
      "source": [
        "## Before you begin"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "before_you_begin:nogpu"
      },
      "source": [
        "### Set your project ID\n",
        "\n",
        "**If you don't know your project ID**, try the following:\n",
        "* Run `gcloud config list`.\n",
        "* Run `gcloud projects list`.\n",
        "* See the support page: [Locate the project ID](https://support.google.com/googleapi/answer/7014113)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "set_project_id"
      },
      "outputs": [],
      "source": [
        "PROJECT_ID = \"[your-project-id]\"  # @param {type:\"string\"}\n",
        "\n",
        "# Set the project id\n",
        "! gcloud config set project {PROJECT_ID}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "region"
      },
      "source": [
        "#### Region\n",
        "\n",
        "You can also change the `REGION` variable used by Vertex AI. Learn more about [Vertex AI regions](https://cloud.google.com/vertex-ai/docs/general/locations)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "region"
      },
      "outputs": [],
      "source": [
        "REGION = \"us-central1\"  # @param {type: \"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "timestamp"
      },
      "source": [
        "#### UUID\n",
        "\n",
        "If you are in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a uuid for each instance session, and append it onto the name of resources you create in this tutorial."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "timestamp"
      },
      "outputs": [],
      "source": [
        "import random\n",
        "import string\n",
        "\n",
        "\n",
        "# Generate a uuid of a specifed length(default=8)\n",
        "def generate_uuid(length: int = 8) -> str:\n",
        "    return \"\".join(random.choices(string.ascii_lowercase + string.digits, k=length))\n",
        "\n",
        "\n",
        "UUID = generate_uuid()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gcp_authenticate"
      },
      "source": [
        "### Authenticate your Google Cloud account\n",
        "\n",
        "Depending on your Jupyter environment, you may have to manually authenticate. Follow the relevant instructions below.\n",
        "\n",
        "**1. Vertex AI Workbench**\n",
        "* Do nothing as you are already authenticated.\n",
        "\n",
        "**2. Local JupyterLab instance, uncomment and run:**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ce6043da7b33"
      },
      "outputs": [],
      "source": [
        "# ! gcloud auth login"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0367eac06a10"
      },
      "source": [
        "**3. Colab, uncomment and run:**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "21ad4dbb4a61"
      },
      "outputs": [],
      "source": [
        "# from google.colab import auth\n",
        "# auth.authenticate_user()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c13224697bfb"
      },
      "source": [
        "**4. Service account or other**\n",
        "* See how to grant Cloud Storage permissions to your service account at https://cloud.google.com/storage/docs/gsutil/commands/iam#ch-examples."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bucket:mbsdk"
      },
      "source": [
        "### Create a Cloud Storage bucket\n",
        "\n",
        "Create a storage bucket to store intermediate artifacts such as datasets."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bucket"
      },
      "outputs": [],
      "source": [
        "BUCKET_URI = f\"gs://your-bucket-name-{PROJECT_ID}-unique\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_bucket"
      },
      "source": [
        "**Only if your bucket doesn't already exist**: Run the following cell to create your Cloud Storage bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "create_bucket"
      },
      "outputs": [],
      "source": [
        "! gsutil mb -l {REGION} {BUCKET_URI}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "setup_vars"
      },
      "source": [
        "### Set up variables\n",
        "\n",
        "Next, set up some variables used throughout the tutorial.\n",
        "### Import libraries and define constants"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "import_aip:mbsdk"
      },
      "outputs": [],
      "source": [
        "import google.cloud.aiplatform as aip"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "init_aip:mbsdk"
      },
      "source": [
        "## Initialize Vertex AI SDK for Python\n",
        "\n",
        "Initialize the Vertex AI SDK for Python for your project and corresponding bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "init_aip:mbsdk"
      },
      "outputs": [],
      "source": [
        "aip.init(project=PROJECT_ID, staging_bucket=BUCKET_URI)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "import_file:u_dataset,csv"
      },
      "source": [
        "#### Location of Cloud Storage training data.\n",
        "\n",
        "Now set the variable `IMPORT_FILE` to the location of the CSV index file in Cloud Storage."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "import_file:salads,csv,iod"
      },
      "outputs": [],
      "source": [
        "IMPORT_FILE = \"gs://cloud-samples-data/vision/salads.csv\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "quick_peek:csv"
      },
      "source": [
        "#### Quick peek at your data\n",
        "\n",
        "This tutorial uses a version of the Salads dataset that is stored in a public Cloud Storage bucket, using a CSV index file.\n",
        "\n",
        "Start by doing a quick peek at the data. You count the number of examples by counting the number of rows in the CSV index file  (`wc -l`) and then peek at the first few rows."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "quick_peek:csv"
      },
      "outputs": [],
      "source": [
        "if \"IMPORT_FILES\" in globals():\n",
        "    FILE = IMPORT_FILES[0]\n",
        "else:\n",
        "    FILE = IMPORT_FILE\n",
        "\n",
        "count = ! gsutil cat $FILE | wc -l\n",
        "print(\"Number of Examples\", int(count[0]))\n",
        "\n",
        "print(\"First 10 rows\")\n",
        "! gsutil cat $FILE | head"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_a_dataset:migration"
      },
      "source": [
        "## Create a dataset"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "datasets_create:migration,new,mbsdk"
      },
      "source": [
        "### [datasets.create-dataset-api](https://cloud.google.com/vertex-ai/docs/datasets/create-dataset-api)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_dataset:image,iod"
      },
      "source": [
        "### Create the Dataset\n",
        "\n",
        "Next, create the `Dataset` resource using the `create` method for the `ImageDataset` class, which takes the following parameters:\n",
        "\n",
        "- `display_name`: The human readable name for the `Dataset` resource.\n",
        "- `gcs_source`: A list of one or more dataset index files to import the data items into the `Dataset` resource.\n",
        "- `import_schema_uri`: The data labeling schema for the data items.\n",
        "\n",
        "This operation may take several minutes."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "create_dataset:image,iod"
      },
      "outputs": [],
      "source": [
        "dataset = aip.ImageDataset.create(\n",
        "    display_name=\"Salads\" + \"_\" + UUID,\n",
        "    gcs_source=[IMPORT_FILE],\n",
        "    import_schema_uri=aip.schema.dataset.ioformat.image.bounding_box,\n",
        ")\n",
        "\n",
        "print(dataset.resource_name)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_dataset:image,iod"
      },
      "source": [
        "*Example Output:*\n",
        "\n",
        "    INFO:google.cloud.aiplatform.datasets.dataset:Creating ImageDataset\n",
        "    INFO:google.cloud.aiplatform.datasets.dataset:Create ImageDataset backing LRO: projects/759209241365/locations/us-central1/datasets/2940964905882222592/operations/1941426647739662336\n",
        "    INFO:google.cloud.aiplatform.datasets.dataset:ImageDataset created. Resource name: projects/759209241365/locations/us-central1/datasets/2940964905882222592\n",
        "    INFO:google.cloud.aiplatform.datasets.dataset:To use this ImageDataset in another session:\n",
        "    INFO:google.cloud.aiplatform.datasets.dataset:ds = aiplatform.ImageDataset('projects/759209241365/locations/us-central1/datasets/2940964905882222592')\n",
        "    INFO:google.cloud.aiplatform.datasets.dataset:Importing ImageDataset data: projects/759209241365/locations/us-central1/datasets/2940964905882222592\n",
        "    INFO:google.cloud.aiplatform.datasets.dataset:Import ImageDataset data backing LRO: projects/759209241365/locations/us-central1/datasets/2940964905882222592/operations/8100099138168815616\n",
        "    INFO:google.cloud.aiplatform.datasets.dataset:ImageDataset data imported. Resource name: projects/759209241365/locations/us-central1/datasets/2940964905882222592\n",
        "    projects/759209241365/locations/us-central1/datasets/2940964905882222592"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "train_a_model:migration"
      },
      "source": [
        "## Train a model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "trainingpipelines_create:migration,new,mbsdk"
      },
      "source": [
        "### [training.automl-api](https://cloud.google.com/vertex-ai/docs/training/automl-api)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_automl_pipeline:image,iod"
      },
      "source": [
        "### Create and run training pipeline\n",
        "\n",
        "To train an AutoML model, you perform two steps: 1) create a training pipeline, and 2) run the pipeline.\n",
        "\n",
        "#### Create training pipeline\n",
        "\n",
        "An AutoML training pipeline is created with the `AutoMLImageTrainingJob` class, with the following parameters:\n",
        "\n",
        "- `display_name`: The human readable name for the `TrainingJob` resource.\n",
        "- `prediction_type`: The type task to train the model for.\n",
        "  - `classification`: An image classification model.\n",
        "  - `object_detection`: An image object detection model.\n",
        "- `multi_label`: If a classification task, whether single (`False`) or multi-labeled (`True`).\n",
        "- `model_type`: The type of model for deployment.\n",
        "  - `CLOUD`: Deployment on Google Cloud\n",
        "  - `CLOUD_HIGH_ACCURACY_1`: Optimized for accuracy over latency for deployment on Google Cloud.\n",
        "  - `CLOUD_LOW_LATENCY_`: Optimized for latency over accuracy for deployment on Google Cloud.\n",
        "  - `MOBILE_TF_VERSATILE_1`: Deployment on an edge device.\n",
        "  - `MOBILE_TF_HIGH_ACCURACY_1`:Optimized for accuracy over latency for deployment on an edge device.\n",
        "  - `MOBILE_TF_LOW_LATENCY_1`: Optimized for latency over accuracy for deployment on an edge device.\n",
        "- `base_model`: (optional) Transfer learning from existing `Model` resource -- supported for image classification only.\n",
        "\n",
        "The instantiated object is the DAG (directed acyclic graph) for the training job."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "create_automl_pipeline:image,iod"
      },
      "outputs": [],
      "source": [
        "dag = aip.AutoMLImageTrainingJob(\n",
        "    display_name=\"salads_\" + UUID,\n",
        "    prediction_type=\"object_detection\",\n",
        "    multi_label=False,\n",
        "    model_type=\"CLOUD\",\n",
        "    base_model=None,\n",
        ")\n",
        "\n",
        "print(dag)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_automl_pipeline:image,iod"
      },
      "source": [
        "*Example output:*\n",
        "\n",
        "    <google.cloud.aiplatform.training_jobs.AutoMLImageTrainingJob object at 0x7f806a6116d0>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "run_automl_pipeline:image"
      },
      "source": [
        "#### Run the training pipeline\n",
        "\n",
        "Next, you run the DAG to start the training job by invoking the method `run`, with the following parameters:\n",
        "\n",
        "- `dataset`: The `Dataset` resource to train the model.\n",
        "- `model_display_name`: The human readable name for the trained model.\n",
        "- `training_fraction_split`: The percentage of the dataset to use for training.\n",
        "- `test_fraction_split`: The percentage of the dataset to use for test (holdout data).\n",
        "- `validation_fraction_split`: The percentage of the dataset to use for validation.\n",
        "- `budget_milli_node_hours`: (optional) Maximum training time specified in unit of millihours (1000 = hour).\n",
        "- `disable_early_stopping`: If `True`, training maybe completed before using the entire budget if the service believes it cannot further improve on the model objective measurements.\n",
        "\n",
        "The `run` method when completed returns the `Model` resource.\n",
        "\n",
        "The execution of the training pipeline will take upto 20 minutes."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "run_automl_pipeline:image"
      },
      "outputs": [],
      "source": [
        "model = dag.run(\n",
        "    dataset=dataset,\n",
        "    model_display_name=\"salads_\" + UUID,\n",
        "    training_fraction_split=0.8,\n",
        "    validation_fraction_split=0.1,\n",
        "    test_fraction_split=0.1,\n",
        "    budget_milli_node_hours=20000,\n",
        "    disable_early_stopping=False,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "run_automl_pipeline:image"
      },
      "source": [
        "*Example output:*\n",
        "\n",
        "    INFO:google.cloud.aiplatform.training_jobs:View Training:\n",
        "    https://console.cloud.google.com/ai/platform/locations/us-central1/training/2109316300865011712?project=759209241365\n",
        "    INFO:google.cloud.aiplatform.training_jobs:AutoMLImageTrainingJob projects/759209241365/locations/us-central1/trainingPipelines/2109316300865011712 current state:\n",
        "    PipelineState.PIPELINE_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.training_jobs:AutoMLImageTrainingJob projects/759209241365/locations/us-central1/trainingPipelines/2109316300865011712 current state:\n",
        "    PipelineState.PIPELINE_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.training_jobs:AutoMLImageTrainingJob projects/759209241365/locations/us-central1/trainingPipelines/2109316300865011712 current state:\n",
        "    PipelineState.PIPELINE_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.training_jobs:AutoMLImageTrainingJob projects/759209241365/locations/us-central1/trainingPipelines/2109316300865011712 current state:\n",
        "    PipelineState.PIPELINE_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.training_jobs:AutoMLImageTrainingJob projects/759209241365/locations/us-central1/trainingPipelines/2109316300865011712 current state:\n",
        "    PipelineState.PIPELINE_STATE_RUNNING\n",
        "    ...\n",
        "    INFO:google.cloud.aiplatform.training_jobs:AutoMLImageTrainingJob run completed. Resource name: projects/759209241365/locations/us-central1/trainingPipelines/2109316300865011712\n",
        "    INFO:google.cloud.aiplatform.training_jobs:Model available at projects/759209241365/locations/us-central1/models/1284590221056278528"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "evaluate_the_model:migration"
      },
      "source": [
        "## Evaluate the model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "models_evaluations_list:migration,new"
      },
      "source": [
        "### [projects.locations.models.evaluations.list](https://cloud.devsite.corp.google.com/ai-platform-unified/docs/reference/rest/v1beta1/projects.locations.models.evaluations/list)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "evaluate_the_model:mbsdk"
      },
      "source": [
        "## Review model evaluation scores\n",
        "After your model has finished training, you can review the evaluation scores for it.\n",
        "\n",
        "First, you need to get a reference to the new model. As with datasets, you can either use the reference to the model variable you created when you deployed the model or you can list all of the models in your project."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "evaluate_the_model:mbsdk"
      },
      "outputs": [],
      "source": [
        "# Get model resource ID\n",
        "models = aip.Model.list(filter=\"display_name=salads_\" + UUID)\n",
        "\n",
        "# Get a reference to the Model Service client\n",
        "client_options = {\"api_endpoint\": f\"{REGION}-aiplatform.googleapis.com\"}\n",
        "model_service_client = aip.gapic.ModelServiceClient(client_options=client_options)\n",
        "\n",
        "model_evaluations = model_service_client.list_model_evaluations(\n",
        "    parent=models[0].resource_name\n",
        ")\n",
        "model_evaluation = list(model_evaluations)[0]\n",
        "print(model_evaluation)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "evaluate_the_model:mbsdk"
      },
      "source": [
        "*Example output:*\n",
        "\n",
        "    name: \"projects/759209241365/locations/us-central1/models/623915674158235648/evaluations/4280507618583117824\"\n",
        "    metrics_schema_uri: \"gs://google-cloud-aiplatform/schema/modelevaluation/classification_metrics_1.0.0.yaml\"\n",
        "    metrics {\n",
        "      struct_value {\n",
        "        fields {\n",
        "          key: \"auPrc\"\n",
        "          value {\n",
        "            number_value: 0.9891107\n",
        "          }\n",
        "        }\n",
        "        fields {\n",
        "          key: \"confidenceMetrics\"\n",
        "          value {\n",
        "            list_value {\n",
        "              values {\n",
        "                struct_value {\n",
        "                  fields {\n",
        "                    key: \"precision\"\n",
        "                    value {\n",
        "                      number_value: 0.2\n",
        "                    }\n",
        "                  }\n",
        "                  fields {\n",
        "                    key: \"recall\"\n",
        "                    value {\n",
        "                      number_value: 1.0\n",
        "                    }\n",
        "                  }\n",
        "                }\n",
        "              }"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "make_batch_predictions:migration"
      },
      "source": [
        "## Make batch predictions"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "batchpredictionjobs_create:migration,new,mbsdk"
      },
      "source": [
        "### [predictions.batch-prediction](https://cloud.google.com/vertex-ai/docs/predictions/batch-predictions)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "get_test_items:batch_prediction"
      },
      "source": [
        "### Get test item(s)\n",
        "\n",
        "Now do a batch prediction to your Vertex model. You will use arbitrary examples out of the dataset as a test items. Don't be concerned that the examples were likely used in training the model -- we just want to demonstrate how to make a prediction."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "get_test_items:automl,iod,csv"
      },
      "outputs": [],
      "source": [
        "test_items = !gsutil cat $IMPORT_FILE | head -n2\n",
        "cols_1 = str(test_items[0]).split(\",\")\n",
        "cols_2 = str(test_items[1]).split(\",\")\n",
        "if len(cols_1) == 11:\n",
        "    test_item_1 = str(cols_1[1])\n",
        "    test_label_1 = str(cols_1[2])\n",
        "    test_item_2 = str(cols_2[1])\n",
        "    test_label_2 = str(cols_2[2])\n",
        "else:\n",
        "    test_item_1 = str(cols_1[0])\n",
        "    test_label_1 = str(cols_1[1])\n",
        "    test_item_2 = str(cols_2[0])\n",
        "    test_label_2 = str(cols_2[1])\n",
        "\n",
        "print(test_item_1, test_label_1)\n",
        "print(test_item_2, test_label_2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "copy_test_items:batch_prediction"
      },
      "source": [
        "### Copy test item(s)\n",
        "\n",
        "For the batch prediction, copy the test items over to your Cloud Storage bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "copy_test_items:batch_prediction"
      },
      "outputs": [],
      "source": [
        "file_1 = test_item_1.split(\"/\")[-1]\n",
        "file_2 = test_item_2.split(\"/\")[-1]\n",
        "\n",
        "! gsutil cp $test_item_1 $BUCKET_URI/$file_1\n",
        "! gsutil cp $test_item_2 $BUCKET_URI/$file_2\n",
        "\n",
        "test_item_1 = BUCKET_URI + \"/\" + file_1\n",
        "test_item_2 = BUCKET_URI + \"/\" + file_2"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "make_batch_file:automl,image"
      },
      "source": [
        "### Make the batch input file\n",
        "\n",
        "Now make a batch input file, which you will store in your local Cloud Storage bucket. The batch input file can be either CSV or JSONL. You will use JSONL in this tutorial. For JSONL file, you make one dictionary entry per line for each data item (instance). The dictionary contains the key/value pairs:\n",
        "\n",
        "- `content`: The Cloud Storage path to the image.\n",
        "- `mime_type`: The content type. In our example, it is a `jpeg` file.\n",
        "\n",
        "For example:\n",
        "\n",
        "                        {'content': '[your-bucket]/file1.jpg', 'mime_type': 'jpeg'}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "make_batch_file:automl,image"
      },
      "outputs": [],
      "source": [
        "import json\n",
        "\n",
        "import tensorflow as tf\n",
        "\n",
        "gcs_input_uri = BUCKET_URI + \"/test.jsonl\"\n",
        "with tf.io.gfile.GFile(gcs_input_uri, \"w\") as f:\n",
        "    data = {\"content\": test_item_1, \"mime_type\": \"image/jpeg\"}\n",
        "    f.write(json.dumps(data) + \"\\n\")\n",
        "    data = {\"content\": test_item_2, \"mime_type\": \"image/jpeg\"}\n",
        "    f.write(json.dumps(data) + \"\\n\")\n",
        "\n",
        "print(gcs_input_uri)\n",
        "! gsutil cat $gcs_input_uri"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "batch_request:mbsdk"
      },
      "source": [
        "### Make the batch prediction request\n",
        "\n",
        "Now that your Model resource is trained, you can make a batch prediction by invoking the batch_predict() method, with the following parameters:\n",
        "\n",
        "- `job_display_name`: The human readable name for the batch prediction job.\n",
        "- `gcs_source`: A list of one or more batch request input files.\n",
        "- `gcs_destination_prefix`: The Cloud Storage location for storing the batch prediction resuls.\n",
        "- `sync`: If set to True, the call will block while waiting for the asynchronous batch job to complete."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "batch_request:mbsdk"
      },
      "outputs": [],
      "source": [
        "batch_predict_job = model.batch_predict(\n",
        "    job_display_name=\"salads_\" + UUID,\n",
        "    gcs_source=gcs_input_uri,\n",
        "    gcs_destination_prefix=BUCKET_URI,\n",
        "    sync=False,\n",
        ")\n",
        "\n",
        "print(batch_predict_job)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "batch_request:mbsdk"
      },
      "source": [
        "*Example output:*\n",
        "\n",
        "    INFO:google.cloud.aiplatform.jobs:Creating BatchPredictionJob\n",
        "    <google.cloud.aiplatform.jobs.BatchPredictionJob object at 0x7f806a6112d0> is waiting for upstream dependencies to complete.\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob created. Resource name: projects/759209241365/locations/us-central1/batchPredictionJobs/5110965452507447296\n",
        "    INFO:google.cloud.aiplatform.jobs:To use this BatchPredictionJob in another session:\n",
        "    INFO:google.cloud.aiplatform.jobs:bpj = aiplatform.BatchPredictionJob('projects/759209241365/locations/us-central1/batchPredictionJobs/5110965452507447296')\n",
        "    INFO:google.cloud.aiplatform.jobs:View Batch Prediction Job:\n",
        "    https://console.cloud.google.com/ai/platform/locations/us-central1/batch-predictions/5110965452507447296?project=759209241365\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob projects/759209241365/locations/us-central1/batchPredictionJobs/5110965452507447296 current state:\n",
        "    JobState.JOB_STATE_RUNNING"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "batch_request_wait:mbsdk"
      },
      "source": [
        "### Wait for completion of batch prediction job\n",
        "\n",
        "Next, wait for the batch job to complete. Alternatively, one can set the parameter `sync` to `True` in the `batch_predict()` method to block until the batch prediction job is completed."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "batch_request_wait:mbsdk"
      },
      "outputs": [],
      "source": [
        "batch_predict_job.wait()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "batch_request_wait:mbsdk"
      },
      "source": [
        "*Example Output:*\n",
        "\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob created. Resource name: projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328\n",
        "    INFO:google.cloud.aiplatform.jobs:To use this BatchPredictionJob in another session:\n",
        "    INFO:google.cloud.aiplatform.jobs:bpj = aiplatform.BatchPredictionJob('projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328')\n",
        "    INFO:google.cloud.aiplatform.jobs:View Batch Prediction Job:\n",
        "    https://console.cloud.google.com/ai/platform/locations/us-central1/batch-predictions/181835033978339328?project=759209241365\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328 current state:\n",
        "    JobState.JOB_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328 current state:\n",
        "    JobState.JOB_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328 current state:\n",
        "    JobState.JOB_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328 current state:\n",
        "    JobState.JOB_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328 current state:\n",
        "    JobState.JOB_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328 current state:\n",
        "    JobState.JOB_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328 current state:\n",
        "    JobState.JOB_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328 current state:\n",
        "    JobState.JOB_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328 current state:\n",
        "    JobState.JOB_STATE_SUCCEEDED\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob run completed. Resource name: projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "get_batch_prediction:mbsdk,iod"
      },
      "source": [
        "### Get the predictions\n",
        "\n",
        "Next, get the results from the completed batch prediction job.\n",
        "\n",
        "The results are written to the Cloud Storage output bucket you specified in the batch prediction request. You call the method iter_outputs() to get a list of each Cloud Storage file generated with the results. Each file contains one or more prediction requests in a JSON format:\n",
        "\n",
        "- `content`: The prediction request.\n",
        "- `prediction`: The prediction response.\n",
        " - `ids`: The internal assigned unique identifiers for each prediction request.\n",
        " - `displayNames`: The class names for each class label.\n",
        " - `bboxes`: The bounding box of each detected object."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "get_batch_prediction:mbsdk,iod"
      },
      "outputs": [],
      "source": [
        "import json\n",
        "\n",
        "import tensorflow as tf\n",
        "\n",
        "bp_iter_outputs = batch_predict_job.iter_outputs()\n",
        "\n",
        "prediction_results = list()\n",
        "for blob in bp_iter_outputs:\n",
        "    if blob.name.split(\"/\")[-1].startswith(\"prediction\"):\n",
        "        prediction_results.append(blob.name)\n",
        "\n",
        "tags = list()\n",
        "for prediction_result in prediction_results:\n",
        "    gfile_name = f\"gs://{bp_iter_outputs.bucket.name}/{prediction_result}\"\n",
        "    with tf.io.gfile.GFile(name=gfile_name, mode=\"r\") as gfile:\n",
        "        for line in gfile.readlines():\n",
        "            line = json.loads(line)\n",
        "            print(line)\n",
        "            break"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "get_batch_prediction:mbsdk,iod"
      },
      "source": [
        "*Example Output:*\n",
        "\n",
        "    Prediction(predictions=[{'ids': ['225634079670796288', '2099131524656922624', '4404974533870616576', '6710817543084310528', '225634079670796288', '2099131524656922624', '6710817543084310528', '2099131524656922624', '225634079670796288', '6710817543084310528', '4404974533870616576', '6710817543084310528', '9016660552298004480', '2099131524656922624', '225634079670796288', '225634079670796288', '225634079670796288', '9016660552298004480', '4404974533870616576', '4404974533870616576', '2099131524656922624', '4404974533870616576', '4404974533870616576', '2099131524656922624', '9016660552298004480', '2099131524656922624', '4404974533870616576', '6710817543084310528', '2099131524656922624', '2099131524656922624', '2099131524656922624', '6710817543084310528', '6710817543084310528', '9016660552298004480', '4404974533870616576', '225634079670796288', '2099131524656922624', '2099131524656922624', '4404974533870616576', '225634079670796288'], 'confidences': [0.993109703, 0.985925615, 0.0453977883, 0.044005096, 0.0327100307, 0.0218445472, 0.019706985, 0.0119887572, 0.0119416527, 0.00427311379, 0.00346497213, 0.00306943036, 0.00277529657, 0.00233747251, 0.00226957048, 0.00169593049, 0.00143932423, 0.00143866683, 0.0012890877, 0.00107431656, 0.00104699621, 0.000975079951, 0.000959897472, 0.000695129682, 0.000612194242, 0.000498361886, 0.000362998835, 0.000331683928, 0.000291648932, 0.000270543911, 0.000256592786, 0.000191880303, 0.000177405163, 0.000124130107, 0.000117361531, 0.00010169336, 8.34302919e-05, 7.46671e-05, 7.43425626e-05, 7.28466039e-05], 'displayNames': ['Salad', 'Baked Goods', 'Cheese', 'Seafood', 'Salad', 'Baked Goods', 'Seafood', 'Baked Goods', 'Salad', 'Seafood', 'Cheese', 'Seafood', 'Tomato', 'Baked Goods', 'Salad', 'Salad', 'Salad', 'Tomato', 'Cheese', 'Cheese', 'Baked Goods', 'Cheese', 'Cheese', 'Baked Goods', 'Tomato', 'Baked Goods', 'Cheese', 'Seafood', 'Baked Goods', 'Baked Goods', 'Baked Goods', 'Seafood', 'Seafood', 'Tomato', 'Cheese', 'Salad', 'Baked Goods', 'Baked Goods', 'Cheese', 'Salad'], 'bboxes': [[0.37747705, 1.0, 0.31908837, 0.981482148], [0.0115042031, 0.553094864, 0.0954515785, 0.728246], [0.00692665577, 0.548264325, 0.204499051, 0.702045619], [0.38464877, 0.958207607, 0.193982765, 0.977637291], [0.17925939, 0.906846285, 0.173706621, 0.842920542], [0.184155583, 0.861931205, 0.132575363, 0.933382452], [0.00305101275, 0.611860037, 0.0430814438, 0.757687569], [0.339314103, 0.976260185, 0.193029925, 1.0], [0.226356596, 0.930693269, 0.581959367, 0.95653224], [0.179834038, 0.908775806, 0.0958697423, 0.939691424], [0.432378292, 0.933267236, 0.383782387, 0.998322189], [0.284259945, 0.931147337, 0.431660354, 0.955191612], [0.380976766, 0.981038809, 0.240252972, 0.948227406], [0.0822101831, 0.80000484, 0.0997514725, 0.613626182], [0.0185090601, 0.71911788, 0.238465369, 0.786021173], [0.226215214, 0.90732789, 0.745254755, 0.976904333], [0.0948396623, 0.77132678, 0.0453846678, 0.7058779], [0.0, 0.600441575, 0.0679637417, 0.726050138], [0.357916325, 0.954997301, 0.135904908, 0.931971073], [0.132627249, 0.893175125, 0.0913171396, 0.866321087], [0.0602070391, 0.563761592, 0.0754725188, 0.508511], [0.0, 0.604042649, 0.0415849313, 0.614845335], [0.229078263, 0.878014445, 0.402972162, 0.969400823], [0.414384753, 0.949710369, 0.0264594965, 0.378633976], [0.241480172, 0.879761755, 0.151504755, 0.885163963], [0.0922716856, 0.839965582, 0.0569753647, 0.344792545], [0.421173453, 0.909894347, 0.0263550486, 0.424664497], [0.469087541, 0.929275, 0.0197226219, 0.430724025], [0.0161168873, 0.577470899, 0.803146243, 0.980094135], [0.274543285, 1.0, 0.508762538, 0.98746717], [0.254469335, 0.936549246, 0.0800963044, 0.473506063], [0.115917385, 0.850970387, 0.0382353887, 0.610701382], [0.215452224, 0.915474534, 0.672911525, 0.974969745], [0.339072973, 0.927099466, 0.545033097, 0.961636961], [0.0, 0.521227062, 0.804131687, 0.996689677], [0.476845741, 0.914084792, 0.00162035227, 0.416255236], [0.00672900677, 0.79107517, 0.23690711, 0.807291448], [0.0806772411, 0.963176131, 0.715660572, 0.979084849], [0.029843241, 0.80046916, 0.0362186842, 0.465307206], [0.375386357, 0.906366706, 0.00989544392, 0.302769661]]}], deployed_model_id='2225068486990757888', explanations=None)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "make_online_predictions:migration"
      },
      "source": [
        "## Make online predictions"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "deploy_model:migration,new,mbsdk"
      },
      "source": [
        "### [predictions.deploy-model-api](https://cloud.google.com/vertex-ai/docs/predictions/deploy-model-api)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "deploy_model:mbsdk,automatic"
      },
      "source": [
        "## Deploy the model\n",
        "\n",
        "Next, deploy your model for online prediction. To deploy the model, you invoke the `deploy` method."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "deploy_model:mbsdk,automatic"
      },
      "outputs": [],
      "source": [
        "endpoint = model.deploy()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "deploy_model:mbsdk,automatic"
      },
      "source": [
        "*Example output:*\n",
        "\n",
        "    INFO:google.cloud.aiplatform.models:Creating Endpoint\n",
        "    INFO:google.cloud.aiplatform.models:Create Endpoint backing LRO: projects/759209241365/locations/us-central1/endpoints/4867177336350441472/operations/4087251132693348352\n",
        "    INFO:google.cloud.aiplatform.models:Endpoint created. Resource name: projects/759209241365/locations/us-central1/endpoints/4867177336350441472\n",
        "    INFO:google.cloud.aiplatform.models:To use this Endpoint in another session:\n",
        "    INFO:google.cloud.aiplatform.models:endpoint = aiplatform.Endpoint('projects/759209241365/locations/us-central1/endpoints/4867177336350441472')\n",
        "    INFO:google.cloud.aiplatform.models:Deploying model to Endpoint : projects/759209241365/locations/us-central1/endpoints/4867177336350441472\n",
        "    INFO:google.cloud.aiplatform.models:Deploy Endpoint model backing LRO: projects/759209241365/locations/us-central1/endpoints/4867177336350441472/operations/1691336130932244480\n",
        "    INFO:google.cloud.aiplatform.models:Endpoint model deployed. Resource name: projects/759209241365/locations/us-central1/endpoints/4867177336350441472"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "endpoints_predict:migration,new,mbsdk"
      },
      "source": [
        "### [predictions.online-prediction-automl](https://cloud.google.com/vertex-ai/docs/predictions/online-predictions-automl)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "get_test_item"
      },
      "source": [
        "### Get test item\n",
        "\n",
        "You will use an arbitrary example out of the dataset as a test item. Don't be concerned that the example was likely used in training the model -- we just want to demonstrate how to make a prediction."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "get_test_item:automl,iod,csv"
      },
      "outputs": [],
      "source": [
        "test_items = !gsutil cat $IMPORT_FILE | head -n1\n",
        "cols = str(test_items[0]).split(\",\")\n",
        "if len(cols) == 11:\n",
        "    test_item = str(cols[1])\n",
        "    test_label = str(cols[2])\n",
        "else:\n",
        "    test_item = str(cols[0])\n",
        "    test_label = str(cols[1])\n",
        "\n",
        "print(test_item, test_label)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "predict_request:mbsdk,iod"
      },
      "source": [
        "### Make the prediction\n",
        "\n",
        "Now that your `Model` resource is deployed to an `Endpoint` resource, you can do online predictions by sending prediction requests to the Endpoint resource.\n",
        "\n",
        "#### Request\n",
        "\n",
        "Since in this example your test item is in a Cloud Storage bucket, you open and read the contents of the image using `tf.io.gfile.Gfile()`. To pass the test data to the prediction service, you encode the bytes into base64 -- which makes the content safe from modification while transmitting binary data over the network.\n",
        "\n",
        "The format of each instance is:\n",
        "\n",
        "    { 'content': { 'b64': base64_encoded_bytes } }\n",
        "\n",
        "Since the `predict()` method can take multiple items (instances), send your single test item as a list of one test item.\n",
        "\n",
        "#### Response\n",
        "\n",
        "The response from the `predict()` call is a Python dictionary with the following entries:\n",
        "\n",
        "- `ids`: The internal assigned unique identifiers for each prediction request.\n",
        "- `displayNames`: The class names for each class label.\n",
        "- `confidences`: The predicted confidence, between 0 and 1, per class label.\n",
        "- `bboxes`: The bounding box of each detected object.\n",
        "- `deployed_model_id`: The Vertex AI identifier for the deployed Model resource which did the predictions."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "predict_request:mbsdk,iod"
      },
      "outputs": [],
      "source": [
        "import base64\n",
        "\n",
        "import tensorflow as tf\n",
        "\n",
        "with tf.io.gfile.GFile(test_item, \"rb\") as f:\n",
        "    content = f.read()\n",
        "\n",
        "# The format of each instance should conform to the deployed model's prediction input schema.\n",
        "instances = [{\"content\": base64.b64encode(content).decode(\"utf-8\")}]\n",
        "\n",
        "prediction = endpoint.predict(instances=instances)\n",
        "\n",
        "print(prediction)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "predict_request:mbsdk,iod"
      },
      "source": [
        "*Example output:*\n",
        "\n",
        "    Prediction(predictions=[{'ids': ['2250776168359788544', '1097854663752941568', '2250776168359788544', '1097854663752941568', '1097854663752941568', '2250776168359788544', '5709540682180329472', '1097854663752941568', '2250776168359788544', '2250776168359788544', '3403697672966635520', '2250776168359788544', '3403697672966635520', '5709540682180329472', '8015383691394023424', '5709540682180329472', '3403697672966635520', '3403697672966635520', '3403697672966635520', '1097854663752941568', '2250776168359788544', '2250776168359788544', '1097854663752941568', '2250776168359788544', '2250776168359788544', '3403697672966635520', '8015383691394023424', '1097854663752941568', '2250776168359788544', '1097854663752941568', '2250776168359788544', '2250776168359788544', '5709540682180329472', '1097854663752941568', '1097854663752941568', '2250776168359788544', '5709540682180329472', '8015383691394023424', '2250776168359788544', '3403697672966635520'], 'displayNames': ['Salad', 'Baked Goods', 'Salad', 'Baked Goods', 'Baked Goods', 'Salad', 'Seafood', 'Baked Goods', 'Salad', 'Salad', 'Cheese', 'Salad', 'Cheese', 'Seafood', 'Tomato', 'Seafood', 'Cheese', 'Cheese', 'Cheese', 'Baked Goods', 'Salad', 'Salad', 'Baked Goods', 'Salad', 'Salad', 'Cheese', 'Tomato', 'Baked Goods', 'Salad', 'Baked Goods', 'Salad', 'Salad', 'Seafood', 'Baked Goods', 'Baked Goods', 'Salad', 'Seafood', 'Tomato', 'Salad', 'Cheese'], 'bboxes': [[0.423218071, 0.979368508, 0.339486301, 0.953063667], [0.00295934081, 0.562559605, 0.0866475701, 0.733601213], [0.446993053, 0.985957086, 0.103116274, 1.0], [0.0, 0.798019052, 0.0843312368, 0.910729289], [0.200755239, 0.878952861, 0.161852047, 0.962031841], [0.141423374, 0.893875718, 0.0966037139, 0.975178], [0.446385473, 0.954654098, 0.322997868, 0.955895185], [0.0540368557, 0.735850155, 0.22394672, 0.783547163], [0.0511586964, 0.590276837, 0.247575879, 0.904480815], [0.291286081, 0.943224788, 0.742888927, 0.981343627], [0.0209743679, 0.534335613, 0.145708829, 0.751705289], [0.286338538, 0.956577778, 0.584680378, 0.96791], [0.493991911, 0.92484349, 0.385724247, 0.975459754], [0.320449412, 0.9483518, 0.16373083, 1.0], [0.458497226, 0.937700331, 0.342862546, 0.935759783], [0.0386219025, 0.711036801, 0.0140583916, 0.901848], [0.484096915, 0.925073504, 0.0282530189, 0.426966846], [0.329300791, 0.922743678, 0.158341929, 1.0], [0.472070664, 0.945299745, 0.0491278172, 0.907515883], [0.395479739, 0.947133362, 0.233833641, 0.983137965], [0.498143733, 0.962923825, 0.000415623188, 0.462042093], [0.168997586, 0.846541345, 0.472939, 0.953329563], [0.0585409701, 0.603083611, 0.0806979761, 0.442619056], [0.149338543, 0.87841326, 0.68745774, 0.975087523], [0.353095531, 0.95818007, 0.0186360981, 0.329624653], [0.127367049, 0.796614647, 0.0658644438, 0.776186824], [0.103480637, 0.728991389, 0.102527976, 0.843147755], [0.0629016757, 0.891313374, 0.0636595339, 0.424591154], [0.0277965665, 0.718145788, 0.0, 0.728123844], [0.431717843, 0.966227055, 0.0133913755, 0.449536532], [0.220863849, 0.897798777, 0.0167301502, 0.446346819], [0.128112122, 0.62433213, 0.576314211, 0.960604429], [0.0, 0.589858651, 0.0557853393, 0.740757704], [0.0763699561, 0.564970851, 0.426471323, 0.898873031], [0.0173879862, 0.97320509, 0.0287637822, 0.322233558], [0.0586977303, 0.64173305, 0.797896147, 0.986974], [0.474722087, 0.961244881, 0.00339941191, 0.444934338], [0.0472798944, 0.610487342, 0.0866674632, 0.682843804], [0.477720022, 0.945115, 0.00805068, 0.81928432], [0.223478109, 0.819148779, 0.297299504, 0.964423835]], 'confidences': [0.99927026, 0.998335898, 0.122888528, 0.0565881766, 0.0334013812, 0.0227512382, 0.019394394, 0.0163640734, 0.0106114028, 0.0095577, 0.0092699714, 0.00591981411, 0.00551535888, 0.00492637418, 0.00376707385, 0.00370016089, 0.00318370084, 0.00264257635, 0.00248732581, 0.00241615647, 0.0022392564, 0.00208666641, 0.00198837952, 0.00168023549, 0.00166417053, 0.0011962672, 0.00100666645, 0.000980117242, 0.000788055069, 0.000682824815, 0.000606221438, 0.000559259, 0.000542622234, 0.000534898543, 0.000527789292, 0.000484947581, 0.000421707897, 0.000392853981, 0.000352483446, 0.0003487629]}], deployed_model_id='5770527293638180864', explanations=None)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "undeploy_model:mbsdk"
      },
      "source": [
        "## Undeploy the model\n",
        "\n",
        "When you are done doing predictions, you undeploy the model from the `Endpoint` resouce. This deprovisions all compute resources and ends billing for the deployed model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "undeploy_model:mbsdk"
      },
      "outputs": [],
      "source": [
        "endpoint.undeploy_all()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cleanup:mbsdk"
      },
      "source": [
        "# Cleaning up\n",
        "\n",
        "To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n",
        "project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n",
        "\n",
        "Otherwise, you can delete the individual resources you created in this tutorial."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cleanup:mbsdk"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "\n",
        "delete_bucket = False\n",
        "\n",
        "# Delete the dataset using the Vertex dataset object\n",
        "\n",
        "dataset.delete()\n",
        "\n",
        "# Delete the model using the Vertex model object\n",
        "model.delete()\n",
        "\n",
        "# Delete the endpoint using the Vertex endpoint object\n",
        "endpoint.delete()\n",
        "\n",
        "# Delete the AutoML or Pipeline trainig job\n",
        "\n",
        "dag.delete()\n",
        "\n",
        "# Delete the batch prediction job using the Vertex batch prediction object\n",
        "batch_predict_job.delete()\n",
        "\n",
        "if delete_bucket or os.getenv(\"IS_TESTING\"):\n",
        "    ! gsutil rm -r $BUCKET_URI"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "sdk-automl-image-object-detection-batch-online.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}