notebooks/official/sdk/SDK_AutoML_Video_Classification.ipynb

{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "modular-concentration"
      },
      "outputs": [],
      "source": [
        "# Copyright 2022 Google LLC\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b88c5cede17b"
      },
      "source": [
        "# Vertex AI SDK for Python: AutoML Video Classification Example\n",
        "\n",
        "<table align=\"left\">\n",
        "<td>\n",
        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/sdk/SDK_AutoML_Video_Classification.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/sdk/SDK_AutoML_Video_Classification.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
        "      View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/sdk/SDK_AutoML_Video_Classification.ipynb\">\n",
        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
        "      Open in Vertex AI Workbench\n",
        "    </a>\n",
        "  </td>                                                                                               \n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "522e8eec0fcb"
      },
      "source": [
        "## Overview\n",
        "\n",
        "This notebook demonstrates how to create an AutoML Video Classification Model, with a Vertex AI video dataset, and how to serve the model for batch prediction. It requires you provide a bucket where the dataset will be stored.\n",
        "\n",
        "Note: you may incur charges for training, prediction, storage or usage of other GCP products in connection with testing this SDK.\n",
        "\n",
        "Learn more about [Classification for video data](https://cloud.google.com/vertex-ai/docs/training-overview#classification_for_videos)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "afc6017b7b45"
      },
      "source": [
        "### Objective\n",
        "\n",
        "The objective of this notebook is to build a AutoML Video Classification Model. The following steps have been followed:  \n",
        "This tutorial uses the following Google Cloud ML services :\n",
        "- `Vertex AI Dataset` resource\n",
        "- `AutoML Training`\n",
        "- `Vertex AI Model` resource\n",
        "- `Vertex AI Batch Prediction`\n",
        "\n",
        "\n",
        "The steps performed include the following:\n",
        "\n",
        "- Set your task name, and GCS prefix\n",
        "- Copy AutoML video demo train data for creating managed dataset\n",
        "- Create a dataset on Vertex AI.\n",
        "- Configure a training job\n",
        "- Launch a training job and create a model on Vertex AI\n",
        "- Copy AutoML Video Demo Prediction Data for creating batch prediction job\n",
        "- Perform batch prediction job on the model\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5bba1b08cba7"
      },
      "source": [
        "### Dataset\n",
        "\n",
        "##### HMDB: a large human motion database\n",
        "Some training data and prediction data for the demo is prepared using the [HMDB Dataset](https://serre-lab.clps.brown.edu/resource/hmdb-a-large-human-motion-database).\n",
        "\n",
        "The HMDB Dataset is licensed under the Creative Commons Attribution 4.0 International License. To view a copy of this license, visit https://creativecommons.org/licenses/by/4.0/\n",
        "\n",
        "For more information about this dataset please visit: https://serre-lab.clps.brown.edu/resource/hmdb-a-large-human-motion-database/"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "248a51c68228"
      },
      "source": [
        "## Costs\n",
        "\n",
        "This tutorial uses the following billable components of Google Cloud:\n",
        "\n",
        "- Vertex AI\n",
        "- Cloud Storage\n",
        "\n",
        "\n",
        "Learn about [Vertex AI\n",
        "pricing](https://cloud.google.com/vertex-ai/pricing), [Cloud Storage\n",
        "pricing](https://cloud.google.com/storage/pricing), and use the [Pricing\n",
        "Calculator](https://cloud.google.com/products/calculator/)\n",
        "to generate a cost estimate based on your projected usage."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "install_aip"
      },
      "source": [
        "## Installation\n",
        "\n",
        "Install the following packages required to execute this notebook. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "coated-remark"
      },
      "outputs": [],
      "source": [
        "! pip3 install --upgrade --quiet google-cloud-aiplatform"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "restart"
      },
      "source": [
        "### Colab only: Uncomment the following cell to restart the kernel.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bzPxhxS5lugp"
      },
      "outputs": [],
      "source": [
        "# Automatically restart kernel after installs so that your environment can access the new packages\n",
        "# import IPython\n",
        "\n",
        "# app = IPython.Application.instance()\n",
        "# app.kernel.do_shutdown(True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d2qpIurSjmpT"
      },
      "source": [
        "## Before you begin\n",
        "\n",
        "### Set up your Google Cloud project\n",
        "\n",
        "**The following steps are required, regardless of your notebook environment.**\n",
        "\n",
        "1. [Select or create a Google Cloud project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\n",
        "\n",
        "2. [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project).\n",
        "\n",
        "3. [Enable the following APIs: Vertex AI API, Cloud Resource Manager API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com,cloudresourcemanager.googleapis.com).\n",
        "\n",
        "4. If you are running this notebook locally, you need to install the [Cloud SDK](https://cloud.google.com/sdk)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "project_id"
      },
      "source": [
        "#### Set your project ID\n",
        "\n",
        "**If you don't know your project ID**, try the following:\n",
        "* Run `gcloud config list`.\n",
        "* Run `gcloud projects list`.\n",
        "* See the support page: [Locate the project ID](https://support.google.com/googleapi/answer/7014113)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "wsePm9c4jmpT"
      },
      "outputs": [],
      "source": [
        "PROJECT_ID = \"[your-project-id]\"  # @param {type:\"string\"}\n",
        "\n",
        "# Set the project id\n",
        "! gcloud config set project {PROJECT_ID}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a54f9d7c1876"
      },
      "source": [
        "#### Region\n",
        "\n",
        "You can also change the `REGION` variable used by Vertex AI. Learn more about [Vertex AI regions](https://cloud.google.com/vertex-ai/docs/general/locations)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3aaadaaf9b30"
      },
      "outputs": [],
      "source": [
        "REGION = \"us-central1\"  # @param {type: \"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "57dad372c81b"
      },
      "source": [
        "#### UUID\n",
        "\n",
        "If you are in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a uuid for each instance session, and append it onto the name of resources you create in this tutorial."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4e166d927e36"
      },
      "outputs": [],
      "source": [
        "import random\n",
        "import string\n",
        "\n",
        "\n",
        "# Generate a uuid of a specifed length(default=8)\n",
        "def generate_uuid(length: int = 8) -> str:\n",
        "    return \"\".join(random.choices(string.ascii_lowercase + string.digits, k=length))\n",
        "\n",
        "\n",
        "UUID = generate_uuid()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5c0404984792"
      },
      "source": [
        "### Authenticate your Google Cloud account\n",
        "\n",
        "Depending on your Jupyter environment, you may have to manually authenticate. Follow the relevant instructions below.\n",
        "\n",
        "**1. Vertex AI Workbench**\n",
        "* Do nothing as you are already authenticated.\n",
        "\n",
        "**2. Local JupyterLab instance, uncomment and run:**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Nt8cEM2GjmpU"
      },
      "outputs": [],
      "source": [
        "# ! gcloud auth login"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XUSL_JcpjmpU"
      },
      "source": [
        "**3. Colab, uncomment and run:**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_2zemfGvjmpU"
      },
      "outputs": [],
      "source": [
        "# from google.colab import auth\n",
        "# auth.authenticate_user()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TCPJ38n7jmpU"
      },
      "source": [
        "**4. Service account or other**\n",
        "* See how to grant Cloud Storage permissions to your service account at https://cloud.google.com/storage/docs/gsutil/commands/iam#ch-examples."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bucket:custom"
      },
      "source": [
        "### Create a Cloud Storage bucket\n",
        "\n",
        "Create a storage bucket to store intermediate artifacts such as datasets.\n",
        "\n",
        "When you submit a training job using the Cloud SDK, you upload a Python package\n",
        "containing your training code to a Cloud Storage bucket. Vertex AI runs\n",
        "the code from this package. In this tutorial, Vertex AI also saves the\n",
        "trained model that results from your job in the same bucket. Using this model artifact, you can then\n",
        "create Vertex AI Model resource and use for prediction."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bucket"
      },
      "outputs": [],
      "source": [
        "BUCKET_URI = \"gs://your-bucket-name-unique\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_bucket"
      },
      "source": [
        "**Only if your bucket doesn't already exist**: Run the following cell to create your Cloud Storage bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Oz8J0vmSlugt"
      },
      "outputs": [],
      "source": [
        "! gsutil mb -l $REGION -p $PROJECT_ID $BUCKET_URI"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "incorporated-edgar"
      },
      "source": [
        "### Import libraries and define constants\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "hispanic-macedonia"
      },
      "outputs": [],
      "source": [
        "import json\n",
        "\n",
        "from google.cloud import aiplatform, storage\n",
        "\n",
        "MY_PROJECT = PROJECT_ID\n",
        "MY_STAGING_BUCKET = BUCKET_URI"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "historical-consciousness"
      },
      "source": [
        "### Set Your Task Name, and GCS Prefix\n",
        "\n",
        "If you want to centeralize all input and output files under the gcs location."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "organizational-salad"
      },
      "outputs": [],
      "source": [
        "TASK_TYPE = \"mbsdk_automl-video-training\"\n",
        "PREDICTION_TYPE = \"classification\"\n",
        "MODEL_TYPE = \"CLOUD\"\n",
        "\n",
        "TASK_NAME = f\"{TASK_TYPE}_{PREDICTION_TYPE}\"\n",
        "BUCKET_NAME = MY_STAGING_BUCKET.split(\"gs://\")[1]\n",
        "GCS_PREFIX = TASK_NAME\n",
        "\n",
        "print(f\"Bucket Name:    {BUCKET_NAME}\")\n",
        "print(f\"Task Name:      {TASK_NAME}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "professional-bulletin"
      },
      "source": [
        "### Copy AutoML Video Demo Train Data for Creating Managed Dataset"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "accurate-producer"
      },
      "outputs": [],
      "source": [
        "automl_video_demo_train_data = (\n",
        "    \"gs://automl-video-demo-data/hmdb_split1_5classes_all.csv\"\n",
        ")\n",
        "\n",
        "\n",
        "gcs_source_train = f\"gs://{BUCKET_NAME}/{TASK_NAME}/data/video_classification.csv\"\n",
        "\n",
        "!gsutil cp $automl_video_demo_train_data $gcs_source_train"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rough-alert"
      },
      "source": [
        "# Run AutoML Video Training with Vertex AI Video Dataset"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "adaptive-slovakia"
      },
      "source": [
        "## Initialize Vertex AI SDK for Python\n",
        "\n",
        "Initialize the *client* for Vertex AI."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "figured-fellow"
      },
      "outputs": [],
      "source": [
        "aiplatform.init(project=PROJECT_ID, staging_bucket=BUCKET_URI)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pleasant-holmes"
      },
      "source": [
        "## Create a Dataset on Vertex AI Dataset resource\n",
        "We will now create a Vertex AI video dataset using the previously prepared csv files."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "uVBfL-0TTjNS"
      },
      "outputs": [],
      "source": [
        "dataset = aiplatform.VideoDataset.create(\n",
        "    display_name=f\"temp-{TASK_NAME}\",\n",
        "    gcs_source=gcs_source_train,\n",
        "    import_schema_uri=aiplatform.schema.dataset.ioformat.video.classification,\n",
        "    sync=False,\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3x4xuyIbVR_N"
      },
      "outputs": [],
      "source": [
        "dataset.wait()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mexican-spending"
      },
      "source": [
        "## Launch a training job and create a model on Vertex AI"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fdb1d50298ef"
      },
      "source": [
        "To train an AutoML model, you perform two steps: 1) create a training pipeline, and 2) run the pipeline.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dynamic-piece"
      },
      "source": [
        "### Configure a Training Job"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a509028760a0"
      },
      "source": [
        "An AutoML training pipeline is created with the `AutoMLVideoTrainingJob` class, with the following parameters:\n",
        "\n",
        "- `display_name`: The human readable name for the `TrainingJob` resource.\n",
        "- `prediction_type`: The type task to train the model for.\n",
        "  - `classification`: A video classification model.\n",
        "  - `object_tracking`: A video object tracking model.\n",
        "  - `action_recognition`: A video action recognition model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "continuous-circular"
      },
      "outputs": [],
      "source": [
        "job = aiplatform.AutoMLVideoTrainingJob(\n",
        "    display_name=f\"temp-{TASK_NAME}\",\n",
        "    prediction_type=PREDICTION_TYPE,\n",
        "    model_type=MODEL_TYPE,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "juvenile-parameter"
      },
      "source": [
        "### Run the Training Job"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a3b60132368e"
      },
      "source": [
        "Next, you run the job to start the training job by invoking the method `run`, with the following parameters:\n",
        "\n",
        "- `dataset`: The `Dataset` resource to train the model.\n",
        "- `model_display_name`: The human readable name for the trained model.\n",
        "- `training_fraction_split`: The percentage of the dataset to use for training.\n",
        "- `test_fraction_split`: The percentage of the dataset to use for test (holdout data).\n",
        "- `sync`: If set to True, the call will block while waiting for the asynchronous batch job to complete.\n",
        "\n",
        "The `run` method when completed returns the `Model` resource.\n",
        "\n",
        "The execution of the training pipeline can take over 24 hours to complete."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8f135100f5d9"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "import sys\n",
        "\n",
        "if os.getenv(\"IS_TESTING\"):\n",
        "    sys.exit(0)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "human-carrier"
      },
      "outputs": [],
      "source": [
        "model = job.run(\n",
        "    dataset=dataset,\n",
        "    training_fraction_split=0.8,\n",
        "    test_fraction_split=0.2,\n",
        "    model_display_name=f\"temp-{TASK_NAME}\",\n",
        "    sync=False,\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "abstract-textbook"
      },
      "outputs": [],
      "source": [
        "model.wait()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "noted-usage"
      },
      "source": [
        "# Make a Batch Prediction request"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ruled-smith"
      },
      "source": [
        "### Copy AutoML video demo prediction data for creating batch prediction job"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "polished-dispatch"
      },
      "outputs": [],
      "source": [
        "automl_video_demo_batch_prediction_data = (\n",
        "    \"gs://automl-video-demo-data/hmdb_split1_predict.jsonl\"\n",
        ")\n",
        "\n",
        "gcs_source_batch_prediction = (\n",
        "    f\"gs://{BUCKET_NAME}/{TASK_NAME}/data/video_classification_batch_prediction.jsonl\"\n",
        ")\n",
        "gcs_destination_prefix_batch_prediction = (\n",
        "    f\"gs://{BUCKET_NAME}/{TASK_NAME}/batch_prediction\"\n",
        ")\n",
        "\n",
        "!gsutil cp $automl_video_demo_batch_prediction_data $gcs_source_batch_prediction"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2bd262d624e3"
      },
      "source": [
        "### Perform batch prediction job on the model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b488cb43682a"
      },
      "source": [
        "Now that your Model resource is trained, you can make a batch prediction by invoking the batch_predict() method, with the following parameters:\n",
        "\n",
        "- `job_display_name`: The human readable name for the batch prediction job.\n",
        "- `gcs_source`: A list of one or more batch request input files.\n",
        "- `gcs_destination_prefix`: The Cloud Storage location for storing the batch prediction resuls.\n",
        "- `sync`: If set to True, the call will block while waiting for the asynchronous batch job to complete."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "piano-middle"
      },
      "outputs": [],
      "source": [
        "batch_predict_job = model.batch_predict(\n",
        "    job_display_name=f\"temp-{TASK_NAME}\",\n",
        "    gcs_source=gcs_source_batch_prediction,\n",
        "    gcs_destination_prefix=gcs_destination_prefix_batch_prediction,\n",
        "    sync=False,\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "e06e39558a8e"
      },
      "outputs": [],
      "source": [
        "batch_predict_job.wait()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8359d586c1e8"
      },
      "source": [
        "### Get the predictions\n",
        "\n",
        "Next, get the results from the completed batch prediction job.\n",
        "\n",
        "The results are written to the Cloud Storage output bucket you specified in the batch prediction request. You call the method iter_outputs() to get a list of each Cloud Storage file generated with the results. Each file contains one or more prediction requests in a JSON format:\n",
        "\n",
        "- `content`: The prediction request.\n",
        "- `prediction`: The prediction response.\n",
        "\n",
        "Prediction response contains following fields\n",
        "\n",
        "- `ids`: The internal assigned unique identifiers for each prediction request.\n",
        "- `displayNames`: The class names for each class label.\n",
        "- `confidences`: The predicted confidence, between 0 and 1, per class label.\n",
        "- `timeSegmentStart`: The time offset in the video to the start of the video sequence.\n",
        "- `timeSegmentEnd`: The time offset in the video to the end of the video sequence."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "visible-scientist"
      },
      "outputs": [],
      "source": [
        "bp_iter_outputs = batch_predict_job.iter_outputs()\n",
        "\n",
        "prediction_results = list()\n",
        "for blob in bp_iter_outputs:\n",
        "    if blob.name.split(\"/\")[-1].startswith(\"prediction\"):\n",
        "        prediction_results.append(blob.name)\n",
        "client = storage.Client()\n",
        "bucket = client.get_bucket(BUCKET_URI.replace(\"gs://\", \"\"))\n",
        "for prediction_result in prediction_results:\n",
        "    gfile_name = f\"{prediction_result}\"\n",
        "    data = bucket.blob(gfile_name).download_as_string()\n",
        "    data = json.loads(data)\n",
        "    print(data)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0472cd54c140"
      },
      "source": [
        "## Clean up\n",
        "<a name=\"section-13\"></a>\n",
        "\n",
        "To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n",
        "project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n",
        "\n",
        "Otherwise, you can delete the individual resources you created in this tutorial:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2f60a4fb2863"
      },
      "outputs": [],
      "source": [
        "delete_bucket = False\n",
        "\n",
        "# Delete the dataset using the Vertex dataset object\n",
        "dataset.delete()\n",
        "\n",
        "# Delete the model using the Vertex model object\n",
        "model.delete()\n",
        "\n",
        "# Delete the AutoML or Pipeline training job\n",
        "job.delete()\n",
        "\n",
        "# Delete the batch prediction job using the Vertex batch prediction object\n",
        "batch_predict_job.delete()\n",
        "\n",
        "# Delete the Cloud Storage bucket\n",
        "if delete_bucket or os.getenv(\"IS_TESTING\"):\n",
        "    ! gsutil -m rm -r $BUCKET_URI"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "name": "SDK_AutoML_Video_Classification.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}