notebooks/official/migration/sdk-automl-tabular-binary-classification-online-prediction.ipynb

{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "copyright"
      },
      "outputs": [],
      "source": [
        "# Copyright 2021 Google LLC\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "title:migration,new"
      },
      "source": [
        "# Vertex AI migration: AutoML tabular binary classification\n",
        "\n",
        "<table align=\"left\">\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/lightweight_functions_component_io_kfp.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Google Colaboratory logo\"><br> Open in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fvertex-ai-samples%2Fmain%2Fnotebooks%2Fnotebook_template.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-enterprise-logo-32px.png\" alt=\"Google Cloud Colab Enterprise logo\"><br> Open in Colab Enterprise\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/lightweight_functions_component_io_kfp.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\"><br> View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/pipelines/lightweight_functions_component_io_kfp.ipynb\" target='_blank'>\n",
        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\"><br> Open in Vertex AI Workbench\n",
        "     </a>\n",
        "   </td>\n",
        "</table>\n",
        "<br/><br/><br/>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fb82f94bbbc7"
      },
      "source": [
        "## Overview\n",
        "\n",
        "\n",
        "This tutorial demonstrates how to use the Vertex AI SDK to create tabular binary classification models and do online prediction using an AutoML model.\n",
        "\n",
        "Learn more about [Migrate to Vertex AI](https://cloud.google.com/vertex-ai/docs/start/migrating-to-vertex-ai). Learn more about [Classification for tabular data](https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/overview)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9f80bba45dd5"
      },
      "source": [
        "### Objective\n",
        "\n",
        "In this tutorial, you create an AutoML tabular binary classification model and deploy for online prediction from a Python script using the Vertex AI SDK. You can alternatively create and deploy models using the `gcloud` command-line tool or using the online Cloud Console.\n",
        "\n",
        "This tutorial uses the following Google Cloud ML services and resources:\n",
        "\n",
        "- Vertex AI managed dataset\n",
        "- Vertex AI Training\n",
        "- Vertex AI endpoints\n",
        "- Vertex AI prediction\n",
        "\n",
        "The steps performed include:\n",
        "\n",
        "- Create a Vertex AI dataset resource.\n",
        "- Train the model.\n",
        "- View the model evaluation.\n",
        "- Deploy the model resource to a serving endpoint resource.\n",
        "- Make a prediction.\n",
        "- Undeploy the model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dataset:bank,lbn"
      },
      "source": [
        "### Dataset\n",
        "\n",
        "The dataset used for this tutorial is the [Bank Marketing](https://pantheon.corp.google.com/storage/browser/_details/cloud-ml-tables-data/bank-marketing.csv) . This dataset doesn't require any feature engineering. The version of the dataset you use in this tutorial is stored in a public Cloud Storage bucket."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "costs"
      },
      "source": [
        "### Costs\n",
        "\n",
        "This tutorial uses billable components of Google Cloud:\n",
        "\n",
        "* Vertex AI\n",
        "* Cloud Storage\n",
        "\n",
        "Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing) and \n",
        "[Cloud Storage pricing](https://cloud.google.com/storage/pricing), and use the \n",
        "[Pricing Calculator](https://cloud.google.com/products/calculator/) to generate a cost estimate based on your projected usage."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "install_aip:mbsdk"
      },
      "source": [
        "### Get started\n",
        "Install Vertex AI SDK for Python and other required packages"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "install_aip:mbsdk"
      },
      "outputs": [],
      "source": [
        "! pip3 install --upgrade --quiet google-cloud-aiplatform fsspec gcsfs \\\n",
        "                                 pandas"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "restart"
      },
      "source": [
        "### Restart runtime (Colab only)\n",
        "To use the newly installed packages, you must restart the runtime on Google Colab."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "D-ZBOjErv5mM"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "\n",
        "if \"google.colab\" in sys.modules:\n",
        "\n",
        "    import IPython\n",
        "\n",
        "    app = IPython.Application.instance()\n",
        "    app.kernel.do_shutdown(True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4de1bd77992b"
      },
      "source": [
        "<div class=\"alert alert-block alert-warning\">,\n",
        "<b>⚠️ The kernel is going to restart. Wait until it's finished before continuing to the next step. ⚠️</b>,\n",
        "</div>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "befa6ca14bc0"
      },
      "source": [
        "### Authenticate your notebook environment (Colab only)\n",
        "Authenticate your environment on Google Colab."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7de6ef0fac42"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "\n",
        "if \"google.colab\" in sys.modules:\n",
        "\n",
        "    from google.colab import auth\n",
        "\n",
        "    auth.authenticate_user()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bd0e79ceaea2"
      },
      "source": [
        "### Set Google Cloud project information\n",
        "Learn more about [setting up a project and a development environment.](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8940d70dfdef"
      },
      "outputs": [],
      "source": [
        "PROJECT_ID = \"[your-project-id]\"  # @param {type:\"string\"}\n",
        "LOCATION = \"us-central1\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bucket:mbsdk"
      },
      "source": [
        "### Create a Cloud Storage bucket\n",
        "\n",
        "Create a storage bucket to store intermediate artifacts such as datasets."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bucket"
      },
      "outputs": [],
      "source": [
        "BUCKET_URI = f\"gs://your-bucket-name-{PROJECT_ID}-unique\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_bucket"
      },
      "source": [
        "**If your bucket doesn't already exist**: Run the following cell to create your Cloud Storage bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "create_bucket"
      },
      "outputs": [],
      "source": [
        "! gsutil mb -l {LOCATION} {BUCKET_URI}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "setup_vars"
      },
      "source": [
        "### Import libraries and define constants"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "import_aip:mbsdk"
      },
      "outputs": [],
      "source": [
        "import google.cloud.aiplatform as aiplatform\n",
        "import pandas as pd"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "init_aip:mbsdk"
      },
      "source": [
        "## Initialize Vertex AI SDK for Python\n",
        "\n",
        "Initialize the Vertex AI SDK for Python for your project and corresponding bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "init_aip:mbsdk"
      },
      "outputs": [],
      "source": [
        "aiplatform.init(project=PROJECT_ID, staging_bucket=BUCKET_URI)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "import_file:u_dataset,csv"
      },
      "source": [
        "#### Location of Cloud Storage training data.\n",
        "\n",
        "Now set the variable `IMPORT_FILE` to the location of the CSV index file in Cloud Storage."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "import_file:bank,csv,lbn"
      },
      "outputs": [],
      "source": [
        "IMPORT_FILE = \"gs://cloud-ml-tables-data/bank-marketing.csv\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "quick_peek:tabular"
      },
      "source": [
        "#### Quick peek at your data\n",
        "\n",
        "This tutorial uses a version of the Bank Marketing dataset that is stored in a public Cloud Storage bucket, using a CSV index file.\n",
        "\n",
        "Start by doing a quick peek at the data. Count the number of examples by counting the number of rows in the CSV index file  (`wc -l`) and then peek at the first few rows.\n",
        "\n",
        "You also need for training to know the heading name of the label column, which is save as `label_column`. For this dataset, it's the last column in the CSV file."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "quick_peek:tabular"
      },
      "outputs": [],
      "source": [
        "count = ! gsutil cat $IMPORT_FILE | wc -l\n",
        "print(\"Number of Examples\", int(count[0]))\n",
        "\n",
        "print(\"First 10 rows\")\n",
        "! gsutil cat $IMPORT_FILE | head\n",
        "\n",
        "heading = ! gsutil cat $IMPORT_FILE | head -n1\n",
        "label_column = str(heading).split(\",\")[-1].split(\"'\")[0]\n",
        "print(\"Label Column Name\", label_column)\n",
        "if label_column is None:\n",
        "    raise Exception(\"label column missing\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_a_dataset:migration"
      },
      "source": [
        "## Create a dataset"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "datasets_create:migration,new,mbsdk"
      },
      "source": [
        "### [datasets.create-dataset-api](https://cloud.google.com/vertex-ai/docs/datasets/create-dataset-api)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_dataset:tabular,lbn"
      },
      "source": [
        "### Create the Dataset\n",
        "\n",
        "Next, create the dataset resource using the `create` method for the `TabularDataset` class, which takes the following parameters:\n",
        "\n",
        "- display_name: The human readable name for the dataset resource.\n",
        "- gcs_source: A list of one or more dataset index files to import the data items into the dataset resource.\n",
        "- bq_source: Alternatively, import data items from a BigQuery table into the dataset resource.\n",
        "\n",
        "This operation may takes several minutes."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "create_dataset:tabular,lbn"
      },
      "outputs": [],
      "source": [
        "dataset = aiplatform.TabularDataset.create(\n",
        "    display_name=\"Bank Marketing\" + \"_\" + \"unique\", gcs_source=[IMPORT_FILE]\n",
        ")\n",
        "\n",
        "print(dataset.resource_name)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_dataset:tabular,lbn"
      },
      "source": [
        "*Example Output:*\n",
        "\n",
        "    INFO:google.cloud.aiplatform.datasets.dataset:Creating TabularDataset\n",
        "    INFO:google.cloud.aiplatform.datasets.dataset:Create TabularDataset backing LRO: projects/759209241365/locations/us-central1/datasets/5048649531491614720/operations/752335608625496064\n",
        "    INFO:google.cloud.aiplatform.datasets.dataset:TabularDataset created. Resource name: projects/759209241365/locations/us-central1/datasets/5048649531491614720\n",
        "    INFO:google.cloud.aiplatform.datasets.dataset:To use this TabularDataset in another session:\n",
        "    INFO:google.cloud.aiplatform.datasets.dataset:ds = aiplatform.TabularDataset('projects/759209241365/locations/us-central1/datasets/5048649531491614720')\n",
        "    projects/759209241365/locations/us-central1/datasets/5048649531491614720"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "train_a_model:migration"
      },
      "source": [
        "## Train a model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "trainingpipelines_create:migration,new,mbsdk"
      },
      "source": [
        "### [training.automl-api](https://cloud.google.com/vertex-ai/docs/training/automl-api)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_automl_pipeline:tabular,lbn"
      },
      "source": [
        "### Create and run training pipeline\n",
        "\n",
        "To train an AutoML model, you perform two steps: 1) create a training pipeline, and 2) run the pipeline.\n",
        "\n",
        "#### Create training pipeline\n",
        "\n",
        "An AutoML training pipeline is created with the `AutoMLTabularTrainingJob` class, with the following parameters:\n",
        "\n",
        "- display_name: The human readable name for the `TrainingJob` resource.\n",
        "- optimization_prediction_type: The type of the task to train the model for.\n",
        "  - classification: A tabuar classification model.\n",
        "  - regression: A tabular regression model.\n",
        "- column_transformations: (Optional): Transformations to apply to the input columns\n",
        "- optimization_objective: The optimization objective to minimize or maximize.\n",
        "  - binary classification:\n",
        "    - minimize-log-loss\n",
        "    - maximize-au-roc\n",
        "    - maximize-au-prc\n",
        "    - maximize-precision-at-recall\n",
        "    - maximize-recall-at-precision\n",
        "  - multi-class classification:\n",
        "    - minimize-log-loss\n",
        "  - regression:\n",
        "    - minimize-rmse\n",
        "    - minimize-mae\n",
        "    - minimize-rmsle\n",
        "\n",
        "The instantiated object is the DAG (directed acyclic graph) for the training pipeline."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "create_automl_pipeline:tabular,lbn"
      },
      "outputs": [],
      "source": [
        "job = aiplatform.AutoMLTabularTrainingJob(\n",
        "    display_name=\"bank_\" + \"unique\",\n",
        "    optimization_prediction_type=\"classification\",\n",
        "    optimization_objective=\"minimize-log-loss\",\n",
        ")\n",
        "\n",
        "print(job)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_automl_pipeline:tabular,lbn"
      },
      "source": [
        "*Example output:*\n",
        "\n",
        "    <google.cloud.aiplatform.training_jobs.AutoMLTabularTrainingJob object at 0x7fc3b6c90f10>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "run_automl_pipeline:tabular"
      },
      "source": [
        "#### Run the training pipeline\n",
        "\n",
        "Next, you run the DAG to start the training job by invoking the method `run`, with the following parameters:\n",
        "\n",
        "- dataset: The dataset resource to train the model.\n",
        "- model_display_name: The human readable name for the trained model.\n",
        "- training_fraction_split: The percentage of the dataset to use for training.\n",
        "- test_fraction_split: The percentage of the dataset to use for test (holdout data).\n",
        "- validation_fraction_split: The percentage of the dataset to use for validation.\n",
        "- target_column: The name of the column to train as the label.\n",
        "- budget_milli_node_hours: (optional) Maximum training time specified in unit of millihours (1000 = hour).\n",
        "- disable_early_stopping: If `True`, training maybe completed before using the entire budget if the service believes it can't further improve on the model objective measurements.\n",
        "\n",
        "The `run` method when completed returns the `Model` resource.\n",
        "\n",
        "The execution of the training pipeline takes upto 20 minutes."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "run_automl_pipeline:tabular"
      },
      "outputs": [],
      "source": [
        "model = job.run(\n",
        "    dataset=dataset,\n",
        "    model_display_name=\"bank_\" + \"unique\",\n",
        "    training_fraction_split=0.6,\n",
        "    validation_fraction_split=0.2,\n",
        "    test_fraction_split=0.2,\n",
        "    budget_milli_node_hours=1000,\n",
        "    disable_early_stopping=False,\n",
        "    target_column=label_column,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "run_automl_pipeline:tabular"
      },
      "source": [
        "*Example output:*\n",
        "\n",
        "    INFO:google.cloud.aiplatform.training_jobs:No column transformations provided, so now retrieving columns from dataset in order to set default column transformations.\n",
        "    INFO:google.cloud.aiplatform.training_jobs:The column transformation of type 'auto' was set for the following columns: ['Day', 'Month', 'MaritalStatus', 'Job', 'Balance', 'Housing', 'Contact', 'Duration', 'Education', 'Previous', 'Loan', 'PDays', 'POutcome', 'Age', 'Default', 'Campaign'].\n",
        "    INFO:google.cloud.aiplatform.training_jobs:View Training:\n",
        "    https://console.cloud.google.com/ai/platform/locations/us-central1/training/4626265542611697664?project=759209241365\n",
        "    INFO:google.cloud.aiplatform.training_jobs:AutoMLTabularTrainingJob projects/759209241365/locations/us-central1/trainingPipelines/4626265542611697664 current state:\n",
        "    PipelineState.PIPELINE_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.training_jobs:AutoMLTabularTrainingJob projects/759209241365/locations/us-central1/trainingPipelines/4626265542611697664 current state:\n",
        "    PipelineState.PIPELINE_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.training_jobs:AutoMLTabularTrainingJob projects/759209241365/locations/us-central1/trainingPipelines/4626265542611697664 current state:\n",
        "    PipelineState.PIPELINE_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.training_jobs:AutoMLTabularTrainingJob projects/759209241365/locations/us-central1/trainingPipelines/4626265542611697664 current state:\n",
        "    PipelineState.PIPELINE_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.training_jobs:AutoMLTabularTrainingJob projects/759209241365/locations/us-central1/trainingPipelines/4626265542611697664 current state:\n",
        "    PipelineState.PIPELINE_STATE_RUNNING\n",
        "    ...\n",
        "    INFO:google.cloud.aiplatform.training_jobs:AutoMLTabularTrainingJob run completed. Resource name: projects/759209241365/locations/us-central1/trainingPipelines/3915928254510268416\n",
        "    INFO:google.cloud.aiplatform.training_jobs:Model available at projects/759209241365/locations/us-central1/models/6324223757200130048"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "evaluate_the_model:migration"
      },
      "source": [
        "## Evaluate the model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "models_evaluations_list:migration,new"
      },
      "source": [
        "### [projects.locations.models.evaluations.list](https://cloud.devsite.corp.google.com/ai-platform-unified/docs/reference/rest/v1beta1/projects.locations.models.evaluations/list)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "evaluate_the_model:mbsdk"
      },
      "source": [
        "## Review model evaluation scores\n",
        "After your model has finished training, you can review the evaluation scores for it.\n",
        "\n",
        "First, you need to get a reference to the new model. As with datasets, you can either use the reference to the model variable you created when you deployed the model or you can list all of the models in your project."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "evaluate_the_model:mbsdk"
      },
      "outputs": [],
      "source": [
        "# Get model resource ID\n",
        "models = aiplatform.Model.list(filter=\"display_name=bank_\" + \"unique\")\n",
        "\n",
        "# Get a reference to the Model Service client\n",
        "client_options = {\"api_endpoint\": f\"{LOCATION}-aiplatform.googleapis.com\"}\n",
        "model_service_client = aiplatform.gapic.ModelServiceClient(\n",
        "    client_options=client_options\n",
        ")\n",
        "\n",
        "model_evaluations = model_service_client.list_model_evaluations(\n",
        "    parent=models[0].resource_name\n",
        ")\n",
        "model_evaluation = list(model_evaluations)[0]\n",
        "print(model_evaluation)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "evaluate_the_model:mbsdk"
      },
      "source": [
        "*Example output:*\n",
        "\n",
        "    name: \"projects/759209241365/locations/us-central1/models/623915674158235648/evaluations/4280507618583117824\"\n",
        "    metrics_schema_uri: \"gs://google-cloud-aiplatform/schema/modelevaluation/classification_metrics_1.0.0.yaml\"\n",
        "    metrics {\n",
        "      struct_value {\n",
        "        fields {\n",
        "          key: \"auPrc\"\n",
        "          value {\n",
        "            number_value: 0.9891107\n",
        "          }\n",
        "        }\n",
        "        fields {\n",
        "          key: \"confidenceMetrics\"\n",
        "          value {\n",
        "            list_value {\n",
        "              values {\n",
        "                struct_value {\n",
        "                  fields {\n",
        "                    key: \"precision\"\n",
        "                    value {\n",
        "                      number_value: 0.2\n",
        "                    }\n",
        "                  }\n",
        "                  fields {\n",
        "                    key: \"recall\"\n",
        "                    value {\n",
        "                      number_value: 1.0\n",
        "                    }\n",
        "                  }\n",
        "                }\n",
        "              }"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "make_batch_predictions:migration"
      },
      "source": [
        "## Make batch predictions"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "batchpredictionjobs_create:migration,new,mbsdk"
      },
      "source": [
        "### [predictions.batch-prediction](https://cloud.google.com/vertex-ai/docs/predictions/batch-predictions)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "make_test_items:automl,batch_prediction"
      },
      "source": [
        "### Make test items\n",
        "\n",
        "You use synthetic data as a test data items. Don't be concerned that you are using synthetic data -- you just want to demonstrate how to make a prediction."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "make_batch_file:automl,tabular,alt"
      },
      "source": [
        "### Make the batch input file\n",
        "\n",
        "Now make a batch input file, which you store in your local Cloud Storage bucket. Unlike image, video and text, the batch input file for tabular is only supported for CSV. For CSV file, you make:\n",
        "\n",
        "- The first line is the heading with the feature (fields) heading names.\n",
        "- Each remaining line is a separate prediction request with the corresponding feature values.\n",
        "\n",
        "For example:\n",
        "\n",
        "    \"feature_1\", \"feature_2\". ...\n",
        "    value_1, value_2, ..."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "make_batch_file:automl,tabular,alt"
      },
      "outputs": [],
      "source": [
        "! gsutil cat $IMPORT_FILE | head -n 1 > tmp.csv\n",
        "! gsutil cat $IMPORT_FILE | tail -n 10 >> tmp.csv\n",
        "\n",
        "! cut -d, -f1-16 tmp.csv > batch.csv\n",
        "\n",
        "gcs_input_uri = BUCKET_URI + \"/test.csv\"\n",
        "\n",
        "! gsutil cp batch.csv $gcs_input_uri"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "batch_request:mbsdk,both_csv"
      },
      "source": [
        "### Make the batch prediction request\n",
        "\n",
        "Now that your Model resource is trained, you can make a batch prediction by invoking the `batch_predict()` method, with the following parameters:\n",
        "\n",
        "- `job_display_name`: The human readable name for the batch prediction job.\n",
        "- `gcs_source`: A list of one or more batch request input files.\n",
        "- `gcs_destination_prefix`: The Cloud Storage location for storing the batch prediction resuls.\n",
        "- `instances_format`: The format for the input instances, either 'csv' or 'jsonl'. Defaults to 'jsonl'.\n",
        "- `predictions_format`: The format for the output predictions, either 'csv' or 'jsonl'. Defaults to 'jsonl'.\n",
        "- `sync`: If set to True, the call will block while waiting for the asynchronous batch job to complete."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "batch_request:mbsdk,both_csv"
      },
      "outputs": [],
      "source": [
        "batch_predict_job = model.batch_predict(\n",
        "    job_display_name=\"bank_\" + \"unique\",\n",
        "    gcs_source=gcs_input_uri,\n",
        "    gcs_destination_prefix=BUCKET_URI,\n",
        "    instances_format=\"csv\",\n",
        "    predictions_format=\"csv\",\n",
        "    sync=False,\n",
        ")\n",
        "\n",
        "print(batch_predict_job)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "batch_request:mbsdk,both_csv"
      },
      "source": [
        "*Example output:*\n",
        "\n",
        "    INFO:google.cloud.aiplatform.jobs:Creating BatchPredictionJob\n",
        "    <google.cloud.aiplatform.jobs.BatchPredictionJob object at 0x7f806a6112d0> is waiting for upstream dependencies to complete.\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob created. Resource name: projects/759209241365/locations/us-central1/batchPredictionJobs/5110965452507447296\n",
        "    INFO:google.cloud.aiplatform.jobs:To use this BatchPredictionJob in another session:\n",
        "    INFO:google.cloud.aiplatform.jobs:bpj = aiplatform.BatchPredictionJob('projects/759209241365/locations/us-central1/batchPredictionJobs/5110965452507447296')\n",
        "    INFO:google.cloud.aiplatform.jobs:View Batch Prediction Job:\n",
        "    https://console.cloud.google.com/ai/platform/locations/us-central1/batch-predictions/5110965452507447296?project=759209241365\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob projects/759209241365/locations/us-central1/batchPredictionJobs/5110965452507447296 current state:\n",
        "    JobState.JOB_STATE_RUNNING"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "batch_request_wait:mbsdk"
      },
      "source": [
        "### Wait for completion of batch prediction job\n",
        "\n",
        "Next, wait for the batch job to complete. Alternatively, one can set the parameter `sync` to `True` in the `batch_predict()` method to block until the batch prediction job is completed."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "batch_request_wait:mbsdk"
      },
      "outputs": [],
      "source": [
        "batch_predict_job.wait()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "batch_request_wait:mbsdk"
      },
      "source": [
        "*Example Output:*\n",
        "\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob created. Resource name: projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328\n",
        "    INFO:google.cloud.aiplatform.jobs:To use this BatchPredictionJob in another session:\n",
        "    INFO:google.cloud.aiplatform.jobs:bpj = aiplatform.BatchPredictionJob('projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328')\n",
        "    INFO:google.cloud.aiplatform.jobs:View Batch Prediction Job:\n",
        "    https://console.cloud.google.com/ai/platform/locations/us-central1/batch-predictions/181835033978339328?project=759209241365\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328 current state:\n",
        "    JobState.JOB_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328 current state:\n",
        "    JobState.JOB_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328 current state:\n",
        "    JobState.JOB_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328 current state:\n",
        "    JobState.JOB_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328 current state:\n",
        "    JobState.JOB_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328 current state:\n",
        "    JobState.JOB_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328 current state:\n",
        "    JobState.JOB_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328 current state:\n",
        "    JobState.JOB_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328 current state:\n",
        "    JobState.JOB_STATE_SUCCEEDED\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob run completed. Resource name: projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "get_batch_prediction:mbsdk,lbn"
      },
      "source": [
        "### Get the predictions\n",
        "\n",
        "Next, get the results from the completed batch prediction job.\n",
        "\n",
        "The results are written to the Cloud Storage output bucket you specified in the batch prediction request. You call the method iter_outputs() to get a list of each Cloud Storage file generated with the results. Each file contains one or more prediction requests in a CSV format:\n",
        "\n",
        "- CSV header + predicted_label\n",
        "- CSV row + prediction, per prediction request"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "get_batch_prediction:mbsdk,lbn"
      },
      "outputs": [],
      "source": [
        "bp_iter_outputs = batch_predict_job.iter_outputs()\n",
        "\n",
        "prediction_results = list()\n",
        "for blob in bp_iter_outputs:\n",
        "    if blob.name.split(\"/\")[-1].startswith(\"prediction.results\"):\n",
        "        prediction_results.append(blob.name)\n",
        "\n",
        "for prediction_result in prediction_results:\n",
        "    gfile_name = f\"gs://{bp_iter_outputs.bucket.name}/{prediction_result}\"\n",
        "    df = pd.read_csv(gfile_name)\n",
        "    print(f\"File name: {gfile_name}\")\n",
        "    print(\"Prediction: \\n\\n\\n\\n\")\n",
        "    print(df)\n",
        "    print(\"\\n\\n\\n\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "get_batch_prediction:mbsdk,lbn"
      },
      "source": [
        "*Example Output:*\n",
        "\n",
        "    File name: gs://vertex-ai-devaip-5j22pmou/prediction-bank_5j22pmou-2022_08_24T01_05_46_028Z/prediction.results-00005-of-00008.csv\n",
        "Prediction: \n",
        "\n",
        "\n",
        "\n",
        "\n",
        "    Age          Job MaritalStatus  Education Default  Balance Housing Loan  \\\n",
        "0    57  blue-collar       married  secondary      no      668      no   no   \n",
        "\n",
        "     Contact  Day Month  Duration  Campaign  PDays  Previous POutcome  \\\n",
        "0  telephone   17   nov       508         4     -1         0  unknown   \n",
        "\n",
        "    Deposit_1_scores  Deposit_2_scores  \n",
        "0          0.847498          0.152502  "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "make_online_predictions:migration"
      },
      "source": [
        "## Make online predictions"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "deploy_model:migration,new,mbsdk"
      },
      "source": [
        "### [predictions.deploy-model-api](https://cloud.google.com/vertex-ai/docs/predictions/deploy-model-api)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "deploy_model:mbsdk,dedicated"
      },
      "source": [
        "## Deploy the model\n",
        "\n",
        "Next, deploy your model for online prediction. To deploy the model, you invoke the `deploy` method, with the following parameters:\n",
        "\n",
        "- `machine_type`: The type of compute machine."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "deploy_model:mbsdk,dedicated"
      },
      "outputs": [],
      "source": [
        "endpoint = model.deploy(machine_type=\"n1-standard-4\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "deploy_model:mbsdk,dedicated"
      },
      "source": [
        "*Example output:*\n",
        "\n",
        "    INFO:google.cloud.aiplatform.models:Creating Endpoint\n",
        "    INFO:google.cloud.aiplatform.models:Create Endpoint backing LRO: projects/759209241365/locations/us-central1/endpoints/4867177336350441472/operations/4087251132693348352\n",
        "    INFO:google.cloud.aiplatform.models:Endpoint created. Resource name: projects/759209241365/locations/us-central1/endpoints/4867177336350441472\n",
        "    INFO:google.cloud.aiplatform.models:To use this Endpoint in another session:\n",
        "    INFO:google.cloud.aiplatform.models:endpoint = aiplatform.Endpoint('projects/759209241365/locations/us-central1/endpoints/4867177336350441472')\n",
        "    INFO:google.cloud.aiplatform.models:Deploying model to Endpoint : projects/759209241365/locations/us-central1/endpoints/4867177336350441472\n",
        "    INFO:google.cloud.aiplatform.models:Deploy Endpoint model backing LRO: projects/759209241365/locations/us-central1/endpoints/4867177336350441472/operations/1691336130932244480\n",
        "    INFO:google.cloud.aiplatform.models:Endpoint model deployed. Resource name: projects/759209241365/locations/us-central1/endpoints/4867177336350441472"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "endpoints_predict:migration,new,mbsdk"
      },
      "source": [
        "### [predictions.online-prediction-automl](https://cloud.google.com/vertex-ai/docs/predictions/online-predictions-automl)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "make_test_item:automl,online_prediction"
      },
      "source": [
        "### Make test item\n",
        "\n",
        "You use synthetic data as a test data item. Don't be concerned that you are using synthetic data -- you just want to demonstrate how to make a prediction."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "make_test_item:automl,tabular,bank"
      },
      "outputs": [],
      "source": [
        "INSTANCE = {\n",
        "    \"Age\": \"58\",\n",
        "    \"Job\": \"managment\",\n",
        "    \"MaritalStatus\": \"married\",\n",
        "    \"Education\": \"teritary\",\n",
        "    \"Default\": \"no\",\n",
        "    \"Balance\": \"2143\",\n",
        "    \"Housing\": \"yes\",\n",
        "    \"Loan\": \"no\",\n",
        "    \"Contact\": \"unknown\",\n",
        "    \"Day\": \"5\",\n",
        "    \"Month\": \"may\",\n",
        "    \"Duration\": \"261\",\n",
        "    \"Campaign\": \"1\",\n",
        "    \"PDays\": \"-1\",\n",
        "    \"Previous\": \"0\",\n",
        "    \"POutcome\": \"unknown\",\n",
        "}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "predict_request:mbsdk,lbn"
      },
      "source": [
        "### Make the prediction\n",
        "\n",
        "Now that your `Model` resource is deployed to an `Endpoint` resource, you can do online predictions by sending prediction requests to the `Endpoint` resource.\n",
        "\n",
        "#### Request\n",
        "\n",
        "The format of each instance is:\n",
        "\n",
        "    [feature_list]\n",
        "\n",
        "Since the predict() method can take multiple items (instances), send your single test item as a list of one test item.\n",
        "\n",
        "#### Response\n",
        "\n",
        "The response from the predict() call is a Python dictionary with the following entries:\n",
        "\n",
        "- `ids`: The internal assigned unique identifiers for each prediction request.\n",
        "- `displayNames`: The class names for each class label.\n",
        "- `confidences`: The predicted confidence, between 0 and 1, per class label.\n",
        "- `deployed_model_id`: The Vertex AI identifier for the deployed model resource which did the predictions."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "predict_request:mbsdk,lbn"
      },
      "outputs": [],
      "source": [
        "instances_list = [INSTANCE]\n",
        "\n",
        "prediction = endpoint.predict(instances_list)\n",
        "print(prediction)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "predict_request:mbsdk,lbn"
      },
      "source": [
        "*Example output:*\n",
        "\n",
        "    Prediction(predictions=[{'classes': ['1', '2'], 'scores': [0.9966018199920654, 0.003398159053176641]}], deployed_model_id='6741053013336522752', explanations=None)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "undeploy_model:mbsdk"
      },
      "source": [
        "## Undeploy the model\n",
        "\n",
        "When you are done doing predictions, undeploy the model from the `Endpoint` resouce. This deprovisions all compute resources and ends billing for the deployed model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "undeploy_model:mbsdk"
      },
      "outputs": [],
      "source": [
        "endpoint.undeploy_all()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cleanup:mbsdk"
      },
      "source": [
        "# Cleaning up\n",
        "\n",
        "To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n",
        "project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n",
        "\n",
        "Otherwise, you can delete the individual resources you created in this tutorial:\n",
        "\n",
        "- Dataset\n",
        "- Model\n",
        "- Endpoint\n",
        "- AutoML Training Job\n",
        "- Batch Job\n",
        "- Cloud Storage Bucket"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cleanup:mbsdk"
      },
      "outputs": [],
      "source": [
        "# Delete the dataset using the Vertex dataset object\n",
        "dataset.delete()\n",
        "\n",
        "# Delete the model using the Vertex model object\n",
        "model.delete()\n",
        "\n",
        "# Delete the endpoint using the Vertex endpoint object\n",
        "endpoint.delete()\n",
        "\n",
        "# Delete the AutoML or Pipeline trainig job\n",
        "job.delete()\n",
        "\n",
        "# Delete the batch prediction job using the Vertex batch prediction object\n",
        "batch_predict_job.delete()\n",
        "\n",
        "# remove locally generated files\n",
        "! rm -r batch.csv\n",
        "! rm -r tmp.csv\n",
        "\n",
        "delete_bucket = False\n",
        "if delete_bucket:\n",
        "    ! gsutil -m rm -r $BUCKET_URI"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "sdk-automl-tabular-binary-classification-online-prediction.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}