notebooks/official/migration/sdk-custom-xgboost-prebuilt-container.ipynb

{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "copyright"
      },
      "outputs": [],
      "source": [
        "# Copyright 2021 Google LLC\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "title:migration,new"
      },
      "source": [
        "# Vertex AI: Vertex AI Migration: Custom XGBoost model with pre-built training container\n",
        "\n",
        "<table align=\"left\">\n",
        "  <td>\n",
        "<a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-custom-xgboost-prebuilt-container.ipynb\" target='_blank'>\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
        "<a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-custom-xgboost-prebuilt-container.ipynb\" target='_blank'>\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
        "      View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "     <td>\n",
        "<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/migration/sdk-custom-xgboost-prebuilt-container.ipynb\" target='_blank'>\n",
        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
        "      Open in Vertex AI Workbench\n",
        "    </a>\n",
        "  </td>                      \n",
        "</table>\n",
        "<br/><br/><br/>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7a8a13b86a8b"
      },
      "source": [
        "## Overview\n",
        "\n",
        "\n",
        "This tutorial demonstrates how to use the Vertex AI SDK for Python to train and deploy a custom tabular classification XGBoost model for batch prediction.\n",
        "\n",
        "Learn more about [Migrate to Vertex AI](https://cloud.google.com/vertex-ai/docs/start/migrating-to-vertex-ai) and [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "618cfedf829a"
      },
      "source": [
        "### Objective\n",
        "\n",
        "In this tutorial, you learn to use `Vertex AI Training` to create a custom trained model and use `Vertex AI Batch Prediction` to do a batch prediction on the trained model.\n",
        "\n",
        "\n",
        "You learn how to create a custom-trained model from a Python script in a Docker container using the Vertex AI SDK for Python, and then do a prediction on the deployed model by sending data.\n",
        "\n",
        "This tutorial uses the following Google Cloud ML services:\n",
        "\n",
        "- `Vertex AI Training`\n",
        "- `Vertex AI Batch Prediction`\n",
        "- `Vertex AI Model` resource\n",
        "- `Vertex AI Endpoint` resource\n",
        "\n",
        "The steps performed include:\n",
        "\n",
        "- Create a `Vertex AI` custom job for training a scikit-learn model.\n",
        "- Upload the trained model artifacts as a `Model` resource.\n",
        "- Make a batch prediction.\n",
        "- Deploy model to a endpoint\n",
        "- Make a online prediction"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dataset:iris,lcn"
      },
      "source": [
        "### Dataset\n",
        "\n",
        "The dataset used for this tutorial is the [Iris dataset](https://www.tensorflow.org/datasets/catalog/iris) from [TensorFlow Datasets](https://www.tensorflow.org/datasets/catalog/overview). This dataset does not require any feature engineering. The version of the dataset you will use in this tutorial is stored in a public Cloud Storage bucket. The trained model predicts the type of Iris flower species from a class of three species: setosa, virginica, or versicolor."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "costs"
      },
      "source": [
        "### Costs\n",
        "\n",
        "This tutorial uses billable components of Google Cloud:\n",
        "\n",
        "* Vertex AI\n",
        "* Cloud Storage\n",
        "\n",
        "Learn about [Vertex AI\n",
        "pricing](https://cloud.google.com/vertex-ai/pricing) and [Cloud Storage\n",
        "pricing](https://cloud.google.com/storage/pricing), and use the [Pricing\n",
        "Calculator](https://cloud.google.com/products/calculator/)\n",
        "to generate a cost estimate based on your projected usage."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "install_aip:mbsdk"
      },
      "source": [
        "## Installation\n",
        "\n",
        "Install the packages required for executing this notebook."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "install_aip:mbsdk"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "\n",
        "! pip3 install --upgrade --quiet google-cloud-aiplatform \\\n",
        "                                 google-cloud-storage\n",
        "\n",
        "if os.getenv(\"IS_TESTING\"):\n",
        "    ! pip3 install --upgrade --quiet tensorflow "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "restart"
      },
      "source": [
        "### Colab only: Uncomment the following cell to restart the kernel"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "D-ZBOjErv5mM"
      },
      "outputs": [],
      "source": [
        "# Automatically restart kernel after installs so that your environment can access the new packages\n",
        "# import IPython\n",
        "\n",
        "# app = IPython.Application.instance()\n",
        "# app.kernel.do_shutdown(True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "before_you_begin:nogpu"
      },
      "source": [
        "## Before you begin"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "before_you_begin:nogpu"
      },
      "source": [
        "### Set your project ID\n",
        "\n",
        "**If you don't know your project ID**, try the following:\n",
        "* Run `gcloud config list`.\n",
        "* Run `gcloud projects list`.\n",
        "* See the support page: [Locate the project ID](https://support.google.com/googleapi/answer/7014113)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "set_project_id"
      },
      "outputs": [],
      "source": [
        "PROJECT_ID = \"[your-project-id]\"  # @param {type:\"string\"}\n",
        "\n",
        "# Set the project id\n",
        "! gcloud config set project {PROJECT_ID}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "region"
      },
      "source": [
        "#### Region\n",
        "\n",
        "You can also change the `REGION` variable used by Vertex AI. Learn more about [Vertex AI regions](https://cloud.google.com/vertex-ai/docs/general/locations)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "region"
      },
      "outputs": [],
      "source": [
        "REGION = \"us-central1\"  # @param {type: \"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "timestamp"
      },
      "source": [
        "#### UUID\n",
        "\n",
        "If you are in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a uuid for each instance session, and append it onto the name of resources you create in this tutorial."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "e87d5856317d"
      },
      "outputs": [],
      "source": [
        "import random\n",
        "import string\n",
        "\n",
        "\n",
        "# Generate a uuid of a specifed length(default=8)\n",
        "def generate_uuid(length: int = 8) -> str:\n",
        "    return \"\".join(random.choices(string.ascii_lowercase + string.digits, k=length))\n",
        "\n",
        "\n",
        "UUID = generate_uuid()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gcp_authenticate"
      },
      "source": [
        "### Authenticate your Google Cloud account\n",
        "\n",
        "Depending on your Jupyter environment, you may have to manually authenticate. Follow the relevant instructions below.\n",
        "\n",
        "**1. Vertex AI Workbench**\n",
        "* Do nothing as you are already authenticated.\n",
        "\n",
        "**2. Local JupyterLab instance, uncomment and run:**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ce6043da7b33"
      },
      "outputs": [],
      "source": [
        "# ! gcloud auth login"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0367eac06a10"
      },
      "source": [
        "**3. Colab, uncomment and run:**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "21ad4dbb4a61"
      },
      "outputs": [],
      "source": [
        "# from google.colab import auth\n",
        "# auth.authenticate_user()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c13224697bfb"
      },
      "source": [
        "**4. Service account or other**\n",
        "* See how to grant Cloud Storage permissions to your service account at https://cloud.google.com/storage/docs/gsutil/commands/iam#ch-examples."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bucket:mbsdk"
      },
      "source": [
        "### Create a Cloud Storage bucket\n",
        "\n",
        "Create a storage bucket to store intermediate artifacts such as datasets."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bucket"
      },
      "outputs": [],
      "source": [
        "BUCKET_URI = f\"gs://your-bucket-name-{PROJECT_ID}-unique\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_bucket"
      },
      "source": [
        "**Only if your bucket doesn't already exist**: Run the following cell to create your Cloud Storage bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "create_bucket"
      },
      "outputs": [],
      "source": [
        "! gsutil mb -l {REGION} {BUCKET_URI}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "setup_vars"
      },
      "source": [
        "### Set up variables\n",
        "\n",
        "Next, set up some variables used throughout the tutorial.\n",
        "### Import libraries and define constants"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "import_aip:mbsdk"
      },
      "outputs": [],
      "source": [
        "import google.cloud.aiplatform as aip"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "init_aip:mbsdk"
      },
      "source": [
        "## Initialize Vertex AI SDK for Python\n",
        "\n",
        "Initialize the Vertex AI SDK for Python for your project and corresponding bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "init_aip:mbsdk"
      },
      "outputs": [],
      "source": [
        "aip.init(project=PROJECT_ID, staging_bucket=BUCKET_URI)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "container:training,prediction,xgboost"
      },
      "source": [
        "#### Set pre-built containers\n",
        "\n",
        "Set the pre-built Docker container image for training and prediction.\n",
        "\n",
        "\n",
        "For the latest list, see [Pre-built containers for training](https://cloud.google.com/ai-platform-unified/docs/training/pre-built-containers).\n",
        "\n",
        "\n",
        "For the latest list, see [Pre-built containers for prediction](https://cloud.google.com/ai-platform-unified/docs/predictions/pre-built-containers)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "container:training,prediction,xgboost"
      },
      "outputs": [],
      "source": [
        "TRAIN_VERSION = \"xgboost-cpu.1-1\"\n",
        "DEPLOY_VERSION = \"xgboost-cpu.1-1\"\n",
        "\n",
        "TRAIN_IMAGE = \"gcr.io/cloud-aiplatform/training/{}:latest\".format(TRAIN_VERSION)\n",
        "DEPLOY_IMAGE = \"gcr.io/cloud-aiplatform/prediction/{}:latest\".format(DEPLOY_VERSION)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "machine:training,prediction"
      },
      "source": [
        "#### Set machine type\n",
        "\n",
        "Next, set the machine type to use for training and prediction.\n",
        "\n",
        "- Set the variables `TRAIN_COMPUTE` and `DEPLOY_COMPUTE` to configure  the compute resources for the VMs you will use for for training and prediction.\n",
        " - `machine type`\n",
        "     - `n1-standard`: 3.75GB of memory per vCPU.\n",
        "     - `n1-highmem`: 6.5GB of memory per vCPU\n",
        "     - `n1-highcpu`: 0.9 GB of memory per vCPU\n",
        " - `vCPUs`: number of \\[2, 4, 8, 16, 32, 64, 96 \\]\n",
        "\n",
        "*Note: The following is not supported for training:*\n",
        "\n",
        " - `standard`: 2 vCPUs\n",
        " - `highcpu`: 2, 4 and 8 vCPUs\n",
        "\n",
        "*Note: You may also use n2 and e2 machine types for training and deployment, but they do not support GPUs*."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "machine:training,prediction"
      },
      "outputs": [],
      "source": [
        "if os.getenv(\"IS_TESTING_TRAIN_MACHINE\"):\n",
        "    MACHINE_TYPE = os.getenv(\"IS_TESTING_TRAIN_MACHINE\")\n",
        "else:\n",
        "    MACHINE_TYPE = \"n1-standard\"\n",
        "\n",
        "VCPU = \"4\"\n",
        "TRAIN_COMPUTE = MACHINE_TYPE + \"-\" + VCPU\n",
        "print(\"Train machine type\", TRAIN_COMPUTE)\n",
        "\n",
        "if os.getenv(\"IS_TESTING_DEPLOY_MACHINE\"):\n",
        "    MACHINE_TYPE = os.getenv(\"IS_TESTING_DEPLOY_MACHINE\")\n",
        "else:\n",
        "    MACHINE_TYPE = \"n1-standard\"\n",
        "\n",
        "VCPU = \"4\"\n",
        "DEPLOY_COMPUTE = MACHINE_TYPE + \"-\" + VCPU\n",
        "print(\"Deploy machine type\", DEPLOY_COMPUTE)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "examine_training_package"
      },
      "source": [
        "### Examine the training package\n",
        "\n",
        "#### Package layout\n",
        "\n",
        "Before you start the training, you will look at how a Python package is assembled for a custom training job. When unarchived, the package contains the following directory/file layout.\n",
        "\n",
        "- PKG-INFO\n",
        "- README.md\n",
        "- setup.cfg\n",
        "- setup.py\n",
        "- trainer\n",
        "  - \\_\\_init\\_\\_.py\n",
        "  - task.py\n",
        "\n",
        "The files `setup.cfg` and `setup.py` are the instructions for installing the package into the operating environment of the Docker image.\n",
        "\n",
        "The file `trainer/task.py` is the Python script for executing the custom training job. *Note*, when we referred to it in the worker pool specification, we replace the directory slash with a dot (`trainer.task`) and dropped the file suffix (`.py`).\n",
        "\n",
        "#### Package Assembly\n",
        "\n",
        "In the following cells, you will assemble the training package."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "examine_training_package"
      },
      "outputs": [],
      "source": [
        "# Make folder for Python training script\n",
        "! rm -rf custom\n",
        "! mkdir custom\n",
        "\n",
        "# Add package information\n",
        "! touch custom/README.md\n",
        "\n",
        "setup_cfg = \"[egg_info]\\n\\ntag_build =\\n\\ntag_date = 0\"\n",
        "! echo \"$setup_cfg\" > custom/setup.cfg\n",
        "\n",
        "setup_py = \"import setuptools\\n\\nsetuptools.setup(\\n\\n    install_requires=[\\n\\n        'tensorflow_datasets==1.3.0',\\n\\n    ],\\n\\n    packages=setuptools.find_packages())\"\n",
        "! echo \"$setup_py\" > custom/setup.py\n",
        "\n",
        "pkg_info = \"Metadata-Version: 1.0\\n\\nName: Iris tabular classification\\n\\nVersion: 0.0.0\\n\\nSummary: Demostration training script\\n\\nHome-page: www.google.com\\n\\nAuthor: Google\\n\\nAuthor-email: aferlitsch@google.com\\n\\nLicense: Public\\n\\nDescription: Demo\\n\\nPlatform: Vertex\"\n",
        "! echo \"$pkg_info\" > custom/PKG-INFO\n",
        "\n",
        "# Make the training subfolder\n",
        "! mkdir custom/trainer\n",
        "! touch custom/trainer/__init__.py"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "taskpy_contents:xgboost,iris"
      },
      "outputs": [],
      "source": [
        "%%writefile custom/trainer/task.py\n",
        "# Single Instance Training for Iris\n",
        "\n",
        "import datetime\n",
        "import os\n",
        "import subprocess\n",
        "import sys\n",
        "import pandas as pd\n",
        "import xgboost as xgb\n",
        "\n",
        "import argparse\n",
        "\n",
        "parser = argparse.ArgumentParser()\n",
        "parser.add_argument('--model-dir', dest='model_dir',\n",
        "                    default=os.getenv('AIP_MODEL_DIR'), type=str, help='Model dir.')\n",
        "args = parser.parse_args()\n",
        "\n",
        "# Download data\n",
        "iris_data_filename = 'iris_data.csv'\n",
        "iris_target_filename = 'iris_target.csv'\n",
        "data_dir = 'gs://cloud-samples-data/ai-platform/iris'\n",
        "\n",
        "# gsutil outputs everything to stderr so we need to divert it to stdout.\n",
        "subprocess.check_call(['gsutil', 'cp', os.path.join(data_dir,\n",
        "                                                    iris_data_filename),\n",
        "                       iris_data_filename], stderr=sys.stdout)\n",
        "subprocess.check_call(['gsutil', 'cp', os.path.join(data_dir,\n",
        "                                                    iris_target_filename),\n",
        "                       iris_target_filename], stderr=sys.stdout)\n",
        "\n",
        "\n",
        "# Load data into pandas, then use `.values` to get NumPy arrays\n",
        "iris_data = pd.read_csv(iris_data_filename).values\n",
        "iris_target = pd.read_csv(iris_target_filename).values\n",
        "\n",
        "# Convert one-column 2D array into 1D array for use with XGBoost\n",
        "iris_target = iris_target.reshape((iris_target.size,))\n",
        "\n",
        "\n",
        "# Load data into DMatrix object\n",
        "dtrain = xgb.DMatrix(iris_data, label=iris_target)\n",
        "\n",
        "\n",
        "# Train XGBoost model\n",
        "bst = xgb.train({}, dtrain, 20)\n",
        "\n",
        "# Export the classifier to a file\n",
        "model_filename = 'model.bst'\n",
        "bst.save_model(model_filename)\n",
        "\n",
        "# Upload the saved model file to Cloud Storage\n",
        "gcs_model_path = os.path.join(args.model_dir, model_filename)\n",
        "subprocess.check_call(['gsutil', 'cp', model_filename, gcs_model_path],\n",
        "    stderr=sys.stdout)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tarball_training_script"
      },
      "source": [
        "#### Store training script on your Cloud Storage bucket\n",
        "\n",
        "Next, you package the training folder into a compressed tar ball, and then store it in your Cloud Storage bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "tarball_training_script"
      },
      "outputs": [],
      "source": [
        "! rm -f custom.tar custom.tar.gz\n",
        "! tar cvf custom.tar custom\n",
        "! gzip custom.tar\n",
        "! gsutil cp custom.tar.gz $BUCKET_URI/trainer_iris.tar.gz"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "train_a_model:migration"
      },
      "source": [
        "## Train a model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "custom_create:migration,new,mbsdk,prebuilt"
      },
      "source": [
        "### [training.create-python-pre-built-container](https://cloud.google.com/vertex-ai/docs/training/create-python-pre-built-container)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_custom_training_job:mbsdk,no_model"
      },
      "source": [
        "### Create and run custom training job\n",
        "\n",
        "\n",
        "To train a custom model, you perform two steps: 1) create a custom training job, and 2) run the job.\n",
        "\n",
        "#### Create custom training job\n",
        "\n",
        "A custom training job is created with the `CustomTrainingJob` class, with the following parameters:\n",
        "\n",
        "- `display_name`: The human readable name for the custom training job.\n",
        "- `container_uri`: The training container image.\n",
        "- `requirements`: Package requirements for the training container image (e.g., pandas).\n",
        "- `script_path`: The relative path to the training script."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "create_custom_training_job:mbsdk,no_model"
      },
      "outputs": [],
      "source": [
        "job = aip.CustomTrainingJob(\n",
        "    display_name=\"iris_\" + UUID,\n",
        "    script_path=\"custom/trainer/task.py\",\n",
        "    container_uri=TRAIN_IMAGE,\n",
        "    requirements=[\"gcsfs==0.7.1\", \"tensorflow-datasets==4.4\"],\n",
        ")\n",
        "\n",
        "print(job)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_custom_training_job:mbsdk,no_model"
      },
      "source": [
        "*Example output:*\n",
        "\n",
        "    <google.cloud.aiplatform.training_jobs.CustomTrainingJob object at 0x7feab1346710>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "run_custom_job:mbsdk,no_model,cpu"
      },
      "source": [
        "#### Run the custom training job\n",
        "\n",
        "Next, you run the custom job to start the training job by invoking the method `run`, with the following parameters:\n",
        "\n",
        "- `replica_count`: The number of compute instances for training (replica_count = 1 is single node training).\n",
        "- `machine_type`: The machine type for the compute instances.\n",
        "- `base_output_dir`: The Cloud Storage location to write the model artifacts to.\n",
        "- `sync`: Whether to block until completion of the job."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "run_custom_job:mbsdk,no_model,cpu"
      },
      "outputs": [],
      "source": [
        "MODEL_DIR = \"{}/{}\".format(BUCKET_URI, UUID)\n",
        "\n",
        "\n",
        "job.run(\n",
        "    replica_count=1, machine_type=TRAIN_COMPUTE, base_output_dir=MODEL_DIR, sync=True\n",
        ")\n",
        "\n",
        "MODEL_DIR = MODEL_DIR + \"/model\"\n",
        "model_path_to_deploy = MODEL_DIR"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "import_model:migration,new"
      },
      "source": [
        "### [general.import-model](https://cloud.google.com/vertex-ai/docs/general/import-model)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "upload_model:mbsdk"
      },
      "source": [
        "## Upload the model\n",
        "\n",
        "Next, upload your model to a `Model` resource using `Model.upload()` method, with the following parameters:\n",
        "\n",
        "- `display_name`: The human readable name for the `Model` resource.\n",
        "- `artifact`: The Cloud Storage location of the trained model artifacts.\n",
        "- `serving_container_image_uri`: The serving container image.\n",
        "- `sync`: Whether to execute the upload asynchronously or synchronously.\n",
        "\n",
        "If the `upload()` method is run asynchronously, you can subsequently block until completion with the `wait()` method."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "upload_model:mbsdk"
      },
      "outputs": [],
      "source": [
        "model = aip.Model.upload(\n",
        "    display_name=\"iris_\" + UUID,\n",
        "    artifact_uri=MODEL_DIR,\n",
        "    serving_container_image_uri=DEPLOY_IMAGE,\n",
        "    sync=False,\n",
        ")\n",
        "\n",
        "model.wait()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "upload_model:mbsdk"
      },
      "source": [
        "*Example output:*\n",
        "\n",
        "    INFO:google.cloud.aiplatform.models:Creating Model\n",
        "    INFO:google.cloud.aiplatform.models:Create Model backing LRO: projects/759209241365/locations/us-central1/models/925164267982815232/operations/3458372263047331840\n",
        "    INFO:google.cloud.aiplatform.models:Model created. Resource name: projects/759209241365/locations/us-central1/models/925164267982815232\n",
        "    INFO:google.cloud.aiplatform.models:To use this Model in another session:\n",
        "    INFO:google.cloud.aiplatform.models:model = aiplatform.Model('projects/759209241365/locations/us-central1/models/925164267982815232')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "make_batch_predictions:migration"
      },
      "source": [
        "## Make batch predictions"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "batchpredictionjobs_create:migration,new,mbsdk"
      },
      "source": [
        "### [predictions.batch-prediction](https://cloud.google.com/vertex-ai/docs/predictions/batch-predictions)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "make_test_items:xgboost,tabular,iris"
      },
      "source": [
        "### Make test items\n",
        "\n",
        "You will use synthetic data as a test data items. Don't be concerned that we are using synthetic data -- we just want to demonstrate how to make a prediction."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "make_test_items:xgboost,tabular,iris"
      },
      "outputs": [],
      "source": [
        "INSTANCES = [[1.4, 1.3, 5.1, 2.8], [1.5, 1.2, 4.7, 2.4]]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "make_batch_file:custom,tabular,list"
      },
      "source": [
        "### Make the batch input file\n",
        "\n",
        "Now make a batch input file, which you will store in your local Cloud Storage bucket.  Each instance in the prediction request is a list of the form:\n",
        "\n",
        "                        [ [ content_1], [content_2] ]\n",
        "\n",
        "- `content`: The feature values of the test item as a list."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "make_batch_file:custom,tabular,list"
      },
      "outputs": [],
      "source": [
        "import tensorflow as tf\n",
        "\n",
        "gcs_input_uri = BUCKET_URI + \"/\" + \"test.jsonl\"\n",
        "with tf.io.gfile.GFile(gcs_input_uri, \"w\") as f:\n",
        "    for i in INSTANCES:\n",
        "        f.write(str(i) + \"\\n\")\n",
        "\n",
        "! gsutil cat $gcs_input_uri"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "batch_request:mbsdk,jsonl,custom,cpu"
      },
      "source": [
        "### Make the batch prediction request\n",
        "\n",
        "Now that your Model resource is trained, you can make a batch prediction by invoking the batch_predict() method, with the following parameters:\n",
        "\n",
        "- `job_display_name`: The human readable name for the batch prediction job.\n",
        "- `gcs_source`: A list of one or more batch request input files.\n",
        "- `gcs_destination_prefix`: The Cloud Storage location for storing the batch prediction resuls.\n",
        "- `instances_format`: The format for the input instances, either 'csv' or 'jsonl'. Defaults to 'jsonl'.\n",
        "- `predictions_format`: The format for the output predictions, either 'csv' or 'jsonl'. Defaults to 'jsonl'.\n",
        "- `machine_type`: The type of machine to use for training.\n",
        "- `sync`: If set to True, the call will block while waiting for the asynchronous batch job to complete."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "batch_request:mbsdk,jsonl,custom,cpu"
      },
      "outputs": [],
      "source": [
        "MIN_NODES = 1\n",
        "MAX_NODES = 1\n",
        "\n",
        "batch_predict_job = model.batch_predict(\n",
        "    job_display_name=\"iris_\" + UUID,\n",
        "    gcs_source=gcs_input_uri,\n",
        "    gcs_destination_prefix=BUCKET_URI,\n",
        "    instances_format=\"jsonl\",\n",
        "    predictions_format=\"jsonl\",\n",
        "    model_parameters=None,\n",
        "    machine_type=DEPLOY_COMPUTE,\n",
        "    starting_replica_count=MIN_NODES,\n",
        "    max_replica_count=MAX_NODES,\n",
        "    sync=False,\n",
        ")\n",
        "\n",
        "print(batch_predict_job)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "batch_request:mbsdk,jsonl,custom,cpu"
      },
      "source": [
        "*Example output:*\n",
        "\n",
        "    INFO:google.cloud.aiplatform.jobs:Creating BatchPredictionJob\n",
        "    <google.cloud.aiplatform.jobs.BatchPredictionJob object at 0x7f806a6112d0> is waiting for upstream dependencies to complete.\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob created. Resource name: projects/759209241365/locations/us-central1/batchPredictionJobs/5110965452507447296\n",
        "    INFO:google.cloud.aiplatform.jobs:To use this BatchPredictionJob in another session:\n",
        "    INFO:google.cloud.aiplatform.jobs:bpj = aiplatform.BatchPredictionJob('projects/759209241365/locations/us-central1/batchPredictionJobs/5110965452507447296')\n",
        "    INFO:google.cloud.aiplatform.jobs:View Batch Prediction Job:\n",
        "    https://console.cloud.google.com/ai/platform/locations/us-central1/batch-predictions/5110965452507447296?project=759209241365\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob projects/759209241365/locations/us-central1/batchPredictionJobs/5110965452507447296 current state:\n",
        "    JobState.JOB_STATE_RUNNING"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "batch_request_wait:mbsdk"
      },
      "source": [
        "### Wait for completion of batch prediction job\n",
        "\n",
        "Next, wait for the batch job to complete. Alternatively, one can set the parameter `sync` to `True` in the `batch_predict()` method to block until the batch prediction job is completed."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "batch_request_wait:mbsdk"
      },
      "outputs": [],
      "source": [
        "batch_predict_job.wait()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "batch_request_wait:mbsdk"
      },
      "source": [
        "*Example Output:*\n",
        "\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob created. Resource name: projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328\n",
        "    INFO:google.cloud.aiplatform.jobs:To use this BatchPredictionJob in another session:\n",
        "    INFO:google.cloud.aiplatform.jobs:bpj = aiplatform.BatchPredictionJob('projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328')\n",
        "    INFO:google.cloud.aiplatform.jobs:View Batch Prediction Job:\n",
        "    https://console.cloud.google.com/ai/platform/locations/us-central1/batch-predictions/181835033978339328?project=759209241365\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328 current state:\n",
        "    JobState.JOB_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328 current state:\n",
        "    JobState.JOB_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328 current state:\n",
        "    JobState.JOB_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328 current state:\n",
        "    JobState.JOB_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328 current state:\n",
        "    JobState.JOB_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328 current state:\n",
        "    JobState.JOB_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328 current state:\n",
        "    JobState.JOB_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328 current state:\n",
        "    JobState.JOB_STATE_RUNNING\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328 current state:\n",
        "    JobState.JOB_STATE_SUCCEEDED\n",
        "    INFO:google.cloud.aiplatform.jobs:BatchPredictionJob run completed. Resource name: projects/759209241365/locations/us-central1/batchPredictionJobs/181835033978339328"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "get_batch_prediction:mbsdk,custom,lcn"
      },
      "source": [
        "### Get the predictions\n",
        "\n",
        "Next, get the results from the completed batch prediction job.\n",
        "\n",
        "The results are written to the Cloud Storage output bucket you specified in the batch prediction request. You call the method iter_outputs() to get a list of each Cloud Storage file generated with the results. Each file contains one or more prediction requests in a JSON format:\n",
        "\n",
        "- `instance`: The prediction request.\n",
        "- `prediction`: The prediction response."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "get_batch_prediction:mbsdk,custom,lcn"
      },
      "outputs": [],
      "source": [
        "import json\n",
        "\n",
        "bp_iter_outputs = batch_predict_job.iter_outputs()\n",
        "\n",
        "prediction_results = list()\n",
        "for blob in bp_iter_outputs:\n",
        "    if blob.name.split(\"/\")[-1].startswith(\"prediction\"):\n",
        "        prediction_results.append(blob.name)\n",
        "\n",
        "tags = list()\n",
        "for prediction_result in prediction_results:\n",
        "    gfile_name = f\"gs://{bp_iter_outputs.bucket.name}/{prediction_result}\"\n",
        "    with tf.io.gfile.GFile(name=gfile_name, mode=\"r\") as gfile:\n",
        "        for line in gfile.readlines():\n",
        "            line = json.loads(line)\n",
        "            print(line)\n",
        "            break"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "get_batch_prediction:mbsdk,custom,lcn"
      },
      "source": [
        "*Example Output:*\n",
        "\n",
        "    {'instance': [1.4, 1.3, 5.1, 2.8], 'prediction': 2.0451931953430176}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "make_online_predictions:migration"
      },
      "source": [
        "## Make online predictions"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "deploy_model:migration,new,mbsdk"
      },
      "source": [
        "### [predictions.deploy-model-api](https://cloud.google.com/vertex-ai/docs/predictions/deploy-model-api)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "deploy_model:mbsdk,cpu"
      },
      "source": [
        "## Deploy the model\n",
        "\n",
        "Next, deploy your model for online prediction. To deploy the model, you invoke the `deploy` method, with the following parameters:\n",
        "\n",
        "- `deployed_model_display_name`: A human readable name for the deployed model.\n",
        "- `traffic_split`: Percent of traffic at the endpoint that goes to this model, which is specified as a dictionary of one or more key/value pairs.\n",
        "If only one model, then specify as { \"0\": 100 }, where \"0\" refers to this model being uploaded and 100 means 100% of the traffic.\n",
        "If there are existing models on the endpoint, for which the traffic will be split, then use model_id to specify as { \"0\": percent, model_id: percent, ... }, where model_id is the model id of an existing model to the deployed endpoint. The percents must add up to 100.\n",
        "- `machine_type`: The type of machine to use for training.\n",
        "- `starting_replica_count`: The number of compute instances to initially provision.\n",
        "- `max_replica_count`: The maximum number of compute instances to scale to. In this tutorial, only one instance is provisioned."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "deploy_model:mbsdk,cpu"
      },
      "outputs": [],
      "source": [
        "DEPLOYED_NAME = \"iris-\" + UUID\n",
        "\n",
        "TRAFFIC_SPLIT = {\"0\": 100}\n",
        "\n",
        "MIN_NODES = 1\n",
        "MAX_NODES = 1\n",
        "\n",
        "endpoint = model.deploy(\n",
        "    deployed_model_display_name=DEPLOYED_NAME,\n",
        "    traffic_split=TRAFFIC_SPLIT,\n",
        "    machine_type=DEPLOY_COMPUTE,\n",
        "    min_replica_count=MIN_NODES,\n",
        "    max_replica_count=MAX_NODES,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "deploy_model:mbsdk,cpu"
      },
      "source": [
        "*Example output:*\n",
        "\n",
        "    INFO:google.cloud.aiplatform.models:Creating Endpoint\n",
        "    INFO:google.cloud.aiplatform.models:Create Endpoint backing LRO: projects/759209241365/locations/us-central1/endpoints/4867177336350441472/operations/4087251132693348352\n",
        "    INFO:google.cloud.aiplatform.models:Endpoint created. Resource name: projects/759209241365/locations/us-central1/endpoints/4867177336350441472\n",
        "    INFO:google.cloud.aiplatform.models:To use this Endpoint in another session:\n",
        "    INFO:google.cloud.aiplatform.models:endpoint = aiplatform.Endpoint('projects/759209241365/locations/us-central1/endpoints/4867177336350441472')\n",
        "    INFO:google.cloud.aiplatform.models:Deploying model to Endpoint : projects/759209241365/locations/us-central1/endpoints/4867177336350441472\n",
        "    INFO:google.cloud.aiplatform.models:Deploy Endpoint model backing LRO: projects/759209241365/locations/us-central1/endpoints/4867177336350441472/operations/1691336130932244480\n",
        "    INFO:google.cloud.aiplatform.models:Endpoint model deployed. Resource name: projects/759209241365/locations/us-central1/endpoints/4867177336350441472"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "endpoints_predict:migration,new,mbsdk"
      },
      "source": [
        "### [predictions.online-prediction-automl](https://cloud.google.com/vertex-ai/docs/predictions/online-predictions-automl)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "make_test_item:xgboost,tabular,iris"
      },
      "source": [
        "### Make test item\n",
        "\n",
        "You will use synthetic data as a test data item. Don't be concerned that we are using synthetic data -- we just want to demonstrate how to make a prediction."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "make_test_item:xgboost,tabular,iris"
      },
      "outputs": [],
      "source": [
        "INSTANCE = [1.4, 1.3, 5.1, 2.8]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "predict_request:mbsdk,custom,lcn"
      },
      "source": [
        "### Make the prediction\n",
        "\n",
        "Now that your `Model` resource is deployed to an `Endpoint` resource, you can do online predictions by sending prediction requests to the `Endpoint` resource.\n",
        "\n",
        "#### Request\n",
        "\n",
        "The format of each instance is:\n",
        "\n",
        "    [feature_list]\n",
        "\n",
        "Since the predict() method can take multiple items (instances), send your single test item as a list of one test item.\n",
        "\n",
        "#### Response\n",
        "\n",
        "The response from the predict() call is a Python dictionary with the following entries:\n",
        "\n",
        "- `ids`: The internal assigned unique identifiers for each prediction request.\n",
        "- `predictions`: The predicted confidence, between 0 and 1, per class label.\n",
        "- `deployed_model_id`: The Vertex AI identifier for the deployed `Model` resource which did the predictions."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "predict_request:mbsdk,custom,lcn"
      },
      "outputs": [],
      "source": [
        "instances_list = [INSTANCE]\n",
        "\n",
        "prediction = endpoint.predict(instances_list)\n",
        "print(prediction)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "undeploy_model:mbsdk"
      },
      "source": [
        "## Undeploy the model\n",
        "\n",
        "When you are done doing predictions, you undeploy the model from the `Endpoint` resouce. This deprovisions all compute resources and ends billing for the deployed model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "undeploy_model:mbsdk"
      },
      "outputs": [],
      "source": [
        "endpoint.undeploy_all()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cleanup:mbsdk"
      },
      "source": [
        "# Cleaning up\n",
        "\n",
        "To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n",
        "project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n",
        "\n",
        "Otherwise, you can delete the individual resources you created in this tutorial."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cleanup:mbsdk"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "\n",
        "delete_bucket = False\n",
        "\n",
        "# Delete the model using the Vertex model object\n",
        "model.delete()\n",
        "\n",
        "# Delete the endpoint using the Vertex endpoint object\n",
        "endpoint.delete()\n",
        "\n",
        "# Delete the training job\n",
        "job.delete()\n",
        "\n",
        "# Delete the batch prediction job using the Vertex batch prediction object\n",
        "batch_predict_job.delete()\n",
        "\n",
        "if delete_bucket or os.getenv(\"IS_TESTING\"):\n",
        "    ! gsutil rm -r $BUCKET_URI"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "sdk-custom-xgboost-prebuilt-container.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}