examples/inferencing/00_batch_prediction_example.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "12b45b0b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Copyright 2024 Google LLC\n",
    "#\n",
    "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "# you may not use this file except in compliance with the License.\n",
    "# You may obtain a copy of the License at\n",
    "#\n",
    "#     https://www.apache.org/licenses/LICENSE-2.0\n",
    "#\n",
    "# Unless required by applicable law or agreed to in writing, software\n",
    "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "# See the License for the specific language governing permissions and\n",
    "# limitations under the License."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8790d7ed",
   "metadata": {},
   "source": [
    "# AutoMLOps - Batch Prediction Example\n",
    "\n",
    "<table align=\"left\">\n",
    "  <td>\n",
    "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/automlops/blob/main/examples/inference/00_batch_prediction_example.ipynb\">\n",
    "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
    "    </a>\n",
    "  </td>\n",
    "  <td>\n",
    "    <a href=\"https://github.com/GoogleCloudPlatform/automlops/blob/main/examples/inference/00_batch_prediction_example.ipynb\">\n",
    "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
    "      View on GitHub\n",
    "    </a>\n",
    "  </td>\n",
    "  <td>\n",
    "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/automlops/main/examples/inference/00_batch_prediction_example.ipynb\">\n",
    "        <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
    "      Open in Vertex AI Workbench\n",
    "    </a>\n",
    "  </td>\n",
    "</table>\n",
    "<br/><br/><br/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f938540",
   "metadata": {},
   "source": [
    "# Overview\n",
    "\n",
    "In this tutorial, you will build a [Vertex AI](https://cloud.google.com/vertex-ai) pipeline, complete with an integrated CI/CD pipeline. This tutorial will walk you through how to use AutoMLOps to define, create and run pipelines.\n",
    "\n",
    "This tutorial assumes you have gone through the [introduction training example](../training/00_introduction_training_example.ipynb) and are using the deployed model endpoint."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22881a6a",
   "metadata": {},
   "source": [
    "# Objective\n",
    "In this tutorial, you will learn how to create and run MLOps pipelines integrated with CI/CD. This tutorial goes through an example pipeline that is defined two ways: first using a custom python syntax, and second using Kubeflow syntax (either option is valid, whichever is preferred is up to you). The example pipeline runs batch prediction using a deployed classification model (see our training example for training this classification model); the pipeline runs the following component:\n",
    "1. create_dataset: A custom component that will export the dataset from BQ to GCS as a jsonl.\n",
    "1. batch_prediction: A custom component that will run a batch prediction job using a deployed model.\n",
    "\n",
    "# Prerequisites\n",
    "\n",
    "In order to use AutoMLOps, the following are required:\n",
    "\n",
    "- Python 3.7 - 3.10\n",
    "- [Google Cloud SDK 407.0.0](https://cloud.google.com/sdk/gcloud/reference)\n",
    "- [beta 2022.10.21](https://cloud.google.com/sdk/gcloud/reference/beta)\n",
    "- `git` installed\n",
    "- `git` logged-in:\n",
    "```\n",
    "  git config --global user.email \"you@example.com\"\n",
    "  git config --global user.name \"Your Name\"\n",
    "```\n",
    "- [Application Default Credentials (ADC)](https://cloud.google.com/docs/authentication/provide-credentials-adc) are setup. This can be done through the following commands:\n",
    "```\n",
    "gcloud auth application-default login\n",
    "gcloud config set account <account@example.com>\n",
    "```\n",
    "\n",
    "# APIs & IAM\n",
    "Based on the user options selection, AutoMLOps will enable up to the following APIs during the provision step:\n",
    "- [aiplatform.googleapis.com](https://cloud.google.com/vertex-ai/docs/reference/rest)\n",
    "- [artifactregistry.googleapis.com](https://cloud.google.com/artifact-registry/docs/reference/rest)\n",
    "- [cloudbuild.googleapis.com](https://cloud.google.com/build/docs/api/reference/rest)\n",
    "- [cloudfunctions.googleapis.com](https://cloud.google.com/functions/docs/reference/rest)\n",
    "- [cloudresourcemanager.googleapis.com](https://cloud.google.com/resource-manager/reference/rest)\n",
    "- [cloudscheduler.googleapis.com](https://cloud.google.com/scheduler/docs/reference/rest)\n",
    "- [compute.googleapis.com](https://cloud.google.com/compute/docs/reference/rest/v1)\n",
    "- [iam.googleapis.com](https://cloud.google.com/iam/docs/reference/rest)\n",
    "- [iamcredentials.googleapis.com](https://cloud.google.com/iam/docs/reference/credentials/rest)\n",
    "- [logging.googleapis.com](https://cloud.google.com/logging/docs/reference/v2/rest)\n",
    "- [pubsub.googleapis.com](https://cloud.google.com/pubsub/docs/reference/rest)\n",
    "- [run.googleapis.com](https://cloud.google.com/run/docs/reference/rest)\n",
    "- [storage.googleapis.com](https://cloud.google.com/storage/docs/apis)\n",
    "- [sourcerepo.googleapis.com](https://cloud.google.com/source-repositories/docs/reference/rest)\n",
    "\n",
    "\n",
    "AutoMLOps will create the following service account and update [IAM permissions](https://cloud.google.com/iam/docs/understanding-roles) during the provision step:\n",
    "1. Pipeline Runner Service Account (defaults to: vertex-pipelines@PROJECT_ID.iam.gserviceaccount.com). Roles added:\n",
    "- roles/aiplatform.user\n",
    "- roles/artifactregistry.reader\n",
    "- roles/bigquery.user\n",
    "- roles/bigquery.dataEditor\n",
    "- roles/iam.serviceAccountUser\n",
    "- roles/storage.admin\n",
    "- roles/cloudfunctions.admin\n",
    "\n",
    "# User Guide\n",
    "\n",
    "For a user-guide, please view these [slides](../../AutoMLOps_User_Guide.pdf).\n",
    "\n",
    "# Costs\n",
    "\n",
    "This tutorial uses billable components of Google Cloud:\n",
    "- Vertex AI\n",
    "- Artifact Registry\n",
    "- Cloud Storage\n",
    "- Cloud Source Repository\n",
    "- Cloud Build\n",
    "- Cloud Run\n",
    "- Cloud Scheduler\n",
    "- Cloud Pub/Sub\n",
    "\n",
    "Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing), and use the [Pricing Calculator](https://cloud.google.com/products/calculator/) to generate a cost estimate based on your projected usage.\n",
    "\n",
    "# Ground-rules for using AutoMLOps\n",
    "1. Do not use variables, functions, code, etc. not defined within the scope of a custom component. These custom components will become containers and will have no reference to the out of scope code.\n",
    "2. Import statements and helper functions must be added inside the function. Provide parameter type hints.\n",
    "3. Test each of your components for accuracy and correctness before running them using AutoMLOps. We cannot fix bugs automatically; bugs are much more difficult to fix once they are made into pipelines.\n",
    "4. If you are using Kubeflow, be sure to define all the requirements needed to run the custom component - it can be easy to leave out packages which will cause the container to fail when running within a pipeline. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12381413",
   "metadata": {},
   "source": [
    "# Dataset\n",
    "For training data, we are using the [dry beans dataset](https://archive.ics.uci.edu/ml/datasets/dry+bean+dataset) which contains metadata on images of seven different types of dry beans taken with a high-resolution camera. The raw dataset can be found [here](https://github.com/GoogleCloudPlatform/automlops/blob/main/example/data/Dry_Beans_Dataset.csv). We will take a subset of this data to show an inferencing example."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a231b629",
   "metadata": {},
   "source": [
    "# Setup Git\n",
    "Set up your git configuration below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "92f90b44",
   "metadata": {},
   "outputs": [],
   "source": [
    "!git config --global user.email 'you@example.com'\n",
    "!git config --global user.name 'Your Name'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5e4d190",
   "metadata": {},
   "source": [
    "# Install AutoMLOps\n",
    "\n",
    "Install AutoMLOps from [PyPI](https://pypi.org/project/google-cloud-automlops/), or locally by cloning the repo and running `pip install .`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "94451868",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip3 install google-cloud-automlops --user"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17db55d5",
   "metadata": {},
   "source": [
    "# Restart the kernel\n",
    "Once you've installed the AutoMLOps package, you need to restart the notebook kernel so it can find the package.\n",
    "\n",
    "**Note: Once this cell has finished running, continue on. You do not need to re-run any of the cells above.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "57c53b86",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "if not os.getenv('IS_TESTING'):\n",
    "    # Automatically restart kernel after installs\n",
    "    import IPython\n",
    "\n",
    "    app = IPython.Application.instance()\n",
    "    app.kernel.do_shutdown(True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "250d511b",
   "metadata": {},
   "source": [
    "# Set your project ID\n",
    "Set your project ID below. If you don't know your project ID, leave the field blank and the following cells may be able to find it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "931ff517",
   "metadata": {},
   "outputs": [],
   "source": [
    "PROJECT_ID = '[your-project-id]'  # @param {type:\"string\"}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "be0be295",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Project ID: automlops-sandbox\n"
     ]
    }
   ],
   "source": [
    "if PROJECT_ID == '' or PROJECT_ID is None or PROJECT_ID == '[your-project-id]':\n",
    "    # Get your GCP project id from gcloud\n",
    "    shell_output = !gcloud config list --format 'value(core.project)' 2>/dev/null\n",
    "    PROJECT_ID = shell_output[0]\n",
    "    print('Project ID:', PROJECT_ID)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "2c36482d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Updated property [core/project].\r\n"
     ]
    }
   ],
   "source": [
    "! gcloud config set project $PROJECT_ID"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d9d9ab0",
   "metadata": {},
   "source": [
    "Set your Model_ID below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "7b2a7e4d",
   "metadata": {},
   "outputs": [],
   "source": [
    "MODEL_ID = 'dry-beans-dt-inferencing'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "503f4a85-c357-4ec4-8ea5-0eeb2c8afd39",
   "metadata": {},
   "source": [
    "# Upload Data\n",
    "This will create a BQ table and upload the Dry Beans csv. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "28557b2d-598c-466b-8116-bc1e34ba090d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dataset automlops-sandbox.test_dataset already exists\n",
      "Table test_dataset.dry-beans-inferencing already exists\n"
     ]
    }
   ],
   "source": [
    "!python3 -m data.load_data_to_bq --project $PROJECT_ID --file data/Dry_Beans_Dataset_Inferencing.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b3fe994",
   "metadata": {},
   "source": [
    "# 1. AutoMLOps Batch Prediction Example\n",
    "This workflow will define and generate a pipeline using AutoMLOps. AutoMLOps provides 2 functions for defining MLOps pipelines:\n",
    "\n",
    "- `AutoMLOps.component(...)`: Defines a component, which is a containerized python function.\n",
    "- `AutoMLOps.pipeline(...)`: Defines a pipeline, which is a series of components.\n",
    "\n",
    "AutoMLOps provides 6 functions for building and maintaining MLOps pipelines:\n",
    "\n",
    "- `AutoMLOps.generate(...)`: Generates the MLOps codebase. Users can specify the tooling and technologies they would like to use in their MLOps pipeline.\n",
    "- `AutoMLOps.provision(...)`: Runs provisioning scripts to create and maintain necessary infra for MLOps.\n",
    "- `AutoMLOps.deprovision(...)`: Runs deprovisioning scripts to tear down MLOps infra created using AutoMLOps.\n",
    "- `AutoMLOps.deploy(...)`: Builds and pushes component container, then triggers the pipeline job.\n",
    "- `AutoMLOps.launchAll(...)`: Runs `generate()`, `provision()`, and `deploy()` all in succession.\n",
    "- `AutoMLOps.monitor(...)`: Creates model monitoring jobs on deployed endpoints.\n",
    "\n",
    "Please see the [readme](https://github.com/GoogleCloudPlatform/automlops/blob/main/README.md) for more information."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2219ee67",
   "metadata": {},
   "source": [
    "## Import AutoMLOps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "3d02ea3e",
   "metadata": {},
   "outputs": [],
   "source": [
    "from google_cloud_automlops import AutoMLOps"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d4476cb4-91c5-42ff-a500-8cc275fedbd1",
   "metadata": {},
   "source": [
    "## Batch Prediction\n",
    "Define a custom component for batch prediction. Import statements and helper functions must be added inside the function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "5371be73-db3f-4d79-bde7-94fcd5ea13b2",
   "metadata": {},
   "outputs": [],
   "source": [
    "@AutoMLOps.component\n",
    "def batch_predict(\n",
    "    project_id: str,\n",
    "    bigquery_destination: str,\n",
    "    bq_dataset_path: str,\n",
    "    instances_format: str,\n",
    "    predictions_format: str,\n",
    "    model_resource_name: str,\n",
    "    endpoint_resource_name: str,\n",
    "    machine_type: str,\n",
    "    accelerator_count: int,\n",
    "    accelerator_type: str,\n",
    "    max_replica_count: int,\n",
    "    starting_replica_count: int\n",
    "):\n",
    "    \"\"\"Runs a batch prediction job.\n",
    "\n",
    "    Args:\n",
    "        bigquery_destination: The BQ uri to store the prediction results.\n",
    "        bq_dataset_path: The BQ uri of the input data to run predictions on.\n",
    "        instances_format: The format in which instances are given, must be one of 'jsonl', 'csv', 'bigquery', 'tf-record', 'tf-record-gzip', or 'file-list'.\n",
    "        predictions_format: The format to output the predictions, must be one of 'jsonl', 'csv', or 'bigquery'.\n",
    "        model_resource_name: The fully-qualified resource name or ID for model e.g. projects/297370817971/locations/{region}/models/4540613586807947264\n",
    "        endpoint_resource_name: The fully-qualified resource name or ID for endpoint e.g. projects/297370817971/locations/{region}/endpoints/1242430547200835584\n",
    "        machine_type: The machine type to serve the prediction requests.\n",
    "        accelerator_count: The number of accelerators to attach to the `machine_type`.\n",
    "        accelerator_type: The type of accelerators that may be attached to the machine as per `accelerator_count`.\n",
    "        max_replica_count: The maximum number of machine replicas the batch operation may be scaled to.\n",
    "        starting_replica_count: The number of machine replicas used at the start of the batch operation.\n",
    "    \"\"\"\n",
    "    import logging\n",
    "    \n",
    "    from google.cloud import aiplatform\n",
    "    from google.cloud.aiplatform.compat.types import job_state_v1\n",
    "    \n",
    "    def _get_endpoint(resource_name: str) -> aiplatform.Endpoint:\n",
    "        return aiplatform.Endpoint(resource_name)\n",
    "\n",
    "    def _get_model(resource_name: str) -> aiplatform.Model:\n",
    "        return aiplatform.Model(resource_name)\n",
    "\n",
    "    def _get_model_from_endpoint(endpoint: aiplatform.Endpoint) -> aiplatform.Model:\n",
    "        current_deployed_model_id = None\n",
    "\n",
    "        traffic_split = endpoint.gca_resource.traffic_split\n",
    "        for key in traffic_split:\n",
    "            if traffic_split[key] == 100:\n",
    "                current_deployed_model_id = key\n",
    "            break\n",
    "\n",
    "        if current_deployed_model_id:\n",
    "            for deployed_model in endpoint.gca_resource.deployed_models:\n",
    "                if deployed_model.id == current_deployed_model_id:\n",
    "                    return aiplatform.Model(deployed_model.model)\n",
    "\n",
    "\n",
    "    logging.info(f'input dataset URI: {bq_dataset_path}')\n",
    "\n",
    "    # Call Vertex AI custom job in another region\n",
    "    aiplatform.init(project=project_id)\n",
    "\n",
    "    if model_resource_name:\n",
    "        model = _get_model(model_resource_name)\n",
    "    elif endpoint_resource_name:\n",
    "        model = _get_model_from_endpoint(_get_endpoint(endpoint_resource_name))\n",
    "    else:\n",
    "        raise ValueError('model or endpoint resource name must be provided!')\n",
    "\n",
    "    logging.info(f'retrieved model URI: {model.uri}')\n",
    "\n",
    "    batch_pred_job = model.batch_predict(\n",
    "        job_display_name='batch-prediction',\n",
    "        bigquery_source=bq_dataset_path,\n",
    "        bigquery_destination_prefix=bigquery_destination,\n",
    "        instances_format=instances_format,\n",
    "        predictions_format=predictions_format,\n",
    "        machine_type=machine_type,\n",
    "        accelerator_count=accelerator_count,\n",
    "        accelerator_type=accelerator_type,\n",
    "        starting_replica_count=starting_replica_count,\n",
    "        max_replica_count=max_replica_count,\n",
    "        sync=True)\n",
    "\n",
    "    logging.info(f'batch prediction job: {batch_pred_job.resource_name}')\n",
    "\n",
    "    batch_pred_job.wait()\n",
    "    if batch_pred_job.state == job_state_v1.JobState.JOB_STATE_SUCCEEDED:\n",
    "        logging.info(f'batch prediction job has finished with info: '\n",
    "                     f'{batch_pred_job.completion_stats}')\n",
    "        logging.info(f'Predictions can be found at: '\n",
    "                     f'{batch_pred_job.output_info.gcs_output_directory}')\n",
    "    else:\n",
    "        raise RuntimeError(batch_pred_job.error)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73d96dcb-a020-4bab-b0e3-1e32f6b2aecf",
   "metadata": {},
   "source": [
    "## Define the Pipeline\n",
    "Define your pipeline. You can optionally give the pipeline a name and description. Define the structure by listing the components to be called in your pipeline; use `.after` to specify the order of execution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "a36e0154",
   "metadata": {},
   "outputs": [],
   "source": [
    "@AutoMLOps.pipeline #(name='automlops-pipeline', description='This is an optional description')\n",
    "def pipeline(project_id: str,\n",
    "             bigquery_destination: str,\n",
    "             bq_dataset_path: str,\n",
    "             instances_format: str,\n",
    "             predictions_format: str,\n",
    "             model_resource_name: str,\n",
    "             endpoint_resource_name: str,\n",
    "             machine_type: str,\n",
    "             accelerator_count: int,\n",
    "             accelerator_type: str,\n",
    "             max_replica_count: int,\n",
    "             starting_replica_count: int\n",
    "            ):\n",
    "\n",
    "    batch_predict_task = batch_predict(\n",
    "             project_id=project_id,\n",
    "             bigquery_destination=bigquery_destination,\n",
    "             bq_dataset_path=bq_dataset_path,\n",
    "             instances_format=instances_format,\n",
    "             predictions_format=predictions_format,\n",
    "             model_resource_name=model_resource_name,\n",
    "             endpoint_resource_name=endpoint_resource_name,\n",
    "             machine_type=machine_type,\n",
    "             accelerator_count=accelerator_count,\n",
    "             accelerator_type=accelerator_type,\n",
    "             max_replica_count=max_replica_count,\n",
    "             starting_replica_count=starting_replica_count)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3874f4f6",
   "metadata": {},
   "source": [
    "## Define the Pipeline Arguments"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "4cb3786b",
   "metadata": {},
   "outputs": [],
   "source": [
    "pipeline_params = {\n",
    "    'project_id': PROJECT_ID,\n",
    "    'bigquery_destination': f'bq://{PROJECT_ID}.test_dataset.dry-beans-inferencing-results',\n",
    "    'bq_dataset_path': f'bq://{PROJECT_ID}.test_dataset.dry-beans-inferencing',\n",
    "    'instances_format': 'bigquery',\n",
    "    'predictions_format': 'bigquery',\n",
    "    'model_resource_name': '',\n",
    "    'endpoint_resource_name': 'projects/45373616427/locations/us-central1/endpoints/2255296260661575680',\n",
    "    'machine_type': 'n1-standard-8',\n",
    "    'accelerator_count': 0,\n",
    "    'accelerator_type': 'ACCELERATOR_TYPE_UNSPECIFIED',\n",
    "    'max_replica_count': 2,\n",
    "    'starting_replica_count': 1\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4cc2f86a",
   "metadata": {},
   "source": [
    "## Generate and Run the pipeline\n",
    "`AutoMLOps.generate(...)` generates the MLOps codebase. Users can specify the tooling and technologies they would like to use in their MLOps pipeline."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "dde57239",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing directories under AutoMLOps/\n",
      "Writing configurations to AutoMLOps/configs/defaults.yaml\n",
      "Writing README.md to AutoMLOps/README.md\n",
      "Writing kubeflow pipelines code to AutoMLOps/pipelines, AutoMLOps/components\n",
      "Writing scripts to AutoMLOps/scripts\n",
      "Writing submission service code to AutoMLOps/services\n",
      "Writing gcloud provisioning code to AutoMLOps/provision\n",
      "Writing cloud build config to AutoMLOps/cloudbuild.yaml\n",
      "Code Generation Complete.\n"
     ]
    }
   ],
   "source": [
    "AutoMLOps.generate(project_id=PROJECT_ID,\n",
    "                   pipeline_params=pipeline_params,\n",
    "                   use_ci=True,\n",
    "                   naming_prefix=MODEL_ID,\n",
    "                   schedule_pattern='59 11 * * 0' # rerun every Sunday at Midnight\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3ff3dd9",
   "metadata": {},
   "source": [
    "`AutoMLOps.provision(...)` runs provisioning scripts to create and maintain necessary infra for MLOps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "8f4124e8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING: Provisioning requires these permissions:\n",
      "-serviceusage.services.enable\n",
      "-serviceusage.services.use\n",
      "-storage.buckets.get\n",
      "-storage.buckets.create\n",
      "-resourcemanager.projects.setIamPolicy\n",
      "-iam.serviceAccounts.list\n",
      "-iam.serviceAccounts.create\n",
      "-iam.serviceAccounts.actAs\n",
      "-pubsub.topics.list\n",
      "-pubsub.topics.create\n",
      "-pubsub.subscriptions.list\n",
      "-pubsub.subscriptions.create\n",
      "-artifactregistry.repositories.list\n",
      "-artifactregistry.repositories.create\n",
      "-cloudbuild.builds.list\n",
      "-cloudbuild.builds.create\n",
      "-cloudscheduler.jobs.list\n",
      "-cloudscheduler.jobs.create\n",
      "-cloudfunctions.functions.get\n",
      "-cloudfunctions.functions.create\n",
      "-source.repos.list\n",
      "-source.repos.create\n",
      "\n",
      "You are currently using: srastatter@google.com. Please check your account permissions.\n",
      "The following are the recommended roles for provisioning:\n",
      "-roles/serviceusage.serviceUsageAdmin\n",
      "-roles/resourcemanager.projectIamAdmin\n",
      "-roles/iam.serviceAccountAdmin\n",
      "-roles/iam.serviceAccountUser\n",
      "-roles/storage.admin\n",
      "-roles/pubsub.editor\n",
      "-roles/artifactregistry.admin\n",
      "-roles/cloudbuild.builds.editor\n",
      "-roles/cloudscheduler.admin\n",
      "-roles/cloudfunctions.admin\n",
      "-roles/source.admin\n",
      "\n",
      "\u001b[0;32m Setting up API services in project automlops-sandbox \u001b[0m\n",
      "Operation \"operations/acat.p2-45373616427-1bcdf775-53ce-437a-be30-d5a4bdf1a27f\" finished successfully.\n",
      "\u001b[0;32m Setting up Artifact Registry in project automlops-sandbox \u001b[0m\n",
      "Listing items under project automlops-sandbox, location us-central1.\n",
      "\n",
      "dry-beans-dt-inferencing-artifact-registry  DOCKER  STANDARD_REPOSITORY  Artifact Registry dry-beans-dt-inferencing-artifact-registry in us-central1.                us-central1                                  Google-managed key  2023-10-27T15:10:14  2023-10-27T15:12:24  168.106\n",
      "Artifact Registry: dry-beans-dt-inferencing-artifact-registry already exists in project automlops-sandbox\n",
      "\u001b[0;32m Setting up Storage Bucket in project automlops-sandbox \u001b[0m\n",
      "gs://automlops-sandbox-dry-beans-dt-inferencing-bucket/\n",
      "GS Bucket: automlops-sandbox-dry-beans-dt-inferencing-bucket already exists in project automlops-sandbox\n",
      "\u001b[0;32m Setting up Pipeline Job Runner Service Account in project automlops-sandbox \u001b[0m\n",
      "Pipeline Runner Service Account         vertex-pipelines@automlops-sandbox.iam.gserviceaccount.com           False\n",
      "Service Account: vertex-pipelines already exists in project automlops-sandbox\n",
      "\u001b[0;32m Setting up IAM roles for Pipeline Job Runner Service Account in project automlops-sandbox \u001b[0m\n",
      "\u001b[0;32m Setting up Cloud Source Repository in project automlops-sandbox \u001b[0m\n",
      "dry-beans-dt-inferencing-repository  automlops-sandbox  https://source.developers.google.com/p/automlops-sandbox/r/dry-beans-dt-inferencing-repository\n",
      "Cloud Source Repository: dry-beans-dt-inferencing-repository already exists in project automlops-sandbox\n",
      "\u001b[0;32m Setting up Queueing Service in project automlops-sandbox \u001b[0m\n",
      "Creating Pub/Sub Topic: dry-beans-dt-inferencing-queueing-svc in project automlops-sandbox\n",
      "Created topic [projects/automlops-sandbox/topics/dry-beans-dt-inferencing-queueing-svc].\n",
      "\u001b[0;32m Deploying Cloud Functions: dry-beans-dt-inferencing-job-submission-svc in project automlops-sandbox \u001b[0m\n",
      "Deploying function (may take a while - up to 2 minutes)...\n",
      "..\n",
      "For Cloud Build Logs, visit: https://console.cloud.google.com/cloud-build/builds;region=us-central1/be8bf0d0-1515-438e-85ee-24ecd873aff5?project=45373616427\n",
      "........................................................................done.\n",
      "availableMemoryMb: 512\n",
      "buildId: be8bf0d0-1515-438e-85ee-24ecd873aff5\n",
      "buildName: projects/45373616427/locations/us-central1/builds/be8bf0d0-1515-438e-85ee-24ecd873aff5\n",
      "dockerRegistry: ARTIFACT_REGISTRY\n",
      "dockerRepository: projects/automlops-sandbox/locations/us-central1/repositories/dry-beans-dt-inferencing-artifact-registry\n",
      "entryPoint: process_request\n",
      "eventTrigger:\n",
      "  eventType: google.pubsub.topic.publish\n",
      "  failurePolicy: {}\n",
      "  resource: projects/automlops-sandbox/topics/dry-beans-dt-inferencing-queueing-svc\n",
      "  service: pubsub.googleapis.com\n",
      "ingressSettings: ALLOW_ALL\n",
      "labels:\n",
      "  deployment-tool: cli-gcloud\n",
      "maxInstances: 3000\n",
      "name: projects/automlops-sandbox/locations/us-central1/functions/dry-beans-dt-inferencing-job-submission-svc\n",
      "runtime: python39\n",
      "serviceAccountEmail: vertex-pipelines@automlops-sandbox.iam.gserviceaccount.com\n",
      "sourceUploadUrl: https://storage.googleapis.com/uploads-961973632599.us-central1.cloudfunctions.appspot.com/5f16fdf8-af98-4c8c-ad71-1a80333c0288.zip\n",
      "status: ACTIVE\n",
      "timeout: 540s\n",
      "updateTime: '2023-10-27T19:17:46.436Z'\n",
      "versionId: '1'\n",
      "\u001b[0;32m Setting up Cloud Build Trigger in project automlops-sandbox \u001b[0m\n",
      "Creating Cloudbuild Trigger on branch automlops in project automlops-sandbox for repo dry-beans-dt-inferencing-repository\n",
      "Created [https://cloudbuild.googleapis.com/v1/projects/automlops-sandbox/locations/us-central1/triggers/15856198-6244-4888-94e5-11a3e97d7bc9].\n",
      "NAME                                    CREATE_TIME                STATUS\n",
      "dry-beans-dt-inferencing-build-trigger  2023-10-27T19:17:50+00:00\n",
      "\u001b[0;32m Setting up Cloud Scheduler Job in project automlops-sandbox \u001b[0m\n",
      "Creating Cloud Scheduler Job: dry-beans-dt-inferencing-schedule in project automlops-sandbox\n",
      "name: projects/automlops-sandbox/locations/us-central1/jobs/dry-beans-dt-inferencing-schedule\n",
      "pubsubTarget:\n",
      "  data: ewogICAgImJxX3RhYmxlIjogImF1dG9tbG9wcy1zYW5kYm94LnRlc3RfZGF0YXNldC5kcnktYmVhbnMtaW5mZXJlbmNpbmciLAogICAgImRhdGFfcGF0aCI6ICJnczovL2F1dG9tbG9wcy1zYW5kYm94LWJ1Y2tldC9kYXRhLmNzdiIsCiAgICAicHJvamVjdF9pZCI6ICJhdXRvbWxvcHMtc2FuZGJveCIsCiAgICAiYmlncXVlcnlfZGVzdGluYXRpb24iOiAiYnE6Ly9hdXRvbWxvcHMtc2FuZGJveC50ZXN0X2RhdGFzZXQuZHJ5LWJlYW5zLWluZmVyZW5jaW5nLXJlc3VsdHMiLAogICAgImJxX2RhdGFzZXRfcGF0aCI6ICJicTovL2F1dG9tbG9wcy1zYW5kYm94LnRlc3RfZGF0YXNldC5kcnktYmVhbnMtaW5mZXJlbmNpbmciLAogICAgImluc3RhbmNlc19mb3JtYXQiOiAiYmlncXVlcnkiLAogICAgInByZWRpY3Rpb25zX2Zvcm1hdCI6ICJiaWdxdWVyeSIsCiAgICAibW9kZWxfcmVzb3VyY2VfbmFtZSI6ICIiLAogICAgImVuZHBvaW50X3Jlc291cmNlX25hbWUiOiAicHJvamVjdHMvNDUzNzM2MTY0MjcvbG9jYXRpb25zL3VzLWNlbnRyYWwxL2VuZHBvaW50cy8yMjU1Mjk2MjYwNjYxNTc1NjgwIiwKICAgICJtYWNoaW5lX3R5cGUiOiAibjEtc3RhbmRhcmQtOCIsCiAgICAiYWNjZWxlcmF0b3JfY291bnQiOiAwLAogICAgImFjY2VsZXJhdG9yX3R5cGUiOiAiQUNDRUxFUkFUT1JfVFlQRV9VTlNQRUNJRklFRCIsCiAgICAibWF4X3JlcGxpY2FfY291bnQiOiAyLAogICAgInN0YXJ0aW5nX3JlcGxpY2FfY291bnQiOiAxLAogICAgImdzX3BpcGVsaW5lX3NwZWNfcGF0aCI6ICJnczovL2F1dG9tbG9wcy1zYW5kYm94LWRyeS1iZWFucy1kdC1pbmZlcmVuY2luZy1idWNrZXQvcGlwZWxpbmVfcm9vdC9kcnktYmVhbnMtZHQtaW5mZXJlbmNpbmcvcGlwZWxpbmVfam9iLmpzb24iCn0=\n",
      "  topicName: projects/automlops-sandbox/topics/dry-beans-dt-inferencing-queueing-svc\n",
      "retryConfig:\n",
      "  maxBackoffDuration: 3600s\n",
      "  maxDoublings: 16\n",
      "  maxRetryDuration: 0s\n",
      "  minBackoffDuration: 5s\n",
      "schedule: 59 11 * * 0\n",
      "state: ENABLED\n",
      "timeZone: Etc/UTC\n",
      "userUpdateTime: '2023-10-27T19:17:52Z'\n"
     ]
    }
   ],
   "source": [
    "AutoMLOps.provision(hide_warnings=False)            # hide_warnings is optional, defaults to True"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e1f5891",
   "metadata": {},
   "source": [
    "`AutoMLOps.deploy(...)` builds and pushes component container, then triggers the pipeline job."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "82fd0a5b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING: Running precheck for deploying requires these permissions:\n",
      "-serviceusage.services.get\n",
      "-resourcemanager.projects.getIamPolicy\n",
      "-storage.buckets.update\n",
      "-iam.serviceAccounts.get\n",
      "-artifactregistry.repositories.get\n",
      "-pubsub.topics.get\n",
      "-pubsub.subscriptions.get\n",
      "-cloudbuild.builds.get\n",
      "-cloudfunctions.functions.get\n",
      "-source.repos.update\n",
      "\n",
      "You are currently using: srastatter@google.com. Please check your account permissions.\n",
      "The following are the recommended roles for deploying with precheck:\n",
      "-roles/serviceusage.serviceUsageViewer\n",
      "-roles/iam.roleViewer\n",
      "-roles/storage.admin\n",
      "-roles/iam.serviceAccountUser\n",
      "-roles/artifactregistry.reader\n",
      "-roles/pubsub.viewer\n",
      "-roles/cloudbuild.builds.editor\n",
      "-roles/cloudfunctions.viewer\n",
      "-roles/source.writer\n",
      "\n",
      "Checking for required API services in project automlops-sandbox...\n",
      "Checking for Artifact Registry in project automlops-sandbox...\n",
      "Checking for Storage Bucket in project automlops-sandbox...\n",
      "Checking for Pipeline Runner Service Account in project automlops-sandbox...\n",
      "Checking for IAM roles on Pipeline Runner Service Account in project automlops-sandbox...\n",
      "Checking for Cloud Source Repo in project automlops-sandbox...\n",
      "Checking for Pub/Sub Topic in project automlops-sandbox...\n",
      "Checking for Pub/Sub Subscription in project automlops-sandbox...\n",
      "Checking for Cloud Functions Pipeline Job Submission Service in project automlops-sandbox...\n",
      "Checking for Cloud Build Trigger in project automlops-sandbox...\n",
      "Precheck successfully completed, continuing to deployment.\n",
      "\n",
      "Initialized empty Git repository in /Users/srastatter/Documents/2023/MLOps-graduation/AutoMLOps-github/examples/inferencing/AutoMLOps/.git/\n",
      "Switched to a new branch 'automlops'\n",
      "[automlops (root-commit) 7ad39d1] init\n",
      " 1 file changed, 157 insertions(+)\n",
      " create mode 100644 .gitignore\n",
      "remote: Waiting for private key checker: 1/1 objects left        \n",
      "To https://source.developers.google.com/p/automlops-sandbox/r/dry-beans-dt-inferencing-repository\n",
      " * [new branch]      automlops -> automlops\n",
      "[automlops 857af15] Run AutoMLOps\n",
      " 22 files changed, 1296 insertions(+)\n",
      " create mode 100644 README.md\n",
      " create mode 100644 cloudbuild.yaml\n",
      " create mode 100644 components/batch_predict/component.yaml\n",
      " create mode 100644 components/component_base/Dockerfile\n",
      " create mode 100644 components/component_base/requirements.txt\n",
      " create mode 100644 components/component_base/src/batch_predict.py\n",
      " create mode 100644 configs/defaults.yaml\n",
      " create mode 100644 pipelines/pipeline.py\n",
      " create mode 100644 pipelines/pipeline_runner.py\n",
      " create mode 100644 pipelines/requirements.txt\n",
      " create mode 100644 pipelines/runtime_parameters/pipeline_parameter_values.json\n",
      " create mode 100755 provision/provision_resources.sh\n",
      " create mode 100755 scripts/build_components.sh\n",
      " create mode 100755 scripts/build_pipeline_spec.sh\n",
      " create mode 100644 scripts/pipeline_spec/.gitkeep\n",
      " create mode 100644 scripts/pipeline_spec/pipeline_job.json\n",
      " create mode 100755 scripts/publish_to_topic.sh\n",
      " create mode 100755 scripts/run_all.sh\n",
      " create mode 100755 scripts/run_pipeline.sh\n",
      " create mode 100644 services/submission_service/Dockerfile\n",
      " create mode 100644 services/submission_service/main.py\n",
      " create mode 100644 services/submission_service/requirements.txt\n",
      "remote: Waiting for private key checker: 21/22 objects left        \n",
      "To https://source.developers.google.com/p/automlops-sandbox/r/dry-beans-dt-inferencing-repository\n",
      "   7ad39d1..857af15  automlops -> automlops\n",
      "Pushing code to automlops branch, triggering build...\n",
      "Cloud Build job running at: https://console.cloud.google.com/cloud-build/builds;region=us-central1\n",
      "Please wait for this build job to complete.\n",
      "\n",
      "#################################################################\n",
      "#                                                               #\n",
      "#                       RESOURCES MANIFEST                      #\n",
      "#---------------------------------------------------------------#\n",
      "#     Generated resources can be found at the following urls    #\n",
      "#                                                               #\n",
      "#################################################################\n",
      "\n",
      "Google Cloud Storage Bucket: https://console.cloud.google.com/storage/automlops-sandbox-dry-beans-dt-inferencing-bucket\n",
      "Artifact Registry: https://console.cloud.google.com/artifacts/docker/automlops-sandbox/us-central1/dry-beans-dt-inferencing-artifact-registry\n",
      "Service Accounts: https://console.cloud.google.com/iam-admin/serviceaccounts?project=automlops-sandbox\n",
      "APIs: https://console.cloud.google.com/apis\n",
      "Cloud Source Repository: https://source.cloud.google.com/automlops-sandbox/dry-beans-dt-inferencing-repository/+/automlops:\n",
      "Cloud Build Jobs: https://console.cloud.google.com/cloud-build/builds;region=us-central1\n",
      "Vertex AI Pipeline Runs: https://console.cloud.google.com/vertex-ai/pipelines/runs\n",
      "Cloud Build Trigger: https://console.cloud.google.com/cloud-build/triggers;region=us-central1\n",
      "Pipeline Job Submission Service (Cloud Functions): https://console.cloud.google.com/functions/details/us-central1/dry-beans-dt-inferencing-job-submission-svc\n",
      "Pub/Sub Queueing Service Topic: https://console.cloud.google.com/cloudpubsub/topic/detail/dry-beans-dt-inferencing-queueing-svc\n",
      "Pub/Sub Queueing Service Subscriptions: https://console.cloud.google.com/cloudpubsub/subscription/list\n",
      "Cloud Scheduler Job: https://console.cloud.google.com/cloudscheduler\n"
     ]
    }
   ],
   "source": [
    "AutoMLOps.deploy(precheck=True,                     # precheck is optional, defaults to True\n",
    "                 hide_warnings=False)               # hide_warnings is optional, defaults to True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "69b86e90",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "environment": {
   "kernel": "python3",
   "name": "common-cpu.m102",
   "type": "gcloud",
   "uri": "gcr.io/deeplearning-platform-release/base-cpu:m102"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  },
  "vscode": {
   "interpreter": {
    "hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}