notebooks/cifar_10_vertex_autoML_pipeline.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/deep-diver/Continuous-Adaptation-with-VertexAI-AutoML-Pipeline/blob/main/notebooks/cifar_10_vertex_autoML_pipeline.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Build Pipeline with Two Branches for GCP AutoML\n",
        "\n",
        "- One for creating a new dataset, training a new model, deploying the model to a new Endpoint\n",
        "- One for importing(adding) an additional dataset, retraining the previouls model, deploying the model to the already exist Endpoint\n",
        "\n",
        "> **NOTICE**: You should change the GCS bucket names, GCP project ID with your owns\n",
        "\n",
        "<img src=\"https://i.ibb.co/kXcVrcm/Screen-Shot-2022-01-14-at-10-55-29-AM.png\" height=\"1000\"/>"
      ],
      "metadata": {
        "id": "MeHHrPr0LSKj"
      },
      "id": "MeHHrPr0LSKj"
    },
    {
      "cell_type": "markdown",
      "source": [
        "# General Setup"
      ],
      "metadata": {
        "id": "FYY-mAoXH58J"
      },
      "id": "FYY-mAoXH58J"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "a359d411-f1f3-45a9-b9e8-666fae820716",
      "metadata": {
        "id": "a359d411-f1f3-45a9-b9e8-666fae820716"
      },
      "outputs": [],
      "source": [
        "!pip install -Uqq kfp\n",
        "!pip install -Uqq google-cloud-aiplatform\n",
        "!pip install -Uqq google-cloud-pipeline-components"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!gcloud init"
      ],
      "metadata": {
        "id": "_DFul6iGnBab",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "8d7dad7a-1356-42e8-eed6-9c8e882ef940"
      },
      "id": "_DFul6iGnBab",
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Welcome! This command will take you through the configuration of gcloud.\n",
            "\n",
            "Settings from your current configuration [handson2] are:\n",
            "component_manager:\n",
            "  disable_update_check: 'True'\n",
            "core:\n",
            "  account: chansung.tester.1015@gmail.com\n",
            "  project: celtic-iridium-338202\n",
            "\n",
            "Pick configuration to use:\n",
            " [1] Re-initialize this configuration [handson2] with new settings \n",
            " [2] Create a new configuration\n",
            " [3] Switch to and re-initialize existing configuration: [default]\n",
            " [4] Switch to and re-initialize existing configuration: [handson]\n",
            "Please enter your numeric choice:  2\n",
            "\n",
            "Enter configuration name. Names start with a lower case letter and contain only \n",
            "lower case letters a-z, digits 0-9, and hyphens '-':  hhh\n",
            "Your current configuration has been set to: [hhh]\n",
            "\n",
            "You can skip diagnostics next time by using the following flag:\n",
            "  gcloud init --skip-diagnostics\n",
            "\n",
            "Network diagnostic detects and fixes local network connection issues.\n",
            "Reachability Check passed.\n",
            "Network diagnostic passed (1/1 checks passed).\n",
            "\n",
            "Choose the account you would like to use to perform operations for this \n",
            "configuration:\n",
            " [1] chansung.tester.1013@gmail.com\n",
            " [2] chansung.tester.1015@gmail.com\n",
            " [3] Log in with a new account\n",
            "Please enter your numeric choice:  1\n",
            "\n",
            "You are logged in as: [chansung.tester.1013@gmail.com].\n",
            "\n",
            "Pick cloud project to use: \n",
            " [1] phonic-agility-338223\n",
            " [2] Create a new project\n",
            "Please enter numeric choice or text value (must exactly match list item):  1\n",
            "\n",
            "Your current project has been set to: [phonic-agility-338223].\n",
            "\n",
            "Do you want to configure a default Compute Region and Zone? (Y/n)?  Y\n",
            "\n",
            "Which Google Compute Engine zone would you like to use as project default?\n",
            "If you do not specify a zone via a command line flag while working with Compute \n",
            "Engine resources, the default is assumed.\n",
            " [1] us-east1-b\n",
            " [2] us-east1-c\n",
            " [3] us-east1-d\n",
            " [4] us-east4-c\n",
            " [5] us-east4-b\n",
            " [6] us-east4-a\n",
            " [7] us-central1-c\n",
            " [8] us-central1-a\n",
            " [9] us-central1-f\n",
            " [10] us-central1-b\n",
            " [11] us-west1-b\n",
            " [12] us-west1-c\n",
            " [13] us-west1-a\n",
            " [14] europe-west4-a\n",
            " [15] europe-west4-b\n",
            " [16] europe-west4-c\n",
            " [17] europe-west1-b\n",
            " [18] europe-west1-d\n",
            " [19] europe-west1-c\n",
            " [20] europe-west3-c\n",
            " [21] europe-west3-a\n",
            " [22] europe-west3-b\n",
            " [23] europe-west2-c\n",
            " [24] europe-west2-b\n",
            " [25] europe-west2-a\n",
            " [26] asia-east1-b\n",
            " [27] asia-east1-a\n",
            " [28] asia-east1-c\n",
            " [29] asia-southeast1-b\n",
            " [30] asia-southeast1-a\n",
            " [31] asia-southeast1-c\n",
            " [32] asia-northeast1-b\n",
            " [33] asia-northeast1-c\n",
            " [34] asia-northeast1-a\n",
            " [35] asia-south1-c\n",
            " [36] asia-south1-b\n",
            " [37] asia-south1-a\n",
            " [38] australia-southeast1-b\n",
            " [39] australia-southeast1-c\n",
            " [40] australia-southeast1-a\n",
            " [41] southamerica-east1-b\n",
            " [42] southamerica-east1-c\n",
            " [43] southamerica-east1-a\n",
            " [44] asia-east2-a\n",
            " [45] asia-east2-b\n",
            " [46] asia-east2-c\n",
            " [47] asia-northeast2-a\n",
            " [48] asia-northeast2-b\n",
            " [49] asia-northeast2-c\n",
            " [50] asia-northeast3-a\n",
            "Did not print [39] options.\n",
            "Too many options [89]. Enter \"list\" at prompt to print choices fully.\n",
            "Please enter numeric choice or text value (must exactly match list item):  8\n",
            "\n",
            "Your project default Compute Engine zone has been set to [us-central1-a].\n",
            "You can change it by running [gcloud config set compute/zone NAME].\n",
            "\n",
            "Your project default Compute Engine region has been set to [us-central1].\n",
            "You can change it by running [gcloud config set compute/region NAME].\n",
            "\n",
            "Your Google Cloud SDK is configured and ready to use!\n",
            "\n",
            "* Commands that require authentication will use chansung.tester.1013@gmail.com by default\n",
            "* Commands will reference project `phonic-agility-338223` by default\n",
            "* Compute Engine commands will use region `us-central1` by default\n",
            "* Compute Engine commands will use zone `us-central1-a` by default\n",
            "\n",
            "Run `gcloud help config` to learn how to change individual settings\n",
            "\n",
            "This gcloud configuration is called [hhh]. You can create additional configurations if you work with multiple accounts and/or projects.\n",
            "Run `gcloud topic configurations` to learn more.\n",
            "\n",
            "Some things to try next:\n",
            "\n",
            "* Run `gcloud --help` to see the Cloud Platform services you can interact with. And run `gcloud help COMMAND` to get help on any gcloud command.\n",
            "* Run `gcloud topic --help` to learn about advanced features of the SDK like arg files and output formatting\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "1082b7a1-bead-49b9-9684-af01075c2348",
      "metadata": {
        "id": "1082b7a1-bead-49b9-9684-af01075c2348"
      },
      "outputs": [],
      "source": [
        "import kfp\n",
        "from google.cloud import aiplatform\n",
        "from google_cloud_pipeline_components import aiplatform as gcc_aip\n",
        "from kfp.v2.dsl import component"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#@title GCS\n",
        "PROJECT_ID = 'phonic-agility-338223'  #@param {type:\"string\"}\n",
        "PIPELINE_ROOT_PATH = 'gs://my-pipeline-1012'  #@param {type:\"string\"}\n",
        "PIPELINE_NAME = 'cifar10-pipeline-automl' #@param {type:\"string\"}"
      ],
      "metadata": {
        "id": "VjTTAQMQno8J"
      },
      "id": "VjTTAQMQno8J",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Custom Components for retraining"
      ],
      "metadata": {
        "id": "NQR4xj85IRW-"
      },
      "id": "NQR4xj85IRW-"
    },
    {
      "cell_type": "markdown",
      "source": [
        "## get_dataset_id component\n",
        "\n",
        "to get dataset ID if one exists"
      ],
      "metadata": {
        "id": "R_h2ojnMIWlR"
      },
      "id": "R_h2ojnMIWlR"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "19c122f3-525c-4720-a0dd-214de415bcdc",
      "metadata": {
        "id": "19c122f3-525c-4720-a0dd-214de415bcdc"
      },
      "outputs": [],
      "source": [
        "from kfp.v2.dsl import Artifact, Output\n",
        "\n",
        "@component(\n",
        "    packages_to_install=[\"google-cloud-aiplatform\", \"google-cloud-pipeline-components\"]\n",
        ")\n",
        "def get_dataset_id(project_id: str, \n",
        "                  location: str,\n",
        "                  dataset_name: str,\n",
        "                  dataset_path: str,\n",
        "                  dataset: Output[Artifact]) -> str:\n",
        "    from google.cloud import aiplatform\n",
        "    from google.cloud.aiplatform.datasets.image_dataset import ImageDataset\n",
        "    from google_cloud_pipeline_components.types.artifact_types import VertexDataset\n",
        "\n",
        "    \n",
        "    aiplatform.init(project=project_id, location=location)\n",
        "    \n",
        "    datasets = aiplatform.ImageDataset.list(project=project_id,\n",
        "                                            location=location,\n",
        "                                            filter=f'display_name={dataset_name}')\n",
        "    \n",
        "    if len(datasets) > 0:\n",
        "        dataset.metadata['resourceName'] = f'projects/{project_id}/locations/{location}/datasets/{datasets[0].name}'\n",
        "        return f'projects/{project_id}/locations/{location}/datasets/{datasets[0].name}'\n",
        "    else:\n",
        "        return 'None'"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from kfp.v2.dsl import Artifact, Output\n",
        "\n",
        "@component(\n",
        "    packages_to_install=[\"google-cloud-aiplatform\", \"google-cloud-pipeline-components\"]\n",
        ")\n",
        "def get_model_id(project_id: str, \n",
        "                 location: str,\n",
        "                 model_name: str,\n",
        "                 model: Output[Artifact]) -> str:\n",
        "    from google.cloud import aiplatform\n",
        "    from google_cloud_pipeline_components.types.artifact_types import VertexModel\n",
        "    \n",
        "    aiplatform.init(project=project_id, location=location)\n",
        "    \n",
        "    models = aiplatform.Model.list(project=project_id,\n",
        "                                   location=location,\n",
        "                                   filter=f'display_name={model_name}')\n",
        "    \n",
        "    if len(models) > 0:\n",
        "        model.metadata['resourceName'] = f'projects/{project_id}/locations/{location}/models/{models[0].name}'\n",
        "        return f'projects/{project_id}/locations/{location}/models/{models[0].name}'\n",
        "    else:\n",
        "        return 'None'"
      ],
      "metadata": {
        "id": "-l3iMiWjE33L"
      },
      "id": "-l3iMiWjE33L",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Pipeline"
      ],
      "metadata": {
        "id": "x_2PvyD5JepU"
      },
      "id": "x_2PvyD5JepU"
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Define pipeline"
      ],
      "metadata": {
        "id": "WnZ5neWXJkEL"
      },
      "id": "WnZ5neWXJkEL"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "e005c3ff-b24d-4389-8313-5d74d05a8f7b",
      "metadata": {
        "id": "e005c3ff-b24d-4389-8313-5d74d05a8f7b"
      },
      "outputs": [],
      "source": [
        "from google.cloud.aiplatform.datasets.image_dataset import ImageDataset\n",
        "from google_cloud_pipeline_components.types.artifact_types import VertexDataset\n",
        "\n",
        "@kfp.dsl.pipeline(\n",
        "    name=PIPELINE_NAME,\n",
        "    pipeline_root=PIPELINE_ROOT_PATH)\n",
        "def pipeline(project_id: str, \n",
        "             location: str,\n",
        "             dataset_name: str,\n",
        "             dataset_path: str,\n",
        "             base_model_name: str):\n",
        "    \n",
        "    dataset_op = get_dataset_id(project_id=project_id,\n",
        "                                location=location,\n",
        "                                dataset_name=dataset_name,\n",
        "                                dataset_path=dataset_path)\n",
        "    \n",
        "    with kfp.dsl.Condition(dataset_op.outputs['Output'] == 'None', name=\"create dataset\"):\n",
        "        ds_op = gcc_aip.ImageDatasetCreateOp(\n",
        "            project=project_id,\n",
        "            display_name=dataset_name,\n",
        "            gcs_source=dataset_path,\n",
        "            import_schema_uri=aiplatform.schema.dataset.ioformat.image.single_label_classification,\n",
        "        )\n",
        "        ds_op.after(dataset_op)\n",
        "        \n",
        "        training_job_run_op = gcc_aip.AutoMLImageTrainingJobRunOp(\n",
        "            project=project_id,\n",
        "            display_name=\"train-cifar10-automl\",\n",
        "            prediction_type=\"classification\",\n",
        "            model_type=\"CLOUD\",\n",
        "            base_model=None,\n",
        "            dataset=ds_op.outputs[\"dataset\"],\n",
        "            model_display_name=\"cifar10-model\",\n",
        "            training_fraction_split=0.6,\n",
        "            validation_fraction_split=0.2,\n",
        "            test_fraction_split=0.2,\n",
        "            budget_milli_node_hours=8000,\n",
        "        )\n",
        "        training_job_run_op.after(ds_op)\n",
        "\n",
        "        create_endpoint_op = gcc_aip.EndpointCreateOp(\n",
        "            project=project_id,\n",
        "            display_name = \"cifar10-automl-endpoint\",\n",
        "        )\n",
        "        create_endpoint_op.after(training_job_run_op)\n",
        "\n",
        "        model_deploy_op = gcc_aip.ModelDeployOp(\n",
        "            model=training_job_run_op.outputs[\"model\"],\n",
        "            endpoint=create_endpoint_op.outputs['endpoint'],\n",
        "            automatic_resources_min_replica_count=1,\n",
        "            automatic_resources_max_replica_count=1,\n",
        "        )\n",
        "        model_deploy_op.after(create_endpoint_op)\n",
        "\n",
        "    with kfp.dsl.Condition(dataset_op.outputs['Output'] != 'None', name=\"update dataset\"):\n",
        "        ds_op = gcc_aip.ImageDatasetImportDataOp(\n",
        "            dataset=dataset_op.outputs['dataset'],\n",
        "            gcs_source=dataset_path,\n",
        "            import_schema_uri=aiplatform.schema.dataset.ioformat.image.single_label_classification\n",
        "        )\n",
        "        ds_op.after(dataset_op)\n",
        "\n",
        "        model_op = get_model_id(\n",
        "            project_id=project_id,\n",
        "            location=location,\n",
        "            model_name=base_model_name\n",
        "        )\n",
        "        model_op.after(ds_op)\n",
        "\n",
        "        with kfp.dsl.Condition(model_op.outputs['Output'] != 'None', name='model exist'):\n",
        "          training_job_run_op = gcc_aip.AutoMLImageTrainingJobRunOp(\n",
        "              project=project_id,\n",
        "              display_name=\"train-cifar10-automl\",\n",
        "              prediction_type=\"classification\",\n",
        "              model_type=\"CLOUD\",\n",
        "              base_model=model_op.outputs['model'],\n",
        "              dataset=ds_op.outputs[\"dataset\"],\n",
        "              model_display_name=\"cifar10-model\",\n",
        "              training_fraction_split=0.6,\n",
        "              validation_fraction_split=0.2,\n",
        "              test_fraction_split=0.2,\n",
        "              budget_milli_node_hours=8000,\n",
        "          )\n",
        "          training_job_run_op.after(model_op)\n",
        "\n",
        "          create_endpoint_op = gcc_aip.EndpointCreateOp(\n",
        "              project=project_id,\n",
        "              display_name = \"cifar10-automl-endpoint\",\n",
        "          )\n",
        "          create_endpoint_op.after(training_job_run_op)\n",
        "\n",
        "          model_deploy_op = gcc_aip.ModelDeployOp(\n",
        "              model=training_job_run_op.outputs[\"model\"],\n",
        "              endpoint=create_endpoint_op.outputs['endpoint'],\n",
        "              automatic_resources_min_replica_count=1,\n",
        "              automatic_resources_max_replica_count=1,\n",
        "              traffic_split={\"0\": 100},\n",
        "          )\n",
        "          model_deploy_op.after(create_endpoint_op)      \n",
        "\n",
        "        with kfp.dsl.Condition(model_op.outputs['Output'] == 'None', name='model not exist'):\n",
        "          training_job_run_op = gcc_aip.AutoMLImageTrainingJobRunOp(\n",
        "              project=project_id,\n",
        "              display_name=\"train-cifar10-automl\",\n",
        "              prediction_type=\"classification\",\n",
        "              model_type=\"CLOUD\",\n",
        "              base_model=None,\n",
        "              dataset=ds_op.outputs[\"dataset\"],\n",
        "              model_display_name=\"cifar10-model\",\n",
        "              training_fraction_split=0.6,\n",
        "              validation_fraction_split=0.2,\n",
        "              test_fraction_split=0.2,\n",
        "              budget_milli_node_hours=8000,\n",
        "          )\n",
        "          training_job_run_op.after(model_op)\n",
        "\n",
        "          create_endpoint_op = gcc_aip.EndpointCreateOp(\n",
        "              project=project_id,\n",
        "              display_name = \"cifar10-automl-endpoint\",\n",
        "          )\n",
        "          create_endpoint_op.after(training_job_run_op)\n",
        "\n",
        "          model_deploy_op = gcc_aip.ModelDeployOp(\n",
        "              model=training_job_run_op.outputs[\"model\"],\n",
        "              endpoint=create_endpoint_op.outputs['endpoint'],\n",
        "              automatic_resources_min_replica_count=1,\n",
        "              automatic_resources_max_replica_count=1,\n",
        "          )\n",
        "          model_deploy_op.after(create_endpoint_op)          "
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Compile pipeline"
      ],
      "metadata": {
        "id": "glsKYKZiJnXx"
      },
      "id": "glsKYKZiJnXx"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "82f31257-ba86-4202-af07-2eef1ae45c69",
      "metadata": {
        "id": "82f31257-ba86-4202-af07-2eef1ae45c69"
      },
      "outputs": [],
      "source": [
        "pipeline_spec_file = 'cifar10_classification_pipeline.json'"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "773accb9-6c09-4b9f-8fa3-796808e9236a",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "773accb9-6c09-4b9f-8fa3-796808e9236a",
        "outputId": "01877551-2c3a-4d18-f435-5a709a9c8b22"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/kfp/v2/compiler/compiler.py:1266: FutureWarning: APIs imported from the v1 namespace (e.g. kfp.dsl, kfp.components, etc) will not be supported by the v2 compiler since v2.0.0\n",
            "  category=FutureWarning,\n"
          ]
        }
      ],
      "source": [
        "from kfp.v2 import compiler\n",
        "\n",
        "compiler.Compiler().compile(\n",
        "        pipeline_func=pipeline,\n",
        "        package_path=pipeline_spec_file)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Create GCP Bucket & Copy the pipeline spec "
      ],
      "metadata": {
        "id": "7vU64-tfJsK5"
      },
      "id": "7vU64-tfJsK5"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "69b948d7-e037-418f-bf08-1bdfc0d71a02",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "69b948d7-e037-418f-bf08-1bdfc0d71a02",
        "outputId": "4ffc9c90-5b69-4bbf-ac3c-a959313614aa"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Creating gs://my-pipeline-1012/...\n",
            "ServiceException: 409 A Cloud Storage bucket named 'my-pipeline-1012' already exists. Try another name. Bucket names must be globally unique across all Google Cloud projects, including those outside of your organization.\n",
            "Copying file://cifar10_classification_pipeline.json [Content-Type=application/json]...\n",
            "/ [1 files][ 68.2 KiB/ 68.2 KiB]                                                \n",
            "Operation completed over 1 objects/68.2 KiB.                                     \n"
          ]
        }
      ],
      "source": [
        "#@title GCS\n",
        "REGION = \"us-central1\" #@param {type:\"string\"}\n",
        "\n",
        "!gsutil mb -l {REGION} {PIPELINE_ROOT_PATH}\n",
        "!gsutil cp {pipeline_spec_file} {PIPELINE_ROOT_PATH}/"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Test Pipeline"
      ],
      "metadata": {
        "id": "H1Lqoq7WJ1Qy"
      },
      "id": "H1Lqoq7WJ1Qy"
    },
    {
      "cell_type": "code",
      "source": [
        "!gcloud config set project {PROJECT_ID}"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nWAs8KHgJLir",
        "outputId": "2f819f0a-0612-439c-b1e2-4ef9fc4d93a9"
      },
      "id": "nWAs8KHgJLir",
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Updated property [core/project].\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import os\n",
        "os.environ['GOOGLE_CLOUD_PROJECT'] = PROJECT_ID"
      ],
      "metadata": {
        "id": "4Z0qGNlNJkDD"
      },
      "id": "4Z0qGNlNJkDD",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "387dfc25-1433-4729-8aeb-fdf03890b4cc",
      "metadata": {
        "id": "387dfc25-1433-4729-8aeb-fdf03890b4cc",
        "outputId": "a0111ccb-875c-4ae9-e9db-a04e2ce45fd1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "INFO:google.cloud.aiplatform.pipeline_jobs:Creating PipelineJob\n",
            "INFO:google.cloud.aiplatform.pipeline_jobs:PipelineJob created. Resource name: projects/161071819378/locations/us-central1/pipelineJobs/cifar10-pipeline-automl-20220114011458\n",
            "INFO:google.cloud.aiplatform.pipeline_jobs:To use this PipelineJob in another session:\n",
            "INFO:google.cloud.aiplatform.pipeline_jobs:pipeline_job = aiplatform.PipelineJob.get('projects/161071819378/locations/us-central1/pipelineJobs/cifar10-pipeline-automl-20220114011458')\n",
            "INFO:google.cloud.aiplatform.pipeline_jobs:View Pipeline Job:\n",
            "https://console.cloud.google.com/vertex-ai/locations/us-central1/pipelines/runs/cifar10-pipeline-automl-20220114011458?project=161071819378\n"
          ]
        }
      ],
      "source": [
        "location = 'us-central1'\n",
        "\n",
        "job = aiplatform.PipelineJob(\n",
        "    display_name=\"automl-image-training-v2\",\n",
        "    template_path=\"cifar10_classification_pipeline.json\",\n",
        "    pipeline_root=PIPELINE_ROOT_PATH,\n",
        "    parameter_values={\n",
        "        'project_id': PROJECT_ID,\n",
        "        'location': REGION,\n",
        "        'dataset_name': 'cifar10-dataset',\n",
        "        'dataset_path': 'gs://my-cifar10-dataset/span-1/annotations.csv',\n",
        "        'base_model_name': 'cifar10-model',\n",
        "    }\n",
        ")\n",
        "\n",
        "job.submit()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "5a205258-d2cc-48b1-a857-f41c9c7dff5f",
      "metadata": {
        "id": "5a205258-d2cc-48b1-a857-f41c9c7dff5f",
        "outputId": "ef87e45a-a1fe-483e-eb33-7ce9b6b7937e"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "INFO:google.cloud.aiplatform.pipeline_jobs:Creating PipelineJob\n"
          ]
        },
        {
          "ename": "InvalidArgument",
          "evalue": "400 Some input parameters of the PipelineSpec.root are missing from PipelineJob.runtimeConfig.parameters: ([base_model_name])",
          "output_type": "error",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31m_InactiveRpcError\u001b[0m                         Traceback (most recent call last)",
            "\u001b[0;32m/opt/conda/lib/python3.7/site-packages/google/api_core/grpc_helpers.py\u001b[0m in \u001b[0;36merror_remapped_callable\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m     65\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0merror_remapped_callable\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 66\u001b[0;31m         \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     67\u001b[0m             \u001b[0;32mreturn\u001b[0m \u001b[0mcallable_\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/opt/conda/lib/python3.7/site-packages/grpc/_channel.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, request, timeout, metadata, credentials, wait_for_ready, compression)\u001b[0m\n\u001b[1;32m    945\u001b[0m                                       wait_for_ready, compression)\n\u001b[0;32m--> 946\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0m_end_unary_response_blocking\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstate\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcall\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    947\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/opt/conda/lib/python3.7/site-packages/grpc/_channel.py\u001b[0m in \u001b[0;36m_end_unary_response_blocking\u001b[0;34m(state, call, with_call, deadline)\u001b[0m\n\u001b[1;32m    848\u001b[0m     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 849\u001b[0;31m         \u001b[0;32mraise\u001b[0m \u001b[0m_InactiveRpcError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstate\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    850\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31m_InactiveRpcError\u001b[0m: <_InactiveRpcError of RPC that terminated with:\n\tstatus = StatusCode.INVALID_ARGUMENT\n\tdetails = \"Some input parameters of the PipelineSpec.root are missing from PipelineJob.runtimeConfig.parameters: ([base_model_name])\"\n\tdebug_error_string = \"{\"created\":\"@1642037466.652545421\",\"description\":\"Error received from peer ipv4:74.125.142.95:443\",\"file\":\"src/core/lib/surface/call.cc\",\"file_line\":1063,\"grpc_message\":\"Some input parameters of the PipelineSpec.root are missing from PipelineJob.runtimeConfig.parameters: ([base_model_name])\",\"grpc_status\":3}\"\n>",
            "\nThe above exception was the direct cause of the following exception:\n",
            "\u001b[0;31mInvalidArgument\u001b[0m                           Traceback (most recent call last)",
            "\u001b[0;32m/tmp/ipykernel_16612/2315919861.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m     13\u001b[0m )\n\u001b[1;32m     14\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 15\u001b[0;31m \u001b[0mjob\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msubmit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;32m/opt/conda/lib/python3.7/site-packages/google/cloud/aiplatform/pipeline_jobs.py\u001b[0m in \u001b[0;36msubmit\u001b[0;34m(self, service_account, network)\u001b[0m\n\u001b[1;32m    279\u001b[0m             \u001b[0mparent\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_parent\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    280\u001b[0m             \u001b[0mpipeline_job\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_gca_resource\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 281\u001b[0;31m             \u001b[0mpipeline_job_id\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mjob_id\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    282\u001b[0m         )\n\u001b[1;32m    283\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/opt/conda/lib/python3.7/site-packages/google/cloud/aiplatform_v1/services/pipeline_service/client.py\u001b[0m in \u001b[0;36mcreate_pipeline_job\u001b[0;34m(self, request, parent, pipeline_job, pipeline_job_id, retry, timeout, metadata)\u001b[0m\n\u001b[1;32m   1025\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1026\u001b[0m         \u001b[0;31m# Send the request.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1027\u001b[0;31m         \u001b[0mresponse\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mrpc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrequest\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mretry\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mretry\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtimeout\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mtimeout\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmetadata\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mmetadata\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1028\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1029\u001b[0m         \u001b[0;31m# Done; return the response.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/opt/conda/lib/python3.7/site-packages/google/api_core/gapic_v1/method.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, timeout, retry, *args, **kwargs)\u001b[0m\n\u001b[1;32m    152\u001b[0m     \u001b[0mclient_info\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mclient_info\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mDEFAULT_CLIENT_INFO\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    153\u001b[0m ):\n\u001b[0;32m--> 154\u001b[0;31m     \"\"\"Wrap an RPC method with common behavior.\n\u001b[0m\u001b[1;32m    155\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    156\u001b[0m     \u001b[0mThis\u001b[0m \u001b[0mapplies\u001b[0m \u001b[0mcommon\u001b[0m \u001b[0merror\u001b[0m \u001b[0mwrapping\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mretry\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mtimeout\u001b[0m \u001b[0mbehavior\u001b[0m \u001b[0ma\u001b[0m \u001b[0mfunction\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/opt/conda/lib/python3.7/site-packages/google/api_core/grpc_helpers.py\u001b[0m in \u001b[0;36merror_remapped_callable\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m     66\u001b[0m         \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     67\u001b[0m             \u001b[0;32mreturn\u001b[0m \u001b[0mcallable_\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 68\u001b[0;31m         \u001b[0;32mexcept\u001b[0m \u001b[0mgrpc\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mRpcError\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mexc\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     69\u001b[0m             \u001b[0msix\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mraise_from\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mexceptions\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfrom_grpc_error\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mexc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mexc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     70\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mInvalidArgument\u001b[0m: 400 Some input parameters of the PipelineSpec.root are missing from PipelineJob.runtimeConfig.parameters: ([base_model_name])"
          ]
        }
      ],
      "source": [
        "location = 'us-central1'\n",
        "\n",
        "job = aiplatform.PipelineJob(\n",
        "    display_name=\"automl-image-training-v2\",\n",
        "    template_path=\"cifar10_classification_pipeline.json\",\n",
        "    pipeline_root=pipeline_root_path,\n",
        "    parameter_values={\n",
        "        'project_id': project_id,\n",
        "        'location': location,\n",
        "        'dataset_name': 'cifar10-dataset2',\n",
        "        'dataset_path': 'gs://my-cifar10-dataset/span-2/annotations.csv',\n",
        "    }\n",
        ")\n",
        "\n",
        "job.submit()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "368237c4-d57b-4415-9c76-5b477cffb2eb",
      "metadata": {
        "id": "368237c4-d57b-4415-9c76-5b477cffb2eb"
      },
      "outputs": [],
      "source": [
        ""
      ]
    }
  ],
  "metadata": {
    "environment": {
      "kernel": "python3",
      "name": "tf2-gpu.2-6.m87",
      "type": "gcloud",
      "uri": "gcr.io/deeplearning-platform-release/tf2-gpu.2-6:m87"
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.12"
    },
    "colab": {
      "name": "cifar-10-vertex-autoML-pipeline",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}