fix, chore, refactor(egen): REST api fixes, KFP v2 refactor, new template #3180
Google Cloud Build / vertex-ai-notebook-execution-test (python-docs-samples-tests)
succeeded
Jul 1, 2024 in 14m 56s
Summary
Build Information
| Trigger | vertex-ai-notebook-execution-test |
| Build | eec7893f-2557-478c-9676-3a73ee22f9fa |
| Start | 2024-07-01T12:58:56-07:00 |
| Duration | 14m14.2s |
| Status | SUCCESS |
Steps
| Step | Status | Duration |
|---|---|---|
| gcr.io/cloud-devrel-public-resources/python-samples-testing-docker:latest | SUCCESS | 45.702s |
| gcr.io/cloud-devrel-public-resources/python-samples-testing-docker:latest | SUCCESS | 576ms |
| gcr.io/cloud-builders/git | SUCCESS | 7.956s |
| gcr.io/cloud-devrel-public-resources/python-samples-testing-docker:latest | SUCCESS | 5.318s |
| gcr.io/cloud-devrel-public-resources/python-samples-testing-docker:latest | SUCCESS | 1m44.126s |
| gcr.io/cloud-devrel-public-resources/python-samples-testing-docker:latest | SUCCESS | 11m24.666s |
Details
starting build "eec7893f-2557-478c-9676-3a73ee22f9fa"
FETCHSOURCE
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint: git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint: git branch -m <name>
Initialized empty Git repository in /workspace/.git/
From https://github.com/GoogleCloudPlatform/vertex-ai-samples
* branch 93969942996008af87452ef58e3f06d84b67c282 -> FETCH_HEAD
HEAD is now at 9396994 chore: addresses the review comments
BUILD
Starting Step #0
Step #0: Pulling image: gcr.io/cloud-devrel-public-resources/python-samples-testing-docker:latest
Step #0: latest: Pulling from cloud-devrel-public-resources/python-samples-testing-docker
Step #0: 80888bc6716f: Pulling fs layer
Step #0: 37c642b4a269: Pulling fs layer
Step #0: f580537b041d: Pulling fs layer
Step #0: fae84b036976: Pulling fs layer
Step #0: 9d85811bd886: Pulling fs layer
Step #0: fae84b036976: Waiting
Step #0: d37fd6b982d1: Pulling fs layer
Step #0: a57f31145b32: Pulling fs layer
Step #0: 72e9bda57337: Pulling fs layer
Step #0: 9d85811bd886: Waiting
Step #0: 1c8b4c0cd55e: Pulling fs layer
Step #0: d0fbf2665f1f: Pulling fs layer
Step #0: a3295a1471c5: Pulling fs layer
Step #0: d37fd6b982d1: Waiting
Step #0: a57f31145b32: Waiting
Step #0: 72e9bda57337: Waiting
Step #0: d0fbf2665f1f: Waiting
Step #0: 1c8b4c0cd55e: Waiting
Step #0: 9e5bd4e62f90: Pulling fs layer
Step #0: df06b8bdaab2: Pulling fs layer
Step #0: 78f9630f5203: Pulling fs layer
Step #0: 240513ab0076: Pulling fs layer
Step #0: a3295a1471c5: Waiting
Step #0: ad94e8022013: Pulling fs layer
Step #0: 02dc8bec8c15: Pulling fs layer
Step #0: 78f9630f5203: Waiting
Step #0: df06b8bdaab2: Waiting
Step #0: 9e5bd4e62f90: Waiting
Step #0: cd968686a9aa: Pulling fs layer
Step #0: cd0e55646a1f: Pulling fs layer
Step #0: 02dc8bec8c15: Waiting
Step #0: cd0e55646a1f: Waiting
Step #0: cd968686a9aa: Waiting
Step #0: ec6fbc22aba4: Pulling fs layer
Step #0: 8a9d0305d6b2: Pulling fs layer
Step #0: bf9b9f9ad000: Pulling fs layer
Step #0: f47546687712: Pulling fs layer
Step #0: 49119bc701a5: Pulling fs layer
Step #0: ec6fbc22aba4: Waiting
Step #0: 8a9d0305d6b2: Waiting
Step #0: bf9b9f9ad000: Waiting
Step #0: f47546687712: Waiting
Step #0: 93f6be0fc88f: Pulling fs layer
Step #0: d0773f577b01: Pulling fs layer
Step #0: 49119bc701a5: Waiting
Step #0: 93f6be0fc88f: Waiting
Step #0: d0773f577b01: Waiting
Step #0: f580537b041d: Download complete
Step #0: 80888bc6716f: Verifying Checksum
Step #0: 80888bc6716f: Download complete
Step #0: fae84b036976: Verifying Checksum
Step #0: fae84b036976: Download complete
Step #0: d37fd6b982d1: Verifying Checksum
Step #0: d37fd6b982d1: Download complete
Step #0: 80888bc6716f: Pull complete
Step #0: 9d85811bd886: Verifying Checksum
Step #0: 9d85811bd886: Download complete
Step #0: a57f31145b32: Verifying Checksum
Step #0: a57f31145b32: Download complete
Step #0: 72e9bda57337: Verifying Checksum
Step #0: 72e9bda57337: Download complete
Step #0: d0fbf2665f1f: Verifying Checksum
Step #0: d0fbf2665f1f: Download complete
Step #0: a3295a1471c5: Verifying Checksum
Step #0: a3295a1471c5: Download complete
Step #0: 9e5bd4e62f90: Verifying Checksum
Step #0: 9e5bd4e62f90: Download complete
Step #0: 37c642b4a269: Verifying Checksum
Step #0: 37c642b4a269: Download complete
Step #0: df06b8bdaab2: Verifying Checksum
Step #0: df06b8bdaab2: Download complete
Step #0: 78f9630f5203: Verifying Checksum
Step #0: 78f9630f5203: Download complete
Step #0: ad94e8022013: Verifying Checksum
Step #0: ad94e8022013: Download complete
Step #0: 02dc8bec8c15: Download complete
Step #0: 240513ab0076: Verifying Checksum
Step #0: 240513ab0076: Download complete
Step #0: cd968686a9aa: Verifying Checksum
Step #0: cd968686a9aa: Download complete
Step #0: cd0e55646a1f: Verifying Checksum
Step #0: cd0e55646a1f: Download complete
Step #0: ec6fbc22aba4: Verifying Checksum
Step #0: ec6fbc22aba4: Download complete
Step #0: bf9b9f9ad000: Download complete
Step #0: 8a9d0305d6b2: Verifying Checksum
Step #0: 8a9d0305d6b2: Download complete
Step #0: f47546687712: Download complete
Step #0: 93f6be0fc88f: Download complete
Step #0: d0773f577b01: Verifying Checksum
Step #0: d0773f577b01: Download complete
Step #0: 1c8b4c0cd55e: Verifying Checksum
Step #0: 1c8b4c0cd55e: Download complete
Step #0: 49119bc701a5: Verifying Checksum
Step #0: 49119bc701a5: Download complete
Step #0: 37c642b4a269: Pull complete
Step #0: f580537b041d: Pull complete
Step #0: fae84b036976: Pull complete
Step #0: 9d85811bd886: Pull complete
Step #0: d37fd6b982d1: Pull complete
Step #0: a57f31145b32: Pull complete
Step #0: 72e9bda57337: Pull complete
Step #0: 1c8b4c0cd55e: Pull complete
Step #0: d0fbf2665f1f: Pull complete
Step #0: a3295a1471c5: Pull complete
Step #0: 9e5bd4e62f90: Pull complete
Step #0: df06b8bdaab2: Pull complete
Step #0: 78f9630f5203: Pull complete
Step #0: 240513ab0076: Pull complete
Step #0: ad94e8022013: Pull complete
Step #0: 02dc8bec8c15: Pull complete
Step #0: cd968686a9aa: Pull complete
Step #0: cd0e55646a1f: Pull complete
Step #0: ec6fbc22aba4: Pull complete
Step #0: 8a9d0305d6b2: Pull complete
Step #0: bf9b9f9ad000: Pull complete
Step #0: f47546687712: Pull complete
Step #0: 49119bc701a5: Pull complete
Step #0: 93f6be0fc88f: Pull complete
Step #0: d0773f577b01: Pull complete
Step #0: Digest: sha256:264987e3a848c6c173ef24fd37de448b06f15302adb3b81d61cb99ff2b4b3a00
Step #0: Status: Downloaded newer image for gcr.io/cloud-devrel-public-resources/python-samples-testing-docker:latest
Step #0: gcr.io/cloud-devrel-public-resources/python-samples-testing-docker:latest
Step #0: [core]
Step #0: account = 1012616486416@cloudbuild.gserviceaccount.com
Step #0: disable_prompts = True
Step #0: disable_usage_reporting = True
Step #0: project = python-docs-samples-tests
Step #0:
Step #0: Your active configuration is: [default]
Finished Step #0
Starting Step #1
Step #1: Already have image (with digest): gcr.io/cloud-devrel-public-resources/python-samples-testing-docker:latest
Step #1: Python version acceptable: 3.12.0 (main, Apr 17 2024, 18:17:28) [GCC 9.4.0]
Finished Step #1
Starting Step #2
Step #2: Already have image (with digest): gcr.io/cloud-builders/git
Step #2:
Step #2: ***** NOTICE *****
Step #2:
Step #2: Official `cloud-sdk` images, including multiple tagged versions across multiple
Step #2: platforms, can be found at
Step #2: https://github.com/GoogleCloudPlatform/cloud-sdk-docker and may be more suitable
Step #2: for some use cases when interacting with Cloud Source Repositories.
Step #2:
Step #2: For additional information, please visit
Step #2: https://github.com/GoogleCloudPlatform/cloud-builders/tree/master/git
Step #2:
Step #2: ***** END OF NOTICE *****
Step #2:
Finished Step #2
Starting Step #3
Step #3: Already have image (with digest): gcr.io/cloud-devrel-public-resources/python-samples-testing-docker:latest
Finished Step #3
Starting Step #4
Step #4: Already have image (with digest): gcr.io/cloud-devrel-public-resources/python-samples-testing-docker:latest
Finished Step #4
Starting Step #5
Step #5: Already have image (with digest): gcr.io/cloud-devrel-public-resources/python-samples-testing-docker:latest
Step #5: /workspace/.cloud-build/execute_changed_notebooks_helper.py:241: SyntaxWarning: invalid escape sequence '\d'
Step #5: "python version = (\d+\.\d+)", markdown, flags=re.IGNORECASE
Step #5: Copying file://source_archived_1df44adf-4e96-4981-8040-1f2e83167a91.tar.gz [Content-Type=application/x-tar]...
Step #5: / [0 files][ 0.0 B/ 27.8 MiB]
/ [1 files][ 27.8 MiB/ 27.8 MiB]
Step #5: Operation completed over 1 objects/27.8 MiB.
Step #5: Checking folders: ['notebooks/official', 'notebooks/notebook_template.ipynb']
Step #5: Looking for notebooks that changed from branch: 7a15b8807136041f766ce0eebb0675a0f5e9ffc4
Step #5: Found 1 notebooks:
Step #5: notebooks/official/pipelines/pipelines_intro_kfp.ipynb
Step #5: Error: file does not exist .
Step #5: Found 1 modified notebooks: ['notebooks/official/pipelines/pipelines_intro_kfp.ipynb']
Step #5: Running notebook: notebooks/official/pipelines/pipelines_intro_kfp.ipynb
Step #5: Running notebook with python 3.9
Step #5: Uploaded source code archive to gs://cloud-build-notebooks-presubmit/code_archives/source_archived_1df44adf-4e96-4981-8040-1f2e83167a91.tar.gz
Step #5: notebooks/official/pipelines/pipelines_intro_kfp.ipynb PASSED in 00:11:19.
Step #5:
Step #5: === RESULTS ===
Step #5:
Step #5: build_tag status duration log_url output_uri output_uri_web logs_bucket
Step #5: ------------------------- -------- ---------- --------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------ -------------------------------------------------------------------------------------------------------------------------------------------------------------- --------------------------------------------------------
Step #5: pipelines_intro_kfp.ipynb PASSED 00:11:19 https://console.cloud.google.com/cloud-build/builds;region=us-central1/7c739286-830a-4b5b-8fb4-dda9f29c90a7?project=1012616486416 gs://cloud-build-notebooks-presubmit/executed_notebooks/PR_3180/BUILD_eec7893f-2557-478c-9676-3a73ee22f9fa/pipelines_intro_kfp.ipynb https://storage.googleapis.com/cloud-build-notebooks-presubmit/executed_notebooks/PR_3180/BUILD_eec7893f-2557-478c-9676-3a73ee22f9fa/pipelines_intro_kfp.ipynb gs://1012616486416.cloudbuild-logs.googleusercontent.com
Step #5: ====================================================================================================
Step #5: The notebook execution build log:
Step #5:
Step #5: ====================================================================================================
Step #5: Downloaded storage object log-7c739286-830a-4b5b-8fb4-dda9f29c90a7.txt from bucket 1012616486416.cloudbuild-logs.googleusercontent.com.
Step #5: Starting Step #4
Step #5: Step #4: Already have image (with digest): gcr.io/cloud-devrel-public-resources/python-samples-testing-docker:latest
Step #5: Step #4:
Step #5: Step #4: === DOWNLOAD EXECUTED NOTEBOOK ===
Step #5: Step #4:
Step #5: Step #4: Please debug the executed notebook by downloading the executed notebook:
Step #5: Step #4: Option 1. Using gsutil. Run the following command in your terminal.
Step #5: Step #4: gsutil cp "gs://cloud-build-notebooks-presubmit/executed_notebooks/PR_3180/BUILD_eec7893f-2557-478c-9676-3a73ee22f9fa/pipelines_intro_kfp.ipynb" .
Step #5: Step #4: Option 2. Using this link.
Step #5: Step #4: https://storage.googleapis.com/cloud-build-notebooks-presubmit/executed_notebooks/PR_3180/BUILD_eec7893f-2557-478c-9676-3a73ee22f9fa/pipelines_intro_kfp.ipynb
Step #5: Step #4:
Step #5: Step #4: ======
Step #5: Step #4:
Step #5: Step #4: Requirement already satisfied: google-cloud-aiplatform in ./workspace/env/lib/python3.9/site-packages (1.57.0)
Step #5: Step #4: Requirement already satisfied: google-cloud-storage in ./workspace/env/lib/python3.9/site-packages (2.17.0)
Step #5: Step #4: Collecting kfp
Step #5: Step #4: Downloading kfp-2.8.0.tar.gz (594 kB)
Step #5: Step #4: �[?25l �[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━�[0m �[32m0.0/594.4 kB�[0m �[31m?�[0m eta �[36m-:--:--�[0m
�[2K �[91m━━�[0m�[90m╺�[0m�[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━�[0m �[32m30.7/594.4 kB�[0m �[31m3.6 MB/s�[0m eta �[36m0:00:01�[0m
�[2K �[91m━━━━━━�[0m�[90m╺�[0m�[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━�[0m �[32m92.2/594.4 kB�[0m �[31m1.6 MB/s�[0m eta �[36m0:00:01�[0m
�[2K �[91m━━━━━━━━━━━━━━━�[0m�[90m╺�[0m�[90m━━━━━━━━━━━━━━━━━━━━━━━�[0m �[32m235.5/594.4 kB�[0m �[31m2.3 MB/s�[0m eta �[36m0:00:01�[0m
�[2K �[91m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━�[0m�[91m╸�[0m�[90m━━━━━�[0m �[32m512.0/594.4 kB�[0m �[31m3.6 MB/s�[0m eta �[36m0:00:01�[0m
�[2K �[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━�[0m �[32m594.4/594.4 kB�[0m �[31m3.5 MB/s�[0m eta �[36m0:00:00�[0m
Step #5: Step #4: �[?25h Installing build dependencies ... �[?25l-� �\� �|� �/� �-� �\� �done
Step #5: Step #4: �[?25h Getting requirements to build wheel ... �[?25l-� �done
Step #5: Step #4: �[?25h Preparing metadata (pyproject.toml) ... �[?25l-� �done
Step #5: Step #4: �[?25hCollecting google-cloud-pipeline-components
Step #5: Step #4: Downloading google_cloud_pipeline_components-2.15.0-py3-none-any.whl.metadata (5.8 kB)
Step #5: Step #4: Requirement already satisfied: google-api-core!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.34.1 in ./workspace/env/lib/python3.9/site-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.34.1->google-cloud-aiplatform) (2.19.1)
Step #5: Step #4: Requirement already satisfied: google-auth<3.0.0dev,>=2.14.1 in ./workspace/env/lib/python3.9/site-packages (from google-cloud-aiplatform) (2.31.0)
Step #5: Step #4: Requirement already satisfied: proto-plus<2.0.0dev,>=1.22.0 in ./workspace/env/lib/python3.9/site-packages (from google-cloud-aiplatform) (1.24.0)
Step #5: Step #4: Requirement already satisfied: protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5 in ./workspace/env/lib/python3.9/site-packages (from google-cloud-aiplatform) (4.25.3)
Step #5: Step #4: Requirement already satisfied: packaging>=14.3 in ./workspace/env/lib/python3.9/site-packages (from google-cloud-aiplatform) (24.1)
Step #5: Step #4: Requirement already satisfied: google-cloud-bigquery!=3.20.0,<4.0.0dev,>=1.15.0 in ./workspace/env/lib/python3.9/site-packages (from google-cloud-aiplatform) (3.25.0)
Step #5: Step #4: Requirement already satisfied: google-cloud-resource-manager<3.0.0dev,>=1.3.3 in ./workspace/env/lib/python3.9/site-packages (from google-cloud-aiplatform) (1.12.3)
Step #5: Step #4: Requirement already satisfied: shapely<3.0.0dev in ./workspace/env/lib/python3.9/site-packages (from google-cloud-aiplatform) (2.0.4)
Step #5: Step #4: Requirement already satisfied: pydantic<3 in ./workspace/env/lib/python3.9/site-packages (from google-cloud-aiplatform) (2.8.0)
Step #5: Step #4: Requirement already satisfied: docstring-parser<1 in ./workspace/env/lib/python3.9/site-packages (from google-cloud-aiplatform) (0.16)
Step #5: Step #4: Requirement already satisfied: google-cloud-core<3.0dev,>=2.3.0 in ./workspace/env/lib/python3.9/site-packages (from google-cloud-storage) (2.4.1)
Step #5: Step #4: Requirement already satisfied: google-resumable-media>=2.6.0 in ./workspace/env/lib/python3.9/site-packages (from google-cloud-storage) (2.7.1)
Step #5: Step #4: Requirement already satisfied: requests<3.0.0dev,>=2.18.0 in ./workspace/env/lib/python3.9/site-packages (from google-cloud-storage) (2.32.3)
Step #5: Step #4: Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in ./workspace/env/lib/python3.9/site-packages (from google-cloud-storage) (1.5.0)
Step #5: Step #4: Requirement already satisfied: click<9,>=8.0.0 in ./workspace/env/lib/python3.9/site-packages (from kfp) (8.1.7)
Step #5: Step #4: Collecting kfp-pipeline-spec==0.3.0 (from kfp)
Step #5: Step #4: Downloading kfp_pipeline_spec-0.3.0-py3-none-any.whl.metadata (329 bytes)
Step #5: Step #4: Collecting kfp-server-api<2.1.0,>=2.0.0 (from kfp)
Step #5: Step #4: Downloading kfp-server-api-2.0.5.tar.gz (63 kB)
Step #5: Step #4: �[?25l �[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━�[0m �[32m0.0/63.4 kB�[0m �[31m?�[0m eta �[36m-:--:--�[0m
�[2K �[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━�[0m �[32m63.4/63.4 kB�[0m �[31m10.0 MB/s�[0m eta �[36m0:00:00�[0m
Step #5: Step #4: �[?25h Installing build dependencies ... �[?25l-� �\� �|� �/� �done
Step #5: Step #4: �[?25h Getting requirements to build wheel ... �[?25l-� �done
Step #5: Step #4: �[?25h Preparing metadata (pyproject.toml) ... �[?25l-� �done
Step #5: Step #4: �[?25hCollecting kubernetes<27,>=8.0.0 (from kfp)
Step #5: Step #4: Downloading kubernetes-26.1.0-py2.py3-none-any.whl.metadata (1.5 kB)
Step #5: Step #4: Requirement already satisfied: PyYAML<7,>=5.3 in ./workspace/env/lib/python3.9/site-packages (from kfp) (6.0.1)
Step #5: Step #4: Collecting requests-toolbelt<1,>=0.8.0 (from kfp)
Step #5: Step #4: Downloading requests_toolbelt-0.10.1-py2.py3-none-any.whl.metadata (14 kB)
Step #5: Step #4: Requirement already satisfied: tabulate<1,>=0.8.6 in ./workspace/env/lib/python3.9/site-packages (from kfp) (0.9.0)
Step #5: Step #4: Collecting urllib3<2.0.0 (from kfp)
Step #5: Step #4: Downloading urllib3-1.26.19-py2.py3-none-any.whl.metadata (49 kB)
Step #5: Step #4: �[?25l �[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━�[0m �[32m0.0/49.3 kB�[0m �[31m?�[0m eta �[36m-:--:--�[0m
�[2K �[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━�[0m �[32m49.3/49.3 kB�[0m �[31m6.7 MB/s�[0m eta �[36m0:00:00�[0m
Step #5: Step #4: �[?25hRequirement already satisfied: Jinja2<4,>=3.1.2 in ./workspace/env/lib/python3.9/site-packages (from google-cloud-pipeline-components) (3.1.4)
Step #5: Step #4: Collecting kfp
Step #5: Step #4: Downloading kfp-2.7.0.tar.gz (441 kB)
Step #5: Step #4: �[?25l �[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━�[0m �[32m0.0/441.8 kB�[0m �[31m?�[0m eta �[36m-:--:--�[0m
�[2K �[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━�[0m �[32m441.8/441.8 kB�[0m �[31m25.4 MB/s�[0m eta �[36m0:00:00�[0m
Step #5: Step #4: �[?25h Installing build dependencies ... �[?25l-� �\� �|� �/� �done
Step #5: Step #4: �[?25h Getting requirements to build wheel ... �[?25l-� �done
Step #5: Step #4: �[?25h Preparing metadata (pyproject.toml) ... �[?25l-� �done
Step #5: Step #4: �[?25hRequirement already satisfied: googleapis-common-protos<2.0.dev0,>=1.56.2 in ./workspace/env/lib/python3.9/site-packages (from google-api-core!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.34.1->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.34.1->google-cloud-aiplatform) (1.63.2)
Step #5: Step #4: Requirement already satisfied: grpcio<2.0dev,>=1.33.2 in ./workspace/env/lib/python3.9/site-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.34.1->google-cloud-aiplatform) (1.64.1)
Step #5: Step #4: Requirement already satisfied: grpcio-status<2.0.dev0,>=1.33.2 in ./workspace/env/lib/python3.9/site-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.34.1->google-cloud-aiplatform) (1.62.2)
Step #5: Step #4: Requirement already satisfied: cachetools<6.0,>=2.0.0 in ./workspace/env/lib/python3.9/site-packages (from google-auth<3.0.0dev,>=2.14.1->google-cloud-aiplatform) (5.3.3)
Step #5: Step #4: Requirement already satisfied: pyasn1-modules>=0.2.1 in ./workspace/env/lib/python3.9/site-packages (from google-auth<3.0.0dev,>=2.14.1->google-cloud-aiplatform) (0.4.0)
Step #5: Step #4: Requirement already satisfied: rsa<5,>=3.1.4 in ./workspace/env/lib/python3.9/site-packages (from google-auth<3.0.0dev,>=2.14.1->google-cloud-aiplatform) (4.9)
Step #5: Step #4: Requirement already satisfied: python-dateutil<3.0dev,>=2.7.2 in ./workspace/env/lib/python3.9/site-packages (from google-cloud-bigquery!=3.20.0,<4.0.0dev,>=1.15.0->google-cloud-aiplatform) (2.9.0.post0)
Step #5: Step #4: Requirement already satisfied: grpc-google-iam-v1<1.0.0dev,>=0.12.4 in ./workspace/env/lib/python3.9/site-packages (from google-cloud-resource-manager<3.0.0dev,>=1.3.3->google-cloud-aiplatform) (0.13.1)
Step #5: Step #4: Requirement already satisfied: MarkupSafe>=2.0 in ./workspace/env/lib/python3.9/site-packages (from Jinja2<4,>=3.1.2->google-cloud-pipeline-components) (2.1.5)
Step #5: Step #4: Requirement already satisfied: six>=1.10 in ./workspace/env/lib/python3.9/site-packages (from kfp-server-api<2.1.0,>=2.0.0->kfp) (1.16.0)
Step #5: Step #4: Requirement already satisfied: certifi in ./workspace/env/lib/python3.9/site-packages (from kfp-server-api<2.1.0,>=2.0.0->kfp) (2024.6.2)
Step #5: Step #4: Requirement already satisfied: setuptools>=21.0.0 in ./workspace/env/lib/python3.9/site-packages (from kubernetes<27,>=8.0.0->kfp) (58.1.0)
Step #5: Step #4: Requirement already satisfied: websocket-client!=0.40.0,!=0.41.*,!=0.42.*,>=0.32.0 in ./workspace/env/lib/python3.9/site-packages (from kubernetes<27,>=8.0.0->kfp) (1.8.0)
Step #5: Step #4: Collecting requests-oauthlib (from kubernetes<27,>=8.0.0->kfp)
Step #5: Step #4: Downloading requests_oauthlib-2.0.0-py2.py3-none-any.whl.metadata (11 kB)
Step #5: Step #4: Requirement already satisfied: annotated-types>=0.4.0 in ./workspace/env/lib/python3.9/site-packages (from pydantic<3->google-cloud-aiplatform) (0.7.0)
Step #5: Step #4: Requirement already satisfied: pydantic-core==2.20.0 in ./workspace/env/lib/python3.9/site-packages (from pydantic<3->google-cloud-aiplatform) (2.20.0)
Step #5: Step #4: Requirement already satisfied: typing-extensions>=4.6.1 in ./workspace/env/lib/python3.9/site-packages (from pydantic<3->google-cloud-aiplatform) (4.12.2)
Step #5: Step #4: Requirement already satisfied: charset-normalizer<4,>=2 in ./workspace/env/lib/python3.9/site-packages (from requests<3.0.0dev,>=2.18.0->google-cloud-storage) (3.3.2)
Step #5: Step #4: Requirement already satisfied: idna<4,>=2.5 in ./workspace/env/lib/python3.9/site-packages (from requests<3.0.0dev,>=2.18.0->google-cloud-storage) (3.7)
Step #5: Step #4: Requirement already satisfied: numpy<3,>=1.14 in ./workspace/env/lib/python3.9/site-packages (from shapely<3.0.0dev->google-cloud-aiplatform) (2.0.0)
Step #5: Step #4: Requirement already satisfied: pyasn1<0.7.0,>=0.4.6 in ./workspace/env/lib/python3.9/site-packages (from pyasn1-modules>=0.2.1->google-auth<3.0.0dev,>=2.14.1->google-cloud-aiplatform) (0.6.0)
Step #5: Step #4: Collecting oauthlib>=3.0.0 (from requests-oauthlib->kubernetes<27,>=8.0.0->kfp)
Step #5: Step #4: Downloading oauthlib-3.2.2-py3-none-any.whl.metadata (7.5 kB)
Step #5: Step #4: Downloading kfp_pipeline_spec-0.3.0-py3-none-any.whl (12 kB)
Step #5: Step #4: Downloading google_cloud_pipeline_components-2.15.0-py3-none-any.whl (1.5 MB)
Step #5: Step #4: �[?25l �[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━�[0m �[32m0.0/1.5 MB�[0m �[31m?�[0m eta �[36m-:--:--�[0m
�[2K �[91m━━━━━━━━━━━━━━━━━━━━━━━�[0m�[91m╸�[0m�[90m━━━━━━━━━━━━━━━━�[0m �[32m0.9/1.5 MB�[0m �[31m52.0 MB/s�[0m eta �[36m0:00:01�[0m
�[2K �[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━�[0m �[32m1.5/1.5 MB�[0m �[31m25.8 MB/s�[0m eta �[36m0:00:00�[0m
Step #5: Step #4: �[?25hDownloading kubernetes-26.1.0-py2.py3-none-any.whl (1.4 MB)
Step #5: Step #4: �[?25l �[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━�[0m �[32m0.0/1.4 MB�[0m �[31m?�[0m eta �[36m-:--:--�[0m
�[2K �[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━�[0m �[32m1.4/1.4 MB�[0m �[31m78.5 MB/s�[0m eta �[36m0:00:00�[0m
Step #5: Step #4: �[?25hDownloading requests_toolbelt-0.10.1-py2.py3-none-any.whl (54 kB)
Step #5: Step #4: �[?25l �[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━�[0m �[32m0.0/54.5 kB�[0m �[31m?�[0m eta �[36m-:--:--�[0m
�[2K �[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━�[0m �[32m54.5/54.5 kB�[0m �[31m7.3 MB/s�[0m eta �[36m0:00:00�[0m
Step #5: Step #4: �[?25hDownloading urllib3-1.26.19-py2.py3-none-any.whl (143 kB)
Step #5: Step #4: �[?25l �[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━�[0m �[32m0.0/143.9 kB�[0m �[31m?�[0m eta �[36m-:--:--�[0m
�[2K �[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━�[0m �[32m143.9/143.9 kB�[0m �[31m20.1 MB/s�[0m eta �[36m0:00:00�[0m
Step #5: Step #4: �[?25hDownloading requests_oauthlib-2.0.0-py2.py3-none-any.whl (24 kB)
Step #5: Step #4: Downloading oauthlib-3.2.2-py3-none-any.whl (151 kB)
Step #5: Step #4: �[?25l �[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━�[0m �[32m0.0/151.7 kB�[0m �[31m?�[0m eta �[36m-:--:--�[0m
�[2K �[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━�[0m �[32m151.7/151.7 kB�[0m �[31m21.2 MB/s�[0m eta �[36m0:00:00�[0m
Step #5: Step #4: �[?25hBuilding wheels for collected packages: kfp, kfp-server-api
Step #5: Step #4: Building wheel for kfp (pyproject.toml) ... �[?25l-� �\� �|� �done
Step #5: Step #4: �[?25h Created wheel for kfp: filename=kfp-2.7.0-py3-none-any.whl size=610417 sha256=68f883a4b7112c9fdff36716d37555a69dbbf2ae9d82676ff976cde40495bae1
Step #5: Step #4: Stored in directory: /builder/home/.cache/pip/wheels/6b/21/04/3a90a57cfed7fb3b132d75ef1617fdeaa85916b04551d5369d
Step #5: Step #4: Building wheel for kfp-server-api (pyproject.toml) ... �[?25l-� �done
Step #5: Step #4: �[?25h Created wheel for kfp-server-api: filename=kfp_server_api-2.0.5-py3-none-any.whl size=114731 sha256=0fa33442dd9e477c9b0275053612049e07cd362a81a28585fe7bd8bb556a8001
Step #5: Step #4: Stored in directory: /builder/home/.cache/pip/wheels/a6/c9/de/9f6964cdffc4d454c0a96d34f9b5ea5e51fc3224ec379222ff
Step #5: Step #4: Successfully built kfp kfp-server-api
Step #5: Step #4: Installing collected packages: urllib3, oauthlib, kfp-pipeline-spec, kfp-server-api, requests-toolbelt, requests-oauthlib, kubernetes, kfp, google-cloud-pipeline-components
Step #5: Step #4: Attempting uninstall: urllib3
Step #5: Step #4: Found existing installation: urllib3 2.2.2
Step #5: Step #4: Uninstalling urllib3-2.2.2:
Step #5: Step #4: Successfully uninstalled urllib3-2.2.2
Step #5: Step #4: Successfully installed google-cloud-pipeline-components-2.15.0 kfp-2.7.0 kfp-pipeline-spec-0.3.0 kfp-server-api-2.0.5 kubernetes-26.1.0 oauthlib-3.2.2 requests-oauthlib-2.0.0 requests-toolbelt-0.10.1 urllib3-1.26.19
Step #5: Step #4: KFP SDK version: 2.7.0
Step #5: Step #4:
Step #5: Step #4:
Step #5: Step #4: Updates are available for some Google Cloud CLI components. To install them,
Step #5: Step #4: please run:
Step #5: Step #4: $ gcloud components update
Step #5: Step #4:
Step #5: Step #4: Creating gs://your-bucket-name-python-docs-samples-tests-d8ddexma/...
Step #5: Step #4:
Executing: 0%| | 0/71 [00:00<?, ?cell/s]
Executing: 1%|▏ | 1/71 [00:00<01:09, 1.01cell/s]
Executing: 8%|▊ | 6/71 [00:01<00:09, 7.09cell/s]
Executing: 10%|▉ | 7/71 [00:20<00:09, 7.09cell/s]
Executing: 11%|█▏ | 8/71 [00:21<03:40, 3.51s/cell]
Executing: 15%|█▌ | 11/71 [00:21<02:03, 2.06s/cell]
Executing: 20%|█▉ | 14/71 [00:23<01:25, 1.49s/cell]
Executing: 24%|██▍ | 17/71 [00:23<00:53, 1.01cell/s]
Executing: 28%|██▊ | 20/71 [00:23<00:34, 1.49cell/s]
Executing: 32%|███▏ | 23/71 [00:27<00:40, 1.17cell/s]
Executing: 37%|███▋ | 26/71 [00:32<00:51, 1.15s/cell]
Executing: 39%|███▉ | 28/71 [00:34<00:49, 1.15s/cell]
Executing: 44%|████▎ | 31/71 [00:35<00:31, 1.27cell/s]
Executing: 48%|████▊ | 34/71 [00:35<00:20, 1.83cell/s]/workspace/workspace/env/lib/python3.9/site-packages/kfp/dsl/component_decorator.py:119: FutureWarning: Python 3.7 has reached end-of-life. The default base_image used by the @dsl.component decorator will switch from 'python:3.7' to 'python:3.8' on April 23, 2024. To ensure your existing components work with versions of the KFP SDK released after that date, you should provide an explicit base_image argument and ensure your component works as intended on Python 3.8.
Step #5: Step #4: return component_factory.create_component_from_func(
Step #5: Step #4:
Step #5: Step #4: /workspace/workspace/env/lib/python3.9/site-packages/kfp/dsl/component_decorator.py:119: FutureWarning: Python 3.7 has reached end-of-life. The default base_image used by the @dsl.component decorator will switch from 'python:3.7' to 'python:3.8' on April 23, 2024. To ensure your existing components work with versions of the KFP SDK released after that date, you should provide an explicit base_image argument and ensure your component works as intended on Python 3.8.
Step #5: Step #4: return component_factory.create_component_from_func(
Step #5: Step #4: Creating PipelineJob
Step #5: Step #4: PipelineJob created. Resource name: projects/1012616486416/locations/us-central1/pipelineJobs/intro-pipeline-4q2q2fik-20240701200604
Step #5: Step #4: To use this PipelineJob in another session:
Step #5: Step #4: pipeline_job = aiplatform.PipelineJob.get('projects/1012616486416/locations/us-central1/pipelineJobs/intro-pipeline-4q2q2fik-20240701200604')
Step #5: Step #4: View Pipeline Job:
Step #5: Step #4: https://console.cloud.google.com/vertex-ai/locations/us-central1/pipelines/runs/intro-pipeline-4q2q2fik-20240701200604?project=1012616486416
Step #5: Step #4: PipelineJob projects/1012616486416/locations/us-central1/pipelineJobs/intro-pipeline-4q2q2fik-20240701200604 current state:
Step #5: Step #4: PipelineState.PIPELINE_STATE_RUNNING
Step #5: Step #4: PipelineJob projects/1012616486416/locations/us-central1/pipelineJobs/intro-pipeline-4q2q2fik-20240701200604 current state:
Step #5: Step #4: PipelineState.PIPELINE_STATE_RUNNING
Step #5: Step #4: PipelineJob projects/1012616486416/locations/us-central1/pipelineJobs/intro-pipeline-4q2q2fik-20240701200604 current state:
Step #5: Step #4: PipelineState.PIPELINE_STATE_RUNNING
Step #5: Step #4: PipelineJob projects/1012616486416/locations/us-central1/pipelineJobs/intro-pipeline-4q2q2fik-20240701200604 current state:
Step #5: Step #4: PipelineState.PIPELINE_STATE_RUNNING
Step #5: Step #4: PipelineJob projects/1012616486416/locations/us-central1/pipelineJobs/intro-pipeline-4q2q2fik-20240701200604 current state:
Step #5: Step #4: PipelineState.PIPELINE_STATE_RUNNING
Step #5: Step #4: PipelineJob run completed. Resource name: projects/1012616486416/locations/us-central1/pipelineJobs/intro-pipeline-4q2q2fik-20240701200604
Step #5: Step #4: Deleting PipelineJob : projects/1012616486416/locations/us-central1/pipelineJobs/intro-pipeline-4q2q2fik-20240701200604
Step #5: Step #4: PipelineJob deleted. . Resource name: projects/1012616486416/locations/us-central1/pipelineJobs/intro-pipeline-4q2q2fik-20240701200604
Step #5: Step #4: Deleting Pipeli
...
[Logs truncated due to log size limitations. For full logs, see https://console.cloud.google.com/cloud-build/builds;region=us-central1/eec7893f-2557-478c-9676-3a73ee22f9fa?project=1012616486416.]
...
-no-warn-script-location 'kfp==2.7.0' '--no-deps' 'typing-extensions\u003e=3.7.4,\u003c5; python_version\u003c\"3.9\"' && \"$0\" \"$@\"\n",
Step #5: Step #4: "sh",
Step #5: Step #4: "-ec",
Step #5: Step #4: "program_path=$(mktemp -d)\n\nprintf \"%s\" \"$0\" \u003e \"$program_path/ephemeral_component.py\"\n_KFP_RUNTIME=true python3 -m kfp.dsl.executor_main --component_module_path \"$program_path/ephemeral_component.py\" \"$@\"\n",
Step #5: Step #4: "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import *\n\ndef consumer(text1: str, text2: str, text3: str) -\u003e str:\n print(f\"text1: {text1}; text2: {text2}; text3: {text3}\")\n return f\"text1: {text1}; text2: {text2}; text3: {text3}\"\n\n"
Step #5: Step #4: ],
Step #5: Step #4: "args": [
Step #5: Step #4: "--executor_input",
Step #5: Step #4: "{{$}}",
Step #5: Step #4: "--function_to_execute",
Step #5: Step #4: "consumer"
Step #5: Step #4: ]
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "exec-hello-world": {
Step #5: Step #4: "container": {
Step #5: Step #4: "image": "python:3.9",
Step #5: Step #4: "command": [
Step #5: Step #4: "sh",
Step #5: Step #4: "-c",
Step #5: Step #4: "\nif ! [ -x \"$(command -v pip)\" ]; then\n python3 -m ensurepip || python3 -m ensurepip --user || apt-get install python3-pip\nfi\n\nPIP_DISABLE_PIP_VERSION_CHECK=1 python3 -m pip install --quiet --no-warn-script-location 'kfp==2.7.0' '--no-deps' 'typing-extensions\u003e=3.7.4,\u003c5; python_version\u003c\"3.9\"' && \"$0\" \"$@\"\n",
Step #5: Step #4: "sh",
Step #5: Step #4: "-ec",
Step #5: Step #4: "program_path=$(mktemp -d)\n\nprintf \"%s\" \"$0\" \u003e \"$program_path/ephemeral_component.py\"\n_KFP_RUNTIME=true python3 -m kfp.dsl.executor_main --component_module_path \"$program_path/ephemeral_component.py\" \"$@\"\n",
Step #5: Step #4: "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import *\n\ndef hello_world(text: str) -\u003e str:\n print(text)\n return text\n\n"
Step #5: Step #4: ],
Step #5: Step #4: "args": [
Step #5: Step #4: "--executor_input",
Step #5: Step #4: "{{$}}",
Step #5: Step #4: "--function_to_execute",
Step #5: Step #4: "hello_world"
Step #5: Step #4: ]
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "exec-two-outputs": {
Step #5: Step #4: "container": {
Step #5: Step #4: "image": "python:3.7",
Step #5: Step #4: "command": [
Step #5: Step #4: "sh",
Step #5: Step #4: "-c",
Step #5: Step #4: "\nif ! [ -x \"$(command -v pip)\" ]; then\n python3 -m ensurepip || python3 -m ensurepip --user || apt-get install python3-pip\nfi\n\nPIP_DISABLE_PIP_VERSION_CHECK=1 python3 -m pip install --quiet --no-warn-script-location 'kfp==2.7.0' '--no-deps' 'typing-extensions\u003e=3.7.4,\u003c5; python_version\u003c\"3.9\"' && python3 -m pip install --quiet --no-warn-script-location 'google-cloud-storage' && \"$0\" \"$@\"\n",
Step #5: Step #4: "sh",
Step #5: Step #4: "-ec",
Step #5: Step #4: "program_path=$(mktemp -d)\n\nprintf \"%s\" \"$0\" \u003e \"$program_path/ephemeral_component.py\"\n_KFP_RUNTIME=true python3 -m kfp.dsl.executor_main --component_module_path \"$program_path/ephemeral_component.py\" \"$@\"\n",
Step #5: Step #4: "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import *\n\ndef two_outputs(\n text: str,\n) -\u003e NamedTuple(\n \"Outputs\",\n [\n (\"output_one\", str), # Return parameters\n (\"output_two\", str),\n ],\n):\n # the import is not actually used for this simple example, but the import\n # is successful, as it was included in the `packages_to_install` list.\n from google.cloud import storage # noqa: F401\n\n o1 = f\"output one from text: {text}\"\n o2 = f\"output two from text: {text}\"\n print(\"output one: {}; output_two: {}\".format(o1, o2))\n return (o1, o2)\n\n"
Step #5: Step #4: ],
Step #5: Step #4: "args": [
Step #5: Step #4: "--executor_input",
Step #5: Step #4: "{{$}}",
Step #5: Step #4: "--function_to_execute",
Step #5: Step #4: "two_outputs"
Step #5: Step #4: ]
Step #5: Step #4: }
Step #5: Step #4: }
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "components": {
Step #5: Step #4: "comp-consumer": {
Step #5: Step #4: "inputDefinitions": {
Step #5: Step #4: "parameters": {
Step #5: Step #4: "text1": {
Step #5: Step #4: "parameterType": "STRING"
Step #5: Step #4: },
Step #5: Step #4: "text2": {
Step #5: Step #4: "parameterType": "STRING"
Step #5: Step #4: },
Step #5: Step #4: "text3": {
Step #5: Step #4: "parameterType": "STRING"
Step #5: Step #4: }
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "outputDefinitions": {
Step #5: Step #4: "parameters": {
Step #5: Step #4: "Output": {
Step #5: Step #4: "parameterType": "STRING"
Step #5: Step #4: }
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "executorLabel": "exec-consumer"
Step #5: Step #4: },
Step #5: Step #4: "comp-hello-world": {
Step #5: Step #4: "inputDefinitions": {
Step #5: Step #4: "parameters": {
Step #5: Step #4: "text": {
Step #5: Step #4: "parameterType": "STRING"
Step #5: Step #4: }
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "outputDefinitions": {
Step #5: Step #4: "parameters": {
Step #5: Step #4: "Output": {
Step #5: Step #4: "parameterType": "STRING"
Step #5: Step #4: }
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "executorLabel": "exec-hello-world"
Step #5: Step #4: },
Step #5: Step #4: "comp-two-outputs": {
Step #5: Step #4: "inputDefinitions": {
Step #5: Step #4: "parameters": {
Step #5: Step #4: "text": {
Step #5: Step #4: "parameterType": "STRING"
Step #5: Step #4: }
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "outputDefinitions": {
Step #5: Step #4: "parameters": {
Step #5: Step #4: "output_one": {
Step #5: Step #4: "parameterType": "STRING"
Step #5: Step #4: },
Step #5: Step #4: "output_two": {
Step #5: Step #4: "parameterType": "STRING"
Step #5: Step #4: }
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "executorLabel": "exec-two-outputs"
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "schemaVersion": "2.1.0",
Step #5: Step #4: "root": {
Step #5: Step #4: "inputDefinitions": {
Step #5: Step #4: "parameters": {
Step #5: Step #4: "text": {
Step #5: Step #4: "parameterType": "STRING",
Step #5: Step #4: "defaultValue": "hi there",
Step #5: Step #4: "isOptional": true
Step #5: Step #4: }
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "dag": {
Step #5: Step #4: "tasks": {
Step #5: Step #4: "consumer": {
Step #5: Step #4: "taskInfo": {
Step #5: Step #4: "name": "consumer"
Step #5: Step #4: },
Step #5: Step #4: "inputs": {
Step #5: Step #4: "parameters": {
Step #5: Step #4: "text1": {
Step #5: Step #4: "taskOutputParameter": {
Step #5: Step #4: "producerTask": "hello-world",
Step #5: Step #4: "outputParameterKey": "Output"
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "text2": {
Step #5: Step #4: "taskOutputParameter": {
Step #5: Step #4: "producerTask": "two-outputs",
Step #5: Step #4: "outputParameterKey": "output_one"
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "text3": {
Step #5: Step #4: "taskOutputParameter": {
Step #5: Step #4: "producerTask": "two-outputs",
Step #5: Step #4: "outputParameterKey": "output_two"
Step #5: Step #4: }
Step #5: Step #4: }
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "dependentTasks": [
Step #5: Step #4: "hello-world",
Step #5: Step #4: "two-outputs"
Step #5: Step #4: ],
Step #5: Step #4: "cachingOptions": {
Step #5: Step #4: "enableCache": true
Step #5: Step #4: },
Step #5: Step #4: "componentRef": {
Step #5: Step #4: "name": "comp-consumer"
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "hello-world": {
Step #5: Step #4: "taskInfo": {
Step #5: Step #4: "name": "hello-world"
Step #5: Step #4: },
Step #5: Step #4: "inputs": {
Step #5: Step #4: "parameters": {
Step #5: Step #4: "text": {
Step #5: Step #4: "componentInputParameter": "text"
Step #5: Step #4: }
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "cachingOptions": {
Step #5: Step #4: "enableCache": true
Step #5: Step #4: },
Step #5: Step #4: "componentRef": {
Step #5: Step #4: "name": "comp-hello-world"
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "two-outputs": {
Step #5: Step #4: "taskInfo": {
Step #5: Step #4: "name": "two-outputs"
Step #5: Step #4: },
Step #5: Step #4: "inputs": {
Step #5: Step #4: "parameters": {
Step #5: Step #4: "text": {
Step #5: Step #4: "componentInputParameter": "text"
Step #5: Step #4: }
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "cachingOptions": {
Step #5: Step #4: "enableCache": true
Step #5: Step #4: },
Step #5: Step #4: "componentRef": {
Step #5: Step #4: "name": "comp-two-outputs"
Step #5: Step #4: }
Step #5: Step #4: }
Step #5: Step #4: }
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "defaultPipelineRoot": "gs://your-bucket-name-python-docs-samples-tests-d8ddexma/pipeline_root/intro",
Step #5: Step #4: "sdkVersion": "kfp-2.7.0",
Step #5: Step #4: "pipelineInfo": {
Step #5: Step #4: "name": "intro-pipeline-4q2q2fik",
Step #5: Step #4: "description": "A simple intro pipeline"
Step #5: Step #4: },
Step #5: Step #4: "deploymentSpec": {
Step #5: Step #4: "executors": {
Step #5: Step #4: "exec-consumer": {
Step #5: Step #4: "container": {
Step #5: Step #4: "image": "python:3.7",
Step #5: Step #4: "command": [
Step #5: Step #4: "sh",
Step #5: Step #4: "-c",
Step #5: Step #4: "\nif ! [ -x \"$(command -v pip)\" ]; then\n python3 -m ensurepip || python3 -m ensurepip --user || apt-get install python3-pip\nfi\n\nPIP_DISABLE_PIP_VERSION_CHECK=1 python3 -m pip install --quiet --no-warn-script-location 'kfp==2.7.0' '--no-deps' 'typing-extensions\u003e=3.7.4,\u003c5; python_version\u003c\"3.9\"' && \"$0\" \"$@\"\n",
Step #5: Step #4: "sh",
Step #5: Step #4: "-ec",
Step #5: Step #4: "program_path=$(mktemp -d)\n\nprintf \"%s\" \"$0\" \u003e \"$program_path/ephemeral_component.py\"\n_KFP_RUNTIME=true python3 -m kfp.dsl.executor_main --component_module_path \"$program_path/ephemeral_component.py\" \"$@\"\n",
Step #5: Step #4: "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import *\n\ndef consumer(text1: str, text2: str, text3: str) -\u003e str:\n print(f\"text1: {text1}; text2: {text2}; text3: {text3}\")\n return f\"text1: {text1}; text2: {text2}; text3: {text3}\"\n\n"
Step #5: Step #4: ],
Step #5: Step #4: "args": [
Step #5: Step #4: "--executor_input",
Step #5: Step #4: "{{$}}",
Step #5: Step #4: "--function_to_execute",
Step #5: Step #4: "consumer"
Step #5: Step #4: ]
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "exec-hello-world": {
Step #5: Step #4: "container": {
Step #5: Step #4: "image": "python:3.9",
Step #5: Step #4: "command": [
Step #5: Step #4: "sh",
Step #5: Step #4: "-c",
Step #5: Step #4: "\nif ! [ -x \"$(command -v pip)\" ]; then\n python3 -m ensurepip || python3 -m ensurepip --user || apt-get install python3-pip\nfi\n\nPIP_DISABLE_PIP_VERSION_CHECK=1 python3 -m pip install --quiet --no-warn-script-location 'kfp==2.7.0' '--no-deps' 'typing-extensions\u003e=3.7.4,\u003c5; python_version\u003c\"3.9\"' && \"$0\" \"$@\"\n",
Step #5: Step #4: "sh",
Step #5: Step #4: "-ec",
Step #5: Step #4: "program_path=$(mktemp -d)\n\nprintf \"%s\" \"$0\" \u003e \"$program_path/ephemeral_component.py\"\n_KFP_RUNTIME=true python3 -m kfp.dsl.executor_main --component_module_path \"$program_path/ephemeral_component.py\" \"$@\"\n",
Step #5: Step #4: "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import *\n\ndef hello_world(text: str) -\u003e str:\n print(text)\n return text\n\n"
Step #5: Step #4: ],
Step #5: Step #4: "args": [
Step #5: Step #4: "--executor_input",
Step #5: Step #4: "{{$}}",
Step #5: Step #4: "--function_to_execute",
Step #5: Step #4: "hello_world"
Step #5: Step #4: ]
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "exec-two-outputs": {
Step #5: Step #4: "container": {
Step #5: Step #4: "image": "python:3.7",
Step #5: Step #4: "command": [
Step #5: Step #4: "sh",
Step #5: Step #4: "-c",
Step #5: Step #4: "\nif ! [ -x \"$(command -v pip)\" ]; then\n python3 -m ensurepip || python3 -m ensurepip --user || apt-get install python3-pip\nfi\n\nPIP_DISABLE_PIP_VERSION_CHECK=1 python3 -m pip install --quiet --no-warn-script-location 'kfp==2.7.0' '--no-deps' 'typing-extensions\u003e=3.7.4,\u003c5; python_version\u003c\"3.9\"' && python3 -m pip install --quiet --no-warn-script-location 'google-cloud-storage' && \"$0\" \"$@\"\n",
Step #5: Step #4: "sh",
Step #5: Step #4: "-ec",
Step #5: Step #4: "program_path=$(mktemp -d)\n\nprintf \"%s\" \"$0\" \u003e \"$program_path/ephemeral_component.py\"\n_KFP_RUNTIME=true python3 -m kfp.dsl.executor_main --component_module_path \"$program_path/ephemeral_component.py\" \"$@\"\n",
Step #5: Step #4: "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import *\n\ndef two_outputs(\n text: str,\n) -\u003e NamedTuple(\n \"Outputs\",\n [\n (\"output_one\", str), # Return parameters\n (\"output_two\", str),\n ],\n):\n # the import is not actually used for this simple example, but the import\n # is successful, as it was included in the `packages_to_install` list.\n from google.cloud import storage # noqa: F401\n\n o1 = f\"output one from text: {text}\"\n o2 = f\"output two from text: {text}\"\n print(\"output one: {}; output_two: {}\".format(o1, o2))\n return (o1, o2)\n\n"
Step #5: Step #4: ],
Step #5: Step #4: "args": [
Step #5: Step #4: "--executor_input",
Step #5: Step #4: "{{$}}",
Step #5: Step #4: "--function_to_execute",
Step #5: Step #4: "two_outputs"
Step #5: Step #4: ]
Step #5: Step #4: }
Step #5: Step #4: }
Step #5: Step #4: }
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "state": "PIPELINE_STATE_RUNNING",
Step #5: Step #4: "jobDetail": {
Step #5: Step #4: "pipelineContext": {
Step #5: Step #4: "name": "projects/1012616486416/locations/us-central1/metadataStores/default/contexts/intro-pipeline-4q2q2fik",
Step #5: Step #4: "displayName": "intro-pipeline-4q2q2fik",
Step #5: Step #4: "etag": "1719864771357",
Step #5: Step #4: "createTime": "2024-07-01T20:06:06.117Z",
Step #5: Step #4: "updateTime": "2024-07-01T20:12:51.357Z",
Step #5: Step #4: "schemaTitle": "system.Pipeline",
Step #5: Step #4: "schemaVersion": "0.0.1",
Step #5: Step #4: "metadata": {}
Step #5: Step #4: },
Step #5: Step #4: "pipelineRunContext": {
Step #5: Step #4: "name": "projects/1012616486416/locations/us-central1/metadataStores/default/contexts/intro-pipeline-job-l3fo7be9",
Step #5: Step #4: "displayName": "intro-pipeline-job-l3fo7be9",
Step #5: Step #4: "etag": "1719864771471",
Step #5: Step #4: "createTime": "2024-07-01T20:12:51.471Z",
Step #5: Step #4: "updateTime": "2024-07-01T20:12:51.471Z",
Step #5: Step #4: "parentContexts": [
Step #5: Step #4: "projects/1012616486416/locations/us-central1/metadataStores/default/contexts/intro-pipeline-4q2q2fik"
Step #5: Step #4: ],
Step #5: Step #4: "schemaTitle": "system.PipelineRun",
Step #5: Step #4: "schemaVersion": "0.0.1",
Step #5: Step #4: "metadata": {}
Step #5: Step #4: },
Step #5: Step #4: "taskDetails": [
Step #5: Step #4: {
Step #5: Step #4: "taskId": "-6740361626681081856",
Step #5: Step #4: "taskName": "intro-pipeline-job-l3fo7be9",
Step #5: Step #4: "createTime": "2024-07-01T20:12:51.290031Z",
Step #5: Step #4: "startTime": "2024-07-01T20:12:51.977871Z",
Step #5: Step #4: "executorDetail": {},
Step #5: Step #4: "state": "RUNNING",
Step #5: Step #4: "execution": {
Step #5: Step #4: "name": "projects/1012616486416/locations/us-central1/metadataStores/default/executions/17468289288802995991",
Step #5: Step #4: "displayName": "intro-pipeline-job-l3fo7be9",
Step #5: Step #4: "state": "RUNNING",
Step #5: Step #4: "etag": "1719864771856",
Step #5: Step #4: "createTime": "2024-07-01T20:12:51.856Z",
Step #5: Step #4: "updateTime": "2024-07-01T20:12:51.856Z",
Step #5: Step #4: "schemaTitle": "system.Run",
Step #5: Step #4: "schemaVersion": "0.0.1",
Step #5: Step #4: "metadata": {
Step #5: Step #4: "vmlmd_lineage_integration": {
Step #5: Step #4: "pipeline_run_component": {
Step #5: Step #4: "parent_task_names": [],
Step #5: Step #4: "task_name": "intro-pipeline-job-l3fo7be9",
Step #5: Step #4: "pipeline_run_id": "intro-pipeline-job-l3fo7be9",
Step #5: Step #4: "project_id": "python-docs-samples-tests",
Step #5: Step #4: "location_id": "us-central1"
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "input:text": "hi there"
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "pipelineTaskStatus": [
Step #5: Step #4: {
Step #5: Step #4: "updateTime": "2024-07-01T20:12:51.795385215Z",
Step #5: Step #4: "state": "RUNNING"
Step #5: Step #4: }
Step #5: Step #4: ]
Step #5: Step #4: },
Step #5: Step #4: {
Step #5: Step #4: "taskId": "-3930115459201892352",
Step #5: Step #4: "taskName": "hello-world",
Step #5: Step #4: "createTime": "2024-07-01T20:12:51.977871Z",
Step #5: Step #4: "startTime": "2024-07-01T20:12:52.546777Z",
Step #5: Step #4: "endTime": "2024-07-01T20:12:52.546777Z",
Step #5: Step #4: "executorDetail": {
Step #5: Step #4: "containerDetail": {
Step #5: Step #4: "mainJob": "projects/1012616486416/locations/us-central1/customJobs/2344517403007778816"
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "state": "SKIPPED",
Step #5: Step #4: "execution": {
Step #5: Step #4: "name": "projects/1012616486416/locations/us-central1/metadataStores/default/executions/6145563962678182717",
Step #5: Step #4: "displayName": "hello-world",
Step #5: Step #4: "state": "CACHED",
Step #5: Step #4: "etag": "1719864772505",
Step #5: Step #4: "createTime": "2024-07-01T20:12:52.185Z",
Step #5: Step #4: "updateTime": "2024-07-01T20:12:52.505Z",
Step #5: Step #4: "schemaTitle": "system.ContainerExecution",
Step #5: Step #4: "schemaVersion": "0.0.1",
Step #5: Step #4: "metadata": {
Step #5: Step #4: "output:Output": "hi there",
Step #5: Step #4: "vmlmd_lineage_integration": {
Step #5: Step #4: "pipeline_run_component": {
Step #5: Step #4: "location_id": "us-central1",
Step #5: Step #4: "project_id": "python-docs-samples-tests",
Step #5: Step #4: "parent_task_names": [
Step #5: Step #4: "intro-pipeline-4q2q2fik-20240701201037"
Step #5: Step #4: ],
Step #5: Step #4: "pipeline_run_id": "intro-pipeline-4q2q2fik-20240701201037",
Step #5: Step #4: "task_name": "hello-world"
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "input:text": "hi there"
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "parentTaskId": "-6740361626681081856",
Step #5: Step #4: "pipelineTaskStatus": [
Step #5: Step #4: {
Step #5: Step #4: "updateTime": "2024-07-01T20:12:52.446723698Z",
Step #5: Step #4: "state": "SKIPPED"
Step #5: Step #4: }
Step #5: Step #4: ]
Step #5: Step #4: },
Step #5: Step #4: {
Step #5: Step #4: "taskId": "2987413568439189504",
Step #5: Step #4: "taskName": "two-outputs",
Step #5: Step #4: "createTime": "2024-07-01T20:12:51.977871Z",
Step #5: Step #4: "startTime": "2024-07-01T20:12:52.736067Z",
Step #5: Step #4: "endTime": "2024-07-01T20:12:52.736067Z",
Step #5: Step #4: "executorDetail": {
Step #5: Step #4: "containerDetail": {
Step #5: Step #4: "mainJob": "projects/1012616486416/locations/us-central1/customJobs/2038272628346585088"
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "state": "SKIPPED",
Step #5: Step #4: "execution": {
Step #5: Step #4: "name": "projects/1012616486416/locations/us-central1/metadataStores/default/executions/14517440568590960592",
Step #5: Step #4: "displayName": "two-outputs",
Step #5: Step #4: "state": "CACHED",
Step #5: Step #4: "etag": "1719864772487",
Step #5: Step #4: "createTime": "2024-07-01T20:12:52.183Z",
Step #5: Step #4: "updateTime": "2024-07-01T20:12:52.487Z",
Step #5: Step #4: "schemaTitle": "system.ContainerExecution",
Step #5: Step #4: "schemaVersion": "0.0.1",
Step #5: Step #4: "metadata": {
Step #5: Step #4: "output:output_one": "output one from text: hi there",
Step #5: Step #4: "vmlmd_lineage_integration": {
Step #5: Step #4: "pipeline_run_component": {
Step #5: Step #4: "project_id": "python-docs-samples-tests",
Step #5: Step #4: "parent_task_names": [
Step #5: Step #4: "intro-pipeline-4q2q2fik-20240701201037"
Step #5: Step #4: ],
Step #5: Step #4: "pipeline_run_id": "intro-pipeline-4q2q2fik-20240701201037",
Step #5: Step #4: "location_id": "us-central1",
Step #5: Step #4: "task_name": "two-outputs"
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "input:text": "hi there",
Step #5: Step #4: "output:output_two": "output two from text: hi there"
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "parentTaskId": "-6740361626681081856",
Step #5: Step #4: "pipelineTaskStatus": [
Step #5: Step #4: {
Step #5: Step #4: "updateTime": "2024-07-01T20:12:52.439926349Z",
Step #5: Step #4: "state": "SKIPPED"
Step #5: Step #4: }
Step #5: Step #4: ]
Step #5: Step #4: },
Step #5: Step #4: {
Step #5: Step #4: "taskId": "5293256577652883456",
Step #5: Step #4: "taskName": "consumer",
Step #5: Step #4: "createTime": "2024-07-01T20:12:51.977871Z",
Step #5: Step #4: "executorDetail": {},
Step #5: Step #4: "state": "PENDING",
Step #5: Step #4: "parentTaskId": "-6740361626681081856",
Step #5: Step #4: "pipelineTaskStatus": [
Step #5: Step #4: {
Step #5: Step #4: "updateTime": "2024-07-01T20:12:52.799213975Z",
Step #5: Step #4: "state": "PENDING"
Step #5: Step #4: }
Step #5: Step #4: ]
Step #5: Step #4: }
Step #5: Step #4: ]
Step #5: Step #4: },
Step #5: Step #4: "labels": {
Step #5: Step #4: "vertex-ai-pipelines-run-billing-id": "4299714162575015936"
Step #5: Step #4: },
Step #5: Step #4: "runtimeConfig": {
Step #5: Step #4: "gcsOutputDirectory": "gs://your-bucket-name-python-docs-samples-tests-d8ddexma/pipeline_root/intro"
Step #5: Step #4: },
Step #5: Step #4: "serviceAccount": "1012616486416-compute@developer.gserviceaccount.com"
Step #5: Step #4: }
Step #5: Step #4: {
Step #5: Step #4: "error": {
Step #5: Step #4: "code": 400,
Step #5: Step #4: "message": "The PipelineJob \"projects/python-docs-samples-tests/locations/us-central1/pipelineJobs/intro-pipeline-job-l3fo7be9\" is in state JOB_STATE_SUCCEEDED and cannot be canceled.",
Step #5: Step #4: "status": "FAILED_PRECONDITION"
Step #5: Step #4: }
Step #5: Step #4: }
Step #5: Step #4: {
Step #5: Step #4: "name": "projects/1012616486416/locations/us-central1/operations/5832465841647517696",
Step #5: Step #4: "metadata": {
Step #5: Step #4: "@type": "type.googleapis.com/google.cloud.aiplatform.v1beta1.DeleteOperationMetadata",
Step #5: Step #4: "genericMetadata": {
Step #5: Step #4: "createTime": "2024-07-01T20:12:57.351407Z",
Step #5: Step #4: "updateTime": "2024-07-01T20:12:57.351407Z"
Step #5: Step #4: }
Step #5: Step #4: },
Step #5: Step #4: "done": true,
Step #5: Step #4: "response": {
Step #5: Step #4: "@type": "type.googleapis.com/google.protobuf.Empty"
Step #5: Step #4: }
Step #5: Step #4: }
Step #5: Step #4:
Executing: 52%|█████▏ | 37/71 [00:35<00:13, 2.58cell/s]
Executing: 56%|█████▋ | 40/71 [00:35<00:08, 3.58cell/s]
Executing: 61%|██████ | 43/71 [00:35<00:05, 4.84cell/s]
Executing: 62%|██████▏ | 44/71 [00:50<00:05, 4.84cell/s]
Executing: 63%|██████▎ | 45/71 [04:45<12:19, 28.45s/cell]
Executing: 68%|██████▊ | 48/71 [04:46<07:21, 19.21s/cell]
Executing: 69%|██████▉ | 49/71 [05:00<07:02, 19.21s/cell]
Executing: 70%|███████ | 50/71 [05:08<06:03, 17.30s/cell]
Executing: 73%|███████▎ | 52/71 [05:08<04:06, 12.96s/cell]
Executing: 75%|███████▍ | 53/71 [05:20<03:53, 12.96s/cell]
Executing: 76%|███████▌ | 54/71 [07:16<07:31, 26.53s/cell]
Executing: 79%|███████▉ | 56/71 [07:17<04:49, 19.28s/cell]
Executing: 83%|████████▎ | 59/71 [07:20<02:29, 12.50s/cell]
Executing: 86%|████████▌ | 61/71 [07:20<01:32, 9.25s/cell]
Executing: 89%|████████▊ | 63/71 [07:21<00:55, 6.93s/cell]
Executing: 92%|█████████▏| 65/71 [07:24<00:31, 5.31s/cell]
Executing: 94%|█████████▍| 67/71 [07:26<00:16, 4.11s/cell]
Executing: 97%|█████████▋| 69/71 [07:28<00:06, 3.24s/cell]
Executing: 100%|██████████| 71/71 [07:30<00:00, 2.58s/cell]
Executing: 100%|██████████| 71/71 [07:31<00:00, 6.35s/cell]
Step #5: Step #4: Copying file://notebooks/official/pipelines/pipelines_intro_kfp.ipynb [Content-Type=application/octet-stream]...
Step #5: Step #4: / [0 files][ 0.0 B/443.3 KiB]
/ [1 files][443.3 KiB/443.3 KiB]
Step #5: Step #4: Operation completed over 1 objects/443.3 KiB.
Step #5: Step #4:
Step #5: Step #4: === EXECUTION FINISHED ===
Step #5: Step #4:
Step #5: Finished Step #4
Step #5: PUSH
Step #5: DONE
Step #5:
Step #5: Updating build results ...
Step #5: adding notebooks/official/pipelines/pipelines_intro_kfp.ipynb
Step #5: Saving accumulative results to build_results/eec7893f-2557-478c-9676-3a73ee22f9fa.json, nentries 1
Step #5:
Step #5: === END RESULTS===
Step #5:
Step #5: Cumulative notebook duration: 00:11:19
Finished Step #5
PUSH
DONE
Loading