Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add SparkR and Presto jobs to WorkflowTemplates; add new optional components; add submit_job_as_operation to v1 (via synth) #21

Merged
merged 3 commits into from May 19, 2020

Conversation

@yoshi-automation
Copy link
Contributor

@yoshi-automation yoshi-automation commented May 13, 2020

This PR was generated using Autosynth. 🌈

Log from Synthtool
2020-05-13 05:23:56,490 synthtool [DEBUG] > Executing /home/kbuilder/.cache/synthtool/python-dataproc/synth.py.
On branch autosynth
nothing to commit, working tree clean
2020-05-13 05:23:56,631 synthtool [DEBUG] > Ensuring dependencies.
2020-05-13 05:23:57,261 synthtool [DEBUG] > Cloning googleapis.
2020-05-13 05:23:57,933 synthtool [DEBUG] > Generating code for: //google/cloud/dataproc/v1beta2:dataproc-v1beta2-py.
2020-05-13 05:24:03,770 synthtool [DEBUG] > Copy: google/cloud/dataproc/v1beta2/clusters.proto to /tmpfs/tmp/tmpups1ph99/google/cloud/dataproc_v1beta2/proto/clusters.proto
2020-05-13 05:24:03,770 synthtool [DEBUG] > Copy: google/cloud/dataproc/v1beta2/jobs.proto to /tmpfs/tmp/tmpups1ph99/google/cloud/dataproc_v1beta2/proto/jobs.proto
2020-05-13 05:24:03,770 synthtool [DEBUG] > Copy: google/cloud/dataproc/v1beta2/operations.proto to /tmpfs/tmp/tmpups1ph99/google/cloud/dataproc_v1beta2/proto/operations.proto
2020-05-13 05:24:03,771 synthtool [DEBUG] > Copy: google/cloud/dataproc/v1beta2/autoscaling_policies.proto to /tmpfs/tmp/tmpups1ph99/google/cloud/dataproc_v1beta2/proto/autoscaling_policies.proto
2020-05-13 05:24:03,771 synthtool [DEBUG] > Copy: google/cloud/dataproc/v1beta2/workflow_templates.proto to /tmpfs/tmp/tmpups1ph99/google/cloud/dataproc_v1beta2/proto/workflow_templates.proto
2020-05-13 05:24:03,771 synthtool [DEBUG] > Copy: google/cloud/dataproc/v1beta2/shared.proto to /tmpfs/tmp/tmpups1ph99/google/cloud/dataproc_v1beta2/proto/shared.proto
2020-05-13 05:24:03,771 synthtool [SUCCESS] > Placed proto files into /tmpfs/tmp/tmpups1ph99/google/cloud/dataproc_v1beta2/proto.
2020-05-13 05:24:03,772 synthtool [SUCCESS] > Generated code into /tmpfs/tmp/tmpups1ph99.
2020-05-13 05:24:03,801 synthtool [INFO] > Replaced 'metadata_type=operations_pb2.ClusterOperationMetadata,' in google/cloud/dataproc_v1beta2/gapic/cluster_controller_client.py.
2020-05-13 05:24:03,804 synthtool [WARNING] > No replacements made in google/cloud/dataproc_v1beta2/gapic/cluster_controller_client.py for pattern \s+<strong>Note:</strong>.*
(.*
)+?.*types.FieldMask., maybe replacement is no longer needed?
2020-05-13 05:24:03,805 synthtool [INFO] > Replaced ', and must\n\\s+conform to the following PCRE regular expression:(.*\n)+?.*No more than 32' in google/cloud/dataproc_v1beta2/proto/workflow_templates_pb2.py.
2020-05-13 05:24:03,807 synthtool [INFO] > Replaced ', and must conform to\n\\s+the following regular expression:(.*\n)+?.* No more than' in google/cloud/dataproc_v1beta2/proto/workflow_templates_pb2.py.
2020-05-13 05:24:03,807 synthtool [DEBUG] > Generating code for: //google/cloud/dataproc/v1:dataproc-v1-py.
2020-05-13 05:24:09,752 synthtool [DEBUG] > Copy: google/cloud/dataproc/v1/clusters.proto to /tmpfs/tmp/tmpvaa4eyup/google/cloud/dataproc_v1/proto/clusters.proto
2020-05-13 05:24:09,753 synthtool [DEBUG] > Copy: google/cloud/dataproc/v1/jobs.proto to /tmpfs/tmp/tmpvaa4eyup/google/cloud/dataproc_v1/proto/jobs.proto
2020-05-13 05:24:09,753 synthtool [DEBUG] > Copy: google/cloud/dataproc/v1/operations.proto to /tmpfs/tmp/tmpvaa4eyup/google/cloud/dataproc_v1/proto/operations.proto
2020-05-13 05:24:09,754 synthtool [DEBUG] > Copy: google/cloud/dataproc/v1/autoscaling_policies.proto to /tmpfs/tmp/tmpvaa4eyup/google/cloud/dataproc_v1/proto/autoscaling_policies.proto
2020-05-13 05:24:09,754 synthtool [DEBUG] > Copy: google/cloud/dataproc/v1/workflow_templates.proto to /tmpfs/tmp/tmpvaa4eyup/google/cloud/dataproc_v1/proto/workflow_templates.proto
2020-05-13 05:24:09,754 synthtool [DEBUG] > Copy: google/cloud/dataproc/v1/shared.proto to /tmpfs/tmp/tmpvaa4eyup/google/cloud/dataproc_v1/proto/shared.proto
2020-05-13 05:24:09,754 synthtool [SUCCESS] > Placed proto files into /tmpfs/tmp/tmpvaa4eyup/google/cloud/dataproc_v1/proto.
2020-05-13 05:24:09,754 synthtool [SUCCESS] > Generated code into /tmpfs/tmp/tmpvaa4eyup.
2020-05-13 05:24:09,788 synthtool [INFO] > Replaced 'metadata_type=operations_pb2.ClusterOperationMetadata,' in google/cloud/dataproc_v1/gapic/cluster_controller_client.py.
2020-05-13 05:24:09,789 synthtool [WARNING] > No replacements made in google/cloud/dataproc_v1/gapic/cluster_controller_client.py for pattern \s+<strong>Note:</strong>.*
(.*
)+?.*types.FieldMask., maybe replacement is no longer needed?
2020-05-13 05:24:09,791 synthtool [INFO] > Replaced ', and must\n\\s+conform to the following PCRE regular expression:(.*\n)+?.*No more than 32' in google/cloud/dataproc_v1/proto/workflow_templates_pb2.py.
2020-05-13 05:24:09,792 synthtool [INFO] > Replaced ', and must conform to\n\\s+the following regular expression:(.*\n)+?.* No more than' in google/cloud/dataproc_v1/proto/workflow_templates_pb2.py.
2020-05-13 05:24:09,793 synthtool [INFO] > Replaced '# Generated by the protocol buffer compiler.  DO NOT EDIT!' in google/cloud/dataproc_v1beta2/proto/clusters_pb2.py.
.coveragerc
.flake8
.github/CONTRIBUTING.md
.github/ISSUE_TEMPLATE/bug_report.md
.github/ISSUE_TEMPLATE/feature_request.md
.github/ISSUE_TEMPLATE/support_request.md
.github/PULL_REQUEST_TEMPLATE.md
.github/release-please.yml
.gitignore
.kokoro/build.sh
.kokoro/continuous/common.cfg
.kokoro/continuous/continuous.cfg
.kokoro/docs/common.cfg
.kokoro/docs/docs.cfg
.kokoro/presubmit/common.cfg
.kokoro/presubmit/presubmit.cfg
.kokoro/publish-docs.sh
.kokoro/release.sh
.kokoro/release/common.cfg
.kokoro/release/release.cfg
.kokoro/trampoline.sh
CODE_OF_CONDUCT.md
CONTRIBUTING.rst
LICENSE
MANIFEST.in
docs/_static/custom.css
docs/_templates/layout.html
docs/conf.py.j2
docs/multiprocessing.rst
noxfile.py.j2
renovate.json
setup.cfg
Running session blacken
Creating virtual environment (virtualenv) using python3.6 in .nox/blacken
pip install black==19.3b0
black docs google tests noxfile.py setup.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/__init__.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/__init__.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1/__init__.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1/gapic/autoscaling_policy_service_client_config.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/docs/conf.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1/gapic/cluster_controller_client_config.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1/gapic/enums.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1/gapic/job_controller_client_config.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1/gapic/transports/autoscaling_policy_service_grpc_transport.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1/gapic/autoscaling_policy_service_client.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1/gapic/cluster_controller_client.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1/gapic/transports/cluster_controller_grpc_transport.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1/gapic/transports/job_controller_grpc_transport.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1/gapic/transports/workflow_template_service_grpc_transport.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1/gapic/workflow_template_service_client_config.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1/gapic/job_controller_client.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1/proto/autoscaling_policies_pb2_grpc.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1/proto/clusters_pb2_grpc.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1/gapic/workflow_template_service_client.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1/proto/jobs_pb2_grpc.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1/proto/autoscaling_policies_pb2.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1/proto/operations_pb2_grpc.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1/proto/operations_pb2.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1/proto/shared_pb2_grpc.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1/proto/shared_pb2.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1/proto/workflow_templates_pb2_grpc.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1/types.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1beta2/__init__.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1beta2/gapic/autoscaling_policy_service_client.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1beta2/gapic/autoscaling_policy_service_client_config.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1beta2/gapic/cluster_controller_client.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1beta2/gapic/cluster_controller_client_config.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1beta2/gapic/enums.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1beta2/gapic/job_controller_client.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1beta2/gapic/job_controller_client_config.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1beta2/gapic/transports/autoscaling_policy_service_grpc_transport.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1beta2/gapic/transports/cluster_controller_grpc_transport.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1beta2/gapic/transports/job_controller_grpc_transport.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1beta2/gapic/transports/workflow_template_service_grpc_transport.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1beta2/gapic/workflow_template_service_client.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1beta2/gapic/workflow_template_service_client_config.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1/proto/clusters_pb2.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1beta2/proto/autoscaling_policies_pb2_grpc.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1/proto/workflow_templates_pb2.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1beta2/proto/clusters_pb2_grpc.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1beta2/proto/autoscaling_policies_pb2.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1beta2/proto/jobs_pb2_grpc.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1beta2/proto/operations_pb2.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1beta2/proto/operations_pb2_grpc.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1beta2/proto/shared_pb2.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1beta2/proto/shared_pb2_grpc.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1/proto/jobs_pb2.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1beta2/proto/workflow_templates_pb2_grpc.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1beta2/types.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/noxfile.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/tests/system/gapic/v1/test_system_cluster_controller_v1.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/tests/system/gapic/v1beta2/test_system_cluster_controller_v1beta2.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/tests/unit/gapic/v1/test_autoscaling_policy_service_client_v1.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/tests/unit/gapic/v1/test_cluster_controller_client_v1.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/tests/unit/gapic/v1/test_job_controller_client_v1.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/tests/unit/gapic/v1/test_workflow_template_service_client_v1.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/tests/unit/gapic/v1beta2/test_autoscaling_policy_service_client_v1beta2.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/tests/unit/gapic/v1beta2/test_cluster_controller_client_v1beta2.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/tests/unit/gapic/v1beta2/test_job_controller_client_v1beta2.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1beta2/proto/clusters_pb2.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1beta2/proto/workflow_templates_pb2.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/tests/unit/gapic/v1beta2/test_workflow_template_service_client_v1beta2.py
reformatted /home/kbuilder/.cache/synthtool/python-dataproc/google/cloud/dataproc_v1beta2/proto/jobs_pb2.py
All done! ✨ 🍰 ✨
69 files reformatted, 7 files left unchanged.
Session blacken was successful.
2020-05-13 05:24:27,535 synthtool [DEBUG] > Wrote metadata to synth.metadata.

@busunkim96 busunkim96 changed the title [CHANGE ME] Re-generated to pick up changes in the API or client library generator. feat: add SparkR and Presto jobs to WorkflowTemplates; add new optional components; add SubmitJobAsOperation to v1 (via synth) May 19, 2020
@busunkim96 busunkim96 changed the title feat: add SparkR and Presto jobs to WorkflowTemplates; add new optional components; add SubmitJobAsOperation to v1 (via synth) feat: add SparkR and Presto jobs to WorkflowTemplates; add new optional components; add submit_job_as_operation to v1 (via synth) May 19, 2020
"""Return a fully-qualified autoscaling_policy string."""
return google.api_core.path_template.expand(
"projects/{project}/regions/{region}/autoscalingPolicies/{autoscaling_policy}",
"projects/{project}/locations/{location}/autoscalingPolicies/{autoscaling_policy}",
Copy link
Collaborator

@busunkim96 busunkim96 May 19, 2020

Check why this resource path helper method is changing

Copy link
Collaborator

@busunkim96 busunkim96 May 19, 2020

Luke says that region and zone were used by APIs before this was standardized to location. Given that the proto annotation specifies both paths this change should not impact users.

@busunkim96 busunkim96 requested a review from bradmiro May 19, 2020
@busunkim96 busunkim96 merged commit 1cf10b6 into master May 19, 2020
3 checks passed
gcf-merge-on-green bot pushed a commit that referenced this issue May 19, 2020
🤖 I have created a release \*beep\* \*boop\* 
---
## [0.8.0](https://www.github.com/googleapis/python-dataproc/compare/v0.7.0...v0.8.0) (2020-05-19)


### Features

* add SparkR and Presto jobs to WorkflowTemplates; add new optional components; add submit_job_as_operation to v1 (via synth) ([#21](https://www.github.com/googleapis/python-dataproc/issues/21)) ([1cf10b6](https://www.github.com/googleapis/python-dataproc/commit/1cf10b6b127a63dbeb34958771c2cc8d8cb37099))
---


This PR was generated with [Release Please](https://github.com/googleapis/release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants