Skip to content
This repository was archived by the owner on Nov 29, 2023. It is now read-only.

Commit 5137a6f

Browse files
chore: upgrade gapic-generator-python to 0.46.3 (#188)
feat: update the Dataproc V1 API client library This includes the following: 1. The new start and stop cluster methods. 2. The ability to specify a metastore config in a cluster. 3. The ability to specify a (BETA) GKE cluster when creating a Dataproc cluster. 4. The ability to configure the behavior for private IPv6 cluster networking. 5. The ability to specify node affinity groups for clusters. 6. The ability to specify shielded VM configurations for clusters. 7. Support for service-account based secure multi-tenancy. 8. The ability to specify cluster labels for picking which cluster should run a job. 9. Components for DOCKER, DRUID, FLINK, HBASE, RANGER, and SOLR 10. The ability to specify a DAG timeout for workflows. feat: support self-signed JWT flow for service accounts fix: add async client to %name_%version/init.py chore: add autogenerated snippets chore: remove auth, policy, and options from the reserved names list chore: enable GAPIC metadata generation chore: sort subpackages in %namespace/%name/init.py
1 parent c1ee50b commit 5137a6f

113 files changed

Lines changed: 6592 additions & 4379 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

dataproc-v1beta2-py.tar.gz

Whitespace-only changes.

docs/dataproc_v1/autoscaling_policy_service.rst

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,6 @@ AutoscalingPolicyService
55
:members:
66
:inherited-members:
77

8-
98
.. automodule:: google.cloud.dataproc_v1.services.autoscaling_policy_service.pagers
109
:members:
1110
:inherited-members:

docs/dataproc_v1/cluster_controller.rst

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,6 @@ ClusterController
55
:members:
66
:inherited-members:
77

8-
98
.. automodule:: google.cloud.dataproc_v1.services.cluster_controller.pagers
109
:members:
1110
:inherited-members:

docs/dataproc_v1/job_controller.rst

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,6 @@ JobController
55
:members:
66
:inherited-members:
77

8-
98
.. automodule:: google.cloud.dataproc_v1.services.job_controller.pagers
109
:members:
1110
:inherited-members:

docs/dataproc_v1/workflow_template_service.rst

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,6 @@ WorkflowTemplateService
55
:members:
66
:inherited-members:
77

8-
98
.. automodule:: google.cloud.dataproc_v1.services.workflow_template_service.pagers
109
:members:
1110
:inherited-members:

docs/dataproc_v1beta2/autoscaling_policy_service.rst

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,6 @@ AutoscalingPolicyService
55
:members:
66
:inherited-members:
77

8-
98
.. automodule:: google.cloud.dataproc_v1beta2.services.autoscaling_policy_service.pagers
109
:members:
1110
:inherited-members:

docs/dataproc_v1beta2/cluster_controller.rst

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,6 @@ ClusterController
55
:members:
66
:inherited-members:
77

8-
98
.. automodule:: google.cloud.dataproc_v1beta2.services.cluster_controller.pagers
109
:members:
1110
:inherited-members:

docs/dataproc_v1beta2/job_controller.rst

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,6 @@ JobController
55
:members:
66
:inherited-members:
77

8-
98
.. automodule:: google.cloud.dataproc_v1beta2.services.job_controller.pagers
109
:members:
1110
:inherited-members:

docs/dataproc_v1beta2/workflow_template_service.rst

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,6 @@ WorkflowTemplateService
55
:members:
66
:inherited-members:
77

8-
98
.. automodule:: google.cloud.dataproc_v1beta2.services.workflow_template_service.pagers
109
:members:
1110
:inherited-members:

google/cloud/dataproc/__init__.py

Lines changed: 72 additions & 58 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,4 @@
11
# -*- coding: utf-8 -*-
2-
32
# Copyright 2020 Google LLC
43
#
54
# Licensed under the Apache License, Version 2.0 (the "License");
@@ -15,28 +14,29 @@
1514
# limitations under the License.
1615
#
1716

18-
from google.cloud.dataproc_v1.services.autoscaling_policy_service.async_client import (
19-
AutoscalingPolicyServiceAsyncClient,
20-
)
2117
from google.cloud.dataproc_v1.services.autoscaling_policy_service.client import (
2218
AutoscalingPolicyServiceClient,
2319
)
24-
from google.cloud.dataproc_v1.services.cluster_controller.async_client import (
25-
ClusterControllerAsyncClient,
20+
from google.cloud.dataproc_v1.services.autoscaling_policy_service.async_client import (
21+
AutoscalingPolicyServiceAsyncClient,
2622
)
2723
from google.cloud.dataproc_v1.services.cluster_controller.client import (
2824
ClusterControllerClient,
2925
)
30-
from google.cloud.dataproc_v1.services.job_controller.async_client import (
31-
JobControllerAsyncClient,
26+
from google.cloud.dataproc_v1.services.cluster_controller.async_client import (
27+
ClusterControllerAsyncClient,
3228
)
3329
from google.cloud.dataproc_v1.services.job_controller.client import JobControllerClient
34-
from google.cloud.dataproc_v1.services.workflow_template_service.async_client import (
35-
WorkflowTemplateServiceAsyncClient,
30+
from google.cloud.dataproc_v1.services.job_controller.async_client import (
31+
JobControllerAsyncClient,
3632
)
3733
from google.cloud.dataproc_v1.services.workflow_template_service.client import (
3834
WorkflowTemplateServiceClient,
3935
)
36+
from google.cloud.dataproc_v1.services.workflow_template_service.async_client import (
37+
WorkflowTemplateServiceAsyncClient,
38+
)
39+
4040
from google.cloud.dataproc_v1.types.autoscaling_policies import AutoscalingPolicy
4141
from google.cloud.dataproc_v1.types.autoscaling_policies import (
4242
BasicAutoscalingAlgorithm,
@@ -80,16 +80,23 @@
8080
from google.cloud.dataproc_v1.types.clusters import EndpointConfig
8181
from google.cloud.dataproc_v1.types.clusters import GceClusterConfig
8282
from google.cloud.dataproc_v1.types.clusters import GetClusterRequest
83+
from google.cloud.dataproc_v1.types.clusters import GkeClusterConfig
84+
from google.cloud.dataproc_v1.types.clusters import IdentityConfig
8385
from google.cloud.dataproc_v1.types.clusters import InstanceGroupConfig
8486
from google.cloud.dataproc_v1.types.clusters import KerberosConfig
8587
from google.cloud.dataproc_v1.types.clusters import LifecycleConfig
8688
from google.cloud.dataproc_v1.types.clusters import ListClustersRequest
8789
from google.cloud.dataproc_v1.types.clusters import ListClustersResponse
8890
from google.cloud.dataproc_v1.types.clusters import ManagedGroupConfig
91+
from google.cloud.dataproc_v1.types.clusters import MetastoreConfig
92+
from google.cloud.dataproc_v1.types.clusters import NodeGroupAffinity
8993
from google.cloud.dataproc_v1.types.clusters import NodeInitializationAction
9094
from google.cloud.dataproc_v1.types.clusters import ReservationAffinity
9195
from google.cloud.dataproc_v1.types.clusters import SecurityConfig
96+
from google.cloud.dataproc_v1.types.clusters import ShieldedInstanceConfig
9297
from google.cloud.dataproc_v1.types.clusters import SoftwareConfig
98+
from google.cloud.dataproc_v1.types.clusters import StartClusterRequest
99+
from google.cloud.dataproc_v1.types.clusters import StopClusterRequest
93100
from google.cloud.dataproc_v1.types.clusters import UpdateClusterRequest
94101
from google.cloud.dataproc_v1.types.jobs import CancelJobRequest
95102
from google.cloud.dataproc_v1.types.jobs import DeleteJobRequest
@@ -155,96 +162,103 @@
155162
from google.cloud.dataproc_v1.types.workflow_templates import WorkflowTemplatePlacement
156163

157164
__all__ = (
158-
"AcceleratorConfig",
159-
"AutoscalingConfig",
160-
"AutoscalingPolicy",
161-
"AutoscalingPolicyServiceAsyncClient",
162165
"AutoscalingPolicyServiceClient",
166+
"AutoscalingPolicyServiceAsyncClient",
167+
"ClusterControllerClient",
168+
"ClusterControllerAsyncClient",
169+
"JobControllerClient",
170+
"JobControllerAsyncClient",
171+
"WorkflowTemplateServiceClient",
172+
"WorkflowTemplateServiceAsyncClient",
173+
"AutoscalingPolicy",
163174
"BasicAutoscalingAlgorithm",
164175
"BasicYarnAutoscalingConfig",
165-
"CancelJobRequest",
176+
"CreateAutoscalingPolicyRequest",
177+
"DeleteAutoscalingPolicyRequest",
178+
"GetAutoscalingPolicyRequest",
179+
"InstanceGroupAutoscalingPolicyConfig",
180+
"ListAutoscalingPoliciesRequest",
181+
"ListAutoscalingPoliciesResponse",
182+
"UpdateAutoscalingPolicyRequest",
183+
"AcceleratorConfig",
184+
"AutoscalingConfig",
166185
"Cluster",
167186
"ClusterConfig",
168-
"ClusterControllerAsyncClient",
169-
"ClusterControllerClient",
170187
"ClusterMetrics",
171-
"ClusterOperation",
172-
"ClusterOperationMetadata",
173-
"ClusterOperationStatus",
174-
"ClusterSelector",
175188
"ClusterStatus",
176-
"Component",
177-
"CreateAutoscalingPolicyRequest",
178189
"CreateClusterRequest",
179-
"CreateWorkflowTemplateRequest",
180-
"DeleteAutoscalingPolicyRequest",
181190
"DeleteClusterRequest",
182-
"DeleteJobRequest",
183-
"DeleteWorkflowTemplateRequest",
184191
"DiagnoseClusterRequest",
185192
"DiagnoseClusterResults",
186193
"DiskConfig",
187194
"EncryptionConfig",
188195
"EndpointConfig",
189196
"GceClusterConfig",
190-
"GetAutoscalingPolicyRequest",
191197
"GetClusterRequest",
198+
"GkeClusterConfig",
199+
"IdentityConfig",
200+
"InstanceGroupConfig",
201+
"KerberosConfig",
202+
"LifecycleConfig",
203+
"ListClustersRequest",
204+
"ListClustersResponse",
205+
"ManagedGroupConfig",
206+
"MetastoreConfig",
207+
"NodeGroupAffinity",
208+
"NodeInitializationAction",
209+
"ReservationAffinity",
210+
"SecurityConfig",
211+
"ShieldedInstanceConfig",
212+
"SoftwareConfig",
213+
"StartClusterRequest",
214+
"StopClusterRequest",
215+
"UpdateClusterRequest",
216+
"CancelJobRequest",
217+
"DeleteJobRequest",
192218
"GetJobRequest",
193-
"GetWorkflowTemplateRequest",
194219
"HadoopJob",
195220
"HiveJob",
196-
"InstanceGroupAutoscalingPolicyConfig",
197-
"InstanceGroupConfig",
198-
"InstantiateInlineWorkflowTemplateRequest",
199-
"InstantiateWorkflowTemplateRequest",
200221
"Job",
201-
"JobControllerAsyncClient",
202-
"JobControllerClient",
203222
"JobMetadata",
204223
"JobPlacement",
205224
"JobReference",
206225
"JobScheduling",
207226
"JobStatus",
208-
"KerberosConfig",
209-
"LifecycleConfig",
210-
"ListAutoscalingPoliciesRequest",
211-
"ListAutoscalingPoliciesResponse",
212-
"ListClustersRequest",
213-
"ListClustersResponse",
214227
"ListJobsRequest",
215228
"ListJobsResponse",
216-
"ListWorkflowTemplatesRequest",
217-
"ListWorkflowTemplatesResponse",
218229
"LoggingConfig",
219-
"ManagedCluster",
220-
"ManagedGroupConfig",
221-
"NodeInitializationAction",
222-
"OrderedJob",
223-
"ParameterValidation",
224230
"PigJob",
225231
"PrestoJob",
226232
"PySparkJob",
227233
"QueryList",
228-
"RegexValidation",
229-
"ReservationAffinity",
230-
"SecurityConfig",
231-
"SoftwareConfig",
232234
"SparkJob",
233235
"SparkRJob",
234236
"SparkSqlJob",
235237
"SubmitJobRequest",
236-
"TemplateParameter",
237-
"UpdateAutoscalingPolicyRequest",
238-
"UpdateClusterRequest",
239238
"UpdateJobRequest",
239+
"YarnApplication",
240+
"ClusterOperationMetadata",
241+
"ClusterOperationStatus",
242+
"Component",
243+
"ClusterOperation",
244+
"ClusterSelector",
245+
"CreateWorkflowTemplateRequest",
246+
"DeleteWorkflowTemplateRequest",
247+
"GetWorkflowTemplateRequest",
248+
"InstantiateInlineWorkflowTemplateRequest",
249+
"InstantiateWorkflowTemplateRequest",
250+
"ListWorkflowTemplatesRequest",
251+
"ListWorkflowTemplatesResponse",
252+
"ManagedCluster",
253+
"OrderedJob",
254+
"ParameterValidation",
255+
"RegexValidation",
256+
"TemplateParameter",
240257
"UpdateWorkflowTemplateRequest",
241258
"ValueValidation",
242259
"WorkflowGraph",
243260
"WorkflowMetadata",
244261
"WorkflowNode",
245262
"WorkflowTemplate",
246263
"WorkflowTemplatePlacement",
247-
"WorkflowTemplateServiceAsyncClient",
248-
"WorkflowTemplateServiceClient",
249-
"YarnApplication",
250264
)

0 commit comments

Comments
 (0)