Skip to content
This repository has been archived by the owner on Jul 13, 2023. It is now read-only.

Commit

Permalink
feat: add support for Dataproc metric configuration (#626)
Browse files Browse the repository at this point in the history
* test: use fully qualified request type name in tests

PiperOrigin-RevId: 475685359

Source-Link: googleapis/googleapis@7a12973

Source-Link: googleapis/googleapis-gen@370c729
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMzcwYzcyOWUyYmEwNjJhMTY3NDQ5YzI3ODgyYmE1ZjM3OWM1YzM0ZCJ9

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* feat: add support for Dataproc metric configuration

Committer: @AkshatBhargava
PiperOrigin-RevId: 475750057

Source-Link: googleapis/googleapis@30517fd

Source-Link: googleapis/googleapis-gen@a41cc2a
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiYTQxY2MyYWIzZGY3OGVlMGI4NGY2NzQ5OWU3NTYyZGUwMmFiMzRjMCJ9

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: sofisl <55454395+sofisl@users.noreply.github.com>
  • Loading branch information
3 people committed Sep 22, 2022
1 parent e1c6d8b commit 79a4958
Show file tree
Hide file tree
Showing 9 changed files with 1,716 additions and 525 deletions.
46 changes: 46 additions & 0 deletions protos/google/cloud/dataproc/v1/clusters.proto
Original file line number Diff line number Diff line change
Expand Up @@ -284,6 +284,9 @@ message ClusterConfig {

// Optional. Metastore configuration.
MetastoreConfig metastore_config = 20 [(google.api.field_behavior) = OPTIONAL];

// Optional. Dataproc metrics configuration.
DataprocMetricConfig dataproc_metric_config = 21 [(google.api.field_behavior) = OPTIONAL];
}

// Dataproc cluster config for a cluster that does not directly control the
Expand Down Expand Up @@ -942,6 +945,49 @@ message MetastoreConfig {
];
}


// Specifies Dataproc OSS Metric.
message Metric {
enum MetricSource {
// Unspecified metric source
METRIC_SOURCE_UNSPECIFIED = 0;

// Default monitoring agent metrics. If this source is enabled,
// Dataproc enables the monitoring agent in Compute Engine, and collects
// default monitoring agent metrics, which are published with an
// agent.googleapis.com prefix.
MONITORING_AGENT_DEFAULTS = 1;

// HDFS metric source
HDFS = 2;

// SPARK metric source
SPARK = 3;

// YARN metric source
YARN = 4;

// Spark History Server metric source
SPARK_HISTORY_SERVER = 5;

// Hiveserver2 metric source
HIVESERVER2 = 6;
}

// Required. Specified source of metric collection
MetricSource metric_source = 1 [(google.api.field_behavior) = REQUIRED];

// Optional. The set of available OSS metrics to collect from the metric
// source.
repeated string metric_overrides = 2 [(google.api.field_behavior) = OPTIONAL];
}

// Specifies a Dataproc metric config
message DataprocMetricConfig {
// Configuration set of metrics to collect from the cluster
repeated Metric metrics = 1 [(google.api.field_behavior) = REQUIRED];
}

// Contains cluster daemon metrics, such as HDFS and YARN stats.
//
// **Beta Feature**: This report is available for testing purposes only. It may
Expand Down
220 changes: 220 additions & 0 deletions protos/protos.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 79a4958

Please sign in to comment.