Google Stackdriver Prometheus exporter
Clone or download
frodenas Merge pull request #35 from olvesh/retry-with-exp-backoff
Added exponential backoff and retries on stackdriver api
Latest commit c367079 Oct 16, 2018

README.md

Google Stackdriver Prometheus Exporter Build Status

A Prometheus exporter for Google Stackdriver Monitoring metrics. It acts as a proxy that requests Stackdriver API for the metric's time-series everytime prometheus scrapes it.

Installation

Binaries

Download the already existing binaries for your platform:

$ ./stackdriver_exporter <flags>

From source

Using the standard go install (you must have Go already installed in your local machine):

$ go install github.com/frodenas/stackdriver_exporter
$ stackdriver_exporter <flags>

Docker

To run the stackdriver exporter as a Docker container, run:

$ docker run -p 9255:9255 frodenas/stackdriver-exporter <flags>

Cloud Foundry

The exporter can be deployed to an already existing Cloud Foundry environment:

$ git clone https://github.com/frodenas/stackdriver_exporter.git
$ cd stackdriver_exporter

Modify the included application manifest file to include the desired properties. Then you can push the exporter to your Cloud Foundry environment:

$ cf push

BOSH

This exporter can be deployed using the Prometheus BOSH Release.

Usage

Credentials and Permissions

The Google Stackdriver Exporter uses the Google Golang Client Library, which offers a variety of ways to provide credentials. Please refer to the Google Application Default Credentials documentation to see how the credentials can be provided.

If you are using IAM roles, the monitoring.metricDescriptors.list and monitoring.timeSeries.list IAM permissions are required. The roles/monitoring.viewer IAM role contains those permissions. See the Access Control Guide for more information.

If you are still using the legacy Access scopes, the https://www.googleapis.com/auth/monitoring.read scope is required.

Flags

Flag / Environment Variable Required Default Description
google.project-id
STACKDRIVER_EXPORTER_GOOGLE_PROJECT_ID
Yes Google Project ID
monitoring.metrics-type-prefixes
STACKDRIVER_EXPORTER_MONITORING_METRICS_TYPE_PREFIXES
Yes Comma separated Google Stackdriver Monitoring Metric Type prefixes (see example and available metrics)
monitoring.metrics-interval
STACKDRIVER_EXPORTER_MONITORING_METRICS_INTERVAL
No 5m Metric's timestamp interval to request from the Google Stackdriver Monitoring Metrics API. Only the most recent data point is used
monitoring.metrics-offset
STACKDRIVER_EXPORTER_MONITORING_METRICS_OFFSET
No 0s Offset (into the past) for the metric's timestamp interval to request from the Google Stackdriver Monitoring Metrics API, to handle latency in published metrics
web.listen-address
STACKDRIVER_EXPORTER_WEB_LISTEN_ADDRESS
No :9255 Address to listen on for web interface and telemetry
web.telemetry-path
STACKDRIVER_EXPORTER_WEB_TELEMETRY_PATH
No /metrics Path under which to expose Prometheus metrics

Metrics

The exporter returns the following metrics:

Metric Description Labels
stackdriver_monitoring_api_calls_total Total number of Google Stackdriver Monitoring API calls made project_id
stackdriver_monitoring_scrapes_total Total number of Google Stackdriver Monitoring metrics scrapes project_id
stackdriver_monitoring_scrape_errors_total Total number of Google Stackdriver Monitoring metrics scrape errors project_id
stackdriver_monitoring_last_scrape_error Whether the last metrics scrape from Google Stackdriver Monitoring resulted in an error (1 for error, 0 for success) project_id
stackdriver_monitoring_last_scrape_timestamp Number of seconds since 1970 since last metrics scrape from Google Stackdriver Monitoring project_id
stackdriver_monitoring_last_scrape_duration_seconds Duration of the last metrics scrape from Google Stackdriver Monitoring project_id

Metrics gathered from Google Stackdriver Monitoring are converted to Prometheus metrics:

  • Metric's names are normalized according to the Prometheus specification using the following pattern:
    1. namespace is a constant prefix (stackdriver)
    2. subsystem is the normalized monitored resource type (ie gce_instance)
    3. name is the normalized metric type (ie compute_googleapis_com_instance_cpu_usage_time)
  • Labels attached to each metric are an aggregation of:
    1. the unit in which the metric value is reported
    2. the metric type labels (see Metrics List)
    3. the monitored resource labels (see Monitored Resource Types)
  • For each timeseries, only the most recent data point is exported.
  • Stackdriver GAUGE and DELTA metric kinds are reported as Prometheus Gauge metrics; Stackdriver CUMULATIVE metric kinds are reported as Prometheus Counter metrics.
  • Only BOOL, INT64, DOUBLE and DISTRIBUTION metric types are supported, other types (STRING and MONEY) are discarded.
  • DISTRIBUTION metric type is reported as a Prometheus Histogram, except the _sum time series is not supported.

Example

If we want to get all CPU (compute.googleapis.com/instance/cpu) and Disk (compute.googleapis.com/instance/disk) metrics for all Google Compute Engine instances, we can run the exporter with the following options:

stackdriver_exporter \
  --google.project-id my-test-project \
  --monitoring.metrics-type-prefixes "compute.googleapis.com/instance/cpu,compute.googleapis.com/instance/disk"

Contributing

Refer to the contributing guidelines.

License

Apache License 2.0, see LICENSE.