Cloud Foundry Firehose Prometheus exporter
Go Makefile
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
architecture Update architecture diagrams Nov 27, 2016
collectors Updates required by a new prometheus golang client Feb 2, 2018
filters Switch to new github repo Nov 25, 2017
firehosenozzle Ensure we handle gracefully the case when the firehose stops retrying Feb 21, 2018
metrics Switch to new github repo Nov 25, 2017
uaatokenrefresher Fix vet typecheck failure Apr 9, 2018
utils Updates required by a new prometheus golang client Feb 2, 2018
vendor Bump vendored packages Mar 1, 2018
.dockerignore Initial version Sep 19, 2016
.gitignore Initial version Sep 19, 2016
.promu.yml Switch to new github repo Nov 25, 2017
.travis.yml Update travis secure properties Feb 2, 2018
CODE_OF_CONDUCT.md Add code of conduct Nov 6, 2016
CONTRIBUTING.md Switch to new github repo Nov 25, 2017
Dockerfile Build docker imag Jul 6, 2017
FAQ.md Switch to new github repo Nov 25, 2017
Gopkg.lock Bump vendored packages Mar 1, 2018
Gopkg.toml Update vendored prometheus golang client Feb 2, 2018
LICENSE Update Copyright dates Feb 8, 2017
Makefile Remove GO15VENDOREXPERIMENT=1 Feb 19, 2018
NOTICE Update Copyright dates Feb 8, 2017
Procfile Switch to kingpin flags Dec 27, 2017
README.md Switch to kingpin flags Dec 27, 2017
VERSION Bump to version v5.0.3 Apr 9, 2018
firehose_exporter.go On metric writes, ignore errors and try to serve as many metrics as p… Apr 9, 2018
manifest.yml Add FIREHOSE_EXPORTER_METRICS_ENVIRONMENT env var to CF manifest file Feb 19, 2018

README.md

Cloud Foundry Firehose Exporter Build Status

A Prometheus exporter proxy for Cloud Foundry Firehose metrics. Please refer to the FAQ for general questions about this exporter.

Architecture overview

Installation

Binaries

Download the already existing binaries for your platform:

$ ./firehose_exporter <flags>

From source

Using the standard go install (you must have Go already installed in your local machine):

$ go install github.com/bosh-prometheus/firehose_exporter
$ firehose_exporter <flags>

Docker

To run the firehose exporter as a Docker container, run:

$ docker run -p 9186:9186 boshprometheus/firehose-exporter <flags>

Cloud Foundry

The exporter can be deployed to an already existing Cloud Foundry environment:

$ git clone https://github.com/bosh-prometheus/firehose_exporter.git
$ cd firehose_exporter

Modify the included application manifest file to include your Cloud Foundry Firehose properties. Then you can push the exporter to your Cloud Foundry environment:

$ cf push

BOSH

This exporter can be deployed using the Prometheus BOSH Release.

Usage

UAA Client

In order to connect to the Cloud Foundry Firehose a client-id and client-secret must be provided. The client-id must have the doppler.firehose authority.

For example, to create a new client-id and client-secret with the right permissions:

uaac target https://<YOUR UAA URL> --skip-ssl-validation
uaac token client get <YOUR ADMIN CLIENT ID> -s <YOUR ADMIN CLIENT SECRET>
uaac client add prometheus-firehose \
  --name prometheus-firehose \
  --secret prometheus-client-secret \
  --authorized_grant_types client_credentials,refresh_token \
  --authorities doppler.firehose

Flags

Flag / Environment Variable Required Default Description
uaa.url
FIREHOSE_EXPORTER_UAA_URL
Yes Cloud Foundry UAA URL
uaa.client-id
FIREHOSE_EXPORTER_UAA_CLIENT_ID
Yes Cloud Foundry UAA Client ID
uaa.client-secret
FIREHOSE_EXPORTER_UAA_CLIENT_SECRET
Yes Cloud Foundry UAA Client Secret
doppler.url
FIREHOSE_EXPORTER_DOPPLER_URL
Yes Cloud Foundry Doppler URL
doppler.subscription-id
FIREHOSE_EXPORTER_DOPPLER_SUBSCRIPTION_ID
No prometheus Cloud Foundry Doppler Subscription ID
doppler.idle-timeout
FIREHOSE_EXPORTER_DOPPLER_IDLE_TIMEOUT
No Cloud Foundry Doppler Idle Timeout duration
doppler.min-retry-delay
FIREHOSE_EXPORTER_DOPPLER_MIN_RETRY_DELAY
No Cloud Foundry Doppler min retry delay duration
doppler.max-retry-delay
FIREHOSE_EXPORTER_DOPPLER_MAX_RETRY_DELAY
No Cloud Foundry Doppler max retry delay duration
doppler.max-retry-count
FIREHOSE_EXPORTER_DOPPLER_MAX_RETRY_COUNT
No Cloud Foundry Doppler max retry count
doppler.metric-expiration
FIREHOSE_EXPORTER_DOPPLER_METRIC_EXPIRATION
No 5 minutes How long Cloud Foundry metrics received from the Firehose are valid
filter.deployments
FIREHOSE_EXPORTER_FILTER_DEPLOYMENTS
No Comma separated deployments to filter
filter.events
FIREHOSE_EXPORTER_FILTER_EVENTS
No Comma separated events to filter. If not set, all events will be enabled (ContainerMetric, CounterEvent, HttpStartStop, ValueMetric)
metrics.namespace
FIREHOSE_EXPORTER_METRICS_NAMESPACE
No firehose Metrics Namespace
metrics.environment
FIREHOSE_EXPORTER_METRICS_ENVIRONMENT
Yes Environment label to be attached to metrics
metrics.cleanup-interval
FIREHOSE_EXPORTER_METRICS_CLEANUP_INTERVAL
No 2 minutes Metrics clean up interval
skip-ssl-verify
FIREHOSE_EXPORTER_SKIP_SSL_VERIFY
No false Disable SSL Verify
web.listen-address
FIREHOSE_EXPORTER_WEB_LISTEN_ADDRESS
No :9186 Address to listen on for web interface and telemetry
web.telemetry-path
FIREHOSE_EXPORTER_WEB_TELEMETRY_PATH
No /metrics Path under which to expose Prometheus metrics
web.auth.username
FIREHOSE_EXPORTER_WEB_AUTH_USERNAME
No Username for web interface basic auth
web.auth.password
FIREHOSE_EXPORTER_WEB_AUTH_PASSWORD
No Password for web interface basic auth
web.tls.cert_file
FIREHOSE_EXPORTER_WEB_TLS_CERTFILE
No Path to a file that contains the TLS certificate (PEM format). If the certificate is signed by a certificate authority, the file should be the concatenation of the server's certificate, any intermediates, and the CA's certificate
web.tls.key_file
FIREHOSE_EXPORTER_WEB_TLS_KEYFILE
No Path to a file that contains the TLS private key (PEM format)

Metrics

For a list of Cloud Foundry Firehose metrics check the Cloud Foundry Component Metrics documentation.

The exporter returns additionally the following internal metrics:

Metric Description Labels
metrics.namespace_total_envelopes_received Total number of envelopes received from Cloud Foundry Firehose environment
metrics.namespace_last_envelope_received_timestamp Number of seconds since 1970 since last envelope received from Cloud Foundry Firehose environment
metrics.namespace_total_metrics_received Total number of metrics received from Cloud Foundry Firehose environment
metrics.namespace_last_metric_received_timestamp Number of seconds since 1970 since last metric received from Cloud Foundry Firehose environment
metrics.namespace_total_container_metrics_received Total number of container metrics received from Cloud Foundry Firehose environment
metrics.namespace_total_container_metrics_processed Total number of container metrics processed from Cloud Foundry Firehose environment
metrics.namespace_container_metrics_cached Number of container metrics cached from Cloud Foundry Firehose environment
metrics.namespace_last_container_metric_received_timestamp Number of seconds since 1970 since last container metric received from Cloud Foundry Firehose environment
metrics.namespace_total_counter_events_received Total number of counter events received from Cloud Foundry Firehose environment
metrics.namespace_total_counter_events_processed Total number of counter events processed from Cloud Foundry Firehose environment
metrics.namespace_counter_events_cached Number of counter events cached from Cloud Foundry Firehose environment
metrics.namespace_last_counter_event_received_timestamp Number of seconds since 1970 since last counter event received from Cloud Foundry Firehose environment
metrics.namespace_total_http_start_stop_received Total number of http start stop received from Cloud Foundry Firehose environment
metrics.namespace_total_http_start_stop_processed Total number of http start stop processed from Cloud Foundry Firehose environment
metrics.namespace_http_start_stop_cached Number of http start stop cached from Cloud Foundry Firehose environment
metrics.namespace_last_http_start_stop_received_timestamp Number of seconds since 1970 since last http start stop received from Cloud Foundry Firehose environment
metrics.namespace_total_value_metrics_received Total number of value metrics received from Cloud Foundry Firehose environment
metrics.namespace_total_value_metrics_processed Total number of value metrics processed from Cloud Foundry Firehose environment
metrics.namespace_value_metrics_cached Number of value metrics cached from Cloud Foundry Firehose environment
metrics.namespace_last_value_metric_received_timestamp Number of seconds since 1970 since last value metric received from Cloud Foundry Firehose environment
metrics.namespace_slow_consumer_alert Nozzle could not keep up with Cloud Foundry Firehose environment
metrics.namespace_last_slow_consumer_alert_timestamp Number of seconds since 1970 since last slow consumer alert received from Cloud Foundry Firehose environment

Contributing

Refer to CONTRIBUTING.md.

License

Apache License 2.0, see LICENSE.