firehose: the metrics gateway for prometheus
Switch branches/tags
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.
firehose-st Configuration with text/plain and extractor script tested Nov 25, 2017
firehose Script configuration simplified Nov 25, 2017
prometheus-firehose compose relates to image, not build Apr 8, 2017
.gitignore Initial commit Dec 8, 2016
LICENSE Update Dec 3, 2017
docker-compose.yml compose relates to image, not build Apr 8, 2017
metrics.env custom application name added Apr 8, 2017


Java EE 7+ metrics gateway for prometheus

The firehose service (16kB Thin WAR) gathers metrics emitted from JSON / text (currently only JSON is supported) HTTP endpoint and converts them into prometheus compliant metrics.

The prometheus metrics are exposed via the: http://[HOST]:[8080]/firehose/resources/metrics/{metrics-name} uri in prometheus format.

Metrics from all registered endpoints can be fetched at once via HTTP http://[HOST]:[8080]/firehose/resources/metrics/


Start firehose in the same docker network as the monitored resources and start it with docker run -d -p 8080:8080 --name firehose airhacks/firehose or deploy the WAR to a Java EE 7 container running on Java 8.

A sample configuration is used for testing. Prometheus, a sample-service and firehose are started:

version: '3.0'
    build: prometheus-firehose
    - "9090:9090"
    - firehose
    image: airhacks/firehose
    env_file: metrics.env
    - "8080:8080"
    - sample-service
    image: airhacks/sample-service
    - "8282:8080"

sample-service is configured in an metrics.env file and provided at starttime:

# Sample service configuration
# prometheus does not allow dashes -> name changed to sampleservice

Configuration management

To gather a remote metric at least the uri with the fully qualified value, e.g.: http://sample-service:8080/sample-service/resources/metrics is required.

The properties (required by prometheus):

  1. application (e.g. "sampleservice"
  2. component (e.g. "MetricsResource"")
  3. units (e.g. "requests")
  4. suffix (e.g. "total")

are usually provided by the monitored endpoint as json, but can be overriden with environment entries, like e.g.


at startup time.

The properties can be also passed as JSON via configuration, e.g.

curl -i -H"Content-type: application/json" -XPUT -d'{"uri":"http://sample-service:8080/sample-service/resources/metrics","application":"sample", "component":"jobprocessing","units":"jobs","suffix":"total}' http://localhost:8080/firehose/resources/configurations/sample-service

Also an extractor script can be configured to extract the data from an incompatible data source. The following script uploads a JavaScript file, which extracts the used heap in bytes from the GlassFish / Payara monitoring endpoint:

curl -i -XPUT -H"Content-type: text/plain" -H"uri: http://localhost:4848/monitoring/domain/server/jvm/memory/usedheapsize-count.json" --upload-file usedheapsize.js http://localhost:8080/firehose/resources/configurations/payarausedheap

The content of the file is:

function apply(input) {
    var metric = JSON.parse(input);
    var value = metric.extraProperties.entity["usedheapsize-count"].count;
    var output = {
        suffix: "size",
        units: "bytes",
        component: "jvmusedheapsize",
        application: "firehosest",
        value: value
    return JSON.stringify(output);

(see firehose-st/src/test/resources)

The following request: curl http://localhost:8080/firehose/resources/metrics/payarausedheap

will return a prometheus-consumable metric:

firehosest_jvmusedheapsize_bytes_size 150629368

The environment variables can be overriden again by the REST configuration endpoint:

  1. create or update a metric

Setup "sample-service" metric to be fetched from http://sample-service:8080/sample-service/resources/metrics and exposed via http://[HOST]:[8080]/firehose/resources/metrics/sample-service:

curl -i -H"Content-type: application/json" -XPUT -d'{"uri":"http://sample-service:8080/sample-service/resources/metrics"}' http://localhost:8080/firehose/resources/configurations/sample-service

This endpoint return 201 for creation and 204 for update.

  1. list all metrics

curl -i http://localhost:8080/firehose/resources/configurations/

  1. delete a metric

delete a metric with the name "sample-service"

curl -XDELETE http://localhost:8080/firehose/resources/configurations/sample-service

Prometheus endpoint

The preconfigured endpoint are available under the URI: http://[HOST]:[8080]/firehose/resources/metrics/{metrics-name}

test it with: e.g. curl -i http://localhost:8080/firehose/resources/metrics/sample-service

You can also fetch all metrics via a single URI: curl http://localhost:8080/firehose/resources/metrics/, what does not require any changes in prometheus configuration in case an additional endpoint is added.


  1. git clone
  2. cd firehose && mvn clean install && docker build -t airhacks/firehose .
  3. cd sample-service && mvn clean install && docker build -t airhacks/sample-service .
  4. docker-compose up -d

Base image for firehose and sample apps were taken from docklands


Java EE Monitoring With firehose and

Java EE Monitoring With firehose and

Monitoring Payara with Firehose, Prometheus and Grafana:

Monitoring Payara with Firehose, Prometheus and Grafana

See you at Java EE Performance, Monitoring and Troubleshooting and/or Java EE Microservices workshops