The main purpose of this library is to provide some wrapper code around the
Dropwizard Metrics Java Library,
but there is some other functionality provided as well. Here are the major features
available in trapperkeeper-metrics
:
- A
Trapperkeeper
service that handles life cycle management for objects from the Dropwizard Metrics library - Support for reading basic metrics configuration info from a
Trapperkeeper
config file, so that the metrics configuration syntax is consistent across all of yourTrapperkeeper
apps - Other utility functions for creating and interacting with metrics in your application
For more detail on these features, read on.
The main entry point into the Dropwizard Metrics API is a class called
MetricRegistry
.
This class requires some basic initialization, so trapperkeeper-metrics
provides a Trapperkeeper service (MetricsService
) that manages the life cycle
of a MetricRegistry
. The service includes a function, get-metrics-registry
,
so that all of your other Trapperkeeper services can access the registry and
register new metrics with it.
For example:
(defservice my-service
[[:MetricsService get-metrics-registry]]
(init [this context]
(let [metrics-registry (get-metrics-registry)
my-metric-name (metrics/host-metric-name "localhost" "my-metric")
my-timer (.timer metrics-registry my-metric-name)]
(metrics/time! my-timer (do-some-work)))
context))
See the source code for the sample app
for a working example. See the utility functions in the trapperkeeper-metrics
puppetlabs.metrics
namespace for some helpers
for constructing other kinds of metrics besides just Timer
. See the
Dropwizard Metrics docs for more
info about all of the available metrics types and their features.
The get-metrics-registry
also allows you to specify two additional fields,
registry-key
and domain
which allow you to create other registries (besides
the default given by (get-metrics-registry)
and allow you to configure the
namespace of the reporter for that registry.
(defservice my-service
[[:MetricsService get-metrics-registry]]
(init [this context]
(let [default-metrics-registry (get-metrics-registry)
my-metrics-registry (get-metrics-registry "my.metrics.domain")
;; This will create the metric
;; `my.metrics.domain:name=puppetlabs.localhost.my-metric`
my-metric-name (metrics/host-metric-name "localhost" "my-metric")
my-timer (.timer metrics-registry my-metric-name)]
(assert (not= default-metrics-registry my-metrics-registry))
(metrics/time! my-timer (do-some-work)))
context)
(start [this context]
;; We can retrieve the same metrics-registry later.
(let [my-metrics-registry (get-metrics-registry "my.metrics.domain")]
(do-some-other-work my-metrics-registry))
context))
For more specific details on the service API, see the Metrics Service
API docs.
The MetricsService
also provides a configuration syntax that users can use to
configure the metrics for a running TK app. This means that all TK services can
provide a consistent interface for interacting with metrics.
For more specific details, see the
MetricsService
configuration docs.
The puppetlabs.metrics
namespace contains some
utility functions for working with metrics. See the source code and docstrings
for more detail. Here are a few bits of basic info:
time!
is a macro that can be used to time some Clojure forms against an existing
Timer
. e.g.:
(let [my-timer (.timer (get-metrics-registry) "my.metric.name")]
(time! my-timer
(do-some-work!)
(do-some-more-work!)))
host-metric-name
can be used to provide a qualified, namespaced metric name.
For best results, it is advisable to use this function to create a name for each
of your application's metrics; this will ensure that the metrics are namespaced
consistently across services. It will also ensure that metrics are namespaced
by hostname, which is critical when consolidating metrics data from multiple
hosts/services. You can use the
server-id
value from the MetricsService
configuration to get the appropriate
hostname for your metric.
(TODO: this part of the API needs to be fleshed out a bit further. We might want to have a more dynamic way to get the hostname/server-id rather than having to put it into the config file. We may want to tweak some other things as well. In the interim, it's probably a good idea to make it clear in your application/service documentation that the specific metric namespaces should not be considered part of a formal API and may change in subsequent releases.)
register
can be used to add a metric to an existing MetricRegistry
.
ratio
, metered-ratio
, and gauge
can be used to construct other types of
Metrics.
To enable the HTTP API for accessing individual metrics add the service to your
bootstrap.cfg
and configure the web-router-service
accordingly:
web-router-service {
"puppetlabs.trapperkeeper.services.metrics.metrics-service/metrics-webservice" : "/metrics"
}
To get a list of all available metric names:
- Request
/metrics/v1/mbeans
. - Use a
GET
request.
Responses return a JSON object mapping a string to a string:
- The key is the name of a valid MBean.
- The value is a URI to use for requesting that MBean's attributes.
To get a the attributes for multiple metrics at the same time:
- Request
/metrics/v1/mbeans
. - Use a
POST
request. - Use a request body which is either a JSON object whose values are metric names, JSON array of metric names or a JSON string of a metric name.
The response format, though always JSON, depends on the request format:
- Requests with a JSON object will return the a JSON object where the values of the original object have been transformed into the Mbeans' attributes for the metric names.
- Requests with a JSON array will return the a JSON array where the items of the original array have been transformed into the Mbeans' attributes for the metric names.
- Requests with a JSON string will return the a JSON object of the Mbean's attributes for the given metric name.
To get the attributes of a particular metric:
- Request
/metrics/v1/mbeans/<name>
, where<name>
is something that was returned in the list of available metrics specified above. - Use a
GET
request.
Responses return a JSON object mapping strings to (strings/numbers/Booleans).
For example, using curl
from localhost:
curl 'http://localhost:8080/metrics/v1/mbeans/java.lang:type=Memory'
{
"ObjectPendingFinalizationCount" : 0,
"HeapMemoryUsage" : {
"committed" : 807403520,
"init" : 268435456,
"max" : 3817865216,
"used" : 129257096
},
"NonHeapMemoryUsage" : {
"committed" : 85590016,
"init" : 24576000,
"max" : 184549376,
"used" : 85364904
},
"Verbose" : false,
"ObjectName" : "java.lang:type=Memory"
}
Since we support sending the metrics data to JMX, there are several existing tools and approaches that can be used to read the data for individual metrics via JMX. JVisualVM is one example; see pe-puppetserver-jruby-jmx-client for an example of how to do this from a JRuby script.
For best results, use this library in combination with the
comidi
library,
and then take advantage of the wrap-with-request-metrics
Ring middleware
to track metrics about all HTTP requests made to your application. See
the comidi
docs for more info.
Some ideas for things we might want to add/change in the future:
There is an existing clojure library that wraps Dropwizard Metrics:
metrics-clojure
. At the time
when we originally wrote our metrics code, this library was very out-of-date
and didn't support some of the features we needed. It also seemed like a pretty
thin facade around the Java library, and we decided that it wasn't worth
adding an extra dependency.
Since then, it's been updated and should be much more compatible with the
more recent versions of Dropwizard Metrics. At a glance, it seems like it
would be possible for other TK services to use metrics-clojure
in
combination with trapperkeeper-metrics
as-is; it mostly just provides utility
functions that should work fine with the MetricRegistry
object surfaced
by trapperkeeper-metrics
. So, if you feel like the abstractions it
provides over the Java library are worthwhile, try it out and let us know if
something doesn't work properly.
At some point in the future we may go ahead and add it as a direct dependency
and refactor things in trapperkeeper-metrics
to use it, but so far there
hasn't been a hugely compelling reason to do so.
There are a few other ideas floating around for how to make it easier for
apps/services that are using trapperkeeper-metrics
to expose the metrics info
to end users. More utility functions for easier integration with status service,
other tools for facilitating consumption / visualization / etc. Stay tuned.