feat: Support for metrics custom labels

CHANGELOG.md

@@ -2,6 +2,10 @@
 
 ## Master
 
+### Features
+
+- Add support for custom labels addition to metrics
+
 ### Breaking Changes
 
 - Add support for `prom-client v13`, which includes a few breaking changes, mainly the following functions are now async (return a promise):

README.md

@@ -38,16 +38,16 @@ API and process monitoring with [Prometheus](https://prometheus.io) for Node.js
 ## Features
 
 - [Collect API metrics for each call](#usage)
-   - Response time in seconds
-   - Request size in bytes
-   - Response size in bytes
-   - Add prefix to metrics names - custom or project name
-   - Exclude specific routes from being collect
-   - Number of open connections to the server
+  - Response time in seconds
+  - Request size in bytes
+  - Response size in bytes
+  - Add prefix to metrics names - custom or project name
+  - Exclude specific routes from being collect
+  - Number of open connections to the server
 - Process Metrics as recommended by Prometheus [itself](https://prometheus.io/docs/instrumenting/writing_clientlibs/#standard-and-runtime-collectors)
 - Endpoint to retrieve the metrics - used for Prometheus scraping
-   - Prometheus format
-   - JSON format (`${path}.json`)
+  - Prometheus format
+  - JSON format (`${path}.json`)
 - Support custom metrics
 - [Http function to collect request.js HTTP request duration](#requestjs-http-request-duration-collector)
 
@@ -60,24 +60,30 @@ app.use(apiMetrics())
 
 ### Options
 
-- metricsPath - Path to access the metrics. `default: /metrics`
-- defaultMetricsInterval - the interval to collect the process metrics in milliseconds. `default: 10000`
-- durationBuckets - Buckets for response time in seconds. `default: [0.001, 0.005, 0.015, 0.05, 0.1, 0.2, 0.3, 0.4, 0.5]`
-- requestSizeBuckets - Buckets for request size in bytes. `default: [5, 10, 25, 50, 100, 250, 500, 1000, 2500, 5000, 10000]`
-- responseSizeBuckets - Buckets for response size in bytes. `default: [5, 10, 25, 50, 100, 250, 500, 1000, 2500, 5000, 10000]`
-- useUniqueHistogramName - Add to metrics names the project name as a prefix (from package.json)
-- metricsPrefix - A custom metrics names prefix, the package will add underscore between your prefix to the metric name.
-- excludeRoutes - Array of routes to exclude. Routes should be in your framework syntax.
-- includeQueryParams - A boolean that indicate if to include query params in route, the query parameters will be sorted in order to eliminate the number of unique labels.
+| Option                   | Type      | Description | Default Value |
+|--------------------------|-----------|-------------|---------------|
+| `metricsPath`            | `String`  | Path to access the metrics | `/metrics` |
+| `defaultMetricsInterval` | `Number`  | Interval to collect the process metrics in milliseconds | `10000` |
+| `durationBuckets`        | `Array<Number>` | Buckets for response time in seconds | `[0.001, 0.005, 0.015, 0.05, 0.1, 0.2, 0.3, 0.4, 0.5]` |
+| `requestSizeBuckets`     | `Array<Number>` | Buckets for request size in bytes | `[5, 10, 25, 50, 100, 250, 500, 1000, 2500, 5000, 10000]` |
+| `responseSizeBuckets`    | `Array<Number>` | Buckets for response size in bytes | `[5, 10, 25, 50, 100, 250, 500, 1000, 2500, 5000, 10000]` |
+| `useUniqueHistogramName` | `Boolean` | Add to metrics names the project name as a prefix (from package.json) | `false` |
+| `metricsPrefix`          | `String`  | A custom metrics names prefix, the package will add underscore between your prefix to the metric name | |
+| `excludeRoutes`          | `Array<String>` | Array of routes to exclude. Routes should be in your framework syntax | |
+| `includeQueryParams`     | `Boolean` | Indicate if to include query params in route, the query parameters will be sorted in order to eliminate the number of unique labels | `false` |
+| `additionalLabels`       | `Array<String>` | Indicating custom labels that can be included on each `http_*` metric. Use in conjunction with `extractAdditionalLabelValuesFn`. |
+| `extractAdditionalLabelValuesFn` | `Function` | A function that can be use to generate the value of custom labels for each of the `http_*` metrics. When using koa, the function takes `ctx`, when using express, it takes `req, res` as arguments | |
 
 ### Access the metrics
 
 To get the metrics in Prometheus format use:
+
 ```sh
 curl http[s]://<host>:[port]/metrics
 ```
 
 To get the metrics in JSON format use:
+
 ```sh
 curl http[s]://<host>:[port]/metrics.json
 ```
@@ -88,18 +94,19 @@ curl http[s]://<host>:[port]/metrics.json
 
 2. If you are using express framework and no route was found for the request (e.g: 404 status code), the request will not be collected. that's because we'll risk memory leak since the route is not a pattern but a hardcoded string.
 
-
 ## Custom Metrics
 
 You can expand the API metrics with more metrics that you would like to expose.
-All you have to do is: 
+All you have to do is:
 
 Require prometheus client
+
 ```js
 const Prometheus = require('prom-client');
 ```
 
 Create new metric from the kind that you like
+
 ```js
 const checkoutsTotal = new Prometheus.Counter({
   name: 'checkouts_total',
@@ -109,6 +116,7 @@ const checkoutsTotal = new Prometheus.Counter({
 ```
 
 Update it:
+
 ```js
 checkoutsTotal.inc({
   payment_method: paymentMethod
@@ -120,45 +128,73 @@ The custom metrics will be exposed under the same endpoint as the API metrics.
 For more info about the Node.js Prometheus client you can read [here](https://github.com/siimon/prom-client#prometheus-client-for-nodejs--)
 
 ### Note
+
 This will work only if you use the default Prometheus registry - do not use `new Prometheus.Registry()`
 
+## Additional Metric Labels
+
+You can define additional metric labels by using `additionalLabels` and `extractAdditionalLabelValuesFn` options.
+
+For instance:
+
+```js
+const apiMetrics = require('prometheus-api-metrics');
+app.use(apiMetrics({
+  additionalLabels: ['customer', 'cluster'],
+  extractAdditionalLabelValuesFn: (req, res) => {
+      const { headers } = req.headers;
+      return {
+        customer: headers['x-custom-header-customer'],
+        cluster: headers['x-custom-header-cluster']
+      }
+  }
+}))
+```
+
 ## Request.js HTTP request duration collector
+
 This feature enables you to easily process the result of Request.js timings feature.
 
 ### Usage
+
 #### Initialize
+
 You can choose to initialized this functionality as a Class or not
 
 **Class:**
+
 ```js
 const HttpMetricsCollector = require('prometheus-api-metrics').HttpMetricsCollector;
 const collector = new HttpMetricsCollector();
 collector.init();
 ```
 
 **Singelton:**
+
 ```js
 const HttpMetricsCollector = require('prometheus-api-metrics').HttpMetricsCollector;
 HttpMetricsCollector.init();
 ```
 
 #### Options
+
 - durationBuckets - the histogram buckets for request duration.
 - countClientErrors - Boolean that indicates whether to collect client errors as Counter, this counter will have target and error code labels.
 - useUniqueHistogramName - Add to metrics names the project name as a prefix (from package.json)
 - prefix - A custom metrics names prefix, the package will add underscore between your prefix to the metric name.
 
-
 For Example:
 
 #### request
+
 ```js
 request({ url: 'http://www.google.com', time: true }, (err, response) => {
     Collector.collect(err || response);
 });
 ```
 
 #### request-promise-native
+
 ```js
 return requestPromise({ method: 'POST', url: 'http://www.mocky.io/v2/5bd9984b2f00006d0006d1fd', route: 'v2/:id', time: true, resolveWithFullResponse: true }).then((response) => {
     Collector.collect(response);
@@ -167,17 +203,18 @@ return requestPromise({ method: 'POST', url: 'http://www.mocky.io/v2/5bd9984b2f0
 });
 ```
 
+**Notes:**
 
-**Notes:** 
 1. In order to use this feature you must use `{ time: true }` as part of your request configuration and then pass to the collector the response or error you got.
 2. In order to use the timing feature in request-promise/request-promise-native you must also use `resolveWithFullResponse: true`
 3. Override - you can override the `route` and `target` attribute instead of taking them from the request object. In order to do that you should set a `metrics` object on your request with those attribute:
-``` js
+```js
 request({ method: 'POST', url: 'http://www.mocky.io/v2/5bd9984b2f00006d0006d1fd', metrics: { target: 'www.google.com', route: 'v2/:id' }, time: true }, (err, response) => {...};
 });
 ```
 
 #### axios
+
 ```js
 const axios = require('axios');
 const axiosTime = require('axios-time');
@@ -192,10 +229,12 @@ try {
 }
 ```
 
-**Notes:** 
-* In order to collect metrics from axios client the [`axios-time`](https://www.npmjs.com/package/axios-time) package is required.
+**Notes:**
+
+- In order to collect metrics from axios client the [`axios-time`](https://www.npmjs.com/package/axios-time) package is required.
 
 ## Usage in koa
+
 This package supports koa server that uses [`koa-router`](https://www.npmjs.com/package/koa-router) and [`koa-bodyparser`](https://www.npmjs.com/package/koa-bodyparser)
 
 ```js
@@ -206,7 +245,7 @@ app.use(koaMiddleware())
 
 ## Test
 
-```
+```sh
 npm test
 ```
 
@@ -219,31 +258,37 @@ npm test
 ```
 
 ### 95th Response Time by specific route and status code
+
 ```
 histogram_quantile(0.95, sum(rate(http_request_duration_seconds_bucket{<SERVICE_LABLE_FIELD>="<SERVICE_LABEL>", route="<ROUTE_NAME>", code="200"}[10m])) by (le))
 ```
 
 ### Median Response Time Overall
+
 ```
 histogram_quantile(0.50, sum(rate(http_request_duration_seconds_bucket{<SERVICE_LABLE_FIELD>="<SERVICE_LABEL>"}[10m])) by (le))
 ```
 
 ### Median Request Size Overall
+
 ```
 histogram_quantile(0.50, sum(rate(http_request_size_bytes_bucket{<SERVICE_LABLE_FIELD>="<SERVICE_LABEL>"}[10m])) by (le))
 ```
 
 ### Median Response Size Overall
+
 ```
 histogram_quantile(0.50, sum(rate(http_response_size_bytes_bucket{<SERVICE_LABLE_FIELD>="<SERVICE_LABEL>"}[10m])) by (le))
 ```
 
 ### Avarage Memory Usage - All services
+
 ```
 avg(nodejs_external_memory_bytes / 1024 / 1024) by (<SERVICE_LABLE_FIELD)
 ```
 
 ### Avarage Eventloop Latency - All services
+
 ```
 avg(nodejs_eventloop_lag_seconds) by (<SERVICE_LABLE_FIELD)
 ```

src/express-middleware.js

@@ -37,9 +37,15 @@ class ExpressMiddleware {
         const route = this._getRoute(req);
 
         if (route && utils.shouldLogMetrics(this.setupOptions.excludeRoutes, route)) {
-            this.setupOptions.requestSizeHistogram.observe({ method: req.method, route: route, code: res.statusCode }, req.metrics.contentLength);
-            req.metrics.timer({ route: route, code: res.statusCode });
-            this.setupOptions.responseSizeHistogram.observe({ method: req.method, route: route, code: res.statusCode }, responseLength);
+            const labels = {
+                method: req.method,
+                route,
+                code: res.statusCode,
+                ...this.setupOptions.extractAdditionalLabelValuesFn(req, res)
+            };
+            this.setupOptions.requestSizeHistogram.observe(labels, req.metrics.contentLength);
+            req.metrics.timer(labels);
+            this.setupOptions.responseSizeHistogram.observe(labels, responseLength);
             debug(`metrics updated, request length: ${req.metrics.contentLength}, response length: ${responseLength}`);
         }
     }
@@ -104,9 +110,7 @@ class ExpressMiddleware {
         }
 
         req.metrics = {
-            timer: this.setupOptions.responseTimeHistogram.startTimer({
-                method: req.method
-            }),
+            timer: this.setupOptions.responseTimeHistogram.startTimer(),
             contentLength: parseInt(req.get('content-length')) || 0
         };
 

src/index.d.ts

@@ -1,5 +1,5 @@
-import { RequestHandler, Response } from 'express';
-import { Middleware } from 'koa';
+import { Request, RequestHandler, Response } from 'express';
+import { Context, Middleware } from 'koa';
 
 export default function middleware(options?: ApiMetricsOpts) : RequestHandler;
 export function koaMiddleware(options?: ApiMetricsOpts) : Middleware;
@@ -20,11 +20,13 @@ export interface ApiMetricsOpts {
   metricsPrefix?: string;
   excludeRoutes?:string[];
   includeQueryParams?: boolean;
+  additionalLabels?: string[];
+  extractAdditionalLabelValuesFn?: ((req: Request, res: Response) => Record<string, unknown>) | ((ctx: Context) => Record<string, unknown>)
 }
 
 export interface CollectorOpts {
   durationBuckets?: number[];
   countClientErrors?: boolean;
   useUniqueHistogramName?: boolean
   prefix?: string;
-}
+}

src/koa-middleware.js

@@ -39,17 +39,15 @@ class KoaMiddleware {
         const route = this._getRoute(ctx) || 'N/A';
 
         if (route && utils.shouldLogMetrics(this.setupOptions.excludeRoutes, route)) {
-            this.setupOptions.requestSizeHistogram.observe({
+            const labels = {
                 method: ctx.req.method,
-                route: route,
-                code: ctx.res.statusCode
-            }, ctx.req.metrics.contentLength);
-            ctx.req.metrics.timer({ route: route, code: ctx.res.statusCode });
-            this.setupOptions.responseSizeHistogram.observe({
-                method: ctx.req.method,
-                route: route,
-                code: ctx.res.statusCode
-            }, responseLength);
+                route,
+                code: ctx.res.statusCode,
+                ...this.setupOptions.extractAdditionalLabelValuesFn(ctx)
+            };
+            this.setupOptions.requestSizeHistogram.observe(labels, ctx.req.metrics.contentLength);
+            ctx.req.metrics.timer(labels);
+            this.setupOptions.responseSizeHistogram.observe(labels, responseLength);
             debug(`metrics updated, request length: ${ctx.req.metrics.contentLength}, response length: ${responseLength}`);
         }
     }
@@ -122,9 +120,7 @@ class KoaMiddleware {
         }
 
         ctx.req.metrics = {
-            timer: this.setupOptions.responseTimeHistogram.startTimer({
-                method: ctx.req.method
-            }),
+            timer: this.setupOptions.responseTimeHistogram.startTimer(),
             contentLength: parseInt(ctx.request.get('content-length')) || 0
         };