API Performance Monitoring Middleware for Node.js

✨ Overview

api-performance-monitoring is a lightweight and extensible Node.js middleware package designed to provide real-time API performance monitoring for your Express (and other compatible) applications. It enables developers to easily track request lifecycle, aggregate key metrics like latency and error rates, and export this data to various observability platforms.

Built with extensibility in mind, api-performance-monitoring aims to be a drop-in solution for understanding and improving your API's health and performance.

🚀 Features

• Request Lifecycle Tracking: Measures request duration, captures HTTP status codes, methods, and resolved route patterns for every incoming API request.
• Metrics Aggregation: Aggregates critical performance metrics:
  • request_count (total requests)
  • request_duration_ms (histograms for p50, p90, p95, p99 percentiles)
  • error_count (categorized by 4xx/5xx status classes)
  • in_flight_requests (current active requests)
  • apdex_score (derived from configurable latency thresholds)
  • Supports dimensions like service, env, method, route, status_class.
• Pluggable Exporters: Export your aggregated metrics to various targets:
  • Prometheus: Exposes metrics in Prometheus exposition format via a /metrics endpoint (requires prom-client for full features).
  • Webhook: Pushes batches of metrics (JSON) to a configurable HTTP endpoint with retry logic.
  • Extensible architecture to add custom exporters.
• Performance Overhead Rules: Designed for minimal overhead with non-blocking processing, bounded memory usage, and configurable sampling.
• Data Safety & Filtering (Planned): Configurable rules for redacting sensitive data from logs and payloads.

📦 Installation

To install the package, run:

npm install api-performance-monitoring
# If you plan to use the Prometheus exporter, also install prom-client
npm install prom-client --save

💡 Usage

Basic Express Application Integration

Integrate the middleware into your Express application:

const express = require('express');
const { expressAdapter } = require('api-performance-monitoring');

const app = express();
const port = process.env.PORT || 3000;

// Initialize the APM monitor and middleware
const { monitor, middleware, route } = expressAdapter({
  service: 'my-api-service',
  env: process.env.NODE_ENV || 'development',
  sampling: 1, // Sample all requests (0.0 - 1.0)
});

// Attach the APM middleware globally
app.use(middleware());

// Listen for raw request events (for custom processing)
monitor.on('request', (evt) => {
  console.log(`[APM Event] ${evt.method} ${evt.route} ${evt.status} ${evt.durationMs.toFixed(2)}ms`);
});

// Example routes
app.get('/', (_req, res) => {
  res.json({ message: 'Welcome to the API!' });
});

// Use the route helper for accurate route naming in metrics
app.get('/users/:id', route('/users/:id'), (req, res) => {
  res.json({ userId: req.params.id, name: 'John Doe' });
});

app.listen(port, () => {
  console.log(`API server listening on http://localhost:${port}`);
});

Metrics Endpoints

To expose metrics via the built-in aggregator's JSON or Prometheus text format:

const express = require('express');
const path = require('path');
const { expressAdapter, Aggregator } = require('api-performance-monitoring');

const app = express();
const port = process.env.PORT || 3000;

const { monitor, middleware, route } = expressAdapter({ service: 'demo-api' });
app.use(middleware());

const agg = new Aggregator({ apdexT: 200 });
monitor.on('request', (evt) => agg.handle(evt));

// JSON metrics endpoint
app.get('/metrics', agg.metricsEndpoint(monitor));

// Prometheus-style text exposition (without prom-client)
app.get('/metrics/prom', agg.promEndpoint(monitor));

app.listen(port, () => {
  console.log(`Demo server listening on http://localhost:${port}`);
});

Pluggable Exporters

Using the Pipeline and Prometheus Exporter with prom-client:

const express = require('express');
const path = require('path');
const {
  expressAdapter,
  Aggregator,
  Pipeline,
  PrometheusExporter,
} = require('api-performance-monitoring');

const app = express();
const port = process.env.PORT || 3000;

const { monitor, middleware } = expressAdapter({ service: 'demo-api' });
app.use(middleware());

const agg = new Aggregator({ apdexT: 200 });
monitor.on('request', (evt) => agg.handle(evt));

const pipeline = new Pipeline({ batchSize: 10, batchIntervalMs: 2000 });
pipeline.attachMonitor(monitor); // Connect monitor events to the pipeline

// Prometheus Exporter (uses prom-client for richer metrics)
const promExp = new PrometheusExporter(agg, { collectIntervalMs: 2000 });
pipeline.registerExporter(promExp);
app.get('/metrics/client', promExp.metricsEndpoint());

// Webhook Exporter (pushes metrics to an external endpoint)
if (process.env.WEBHOOK_URL) {
  const WebhookExporter = require('api-performance-monitoring').WebhookExporter;
  const wh = new WebhookExporter({ url: process.env.WEBHOOK_URL, retries: 2 });
  pipeline.registerExporter(wh);
}

app.listen(port, () => {
  console.log(`Demo server listening on http://localhost:${port}`);
});

Graceful Shutdown

Ensure all queued metrics are flushed on application shutdown:

process.on('SIGTERM', async () => {
  console.log('SIGTERM received, flushing metrics...');
  await monitor.flush(5000); // Wait up to 5 seconds to flush
  await monitor.shutdown(5000); // Shutdown exporters gracefully
  console.log('Metrics flushed, exiting.');
  process.exit(0);
});

⚙️ Configuration

(A detailed table of configurable options for Monitor, Aggregator, Pipeline, and Exporters will be added here, including defaults and explanations.)

📊 How it Works (Workflow Diagram)

Here's a high-level overview of the data flow:

graph TD
    A[Incoming HTTP Request] --> B{APM Middleware};
    B --> C{Start Timer & Collect Request Info};
    C --> D{Pass to Express Route Handler};
    D -- Response --> E{End Timer & Collect Response Info};
    E --> F[Emit 'request' Event];
    F --> G[Aggregator (Metrics Calculation & Storage)];
    G --> H[Pipeline (Batching & Dispatching)];
    H --> I(Prometheus Exporter);
    H --> J(Webhook Exporter);
    H --> K(Custom Exporter);
    G -- Snapshot --> I;

Loading

👨‍💻 Development

Running the Demo Server

To see api-performance-monitoring in action locally, start the demo Express server:

npm run dev

Access metrics at:

• JSON Metrics: http://localhost:3000/metrics
• Prometheus (Aggregator): http://localhost:3000/metrics/prom
• Prometheus (Prom-Client): http://localhost:3000/metrics/client

Running Tests

To ensure everything is working correctly:

npm test

🤝 Contributing

We welcome contributions! Please see our CONTRIBUTING.md for more details.

📄 License

This project is licensed under the ISC License. See the LICENSE file for details.