Skip to content

simpletun/performance-library

BranchesTags

Repository files navigation

cluster-load-runner

A comprehensive Node.js library for building distributed HTTP performance testing frameworks using the cluster module. This library provides the core infrastructure for creating multi-threaded, coordinated load tests with support for data providers, custom workers, and multiple output formats.

What is this for?

The cluster-load-runner provides a master-worker architecture for performance testing. It handles:

  • Process Management: Spawns and manages worker processes using Node.js cluster module
  • Inter-Process Communication: Coordinates message passing between master and worker processes
  • Data Providers: Built-in providers for file and MySQL data sources that feed test data to workers
  • HTTP Request Utilities: Simplified HTTP request handling with timing and result reporting
  • Result Collection: Aggregates performance metrics from all workers and outputs results in multiple formats (CSV, JSON, NewRelic, InfluxDB, OTEL, stdout)
  • Ramp-up Strategies: Gradually increase load over time with configurable thread scheduling
  • Utility Functions: Random data generation, caching, mathematical operations, and more

Architecture Overview

The library uses a master-worker pattern:

  • Master Process (master.js): Spawns workers, coordinates data flow between workers, collects and analyzes results
  • Worker Processes (worker.js): Execute the actual load test scenarios, make HTTP requests, report results
  • Providers: Special workers that supply data to regular workers (e.g., reading lines from CSV files)
  • Custom Worker Types: Your application-specific test logic that imports this library

Getting Started

Option 1: Clone the Performance Framework (Recommended)

The easiest way to get started is to clone the performance-framework repository as a starting point. It already has the correct structure, example scenarios, and worker implementations you can use as templates.

# Clone the performance-framework
git clone git@github.com:simpletun/performance-framework.git my-performance-tests
cd my-performance-tests

# Install dependencies
npm install

# Review the example scenarios
ls scenarios/

# Review the example workers
ls src/workers/

# Run an example scenario
npm start perf-test-example

From there, you can:

  1. Modify the existing scenario files or create new ones in scenarios/
  2. Modify the existing workers or create new ones in src/workers/
  3. Update package.json with your project details
  4. Customize for your specific testing needs

Option 2: Start from Scratch

If you prefer to build from scratch, here's the basic structure you'll need:

Quick Start Example

The performance-framework is a reference implementation that shows how to use this library. Here's the basic structure:

1. Entry Point (src/start.js)

import cluster from 'cluster';

if (cluster.isPrimary) {
    await import('cluster-load-runner/master');
} else {
    await import('cluster-load-runner/worker');
}

This simple entry point determines whether the process is the primary or a worker and loads the appropriate module from cluster-load-runner.

2. Project Structure

your-performance-project/
├── src/
│   ├── start.js                    # Entry point
│   └── workers/
│       ├── your-worker-type.js     # Custom worker implementations
│       └── another-worker.js
├── scenarios/
│   ├── your-scenario.js            # Test scenario configurations
│   └── another-scenario.js
└── package.json

Defining Scenario Documents

Scenarios are JavaScript files that default-export configuration for your performance test. They define what workers to run, how many threads to use, and test parameters.

Basic Scenario Structure

Create a file in your scenarios/ directory (e.g., scenarios/my-test.js):

const second = 1000;
const minute = 60 * second;

// Global scenario configuration
const scenario = {
    duration: 5 * minute  // Test runs for 5 minutes
};

const server = {
    ssl: true,
    hostname: 'api.example.com',
    headers: {
        'Content-Type': 'application/json'
    }
};

export default {
    // Providers (optional) - workers that supply data to other workers
    providers: [
        {
            workerType: 'file-data-provider',     // Built-in provider from cluster-load-runner
            workerGroup: 'dataReader',            // Name used by workers to request data
            threads: 1,                           // Usually 1 thread for providers
            fileName: 'test-data.csv',            // File to read from
            recycleOnEof: true,                   // Loop back to start when file ends
            chunkSize: Infinity,
            bufferSize: 512 * 1024
        }
    ],

    // Workers - your custom test logic
    workers: [
        {
            workerType: 'my-custom-worker',       // Name of your worker file (src/workers/my-custom-worker.js)
            threads: 10,                          // Number of parallel workers
            subThreads: 5,                        // Each worker runs 5 concurrent loops
            thinkFrom: 200,                       // Minimum delay between requests (ms)
            thinkTo: 500,                         // Maximum delay between requests (ms)
            server,                               // Server configuration
            scenario                              // Scenario configuration
        }
    ]
};

You can also use ramp-up for gradual load increase:

import { evenRampUp } from 'cluster-load-runner';

const minute = 60 * 1000;
const server = { ssl: true, hostname: 'api.example.com' };
const scenario = { duration: 10 * minute };

export default {
    providers: [],
    workers: [
        {
            workerType: 'my-custom-worker',
            threads: evenRampUp(50, 2 * minute),  // Ramp from 0 to 50 threads over 2 minutes
            subThreads: 3,
            server,
            scenario
        }
    ]
};

Scenario Configuration Options

Providers Configuration

Option Type Description
workerType string Type of provider: 'file-data-provider' or 'mysql-data-provider'
workerGroup string Unique name workers use to request data from this provider
threads number Number of provider instances (usually 1)
fileName string (File provider) CSV file to read from
recycleOnEof boolean (File provider) Loop back to start when reaching end of file
chunkSize number (File provider) Read chunk size
bufferSize number (File provider) Buffer size for reading

Workers Configuration

Option Type Description
workerType string Name of your worker file (without .js extension)
threads number/array Number of workers, or ramp-up array from evenRampUp()
subThreads number How many concurrent loops each worker runs
thinkFrom number Minimum delay between requests (milliseconds)
thinkTo number Maximum delay between requests (milliseconds)
server object Server connection details (hostname, ssl, headers)
scenario object Reference to scenario configuration (duration, etc.)
workerGroup string (Optional) Group name for workers that need to coordinate
(custom) any Any custom configuration your worker needs

Creating a New Worker Type

Workers are the heart of your performance test. They define what requests to make and how to make them.

Step 1: Create Worker File

Create a new file in src/workers/ directory. The filename (without extension) becomes your workerType.

Example: src/workers/api-test.js

import {
    config,           // Configuration from scenario
    shutdown,         // Function to stop worker
    onMessage,        // Listen for messages from master
    makeRequest,      // Make HTTP requests with timing
    sleep,            // Sleep utility
    randomNumberFrom, // Random number generator
    logger,           // Logging utility
    FileReadMessenger // Request data from file provider
} from 'cluster-load-runner';

// If using a data provider, create a messenger
const dataMessenger = new FileReadMessenger({
    workerGroup: 'dataReader'  // Must match provider's workerGroup in scenario
});

// Handle stop message from master
onMessage('stop', () => {
    shutdown();
});

// Handle start message - begins the test
onMessage('start', async () => {
    logger.info(`Starting test for ${config.scenario.duration / 1000}s`);

    // Start multiple concurrent loops (subThreads)
    for (let i = 0; i < config.subThreads; i++) {
        startSubThread();
    }
});

// Each subthread runs independently
const startSubThread = async () => {
    // Loop until master sends 'stop' message
    while (true) {
        try {
            await performTest();
        } catch (error) {
            logger.error(`Test error: ${error.message}`);
        }

        // Random "think time" between requests
        await sleep(randomNumberFrom(config.thinkFrom, config.thinkTo));
    }
};

// Your actual test logic
const performTest = async () => {
    // Get test data from provider (if using one)
    const testData = await dataMessenger.getLine(config.randomLine);

    // Make HTTP request - automatically times and reports results to master
    await makeRequest({
        transactionName: 'API Test Request',  // Shows up in results
        requestConfig: {
            path: `/api/endpoint/${testData}`,
            method: 'GET'
        }
    });
};

Step 2: Use in Scenario

Reference your worker in a scenario file:

export default {
    providers: [],
    workers: [
        {
            workerType: 'api-test',  // Matches filename: src/workers/api-test.js
            threads: 10,
            subThreads: 5,
            thinkFrom: 200,
            thinkTo: 500,
            randomLine: true,
            server: {
                ssl: true,
                hostname: 'api.example.com'
            },
            scenario: {
                duration: 5 * 60 * 1000
            }
        }
    ]
};

Worker Best Practices

  1. Always handle 'stop' and 'start' messages: These control your worker's lifecycle
  2. Use subThreads for concurrency: Each worker can run multiple concurrent test loops
  3. Add think time: Use sleep() between requests to simulate realistic user behavior
  4. Error handling: Wrap test logic in try-catch to prevent worker crashes
  5. Use makeRequest(): This utility automatically times requests and reports results to the master
  6. Log appropriately: Use logger.debug(), logger.info(), logger.error() for different verbosity levels

Available Utilities from cluster-load-runner

The library exports many utilities for building workers:

// Worker lifecycle
config          // Your scenario configuration
shutdown()      // Stop this worker
onMessage()     // Listen for messages from master
sendMessage()   // Send messages to master

// HTTP utilities
makeRequest()   // Make HTTP request with automatic timing and reporting
request()       // Lower-level HTTP request

// Data providers
FileReadMessenger    // Request data from file-data-provider
MysqlQueryMessenger  // Request data from mysql-data-provider

// Utilities
sleep()                  // Async sleep
randomNumberFrom()       // Random number in range
randomInt()              // Random integer
randomItem()             // Pick random item from array
randomItems()            // Pick multiple random items from array
coinFlip()               // Random boolean
logger                   // Winston logger instance

// Math utilities
mean()
variance()
populationStandardDeviation()
sampleStandardDeviation()
combinedAverage()
combinedSampleStandardDeviation()
round()

// Ramp-up
evenRampUp()     // Generate a ramp-up schedule for threads

// Caching
cache                // Cache utility
cachedFunction()     // Memoization wrapper

Running Tests

After setting up scenarios and workers in your consuming project:

# Run a specific scenario
npm start your-scenario

# The master process will:
# 1. Load the scenario configuration
# 2. Spawn provider workers
# 3. Spawn test workers
# 4. Coordinate data flow
# 5. Collect and report results
# 6. Output results to CSV/JSON/stdout

Advanced: Inter-Worker Communication

Workers can communicate with each other through the master process:

Broadcast to Worker Group

// In scenario, assign workers to a group
export default {
    providers: [],
    workers: [
        {
            workerType: 'receiver-worker',
            workerGroup: 'receivers',  // Group name
            threads: 5
        }
    ]
};

// In another worker, broadcast to the group
sendMessage('broadcast', {
    workerGroup: 'receivers',
    messages: [
        { type: 'custom-message', data: 'Hello all receivers!' }
    ]
});

Round-Robin to Worker Group

// Send messages in round-robin fashion to group
sendMessage('roundrobin', {
    workerGroup: 'receivers',
    messages: [
        { type: 'task', id: 1 },
        { type: 'task', id: 2 },
        { type: 'task', id: 3 }
    ]
});

Direct Message to Specific Worker

// Send to specific worker by PID
sendMessage('direct', {
    to: targetPid,
    message: { type: 'custom', data: 'Hello specific worker!' }
});

Output Formats

Results can be output in multiple formats, selected via the --output run mode flag:

  • CSV (output:csv, default): Detailed per-request results in CSV format
  • JSON (output:json): Results in JSON format
  • NewRelic (output:newrelic): Send metrics to NewRelic APM
  • InfluxDB (output:influxdb): Send metrics to an InfluxDB instance
  • OTEL (output:otel): Export metrics via OpenTelemetry (OTLP/HTTP) — compatible with Grafana Alloy, OpenTelemetry Collector, and any OTLP-compatible backend
  • Stdout (output:stdout): Print results to console

OTEL Output

The OTEL output type exports all request metrics as standard OpenTelemetry metrics using OTLP/HTTP, making it compatible with Grafana Alloy, Grafana Mimir, Prometheus, and any other OTLP-capable backend.

Usage:

npm start my-scenario output:otel

Configuration (constructor options or environment variables):

Option Env var Default
endpoint OTEL_EXPORTER_OTLP_ENDPOINT http://localhost:4318
headers OTEL_EXPORTER_OTLP_HEADERS (key=val,key=val) {}
serviceName OTEL_SERVICE_NAME project directory name
exportIntervalMs — 10000
runId — auto-generated timestamp + random suffix

Metrics exported:

Metric name Type Unit Description
performance.request.duration Histogram ms Total request duration
performance.request.latency Histogram ms Time to first byte (TTFB)
performance.request.connect_time Histogram ms TCP connection time
performance.request.bytes_received Histogram bytes Response body size
performance.request.bytes_sent Histogram bytes Request body size
performance.requests Counter requests Total request count

Each metric carries these attributes: request.name, http.response.status_code, http.response.status_message, request.success, request.error (if present), worker.type, process.pid, worker.thread_count. Run-level dimensions (run.id, scenario.name, project.name, service.name) are attached as OTEL Resource attributes, which are sent once per export batch rather than on every data point.

Minimal Grafana Alloy config to receive from this library:

otelcol.receiver.otlp "default" {
  http { endpoint = "0.0.0.0:4318" }

  output {
    metrics = [otelcol.exporter.prometheus.default.input]
  }
}

otelcol.exporter.prometheus "default" {
  // Required: promotes run.id, scenario.name, etc. from Resource attributes
  // to per-metric labels so they can be used as dashboard filter variables.
  resource_to_telemetry_conversion = true
  forward_to = [prometheus.remote_write.local.receiver]
}

prometheus.remote_write "local" {
  endpoint {
    url = "http://localhost:9090/api/v1/write"
  }
}

OTEL Tracing

To additionally emit per-request trace spans (separate from metrics), use the otel-traces run mode flag:

npm start my-scenario output:otel,otel-traces

This requires a TracerProvider to be configured externally in the consuming application. The legacy signalfx flag is still supported as an alias.

The master process automatically calculates:

  • Average response time
  • Min/Max response times
  • 95th percentile statistics
  • Success/error counts
  • Total request count

Developing cluster-load-runner

This project uses native ES modules ("type": "module") — there is no build step. The src/ directory is published directly to npm.

Testing

npm test           # Run tests
npm run coverage   # Generate coverage report
npm run lint       # Run linter

Project Structure

cluster-load-runner/
├── src/
│   ├── index.js           # Main exports
│   ├── master.js          # Master process implementation
│   ├── worker.js          # Worker process base implementation
│   ├── providers/         # Built-in data providers
│   │   ├── file-data-provider.js
│   │   └── mysql-data-provider.js
│   ├── outputs/           # Output formatters
│   │   ├── csv.js
│   │   ├── influxdb.js
│   │   ├── json.js
│   │   ├── newrelic.js
│   │   ├── otel.js
│   │   └── stdout.js
│   └── utils/             # Utility functions
│       ├── logger.js
│       ├── makeRequest.js
│       ├── fileReadMessenger.js
│       ├── rampup.js
│       ├── random.js
│       └── ...
└── types.d.ts             # TypeScript type definitions

The src/index.js file defines the library's public API. All exports from this file are available to consumers of the library.

About

Library for running performance tests

Resources

Readme

License

MIT license

Contributing

Contributing
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

2 tags

Packages

 
 
 

Contributors

Languages