TooFast ⚡️

The Node.js performance testing tool with unit-test-like API.

• Runs each test in a separate process;
• Measures execution time using performance;
• Runs test functions in multiple batches to reduce garbage collection interference;
• Warms up test functions;
• Measures memory consumption using process.memoryUsage.

npm install toofast --save-dev

• Usage
• CLI options
• Test API
  • test
  • describe
  • Lifecycle hooks
• Test options
• Configuration

Usage

Let's write a performance test for a function that computes a factorial.

Create a file factorial.perf.js:

function factorial(x) {
  return x === 0 ? 1 : x * factorial(x - 1);
}

describe('factorial', () => {

  test('of 33', measure => {
    measure(() => {
      factorial(33);
    });
  });

  test('of 42', measure => {
    measure(() => {
      factorial(42);
    });
  });
});

Call toofast in the same directory with this file:

npx toofast

CLI options

toofast [options] ...files

...files

The list of glob patterns of included test files. If config file was not found, then files that match **/*.perf.js are included.

-c <file>, --config <file>

The configuration file path.

-t <pattern>, --testNamePattern <pattern>

The name glob pattern of describe and test blocks that should be run. If specified multiple times then blocks that match any of the patterns are run.

Test API

🔎 Programmatic API documentation is available here.

TooFast injects several global callbacks in test files that register lifecycle hooks and trigger test execution.

test

The minimum setup that you need in a test file is the test callback which runs a test. For example, let's say there's a function factorial() which performance must be measured. Your whole test could be:

test('factorial of 33', measure => {
  measure(() => {
    factorial(33);
  });
});

The measure callback starts the performance measurement. It can be invoked multiple times inside a test block to collect a data population from which an average results are derived.

test('factorial of 33 and 42', measure => {

  measure(() => {
    factorial(33);
  });

  measure(() => {
    factorial(42);
  });
});

The measure callback returns a promise that is resolved as soon as performance measurement is completed.

Test lifecycle is initiated for each test block and run in a separate process.

describe

Creates a block that groups together several related tests.

describe('factorial', () => {

  test('of 42', measure => {
    measure(() => {
      factorial(42);
    });
  });
});

describe blocks can be nested:

describe('Math', () => {

  describe('factorial', () => {
    // Tests go here
  });
});

Lifecycle hooks

There are several global functions injected by TooFast that register hooks. Hooks are invoked at different phases of the performance test suite lifecycle: beforeEach, afterEach, afterWarmup, beforeBatch, afterBatch, beforeIteration, and afterIteration.

The chart below demonstrates when they are called.

flowchart TD

describe --> testLifecycle

subgraph testLifecycle [Test lifecycle]
    direction LR

    subgraph warmup [Warmup]
        direction TB
        warmupBeforeBatch(beforeBatch) -->
        warmupBeforeIteration(beforeIteration) -->
        warmupMeasure[measure] -->
        warmupAfterWarmup(afterWarmup) -->
        warmupAfterIteration(afterIteration) -->
        warmupAfterBatch(afterBatch)
    end

    subgraph batch [Batch]
        direction TB
        testBeforeBatch(beforeBatch) -->
        testBeforeIteration(beforeIteration) -->
        testMeasure[measure] -->
        testAfterIteration(afterIteration) -->
        testAfterBatch(afterBatch)
    end

    beforeEach(beforeEach) -->
    test -->
    warmup -->
    batch -->
    afterEach(afterEach)
end

Hooks can be registered at root level, or inside a describe or test block. Registered hooks affect measure calls that are nested in the same enclosing block.

Hooks are always registered before any measurements are started, so the code below would first register beforeEach and beforeIteration hooks and only after that would run measure.

describe('factorial', () => {

  beforeEach(() => {
    // Runs before each test
  });

  test('of 42', measure => {
    measure(() => {
      factorial(42);
    });

    beforeIteration(() => {
      // Runs before each measurement iteration
    });
  });
});

Test options

Provide test options to test, describe and measure functions. Options of nested blocks are merged.

describe('factorial', { batchTimeout: 500 }, () => {

  test('of 42', { targetRme: 0.2 }, measure => {

    measure({ warmupIterationCount: 5 }, () => {
      factorial(42);
    });
  });
});

measureTimeout

The maximum measure duration in milliseconds. Doesn't include the duration of warmup iterations. Defaults to 10_000.

targetRme

The maximum relative margin of error that must be reached for each measurement [0, 1]. Defaults to 0.01.

warmupIterationCount

The maximum number of warmup iterations that are run before each measurement. Defaults to 1. Set to 0 to disable warmup.

batchIterationCount

The maximum number of iterations in a batch. Unlimited by default.

batchTimeout

The maximum duration of batched measurements in milliseconds. Defaults to 1_000.

batchIntermissionTimeout

The delay between batched measurements in milliseconds. VM is expected to run garbage collector during this delay. Defaults to 200.

You can also register hooks specific for a particular measure call.

test('factorial', measure => {
  measure(
    {
      beforeBatch() {
        gc();
      }
    },
    () => {
      factorial(42);
    }
  );
});

Configuration

By default, TooFast searches for .toofastrc, toofast.json, or toofast.config.js in the current directory.

Configuration file should export the object that satisfies the Config interface:

testOptions

The default test options used for all tests.

include

The array of glob patterns of included test files. File paths are resolved relative to the config file.

setupFiles

The array of glob patters of files that are evaluated in the test environment before any test suites are run. File paths are resolved relative to the config file.