(No longer maintained) Basic performance tests for promise implementations
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
bin
lib
.gitignore
README.md
package.json

README.md

This project is no longer maintained


Promise implementation performance tests

This is a set of basic performance tests for promise implementations. As is almost always the case, take these with the usual grains of salt. That said, they should give a reasonable ballpark comparison of the performance characteristics of common, basic operations that most promise libraries provide.

Of course, performance is not the only thing to consider in a promise library. Interoperability via a proposed standard, such as Promises/A, API convenience, safety, and even code size (for browser applications) are all important, application-specific considerations.

Running the tests

Right now, the tests are runnable en masse via npm test in unix-like environments, and individually via node in other envs.

Setup

  1. Clone the repo
  2. npm install to install the promise implementations to be tested
  3. Run tests:
    • Run all tests: npm test
    • Run a single test with node: node <test>

Implementation-specific notes

when.js

when.js uses synchronous resolutions, and no longer uses Object.freeze() as of v1.6.0, to avoid this unfortunate v8-imposed performance penalty.

avow

avow is an example Promises/A+ implementation. In its default configuration, it uses asynchronous resolutions and does not call Object.freeze. However, it can be configured to use synchronous resolutions, and/or Object.freeze. Performance tests are run using the default configuration.

Q

Q uses asynchronous resolutions, and calls Object.freeze on its promises, and so it incurs the v8-imposed performance penalty.

RSVP

RSVP uses asynchronous resolutions, and doesn't use Object.freeze.

deferred

deferred uses synchronous resolutions, and doesn't use Object.freeze.

laissez-faire

lassez-faire uses synchronous resolutions, and doesn't use Object.freeze.

jQuery Deferred

jQuery uses synchronous resolutions, and it doesn't use Object.freeze.

These tests use jQuery via jquery-browserify, with jsdom for support. This approach was taken from the Promises Test Suite, and currently, appears to be the only way to use jQuery 1.8.x in node.

jQuery Deferred is not intended to be fully Promises/A compliant in its forwarding behavior. We've done our best to design the tests so that that does not affect the performance characteristics. While this does affect the computation results of some tests, it can be ignored for most performance testing purposes.

Test Environment

These tests were run on a MacBook Pro Intel Core i7, 2.3Ghz, 8g RAM, 256g SSD, using Node.js v0.8.14 installed via nvm and the following library versions (npm ls):

├── avow@1.0.0
├─┬ deferred@0.6.1
│ ├── es5-ext@0.9.1
│ ├── event-emitter@0.2.1
│ └── next-tick@0.1.0
├── jquery-browserify@1.8.1
├─┬ jsdom@0.2.19
│ ├─┬ contextify@0.1.3
│ │ └── bindings@1.0.0
│ ├── cssom@0.2.5
│ ├── cssstyle@0.2.3
│ ├── htmlparser@1.7.6
│ └─┬ request@2.11.4
│   ├─┬ form-data@0.0.3
│   │ ├── async@0.1.9
│   │ └─┬ combined-stream@0.0.3
│   │   └── delayed-stream@0.0.5
│   └── mime@1.2.7
├── laissez-faire@0.1.2
├── q@0.8.9
├── rsvp@1.0.0
└── when@1.6.1

Test Results

Each test is sorted from best to worst time. Times are in milliseconds, and Diff is the percentage difference from the best time (computed via ((current - best) / best) * 100)).

==========================================================
Test: promise-fulfill x 10000
----------------------------------------------------------
Name      Time ms   Avg ms   Diff %
when            4   0.0004        -
laissez         5   0.0005    25.00
deferred        7   0.0007    75.00
avow           24   0.0024   500.00
rsvp           74   0.0074  1750.00
q             145   0.0145  3525.00
jquery        154   0.0154  3750.00
Should be empty: []

==========================================================
Test: promise-reject x 10000
----------------------------------------------------------
Name      Time ms   Avg ms   Diff %
when            8   0.0008        -
avow           13   0.0013    62.50
rsvp           48   0.0048   500.00
laissez        82   0.0082   925.00
deferred       83   0.0083   937.50
q             158   0.0158  1875.00
jquery        196   0.0196  2350.00

==========================================================
Test: promise-sequence x 10000
----------------------------------------------------------
Name      Time ms   Avg ms   Diff %
when            8   0.0008        -
laissez        13   0.0013    62.50
deferred       19   0.0019   137.50
avow          106   0.0106  1225.00
rsvp          242   0.0242  2925.00
jquery        462   0.0462  5675.00
q             601   0.0601  7412.50

==========================================================
Test: defer-create x 10000
----------------------------------------------------------
Name      Time ms   Avg ms   Diff %
laissez         1   0.0001        -
when            5   0.0005   400.00
avow           18   0.0018  1700.00
deferred       28   0.0028  2700.00
rsvp           60   0.0060  5900.00
jquery        160   0.0160 15900.00
q             238   0.0238 23700.00

==========================================================
Test: defer-fulfill x 10000
----------------------------------------------------------
Name      Time ms   Avg ms   Diff %
avow           28   0.0028        -
when           35   0.0035    25.00
laissez        42   0.0042    50.00
rsvp          183   0.0183   553.57
deferred      227   0.0227   710.71
jquery        401   0.0401  1332.14
q             817   0.0817  2817.86
Should be empty: []

==========================================================
Test: defer-reject x 10000
----------------------------------------------------------
Name      Time ms   Avg ms   Diff %
when           27   0.0027        -
avow           31   0.0031    14.81
laissez        99   0.0099   266.67
rsvp          290   0.0290   974.07
deferred      300   0.0300  1011.11
jquery        398   0.0398  1374.07
q             889   0.0889  3192.59

==========================================================
Test: defer-sequence x 10000
----------------------------------------------------------
Name      Time ms   Avg ms   Diff %
laissez        12   0.0012        -
when           18   0.0018    50.00
avow           97   0.0097   708.33
deferred      189   0.0189  1475.00
rsvp          306   0.0306  2450.00
jquery        411   0.0411  3325.00
q            1023   0.1023  8425.00

==========================================================
Test: map x 10000
----------------------------------------------------------
Name      Time ms   Avg ms   Diff %
when           12   0.0012        -
deferred       29   0.0029   141.67

==========================================================
Test: reduce-small x 609
NOTE: in node v0.8.14, deferred.reduce causes a
stack overflow for an array length >= 610
----------------------------------------------------------
Name      Time ms   Avg ms   Diff %
when            1   0.0016        -
deferred        4   0.0066   300.00

==========================================================
Test: reduce-large x 10000
NOTE: in node v0.8.14, deferred.reduce causes a
stack overflow for an array length >= 610
----------------------------------------------------------
Name      Time ms   Avg ms   Diff %
when           13   0.0013        -
deferred [RangeError: Maximum call stack size exceeded]