Node.js vs vert.x
WORK IN PROGRESS
Simple and naive Node.js vs vert.x benchmark. I've written it to check if there's any significant performance difference between Node.js (which is getting more popular than even before and undoubtedly more stable and mature) and new kid on the block -- vert.x (which shares the asynchronous and message-oriented philosophy with Node.js). I'm playing with Node since v0.2, this benchmark took me 2 hours of work mostly due to trial-and-error approach to vert.x, so the code may not be the most optimized and sophisticated.
Node.js server uses Restify framework and Mongoose ORM. Vert.x is a complete stack so there's no need for external modules. The database is MongoDB.
- Processor: Intel(R) Core(TM) i5-2410M CPU @ 2.30GHz
- Memory: 4GB DDR3 SDRAM @ 1333MHz (1725MB used)
- Hard drive: KINGSTON SSD SVP100S
- Operating System: Ubuntu 11.04
Yes. It's a laptop.
- Node.js v0.6.6
ulimit = unlimited
Default V8/JVM and Node.js/vert.x settings unless stated otherwise.
As far as I know, vert.x uses one 'vertex' instance by default (I'm certain about that in Node's case). Multiple instances were tested for vert.x, I will add node-cluster benchmarks in the future.
Benchmarks were performed using Siege. All test data is generated on the fly.
Posting and selecting from database
Request adds a random JSON document to database:
author: String(16) date: Date content: String(160)
and gets 100 already stored documents in response (no sorting).
Benchmarking command: siege -c100 -d1 -r100 http://localhost:1337/post
|[1.1] Node.js (default)||[1.2] vert.x (default)|
|Transactions||10000 hits||10000 hits|
|Availability||100.00 %||100.00 %|
|Elapsed time||203.10 secs||64.04 secs|
|Data transferred||257.50 MB||283.23 MB|
|Response time||1.45 secs||0.04 secs|
|Transaction rate||49.24 trans/sec||156.15 trans/sec|
|Throughput||1.27 MB/sec||4.42 MB/sec|
|Memory consumed||35 MB||50 MB -> 1 GB|
Couldn't make multiple vert.x instances work:
Returning 'hello world' response
Short JSON was returned in response to every request, no calculations performed.
Benchmarking command: siege -c100 -b -r1000 http://localhost:1337/hello
|[2.1] Node.js (default)||[2.2] vert.x (default)||[2.3] vert.x (4 instances)|
|Transactions||39515 hits||100000 hits||100000 hits|
|Availability||97.37 %||100.00 %||100.00 %|
|Elapsed time||39.72 secs||22.41 secs||13.84 secs|
|Data transferred||1.21 MB||1.81 MB||1.81 MB|
|Response time||0.09 secs||0.02 secs||0.01 secs|
|Transaction rate||994.84 trans/sec||4462.29 trans/sec||7225.43 trans/sec|
|Throughput||0.03 MB/sec||0.08 MB/sec||0.13 MB/sec|
|Memory consumed||35 MB||60 MB||83 MB|
- 2.1 was aborted by Siege due to 'excessive socket failure'.
Random 10000 characters long strings were returned to this requests. No buffers were used, just plain old strings and concatenation (one character at a time).
Benchmarking command: siege -c100 -b -r100 http://localhost:1337/concat
|[3.1] Node.js (default)||[3.2] vert.x (default)||[3.3] vert.x (4 instances)|
|Transactions||10000 hits||10000 hits||10000 hits|
|Availability||100.00 %||100.00 %||100.00 %|
|Elapsed time||20.17 secs||61.27 secs||35.83 secs|
|Data transferred||95.49 MB||95.49 MB||95.49 MB|
|Response time||0.20 secs||0.61 secs||0.34 secs|
|Transaction rate||495.79 trans/sec||163.21 trans/sec||279.10 trans/sec|
|Throughput||4.73 MB/sec||1.56 MB/sec||2.67 MB/sec|
|Memory consumed||30 MB||320 MB||360 MB|
The most famous benchmark out there. Calculating fib(30) recursively.
Benchmarking command: siege -c100 -b -r10 http://localhost:1337/fibonacci
|[4.1] Node.js (default)||[4.2] vert.x (default)||[4.3] vert.x (4 instances)|
|Transactions||1000 hits||1000 hits||1000 hits|
|Availability||100.00 %||100.00 %||100.00 %|
|Elapsed time||26.88 secs||101.46 secs||64.01 secs|
|Data transferred||0.04 MB||0.02 MB||0.02 MB|
|Response time||2.54 secs||9.63 secs||5.72 secs|
|Transaction rate||37.20 trans/sec||9.86 trans/sec||15.62 trans/sec|
|Throughput||0.00 MB/sec||0.00 MB/sec||0.00 MB/sec|
|Memory consumed||15 MB -> 30 MB||300 MB||335 MB|
[ETA: end of June]
- Dummy database constrained benchmarks favors 3x vert.x over Node.js.
- As for 'requests per second' it's 3.5x faster (7x using both processor cores, but this is unfair).
- Vert.x sent more data down the tubes,
- Native string manipulation (simple concat) in V8 seems faster than JVM's.
- It looks like recursive Fibonacci's fans should go with Node.js.
- Naming your vert.x application file 'vertx.js' is a bad idea.
- Vert.x crashed with 'java.lang.OutOfMemoryError: GC overhead limit exceeded' exception because of the MongoDB socket timeouts.