Which is the fastest?

This is a fork of Taichiro Suzuki's "Which is the fastest?" benchmarks, designed for use as a performance test when working on the HTTP modules.

Result

Last update: 2017-10-04

OS: Darwin (version: 16.6.0, arch: x86_64)
CPU Cores: 8

Swift Results

Language (Runtime)        Framework (Middleware)          Max [sec]       Min [sec]       Ave [sec]
------------------------- ------------------------- --------------- --------------- ---------------
swift                     vapor                            8.563006        6.936387        7.576272
swift                     perfect                          8.909572        8.094777        8.528549
swift                     kitura                           8.439633        7.952627        8.289233
swift                     http                             6.717852        6.031183        6.382276

 -- Ranking (Language) -- 
1. swift (http) 6.3822756

 -- Ranking (Framework) -- 
1. http 6.3822756
2. vapor 7.576272399999999
3. kitura 8.2892332
4. perfect 8.5285488

Current target frameworks (middlewares)

• Ruby
  • Rails
  • Sinatra
  • Roda
• Crystal
  • Kemal
  • raze
  • router.cr
• Go
  • Echo
  • gorilla-mux
  • iris
  • fasthttprouter
• Rust
  • IRON
  • nickel.rs
  • Rocket (nightly)
• node
  • express
  • express/cluster
• Elixir
  • Plug
  • Phoenix
• Swift
  • Vapor
  • Perfect
  • Kitura
  • HTTP
• Scala
  • Akka-Http (Routing DSL)
• C#
  • ASP.NET Core
• Python
  • sanic
  • japronto

See Development section when you want to add new languages or frameworks.

The rule

We want to know the response time, not a usability. So full-stack framework is at a disadvantage.

• Each server has no special logics.
• Each server's executable is named as server_[Lauguage]_[Framework]. (For example, server_ruby_sinatra)
• There are only 3 routes
  • GET '/' return status code 200 with empty body
  • GET '/user/:id' return status code 200 with the id
  • POST '/user' return status code 200 with empty body

Installation

Required environment -> See Current target frameworks(middlewares)

By using Neph

Neph is a modern command line job processor that can be substitute for make command.

To compile servers and benchmarker,

> neph

For each language,

> neph -j ruby

For each framework,

> neph -j rails

See neph.yml

By using make

To compile servers and benchmarker,

> make

For each language,

> make ruby

For each framework,

> make rails

Usage

You can take a benchmark by

> bin/benchmarker

For each language

> bin/benchmarker ruby

For each framework

> bin/benchmarker rails

For comparison (Comparing rails, kemal and router.cr in this example)

> bin/benchmarker rails crystal

If you take it manually, you can run each server by

> bin/server_[Language]_[Framework]

and run client by

> time bin/client

You can set # of threads and # of the loops of the request(there are 3 requests in a loop) by

> time bin/client -t 16 -r 1000

In the above example, 16 threads requests 1000 * 3 times. So 48000 requests are sent in total.

Using Docker

Setup servers by using docker is under WIP. Currently, crystal and ruby servers are supported. For example

docker-compose up rails

Then you can run your client by

time ./bin/client

Development

• Give me PR when you want to add other web frameworks
• Give me PR when you can tuning each framework (under the rule)

Where should I modify when adding new framework

• /[language]/[framework]/[codes] <- Project iteself
• benchmarker/benchmarker.cr <- Adding it as a target to
• README.md <- Adding it as a target framework of the list
• Makefile
• neph.yml (optional)

Anyway, you don't have to care about details since maintainer can fix them after merging it. The result will be updated by maintainer.

Contributing

1. Fork it (https://github.com/tbrand/which_is_the_fastest/fork)
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create a new Pull Request

Contributors

• tbrand Taichiro Suzuki - creator, maintainer
• OvermindDL1 OvermindDL1 - maintainer

Donate

1AE9P6TUVik1rJGQhaSqGWRk1oAQ3DJnmo