bench some web microframeworks (Ruby)

Because we don't have enough drama, let's bench some hello world apps!

Why?

As part of some research, collected some benchmark around Ruby web microframeworks.

The idea is bench the performance from Rack, throught the stack of the framework to the minimum representation of an application (Hello World apps).

It is not the goal to bechmark hits to the database or external services, nor to do a silly fibonacci-as-a-service, ok?

What?

The following microframeworks were considered when doing this research:

• Brooklyn - brooklyn.ru
• Cuba - cuba.ru
• Gin - gin.ru
• Hobbit - hobbit.ru
• Lotus (Router) - lotus-router.ru
• Nancy - nancy.ru
• NYNY - nyny.ru
• Mustermann - mustermann.ru
• Rack - rack.ru + rack-response.ru
• Rails - rails.ru
• Ramaze - ramaze.ru
• Rambutan - rambutan.ru
• Roda - roda.ru
• Scorched - scorched.ru
• Sinatra - sinatra.ru

Please note that while Rails has been added to the list, it is just a minimalistic representation (using Metal, no full middleware stack, etc). You shouldn't take the performance numbers mentioned here about Rails (or any other) as scientific and decision-taking references.

How?

Used wrk to benchmark, locally, a burst of requests (in 2 threads) over 10 seconds. The command line used was:

$ wrk -t 2 http://localhost:9292/

All the frameworks were run using Puma on Ruby 2.1, in production mode and using 16 threads:

$ puma -e production -t 16:16 apps/<framework.ru>

Run benchmark for all frameworks

# use `bundle exec`, if needed
$ bench/summary-memory
$ bench/summary-speed

Have some numbers around?

Yup, I do:

Requests/sec

Framework            Requests/sec  % from best
----------------------------------------------
rack                      8426.71       100.0%
mustermann                7431.67       88.19%
hobbit                    7222.83       85.71%
cuba                      7114.43       84.43%
roda                      7024.97       83.37%
lotus-router              6943.77        82.4%
rack-response             6534.07       77.54%
brooklyn                  6312.10       74.91%
rambutan                  5399.59       64.08%
nancy                     5214.94       61.89%
gin                       3735.51       44.33%
nyny                      3537.24       41.98%
sinatra                   2525.73       29.97%
rails                     2482.94       29.47%
scorched                  1504.56       17.85%
ramaze                    1277.85       15.16%

Memory Allocation/Request

Framework       Allocs/Req Memsize/Req
--------------------------------------
rack                    60        1704
roda                    65        1904
hobbit                  70        1976
mustermann              73        2040
cuba                    79        1936
rack-response           83        3072
lotus-router            92        2272
brooklyn                98        2432
nancy                  107        3440
rambutan               116        3448
nyny                   169        5248
rails                  212        9109
gin                    214        9419
sinatra                255        6083
ramaze                 579       21837
scorched              1711      115524

These numbers were collected on:

• OSX, 10.8.5, MacBook Pro i5 (2.5GHz), 8GB 1600 MHz DDR3.
• ruby 2.1.3p242 (clang 5.1 - clang-503.0.40)