Right after announcing Padrino, many developers began to request benchmarks in order give them a better understanding of how our framework compared in terms of performance with the existing ruby web frameworks.
Personally, no one on our team is a big fan of benchmarks since they can often be misleading and real world usage is generally quite different. However, we decided that providing a set of reasonable benchmarking results would help people get at least a general sense of Padrino’s speed in relation to other popular alternatives.
Within our benchmarks, we opted to test three different sample applications (three repo branches):
- text_render
-
a bare minimum app serves as a baseline simply rendering an inline string for the response.
- template_render
-
a simple app serving a small ‘view’ template for the response.
- more_advanced
-
‘full-stack’ app including sessions, haml, layouts, templates, and helpers.
The application frameworks tested in these benchmarks:
-
Rack 1.1.0 (site)
-
Padrino 0.9.10 (site)
-
Sinatra 1.0.0 (site)
-
Rails 3.beta3 (site)
-
Merb 1.1.0 (site)
-
Camping 2.0.0 (site)
-
Ramaze 2010.04.04 (site)
Our intention is for these benchmarks to include all ‘popular’ ruby framework alternatives. If you would like to add another framework, please fork this project and let us know!
While constructing ‘equivalent’ applications for each of the frameworks for each of the branches, occasionally decisions had to be made in order to achieve the same functionality throughout. Here are a few decisions made during the development of these benchmarks:
-
The camping applications utilize ‘markaby’ instead of ‘erb’ and ‘haml’ due to a current framework limitation.
-
The rack applications utilize the ‘Tilt’ library for the rendering of ‘erb’ and ‘haml’ templates (and layouts).
-
The sinatra and rack applications required us to build simplified ‘helpers’ to keep the tests accurate.
We want to be fair for each of the frameworks tested and stick to the common ‘real-world’ usage. If anybody has any suggestions on how to improve the accuracy of these benchmarks, please do let us know or fork the project!
Also worth noting is that running benchmarks can give yield quite different results on different machines. This can be primarily due to the fact that your system can be busy with other background processes during execution. Padrino, Merb and Sinatra all also have quite similar performance results. This means that often in certain executions, Padrino might run faster and yet in others Merb or Sinatra might appear faster. Be sure to run the benchmarks 5 or 6 times and then choose the most frequent result.
These benchmarks were executed on the following machine:
Model Name:: MacBook Pro Model Identifier:: MacBookPro5.1 Processor Name:: Intel Core 2 Duo Processor Speed:: 2.4 GHz Number Of Processors:: 1 Total Number Of Cores:: 2 L2 Cache:: 3 MB Memory:: 4 GB Bus Speed:: 1.07 GHz
with ruby 1.8.7 (2010-01-10 patchlevel 249) [i686-darwin10.2.0]
The original source code for running these benchmarks is attributed to the oleganza repo. This fork was developed by DAddYE and Nathan Esquenazi for use in benchmarking Padrino. The code was heavily adapted and recent versions of each popular ruby web framework were included along with developing three different benchmark tests for additional accuracy.