PHP Framework Benchmark
Switch branches/tags
Nothing to show
Clone or download
kenjis Merge pull request #89 from Nyholm/patch-1
[Symfony] Warm up the cache before we run the tests
Latest commit 2965b9c Jul 21, 2018
Permalink
Failed to load latest commit information.
aura-2.0 composer update Feb 14, 2017
bear-0.10 Remove htaccess Mar 31, 2016
bear-1.0 composer update Feb 14, 2017
benchmarks The "time" result is taken from 10 request (average) Feb 18, 2017
bin Add script to output current framework list as array Dec 6, 2015
cake-3.0 Remove htaccess Mar 31, 2016
cake-3.1 Remove htaccess Mar 31, 2016
cake-3.2 composer update Feb 14, 2017
ci-3.0 Remove CodeIgniter User Guide Feb 20, 2017
ci-3.1 Remove CodeIgniter User Guide Feb 20, 2017
ci-4.0-dev Update ci-4.0-dev URL Dec 7, 2016
cygnite-1.3 composer update Feb 14, 2017
fatfree-3.5 composer update Dec 7, 2016
fuel-1.8 composer update Dec 7, 2016
ice-1.0 Remove htaccess Mar 31, 2016
ice-1.1 Remove htaccess Apr 1, 2016
img Update graph Feb 14, 2017
kumbia-1.0-dev composer update Feb 14, 2017
laravel-4.2 Remove htaccess Mar 31, 2016
laravel-5.0 Remove htaccess Mar 31, 2016
laravel-5.1 Remove htaccess Mar 31, 2016
laravel-5.2 composer update Dec 7, 2016
laravel-5.3 composer update Feb 14, 2017
libs Change graph size Feb 14, 2017
lumen-5.0 Extract the peak memory output code to a file Oct 25, 2015
lumen-5.1 composer update Feb 14, 2017
no-framework Extract the peak memory output code to a file Oct 25, 2015
nofuss-1.2 Remove htaccess Mar 31, 2016
output Update benchmarks Feb 14, 2017
phalcon-1.3 Remove htaccess Mar 31, 2016
phalcon-2.0 Remove htaccess Mar 31, 2016
phpixie-3.2 composer update Feb 14, 2017
radar-1.0-dev composer update Feb 14, 2017
siler-0.6 Fix siler-0.6 code Feb 15, 2017
silex-1.2 Restore silex-1.2/ Feb 1, 2017
silex-1.3 composer update Dec 7, 2016
silex-2.0 composer update Feb 14, 2017
slim-2.6 Fix Slim version and composer update Mar 31, 2016
slim-3.0 composer update Dec 7, 2016
staticphp-0.9 Add missing folder Feb 14, 2017
symfony-2.5 Remove htaccess Mar 31, 2016
symfony-2.6 Remove htaccess Mar 31, 2016
symfony-2.7 composer update Dec 7, 2016
symfony-3.0 composer update Feb 14, 2017
symfony-4.0 Warm up the cache before we run the tests Jun 16, 2018
tipsy-0.10 composer update Feb 14, 2017
typo3f-2.3 composer update Feb 14, 2017
typo3f-3.0 composer update Feb 14, 2017
yii-2.0 composer update Feb 14, 2017
ze-1.0 composer update Feb 14, 2017
zf-1.12 Add output of memory and time Dec 17, 2016
zf-2.4 Remove htaccess Mar 31, 2016
zf-2.5 composer update Dec 7, 2016
zf-3.0 composer update Feb 14, 2017
.gitignore Update gitignore Oct 15, 2015
README.md Correct name of TYPO3Flow to Flow-Framework and correct link Dec 13, 2017
benchmark.sh Improve bash scripts Oct 14, 2015
index.php Fix Phalcon link Jul 3, 2016
list.sh Add Symfony 4.0 Mar 4, 2018
setup.sh Remove all .htaccess Mar 7, 2016
update.sh Add script to composer update Oct 24, 2015

README.md

PHP Framework Benchmark

This project attempts to measure minimum overhead (minimum bootstrap cost) of PHP frameworks in the real world.

So I think the minimum applications to benchmark should not include:

  • cost of template engine (HTML output)
  • cost of database manipulation
  • cost of debugging information

Components like Template engine or ORM/Database libraries are out of scope in this project.

Benchmarking Policy

This is master branch.

  • Install a framework according to the official documentation.
  • Use the default configuration.
    • Don't remove any components/configurations even if they are not used.
    • With minimum changes to run this benchmark.
  • Set environment production/Turn off debug mode.
  • Run optimization which you normally do in your production environment, like Composer's --optimize-autoloader.
  • Use controller or action class if a framework has the functionality.

Some people may think using default configuration is not fair. But I think a framework's default configuration is an assertion of what it is. Default configuration is a good starting point to know a framework. And I can't optimize all the frameworks. Some frameworks are optimized, some are not, it is not fair. So I don't remove any components/configurations.

But if you are interested in benchmarking with optimization (removing components/configurations which are not used), See optimize branch.

If you find something wrong with my code, please feel free to send Pull Requests. But please note optimizing only for "Hello World" is not acceptable. Building fastest "Hello World" application is not the goal in this project.

Results

Benchmarking Environment

  • CentOS 6.8 64bit (VM; VirtualBox)
    • PHP 5.6.30 (Remi RPM)
      • Zend OPcache v7.0.6-dev
    • Apache 2.2

Hello World Benchmark

These are my benchmarks, not yours. I encourage you to run on your (production equivalent) environments.

(2017/02/14)

Benchmark Results Graph

framework requests per second relative peak memory relative
siler-0.6 2,069.69 20.3 0.25 1.0
kumbia-1.0-dev 1,753.60 17.2 0.29 1.2
staticphp-0.9 1,665.28 16.3 0.27 1.1
phalcon-2.0 1,618.39 15.9 0.26 1.1
tipsy-0.10 1,376.97 13.5 0.32 1.3
fatfree-3.5 965.16 9.5 0.41 1.7
ci-3.0 753.09 7.4 0.42 1.7
nofuss-1.2 667.24 6.5 0.40 1.6
slim-3.0 550.43 5.4 0.61 2.5
bear-1.0 502.52 4.9 0.73 3.0
lumen-5.1 415.57 4.1 0.85 3.5
yii-2.0 410.08 4.0 1.32 5.4
ze-1.0 403.34 4.0 0.75 3.1
cygnite-1.3 369.12 3.6 0.71 2.9
fuel-1.8 344.26 3.4 0.63 2.6
silex-2.0 342.81 3.4 0.78 3.2
phpixie-3.2 267.24 2.6 1.25 5.1
aura-2.0 233.54 2.3 0.88 3.6
cake-3.2 174.91 1.7 1.95 7.9
zf-3.0 133.87 1.3 2.24 9.1
symfony-3.0 131.50 1.3 2.18 8.9
laravel-5.3 101.94 1.0 2.83 11.5

Note(1): All the results are run on php with phalcon.so and ice.so. If you don't load phalcon.so or ice.so, the rps except for Phalcon or Ice probably increase a bit.

Note(2): This benchmarks are limited by ab performance. See #62.

How to Benchmark

If you want to benchmark PHP extension frameworks like Phalcon, you need to install the extenstions.

Install source code as http://localhost/php-framework-benchmark/:

$ git clone https://github.com/kenjis/php-framework-benchmark.git
$ cd php-framework-benchmark
$ bash setup.sh

Run benchmarks:

$ bash benchmark.sh

See http://localhost/php-framework-benchmark/.

If you want to benchmark some frameworks:

$ bash setup.sh fatfree-3.5/ slim-3.0/ lumen-5.1/ silex-1.3/
$ bash benchmark.sh fatfree-3.5/ slim-3.0/ lumen-5.1/ silex-1.3/

Linux Kernel Configuration

I added below in /etc/sysctl.conf

# Added
net.netfilter.nf_conntrack_max = 100000
net.nf_conntrack_max = 100000
net.ipv4.tcp_max_tw_buckets = 180000
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_fin_timeout = 10

and run sudo sysctl -p.

If you want to see current configuration, run sudo sysctl -a.

Apache Virtual Host Configuration

<VirtualHost *:80>
  DocumentRoot /home/vagrant/public
</VirtualHost>

References

Other Benchmarks