Small changes related to CakePHP best practices and performance tweaks

[lorenzo]

I have made a few changes to correct how the frameworks was being used, specially echoing in the controller which is not an accepted practice in the community.

I have also made some slight changes for performance in a way that they are still fair to the other frameworks in this repository:

• Only connected a single route. The majority of the other framework samples only had 1 route.
• Removed development plugins, there is no need for them
• Removed the development oriented "AssetsDispatcher". That's an usual common practice before going live.
• Removed the use of unused component in the application, loading code that is not needed by default is not something you would do in a normal application.

[lorenzo]

Also, I'm curious to know if you did "composer dumpautolaod -o" before running the tests in all projects?

Edit: I can see this commits implements it #2

[lorenzo]

Update: After looking at the PR accepted for Symfony I decided to add another commit. In that PR, most configuration was removed. For example, no logging component was loaded:

https://github.com/kenjis/php-framework-benchmark/pull/3/files#diff-916370cff7f64fa71f0e69c922b515aaL14

I have done similar changes in the Cake repo.

[alexanderschnitzler]

I wouldn't adjust Cake but revert the symfony-patch again. If I understood correctly this benchmarks tests the output of Hello World! through a single route in any framework with their basic configuration. So, stripping off everything that slows down a Hello World, is not fair and not want I want to see compared.

[lorenzo]

@alexanderschnitzler I wouldn't mind removing the second commit (despite the fact that it is something I do in my projects: removing stuff I am not using). The first commit should stay, for the reasons exposed in the initial PR description.

[neomerx]

Guys you need to move your framework up in the testing script. It will have more impressive results on the server which is not holding thousands opened connections yet. And then invoke your less favorite framework test. Sure screwing PHP when finished would be even better.

[lorenzo]

@neomerx Unfortunately this kind of benchmarks keep appearing and become quite popular despite the fact they provide no real value or insight on the actual framework performance or its ability to scale. But we still need to point out the misconfiguration problems so people looking at these numbers don't get an entirely wrong idea of what a framework is capable of

[neomerx]

@lorenzo Have you seen what slim does? That's the whole app

$app = new \Slim\Slim();
$app->get('/hello/index', function () {
    echo 'Hello World!';
});
$app->run();

When every framework would be chopped to routing + echo you will compete for place in testing script

[lorenzo]

@neomerx Sure, I saw that. Looking at that motivated me to submit this PR. I could have done this to make any framework it the fastest framework, though:

<?php
// index.php
die('Hello World!');

[alexanderschnitzler]

Well, the concept of slim is to be slim. And faster doesn't necessarily mean better. So no framework should be optimized at all. Let's just test a Hello World with each and every framework and that's it. If slim wins that battle due to missing components, that's the way it is.

In another benchmark that tests functionalities like security or asset management, slim might not even be present at all, but who cares? So, please do not adjust your framework to make the test results look better.

[neomerx]

@lorenzo I can't get if

<?php
// index.php
die('Hello World!');

is optimized cake or symfony (:sarcasm:) but I have a strong feeling that supporting such tests is terribly wrong instead we should educate why they are wrong

[lorenzo]

@neomerx I'm not sure what you mean.

@alexanderschnitzler I have absolutely no problem with that. But some people do care, as they see nice looking graphs portraying one thing to be "better" than other based on absolutely nothing.

[neomerx]

@lorenzo updated above

[kenjis]

@lorenzo Thank you for you PR.

I applied your patch and ran on my benchmarking environment:

framework	requests per second	relative	peak memory	relative
silex-1.2	392.00	5.5	1.00	2.0
cake-3.0	380.19	5.3	1.00	2.0
symfony-2.6	296.75	4.2	1.00	2.0

But I'm not going to merge your PR as it is. I feel it goes too far.

[lorenzo]

@kenjis What needs to be done so you feel this is not going too far? An given how similar the changes were when compared to symfony, why did you not feel the same of #3?

[harikt]

In my humble opinion, we should not remove the components the framework provides. The framework provides the needed and the necessary components for the user. That means it really is needed. I don't think every developer will optimize a framework components until they notice performance issues or someone says to do so.

Each framework have its own good / bad things.

Regarding Slim.

Slim has changed a lot from 2.0 to 3.0. You can change something like

$app = new \Slim\App();
$app->get('/books', 'Bookshelf\BookController:listBooks')->SetName('list-books');
$app['Bookshelf\BookController'] = function ($c) {
    return new Bookshelf\BookController($c['view']);
};
$app->run();

Courtesy https://github.com/akrabat/slim-bookshelf/ .

Micro frameworks are nice, to start small and grow. It may be bolted who knows. But in the end, I do believe they will use something like cakephp/orm , eloquent, doctrine or something else under the hood. But I am not seeing a very huge difference in performance impact happening to micro frameworks for they can switch between the bolted things easily.

[lorenzo]

@harikt As long as comparisons like this one are made, I think it is fair to change the code to reflect a more realistic case. Again, why were the changes for symfony accepted?

[harikt]

@lorenzo I have not seen or tried to check until now. It was the tweet from Koriym that made me here.

Before benchmarking I think a few things need to be taken care .

1 ) Restart apache , if possible make only one virtual host ;)
2 ) composer dump-autoload optimize
3 ) Warm the page, cache etc // symfony guys don't complain ;)
4 ) Test and make sure the page you are requesting is not getting a 404 or something else.
5 ) ab test :) .

[kenjis]

@lorenzo I'm sorry to have kept you waiting. I've been thinking about this benchmarks.
I understand it is not fair that only optimization for Symfony is accepted.

[kenjis]

I changed my benchmarking policy: #10
So I reverted symfony's optimization code.

But I created a new branch optimize, and accepted your PR.
11824e7
ca55d14
1bc29f4

If I did something wrong or missing, please inform me.

[lorenzo]

Thanks @kenjis, the changes look fine to me :)

[kenjis]

@lorenzo You are welcome. Thanks for your feedback!