Check for performance improvements in Symfony 5.0

[javiereguiluz]

Description
The *.4 Symfony versions are always the slowest in a branch (e.g. 4.4 is the slowest 4.* version) because of all the deprecated features.

For the same reason, the *.0 versions are usually the fastest Symfony versions. However, for 5.0 this didn't happen. See phpbenchmarks.com results:

Maybe we could do some Blackfire profiling to find possible ways of improving Symfony 5.0 performance. Thanks!

[stof]

The symfony 5.0 benchmark has APP_ENV=dev in their .env file. So it looks like they benchmarked the dev environment, not the prod environment: https://github.com/phpbenchmarks/symfony/blob/d42d5dc67d54de676460f9b91b3bb326b04696b3/.env#L17

[stof]

btw, the 4.4 branch of the benchmark has the same issue

[javiereguiluz]

Thanks Christophe! I've sent a PR to fix it for 5.0 (phpbenchmarks/symfony#13). Let's see what do they think about that.

[steevanb]

Hi, i'm phpbenchmarks owner.

Benchmarks were done in PROD env, be sure of that: https://github.com/phpbenchmarks/symfony/blob/5.0_hello-world/.phpbenchmarks/initBenchmark.sh#L22

If they were done in DEV env, believe me, results will be really slower ;)

[stof]

@steevanb composer dump-env prod does not mean that the APP_ENV variable will be forced to be set to prod. It means that will will load .env and .env.prod (and the .local overrides) to generate the optimized PHP file to skip the .env parsing.

[nicolas-grekas]

composer dump-env prod does not mean that the APP_ENV variable will be forced to be set to prod

it still means that the default is going to be prod, so that's OK, it works for the purpose

[steevanb]

@stof Are you sure about it? On all my Symfony projects, I don't have .env.prod, and composer dump-env prod define APP_ENV to prod in .env.local.php.

[stof]

hmm, indeed, I forgot that dump-env has a special case for APP_ENV

[steevanb]

To be sure there is no problem with this benchmarks, i've relaunched Symfony 5.0 Hello World benchmark with 3 test cases:

1. APP_ENV=prod, APP_DEBUG not defined
2. APP_ENV=prod, APP_DEBUG=false
3. APP_ENV=dev, APP_DEBUG=true

Line Best is the best time before this tests, it's time you can see on phpbenchmarks.com.
Line Actual is time for this benchmark, with the new configuration.

APP_ENV=prod, APP_DEBUG not defined

	Calls	Fails	Total time	rq/sec	ms/rq
Best	50000	0	53609	932.67	1.072
Bench	50000	0	68518	729.73	1.370

APP_ENV=prod, APP_DEBUG=false

	Calls	Fails	Total time	rq/sec	ms/rq
Best	50000	0	53609	932.67	1.072
Bench	50000	0	63742	784.42	1.275

APP_ENV=dev, APP_DEBUG=true

	Calls	Fails	Total time	rq/sec	ms/rq
Best	50000	0	53609	932.67	1.072
Bench	50000	0	110887	450.91	2.218

As you can see, results are very similars when APP_DEBUG is not defined or is =false.

But with APP_ENV=dev, results are slower.

Too me, results available on phpbenchmarks.com are the right ones: APP_ENV is well defined to prod and APP_DEBUG does not seems to change anything.

[javiereguiluz]

@steevanb thanks for double checking the benchmarks (and thanks for creating and maintain that site!). We'll look for other posible solutions to improve performance.

[steevanb]

Thank you! If you need help to improve performances, you can contact me ;)

[stof]

As you can see, results are very similars when APP_DEBUG is not defined or is =false.

that's expected, as the default value for APP_DEBUG is false when running in prod environment.

[steevanb]

Yes, this test was just here to prove it ;)