Speed up generated autoloaders #1585

Closed
dlsniper opened this Issue Feb 14, 2013 · 15 comments

Projects

None yet

10 participants

@dlsniper

Hello,

I've did some research with a friend, @talpah, and, as expected, the generated autoloaders can be improved in terms of speed by removing the

// autoload_classmap.php generated by Composer

$vendorDir = dirname(__DIR__);
$baseDir = dirname($vendorDir);

part of the autoload_classmap.php and autoload_namespaces.php and replacing them with real paths and no concatenation instead.

The benefits should be quite clear, especially for large arrays, as for each entry in them instead of having two operations done, we'll do only one.

I've yet to test it on what happens when using op-code caching but since not all people can benefit from it, I think it would be a nice addition.

What's the opinion on this? Would this be a good addition? Aside from the cases where you move the application without regenerating the autoloaders, I don't see many problems.

Also, any pointers on how this could be done in a clean manner?

Thank you!

Contributor
stof commented Feb 14, 2013

Replacing them with real paths means you cannot run composer on one machine and copy the vendor folder to another machine anymore, which means you cannot install vendors locally before deploying (you have to run composer on the prod server) and you cannot use the composer autoloader for code packaged in a phar (composer itself for instance).

Contributor

The only optimization you could do would be to move those two lines to autoload_real, so you would remove duplicity.

But what you're asking would make composer absolutely usless for me as I neve run composer install directly on production folder - that would be insane.

Owner
Seldaek commented Feb 15, 2013

I won't accept this as a general change because it causes problems with moving projects around. It could be done as an extreme optimization in dump-autoload though if it is proven to be noticeably faster with opcode caching (and I would tend to say only Zend Optimizer+ matters if it gets included in 5.5). People that don't install an opcode cache have bigger performance issues than micro optimizations like this one.

Contributor
xrstf commented Feb 24, 2013

Please don't change the behaviour as it is right now. Our company is relying on being able to run Composer on dev machines and then deploy to [crappy] PHP 5.2 hostings.

It is, IMHO, wrong having that concatenation done on every elements. It's is quite common to have thousands of entries and it means there's thousands of concatenation on every requests, only if a part of those are used.
Concatenation could be done while using the classmap rather than while defining it.

That said, any serious deployment strategy takes running commands into account.

Contributor
stof commented Mar 12, 2013

@patrickallaert Doing the concatenation when dumping the autoloader means that you cannot move the generated file to another location. this breaks in 2 cases (at least):

  • running composer locally before deploying (you would need to dump the autoloader again on all your prod servers)
  • using the composer autoloader in a phar archive (composer itself being an example)

@patrickallaert Doing the concatenation when dumping the autoloader means that you cannot move the generated file to another location. this breaks in 2 cases (at least):

@stof I do understand that, but you haven't taken one of my suggestion into account:

Concatenation could be done while using the classmap rather than while defining it.

I mean:

$classMap = array("ClassName" => "relative/path/to/ClassName.php");
[...]
require $baseDir . $classMap[$className];

instead of:

$classMap = array("ClassName" => $baseDir . "/relative/path/to/ClassName.php");
[...]
require $classMap[$className];

It means that concatenation is done only for the classes that are used and not just all of the entries.

@Seldaek Seldaek closed this Jan 1, 2014

Any update on this?

I've had the same issue on OpenShift with php 5.5 (using Zend Optimizer+):
Sometimes it takes up to two seconds to run ComposerAutoloaderInit###::getLoader() in production (as reported by New Relic)!

So the following lines are part of my deployment script now:

echo "Optimizing composer autoloader."
sed -i "s|\$vendorDir . '|'${OPENSHIFT_REPO_DIR}vendor|" ${OPENSHIFT_REPO_DIR}/vendor/composer/autoload_classmap.php
sed -i "s|\$baseDir . '/|'${OPENSHIFT_REPO_DIR}|" ${OPENSHIFT_REPO_DIR}/vendor/composer/autoload_classmap.php

I would be happy to have this option built-in in composer!

milesj commented May 22, 2014

Make it an option during composer install?

composer install --optimize-autoloader --with-absolute-paths, or composer install -o -a, but also for composer update, of course.

rskvazh commented Jun 19, 2014

@sebastien-fauvel, +1. We have same problem with many memcpy() calls.
PHP 5.5 + Opcache.
https://gist.github.com/rskvazh/d616dafd78f10993e2ef

Without concatenation performance increased by 25% on our EC2-servers.

dincho commented Jan 12, 2015

composer install --optimize-autoloader --with-absolute-paths, or composer install -o -a, but also for composer update, of course.

It would be better $vendorDir to be passed as parameter, so you can generate the classmap on "deployer" machine for other/production machines.

Something like:

composer install --optimize-autoloader --with-absolute-path=/home/production/app/vendor

Just FYI, I see this is old and closed.

@dincho could you open a new issue with your proposal?

dincho commented Jan 12, 2015

I'm not sure that It's OK to do so once it's already closed. IMO the contributor that closed it should take that decision. In this case @Seldaek. I don't want to flood with new issues with the same subject.

Owner
Seldaek commented Jan 12, 2015

Feel free to open a new issue for this yes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment