Composer update runs out of memory

[jrooi]

Running the "composer update" command results in a fatal error:

Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 79 bytes) in phar:///usr/local/bin/composer/src/Composer/DependencyResolver/RuleWatchGraph.php on line 100

Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 71 bytes) in phar:///usr/local/bin/composer/src/Composer/DependencyResolver/Solver.php on line 700

Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 72 bytes) in phar:///usr/local/bin/composer/src/Composer/DependencyResolver/RuleWatchGraph.php on line 104

I already tried removing the vendors in my Symfony2 project. After removing the vendors and running a "composer install" command I get no errors. Running a "composer update" directly after the install results in the same memory error. The current memory limit is set to 512MB, that should be enough, right?

What can be the cause of the sudden memory issue? Can I take other steps in solving this issue? As far as I can tell, the dependencies haven't changed and I don't include many vendors.

Any help is much appreciated!

Kind regards,
Jeroen van Rooij

Ps. I'm using the latest composer version and al the requires in the json are set to a fixed version

[Seldaek]

Could you maybe share your composer.json so it's at least possible to try and reproduce it?

[jrooi]

Yeah ofcourse, sorry!

{
    "name": "lsg/lsg.nl",
    "description": "Lsg website",
    "autoload": {
        "psr-0": { "": "src/" }
    },
    "repositories": [
        { "type": "vcs", "url": "git@github.com:lsg-it/ExtensionBundle.git" },
        { "type": "vcs", "url": "git@github.com:lsg-it/CmsBundle.git" }
    ],
    "require": {
        "php": ">=5.3.3",
        "symfony/symfony": "v2.1.7",
        "doctrine/orm": "2.3.1",
        "doctrine/doctrine-bundle": "v1.0.0",
        "doctrine/doctrine-fixtures-bundle": "2.1.*",
        "twig/extensions": "1.0.*",
        "symfony/assetic-bundle": "v2.1.1",
        "symfony/swiftmailer-bundle": "v2.1.7",
        "symfony/monolog-bundle": "v2.1.7",
        "sensio/distribution-bundle": "v2.1.7",
        "sensio/framework-extra-bundle": "v2.1.7",
        "sensio/generator-bundle": "v2.1.7",
        "jms/security-extra-bundle": "1.2.0",
        "jms/di-extra-bundle": "1.1.1",
        "nelmio/solarium-bundle": "v1.1.0",
        "friendsofsymfony/facebook-bundle": "v1.1.0",
        "hwi/oauth-bundle": "*",
        "gedmo/doctrine-extensions": "v2.3.4",
        "lsg/cms-bundle": "master",
        "lsg/extension-bundle": "master"
    },
    "scripts": {
        "post-install-cmd": [
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::buildBootstrap",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::clearCache",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installAssets",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installRequirementsFile"
        ],
        "post-update-cmd": [
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::buildBootstrap",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::clearCache",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installAssets",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installRequirementsFile"
        ]
    },
    "minimum-stability": "dev",
    "extra": {
        "symfony-app-dir": "app",
        "symfony-web-dir": "web",
        "symfony-assets-install": "symlink"
    }
}

Those two Lsg requires are internal, private bundles. The json of those two bundles are:

{
    "name": "lsg/cms-bundle",
    "type": "symfony-bundle",
    "description": "LSG CMS Bundle",
   "repositories": [
        { "type": "vcs", "url": "git@github.com:lsg-it/ExtensionBundle.git" }
    ],
    "require": {
        "php": ">=5.3.3",
        "symfony/symfony": "2.1.*",
        "white-october/pagerfanta-bundle": "2.1.*",
        "craue/formflow-bundle": "1.1.0",
        "lsg/extension-bundle": "master"
    },
    "autoload": {
        "psr-0": {
            "Lsg\\CmsBundle": ""
        }
    },
    "target-dir": "Lsg/CmsBundle"
}

And the last one:

{
    "name": "lsg/extension-bundle",
    "type": "symfony-bundle",
    "description": "Standard LSG extensions",
    "require": {
        "php": ">=5.3.3",
        "symfony/symfony": "2.1.*",
        "gedmo/doctrine-extensions": "v2.3.4",
        "twig/twig": "1.*"
    },
    "autoload": {
        "psr-0": {
            "Lsg\\ExtensionBundle": ""
        }
    },
    "target-dir": "Lsg/ExtensionBundle"
}

[todsul]

We're receiving this error too. Increasing to 1G in php.ini worked.

.json below...

{
"name": "symfony/framework-standard-edition",
"description": "The "Symfony Standard Edition" distribution",
"autoload": {
"psr-0": { "": "src/" }
},
"require": {
"php": ">=5.3.3",
"symfony/symfony": "2.1.",
"doctrine/orm": ">=2.2.3,<2.4-dev",
"doctrine/doctrine-bundle": "1.0.",
"twig/extensions": "1.0.",
"symfony/assetic-bundle": "2.1.",
"symfony/swiftmailer-bundle": "2.1.",
"symfony/monolog-bundle": "2.1.",
"sensio/distribution-bundle": "2.1.",
"sensio/framework-extra-bundle": "2.1.",
"sensio/generator-bundle": "2.1.",
"jms/security-extra-bundle": "1.2.",
"jms/di-extra-bundle": "1.1.",
"dholmes/error-handling": "master",
"dholmes/doctrine-extras": "master",
"friendsofsymfony/jsrouting-bundle": "1.0.2",
"amazonwebservices/aws-sdk-for-php": "1.5.15",
"beberlei/DoctrineExtensions": "v0.1",
"doctrine/doctrine-fixtures-bundle": "dev-master",
"EHER/phpunit": ">=1.2",
"phake/phake": "dev-master",
"stripe/stripe-php": "v1.7.7",
"imagine/Imagine": "dev-master",
"michelf/php-markdown": "1.2.5",
"cboden/Ratchet": "0.2.",
"react/zmq": "dev-master"
},
"repositories": [
{
"type": "vcs",
"url": "https://github.com/danielholmes/error-handling"
},
{
"type": "vcs",
"url": "https://github.com/danielholmes/doctrine-extras"
},
{
"type": "package",
"package": {
"name": "michelf/php-markdown",
"version": "1.2.5",
"source": {
"url": "git://github.com/michelf/php-markdown.git",
"type": "git",
"reference": "origin/extra"
}
}
}
],
"scripts": {
"post-install-cmd": [
"Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::buildBootstrap",
"Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::clearCache",
"Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::installAssets",
"Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::installRequirementsFile"
],
"post-update-cmd": [
"Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::buildBootstrap",
"Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::clearCache",
"Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::installAssets",
"Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::installRequirementsFile"
]
},
"minimum-stability": "dev",
"extra": {
"symfony-app-dir": "app",
"symfony-web-dir": "web"
}
}

[mac-cain13]

I also can confirm this issue, after updating composer I can't update our projects without raising the memory_limit from 512MB to 1GB. Seems that memory usage increased quite a lot. Here is one of our composer.json files that are problematic:

{
    "name": "symfony/framework-standard-edition",
    "description": "The \"Symfony Standard Edition\" distribution",
    "autoload": {
        "psr-0": { "": "src/" }
    },
    "require": {
        "php": ">=5.3.3",
        "symfony/symfony": "2.1.*",
        "doctrine/orm": ">=2.2.3,<2.4-dev",
        "doctrine/doctrine-bundle": "1.0.*-dev",
        "twig/extensions": "1.0.*-dev",
        "symfony/assetic-bundle": "2.1.*-dev",
        "symfony/swiftmailer-bundle": "2.1.*-dev",
        "symfony/monolog-bundle": "2.1.*-dev",
        "sensio/distribution-bundle": "2.1.*-dev",
        "sensio/framework-extra-bundle": "2.1.*-dev",
        "sensio/generator-bundle": "2.1.*-dev",
        "jms/security-extra-bundle": "1.2.*",
        "jms/di-extra-bundle": "1.1.*-dev",
        "friendsofsymfony/rest-bundle" : "0.10.*-dev",
        "doctrine/doctrine-migrations-bundle" : "dev-master",
        "richsage/rms-push-notifications-bundle" : "*",
        "imagine/Imagine": "dev-develop",
        "stof/doctrine-extensions-bundle": "dev-master",
        "mlpz/postmark-bundle": "*",
        "jms/serializer-bundle" : "dev-master",
        "craue/twigextensions-bundle": "dev-master",
        "wrep/bugsnag-php-symfony" : "dev-master",
        "wrep/notificato-symfony" : "1.0.*",
        "wrep/daemonizable-command": "dev-master"
    },
    "scripts": {
        "post-install-cmd": [
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::buildBootstrap",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::clearCache",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installAssets",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installRequirementsFile"
        ],
        "post-update-cmd": [
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::buildBootstrap",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::clearCache",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installAssets",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installRequirementsFile"
        ]
    },
    "config": {
        "bin-dir": "bin"
    },
    "minimum-stability": "dev",
    "extra": {
        "symfony-app-dir": "app",
        "symfony-web-dir": "web"
    }
}

I just added the wrep/daemonizable-command package and runned composer update wrep/daemonizable-command, but that crashed with this error:

Loading composer repositories with package information
Updating dependencies (including require-dev)
PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 32 bytes) in phar:///usr/bin/composer/src/Composer/DependencyResolver/RuleWatchGraph.php on line 52
PHP Stack trace:
PHP   1. {main}() /usr/bin/composer:0
PHP   2. require() /usr/bin/composer:15
PHP   3. Composer\Console\Application->run() phar:///usr/bin/composer/bin/composer:43
PHP   4. Symfony\Component\Console\Application->run() phar:///usr/bin/composer/src/Composer/Console/Application.php:83
PHP   5. Composer\Console\Application->doRun() phar:///usr/bin/composer/vendor/symfony/console/Symfony/Component/Console/Application.php:119
PHP   6. Symfony\Component\Console\Application->doRun() phar:///usr/bin/composer/src/Composer/Console/Application.php:117
PHP   7. Symfony\Component\Console\Application->doRunCommand() phar:///usr/bin/composer/vendor/symfony/console/Symfony/Component/Console/Application.php:212
PHP   8. Symfony\Component\Console\Command\Command->run() phar:///usr/bin/composer/vendor/symfony/console/Symfony/Component/Console/Application.php:882
PHP   9. Composer\Command\UpdateCommand->execute() phar:///usr/bin/composer/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:244
PHP  10. Composer\Installer->run() phar:///usr/bin/composer/src/Composer/Command/UpdateCommand.php:102
PHP  11. Composer\Installer->doInstall() phar:///usr/bin/composer/src/Composer/Installer.php:208
PHP  12. Composer\DependencyResolver\Solver->solve() phar:///usr/bin/composer/src/Composer/Installer.php:446
PHP  13. Composer\DependencyResolver\RuleWatchGraph->insert() phar:///usr/bin/composer/src/Composer/DependencyResolver/Solver.php:170
PHP  14. SplDoublyLinkedList->unshift() phar:///usr/bin/composer/src/Composer/DependencyResolver/RuleWatchGraph.php:52

Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 32 bytes) in phar:///usr/bin/composer/src/Composer/DependencyResolver/RuleWatchGraph.php on line 52

Call Stack:
    0.0035     630856   1. {main}() /usr/bin/composer:0
    0.0091     817720   2. require('phar:///usr/bin/composer/bin/composer') /usr/bin/composer:15
    0.0454    5095992   3. Composer\Console\Application->run() phar:///usr/bin/composer/bin/composer:43
    0.0502    5671944   4. Symfony\Component\Console\Application->run() phar:///usr/bin/composer/src/Composer/Console/Application.php:83
    0.0519    5917824   5. Composer\Console\Application->doRun() phar:///usr/bin/composer/vendor/symfony/console/Symfony/Component/Console/Application.php:119
    0.0530    6057016   6. Symfony\Component\Console\Application->doRun() phar:///usr/bin/composer/src/Composer/Console/Application.php:117
    0.0544    6057016   7. Symfony\Component\Console\Application->doRunCommand() phar:///usr/bin/composer/vendor/symfony/console/Symfony/Component/Console/Application.php:212
    0.0544    6057016   8. Symfony\Component\Console\Command\Command->run() phar:///usr/bin/composer/vendor/symfony/console/Symfony/Component/Console/Application.php:882
    0.0555    6055984   9. Composer\Command\UpdateCommand->execute() phar:///usr/bin/composer/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:244
    0.2593   13311072  10. Composer\Installer->run() phar:///usr/bin/composer/src/Composer/Command/UpdateCommand.php:102
    0.2611   13478184  11. Composer\Installer->doInstall() phar:///usr/bin/composer/src/Composer/Installer.php:208
    3.2977   53035864  12. Composer\DependencyResolver\Solver->solve() phar:///usr/bin/composer/src/Composer/Installer.php:446
   53.1716  536465808  13. Composer\DependencyResolver\RuleWatchGraph->insert() phar:///usr/bin/composer/src/Composer/DependencyResolver/Solver.php:170
   53.1716  536466480  14. SplDoublyLinkedList->unshift() phar:///usr/bin/composer/src/Composer/DependencyResolver/RuleWatchGraph.php:52

[jameshalsall]

👍 I'm also receiving this issue on PHP 5.3.15 on Mac OSX. Memory limit is at 512mb and never had a problem in the past (until updating composer a couple of days ago).

EDIT: I can confirm that upping a 1gb limit fixes, but surely an issue.

Composer JSON:

{
    "autoload": {
        "psr-0":         {
            "": "src/"
        }
    },
    "config": {
        "bin-dir": "bin"
    },
    "description": "Tickit Application",
    "extra": {
        "symfony-app-dir": "app",
        "symfony-web-dir": "web"
    },
    "minimum-stability": "dev",
    "name": "tickit-project/tickit",
    "require": {
        "doctrine/doctrine-bundle": "1.2.x-dev",
        "doctrine/doctrine-fixtures-bundle": "dev-master",
        "doctrine/orm": "2.3.*",
        "friendsofsymfony/user-bundle": "2.0.x-dev",
        "jms/di-extra-bundle": "1.3.x-dev",
        "jms/security-extra-bundle": "1.4.x-dev",
        "php": ">=5.3.3",
        "sensio/distribution-bundle": "2.3.x-dev",
        "sensio/framework-extra-bundle": "2.3.x-dev",
        "sensio/generator-bundle": "2.3.x-dev",
        "stof/doctrine-extensions-bundle": "1.1.x-dev",
        "symfony/assetic-bundle": "2.1.x-dev",
        "symfony/monolog-bundle": "2.2.x-dev",
        "symfony/swiftmailer-bundle": "2.2.x-dev",
        "symfony/symfony": "2.3.x-dev",
        "symfony/console": "2.3.x-dev",
        "twig/extensions": "1.0.x-dev",
        "psr/log": "1.0.0",
        "tickit-project/cache-bundle": "1.0.x-dev",
        "squizlabs/php_codesniffer": "1.*",
        "behat/symfony2-extension": "dev-master",
        "satooshi/php-coveralls": "dev-master"
    },
    "scripts": {
        "post-install-cmd": [
"Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::buildBootstrap",
"Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::clearCache",
"Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installAssets",
"Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installRequirementsFile"
    ],
        "post-update-cmd": [
"Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::buildBootstrap",
"Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::clearCache",
"Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installAssets",
"Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installRequirementsFile"
    ]
    }
}

[mddubs]

Same issue since updating Composer. 1GB workaround does work, but something must be wrong.

Composer: 950fc7e
Mac OSX: 10.8.3
PHP: 5.3.15

[michaldudek]

Now I've hit 2GB memory limit ;) I'm getting 16GB RAM tonight so gonna increase the php memory limit to around 10, should last for few months of updates :P

[michaldudek]

Updating my CLI to php54 fixed it for now.

[robocoder]

Yup, upgrading to php5.4.x fixes a memory issue when using php 5.3.x. For example, I was running composer install without a .lock file. With php5.3.10, composer required over 3GB of memory. With php5.4.16, composer consumed less than 1GB.

[Dynom]

Needing 512MiB RAM for a package-manager is already a bit much, but I also exceed it. The interesting part is that I exceed 512 MiB RAM (up to around 800 MiB RAM according to my process output) when I use my private Satis repository (which is actually just a cache for all Github public repositories). I do not exceed it otherwise. Just by adding my satis repository, composer consumes almost 400 MiB more RAM.

I'm on PHP 5.4.14.

To all the others raising their PHP CLI memory_limit globally, I recommend against that. Instead I suggest to put it in the command line, like so:

php -dmemory_limit=1G bin/composer.phar update .. 

Otherwise you could potentially miss certain unintended memory related behaviour in your own work.

[tyler-sommer]

Chiming in with my experience. I'm also using a private Satis repository in addtion to the zend framework repository. I was having problems with composer update on PHP 5.3.26 using over 2GB of memory.

Switching to 5.4 solved everything, now using under 400MB.

[jameshalsall]

ping @Seldaek

[Taiger]

Thanks @Dynom, this works for those of us on PHP <5.4

php -dmemory_limit=1G composer.phar update

[jjanvier]

Same problem

[iVariable]

Same here. Mac OS X. PHP 5.3.15.
Fixed by updating php to 5.4.10.

[baohx2000]

Happening to me on 5.5. Guessing something somewhere is not cleaning up after itself and leaving objects hanging around?

[deguif]

Same problem with php 5.3.15 on Mac OS X.

[atmediauk]

Same problem here. I can't get it to composer update as I only have 1gb total for this particular vps.

PHP 5.4.11 (cli) (built: Feb 20 2013 19:02:54)

----> Update
Not sure if it helps but I noticed that my local php version was: 5.4.19 so although I had 5.4 on the production server it was a few updates out.

I updated the production server from 5.4.11 to 5.4.19 and composer update seems to be working now.

[kmmathis]

Had this issue until I upgraded to PHP 5.4.19. composer update now appears to work.

[IAkumaI]

I'm also have a problem, but yesterday it worked fine with the same composer.json

[MPLLC]

Ugh, this is a killer on shared hosting.

[Seldaek]

@kevinm1 it's not, you are not supposed to run composer update on your production server. You should run update on your dev box then deploy the composer.lock file and run install which will run with very low memory footprint and very fast since it doesn't have to check for new packages but merely applies what's in the lock file.

[Imunhatep]

Yep, same here - have to increase memory limit to 1G. And indeed it eats up to 980M.

Interesting thing is that this started to happen only after I added local Satis composer repository which in fact holds only cached .zip packages mostly if not only from github.

Without local Satis repository, composer uses around of 240mb.
And if use local Satis repository with turned off packagist it also uses around 240mb. Example:

"repositories": [
{ "type": "composer", "url": "http://packagist.example.com" },
{ "packagist": false }
],

Happens on Gentoo with PHP 5.4.16

[kureikain]

I was on PHP 5.5.5. Cannot solve this. Tried to disable lots of extension: opcode, xdebug,..I actually ended up creating swap file. My VPS is 512MB (a cheap digital ocean instance) with a handful of service: nginx, mariadb, mongodb, redis. puma.

to clarify, I was not able to run a simple single command for installing laravel

composer create-project laravel/laravel

Here is my simple work around, hopefully help s.o:

df -h 
dd if=/dev/zero of=/swapfile bs=1M count=1024
sudo dd if=/dev/zero of=/swapfile bs=1M count=1024
mkswap /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
echo 'echo "/swapfile  none  swap  defaults  0  0" >> /etc/fstab' | sudo sh

free -m

confirm u see your swap there:
total used free shared buffers cached
Mem: 494 335 158 0 19 62
-/+ buffers/cache: 254 240
Swap: 1023 3 1020

Then install composer, and wat your RAM ;(

watch free -m

[kmmathis]

Most of the people getting errors related to this ticket seem to be trying to create projects and/or updating with composer on their servers. Best practice is running your composer create-project and composer update commands locally, using git to pull your project updates to the server, and then simply running composer install on the server to update your production dependancies.

[kureikain]

I did what u said, and it fail right on that command...I actually ended up creating a swap file to expand memory..

[uwej711]

While php -dmemory_limit=1G composer.phar update is ok, it is rather a bit to type (especially if you install it to /usr/bin). Any thoughts about passing the memory limit optionally directly to composer, something like

composer --memory=1G update

[mrchimp]

@kmmathis I'm getting the same problem when I do composer install.

[Glideh]

composer install also gives me the same issue (PHP 5.3.3).
@uwej711 you can still php -dmemory_limit=xxx /usr/local/bin/composer ...
It passes with 512M memory_limit

[nfx]

maybe there is a substantial problem building dependency graph, that needs to be addressed?...

[Dynom]

I think you might be on to something there @nfx ;-)

I'd like to remind everyone, including myself, that Composer is OSS and can be forked and improved by yourself!

[frangeris]

Same problem here, memory_limit = 512M;, the only way to solve this is increasing the memory? another way?

[mrchimp]

@frangeris Increasing swap space seems to have helped.

[euskadi31]

👍

[djlambert]

I received this error with PHP 5.5.6 x64 on Win7, memory_limit = 1G resolved the issue. 512M did not.

[jkup]

Same as others, 1G solved my issue but even at 512M it had errors.

[lenybernard]

Same problem ;-(

[musixite]

@Seldaek "deploy the composer.lock file and run install"

I know this is probably basic for you but could you please tell me how to do this on the command line?
I have an idea but I wouldn't want to screw up :)

Thanks!

• Vincent

[mrchimp]

@musixite The composer.lock is basically a copy of composer.json (with some timestamps added). When you run composer update it will attempt to get new versions of your dependencies and will then update the composer.lock file with these version numbers. This takes quite a lot of memory.

composer install on the other hand will look in composer.lock and install the versions specified there. This takes less memory.

So update should only be used when developing as it could break things. Only install should be used in production as it will only install things that have (hopefully!) been tested.

What @Seldaek is saying is that you should push to your project to your production server with a composer.lock file attached and then run composer install.

Hope that helps!

[geoffreytran]

Running into this issue also. There shouldn't be a reason for composer to need this much memory.

[proligde]

Same here with 512M RAM using PHP 5.5.3

[judgej]

Getting this problem on our dev server too, for the first time. Trying to set the memory limit in the command line just gives us this:

$ php -dmemory_limit=1G composer.phar update 
Loading composer repositories with package information
Updating dependencies (including require-dev)
Killed
$

However, running the command about six times eventually completed without an error. I don't know if each run leaves enough cached data behind to pick up where it left off, working further on each run, or something else was affecting how much memory it needed.

[Seldaek]

@judgej I guess you are just at the edge of your server's memory so it gets killed or not depending on the amount of ram used when you start composer. Anyway you should avoid running update on servers at all, rather commit the composer.lock file and run install on servers, which uses very little memory.

[judgej]

Servers are where I kind of do my development. The composer.lock and install steps I can understand for migrating to other server instances - test/staging/production/whatever - but am I in the minority doing work on a "server" rather than my own laptop or workstation? I have too many things on the go at once to work any other way (or maybe I need a bigger laptop with a zillion VMs :-).

On the plus side, at least I know I'm not blowing the 1G limit by too much ;-)

[judgej]

So out of curiosity, is there a fundamental reason why an update has to use that much memory, or is it something that is lower on the priorities to fix or optimise because people should only be doing this on a machine with enough resources? Just wondering if there is hope for using less memory (with fixes by whoever), or if it ain't gonna happen (because it can't) and it must be handled by the resources we must throw at it.

[Seldaek]

@judgej I don't know if you are in the minority or not, I think it depends on the sort of demographic you are looking at. In the people I know doing mostly the things I do, most work on their own laptops, either in VMs or in the native OS. That usually gives you more allowances regarding memory. I'm sure other communities/groups tend to still rely more on dev servers. By the way the Killed line does not mean it reached the php limit (1G) but that the OS ran out of memory and decided to kill that process since it's hoarding all of it.

As for why it's using so much, unfortunately it's not such an easy fix, I wish I'll have time to dedicate to that some day, but right now it's using lots because it loads tons of packages in memory and php isn't the most memory efficient beast out there. Upgrading to php 5.4 or even 5.5 should help on that front by the way, 5.3 is quite wasteful.

[judgej]

Being able to see how memory is being used graphically would be cool, one day, for seeing just where it all goes.

I kind of guessed the Killed came from a different source, as PHP did not have a chance to report any reason for aborting. Just ZAP - you're dead.

Thanks for spending your time explaining, much appreciated :-)

[froddd]

Seeing all this, is there much point in excluding the lock file from version control? If we're encouraging install rather than update on deployed servers (testing/staging/production), surely the lock file should always be deployed?

[stof]

@froddd there is strictly no point about excluding the lock file from version control for projects (and the doc advices to version it). If you run composer update during your deployment process, you have strictly no guarantee that you will get the same dependencies than during your local development or your CI build (you are likely to get newer versions). This means that running composer update on deployment implies deploying untested code.
The only case where it can make sense to exclude the lock file is when developing a library which is only meant to be used as a dependency (so its lock file will impact only its own development and testing), and you want your CI server to get the newest version of the dependencies each time

[TehWan]

Could it be possible to add to the system requirements a note about the high memory usage? @judgej is not the only one who needs to run it on low memory systems, and most of my VMs usually don't have 512MB+ free for PHP, it's more in the range of 128-256MB. We don't all have high-end computers in an enterprise environment. ;)

[judgej]

I actually had to copy the composer.json to a production server with lots of memory, run composer update on it, delete the vendor directory after it had fetched all its packages, then copy the resulting composer.lock file back to my development machine to run a composer install. That worked, but is far from ideal.

I have been thinking about these memory issues for the last few days. Is there a discussion or overview somewhere that lays out the issues that is causing the memory usage to be so high? I'm wondering if we understand the problems more, we may be able to help. Is there a way to work out the dependencies without opening up all the packages in memory, i.e. just getting hold of the composer.json files for each package without the rest of the files it contains (I'm assuming composer.json is all that is needed - but I may be wrong)?

Or could packages be extracted to a temporary folder rather then into memory, using a shell command (when run on Linux), since the shell command does not use PHP's memory allocation and memory is cleaned up a little better when the shell command ends.

This is becoming a bigger issue as more frameworks use composer for all their packages. I'd like to help, so if there are any pointers on where to get started (a description of perhaps why this is not a trivial problem to solve) then that would help a lot.

[stof]

@judgej a Package object for Composer contains only the information of the composer.json file (and not even all of them during the solving of dependencies). Composer does not download the packages themselves until it install them. your suggestion are all talking about moving the extraction of packages out of PHP, while the issue comes from the amount of extracted data which is needed by the PHP code

The issue is that the algorithm resolving dependencies needs to be fed with all potential matches for a constraint, and when you require ~2.0 for symfony/symfony, it already represents 65 packages only for it (without considering dependencies of symfony yet) due to the number of released versions supporting composer, and twice more when requiring symfony/console (as symfony/symfony needs to be considered as well).
As you can see, this requires loading lots of information in memory. Some memory optimizations have already been implemented to limit the data loaded in the algorithm first (avoiding to include the 2.0 packages if you require ~2.1 for instance), but optimizing it further (while keeping the really complex algorithm free of bugs) is a really complex task.

We are talking about optimizing the most complex part of composer here, for which a long time of development was already used (to give you a quick idea, during the first months of development of Composer, @naderman worked quite only on the dependency solver while @Seldaek was building the surrounding infrastructure).

[judgej]

@stof Thanks, that makes a lot of sense. So it is not necessary to download a package to see its metadata (composer.json), which is good. However, you need to download the full (-ish, depending on constraints) history of each package so the information needed to make the right dependency choices is available. I can see there would be many possible combinations to work through, with a lot of data, and PHP is not good at tidying up memory as it goes along.

How does this stuff all fit in one tiny script? ;-) Incredible work.

I wonder how it would work if taken online, with a database to manage the data? Just composer.json in and composer.lock out? A throw-away thought...but I've got to ask the stupid questions.

[tantam]

get similar issue, when i try to run "composer update" with "minimum-stability": "dev" , i just change to "minimum-stability": "stable" and it's work :)

[ChristopheCubat]

Hitting 3G now... at this pace... (1G a month)... I will be need to change my computer in 4 month... :-S

[baohx2000]

HHVM can run phar files now. One of my coworkers tried it out with composer and it used less cpu, time, and memory.

[naderman]

Indeed, if you have HHVM available, that'll help a lot.

[stof]

yeah, HHVM improves things a lot (especially regarding time as it is far more efficient at running complex computations)

[svenvarkel]

Same problem here. To all those who recommend to increase memory limit - memory leaks cannot and must not be fixed by throwing more memory at it.

[naderman]

Well it's not a leak. It simply needs a lot of memory.

[BevanR]

I had to increase to 1024-2048MB for

{
    "require": {
        "drupal/drupal-extension": "dev-master"
    },
    "require-dev": {
        "igorw/composer-yaml": "*"
    },
    "minimum-stability": "dev",
    "prefer-stable": true
}

[TheAifam5]

I have problem. Composer use all of available ram(6GB) and 1GB Swap(max: 7GB) and freeze system... My linux lost stability and i must restart ...
I have Arch Linux with linux-ck kernel.

[TheAifam5]

Please add information to disable XDebug(zend plugin)... After disabled, Composer work perfectly, without timeout, without too much eat RAM, out of memory and freezing system. ;)

[namreg]

thanks @theaifam5!
you save my day!

[joelrotelli]

@theaifam5 How do you disable xdebug please ?

Thanks

[BevanR]

Comment out any lines mentioning it in your php.ini file.

[namreg]

@joelrotelli you need just disable profiling feature in xdebug.
that's my debug config

zend_extension="xdebug.so"
xdebug.idekey=PHPSTORM
xdebug.profiler_enable=0
xdebug.profiler_enable_trigger=1
debug.remote_autostart=on
xdebug.remote_port=9001
xdebug.remote_enable=on
xdebug.remote_handler="dbgp"

[joelrotelli]

OK thanks.

Passing to PHP5.4 solve my problem too.

[TheAifam5]

so... if you used PHP Web Server, copy a php.ini from /etc/php to current directory(eg: your website). Run(if need, edit path and IP): php -S 127.0.0.1:80 -t "/home/theaifam5/www/public" -c ./php.ini
Edit copied php.ini, put all from @namreg .
Ok.. now server read local file php.ini because you specified which file.

So, now you must disable XDebug globally in /etc/php/php.ini. Replace line zend_extension="xdebug.so" to ;zend_extension="xdebug.so". If line not exist, check this path(exists on Arch Linux): /etc/php/conf.d/xdebug.ini and replace ;)

So. Globally XDebug is disabled, but for server have own configuration with enabled XDebug.. XDebug is enabled only for webserver because using own configuration.

If you using PHPStorm, add to your configration in PHP Built-in Web Server to Interpreter options: -c php-xdebug.ini.
Now PHPStorm start a webserver with XDebug, but all commands typed in console(everywhere, and in PHPStorm) starting PHP without XDebug..

:)

Yea, nice commit: 10f8e56

Shit not working(commit), no showing any warnings about XDebug ;p Bravo !

[ammont]

Same problem here

{
    "name": "laravel/laravel",
    "description": "The Laravel Framework.",
    "keywords": ["framework", "laravel"],
    "license": "MIT",
    "require": {
        "laravel/framework": "4.1.*",
        "phpunit/phpunit": "3.7.*",
        "barryvdh/laravel-debugbar": "1.*",
        "ammont/fa-slugify": "dev-master",
        "miladr/jalali": "dev-master",
        "intervention/image": "dev-master",
        "cviebrock/image-validator": "1.0.*"
    },
    "autoload": {
        "classmap": [
            "app/commands",
            "app/controllers",
            "app/models",
            "app/database/migrations",
            "app/database/seeds",
            "app/tests/TestCase.php",
            "app/Common.php"
        ]
    },
    "scripts": {
        "post-install-cmd": [
            "php artisan clear-compiled",
            "php artisan optimize"
        ],
        "post-update-cmd": [
            "php artisan clear-compiled",
            "php artisan optimize",
            "php artisan debugbar:publish"
        ],
        "post-create-project-cmd": [
            "php artisan key:generate"
        ]
    },
    "config": {
        "preferred-install": "dist"
    },
    "minimum-stability": "dev",
    "prefer-stable": true
}

[nvartolomei]

same problem with php56 installed via homebrew with fpm and debug, it was ok with php shipped with yosemite

[wiesson]

same here as @nvartolomei (vagrant/puphpet, php56, ubuntu14.04, windows)

[ikari-pl]

@wiesson and others using laravel: try applying what @barryvdh suggested to me: #1898 (comment) — directly adding symfony subdependencies from laravel dramatically reduces the memory usage for this command. For me it worked like a charm.

[energylevels]

I dunno why this needs so much memory but makes it a non starter on limited shared hosting environments ....

[MPLLC]

If we're supposed to only commit lock files to production, why, then, does composer not allow the installation of vendor packages without the json file?

And, again, this memory issue is killing me on shared hosting. Something needs to be done, as it's ridiculous to ask people to increase memory when they might not be able to.

[cjunge-work]

@kevinm1 I think you've misunderstood the suggestion. The composer.json file must always been committed, but the lock file should also be committed as it is used in a composer install.

If the composer.lock file does not exist, then using composer install essentially runs an update, and saves the generated dependency map to the lock file. composer install then uses the composer.lock file to install the dependencies, but it also does some sanity checking to see if the composer.json file has changed.

Hope that helps.

[nvartolomei]

Both files are needed.

You should run update only on dev machine (environment).

Problems like this are not always meant to be fixed by the way, hardware are better now so we can use more of it, or if you want you can rewrite entire composer in assembler 😶.

On 19 nov. 2014, at 00:30, Kevin Major notifications@github.com wrote:

If we're supposed to only commit lock files to production, why, then, does composer not allow the installation of vendor packages without the json file?

And, again, this memory issue is killing me on shared hosting. Something needs to be done, as it's ridiculous to ask people to increase memory when they might not be able to.

—
Reply to this email directly or view it on GitHub.

[moafred]

Thanks to @nasr, disable xdebug fix this for me

Disable xdebug while running the "composer update" command.
The xdebug cause this slowness for me.

[Seldaek]

Recent patches improve memory usage slightly (0-50% reduction it seems depending on packages), but if you have large sets of packages it might still use a lot, and using a composer.lock to deploy to production (especially if prod has low memory like shared hosting) is still the best practice, not just for performance reasons.

[CaptainN]

I had this problem the other day. It turned out to have nothing to do with any real bug, it's just that the documentation on the composer website about how to set up composer globally on OSX using brew doesn't tell you that brew installs an old outdated version of composer, and that you have to update it with: composer selfupdate

After running composer selfupdate I didn't have this bug anymore. It took quite a bit of searching to figure that out. It would probably be easier, and maybe even reduce reports, if the composer getting started page explained this.

[startinggravity]

Wonderful catch, and so timely too! Thanks, @CaptainN. Your solution fixed the memory problem I ran into while installing Codesniffer.

[matyhaty]

Running

sudo composer selfupdate

Fixed it for me. I was having memory issues even when set to 5120000M!!!

[collinanderson]

==> /usr/local/bin/composer install hangs for me when running brew install wp-cli. selfupdate doesn't seem to help.

[jeankleemann]

I had same problem with:
composer create-project sylius/sylius-standard -s dev
solved for me with:
php -dmemory_limit=2G /usr/local/bin/composer.phar create-project sylius/sylius-standard -s dev

[seanlindo]

@CaptainN This worked perfectly.

[mrofi]

I have same issue when I would install laravel 4.2 on my openshift gear.
Even I use a small gear that only has 512 MB RAM, this solved the issue
php -dmemory_limit=2G /var/lib/openshift/.cartridge_repository/redhat-php/0.0.23/usr/bin/composer.phar install

[velikanov]

@Seldaek 👍 👍 👍 for running composer install on production!
thank you man!

[mailightkun]

This solved my problem base on their Suggestion:
https://getcomposer.org/doc/articles/troubleshooting.md#proc-open-fork-failed-errors

[phirschybar]

this worked for me as well: https://getcomposer.org/doc/articles/troubleshooting.md#proc-open-fork-failed-errors

[kumarramalingam]

Solve this problem.

Update PHP Version.
Ubuntu Terminal window:

1.sudo apt-get update

2.sudo apt-get install lamp-server^

That's all.

[cbrunnkvist]

"You're on the wrong server" / "You don't have enough swap" / "Framework X is an edge-case" / "Problems like this are not always meant to be fixed"?? Whaaaat, seriously? Is everybody (with the brilliant exception of @stof and @judgej) turning into total apologists? 😟

The issue still remains: something is wrong, because it is insane that such an important and central tool such as the dependency manager, should gobble up gigglypiles of memory just in order to complete it's most basic tasks.

[cbrunnkvist]

For reference: mem usage stays stable on my Yosemite Mac (PHP 5.5.20) but totally blows up on my Ubuntu Trusty deployment (PHP 5.5.9-1ubuntu4.9). None of the workarounds above has any effecty ~ it must be some triggering some leak in that older ("stable") version of PHP.

[sbuzonas]

The issue still remains: something is wrong, because it is insane that such an important and central tool such as the dependency manager, should gobble up gigglypiles of memory just in order to complete it's most basic tasks.

Dependency resolution is not it's most basic task, on the contrary it is the most complex task. Being such a critical piece of the application, it is rather difficult to make improvements to. Servers generally have less memory than a development machine and it is recommended to do the update in your development environment and simply do an install from the lock on a server.

The solver is computing a solution for a graph theory equation. It basically does a tsort with an unbounded set discovering additional dependencies to consider for resolution throughout the process. The larger the graph the more memory required to compute the result. When you introduce potential nodes to the graph via discovery through a service, like packagist, that provides a large number of alternatives it increases the complexity and required memory/compute time. A traditional tsort algorithm has a complexity of O(log^2 n).

[cbrunnkvist]

I would stress that my use of "basic" is not in the sense of "simplistic and of low complexity" - but in the meaning that this task is quintessential, and critical, to the goals of the program.

If we can conclude that there is nothing obviously wrong with the algo. implementation itself, then one way to start dealing with the problem could be to build up a dataset of known PHP version where bloat occurs and at the very least display a warning at execution time. Ideas?

[sbuzonas]

It isn't a specific PHP version that causes bloat, it is a factor. Packages can have dependencies on different PHP versions. If you have a 5.3 installation of PHP you are eliminating candidates of packages that require php >5.4.

The largest factor is the dependencies themselves. For instance "symfony/symfony": "^2.7" is a much simpler constraint because it performs a replaces for all of it's subpackages with self.version... this eliminates a huge subset of possibilities because each sub package does not need to be evaluated.

The bloat generally occurs whenever you have a dependency somewhere down the line that has a large number of potential candidates. For instance, if you only require a subset of symfony packages they will need to be calculated individually to satisfy each constraint and dependency. If you drop that constraint down to ^2.3 you introduce a significant number of possibilities to evaluate.

The ideal goal of maintaining backwards compatibility with other open source packages, a good motivation IMO, is the main killer of the memory utilization of composer. When a new version is released that meets your constraint it needs to be evaluated.

[cbrunnkvist]

"Packages can have dependencies on different PHP versions"

💡 Aaah darn, good point...

[mpemberton5]

I turned off xdebug and still had problem. Turned off OpCode for cli and worked like a dream (and fast too).

[orhan-mobven]

I turned off xdebug and it works now.

[fabianhenzler]

Having it with HHVM too

[ronnyandre]

Thanks @kureikain
Your solution solved my problem! 👍

[chrisg123]

I was having this issue. Updating composer (composer self-update) seems to have solved it for me...

[Aldibe]

i got this error when running composer.phar update. and i think it is similar to this issue, but i don't know how to fix it..

PHP Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to allocate 144115188075867549 bytes) in phar:///bin/composer.phar/src/Composer/Util/RemoteFilesystem.php on line 179

this is my composer.json

{ "description" : "The CodeIgniter framework", "name" : "codeigniter/framework", "license": "MIT", "require": { "php": ">=5.2.4", "videlalvaro/php-amqplib": "2.5.*" }, "require-dev": { "mikey179/vfsStream": "1.1.*", "videlalvaro/php-amqplib": "2.5.*" } }

is it possible/normal for composer to allocate that much memory??

[stof]

during an update, it can indeed use lots of memory

[kkomelin]

I might have missed some of the comments above but since this issue is still open I'll add my two coins.

My system:
VPS: 1Gb RAM, Ubuntu, ssd, no swap (I don't need it)

My situation:
I successfully updated composer to the latest version using self-update command. Then I tried to update Drush to the latest but received the memory error discussed in this issue.
I used the following command: composer global require drush/drush

I did this way because it's required by the official Drush documentation which is used by thousands so the suggestion to use composer on my local machine and then move to Git is not for me obviously.

Of course, I can temporary increase memory to 1G but folks... really... it's too much for one utility tool and it's probably not a PHP problem but a problem of the composer architecture.

So, maintainers please think about this when you're planning roadmap for future versions because I believe that no one utility tool deserves 1G of memory even if it's local machine memory.
Thanks.

[stefandoorn]

@barryvdh Ain't your suggestion more like this, just a suggestion:

• Composer update
• Composer copies composer.json to composer.json.tmp
• Get all dependencies from root projects and add to require part in composer.json.tmp
• Internally run composer update again but then using the temporary file, which seems faster and less memory in your examples
• After update, remove composer.json.tmp file and write composer.lock file

[TengfeiQi]

这个问题好蛋疼

Fatal error: Uncaught exception 'ErrorException' with message 'proc_open(): fork failed - Cannot allocate memory' in phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:940

[SiliconMind]

I'd consider this more than serious. I just tried to install Symfony CMF. I needed to increase memory limit to 2G in order to avoid constant out of memory issues. Come one guys! There is something seriously wrong with the composer. I'm managing dozens of dev machines and it was the first time for past few years that I had to increase it's memory over 1G and for what? A composer install?

With recent boom for small cloud dev machines this will become a serious issue.

[judgej]

@SiliconMind Be more specific about the version(s) you want to install. That is the key. Don't give composer the option to have to investigate every possible combination of every single package that is needed to install your framework; if you do, then it will be trying out many thousands of possible combinations to find the perfect latest compatible set of packages available, and it is that which eats memory.

[staabm]

Did you check the tipps mentioned at https://getcomposer.org/doc/articles/troubleshooting.md#memory-limit-errors ?

[staabm]

Also it would be very helpfull when you could create a blackfire profile of your run (after you increased php-memory-limit to a value high enough that it doesnt crash)

[SiliconMind]

@judgej You know, I can handle myself a lot but I don't think that's the point. Consider someone less experienced that visits Symfony's CMF install instructions and sees this: composer create-project symfony-cmf/standard-edition cmf '~1.2'.

You can't expect that average John Developer will know composer inside out. And frankly, the versions for Symfony CMF are pretty precise. Maybe related packages are not specific enough, but you can't expect that all end users will now start to edit related composer.json files in hope of finding the source of the issue.

[SiliconMind]

@staabm xdebug was disabled. It was vanilla debian install without any additional stuff. As for the profile - I'm sorry, I can't do this now.

[garethmcc]

I have been plagued by this issue for over two years now and the official response is still "add swap"? This should not be a problem and its ridiculous that it isn't resolved yet. If a memory issue such as this made it into anything I released I would lose my job.