New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Composer update runs out of memory #1898

Closed
jrooi opened this Issue May 14, 2013 · 181 comments

Comments

Projects
None yet
@jrooi

jrooi commented May 14, 2013

Running the "composer update" command results in a fatal error:

Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 79 bytes) in phar:///usr/local/bin/composer/src/Composer/DependencyResolver/RuleWatchGraph.php on line 100

Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 71 bytes) in phar:///usr/local/bin/composer/src/Composer/DependencyResolver/Solver.php on line 700

Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 72 bytes) in phar:///usr/local/bin/composer/src/Composer/DependencyResolver/RuleWatchGraph.php on line 104

I already tried removing the vendors in my Symfony2 project. After removing the vendors and running a "composer install" command I get no errors. Running a "composer update" directly after the install results in the same memory error. The current memory limit is set to 512MB, that should be enough, right?

What can be the cause of the sudden memory issue? Can I take other steps in solving this issue? As far as I can tell, the dependencies haven't changed and I don't include many vendors.

Any help is much appreciated!

Kind regards,
Jeroen van Rooij

Ps. I'm using the latest composer version and al the requires in the json are set to a fixed version

@Seldaek

This comment has been minimized.

Member

Seldaek commented May 15, 2013

Could you maybe share your composer.json so it's at least possible to try and reproduce it?

@jrooi

This comment has been minimized.

jrooi commented May 15, 2013

Yeah ofcourse, sorry!

{
    "name": "lsg/lsg.nl",
    "description": "Lsg website",
    "autoload": {
        "psr-0": { "": "src/" }
    },
    "repositories": [
        { "type": "vcs", "url": "git@github.com:lsg-it/ExtensionBundle.git" },
        { "type": "vcs", "url": "git@github.com:lsg-it/CmsBundle.git" }
    ],
    "require": {
        "php": ">=5.3.3",
        "symfony/symfony": "v2.1.7",
        "doctrine/orm": "2.3.1",
        "doctrine/doctrine-bundle": "v1.0.0",
        "doctrine/doctrine-fixtures-bundle": "2.1.*",
        "twig/extensions": "1.0.*",
        "symfony/assetic-bundle": "v2.1.1",
        "symfony/swiftmailer-bundle": "v2.1.7",
        "symfony/monolog-bundle": "v2.1.7",
        "sensio/distribution-bundle": "v2.1.7",
        "sensio/framework-extra-bundle": "v2.1.7",
        "sensio/generator-bundle": "v2.1.7",
        "jms/security-extra-bundle": "1.2.0",
        "jms/di-extra-bundle": "1.1.1",
        "nelmio/solarium-bundle": "v1.1.0",
        "friendsofsymfony/facebook-bundle": "v1.1.0",
        "hwi/oauth-bundle": "*",
        "gedmo/doctrine-extensions": "v2.3.4",
        "lsg/cms-bundle": "master",
        "lsg/extension-bundle": "master"
    },
    "scripts": {
        "post-install-cmd": [
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::buildBootstrap",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::clearCache",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installAssets",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installRequirementsFile"
        ],
        "post-update-cmd": [
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::buildBootstrap",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::clearCache",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installAssets",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installRequirementsFile"
        ]
    },
    "minimum-stability": "dev",
    "extra": {
        "symfony-app-dir": "app",
        "symfony-web-dir": "web",
        "symfony-assets-install": "symlink"
    }
}

Those two Lsg requires are internal, private bundles. The json of those two bundles are:

{
    "name": "lsg/cms-bundle",
    "type": "symfony-bundle",
    "description": "LSG CMS Bundle",
   "repositories": [
        { "type": "vcs", "url": "git@github.com:lsg-it/ExtensionBundle.git" }
    ],
    "require": {
        "php": ">=5.3.3",
        "symfony/symfony": "2.1.*",
        "white-october/pagerfanta-bundle": "2.1.*",
        "craue/formflow-bundle": "1.1.0",
        "lsg/extension-bundle": "master"
    },
    "autoload": {
        "psr-0": {
            "Lsg\\CmsBundle": ""
        }
    },
    "target-dir": "Lsg/CmsBundle"
}

And the last one:

{
    "name": "lsg/extension-bundle",
    "type": "symfony-bundle",
    "description": "Standard LSG extensions",
    "require": {
        "php": ">=5.3.3",
        "symfony/symfony": "2.1.*",
        "gedmo/doctrine-extensions": "v2.3.4",
        "twig/twig": "1.*"
    },
    "autoload": {
        "psr-0": {
            "Lsg\\ExtensionBundle": ""
        }
    },
    "target-dir": "Lsg/ExtensionBundle"
}
@todsul

This comment has been minimized.

todsul commented May 15, 2013

We're receiving this error too. Increasing to 1G in php.ini worked.

.json below...

{
"name": "symfony/framework-standard-edition",
"description": "The "Symfony Standard Edition" distribution",
"autoload": {
"psr-0": { "": "src/" }
},
"require": {
"php": ">=5.3.3",
"symfony/symfony": "2.1.",
"doctrine/orm": ">=2.2.3,<2.4-dev",
"doctrine/doctrine-bundle": "1.0.
",
"twig/extensions": "1.0.",
"symfony/assetic-bundle": "2.1.
",
"symfony/swiftmailer-bundle": "2.1.",
"symfony/monolog-bundle": "2.1.
",
"sensio/distribution-bundle": "2.1.",
"sensio/framework-extra-bundle": "2.1.
",
"sensio/generator-bundle": "2.1.",
"jms/security-extra-bundle": "1.2.
",
"jms/di-extra-bundle": "1.1.",
"dholmes/error-handling": "master",
"dholmes/doctrine-extras": "master",
"friendsofsymfony/jsrouting-bundle": "1.0.2",
"amazonwebservices/aws-sdk-for-php": "1.5.15",
"beberlei/DoctrineExtensions": "v0.1",
"doctrine/doctrine-fixtures-bundle": "dev-master",
"EHER/phpunit": ">=1.2",
"phake/phake": "dev-master",
"stripe/stripe-php": "v1.7.7",
"imagine/Imagine": "dev-master",
"michelf/php-markdown": "1.2.5",
"cboden/Ratchet": "0.2.
",
"react/zmq": "dev-master"
},
"repositories": [
{
"type": "vcs",
"url": "https://github.com/danielholmes/error-handling"
},
{
"type": "vcs",
"url": "https://github.com/danielholmes/doctrine-extras"
},
{
"type": "package",
"package": {
"name": "michelf/php-markdown",
"version": "1.2.5",
"source": {
"url": "git://github.com/michelf/php-markdown.git",
"type": "git",
"reference": "origin/extra"
}
}
}
],
"scripts": {
"post-install-cmd": [
"Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::buildBootstrap",
"Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::clearCache",
"Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::installAssets",
"Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::installRequirementsFile"
],
"post-update-cmd": [
"Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::buildBootstrap",
"Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::clearCache",
"Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::installAssets",
"Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::installRequirementsFile"
]
},
"minimum-stability": "dev",
"extra": {
"symfony-app-dir": "app",
"symfony-web-dir": "web"
}
}

@mac-cain13

This comment has been minimized.

mac-cain13 commented May 17, 2013

I also can confirm this issue, after updating composer I can't update our projects without raising the memory_limit from 512MB to 1GB. Seems that memory usage increased quite a lot. Here is one of our composer.json files that are problematic:

{
    "name": "symfony/framework-standard-edition",
    "description": "The \"Symfony Standard Edition\" distribution",
    "autoload": {
        "psr-0": { "": "src/" }
    },
    "require": {
        "php": ">=5.3.3",
        "symfony/symfony": "2.1.*",
        "doctrine/orm": ">=2.2.3,<2.4-dev",
        "doctrine/doctrine-bundle": "1.0.*-dev",
        "twig/extensions": "1.0.*-dev",
        "symfony/assetic-bundle": "2.1.*-dev",
        "symfony/swiftmailer-bundle": "2.1.*-dev",
        "symfony/monolog-bundle": "2.1.*-dev",
        "sensio/distribution-bundle": "2.1.*-dev",
        "sensio/framework-extra-bundle": "2.1.*-dev",
        "sensio/generator-bundle": "2.1.*-dev",
        "jms/security-extra-bundle": "1.2.*",
        "jms/di-extra-bundle": "1.1.*-dev",
        "friendsofsymfony/rest-bundle" : "0.10.*-dev",
        "doctrine/doctrine-migrations-bundle" : "dev-master",
        "richsage/rms-push-notifications-bundle" : "*",
        "imagine/Imagine": "dev-develop",
        "stof/doctrine-extensions-bundle": "dev-master",
        "mlpz/postmark-bundle": "*",
        "jms/serializer-bundle" : "dev-master",
        "craue/twigextensions-bundle": "dev-master",
        "wrep/bugsnag-php-symfony" : "dev-master",
        "wrep/notificato-symfony" : "1.0.*",
        "wrep/daemonizable-command": "dev-master"
    },
    "scripts": {
        "post-install-cmd": [
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::buildBootstrap",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::clearCache",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installAssets",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installRequirementsFile"
        ],
        "post-update-cmd": [
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::buildBootstrap",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::clearCache",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installAssets",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installRequirementsFile"
        ]
    },
    "config": {
        "bin-dir": "bin"
    },
    "minimum-stability": "dev",
    "extra": {
        "symfony-app-dir": "app",
        "symfony-web-dir": "web"
    }
}

I just added the wrep/daemonizable-command package and runned composer update wrep/daemonizable-command, but that crashed with this error:

Loading composer repositories with package information
Updating dependencies (including require-dev)
PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 32 bytes) in phar:///usr/bin/composer/src/Composer/DependencyResolver/RuleWatchGraph.php on line 52
PHP Stack trace:
PHP   1. {main}() /usr/bin/composer:0
PHP   2. require() /usr/bin/composer:15
PHP   3. Composer\Console\Application->run() phar:///usr/bin/composer/bin/composer:43
PHP   4. Symfony\Component\Console\Application->run() phar:///usr/bin/composer/src/Composer/Console/Application.php:83
PHP   5. Composer\Console\Application->doRun() phar:///usr/bin/composer/vendor/symfony/console/Symfony/Component/Console/Application.php:119
PHP   6. Symfony\Component\Console\Application->doRun() phar:///usr/bin/composer/src/Composer/Console/Application.php:117
PHP   7. Symfony\Component\Console\Application->doRunCommand() phar:///usr/bin/composer/vendor/symfony/console/Symfony/Component/Console/Application.php:212
PHP   8. Symfony\Component\Console\Command\Command->run() phar:///usr/bin/composer/vendor/symfony/console/Symfony/Component/Console/Application.php:882
PHP   9. Composer\Command\UpdateCommand->execute() phar:///usr/bin/composer/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:244
PHP  10. Composer\Installer->run() phar:///usr/bin/composer/src/Composer/Command/UpdateCommand.php:102
PHP  11. Composer\Installer->doInstall() phar:///usr/bin/composer/src/Composer/Installer.php:208
PHP  12. Composer\DependencyResolver\Solver->solve() phar:///usr/bin/composer/src/Composer/Installer.php:446
PHP  13. Composer\DependencyResolver\RuleWatchGraph->insert() phar:///usr/bin/composer/src/Composer/DependencyResolver/Solver.php:170
PHP  14. SplDoublyLinkedList->unshift() phar:///usr/bin/composer/src/Composer/DependencyResolver/RuleWatchGraph.php:52

Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 32 bytes) in phar:///usr/bin/composer/src/Composer/DependencyResolver/RuleWatchGraph.php on line 52

Call Stack:
    0.0035     630856   1. {main}() /usr/bin/composer:0
    0.0091     817720   2. require('phar:///usr/bin/composer/bin/composer') /usr/bin/composer:15
    0.0454    5095992   3. Composer\Console\Application->run() phar:///usr/bin/composer/bin/composer:43
    0.0502    5671944   4. Symfony\Component\Console\Application->run() phar:///usr/bin/composer/src/Composer/Console/Application.php:83
    0.0519    5917824   5. Composer\Console\Application->doRun() phar:///usr/bin/composer/vendor/symfony/console/Symfony/Component/Console/Application.php:119
    0.0530    6057016   6. Symfony\Component\Console\Application->doRun() phar:///usr/bin/composer/src/Composer/Console/Application.php:117
    0.0544    6057016   7. Symfony\Component\Console\Application->doRunCommand() phar:///usr/bin/composer/vendor/symfony/console/Symfony/Component/Console/Application.php:212
    0.0544    6057016   8. Symfony\Component\Console\Command\Command->run() phar:///usr/bin/composer/vendor/symfony/console/Symfony/Component/Console/Application.php:882
    0.0555    6055984   9. Composer\Command\UpdateCommand->execute() phar:///usr/bin/composer/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:244
    0.2593   13311072  10. Composer\Installer->run() phar:///usr/bin/composer/src/Composer/Command/UpdateCommand.php:102
    0.2611   13478184  11. Composer\Installer->doInstall() phar:///usr/bin/composer/src/Composer/Installer.php:208
    3.2977   53035864  12. Composer\DependencyResolver\Solver->solve() phar:///usr/bin/composer/src/Composer/Installer.php:446
   53.1716  536465808  13. Composer\DependencyResolver\RuleWatchGraph->insert() phar:///usr/bin/composer/src/Composer/DependencyResolver/Solver.php:170
   53.1716  536466480  14. SplDoublyLinkedList->unshift() phar:///usr/bin/composer/src/Composer/DependencyResolver/RuleWatchGraph.php:52
@jameshalsall

This comment has been minimized.

jameshalsall commented May 18, 2013

👍 I'm also receiving this issue on PHP 5.3.15 on Mac OSX. Memory limit is at 512mb and never had a problem in the past (until updating composer a couple of days ago).

EDIT: I can confirm that upping a 1gb limit fixes, but surely an issue.

Composer JSON:

{
    "autoload": {
        "psr-0":         {
            "": "src/"
        }
    },
    "config": {
        "bin-dir": "bin"
    },
    "description": "Tickit Application",
    "extra": {
        "symfony-app-dir": "app",
        "symfony-web-dir": "web"
    },
    "minimum-stability": "dev",
    "name": "tickit-project/tickit",
    "require": {
        "doctrine/doctrine-bundle": "1.2.x-dev",
        "doctrine/doctrine-fixtures-bundle": "dev-master",
        "doctrine/orm": "2.3.*",
        "friendsofsymfony/user-bundle": "2.0.x-dev",
        "jms/di-extra-bundle": "1.3.x-dev",
        "jms/security-extra-bundle": "1.4.x-dev",
        "php": ">=5.3.3",
        "sensio/distribution-bundle": "2.3.x-dev",
        "sensio/framework-extra-bundle": "2.3.x-dev",
        "sensio/generator-bundle": "2.3.x-dev",
        "stof/doctrine-extensions-bundle": "1.1.x-dev",
        "symfony/assetic-bundle": "2.1.x-dev",
        "symfony/monolog-bundle": "2.2.x-dev",
        "symfony/swiftmailer-bundle": "2.2.x-dev",
        "symfony/symfony": "2.3.x-dev",
        "symfony/console": "2.3.x-dev",
        "twig/extensions": "1.0.x-dev",
        "psr/log": "1.0.0",
        "tickit-project/cache-bundle": "1.0.x-dev",
        "squizlabs/php_codesniffer": "1.*",
        "behat/symfony2-extension": "dev-master",
        "satooshi/php-coveralls": "dev-master"
    },
    "scripts": {
        "post-install-cmd": [
"Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::buildBootstrap",
"Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::clearCache",
"Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installAssets",
"Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installRequirementsFile"
    ],
        "post-update-cmd": [
"Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::buildBootstrap",
"Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::clearCache",
"Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installAssets",
"Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installRequirementsFile"
    ]
    }
}
@mddubs

This comment has been minimized.

mddubs commented May 23, 2013

Same issue since updating Composer. 1GB workaround does work, but something must be wrong.

Composer: 950fc7e
Mac OSX: 10.8.3
PHP: 5.3.15

@michaldudek

This comment has been minimized.

michaldudek commented Jun 6, 2013

Now I've hit 2GB memory limit ;) I'm getting 16GB RAM tonight so gonna increase the php memory limit to around 10, should last for few months of updates :P

@michaldudek

This comment has been minimized.

michaldudek commented Jun 6, 2013

Updating my CLI to php54 fixed it for now.

@robocoder

This comment has been minimized.

Contributor

robocoder commented Jun 21, 2013

Yup, upgrading to php5.4.x fixes a memory issue when using php 5.3.x. For example, I was running composer install without a .lock file. With php5.3.10, composer required over 3GB of memory. With php5.4.16, composer consumed less than 1GB.

@Dynom

This comment has been minimized.

Dynom commented Jul 4, 2013

Needing 512MiB RAM for a package-manager is already a bit much, but I also exceed it. The interesting part is that I exceed 512 MiB RAM (up to around 800 MiB RAM according to my process output) when I use my private Satis repository (which is actually just a cache for all Github public repositories). I do not exceed it otherwise. Just by adding my satis repository, composer consumes almost 400 MiB more RAM.

I'm on PHP 5.4.14.

To all the others raising their PHP CLI memory_limit globally, I recommend against that. Instead I suggest to put it in the command line, like so:

php -dmemory_limit=1G bin/composer.phar update .. 

Otherwise you could potentially miss certain unintended memory related behaviour in your own work.

@tyler-sommer

This comment has been minimized.

tyler-sommer commented Jul 18, 2013

Chiming in with my experience. I'm also using a private Satis repository in addtion to the zend framework repository. I was having problems with composer update on PHP 5.3.26 using over 2GB of memory.

Switching to 5.4 solved everything, now using under 400MB.

@jameshalsall

This comment has been minimized.

jameshalsall commented Jul 18, 2013

ping @Seldaek

@Taiger

This comment has been minimized.

Taiger commented Jul 19, 2013

Thanks @Dynom, this works for those of us on PHP <5.4

php -dmemory_limit=1G composer.phar update
@jjanvier

This comment has been minimized.

jjanvier commented Jul 23, 2013

Same problem

@iVariable

This comment has been minimized.

iVariable commented Aug 3, 2013

Same here. Mac OS X. PHP 5.3.15.
Fixed by updating php to 5.4.10.

@baohx2000

This comment has been minimized.

baohx2000 commented Aug 27, 2013

Happening to me on 5.5. Guessing something somewhere is not cleaning up after itself and leaving objects hanging around?

@deguif

This comment has been minimized.

Contributor

deguif commented Aug 28, 2013

Same problem with php 5.3.15 on Mac OS X.

@atmediauk

This comment has been minimized.

atmediauk commented Aug 28, 2013

Same problem here. I can't get it to composer update as I only have 1gb total for this particular vps.

PHP 5.4.11 (cli) (built: Feb 20 2013 19:02:54)

----> Update
Not sure if it helps but I noticed that my local php version was: 5.4.19 so although I had 5.4 on the production server it was a few updates out.

I updated the production server from 5.4.11 to 5.4.19 and composer update seems to be working now.

@kmmathis

This comment has been minimized.

kmmathis commented Aug 30, 2013

Had this issue until I upgraded to PHP 5.4.19. composer update now appears to work.

@IAkumaI

This comment has been minimized.

IAkumaI commented Sep 27, 2013

I'm also have a problem, but yesterday it worked fine with the same composer.json

@MPLLC

This comment has been minimized.

MPLLC commented Oct 2, 2013

Ugh, this is a killer on shared hosting.

@Seldaek

This comment has been minimized.

Member

Seldaek commented Oct 3, 2013

@kevinm1 it's not, you are not supposed to run composer update on your production server. You should run update on your dev box then deploy the composer.lock file and run install which will run with very low memory footprint and very fast since it doesn't have to check for new packages but merely applies what's in the lock file.

@Imunhatep

This comment has been minimized.

Imunhatep commented Oct 15, 2013

Yep, same here - have to increase memory limit to 1G. And indeed it eats up to 980M.

Interesting thing is that this started to happen only after I added local Satis composer repository which in fact holds only cached .zip packages mostly if not only from github.

Without local Satis repository, composer uses around of 240mb.
And if use local Satis repository with turned off packagist it also uses around 240mb. Example:

"repositories": [
{ "type": "composer", "url": "http://packagist.example.com" },
{ "packagist": false }
],

Happens on Gentoo with PHP 5.4.16

@v9n

This comment has been minimized.

v9n commented Oct 20, 2013

I was on PHP 5.5.5. Cannot solve this. Tried to disable lots of extension: opcode, xdebug,..I actually ended up creating swap file. My VPS is 512MB (a cheap digital ocean instance) with a handful of service: nginx, mariadb, mongodb, redis. puma.

to clarify, I was not able to run a simple single command for installing laravel

composer create-project laravel/laravel

Here is my simple work around, hopefully help s.o:

df -h 
dd if=/dev/zero of=/swapfile bs=1M count=1024
sudo dd if=/dev/zero of=/swapfile bs=1M count=1024
mkswap /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
echo 'echo "/swapfile  none  swap  defaults  0  0" >> /etc/fstab' | sudo sh

free -m

confirm u see your swap there:
total used free shared buffers cached
Mem: 494 335 158 0 19 62
-/+ buffers/cache: 254 240
Swap: 1023 3 1020

Then install composer, and wat your RAM ;(

watch free -m
@kmmathis

This comment has been minimized.

kmmathis commented Oct 21, 2013

Most of the people getting errors related to this ticket seem to be trying to create projects and/or updating with composer on their servers. Best practice is running your composer create-project and composer update commands locally, using git to pull your project updates to the server, and then simply running composer install on the server to update your production dependancies.

@v9n

This comment has been minimized.

v9n commented Oct 21, 2013

I did what u said, and it fail right on that command...I actually ended up creating a swap file to expand memory..

@uwej711

This comment has been minimized.

uwej711 commented Oct 21, 2013

While php -dmemory_limit=1G composer.phar update is ok, it is rather a bit to type (especially if you install it to /usr/bin). Any thoughts about passing the memory limit optionally directly to composer, something like

composer --memory=1G update
@mrchimp

This comment has been minimized.

mrchimp commented Nov 5, 2013

@kmmathis I'm getting the same problem when I do composer install.

@Glideh

This comment has been minimized.

Glideh commented Nov 5, 2013

composer install also gives me the same issue (PHP 5.3.3).
@uwej711 you can still php -dmemory_limit=xxx /usr/local/bin/composer ...
It passes with 512M memory_limit

@nfx

This comment has been minimized.

Contributor

nfx commented Nov 19, 2013

maybe there is a substantial problem building dependency graph, that needs to be addressed?...

@kumarramalingam

This comment has been minimized.

kumarramalingam commented Mar 19, 2015

Solve this problem.

Update PHP Version.
Ubuntu Terminal window:

1.sudo apt-get update

2.sudo apt-get install lamp-server^

That's all.

@cbrunnkvist

This comment has been minimized.

cbrunnkvist commented Jun 12, 2015

"You're on the wrong server" / "You don't have enough swap" / "Framework X is an edge-case" / "Problems like this are not always meant to be fixed"?? Whaaaat, seriously? Is everybody (with the brilliant exception of @stof and @judgej) turning into total apologists? 😟

The issue still remains: something is wrong, because it is insane that such an important and central tool such as the dependency manager, should gobble up gigglypiles of memory just in order to complete it's most basic tasks.

@cbrunnkvist

This comment has been minimized.

cbrunnkvist commented Jun 12, 2015

For reference: mem usage stays stable on my Yosemite Mac (PHP 5.5.20) but totally blows up on my Ubuntu Trusty deployment (PHP 5.5.9-1ubuntu4.9). None of the workarounds above has any effecty ~ it must be some triggering some leak in that older ("stable") version of PHP.

@sbuzonas

This comment has been minimized.

Contributor

sbuzonas commented Jun 15, 2015

The issue still remains: something is wrong, because it is insane that such an important and central tool such as the dependency manager, should gobble up gigglypiles of memory just in order to complete it's most basic tasks.

Dependency resolution is not it's most basic task, on the contrary it is the most complex task. Being such a critical piece of the application, it is rather difficult to make improvements to. Servers generally have less memory than a development machine and it is recommended to do the update in your development environment and simply do an install from the lock on a server.

The solver is computing a solution for a graph theory equation. It basically does a tsort with an unbounded set discovering additional dependencies to consider for resolution throughout the process. The larger the graph the more memory required to compute the result. When you introduce potential nodes to the graph via discovery through a service, like packagist, that provides a large number of alternatives it increases the complexity and required memory/compute time. A traditional tsort algorithm has a complexity of O(log^2 n).

@cbrunnkvist

This comment has been minimized.

cbrunnkvist commented Jun 15, 2015

I would stress that my use of "basic" is not in the sense of "simplistic and of low complexity" - but in the meaning that this task is quintessential, and critical, to the goals of the program.

If we can conclude that there is nothing obviously wrong with the algo. implementation itself, then one way to start dealing with the problem could be to build up a dataset of known PHP version where bloat occurs and at the very least display a warning at execution time. Ideas?

@sbuzonas

This comment has been minimized.

Contributor

sbuzonas commented Jun 15, 2015

It isn't a specific PHP version that causes bloat, it is a factor. Packages can have dependencies on different PHP versions. If you have a 5.3 installation of PHP you are eliminating candidates of packages that require php >5.4.

The largest factor is the dependencies themselves. For instance "symfony/symfony": "^2.7" is a much simpler constraint because it performs a replaces for all of it's subpackages with self.version... this eliminates a huge subset of possibilities because each sub package does not need to be evaluated.

The bloat generally occurs whenever you have a dependency somewhere down the line that has a large number of potential candidates. For instance, if you only require a subset of symfony packages they will need to be calculated individually to satisfy each constraint and dependency. If you drop that constraint down to ^2.3 you introduce a significant number of possibilities to evaluate.

The ideal goal of maintaining backwards compatibility with other open source packages, a good motivation IMO, is the main killer of the memory utilization of composer. When a new version is released that meets your constraint it needs to be evaluated.

@cbrunnkvist

This comment has been minimized.

cbrunnkvist commented Jun 16, 2015

"Packages can have dependencies on different PHP versions"

💡 Aaah darn, good point...

@mpemberton5

This comment has been minimized.

mpemberton5 commented Jul 3, 2015

I turned off xdebug and still had problem. Turned off OpCode for cli and worked like a dream (and fast too).

@orhan-mobven

This comment has been minimized.

orhan-mobven commented Jul 16, 2015

I turned off xdebug and it works now.

@fabianhenzler

This comment has been minimized.

fabianhenzler commented Aug 27, 2015

Having it with HHVM too

@ronnyandre

This comment has been minimized.

ronnyandre commented Sep 1, 2015

Thanks @kureikain
Your solution solved my problem! 👍

@chrisg123

This comment has been minimized.

chrisg123 commented Sep 26, 2015

I was having this issue. Updating composer (composer self-update) seems to have solved it for me...

@Aldibe

This comment has been minimized.

Aldibe commented Oct 23, 2015

i got this error when running composer.phar update. and i think it is similar to this issue, but i don't know how to fix it..

PHP Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to allocate 144115188075867549 bytes) in phar:///bin/composer.phar/src/Composer/Util/RemoteFilesystem.php on line 179

this is my composer.json

{ "description" : "The CodeIgniter framework", "name" : "codeigniter/framework", "license": "MIT", "require": { "php": ">=5.2.4", "videlalvaro/php-amqplib": "2.5.*" }, "require-dev": { "mikey179/vfsStream": "1.1.*", "videlalvaro/php-amqplib": "2.5.*" } }

is it possible/normal for composer to allocate that much memory??

@stof

This comment has been minimized.

Contributor

stof commented Oct 23, 2015

during an update, it can indeed use lots of memory

@kkomelin

This comment has been minimized.

kkomelin commented Oct 27, 2015

I might have missed some of the comments above but since this issue is still open I'll add my two coins.

My system:
VPS: 1Gb RAM, Ubuntu, ssd, no swap (I don't need it)

My situation:
I successfully updated composer to the latest version using self-update command. Then I tried to update Drush to the latest but received the memory error discussed in this issue.
I used the following command: composer global require drush/drush

I did this way because it's required by the official Drush documentation which is used by thousands so the suggestion to use composer on my local machine and then move to Git is not for me obviously.

Of course, I can temporary increase memory to 1G but folks... really... it's too much for one utility tool and it's probably not a PHP problem but a problem of the composer architecture.

So, maintainers please think about this when you're planning roadmap for future versions because I believe that no one utility tool deserves 1G of memory even if it's local machine memory.
Thanks.

@stefandoorn

This comment has been minimized.

stefandoorn commented Nov 13, 2015

@barryvdh Ain't your suggestion more like this, just a suggestion:

  • Composer update
  • Composer copies composer.json to composer.json.tmp
  • Get all dependencies from root projects and add to require part in composer.json.tmp
  • Internally run composer update again but then using the temporary file, which seems faster and less memory in your examples
  • After update, remove composer.json.tmp file and write composer.lock file
@TengfeiQi

This comment has been minimized.

TengfeiQi commented Dec 4, 2015

这个问题好蛋疼

Fatal error: Uncaught exception 'ErrorException' with message 'proc_open(): fork failed - Cannot allocate memory' in phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:940
@SiliconMind

This comment has been minimized.

SiliconMind commented Dec 14, 2015

I'd consider this more than serious. I just tried to install Symfony CMF. I needed to increase memory limit to 2G in order to avoid constant out of memory issues. Come one guys! There is something seriously wrong with the composer. I'm managing dozens of dev machines and it was the first time for past few years that I had to increase it's memory over 1G and for what? A composer install?

With recent boom for small cloud dev machines this will become a serious issue.

@judgej

This comment has been minimized.

judgej commented Dec 14, 2015

@SiliconMind Be more specific about the version(s) you want to install. That is the key. Don't give composer the option to have to investigate every possible combination of every single package that is needed to install your framework; if you do, then it will be trying out many thousands of possible combinations to find the perfect latest compatible set of packages available, and it is that which eats memory.

@staabm

This comment has been minimized.

Contributor

staabm commented Dec 14, 2015

@staabm

This comment has been minimized.

Contributor

staabm commented Dec 14, 2015

Also it would be very helpfull when you could create a blackfire profile of your run (after you increased php-memory-limit to a value high enough that it doesnt crash)

@SiliconMind

This comment has been minimized.

SiliconMind commented Dec 14, 2015

@judgej You know, I can handle myself a lot but I don't think that's the point. Consider someone less experienced that visits Symfony's CMF install instructions and sees this: composer create-project symfony-cmf/standard-edition cmf '~1.2'.

You can't expect that average John Developer will know composer inside out. And frankly, the versions for Symfony CMF are pretty precise. Maybe related packages are not specific enough, but you can't expect that all end users will now start to edit related composer.json files in hope of finding the source of the issue.

@SiliconMind

This comment has been minimized.

SiliconMind commented Dec 14, 2015

@staabm xdebug was disabled. It was vanilla debian install without any additional stuff. As for the profile - I'm sorry, I can't do this now.

@garethmcc

This comment has been minimized.

garethmcc commented Feb 5, 2016

I have been plagued by this issue for over two years now and the official response is still "add swap"? This should not be a problem and its ridiculous that it isn't resolved yet. If a memory issue such as this made it into anything I released I would lose my job.

@composer composer locked and limited conversation to collaborators Feb 5, 2016

@composer composer locked and limited conversation to collaborators Feb 5, 2016

@Seldaek Seldaek closed this Mar 2, 2016

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.