Failed to rename from tmp directory to vendor #1765

Closed
rmasters opened this Issue Apr 3, 2013 · 11 comments

Projects

None yet

7 participants

@rmasters
rmasters commented Apr 3, 2013

For some packages (reproducibly, this happens with gedmo/doctrine-extensions@v2.3.5) Composer fails to copy directories from the temporary clone to the vendors directory.

In this case specifically, the first directory to fail is /tmp/.../doctrine-extensions/schemas. On a hunch I patched Composer\Util\Filesystem to use mv -p src dest (the -p flag added), which fixed this. However it now fails on /tmp/.../doctrine-extensions/doc:

vagrant@quantal64:/var/www/symfony$ sudo php ../composer/composer.phar update -v
Loading composer repositories with package information
Updating dependencies (including require-dev)
  - Installing gedmo/doctrine-extensions (v2.3.5)
    Loading from cache
    Extracting archive

  [RuntimeException]
  Could not rename "/tmp/cmp08399/l3pp4rd-DoctrineExtensions-1cbcf3a/doc" to "/var/www/project/vendor/gedmo/doctrine-extensions/doc".

Exception trace:
 () at phar:///var/www/composer/composer.phar/src/Composer/Util/Filesystem.php:188
 Composer\Util\Filesystem->rename() at phar:///var/www/composer/composer.phar/src/Composer/Downloader/ArchiveDownloader.php:61
 Composer\Downloader\ArchiveDownloader->download() at phar:///var/www/composer/composer.phar/src/Composer/Downloader/DownloadManager.php:177
 Composer\Downloader\DownloadManager->download() at phar:///var/www/composer/composer.phar/src/Composer/Installer/LibraryInstaller.php:155
 Composer\Installer\LibraryInstaller->installCode() at phar:///var/www/composer/composer.phar/src/Composer/Installer/LibraryInstaller.php:86
 Composer\Installer\LibraryInstaller->install() at phar:///var/www/composer/composer.phar/src/Composer/Installer/InstallationManager.php:151
 Composer\Installer\InstallationManager->install() at phar:///var/www/composer/composer.phar/src/Composer/Installer/InstallationManager.php:138
 Composer\Installer\InstallationManager->execute() at phar:///var/www/composer/composer.phar/src/Composer/Installer.php:488
 Composer\Installer->doInstall() at phar:///var/www/composer/composer.phar/src/Composer/Installer.php:204
 Composer\Installer->run() at phar:///var/www/composer/composer.phar/src/Composer/Command/UpdateCommand.php:102
 Composer\Command\UpdateCommand->execute() at phar:///var/www/composer/composer.phar/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:240
 Symfony\Component\Console\Command\Command->run() at phar:///var/www/composer/composer.phar/vendor/symfony/console/Symfony/Component/Console/Application.php:193
 Symfony\Component\Console\Application->doRun() at phar:///var/www/composer/composer.phar/src/Composer/Console/Application.php:117
 Composer\Console\Application->doRun() at phar:///var/www/composer/composer.phar/vendor/symfony/console/Symfony/Component/Console/Application.php:106
 Symfony\Component\Console\Application->run() at phar:///var/www/composer/composer.phar/src/Composer/Console/Application.php:83
 Composer\Console\Application->run() at phar:///var/www/composer/composer.phar/bin/composer:39
 require() at /var/www/composer/composer.phar:15

I get the same stack trace for the fail on /tmp/.../doctrine-extensions/Doctrine.

This happens when I run as root as well, so I guess this isn't permissions related and I'm finding it hard to debug. Any ideas?

@jshanman
jshanman commented Apr 9, 2013

I am having the same issue

@rmasters

I've been doing some more digging on this. For context - I'm working on a Ubuntu server Virtualbox VM (via Vagrant+Chef) (which I should have mentioned before).

In this case, my project is on a shared folder (so it's accessible by the host and guest) while /tmp is on the guest's virtual-hard-disk.

I think the problem is that after the files are downloaded to /tmp we're moving them from a filesystem of one type (tmpfs) to another (vboxsf). Both mv and cp -r fail with this, see this Superuser answer, with errors as below:

mv: inter-device move failed: `/tmp/cmpc3a08/l3pp4rd-DoctrineExtensions-1cbcf3a/schemas' to `/var/www/symfony/vendor/gedmo/doctrine-extensions/schemas'; unable to remove target: Is a directory

The solution to this is initialising the TMPDIR environment variable to a folder on the same filesystem:

# Assuming ~/project is a shared folder,
cd ~/project
mkdir tmp
TMPDIR="tmp/" composer update

I'm not very knowledgeable on filesystems, so I'm not sure if we can check if tmpdir and target are the same type (as it could be tmpfs and ext3 which are apparently compatible?). Is there a flag to use on mv that helps?

@rredpoppy

Thanks a milion, you saved my life 👍

@Seldaek Seldaek closed this in 27d958a Jun 18, 2013
@Seldaek
Member
Seldaek commented Jun 18, 2013

OK I switched it so that composer now extracts archives in vendor/composer/ first and then moves the files to the right location. This hopefully fixes this issue. If you could confirm though it would be great.

@radnan
Contributor
radnan commented Jun 20, 2013

This should really be a configurable option like "tmp-dir" or something with the system /tmp being the default. Otherwise you end up with with a bunch of empty vendor/ directories all over the place (when using the create-project command for example).

@stof
Contributor
stof commented Jun 20, 2013

Using the system tmp dir was the previous behavior, and it was causing issues because the permissions on the tmp dir may be different than in the project

@radnan
Contributor
radnan commented Jun 20, 2013

@stof I understand that. What I'm proposing is having something like the following:

$temporaryDir = $this->config->has('tmp-dir') ? rtrim($this->config->get('tmp-dir'), '/') . '/' : sys_get_temp_dir() . '/cmp';
$temporaryDir .= substr(md5(uniqid('', true)), 0, 8);

That way you can still override it, but the default would be the system /tmp.

@stof
Contributor
stof commented Jun 20, 2013

it was already overwritable through an environment variable. But the issue was that using the system temp folder as default was a broken default. composer should work by default

@radnan
Contributor
radnan commented Jun 20, 2013

In that case the create-project command should clean up after itself and delete the tmp directory.

@Seldaek
Member
Seldaek commented Jun 21, 2013

@radnan yeah definitely that is a regression. I'll open a new issue for it. Thanks for letting us know.

@oleg-andreyev

Getting same problem with rename from vendor/composer/ folder:

[09:32 dev@saclay ~/auto_branches/opc-1175/current]$ php composer.phar install -vvv
Downloading composer.json
Downloading composer.lock
Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file
Downloading composer.lock
  - Installing doctrine/lexer (v1.0)
Reading /home/sites/dev/.composer/cache/files/doctrine/lexer/1.0.0.0-v1.0.zip from cache
    Loading from cache
    Extracting archive

    REASON: Required by root: doctrine/lexer

  - Installing doctrine/annotations (v1.1.2)
Reading /home/sites/dev/.composer/cache/files/doctrine/annotations/1.1.2.0-v1.1.2.zip from cache
    Loading from cache
    Extracting archive



  [RuntimeException]
  Could not rename "vendor/composer/615b1f10/doctrine-annotations-65842e9/lib" to "/home/sites/dev/auto_branches/opc-1175/current/vendor/doctrine/annotations/lib".



Exception trace:
 () at phar:///home/sites/dev/auto_branches/opc-1175/current/composer.phar/src/Composer/Util/Filesystem.php:188
 Composer\Util\Filesystem->rename() at phar:///home/sites/dev/auto_branches/opc-1175/current/composer.phar/src/Composer/Downloader/ArchiveDownloader.php:61
 Composer\Downloader\ArchiveDownloader->download() at phar:///home/sites/dev/auto_branches/opc-1175/current/composer.phar/src/Composer/Downloader/DownloadManager.php:180
 Composer\Downloader\DownloadManager->download() at phar:///home/sites/dev/auto_branches/opc-1175/current/composer.phar/src/Composer/Installer/LibraryInstaller.php:157
 Composer\Installer\LibraryInstaller->installCode() at phar:///home/sites/dev/auto_branches/opc-1175/current/composer.phar/src/Composer/Installer/LibraryInstaller.php:86
 Composer\Installer\LibraryInstaller->install() at phar:///home/sites/dev/auto_branches/opc-1175/current/composer.phar/src/Composer/Installer/InstallationManager.php:151
 Composer\Installer\InstallationManager->install() at phar:///home/sites/dev/auto_branches/opc-1175/current/composer.phar/src/Composer/Installer/InstallationManager.php:138
 Composer\Installer\InstallationManager->execute() at phar:///home/sites/dev/auto_branches/opc-1175/current/composer.phar/src/Composer/Installer.php:505
 Composer\Installer->doInstall() at phar:///home/sites/dev/auto_branches/opc-1175/current/composer.phar/src/Composer/Installer.php:208
 Composer\Installer->run() at phar:///home/sites/dev/auto_branches/opc-1175/current/composer.phar/src/Composer/Command/InstallCommand.php:97
 Composer\Command\InstallCommand->execute() at phar:///home/sites/dev/auto_branches/opc-1175/current/composer.phar/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:244
 Symfony\Component\Console\Command\Command->run() at phar:///home/sites/dev/auto_branches/opc-1175/current/composer.phar/vendor/symfony/console/Symfony/Component/Console/Application.php:882
 Symfony\Component\Console\Application->doRunCommand() at phar:///home/sites/dev/auto_branches/opc-1175/current/composer.phar/vendor/symfony/console/Symfony/Component/Console/Application.php:212
 Symfony\Component\Console\Application->doRun() at phar:///home/sites/dev/auto_branches/opc-1175/current/composer.phar/src/Composer/Console/Application.php:117
 Composer\Console\Application->doRun() at phar:///home/sites/dev/auto_branches/opc-1175/current/composer.phar/vendor/symfony/console/Symfony/Component/Console/Application.php:119
 Symfony\Component\Console\Application->run() at phar:///home/sites/dev/auto_branches/opc-1175/current/composer.phar/src/Composer/Console/Application.php:83
 Composer\Console\Application->run() at phar:///home/sites/dev/auto_branches/opc-1175/current/composer.phar/bin/composer:43
 require() at /home/sites/dev/auto_branches/opc-1175/current/composer.phar:15
@renanbr renanbr added a commit to renanbr/composer that referenced this issue Aug 6, 2013
@Seldaek @renanbr Seldaek + renanbr Extract archives into vendor dir to avoid permission errors, fixes #1765
, fixes #1714
72afcfa
@digitalkaoz digitalkaoz pushed a commit to digitalkaoz/composer that referenced this issue Nov 22, 2013
@Seldaek Seldaek Extract archives into vendor dir to avoid permission errors, fixes #1765
, fixes #1714
e484d79
@digitalkaoz digitalkaoz pushed a commit to digitalkaoz/composer that referenced this issue Nov 22, 2013
@Seldaek Seldaek Also try a php copy-and-remove on unix if mv failed, refs #1765 4e58aa1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment