Plugin breaks composer status command

[kdambekalns]

In any Flow setup run composer status, either v1 or v2 to see an error:

$ composer status -v


  [ErrorException]
  chdir(): No such file or directory (errno 2)


Exception trace:
 () at phar:///…/composer.phar/src/Composer/Package/Comparer/Comparer.php:68
 Composer\Util\ErrorHandler::handle() at n/a:n/a
 chdir() at phar:///…/composer.phar/src/Composer/Package/Comparer/Comparer.php:68
 Composer\Package\Comparer\Comparer->doCompare() at phar:///…/composer.phar/src/Composer/Downloader/FileDownloader.php:307
 Composer\Downloader\FileDownloader->getLocalChanges() at phar:///…/composer.phar/src/Composer/Command/StatusCommand.php:118
 Composer\Command\StatusCommand->doExecute() at phar:///…/composer.phar/src/Composer/Command/StatusCommand.php:76
 Composer\Command\StatusCommand->execute() at phar:///…/composer.phar/vendor/symfony/console/Command/Command.php:245
 Symfony\Component\Console\Command\Command->run() at phar:///…/composer.phar/vendor/symfony/console/Application.php:835
 Symfony\Component\Console\Application->doRunCommand() at phar:///…/composer.phar/vendor/symfony/console/Application.php:185
 Symfony\Component\Console\Application->doRun() at phar:///…/composer.phar/src/Composer/Console/Application.php:281
 Composer\Console\Application->doRun() at phar:///…/composer.phar/vendor/symfony/console/Application.php:117
 Symfony\Component\Console\Application->run() at phar:///…/composer.phar/src/Composer/Console/Application.php:113
 Composer\Console\Application->run() at phar:///…/composer.phar/bin/composer:61
 require() at /…/composer.phar:24

Since this issue is no longer caused by Composer itself (see composer/composer#7519) I debugged and found it's caused by our plugin not returning an absolute path in getInstallPath()

[kdambekalns]

Well, according to https://getcomposer.org/doc/articles/custom-installers.md our installer behaves correctly:

getInstallPath(), this method should return the location where the package is to be installed, relative from the location of composer.json.

[kdambekalns]

The error is on our side, but different: the returned path must not end with a slash!

[albe]

so it fails at the chdir() with the installPath of the package if it ends with a slash? Because I can find no trace in the composer code that would change the path returned from the installer anywhere. That sounds odd. (it does by appending _compare: https://github.com/composer/composer/blob/master/src/Composer/Downloader/FileDownloader.php#L443)
And a quick 3v4l/check with

<?php

mkdir('/tmp/test');
chdir('/tmp/test/');
chdir('/tmp/test');
chdir('/tmp/test2');
chdir('/tmp/test2/');

verifies that chdir does work with paths even if they end with a slash, but it fails with this exact error if the path does indeed not exist (no matter if with ending slash or not). So could the issue here really be, that our installer returns a path that it does not make sure exists beforehand (and that is created later on, then the "fix" here only seems to actually change things)?

Given the _compare path created in composer, it's more likely the download which should create the path fails (because it won't create the target directory recursively? nope, it is recursive https://github.com/composer/composer/blob/master/src/Composer/Downloader/FileDownloader.php#L131 https://github.com/composer/composer/blob/master/src/Composer/Util/Filesystem.php#L186). So, well... yeah, somehow the ending / is a problem I guess.

[Seldaek]

It seems to fail there going into $targetDir.'_compare', which would mean in this case if $targetDir has a trailing slash it should be installing a fresh copy of the package in a _compare subdirectory of the package itself.. Then it fails to chdir into that. It doens't quite make sense to me why this would be a problem, nor why the dir would be missing right after it installed the package.

Can you run a composer status -vvv @kdambekalns ?

Also can you maybe check what's in the package's dir, is there a _compare dir right after the run fails? Does it contain a clean copy of the package in question?

[kdambekalns]

The chdir call at the point it breaks tries to enter Packages/Sites/Acme.AcmeCom/_compare. It exists:

$ ls -l Packages/Sites
total 0
lrwxrwxrwx  1 karsten  staff   41 Feb 12  2020 Acme.AcmeCom -> ../../DistributionPackages/Acme.AcmeCom

$ ls -l Packages/Sites/Acme.AcmeCom
lrwxrwxrwx  1 karsten  staff  41 Feb 12  2020 Packages/Sites/Acme.AcmeCom -> ../../DistributionPackages/Acme.AcmeCom

$ ls -l Packages/Sites/Acme.AcmeCom/
total 8
drwxr-xr-x  7 karsten  staff   224 Oct 26 13:14 Configuration
drwxr-xr-x  3 karsten  staff    96 Sep 11  2019 Migrations
drwxr-xr-x  5 karsten  staff   160 Oct 29 15:05 Resources
lrwxr-xr-x  1 karsten  staff    44 Nov  5 16:13 _compare -> ../../../DistributionPackages/Acme.AcmeCom
-rw-r--r--  1 karsten  staff  3493 Apr  1  2020 composer.json

$ ls -l Packages/Sites/Acme.AcmeCom/_compare
lrwxr-xr-x  1 karsten  staff  44 Nov  5 16:13 Packages/Sites/Acme.AcmeCom/_compare -> ../../../DistributionPackages/Acme.AcmeCom

$ ls -l Packages/Sites/Acme.AcmeCom/_compare/
ls: Packages/Sites/Acme.AcmeCom/_compare/: No such file or directory

With my proposed fix (no trailing slash) this is the situation:

$ ls -l Packages/Sites
total 0
lrwxrwxrwx  1 karsten  staff   41 Feb 12  2020 Acme.AcmeCom -> ../../DistributionPackages/Acme.AcmeCom
lrwxr-xr-x  1 karsten  staff   41 Nov  5 16:20 Acme.AcmeCom_compare -> ../../DistributionPackages/Acme.AcmeCom

Might very well be an edge case involving path repositories…

For completeness, this is the full output when it breaks:

$ composer status -vvv
Reading ./composer.json
Loading config file /Users/karsten/.composer/config.json
Loading config file /Users/karsten/.composer/auth.json
Loading config file ./composer.json
Checked CA file /usr/local/etc/openssl@1.1/cert.pem: valid
Executing command (/…): git branch -a --no-color --no-abbrev -v
Executing command (/…): git rev-list remotes/origin/master..development
Reading /Users/karsten/.composer/composer.json
Loading config file /Users/karsten/.composer/config.json
Loading config file /Users/karsten/.composer/auth.json
Loading config file /Users/karsten/.composer/composer.json
Loading config file /Users/karsten/.composer/auth.json
Reading /Users/karsten/.composer/auth.json
Reading /…/Packages/Libraries/composer/installed.json
The "ocramius/package-versions" plugin was skipped because it requires a Plugin API version ("^1.0.0") that does not match your Composer installation ("2.0.0"). You may need to run composer update with the "--no-plugins" option.
Reading /Users/karsten/.composer/vendor/composer/installed.json
Loading plugin Neos\ComposerPlugin\Plugin
Loading plugin PackageVersions\Installer
Running 2.0.2 (2020-10-25 23:03:59) with PHP 7.4.12 on Darwin / 19.6.0
Reading /Users/karsten/.composer/cache/files/behat/transliterator/d9ab5cd48ae5eb9bcd43dc2aaeac55af8fb1f093.zip from cache
Executing async command (CWD): unzip -qq  '/…/Packages/Libraries/composer/tmp-4706fa5185beb5b89415a76f78fc1a2b' -d '/…/Packages/Libraries/composer/f0ca26bf'
Executing command (CWD): rm -rf '/…/Packages/Libraries/behat/transliterator_compare'
Executing command (CWD): rm -rf '/…/Packages/Libraries/composer/f0ca26bf'
Executing command (CWD): rm -rf '/…/Packages/Libraries/behat/transliterator_compare'
Reading /Users/karsten/.composer/cache/files/clue/stream-filter/b5b323cbf28c105fa394b1743a7886d3ba1f723a.zip from cache
Executing async command (CWD): unzip -qq  '/…/Packages/Libraries/composer/tmp-5a586e1112012489230dc62c1fb2c5e6' -d '/…/Packages/Libraries/composer/92a6c20e'
Executing command (CWD): rm -rf '/…/Packages/Libraries/clue/stream-filter_compare'
Executing command (CWD): rm -rf '/…/Packages/Libraries/composer/92a6c20e'
Executing command (CWD): rm -rf '/…/Packages/Libraries/clue/stream-filter_compare'
Reading /Users/karsten/.composer/cache/files/composer/ca-bundle/63c568debe0aba21e44215983804b8205bc3e03c.zip from cache
Executing async command (CWD): unzip -qq  '/…/Packages/Libraries/composer/tmp-20a29769ac45b86f62177bcf7cbbca72' -d '/…/Packages/Libraries/composer/51aa95c7'
Executing command (CWD): rm -rf '/…/Packages/Libraries/composer/ca-bundle_compare'
Executing command (CWD): rm -rf '/…/Packages/Libraries/composer/51aa95c7'
Executing command (CWD): rm -rf '/…/Packages/Libraries/composer/ca-bundle_compare'
Reading /Users/karsten/.composer/cache/files/composer/composer/0974db4c23909b971b0338cfe83a164e3f2aa651.zip from cache
Executing async command (CWD): unzip -qq  '/…/Packages/Libraries/composer/tmp-52ea6f56fb82ddb2d333824a31149ccb' -d '/…/Packages/Libraries/composer/264cda8d'
Executing command (CWD): rm -rf '/…/Packages/Libraries/composer/composer_compare'
Executing command (CWD): rm -rf '/…/Packages/Libraries/composer/264cda8d'
Executing command (CWD): rm -rf '/…/Packages/Libraries/composer/composer_compare'
Reading /Users/karsten/.composer/cache/files/composer/semver/763b6d5a5088410e318ef7a97483d50842e7fddd.zip from cache
Executing async command (CWD): unzip -qq  '/…/Packages/Libraries/composer/tmp-b82c025594a2e4fd08770aa3de4aa54e' -d '/…/Packages/Libraries/composer/a518182b'
Executing command (CWD): rm -rf '/…/Packages/Libraries/composer/semver_compare'
Executing command (CWD): rm -rf '/…/Packages/Libraries/composer/a518182b'
Executing command (CWD): rm -rf '/…/Packages/Libraries/composer/semver_compare'
Reading /Users/karsten/.composer/cache/files/composer/spdx-licenses/d6448a134c74473ca6a634b382a8f252307a5b08.zip from cache
Executing async command (CWD): unzip -qq  '/…/Packages/Libraries/composer/tmp-6acf1777a577264cfe471d9f9edfeea0' -d '/…/Packages/Libraries/composer/6a43802b'
Executing command (CWD): rm -rf '/…/Packages/Libraries/composer/spdx-licenses_compare'
Executing command (CWD): rm -rf '/…/Packages/Libraries/composer/6a43802b'
Executing command (CWD): rm -rf '/…/Packages/Libraries/composer/spdx-licenses_compare'
Reading /Users/karsten/.composer/cache/files/composer/xdebug-handler/d207f8dcdbb1e8ea9a22db2d3bf23503ddc7e5d7.zip from cache
Executing async command (CWD): unzip -qq  '/…/Packages/Libraries/composer/tmp-03bf4d7429eaea52db9937195aec640a' -d '/…/Packages/Libraries/composer/896ac602'
Executing command (CWD): rm -rf '/…/Packages/Libraries/composer/xdebug-handler_compare'
Executing command (CWD): rm -rf '/…/Packages/Libraries/composer/896ac602'
Executing command (CWD): rm -rf '/…/Packages/Libraries/composer/xdebug-handler_compare'


  [ErrorException]
  chdir(): No such file or directory (errno 2)


Exception trace:
 () at phar:///Users/karsten/bin/composer.phar/src/Composer/Package/Comparer/Comparer.php:68
 Composer\Util\ErrorHandler::handle() at n/a:n/a
 chdir() at phar:///Users/karsten/bin/composer.phar/src/Composer/Package/Comparer/Comparer.php:68
 Composer\Package\Comparer\Comparer->doCompare() at phar:///Users/karsten/bin/composer.phar/src/Composer/Downloader/FileDownloader.php:445
 Composer\Downloader\FileDownloader->getLocalChanges() at phar:///Users/karsten/bin/composer.phar/src/Composer/Command/StatusCommand.php:117
 Composer\Command\StatusCommand->doExecute() at phar:///Users/karsten/bin/composer.phar/src/Composer/Command/StatusCommand.php:76
 Composer\Command\StatusCommand->execute() at phar:///Users/karsten/bin/composer.phar/vendor/symfony/console/Command/Command.php:245
 Symfony\Component\Console\Command\Command->run() at phar:///Users/karsten/bin/composer.phar/vendor/symfony/console/Application.php:835
 Symfony\Component\Console\Application->doRunCommand() at phar:///Users/karsten/bin/composer.phar/vendor/symfony/console/Application.php:185
 Symfony\Component\Console\Application->doRun() at phar:///Users/karsten/bin/composer.phar/src/Composer/Console/Application.php:309
 Composer\Console\Application->doRun() at phar:///Users/karsten/bin/composer.phar/vendor/symfony/console/Application.php:117
 Symfony\Component\Console\Application->run() at phar:///Users/karsten/bin/composer.phar/src/Composer/Console/Application.php:121
 Composer\Console\Application->run() at phar:///Users/karsten/bin/composer.phar/bin/composer:63
 require() at /Users/karsten/bin/composer.phar:24

status [-v|vv|vvv|--verbose]

[kdambekalns]

A "fool proof" way to fix this might be to make sure $targetPath has no trailing slash in FileDownloader->getLocalChanges()…

[Seldaek]

Yeah that definitely looks like a path repo edge case due to symlinks. I think we can fix this in Composer by trimming the return value from the installers, it should make it more resilient than documenting it :) I'll get on that.

[kdambekalns]

Fellow reviewers: we can still merge the PR, doesn't harm!

[Seldaek]

No you should absolutely fix it here because I can't fix existing composer versions. It's unfortunately not easy to fix it everywhere in one place as I hoped.. But I did fix it where it's most likely to break: composer/composer@bc93369

Can you please try again @kdambekalns using composer self-update --snapshot to see if it works for you?

[kdambekalns]

@Seldaek Yes, with that change it works even when our installer returns paths with trailing slashes. ✅