relative symlinks do not work on Windows #62

fritzmg · 2016-11-21T12:18:49Z

When trying to install a Contao 3 extension in Contao 4 that is compatible with Contao 4, the composer-plugin tries to create a relative symlink from /system/modules/… to /vendor/… » https://github.com/contao-community-alliance/composer-plugin/blob/3.0.4/src/Installer/AbstractModuleInstaller.php#L240

However, this will fail on Windows. See contao/core-bundle#208 (comment).

The contao/core-bundle for example will fall back to absolute symlinks in such a case, and so does Symfony.

The text was updated successfully, but these errors were encountered:

aschempp · 2016-11-28T07:31:07Z

@discordier any reason we're not using AppVeyor to test on windows?

discordier · 2016-11-28T09:05:04Z

Not really. Will set it up.

discordier · 2016-11-28T14:55:45Z

@fritzmg could you provide a unit test? I just set up appveyor which should then point out the problem.

fritzmg · 2016-11-28T14:59:32Z

I don't know, I have never created unit tests 😁
I'll see what I can do.

bezin · 2016-12-08T17:53:31Z

I just stumbled upon the same issue while trying to make an Contao 3.5 extension compatible with Contao 4 and I can confirm this.

discordier · 2016-12-08T20:53:59Z

Unit test? Example information? Anything?

These "mee too" comments lead us to nowhere, we need test data to reproduce a problem.
This means you should supply us with the minimum file layout and configuration to reproduce the problem.

In your comment you write that:

you "can confirm this", this was undisputed.
you had the problem while trying to make an Contao 3.5 extension compatible - which extension? We can not have a look ourselves what went wrong.

Please supply relevant information. Thanks.

fritzmg · 2016-12-09T08:58:55Z

There is already a test for symlinks: https://github.com/contao-community-alliance/composer-plugin/blob/3.0.4/tests/ContaoModuleInstallerTest.php#L99-L130

aschempp · 2016-12-13T07:23:23Z

@discordier do we have AppVeyor now?

discordier · 2016-12-13T17:57:16Z

We should, however we have build issues as it seems, will bump the appveyor config from core-bundle and check if it works out.
https://ci.appveyor.com/project/contaocommunityalliance/composer-plugin

discordier · 2016-12-13T19:15:00Z

Appveyor is up and running and failing tests like a charm.
However, there are completely different tests that are failing as it seems. some related to path naming.
Will try to sort them out.

yet the original issue stands, does the above linked test reproduce the issue by @fritzmg or do we need another unit test for this?

discordier · 2016-12-14T02:47:40Z

Ok, I fixed several problems already. However, appveyor still breaks on 3 tests and I fail to see why.

I fired up a Windows at a customer and ran the tests there. Only one failing there.

@fritzmg You seem to have comprehensive experience with Windows debugging of the symlink issues, could you have a look on the remaining test cases and tell me why they might fail to create the links?

Funny part is, during my testing the creation of the symlink went just fine, just the validation of a relative symlink failed due to Windows failing to execute the readlink() call and erroring out with:

C:\projekte\composer-plugin>vendor\bin\phpunit.bat
PHPUnit 4.8.31 by Sebastian Bergmann and contributors.

...............E......................

Time: 24.31 seconds, Memory: 8.00MB

There was 1 error:

1) ContaoCommunityAlliance\Composer\Plugin\Test\LegacyContaoModuleInstallerTest::testSourcesOnInstallIfSymlinkIsRelative

readlink(): readlink failed to read the symbolic link ([...]/composer-plugin-test-8679005b/vendor/foo/bar/../../../system/modules/foobar/config/config.php), error 123)

[...]\vendor\symfony\phpunit-bridge\DeprecationErrorHandler.php:73
[...]\tests\LegacyContaoModuleInstallerTest.php:107

FAILURES!
Tests: 38, Assertions: 93, Errors: 1.

fritzmg · 2016-12-14T07:01:38Z

Which PHP version is used for testing?

discordier · 2016-12-14T16:51:13Z

5.6 and 7.0 so far.

fritzmg · 2016-12-14T17:04:55Z

Hm, it's weird that the creation of the relative symlink works. It still has not been fixed in any PHP version, including PHP 7: php/php-src#1243

I have no experience with AppVeyor, how exactly are these tests run? Can I run these tests locally?

fritzmg · 2016-12-14T17:19:52Z

I ran

php phpunit.phar --bootstrap tests/bootstrap.php tests/ContaoModuleInstallerTest

locally and got

There were 2 errors:

1) ContaoCommunityAlliance\Composer\Plugin\Test\ContaoModuleInstallerTest::testSourcesOnInstall
RuntimeException: Failed to create symlink C:/…/Temp/composer-plugin-test-700d65ff/system/modules/foobar/config/config.php

C:\…\composer-plugin\src\Installer\AbstractModuleInstaller.php:241
C:\…\composer-plugin\src\Installer\AbstractModuleInstaller.php:91
C:\…\composer-plugin\tests\ContaoModuleInstallerTest.php:122

2) ContaoCommunityAlliance\Composer\Plugin\Test\ContaoModuleInstallerTest::testSourcesOnInstallIgnoresIfLinkIsAlreadyCorrect
RuntimeException: Installation target "C:/…/Temp/composer-plugin-test-822ba422/system/modules/foobar/config/config.php" already exists

C:\…\composer-plugin\src\Installer\AbstractModuleInstaller.php:529
C:\…\composer-plugin\src\Installer\AbstractModuleInstaller.php:459
C:\…\composer-plugin\src\Installer\AbstractModuleInstaller.php:229
C:\…\composer-plugin\src\Installer\AbstractModuleInstaller.php:91
C:\…\composer-plugin\tests\ContaoModuleInstallerTest.php:158

So the creation of the symlink fails as expected.

fritzmg · 2017-02-13T08:53:51Z

Any updates on this? This makes developing a Contao 4 website currently rather inconvenient for me. I can make a PR for the absolute symlink fallback if you want.

aschempp · 2017-02-13T10:06:36Z

So the necessary changes is basically this?
contao/core-bundle@77c1c10#diff-eebbebd36be2522e4a8c571895b59325R183

Do you get the IOException?

fritzmg · 2017-02-13T10:10:15Z

So the necessary changes is basically this?
contao/core-bundle@77c1c10#diff-eebbebd36be2522e4a8c571895b59325R183

I guess, I haven't yet looked at how it was actually implemented in Contao 4.

Do you get the IOException?

No, but the IOException comes from Symfony. The composer plugin does not use Symfony for symlinks, does it?

fritzmg · 2017-02-13T22:41:55Z

Also Contao does not use a fallback on failure at all. It simply checks if the directory separator is \ and if so, it creates absolute symlinks: https://github.com/contao/core-bundle/blob/4.3.3/src/Util/SymlinkUtil.php#L38-L39

~~But currently even that fails in my testing environment with PHP 7.1.1.~~

discordier · 2017-02-14T21:05:13Z

So both are failing for you? core-bundle and composer-plugin?

fritzmg · 2017-02-21T14:53:17Z

So both are failing for you? core-bundle and composer-plugin?

No, core-bundle works, there was some other issue. But composer-plugin still gives me the initially described problem.

$ composer require fritzmg/contao-inherit-article
Using version ^1.3 for fritzmg/contao-inherit-article
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
  - Installing contao-community-alliance/composer-plugin (3.0.4)
    Downloading: 100%

  - Installing fritzmg/contao-inherit-article (1.3.3)
    Downloading: 100%


Installation failed, reverting ./composer.json to its original content.


  [RuntimeException]
  Failed to create symlink C:/xampp/htdocs/c43dev/system/modules/inherit_article

aschempp · 2017-03-26T18:58:33Z

@discordier I saw you added a branch related to this, can this be finalized so we can release 3.0.5 ?

discordier · 2017-03-27T16:27:42Z

We can surely try to finalize this, yet I lack the time to do this right now.

Maybe I could try to squeeze it in next week.

m-vo · 2017-04-05T14:59:06Z

(I stumbled across this topic when trying to add codefog/haste (which requires the ComposerPlugin) to a contao 4.3 installation on a windows machine (PHP7) resulting in the same Failed to create symlink xyz error.)

...so just to sum it up:

addSymlinks(...) gets called during install and normalizes source and target paths
it then calls Composer\Util\Filesystem->relativeSymlink(...)

[...]
public function relativeSymlink($target, $link) {
  $cwd = getcwd();
  $relativePath = $this->findShortestPath($link, $target);
  chdir(dirname($link));
  $result = @symlink($relativePath, $link);
  chdir($cwd);
  return (bool) $result;
}

there @symlink($relativePath, $link) fails with the error symlink(): Could not fetch file information(error 3)
this is exactly the same output I get if running the commands (chdir(...) + symlink(...)) directly on the command line
if I understand correctly this is the expected behaviour as creating relative symlinks unfortunately does not work with PHP on windows (at least not in this way)

Am I right to assume that the only working solution then is to use absolute links as a fallback? Sorry if this was clear all the time - I am just trying to understand the core of the problem and maybe help debugging/testing. 😉

EDIT: btw., thats how the actual paths look like:

$link = "X:/xampp/htdocs/test/system/modules/haste"
$relativePath = "../../vendor/codefog/contao-haste"

fritzmg · 2017-04-05T15:50:14Z

Am I right to assume that the only working solution then is to use absolute links as a fallback?

Yes 😉 . Symfony does the same fallback.

m-vo · 2017-04-05T17:09:04Z

Right, I did a quick test and it seems to work without a problem so far!

For my test, I just replaced this (https://github.com/contao-community-alliance/composer-plugin/blob/3.0.4/src/Installer/AbstractModuleInstaller.php#L240-L242):

if (!$this->filesystem->relativeSymlink($source, $target)) {
    throw new \RuntimeException('Failed to create symlink ' . $target);
}

with this:

if('\\' == DIRECTORY_SEPARATOR) {
    if (!@symlink($source, $target)) {
        throw new \RuntimeException('Failed to create absolute symlink (fallback) ' . $target);
    }
}
else {
    if (!$this->filesystem->relativeSymlink($source, $target)) {
        throw new \RuntimeException('Failed to create symlink ' . $target);
    }
}

fritzmg · 2017-04-21T06:43:56Z

@m-vo looks good, tested it as well. Can you please make a PR?

m-vo · 2017-04-23T11:14:28Z

@fritzmg Allright, I'll try my best (might take until mid next week, though).

fritzmg · 2017-04-23T12:49:35Z

Since it's a simple fix, you could also just use the edit button here on GitHub ;)

m-vo · 2017-04-23T13:00:03Z

You're so right :-) First time I tried the online editor, hope it worked.

aschempp · 2017-04-25T10:21:01Z

Fixed in #67

discordier added bug need-windows labels Nov 21, 2016

discordier added this to the 3.0.5 milestone Nov 21, 2016

aschempp self-assigned this Nov 24, 2016

discordier assigned discordier and unassigned aschempp Dec 14, 2016

m-vo mentioned this issue Apr 23, 2017

symlink fallback for windows #67

Merged

aschempp closed this as completed Apr 25, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

relative symlinks do not work on Windows #62

relative symlinks do not work on Windows #62

fritzmg commented Nov 21, 2016 •

edited

aschempp commented Nov 28, 2016 •

edited

discordier commented Nov 28, 2016

discordier commented Nov 28, 2016

fritzmg commented Nov 28, 2016

bezin commented Dec 8, 2016 •

edited

discordier commented Dec 8, 2016

fritzmg commented Dec 9, 2016

aschempp commented Dec 13, 2016

discordier commented Dec 13, 2016

discordier commented Dec 13, 2016

discordier commented Dec 14, 2016

fritzmg commented Dec 14, 2016

discordier commented Dec 14, 2016

fritzmg commented Dec 14, 2016 •

edited

fritzmg commented Dec 14, 2016 •

edited

fritzmg commented Feb 13, 2017

aschempp commented Feb 13, 2017

fritzmg commented Feb 13, 2017

fritzmg commented Feb 13, 2017 •

edited

discordier commented Feb 14, 2017

fritzmg commented Feb 21, 2017

aschempp commented Mar 26, 2017

discordier commented Mar 27, 2017

m-vo commented Apr 5, 2017 •

edited

fritzmg commented Apr 5, 2017

m-vo commented Apr 5, 2017

fritzmg commented Apr 21, 2017

m-vo commented Apr 23, 2017

fritzmg commented Apr 23, 2017

m-vo commented Apr 23, 2017

aschempp commented Apr 25, 2017

relative symlinks do not work on Windows #62

relative symlinks do not work on Windows #62

Comments

fritzmg commented Nov 21, 2016 • edited

aschempp commented Nov 28, 2016 • edited

discordier commented Nov 28, 2016

discordier commented Nov 28, 2016

fritzmg commented Nov 28, 2016

bezin commented Dec 8, 2016 • edited

discordier commented Dec 8, 2016

fritzmg commented Dec 9, 2016

aschempp commented Dec 13, 2016

discordier commented Dec 13, 2016

discordier commented Dec 13, 2016

discordier commented Dec 14, 2016

fritzmg commented Dec 14, 2016

discordier commented Dec 14, 2016

fritzmg commented Dec 14, 2016 • edited

fritzmg commented Dec 14, 2016 • edited

fritzmg commented Feb 13, 2017

aschempp commented Feb 13, 2017

fritzmg commented Feb 13, 2017

fritzmg commented Feb 13, 2017 • edited

discordier commented Feb 14, 2017

fritzmg commented Feb 21, 2017

aschempp commented Mar 26, 2017

discordier commented Mar 27, 2017

m-vo commented Apr 5, 2017 • edited

fritzmg commented Apr 5, 2017

m-vo commented Apr 5, 2017

fritzmg commented Apr 21, 2017

m-vo commented Apr 23, 2017

fritzmg commented Apr 23, 2017

m-vo commented Apr 23, 2017

aschempp commented Apr 25, 2017

fritzmg commented Nov 21, 2016 •

edited

aschempp commented Nov 28, 2016 •

edited

bezin commented Dec 8, 2016 •

edited

fritzmg commented Dec 14, 2016 •

edited

fritzmg commented Dec 14, 2016 •

edited

fritzmg commented Feb 13, 2017 •

edited

m-vo commented Apr 5, 2017 •

edited