Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement a winsymlinks mode that prefers native symlinks, falling back to the deep copy mode #114

Open
wants to merge 2 commits into
base: msys2-3.5.3
Choose a base branch
from

Conversation

dscho
Copy link
Collaborator

@dscho dscho commented Nov 7, 2022

It surprises new MSYS2 users no end that ln -s does not create symbolic links at all, but deep copies (and with an exit code indicating success!). This did not at all match the expectations of those users who were familiar with Unix' concept of symbolic links and thought that they could rely on MSYS2 providing those, too, or fail with a non-zero exit code.

Historical reasons are at play here: When MSYS2 was started (or was that already the behavior of MSys? I forget...), symbolic links were not supported on Windows, at least not really: you had to have administrator privileges to create them (but not to delete them... 🤷) in Windows Vista, and before that, Windows simply had no idea about symbolic links.

So what about Cygwin? Well, Cygwin had something like support for symbolic links, using .lnk files for the emulation. The only problem? You had to stay within Cygwin's walled garden to make use of them. All non-Cygwin applications would react with a less or more unpleasant "huh?!?" when encountering those "symbolic links".

That's why MSYS2 chose to deep-copy by default. At least that way ./configure would still work for those projects that required symbolic links. This was instrumental in getting MSYS2's package ecosystem off the ground.

Even when a Windows 10 update introduced support for creating symbolic links without elevation as long as Windows was run in Developer Mode, the created symbolic links are not completely what Unix/Linux/macOS users may be used to, as Windows discerns between directory symlinks and file symlinks.

Be that as it may, now that we've dropped Windows 7 and Windows 8 support, it may be a good time to start switching the default to creating actual symbolic links by default.

Since we still support Windows 8.1 (and a couple of Windows 10 versions that do not allow creating symbolic links in non-elevated operations, even in Developer Mode), we cannot simply switch to a mode where the MSYS2 runtime creates symbolic links when asked for, but we have to have a mode where the MSYS2 runtime first checks whether that is possible with the Windows version on which it is running, and if not, falls back to the deep-copy.

This PR does precisely that: implement that mode, but does not yet flip the default away from deepcopy. The reason is that I want this to be tested by volunteers (myself included) first, and once it is deemed robust and stable enough, flip the default to nativeordeepcopy.

This addresses #113.

@dscho dscho self-assigned this Nov 7, 2022
@lazka lazka added the symlinks label Jan 27, 2024
@dscho dscho changed the base branch from msys2-3_3_6-release to msys2-3.5.3 July 24, 2024 12:58
@dscho dscho force-pushed the native-symlinks-with-deepcopy-fallback branch from e69c9f4 to cb31af7 Compare July 24, 2024 12:58
This adds the missing documentation (in case we ever publish the
information about the MSYS variable on msys2.org).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
…llback

When native symlinks are available, it is a shame to create deep copies
by default.

However, since there are many scenarios where symlinks are not available
(e.g. when running on FAT, or on older Windows versions, or when
Developer Mode is not enabled), we've got to have a fallback.

In the regular Cygwin world, it is legitimate to fall back to WSL
symlinks and/or to the system file emulation (where a file is created
that is marked with the "system" attribute and with content that adheres
to a specific, magic form that is recognized specifically by the Cygwin
runtime).

However, in the world of MSYS2, the assumption is that the result of the
operation should be as interoperable with regular Win32 programs as
possible. Hence the default to "deepcopy".

As a "best of both worlds" mode, let's implement one that tries to
create native symlinks by default, and if that fails, uses the
"deepcopy" method as a fallback.

This addresses msys2#113.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
@dscho dscho force-pushed the native-symlinks-with-deepcopy-fallback branch from cb31af7 to b8e842e Compare July 24, 2024 19:51
@dscho dscho marked this pull request as ready for review July 24, 2024 20:28
@dscho dscho requested a review from lazka July 24, 2024 20:29
@jcrben
Copy link

jcrben commented Aug 25, 2024

Well, Cygwin had something like support for symbolic links, using .lnk files for the emulation.

Cygwin not only had, it still has symlinks like that - except the docs suggest that the *.lnk are not the default. The cygwin docs say this about their default symlinks https://cygwin.com/cygwin-ug-net/using.html#pathnames-symlinks

special reparse points shared with WSL (on NTFS on Windows 10 1607 or later)
plain files with the system attribute, containing a magic cookie followed by the path to which the link points.

The "shortcut" style links are something different.

The new Developer Mode feature is OK, but it's got a big warning on it when you enable it and enterprises therefore don't want to enable it:

Turning on developer mode, including installing and running apps from outside the Microsoft Store, could expose your device and personal data to security risks or harm your device.

https://stackoverflow.com/questions/74613686/what-does-the-warning-in-developer-mode-mean

I set export MSYS=winsymlinks:sys as I believe this restores the cygwin default special reparse point symlinks, which is fine by me. I also found https://carltonf.github.io/post/symbolic-links-on-windows sort of helpful in dumbing it down

@dscho
Copy link
Collaborator Author

dscho commented Aug 26, 2024

MSYS=winsymlinks:sys

This mode is incompatible with regular Win32 programs; They won't understand those symlinks and misinterpret them all the time. That is an okay stance to take for Cygwin, which wants you to stay within its ecosystem. It is not an okay stance for MSYS2 which wants to integrate with native Win32 programs as much as possible.

The new Developer Mode feature is OK, but it's got a big warning on it when you enable it and enterprises therefore don't want to enable it:

Turning on developer mode, including installing and running apps from outside the Microsoft Store, could expose your device and personal data to security risks or harm your device.

Yes, developer mode comes with a lot of liberties that developing code requires. Nevertheless, it is the mode in which Windows users can create symbolic links in modern Windows without requiring an elevated process (which would increase the security risk a lot more than Developer Mode).

In any case, the mode I introduce here, and which I propose to promote to eventually be the default, retains backwards-compatibility in a fashion, by falling back to MSYS2's current behavior if symbolic links cannot be created.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants