Windows - Symlink Issues

[ulrikstrid]

Normally only administrators can create symlinks on Windows unless you enable Developer Mode on Windows 10.

https://blogs.windows.com/buildingapps/2016/12/02/symlinks-windows-10/

[bryphe]

Good catch. We should definitely add this to our documentation.

[jordwalke]

Is it possible to remove this limitation? Is this only to allow the standard library to be linked?

[bryphe]

Ya, running through this flow and debugging with @ulrikstrid and @jaredly - I think it would be better to remove this limitation. The npm install -g esy -> esy install -> esy build flow should work out of the box w/o intervention.

A few ideas of dealing with this:

• LONG TERM FIX: Remove symlinks altogether. Use a copy flow instead.
• SHORT TERM FIXES:
  • OPTION 1: Request elevation on Windows, prior to creating a symlink or running esy commands.
  • OPTION 2: Just quit with an error message - like "esy must be run with administrator permissions"

[bryphe]

PR #540 implements the error message solution. We should still continue to pursue the long term solution.

[fspeech]

I have developer mode enabled. According to https://blogs.windows.com/buildingapps/2016/12/02/symlinks-windows-10/ this allows sym link without privilege escalation, yet I am getting this error message. According to the linked page, you should also check SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE. Could you check the right flag or at least offer the user the option to proceed instead of terminating the program?

[manu-unter]

We currently use Unix.has_symlinks to detect if the user can create symlinks with the given setup. That code is here. It seems to check SeCreateSymbolicLinkPrivilege only if I read that correctly. I just checked my privileges on a Windows machine with Developer Mode enabled and found that privilege enabled for me, which means this function should return true with Developer Mode enabled. It doesn't seem to do so though. Maybe we have a bug there?

Another point is the fact that we will need to pass a new flag - SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE - to CreateSymbolicLink in this piece of code so that the symlinks will be created without an elevated shell.

Once we figured these two out, I think it should be possible to support running esy without elevation.

It would require a new version of ocaml, though :(

[ManasJayanth]

Esy itself doesn't critically need symlinks. Tarring/untarring of sources are a bigger issue. For instance, (pasting logs from slow-tests)

 ** Iteration: 2
EXEC: C:/Users/manas/development/esy/bin/esy.cmd install
info install 0.6.0 (using esy.json)
info fetching: done
error command failed: "tar" "xf" "/cygdrive/c/Users/manas/AppData/Local/Temp/2ae948cf0d340541/reason/_esy/default/dist/f34f64b259f0f67c9f07cb2331f9cbf8" "-C" "/cygdrive/c/Users/manas/AppData/Local/Temp/esy-fdd963"
      stderr:
               tar: ocamlbuild-0.14.0/examples/07-dependent-projects/libdemo: Cannot create symlink to '../04-library/libdemo': Operation not permitted
               tar: Exiting with failure status due to previous errors

      stdout:

error: error running command
  unpacking @opam/ocamlbuild@opam:0.14.0
  installing @opam/ocamlbuild@opam:0.14.0
esy: exiting due to errors above
Exception: Error: Command failed: C:/Users/manas/development/esy/bin/esy.cmd install
 ** Iteration: 3
EXEC: C:/Users/manas/development/esy/bin/esy.cmd install
info install 0.6.0 (using esy.json)
info fetching: done
error command failed: "tar" "xf" "/cygdrive/c/Users/manas/AppData/Local/Temp/2ae948cf0d340541/reason/_esy/default/dist/f34f64b259f0f67c9f07cb2331f9cbf8" "-C" "/cygdrive/c/Users/manas/AppData/Local/Temp/esy-1d6681"
      stderr:
               tar: ocamlbuild-0.14.0/examples/07-dependent-projects/libdemo: Cannot create symlink to '../04-library/libdemo': Operation not permitted
               tar: Exiting with failure status due to previous errors

      stdout:

error: error running command
  unpacking @opam/ocamlbuild@opam:0.14.0
  installing @opam/ocamlbuild@opam:0.14.0
esy: exiting due to errors above

GNU tar seems to rely on symlinks

[DrYSG]

So my company does give us an windows image to enable developer mode. But I do have Admin privileges. Right now, I have to run VSCODE as an admin, and then run esy from the elevated terminal.

I am getting stuck here: esy-ocaml/hello-reason#62

it would be nice if there is progress on this:
#1004

[manu-unter]

I'm happy to say that ocaml 4.12 will hopefully not require elevation anymore when developer mode is enabled.
As soon as that lands, we could update the error message to inform the user about this new option, with something like "run esy in an elevated shell or enable developer mode if you are using ocaml 4.12 or higher" - of course with a bit more detail than that.

What do you think?

[ManasJayanth]

@manuhornung I'm considering (as suggested by Bryan) borrowing the C stub from you PR so that we dont have to wait for 4.12. What do you think? We can update the instructions right away then.

[manu-unter]

Oh sure, feel free to copy it downstream for the time being. I won't find the time to focus on that since I'll be working on the hinting / antialias issue and the conceptual work for a new component API, but I'll be happy if you want to take this over!

[Fuwn]

Came across this issue myself when trying to run esy install; if anyone comes across this issue as well, I recommend you use gsudo to run the commands as an elevated user.

[ruifortes]

not trying to be a prick (I don't know enough for that) but... how come https://pnpm.js.org/ has no problem with symlinks?

[manu-unter]

@ruifortes to answer your question literally: I would assume that the code behind pnpm already builds on the newer Windows APIs which is why there are no issues there. We worked towards upgrading to those in OCaml 4.12 too.

The good news is that OCaml 4.12 has been released a month ago and we could now test if the issues are fixed and change the error message accordingly

[ahmadrasyidsalim]

Even when developer mode is enabled, I got the error 😢

PowerShell 7.1.3
Copyright (c) Microsoft Corporation.

https://aka.ms/powershell
Type 'help' to get help.

PS C:\Users\ahmad> npm i -g esy
C:\Program Files\nodejs\esy -> C:\Program Files\nodejs\node_modules\esy\bin\esy

> esy@0.6.10 postinstall C:\Program Files\nodejs\node_modules\esy
> node -e "process.env['OCAML_VERSION'] = process.platform == 'linux' ? '4.10.1002-musl.static.flambda': '4.10.0'; process.env['OCAML_PKG_NAME'] = 'ocaml'; require('./postinstall.js')"

Installing native compiler toolchain for Windows...
Native compiler toolchain installed successfully.
importing: esy-4336793f
Done!
+ esy@0.6.10
added 1 package in 110.853s
PS C:\Users\ahmad> esy --help
Fatal error: exception Unix.Unix_error(_, "create_process", "C:\Program Files\??\C\Users\ahmad\AppData\Roaming\nvm\v14.16.0\node_modules\esy\3/i/esy-4336793f/bin/esy.exe")
PS C:\Users\ahmad>

Fatal error: exception Unix.Unix_error(_, "create_process", "C:\Program Files\??\C\Users\ahmad\AppData\Roaming\nvm\v14.16.0\node_modules\esy\3/i/esy-4336793f/bin/esy.exe")

[EduardoRFS]

@ahmadrasyidsalim yeah that's because esy was not built using ocaml@4.12.x

[alminisl]

@ahmadrasyidsalim yeah that's because esy was not built using ocaml@4.12.x

and there is no way to install Ocaml on windows.. I tried for hours.. so no way to build it on windows.

[EduardoRFS]

@alminisl you can build esy using esy, and that's what we do on the CI.

[alminisl]

@alminisl you can build esy using esy, and that's what we do on the CI.

will try that, thanks

[manu-unter]

Super nice, thanks for taking care of this!

On April 27, 2021, laike9m ***@***.***> wrote: Closed #389 <#389> via #1292 <#1292>.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#389 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe- auth/ABCOXPUQ5BJHOI2QV4ESLNTTKYDOZANCNFSM4FRIRKCA>.