Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Environments for PowerShell and Windows command line #4816

Merged
merged 4 commits into from
Jul 26, 2022

Conversation

jonahbeckford
Copy link
Contributor

Splitting up #4813:

For 0f0bd98 ("Show eval expression for PowerShell on native Windows"), opam 2.2 should finally see the activation of parent_putenv (and the opam-putenv auxiliary)... on Windows, there's no need to output shell instructions - on Windows, opam env can actually update the environment variables of the shell (see dra27/opam@2934dc4#diff-e21e37a0339dbaafa95fd0b6d57221f656f18d0771c06f6f837bb98c3a71b91aR193). However, that doesn't necessarily preclude merging this first!

For continuity this initial PR is as-is from the original PR.

@kit-ty-kate kit-ty-kate requested a review from dra27 August 30, 2021 18:03
@kit-ty-kate kit-ty-kate added this to PR in progress in Opam 2.2.0 via automation Aug 30, 2021
@kit-ty-kate kit-ty-kate added this to the 2.2.0~alpha milestone Aug 30, 2021
@jonahbeckford
Copy link
Contributor Author

Question: Somewhat related to this PR ... opam env is showing Windows paths like it should be on my Windows distribution:

OPAM_SWITCH_PREFIX='C:\Users\beckf\AppData\Local\.opam\diskuv-boot-DO-NOT-DELETE'; export OPAM_SWITCH_PREFIX;
MANPATH=';C:\Users\beckf\AppData\Local\.opam\diskuv-boot-DO-NOT-DELETE\man'; export MANPATH;
PATH='C:\Users\beckf\AppData\Local\.opam\diskuv-boot-DO-NOT-DELETE\bin;Z:\ProgramFiles\VMware\VMware Player\bin\;...;Z:\ProgramFiles\CMake\bin'; export PATH;

However, these Windows paths can only work in native Windows (ex. PowerShell) but not in MSYS2. Among other things that means eval $(opam env) won't work as-is in MSYS2.

Should opam env be aware of its context?

@dra27
Copy link
Member

dra27 commented Aug 31, 2021

Indeed, the check here is not quite correct: even on Windows, opam needs to be looking at its environment to work out the shell.

To remain compatible with running native opam inside MSYS2, we should definitely inspect the SHELL environment variable first. With this patch, OpamStubs.(getParentProcessName (getCurrentProcessID ())) will return cmd.exe if opam is called from cmd and powershell.exe if it's called from PowerShell. The API might not quite be right for, say, calling it from Dune (where the parent process name will obviously by dune.exe) - it might be that the API wants to search for the first parent called cmd.exe or powershell.exe to classify the shell, but this is a reasonable first cut!

diff --git a/src/core/opamStubs.dummy.ml b/src/core/opamStubs.dummy.ml
index a7afdee28..6dbb477c5 100644
--- a/src/core/opamStubs.dummy.ml
+++ b/src/core/opamStubs.dummy.ml
@@ -34,5 +34,6 @@ let process_putenv _ = that's_a_no_no
 let shGetFolderPath _ = that's_a_no_no
 let sendMessageTimeout _ _ _ _ _ = that's_a_no_no
 let getParentProcessID = that's_a_no_no
+let getParentProcessName = that's_a_no_no
 let getConsoleAlias _ = that's_a_no_no
 let win_create_process _ _ _ _ _ = that's_a_no_no
diff --git a/src/core/opamStubs.mli b/src/core/opamStubs.mli
index cbbcd67a5..891264164 100644
--- a/src/core/opamStubs.mli
+++ b/src/core/opamStubs.mli
@@ -124,6 +124,12 @@ val getParentProcessID : int32 -> int32

     @raise Failure If walking the process tree fails to find the process. *)

+val getParentProcessName : int32 -> string
+(** Windows only. [getParentProcessName pid] returns the executable name of the
+    parent of [pid].
+
+    @raise Failure If walking the process tree fails to find the process. *)
+
 val getConsoleAlias : string -> string -> string
 (** Windows only. [getConsoleAlias alias exeName] retrieves the value for a
     given executable or [""] if the alias is not defined. See
diff --git a/src/stubs/win32/opamWin32Stubs.ml b/src/stubs/win32/opamWin32Stubs.ml
index 9fe73f738..fdd1e32f3 100644
--- a/src/stubs/win32/opamWin32Stubs.ml
+++ b/src/stubs/win32/opamWin32Stubs.ml
@@ -33,4 +33,5 @@ external process_putenv : int32 -> string -> string -> bool = "OPAMW_process_put
 external shGetFolderPath : int -> 'a -> string = "OPAMW_SHGetFolderPath"
 external sendMessageTimeout : nativeint -> int -> int -> 'a -> 'b -> 'c -> int * 'd = "OPAMW_SendMessageTimeout_byte" "OPAMW_SendMessageTimeout"
 external getParentProcessID : int32 -> int32 = "OPAMW_GetParentProcessID"
+external getParentProcessName : int32 -> string = "OPAMW_GetParentProcessName"
 external getConsoleAlias : string -> string -> string = "OPAMW_GetConsoleAlias"
diff --git a/src/stubs/win32/opamWindows.c b/src/stubs/win32/opamWindows.c
index 8e357dad1..39b82c6a8 100644
--- a/src/stubs/win32/opamWindows.c
+++ b/src/stubs/win32/opamWindows.c
@@ -726,6 +726,26 @@ CAMLprim value OPAMW_GetParentProcessID(value processId)
   OPAMreturn(caml_copy_int32(entry.th32ParentProcessID));
 }

+CAMLprim value OPAMW_GetParentProcessName(value processId)
+{
+  CAMLparam1(processId);
+
+  PROCESSENTRY32 entry;
+  DWORD parent_pid;
+  char* msg;
+  HANDLE hProcessSnapshot = CreateToolhelp32Snapshot(TH32CS_SNAPPROCESS, 0);
+
+  if ((msg = getProcessInfo(hProcessSnapshot, Int32_val(processId), &entry)))
+    caml_failwith(msg);
+
+  if ((msg = getProcessInfo(hProcessSnapshot, entry.th32ParentProcessID, &entry)))
+    caml_failwith(msg);
+
+  CloseHandle(hProcessSnapshot);
+
+  CAMLreturn(caml_copy_string(entry.szExeFile));
+}
+
 CAMLprim value OPAMW_GetConsoleAlias(value alias, value exeName)
 {
   CAMLparam2(alias, exeName);

The story - as with everything on Windows - is complicated (!!). In principle, one should be using an opam matching the compiler you're working with. So with a native Windows opam, you should be using that in native Windows to create native Windows switches. From that, for example, one shouldn't expect to use Ubuntu's opam in WSL to create native Windows switches (unless you're using opam-cross, but that's a whole other story). So pedantically, I don't expect native opam to be used "in MSYS2" or "in Cygwin". However, the same is not true for "in MSYS2's bash" or "in Cygwin's bash" - i.e. it's reasonable to be running in their shells, since they provide a fully working environment (the same is not quite true for calling native Windows opam inside WSL1's bash, because the commands exposed by WSL1 are not usable by that native Windows opam).

@jonahbeckford jonahbeckford changed the title Show eval expression for PowerShell on native Windows Environments for PowerShell and Windows command line Sep 1, 2021
@jonahbeckford
Copy link
Contributor Author

Okay, I overhauled the PR.

What is missing is the contents of the environment (ex. PATH should be cygpath'd within Cygwin and MSYS2), but I think that belongs in a separate PR.

It prints the following ...

MSYS2 and Cygwin

/z/source/opam$ env OPAMSWITCH= _build/default/src/client/opamMain.exe switch --root $LOCALAPPDATA/.opam
#  switch                                                      compiler                            description
   C:\Users\beckf\AppData\Local\Programs\DiskuvOCaml\0\system  ocaml-variants.4.12.0+msvc64+msys2  C:\Users\beckf\AppData\Local\Programs\DiskuvOCaml\0\system
   C:\Users\beckf\AppData\Local\Programs\DiskuvOCaml\1\system  ocaml-variants.4.12.0+msvc64+msys2  C:\Users\beckf\AppData\Local\Programs\DiskuvOCaml\1\system
   Z:\source\diskuv-ocaml-starter\build\dev\Debug              ocaml-variants.4.12.0+msvc64+msys2  Z:\source\diskuv-ocaml-starter\build\dev\Debug
→  diskuv-boot-DO-NOT-DELETE                                                                       diskuv-boot-DO-NOT-DELETE

[WARNING] The environment is not in sync with the current switch.
          You should run: eval $(opam env)
/z/source/opam$ env _build/default/src/client/opamMain.exe env --root $LOCALAPPDATA/.opam
OPAM_SWITCH_PREFIX='Z:\source\diskuv-ocaml-starter\build\dev\Debug\_opam'; export OPAM_SWITCH_PREFIX;
OCAMLLIB='Z:\source\diskuv-ocaml-starter\build\dev\Debug\_opam/lib/ocaml'; export OCAMLLIB;
CC='cl.exe'; export CC;
...

Command Prompt

C:\Users\beckf>Z:\source\opam\_build\default\src\client\opamMain.exe switch --root %LOCALAPPDATA%\.opam
#  switch                                                      compiler                            description
   C:\Users\beckf\AppData\Local\Programs\DiskuvOCaml\0\system  ocaml-variants.4.12.0+msvc64+msys2
          C:\Users\beckf\AppData\Local\Programs\DiskuvOCaml\0\system
   C:\Users\beckf\AppData\Local\Programs\DiskuvOCaml\1\system  ocaml-variants.4.12.0+msvc64+msys2
          C:\Users\beckf\AppData\Local\Programs\DiskuvOCaml\1\system
   Z:\source\diskuv-ocaml-starter\build\dev\Debug              ocaml-variants.4.12.0+msvc64+msys2
          Z:\source\diskuv-ocaml-starter\build\dev\Debug
→  diskuv-boot-DO-NOT-DELETE
          diskuv-boot-DO-NOT-DELETE

[WARNING] The environment is not in sync with the current switch.
          You should run: for /f "tokens=*" %i in ('opam env --root=C:\Users\beckf\AppData\Local\.opam') do @%i

C:\Users\beckf>Z:\source\opam\_build\default\src\client\opamMain.exe env --root %LOCALAPPDATA%\.opam --switch Z:\source\diskuv-ocaml-starter\build\dev\Debug --set-switch
[NOTE] To make opam select C:\Users\beckf\AppData\Local\.opam as its root in the current shell, add --set-root or set
       OPAMROOT
SET OPAMSWITCH=Z:\source\diskuv-ocaml-starter\build\dev\Debug
SET OPAM_SWITCH_PREFIX=Z:\source\diskuv-ocaml-starter\build\dev\Debug\_opam
SET OCAMLLIB=Z:\source\diskuv-ocaml-starter\build\dev\Debug\_opam/lib/ocaml
SET CC=cl.exe
...

C:\Users\beckf>for /f "tokens=*" %i in ('Z:\source\opam\_build\default\src\client\opamMain.exe env --root=C:\Users\beckf\AppData\Local\.opam --switch Z:\source\diskuv-ocaml-starter\build\dev\Debug --set-root --set-switch') do @%i

C:\Users\beckf>Z:\source\opam\_build\default\src\client\opamMain.exe switch
#  switch                                                      compiler                            description
   C:\Users\beckf\AppData\Local\Programs\DiskuvOCaml\0\system  ocaml-variants.4.12.0+msvc64+msys2
          C:\Users\beckf\AppData\Local\Programs\DiskuvOCaml\0\system
   C:\Users\beckf\AppData\Local\Programs\DiskuvOCaml\1\system  ocaml-variants.4.12.0+msvc64+msys2
          C:\Users\beckf\AppData\Local\Programs\DiskuvOCaml\1\system
→  Z:\source\diskuv-ocaml-starter\build\dev\Debug              ocaml-variants.4.12.0+msvc64+msys2
          Z:\source\diskuv-ocaml-starter\build\dev\Debug
   diskuv-boot-DO-NOT-DELETE
          diskuv-boot-DO-NOT-DELETE

[NOTE] Current switch is set locally through the OPAMSWITCH variable.
       The current global system switch is diskuv-boot-DO-NOT-DELETE.

PowerShell

PS Z:\source\opam> $env:OPAMSWITCH=""; Z:\source\opam\_build\default\src\client\opamMain.exe switch --root $env:LOCALAPPDATA/.opam
#  switch                                                      compiler                            description
   C:\Users\beckf\AppData\Local\Programs\DiskuvOCaml\0\system  ocaml-variants.4.12.0+msvc64+msys2  C:\Users\beckf\AppData\Local\Programs\DiskuvOCaml\0\system
   C:\Users\beckf\AppData\Local\Programs\DiskuvOCaml\1\system  ocaml-variants.4.12.0+msvc64+msys2  C:\Users\beckf\AppData\Local\Programs\DiskuvOCaml\1\system
   Z:\source\diskuv-ocaml-starter\build\dev\Debug              ocaml-variants.4.12.0+msvc64+msys2  Z:\source\diskuv-ocaml-starter\build\dev\Debug
→  diskuv-boot-DO-NOT-DELETE                                                                       diskuv-boot-DO-NOT-DELETE

[WARNING] The environment is not in sync with the current switch.
          You should run: (& opam env) -split '\r?\n' | ForEach-Object { Invoke-Expression $_ }

PS Z:\source\opam> Z:\source\opam\_build\default\src\client\opamMain.exe env --root $env:LOCALAPPDATA/.opam  --set-root --set-switch
$env:OPAMROOT = 'C:\Users\beckf\AppData\Local\.opam'
$env:OPAMSWITCH = 'Z:\source\diskuv-ocaml-starter\build\dev\Debug'
$env:OPAM_SWITCH_PREFIX = 'Z:\source\diskuv-ocaml-starter\build\dev\Debug\_opam'
...

PS Z:\source\opam> (& Z:\source\opam\_build\default\src\client\opamMain.exe env --root $env:LOCALAPPDATA/.opam --switch Z:\source\diskuv-ocaml-starter\build\dev\Debug --set-switch --set-root) -split '\\r?\\n' | ForEach-Object { Invoke-Expression $_ }

PS Z:\source\opam> Z:\source\opam\_build\default\src\client\opamMain.exe switch --root $env:LOCALAPPDATA/.opam                                                                                                   #  switch                                                      compiler                            description
   C:\Users\beckf\AppData\Local\Programs\DiskuvOCaml\0\system  ocaml-variants.4.12.0+msvc64+msys2  C:\Users\beckf\AppData\Local\Programs\DiskuvOCaml\0\system
   C:\Users\beckf\AppData\Local\Programs\DiskuvOCaml\1\system  ocaml-variants.4.12.0+msvc64+msys2  C:\Users\beckf\AppData\Local\Programs\DiskuvOCaml\1\system
→  Z:\source\diskuv-ocaml-starter\build\dev\Debug              ocaml-variants.4.12.0+msvc64+msys2  Z:\source\diskuv-ocaml-starter\build\dev\Debug
   diskuv-boot-DO-NOT-DELETE                                                                       diskuv-boot-DO-NOT-DELETE

[NOTE] Current switch is set locally through the OPAMSWITCH variable.
       The current global system switch is diskuv-boot-DO-NOT-DELETE.

@rjbou
Copy link
Collaborator

rjbou commented Dec 8, 2021

I can't push on your repo @jonahbeckford . I pushed in rjbou/feature-eval-on-pwsh, if you can port last commits, and feel free to squash commits (there is one that is just style related)

@jonahbeckford
Copy link
Contributor Author

Thanks @rjbou ; I merged, squashed and pushed your changes. (And my future opam PRs will be pushable)

@rjbou rjbou moved this from PR in progress to PR to review in Opam 2.2.0 Mar 23, 2022
@jonahbeckford
Copy link
Contributor Author

Rebased. Can see CI problems with List.filter_map in OCaml 4.03.0, so fix pending.

@jonahbeckford
Copy link
Contributor Author

Rebased and List.filter_map usage fixed.

@dra27 dra27 self-assigned this Jun 27, 2022
Copy link
Member

@dra27 dra27 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sorry this has taken me such a ridiculously long time to review, but thank you very much for this! This goes very nicely with one of my oldest still-to-be-merged commits for opam (from 2015!) for the Command Prompt but I hadn't tackled PowerShell at all, and several parts of your cmd implementation here are really very nice (I particularly like having a clear - if, of course, arcane - equivalent for eval $(opam env)).

I have a few questions for small changes, but I propose otherwise that we get this merged and then I'll get on with pulling in the rest of my opam init stuff (which covers the shell script generation, and so forth).

src/core/opamStd.ml Outdated Show resolved Hide resolved
src/core/opamStd.ml Outdated Show resolved Hide resolved
src/client/opamConfigCommand.ml Show resolved Hide resolved
src/state/opamEnv.ml Outdated Show resolved Hide resolved
src/state/opamEnv.ml Outdated Show resolved Hide resolved
src/state/opamEnv.ml Outdated Show resolved Hide resolved
src/client/opamArg.ml Outdated Show resolved Hide resolved
src/core/opamStd.ml Outdated Show resolved Hide resolved
Comment on lines +1075 to +1073
| SH_pwsh | SH_win_powershell ->
if Sys.win32 then win_my_powershell "Microsoft.Powershell_profile.ps1" else
List.fold_left Filename.concat (home ".config") ["powershell"; "Microsoft.Powershell_profile.ps1"]
| SH_sh -> home ".profile"
| SH_win_cmd ->
(* cmd.exe does not have a concept of profiles *)
home ".profile"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have some related work on this which I propose coming to also in a subsequent PR - IIUC Diskuv doesn't use opam init to do this at all?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the question whether Diskuv uses opam init to populate entries (PATH, auto-complete) in the user's profile? No, Diskuv does not use opam init for that.

The installer takes care of putting Opam and with-dkml.exe (the wrapper) and a system dune + utop in the PATH via the registry in the source code https://github.com/diskuv/dkml-component-ocamlcompiler/blob/1e1990447a2fafdca0b2a18e845821952dc0cce2/assets/staging-files/win32/setup-userprofile.ps1#L1538-L1541 .

The uninstaller (on my TODO) will do the reverse.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent - the main thing I wanted to be sure was that me messing around with this in a subsequent PR wouldn't risk breaking things for you! I've got a patch which takes advantage of clink (if the user has it installed) to do the ".profile" trick for cmd, as it provides start-up scripts in Lua.

src/core/opamStd.ml Show resolved Hide resolved
src/client/opamArg.ml Outdated Show resolved Hide resolved
@dra27
Copy link
Member

dra27 commented Jul 25, 2022

Thanks for the updates! There's the CLI versioning question above and also the question as to whether the guessing of the shell for /usr/bin/env affects Diskuv and then this is good to merge.

Copy link
Member

@dra27 dra27 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to go, thank you!

jonahbeckford and others added 4 commits July 26, 2022 15:36
It's virtually impossible to define a general escaping function for the
Command Prompt, because the expansion is context-specific. However, as
set is the _only_ command, it's possible to use its double-quote escaped
form which then requires no further escaping of the strings.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Opam 2.2.0
  
Done
Development

Successfully merging this pull request may close these issues.

None yet

4 participants