Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bypass or reduce PowerShell usage on Windows #1387

Closed
tarsa opened this issue Sep 29, 2019 · 7 comments

Comments

@tarsa
Copy link

@tarsa tarsa commented Sep 29, 2019

Hi,

I'm working at my company on upgrading SBT from SBT 0.13.x to newest SBT and gave SBT coursier integration a try. While I had multiple issues (like differently working dependency resolution, so old conflict manager settings stopped working), the one that was most annoying was a 2 minutes lag on basically every SBT operation (on fresh SBT instance), like running a build, importing project in IntelliJ, etc I traced that lag down to invocation of powershell.exe to get two folder locations using Known Folder API inside https://github.com/soc/directories-jvm project. There seems to be no way of bypassing that check, even if I pass -Dsbt.coursier.home=C:\some\folder to SBT then it uses that C:\some\folder for Coursier artifacts, but still invoke powershell inducing a 2 minutes lag.

What could be extremely useful would be a directories cache. E.g. if passing -Dsbt.coursier.directories.cache=C:\some\folder\with\directories.conf would allow to avoid powershell invocation then that would be golden.

PS: I don't fully know why there's such a big lag in powershell.exe invocation, but my work laptop is overloaded with powershell modules. I guess managing that would be a little painful as I would need to ask technical support to do something with it (if anything can be done). The bloatware is managed by company and I don't have full control over my OS due to security concerns (I'm working for financial company).

@alexarchambault

This comment has been minimized.

Copy link
Member

@alexarchambault alexarchambault commented Sep 30, 2019

#1389 should fix that. With it, powershell.exe shouldn't be called if the cache path is explicitly set. It'll be included in the next release.

@alexarchambault

This comment has been minimized.

Copy link
Member

@alexarchambault alexarchambault commented Sep 30, 2019

cc @soc (that issue could impact other directories-jvm users)

@soc

This comment has been minimized.

Copy link
Contributor

@soc soc commented Sep 30, 2019

@tarsa Thanks! I fear this is either an issue with the local install or some general powershell/.NET issue.
Might make sense to file a ticket with your operating system vendor in the latter case.

@alexarchambault Before Java ships with Project Panama (improved interop with native APIs), I guess each application has to decide on its own what's the best way to deal with not being able to retrieve the standard dirs from the OS. Not sure I can offer semantics that are general enough to be useful. :-/

@tarsa Would it be possible for you to run the PowerShell command on it's own? See https://github.com/soc/directories-jvm/blob/master/src/main/java/io/github/soc/directories/Util.java#L119.

This would allow us to pinpoint the origin of the hang:

  • If running the powershell command on its own doesn't hang, then it has to be an issue with the JVM creating a process (which I guess is unlikely).
  • If the PowerShell command hangs, you could try running the embedded C# code on its own.
@tarsa

This comment has been minimized.

Copy link
Author

@tarsa tarsa commented Sep 30, 2019

@alexarchambault Thanks for the change. I'll test it when next SBT version that integrates it comes out.

@soc I am running SBT and PowerShell through Cygwin console (that shouldn't have much impact on its own).

Running the command that directories-jvm is running takes a lot of time:

$ time "powershell.exe" -Command "& {
[Console]::OutputEncoding = [System.Text.Encoding]::UTF8
Add-Type @\"
using System;
using System.Runtime.InteropServices;
public class Dir {
   [DllImport(\"shell32.dll\")]
   private static extern int SHGetKnownFolderPath([MarshalAs(UnmanagedType.LPStruct)] Guid rfid, uint dwFlags, IntPtr hToken, out IntPtr pszPath);
   public static string GetKnownFolderPath(string rfid) {
       IntPtr pszPath;
       if (SHGetKnownFolderPath(new Guid(rfid), 0, IntPtr.Zero, out pszPath) != 0) return \"\";
       string path = Marshal.PtrToStringUni(pszPath);
       Marshal.FreeCoTaskMem(pszPath);
       return path;
   }
}
\"@
[Dir]::GetKnownFolderPath(\"3EB685DB-65F9-4CF6-A03A-E3EF65729F3D\")
[Dir]::GetKnownFolderPath(\"F1B32785-6FBA-4FCF-9D55-7B8E7F157091\")
}"
C:\Users\abcdefgh\AppData\Roaming
C:\Users\abcdefgh\AppData\Local

real    1m39.689s
user    0m0.015s
sys     0m0.047s

However, if I start powershell.exe from cygwin and then enter commands one by one then no lag occurs.

Judging from the timings it seems nothing CPU heavy is going on. Maybe some network resource is unavailable and powershell waits for a timeout? I tried preparing a script that would be both fast and correct, but that didn't pan out. I'm not an expert in powershell, though.

@soc

This comment has been minimized.

Copy link
Contributor

@soc soc commented Oct 1, 2019

@tarsa Thanks for investigating! This has dramatically cut down the possible sources of this error (although I don't have an idea yet what could be the cause...).

@tarsa

This comment has been minimized.

Copy link
Author

@tarsa tarsa commented Oct 1, 2019

I think I nailed the reason for lag. A error message flashed once during import in IntelliJ with Coursier enabled. Message was saying that powershell was trying to load modules from network attached storage that was unreachable. I don't know why the message from powershell isn't displaying reliably.

I tried to somehow force powershell to skip loading modules when executing script block in batch mode but I have failed. -NoProfile didn't make execution shorter.

@alexarchambault

This comment has been minimized.

Copy link
Member

@alexarchambault alexarchambault commented Oct 15, 2019

Going to close this one. The latest 2.0.0-RC4 includes #1389, which should not run Powershell when it's not needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.