java.io.IOException: Cannot run program "powershell.exe": CreateProcess error=2

[eed3si9n]

This was originally reported by @xshi0001 as sbt/sbt#5386

steps

sbt version: 1.3.4 window10

problem

xshi0@LAPTOP-L904TFAI MINGW64 /d/WorkPlace/Flink/2-demo/springBoot-flink
$ sbt
[warn] No sbt.version set in project/build.properties, base directory: D:\WorkPlace\Flink\2-demo\springBoot-flink
[error] java.io.IOException: Cannot run program "powershell.exe": CreateProcess error=2, 系统找不到指定的文件。
[error] Use 'last' for the full log.
Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? l

[soc]

Thanks – it feels like this may be caused by some recent change in Windows. What do you think?

Are you using a patched version of the code that ships with the fixes for #26? (If yes, could you try without the patch?)

[eed3si9n]

For the case of Scala team, @alexarchambault discovered that our Windows worker machine had limited PATH to

/bin:/usr/bin:/cygdrive/c/Program Files/Java/jdk1.8.0_121/bin:/cygdrive/c/Program Files/Git/Cmd:/cygdrive/c/ProgramData/chocolatey/bin

in scala/scala-dev#666 (comment).

I'm not sure how common this practice is but considering running anything on Windows, I can see it's preferable to lock things down.

[soc]

Agreed – I wonder whether effort should be made to handle gracefully (whatever that would mean) or if the exception is actually the right approach in this situation.

[eed3si9n]

sbt uses a zero-dependency file watching library called File tree views https://github.com/swoval/swoval/tree/develop/files written by @eatkins.
The JAR contains swoval-files0.dll for Windows, which is about 9kb. So maybe raw JNI is not so bad?

[soc]

If someone figures out a way to reliably build and maintain such a library on all supported Windows architectures in a safe language and sends a PR, I'm more than happy to review it.

[eatkins]

If someone figures out a way to reliably build and maintain such a library on all supported Windows architectures in a safe language and sends a PR, I'm more than happy to review it.

I don't think you need to support all architectures. The swoval file tree views library doesn't support 32 bit and falls back to jvm built-ins in that case. A similar thing could be done here with a fallback to powershell if there is no native implementation available for the runtime platform. I also wrote it in such a way that the user can choose the implementation either at compile time through the public api or at runtime via a system property.

What counts as a safe language? In this context, I almost read this as a requirement that rust be used because it is the only widely used safe language without a runtime that I'm aware of at least. For the purposes of this library, that seems like an overly harsh restriction. Given a cursory look at the problem, I suspect this could be implemented with just stack allocations in c or c++. We haven't had any reports of memory leaks or segfaults with sbt 1.3.x even though it uses native jni implementations written in c++ on all 64 bit platforms which I'm guessing covers > 90% of sbt users. I definitely agree with the general desire to avoid unsafe languages but in this particular case it doesn't seem like that big a deal.

That all being said, jna would also seem to be an option here. If done using jna, it could even be a provided dependency with a similar fallback mechanism to the jni case described above if the jna is not available on the system.

[soc]

Hi @eatkins,

thanks for writing down your thoughts! Comments inline:

I don't think you need to support all architectures.

I think the main difference between swoval and directories is that not supporting all architectures only makes things a bit slower like in your case; in my case it doubles the codepaths the program could take, it doubles the things that can go wrong, and makes it harder to test things.

It also doesn't eliminate issues that could be caused by Powershell/.NET/etc. problems, it just pushes them to more obscure configurations that are even harder to test/debug.

What counts as a safe language?

I think Rust libraries might be too large; ATS could be an option though.

To be honest, I'd rather have assembly than touching that shit show of C/C++ ever again.
At least assembly doesn't go to extraordinary lengths to prove one made a mistake somewhere – and therefore the compiler is allowed to do "whatever".

That all being said, jna would also seem to be an option here.

JNA was discussed in #16.

---

What makes sense in any case is to document how the directories are computed on Windows, I believe.
As mentioned in the other issue as soon as the JVM ships with a credible solution to do native interop, I'll make use of it. Not sure if there is a good solution until then.

Cheers,

Simon

[eatkins]

I know little about Windows, but I assume that powershell is probably in a standard location on most machines? Could the library fallback to looking for powershell in the standard location(s) if the process returns an IOException instead of relying on the PATH? I'm imagining something like (ignoring concurrency issues):

private static final String systemPowershell;
private static String[] getWinDirs(final String powershell, boolean doubleEscapeQuotes, String... guids) {
  ...
  return runCommands(guidsLength, Charset.forName("UTF-8"), powerShell, ...)
}
static String[] getWinDirs(boolean doubleEscapeQuotes, String... guids) {
  try {
    final String powerShell = systemPowershell == null ? "powershell.exe " : systemPowerShell;
    return getWinDirs(cmd, doubleEscapeQuotes, guids);
  } catch (final IOException e) {
     final String standardPowerShellLocation = "C:\\Program Files\\...\\powershell.exe"; 
     if (java.nio.file.Files.exists(java.nio.file.Paths.get(standardPowerShellLocation))) {
        systemPowerShell = standardPowerShellLocation;
        return getWinDirs(systemPowerShell, doubleEscapeQuotes, guids);
     }
  }
}

As mentioned in the other issue as soon as the JVM ships with a credible solution to do native interop, I'll make use of it. Not sure if there is a good solution until then.

The problem for sbt is that we expect to support users on jdk 8 indefinitely. Since sbt has a transitive dependency on this library via coursier, we are invested in either fixing this issue here or moving off of the functionality provided by the library.

[soc]

I think it is reasonable to try the (or a few) standard locations for Powershell, such a patch would be very welcome!

[eatkins]

Cool. I am unlikely to do this myself as I don't really have the bandwidth to spin up a VM and write the patch and tests but I'm hoping someone more invested in Windows can try this idea.

[alexarchambault]

This website mentions %SystemRoot%\system32\WindowsPowerShell\v1.0\powershell.exe, as does this StackOverflow comment. That's in line with a Windows 10 install I have.

I guess we can read SystemRoot ourselves from the env, with a fallback to C:\Windows.

[soc]

Agreed. Would it be possible for you to create/test a patch that checks those two things in addition to using path?

I'd look into migrating to Base64 encoding the command as you suggested in #26 (comment).

With those two things in place, I'd cut a new release soonish if that works for you?

[soc]

@alexarchambault #30