Cross platform shebangs by special-casing #!/usr/bin/env on Windows

[cspotcode]

I can't figure out how to write a cross-platform python or node shebang recipe. #!python works on Windows, and #!/usr/bin/env python works on Linux. But what works on both?

On Linux we can write a reliable shebang using: #!/usr/bin/env bash or #!/usr/bin/env python3

It is expected that /usr/bin/env will always exist at that path, and it will look up the positional argument on your $PATH and call it.

I'm proposing that just can safely special-case this /usr/bin/env prefix on Windows to behave the same: look up the positional argument on your $PATH and execute it. Essentially, on Windows, #!/usr/bin/env whatever will be handled the way it handles #!whatever today.

I can't think of a situation where it is desirable for /usr/bin/env to either do something else on Windows, or to fail with an error. So I don't think special-casing will break anyone's use-case.

[runeimp]

@cspotcode something like this following should take care of things. 😉

shebang := if os() == 'windows' {
	'python'
} else {
	'/usr/bin/env python3'
}

recipe:
	#!{{shebang}}

	# Script code goes here

[cspotcode]

I did that, but it's still error-prone. If the variable includes the #! then it doesn't trigger just's shebang logic.

[runeimp]

Note that the #! part is outside the curly braces. As long as that is the first line of the script (which is required for any use of shebang) you are good. I do this all the time.

[cspotcode]

Do we think the /usr/bin/env special-casing is a net win, given that windows shebangs are already special-cased?

[runeimp]

@cspotcode There are a few ways to interpret what you said as I'm not sure if you're referring to how Just manages shebangs at all on Windows or that having a shebang start with /usr/bin/env works but isn't part of the original UNIX spec?

[cspotcode]

The idea is that common devtools are often available on PATH but at a path that may vary or be onerous to add to a shebang. For example, on my Linux box:

❯ which node
/home/cspotcode/.volta/bin/node

The practical solution here, acknowledging that it's not part of an original UNIX spec, is #!/usr/bin/env node. This form of shebang is well-known and works reliably.

We can add logic to just so that, only when interpreting a shebang on Windows, just will detect a /usr/bin/env prefix.

When just sees #!/usr/bin/env python on Windows, it will do what it already does for #!python

This means that users writing a well-known and well-understood shebang of the form #!/usr/bin/env python will have that shebang work correctly on Windows, too: it will call the python executable found on $PATH

[runeimp]

I always thought having to specify the exact path to the "interpreter" was lame in the shebang spec. It being an option was completely reasonable but required is just problematic. I believe I've already proposed to @casey to have how shebang is handled on Windows also work for other operating systems. I forget where he landed on that exactly at the moment. But I'd prefer that to Just ignoring /usr/bin/env on Windows. If I could only have one option that is.

[cspotcode]

When users rely on cygpath to convert /usr/bin/env into C:\cygwin64\bin\env.exe, what features of env are they typically relying on? -i, -S, -C?

[runeimp]

Hm, -C isn't even an option on my Mac, or in Bash according to https://ss64.com/osx/env.html . None-the-less those options are almost never used in any shebang line that I've ever seen. Using env is typically used only for it's side effect of searching PATH for the binary so that the command in the shebang isn't fixed and instead uses the first version found in PATH.

[cspotcode]

That's my thinking as well. -S allows passing args, so I see that one used in the wild a fair bit.

I was playing around with writing a PR for this. It's predicated on the assumption that Windows users never actually care that they're invoking cygwin's env.exe.

[runeimp]

Check out our discussion at #1256 to get more insight. You could write a PR as @casey is appreciative of help coding. Hopefully he will jump in this discussion a bit though. Hey @casey ! 😀

[casey]

Thanks for opening this issue, and sorry for not responding here!

I definitely agree that there's a problem here, but I think I'm on balance against special-casing /usr/bin/env on windows.

/usr/bin/env has different capabilities on different platforms, and even different unix kernels split shebang lines differently, so trying to get shebang lines to behave consistently across platforms feels like a losing battle to me. My preferred approach to this is to introduce a new script annotation, which allows passing a command to run the script which is looked up in $PATH like usual, and be fully cross platform with no special cases, e.g.:

[script('python3')]
foo:
  print('hello')

[beetleb]

Just ran into this myself and took a lot of searching to find the solution given here (#1549 (comment)).

While it's inelegant, it does work. I would strongly recommend listing it somewhere in the Docs until a "proper" solution is added to just. I think enough people write cross platform recipes to warrant its inclusion in the docs.

[runeimp]

I thought an example like mine was in the docs for some time. Maybe it was removed due to some redundancy in the docs at the time. @casey do you know anything about that example? I think you wrote the one that ended up in the docs, no?