Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
How should Proc::Async.new, run and shell call cmd.exe? #20
On Windows it is not possible to call
Short problem description
On Windows the API to start programs (CreateProcess) does not take an array, but a single string. There is a convention nearly all programs adhere to of how to turn an array of arguments into a single string (a well defined quoting). This convention is what rakudo currently implements (via libuv, which does the actual quoting).
Perl 6 functionality affected:
How could the API look that in addition to the current behaviour also allows to call
How command line processing on Windows works
Also don't miss this bugreport that already has a proposal of how to handle this in Perl 6.
Windows does not have the concept of an ARGV array. At the API level there is only one single string that is passed. There is a convention (CommandLineToArgv() implements that) of how a string array is converted to a single string and back. With rare exceptions all programs adhere to this convention. The most prominent exception is
When calling an
The CommandLineToArgv() quoting (CLTA)
If the argument contains
The cmd quoting
Five possible usage scenarios
In decreasing order of commonness.
Number 4. and 5. won't happen in any normal usage scenarios.
What others do
libuv quotes arguments compatible to
node exposes a
A good API should:
Proc::Async idea 1
Add a new enum
Proc::Async idea 2
Add a new flag to the
I like the java way of doing it (+ a flag to disable it if they don't do that) because it doesn't require the programmer to have as much of an understanding of e.g. Windows to write platform-independent code. Not thrilled about the idea of speccing some potentially complex and heuristic-based logic though.
Despite reading a bunch, I'm still confused about this issue or have some minor concerns. This comment provides input from others and from myself. Please read both parts of it (about the reddit post, and then my 2c) before posting anything. Thanks.
First, please read the following reddit thread that I started yesterday to get feedback on this issue:
Some of the comments in the thread so far correspond to some of my areas of confusion/concern.
If you wish, please respond to comments directly on reddit. If you prefer not to comment on reddit, please respond to their questions and comments in this thread and I will then follow up on reddit.
In the meantime, the rest of this comment is my 2c on the API / paintwork, either reflecting my confusion or, if I'm not confused, reflecting my concerns and attempts to reduce confusion:
As I understand it you either get the default, and live with it, or use one of three Windows specific options. Is that right?
Calling the default "normal" is imo weighted slightly wrongly in several ways. It has an accidental hint of derogatory attitude toward Windows that, while arguably deserved, equally arguably is out of place. It accidentally implies a reason for anxiety about switching away from it to an "abnormal" alternative. Again, that's arguably deserved but doesn't belong in the API. And "normal" doesn't hint at any particular relationship with the alternatives except inasmuch as it hints that it is perhaps notably different to all of them.
The word "default" conveys something close to "normal" so it's not a win in that regard. And it also has an unclear relationship with the alternatives. But -- and this is why I suggest it -- it slightly biases toward the sense it might just be the same as one of them. I consider this a good thing because...
Calling the next listed option "CommandLineArgvW" is so that the first page listed when googling for it says:
I consider it useful in its own right that a google will immediately produce a result that is both exactly the right direction and both obviously and authoritatively exactly the right direction. Which is one of the reasons I'm suggesting it as the enum identifier.
But more pertinent to my point about "default", I think the word "similar" in MS's doc helps point an investigating user in the right direction inasmuch as it is simultaneously similar to the standard C run-time processing and not quite the same. And it makes it easy for our doc to leverage that when describing what the "default" option does, something like:
Here is how I intended the respective parameters to influence processing:
Now on to your proposal.
Some links I found during my research of this issue that might be useful.
A key comment in a libuv issue: libuv/libuv#2627 (comment)
Two SOs that may only be useful because they're so monumental and scary that it emphasizes the need for a "Give enough control to the user to call any program the way he wants." escape hatch: https://stackoverflow.com/questions/6828751/batch-character-escaping and https://stackoverflow.com/questions/4094699/how-does-the-windows-command-interpreter-cmd-exe-parse-scripts
A link to fellow PL folk (Julia) in the process of figuring out related issues: JuliaLang/julia#34111
@patrickbkr You have it about right for my paintjob. To sort the one point of confusion, my intent was that
Thanks for following up in that reddit @patrickbkr. I suspect you got nothing at all out of it but I wanted to try. Also, I promised folk in my reddit post I'd follow up today if no one else had replied to them. There is one remaining as-yet-unanswered follow up to a comment you made @patrickbkr, and one other that didn't get any response so I said I'd post it here:
@patrickbkr To try to be clear, I presumed there was no API to fall back on at the moment. But I also thought that your proposal, which I imagine is what the poster was asking about, was to provide an API that, among other cases, covered this passthru case so that a user could indeed do whatever they wanted provided they called the code they needed to call. I thought this was perhaps (probably?) available via the
Thinking about this some more. I'm a bit uneasy about my no. 5 use case.
I don't see when one would actually need such double quoting. But that's exactly what the Microsoft blog promotes. So I guess I miss the point there. It would be a bad idea to just ignore that use case then.
My two API ideas each have a tradeoff:
Idea 1: We are limited to three given built in quoting systems that are provided. Given you need a different one, you have to set it to
Idea 2: This proposal uses separate functions. Of the top off my head I can't remember any other place in the Raku setting where there are such dependent functions. i. e. One needs to know about the existence of the
I currently lean toward Idea 1. We can add other options later when actual use cases show up. And it's still possible to turn automatic quoting off entirely and do it all by yourself.
It seems like you're saying that you don't get why that MS person recommend this, but you nevertheless think they must surely have a point, perhaps even a good one.
As another data point, let me quote a 2017 perlmonks post:
afoken's profile page includes plenty of evidence they're to be taken seriously on this topic. More importantly, their comment has "Reputation: 22 (no significant downvotes)?" and no rebuttal. For a comment like it, that voting/reply/rebuttal status on perlmonks speaks very loudly.
I grant that it's probably written from the perspective of someone whose main focus is perl but, to the degree there is any conflict between the perspective the MS writer provides, and the one afoken provides, but you must make simplifying presumptions (even if it's the presumption "I can't afford to make such-and-such a presumption"), I'd recommend erring on the side of being consistent with afoken's perspective.
There is no reliable way to escape parameters for use with CMD. CMD splits its argument string on space (and
So I once again change my mind and now favour my Idea No. 2: Provide a toggle to turn off argument quoting on Windows. But I wouldn't provide separate quoting subs anymore. The CLTR quoting sub is not needed as that quoting is already active by default. A CMD quoting sub should be omitted because it can't be made to work right.
This is pretty much what node.js decided to do.
I have a basic implementation of this running.
I wonder if its worth having a test that checks the command line arguments come out correct via something like
Do you happen to know if we can get the escaped command the JVM would run in such a way that we could set
I wanted to modify zefs AppVeyor to use a path with a space to see what would happen, but I have not gotten around to that. However -- this all appears sane to me.