Command-line arguments to Windows exe not quoted when they contain tabs

[cspotcode]

Your Windows build number:

Microsoft Windows [Version 10.0.15063]

What you're doing and what's happening:

When I invoke a Windows executable from WSL and one of the command-line arguments contains a tab, WSL is not wrapping it in double-quotes. Windows executables usually split command-line args on tabs.

# from bash shell
# Argument with spaces is quoted
$ /mnt/c/EchoArgs.exe "$(printf 'a  b')"
Arg 0 is <a  b>

Command line:
"C:\EchoArgs.exe" "a  b"

# Argument with tabs is not quoted
$ /mnt/c/EchoArgs.exe "$(printf 'a\t\tb')"
Arg 0 is <a>
Arg 1 is <b>

Command line:
"C:\EchoArgs.exe" a         b

EchoArgs.exe is from Pscx; real path is C:\Program Files (x86)\PowerShell Community Extensions\Pscx3\Pscx\Apps\EchoArgs.exe but I truncated it above for readability.

What's wrong / what should be happening instead:

WSL should be wrapping args that contain tabs in double-quotes like it does for args containing spaces.

It appears that double-quotes are not necessary for \v, \r, \n, or \f characters, so those don't cause any problems.

Sorry if this is a duplicate; I couldn't find a preexisting issue.

[sunjoong]

@cspotcode - Hmm... what if use arguments that make quote? I could not sure... but...

EchoArgs.exe '"' "$(printf 'a\t\tb')" '"'

Or

EchoArgs.exe \" "$(printf 'a\t\tb')" \"

This reminded me #2069 to make this elisp function;

(defun browse-url-edge (url &optional new-window)
  (shell-command
   (concat "/mnt/c/Windows/System32/WindowsPowerShell/v1.0/powershell.exe "
           "start microsoft-edge:'\\\"" url "\\\"'")))

UPDATE1: \""$(printf 'a\t\tb')"\" looked good;

root@SUNJOONG-DESKTOP:~# /mnt/c/EchoArgs.exe \""$(printf 'a\t\tb')"\"
Arg 0 is <a             b>

Command line:
C:\EchoArgs.exe "a              b"

root@SUNJOONG-DESKTOP:~#

But... it seems to make another problem;

root@SUNJOONG-DESKTOP:~# /mnt/c/EchoArgs.exe \""$(printf 'a  b')"\"
Arg 0 is <a>
Arg 1 is <b>

Command line:
C:\EchoArgs.exe ""a  b""

root@SUNJOONG-DESKTOP:~#

And... \"$(printf 'a\t\tb')\" (ommint middle quotes) will change whitespace characaterS to A single blank character;

root@SUNJOONG-DESKTOP:~# /mnt/c/EchoArgs.exe \"$(printf 'a  b')\"
Arg 0 is <a b>

Command line:
C:\EchoArgs.exe "a b"

root@SUNJOONG-DESKTOP:~# /mnt/c/EchoArgs.exe \"$(printf 'a\t\tb')\"
Arg 0 is <a b>

Command line:
C:\EchoArgs.exe "a b"

root@SUNJOONG-DESKTOP:~#

But... it might be natural, comparing bash -c;

root@SUNJOONG-DESKTOP:~# function arg1 {
> echo $1
> }
root@SUNJOONG-DESKTOP:~# arg1 "$(bash -c "printf 'a\t\tb'")"
a b
root@SUNJOONG-DESKTOP:~# arg1 "$(bash -c "printf 'a  b'")"
a b
root@SUNJOONG-DESKTOP:~#

UPDATE2: I got a some idea just now but does not have EchoArgs.exe. EchoArgs.exe looked like to need dot.NET3. I had haven a plan to restore OS backup image, so I tested above before that, but I have clean image now and... feel uncomfortable with restoring again.

What if EchoArgs.exe \"$(printf %q 'a\t\tb')\" or EchoArgs.exe "$(printf %q 'a\t\tb')" ?

I tested %q qualifier with above arg1 function;

sunjoong@SUNJOONG-DESKTOP ~ $ arg1 "$(bash -c "printf %q 'a\t\tb'")"
a\\t\\tb
sunjoong@SUNJOONG-DESKTOP ~ $ arg1 "$(bash -c "printf %q 'a  b'")"
a\ \ b
sunjoong@SUNJOONG-DESKTOP ~ $

UPDATE3:

sunjoong@SUNJOONG-DESKTOP ~/temp $ /mnt/c/msys64/home/sunjoong/echoargs_mingw.exe "$(printf %q 'a\t\tb')"
This program was called with 2 arguments.
The name of this program is [C:\msys64\home\sunjoong\echoargs_mingw.exe]
argv[0] = C:\msys64\home\sunjoong\echoargs_mingw.exe
argv[1] = a\\t\\tb
sunjoong@SUNJOONG-DESKTOP ~/temp $ /mnt/c/msys64/home/sunjoong/echoargs_mingw.exe "$(printf %q 'a  b')"
This program was called with 2 arguments.
The name of this program is [C:\msys64\home\sunjoong\echoargs_mingw.exe]
argv[0] = C:\msys64\home\sunjoong\echoargs_mingw.exe
argv[1] = a\ \ b
sunjoong@SUNJOONG-DESKTOP ~/temp $

[cspotcode]

You can create your own EchoArgs.exe. Just compile a simple C program that calls printf() for each entry in the argv array. You can also write it in Python, Node, or Java.

Some of the solutions you're attempting are being affected by bash's argument splitting, so you should modify your tests to output the argv array in Linux first, and then pass it to EchoArgs.exe, so you can see the difference. Otherwise you'll be accidentally splitting arguments before passing them to the Windows exe.

It's possible to manually wrap arguments in double-quotes, but this must be done only for arguments that contain tabs and not spaces. It cannot be done for arguments containing spaces. Essentially, the workaround must understand WSL's inconsistent quoting behavior. I don't think this is what the WSL developers intended, which is why I reported this behavior as a bug.

[sunjoong]

@cspotcode - Ah... I think you mean...

sunjoong@SUNJOONG-DESKTOP ~/temp $ /mnt/c/msys64/home/sunjoong/echoargs_mingw.exe "$(printf 'a\t\tb')"
This program was called with 3 arguments.
The name of this program is [C:\msys64\home\sunjoong\echoargs_mingw.exe]
argv[0] = C:\msys64\home\sunjoong\echoargs_mingw.exe
argv[1] = a
argv[2] = b
sunjoong@SUNJOONG-DESKTOP ~/temp $

should be same as

sunjoong@SUNJOONG-DESKTOP ~/temp $ ./echoargs_linux "$(printf 'a\t\tb')"
This program was called with 2 arguments.
The name of this program is [./echoargs_linux]
argv[0] = ./echoargs_linux
argv[1] = a             b
sunjoong@SUNJOONG-DESKTOP ~/temp $

or

sunjoong@SUNJOONG-DESKTOP MINGW64 ~
$ ./echoargs_mingw.exe "$(printf 'a\t\tb')"
This program was called with 2 arguments.
The name of this program is [C:\msys64\home\sunjoong\echoargs_mingw.exe]
argv[0] = C:\msys64\home\sunjoong\echoargs_mingw.exe
argv[1] = a             b

sunjoong@SUNJOONG-DESKTOP MINGW64 ~
$

UPDATE: Yeah... feeling strange, comaring with PowerShell;

PS C:\msys64\home\sunjoong> .\echoargs_mingw.exe (bash.exe -c "printf 'a\t\tb'")
This program was called with 2 arguments.
The name of this program is [C:\msys64\home\sunjoong\echoargs_mingw.exe]
argv[0] = C:\msys64\home\sunjoong\echoargs_mingw.exe
argv[1] = a             b
PS C:\msys64\home\sunjoong>

[TSlivede]

The problem is not only about tabs, it's the whole "convert argv array to single CommandLine string for windows". AFAIK almost all Windows executables follow these rules, so WSL should build the CommandLine accordingly:

• If an argument contains whitespace (including tab) it should be surrounded by quotes.
• If an argument contains " those should be escaped with backslashes (see Transferring quotes from bash to windows app doesn't work as expected #1625)
• etc.

Even many applications that don't follow these rules could benefit from this: AFAIK cygwin's slightly incompatible commandline parsing (or even bash.exe's current strange commandline parsing ) would profit from this change.
(Many of these apps would profit from quoting arguments not only if they contain space or tab but also if they contain ' - this is perfectly compatible for apps that do follow MS rules )

A similar issue exists for powershell (PowerShell/PowerShell#1995). Fixing this in powershell requires many thoughts about backward compatibility.
For WSL fixing should be easier, because WSL is still beta.

[benhillis]

Thanks again for reporting this issue, I have drafted a fix that resolves this and #1625.

[sunilmut]

This should be fixed in Fall Creators Update or any build > 16273