Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PowerShell 7.3 doesn't pass variable values / cmdlet results arguments to child processes when comma is present #18502

Closed
5 tasks done
conioh opened this issue Nov 9, 2022 · 29 comments
Labels
Needs-Triage The issue is new and needs to be triaged by a work group. Resolution-No Activity Issue has had no activity for 6 months or more WG-Engine core PowerShell engine, interpreter, and runtime

Comments

@conioh
Copy link

conioh commented Nov 9, 2022

Prerequisites

Steps to reproduce

When launching an executable from PowerShell, with $PSNativeCommandArgumentPassing <> "Legacy" if the arguments contain a comma, the variables/cmdlets are evaluated, but the results of the evaluation don't go to the child process.

C:\Windows\System32> $PSNativeCommandArgumentPassing = "Windows"
C:\Windows\System32> $x = "lame"
C:\Windows\System32> cmd /c echo x=$x
x=lame
C:\Windows\System32> cmd /c echo a,x=$x
a,x=$x
C:\Windows\System32> Trace-Command -PSHost -Name "ParameterBinding" { cmd /c echo x=$x }
DEBUG: 2022-11-09 20:39:02.1548 ParameterBinding Information: 0 : BIND NAMED native application line args [C:\Windows\system32\cmd.exe]
DEBUG: 2022-11-09 20:39:02.1551 ParameterBinding Information: 0 :     BIND argument [/c echo x=lame]
DEBUG: 2022-11-09 20:39:02.1702 ParameterBinding Information: 0 : CALLING BeginProcessing
x=lame
C:\Windows\System32> Trace-Command -PSHost -Name "ParameterBinding" { cmd /c echo a,x=$x }
DEBUG: 2022-11-09 20:39:07.6131 ParameterBinding Information: 0 : BIND NAMED native application line args [C:\Windows\system32\cmd.exe]
DEBUG: 2022-11-09 20:39:07.6134 ParameterBinding Information: 0 :     BIND argument [/c echo a,x=$x]
DEBUG: 2022-11-09 20:39:07.6194 ParameterBinding Information: 0 : CALLING BeginProcessing
a,x=$x
C:\Windows\System32> $PSNativeCommandArgumentPassing = "Standard"
C:\Windows\System32> Trace-Command -PSHost -Name "ParameterBinding" { cmd /c echo x=$x }
DEBUG: 2022-11-09 20:39:22.6507 ParameterBinding Information: 0 : BIND NAMED native application line args [C:\Windows\system32\cmd.exe]
DEBUG: 2022-11-09 20:39:22.6510 ParameterBinding Information: 0 :     BIND cmd line arg [/c] to position [0]
DEBUG: 2022-11-09 20:39:22.6513 ParameterBinding Information: 0 :     BIND cmd line arg [echo] to position [1]
DEBUG: 2022-11-09 20:39:22.6515 ParameterBinding Information: 0 :     BIND cmd line arg [x=lame] to position [2]
DEBUG: 2022-11-09 20:39:22.6681 ParameterBinding Information: 0 : CALLING BeginProcessing
x=lame
C:\Windows\System32> Trace-Command -PSHost -Name "ParameterBinding" { cmd /c echo a,x=$x }
DEBUG: 2022-11-09 20:39:25.2418 ParameterBinding Information: 0 : BIND NAMED native application line args [C:\Windows\system32\cmd.exe]
DEBUG: 2022-11-09 20:39:25.2422 ParameterBinding Information: 0 :     BIND cmd line arg [/c] to position [0]
DEBUG: 2022-11-09 20:39:25.2425 ParameterBinding Information: 0 :     BIND cmd line arg [echo] to position [1]
DEBUG: 2022-11-09 20:39:25.2428 ParameterBinding Information: 0 :     BIND cmd line arg [a,x=$x] to position [2]
DEBUG: 2022-11-09 20:39:25.2666 ParameterBinding Information: 0 : CALLING BeginProcessing
a,x=$x
C:\Windows\System32> $PSNativeCommandArgumentPassing = "Legacy"
C:\Windows\System32> Trace-Command -PSHost -Name "ParameterBinding" { cmd /c echo x=$x }
DEBUG: 2022-11-09 20:39:36.2214 ParameterBinding Information: 0 : BIND NAMED native application line args [C:\Windows\system32\cmd.exe]
DEBUG: 2022-11-09 20:39:36.2218 ParameterBinding Information: 0 :     BIND argument [/c echo x=lame]
DEBUG: 2022-11-09 20:39:36.2561 ParameterBinding Information: 0 : CALLING BeginProcessing
x=lame
C:\Windows\System32> Trace-Command -PSHost -Name "ParameterBinding" { cmd /c echo a,x=$x }
DEBUG: 2022-11-09 20:39:38.4751 ParameterBinding Information: 0 : BIND NAMED native application line args [C:\Windows\system32\cmd.exe]
DEBUG: 2022-11-09 20:39:38.4755 ParameterBinding Information: 0 :     BIND argument [/c echo a,x=lame]
DEBUG: 2022-11-09 20:39:38.4822 ParameterBinding Information: 0 : CALLING BeginProcessing
a,x=lame
C:\Windows\System32>

We can see that the cmdlet is executed (or attempted):

C:\Windows\System32> $PSNativeCommandArgumentPassing = "Standard"
C:\Windows\System32> Trace-Command -PSHost -Name "ParameterBinding" { cmd /c echo cd=$(Get-Location) }
DEBUG: 2022-11-09 20:48:15.0675 ParameterBinding Information: 0 : BIND NAMED cmd line args [Get-Location]
DEBUG: 2022-11-09 20:48:15.0677 ParameterBinding Information: 0 : BIND POSITIONAL cmd line args [Get-Location]
DEBUG: 2022-11-09 20:48:15.0679 ParameterBinding Information: 0 : BIND cmd line args to DYNAMIC parameters.
DEBUG: 2022-11-09 20:48:15.0681 ParameterBinding Information: 0 : MANDATORY PARAMETER CHECK on cmdlet [Get-Location]
DEBUG: 2022-11-09 20:48:15.0683 ParameterBinding Information: 0 : CALLING BeginProcessing
DEBUG: 2022-11-09 20:48:15.0685 ParameterBinding Information: 0 : CALLING ProcessRecord
DEBUG: 2022-11-09 20:48:15.0687 ParameterBinding Information: 0 : CALLING EndProcessing
DEBUG: 2022-11-09 20:48:15.0762 ParameterBinding Information: 0 : BIND NAMED native application line args [C:\Windows\system32\cmd.exe]
DEBUG: 2022-11-09 20:48:15.0767 ParameterBinding Information: 0 :     BIND cmd line arg [/c] to position [0]
DEBUG: 2022-11-09 20:48:15.0770 ParameterBinding Information: 0 :     BIND cmd line arg [echo] to position [1]
DEBUG: 2022-11-09 20:48:15.0773 ParameterBinding Information: 0 :     BIND cmd line arg [cd=C:\Windows\System32] to position [2]
DEBUG: 2022-11-09 20:48:15.0943 ParameterBinding Information: 0 : CALLING BeginProcessing
cd=C:\Windows\System32
C:\Windows\System32> Trace-Command -PSHost -Name "ParameterBinding" { cmd /c echo foo,cd=$(Get-Location) }
DEBUG: 2022-11-09 20:48:18.2709 ParameterBinding Information: 0 : BIND NAMED cmd line args [Get-Location]
DEBUG: 2022-11-09 20:48:18.2712 ParameterBinding Information: 0 : BIND POSITIONAL cmd line args [Get-Location]
DEBUG: 2022-11-09 20:48:18.2714 ParameterBinding Information: 0 : BIND cmd line args to DYNAMIC parameters.
DEBUG: 2022-11-09 20:48:18.2717 ParameterBinding Information: 0 : MANDATORY PARAMETER CHECK on cmdlet [Get-Location]
DEBUG: 2022-11-09 20:48:18.2719 ParameterBinding Information: 0 : CALLING BeginProcessing
DEBUG: 2022-11-09 20:48:18.2721 ParameterBinding Information: 0 : CALLING ProcessRecord
DEBUG: 2022-11-09 20:48:18.2723 ParameterBinding Information: 0 : CALLING EndProcessing
DEBUG: 2022-11-09 20:48:18.2815 ParameterBinding Information: 0 : BIND NAMED native application line args [C:\Windows\system32\cmd.exe]
DEBUG: 2022-11-09 20:48:18.2819 ParameterBinding Information: 0 :     BIND cmd line arg [/c] to position [0]
DEBUG: 2022-11-09 20:48:18.2821 ParameterBinding Information: 0 :     BIND cmd line arg [echo] to position [1]
DEBUG: 2022-11-09 20:48:18.2824 ParameterBinding Information: 0 :     BIND cmd line arg [foo,cd=$(Get-Location)] to position [2]
DEBUG: 2022-11-09 20:48:18.2999 ParameterBinding Information: 0 : CALLING BeginProcessing
foo,cd=$(Get-Location)
C:\Windows\System32> Trace-Command -PSHost -Name "ParameterBinding" { cmd /c echo foo,cd=$(Get-Locatio) }
DEBUG: 2022-11-09 20:48:24.7753 ParameterBinding Information: 0 : BIND PIPELINE object to parameters: [Out-Default]
DEBUG: 2022-11-09 20:48:24.7756 ParameterBinding Information: 0 :     PIPELINE object TYPE = [System.Management.Automation.ErrorRecord]
DEBUG: 2022-11-09 20:48:24.7759 ParameterBinding Information: 0 :     RESTORING pipeline parameter's original values
DEBUG: 2022-11-09 20:48:24.7762 ParameterBinding Information: 0 :     Parameter [InputObject] PIPELINE INPUT ValueFromPipeline NO COERCION
DEBUG: 2022-11-09 20:48:24.7765 ParameterBinding Information: 0 :     BIND arg [The term 'Get-Locatio' is not recognized as a name of a cmdlet, function, script file, or executable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.] to parameter [InputObject]
DEBUG: 2022-11-09 20:48:24.7768 ParameterBinding Information: 0 :         BIND arg [The term 'Get-Locatio' is not recognized as a name of a cmdlet, function, script file, or executable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.] to param [InputObject] SUCCESSFUL
DEBUG: 2022-11-09 20:48:24.7771 ParameterBinding Information: 0 : MANDATORY PARAMETER CHECK on cmdlet [Out-Default]
DEBUG: 2022-11-09 20:48:24.7774 ParameterBinding Information: 0 : CALLING ProcessRecord
DEBUG: 2022-11-09 20:48:24.7778 ParameterBinding Information: 0 :     BIND NAMED cmd line args [Out-LineOutput]
DEBUG: 2022-11-09 20:48:24.7785 ParameterBinding Information: 0 :         BIND arg [Microsoft.PowerShell.Commands.Internal.Format.ConsoleLineOutput] to parameter [LineOutput]
DEBUG: 2022-11-09 20:48:24.7803 ParameterBinding Information: 0 :             COERCE arg to [System.Object]
DEBUG: 2022-11-09 20:48:24.7830 ParameterBinding Information: 0 :                 Parameter and arg types the same, no coercion is needed.
DEBUG: 2022-11-09 20:48:24.7845 ParameterBinding Information: 0 :             BIND arg [Microsoft.PowerShell.Commands.Internal.Format.ConsoleLineOutput] to param [LineOutput] SUCCESSFUL
DEBUG: 2022-11-09 20:48:24.7853 ParameterBinding Information: 0 :     BIND POSITIONAL cmd line args [Out-LineOutput]
DEBUG: 2022-11-09 20:48:24.7858 ParameterBinding Information: 0 :     MANDATORY PARAMETER CHECK on cmdlet [Out-LineOutput]
DEBUG: 2022-11-09 20:48:24.7863 ParameterBinding Information: 0 :     CALLING BeginProcessing
DEBUG: 2022-11-09 20:48:24.7868 ParameterBinding Information: 0 :     BIND PIPELINE object to parameters: [Out-LineOutput]
DEBUG: 2022-11-09 20:48:24.7873 ParameterBinding Information: 0 :         PIPELINE object TYPE = [System.Management.Automation.ErrorRecord]
DEBUG: 2022-11-09 20:48:24.7878 ParameterBinding Information: 0 :         RESTORING pipeline parameter's original values
DEBUG: 2022-11-09 20:48:24.7885 ParameterBinding Information: 0 :         Parameter [InputObject] PIPELINE INPUT ValueFromPipeline NO COERCION
DEBUG: 2022-11-09 20:48:24.7890 ParameterBinding Information: 0 :         BIND arg [The term 'Get-Locatio' is not recognized as a name of a cmdlet, function, script file, or executable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.] to parameter [InputObject]
DEBUG: 2022-11-09 20:48:24.7895 ParameterBinding Information: 0 :             BIND arg [The term 'Get-Locatio' is not recognized as a name of a cmdlet, function, script file, or executable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.] to param [InputObject] SUCCESSFUL
DEBUG: 2022-11-09 20:48:24.7901 ParameterBinding Information: 0 :     MANDATORY PARAMETER CHECK on cmdlet [out-lineoutput]
DEBUG: 2022-11-09 20:48:24.7904 ParameterBinding Information: 0 :     CALLING ProcessRecord
DEBUG: 2022-11-09 20:48:24.7911 ParameterBinding Information: 0 :         BIND NAMED cmd line args [Format-Default]
DEBUG: 2022-11-09 20:48:24.7916 ParameterBinding Information: 0 :         BIND POSITIONAL cmd line args [Format-Default]
DEBUG: 2022-11-09 20:48:24.7920 ParameterBinding Information: 0 :         MANDATORY PARAMETER CHECK on cmdlet [Format-Default]
DEBUG: 2022-11-09 20:48:24.7924 ParameterBinding Information: 0 :         CALLING BeginProcessing
DEBUG: 2022-11-09 20:48:24.7928 ParameterBinding Information: 0 :         BIND PIPELINE object to parameters: [Format-Default]
DEBUG: 2022-11-09 20:48:24.7932 ParameterBinding Information: 0 :             PIPELINE object TYPE = [System.Management.Automation.ErrorRecord]
DEBUG: 2022-11-09 20:48:24.7937 ParameterBinding Information: 0 :             RESTORING pipeline parameter's original values
DEBUG: 2022-11-09 20:48:24.7946 ParameterBinding Information: 0 :             Parameter [InputObject] PIPELINE INPUT ValueFromPipeline NO COERCION
DEBUG: 2022-11-09 20:48:24.7954 ParameterBinding Information: 0 :             BIND arg [The term 'Get-Locatio' is not recognized as a name of a cmdlet, function, script file, or executable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.] to parameter [InputObject]
DEBUG: 2022-11-09 20:48:24.7967 ParameterBinding Information: 0 :                 BIND arg [The term 'Get-Locatio' is not recognized as a name of a cmdlet, function, script file, or executable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.] to param [InputObject] SUCCESSFUL
DEBUG: 2022-11-09 20:48:24.7974 ParameterBinding Information: 0 :         MANDATORY PARAMETER CHECK on cmdlet [format-default]
DEBUG: 2022-11-09 20:48:24.7978 ParameterBinding Information: 0 :         CALLING ProcessRecord
DEBUG: 2022-11-09 20:48:24.8372 ParameterBinding Information: 0 :             BIND NAMED cmd line args [Set-StrictMode]
DEBUG: 2022-11-09 20:48:24.8377 ParameterBinding Information: 0 :                 BIND arg [True] to parameter [Off]
DEBUG: 2022-11-09 20:48:24.8381 ParameterBinding Information: 0 :                     COERCE arg to [System.Management.Automation.SwitchParameter]
DEBUG: 2022-11-09 20:48:24.8384 ParameterBinding Information: 0 :                         Parameter and arg types the same, no coercion is needed.
DEBUG: 2022-11-09 20:48:24.8388 ParameterBinding Information: 0 :                     BIND arg [True] to param [Off] SUCCESSFUL
DEBUG: 2022-11-09 20:48:24.8392 ParameterBinding Information: 0 :             BIND POSITIONAL cmd line args [Set-StrictMode]
DEBUG: 2022-11-09 20:48:24.8395 ParameterBinding Information: 0 :             MANDATORY PARAMETER CHECK on cmdlet [Set-StrictMode]
DEBUG: 2022-11-09 20:48:24.8399 ParameterBinding Information: 0 :             CALLING BeginProcessing
DEBUG: 2022-11-09 20:48:24.8402 ParameterBinding Information: 0 :             CALLING ProcessRecord
DEBUG: 2022-11-09 20:48:24.8406 ParameterBinding Information: 0 :             CALLING EndProcessing
DEBUG: 2022-11-09 20:48:24.8539 ParameterBinding Information: 0 :             BIND NAMED cmd line args [Set-StrictMode]
DEBUG: 2022-11-09 20:48:24.8544 ParameterBinding Information: 0 :                 BIND arg [1] to parameter [Version]
DEBUG: 2022-11-09 20:48:24.8548 ParameterBinding Information: 0 :                     Executing DATA GENERATION metadata: [Microsoft.PowerShell.Commands.SetStrictModeCommand+ArgumentToPSVersionTransformationAttribute]
DEBUG: 2022-11-09 20:48:24.8553 ParameterBinding Information: 0 :                         result returned from DATA GENERATION: 1.0
DEBUG: 2022-11-09 20:48:24.8556 ParameterBinding Information: 0 :                     COERCE arg to [System.Version]
DEBUG: 2022-11-09 20:48:24.8560 ParameterBinding Information: 0 :                         Parameter and arg types the same, no coercion is needed.
DEBUG: 2022-11-09 20:48:24.8564 ParameterBinding Information: 0 :                     Executing VALIDATION metadata: [Microsoft.PowerShell.Commands.SetStrictModeCommand+ValidateVersionAttribute]
DEBUG: 2022-11-09 20:48:24.8568 ParameterBinding Information: 0 :                     BIND arg [1.0] to param [Version] SUCCESSFUL
DEBUG: 2022-11-09 20:48:24.8571 ParameterBinding Information: 0 :             BIND POSITIONAL cmd line args [Set-StrictMode]
DEBUG: 2022-11-09 20:48:24.8575 ParameterBinding Information: 0 :             MANDATORY PARAMETER CHECK on cmdlet [Set-StrictMode]
DEBUG: 2022-11-09 20:48:24.8578 ParameterBinding Information: 0 :             CALLING BeginProcessing
DEBUG: 2022-11-09 20:48:24.8582 ParameterBinding Information: 0 :             CALLING ProcessRecord
DEBUG: 2022-11-09 20:48:24.8585 ParameterBinding Information: 0 :             CALLING EndProcessing
DEBUG: 2022-11-09 20:48:24.8592 ParameterBinding Information: 0 :             BIND NAMED cmd line args [Set-StrictMode]
DEBUG: 2022-11-09 20:48:24.8595 ParameterBinding Information: 0 :                 BIND arg [1] to parameter [Version]
DEBUG: 2022-11-09 20:48:24.8599 ParameterBinding Information: 0 :                     Executing DATA GENERATION metadata: [Microsoft.PowerShell.Commands.SetStrictModeCommand+ArgumentToPSVersionTransformationAttribute]
DEBUG: 2022-11-09 20:48:24.8604 ParameterBinding Information: 0 :                         result returned from DATA GENERATION: 1.0
DEBUG: 2022-11-09 20:48:24.8607 ParameterBinding Information: 0 :                     COERCE arg to [System.Version]
DEBUG: 2022-11-09 20:48:24.8610 ParameterBinding Information: 0 :                         Parameter and arg types the same, no coercion is needed.
DEBUG: 2022-11-09 20:48:24.8614 ParameterBinding Information: 0 :                     Executing VALIDATION metadata: [Microsoft.PowerShell.Commands.SetStrictModeCommand+ValidateVersionAttribute]
DEBUG: 2022-11-09 20:48:24.8618 ParameterBinding Information: 0 :                     BIND arg [1.0] to param [Version] SUCCESSFUL
DEBUG: 2022-11-09 20:48:24.8621 ParameterBinding Information: 0 :             BIND POSITIONAL cmd line args [Set-StrictMode]
DEBUG: 2022-11-09 20:48:24.8625 ParameterBinding Information: 0 :             MANDATORY PARAMETER CHECK on cmdlet [Set-StrictMode]
DEBUG: 2022-11-09 20:48:24.8628 ParameterBinding Information: 0 :             CALLING BeginProcessing
DEBUG: 2022-11-09 20:48:24.8632 ParameterBinding Information: 0 :             CALLING ProcessRecord
DEBUG: 2022-11-09 20:48:24.8635 ParameterBinding Information: 0 :             CALLING EndProcessing
Get-Locatio: The term 'Get-Locatio' is not recognized as a name of a cmdlet, function, script file, or executable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
DEBUG: 2022-11-09 20:48:24.8905 ParameterBinding Information: 0 : BIND NAMED native application line args [C:\Windows\system32\cmd.exe]
DEBUG: 2022-11-09 20:48:24.8912 ParameterBinding Information: 0 :     BIND cmd line arg [/c] to position [0]
DEBUG: 2022-11-09 20:48:24.8917 ParameterBinding Information: 0 :     BIND cmd line arg [echo] to position [1]
DEBUG: 2022-11-09 20:48:24.8928 ParameterBinding Information: 0 :     BIND cmd line arg [foo,cd=$(Get-Locatio)] to position [2]
DEBUG: 2022-11-09 20:48:24.9049 ParameterBinding Information: 0 : CALLING BeginProcessing
foo,cd=$(Get-Locatio)
C:\Windows\System32>

Expected behavior

C:\Foo> docker run --rm -it --mount type=bind,src=$(Get-Location),dst=C:\target mcr.microsoft.com/windows/servercore:10.0.20348.887 powershell.exe
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

Install the latest PowerShell for new features and improvements! https://aka.ms/PSWindows

PS C:\> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      5.1.20348.859
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.20348.859
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1


PS C:\>

Actual behavior

C:\Foo> docker run --rm -it --mount type=bind,src=$(Get-Location),dst=C:\target mcr.microsoft.com/windows/servercore:10.0.20348.887 powershell.exe
docker: Error response from daemon: invalid mount config for type "bind": invalid mount path: '$(Get-Location)' mount path must be absolute.
See 'docker run --help'.
C:\Foo>

Error details

No response

Environment data

C:\> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      7.3.0
PSEdition                      Core
GitCommitId                    7.3.0
OS                             Microsoft Windows 10.0.22621
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Visuals

No response

@conioh conioh added the Needs-Triage The issue is new and needs to be triaged by a work group. label Nov 9, 2022
@mklement0
Copy link
Contributor

Wow - that is very unfortunate. The workaround for now is to double-quote the argument (cmd /c echo "a,x=$x").

The problem isn't Windows-specific; Unix repro: /bin/echo a,$HOME and even /bin/echo a, $HOME; that is, a trailing unquoted , in an argument prevents expansion of the next one.

Note that there's an older, similar bug: /bin/echo -foo=$HOME doesn't expand either (there are several bugs related to --prefixed arguments):

@conioh
Copy link
Author

conioh commented Nov 9, 2022

@mklement0, thanks. This is very helpful.

I'm sure there are lots of other bugs here (I know some of them up close ☹️), but this is a regression which makes it even worse.

@237dmitry
Copy link

$ $PSNativeCommandArgumentPassing = 'Legacy' 
$ /bin/echo a,$PShome
a,/opt/microsoft/powershell/7

But:

$ /bin/echo -foo=$PShome
-foo=$PShome

@JamesWTruher
Copy link
Member

There's specialized code added for this new behavior (fixing a different issue) which treats arguments with commas slightly different than before. Variables are no longer expanded when they appear with commas. However, the behavior you want can be easily achieved by creating an expandable string ala "a,b,x=$host" that will then expand the variables.

@mklement0
Copy link
Contributor

mklement0 commented Nov 15, 2022

@JamesWTruher, the workaround has already been mentioned.

What this issue is about is a (previously) unacknowledged-as-such breaking change, for which there appears to be no good (conceptual) reason - "There's specialized code added for this new behavior" certainly doesn't qualify.

Yes, even without the bug at hand compound string tokens behave in inconsistent, but at least predictable ways:

  • In short: only if a compound token starts with an unquoted token are variable references / subexpressions considered part of the same argument.

Therefore, even though it is generally advisable to enclose compound tokens that are to form a single argument in "...", there is no justification for this breaking change, which appears to be the accidental fallout from a change in the underlying implementation.

@conioh
Copy link
Author

conioh commented Nov 15, 2022

There's specialized code added for this new behavior (fixing a different issue) which treats arguments with commas slightly different than before.

Great. Then it should be easy to remove this specialized code. Please do that.

Variables are no longer expanded when they appear with commas.

image

Neither the blog post nor the documentation mention any of this.
And what does "with" mean?
And how does it make sense to execute the cmdlet but ignore its result? That's doubly insane.

PS C:\> $PSNativeCommandArgumentPassing = "Legacy"
PS C:\> cmd /c echo foo,cd=$(Write-Host "completely insane"; "return")
completely insane
foo,cd=return
PS C:\> $PSNativeCommandArgumentPassing = "Windows"
PS C:\> cmd /c echo foo,cd=$(Write-Host "completely insane"; "return")
completely insane
foo,cd=$(Write-Host"completelyinsane";"return")
PS C:\>

Not executing the cmdlet/script block if you're not going to use its return value isn't the bare minimum, it's way below the minimum.
Even assuming breaking everything and not expanding variables when they're "with" commas is somehow fine, how do you consider this even remotely acceptable?
You want to treat this as a "raw string"? Fine. then don't execute it. 🤦

@JamesWTruher
Copy link
Member

JamesWTruher commented Nov 28, 2022

this was an explicit breaking change as part of the native argument passing changes.
The new behavior requires that you use an expandable string.
cmd /c echo "foo,cd=$(Write-Host 'completely insane'; 'return')"

this behavior is only when commas are used in the open.
generally when a comma is used, it would be parsed as an array.

@SteveL-MSFT
Copy link
Member

@conioh a friendly reminder of our Code of Conduct. Let's keep the discussion civil and professional. We understand your frustration here, but we need to think through both the short term and long term effects.

@conioh
Copy link
Author

conioh commented Nov 28, 2022

this was an explicit breaking change as part of the native argument passing changes. The new behavior requires that you use an expandable string. cmd /c echo "foo,cd=$(Write-Host 'completely insane'; 'return')"

this behavior is only when commas are used in the open. generally when a comma is used, it would be parsed as an array.

@JamesWTruher, may we know

  1. What was the bug you referred to above?
  2. Why does fixing it require this change? Was it discussed on GitHub publicly? If so can we see the discussion?
  3. Why wasn't this documented? Can we expect it to be documented? If so, when?

@conioh
Copy link
Author

conioh commented Nov 28, 2022

@conioh a friendly reminder of our Code of Conduct. Let's keep the discussion civil and professional.

@SteveL-MSFT, thank you for the reminder!
I would like to remind you that that so far the responses here were not what I would consider professional. Elsewhere Mr. Klement has also expressed a similar view. I would love nothing more than that to achieve a professional discussion, and once we're there we should strive to keep it so.

We understand your frustration here, but we need to think through both the short term and long term effects.

Please consider the long-term effects of ¹seemingly unjustified ²undocumented, ³breaking changes for which ⁴no explanation but "trust me, we had a reason" is given on trust towards you, PowerShell and the company you work for.

@SteveL-MSFT
Copy link
Member

@conioh I think it's fair feedback, that members of the team should provide more context and reasoning on changes (or at least link to other issues where it was discussed)

@mklement0
Copy link
Contributor

this was an explicit breaking change as part of the native argument passing changes.

There is no justification for changing the expansion behavior in the context of $PSNativeCommandArgumentPassing, which is solely about quoting - especially since there is no discernible conceptual reason to do so:

  • The new behavior offers no improvement whatsoever - on the contrary: it is unexpected and has unwanted side effects (unexpected evaluation despite non-expansion)
  • It therefore needlessly breaks existing code.

generally when a comma is used, it would be parsed as an array.

That applies to arguments passed to PowerShell commands, but has never applied to external (native) programs (which have no notion of arrays) - and that shouldn't change.

(Aside from that, this aspect is incidental to the discussion of whether expansion should occur.)

In the past, passing something like foo,cd=$(1+2) to an external program was treated as if it were a single, expandable string ("foo,cd=$(1+2)"), and there is no reason to change that.

@mklement0
Copy link
Contributor

mklement0 commented Dec 7, 2022

On the meta issue, @SteveL-MSFT

members of the team should provide more context and reasoning on changes

That is encouraging to hear. 🤞for the future.

As for the present:

Your comment came not only after the initial, bewilderingly cavalier team-member response, but after a follow-up response that also failed to provide a reasoning (while making the spurious claim that "the new behavior requires that you use an expandable string").

I do hope that a proper response is forthcoming, one that addresses the specific arguments made above, namely that this is bug that, even if the change were intentional, would have no conceptual justification, and is therefore an unacceptable breaking change.

@mklement0
Copy link
Contributor

@237dmitry, as for -foo=$PShome: This is a bug that (long) predates the 7.3.0 changes, and is one of several related bugs: see #14587 for the specific bug.

@conioh
Copy link
Author

conioh commented Dec 15, 2022

this was an explicit breaking change as part of the native argument passing changes. The new behavior requires that you use an expandable string. cmd /c echo "foo,cd=$(Write-Host 'completely insane'; 'return')"

this behavior is only when commas are used in the open. generally when a comma is used, it would be parsed as an array.

@JamesWTruher, I ask again, after more than two weeks, may we know

  1. What was the bug you referred to above?
  2. Why does fixing it require this change? Was it discussed on GitHub publicly? If so can we see the discussion?
  3. Why wasn't this documented? Can we expect it to be documented? If so, when?

CC @SteveL-MSFT

@conioh
Copy link
Author

conioh commented Dec 29, 2022

this was an explicit breaking change as part of the native argument passing changes. The new behavior requires that you use an expandable string. cmd /c echo "foo,cd=$(Write-Host 'completely insane'; 'return')"

this behavior is only when commas are used in the open. generally when a comma is used, it would be parsed as an array.

@JamesWTruher, I ask again, after a month, may we know

  1. What was the bug you referred to above?
  2. Why does fixing it require this change? Was it discussed on GitHub publicly? If so can we see the discussion?
  3. Why wasn't this documented? Can we expect it to be documented? If so, when?

CC @SteveL-MSFT Are these questions difficult to answer or is the information classified? Is it TOP SECRET and available only under a special access program? #professionalism_and_all_that_jazz

@SteveL-MSFT
Copy link
Member

SteveL-MSFT commented Dec 29, 2022

@conioh team members are on vacation currently (as am I). Mondays are days the team specifically focuses on community issues and PRs (although given the number of them, you cannot expect priority attention here). In general, being a bit nicer will make people (not just team members, but community members) more likely to want to respond.

@conioh
Copy link
Author

conioh commented Dec 30, 2022

@conioh team members are on vacation currently (as am I).

Of course they are currently on vacation, but were they the entire past month? I find it unlikely, but even if they were on vacation for a month, it's reasonable to expect from their employer and manager to assign someone else to fill in. At least that's what happened when my colleagues went on long vacations. Don't know how they do it over there at Microsoft.

Mondays are days the team specifically focuses on community issues and PRs (although given the number of them, you cannot expect priority attention here).

Given that this is basically a security vulnerability I can certainly expect priority here and I indeed expect it.
You're executing as code a string that should be executed.
Not only that, you're claiming it is justified in order to resolve some secret bug that you won't even link to.
And on top of that now you're claiming that giving this link is such a burden that it's reasonable to wait for over a month such to get the link, not a fix or anything like that.

In general, being a bit nicer will make people (not just team members, but community members) more likely to want to respond.

Perhaps. On the other hand between the two of us I'm the one who got an unsolicited upvote (I'm sure you can call all you buddies to give you a hundred upvotes but that won't change or prove anything) and you're the one who raised the issue of professionalism. Could have avoided that, but now that you have I think it's a great issue to explore and I'm just providing another perspective on it. "In general, being a bit more competent and responsible will make people (not just users, but the open source community and the general public) more likely to want to respect you."

@HumanEquivalentUnit
Copy link
Contributor

Given that this is basically a security vulnerability

Can you expand on that, give an example scenario? (e.g. What security boundary is crossed, what could an attacker do now which they couldn't do before?).

@mklement0
Copy link
Contributor

mklement0 commented Feb 9, 2023

@HumanEquivalentUnit, we don't need to go down that route, given the following:

  • There is no good conceptual reason for this change - it clearly is the fallout from an implementation change.

  • If this - highly unfortunate, counterintuitive, frivolous - change is allowed to stand (why?), it is obviously pointless and can have unwanted side effects to actually evaluate the subexpression, if it is meant to be passed as a string literal to the native program.

@HumanEquivalentUnit
Copy link
Contributor

@mklement0 conioh's 'provocative style' and your focus on the 'good conceptual reason' had obscured the important part for me. In case that is similar for anyone else...

@JamesWTruher with the new behaviour, if you want to pass the text literally then subexpressions ought not to be evaluated, but they are:

PS C:\> cmd /c echo a,x=$(get-item c:\imaginary)
Get-Item: Cannot find path 'C:\imaginary' because it does not exist.
a,x=$(get-itemc:\imaginary)

That get-item should not be executed in this code to trigger the exception, is that right?

@mklement0
Copy link
Contributor

mklement0 commented Feb 10, 2023

@HumanEquivalentUnit, that is the wrong thing to focus on.

The primary focus should be:

  • Why allow this frivolous breaking change to begin with?
  • Why - if the change was intentional, as was claimed - not offer a rationale for it (instead of vaguely saying "we changed something, and - trust us - we meant to" and repeatedly noting an - already mentioned - workaround)?
  • if it truly is intentional - which seems doubtful - and a satisfactory rationale has been provided, then its inconsistent behavior should be addressed - but I certainly hope it doesn't have to get to that point.

At least that has been my focus throughout this thread.

Honestly, you can't fault @conioh for getting frustrated with the team's responses here, primarily by @JamesWTruher (part of a recurring pattern, such as this response, for example), and secondarily - through appeasement and silence - by @SteveL-MSFT.

@HumanEquivalentUnit
Copy link
Contributor

@mklement0 your comment of April 2021 cited "Fiat decisions handed down with little to no explanation [...] followed by unwillingness to explain or debate the decision." - has that changed, are you honestly expecting a satisfying reason to appear?

I suspect the subexpressions still being evaluated has not been noticed; it is an unexpected and potentially dangerous behaviour which ought to be addressed even if the change is staying and we never get an explanation (I think both are likely). Letting it be lost in the noise of comments like "(I'm sure you can call all you buddies to give you a hundred upvotes but that won't change or prove anything)" making the PS Team roll their eyes and stop reading would be a shame.

@mklement0
Copy link
Contributor

are you honestly expecting a satisfying reason to appear?

If enough people push back long enough, hopefully, yes.

I am not condoning abrasive comments, and you're right that they make the arguments less likely to be heard; I was just pointing out that it is not hard to see what triggered them.

Yes, a resolution is necessary here, but, given that we're dealing with what is to me an obvious bug that is an accidental fallout from an implementation change, I think it's important to stay focused on that, which I've tried to.

I suspect the subexpressions still being evaluated has not been noticed;

It's been discussed repeatedly and explicitly here, and @conioh even created a separate issue focused on this: #18668

That, along with your comments here, should leave no doubt that we're not only dealing with a breaking change, but also potentially dangerous side effects.

@SteveL-MSFT SteveL-MSFT added the WG-Engine core PowerShell engine, interpreter, and runtime label Feb 13, 2023
@SteveL-MSFT
Copy link
Member

Let's have the Engine WG discuss and make a recommendation

Copy link
Contributor

This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.

1 similar comment
Copy link
Contributor

This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.

@microsoft-github-policy-service microsoft-github-policy-service bot added Resolution-No Activity Issue has had no activity for 6 months or more labels Nov 15, 2023
Copy link
Contributor

This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.

Copy link
Contributor

This issue has been marked as "No Activity" as there has been no activity for 6 months. It has been closed for housekeeping purposes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs-Triage The issue is new and needs to be triaged by a work group. Resolution-No Activity Issue has had no activity for 6 months or more WG-Engine core PowerShell engine, interpreter, and runtime
Projects
None yet
Development

No branches or pull requests

7 participants