Stop-parsing token is broken

[mklement0]

Prerequisites

• Write a descriptive title.
• Make sure you are able to repro it on the latest released version
• Search the existing issues.
• Refer to the FAQ.
• Refer to Differences between Windows PowerShell 5.1 and PowerShell.

Steps to reproduce

This was previously reported in #15261, which was inappropriately closed as by-design.

The purpose of --% (which only makes sense on Windows) is to copy the remainder of the command line verbatim to the process command line constructed behind the scenes, so as to provide a way to explicitly control the double-quoting for target executables that have non-standard quoting requirements.

As such, it should not be affected by the effective value of $PSNativeCommandArgumentPassing

(Note: It is debatable how arguments that come before --% on a given command line should be handled; for backward compatibility I think it would make the most sense to process them as if 'Legacy' were in effect; in other words: --% would be an implicit, command-scoped opt-in to $PSNativeCommandArgumentPassing = 'Legacy').

However, as of v7.3.0 it is, and the behavior is broken by default (the only way to make it work again is with $PSNativeCommandArgumentPassing = 'Legacy', which should not be necessary).

# Preparation: Create an executable that echoes the raw command line, as `.\echoCmdLine.exe`
# Note: Must call via *Windows PowerShell*
powershell -NoProfile {
  Add-Type -OutputType ConsoleApplication -OutputAssembly ./echoCmdLine.exe -TypeDefinition @'
  using System;
  static class ConsoleApp {
    static int Main(string[] args) {
      Console.WriteLine(Environment.CommandLine);
      return 0;
    }
  }
'@
}

# Set $PSNativeCommandArgumentPassing  to something *other* than 'Legacy'
$PSNativeCommandArgumentPassing = 'Windows'  # (the default on Windows)

# Using --%, pass *verbatim* `/p:Category="CI,Nightly"` on the process command line.
./echoCmdLine --% /p:Category="CI,Nightly"

Expected behavior

"C:\Users\jdoe\echoCmdLine.exe"  /p:Category="CI,Nightly"

That is, the part of the original command line after --% should be copied as-is to the process command line.

That is how it has worked since v3 (when --% was introduced) and should continue to work, irrespective of the value of $PSNativeCommandArgumentPassing = 'Legacy', in both PowerShell editions.

Actual behavior

"C:\Users\jdoe\echoCmdLine.exe" "/p:Category=\"CI,Nightly\""

Note how the argument was inappropriately enclosed in "...", with embedded " escaped as \".

That is, the 'Standard' behavior was inappropriately used (although cmd.exe-style environment variables, e.g. %OS%, do still get expanded).

Error details

No response

Environment data

PowerShell Core 7.3.0 on Windows

Visuals

No response

[SteveL-MSFT]

Agree that --% means verbatim, so no additional processing should happen.

[Scordo]

This would really ease our backward compatibility issues. Please fix this in 7.3.1 as soon as possible :-)

[Scordo]

@SteveL-MSFT Any plans to fix this? I think this is the only thing that would make it possible to make old scripts (7.2.*) compatible to new versions of powershell (7.3+) where escaped quotes are used. Btw. I still cant find any documentation on how to handle these problems. How can I call an executable which does not use standard command line escaping like msbuild in a compatible manner in all versions of powershell 5.1 to 7.4?

[nickcherrill]

The title fits our issue, though the details of how our workflow was interrupted by changes to --% is not in the original post. I don't see a similar bug raised and it relates to the same change, so I'll include my issue here.

I have found this is still broken in v7.4.1 release. We were updated to v7.4.1 from v7.2.18 on the MS Hosted Windows agents for DevOps.

This broke our existing pipelines, though I understand it is intended to be a breaking change as detailed here.

Our use case:
We use this in our deployment pipelines to pass optional parameters to the Azure Resource Manager via Azure CLI run in Powershell.
The parameters can include generated values, we originally put the --% stop parsing token into the command to avoid a symbol being processed as part of the command.

It seems when the optional parameters after --% is blank, our pipeline fails on Az CLI stating unrecognized arguments, which is more Az CLI than Powershell but this behaviour started when the Powershell version was upgraded.
I'm yet to test, but I'll try with $PSNativeCommandArgumentPassing = 'Legacy'.