Inconsistent, unpredictable behavior of "-"-prefixed barewords in argument mode

[mklement0]

^{Follow-up to #4576 and #4591}

Context:

Note: By bareword I mean a run of non-whitespace characters not single- or double-quoted as a whole (e.g., foo), used in argument mode, and normally interpreted as an expandable string (e.g., $HOME/boy).

While --prefixed barewords look like parameter names (e.g.,-foo), they sometimes act as arguments (values to bind to parameters), and the ability to use them as such is important in two contexts:

• In the context of the external CLI, when passing the name of a script file that happens to start with -; e.g., powershell -File -foo.ps1

• When passing arguments through that themselves happen to be command line or command-line fragments, which is enabled by cmdlets/advanced functions that declare a ValueFromRemainingArguments parameter; e.g.,
  Get-Command -Syntax Get-ChildItem -Path cert: passes -Path cert: through (the -Path bareword is not interpreted as Get-Command's parameter) in order to discover the certificate provider's dynamic parameters.

Note that, except when using the external CLI, you can always use quoting to explicitly mark a bareword as an argument.

Generally, though, if there's no ambiguity, not having to quote tokens that don't strictly need it is a matter of convenience.

The problem:

There are two major problems with how --prefixed barewords are currently parsed and interpreted:

• Pragmatically speaking, their syntactic role (parameter name vs. argument) is virtually unpredictable by the end user.

  • E.g., the -a in Get-Item -LiteralPath -a is an argument, whereas the -c in Get-Item -LiteralPath -c is parameter name-Credential.

• They are not parsed and expanded in the same way as barewords that do not start with -.

  • E.g., bareword a$HOME expands $HOME, while -a$HOME does not.

More examples below.

Solution options:

• Leave things as they are and document the current behavior, recommending that quoting be used for predictability.

• Change the parameter binding as proposed in Parameter values disambiguated with their explicit parameter names should not require quoting #4576: Once a parameter has been (unambiguously) identified, its specific definition - switch vs. parameter-with-argument- alone determines whether the next token is its argument or a different parameter:

  • E.g., in Get-Item -LiteralPath -p, -p would unambiguously be an argument, because -LiteralPath syntactically requires an argument.

    • An argument can be made that in cases where the user intent was likely different - e.g., Get-Item -LiteralPath -Path - that unconditional interpretation of --prefixed barewords as parameter names is preferable in terms of the user experience (Missing an argument for parameter 'LiteralPath' as opposed to Cannot find path '-Path' because it does not exist); a point that @markekraus has made.

    • That said, I think the conceptual simplicity and clarity of the let-the-identified-parameter-determine-the-next-token's-syntactic-role approach is more beneficial in the grand scheme of things.

  • Note: @lzybkr has already discounted this option based on implementation cost, but I'm resubmitting it for consideration now that we have a fuller picture of all the issues involved:

I can't see adding more complexity to the parameter binder to change this behavior.

• Disallow --prefixed barewords altogether, as proposed in "-"-prefixed barewords in argument mode should always be interpreted as parameter names #4591: In order for a --prefixed tokens to be recognized as an argument, it would have to be single- or double-quoted .

  • E.g., Get-Item -LiteralPath -a would then break, because Get-Item has no -a* parameter, and Get-Item -LiteralPath '-a' would have to used consistently. (But, in contrast, PowerShell's outside CLI would still need to accept something like -File -a.)

  • The convenience of being able to specify --prefixed tokens as arguments is lost, but at least the behavior is predictable.

    • That said, the waters would still be muddy with respect to commands that have a ValueFromRemainingArguments parameter, where, based on the current behavior, a --prefixed barewords if it happens not to match a regular parameter would still be interpreted as an argument.

Examples:

### Virtually unpredictable syntactic role

# BREAKS
# -c, because it happens to match parameter name -Credential, is interpreted as a *parameter name*.
# Note that, purely based on -LiteralPath being a parameter that *requires an argument*, there is
# no good reason to parse -c as a parameter name. The syntax of Get-Item unambigously implies that
# the token following -LiteralPath must be an *argument*.
Get-Item -LiteralPath -c

# SUCCEEDS
# -a, because it happens not to match any of Get-Item's parameters, is interpreted as an *argument*
Get-Item -LiteralPath -a


# Even though a "-"-prefixed token as the 1st argument is normally *invariably* interpreted as 
# a *parameter name*, that is not the case for cmdlets with a ValueFromRemainingArguments parameter,
# such as Write-Output:

# BREAKS
# -n happens to match parameter (switch) -NoEnumeration, and the call is therefore missing
# arguments to output (-InputObject)
Write-Output -n
# Solution: Use -- to signal the start of all positional arguments.
Write-Output -- -n

# SUCCEEDS, because -z happens not to match a parameter name and is therefore bound to -InputObject.
Write-Output -z

### Parsing / expansion inconsistencies

# As part of an array, a "-"-prefixed bareword doesn't work as the 1st element.
Write-Output -a, b	# BREAKS
Write-Output a, -b  # SUCCEEDS


# "-"-prefixed barewords are non-expanding
Write-Output a$HOME   # expands $HOME
Write-Output -a$HOME  # doesn't expand $HOME


# A "-"-prefixed bareword containing a "."-prefixed suffix is parsed as *2* arguments
Get-Item -LiteralPath a.ps1  # SUCCEEDS
Get-Item -LiteralPath -a.ps1 # BREAKS: Get-Item receives *2* arguments: '-a' and '.ps1'

Environment data

PowerShell Core v6.0.0-beta.5 on macOS 10.12.6
PowerShell Core v6.0.0-beta.5 on Ubuntu 16.04.3 LTS
PowerShell Core v6.0.0-beta.5 on Microsoft Windows 10 Pro (64-bit; v10.0.15063)
Windows PowerShell v5.1.15063.483 on Microsoft Windows 10 Pro (64-bit; v10.0.15063)

[markekraus]

@mklement0

But, in contrast, PowerShell's outside CLI would still need to accept something like -File -a.

As a point of clarity, this is not something under PowerShell's control as the calling shell (bash, another PowerShell, CMD, ksh, etc) has final say in the matter. What you are specifically requesting here is to maintain the current argument parsing for arguments passed from the calling shell to the PowerShell binary upon initialization. Correct?

[mklement0]

@markekraus:

What you are specifically requesting here is to maintain the current argument parsing for arguments passed from the calling shell to the PowerShell binary upon initialization. Correct?

Correct: powershell -File -foo.ps1 currently works already, and should continue to work.
(And as one aspect of conceptual simplicity and consistency, the same should work predictably PowerShell-internally as well (when calling functions/cmdlets)).

Calling from a POSIX-like shell (Unix; e.g., bash), powershell -File -foo.ps1 is indistinguishable from variants powershell -File '-foo.ps1' and powershell -File "-foo.ps1", because the POSIX-like shell performs quote removal before passing the array of arguments as literals to the target command: powershell sees just literals -File and -foo.ps1 in all cases.

Conceivably, PowerShell could - but shouldn't, in my view - then place additional requirements on the arguments passed, based on its syntax requirement, such as for the received -foo.ps1 argument to still be represented as '-foo.ps1', for instance, if
the decision were made (I hope not) that --prefixed tokens are only recognized as arguments when quoted.

If such a requirement existed, you'd have to call powershell -File "'-foo.ps1'" (or powershell -File '"-foo.ps1"') - and such two-unrelated-quoting-layers awkwardness is best avoided.

Unfortunately, we already have that awkwardness with respect to how -Command parsing works: because it assembles all arguments into a single string that it then interprets based on PowerShell rules, you do need two-layer quoting; e.g.:

# Oops! "The string is missing the terminator: '."
# PowerShell ended up interpreting `Write-Output don't` as the script.
powershell -noprofile -command Write-Output "don't"

# OK, but clearly cumbersome:
# With shell-variable-based arguments, it gets even trickier.
powershell -noprofile -command Write-Output "\"don't\""

See #4024 (comment) for more.

[microsoft-github-policy-service]

This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.

[microsoft-github-policy-service]

This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.

[microsoft-github-policy-service]

This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.

[microsoft-github-policy-service]

This issue has been marked as "No Activity" as there has been no activity for 6 months. It has been closed for housekeeping purposes.