Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Suggestion: implement Bash-style && and || control operators for chaining commands based on success / failure - short-circuiting Boolean execution-flow control operators #3241

Open
mklement0 opened this issue Mar 2, 2017 · 35 comments · May be fixed by #9849

Comments

@mklement0
Copy link
Contributor

commented Mar 2, 2017

Bash-style && and || control operators would be handy additions to the language.

For instance, instead of writing (the examples are contrived, but hopefully illustrate the point):

# If command A succeeds, execute command B too.
csc /nologo file.cs
if (0 -eq $LASTEXITCODE) { git commit -m 'ok' }

# if command A fails, execute command B.
csc /nologo file.cs
if (0 -ne $LASTEXITCODE) { git commit -m 'failed'  }

one might be able to write:

# If command A succeeds, execute command B too.
csc /nologo file.cs && git commit -m 'ok'

# if command A fails, execute command B.
csc /nologo file.cs || git commit -m 'failed'

The examples use external utilities, but these operators would be equally handy for PowerShell-native commands (where $? rather than $LASTEXITCODE would have to be examined).


&& and || were on the list of things to implement (still are) but did not pop up as the next most useful thing to add.

There are several other questions on Stack Overflow asking for this feature and I've tried to summarize the state of affairs in this answer.

@SteveL-MSFT SteveL-MSFT added this to the 6.1.0 milestone Mar 4, 2017

@mklement0 mklement0 changed the title Suggestion: implement Bash-style && and || control operators Suggestion: implement Bash-style && and || control operators for chaining commands based on success / failure Aug 14, 2017

@i-give-up

This comment has been minimized.

Copy link

commented Oct 3, 2017

I believe the proper term for && and || is "short-circuit operator".

@mklement0 mklement0 changed the title Suggestion: implement Bash-style && and || control operators for chaining commands based on success / failure Suggestion: implement Bash-style && and || control operators for chaining commands based on success / failure - short-circuiting Boolean execution-flow control operators Oct 3, 2017

@mklement0

This comment has been minimized.

Copy link
Contributor Author

commented Oct 3, 2017

@i-give-up: While short-circuiting is an important aspect of these operators' behavior, both Bash and POSIX call them control operators.

If you wanted to get as descriptive as possible, I guess you could call them short-circuiting Boolean execution-flow control operators.

So as to make this issue more easily discoverable, I've appended that term to the issue's title.

@lenkite

This comment has been minimized.

Copy link

commented Oct 18, 2017

Short circuit operators are such a fundamental part of the daily shell use at the command line and it is a massive burden to work without them. And the -and/-or operators are not a replacement since they simply output the boolean result of the operation making them useless for interactive use.

@stofte

This comment has been minimized.

Copy link

commented Nov 9, 2017

I dont understand what the -and operator actually does (and presumably the -or as well).

It's simply baffling to have the following code and see "True" come out when you run it

function Foo { return $True }
function Bar { return Foo -and $False }
Write-Host "Bar: $(Bar)"

The OPs stackoverflow really shows everything that is fubared about this design.

@mklement0

This comment has been minimized.

Copy link
Contributor Author

commented Nov 9, 2017

@stofte: PowerShell has 2 personalities (fundamental syntax forms): argument mode (like a shell) and expression mode (like a programming language) - see Get-Help about_Parsing.

-and and -or belong to the expression-mode world and operate on (often inferred) Boolean values.
They have no equivalent in argument mode, and that's the crux of the problem.

Your example, Foo -and $False is parsed in argument mode, because the statement's first token is a command. Therefore, you're passing tokens -and and -$False as arguments (parameter values) to function Foo - where, in your case, they're ignored and the output is simply $True.

If you want to use a command (argument mode) as part of an expression (expression mode; e.g., with -and), you must enclose it in parentheses:

return (Foo) -and $False
@stofte

This comment has been minimized.

Copy link

commented Nov 10, 2017

@mklement0 Thanks for the explanation. It's still not entirely clear how argument modes relates and interacts with outcome vs output parsing/interpretation however.

PS is probably the most confusing language I've come across in my years. It seems to assemble a hodge-podge of ideas in order to be everything for everyone (anyway, ranting over!)

@dragonwolf83

This comment has been minimized.

Copy link

commented Nov 10, 2017

@stofte You pass parameters to functions via the - character so any - characters after the function are going to be assumed to be parameters. So Foo -and $false is really trying to pass -and as the parameter name with $false as the value but since Foo doesn't define any parameters, it just ignores it.

Let's change your script a bit to add parameters and see how it works.

function Foo 
{ 
    Param($and) 
    Write-Host "And = $and"
    
    return $True 

}
function Bar { Foo -and $False }
Write-Host "Bar: $(Bar)"

Now, Foo will accept the parameter and pass $false into the function. However, it still returns $true since that is what the function outputs.

So for PowerShell to know if it shouldn't be trying to pass -and as a parameter, you need to use parenthesis to control order of operations. (Foo) -and $false now will run Foo first, then evaluate the remaining statement $true -and -$false.

BTW, PowerShell ISE and Visual Studio Code is smart enough to tell you all of this via Intellisense. It knows that the -and operator can't be used in your original code and won't give it as an option. I highly recommend using Intellisense to make sure keywords are usable in the right context.

Now for why functions are using - to pass parameters is because the idea behind PowerShell is to create command-line applications through functions instead of having to compile an application. So a Function mimics much of the command-line experience to make it happen.

@jurriaanr

This comment has been minimized.

Copy link

commented Apr 13, 2018

I can't believe I still have to start up cmd for this functionality, my bash using colleagues are making fun of me

@rjmholt

This comment has been minimized.

Copy link
Member

commented Apr 21, 2018

I've been thinking about this a bit.

A thought I have is that if && and || only look at $LASTEXITCODE it makes them pretty specific -- two new operators that only work in the context of native commands. And without good knowledge of that fact, something like

csc.exe program.cs

Foo && Bar

could prove very confusing, since it either uses the $LASTEXITCODE value, in which case Bar being executed depends on the success of the csc.exe line. OR, if just assumes that a non-native command succeeds (or whatever) that is just as confusing. OR, we throw an error because Foo isn't a native command...

So I was thinking about something more like JavaScript's (please, hear me out...) short circuit operator implementation. That way you could do:

> "" && "Hello"

> "Hi" && "Hello"
Hello

>"Hi" || "Hello"
Hi

> "" || "Hello"
Hello

Specifically, Foo && Bar evaluates Foo and if the result coerces to $false, it returns that (pre-coercion) value, otherwise it returns the result of Bar. With Foo || Bar, Foo is evaluated and if the result coerces to $true it returns that (pre-coercion) result, otherwise we evaluate and return the result of Bar. The operators would have equal precedence and be left-associative, so you can chain them like so:

> "" || "Hello" && "Goodbye"
Goodbye

That last thing is pretty confusing, but would be useful for chains of only one operator (Step1 && Step2 && Step3, Try1 || Failover1 || Failover2) and I see no particular reason not to support a mixed chain.

Notice that this also gives us a null-coalescing functionality like so: $x = $v || "default". With a mixed chain you'd even get a (rather confusing) ternary operator: $condition && $ifTrue || $ifFalse.

However, the problem is that this wouldn't scratch the itch of wanting a way to chain native commands. So, we could make it that when one of these operators evaluates a native command as an argument, it proceeds based on the $LASTEXITCODE value, so that the following work:

> csc.exe badprogram.cs && RunProgram
# error output from csc.exe, RunProgram not evaluated

> SetupProject && csc.exe goodprogram.cs
# output from csc.exe, SetupProject evaluated

I think that's a bit complex, but unfortunately it seems to be most sensible and useful option I can see.

Anyway, please let me know your thoughts on this -- I'm just spitballing ideas here (although I'm always in favour of consistency). After some initial thoughts/comments here, I'd like to write up an RFC with a more concrete proposal and then implement a prototype.

@saschanaz

This comment has been minimized.

Copy link

commented Apr 21, 2018

So I was thinking about something more like JavaScript's (please, hear me out...) short circuit operator implementation

I still think we need to use $LASTEXITCODE. For example, think about npm run build && npm run deploy. npm run build can emit some text even when it fails, and I want to prevent npm run deploy from running after a failed build.

@rjmholt

This comment has been minimized.

Copy link
Member

commented Apr 21, 2018

Yes, but exclusively $LASTEXITCODE? Or in the way I describe at end where non-native commands work by falsey/truthy-ness and native commands use $LASTEXITCODE?

My concern with exclusively using $LASTEXITCODE is as I say above: if you run a native command and then use && later, like:

npm run build

DoSomething && DoAnotherThing

what should && do?

@saschanaz

This comment has been minimized.

Copy link

commented Apr 21, 2018

So, we could make it that when one of these operators evaluates a native command as an argument, it proceeds based on the $LASTEXITCODE value, so that the following work

if you run a native command and then use && later, like:

Throwing when when something is not (edit) a native command may look simpler. The suggested behavior looks inconsistent for me when:

> npm --version && "Hello"
5.6.0
Hello
> $v = npm --version; $v && "Hello"
Hello
@mklement0

This comment has been minimized.

Copy link
Contributor Author

commented Apr 21, 2018

Please note that && and || in Bash are control(-only) operators:

  • they only decide what commands get executed in principle, based on the commands' exit codes
  • but do not interfere with the commands' output streams
  • and what the commands do or do not output (send to the output streams) is irrelevant for the determination of execution success.
#!/bin/bash
echo one && echo two  # prints *both* 'one' and 'two'

In other words: It's a way of conditionally chaining commands with their usual output behavior, if they get to execute.

By analogy, in PowerShell their detection of successful execution should be based on:

  • $LASTEXITCODE -eq 0 for external utilities (native commands)

  • $? -eq $True for PowerShell commands (cmdlets / functions / scripts)

&& and || primary belong to the realm of commands (something called by a name, with arguments parsed in argument mode) and has limited applicability to expressions:

"a" && "b"  # should output *both*  'a' and 'b'

In a simple expression such as "a", there's nothing that can fail, so use of && makes little sense.

What's worse, because && / || only act on execution success, even falsy expressions are "true" from their perspective, because it isn't their value (output) that matters:

$False && "hi"  # !! should output *both*  'False' and 'hi' - both expressions "ran" without error

So, one way to bypass potential confusion is to restrict use of && and || to pipelines only; in other words: all operands must be either simple commands or pipelines.

There are several problems with this approach, however:

  • Pipelines can start with an expression, which could be confusing. E.g.,
    'foo' | Write-Output && ... would be allowed, but 'foo' && ... would not.

  • Arguably, using an expression as the RHS of && and || would be useful; e.g.,
    Get-ChildItem -ea Ignore -Name *.log || 'log1.log'

  • More generally, wrapping commands in expressions can be useful; e.g.,
    (Get-ChildItem -ea ignore *.log).LastWriteTime || [datetime] 0

  • There are pure expressions that can fail, generating a statement-terminating error in the case of (implicitly) calling a .NET framework method that throws an exception; e.g.:

    • [int]::parse('hi')
    • $div = 0; 1 / $div

If we do allow expressions, the problems are:

  • Sowing confusion among users that come from the C# / JavaScript, ... world who are used to && and || having the semantics of what -and and -or do in PowerShell (such languages have no counterpart to what Bash's && and || do).

  • Similarly, pointless constructs such as 1 && 2 are enabled (applying the operators to expressions that never fail); the example is equivalent to 1; 2, i.e., simple serial execution.

  • $? currently also reflects expression success, which wipes out the $? value from any wrapped command, which happens with something as simple as enclosing a command in (...); e.g.,
    Get-Item NoSuchFile; $? outputs $False whereas
    (Get-Item NoSuchFile); $? outputs $True
    The current $? behavior is summarized here, with clarifications by @BrucePay here.

    • This severely limits $?'s usefulness in general, but especially in the context of implementing && and ||

    • The solution would be to make only commands (outside conditionals) set $?, not expressions, with the exception of a .NET operation causing a statement-terminating error, such as [int]::Parse('hi').

@rjmholt

This comment has been minimized.

Copy link
Member

commented Apr 23, 2018

Yes I see what you mean.

In my mind then, && and || must always write the result of an evaluation to the relevant stream as you say and proceed based on the value of $? (assuming that always accords with $LASTEXITCODE in the case of native commands, and if not, we check $LASTEXITCODE -eq 0 in those cases).

I'm motivated by @stofte's sentiment earlier:

PS is probably the most confusing language I've come across in my years. It seems to assemble a hodge-podge of ideas in order to be everything for everyone

PowerShell contains a lot of concepts that Bash doesn't, and for && and || to be worthy first-class language-level operators, they need to find holistic application in the language. Since commands can be native or non-native and expressions can have side effects, it still makes sense to use control operators with all of them (to some extent).

To that end, I think that allowing && and || only in pipelines or only with native commands is not the way to go.

Instead I think && and || should be usable wherever ; is usable, but where <x>;<y> evaluates <x> and then <y>, <x> && <y> evaluates <x> and if $? is true evaluates <y>, while <x> || <y> evaluates <x> and if $? is false evaluates <y>. && and || should be success-conditional sequencing operators.

That leaves us with the issue of how the $? value is set. But I feel like it makes sense to implement && and || with respect to $? and then make sure that $? has a straightforward meaning and contract as a separate issue.

@BrucePay

This comment has been minimized.

Copy link
Collaborator

commented Apr 23, 2018

@rjmholt Lexically, && and || are statement separators and therefore should work with any simple statements including expressions.

They should operate off of $? and there is no need to check $LASTEXITCODE since this is taken care of automatically (i.e. if a native command exits with a non-zero exit code, then $? will be false.). For example:

PS[1] (140) > true.exe ; $?
True
PS[1] (141) > false.exe ; $?
False

And expressions do set $?:

PS[1] (135) > 1/1 ; $?
1
True
PS[1] (136) > 1/0 ; $?
Attempted to divide by zero.
At line:1 char:1
+ 1/0 ; $?
+ ~~~
    + CategoryInfo          : NotSpecified: (:) [], RuntimeException
    + FullyQualifiedErrorId : RuntimeException

False

so they will work fine with && and ||.

One difference between ; and && is that the flow control operators should throw an IncompleteParse exception if the RHS is missing so we can handle the interactive experience. And the code will have to be formatted multiline like:

command1 &&
    command2 &&
        command3 

as is the case with pipelines (which is prompting much discussion in #3020)

We might also spend a little bit of time thinking about exceptions to see if they should be involved in any way...

@rjmholt

This comment has been minimized.

Copy link
Member

commented Apr 23, 2018

@BrucePay ok, that's what I was imagining should be the case.

Can you expand on the exceptions part?

I'm personally not sure whether it's better to have:

throw "Bad" || CleanupThings

continue to CleanupThings (especially since throw sets $? = $false), or if we consider exceptions to not be caught by control operators.

The problem I see with the first case is that if || continues after an exception, we either rethrow the exception at the end of the statement sequence (which is probably not helpful at that stage), or we absorb the exception, which means statement control implicitly catches all exceptions and users won't get warned about possibly bad failures.

But in the second case, || works fine unless you throw an exception, at which point it no longer "executes the RHS after the LHS fails". So any sequence of statements that could fail or throw an exception would have to use || inside a try/catch block.

@mklement0

This comment has been minimized.

Copy link
Contributor Author

commented Apr 24, 2018

Good point about $? already reflecting $LASTEXITCODE -eq 0 immediately after calling a "native" command (I wish we'd stop calling them that, given that there's nothing native about, say, calling node - something like external programs seems more apt).

And expressions do set $?:

Indeed they do, and that they do so is unhelpful, as I've tried to argue.

Given a statement such as:

(Get-Content nosuchfile) -notmatch '^\s*#' || ...

Don't you think users are more likely to expect the || to act on Get-Content's success rather than the wrapping expression's "success", which has little meaning other than when involving a .NET exception?

@mklement0

This comment has been minimized.

Copy link
Contributor Author

commented Apr 24, 2018

@rjmholt:

This comment by @BrucePay tells me that Throw terminates the current thread of execution, which in your example would not give the the RHS of || a chance to execute.

Interactively, you can verify this as follows, using ; instead of the (not-yet-extant) ||:

PS> Throw "bad"; 'after'

In the example above, 'after' never gets to execute, because the entire command line is terminated (which can be conceived of as an ad-hoc script).

We still don't have an official term for the kind of error Throw generates (see #3798 (comment)), but for now I'm calling them fatal errors.

My vote is for not changing this behavior, i.e., for && and || only to act on non-fatal errors (which by default are non-terminating errors and statement-terminating errors).

@rjmholt

This comment has been minimized.

Copy link
Member

commented Apr 24, 2018

I find that behaviour very interesting, given this comparison:

> 1/0; "Hello"
Attempted to divide by zero.
At line:1 char:1
+ 1/0; "Hello"
+ ~~~
+ CategoryInfo          : NotSpecified: (:) [], RuntimeException
+ FullyQualifiedErrorId : RuntimeException

Hello
> throw "Bad"; "Hello"
Bad
At line:1 char:1
+ throw "Bad"; "Hello"
+ ~~~~~~~~~~~
+ CategoryInfo          : OperationStopped: (Bad:String) [], RuntimeException
+ FullyQualifiedErrorId : Bad
> try { 1/0; "Hello" } catch { "Exception" }
Exception
> try { throw "Bad"; "Hello" } catch { "Exception" }
Exception

But yes, I agree.

@mklement0

This comment has been minimized.

Copy link
Contributor Author

commented Apr 25, 2018

@rjmholt:

Yes:

  • 1 / 0 is a statement-terminating error, which is why "Hello", as the next statement, still executes.

  • throw "Bad" is a script-terminating error (fatal error), which means that the current thread of execution is terminated by default, so "Hello" never gets to execute.

On the other hand, try / catch (and also trap) can trap either kind of error (but not non-terminating ones such as Get-Item /NoSuch).

I've tried to document the bewildering complexity and inconsistency of PowerShell's current error handling in this doc issue.


In a technology-debt-free world, 1 / 0 would ideally be a script-terminating error as well, but changing this behavior now would be a massively breaking change.

By contrast, what we can do - given that $? is currently virtually useless with non-statement-error-throwing expressions - is:

  • Nothing needs to change with respect to statement-terminating errors, be they generated by a command or by an exception produced by an operator or direct .NET method call: The statement as a whole is terminated and $? is set to $False.

  • Otherwise - if there are no errors or only non-terminating errors:

    • Make expressions not set $?,

    • ... which then allows the overall expression to reflect an embedded command's success .

In other words: unless a statement-terminating error occurs, an expression by itself wouldn't even touch $?.


With $? relegated to the world of commands, the question then becomes how to handle multiple commands embedded in an expression:

Assuming we don't want to go as far as to provide the success status of all commands involved (analogous to Bash's automatic $PIPESTATUS array variable):

The precedent set by the pipeline is to have $? reflect $False if any command involved generated a non-terminating error.
In other words: unlike in Bash, it is not the last command's status that is reflected. E.g.:

Get-Item /, /NoSuch | % { 'hi' }; $?
hi
...       # error output
False     # A reflection of Get-Item's non-terminating error.

Therefore I suggest exhibiting analogous behavior in the case of multiple commands being embedded in an expression.

@felixfbecker

This comment has been minimized.

Copy link
Contributor

commented Apr 26, 2018

This proposal seems to resolve mostly around native commands, where failure is indicated by a non-zero exit code vs cmdlets, where failure is indicated through an exception. As some have pointed out, I believe there is bad potential here for inconsistency. If I run ls || "hi" it shouldn't work any different than Get-ChildItem || "hi", otherwise external programs become less "integrated" into PowerShell, not more, and I believe seemless integration of native commands should be an important goal in PS.

I would like to understand the actual use case of this proposal a bit more as it seems related to #3415 (see #3996 (comment)). I.e. in what scenerio is this operator needed and what problem does it actually solve.

As seen in the OP it is really just a shorter one-line syntax for something that can already be done with an if statement. As such it seems like the primary use case for this is in the REPL, because I believe in scripts it is more readable if you write out the if (or try/catch with linked proposal) and not put multiple commands in one line. But in the REPL, I wonder why you wouldn't just run the commands manually and only run the second if the first failed.

I think the more readable alternative to chaining native commands with && is having the proposed set -e equivalent, which would just allow $CommandErrorActionPreference='Stop' to turn any non-zero exit code into an exception, with the possibility to catch it (the equivalent to ||) with a normal try/catch (like it works for cmdlets, so it's consistent). Concrete example of the OP:

# If command A succeeds, execute command B too.
$CommandErrorActionPreference = 'Stop'
csc /nologo file.cs
git commit -m 'ok'

# if command A fails, execute command B.
try {
  csc /nologo file.cs
} catch {
  git commit -m 'failed'
}

What do the proponents of this proposal think about this? In a world where we had this capability, would you still want && and || as much? If yes, for what reason?

@saschanaz

This comment has been minimized.

Copy link

commented Apr 26, 2018

try-catch would work, but what about multiple commands more than 2? npm run fetch && npm run build && npm run baseline-accept becomes try { npm run fetch } catch { try { npm run build } catch { npm run baseline-accept } } and this is much more verbose.

Edit: Or right, I confused && and || 😅

@felixfbecker

This comment has been minimized.

Copy link
Contributor

commented Apr 26, 2018

@saschanaz I think you are confusing && and ||.
npm run fetch && npm run build && npm run baseline-accept would become

$CommandErrorActionPreference='Stop'
npm run fetch
npm run build
npm run baseline-accept

or

$CommandErrorActionPreference='Stop'; npm run fetch; npm run build; npm run baseline-accept

This line in bash does the same thing:

set -e; npm run fetch; npm run build; npm run baseline-accept
@rjmholt

This comment has been minimized.

Copy link
Member

commented Apr 26, 2018

Personally, while I try to resist the "add the kitchen sink" approach to PowerShell, this is a feature common to most other shells. It's handy, and as a user of other shells as well, in my muscle memory.

PowerShell tries to differentiate itself on its interactive experience, and having to type out a try/catch block doesn't feel as one-liner interactive as just chaining things with an operator. And with PSCore being cross-platform, there's more native utility interaction experience to handle than ever, and we want to minimise friction for new users who want to do things like apt update && apt upgrade.

@saschanaz also makes a good point that try/catch is much harder to compose than the left-to-right operator execution.

I share exactly your concern about inconsistency between native utilities and PowerShell commands being exacerbated, but I think that's why we need to discuss this feature carefully. If we make the rule for && and || simple and base it on existing concepts (like that && proceeds when $? is true and || proceeds when $? is false -- and neither catch a fatal/pipeline-terminating error), then all we have to do is make sure those existing concepts work and make sense (i.e. make sense of PowerShell's error handling -- no small task, but easier than having to do that while also explaining complicated workaround rules for these operators).

In terms of set -e, I think having an explicit, composable operator is still desired. Even though you can use set -e in an interactive context, it relies on an implicit state in execution. So it's arguably harder to read (especially for the uninitiated) and also harder to analyse. The other problem is that it doesn't provide || functionality. I think set -e is also useful, but in my mind it's more designed to indicate fast-fail behaviour in larger scripts, rather than for chaining one-liners.

@felixfbecker

This comment has been minimized.

Copy link
Contributor

commented Apr 26, 2018

If we make the rule for && and || simple and base it on existing concepts (like that && proceeds when $? is true and || proceeds when $? is false -- and neither catch a fatal/pipeline-terminating error), then all we have to do is make sure those existing concepts work and make sense (i.e. make sense of PowerShell's error handling -- no small task, but easier than having to do that while also explaining complicated workaround rules for these operators).

Okay, so just to be clear, || and && would only be effective for cmdlets if $ErrorActionPreference -ne 'Stop' and for external commands if $CommandErrorActionPreference -ne 'Stop'. Otherwise there would be an exception just like if you didn't use those operators.
I think this makes sense because in an REPL context $ErrorActionPreference is usually Continue, whereas in a script where you might want to set it to Stop you also would want the readability of having one command per line, so the operators are not as appropiate. 👍

@felixfbecker

This comment has been minimized.

Copy link
Contributor

commented Apr 26, 2018

So I think the coupling between the two proposal is solved. For me the only question left is whether it should be && and || or -and/-or (or maybe new operators), since PowerShell usually breaks legacy conventions in favor of readability. But I guess anything starting with - doesn't work well with native commands and output redirection already uses special characters inspired by other shells for its operators.

@rjmholt

This comment has been minimized.

Copy link
Member

commented Apr 26, 2018

@felixfbecker You make a very good point about $[Command]ErrorActionPreference = 'Stop' but I think your proposed behaviour is probably the right one. Others may want to add their insights there. But it makes sense to me that setting those variables means throwing a fatal error, which means there's no continuing over ||.

Because of the parser's "modes", an operator like -and won't work after a command due to ambiguity (DoSomething -or CleanUp makes -or look like a parameter passing CleanUp as a bareword string). It's theoretically possible to implement, but it would break a number of things, is less efficient and would be a lot of work, so is pretty squarely ruled out. The other thing is that -and and -or already have functionality, so changing that would be a breaking change, which again is ruled out by the PowerShell breaking change contract.

My inclination is that && and || have good recognition and have been on the list of things to add in that verbatim form for some time. But there's no reason to rule out alternatives. They just can't start with -.

@mklement0

This comment has been minimized.

Copy link
Contributor Author

commented Apr 27, 2018

This proposal seems to resolve mostly around native commands, where failure is indicated by a non-zero exit code

No, the proposal is to apply the operators equally to PowerShell commands and external programs, based on $? (though the operators are patterned after POSIX-like shells).


vs cmdlets, where failure is indicated through an exception

While .NET exceptions underlie PowerShell's error handling, it's not useful to discuss it in those terms - they are an implementation detail.

In fact, cmdlets (as opposed to advanced functions) never create fatal errors, they only create two kinds of errors:

  • non-terminating: the statement that the error-reporting cmdlet is part of keeps executing, if there are more input objects to process.

  • statement-terminating: the statement the error-reporting cmdlet is part of is terminated, but execution continues with the next statement.

Script-terminating errors (fatal-by-default errors) can only arise in two situations:

  • $ErrorActionPreference = 'Stop'

  • -ErrorAction Stop, but, confusingly, only when encountering non-terminating errors.

In short: by default, the typical case is for execution to continue, either intra-statement (non-terminating error), or with the next statement (statement-terminating).


I think the more readable alternative to chaining native commands with && is having the proposed set -e equivalent

set -e and && / || are separate features, each useful in their own right, and problematic in combination (see below)

In POSIX-like shells, users often forgo set -e and its complex rules in favor of explicit error handling with ||.


The other thing is that -and and -or already have functionality

Indeed, they do; furthermore, -and and -or belong to the world of expressions, based on return values / outputs - and that obviously shouldn't change.

&& and || belong to the world of commands - be they cmdlets or external programs - and they act on success vs. failure via $?, without interfering with outputs.

This distinction is somewhat tricky, given that && / || have the same semantics as PS's -and / -or in C-inspired "real" programming languages, but it's just one of the challenges that comes with PS's hybrid nature, and we just need to make sure that the difference is well-documented.


$CommandErrorActionPreference='Stop'

The word Command in your proposed variable name is problematic, because command is PowerShell's umbrella term for anything invoked by name / filesystem path - whether PowerShell-internal or external.

Thus, $ExternalErrorActionPreference makes more sense to me, to highlight that the preference refers to external programs ("native" is a confusing term, and I suggest we avoid it).
I'll use this name in the remainder of this comment.


I believe in scripts it is more readable if you write out the if (or try/catch with linked proposal)

To me, && and || are concise and expressive constructs that beat the "noise" of if statements and try / catch handlers - on the command line as well as in scripts.

Okay, so just to be clear, || and && would only be effective for cmdlets if $ErrorActionPreference -ne 'Stop' and for external commands if $CommandErrorActionPreference -ne 'Stop'. Otherwise there would be an exception just like if you didn't use those operators.

  • && and || should operate on $? only - the combination with $ExternalErrorActionPreference needs to be discussed separately (see below).

  • as stated, cmdlet errors continue execution by default - intra-statement with non-terminating errors, next-statement with statement-terminating errors.


That leaves the question: how should && and || act in the presence of $ExternalErrorActionPreference = 'Stop'?

As I've argued at #3996 (comment), it's worth borrowing one exception from the set -e exceptions in POSIX-like shells:

Any command (statement) except the last in a && / || chain should not trigger a fatal error while $ExternalErrorActionPreference = 'Stop' is in effect, given that the use of && and || clearly signals an anticipation of potential failure.

@rjmholt

This comment has been minimized.

Copy link
Member

commented Apr 27, 2018

@mklement0 I think we are almost entirely agreeing with you here. I definitely appreciate your ability to dissect a problem too. I'm not sure I share your fully taxonomised conceptual model of PowerShell's semantics, but I think we are generally on the same wavelength. (The concern about being native-command-only I think was because that was floated earlier in the thread).

The only part I'm not sure about is the $ExternalErrorActionPreference = 'Stop' behaviour. On one hand, you make a good point that bash does it and our inspiration for these operators is bash. But on the other hand, it also makes sense that $ExternalErrorActionPreference = 'Stop' means if any external command has an error, we stop. Having to explain to users that "well, actually, if you use || we keep going even though you specified Stop" is one of those things I don't look forward to, but it might be a matter of naming the variable. As often, the UNIX world gets away with it because set -e doesn't promise much in the name.

On the name of the variable:

Thus, $ExternalErrorActionPreference makes more sense to me, to highlight that the preference refers to external programs ("native" is a confusing term, and I suggest we avoid it).
I'll use this name in the remainder of this comment.

For better or worse, PowerShell refers to external utilities as "native" everywhere I've seen. I think the horse has bolted on that vocabulary and trying to change it will just result in two competing terms that confuse more people.

@mklement0

This comment has been minimized.

Copy link
Contributor Author

commented Apr 27, 2018

@rjmholt:

Thanks; I'm glad we're mostly in agreement.

I'm not sure I share your fully taxonomised conceptual model of PowerShell's semantics

Please tell me where you disagree; it'll help future conversations.


if you use || we keep going even though you specified Stop" is one of those things I don't look forward to

I do get the appeal of avoiding exceptions in general, but this particular one makes sense to me.

To recap: The sole reason to use || is to deal with anticipated failures.

To borrow my example from #3996 (comment):

# Get files from /dir1, if available, otherwise from /dir2
$files = ls /dir1 2>$null || ls /dir2

The use of || here signals an anticipation of a failure of the 1st command.

You can think of || as an ad hoc override of any $ExternalErrorActionPreference = 'Stop', similar to overriding $ErrorActionPreference = 'Stop' with, say, -ErrorAction Ignore.

However, $ExternalErrorActionPreference = 'Stop' would kick in if the 2nd ls command failed, since its failure is not anticipated.

By contrast, with && the case is less clear-cut: in POSIX-like shells, it is often used to prevent execution of subsequent commands if earlier ones failed.

So if the intent is simply to prevent follow-on failures, an exception is perhaps not needed, and $ExternalErrorActionPreference = 'Stop' could treat a failure as fatal even if it is the first && operand - but that would need to be overruled if a || is present later.

In POSIX-like shells && is granted the same exception as ||, and perhaps modifying that exception to only apply to || is too confusing (and perhaps even impractical to implement - I can't speak to that), both in itself and for users coming from POSIX-like shells.

I'm not sure what the right answer is.


If we make no exception:

Effectively, we're then forcing users to choose between use of $ExternalErrorActionPreference = 'Stop' and &&/|| use, because having to locally try / catch potential failures negates the concision and expressiveness of &&/||:

# Need try / catch to guard against $ExternalErrorActionPreference = 'Stop'
$files = try { ls /dir1 2>$null } catch {} || ls /dir2

This could be somewhat mitigated by a yet-to-be-introduced new syntax that provides the analog of
-ErrorAction to external-program invocations.
It is tricky to come up with the right syntax, however, since it would either have to be a meta-parameter similar to --% or perhaps a modified version of the & operator, and keeping it concise risks its being arcane:

$files = &(i) ls /dir1 2>$null || ls /dir2  # &(i) is the hypothetical analog to -ErrorAction Ignore

As for the term "native":

At the end of the day, what matters is that people know what a term stands for.
And human language in general is full of misnomers that no one thinks twice about, if they've been around long enough.

So I don't think this is the most important issue, but please bear with me:

For better or worse, PowerShell refers to external utilities as "native" everywhere I've seen.

  • Naming should be as precise as possible when it comes to code elements.

  • "Native" is an established term, but it's not really used in the official documentation, as far as I can tell, so the documentation could lead the way here (e.g., about_Parsing currently refers to "executable programs" in the context of discussing --%).

  • In the specific case at hand, the term "external program" (perhaps "utility" is too "Unix-y") or more broadly "external" to refer to functionality outside of PowerShell avoids the ambiguity of "native" while still being concise and "self-documenting", so even those used to the term "native" should be able to infer its meaning.
    (And perhaps some day "external" will become the predominantly used term - a guy can dream, can't he?)

@mikemaccana

This comment has been minimized.

Copy link

commented Apr 6, 2019

@SteveL-MSFT Since there won't be a 6.3 - consider it for 7?

@SteveL-MSFT

This comment has been minimized.

Copy link
Member

commented Apr 7, 2019

@mikemaccana I've renamed the 6.3-Consider milestone to 7.0-Consider

@vladimiry

This comment has been minimized.

Copy link

commented Apr 27, 2019

@mklement0

This comment has been minimized.

Copy link
Contributor Author

commented Apr 27, 2019

@vladimiry Thanks, but this issue is about a concise, success-conditional idiom for sequencing commands; by contrast, your link is (primarily) about unconditional parallel execution via PowerShell workflows, which aren't supported in PowerShell Core.

@rjmholt

This comment has been minimized.

Copy link
Member

commented Jun 12, 2019

Please see PowerShell/PowerShell-RFC#192

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.