New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Asynchronous behavior of implicit Format-Table use can have surprising side effects - console output may appear out of order #4594
Comments
The current behavior is |
Re as-designed: I assumed as much, but it's another gotcha that calls for Re Or are we talking about different things? |
& {
[pscustomobject] @{ one = 1 } # implicit Format-Table call - in PSv5+, waits 300 ms. before outputting
'hi' | Out-Host # explicit, synchronous, non-pipeline output
Read-Host -Prompt 'Press any key'
} | Format-Table |
It seems internally we call Format-Default in the case. |
Good point, but what I meant was applying [pscustomobject] @{ one = 1 } | Format-Table # outputs synchronously
'hi' | Out-Host # explicit, synchronous, non-pipeline output
Read-Host -Prompt 'Press any key' Now that I think about it, even that use of |
It is dup #1747 |
Indeed it is, thanks - due to its generic title, I didn't discover it. Given that this issue describes the problem more specifically and includes a command to reproduce it, can I suggest we instead close #1747 as a duplicate? Here's its summary: @Jaykul states:
@SteveL-MSFT states:
@lzybkr states:
@joeyaiello states:
|
Currently I don't know what way is better. |
- The current WinRM specs use the Write-Output cmdlet. The Write-Output cmdlet is designed for writing objects to the PowerShell pipeline, *not* for writing stdout to a host. PowerShell has a few different methods of writing more directly to stdout, the most popular of which is Write-Host. Write-Host is typically frowned upon by the PowerShell community, but because it is not appropriate for use where pipelines may be involved: http://www.jsnover.com/blog/2013/12/07/write-host-considered-harmful/ When the goal is to emit text to be consumed by other tooling that doesn't integrate with the PowerShell pipeline, Write-Host becomes an important tool, and likely the case the tests are interested in (independent ordered text-based writes to stdout). Such counterpoints are provided at: https://psitem.com/2015/01/16/is-write-host-really-harmful/ The other alternatives to calling Write-Host are Out-Host, [Console]::WriteLine and the lesser known $Host.UI.WriteLine - While Write-Output can generate output for both stdout *and* for the PowerShell pipeline, there is also a problem with using it if output ordering is important. In PowerShell 5, changes were introduced to its behavior, so that implicit table formatting receives a 300ms delay. This often results in out-of-order output when Write-Output is used, and may especially be the case when pipeline writes with Write-Output are mixed with direct writes to stdout with Write-Host PowerShell/PowerShell#4594 https://windowsserver.uservoice.com/forums/301869-powershell/suggestions/14925213-bug-console-output-appears-out-of-order
- The current WinRM specs use the Write-Output cmdlet. The Write-Output cmdlet is designed for writing objects to the PowerShell pipeline, *not* for writing stdout to a host. PowerShell has a few different methods of writing more directly to stdout, the most popular of which is Write-Host. Write-Host is typically frowned upon by the PowerShell community, but because it is not appropriate for use where pipelines may be involved: http://www.jsnover.com/blog/2013/12/07/write-host-considered-harmful/ When the goal is to emit text to be consumed by other tooling that doesn't integrate with the PowerShell pipeline, Write-Host becomes an important tool, and likely the case the tests are interested in (independent ordered text-based writes to stdout). Such counterpoints are provided at: https://psitem.com/2015/01/16/is-write-host-really-harmful/ The other alternatives to calling Write-Host are Out-Host, [Console]::WriteLine and the lesser known $Host.UI.WriteLine - While Write-Output can generate output for both stdout *and* for the PowerShell pipeline, there is also a problem with using it if output ordering is important. In PowerShell 5, changes were introduced to its behavior, so that implicit table formatting receives a 300ms delay. This often results in out-of-order output when Write-Output is used, and may especially be the case when pipeline writes with Write-Output are mixed with direct writes to stdout with Write-Host PowerShell/PowerShell#4594 https://windowsserver.uservoice.com/forums/301869-powershell/suggestions/14925213-bug-console-output-appears-out-of-order
- The current WinRM specs use the Write-Output cmdlet. The Write-Output cmdlet is designed for writing objects to the PowerShell pipeline, *not* for writing stdout to a host. PowerShell has a few different methods of writing more directly to stdout, the most popular of which is Write-Host. Write-Host is typically frowned upon by the PowerShell community, but because it is not appropriate for use where pipelines may be involved: http://www.jsnover.com/blog/2013/12/07/write-host-considered-harmful/ When the goal is to emit text to be consumed by other tooling that doesn't integrate with the PowerShell pipeline, Write-Host becomes an important tool, and likely the case the tests are interested in (independent ordered text-based writes to stdout). Such counterpoints are provided at: https://psitem.com/2015/01/16/is-write-host-really-harmful/ The other alternatives to calling Write-Host are Out-Host, [Console]::WriteLine and the lesser known $Host.UI.WriteLine - While Write-Output can generate output for both stdout *and* for the PowerShell pipeline, there is also a problem with using it if output ordering is important. In PowerShell 5, changes were introduced to its behavior, so that implicit table formatting receives a 300ms delay. This often results in out-of-order output when Write-Output is used, and may especially be the case when pipeline writes with Write-Output are mixed with direct writes to stdout with Write-Host PowerShell/PowerShell#4594 https://windowsserver.uservoice.com/forums/301869-powershell/suggestions/14925213-bug-console-output-appears-out-of-order - Modify 2 tests so that Write-Host emits a single string with multiple embeds, rather than calling Write-Output with multiple string objects
- The current WinRM specs use the Write-Output cmdlet. The Write-Output cmdlet is designed for writing objects to the PowerShell pipeline, *not* for writing stdout to a host. PowerShell has a few different methods of writing more directly to stdout, the most popular of which is Write-Host. Write-Host is typically frowned upon by the PowerShell community, but because it is not appropriate for use where pipelines may be involved: http://www.jsnover.com/blog/2013/12/07/write-host-considered-harmful/ When the goal is to emit text to be consumed by other tooling that doesn't integrate with the PowerShell pipeline, Write-Host becomes an important tool, and likely the case the tests are interested in (independent ordered text-based writes to stdout). Such counterpoints are provided at: https://psitem.com/2015/01/16/is-write-host-really-harmful/ The other alternatives to calling Write-Host are Out-Host, [Console]::WriteLine and the lesser known $Host.UI.WriteLine - While Write-Output can generate output for both stdout *and* for the PowerShell pipeline, there is also a problem with using it if output ordering is important. In PowerShell 5, changes were introduced to its behavior, so that implicit table formatting receives a 300ms delay. This often results in out-of-order output when Write-Output is used, and may especially be the case when pipeline writes with Write-Output are mixed with direct writes to stdout with Write-Host PowerShell/PowerShell#4594 https://windowsserver.uservoice.com/forums/301869-powershell/suggestions/14925213-bug-console-output-appears-out-of-order - Modify 2 tests so that Write-Host emits a single string with multiple embeds, rather than calling Write-Output with multiple string objects - Introduce additional test demonstrating ways of generating stdout in PowerShell - namely: Write-Output explicitly sent to Format-Table to avoid ordering issue Write-Host Out-Host $Host.UI.WriteLine [Console]::WriteLine Write-Information
I had the idea of a -NoWait parameter for Format-Table. You could set it in advance with $PSDefaultParameterValues, like setting -Encoding to Ascii for Out-File and '>'. Plus -NoWait's very existence would help document this behavior. By the way, in a script, implicitly run format-table could wait any amount of time for the second object before showing output. # no output until script finishes
foreach ($i in 1..60) {
if ($i -eq 1) { [pscustomobject]@{message = $i} }
sleep 1
} |
Really Format-Table doesn't do any delay, it only creates formatting objects. PowerShell/src/System.Management.Automation/FormatAndOutput/common/BaseOutputtingCommand.cs Line 158 in ca202da
and implemented here as a queue: PowerShell/src/System.Management.Automation/FormatAndOutput/common/OutputQueue.cs Lines 71 to 77 in ca202da
If we could send a message and catch it in the condition we could resolve the issue. |
The following test fails, as of PSVersion 7.0.2 -SPLIT ([CONSOLEKEYINFO], [VERSION] | % { NEW-OBJECT $_ } | OUT-STRING) | SHOULD -CONTAIN 'MAJOR' A partial workaround would be to put a ref object at the beginning of the pipeline, e.g.: & { [VERSION]
NEW-OBJECT CONSOLEKEYINFO
READ-HOST } This causes the default formatter to switch to FORMAT-LIST and also flushes the output. Why it is so remains a mystery. |
It is only related in the abstract sense that is also a surprising Your workaround isn't one, because none is required for the types in your example (because they have formatting data associated with them), and it is ineffective for types that do need a workaround: # NOT a workaround: Read-Host prompts first.
& {
[pscustomobject] @{ one = 1; two = 2; three = 3 }
READ-HOST
} |
Workaround: & {
[VERSION]
[pscustomobject] @{ one = 1; two = 2; three = 3 }
READ-HOST
} Works like a charm 😺 |
It's still not a workaround: It only happens to work, because If you use a second The only way to always avoid the asynchronous behavior is to pipe to |
@yecril71pl This issue exists only in v5+, but yours behave that way since v1. |
The block is necessary if you paste it into the console.
Geez, dude, who do you think I am? & {
[VERSION]
[pscustomobject] @{ one = 1; two = 2; three = 3 }
[pscustomobject] @{ one = 1; two = 2; three = 3 }
READ-HOST
}
As I said,
If I ever encounter a situation where the workaround does not work, I shall report back, with your permission. |
I meant: use a second # Does NOT work: Read-Host prompts first.
& {
[pscustomobject] @{ one = 1; two = 2; three = 3 }
[VERSION]
READ-HOST
} I don't think having to place an unrelated object whose type has associated formatting data, just so that it triggers synchronous output behavior (and obviously becomes part of the output), qualifies as a workaround. There currently is no good workaround: you're forced to choose between synchronous display (e.g., via On a meta note:
We are discussing technical issues here, not personalities. |
As I said,
I appreciate your disgust, however, it is sufficient for me and I do not know anything better.
It would have been blatantly incompetent of me if I had not checked that the workaround works with two objects in the pipeline. |
Nah just would have been a mistake, we all make 'em. It's also pretty much impossible to judge someone's competency from a handful of issues. Even harder to actually keep a mental map of exactly how competent every single community member is. |
Excellent point, @SeeminglyScience. Additionally, in this specific case, the instruction given by me that triggered @yecril71pl's response was misinterpreted: The instruction was poorly phrased (since corrected), but the paragraph that preceded it should have made it clear that the issue is not about how many objects there are in the pipeline, but whether or not the type of the FIRST one happens to have associated formatting data. And that distinction is indeed quite subtle, not something anyone could be expected to recognize readily. Therefore: We all make mistakes, and we all misinterpret others' comments on occasion. This only becomes problematic if one takes things personally. @yecril71pl , another problematic aspect is to use incendiary language such as "I appreciate your disgust"; it adds nothing to the discussion and serves merely to antagonize. |
Since this behavior happens only with I know |
@metablaster, the problem isn't specific to PS> [regex] ''; write-host hi # !! 'hi' prints first
hi
Options RightToLeft MatchTimeout
------- ----------- ------------
None False -00:00:00.0010000 As an aside: Use of |
I took a look at your link where you said However your mention of "format data" solved all of my problems and I'm still sticking with Thanks so much! foreach ($i in 1..3)
{
Write-Information -MessageData "Performing operation $i" -INFA "Continue"
[PSCustomObject]@{
TemplateKey1 = "A$i"
TemplateKey2 = "B$i"
TemplateKey3 = "C$i"
# Cutom type name for Format.ps1xml
PSTypeName = "TestType"
}
} Actual AND expected behavior
|
Glad to hear it. Yes, associating formatting data with any type avoids the problem. Re:
I should have made it clearer that this was an entirely incidental aspect of why I linked to that comment - I linked to it solely for the background information on |
A more insidious side effect of the asynchronous behavior is that CLI calls from outside PowerShell can result in data loss due to missing output - see this Stack Overflow question and |
This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you. |
1 similar comment
This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you. |
This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you. |
This issue has been marked as "No Activity" as there has been no activity for 6 months. It has been closed for housekeeping purposes. |
Glad to see all these bugs have been resolved. |
lol, reminds me of one reason for not resolving a bug from GNU mailing list labeled as "Nobody cares" |
A follow-up to #4552.
Reference https://windowsserver.uservoice.com/forums/301869-powershell/suggestions/14925213-bug-console-output-appears-out-of-order
Users have been stumbling over this for a while (see this 2-year-old connect.microsoft.com bug report and various SO questions).
@PetSerAl provides an in-depth explanation of the problem in this SO answer.
In short, he states that, starting in PSv5, an implicitly applied
Format-Table
call now waits 300 ms. before producing output in order to collect a reasonable number of objects to output so as to better determine suitable column widths.That means that subsequent commands get to execute before the
Format-Table
output kicks in, which results in surprising behavior if these commands produce non-pipeline display output (e.g., withWrite-Host
) or prompt for user input, for instance.Update: A related problem (irrespective of whether the asynchronous output is a problem) is that the implicit output formatting applied to the first command in a script locks in the implicit formatting for all remaining commands, which, if
Format-Table
is locked in, can lead to objects output by subsequent commands "disappearing", as originally detailed in #4552 (e.g.,[pscustomobject] @{ one = 1; two = 2}; [pscustomobject] @{ three = 3}
makes the 2nd custom object "disappear", because it doesn't share any properties with the 1st one) - see also: #12825Steps to reproduce
Expected behavior
PSv4- produces (note how the implicit
Format-Table
output comes after the prompt):Actual behavior
PSv5+ produces:
Environment data
The text was updated successfully, but these errors were encountered: