Enable formatting of different object types in same pipeline

[JohnLBevan]

When running some commands in sequence, if no Format- command is explicitly stated, output may not show. I believe this occurs when one command formats data as a table, and the subsequent command erroneously attempts to put its data into this same table.

Issue first observed by LukeZeimet at: https://stackoverflow.com/questions/45637800/powershell-if-condition/45638510?noredirect=1#comment78237014_45638510

Steps to reproduce

Running the following only shows the output of Test-Connection, not of Get-WmiObject, despite Get-WmiObject giving results / working when run alone:

$computername = 'myComputer'
Test-Connection $computername | select @{Name="Computername";Expression={$_.Address}}, 'Ipv4Address' 
Get-WmiObject win32_SystemEnclosure -computername $computername | select serialnumber

Test-Connection $computername | select @{Name="Computername";Expression={$_.Address}}, 'Ipv4Address' 
Get-WmiObject win32_SystemEnclosure -computername $computername | select @{Name="Ipv4Address";Expression={$_.serialnumber}}
 

Expected behavior

Output:

Computername  IPV4Address  
------------  -----------  
myComputer 123.456.789.321
myComputer 123.456.789.321
myComputer 123.456.789.321
myComputer 123.456.789.321
serialnumber
------------
None
Computername  IPV4Address  
------------  -----------  
myComputer 123.456.789.321
myComputer 123.456.789.321
myComputer 123.456.789.321
myComputer 123.456.789.321
IPV4Address  
------------
None

Actual behavior

Computername  IPV4Address  
------------  -----------  
myComputer 123.456.789.321
myComputer 123.456.789.321
myComputer 123.456.789.321
myComputer 123.456.789.321

myComputer 123.456.789.321
myComputer 123.456.789.321
myComputer 123.456.789.321
myComputer 123.456.789.321
                      None

Environment data

> $PSVersionTable
Name                           Value                                                                                                                                                          
----                           -----                                                                                                                                                          
PSVersion                      5.1.14409.1012                                                                                                                                                 
PSEdition                      Desktop                                                                                                                                                        
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}                                                                                                                                        
BuildVersion                   10.0.14409.1012                                                                                                                                                
CLRVersion                     4.0.30319.42000                                                                                                                                                
WSManStackVersion              3.0                                                                                                                                                            
PSRemotingProtocolVersion      2.3                                                                                                                                                            
SerializationVersion           1.1.0.1   

[SteveL-MSFT]

Yes, the 300 ms wait was added to improve formatting of column widths, however, this issues seems to be an unintended side-effect.

[mklement0]

@SteveL-MSFT:

I'm glad to hear it.

Unfortunately I just realized that the behavior described in this issue is as designed and not related to the 300 ms. wait - see below (I've also removed my original explanation above).

So, unless I'm wrong again, I suggest:

• closing this issue as as-designed; @JohnLBevan points out that the behavior is gotcha, however, so a documentation-needed label is warranted too.

• creating a new issue that focuses on the 300 ms. wait - see Asynchronous behavior of implicit Format-Table use can have surprising side effects - console output may appear out of order #4594

---

^{(Adapted from https://stackoverflow.com/a/45705068/45375.)}

All output produced by a given script - even across separate commands - is sent to the same pipeline.
(You can think of a command line submitted interactively as an implicit script.)

While you can send any mix of data types to a pipeline, its default display formatting is optimized for objects of the same type, as that is the more typical case.

^{The following is in part based on experimentation - do tell me if I got something wrong.}

In the absence of explicit formatting commands (Format-Table, Format-List, ...), PowerShell automatically chooses a suitable display format, based either on a given type's preconfigured formatting data (see Get-Help about_Format.ps1xml) or, in their absence, based on simple rules (4 or fewer properties? -> Format-Table; 5 or more? -> Format-List).

However, it is the first object sent to the pipeline that determines the display format for all objects:

• In case Format-Table is selected, that also means that the first object alone determines the set of table columns (properties), which can cause subsequent objects to "disappear", if they don't have the same properties.

  • The only data types unaffected by this appear to be the .NET base types, such as [int] and [string] - they print normally, irrespective of what display columns were selected by the first object.

• In case Format-List is selected, there is no problem, as each object's properties are listed individually.

Again, note that even "disappearing" objects are just a display problem: the objects are there, and sending them to another command for further processing works just fine.

---

Why using an explicit formatting command helps:

By explicitly piping to a Format-* cmdlet (e.g, [pscustomobject] @{ one = 1; two = 2 } | Format-Table), you're actually sending formatting objects (various [Microsoft.PowerShell.Commands.Internal.Format.*] types) to the pipeline, and PowerShell then effectively passes them through for display.

An alternative is to use a generic workaround: if you pipe to Out-Host instead (e.g., [pscustomobject] @{ one = 1; two = 2 } | Out-Host), in which case:

• you bypass the pipeline and print directly to the console (if you're running PowerShell in a regular console window),
• and the object's default formatting view is applied.

It is important to note that these workarounds are suitable only for display purposes, because the original objects are lost in the process:

• When you pipe to a Format-* cmdlet explicitly, you replace the original object with objects containing formatting instructions, which are useless for further processing.

• When you pipe to Out-Host, you send nothing to the script's pipeline.

[JohnLBevan]

Thanks @mklement0; agree that this behaviour makes sense.

However, it's a definite gotcha for the uninitiated / unwary.

Many people will want to avoid putting formatting commands in their functions as that then limits their reuse. Whilst they could avoid the issue by returning the different result types as different properties on a custom object (i.e. thus keeping them clearly distinct in the pipeline, whilst allowing them to be formatted as required down the line), this kind of thing will catch out a lot of people who've not hit it previously.

It would be helpful if the implicit Format-Table were able to check the pipeline values' .PSObject.TypeNames values and where they differ from the previous object, start a new table. That said, this suggestion would be:

• a breaking chage
• wouldn't resolve the issue in the many cases where all values were PSCustomObjects.

However, some solution along those lines would be helpful... Though I suspect it would be an enhancement rather than a bug...

[SteveL-MSFT]

I agree that this trips up people and open to suggestions on how to enhance this, however, it should probably be a RFC

[mklement0]

Just some quick food for thought on how to address this:

Building on @JohnLBevan's suggestions, here's a quick-and-dirty prototype for a potential new Format-Grouped cmdlet (a working title):

Function Format-Grouped { 
  $Input | Group-Object { 
    if ($_ -is [System.Management.Automation.PsCustomObject])
      { [string] $_.psobject.properties.Name }
   else 
      { $_.GetType().FullName } 
  } | 
    ForEach-Object { $_.Group | Out-String }
}

It groups the pipeline objects by type and outputs each group of objects of the same type with that type's default formatting; in the case of custom objects, they're grouped by shared array of property names.

A much simpler, streaming function that simply formats each input object individually, which notably means that table-formatted objects each get a table header, even if they occur in blocks.

Filter Format-Each { $_ | Out-String }

Example with regular types:

> & { Get-ChildItem -File /; Get-Process | Select-Object -First 2 } | Format-Grouped


    Directory: /


Mode                LastWriteTime         Length Name                                                                                                                              
----                -------------         ------ ----                                                                                                                              
--r---          7/30/16  10:00 PM            313 installer.failurerequests                                                                                                         
--r---          8/10/17   3:53 PM            146 lastlogout_hook.txt                                                                                                               



 NPM(K)    PM(M)      WS(M)     CPU(s)     Id  SI ProcessName                                                                                                                      
 ------    -----      -----     ------     --  -- -----------                                                                                                                      
      0     0.00       0.00       0.00      0 942                                                                                                                                  
      0     0.00       0.00       0.00      1   1                                                                                                                                  

Example with custom objects:

> & { [pscustomobject] @{ one = 1; two = 2 }; [pscustomobject] @{ four = 4} } | Format-Grouped

one two
--- ---
  1   2



four
----
   4

[JohnLBevan]

@SteveL-MSFT is it best that I raise a new issue on here to track this as an RFC, or could we retag this existing issue for that purpose?

[iSazonov]

@JohnLBevan We continue in #4594

[SteveL-MSFT]

Didn't see @iSazonov reply which sounds good, re-resolving

[mklement0]

@SteveL-MSFT:

Perhaps the solution to the asynchronous problem will implicitly give us the behavior that @JohnLBevan suggests here, but we should keep in mind that they are really two separate issues:

• surprising asynchronous output behavior

• the first implicit Format-Table use locking in the output format for all subsequent commands, even if they output different data

I've folded a description of the latter into #4594, at least for now.