measure-object -maximum changes the type of its input

[jszabo98]

Here the maximum property is a double, while the input was an int64 or long. A decimal (128-bit) input would also result in the maximum property being a double.

Steps to reproduce

1000000000000000 | measure-object -Maximum

Expected behavior

Count             : 1
Average           :
Sum               :
Maximum           : 1000000000000000
Minimum           :
StandardDeviation :
Property          :

Actual behavior

Count             : 1
Average           :
Sum               :
Maximum           : 1E+15
Minimum           :
StandardDeviation :
Property          :

Environment data

Name                           Value
----                           -----
PSVersion                      7.0.0
PSEdition                      Core
GitCommitId                    7.0.0
OS                             Microsoft Windows 10.0.16299
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

[iSazonov]

It is a fact that we convert all input objects to Double. It is by-design and it is a compromise because in general we don't know object types in pipeline.

[mklement0]

@iSazonov, I understand that there's no trivial solution, but the current behavior is clearly both surprising and unhelpful:

PS> (10000000000000199 | Measure-Object -Sum).Sum
1.00000000000002E+16

Casting to [bigint] shows that precision was lost; that is, the summation doesn't work as intended, and you may not even notice:

PS> [bigint] (10000000000000199 | Measure-Object -Sum).Sum
10000000000000200

[iSazonov]

I understand that there's no trivial solution

I can't think of anything better than explicit type conversion of input object (property) like -AsType [type].

[vexx32]

Get-Random does some manual type figuring-out for numbers. It's plausible we could adopt a similar approach in Measure-Object, type the parameter as object and then verify the type is a supported one as it comes in.

[mklement0]

Agreed in principle, @vexx32, but note that it's not about parameters in this case, but about the numeric types encountered in the input.

I think we need something like the automatic type widening (promotion) found in PowerShell's number-literal parsing, where you start with the type of the first number type encountered in the input and widen as needed during processing:

For integer input types:

int -> long -> decimal -> double

Encountering a [double] (or the rarely used [float]) in the input would instantly switch to [double]. -Average and -StandardDeviation still have to output the final result as a [double].

As an aside: there's a troubling inconsistency between type-widening in number-literal parsing and calculations (in expressions that don't use casts):

# A number literal that is beyond [long]::MaxValue is promoted to [decimal]
# The number below is [long]::MaxValue + 1 (cast to [decimal] to see the precise value)
PS> (9223372036854775808).GetType().Name
Decimal

# Performing the equivalent as a *calculation* promotes to [double], 
# with loss of precision.
PS> ([long]::MaxValue + 1).GetType().Name
Double # !!

Thus, with implicit type conversions you may lose precision without realizing it; consider the difference between:

# OK - LHS forced to [decimal] preserves [decimal]
PS> [decimal] [long]::MaxValue +1
9223372036854775808

and:

# !! Implicit calculation coerces to [double] with precision loss, which the subsequent
# !! [decimal] cast cannot recover.
PS> [decimal] ([long]::MaxValue +1)
9223372036854780000 # !! Loss of precision

@vexx32, @SeeminglyScience, any thoughts?

Obviously, changing fundamental behavior like that would be extremely fraught.

[vexx32]

My initial inclination is to avoid widening to double unless we get input that is either float or double and instead widen from int64 to BigInteger as that'll get us more directly representative results. For decimal inputs, widening to double does make more sense, despite the loss of precision.

A property like Count on Measure-Object, for example, could definitely stand to be BigInteger on the high end since a double-type Count isn't especially meaningful; all it gives you is a rough order of magnitude if you have enough objects. Other properties could, too, but it would be more dependent on the format the data is input in.

As for the inconsistencies between parsing and calculations -- I tend to think they should behave pretty similarly. Part of the trouble there is likely that .NET's default widening behaviours don't quite match PowerShell's (at least where the parsing is concerned). Whether that should / could be addressed in a reasonable manner, I'm not sure. It would probably take a fair bit of tinkering around with LanguagePrimitives and the conversion methods there, plus I'm sure a good deal of manual work in the arithmetic operator binders.

[bpayette]

Measure-Object returns a specific type: Microsoft.PowerShell.Commands.GenericMeasureInfo which defines the fields as follows:

  System.Nullable`1[System.Double] Average
  System.Nullable`1[System.Double] Sum
  System.Nullable`1[System.Double] Maximum
  System.Nullable`1[System.Double] Minimum

This is why everything comes out as double. "Fixing" this would require changing the type of the returned object and that would be a breaking change.

[mklement0]

Good point, @bpayette, so - unfortunately - it would have to be an opt-in enhancement via a parameter, say a -AsAutoNumber switch (just a first suggestion, struggling to come up with a good name), similar to what @iSazonov proposed, though the idea would be to not have to ask for a specific type and let the widening algorithm pick the appropriate type.

@vexx32, agreed re float and double (in line with what I proposed), and also re [bigint].

---

Re for the inconsistencies between parsing and calculations: I think PowerShell's number-literal parsing is fine (albeit different from C#'s); it's the calculation behavior of any-result-larger-than-[long]::MaxValue-becomes-a-[double] that I find troubling, but I'll take this topic elsewhere.

[vexx32]

I think PowerShell's number-literal parsing is fine

Glad to hear it, I'd hate to have to rewrite it again 😂

[iSazonov]

the idea would be to not have to ask for a specific type and let the widening algorithm pick the appropriate type

I do not think this is reliable. Also it complicates too the code. I think in real scenarios long type is enough.

[mklement0]

I do not think this is reliable.

In what way would it not be reliable?

I think in real scenarios long type is enough.
I think in real scenarios long type is enough.

See https://stackoverflow.com/a/60609025/45375, for example.

The inputs are [long]s, but the invariable conversion to [double] causes loss of precision for [long]s beyond 9007199254740991, which is particularly insidious in this case, given that you'd never expect -Maximum to modify the value:

PS> [bigint] (132273694413991065 | Measure-Object -Maximum).Maximum
132273694413991072  # !! Different number, to lossy conversion to [double]

(Strictly speaking, no widening is required in this case - only preservation of the input type).

Stack Overflow

How to prevent PowerShell's Measure-Object cmdlet from truncating your data?

I am tryng to compare the following data to obtain the largest number:

$UserDeets

name lastLogon

---

Frank Ti 132273694413991065
Frank Ti 1322797428841...

[iSazonov]

I do not think this is reliable.
In what way would it not be reliable?

:-) #12103 (comment) Do you want to ask @vexx32 ? :-)
We could switch to BigInt but we lost performance. The same is for Dynamic type. I do not think that we'd want this.

I don't understand your stackoverflow example. My thought is to keep double type by default and add new parameter to switch to long type for inputs and results.

[vexx32]

I wasn't aware Measure-Object was a performance-critical scenario. 😉

Personally I think it makes the most sense to adopt the widest / least precise input type for the overall output. Most likely, this would mean the actual class property would have to be Object or at least ValueType (can you make properties ValueType? I feel like I remember that's not really allowed or something).