Select-Object -unique is inconsistently case sensitive

[bobfrankly]

Steps to reproduce

"derp","Derp" | Select-Object -unique

Expected behavior

Single Return of derp

Actual behavior

[PSCustomObject]@{Name = "derp"},[PSCustomObject]@{Name = "Derp"} | Select-Object -unique
# Returns one 'derp'

"derp","Derp" | Select-Object -unique
# Returns two, "Derp", and "derp"

"derp","Derp" | Sort-Object -unique
# Returns One 'derp'

# Also: This is also the way it behaves in 5.1, if it helps

Environment data

Name                           Value
----                           -----
PSVersion                      7.0.0
PSEdition                      Core
GitCommitId                    7.0.0
OS                             Microsoft Windows 10.0.18362
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

[marshallwp]

Odd, I'd assume case sensitivity would only be an issue on non-Windows implementations per the Case-Sensitivity In Powershell known issue.

How curious.

[marshallwp]

Well, I'm stumped. I looked through the portion of Select-Object code that gets invoked when only the -Unique flag is supplied. I went down every path I could find from the different comparison methods to cultural differences, to type conversion, but everything appears consistently implemented.

Aside from the questionable choice to hard-code case-sensitivity into Select-Object -Unique when declaring the ObjectCommandComparer, I couldn't find much.

I mean, that would explain why "derp","Derp" | Select-Object -unique returns both values, but it doesn't explain why [PSCustomObject]@{Name = "derp"},[PSCustomObject]@{Name = "Derp"} | Select-Object -unique returns only a single value. As far as I can tell, they should both be case sensitive under the current code.

[BrandynThornton]

It looks like the issue reported is due to not specifying a Property for comparison. Without the Property the comparison is between the base objects of each PSCustomObject which are equal. The Sort-Object cmdlet has a separate switch parameter to set case sensitivity.

Updated example showing difference in behavior, note the two Name values in the first example are different words and not just different case. I would agree that it would be nice if this defaulted to compare all properties rather than none but that could be a breaking change.

[PSCustomObject]@{Name = "derp"},[PSCustomObject]@{Name = "lerp"} | Select-Object -unique
# Returns one 'derp'

[PSCustomObject]@{Name = "derp"},[PSCustomObject]@{Name = "Derp"} | Select-Object -unique -Property Name
# Returns two, 'derp' and 'Derp'

"derp","Derp" | Sort-Object -unique -CaseSensitive
# Returns two, 'derp' and 'Derp'

# Also: This is also the way it behaves in 5.1, if it helps

[marshallwp]

Good catch! Though that behavior is rather alarming as it is both clearly incorrect and an easy mistake for a programmer to make. The help for Select-Object explicitly states that "When you select properties, Select-Object returns new objects that have only the specified properties," so using the -Property parameter to get wholly unique objects without discarding properties is contraindicated.

If that change is judged to be too breaking, we need to update the documentation to make it clear that to select unique objects, the proper syntax is Select-Object -Unique -Property *. Probably add a warning message to the same effect as well.

[bobfrankly]

So I ran across this issue dealing with REST style API, which doesn't return json or xml objects, just an array of strings. It's similar to what you'd expect from Get-Content on a raw text file. If a property is required for Select-Object to behave as expected, what is one supposed to use to pull uniques out of an array of strings?

[vexx32]

@bobfrankly Sort-Object -Unique works, but (naturally) is a sorted list.

[bobfrankly]

@vexx32 I get that it works, it's listed in original bug report. But what's the design supposed to be? That Select-Object isn't for arrays of strings? Example 4 in the Select-Object appears to state that it's supposed to be used exactly that way. Am I misunderstanding?

[vexx32]

@bobfrankly I would be inclined to assume that, given PowerShell is typically by default case-insensitive, this case is more likely to be a bug than a design choice.

[dillardd]

Just ran into this today. Sort-Object was the workaround, but it is definitely still broken. Pwsh 7.1.3 on Window 10.

[Szeraax]

Wow, I had no idea that this sort of a bug was present in Select-Object.

In my case, I needed to preserve order and preserve casing from the original addition.

My solution uses a case insensitive hashset that will only add to the list IF the input is not present in the list. Good enough for my small lists. (Thanks SeeminglyScience):

$List = [Collections.Generic.HashSet[string]]::new([StringComparer]::OrdinalIgnoreCase)
"a","A","B","b" | Foreach-Object {if (-not $List.Contains($_)) {$null = $List.Add($_)}}
$List