math: remove Compare, Compare32

[rsc]

In #56491 we added math.Compare. The original motivation was:

The slices.Sort doesn't support sorting slices of floating-point numbers containing Not-a-number (NaN) values, It's not very intuitive.

Once the slices package is merged into stdlib, the sort.{TYPE}s(e.g. sort.Ints, sort.Float64s) APIs may be semi-deprecated. But the slices package doesn't provide a simple way to sort slices of floats containing NaNs, it recommends using slices.SortFunc(x, func(a, b float64) bool {return a < b || (math.IsNaN(a) && !math.IsNaN(b))}), this function is very complex compared to using sort.Float64s.

This need is addressed not only by math.Compare but also by the recently added cmp.Compare.

We ended up with different semantics with respect to NaNs for cmp.Compare and math.Compare. For cmp.Compare we order NaNs all less than negative infinity, the same as sort.Float64s always has. For math.Compare we used the total order predicate from IEEE 754, which sorts NaNs with a negative sign less than negative infinity and NaNs with a positive sign greater than positive infinity. This is strange because NaNs don't even have signs technically, and there's no easy way to get at the sign.

Given that
(1) no one was asking for IEEE 754 NaN total ordering, and
(2) cmp.Compare now addresses the original motivation for math.Compare,
I believe we should remove math.Compare and math.Compare32.

We can always add them back in a future release if someone really does need the IEEE 754 semantics, but we don't have evidence for that today.

[smasher164]

Given that
(1) no one was asking for IEEE 754 NaN total ordering, and
(2) cmp.Compare now addresses the original motivation for math.Compare,
I believe we should remove math.Compare and math.Compare32.

The reasoning makes sense to me. I will point out that another reason for math.Compare was the fact that it solved #33440, namely sorting -0 before +0. Can cmp.Compare accommodate this?

[rsc]

cmp.Compare does not handle -0 before +0, nor can it do so easily. I am not sure about the need to "solve" #33440. Sort allows equal elements to appear in any order, and -0 == +0 by definition. What is the context where this matters?

[smasher164]

What is the context where this matters?

Just that we special-case one float value's comparison (NaN), whereas we allow 0 to follow the language-defined equality. This is more an argument about consistency, than necessity. However this isn't a show-stopper by any means, so I'm in support of this change.

[eliben]

@dsnet's request in #33440 was "Either this case be documented or we fix the Float64Slice.Less method."

cmp.Compare treats negative and positive FP zeros as equal, but it does document it:

// Compare returns
//
//	-1 if x is less than y,
//	 0 if x equals y,
//	+1 if x is greater than y.
//
// For floating-point types, a NaN is considered less than any non-NaN,
// a NaN is considered equal to a NaN, and -0.0 is equal to 0.0.

So it seems like this fulfills the original goal of #33440

[rsc]

Just that we special-case one float value's comparison (NaN), whereas we allow 0 to follow the language-defined equality. This is more an argument about consistency, than necessity.

It's perhaps worth pointing out that NaN must be handled as a special case, because the ordinary language comparisons stop being an ordering relationship at all when NaNs are involved, and if you pass an inconsistent ordering relationship to sort, you are very likely not to get sorted results at all. That is, because neither 1 < NaN nor NaN < 1 is true, sort will assume 1 == NaN. But by the same logic it will also assume 2 == NaN and so on. Because sort expects transitivity in the comparison, it may well conclude that 1 == 2 and end up sorting [1, 2, NaN] as [2, NaN, 1] or any other confusing result. Something must be done with NaNs to make sort work at all. (Unless we define that if there are any NaNs anywhere in the input then the entire result of sort is completely undefined.)

In contrast, -0 and +0 do not need to be handled as a special case: they are indistinguishable from each other in ordinary operations, and they cause no problems for sort at all.

A lesser concern, but it is also true that handling NaNs is very easy and can be done generically (x != x is a simple test for x being a NaN that still does the right thing even when x is not a float at all), while handling -0 vs +0 is not easy and cannot be done generically (the only way to tell them apart is to access the underlying float64 bit patterns, and that cannot be written generically).

[gopherbot]

Change https://go.dev/cl/499416 mentions this issue: Revert "math: add Compare and Compare32"