sort: Subtle bug in (Float64Array) Less #2092
Description
What steps will reproduce the problem?
1. Compile:
package main
import "math"
import "sort"
import "fmt"
func main() {
s:=[]float64{1,2,3,math.NaN(),1,2,3}
sort.SortFloat64s(s)
fmt.Println(s)
}
2. Run
What is the expected output?
[1 1 2 2 3 3 NaN]
or
[NaN 1 1 2 2 3 3]
or
something sorted with NaN at an arbitrary position, since it is not ordered anyway
(impossible with current sort interface, imho)
What do you see instead?
[1 2 3 NaN 1 2 3]
swapping numbers around in the initial vector produces all sorts of funny results.
First reproduced it with webinterface on golang.org should be version and architecture
independent
Reason is how comparison operators works with NaN and float64, (Float64Array) Less
always returns false if NaN is involved.
Fixing the case above would be trivial, just make Less return !(p[i]>p[j]) instead of
p[i]<p[j]. Unfortunately this should lead to infinite loops if more than one NaN is
involved.
Will send correct fix later.