Repository navigation
proposal: spec: disallow NaN keys in maps #20660
Description
Motivation
Go currently supports NaN keys in maps through some clever tricks.
The current spec is ambiguous on this behavior. It says:
[I]f the map contains an entry with key
x,a[x]is the map value with keyxand the type ofa[x]is the value […]
but "with key x" could plausibly mean "with key represented by x". (In practice, today it means "with key equal to x".)
That leads to surprising and subtle bugs (such as #14427) and adds complexity even in correct code (#20138 (comment)).
For example, the following two functions appear to be equivalent, but the latter version silently copies zeroes instead of the intended values for NaN keys:
func copy(m map[float64]int) map[float64]int {
c := make(map[float64]int, len(m))
for k, v := range m {
c[k] = v
}
}func copy(m map[float64]int) map[float64]int {
c := make(map[float64]int, len(m))
for k := range m {
c[k] = m[k]
}
}Proposal
We already reject incomparable values as map keys:
If the key type is an interface type, these comparison operators must be defined for the dynamic key values; failure will cause a run-time panic.
I propose that we should apply that standard to NaNs as well. Specifically, I propose that we add the following sentence to https://golang.org/ref/spec#Index_expressions:
Assigning to an element of a map with a key that is not equal to itself causes a run-time panic.
This proposal is along the lines of #19624, in that it produces panics for arithmetic operations which are otherwise likely to result in silent corruption in typical user code.
Workarounds
Users can easily avoid the proposed panics by calling checking whether the key is equal to itself before inserting a map entry.math.IsNaN(k)
Users who expect NaNs with the same representation to map to the same element can transform the keys using math.Float64bits or math.Float32bits:
if k == 0 { // -0.0 or +0.0
m[math.Float64bits(0)] = v
} else {
m[math.Float64bits(k)] = v
}Users who really do intend to store all of the entries for NaN keys separately can explicitly pair the map with a slice:
type nanElem struct {
k float64
v V
}
var (
m map[float64]V
nans []nanElem
)
if k != k {
nans = append(nans, nanElem{k, v})
} else {
m[k] = v
}
Activity
[-]proposal: Go 2: disallow NaN keys in maps[/-][+]proposal: disallow NaN keys in maps[/+]randall77 commentedon Jun 13, 2017
FWIW, there's a fair amount of code in the hashmap implementation that deals with the possibility of k!=k for keys k (search for reflexivekey in runtime/hashmap.go). It would be nice to strip that out.
That said, I don't think it is a big performance impediment. In the common case it's just a load/compare/predictable branch in a few places.
dsnet commentedon Jun 13, 2017
The strange behavior with maps only because == is not symmetric when applied on NaNs.
An alternative proposal would be to define == to be symmetric for NaNs only for maps. Technically there are multiple types of NaNs (signaling and quiet NaNs and a variety of valid bit sequences that represent NaNs). I similarly argue that they should all be equal to each other for maps in the same way that
-0.0 == +0.0.bcmills commentedon Jun 13, 2017
@randall77 It may be that this proposal would simplify the hashmap implementation, but I would prefer to focus on the correctness and complexity implications for user code.
bcmills commentedon Jun 13, 2017
@dsnet As @rsc notes in the blog post, allowing
m[k1]andm[k2]to be same element whenk1 != k2would also be surprising. It is straightforward to equate NaNs in user code if desired using themath.FloatNbitsapproach outlined above, and I would argue that the explicit code makes the change in equality less surprising.randall77 commentedon Jun 13, 2017
@bcmills : I agree, just wanted to let people know what the effects on the implementation would be (tl;dr a small net positive).
cznic commentedon Jun 14, 2017
All and every value of type {float32,float64} is comparable (guaranteed by the specs). The result of the comparison is irrelevant here.
Map keys must be comparable is a simple, non-complicated rule. I object to making it not only more complex, but suprisingly non-orthogonal.
rsc commentedon Jun 14, 2017
Not true. NaNs can be inside arrays or structs.
All our current restrictions are type-based, not value-based. Even the interface panic restriction is about the type inside the interface, not the value. I'd really like to avoid introducing and explaining new value-based restrictions.
bcmills commentedon Jun 14, 2017
Good point. But it's still easy to check, especially given the "not equal to itself" phrasing I've proposed:
bcmills commentedon Jun 14, 2017
The main issue I want to address is the surprising inability to access elements by key in an irreflexively-keyed map. If you don't want to introduce value-based restrictions, perhaps we could at least change map entries from "with key equal to
x" to "with key represented byx"?rsc commentedon Jun 14, 2017
It's certainly an idea worth evaluating when we get to language changes again (hence the Go2 label). Like all the Go2-labelled things, I'm not going to try to evaluate them now.
30 remaining items
deanveloper commentedon Jul 28, 2021
To add on to @DeedleFake, here is JavaScript Map:
bcmills commentedon Aug 6, 2021
This comment thread on CL 338609 is another interesting data point.
(It seems that the behavior of NaN map keys runs counter to the intuition of even @robpike!)
robpike commentedon Aug 6, 2021
I wouldn't say that. It was how NaNs worked (or not) in reflect.DeepEqual I didn't know.
docmerlin commentedon Sep 12, 2022
I don't like the idea of runtime panics on NaN being assigned to maps. Instead we should change the map behavior to allow NaN to be a valid key that can be queried by a NaN.
docmerlin commentedon Sep 12, 2022
IMO behind the scenes it should be doing something like what was proposed above for a workaround:
robpike commentedon Sep 13, 2022
I'd really like to avoid introducing NaNs, but I missed that boat. :(
atdiar commentedon Sep 29, 2022
It's interesting that if we consider a value to be/have a subtype (if a type is a set of values, a subtype can be represented as a subset... e.g. float64 with the added constraint that the set of values is restricted to NaN), we can reproduce the same logic used for the
comparableinterface.I don't know if this would be backward-compatible though (because I think it would require some additional checks, equivalent to implementing a negated constraint such as non-reflexive comparability constraint checks). Could think about it.
ianlancetaylor commentedon Jan 24, 2023
Just a note that as of the upcoming 1.21 release the language will have a new builtin,
clear, which removes a part of the argument for this change. (But there are other arguments for this change, and we could in principle still adopt it.)_bcmills commentedon Oct 13, 2023
#63312 is another issue of a package in the standard library neglecting to document an exception to its behavior for NaN keys.
The documentation for
maps.DeleteFuncsays “DeleteFunc deletes any key/value pairs from m for which del returns true”, but today it does not delete such pairs if the key for the pair is not equal to itself.candlerb commentedon Oct 13, 2023
Panicking on
m[k] = vif k != k seems sane and reasonable to me, and although it would be backwards-incompatible, it would be in such a way as to draw attention to itself anywhere that it mattered.The utility of being able to store multiple values in a hash with the "same" key, which you can't actually retrieve (without iterating over the whole map) or delete (without clearing the whole map) is very low. Anybody who currently does this, very likely does so by accident IMO.
Special-casing NaN when used as a hash key sounds appealing at first, but it would also have to apply to NaNs deeply nested in structs.
Nobody likes the idea of another line of code that could panic at runtime, but this is a similar case to interface comparisons, which are almost always allowed but can panic in certain cases (https://go.dev/play/p/eK9pF8naT8q) - or indeed a divide-by-zero error or a nil pointer dereference.
I suppose a counter-argument is it might encourage more use of panic/recover in a try/catch kind of way, like this, but again, I don't suppose NaN map keys are used much.
There is another possible behaviour:
m[k] = vsilently does nothing if k != k (i.e. nothing is inserted into the map, and no panic). It gives roughly the same semantics as today:m[k]does not return v;delete(m, k)does nothing. The only difference is that you won't find k/v when iterating over the map.