a vector is always == to itself, even when containing missing

[rfourquet]

This tries to fix the following inconsistency:

julia> v = [missing]; (v == v, v == [missing])
(missing, missing)

julia> d = Dict(0=>missing); (d == d, d == Dict(0=>missing))
(true, missing)

The docstring for == says

For collections, missing is returned if at least one of the operands contains a missing value and all
non-missing values are equal.

Which would suggest d == d should be missing. But another point of vue would suggest that v == v should be true, as well expressed by @oxinabox:

But we know the missings the two d in d == d are the same.
Because those are the same reference.
We are not comparing two different forms with unfilled boxes, we are comparing a form to itself.
No matter what the answer for the missing on the left is, it will certainly be the same for the missing on the right

This PR makes (v == v) == true. But this raises the quesion for some immutable types: what should (missing,) == (missing,) be? we don't know if both tuples have been created from the same source or not...

cc. @nalimilan
(apologies if this was already discussed to death and consciously decided upon the current behavior)

[JeffBezanson]

But this raises the quesion for some immutable types: what should (missing,) == (missing,) be? we don't know if both tuples have been created from the same source or not...

Precisely --- I'm inclined not to do this, largely because it only makes sense for mutable types. We should change Dict comparison instead. == should generally be the same on mutable and immutable versions of the same collections. Are there counterexamples to that?

[rapus95]

Tl;Dr: we definitively should change the mental model of missings.

Just for the point @oxinabox brought up, rephrasing it to leverage the precise point into focus: I first found v==v to return missing the better solution out of consistency reasons. But only until I remembered the actual meaning of missing (="There exists some value for it but that is unknown to you"). So, in case of identical objects (= indistinguishable in any sense) no matter what the value of the missing would be, it would be the same. semantically & syntactically identical objects should always be equal.

reflexivity of variables should hold even for missings as these represent values you don't know. But no matter which value, it will be equal to itself. Otherwise we lose an important property of ==.

That brings me to the conclusion that missing===missing should be false because they most probably represent different unknown values. 😅 While x=missing; x===x should be true as it will always represent the same unknown value. And then we should keep a===b => a==b. I'm not even sure if that could be accomplished in Julia right now but it would be the correct way w.r.t. the current meaning of missing.

Seems like the only formally correct way would be to suffer from the same problem that Strings and Symbols have and turn Missing into a mutable empty struct to be able to distinguish different instances. 😭

EDIT: Or change the mental model of missing to something else which properly reasons for that case (which definitively would the the better path to keep good performance I guess). The current model is flawed in some sense anyway as that model would suggest missing isa Int == missing.

CAUTION: The following thoughts might insult you or provoke some very bad feelings, so be warned and only keep reading on your own responsibility.
Would it be possible to make a Missing{D} where D holds the Domain aka Type? And current usage of missing would be Missing{Any}? By that we could have missing isa Int == eltype(typeof(missing))<:Int. Edit: Well, we'd need dynamic supertypes so that wouldn't work anyway right now but it would be cool. Edit2: Thinking further that actually proposes turning Union{T, Missing} into T. Which is definitively problematic since it ruins our non-null domains. So screw that idea of Missing{D}. Edit3: The mental model which might work better is to assume that the missings in Union{T, Missing} may only represent unknown values of domain/type T. Edit4: This is the same 🤦‍♂ as it suggests Union{T, Missing} <: T aswell.

[tkf]

And then we should keep a===b => a==b.

Even without missing, we already have

julia> NaN === NaN
true

julia> NaN == NaN
false

julia> isequal(NaN, NaN)
true

julia> x = [NaN]
1-element Array{Float64,1}:
 NaN

julia> x === x
true

julia> x == x
false

julia> isequal(x, x)
true

[rapus95]

make NaN === NaN be missing and watch the world burn 😄.
From the consistency point here both can be reasonable depending on whether you group all not a number types together. Also that's within the floating point domain and in some sense out of our typing reach. But for missing it's explicitely stated to mean a given unknown value. So those in general should be handled as if they indeed were different (even if they share the same bit layout). That's what I mean with meaning vs layout. If we had function types which capture argument and return types, then identity would have a similar problem. identity:A->A != identity:B->B <=> A!=B even though they share the same functional/binary layout/abstraction.

[KristofferC]

Seems like the only formally correct way would be to suffer from the same problem that Strings and Symbols have and turn Missing into a mutable empty struct to be able to distinguish different instances.

Not sure what this mean but note:

julia> "foo" === "foo"
true

julia> :bar === :bar
true

[KristofferC]

So, in case of identical objects (= indistinguishable in any sense) no matter what the value of the missing would be, it would be the same. semantically & syntactically identical objects should always be equal.

This seems like a non-useful way of thinking about the Julia missing because it doesn't and cannot represent that concept, since missing has no identity. Instead, thinking about missing as just an object that obeys the documented special cases (three valued logic etc) and then use that to build whatever higher level structures you want seems more productive. For example a package could define

mutable struct MissingWithIdentity
    missing::Missing
end

and define == on MissingWithIdentity etc.

[rapus95]

You need to overload each occurence of missing for your own type aswell then. But that's not the point. The point is that each properly defined type should be reflexive. (NaN being a float is a bad design from consistency points of view but can't be changed). So this is not only about missing but about default equality implementations in general. Default implementation should follow the lines of
==(a,b) = a===b || recurse(==, a, b). That weird combination is only necessary because there are types which aren't reflexive like Missing and those have the right to exist. But if your objects shall not allow identity as being equal (and thus be non-reflexive) they may rather define their own ==. Defaults in Julia usually intend to go the canonical way. And the canonical way is that == is reflexive.

[JeffBezanson]

This issue comes up for any sort of computing with domains. Intervals are the classic example. Giving missing for x==x is not best thought of as "wrong" but as a sound approximation. Our missings are indistinguishable, which is a highly practical design choice. Claiming that only distinguishable, identity-carrying missings are "correct" is just silly.

And yes, there is a point in julia where abstraction stops, and it's ===. No matter what behaviors are defined for X, === ignores all of them and just asks "is this thing X?" No matter how much confusion and uncertainty X is supposed to represent, you want to at least be able to tell when you have that thing.

[nalimilan]

I agree with @JeffBezanson that the correct fix is to change Dict equality so that == can return missing even if the two arguments are ===.

@rapus95 Please don't rehash all discussions that happened before missing was introduced. Have a look at issues regarding Nullable, for example #22682.

[rapus95]

Not sure what this mean but note:

julia> "foo" === "foo"
true

julia> :bar === :bar
true

I meant that:

julia> Base.isimmutable(:a)
false

julia> fieldnames(Symbol)
()

an empty mutable struct.

---

Our missings are indistinguishable, which is a highly practical design choice.

Definitively

Claiming that only distinguishable, identity-carrying missings are "correct" is just silly.

@JeffBezanson that sounds a bit harsh but to be fair, I claimed it to be the only correct way for the given mental model @oxinabox, me and probably some others are carrying. And for that model I'd still say the claim to be true.

And yes, there is a point in julia where abstraction stops, and it's ===. No matter what behaviors are defined for X, === ignores all of them and just asks "is this thing X?" No matter how much confusion and uncertainty X is supposed to represent, you want to at least be able to tell when you have that thing.

I definitively support that aswell because without it my custom equals a===b || a==b wouldn't work either.

So, let me rephrase it once again (that's why I added the EDIT in the first place):
The current mental model for missings (which seems to have some spread among users of the community, including myself) is flawed. (And taken the current way it works as truth, that model is wrong.) We should try to clarify it and find and propose an explicit mental model which doesn't have those flaws. And as you say yourself that there are points in Julia where abstraction stops & that model breaks, I guess you have a similar feeling about that. If anyone feels like the model I have (=missing is a placeholder for a concrete but currently unknown value) is the right one he may show me how to solve those expectations being violated. 👼

Tl;Dr: That mental model is flawed, not the implementation (which is good & very performant 👍); I don't want to change the implementation but am calling for a better model.

Regarding a===b => a==b my gut still says == being reflexive would be a nice attribute but I guess that's only relevant in the binary case, where it holds IIRC. With missing that's no more binary and as such requiring to follow the binary logic would be silly.

[JeffBezanson]

missing is a placeholder for a concrete but currently unknown value

Any documentation on this should not use the word "currently", since that implies it might change to being known at some point. Of course, a mutable container can do that, but missing itself does not have that notion attached to it. I would say something like that it's a token used by convention to represent an (identityless) unknown value e.g. in a statistical dataset --- unknown to the user, but not to the programming language. And I agree the docstring for missing is very lacking right now.

[andyferris]

Here's another one:

julia> isequal(missing, missing)
true

julia> Set([missing]) == Set([missing])
true

Personally, I actually think Set gets it right here. We rely on in and hash/isless and isequal in combination to possibly make this efficient. I see the sensible definition for sets is to satisfy:

(set1 == set2) == (set1 ⊆ set2 && set2 ⊆ set1) == (length(set1) == length(set2) && set1 ⊆ set2)

Since isequal(a, b) neither implies nor is implied by a == b, if we wanted to determine if two sets were == using == on the elements we would require a O(N^2) implementation. While savvy users could of course use isequal(set1, set2), the existence of a slow ==(set1, set2) would be the kind of massive footgun for users that I thought we were trying to eliminate with the creation of Julia (and the isequal version is the one to use when you care about the ordering of the elements inside the set - e.g. the hash of a container depends on this).

But a set is just a collection; I feel we should have been consistent and made arrays and dictionaries and tuples to use have a "reflexive" == by using isequal to compare the values too. In my mind it would simplify handling of containers to know ==(::Array, ::Array) returns a Bool, for example, or knowing that Set(vector) == Set(vector) gives the same result as vector == vector. Operating without these kinds of assumptions makes life challenging. From the example spoken of above, it could make sense to compare two identically filled "forms" as being the same via ==, even if on both forms some parts of the form are left missing.

(I also think there is something here to preserve along the lines of "the interface of arrays should be independent to the interface of the elements", but I don't have good enough language to express that idea. Basically, it would seem more composable if container implementers didn't have to know that missing existed, instead missing should be made to work inside generic containers - at that point we'd know we got the design "right").

Basically, I think == comparisons of containers using isequal on the elements is the only way to satisfy consistency, efficiency and compositionality considerations (though I realize this is rehashing territory discussed during Julia v0.7 development and thus may be unpopular).

[nalimilan]

Yes, things would certainly be cleaner if containers didn't have to have a special case for missing. OTOH it would be a be weird to have == for collections call isequal rather than == on their elements. People could be bitten by corner cases like [0.0] == [-0.0] or even worse [0] == [-0.0], which would be false with isequal and yet probably not what is commonly intended.

[andyferris]

People could be bitten by corner cases like [0.0] == [-0.0] or even worse [0] == [-0.0]

I think -0.0 is a great examplar.

What do we mean by "bitten" here?

For the longest time I assumed == implies isequal (and both are implied by ===). Whenever these are violated I feel surprised/bitten. If I want to create a unique set of numbers (which might have been calculated algorithmically), I am "bitten" if my Set contains both a positive and negative zero. I am bitten if x in collection but there is no any(==(x), collection). I am bitten when the tautology for x in collection; @assert x in collection; end is violated.

---

I am frequently not sure what the purpose of == and isequal are meant to be. I think == is meant to be the notion of equality in the appropriate domain, such as Number. OTOH isequal is clearly required for constructing sets and dictionaries (together with a consistent hash and/or isless), but beyond that we are allowed put things with different bit patterns (like the NaNs) into a single bucket, but what are the "rules" around that? Do we expect things that are isequal to have the same repr, for example?

Out of curiousity - if I really wanted "== implies isequal then do we need to do any more than making isequal(0.0, -0.0)? Are there other values of Base "scalar" types that don't have == implies isequal, or === does not imply ==? Note that "fixing" the scalar types automatically has the consequence of simplifying some of the container comparison behaviour, and I think isequal(0.0, -0.0) gives us a better Set/Dict (plus opens the door to using == inside Set comparisons without replying on a O(N^2) algorithm).

(Note: the other path to efficient recursive == on sets is to use ordered collections and ordered comparisons, and leave issetequal to use in/isequal internally).

[tkf]

Are there other values of Base "scalar" types that don't have == implies isequal

It's not in Base, but you have CompoundPeriod

julia> Hour(1) + Minute(10) == Minute(70)
true

julia> isequal(Hour(1) + Minute(10), Minute(70))
false

Reading #18485, IIUC, the motivation behind isequal(0.0, -0.0) == false is for using isless(-0.0, 0.0) for sorting. It makes sense, but it's slightly unfortunate that it makes Julia a bit of oddball when it comes to Dict(-0.0 => 1)[0.0]. Trying {-0.0: 1}[0.0] in Python and something similar in Ruby, Go, and Clojure all work. Poking around this a bit more I found that Rust does not implement hash for float for the similar reason. Ref: https://internals.rust-lang.org/t/f32-f64-should-implement-hash/5436

[tkf]

I just found this great write-up by @StefanKarpinski explaining isequal on float: https://discourse.julialang.org/t/possible-bug-in-unique-set/5371/9

(It would be nice if design documentations/explanations like this are in a predicable place like Julep... #33239)

[rapus95]

I think, either way, we should add exactly those contracted properties to the docs of the comparison operators. I suggest to tag each ordering system's operators with a keyword (like "hashing" and "intuitive/IEEE" for now) and to define the following properties on those different "universes":

usecase	keyword		operators		notes
		equality (eqop)	less (lsop)	hash (hsop)
arrays & content comparison	intuitive/IEEE	==	<		@fastmath partially disables IEEE
dicts & sets & hash based sorting	hashing	isequal	isless	hash	defaults to intuitive/IEEE

Now, for the given categories we have required contracts and suggestions

name	predicate	hashing	intuitive/IEEE
totality	oneof(lsop(x, y), lsop(y,x), eqop(x,y))	✔️	⚠️(only at max one of)
	!lsop(x,y) && !lsop(y,x) == eqop(x,y)	✔️	⚠️(only partial order)
reflexivity	ifelse(x===y, eqop(x,y), true)	✔️	⚠️
transitivity	ifelse(eqop(x,y) && eqop(y,z), eqop(x,z), true)	✔️	⚠️
symmetricy	eq(x,y) == eq(y,x)	✔️	⚠️
hash consistency	ifelse(eqop(x,y), hsop(x)==hsop(y), true)	✔️
missingness propagates	iseq(x, missing) == missing isls(x, missing) == missing hsop(missing) == missing	❌	✔️

✔️ required/contracted
⚠️ intended but not contracted
❌ never the case, eventually forbidden

Further improvements & additions would be very welcome!

Edit: How about adding a 3rd universe by splitting up intuitive & IEEE? That'd at least help to define the actions that may happen in @fastmath.
Edit2: So there's no comparison operator besides === which is guaranteed to hold for the identity?

[andyferris]

Thanks @tkf, that is very helpful.

(If I were to critique Stefan's rationale I would say that jumbled-up ordering of -0.0 and 0.0 upon sort may be unpretty, but seems less likely to result in bugs than having both 0.0 and -0.0 in a Set or the keys of a Dict. The argument of following IEEE 754 total ordering seems rather weakened by the fact that, as I understand it, we don't follow IEEE total ordering since we ignore the sign bit of NaN (thank goodness!))

One thing I would like to get out of this (as a library writer implementing things like sets and ordered collections, or algorithms for Base) is an answer to a question I've had for a while: can I assume in generic code that (a == b) === true implies isequal(a, b) excepting AbstractFloats with 0.0/-0.0? (We need some assumptions here in order to efficiently implement e.g. any(==(x), ::Set), or set comparisons using == on the elements, or searching sorted collections quickly with a < predicate like findall(<(0), my_sorted_collection)). If so, would a PR adding documentation to this effect be welcomed?

Can one safely assume that === implies isequal in generic code, or is that a no-go? If so, might we document that?

EDIT: Damn, I just remembered that unordered sets (and dictionaries) can't satisfy set1 == set2 implies isequal(set1, set2) - oops! (Although ordering those might be good for other reasons...)

[tkf]

can I assume in generic code that (a == b) === true implies isequal(a, b) excepting AbstractFloats with 0.0/-0.0?

What about CompoundPeriod? #34744 (comment)

[JeffBezanson]

It's important to distinguish keys and values here. Sets don't contain their elements (and dicts their keys) in the same way that arrays do. For sets and dicts, the container determines what equality predicate to use. Dict uses isequal and IdDict uses ===, for example, so == on Dicts compares keys with isequal and values with ==.

Can one safely assume that === implies isequal in generic code, or is that a no-go? If so, might we document that?

Yes, === should imply isequal.

but it's slightly unfortunate that it makes Julia a bit of oddball when it comes to Dict(-0.0 => 1)[0.0]. Trying {-0.0: 1}[0.0] in Python and something similar in Ruby, Go, and Clojure all work

The motivation for this is memoization. A numerical function can easily give a very different answer for 0.0 and -0.0, so they should get separate lookup table entries. Refusing to hash floating point numbers is just a cop-out. People also seem to think the IEEE committee is listening on their Alexa or something, and will get them in trouble if they ever try to distinguish 0.0 and -0.0. No. The spec only says how == should work; it does not care how you look things up in dictionaries.

[tkf]

The motivation for this is memoization.

Thanks for the explanation. I feel this is much stronger motivation than sort([0.0, -0.0]).

It's important to distinguish keys and values here.

I think I understand this, and, because of this, I think it's important to document the behavior like

julia> Dict(1 => missing) == Dict(1 => missing)
missing

julia> Dict(missing => 1) == Dict(missing => 1)
true

and also

julia> Dict(0.0 => true, -0.0 => false)[0.0]
true

as I don't think it is immediately obvious that this is how Dict etc. works in Julia. A short footnote for the justification (memoization and sorting 0.0/-0.0) would be great, too. (Maybe I'll try creating a PR at some point.)

[rapus95]

@JeffBezanson what specifically didn't you like about adding a tabular overview of the current comparison operators/systems aswell as contracting them in how they work for given circumstances? (For knowing what to do when implementing my own variant aswell as knowing what I can rely on)

Yes, === should imply isequal.

I'll add that to the table.

[andyferris]

Yes, === should imply isequal.

Cool, I submitted a PR (#34798) to document that.

@JeffBezanson -0.0 for memoization of numerical functions is a really good example of where this might be useful in practice - thanks. (I suppose one could use an IdDict for this behavior as necessary, but I'm not particularly sure I want to drag out this discussion! 😆). Your point about different comparison functions for keys is interesting - I wonder if introspecting this function should be part of the abstract interface? (e.g key_comparitor(::Dict) = isequal, key_comparitor(::IdDict) = === - and should AbstractDict support arbitrary reflexive/transitive predicates?). The opposite way to go would be to instead always use isequal on the keys and wrap each key in a wrapper type that forwards isequal to === and hash to objectid (which could make for a simpler, more composable AbstractDict interface).

Regarding this PR - I for one would prefer array equality to strictly depend on the elements (and axes) and not differ between vector and copy(vector). I feel the exception with them being the same object may lead to confusion, ambiguity or bugs (and seems to create a distinction between mutable/immutable types). @rfourquet for now when I want this behavior I would tend to use isequal rather than == since it provides sensible & reliable guarantees around "being the same thing". I also happen to feel there is plenty of opportunity to improve on dictionaries (and maybe sets?) to remove the kinds of differences mentioned in the OP.

[rapus95]

I feel like those suggestions are a good idea for the new Dictionaries.jl package. That key_comparator(::AbstractDictionary) would be an interesting trait. At some point we might even have AbstractArray sharing traits with AbstractDictionary (which is what Dictionaries.jl aims for IIRC)

[StefanKarpinski]

what specifically didn't you like about adding a tabular overview of the current comparison operators/systems aswell as contracting them in how they work for given circumstances?

I don't have any issues with tables, but that one seems to me to confuse much more than it clarifies. I don't really understand what it's trying to convey. It seems to make the situation seem much more ad hoc and confusing than it actually is. Perhaps some table would be helpful.

[JeffBezanson]

I wonder if introspecting this function should be part of the abstract interface? (e.g key_comparitor(::Dict) = isequal, key_comparitor(::IdDict) = ===

Yes I think we should add that.

[rapus95]

I don't have any issues with tables, but that one seems to me to confuse much more than it clarifies. I don't really understand what it's trying to convey. It seems to make the situation seem much more ad hoc and confusing than it actually is. Perhaps some table would be helpful.

Well, after reading all those function docs again I feel like I haven't been on the newest version regarding their documentation. I guess the only thing I'm missing is a paragraph about "Comparables" in the Interfaces section of the docs.

That'd guide you on which functions to overload in a similar manner to how iteration is explained.
For example:
Does that type induce a total or a partial order? Then define
total: isequal & isless
partial: == & <
Shall it be usable as dictionary keys?
define hash & check isequal(x,y)=>hash(x)==hash(y) holds
[Insert the list of defaults for those functions similar to the approach at Iterations]

Btw why do we fallback from isequal to == when the former requires to be reflexive and the latter may not be? Especially since for isless/< we go the other way around and have < fallback to isless

---

Well, and I find the docs of isless confusing.

Test whether x is less than y, according to a fixed total order. isless is not defined on all pairs of values (x, y).

That sounds contradictory. The second sentence should rather clarify that outside of the common domain it may not be defined. Like:

isless may not be defined for pairs (x,y) when x and y are from different domains/types".

We also should note x===y => isequal(x,y) as that is not required from ==.
Finally I found the order of function docs misordered isless(::Tuple, ::Tuple) should come after the generic docs.
Shall I open a new issue for those 3 points?

[StefanKarpinski]

That sounds contradictory.

Agree, that does seem contradictory. Clarification would be good. I believe @JeffBezanson wrote that in the first place. Can you clarify what you meant? Perhaps it's that given x and y either they are of comparable types or not:

• if comparable then isless(x, y) ⊻ isless(y, x) ⊻ isequal(x, y)
• if incomparable then !isequal(x, y) and isless(x, y), isless(y, x) are errors

Whether two values are comparable or not depends only on their types, not specific values.

[JeffBezanson]

Yes that's basically what it means. It's just to point out that while isequal works on all pairs of values, isless might be an error. I think it's possible though that isequal(x,y) could be true, but isless(x,y) would still be an error, for types that don't have any kind of order.

[andyferris]

Could we put a stable total order on (at least concrete) Types and have isless default to that?

[tkf]

If the philosophy behind the compatibility between hasing and sorting is to support consistent behavior of hash-based and comparison-based containers, I think making isless a total function makes sense.

[JeffBezanson]

I don't think that follows; having an equivalence relation is weaker than having a total order.

[tkf]

(I opened #34815 for discussing totality of isless, as I think this is not the main topic here.)