ismissing(x) much slower than x === missing

[nalimilan]

Since #27651 a loop involving x === missing can be very fast, but using the more generic ismissing(x) leads to much less efficient code being generated. Apparently, the compiler isn't able to detect that !ismissing(x) implies that x::Int.

function mysum(X)
   s = zero(eltype(X))
   @inbounds @simd for x in X
       if x !== missing
           s += x
       end
   end
   s
end

function mysum2(X)
   s = zero(eltype(X))
   @inbounds @simd for x in X
       if !ismissing(x)
           s += x
       end
   end
   s
end


X1 = rand(Int, 10_000_000);
X2 = Vector{Union{Missing,Int}}(X1);
X3 = ifelse.(rand(length(X2)) .< 0.9, X2, missing);

julia> @btime mysum(X1);
  5.631 ms (1 allocation: 16 bytes)

julia> @btime mysum(X2);

  5.785 ms (1 allocation: 16 bytes)

julia> @btime mysum(X3);
  5.670 ms (1 allocation: 16 bytes)

julia> @btime mysum2(X1);
  5.676 ms (1 allocation: 16 bytes)

julia> @btime mysum2(X2);
  16.377 ms (1 allocation: 16 bytes)

julia> @btime mysum2(X3);
  20.681 ms (1 allocation: 16 bytes)

[StefanKarpinski]

What if you change the definition to ismissing(x::Any) = x === missing?

[vtjnash]

Probably a good idea (since it assists with more optimal inlining), but won't affect inference (I think we have a issue open about improving that)

[StefanKarpinski]

What's the inference issue?

[JeffBezanson]

#26417

[nalimilan]

What if you change the definition to ismissing(x::Any) = x === missing?

One of the points of having ismissing is that it could be overloaded in the future by a TypedMissing type which would behave like Missing, but also carry an additional information about the reason why it's missing which could be used explicitly, e.g. when recoding/imputing (see e.g. https://www.hl7.org/fhir/v3/NullFlavor/cs.html).

Though I guess having this definition as a fallback would be OK if that's more efficient. I'll let @vtjnash decide whether it's a good idea.

[Keno]

What's the inference issue?

=== is special in inference and inference happens before inlining, so even if they're the same after inlining, Inference doesn't know that.