Failing tests on custom number types

[giordano]

E.g.

   Expression: ≈(#= /home/runner/work/Reactant.jl/Reactant.jl/test/custom_number_types.jl:13 =# @jit(sumall(x_ra)), sum(x_64), atol = 0.1, rtol = 0.1)
   Evaluated: ConcreteRNumber{Float64}(12.0) ≈ 14.109375 (atol=0.1, rtol=0.1)

It's really easy to generate a set of Float8_4s which would cause the test failure:

julia> x = Float8_4[Float8_4(0.5625) Float8_4(0.03125) Float8_4(0.3125); Float8_4(0.6875) Float8_4(0.375) Float8_4(0.078125); Float8_4(0.34375) Float8_4(0.5625) Float8_4(0.375); Float8_4(0.6875) Float8_4(0.203125) Float8_4(0.4375); Float8_4(0.3125) Float8_4(0.04296875) Float8_4(0.75); Float8_4(0.34375) Float8_4(0.5625) Float8_4(0.625); Float8_4(0.203125) Float8_4(0.625) Float8_4(0.5625); Float8_4(0.6875) Float8_4(0.8125) Float8_4(0.234375); Float8_4(0.4375) Float8_4(0.75) Float8_4(0.078125); Float8_4(0.875) Float8_4(0.4375) Float8_4(0.5)];

julia> x_ra = Reactant.to_rarray(x);

julia> sum(x)
Float8_4(12.0)

julia> reinterpret(Float8_4, Reactant.to_number(@jit(sum(x_ra))))
Float8_4(14.0)

The result is a bit off and no conversion is involved here.

It's also easy to construct an example involving fewer number which would fail the 0.1 atol/rtol tolerance test:

julia> x = Float8_4[Float8_4(0.5625), Float8_4(1.0), Float8_4(0.3125), Float8_4(0.3125)];

julia> x_ra = Reactant.to_rarray(x);

julia> sum(x)
Float8_4(2.25)

julia> reinterpret(Float8_4, Reactant.to_number(@jit(sum(x_ra))))
Float8_4(2.0)

I wonder if the difference is due to different summation orders between sum and Reactant, for example

julia> x[1] + x[2] + x[3] + x[4]
Float8_4(2.25)

julia> x[2] + x[3] + x[4] + x[1]
Float8_4(2.0)

Maybe the test should be changed to checking the result within a 2prevfloat and 2nextfloat (or Float8s.jl should have an eps method to be able to do isapprox natively on Float8_4)? I guess the alternative would be to stop using randomly generated numbers (#620), not sure if the motivation for that ticket was this test.