Skip to content

report error magnitude on @test x ≈ y failure #55613

@stevengj

Description

@stevengj

As commented on discourse, it would be nice to update the @test macro to report the absolute and relative errors on a failure of an x ≈ y test.

We already specialize the test output on failed comparisons to report the operands:

julia> x, y = 1, 1 + 1e-5;

julia> @test x  y
Test Failed at REPL[63]:1
  Expression: x  y
   Evaluated: 1  1.00001

but it would be good to show the absolute error norm(x - y) and/or the relative error norm(x - y) / max(norm(x), norm(y)) e.g.

julia> @test x  y
Test Failed at REPL[63]:1
  Expression: x  y
   Evaluated: 1  1.00001
    (absolute error 1e-05 exceeds atol=0,
     relative error 1e-05 exceeds rtol=1.49e-08)

(where it should suffice to print the errors with %0.3g precision.) Or maybe only print the error that failed (i.e. either the atol or rtol, whichever is bigger).

Metadata

Metadata

Assignees

No one assigned

    Labels

    testsystemThe unit testing framework and Test stdlib

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions