Change in behavior of UndefVarError(::Symbol) constructor breaks @test_throws in 1.11.0-beta1

[DilumAluthge]

MWE

Here is the MWE:

using Test: @test_throws

module MyModule end

foo() = MyModule.x

@test_throws UndefVarError(:x) foo()

On Julia 1.10.2, this MWE works fine:

  | | |_| | | | (_| |  |  Version 1.10.2 (2024-03-01)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release

julia> using Test: @test_throws

julia> module MyModule end
Main.MyModule

julia> foo() = MyModule.x
foo (generic function with 1 method)

julia> @test_throws UndefVarError(:x) foo()
Test Passed
      Thrown: UndefVarError

On Julia 1.11.0-beta1, the MWE is broken:

  | | |_| | | | (_| |  |  Version 1.11.0-beta1 (2024-04-10)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release

julia> using Test: @test_throws

julia> module MyModule end
Main.MyModule

julia> foo() = MyModule.x
foo (generic function with 1 method)

julia> @test_throws UndefVarError(:x) foo()
ERROR: UndefRefError: access to undefined reference
Stacktrace:
 [1] do_test_throws(result::Test.ExecutionResult, orig_expr::Any, extype::Any)
   @ Test ~/.asdf/installs/julia/1.11.0-beta1/share/julia/stdlib/v1.11/Test/src/Test.jl:815
 [2] top-level scope
   @ REPL[4]:1

Similarly, on Julia nightly (1.12.0-DEV.339 / b9aeafa), the MWE is broken in the same way: (click to expand)

  | | |_| | | | (_| |  |  Version 1.12.0-DEV.339 (2024-04-13)
 _/ |\__'_|_|_|\__'_|  |  Commit b9aeafa171e (0 days old master)

julia> using Test: @test_throws

julia> module MyModule end
Main.MyModule

julia> foo() = MyModule.x
foo (generic function with 1 method)

julia> @test_throws UndefVarError(:x) foo()
ERROR: UndefRefError: access to undefined reference
Stacktrace:
 [1] do_test_throws(result::Test.ExecutionResult, orig_expr::Any, extype::Any)
   @ Test ~/.asdf/installs/julia/nightly/share/julia/stdlib/v1.12/Test/src/Test.jl:815
 [2] top-level scope
   @ REPL[4]:1

The issue here is that the behavior of the UndefVarError(::Symbol) constructor has changed between 1.10.2 and 1.11.0-beta1.

The UndefVarError(::Symbol) constructor is in the manual. So, some questions are:

1. Do we think that the behavior of the UndefVarError(::Symbol) constructor is public API?
2. Do we think this change is a breaking change?
3. Is there a way we can patch @test_throws to avoid this specific use case from breaking?

My initial thought is that this isn't a breaking change, and that we don't need to fix it in Julia; instead, we can just tell users to fix their code, e.g.:

@@ -1,7 +1,11 @@
 using Test: @test_throws

 module MyModule end

 foo() = MyModule.x

-@test_throws UndefVarError(:x) foo()
+if Base.VERSION >= v"1.11-"
+    @test_throws UndefVarError(:x, MyModule) foo()
+else
+    @test_throws UndefVarError(:x) foo()
+end

But I'm curious to hear what other people think.

[DilumAluthge]

If we do decide that we want to fix this in Julia itself, here are two possible ways to fix it: #54083, #54084

[Keno]

I prefer #54084