Suggestion: abbreviate stack traces by default

[BioTurboNick]

I'd like to suggest abbreviating the display of stack traces in errors by default, with the ability to display the full trace immediately after if you wish to see it.

As an example, this stack trace from when BenchmarkTools.@btime errors:

ERROR: UndefVarError: k not defined
Stacktrace:
 [1] var"##core#271"()
   @ Main ~\.julia\packages\BenchmarkTools\Kt6kv\src\execution.jl:371
 [2] var"##sample#272"(__params::BenchmarkTools.Parameters)
   @ Main ~\.julia\packages\BenchmarkTools\Kt6kv\src\execution.jl:377
 [3] _run(b::BenchmarkTools.Benchmark{Symbol("##benchmark#270")}, p::BenchmarkTools.Parameters; verbose::Bool, pad::String, kwargs::Base.Iterators.Pairs{Symbol, Integer, NTuple{4, Symbol}, NamedTuple{(:samples, :evals, :gctrial, :gcsample), Tuple{Int64, Int64, Bool, Bool}}})
   @ Main ~\.julia\packages\BenchmarkTools\Kt6kv\src\execution.jl:405
 [4] #invokelatest#2
   @ .\essentials.jl:710 [inlined]
 [5] #run_result#39
   @ ~\.julia\packages\BenchmarkTools\Kt6kv\src\execution.jl:32 [inlined]
 [6] run(b::BenchmarkTools.Benchmark{Symbol("##benchmark#270")}, p::BenchmarkTools.Parameters; progressid::Nothing, nleaves::Float64, ndone::Float64, kwargs::Base.Iterators.Pairs{Symbol, Integer, NTuple{5, Symbol}, NamedTuple{(:verbose, :samples, :evals, :gctrial, :gcsample), Tuple{Bool, Int64, Int64, Bool, Bool}}})
   @ BenchmarkTools ~\.julia\packages\BenchmarkTools\Kt6kv\src\execution.jl:94
 [7] #warmup#47
   @ ~\.julia\packages\BenchmarkTools\Kt6kv\src\execution.jl:141 [inlined]
 [8] warmup(item::BenchmarkTools.Benchmark{Symbol("##benchmark#270")})
   @ BenchmarkTools ~\.julia\packages\BenchmarkTools\Kt6kv\src\execution.jl:141
 [9] top-level scope
   @ ~\.julia\packages\BenchmarkTools\Kt6kv\src\execution.jl:481

An improved experience might be something like:

ERROR: UndefVarError: k not defined
in var"##core#271"()
   @ Main ~\.julia\packages\BenchmarkTools\Kt6kv\src\execution.jl:371
Use `stacktrace(errs)` to retrieve the full stack trace.

Related to #36517, where @vtjnash mentioned the possibility of a errs variable that stores an error similar to ans storing the last result.

[KristofferC]

An improved experience might be something like:

So always take the first? How about:

julia> "foo"[4]
ERROR: BoundsError: attempt to access 3-codeunit String at index [4]
Stacktrace:
 [1] checkbounds
   @ ./strings/basic.jl:216 [inlined]
 [2] codeunit
   @ ./strings/string.jl:102 [inlined]
 [3] getindex(s::String, i::Int64)
   @ Base ./strings/string.jl:223
 [4] top-level scope
   @ REPL[2]:1

Would only showing the first entry there really be better?

[BioTurboNick]

I think so? The error message itself is pretty clear, the stack trace seems dispensable?

[BioTurboNick]

Or you could show the first non-Base/stdlib frame

[KristofferC]

I think so? The error message itself is pretty clear, the stack trace seems dispensable?

But how would you know where in your code the problem originated from?

[JeffBezanson]

This has come up a couple times in the past and I generally resist hiding information. To me, the tradeoff is not good since at best this just avoids a bit of visual noise, but at worst it makes debugging harder. We will start to see bug reports without the whole stack trace. Obviously many people will request the full trace before filing an issue, but it's one more step in the way of getting the information we need. To be a bit hyperbolic, it's like complaining that a smoke detector is too loud.

[BioTurboNick]

I totally understand that view.

To a novice user though, extraneous information can be daunting and actively prevent them from understanding where the problem is, potentially leading to unnecessary support requests or frustration.

I was partly inspired by the complaint posted on Slack from the JuliaOptics guy, but I've seen that sort of complaint before and it was a bit of a barrier for me getting started.

Readily available, discoverable, but not shown by default strikes a good balance there IMO. I made a similar argument in favor of low-cost notice of deprecation warnings, against them being hidden entirely.

Take this example:

a() = "foo"[4]
b() = a()
c() = b()
d() = c()
e() = d()
e()

ERROR: BoundsError: attempt to access 3-codeunit String at index [4]
Stacktrace:
 [1] checkbounds
   @ .\strings\basic.jl:216 [inlined]
 [2] codeunit
   @ .\strings\string.jl:102 [inlined]
 [3] getindex
   @ .\strings\string.jl:223 [inlined]
 [4] a
   @ .\REPL[6]:1 [inlined]
 [5] b
   @ .\REPL[7]:1 [inlined]
 [6] c
   @ .\REPL[8]:1 [inlined]
 [7] d
   @ .\REPL[9]:1 [inlined]
 [8] e()
   @ Main .\REPL[10]:1
 [9] top-level scope
   @ REPL[11]:1

vs.

ERROR: BoundsError: attempt to access 3-codeunit String at index [4]
in a
   @ .\REPL[6]:1 [inlined]
 Use `stacktrace(errs)` to retrieve the full stack trace.

If a stack trace can be filtered for Base/stdlib methods and show the top one, it would be a lot more clear where those more simple errors originate.

Maybe there are good reasons not to do it right now, but it might be useful as Julia matures and grows.

Amending with another example. HTTP requests fail with an exception. But it's just information, the stack trace isn't useful there, and takes up ~20 lines depending on window width:

ERROR: HTTP.ExceptionRequest.StatusError(404, "GET", "/proteins/api/protein/P08069", HTTP.Messages.Response:
"""
HTTP/1.1 404 Not Found
Server: grizzly/2.3.28
Vary: Accept
Content-Type: application/json
Access-Control-Allow-Credentials: true
Access-Control-Expose-Headers: Content-Type, X-UniProt-Release, Link, X-Pagination-TotalRecords
Strict-Transport-Security: max-age=0
Date: Sat, 27 Mar 2021 03:25:38 GMT
Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: origin, content-type, accept, authorization
Access-Control-Allow-Methods: GET, POST, PUT, DELETE, OPTIONS, HEAD
Content-Length: 106

{"requestedURL":"https://www.ebi.ac.uk/proteins/api/protein/P08069","errorMessage":["HTTP 404 Not Found"]}""")
Stacktrace:
 [1] request(::Type{ExceptionLayer{ConnectionPoolLayer{StreamLayer{Union{}}}}}, ::URIs.URI, ::Vararg{Any,N} where N; kw::Base.Iterators.Pairs{Symbol,Any,Tuple{Symbol,Symbol,Symbol},NamedTuple{(:iofunction, :reached_redirect_limit, :accept),Tuple{Nothing,Bool,String}}}) at C:\Users\nicho\.julia\packages\HTTP\cxgat\src\ExceptionRequest.jl:22
 [2] (::Base.var"#56#58"{Base.var"#56#57#59"{ExponentialBackOff,HTTP.RetryRequest.var"#2#3"{Bool,HTTP.Messages.Request},typeof(HTTP.request)}})(::Type{T} where T, ::Vararg{Any,N} where N; kwargs::Base.Iterators.Pairs{Symbol,Any,Tuple{Symbol,Symbol,Symbol},NamedTuple{(:iofunction, :reached_redirect_limit, :accept),Tuple{Nothing,Bool,String}}}) at .\error.jl:288
 [3] #request#1 at C:\Users\nicho\.julia\packages\HTTP\cxgat\src\RetryRequest.jl:44 [inlined]
 [4] request(::Type{MessageLayer{RetryLayer{ExceptionLayer{ConnectionPoolLayer{StreamLayer{Union{}}}}}}}, ::String, ::URIs.URI, ::Array{Pair{SubString{String},SubString{String}},1}, ::Array{UInt8,1}; http_version::VersionNumber, target::String, parent::Nothing, iofunction::Nothing, kw::Base.Iterators.Pairs{Symbol,Any,Tuple{Symbol,Symbol},NamedTuple{(:reached_redirect_limit, :accept),Tuple{Bool,String}}}) at C:\Users\nicho\.julia\packages\HTTP\cxgat\src\MessageRequest.jl:66
 [5] request(::Type{BasicAuthLayer{MessageLayer{RetryLayer{ExceptionLayer{ConnectionPoolLayer{StreamLayer{Union{}}}}}}}}, ::String, ::URIs.URI, ::Array{Pair{SubString{String},SubString{String}},1}, ::Array{UInt8,1}; kw::Base.Iterators.Pairs{Symbol,Any,Tuple{Symbol,Symbol},NamedTuple{(:reached_redirect_limit, :acccept),Tuple{Bool,String}}}) at C:\Users\nicho\.julia\packages\HTTP\cxgat\src\BasicAuthRequest.jl:28
 [6] request(::Type{RedirectLayer{BasicAuthLayer{MessageLayer{RetryLayer{ExceptionLayer{ConnectionPoolLayer{StreamLayer{Union{}}}}}}}}}, ::String, ::URIs.URI, ::Array{Pair{SubString{String},SubString{String}},1}, ::Array{UInt8,1}; redirect_limit::Int64, forwardheaders::Bool, kw::Base.Iterators.Pairs{Symbol,Stringg,Tuple{Symbol},NamedTuple{(:accept,),Tuple{String}}}) at C:\Users\nicho\.julia\packages\HTTP\cxgat\src\RedirectRequest.jl:24
 [7] request(::String, ::String, ::Array{Pair{SubString{String},SubString{String}},1}, ::Array{UInt8,1}; headers::Array{Pair{SubString{String},SubString{String}},1}, body::Array{UInt8,1}, query::Nothing, kw::Base.Iterators.Pairs{Symbol,String,Tuple{Symbol},NamedTuple{(:accept,),Tuple{String}}}) at C:\Users\nicho\\.julia\packages\HTTP\cxgat\src\HTTP.jl:315
 [8] top-level scope at REPL[17]:1

[david-macmahon]

Is there some way to override how the REPL displays exceptions? I'm imagining something like a user-supplied show method in startup.jl to handle how Exceptions are displayed, but I don't know how feasible something like that is.

[BioTurboNick]

Just learned about isinteractive. Is it the case that isinteractive will be false e.g. in CI runs, where if a feature like this was adopted, the whole stack could be dumped normally in that context?

[StefanKarpinski]

I want to add a voice from the "core dev" side in support of abbreviated stack traces. I think this is a major usability issue for the language. Two-language solutions have an advantage here: the boundary between the high-level (e.g. Python) and low-level language (e.g. C) is a natural stopping point for errors — if something is wrong about how the user is calling a function, it will get caught before calling into C or not at all; if something goes wrong inside of the C call, the user doesn't get a C stack trace, they see an error raised by the C function. Imagine if Python users were subjected to C stack traces deep into the CPython runtime? That's basically what we're currently subjecting Julia users to. Yes, the stack is all Julia calls, but if something goes deep inside of stdlib, does the typical user they care about a dozen internal function calls between the public stdlib function they called and where they error actually occurred? They do not. Printing all that extra information makes it much harder for the typical user to understand what actually went wrong and fix it. Frankly, when I get stack traces, I find it annoyingly difficult to figure out where the actual problem is in the wall of text that we dump.

[stevengj]

What if we restricted the stack trace by default to show (a) lines in code defined in Main or any modules in untracked/dev packages, (b) the entry points into all other code, and (c) the topmost stackframe.

The example above would become something like:

 [1] request(::Type{ExceptionLayer{...}}, ::URIs.URI, ::Vararg{Any,N} where N; kw::Base.Iterators.Pairs{...}) at C:\Users\nicho\.julia\packages\HTTP\cxgat\src\ExceptionRequest.jl:22
 [omitted stack frames 2–6]
 [7] request(::String, ::String, ::Array{Pair{...}},1}, ::Array{UInt8,1}; headers::Array{Pair{...},1}, body::Vector{UInt8}, query::Nothing, kw::Base.Iterators.Pairs{...}}) at C:\Users\nicho\\.julia\packages\HTTP\cxgat\src\HTTP.jl:315
 [8] top-level scope at REPL[17]:1

The other thing that would help a lot is to abbreviate the display of long parameterized type names, as I've done manually above. So many times I've been faced with an unhelpful "wall of types".

[BioTurboNick]

That's basically what my split-off package does right now if you haven't taken a look yet, @stevengj. My plan is to refine the idea there and then it can be brought back when it seems most issues are addressed.