Decorate exceptions with backtrace information

[bgamari]

The proposal has been accepted; the following discussion is mostly of historic interest.

[isovector]

I love this.

[saurabhnanda]

Add a HasCallStack constraint to toException, incurring potentially unnecessary runtime cost and changing the type of a fairly widely used function (albeit in a backwards compatible way)

@bgamari FWIW I had attempted some benchmarks a long time ago:

• benchmarking HasCallStack in pure code
• benchmarking HasCallStack in IO-heavy code -- there are many variations of these benchmarks in that discussion thread.

[elaforge]

Isn't this a HasCallStackBacktrace, not a ExecutionBacktrace as implied above? Also it's not clear how the Backtrace type enables this, at least not without a definition for throw.

[jberryman]

Thanks for doing this work! Can you lead with the UX story and clarify a bit?

E.g.: I have a web app with various layers of exception handling with a topmost handler that logs anything that reaches there. Occasionally I get a mysterious head: empty list exception presumably from a dependency, but only in production. Does this help me? Would it have helped me diagnose the issue without me needing to anticipate it, or would I need to fiddle with stuff, release a new version, then hope the error is triggered? Say I've written my app already without being aware of this new backtrace exceptions stuff; what do I need to do exactly to my code to get a better story for the production scenario above?

If this is only really useful with a local profiling build, why not just xc?

I already don't use xc or HasCallStack since I guess I don't find them useful/ergonomic enougg. It sounds like if DWARF stack unwinding were fast enough to use in production then I might be able to use this?

[enobayram]

How much overhead would it bring to introduce a withBacktraceMechanism instead of a setGlobalBacktraceMechanism?

Also, would I ever want to have backtraces from multiple mechanisms for the same exception? My understanding is that HasCallstack gives you lexical traces but other methods give you execution traces, so even if performance weren't a concern, there's no strictly better-worse relationship between these mechanisms. Then, would it make sense to store a [Backtrace] instead of a Maybe Backtrace inside SomeException? Or maybe there are ways to obtain this behavior in other ways?

As a side note, I feel like the most significant feature suggested here is exposing to Haskell the stack trace powered by the DWARF debug information (Don't get me wrong, the change to SomeException is also important and significant). So, would it make sense to introduce it in a preliminary proposal?

P.S: Just a personal note; There have been times when I would sell my soul to the devil on the spot just to have the slightest bit of context for a given exception thrown in production. Any hint whatsoever...

[phadej]

foo someExc = do
    SomeException exc <- someExc

will desugar using MonadFail, currently it doesn't. See #327 #319. I don't consider PatternSynonyms to be good enough to offer compatibility layer for something as core feature as exceptions. I think this proposal would be better without until the pattern synonym.

Note, exceptions are also thrown in mtl style programs, where m might be on purpose not MonadFail so errors are reported using specific mechanism, potentially building on using exceptions.

[pepeiborra]

An alternative to the pattern synonym: leave SomeException alone and introduce a new datatype SomeExceptionWithLocation to become the root of the exception hierarchy, while SomeException would only stay around for backwards compatibility.

One also needs to keep fromException and toException in the Exception type class for backwards compatibility.

data SomeExceptionWithLocation where
    SomeExceptionWithLocation
      :: forall e. Exception e
      => Maybe Backtrace   -- ^ backtrace, if available
      -> e                 -- ^ the exception
      -> SomeExceptionWithLocation

 -- Two new members are added to the Exception type class, with default implementations.
class Exception e where
  toExceptionWithLocation :: e -> SomeExceptionWithLocation
  toExceptionWithLocation = SomeExceptionWithLocation Nothing

  fromExceptionWithLocation :: SomeExceptionWithLocation -> Maybe e
  fromExceptionWithLocation (SomeExceptionWithLocation _ e) = fromException e

  -- we keep the old members for backwards compatibility
  toException :: e -> SomeException
  fromException :: SomeException -> Maybe e

{- NOTE changing the default implementations of toException and fromException to be
    defined via the WithLocation variants will loop for fully default instances.
 
  toException e = case toExceptionWithLocation e of 
    SomeExceptionWithLocation _ someE -> SomeException someE

  fromException e = fromExceptionWithLocation (SomeExceptionWithLocation Nothing e)
-}

instance Exception SomeExceptionWithLocation where
  toExceptionWithLocation = id
  fromExceptionWithLocation = Just

instance Exception SomeException where
  toException = id
  fromException = Just

-- Minor changes are needed to 'throw' and 'catch' 
throw = raise# . toExceptionWithLocation
catch = ...

[Shimuuar]

Another problem with SomeException pattern synonym is case exc of SomeException e -> SomeException e is no longer so whenever code inspects exception and then rethrows it it will strip and replace stack trace information.

[Ericson2314]

I think it's nice to point out that the extra state lives purely in base using regular synchronization primitives? If one was really worried they could alwaysrebuild base to statically force one sort of backtrace and remove the state.

[cartazio]

is this a ghc proposal or a base / core libraries proposal? this seems to be strictly libraries based ( cc @chessai )

[cartazio]

another question: why are we assuming/presuming that you'll only provide one of the several types of backtrace? what obstacles or reasons for/against having an n-tuple of possible subset products of these various backtraces?

[goldfirere]

I agree about concerns around pattern synonyms. But maybe using one in this way would be a nice forcing function to get GHC to fix weaknesses around them.

I'm still a little unsure of what this all means, though. Here are a few questions:

• Suppose I don't take any particular action w.r.t. this proposal and my program throws an exception. Will it have tracing info? That is, what's the default behavior?

• DWARF traces are evidently time-consuming to build. Is it possible to build a DWARF trace only if an exception won't be caught? Or do we not have that information at the right time?

• Is DWARF available on all platforms? I somehow thought it was only certain Linux flavors.

• Would this mechanism be used by all exceptions? This worries me. If a library uses lots of exceptions internally (with the expectation of catching them all), then would this introduce a performance regression?

Don't get me wrong -- I love the idea of having stack traces on exceptions. I just don't really understand how this will work in practice.

[Philonous]

As someone has pointed out on reddit¹ there's combinators in base that might need adjustment to preserve backtraces when they re-throw (e.g. tryJust²).

1: https://reddit.com/r/haskell/comments/gfg5ac/ghc_proposal_decorate_exceptions_with_backtrace/fpvtxe8/
2: https://hackage.haskell.org/package/base-4.14.0.0/docs/src/Control.Exception.Base.html#tryJust

[bgamari]

@int-index, as far as I can tell there are no major objections to the proposed handling of rethrowing. I would consequently like to submit the proposal for consideration by the committee.

[int-index]

I hereby declare the proposal to be accepted.

[int-index]

I took the liberty to squash the commits, so the GitHub interface didn't detect the merge. Nevertheless, you can find the merged document at https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0330-exception-backtraces.rst

[nomeata]

Yeah, I usually don’t bother squashing. Thanks for merging!

[simonmar]

I know this is closed, but I didn't want to open another thread somewhere else... I was just taking another look at this and noticed that with the current design if we catch and rethrow a SomeException then we'll get a duplicate context (one from the original exception, and one from the WhileHandling) whereas this doesn't happen when catching and rethrowing other types of exception. That's a slightly unsatisfying inconsistency. Should we make it so that the original context is stripped by catch in this case?

[sidkshatriya]

From the Proposal:

• HasCallStack:
  • Pros: Can be used on all platforms; provides precise backtraces
  • Cons: Requires manual modification of the source program; runtime overhead
• Cost-centre profiler (via GHC.Stack.CCS.getCurrentCCS):
  • Pros: Can be used on all platforms; fairly precise backtraces
  • Requires profiled executable (-prof); runtime overhead; may require
    manual SCC pragmas
• DWARF debug information in conjunction with GHC's built-in stack unwinder <https://www.haskell.org/ghc/blog/20200405-dwarf-3.html>_:
  • Pros: No runtime overhead; can trace through foreign code
  • Cons: Highly platform-specific (currently only available on Linux); slow
    backtrace collection; imprecise backtraces; large binary size overhead
    (built with -g3)
• Info-table provenance (IPE) information (via GHC.Stack.CloneStack):
  • Pros: Can be used on all platforms; no runtime overhead
  • Cons: Large binary size overhead; no visibility into foreign code; must be
    built with -finfo-table-map

(Some text in the quote above has been bolded by me)

So HasCallStack results in precise backtraces, Cost center is "fairly precise", DWARF is "imprecise".

What about IPE backtraces ? How can we qualify the qualify of the backtraces on that ? Precise ? Imprecise ? As precise as HasCallStack ?

[tbidne]

@sidkshatriya I (sort of) asked this question on discourse. I have not received a response, but perhaps my examples are useful to you. For me the answer seems to be "fairly imprecise", but I would love to learn that I have done something wrong.

[sidkshatriya]

Thanks @tbidne !

@bgamari Sorry for the ping -- Your input on #330 (comment) would be much appreciated too !