proposal: errors: add (stack)trace at error annotation

[flibustenet]

This proposal try to address the lack of traceback in errors but keep the way we handle errors as value annotated where the error occur.

Errors are values, a string or a custom error.
To retrieve the stack of errors we need to follow the chain of strings, each annotation should be written carefully to retrieve easily from where the error comes. If we just return err one time we loose the path.

Because of this we can embed all the stack in a custom error at the higher lever and print it at the end with the hope that we didn't loose the path with a return fmt.Errorf("... %v"). Like that we can just return err everywhere, we'll just use the full traceback to see what's happen like in most of the others languages.
Often we'll have both, a huge traceback and a chain of annotations in the end and full of return err everywhere. This is not fun and not idiomatic in Go.

Sometimes we just need an annotation, in a lib for example, and sometimes it's interesting to have the method and the exact line in an application. In a web server we generaly need only the stack after the handler and before the lib but not in the server and middlewares.

My proposal is to can add easily the trace and just this trace when we annotate the error. Something like fmt.Errorf("@trace I call foo: %v", err)

The result is a minimal traceback like that:
https://go.dev/play/p/VJSXkbOX7J_c

> main.main() prog.go:44
 f1 call f2: 
> main.f1() prog.go:13
 f2 call f3: 
> main.f2() prog.go:22
 simple error with no trace

I believe it follow the Go way of handling errors.
The error is still a value, a value with a trace as annotation (not to do for an API).
We decide at each error if we need the trace or not and not for the whole chain.
We encourage to annotate.
Explicit.
Easy to implement.

It's easy to experiment, just replace fmt.Errorf with a function like this:

func Errorf(s string, vals ...any) error {
	pc, file, line, ok := runtime.Caller(2)
	if ok {
		info := fmt.Sprintf("\n> %s() %s:%d\n",
			runtime.FuncForPC(pc).Name(), filepath.Base(file), line)
		s = strings.Replace(s, "@trace", info, 1)
	}
	return fmt.Errorf(s, vals...)
}

Hope it give at least inspiration.

[ianlancetaylor]

Note that you can already call a function that records a stack trace for an error, such as https://pkg.go.dev/github.com/go-errors/errors. Personally I think that if we want to add this kind of functionality into the standard library, we should do it using a different function, not using an annotation in the string passed to fmt.Errorf.

[flibustenet]

@ianlancetaylor I'm a aware of ways to add a full traceback and I writted about this, my english is not good.
This as few caveats:
The traceback is full even if you don't need all the stack.
The annotations are all at the end and not near the line.
Returning a new error in the flow loose the trace.
It encourage to just return err every where else.
It mimick a panic. and confuse beginners.

[earthboundkid]

There are a number of third party packages with traces. The unique thing the standard library can do is to standardize how the traces are exposed. Maybe an interface in runtime like

type ErrorTrace interface{
  error
  Trace() []Frames
  }

[ianlancetaylor]

@flibustenet Thanks, I think I see what you are saying now. But I don't see why this has to be in the standard library. It can be a function defined in a separate package.

[flibustenet]

@carlmjohnson my point was to embed only one Frame and not all the stack.
@ianlancetaylor thanks to try to understand my english ! I thought about this because of the way wrapping was resolved with a simple addon to fmt.Errorf with %w, i suggest the same kind of things for trace.
I don't make this proposal because I need it, of course it's nothing more easy to do it alone (and it works !) but I like to imagine for Go something more original than legacy traceback.

[earthboundkid]

I have errorx.Trace which adds a single trace line like this:

func traceExample(ok bool) (err error) {
	defer errorx.Trace(&err)
	if !ok {
		// returns "problem in github.com/carlmjohnson/errorx_test.traceExample (trace_example_test.go:13): oh no!"
		return errors.New("oh no!") 
	}
	return nil
}

I think it's more interesting to think about what can the standard library do that user code cannot do. Otherwise, I can just use my own code. :-)

[flibustenet]

@carlmjohnson fine to see similar ideas (I didn't search much).

[neild]

The original set of error proposals for 1.13 included adding stack frames to errors created with errors.New and fmt.Errorf. That change was implemented, but ultimately rolled back. As I recall, the concerns were startup overhead (var ErrFoo = errors.New("...") calls at init time got a lot more expensive), and reflect.DeepEqual no longer reporting errors with the same text as identical.

This proposal avoids these issues by making stack frames opt-in; you need to ask for one. The downside is that you don't get stack frames for errors unless the creator asks for them. Either this means most errors don't have stack frames, or we need to update the entire Go ecosystem to start asking for stack frames in errors.

Updating the entire Go ecosystem seems infeasible.

I don't know what the right approach to getting stacks into errors is. Clearly this would be useful; we dropped it from 1.13 because we couldn't make it work right, not because the goal was a bad idea. I think any successful proposal here will need to figure out how to get stacks into existing errors without rewriting the world, and without running into the issues the 1.13 implementation did.

[jaloren]

@ianlancetaylor you raise an important point. Why put this in the standard library when this can be written in third party code, especially when there are so many conflicting opinions on how error annotations should be handled?

IMHO, the answer is that in some cases, having a single way to approach a problem in the standard library with a canonical interface is a great help to the go community. The go community has high trust in the standard library in terms of quality and the backwards compatibility guarantee. And for common needs it also removes one less third party dependency i need to pull in or have a discussion about with my team. Finally, it provides a contract that allows third party libraries to opt into so an application can handle cross cutting concerns in a consistent way.

Consider the following examples:

• we are adding structured logging to the std library soon. The proposal process was grueling. I think all told there was well over 600 comments. Structured logging with log levels is something that anyone could write and.....we did. I think at one point there were over 10 popular logging libraries out there.
• the context library
• errors.Join

I've been writing go since 2016. Error annotation comes up constantly on my team both writing go code, reading it, and most importantly debugging it. I am sure everyone has had a story of getting an error printed to standard out that repeats the same error message multiple times (hello improper wrapping), being forced to grep through the go source files, and them making an educated guess which error was the source of the problem.