errors: add support for wrapping multiple errors

[neild]

For the most recent version of this proposal, see: #53435 (comment) below.

---

This is a variation on the rejected proposal #47811 (and perhaps that proposal should just be reopened), and an expansion on a comment in it.

Background

Since Go 1.13, an error may wrap another by providing an Unwrap method returning the wrapped error. The errors.Is and errors.As functions operate on chains of wrapped errors.

A common request is for a way to combine a list of errors into a single error.

Proposal

An error wraps multiple errors if its type has the method

Unwrap() []error

Reusing the name Unwrap avoids ambiguity with the existing singular Unwrap method. Returning a 0-length list from Unwrap means the error doesn't wrap anything. Callers must not modify the list returned by Unwrap. The list returned by Unwrap must not contain any nil errors.

We replace the term "error chain" with "error tree".

The errors.Is and errors.As functions are updated to unwrap multiple errors. Is reports a match if any error in the tree matches. As finds the first matching error in a preorder traversal of the tree.

The errors.Join function provides a simple implementation of a multierr. It does not flatten errors.

// Join returns an error that wraps the given errors.
// Any nil error values are discarded.
// The error formats as the text of the given errors, separated by sep.
// Join returns nil if errs contains no non-nil values.
func Join(sep string, errs ...error) error

The fmt.Errorf function permits multiple instances of the %w formatting verb.

The errors.Split function retrieves the original errors from a combined error.

// Split returns the result of calling the Unwrap method on err,
// if err's type contains an Unwrap method returning []error.
// Otherwise, Split returns nil.
func Split(err error) []error

The errors.Unwrap function is unaffected: It returns nil when called on an error with an Unwrap() []error method.

Questions

Prior proposals have been declined on the grounds that this functionality can be implemented outside the standard library, and there was no good single answer to several important questions.

Why should this be in the standard library?

This proposal adds something which cannot be provided outside the standard library: Direct support for error trees in errors.Is and errors.As. Existing combining errors operate by providing Is and As methods which inspect the contained errors, requiring each implementation to duplicate this logic, possibly in incompatible ways. This is best handled in errors.Is and errors.As, for the same reason those functions handle singular unwrapping.

In addition, this proposal provides a common method for the ecosystem to use to represent combined errors, permitting interoperation between third-party implementations.

How are multiple errors formatted?

A principle of the errors package is that error formatting is up to the user. This proposal upholds that principle: The errors.Join function combines error text with a user-provided separator, and fmt.Errorf wraps multiple errors in a user-defined layout. If users have other formatting requirements, they can still create their own error implementations.

How do Is and As interact with combined errors?

Every major multierror package that I looked at (see "Prior art" below) implements the same behavior for Is and As: Is reports true if any error in the combined error matches, and As returns the first matching error. This proposal follows common practice.

Does creating a combined error flatten combined errors in the input?

The errors.Join function does not flatten errors. This is simple and comprehensible. Third-party packages can easily provide flattening if desired.

Should Split unwrap errors that wrap a single error?

The errors.Split function could call the single-wrapping Unwrap() error method when present, converting a non-nil result into a single-element slice. This would allow traversing an error tree with only calls to Split.

This might allow for a small improvement in the convenience of code which manually traverses an error tree, but it is rare for programs to manually traverse error chains today. Keeping Split as the inverse of Join is simpler.

Why does the name of the Split function not match the Unwrap method it calls?

Giving the single- and multiple-error wrapping methods the same name neatly avoids any questions of how to handle errors that implement both.

Split is a natural name for the function that undoes a Join.

While we could call the method Split, or the function UnwrapMultiple, or some variation on these options, the benefits of the above points outweigh the value in aligning the method name with the function name.

Prior art

There have been several previous proposals to add some form of combining error, including:

https://go.dev/issue/47811: add Errors as a standard way to represent multiple errors as a single error
https://go.dev/issue/48831: add NewJoinedErrors
https://go.dev/issue/20984: composite errors
https://go.dev/issue/52607: add With(err, other error) error
fmt.Errorf("%w: %w", err1, err2) is largely equivalent to With(err1, err2).

Credit to @jimmyfrasche for suggesting the method name Unwrap.

There are many implementations of combining errors in the world, including:

https://pkg.go.dev/github.com/hashicorp/go-multierror (8720 imports)
https://pkg.go.dev/go.uber.org/multierr (1513 imports)
https://pkg.go.dev/tailscale.com/util/multierr (2 imports)

[DeedleFake]

How does walking up the tree work in errors.Is() and errors.As()? Is it depth-first, i.e. Unwrap() to []error and then fully walk each resulting tree, or is it breadth-first, i.e. Unwrap() to []error and then do each of those before unwrapping them and doing the next set. I assume that it's depth-first, as that's simpler, but breadth-first could also make sense. It would also be possible to convert from a default of breadth-first to a depth-first by implementing custom unwrapping logic, but going the other way could be difficult.

[neild]

How does walking up the tree work in errors.Is() and errors.As()?

Depth-first. Unwrap to an []error and walk each error in the list in turn. This is what every existing multierr implementation I looked at does.

[earthboundkid]

This would also subsume #50819 and I imagine any other similar proposals.

Maybe it should wait for a generic iteration standard, but should there be some errors.Each function that goes through each link in the chain? My suspicion is there’s not much to do besides As-ing and Is-ing, but it does seem like an absence, since there will be some internal function that works like that, which could potentially be exported.

[neild]

Is and As are separate implementations for efficiency reasons, so there's no internal Each function. This proposal does make the case for an exported Each or similar stronger, since iteration becomes more complicated.

We should probably wait and see what happens with generic iteration, however.

[balasanjay]

Does Unwrap return a reference to the internal slice in the error?
If so, that would make the internal slice reference externally mutable, which seems unfortunate (but not a dealbreaker).

If not, and it returns a copy, then it would allocate, which also seems unfortunate (but probably not a dealbreaker).

[neild]

Does Unwrap return a reference to the internal slice in the error?

Unspecified, but the contract for it says that the caller must not modify the slice, so it may.

Perhaps Split should copy the result into a new slice. This would leave Is and As potentially allocation-free, while adding a layer of safety (and an allocation) to the relatively unusual case of manually inspecting the individual errors.

[josharian]

This strikes me as a significant step forward in figuring out the multi-error question. Thank you!

The only part of this that doesn't seem entirely spot-on is errors.Split.

What is the use case for errors.Split?

Figuring errors.Split would be used like errors.Unwrap, I grepped through the code sitting around on my laptop and found only a single use of errors.Unwrap. It was used to check recursively whether any error in the chain satisfied a predicate. It would be interesting to analyze other uses in a larger corpus.

Walking an arbitrary error tree using errors.Split and errors.Unwrap will be annoying. I wonder whether a callback-based errors.Walk that walks the entire error tree would be a better API. There are a lot of details to specify for such an API, but with a bit of care, it could be more powerful and flexible than errors.Split. It could be used to implement errors.Is and errors.As. And it sidesteps questions about ownership and modification of returned error slices.

(The details about errors.Walk include things like: Does it call back for all errors, or only leaf errors? Does it use a single callback like io/fs.WalkDir or separate pre- and post- callbacks like golang.org/x/tools/ast/astutil.Apply? Does it support early stopping?)

On the topic of slice copies, a minor question. Does errors.Join make a copy of the variadic arg? I'd argue for yes, and ameliorate the pain a bit by adding a small default backing store to the struct. Aliasing bugs are miserable.

[earthboundkid]

I find that the main reason I unwrap my own multierrors is so that I can report them separately to my logging service or display them as a list in a CLI. The original xerrors formatting proposal was dropped for being too complicated but I think that an easy to use error walking mechanism might make it easy to let users figure out how they want to display and format all the suberrors on their own.

[rsc]

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

[hopehook]

If I hadn't read the comments, I might have thought that after the "Join" error, I could use "Split" to parse it out.

"errors.Split" does the same thing as "errors.Unwrap", but deals with multiple errors. So instead of "Split" which is the opposite of "Join", it should be "UnwrapMultiple" or "Walk" which is more appropriate.