Skip to content

Conversation

@jeffswenson
Copy link
Contributor

@jeffswenson jeffswenson commented Oct 6, 2025

benchmark: add a benchmark for the errors package

This is a benchmark created by @rafiss. It demonstrates errors.Is
is very inefficient when the reference error does not match the
input error.

errbase: optimize errors.Is

Previously, errors.Is was very inefficient if the reference error did
not match any errors in the chain.

There were two significant sources of inefficiency:

  1. The code would pessimistically construct the error mark for every
    error in the input error chain. This is O(chain_length^2). This was
    a lot of unnecessary allocations and caused the runtime to be O(n^2)
    in the average case instead of O(n).
  2. The code compared the Error() message before comparing the types.
    It is possible to compare error types with zero allocations whereas
    computing the message often requires an allocation.

refs: cockroachdb/cockroach#154555

This change is Reviewable

@cockroachlabs-cla-agent
Copy link

cockroachlabs-cla-agent bot commented Oct 6, 2025

CLA assistant check
All committers have signed the CLA.

This is a benchmark created by @rafi. It demonstrates `errors.Is`
is very inefficient when the reference error does not match the
input error.
Previously, `errors.Is` was very inefficient if the reference error did
not match any errors in the chain.

There were two significant sources of inefficiency:
1. The code would pessimistically construct the error mark for every
   error in the input error chain. This is O(chain_length^2). This was
   a lot of unnecessary allocations and caused the runtime to be O(n^2)
   in the average case instead of O(n).
2. The code compared the `Error()` message before comparing the types.
   It is possible to compare error types with zero allocations whereas
   computing the message often requires an allocation.
@jeffswenson jeffswenson force-pushed the jeffswenson-optimize-errors-is branch from 3ff6ee5 to 3a21e3d Compare October 7, 2025 15:35
@jeffswenson jeffswenson requested a review from dhartunian October 7, 2025 15:38
if m1.msg != m2.msg {
return false
}
if len(m1.types) != len(m2.types) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bug that is pretty unlikely to occur in the original implementation that checks error message before types, but its somewhat likely now that we check the types first.

return true
}
for _, reference := range references {
if Is(err, reference) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the reason we didn't use this implementation originally is Is was pretty slow. So it was better to use the quick check before the slow mark check. Now that Is is efficient, we don't need to duplicate the implementations.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the consolidation.

@jeffswenson
Copy link
Contributor Author

jeffswenson commented Oct 7, 2025

% benchstat before.txt after.txt
goos: darwin
goarch: arm64
pkg: github.com/cockroachdb/errors
cpu: Apple M1 Pro
                                        │   before.txt   │              after.txt              │
                                        │     sec/op     │   sec/op     vs base                │
ErrorsIs/NilError-10                         3.122n ± 1%   3.156n ± 2%   +1.12% (p=0.004 n=10)
ErrorsIs/SimpleError-10                     656.00n ± 3%   42.88n ± 2%  -93.46% (p=0.000 n=10)
ErrorsIs/WrappedError-10                   3512.00n ± 0%   82.72n ± 2%  -97.64% (p=0.000 n=10)
ErrorsIs/WrappedWithStack-10               1099.00n ± 3%   61.94n ± 1%  -94.36% (p=0.000 n=10)
ErrorsIs/NetworkError-10                   2597.50n ± 2%   99.72n ± 1%  -96.16% (p=0.000 n=10)
ErrorsIs/DeeplyWrappedNetworkError-10      26738.0n ± 5%   193.1n ± 1%  -99.28% (p=0.000 n=10)
ErrorsIs/MultipleWrappedErrors-10          18197.0n ± 8%   203.4n ± 1%  -98.88% (p=0.000 n=10)
ErrorsIs/NetworkErrorWithLongAddress-10    10504.5n ± 3%   122.0n ± 1%  -98.84% (p=0.000 n=10)
ErrorsIs/WithMessage-10                    2169.50n ± 2%   62.90n ± 1%  -97.10% (p=0.000 n=10)
ErrorsIs/MultipleWithMessage-10             7780.0n ± 3%   113.8n ± 1%  -98.54% (p=0.000 n=10)
ErrorsIs/WithMessageAndStack-10             5089.5n ± 5%   101.5n ± 1%  -98.00% (p=0.000 n=10)
ErrorsIs/NetworkErrorWithMessage-10        6485.50n ± 8%   93.93n ± 0%  -98.55% (p=0.000 n=10)
ErrorsIs/NetworkErrorWithEverything-10     43370.0n ± 3%   236.7n ± 5%  -99.45% (p=0.000 n=10)
ErrorsIs/DeeplyNested100Levels-10         4524.823µ ± 2%   2.764µ ± 2%  -99.94% (p=0.000 n=10)
geomean                                      5.231µ        102.8n       -98.03%

@jeffswenson jeffswenson marked this pull request as ready for review October 7, 2025 15:59
Copy link
Contributor

@dhartunian dhartunian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm: thanks for this improvement!

@dhartunian reviewed 1 of 3 files at r1, all commit messages.
Reviewable status: 1 of 3 files reviewed, 2 unresolved discussions (waiting on @jeffswenson)

return true
}
for _, reference := range references {
if Is(err, reference) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the consolidation.

@jeffswenson
Copy link
Contributor Author

Thanks for the review!

@jeffswenson jeffswenson merged commit 6480126 into master Oct 10, 2025
5 of 6 checks passed
jeffswenson added a commit to jeffswenson/cockroach that referenced this pull request Oct 10, 2025
Update the errors package to get the performance optimization from
cockroachdb/errors#153.

Release note: none
Informs: cockroachdb#154125
jeffswenson added a commit to jeffswenson/cockroach that referenced this pull request Oct 10, 2025
Update the errors package to get the performance optimization from
cockroachdb/errors#153.

Release note: none
Informs: cockroachdb#154125
craig bot pushed a commit to cockroachdb/cockroach that referenced this pull request Oct 10, 2025
155254: go.mod: update cockroachdb/errors package r=jeffswenson a=jeffswenson

Update the errors package to get the performance optimization from cockroachdb/errors#153.

Release note: none
Informs: #154125

Co-authored-by: Jeff Swenson <jeffswenson@betterthannull.com>
angles-n-daemons pushed a commit to angles-n-daemons/cockroach that referenced this pull request Oct 22, 2025
Update the errors package to get the performance optimization from
cockroachdb/errors#153.

Release note: none
Informs: cockroachdb#154125
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants