A retry package owned by bedrock

[Zaba505]

For many of the same reasons as the circuit breaker request, I would like bedrock to provide a general retry package. I really like the 2-step api of the circuit breaker package and wonder if a similar api could be used for a retry package too.

[Zaba505]

Review

A retry package in bedrock is defensible, but not for the same reasons as the circuit breaker, and only if it does more than a loop does. The two-step API carries over in part. Sitting inline in the caller's code carries over completely. The ticket and Release carry over only if something is held between attempts. Before this turns into stories, it needs its own reasons, a scope that includes a retry budget, and answers to the design questions in section 7.

1. Drop the freshness argument

It doesn't apply to retry. The leading libraries are among the most active in the ecosystem:

• cenkalti/backoff v7.0.1 (2026-09-30), after v6.0.0 and v7.0.0 in June;
• sethvargo/go-retry v0.5.0 (2026-10-05);
• failsafe-go v0.9.8 (2026-09-30);
• avast/retry-go v5.0.0 (2025-11-24), last pushed 2026-08.

There is a stale tail: jpillora/backoff v1.0.0 from 2019, lestrrat-go/backoff v2.0.8 from 2021 and Rican7/retry v0.3.1 from 2021. Nobody would point to it, though. Rewording won't save this argument the way it could in #595, so drop it.

2. Say exactly how existing retry libraries break the ethos

This is where the real case is. Each point below comes from a library's default path:

• Hidden defaults ("explicit over implicit").
  • cenkalti/backoff v7: Retry(ctx, op) with no options retries every error for up to 15 minutes. It backs off exponentially from 500ms, ×1.5, ±50%, up to 60s, with no limit on attempts.
  • avast/retry-go v5: 10 attempts, a 100ms delay plus up to 100ms of jitter, and context.Background() unless retry.Context(ctx) is passed. If a caller forgets that option, SIGTERM can't stop the loop, which is exactly what Process.NotifyContext exists to prevent.
  • failsafe-go: 3 attempts with no delay.
  • AWS SDK for Go v2: the standard retryer replaces a MaxAttempts of 0 or less with 3.
• Retrying everything by default ("fail loudly"). cenkalti retries every error not wrapped in backoff.Permanent, avast every error not wrapped in retry.Unrecoverable, and failsafe-go every error. circuit has a required classifier with no default, which is the opposite.
• Reading the process ("the process is the boundary"). The AWS standard retryer reads AWS_NEW_RETRIES_2026 with os.Getenv to choose its retry costs. Library code reading the environment is exactly what bedrock rules out.
• Wrapping the caller's code. All four take a closure: Retry[T](ctx, op), Retrier.Do(fn), Do(ctx, b, f) and failsafe.With(policies...).Get(fn). A circuit breaker should be provided by bedrock #595 settled that bedrock doesn't wrap the caller's code. That is the request's real reason, so it should say so.

sethvargo/go-retry is the exception, and the one the request has to answer. It retries only errors marked with RetryableError, and it requires a Backoff. The request should say why it isn't enough. From its source:

• Do wraps the call, and classifying means rewriting errors inside the closure.
• NewExponential and NewConstant panic on a duration that isn't positive. Durations come from configuration, and for those fit and circuit return an error.
• If the context ends during a wait, DoValue returns only context.Cause(ctx), and the last attempt's error is lost.
• WithMaxRetries counts inside the Backoff and never resets. Suppose a Backoff is built once at startup and shared, the way a bedrock Breaker is. Then it allows max retries for the life of the process, not per call.

The request shouldn't use panics as an argument. Unlike gobreaker, none of the four libraries calls recover in its library code.

3. Make it more than a loop: put a retry budget in scope

The circuit breaker earned a package because it holds shared state with invariants that are hard to keep under concurrency. A per-call retry loop holds none: it's an attempt counter, a delay and a wait, about 20 lines inline. Without shared state, a bedrock retry package is a helper teams can write themselves, and owning it is hard to justify.

The shared state in retrying is a retry budget: a limit, per dependency, on how much of the traffic may be retries. Per-call limits don't stop a retry storm. They multiply across layers: 3 attempts at each of 3 layers is up to 27 attempts at the bottom. They also multiply across every caller during an outage. Prior art:

• gRPC retry throttling, gRFC A6. A token_count per server starts at maxTokens. Each failed RPC subtracts 1 and each success adds tokenRatio. Nothing is retried while the count is at or below maxTokens/2.
• AWS SDK for Go v2, standard retryer. It keeps a bucket of 500 tokens. A retry costs 5 and a retry after a timeout costs 10. A first attempt that succeeds adds 1.
• failsafe-go, budget. It caps the share of in-flight executions that are retries or hedges.
• Google's SRE book, Handling Overload. It allows at most 3 attempts per request, and retries only while they're under 10% of a client's requests.

cenkalti/backoff, avast/retry-go and sethvargo/go-retry have no budget. The budget is retry's counterpart of the breaker's state machine: one per process and dependency, built at startup and shared by every goroutine. It's what justifies the package.

4. How the two-step API carries over

What carries over: being inline. It matters even more for retry than for breaking, because whether a retry is safe depends on the request as well as the error. The method may not be idempotent, the request may carry an idempotency key, the body may not be replayable (http.Request.GetBody), or the request may never have been sent. A classifier func(ctx, err) can't see any of that. Inline code can.

What doesn't carry over directly:

• It's a loop, not two steps. The circuit idiom is admit, call, settle. Retry is start, then for each attempt: call, report the result, get a decision (stop, or wait d and try again), and wait. Three shapes are worth comparing:

  // A: the package decides and waits.
  a := r.Start(ctx)
  for {
  	err = call(ctx)
  	if !a.Retry(ctx, err) { // classify, charge the budget, wait out the delay
  		break
  	}
  }
  
  // B: the package decides, and the caller waits.
  a := r.Start(ctx)
  for {
  	err = call(ctx)
  	d, ok := a.Next(err)
  	if !ok || retry.Wait(ctx, d) != nil {
  		break
  	}
  }
  
  // C: range over a func. The iterator waits between attempts and stops when Done says to.
  for a := range r.Attempts(ctx) {
  	err = call(ctx)
  	a.Done(err)
  }

  C is the shortest. The iterator's own frame can also free what the budget holds if the body panics, with no recover and no deferred Release. However, the loop body compiles to a function that the package calls, which comes close to the wrapping A circuit breaker should be provided by bedrock #595 ruled out.

• Nothing may be held between attempts. Release exists because a ticket holds a probe slot. Whether a retry holds anything depends on the kind of budget:

  • A token bucket, as in gRPC, spends tokens and holds nothing, so there's nothing to release.
  • An in-flight rate, as in failsafe-go, holds a slot for each running retry. The whole circuit idiom then carries over, with defer Release() as the panic guard.
  • AWS already has a two-step shape in the circuit sense: GetRetryToken returns a func(error) error that refunds the cost when the retry succeeds.

• There are no late results. circuit's Ticket also serves results that arrive after the function returns, such as a streamed body. A retry decision can't wait for one of those. Once a response is handed to the caller, the call is committed, in gRFC A6's term, and can't be retried. So retry has no hand-off case, and its docs have to say when a call is committed.

5. Why bedrock rather than humus

• The split is the one fit and circuit use. bedrock ships the model: attempt bookkeeping, the budget and the delay decision. humus ships the wiring: an http.RoundTripper that replays bodies with GetBody and reads Retry-After, gRPC interceptors, OTel instruments and config.Struct settings.

• gRPC already retries. grpc-go implements gRFC A6, throttling included, from the service config. A retrying interceptor on top of it retries twice over, so the request should acknowledge that overlap.

• There's a use case outside RPC. humus's Init doc already mentions retrying a database connection at startup. There the context comes from p.NotifyContext, so SIGTERM must end the wait. That use wants long schedules and no budget, while per-request RPC retries want short schedules and a budget. The request should say which use it targets, or that it targets both.

• Composing with circuit.

  • Retry goes outside the breaker, so each attempt goes through Allow.
  • Retry's library code should import only the standard library, as circuit's does. A *circuit.RejectedError therefore reaches retry as an ordinary error for the caller to classify. Usually the answer is stop, because retrying an open breaker defeats it. Otherwise it's to wait out RetryAfter when the deadline allows.
  • So retry needs a generic way to take a minimum delay from the server as a time.Duration: Retry-After, gRPC's grpc-retry-pushback-ms, or RejectedError.RetryAfter.
  • The composed idiom needs an example. defer t.Release() inside the retry loop is still correct, because Release does nothing after Done. But it keeps one deferred call per attempt until the function returns, so callers will reach for a per-attempt helper.

• A fourth module. It would be github.com/z5labs/bedrock/retry, nested like fit and circuit. A separate module fits, since circuit's doc already puts retry and backoff out of its scope. It costs:

  • build, vet, test and lint steps in CI;
  • a depguard rule;
  • the coverage gate;
  • the CLAUDE.md commands and Package Layout.

  Release tagging costs nothing, because tag-modules.sh already finds every nested go.mod.

6. Draw the scope boundary explicitly

• In:
  • a per-call limit on attempts;
  • a caller-supplied delay schedule;
  • a required classification supplied by the caller;
  • a minimum delay from the server;
  • a retry budget per dependency.
• Out:
  • hedging, which is retry's sibling but runs attempts concurrently and cancels the losers;
  • timeouts per attempt, which the caller sets with context.WithTimeout around each attempt; being inline makes that natural;
  • fallbacks, rate limits and bulkheads;
  • a budget shared between processes;
  • a registry of named retriers, which would be global state;
  • replaying request bodies and parsing Retry-After or pushback, which is protocol work for humus.

7. Design questions to settle before stories

1. Who waits. Shapes A and C put a timer in the package. B keeps the timer out, as circuit does, at the cost of a wait at every call site or a small exported Wait(ctx, d). Whichever is chosen, a delay that would end after the context's deadline should stop the loop at once rather than sleep until the deadline passes.

2. Loop shape. Choose A, B or C. Also decide how the caller learns why the loop stopped without losing the last attempt's error. It may have run out of attempts, been denied by the budget, been classified as final, or seen its context end during a wait. cenkalti's *RetryError{LastErr, Cause}, added in v6, is prior art.

3. Classifying.

   • It's required, with no default, as in circuit.
   • The outcomes are at least success, retry and stop, plus a minimum delay.
   • Either the classifier is configured once in New, as circuit's is, or the decision is passed inline at each attempt, where the request is in scope.
   • Either way, the budget has to see every result, first attempts included.
   • Context errors need the same care as in circuit. The caller cancelling means stop. A deadline on one attempt is retryable, and the deadline on the whole call is not.

4. Budget. Decide whether it's in v1. The recommendation is yes, for the reasons in section 3. Then choose the kind:

   • a token bucket after gRFC A6, which holds nothing and needs no Release;
   • an in-flight rate, which holds a slot for each retry and brings the whole circuit idiom with it.

   Either way, its parameters come from configuration and are validated as circuit's are.

5. Schedule. It's a caller-supplied func(attempt int) time.Duration, as circuit ships no Policy. Recommendation: also ship one exponential, full-jitter constructor in which every parameter is required.

   • Unlike a trip policy, the formula isn't an org's convention.
   • The arithmetic is easy to get wrong: cenkalti v7.0.1 fixed ExponentialBackOff returning a negative delay on overflow.
   • If it ships, decide how a test pins the jitter.

6. Shared versus per-call state. The schedule, the budget and the classifier are built once at startup and shared. The attempt count belongs to one call. sethvargo/go-retry's shared WithMaxRetries counter is the trap to avoid.

7. Validation, observer and clock. These follow circuit:

   • nil functions panic;
   • New returns one joined error naming every invalid setting;
   • time comes from time.Now, with no injected clock, and tests run under testing/synctest.

   The caller already sees every attempt and every decision inline, so an observer may not be needed, just as circuit doesn't observe results. A read of the budget's state, like Breaker.State(), may be all a gauge needs.

[Zaba505]

Proposed API: the iterator, a user-provided classifier and a retry budget

This proposes an API for retry. It builds on the two decisions settled after the review:

• The loop: a call is retried with the range-over-func loop, option C.
• The classifier: the error-checking function is user-provided, so retry stays general and never imports circuit. A framework's classifier returns Stop for circuit's rejections.

It answers the rest of the open questions. The four that needed the maintainer are settled too, under "Decided".

How it was produced:

• A prototype was built to the settled shape.
• Two independent critics reviewed it: one tried to break it on correctness and concurrency, and the other checked the API against bedrock's principles and circuit's conventions.
• Every finding they demonstrated was fixed. The fixes are listed under "What the review changed".

How it was checked:

• Imports: only context errors fmt iter math math/rand/v2 sync sync/atomic time.
• Formatting and vet: gofmt is clean and go vet passes.
• golangci-lint: the repo's config, plus a retry/** depguard rule that allows only the standard library, reports 0 issues. The root module, fit and circuit stay clean under the changed config.
• Tests: go test -race -cover reports 100.0% statement coverage, with every function at 100%. 93 tests and examples pass, three runs in a row.
• The critic's adversarial suite passes, including 20,000 iterations racing Done against the end of an attempt. Four of its tests fail, but each asserted behaviour that the review changed on purpose.
• humus sketch: it builds on the published bedrock v0.23.1 and circuit/v0.23.1, with no change to retry's API. It covers settings read through config.Struct, an HTTP classifier that stops on circuit.ErrRejected, and an http.RoundTripper that retries each request inline, with each attempt going through a breaker. Twelve scenarios pass under testing/synctest and -race.

Both are prototypes. Nothing is in the repository yet.

The API at a glance

Module github.com/z5labs/bedrock/retry, in directory retry/, nested like fit/ and circuit/:

type Outcome int  // Success, Retry, Stop; the zero value is invalid
type Decision int // Again, Succeeded, Stopped, Exhausted, Throttled, Expired; Done never returns zero

const TokenLimit = 1 << 16
type BudgetConfig struct { MaxTokens int; TokenRatio float64 } // both required
func NewBudget(cfg BudgetConfig) (*Budget, error)
func (b *Budget) Tokens() float64 // a pure read, for a gauge

func Exponential(base, maxDelay time.Duration) (func(attempt int) time.Duration, error)

func New(maxAttempts int,
	schedule func(attempt int) time.Duration,
	classify func(context.Context, error) (Outcome, time.Duration),
	budget *Budget) (*Retrier, error) // budget may be nil

func (r *Retrier) Attempts(ctx context.Context) iter.Seq[*Attempt]
func (a *Attempt) Done(err error) Decision
func (a *Attempt) Number() int // from 1

The one idiom

var err error
for a := range r.Attempts(ctx) {
	err = call(ctx)
	a.Done(err)
}
return err

• Every attempt reaches Done before the loop body ends, whether the body then continues, breaks or returns. An attempt that ends without Done panics.
  • circuit can't check this, but the iterator sees the body return, so a lost result is caught instead of silently starving the budget.
  • A body that panics or calls runtime.Goexit is the exception. The panic or Goexit passes through unchanged and ends its attempt.
• After Again, the iterator waits max(schedule(n), the classifier's delay) on a timer, unless ctx ends first.
• What only the caller knows stays in the caller's code:
  • a request that isn't idempotent breaks out after Done, which skips the wait;
  • a body is rewound, and the previous result cleaned up, when Number() > 1;
  • a timeout for each attempt is context.WithTimeout inside the loop.

Composing with circuit

From the humus sketch. The breaker goes inside the loop, so each attempt is one call the breaker admits or rejects. The breaker's own idiom sits in a function per attempt, so its deferred Release runs once per attempt:

func ClassifyHTTP(ctx context.Context, err error) (retry.Outcome, time.Duration) {
	switch {
	case err == nil:
		return retry.Success, 0
	case errors.Is(err, circuit.ErrRejected):
		return retry.Stop, 0 // the breaker is telling the caller not to call
	case errors.Is(err, ErrRewind):
		return retry.Stop, 0
	case ctx.Err() != nil:
		return retry.Stop, 0
	}
	if se, ok := errors.AsType[*StatusError](err); ok {
		switch se.Code {
		case 429, 502, 503, 504:
			return retry.Retry, se.RetryAfter
		}
		return retry.Stop, 0
	}
	return retry.Retry, 0 // a transport error
}

func (t Transport) RoundTrip(req *http.Request) (*http.Response, error) {
	ctx := req.Context()
	retryable := idempotent(req) && (req.Body == nil || req.Body == http.NoBody || req.GetBody != nil)

	var resp *http.Response
	var err error
	for a := range t.Retrier.Attempts(ctx) {
		attemptReq := req
		if a.Number() > 1 {
			// Only now is the last response replaced: if the context had
			// ended during the wait, the loop would have ended with it.
			discard(resp)
			resp = nil
			if attemptReq, err = rewind(req); err != nil {
				a.Done(err) // an ErrRewind, which ClassifyHTTP stops on
				break
			}
		}

		resp, err = t.attempt(attemptReq)
		if a.Done(result(resp, err)) != retry.Again || !retryable {
			break
		}
	}
	return resp, err
}

func (t Transport) attempt(req *http.Request) (*http.Response, error) {
	ticket, err := t.Breaker.Allow(req.Context())
	if err != nil {
		return nil, err // a *circuit.RejectedError
	}
	defer ticket.Release()

	resp, err := t.Next.RoundTrip(req)
	ticket.Done(result(resp, err))
	return resp, err
}

A test pins the composition.

• Two 503s open the breaker.
• The third attempt is rejected without reaching the server. The classifier stops there, with no further wait.
• RoundTrip returns the *circuit.RejectedError, which still matches circuit.ErrRejected through http.Client's *url.Error.

Answers

1. What the classifier returns

• Decision: (Outcome, time.Duration).
  • The Outcome is Success, Retry or Stop. The zero value panics, as circuit's does.
  • The duration is the least time to wait before a Retry, such as Retry-After. The delay is max(schedule(n), it).
  • Success and Stop ignore the duration. With Retry, a negative one panics, so a value computed from a server's date has to be clamped first.
• Rejected:
  • a RetryAfter() method discovered on the error, which is implicit;
  • wrapping errors, as in cenkalti's RetryAfter, which rewrites the caller's errors.
• Known difference: gRFC A6's pushback replaces the backoff, while max treats it as a minimum.

2. How the caller learns why the loop stopped

• Decision: Done returns a Decision, either Again or one of five reasons: Succeeded, Stopped, Exhausted, Throttled or Expired.
  • The caller's err holds the last result. retry never wraps or replaces it.
  • Metrics for Throttled and Exhausted come from the Decision. Outcome metrics come from wrapping the classifier. There's no observer, for circuit's reason: the caller already sees everything.
• The one case Done can't report: ctx ending during the wait after Again. The loop then ends with Again as its last Decision, although the body didn't break, and ctx.Err() is not nil.
  • That's why the previous result is cleaned up when the next attempt starts, not when Done returns Again. A call whose wait is cut short still ends with its last result.
  • The startup idiom is if last == retry.Again { return context.Cause(ctx) }.

3. Retry budget

• Decision: a token bucket, after gRFC A6.
  • Sharing: it's a separate type, shared between Retriers, because A6 throttles per server while retry policy is set per method.
  • nil means no budget, for startup retries. It's visible at the call site.
• Accounting:
  • Each Retry takes one token, even on the last attempt, as grpc-go does.
  • Each Success adds TokenRatio, and Stop changes nothing.
  • A Retry whose ctx is done costs nothing and is Expired, because a caller giving up says nothing about the dependency.
  • Retries stop while the bucket holds half of MaxTokens or fewer. First attempts are never limited.
• Validation:
  • MaxTokens is 3 to TokenLimit, because 1 or 2 tokens can never allow a retry.
  • TokenRatio is 0.001 to TokenLimit, rounded to thousandths. Tokens are counted in thousandths, so 0.1 adds up exactly.
• Rejected: an in-flight rate, as in failsafe-go. It holds a slot for each running retry, which would need a Release.

4. Schedule

• Decision: a caller-supplied func(attempt int) time.Duration. It depends only on the attempt number, so one schedule is shared by every call.
• Exponential ships with full jitter:
  • the delay after attempt n is uniform in [0, min(maxDelay, base·2ⁿ⁻¹));
  • the shift never overflows, checked against math/big up to MaxInt64;
  • both arguments are required, and invalid ones return joined errors.
  • It draws from math/rand/v2's top-level functions, so tests pass their own schedule.
• Rejected: decorrelated jitter, which needs state for each call.

5. Validation, observer and clock
These follow circuit.

• Validation:
  • Nil functions panic.
  • Configuration errors are returned: New for maxAttempts < 1, NewBudget and Exponential, each joined.
  • math.MaxInt attempts means "until the context ends", which is explicit. There's no 0-means-unlimited.
• Observer: there's none, as covered in answer 2.
• Clock: time.Now, and one timer per wait. There are no goroutines, and tests run under testing/synctest.

What the review changed

1. A late Done after the loop body panicked or called runtime.Goexit decided Again and charged the Budget. A test proved it. The iterator now ends the attempt as the panic or Goexit passes through, with no recover.
2. A negative delay with Success or Stop panicked, contradicting the docs. It's now checked only with Retry.
3. Callers that gave up drained the shared Budget. Six cancelled calls cost six tokens and throttled healthy callers. They now decide Expired without taking a token.
4. break or return without Done silently starved the Budget. Ten calls that each returned their success before reporting it drained a budget to 0, with no panic. Ending an attempt without Done now always panics.
5. "Clean up on Again" was the wrong advice, because the wait can be cut short. The previous result is now cleaned up when the next attempt starts.
6. A MaxTokens of 1 or 2 could never allow a retry. It's rejected.
7. Docs:
   • the order of Expired;
   • a cancellation timed to the instant a wait ends;
   • the := shadowing trap;
   • when a call is committed;
   • a startup example.

Concerns

• The loop body compiles to a function the iterator calls. That's the shape A circuit breaker should be provided by bedrock #595 ruled out for circuit, and it's settled for retry. It's also what lets retry catch an attempt that ends without Done.
• Again is provisional. The wait can be cut short, so the caller cleans up a result when the next attempt starts, not on Again. That keeps an HTTP connection busy during the wait, unless humus buffers a small body and closes the connection.
• := in the loop body declares new variables, so the function returns the outer ones, which the loop never set. Nothing catches this; the docs warn about it.
• A negative delay with Retry panics on the request path. That happens if a classifier forgets to clamp a server's date. It's decided: it panics, and the docs say to clamp.
• There's no Attempt.Delay() yet, so humus can't log "retrying in X". It can be added later without breaking anything.
• grpc-go already retries, with A6 throttling, from the service config. humus should use one or the other for a given client, not both.

Decided

The maintainer settled these four, each as recommended:

1. A negative delay with Retry panics, rather than being clamped to 0 as cenkalti v7.0.1 does. It fails loudly, and the docs say to clamp a value built from a header.
2. No budget is a nil *Budget, not an explicit "unlimited" value. It's visible at the call site, and a sentinel would add an exported name.
3. New(maxAttempts int, …) stays positional, rather than taking Config{MaxAttempts} for parity with circuit. A Config with one field is ceremony, and circuit's exists because four fields are validated together.
4. Attempt.Delay() waits until humus asks for it. It can be added later without breaking anything.

Adoption cost

A fourth module:

• per-module build, vet, test and lint steps in build.yaml, and in CLAUDE.md's commands;
• a depguard rule that allows only $gostd for retry/**, and the bedrock rule extended to exclude it;
• the coverage gate;
• the Package Layout and Conventions sections of CLAUDE.md, and a README section;
• nothing for releases, because tag-modules.sh already finds every nested go.mod.

Full API with doc comments (go doc -all)

package retry // import "github.com/z5labs/bedrock/retry"

Package retry retries calls to a dependency.

# Calls

A call is retried inline, in the caller's own code, by ranging over the attempts
that Retrier.Attempts yields. Each attempt makes the call and reports its result
with Attempt.Done:

    var err error
    for a := range r.Attempts(ctx) {
    	err = call(ctx)
    	a.Done(err)
    }
    return err

Done classifies the result and returns a Decision: Again when another attempt
follows, or the reason none does. After Again, the next attempt is yielded once
the delay has passed, unless ctx ends first. When the loop ends, err holds the
result of the last attempt.

Every attempt must reach Done before the loop body ends, whether the body
then continues, breaks or returns, so that every result is classified.
A Success that never reaches Done never refills the Budget, and a failure that
never reaches it is never counted, so an attempt that ends without Done panics.
Call Done straight after the call, before any check of its error. Assign the
call's results to variables declared before the loop: a := inside the loop body
declares new ones, and the function returns the outer ones, which the loop never
set.

Because the loop is the caller's own code, the caller decides what only it knows
about a call:

  - a call that must not be retried, such as a request that is not idempotent,
    breaks out of the loop after Done, whatever Done decided;
  - a request that must be prepared again for a retry, such as one whose
    body must be rewound, is prepared when Attempt.Number is greater than 1,
    and the result of the attempt before it, such as the body of a response,
    is cleaned up then too;
  - a deadline for each attempt is set inside the loop with context.WithTimeout.

Again is what Done decides, not yet what happens. The context may end during
the wait that follows, and then the loop ends with no further attempt: the last
Decision was Again, although the loop body did not break, and ctx.Err() is not
nil. A result is therefore cleaned up when the attempt that replaces it starts,
not when Done returns Again, so a call whose wait is cut short still ends with
its last result.

A call can be retried only until its result is handed to its caller: once the
caller reads a response body or a stream, the call is committed, and a failure
after that is the caller's to handle.

Each call belongs either at startup or on the request path:

    | Call                                       | When                                         |
    |--------------------------------------------|----------------------------------------------|
    | Exponential, NewBudget, New                | Startup, inside App.Run, once per dependency |
    | ranging over Attempts, with Done each time | Per outbound call                            |
    | the classifier and the schedule            | Per Done, with nothing locked                |
    | Budget.Tokens                              | Whenever, such as in a metrics callback      |

One Retrier is shared by every goroutine that calls its dependency, so the
classifier and the schedule must be safe for concurrent use.

# Classifying results

retry does not decide what can be retried. Every result passed to Done goes
to the classify function passed to New, together with the context passed to
Attempts, and classify returns an Outcome:

  - Success: the call succeeded, and no attempt follows;
  - Retry: the call failed in a way another attempt may fix;
  - Stop: the call failed in a way another attempt will not fix, or must not try
    to.

For Retry, classify also returns the least time to wait before the next attempt,
such as a server's Retry-After, or 0. retry has no default classifier and treats
no error specially, a nil error and the context errors included.

The context passed to classify is the context of the whole call, so ctx.Err()
tells the caller giving up from an attempt failing:

  - ctx.Err() is not nil: the caller cancelled the call or its deadline ran out,
    and another attempt cannot help. Done decides Expired for a Retry here,
    and takes no token from the Budget for it, because the caller giving up says
    nothing about the dependency; a classifier that returns Stop says the same
    thing directly;
  - ctx.Err() is nil, and err wraps context.DeadlineExceeded: a deadline set for
    that one attempt ran out, which a retry may fix.

A result that is not an error, such as an HTTP status code, reaches classify as
an error value that the caller builds.

# Composing with a circuit breaker

A circuit breaker goes inside the retry loop, so that each attempt is a call
the breaker admits or rejects. A breaker that rejects a call is telling its
caller not to call, so retrying the rejection at once defeats the breaker.
retry imports no breaker. The caller's classifier returns Stop for the breaker's
rejection, such as an error that matches github.com/z5labs/bedrock/circuit's
ErrRejected, and the attempt makes its call through the breaker in a function of
its own, so that the breaker's deferred guard runs once per attempt:

    for a := range r.Attempts(ctx) {
    	err = callThrough(ctx, b) // b.Allow, defer t.Release, the call, t.Done
    	a.Done(err)
    }

# Limiting retries

Retries add load to a dependency when it can least take it. A limit on attempts
per call bounds one call, but not every call together: when a dependency fails,
every caller retries at once, and a service that retries at each of several
layers multiplies its attempts. A Budget bounds every call that shares it,
as gRPC's retry throttling does: each attempt classified Retry takes a token,
each Success adds a fraction of one back, and retries stop while half the tokens
or fewer are left. First attempts are never limited, so a call is never refused
for lack of budget, only its retries.

A Budget belongs to calls on the request path. A Retrier that retries a
connection at startup, until a deadline or a signal stops it, has no Budget:
pass nil to New, and math.MaxInt attempts, so that only the context ends the
call, as New shows.

# Schedules

The delay before each retry comes from the schedule passed to New, a function
from the number of the attempt that failed to a delay. Exponential builds the
usual one, exponential backoff with full jitter. A schedule is a function of the
attempt number alone, so one schedule is shared by every call.

# Panics

retry never recovers a panic. A panic in the loop body, or a call to
runtime.Goexit, passes through the iterator unchanged and ends the attempt it
was in, and retry holds nothing between attempts, so nothing is left behind.
Misuse panics too, because it is a bug in the caller: an attempt that ends
without Done, Done called twice or after its attempt ended, and a classifier or
schedule that returns a value that is out of range.

# Time

retry reads time.Now and sets a timer for each wait, and nothing else:
no clock is injected, and no goroutine runs in the background. A test runs a
Retrier inside a testing/synctest bubble, where the waits take no real time. A
cancellation timed to the very instant a wait ends may land before or after the
next attempt starts, so a test cancels before or after that instant, not at it.

# Out of scope

retry does not:

  - hedge calls, time out attempts, rate limit or bound the concurrency of
    calls;
  - share a Budget's tokens between processes: each process has its own Budgets;
  - keep a registry of named Retriers: the caller holds each Retrier it builds;
  - replay request bodies or parse Retry-After or gRPC pushback, which belong to
    the framework that wraps a transport;
  - ship classifiers.

CONSTANTS

const TokenLimit = 1 << 16
    TokenLimit is the largest BudgetConfig.MaxTokens and BudgetConfig.TokenRatio
    that NewBudget accepts. The bound turns a typo in configuration into an
    error from NewBudget, and keeps the count of tokens exact.


FUNCTIONS

func Exponential(base, maxDelay time.Duration) (func(attempt int) time.Duration, error)
    Exponential returns a schedule for New that backs off exponentially
    with full jitter: the delay after attempt n is drawn uniformly from [0,
    min(maxDelay, base·2ⁿ⁻¹)). Full jitter spreads out the retries of callers
    that failed together, so they do not all try again at once.

    The schedule draws from math/rand/v2's top-level functions, so it is safe
    for concurrent use, and one schedule may be shared by every call. It panics
    if it is called with an attempt less than 1.

    Exponential returns an error naming every invalid argument, because both
    come from configuration: base must be positive, and maxDelay at least base.


TYPES

type Attempt struct {
	// Has unexported fields.
}
    Attempt is one attempt of a call, yielded by Retrier.Attempts. Every
    attempt reports its result with Done, once, before the loop body ends,
    whether the body then continues, breaks or returns. An attempt that ends
    without Done panics, because a result that is never reported is lost: a
    lost Success never refills the Budget, and a lost failure is never counted.
    The exception is a loop body that panics or calls runtime.Goexit, which ends
    its attempt as it passes through.

func (a *Attempt) Done(err error) Decision
    Done reports the result of a's call, and returns what happens next.
    It classifies err with the context passed to Retrier.Attempts, updates the
    Budget, and decides whether another attempt follows:

      - Success: Succeeded, and the Budget gains BudgetConfig.TokenRatio;
      - Stop: Stopped, and the Budget is unchanged;
      - Retry, with the context done: Expired, and the Budget is unchanged,
        because the caller giving up says nothing about the dependency;
      - Retry otherwise: one token is taken from the Budget, and then, in this
        order, Exhausted if a was the last attempt allowed, Throttled if the
        Budget allows no retry, Expired if the context's deadline comes before
        the delay would end, and Again otherwise.

    Call Done straight after the call, with its error, before any check of the
    error, so that every result is classified. A caller that must not retry this
    particular call, such as one whose request is not idempotent, breaks out of
    the loop after Done, which skips the wait.

    Again is what Done decides, not yet what happens: the wait that follows may
    be cut short by the context, and then no attempt follows. A result that
    the next attempt replaces, such as the body of a response that is retried,
    is therefore cleaned up when the next attempt starts, when Number is greater
    than 1, so that a call whose wait is cut short still ends with its last
    result.

    Done panics if it is called twice for one attempt, or after its
    attempt ended by a panic or runtime.Goexit, or if a was not yielded by
    Retrier.Attempts. It panics too if the classifier returns a value that is
    not an Outcome, or a negative delay with Retry, or if the schedule returns
    a negative delay. A panic from the classifier leaves the Budget as it was.
    One from the schedule comes after the Budget counted the Retry.

func (a *Attempt) Number() int
    Number returns the number of a, from 1. A caller that must prepare a request
    again for each retry, such as one that rewinds an HTTP request body, does so
    when Number is greater than 1.

    Number panics if a was not yielded by Retrier.Attempts.

type Budget struct {
	// Has unexported fields.
}
    Budget limits the retries of every call that shares it, as gRPC's retry
    throttling does (gRFC A6). It starts with BudgetConfig.MaxTokens tokens.
    Every attempt classified Retry takes one, down to zero, and every attempt
    classified Success adds BudgetConfig.TokenRatio, up to MaxTokens. Stop
    changes nothing. An attempt classified Retry is followed by another only
    while the Budget holds more than half of MaxTokens once its token is taken.

    Share one Budget between every Retrier that calls the same dependency.
    While the dependency fails, the Budget runs down and retries stop,
    so a failing dependency gets at most one attempt per call instead of one per
    allowed attempt. First attempts are never limited.

    Its fields are unexported, so NewBudget is the only way to build one,
    and every method of a Budget not built by NewBudget panics.

func NewBudget(cfg BudgetConfig) (*Budget, error)
    NewBudget returns a full Budget with the settings cfg. It returns an error
    naming every invalid field of cfg, because cfg comes from configuration and
    App.Run should exit non-zero at startup instead.

func (b *Budget) Tokens() float64
    Tokens returns how many tokens b holds. It changes nothing and calls no
    function of the caller's, so a metrics callback may call it.

    Tokens panics if b was not built by NewBudget.

type BudgetConfig struct {
	// MaxTokens is how many tokens a full Budget holds, from 3 to
	// TokenLimit. A Budget starts full, and allows no retry while it holds
	// half of MaxTokens or fewer, so a Budget of 1 or 2 tokens, which one
	// failure takes to half, would never allow a retry.
	MaxTokens int

	// TokenRatio is how many tokens each Success adds, from 0.001 to
	// TokenLimit. It is rounded to the nearest thousandth of a token. A
	// TokenRatio of MaxTokens or more refills the Budget on every Success.
	TokenRatio float64
}
    BudgetConfig holds the settings of a Budget that come from configuration.
    Both fields are required: NewBudget rejects a zero or out-of-range value
    rather than replacing it with a default.

type Decision int
    Decision is what Attempt.Done decides after an attempt: Again, or the reason
    no attempt follows. Done never returns the zero Decision.

const (
	// Again means another attempt follows once the delay has passed, unless
	// the context ends first.
	Again Decision = iota + 1

	// Succeeded means the classifier returned Success.
	Succeeded

	// Stopped means the classifier returned Stop.
	Stopped

	// Exhausted means the classifier returned Retry for the last attempt the
	// Retrier allows.
	Exhausted

	// Throttled means the classifier returned Retry, and the Budget allows
	// no retry.
	Throttled

	// Expired means the classifier returned Retry, and the context is done,
	// or its deadline comes before the next attempt would start.
	Expired
)
func (d Decision) String() string
    String returns "again", "succeeded", "stopped", "exhausted", "throttled" or
    "expired", or "Decision(n)" for a value that is not a Decision.

type Outcome int
    Outcome is how a Retrier treats the result of one attempt, as the classifier
    passed to New decides it. The zero Outcome is not an outcome, so a
    classifier that returns one by mistake panics instead of deciding silently.

const (
	// Success ends the call: no attempt follows. It adds
	// BudgetConfig.TokenRatio tokens to the Budget.
	Success Outcome = iota + 1

	// Retry is a failure that another attempt may fix, such as a dependency
	// that is briefly unavailable. It takes one token from the Budget, and
	// another attempt follows if the attempt limit, the Budget and the
	// context all allow one.
	Retry

	// Stop is a failure that another attempt will not fix, or must not
	// try to, such as a call rejected by an open circuit breaker. It ends
	// the call, and leaves the Budget as it is.
	Stop
)
func (o Outcome) String() string
    String returns "success", "retry" or "stop", or "Outcome(n)" for a value
    that is not an Outcome.

type Retrier struct {
	// Has unexported fields.
}
    Retrier retries the calls to one dependency. Build one with New at startup
    and share it between every goroutine that calls that dependency. Its fields
    are unexported, so New is the only way to build one, and every method of a
    Retrier not built by New panics.

func New(maxAttempts int, schedule func(attempt int) time.Duration, classify func(context.Context, error) (Outcome, time.Duration), budget *Budget) (*Retrier, error)
    New returns a Retrier that makes at most maxAttempts attempts per call,
    classifies the result of each with classify, waits between attempts as
    schedule says, and limits its retries with budget.

    classify returns the Outcome of an attempt's error and, for Retry, the least
    time to wait before the next attempt, such as a server's Retry-After, or 0
    to wait only as long as schedule says. The delay before the next attempt
    is the larger of the two. For Retry the least time must not be negative,
    so one computed from a server's date is clamped at 0 first; for Success and
    Stop it is ignored. schedule returns the delay after attempt n, from 1,
    as Exponential does. Both must be safe for concurrent use, and both are
    called by Attempt.Done, on the goroutine that calls it, with nothing locked.

    budget may be shared with other Retriers, and may be nil for a Retrier
    whose retries are limited only by maxAttempts, such as one that retries a
    connection at startup. A Retrier with a Budget is for calls on the request
    path, where retries of every caller add up.

    New panics if schedule or classify is nil, which is a programming error.
    It returns an error if maxAttempts is less than 1, because maxAttempts comes
    from configuration and App.Run should exit non-zero at startup instead.
    A maxAttempts of math.MaxInt retries until a limit other than the attempt
    count stops the call.

func (r *Retrier) Attempts(ctx context.Context) iter.Seq[*Attempt]
    Attempts returns the attempts of one call to r's dependency. It is the only
    way to make a call through r, and it sits inline in the caller's code:

        var err error
        for a := range r.Attempts(ctx) {
        	err = call(ctx)
        	a.Done(err)
        }
        return err

    The first attempt is yielded at once. After an attempt for which
    Attempt.Done returned Again, the next one is yielded once the delay has
    passed, unless ctx ends first. The loop ends:

      - after Attempt.Done returns any other Decision;
      - when the loop body breaks or returns after Done, which skips the wait
        after an Again, for a caller that must not retry a particular call;
      - when ctx ends during the wait. Then the last Decision was Again,
        although the loop body did not break, and ctx.Err() is not nil.

    Each range over the sequence is a new call, which starts from attempt 1. ctx
    is the context of the call. The classifier receives it with every result,
    and its deadline ends the call when the next attempt would start too late.

    Attempts panics if ctx is nil, or if r was not built by New.

Examples: a flaky call, a startup connection, a call that must not be retried, a Budget that throttles

package retry_test

import (
	"context"
	"errors"
	"fmt"
	"math"
	"time"

	"github.com/z5labs/bedrock/retry"
)

var errNotFound = errors.New("not found")

// classify retries every failure but two: the caller giving up, and a
// result that another attempt would only repeat.
func classify(ctx context.Context, err error) (retry.Outcome, time.Duration) {
	switch {
	case err == nil:
		return retry.Success, 0
	case ctx.Err() != nil:
		return retry.Stop, 0 // the caller cancelled the call, or its deadline ran out
	case errors.Is(err, errNotFound):
		return retry.Stop, 0
	default:
		return retry.Retry, 0
	}
}

// A call that fails twice and then succeeds. Each attempt reports its result
// with Done, and the loop ends when Done decides that no attempt follows.
func Example() {
	ctx := context.Background()

	// At startup, from configuration. The delays are short so that the
	// example runs quickly.
	schedule, err := retry.Exponential(time.Millisecond, 10*time.Millisecond)
	if err != nil {
		fmt.Println(err)
		return
	}
	budget, err := retry.NewBudget(retry.BudgetConfig{MaxTokens: 10, TokenRatio: 0.1})
	if err != nil {
		fmt.Println(err)
		return
	}
	r, err := retry.New(4, schedule, classify, budget)
	if err != nil {
		fmt.Println(err)
		return
	}

	// Per outbound call.
	unavailable := errors.New("payments: unavailable")
	results := []error{unavailable, unavailable, nil}

	var callErr error
	for a := range r.Attempts(ctx) {
		callErr = results[a.Number()-1]
		fmt.Printf("attempt %d: %v: %v\n", a.Number(), callErr, a.Done(callErr))
	}
	fmt.Println("call:", callErr)
	fmt.Println("tokens:", budget.Tokens())
	// Output:
	// attempt 1: payments: unavailable: again
	// attempt 2: payments: unavailable: again
	// attempt 3: <nil>: succeeded
	// call: <nil>
	// tokens: 8.1
}

// Retrying a connection at startup, until it succeeds or the context ends.
// Nothing else is calling the dependency yet, so there is no Budget, and
// math.MaxInt attempts leave the context as the only limit. In App.Run the
// context comes from p.NotifyContext, so a SIGTERM ends the wait.
func ExampleNew() {
	ctx, cancel := context.WithTimeout(context.Background(), time.Minute)
	defer cancel()

	schedule, err := retry.Exponential(time.Millisecond, 5*time.Millisecond)
	if err != nil {
		fmt.Println(err)
		return
	}
	r, err := retry.New(math.MaxInt, schedule, func(_ context.Context, err error) (retry.Outcome, time.Duration) {
		if err == nil {
			return retry.Success, 0
		}
		return retry.Retry, 0
	}, nil)
	if err != nil {
		fmt.Println(err)
		return
	}

	refused := errors.New("dial tcp 10.0.0.7:5432: connection refused")
	tries := 0
	connect := func(context.Context) error {
		tries++
		if tries < 4 {
			return refused
		}
		return nil
	}

	var connErr error
	var last retry.Decision
	for a := range r.Attempts(ctx) {
		connErr = connect(ctx)
		last = a.Done(connErr)
	}
	if last == retry.Again {
		// The loop ended after an Again without breaking: the context
		// ended during the wait.
		fmt.Println("gave up:", context.Cause(ctx))
		return
	}
	fmt.Printf("connected after %d attempts: %v\n", tries, connErr)
	// Output:
	// connected after 4 attempts: <nil>
}

// A call that must not be made twice, such as a payment, still reports its
// result, so that the Budget counts it, and then breaks out of the loop
// instead of retrying, whatever Done decided.
func ExampleAttempt_Done() {
	r, err := retry.New(3, func(int) time.Duration { return time.Second }, classify, nil)
	if err != nil {
		fmt.Println(err)
		return
	}

	var callErr error
	for a := range r.Attempts(context.Background()) {
		callErr = errors.New("payments: connection reset")
		fmt.Println("decision:", a.Done(callErr))
		break // the payment may have been made
	}
	fmt.Println("call:", callErr)
	// Output:
	// decision: again
	// call: payments: connection reset
}

// A dependency that is down runs the shared Budget down, and from then on
// each call makes one attempt instead of five.
func ExampleNewBudget() {
	budget, err := retry.NewBudget(retry.BudgetConfig{MaxTokens: 4, TokenRatio: 0.5})
	if err != nil {
		fmt.Println(err)
		return
	}
	r, err := retry.New(5, func(int) time.Duration { return 0 }, classify, budget)
	if err != nil {
		fmt.Println(err)
		return
	}

	down := errors.New("inventory: unavailable")
	for i := range 3 {
		var decisions []retry.Decision
		for a := range r.Attempts(context.Background()) {
			decisions = append(decisions, a.Done(down))
		}
		fmt.Printf("call %d: %v, tokens left: %v\n", i+1, decisions, budget.Tokens())
	}
	// Output:
	// call 1: [again throttled], tokens left: 2
	// call 2: [throttled], tokens left: 1
	// call 3: [throttled], tokens left: 0
}

// Configuration that is out of range is an error at startup, naming every
// setting that is wrong.
func ExampleExponential() {
	_, err := retry.Exponential(0, -time.Second)
	fmt.Println(err)
	// Output:
	// retry: base delay must be positive, got 0s
	// retry: max delay must be at least the base delay, 0s, got -1s
}

humus sketch: config.Struct settings, an HTTP classifier, and a RoundTripper composed with circuit

// Package humusretry sketches the wiring z5labs/humus builds on
// github.com/z5labs/bedrock/retry, using only its exported API, together with
// github.com/z5labs/bedrock/circuit: settings read with config.Struct, an
// HTTP classifier that stops on a circuit rejection, and an
// http.RoundTripper that retries each request inline, with each attempt
// going through a circuit breaker.
package humusretry

import (
	"context"
	"errors"
	"fmt"
	"io"
	"net/http"
	"strconv"
	"time"

	"github.com/z5labs/bedrock"
	"github.com/z5labs/bedrock/circuit"
	"github.com/z5labs/bedrock/config"
	"github.com/z5labs/bedrock/retry"
)

// ---------------------------------------------------------------------------
// Configuration. Any default is humus's to write out with .Default(...);
// bedrock has none.

// Settings is everything a humus retrier reads from its environment.
type Settings struct {
	MaxAttempts int
	BaseDelay   time.Duration
	MaxDelay    time.Duration
	Budget      retry.BudgetConfig
}

// ReadSettings reads the Settings of the retrier whose variables start with
// prefix, such as "PAYMENTS_RETRY_".
func ReadSettings(env bedrock.Environment, prefix string) config.Value[Settings] {
	return config.Struct(func(r *config.Resolver) Settings {
		v := func(name string) config.Value[string] { return config.Env(env, prefix+name) }
		return Settings{
			MaxAttempts: r.Get(v("MAX_ATTEMPTS").Parse(strconv.Atoi)),
			BaseDelay:   r.Get(v("BASE_DELAY").Parse(time.ParseDuration)),
			MaxDelay:    r.Get(v("MAX_DELAY").Parse(time.ParseDuration)),
			Budget: retry.BudgetConfig{
				MaxTokens:  r.Get(v("BUDGET_MAX_TOKENS").Parse(strconv.Atoi)),
				TokenRatio: r.Get(v("BUDGET_TOKEN_RATIO").Parse(config.ParseFloat64)),
			},
		}
	})
}

// NewRetrier builds a Retrier from s, with its own Budget, classifying with
// ClassifyHTTP. It is called once per dependency in App.Run.
func NewRetrier(s Settings) (*retry.Retrier, error) {
	schedule, scheduleErr := retry.Exponential(s.BaseDelay, s.MaxDelay)
	budget, budgetErr := retry.NewBudget(s.Budget)
	if err := errors.Join(scheduleErr, budgetErr); err != nil {
		return nil, err
	}
	return retry.New(s.MaxAttempts, schedule, ClassifyHTTP, budget)
}

// ---------------------------------------------------------------------------
// Classifier.

// StatusError is the result an attempt reports for a response with a status
// of 400 or more, so that the classifier decides which statuses count.
// RoundTrip still returns the response to its caller when it is not retried.
type StatusError struct {
	Code       int
	RetryAfter time.Duration // from the Retry-After header, or 0
}

func (e *StatusError) Error() string { return "http: status " + strconv.Itoa(e.Code) }

// ErrRewind is the result of an attempt whose request body could not be
// read again, so no request was sent.
var ErrRewind = errors.New("humusretry: request body could not be rewound")

// ClassifyHTTP retries transport errors and the statuses that mean "try
// again": 429, 502, 503 and 504, waiting at least as long as the server's
// Retry-After. It stops on a rejection by a circuit breaker, which is
// telling the caller not to call, on the caller giving up, on a body that
// could not be rewound, and on every other status, which another attempt
// would only repeat.
func ClassifyHTTP(ctx context.Context, err error) (retry.Outcome, time.Duration) {
	switch {
	case err == nil:
		return retry.Success, 0
	case errors.Is(err, circuit.ErrRejected):
		return retry.Stop, 0
	case errors.Is(err, ErrRewind):
		return retry.Stop, 0
	case ctx.Err() != nil:
		return retry.Stop, 0
	}
	if se, ok := errors.AsType[*StatusError](err); ok {
		switch se.Code {
		case http.StatusTooManyRequests, http.StatusBadGateway, http.StatusServiceUnavailable, http.StatusGatewayTimeout:
			return retry.Retry, se.RetryAfter
		}
		return retry.Stop, 0
	}
	return retry.Retry, 0 // a transport error, such as a reset connection
}

// ---------------------------------------------------------------------------
// net/http.

// Transport is an http.RoundTripper that retries requests with Retrier, and
// sends each attempt through Breaker.
type Transport struct {
	Retrier *retry.Retrier
	Breaker *circuit.Breaker
	Next    http.RoundTripper
}

// RoundTrip implements http.RoundTripper. A request is retried only if it
// is idempotent, by its method or an Idempotency-Key header, and its body
// can be sent again. Any other request still makes its one attempt through
// the Retrier, so that its result counts in the Budget.
func (t Transport) RoundTrip(req *http.Request) (*http.Response, error) {
	ctx := req.Context()
	retryable := idempotent(req) && (req.Body == nil || req.Body == http.NoBody || req.GetBody != nil)

	var resp *http.Response
	var err error
	for a := range t.Retrier.Attempts(ctx) {
		attemptReq := req
		if a.Number() > 1 {
			// Only now is the last response replaced: if the context had
			// ended during the wait, the loop would have ended with it.
			discard(resp)
			resp = nil
			if attemptReq, err = rewind(req); err != nil {
				a.Done(err) // an ErrRewind, which ClassifyHTTP stops on
				break
			}
		}

		resp, err = t.attempt(attemptReq)
		if a.Done(result(resp, err)) != retry.Again || !retryable {
			break
		}
	}
	return resp, err
}

// attempt sends req through the Breaker. The breaker settles on the status
// line here; the #595 sketch shows settling on the body instead.
func (t Transport) attempt(req *http.Request) (*http.Response, error) {
	ticket, err := t.Breaker.Allow(req.Context())
	if err != nil {
		return nil, err // a *circuit.RejectedError
	}
	defer ticket.Release()

	resp, err := t.Next.RoundTrip(req)
	ticket.Done(result(resp, err))
	return resp, err
}

// result is what an attempt reports to the Retrier and the Breaker.
func result(resp *http.Response, err error) error {
	if err != nil || resp.StatusCode < 400 {
		return err
	}
	return &StatusError{Code: resp.StatusCode, RetryAfter: retryAfter(resp.Header.Get("Retry-After"))}
}

// retryAfter parses a Retry-After header, in seconds or as an HTTP date.
func retryAfter(v string) time.Duration {
	if secs, err := strconv.Atoi(v); err == nil && secs >= 0 {
		return time.Duration(secs) * time.Second
	}
	if at, err := http.ParseTime(v); err == nil {
		return max(0, time.Until(at))
	}
	return 0
}

func idempotent(req *http.Request) bool {
	switch req.Method {
	case "", http.MethodGet, http.MethodHead, http.MethodOptions, http.MethodTrace, http.MethodPut, http.MethodDelete:
		return true
	}
	return req.Header.Get("Idempotency-Key") != ""
}

// rewind returns a copy of req with a fresh body, for a retry.
func rewind(req *http.Request) (*http.Request, error) {
	r := req.Clone(req.Context())
	if req.GetBody != nil {
		body, err := req.GetBody()
		if err != nil {
			return nil, fmt.Errorf("%w: %w", ErrRewind, err)
		}
		r.Body = body
	}
	return r, nil
}

// discard drains a little of a response that will not be returned, so that
// its connection can be reused, and closes it.
func discard(resp *http.Response) {
	if resp == nil {
		return
	}
	_, _ = io.CopyN(io.Discard, resp.Body, 4<<10)
	_ = resp.Body.Close()
}

[Zaba505]

Prototype source for the proposal above

The proposal above describes the prototype, and this is its library source, so that the stories can include it. The prototype lives in a session scratchpad, so this comment is its durable copy. The tests aren't included. They're 93 tests and examples at 100% statement coverage, and the stories will ask for that coverage again.

retry/go.mod:

module github.com/z5labs/bedrock/retry

go 1.27.0

require github.com/stretchr/testify v1.12.1

require go.yaml.in/yaml/v3 v3.0.5 // indirect

The .golangci.yaml change: the bedrock rule excludes retry/** and denies the retry module, and a new rule allows only the standard library:

@@ -22,6 +22,7 @@
             - "!$test"
             - "!${base-path}/fit/**"
             - "!${base-path}/circuit/**"
+            - "!${base-path}/retry/**"
           allow:
             - $gostd
             - github.com/z5labs/bedrock
@@ -30,6 +31,8 @@
               desc: the bedrock module does not depend on the fit module
             - pkg: github.com/z5labs/bedrock/circuit
               desc: the bedrock module does not depend on the circuit module
+            - pkg: github.com/z5labs/bedrock/retry
+              desc: the bedrock module does not depend on the retry module
         # Library code of the fit module imports only the standard library
         # and OpenTelemetry's baggage. It does not import the bedrock module.
         # Tests may use any module.
@@ -51,6 +54,16 @@
             - "!$test"
           allow:
             - $gostd
+        # Library code of the retry module imports only the standard
+        # library. It does not import the bedrock module, the fit module or
+        # the circuit module. Tests may use any module.
+        retry:
+          list-mode: strict
+          files:
+            - "${base-path}/retry/**"
+            - "!$test"
+          allow:
+            - $gostd
     forbidigo:
       # Examples are documentation that users copy, so they are held to the
       # same rule as everything else.

retry/retry.go

// Copyright (c) 2026 Z5Labs and Contributors
//
// This software is released under the MIT License.
// https://opensource.org/licenses/MIT

package retry

import "fmt"

// Outcome is how a [Retrier] treats the result of one attempt, as the
// classifier passed to [New] decides it. The zero Outcome is not an outcome,
// so a classifier that returns one by mistake panics instead of deciding
// silently.
type Outcome int

const (
	// Success ends the call: no attempt follows. It adds
	// BudgetConfig.TokenRatio tokens to the Budget.
	Success Outcome = iota + 1

	// Retry is a failure that another attempt may fix, such as a dependency
	// that is briefly unavailable. It takes one token from the Budget, and
	// another attempt follows if the attempt limit, the Budget and the
	// context all allow one.
	Retry

	// Stop is a failure that another attempt will not fix, or must not
	// try to, such as a call rejected by an open circuit breaker. It ends
	// the call, and leaves the Budget as it is.
	Stop
)

// String returns "success", "retry" or "stop", or "Outcome(n)" for a value
// that is not an Outcome.
func (o Outcome) String() string {
	switch o {
	case Success:
		return "success"
	case Retry:
		return "retry"
	case Stop:
		return "stop"
	default:
		return fmt.Sprintf("Outcome(%d)", int(o))
	}
}

// Decision is what [Attempt.Done] decides after an attempt: [Again], or the
// reason no attempt follows. Done never returns the zero Decision.
type Decision int

const (
	// Again means another attempt follows once the delay has passed, unless
	// the context ends first.
	Again Decision = iota + 1

	// Succeeded means the classifier returned Success.
	Succeeded

	// Stopped means the classifier returned Stop.
	Stopped

	// Exhausted means the classifier returned Retry for the last attempt the
	// Retrier allows.
	Exhausted

	// Throttled means the classifier returned Retry, and the Budget allows
	// no retry.
	Throttled

	// Expired means the classifier returned Retry, and the context is done,
	// or its deadline comes before the next attempt would start.
	Expired
)

// String returns "again", "succeeded", "stopped", "exhausted", "throttled"
// or "expired", or "Decision(n)" for a value that is not a Decision.
func (d Decision) String() string {
	switch d {
	case Again:
		return "again"
	case Succeeded:
		return "succeeded"
	case Stopped:
		return "stopped"
	case Exhausted:
		return "exhausted"
	case Throttled:
		return "throttled"
	case Expired:
		return "expired"
	default:
		return fmt.Sprintf("Decision(%d)", int(d))
	}
}

retry/budget.go

// Copyright (c) 2026 Z5Labs and Contributors
//
// This software is released under the MIT License.
// https://opensource.org/licenses/MIT

package retry

import (
	"errors"
	"fmt"
	"math"
	"sync"
)

// TokenLimit is the largest BudgetConfig.MaxTokens and
// BudgetConfig.TokenRatio that NewBudget accepts. The bound turns a typo in
// configuration into an error from NewBudget, and keeps the count of tokens
// exact.
const TokenLimit = 1 << 16

// BudgetConfig holds the settings of a Budget that come from configuration.
// Both fields are required: NewBudget rejects a zero or out-of-range value
// rather than replacing it with a default.
type BudgetConfig struct {
	// MaxTokens is how many tokens a full Budget holds, from 3 to
	// TokenLimit. A Budget starts full, and allows no retry while it holds
	// half of MaxTokens or fewer, so a Budget of 1 or 2 tokens, which one
	// failure takes to half, would never allow a retry.
	MaxTokens int

	// TokenRatio is how many tokens each Success adds, from 0.001 to
	// TokenLimit. It is rounded to the nearest thousandth of a token. A
	// TokenRatio of MaxTokens or more refills the Budget on every Success.
	TokenRatio float64
}

// Budget limits the retries of every call that shares it, as gRPC's retry
// throttling does (gRFC A6). It starts with BudgetConfig.MaxTokens tokens.
// Every attempt classified Retry takes one, down to zero, and every attempt
// classified Success adds BudgetConfig.TokenRatio, up to MaxTokens. Stop
// changes nothing. An attempt classified Retry is followed by another only
// while the Budget holds more than half of MaxTokens once its token is
// taken.
//
// Share one Budget between every Retrier that calls the same dependency.
// While the dependency fails, the Budget runs down and retries stop, so a
// failing dependency gets at most one attempt per call instead of one per
// allowed attempt. First attempts are never limited.
//
// Its fields are unexported, so NewBudget is the only way to build one, and
// every method of a Budget not built by NewBudget panics.
type Budget struct {
	// Tokens are counted in thousandths, so that a TokenRatio such as 0.1
	// adds up exactly.
	max, ratio int64

	mu     sync.Mutex
	tokens int64
}

// NewBudget returns a full Budget with the settings cfg. It returns an error
// naming every invalid field of cfg, because cfg comes from configuration
// and App.Run should exit non-zero at startup instead.
func NewBudget(cfg BudgetConfig) (*Budget, error) {
	var errs []error
	if cfg.MaxTokens < 3 || cfg.MaxTokens > TokenLimit {
		errs = append(errs, fmt.Errorf("retry: max tokens must be between 3 and %d, got %d", TokenLimit, cfg.MaxTokens))
	}
	if !(cfg.TokenRatio >= 0.001 && cfg.TokenRatio <= TokenLimit) {
		errs = append(errs, fmt.Errorf("retry: token ratio must be between 0.001 and %d, got %v", TokenLimit, cfg.TokenRatio))
	}
	if err := errors.Join(errs...); err != nil {
		return nil, err
	}

	full := int64(cfg.MaxTokens) * 1000
	return &Budget{
		max:    full,
		ratio:  int64(math.Round(cfg.TokenRatio * 1000)),
		tokens: full,
	}, nil
}

// Tokens returns how many tokens b holds. It changes nothing and calls no
// function of the caller's, so a metrics callback may call it.
//
// Tokens panics if b was not built by NewBudget.
func (b *Budget) Tokens() float64 {
	b.mustBeBuilt("Tokens")

	b.mu.Lock()
	defer b.mu.Unlock()
	return float64(b.tokens) / 1000
}

// succeeded adds TokenRatio tokens to b, up to MaxTokens.
func (b *Budget) succeeded() {
	b.mu.Lock()
	defer b.mu.Unlock()
	b.tokens = min(b.max, b.tokens+b.ratio)
}

// failed takes one token from b, down to zero, and reports whether a retry
// may follow.
func (b *Budget) failed() bool {
	b.mu.Lock()
	defer b.mu.Unlock()
	b.tokens = max(0, b.tokens-1000)
	return b.tokens > b.max/2
}

func (b *Budget) mustBeBuilt(method string) {
	if b == nil || b.max == 0 {
		panic("retry: Budget." + method + " called on a Budget not built by NewBudget")
	}
}

retry/schedule.go

// Copyright (c) 2026 Z5Labs and Contributors
//
// This software is released under the MIT License.
// https://opensource.org/licenses/MIT

package retry

import (
	"errors"
	"fmt"
	"math/rand/v2"
	"time"
)

// Exponential returns a schedule for [New] that backs off exponentially with
// full jitter: the delay after attempt n is drawn uniformly from
// [0, min(maxDelay, base·2ⁿ⁻¹)). Full jitter spreads out the retries of
// callers that failed together, so they do not all try again at once.
//
// The schedule draws from math/rand/v2's top-level functions, so it is safe
// for concurrent use, and one schedule may be shared by every call. It
// panics if it is called with an attempt less than 1.
//
// Exponential returns an error naming every invalid argument, because both
// come from configuration: base must be positive, and maxDelay at least
// base.
func Exponential(base, maxDelay time.Duration) (func(attempt int) time.Duration, error) {
	var errs []error
	if base <= 0 {
		errs = append(errs, fmt.Errorf("retry: base delay must be positive, got %v", base))
	}
	if maxDelay < base {
		errs = append(errs, fmt.Errorf("retry: max delay must be at least the base delay, %v, got %v", base, maxDelay))
	}
	if err := errors.Join(errs...); err != nil {
		return nil, err
	}

	return func(attempt int) time.Duration {
		if attempt < 1 {
			panic(fmt.Sprintf("retry: schedule called with attempt %d, which is less than 1", attempt))
		}
		// base<<(attempt-1) is at most maxDelay whenever base is at most
		// maxDelay>>(attempt-1), so the shift never overflows.
		ceiling := maxDelay
		if base <= maxDelay>>(attempt-1) {
			ceiling = base << (attempt - 1)
		}
		return rand.N(ceiling)
	}, nil
}

retry/retrier.go

// Copyright (c) 2026 Z5Labs and Contributors
//
// This software is released under the MIT License.
// https://opensource.org/licenses/MIT

package retry

import (
	"context"
	"fmt"
	"iter"
	"sync/atomic"
	"time"
)

// Retrier retries the calls to one dependency. Build one with New at startup
// and share it between every goroutine that calls that dependency. Its
// fields are unexported, so New is the only way to build one, and every
// method of a Retrier not built by New panics.
type Retrier struct {
	maxAttempts int
	schedule    func(attempt int) time.Duration
	classify    func(context.Context, error) (Outcome, time.Duration)
	budget      *Budget
}

// New returns a Retrier that makes at most maxAttempts attempts per call,
// classifies the result of each with classify, waits between attempts as
// schedule says, and limits its retries with budget.
//
// classify returns the Outcome of an attempt's error and, for Retry, the
// least time to wait before the next attempt, such as a server's
// Retry-After, or 0 to wait only as long as schedule says. The delay before
// the next attempt is the larger of the two. For Retry the least time must
// not be negative, so one computed from a server's date is clamped at 0
// first; for Success and Stop it is ignored. schedule returns the delay after attempt n, from 1, as
// [Exponential] does. Both must be safe for concurrent use, and both are
// called by Attempt.Done, on the goroutine that calls it, with nothing
// locked.
//
// budget may be shared with other Retriers, and may be nil for a Retrier
// whose retries are limited only by maxAttempts, such as one that retries a
// connection at startup. A Retrier with a Budget is for calls on the
// request path, where retries of every caller add up.
//
// New panics if schedule or classify is nil, which is a programming error.
// It returns an error if maxAttempts is less than 1, because maxAttempts
// comes from configuration and App.Run should exit non-zero at startup
// instead. A maxAttempts of math.MaxInt retries until a limit other than
// the attempt count stops the call.
func New(maxAttempts int, schedule func(attempt int) time.Duration, classify func(context.Context, error) (Outcome, time.Duration), budget *Budget) (*Retrier, error) {
	if schedule == nil {
		panic("retry: New called with a nil schedule")
	}
	if classify == nil {
		panic("retry: New called with a nil classify function")
	}
	if budget != nil && budget.max == 0 {
		panic("retry: New called with a Budget not built by NewBudget")
	}
	if maxAttempts < 1 {
		return nil, fmt.Errorf("retry: max attempts must be at least 1, got %d", maxAttempts)
	}
	return &Retrier{
		maxAttempts: maxAttempts,
		schedule:    schedule,
		classify:    classify,
		budget:      budget,
	}, nil
}

// Attempts returns the attempts of one call to r's dependency. It is the only
// way to make a call through r, and it sits inline in the caller's code:
//
//	var err error
//	for a := range r.Attempts(ctx) {
//		err = call(ctx)
//		a.Done(err)
//	}
//	return err
//
// The first attempt is yielded at once. After an attempt for which
// Attempt.Done returned Again, the next one is yielded once the delay has
// passed, unless ctx ends first. The loop ends:
//
//   - after Attempt.Done returns any other Decision;
//   - when the loop body breaks or returns after Done, which skips the
//     wait after an Again, for a caller that must not retry a particular
//     call;
//   - when ctx ends during the wait. Then the last Decision was Again,
//     although the loop body did not break, and ctx.Err() is not nil.
//
// Each range over the sequence is a new call, which starts from attempt 1.
// ctx is the context of the call. The classifier receives it with every
// result, and its deadline ends the call when the next attempt would start
// too late.
//
// Attempts panics if ctx is nil, or if r was not built by New.
func (r *Retrier) Attempts(ctx context.Context) iter.Seq[*Attempt] {
	r.mustBeBuilt("Attempts")
	if ctx == nil {
		panic("retry: Retrier.Attempts called with a nil context")
	}

	return func(yield func(*Attempt) bool) {
		// A loop body that panics or calls runtime.Goexit never returns to
		// the iterator, so end its attempt on the way out: a Done called on
		// it later panics instead of deciding for a call that is gone.
		// Nothing is recovered.
		var a *Attempt
		defer func() {
			if a != nil {
				a.state.CompareAndSwap(open, ended)
			}
		}()

		for n := 1; ; n++ {
			a = &Attempt{r: r, ctx: ctx, n: n}
			more := yield(a)
			d, delay := a.end()
			if !more || d != Again || !sleep(ctx, delay) {
				return
			}
		}
	}
}

// Attempt is one attempt of a call, yielded by Retrier.Attempts. Every
// attempt reports its result with Done, once, before the loop body ends,
// whether the body then continues, breaks or returns. An attempt that ends
// without Done panics, because a result that is never reported is lost:
// a lost Success never refills the Budget, and a lost failure is never
// counted. The exception is a loop body that panics or calls
// runtime.Goexit, which ends its attempt as it passes through.
type Attempt struct {
	r   *Retrier
	ctx context.Context
	n   int

	state    atomic.Uint32 // open, deciding, decided or ended
	decision Decision      // set before state becomes decided
	delay    time.Duration // likewise
}

// The states of an Attempt.
const (
	open uint32 = iota
	deciding
	decided
	ended
)

// Number returns the number of a, from 1. A caller that must prepare a
// request again for each retry, such as one that rewinds an HTTP request
// body, does so when Number is greater than 1.
//
// Number panics if a was not yielded by Retrier.Attempts.
func (a *Attempt) Number() int {
	a.mustBeFromAttempts("Number")
	return a.n
}

// Done reports the result of a's call, and returns what happens next. It
// classifies err with the context passed to Retrier.Attempts, updates the
// Budget, and decides whether another attempt follows:
//
//   - Success: Succeeded, and the Budget gains BudgetConfig.TokenRatio;
//   - Stop: Stopped, and the Budget is unchanged;
//   - Retry, with the context done: Expired, and the Budget is unchanged,
//     because the caller giving up says nothing about the dependency;
//   - Retry otherwise: one token is taken from the Budget, and then, in
//     this order, Exhausted if a was the last attempt allowed, Throttled if
//     the Budget allows no retry, Expired if the context's deadline comes
//     before the delay would end, and Again otherwise.
//
// Call Done straight after the call, with its error, before any check of
// the error, so that every result is classified. A caller that must not
// retry this particular call, such as one whose request is not idempotent,
// breaks out of the loop after Done, which skips the wait.
//
// Again is what Done decides, not yet what happens: the wait that follows
// may be cut short by the context, and then no attempt follows. A result
// that the next attempt replaces, such as the body of a response that is
// retried, is therefore cleaned up when the next attempt starts, when
// Number is greater than 1, so that a call whose wait is cut short still
// ends with its last result.
//
// Done panics if it is called twice for one attempt, or after its attempt
// ended by a panic or runtime.Goexit, or if a was not yielded by
// Retrier.Attempts. It panics too if
// the classifier returns a value that is not an Outcome, or a negative
// delay with Retry, or if the schedule returns a negative delay. A panic
// from the classifier leaves the Budget as it was. One from the schedule
// comes after the Budget counted the Retry.
func (a *Attempt) Done(err error) Decision {
	a.mustBeFromAttempts("Done")
	if !a.state.CompareAndSwap(open, deciding) {
		if a.state.Load() == ended {
			panic(fmt.Sprintf("retry: Attempt.Done called after attempt %d ended", a.n))
		}
		panic(fmt.Sprintf("retry: Attempt.Done called twice for attempt %d", a.n))
	}

	d, delay := a.r.decide(a.ctx, a.n, err)
	a.decision, a.delay = d, delay
	a.state.Store(decided)
	return d
}

// end ends a once the loop body has returned to the iterator, and returns
// the Decision Done made and the delay before the next attempt.
func (a *Attempt) end() (Decision, time.Duration) {
	if a.state.CompareAndSwap(open, ended) {
		panic(fmt.Sprintf("retry: attempt %d ended without a call to Attempt.Done", a.n))
	}
	if a.state.Load() != decided {
		panic(fmt.Sprintf("retry: attempt %d ended before its call to Attempt.Done returned", a.n))
	}
	return a.decision, a.delay
}

// decide classifies err, the result of attempt n, and decides what follows.
func (r *Retrier) decide(ctx context.Context, n int, err error) (Decision, time.Duration) {
	o, after := r.classify(ctx, err)
	switch o {
	case Success:
		if r.budget != nil {
			r.budget.succeeded()
		}
		return Succeeded, 0
	case Stop:
		return Stopped, 0
	case Retry:
	default:
		panic(fmt.Sprintf("retry: the classifier returned %v, which is not an Outcome", o))
	}
	if after < 0 {
		panic(fmt.Sprintf("retry: the classifier returned a negative delay, %v, for a Retry", after))
	}

	// The caller gave up, or ran out of time, which says nothing about the
	// dependency, so it costs the Budget nothing.
	if ctx.Err() != nil {
		return Expired, 0
	}
	allowed := r.budget == nil || r.budget.failed()
	switch {
	case n >= r.maxAttempts:
		return Exhausted, 0
	case !allowed:
		return Throttled, 0
	}

	delay := r.schedule(n)
	if delay < 0 {
		panic(fmt.Sprintf("retry: the schedule returned a negative delay, %v, for attempt %d", delay, n))
	}
	delay = max(delay, after)
	if deadline, ok := ctx.Deadline(); ok && time.Until(deadline) <= delay {
		return Expired, 0
	}
	return Again, delay
}

// sleep waits for d, or until ctx ends, and reports whether ctx is still
// live once the wait is over, so that a cancellation that has happened by
// then ends the call even if the timer fired too.
func sleep(ctx context.Context, d time.Duration) bool {
	if d > 0 {
		t := time.NewTimer(d)
		defer t.Stop()
		select {
		case <-ctx.Done():
		case <-t.C:
		}
	}
	return ctx.Err() == nil
}

func (r *Retrier) mustBeBuilt(method string) {
	if r == nil || r.classify == nil {
		panic("retry: Retrier." + method + " called on a Retrier not built by New")
	}
}

// mustBeFromAttempts panics if a was not yielded by Retrier.Attempts.
func (a *Attempt) mustBeFromAttempts(method string) {
	if a == nil || a.r == nil {
		panic("retry: Attempt." + method + " called on an Attempt not yielded by Retrier.Attempts")
	}
}

retry/doc.go

// Copyright (c) 2026 Z5Labs and Contributors
//
// This software is released under the MIT License.
// https://opensource.org/licenses/MIT

// Package retry retries calls to a dependency.
//
// # Calls
//
// A call is retried inline, in the caller's own code, by ranging over the
// attempts that [Retrier.Attempts] yields. Each attempt makes the call and
// reports its result with [Attempt.Done]:
//
//	var err error
//	for a := range r.Attempts(ctx) {
//		err = call(ctx)
//		a.Done(err)
//	}
//	return err
//
// Done classifies the result and returns a [Decision]: [Again] when another
// attempt follows, or the reason none does. After Again, the next attempt
// is yielded once the delay has passed, unless ctx ends first. When the loop
// ends, err holds the result of the last attempt.
//
// Every attempt must reach Done before the loop body ends, whether the body
// then continues, breaks or returns, so that every result is classified. A
// Success that never reaches Done never refills the [Budget], and a
// failure that never reaches it is never counted, so an attempt that ends
// without Done panics. Call Done straight after the call, before any check
// of its error. Assign the call's results to variables declared before
// the loop: a := inside the loop body declares new ones, and the function
// returns the outer ones, which the loop never set.
//
// Because the loop is the caller's own code, the caller decides what only
// it knows about a call:
//
//   - a call that must not be retried, such as a request that is not
//     idempotent, breaks out of the loop after Done, whatever Done decided;
//   - a request that must be prepared again for a retry, such as one whose
//     body must be rewound, is prepared when [Attempt.Number] is greater
//     than 1, and the result of the attempt before it, such as the body of
//     a response, is cleaned up then too;
//   - a deadline for each attempt is set inside the loop with
//     context.WithTimeout.
//
// Again is what Done decides, not yet what happens. The context may end
// during the wait that follows, and then the loop ends with no further
// attempt: the last Decision was Again, although the loop body did not
// break, and ctx.Err() is not nil. A result is therefore cleaned up when
// the attempt that replaces it starts, not when Done returns Again, so a
// call whose wait is cut short still ends with its last result.
//
// A call can be retried only until its result is handed to its caller:
// once the caller reads a response body or a stream, the call is committed,
// and a failure after that is the caller's to handle.
//
// Each call belongs either at startup or on the request path:
//
//	| Call                                       | When                                         |
//	|--------------------------------------------|----------------------------------------------|
//	| Exponential, NewBudget, New                | Startup, inside App.Run, once per dependency |
//	| ranging over Attempts, with Done each time | Per outbound call                            |
//	| the classifier and the schedule            | Per Done, with nothing locked                |
//	| Budget.Tokens                              | Whenever, such as in a metrics callback      |
//
// One Retrier is shared by every goroutine that calls its dependency, so the
// classifier and the schedule must be safe for concurrent use.
//
// # Classifying results
//
// retry does not decide what can be retried. Every result passed to Done
// goes to the classify function passed to [New], together with the context
// passed to Attempts, and classify returns an [Outcome]:
//
//   - [Success]: the call succeeded, and no attempt follows;
//   - [Retry]: the call failed in a way another attempt may fix;
//   - [Stop]: the call failed in a way another attempt will not fix, or
//     must not try to.
//
// For Retry, classify also returns the least time to wait before the next
// attempt, such as a server's Retry-After, or 0. retry has no default
// classifier and treats no error specially, a nil error and the context
// errors included.
//
// The context passed to classify is the context of the whole call, so
// ctx.Err() tells the caller giving up from an attempt failing:
//
//   - ctx.Err() is not nil: the caller cancelled the call or its deadline
//     ran out, and another attempt cannot help. Done decides [Expired] for
//     a Retry here, and takes no token from the Budget for it, because the
//     caller giving up says nothing about the dependency; a classifier that
//     returns Stop says the same thing directly;
//   - ctx.Err() is nil, and err wraps context.DeadlineExceeded: a deadline
//     set for that one attempt ran out, which a retry may fix.
//
// A result that is not an error, such as an HTTP status code, reaches
// classify as an error value that the caller builds.
//
// # Composing with a circuit breaker
//
// A circuit breaker goes inside the retry loop, so that each attempt is a
// call the breaker admits or rejects. A breaker that rejects a call is
// telling its caller not to call, so retrying the rejection at once
// defeats the breaker. retry imports no breaker. The caller's classifier
// returns Stop for the breaker's rejection, such as an error that matches
// github.com/z5labs/bedrock/circuit's ErrRejected, and the attempt makes
// its call through the breaker in a function of its own, so that the
// breaker's deferred guard runs once per attempt:
//
//	for a := range r.Attempts(ctx) {
//		err = callThrough(ctx, b) // b.Allow, defer t.Release, the call, t.Done
//		a.Done(err)
//	}
//
// # Limiting retries
//
// Retries add load to a dependency when it can least take it. A limit on
// attempts per call bounds one call, but not every call together: when a
// dependency fails, every caller retries at once, and a service that
// retries at each of several layers multiplies its attempts. A [Budget]
// bounds every call that shares it, as gRPC's retry throttling does: each
// attempt classified Retry takes a token, each Success adds a fraction of
// one back, and retries stop while half the tokens or fewer are left.
// First attempts are never limited, so a call is never refused for lack of
// budget, only its retries.
//
// A Budget belongs to calls on the request path. A Retrier that retries a
// connection at startup, until a deadline or a signal stops it, has no
// Budget: pass nil to New, and math.MaxInt attempts, so that only the
// context ends the call, as [New] shows.
//
// # Schedules
//
// The delay before each retry comes from the schedule passed to New, a
// function from the number of the attempt that failed to a delay.
// [Exponential] builds the usual one, exponential backoff with full jitter.
// A schedule is a function of the attempt number alone, so one schedule is
// shared by every call.
//
// # Panics
//
// retry never recovers a panic. A panic in the loop body, or a call to
// runtime.Goexit, passes through the iterator unchanged and ends the
// attempt it was in, and retry holds nothing between attempts, so nothing
// is left behind. Misuse panics too, because it is a bug in the caller: an
// attempt that ends without Done, Done called twice or after its attempt
// ended, and a classifier or schedule that returns a value that is out of
// range.
//
// # Time
//
// retry reads time.Now and sets a timer for each wait, and nothing else: no
// clock is injected, and no goroutine runs in the background. A test runs a
// Retrier inside a testing/synctest bubble, where the waits take no real
// time. A cancellation timed to the very instant a wait ends may land before
// or after the next attempt starts, so a test cancels before or after that
// instant, not at it.
//
// # Out of scope
//
// retry does not:
//
//   - hedge calls, time out attempts, rate limit or bound the concurrency of
//     calls;
//   - share a Budget's tokens between processes: each process has its own
//     Budgets;
//   - keep a registry of named Retriers: the caller holds each Retrier it
//     builds;
//   - replay request bodies or parse Retry-After or gRPC pushback, which
//     belong to the framework that wraps a transport;
//   - ship classifiers.
package retry

[Zaba505]

Closing as resolved. The design is settled, and it's filed as three stories. Each is blocked by the one before it:

1. story(retry): nested module with the value types, Budget and Exponential #607: the nested module github.com/z5labs/bedrock/retry, with Outcome, Decision, Budget and Exponential, and the CI, lint and coverage plumbing for a fourth module.
2. story(retry): Retrier and Attempt behind the range-over-func idiom #608: Retrier and Attempt, behind the range-over-func loop, for a := range r.Attempts(ctx) with a.Done(err) for each attempt.
3. story(retry): package doc, examples and README section for the range-over-func idiom #609: the package doc, the examples and the README section.

Settled decisions:

• One way to use it: the range-over-func loop, inline in the caller's code. There is no Do and no generic type.
• Classifier: the caller provides it, and retry ships none. retry never imports circuit; a caller's classifier returns Stop for circuit.ErrRejected.
• Every attempt reaches Done: an attempt that ends without it panics, whether the loop continues, breaks or returns.
• Negative delays: a negative delay with Retry panics, and isn't clamped.
• No budget: a nil *Budget means none, and there's no "unlimited" value.
• maxAttempts: a plain argument to New, with no Config.
• Attempt.Delay(): left out until humus asks for it.

The answers to the remaining design questions are in the API proposal. The stories embed the source they need.