Hedging

Hedging Pattern Guide

The Hedging pattern launches additional parallel attempts when the primary one is too slow or fails, returning the first attempt that succeeds. It trades extra work for lower tail latency.

When to Use

• Reducing tail latency for calls to replicated or redundant services
• Flaky endpoints where an occasional slow or failed call shouldn't block the whole request
• Operations that are safe to run more than once at the same time (idempotent)

When NOT to Use

• Non-idempotent operations (e.g. anything that creates a resource, charges a payment, or has other side effects) - losing attempts keep running in the background and can still complete
• Calls to a single, non-replicated backend where extra parallel attempts just add load without improving the odds of success
• Expensive operations where launching multiple attempts in parallel is not acceptable from a cost or resource standpoint

Quick Start

uses Murphy.Policy.Hedging;

var
  Policy: IHedgingPolicy;
begin
  Policy := THedgingBuilder
    .Handle(ENetworkError)
    .MaxAttempts(2)
    .Delay(TTimeSpan.FromMilliseconds(500))
    .Build;

  Policy.Execute(procedure begin CallReplicatedService; end);
end;

Common Scenarios

Reducing Tail Latency

procedure FetchFromReplica;
var
  Policy: IHedgingPolicy;
begin
  Policy := THedgingBuilder
    .Handle(Exception)
    .MaxAttempts(1)                        // One extra attempt
    .Delay(TTimeSpan.FromMilliseconds(300)) // Hedge if slower than 300ms
    .Build;

  Policy.Execute(
    procedure
    begin
      Result := CallAnyReplica;
    end);
end;

Observing Hedged Attempts

var Policy: IHedgingPolicy;
begin
  Policy := THedgingBuilder
    .Handle(ENetworkError)
    .MaxAttempts(3)
    .Delay(TTimeSpan.FromMilliseconds(200))
    .OnHedging(procedure(Context: THedgingContext)
               begin
                 WriteLn(Format('Launching hedged attempt #%d', [Context.AttemptNumber]));
               end)
    .Build;

  Policy.Execute(procedure begin CallReplicatedService; end);
end;

Handling Total Failure

var Policy: IHedgingPolicy;
begin
  Policy := THedgingBuilder.Handle(ENetworkError).MaxAttempts(2).Build;

  try
    Policy.Execute(procedure begin CallReplicatedService; end);
  except
    on E: ENetworkError do
      WriteLn('All hedged attempts failed: ' + E.Message);
  end;
end;

Best Practices

1. Only hedge idempotent actions - losing attempts are abandoned but keep running in the background, so side effects can still happen
2. Set the delay based on expected latency - hedge too early and you waste resources on every call; too late and tail latency isn't improved
3. Cap MaxAttempts - each extra attempt multiplies load on the backend; negative values raise EArgumentException
4. Use OnHedging for observability - track how often hedged attempts fire to tune the delay
5. Combine with Timeout - bound how long the winning attempt itself is allowed to take

See Also

• Hedging API Reference
• Combining Patterns

---

Back to Index