RateLimit

Rate Limit Pattern Guide

The Rate Limit pattern controls the rate of operations to prevent system overload using a token bucket algorithm.

When to Use

• Calling rate-limited third-party APIs
• Protecting your own services from overload
• Preventing resource exhaustion
• Controlling costs (pay-per-request APIs)

When NOT to Use

• Operations without rate limits
• Single or infrequent operations
• When client-side queuing is better

Quick Start

uses Murphy.Policy.RateLimit;

var
  Policy: IRateLimitPolicy;
begin
  Policy := TRateLimitBuilder
    .Handle(Exception)
    .Allow(100)
    .Within(TTimeSpan.FromMinutes(1))
    .Build;

  Policy.Execute(procedure begin MakeAPICall; end);
end;

Common Scenarios

API Client Rate Limiting

type
  TTwitterClient = class
  private
    FRateLimit: IRateLimitPolicy;
  public
    constructor Create;
    procedure Tweet(Status: string);
  end;

constructor TTwitterClient.Create;
begin
  // Twitter: 300 tweets per 3 hours
  FRateLimit := TRateLimitBuilder
    .Handle(Exception)
    .Allow(300)
    .Within(TTimeSpan.FromHours(3))
    .Build;
end;

procedure TTwitterClient.Tweet(Status: string);
begin
  try
    FRateLimit.Execute(procedure begin PostTweet(Status); end);
  except
    on E: ERateLimitRejectedException do
      ShowMessage('Please wait before tweeting again');
  end;
end;

Batch Processing

procedure ProcessItems(Items: TList<string>);
var
  Policy: IRateLimitPolicy;
begin
  Policy := TRateLimitBuilder
    .Handle(Exception)
    .Allow(100)
    .Within(TTimeSpan.FromMinutes(1))
    .Build;

  for var Item in Items do
  begin
    try
      Policy.Execute(procedure begin ProcessItem(Item); end);
    except
      on E: ERateLimitRejectedException do
      begin
        WriteLn('Rate limit reached, waiting 60 seconds...');
        Sleep(60000);
        Policy.Execute(procedure begin ProcessItem(Item); end);
      end;
    end;
  end;
end;

Multiple Time Windows

// Enforce both per-second AND per-hour limits
var
  PerSecond: IRateLimitPolicy;
  PerHour: IRateLimitPolicy;
begin
  PerSecond := TRateLimitBuilder.Handle(Exception)
    .Allow(1).Within(TTimeSpan.FromSeconds(1)).Build;

  PerHour := TRateLimitBuilder.Handle(Exception)
    .Allow(1000).Within(TTimeSpan.FromHours(1)).Build;

  // Check both limits
  PerSecond.Execute(procedure
    begin
      PerHour.Execute(procedure begin MakeRequest; end);
    end);
end;

Best Practices

1. Match API limits - Configure to match actual rate limits
2. Handle rejections gracefully - Queue or retry after delay
3. Add margin - Set limit slightly lower than actual (e.g., 95 instead of 100)
4. Monitor usage - Track how often limits are hit
5. Consider multiple windows - Some APIs have per-second AND per-hour limits

Token Bucket Behavior

• Bucket refills completely when time window expires
• Allows bursts (all tokens can be used immediately)
• Does not carry over unused tokens

See Also

• Rate Limit API Reference
• Combining Patterns

---

Back to Index