Skip to content

RateLimit Policy

Jump to bottom
Marco Breveglieri edited this page Jul 20, 2026 · 2 revisions

Rate Limit Policy API

Unit: Murphy.Policy.RateLimit

The Rate Limit policy controls the rate of operations to prevent system overload using a token bucket algorithm.

Overview

Rate limiting prevents:

  • API throttling (exceeding third-party rate limits)
  • Resource exhaustion (too many concurrent operations)
  • System overload (protecting your own services)
  • Uncontrolled bursts of activity

Table of Contents

ERateLimitRejectedException Class

Exception thrown when the rate limit has been exceeded.

Declaration

type
  ERateLimitRejectedException = class(Exception)
  end;

Description: Simple exception class indicating that the rate limit was exceeded.

Usage:

try
  RateLimitPolicy.Execute(procedure begin MakeAPICall; end);
except
  on E: ERateLimitRejectedException do
  begin
    WriteLn('Rate limit exceeded - too many calls');
    // Wait and retry, or handle appropriately
  end;
end;

IRateLimitPolicy Interface

The interface for the Rate Limit policy pattern.

Declaration

type
  IRateLimitPolicy = interface(IPolicy)
    ['{B4AFFB0E-C958-4D89-A2BC-79659915581E}']
    function Allow(ACalls: Integer): IRateLimitPolicy;
    procedure Execute(AProc: TProc);
    function Within(ADuration: TTimeSpan): IRateLimitPolicy;
  end;

Methods

Allow

function Allow(ACalls: Integer): IRateLimitPolicy;

Description: Configures the maximum number of calls allowed within the time window.

Parameters:

  • ACalls: Integer - Maximum number of calls allowed

Returns: IRateLimitPolicy - Self for method chaining

Default: 20 calls

Example:

// Allow 100 calls
Policy := TRateLimitBuilder
  .Handle(Exception)  // Exception type doesn't affect rate limiting
  .Allow(100)
  .Within(TTimeSpan.FromMinutes(1))
  .Build;

Execute

procedure Execute(AProc: TProc);

Description: Executes the provided procedure if rate limit allows, otherwise throws ERateLimitRejectedException.

Parameters:

  • AProc: TProc - Anonymous procedure to execute

Behavior:

  1. Checks if token bucket has expired (time window passed)
  2. If expired: Refills bucket with tokens
  3. If tokens available: Consumes one token and executes AProc
  4. If no tokens available: Throws ERateLimitRejectedException

Example:

try
  RateLimitPolicy.Execute(
    procedure
    begin
      WriteLn('Making API call...');
      CallAPI;
    end);
except
  on E: ERateLimitRejectedException do
    WriteLn('Rate limit exceeded!');
end;

Within

function Within(ADuration: TTimeSpan): IRateLimitPolicy;

Description: Configures the time window for the rate limit.

Parameters:

  • ADuration: TTimeSpan - Time window duration

Returns: IRateLimitPolicy - Self for method chaining

Default: 1 second

Example:

// 100 calls per minute
Policy := TRateLimitBuilder
  .Handle(Exception)
  .Allow(100)
  .Within(TTimeSpan.FromMinutes(1))
  .Build;

// 10 calls per second
Policy := TRateLimitBuilder
  .Handle(Exception)
  .Allow(10)
  .Within(TTimeSpan.FromSeconds(1))
  .Build;

// 1000 calls per hour
Policy := TRateLimitBuilder
  .Handle(Exception)
  .Allow(1000)
  .Within(TTimeSpan.FromHours(1))
  .Build;

TRateLimitPolicy Class

Concrete implementation of the Rate Limit policy using token bucket algorithm.

Declaration

type
  TRateLimitPolicy = class(TPolicy, IRateLimitPolicy)
  private
    FAllowedCalls: Integer;
    FBucketIds: TStack<TGUID>;
    FBucketTime: TDateTime;
    FWithinDuration: TTimeSpan;
  protected
    function IsBucketExpired: Boolean;
  public
    constructor Create(AExceptionTypes: TArray<ExceptClass>); override;
    destructor Destroy; override;
    function Allow(ACalls: Integer): IRateLimitPolicy;
    procedure Execute(AProc: TProc);
    function Within(ADuration: TTimeSpan): IRateLimitPolicy;
  end;

Constructor

constructor Create(AExceptionTypes: TArray<ExceptClass>); override;

Defaults:

  • FAllowedCalls: 20
  • FWithinDuration: 1 second
  • FBucketTime: MinDateTime (will trigger immediate bucket fill)
  • FBucketIds: Empty stack

Protected Methods

IsBucketExpired

function IsBucketExpired: Boolean;

Description: Checks if the current token bucket time window has expired.

Returns: Boolean

  • True - Time window has expired, bucket should be refilled
  • False - Still within current time window

Implementation (Murphy.Policy.RateLimit.pas:105):

Result := (FBucketTime + FWithinDuration) < Now;

Token Bucket Algorithm

The implementation uses a simple token bucket:

  1. Bucket Creation: When expired or first use, fill bucket with FAllowedCalls tokens (GUIDs)
  2. Token Consumption: Each Execute call pops one token from the bucket
  3. Rejection: If bucket is empty, throw ERateLimitRejectedException
  4. Refill: When time window expires, clear and refill bucket

Characteristics:

  • Allows bursts (can use all tokens immediately)
  • Simple and predictable
  • No gradual token replenishment (refills completely when window expires)

TRateLimitBuilder Class

Builder class for creating rate limit policies.

Declaration

type
  TRateLimitBuilder = class sealed(TPolicyBuilder<IRateLimitPolicy>)
  public
    class function Handle(AExceptionTypes: TArray<ExceptClass>): IRateLimitPolicy; override;
  end;

Class Methods

Handle

class function Handle(AExceptionTypes: TArray<ExceptClass>): IRateLimitPolicy; override;

Description: Creates a rate limit policy.

Note: Exception type parameter is not used by rate limiting logic (inherited from base), but provided for consistency.

Usage:

Policy := TRateLimitBuilder
  .Handle(Exception)  // Can use any exception type
  .Allow(50)
  .Within(TTimeSpan.FromMinutes(1))
  .Build;

Usage Examples

Example 1: Basic Rate Limiting

var
  Policy: IRateLimitPolicy;
  I: Integer;
begin
  // Allow 5 calls per 10 seconds
  Policy := TRateLimitBuilder
    .Handle(Exception)
    .Allow(5)
    .Within(TTimeSpan.FromSeconds(10))
    .Build;

  for I := 1 to 10 do
  begin
    try
      Policy.Execute(
        procedure
        begin
          WriteLn(Format('Call %d at %s', [I, TimeToStr(Now)]));
          MakeAPICall;
        end);
    except
      on E: ERateLimitRejectedException do
      begin
        WriteLn(Format('Call %d rejected - rate limit exceeded', [I]));
        Sleep(2000);  // Wait before retrying
      end;
    end;
  end;
end;

Example 2: API Client with Rate Limiting

type
  TAPIClient = class
  private
    FRateLimitPolicy: IRateLimitPolicy;
  public
    constructor Create;
    function Get(const AURL: string): string;
  end;

constructor TAPIClient.Create;
begin
  // GitHub API: 60 requests per hour for unauthenticated requests
  FRateLimitPolicy := TRateLimitBuilder
    .Handle(Exception)
    .Allow(60)
    .Within(TTimeSpan.FromHours(1))
    .Build;
end;

function TAPIClient.Get(const AURL: string): string;
begin
  try
    FRateLimitPolicy.Execute(
      procedure
      begin
        Result := HTTP.Get(AURL);
      end);
  except
    on E: ERateLimitRejectedException do
    begin
      raise Exception.Create('API rate limit exceeded. Please wait before making more requests.');
    end;
  end;
end;

Example 3: Rate Limiting with Retry on Rejection

var
  Policy: IRateLimitPolicy;
  MaxRetries: Integer;
  RetryCount: Integer;
begin
  Policy := TRateLimitBuilder
    .Handle(Exception)
    .Allow(10)
    .Within(TTimeSpan.FromSeconds(1))
    .Build;

  MaxRetries := 5;
  RetryCount := 0;

  while RetryCount < MaxRetries do
  begin
    try
      Policy.Execute(
        procedure
        begin
          ProcessItem;
        end);

      Break;  // Success - exit retry loop
    except
      on E: ERateLimitRejectedException do
      begin
        Inc(RetryCount);
        WriteLn(Format('Rate limited. Retry %d/%d...', [RetryCount, MaxRetries]));
        Sleep(200);  // Wait 200ms before retry
      end;
    end;
  end;

  if RetryCount >= MaxRetries then
    raise Exception.Create('Failed after maximum retries due to rate limiting');
end;

Example 4: Multiple Rate Limits (Different Time Windows)

var
  PerSecondPolicy: IRateLimitPolicy;
  PerMinutePolicy: IRateLimitPolicy;
begin
  // Twitter-style: 15 calls per 15 minutes AND max 1 per second
  PerSecondPolicy := TRateLimitBuilder
    .Handle(Exception)
    .Allow(1)
    .Within(TTimeSpan.FromSeconds(1))
    .Build;

  PerMinutePolicy := TRateLimitBuilder
    .Handle(Exception)
    .Allow(15)
    .Within(TTimeSpan.FromMinutes(15))
    .Build;

  // Apply both limits
  try
    PerSecondPolicy.Execute(
      procedure
      begin
        PerMinutePolicy.Execute(
          procedure
          begin
            MakeAPICall;
          end);
      end);
  except
    on E: ERateLimitRejectedException do
      WriteLn('Rate limit exceeded');
  end;
end;

Example 5: Batch Processing with Rate Limiting

procedure ProcessItemsWithRateLimit(Items: TList<string>);
var
  Policy: IRateLimitPolicy;
  Item: string;
  ProcessedCount: Integer;
begin
  // Process max 100 items per minute
  Policy := TRateLimitBuilder
    .Handle(Exception)
    .Allow(100)
    .Within(TTimeSpan.FromMinutes(1))
    .Build;

  ProcessedCount := 0;

  for Item in Items do
  begin
    try
      Policy.Execute(
        procedure
        begin
          WriteLn(Format('Processing: %s', [Item]));
          ProcessItem(Item);
          Inc(ProcessedCount);
        end);
    except
      on E: ERateLimitRejectedException do
      begin
        WriteLn(Format('Rate limit reached after %d items. Waiting...', [ProcessedCount]));
        Sleep(60000);  // Wait 1 minute for bucket to reset

        // Retry this item
        Policy.Execute(procedure begin ProcessItem(Item); end);
      end;
    end;
  end;

  WriteLn(Format('Processed %d items', [ProcessedCount]));
end;

Important Notes

Token Bucket Behavior

The current implementation:

  • Refills completely when time window expires (not gradually)
  • Allows bursts - all tokens can be consumed immediately
  • Resets on expiration - doesn't carry over unused tokens

Example:

// Allow 10 calls per minute
Policy := TRateLimitBuilder.Handle(Exception).Allow(10).Within(TTimeSpan.FromMinutes(1)).Build;

// You can make 10 calls immediately (burst)
for I := 1 to 10 do
  Policy.Execute(procedure begin CallAPI; end);  // All succeed

// 11th call fails
Policy.Execute(procedure begin CallAPI; end);  // Throws ERateLimitRejectedException

// After 1 minute, bucket refills and you can make 10 more calls

Thread Safety

Current Status: Rate limit policy is not thread-safe.

Recommendations:

  • Use separate policy instances per thread
  • Or protect shared instance with synchronization primitives (TMonitor, TCriticalSection)

Combining with Other Patterns

Rate limiting works well with:

  • Retry: Retry after delay when rate limit exceeded
  • Circuit Breaker: Prevent overwhelming rate-limited services
  • Fallback: Provide cached data when rate limit prevents fresh data fetch

See Also

← Fallback Policy | API Reference Index | Timeout Policy →

Clone this wiki locally