[Enhancement] Configurable dispatch rate limiter backoff to reduce the 1-second latency penalty when limits are reached

[lhotari]

Search before asking

• I searched in the issues and found nothing similar.

Motivation

The current dispatch rate limiter implementation introduces a fixed 1-second additional latency when the rate limit is reached. This delay is hardcoded as MESSAGE_RATE_BACKOFF_MS = 1000 in the PersistentTopic class:

pulsar/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java

Line 240 in e547bea

public static final int MESSAGE_RATE_BACKOFF_MS = 1000;

Solution

Simply making MESSAGE_RATE_BACKOFF_MS configurable would be insufficient for several reasons:

1. Token Replenishment Frequency: Tokens are currently added to the rate limiter once per second. If the backoff time were reduced (e.g., to 100ms) without changing the token addition frequency, dispatchers would check for available tokens too frequently, wasting CPU resources.

2. Implementation Differences:

   • Classic RateLimiterImpl: Uses a scheduled job to add permits periodically (controlled by ratePeriod).
   • PIP-322 AsyncTokenBucket: Calculates tokens on-demand when the limiter is used (controlled by addTokensResolutionNanos), enabling better scaling to millions of rate limiter instances without any overhead of scheduled jobs.

The backoff mechanism is triggered in the AbstractBaseDispatcher class:

pulsar/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractBaseDispatcher.java

Lines 417 to 428 in e547bea

	if (readLimits.getLeft() == 0 || readLimits.getRight() == 0) {
	if (log.isDebugEnabled()) {
	log.debug("[{}] message-read exceeded {} message-rate {}/{}, schedule after {}ms", getName(),
	limiterType.name().toLowerCase(),
	rateLimiter.getDispatchRateOnMsg(), rateLimiter.getDispatchRateOnByte(),
	MESSAGE_RATE_BACKOFF_MS);
	}
	reScheduleRead();
	readLimits.setLeft(-1);
	readLimits.setRight(-1L);
	return false;
	}

Possible solution:

• Making MESSAGE_RATE_BACKOFF_MS configurable in dispatch rate limiter configurations.
  • One possibility would be to internally configure it based on the rate period. However currently rate period's time unit is fixed to seconds for configuration:
    • https://github.com/apache/pulsar/blob/master/pulsar-client-admin-api/src/main/java/org/apache/pulsar/common/policies/data/DispatchRate.java
    • Another problem is that for users, it's more natural to configure the rate for 1 second or 1 minute. It could be better way to configure the backoff time.

Fairness (as defined in fair queuing and fairness measure) Considerations:

The current fixed 1-second backoff, which matches the 1-second token replenishment interval, may inadvertently provide some level of fairness in resource allocation. Changing this ratio could impact the fairness properties of the system.

Fairness is currently unaddressed in Pulsar's dispatch rate limiting. Addressing fairness is crucial to improving Pulsar's rate limiting and capacity management capabilities as described in the Pulsar 4.0 blog post.

To make Pulsar competitive with Confluent's Kora, which according to the Kora paper includes features like "backpressure and auto-tuning" and "dynamic quota management," we need to enhance Pulsar's approach to fairness in resource allocation, including in dispatch rate limiting.

Alternatives

No response

Anything else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!