Skip to content

feat: implement EWMA rate limiter with strategy support (#404)#476

Merged
cameri merged 1 commit intocameri:mainfrom
saniddhyaDubey:feature/issue-404-ewma-rate-limiter
Apr 19, 2026
Merged

feat: implement EWMA rate limiter with strategy support (#404)#476
cameri merged 1 commit intocameri:mainfrom
saniddhyaDubey:feature/issue-404-ewma-rate-limiter

Conversation

@saniddhyaDubey
Copy link
Copy Markdown
Collaborator

Description

Implements an Exponentially Weighted Moving Average (EWMA) rate limiter as the new default strategy, replacing the sliding window approach. The EWMA algorithm stores only two values per key (rate and timestamp) in a Redis hash, making it significantly more memory-efficient than the sliding window which stores every request in a sorted set. The entire EWMA calculation is handled atomically via a Lua script in Redis, preventing race conditions.

Related Issue

#404

Motivation and Context

The sliding window rate limiter stores every request timestamp in a Redis sorted set, causing memory usage to grow linearly with traffic. For high-traffic relays with thousands of concurrent clients, this becomes a primary cost driver. EWMA solves this by maintaining only two values per key regardless of traffic volume, while still accurately penalizing bursty behavior through exponential decay.

How Has This Been Tested?

  • Unit tests — added unit tests for calculateEWMA covering first request, burst behavior, time decay, and long inactivity scenarios
  • Integration test — added a Cucumber integration test that pre-seeds Redis with a rate value above the limit, sends a WebSocket event, and verifies the relay correctly responds with a rate limited notice
  • Manual testing — ran the relay locally with Docker Compose, verified EWMA hash keys (rate and timestamp fields) are correctly stored and updated in Redis on each request, and confirmed rate limiting triggers under load

Screenshots (if appropriate):

Screenshot 2026-04-16 at 2 31 46 PM

End-to-end EWMA rate limiting verification:

  • left terminal shows wscat receiving NOTICE: rate limited after exceeding the message limit
  • top-right shows server logs confirming Using strategy: ewma,
  • bottom-right shows Redis hash with rate and timestamp fields being updated in real time on each request, confirming EWMA is storing decay state correctly.

Types of changes

  • New feature (non-breaking change which adds functionality)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my code changes.
  • All new and existing tests passed.

@saniddhyaDubey
Copy link
Copy Markdown
Collaborator Author

Hey @cameri , also wanted to know, if this custom LUA script solution looks good, I would like to move the sliding-window algo into a LUA script based too to achieve atomicity. Let me know what do you think about it.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces an EWMA-based rate limiting strategy (as the new default) with a configurable strategy switch to retain sliding-window support, aiming to reduce Redis memory usage and avoid race conditions via an atomic Lua script.

Changes:

  • Add EWMA rate limiter implementation and Redis Lua script for atomic EWMA updates.
  • Add strategy selection (ewma vs sliding_window) via settings and update factories/handlers to use the new unified rate limiter factory.
  • Add unit + integration test coverage and update default settings + configuration docs.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
test/unit/utils/sliding-window-rate-limiter.spec.ts Adjust test cache stub typing to satisfy updated adapter interface.
test/unit/utils/ewma-rate-limiter.spec.ts Add unit tests for EWMA math and EWMA limiter hit behavior.
test/integration/features/rate-limiter/rate-limiter.ts Add Cucumber steps to preseed Redis EWMA state and trigger rate limiting via WS.
test/integration/features/rate-limiter/rate-limiter.feature Add integration scenario asserting rate limited notice.
src/utils/ewma-rate-limiter.ts Implement EWMA calculation helper and new EWMARateLimiter.
src/handlers/request-handlers/rate-limiter-middleware.ts Switch middleware to use the new rateLimiterFactory.
src/handlers/event-message-handler.ts Replace injected sliding-window factory with generic rate limiter factory.
src/factories/websocket-adapter-factory.ts Wire rateLimiterFactory into the websocket adapter construction.
src/factories/rate-limiter-factory.ts Add strategy selection and make EWMA the default limiter.
src/factories/message-handler-factory.ts Pass rateLimiterFactory into message handlers.
src/factories/controllers/post-invoice-controller-factory.ts Update controller wiring to use rateLimiterFactory.
src/factories/controllers/get-admission-check-controller-factory.ts Update controller wiring to use rateLimiterFactory.
src/adapters/web-socket-adapter.ts Replace injected sliding-window factory usage with generic rate limiter factory.
src/adapters/redis-adapter.ts Add Redis Lua EWMA script + adapter method to execute it; add hash helpers.
src/@types/settings.ts Add limits.rateLimiter settings typing (strategy + halfLife).
src/@types/adapters.ts Extend ICacheAdapter with EWMA/hash-related methods.
resources/default-settings.yaml Add default limits.rateLimiter configuration (ewma + halfLife).
CONFIGURATION.md Document new rate limiter settings options.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 9 to 23
let instance: IRateLimiter = undefined

export const slidingWindowRateLimiterFactory = () => {
export const rateLimiterFactory = () => {
if (!instance) {
const cache: ICacheAdapter = new RedisAdapter(getCacheClient())
instance = new SlidingWindowRateLimiter(cache)
const settings = createSettings()
const strategy = settings.limits?.rateLimiter?.strategy ?? 'ewma'
console.log('Using strategy:', strategy)

if (strategy === 'sliding_window') {
instance = new SlidingWindowRateLimiter(cache)
} else {
instance = new EWMARateLimiter(cache)
}
}
Copy link

Copilot AI Apr 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rateLimiterFactory is a module-level singleton, but the app supports live settings reload via SettingsStatic.watchSettings(). As written, changes to limits.rateLimiter.strategy (and any future EWMA tuning like half-life) won’t take effect until a process restart because instance is never recreated. Consider either (a) selecting strategy on each call and memoizing per-strategy, or (b) tracking the last-used strategy/config and recreating the limiter when settings change (and providing a reset hook for tests).

Copilot uses AI. Check for mistakes.
Comment thread src/factories/rate-limiter-factory.ts Outdated
instance = new SlidingWindowRateLimiter(cache)
const settings = createSettings()
const strategy = settings.limits?.rateLimiter?.strategy ?? 'ewma'
console.log('Using strategy:', strategy)
Copy link

Copilot AI Apr 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid console.log here; the codebase already uses createLogger(...) for structured logging. Using console.log will bypass log level controls and is noisy in production/test output.

Suggested change
console.log('Using strategy:', strategy)

Copilot uses AI. Check for mistakes.
Comment on lines +11 to +13
step: number
): number => {
const lambda = Math.log(2) / period
Copy link

Copilot AI Apr 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The EWMA decay constant is currently computed from period, but the configuration/docs introduce limits.rateLimiter.halfLife as the half-life parameter. As-is, halfLife has no effect and users can’t tune decay independent of the enforcement window. Consider changing the function signature to accept halfLifeMs (or reading it from settings) and using that in the lambda calculation.

Suggested change
step: number
): number => {
const lambda = Math.log(2) / period
step: number,
halfLifeMs: number = period,
): number => {
const lambda = Math.log(2) / halfLifeMs

Copilot uses AI. Check for mistakes.
Comment thread src/adapters/redis-adapter.ts Outdated
Comment thread src/adapters/redis-adapter.ts Outdated
Comment on lines +14 to +21
Before({ tags: '@rate-limiter' }, async function() {
testCacheClient = createClient(getCacheConfig())
await testCacheClient.connect()
SettingsStatic._settings = pipe(
assocPath(['limits', 'message', 'rateLimits'], [{ period: 60000, rate: 10 }]),
assocPath(['limits', 'message', 'ipWhitelist'], []),
)(SettingsStatic._settings) as any
})
Copy link

Copilot AI Apr 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This scenario assumes the EWMA strategy is active (it seeds a Redis hash with rate/timestamp). If a contributor has limits.rateLimiter.strategy: sliding_window in their local settings, this test will fail. To make the test deterministic, set limits.rateLimiter.strategy to ewma in the Before hook (and restore it in After if needed).

Copilot uses AI. Check for mistakes.
Comment thread CONFIGURATION.md Outdated
Comment thread src/@types/adapters.ts Outdated
Comment thread src/adapters/redis-adapter.ts Outdated
Comment thread src/adapters/redis-adapter.ts Outdated
Comment thread src/utils/ewma-rate-limiter.ts Outdated
@cameri
Copy link
Copy Markdown
Owner

cameri commented Apr 18, 2026

@saniddhyaDubey do you mind addressing the conflicts?

@saniddhyaDubey
Copy link
Copy Markdown
Collaborator Author

@saniddhyaDubey do you mind addressing the conflicts?

Already done, final checks are being done from my side!

@saniddhyaDubey saniddhyaDubey force-pushed the feature/issue-404-ewma-rate-limiter branch from 41c0d0b to 2acb629 Compare April 18, 2026 20:44
@coveralls
Copy link
Copy Markdown
Collaborator

coveralls commented Apr 18, 2026

Coverage Status

coverage: 68.698% (+0.4%) from 68.308% — saniddhyaDubey:feature/issue-404-ewma-rate-limiter into cameri:main

@saniddhyaDubey
Copy link
Copy Markdown
Collaborator Author

Unit test failures in event-message-handler.spec.ts are caused by upstream changes introducing INip05VerificationRepository - not related to this PR.

@saniddhyaDubey saniddhyaDubey force-pushed the feature/issue-404-ewma-rate-limiter branch 2 times, most recently from ba163c6 to afa893c Compare April 18, 2026 23:31
@saniddhyaDubey saniddhyaDubey force-pushed the feature/issue-404-ewma-rate-limiter branch from afa893c to 89312db Compare April 19, 2026 14:02
@cameri cameri self-assigned this Apr 19, 2026
@cameri cameri merged commit 49322a9 into cameri:main Apr 19, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants