feat: implement EWMA rate limiter with strategy support (#404)#476
Conversation
|
Hey @cameri , also wanted to know, if this custom LUA script solution looks good, I would like to move the sliding-window algo into a LUA script based too to achieve atomicity. Let me know what do you think about it. |
There was a problem hiding this comment.
Pull request overview
This PR introduces an EWMA-based rate limiting strategy (as the new default) with a configurable strategy switch to retain sliding-window support, aiming to reduce Redis memory usage and avoid race conditions via an atomic Lua script.
Changes:
- Add EWMA rate limiter implementation and Redis Lua script for atomic EWMA updates.
- Add strategy selection (
ewmavssliding_window) via settings and update factories/handlers to use the new unified rate limiter factory. - Add unit + integration test coverage and update default settings + configuration docs.
Reviewed changes
Copilot reviewed 18 out of 18 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| test/unit/utils/sliding-window-rate-limiter.spec.ts | Adjust test cache stub typing to satisfy updated adapter interface. |
| test/unit/utils/ewma-rate-limiter.spec.ts | Add unit tests for EWMA math and EWMA limiter hit behavior. |
| test/integration/features/rate-limiter/rate-limiter.ts | Add Cucumber steps to preseed Redis EWMA state and trigger rate limiting via WS. |
| test/integration/features/rate-limiter/rate-limiter.feature | Add integration scenario asserting rate limited notice. |
| src/utils/ewma-rate-limiter.ts | Implement EWMA calculation helper and new EWMARateLimiter. |
| src/handlers/request-handlers/rate-limiter-middleware.ts | Switch middleware to use the new rateLimiterFactory. |
| src/handlers/event-message-handler.ts | Replace injected sliding-window factory with generic rate limiter factory. |
| src/factories/websocket-adapter-factory.ts | Wire rateLimiterFactory into the websocket adapter construction. |
| src/factories/rate-limiter-factory.ts | Add strategy selection and make EWMA the default limiter. |
| src/factories/message-handler-factory.ts | Pass rateLimiterFactory into message handlers. |
| src/factories/controllers/post-invoice-controller-factory.ts | Update controller wiring to use rateLimiterFactory. |
| src/factories/controllers/get-admission-check-controller-factory.ts | Update controller wiring to use rateLimiterFactory. |
| src/adapters/web-socket-adapter.ts | Replace injected sliding-window factory usage with generic rate limiter factory. |
| src/adapters/redis-adapter.ts | Add Redis Lua EWMA script + adapter method to execute it; add hash helpers. |
| src/@types/settings.ts | Add limits.rateLimiter settings typing (strategy + halfLife). |
| src/@types/adapters.ts | Extend ICacheAdapter with EWMA/hash-related methods. |
| resources/default-settings.yaml | Add default limits.rateLimiter configuration (ewma + halfLife). |
| CONFIGURATION.md | Document new rate limiter settings options. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| let instance: IRateLimiter = undefined | ||
|
|
||
| export const slidingWindowRateLimiterFactory = () => { | ||
| export const rateLimiterFactory = () => { | ||
| if (!instance) { | ||
| const cache: ICacheAdapter = new RedisAdapter(getCacheClient()) | ||
| instance = new SlidingWindowRateLimiter(cache) | ||
| const settings = createSettings() | ||
| const strategy = settings.limits?.rateLimiter?.strategy ?? 'ewma' | ||
| console.log('Using strategy:', strategy) | ||
|
|
||
| if (strategy === 'sliding_window') { | ||
| instance = new SlidingWindowRateLimiter(cache) | ||
| } else { | ||
| instance = new EWMARateLimiter(cache) | ||
| } | ||
| } |
There was a problem hiding this comment.
rateLimiterFactory is a module-level singleton, but the app supports live settings reload via SettingsStatic.watchSettings(). As written, changes to limits.rateLimiter.strategy (and any future EWMA tuning like half-life) won’t take effect until a process restart because instance is never recreated. Consider either (a) selecting strategy on each call and memoizing per-strategy, or (b) tracking the last-used strategy/config and recreating the limiter when settings change (and providing a reset hook for tests).
| instance = new SlidingWindowRateLimiter(cache) | ||
| const settings = createSettings() | ||
| const strategy = settings.limits?.rateLimiter?.strategy ?? 'ewma' | ||
| console.log('Using strategy:', strategy) |
There was a problem hiding this comment.
Avoid console.log here; the codebase already uses createLogger(...) for structured logging. Using console.log will bypass log level controls and is noisy in production/test output.
| console.log('Using strategy:', strategy) |
| step: number | ||
| ): number => { | ||
| const lambda = Math.log(2) / period |
There was a problem hiding this comment.
The EWMA decay constant is currently computed from period, but the configuration/docs introduce limits.rateLimiter.halfLife as the half-life parameter. As-is, halfLife has no effect and users can’t tune decay independent of the enforcement window. Consider changing the function signature to accept halfLifeMs (or reading it from settings) and using that in the lambda calculation.
| step: number | |
| ): number => { | |
| const lambda = Math.log(2) / period | |
| step: number, | |
| halfLifeMs: number = period, | |
| ): number => { | |
| const lambda = Math.log(2) / halfLifeMs |
| Before({ tags: '@rate-limiter' }, async function() { | ||
| testCacheClient = createClient(getCacheConfig()) | ||
| await testCacheClient.connect() | ||
| SettingsStatic._settings = pipe( | ||
| assocPath(['limits', 'message', 'rateLimits'], [{ period: 60000, rate: 10 }]), | ||
| assocPath(['limits', 'message', 'ipWhitelist'], []), | ||
| )(SettingsStatic._settings) as any | ||
| }) |
There was a problem hiding this comment.
This scenario assumes the EWMA strategy is active (it seeds a Redis hash with rate/timestamp). If a contributor has limits.rateLimiter.strategy: sliding_window in their local settings, this test will fail. To make the test deterministic, set limits.rateLimiter.strategy to ewma in the Before hook (and restore it in After if needed).
|
@saniddhyaDubey do you mind addressing the conflicts? |
Already done, final checks are being done from my side! |
41c0d0b to
2acb629
Compare
|
Unit test failures in |
ba163c6 to
afa893c
Compare
afa893c to
89312db
Compare
Description
Implements an Exponentially Weighted Moving Average (EWMA) rate limiter as the new default strategy, replacing the sliding window approach. The EWMA algorithm stores only two values per key (rate and timestamp) in a Redis hash, making it significantly more memory-efficient than the sliding window which stores every request in a sorted set. The entire EWMA calculation is handled atomically via a Lua script in Redis, preventing race conditions.
Related Issue
#404
Motivation and Context
The sliding window rate limiter stores every request timestamp in a Redis sorted set, causing memory usage to grow linearly with traffic. For high-traffic relays with thousands of concurrent clients, this becomes a primary cost driver. EWMA solves this by maintaining only two values per key regardless of traffic volume, while still accurately penalizing bursty behavior through exponential decay.
How Has This Been Tested?
calculateEWMAcovering first request, burst behavior, time decay, and long inactivity scenariosrate limitednoticerateandtimestampfields) are correctly stored and updated in Redis on each request, and confirmed rate limiting triggers under loadScreenshots (if appropriate):
End-to-end EWMA rate limiting verification:
Types of changes
Checklist: