Repository navigation
Per-customer API keys and quotas in a Worker: what had to change from the Node.js version #16652
supertj
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
We build an SDK that checks API keys, applies each customer's rate limit and quota, and counts their usage inside your own server. On Node.js it runs background timers: sync the key list every 15 seconds, send counts every 60. None of that works in a Worker, where an isolate can be frozen between requests and there's no process to hook on shutdown. Here's what the Workers setup looks like now (Hono on Workers,
@usagerail/node0.1.2 or later):What changed for Workers, and why:
waitUntil. Withserverless: truethere are no timers. After each response, the middleware handsc.executionCtx.waitUntilthe job of sending counts, refreshing the key list and renewing quota leases. One isolate sends at most one batch every 0.5 seconds; a batch that fails stays in memory and goes out with the next request.serverless: true, the Worker fails to start.nodejs_compatis required, because the SDK usesnode:crypto.envfromcloudflare:workersis available there, so one middleware instance is shared by every request on an isolate.The risk with
waitUntilis the platform ending an isolate before that work finishes, which would lose counts that were never sent. In our October 2026 tests with 0.1.2 on the Workers Free plan, the recorded count matched the number of successful responses in every run: 60,000 requests at 100 a second, 120 requests a minute apart over two hours, and bursts of up to 260 simultaneous requests on new deployments that started up to 48 new isolates. The 48-isolate run had one connection error on the load generator's side, so matching totals there could still hide one missed response.Disclosure: I work on UsageRail. Full Workers setup: https://usagerail.com/docs/cloudflare-workers
All reactions