Skip to content

fix(ban-all): expand urgent queue capacity - #147

Merged
lorenzocorallo merged 1 commit into
mainfrom
fix/expand-banall-capacity
Sep 4, 2026
Merged

fix(ban-all): expand urgent queue capacity#147
lorenzocorallo merged 1 commit into
mainfrom
fix/expand-banall-capacity

Conversation

@lorenzocorallo

Copy link
Copy Markdown
Member

Summary

  • raise the BanAll outstanding-job cap from 30,000 to 60,000
  • raise the executor rate from 8 to 12 jobs per second so accepted work does not create a two-hour tail
  • update regression coverage and retention calculations for the new bounds

Production evidence

At the time of the incident:

  • 29,754 outstanding executor jobs used 44.16 MiB of Redis memory
  • Redis had a 320 MiB memory limit and peaked at 46.79 MiB
  • the 1 GiB PVC had 843 MiB free
  • AOF writes were healthy, with no rejected connections or evictions
  • 46 BanAll parents were waiting and the executor drained exactly 8 jobs per second

At the new cap, current observed memory density projects well below both the Redis memory limit and the container limit. The 12-per-second rate limits worst-case BanAll Telegram traffic to 24 calls per second when every job also needs explicit deletion.

Operational effect

  • supports about 92 simultaneous 648-chat flows
  • reduces the first-attempt drain time of a full 60,000-job queue to about 83 minutes
  • preserves the admission bound and all retry, retention, and permanent-error behavior

Verification

  • pnpm test (79 tests)
  • pnpm typecheck
  • pnpm check
  • pnpm build

@coderabbitai

coderabbitai Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

Next included review available in 13 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 8bc16821-8627-4041-a572-66ae86cfa615

📥 Commits

Reviewing files that changed from the base of the PR and between d1283b5 and 87cb028.

📒 Files selected for processing (2)
  • src/modules/moderation/ban-all-flow.ts
  • tests/ban-all-flow.test.ts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@lorenzocorallo
lorenzocorallo merged commit 39b1ee5 into main Sep 4, 2026
2 checks passed
@lorenzocorallo
lorenzocorallo deleted the fix/expand-banall-capacity branch September 4, 2026 13:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant