fix(metadata): lazy-load per-user metadata to stop recvq flood (#116)#212
Open
ValwareIRC wants to merge 2 commits into
Open
fix(metadata): lazy-load per-user metadata to stop recvq flood (#116)#212ValwareIRC wants to merge 2 commits into
ValwareIRC wants to merge 2 commits into
Conversation
On a 200+ user channel, joining used to disconnect the client. Two loops fired METADATA LIST for every member in NAMES and again on WHO completion -- ~400 commands in a tight burst -- which trips UnrealIRCd's recvq flood protection. The connect-time SUB was a secondary firehose, asking the server to push 8 keys * N users on every JOIN. Switch to a fully lazy model: - Narrow the SUB at connect from 8 keys to 2 (display-name + avatar) so live updates for the user-visible bits still come through but the server-side metadata firehose on JOIN is ~75% smaller. - Remove the per-user METADATA LIST loops in the NAMES and WHO handlers. Lazy triggers below cover everyone the user actually sees. - CHANMSG and USERMSG: lazy-fetch the sender's metadata the first time they speak. metadataList() already dedups and consults the localStorage cache, so this is one fetch per unique active speaker per session. - MemberList: IntersectionObserver on each UserItem plus a 250 ms scroll-idle debounce. When the scroll comes to rest, fetch metadata for currently-visible nicks we haven't already requested. Mid-scroll visibility events don't fire fetches. - All METADATA LIST calls now go through a tiny drip queue (src/store/metadataLazyQueue.ts) that paces them at one every 200 ms. Bursty sources can never trip the server-side flood threshold again. Fixes #116
Contributor
📝 WalkthroughWalkthroughMetadata requests are queued and paced at 200ms intervals. Lazy triggers occur in message handlers on first speak, MemberList scroll visibility, and connection setup now subscribes only to avatar and display-name. Eager batch fetches are removed from channel NAMES, WHO completion, and connection handlers. ChangesLazy Metadata Loading System
Sequence Diagram(s)sequenceDiagram
participant User
participant MemberList as MemberList<br/>(UI)
participant IntersectionObserver as IntersectionObserver
participant MessageHandler as Message<br/>Handler
participant Store
participant LazyQueue as Lazy Queue
participant IRC as IRC Client
User->>MemberList: scroll member list
MemberList->>IntersectionObserver: observe visible UserItems
IntersectionObserver->>MemberList: intersection changed
MemberList->>MemberList: debounce and collect usernames
MemberList->>Store: metadataList(serverId, username)
User->>MessageHandler: user speaks in channel
MessageHandler->>Store: metadataList(serverId, sender)
Store->>LazyQueue: enqueueMetadataList(client, serverId, target)
LazyQueue->>LazyQueue: deduplicate and buffer
LazyQueue->>LazyQueue: start 200ms tick interval
loop every 200ms
LazyQueue->>IRC: client.metadataList(serverId, target)
LazyQueue->>LazyQueue: shift next queued entry
end
LazyQueue->>LazyQueue: clear timer when empty
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Automated deployment preview for the PR in the Cloudflare Pages. |
ValwareIRC
added a commit
that referenced
this pull request
May 13, 2026
After PR #212 (lazy metadata) joining a 500-user channel still felt slow: the member list ticked in one nick at a time and metadata requests fired in dribs. The cause was unrelated to metadata. Each 352 (RFC 1459 WHO_REPLY) and 354 (WHOX RPL_WHOSPCRPL) was running a full store.setState that mapped over every server, every channel, every user, and every private chat -- once per nick. With 500 users that's 500 React re-renders of MemberList, and the IntersectionObserver re-fires for each item that scrolls into view, scheduling another lazy fetch burst each tick. Add a tiny batcher (src/store/whoReplyBatcher.ts) with idle + ceiling timers, last-write-wins per nick, and per-(serverId,channel) buckets. WHO_END flushes immediately so a channel renders the moment the server says it is done. WHO_REPLY and WHOX_REPLY now push into the buffer instead of running setState. PM-only fallbacks (no channel) stay inline -- they are single lines, not bursts. Tests cover idle flush, last-write-wins, explicit flush via WHO_END, ceiling fallback for a trickling WHO, merge-into-existing-user, and per-(server,channel) scoping. Full suite green (795 / 1 skipped), build clean.
2 tasks
matheusfillipe
previously approved these changes
May 21, 2026
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/store/handlers/channels.ts`:
- Line 408: There is trailing whitespace on the empty line in
src/store/handlers/channels.ts that fails the Biome formatter; remove the extra
spaces at the end of that line (e.g., after the closing block near the channel
handler/export) so the file contains no trailing whitespace and the CI
formatting check will pass.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 2151a41a-adaa-41df-9ce6-d2ede408c112
📒 Files selected for processing (8)
src/components/layout/MemberList.tsxsrc/store/handlers/channels.tssrc/store/handlers/connection.tssrc/store/handlers/messages.tssrc/store/handlers/whois.tssrc/store/index.tssrc/store/metadataLazyQueue.tstests/store/metadataLazyQueue.test.ts
| } | ||
| } | ||
|
|
||
|
|
Contributor
There was a problem hiding this comment.
Remove trailing whitespace to unblock Biome format check.
Line 408 has trailing whitespace; this matches the reported formatting failure in CI and should be cleaned to restore pipeline green.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/store/handlers/channels.ts` at line 408, There is trailing whitespace on
the empty line in src/store/handlers/channels.ts that fails the Biome formatter;
remove the extra spaces at the end of that line (e.g., after the closing block
near the channel handler/export) so the file contains no trailing whitespace and
the CI formatting check will pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #116.
Root cause
On a 200+ user channel, joining the channel disconnects the client. Two hidden firehoses fire at JOIN time:
metadataList(serverId, user), sendingMETADATA <nick> LISTonce per user — ~200 lines in a burst.The connect-time
METADATA * SUB(store/handlers/connection.ts) was also subscribing to 8 keys, telling the server to push 8 × N values for every user we share a channel with — the spec-blessed "metadata firehose".UnrealIRCd's recvq protection then ate the connection.
Fix — fully lazy per-user metadata
CHANMSG/USERMSG)METADATA LISTfor them.IntersectionObserveron eachUserItem+ 250 ms scroll-idle debounce. When the scroll comes to rest, fetch metadata for currently-visible nicks not yet requested. Mid-scroll visibility events don't fire fetches.The existing
userMetadataRequested: Record<serverId, Set<nick>>store state deduplicates across all sources, so each unique nick is fetched at most once per session.To bound bursts (e.g. scroll-stops on a wall of unfamiliar nicks), all
METADATA LISTcalls now go through a tiny drip queue — src/store/metadataLazyQueue.ts — that paces dispatches at one every 200 ms. FIFO, coalesces duplicates, idles when empty.SUB narrowing
Kept a narrow SUB on
display-name+avatarso the two user-visible bits still update live (rename / new avatar). Droppedurl,website,status,location,color,bot. ~75% smaller firehose on JOIN, still nice live updates for the things people notice.Test plan
npm run format,npm run fix:unsafe,npm run test(789 pass / 1 skipped),npm run buildcleantests/store/metadataLazyQueue.test.tscovers: FIFO drip, coalesce duplicates, per-server isolation, idle-resume after queue emptiesSummary by CodeRabbit
Refactor
Tests