fix(mobile): retry channel-sections startup sync when relay rate-limits cold start - #3004
Conversation
…ts cold start On mobile cold start, ChannelsNotifier fires a burst of per-channel REQs that exhausts the relay's per-connection rate-limit quota. The sections manager then has BOTH its one-shot history fetch and its live subscription rejected with 'rate-limited: quota exceeded'. Both errors were silently swallowed with no retry, so the manager kept the local (empty/default) store forever and desktop-created channel groups never appeared on mobile. Restarts replay the same storm, so the failure is sticky. Fix: track whether the startup fetch and the live subscription have succeeded, and retry _syncWithRelay with exponential backoff (2s base, capped at 30s) until both land. The retry timer is cancelled on dispose and previously swallowed errors are now logged. Verified live on the Android emulator: cold-start logs show the manager rate-limited, then a 2s retry succeeding and desktop-created groups rendering. Regression tests cover retry-until-adopted, retry stopping after success, and dispose cancelling pending retries. Co-authored-by: npub1223z34hd7vtwc6qj4s7flsxkj644nlre2nthu7lrrmkumhu3xddsrx9r6w <52a228d6edf316ec6812ac3c9fc0d696ab59fc7954d77e7be31eedcddf91335b@buzz.block.builderlab.xyz> Signed-off-by: npub1223z34hd7vtwc6qj4s7flsxkj644nlre2nthu7lrrmkumhu3xddsrx9r6w <52a228d6edf316ec6812ac3c9fc0d696ab59fc7954d77e7be31eedcddf91335b@buzz.block.builderlab.xyz>
wpfleger96
left a comment
There was a problem hiding this comment.
Reviewed at head c5f1d9a38. Combined and deduplicated feedback from three independent reviewers (Paul, Thufir, Duncan) — each formed findings separately before seeing the others.
This is a good PR: small, single-purpose, and the emulator reproduction in the description is exactly the right kind of evidence. Diagnosing the catch (_) swallowing as the reason the manager kept empty state forever is correct, and the retry is the right shape. No CRITICAL findings.
One IMPORTANT: the retry can fail to fire in the very scenario it targets, because subscribe() reports success on a readiness timeout. Details inline. Two MINORs on the backoff, also inline.
Worth stating what we verified rather than assumed — the null-vs-false distinction the retry rests on does hold. A rate-limited REQ produces a relay CLOSED (crates/buzz-relay/src/connection.rs:655-679 routes every admission rejection through request_rejection_message, which emits CLOSED whenever a sub_id is present, and a REQ always has one), which relay_session.dart:503-509 turns into a thrown error rather than an empty result. So a rate-limit can't masquerade as "relay has no sections blob." We specifically went looking for a hole there and there isn't one.
CI is green, and none of the findings below are CI-detectable.
…ess timeout subscribe() resolves successfully when its 500ms readiness wait times out, so under cold-start load a rate-limit CLOSED can land after the manager already recorded the subscription as established. relay_session then removes the live subscription and only invokes onClosed — which this manager didn't pass — leaving a dead subscription with no retry. Pass onClosed (same pattern as observer_subscription.dart), clear the subscription state, and re-schedule _syncWithRelay. Regression tests cover: late CLOSED triggers exactly one replacement subscription and remote state is adopted on it; late CLOSED after dispose schedules nothing. Addresses wpfleger96 review (IMPORTANT) on #3004. Co-authored-by: npub1223z34hd7vtwc6qj4s7flsxkj644nlre2nthu7lrrmkumhu3xddsrx9r6w <52a228d6edf316ec6812ac3c9fc0d696ab59fc7954d77e7be31eedcddf91335b@buzz.block.builderlab.xyz> Signed-off-by: npub1223z34hd7vtwc6qj4s7flsxkj644nlre2nthu7lrrmkumhu3xddsrx9r6w <52a228d6edf316ec6812ac3c9fc0d696ab59fc7954d77e7be31eedcddf91335b@buzz.block.builderlab.xyz>
The inner min(attempt, 5) is dead at the default 2s base (the outer 30s clamp dominates), but it is deliberate: the consecutive-failure counter is unbounded and an unchecked << past 62 wraps negative, turning the retry Timer into a hot loop. Say so in a comment instead of leaving reviewers to wonder. Addresses wpfleger96 review (MINOR, backoff cap) on #3004. Co-authored-by: npub1223z34hd7vtwc6qj4s7flsxkj644nlre2nthu7lrrmkumhu3xddsrx9r6w <52a228d6edf316ec6812ac3c9fc0d696ab59fc7954d77e7be31eedcddf91335b@buzz.block.builderlab.xyz> Signed-off-by: npub1223z34hd7vtwc6qj4s7flsxkj644nlre2nthu7lrrmkumhu3xddsrx9r6w <52a228d6edf316ec6812ac3c9fc0d696ab59fc7954d77e7be31eedcddf91335b@buzz.block.builderlab.xyz>
…ounded retries The attempt counter previously never reset, so a transient failure after a rough cold start would wait the full 30s ceiling instead of the 2s base. Reset it once fetch and subscription have both succeeded, and document that lifetime-unbounded retries are an intentional policy so a future reader doesn't bound the loop and reintroduce the permanent-give-up bug. Regression test proves a post-recovery failure retries at ~base delay rather than the climbed backoff. Addresses wpfleger96 review (MINOR, counter reset) on #3004. Co-authored-by: npub1223z34hd7vtwc6qj4s7flsxkj644nlre2nthu7lrrmkumhu3xddsrx9r6w <52a228d6edf316ec6812ac3c9fc0d696ab59fc7954d77e7be31eedcddf91335b@buzz.block.builderlab.xyz> Signed-off-by: npub1223z34hd7vtwc6qj4s7flsxkj644nlre2nthu7lrrmkumhu3xddsrx9r6w <52a228d6edf316ec6812ac3c9fc0d696ab59fc7954d77e7be31eedcddf91335b@buzz.block.builderlab.xyz>
* origin/main: Revert "fix(cli,relay): resolve agents by verified owner" (block#3168) feat(desktop): redesign agent runtime settings (block#3093) fix(mobile): match markContextRead signature in activity test fake (block#3158) fix(desktop): use forward slashes for git credential.helper on Windows (block#3023) fix(mobile): tapping threaded message in Inbox navigates to top level of channel (block#2103) fix(mobile): retry channel-sections startup sync when relay rate-limits cold start (block#3004)
Co-authored-by: Kalvin Chau <kalvin@block.xyz> Signed-off-by: Kalvin Chau <kalvin@block.xyz> * origin/main: chore(deps): update plugin org.jetbrains.kotlin.android to v2.2.21 (#3058) resolve findings (#3150) Revert "fix(cli,relay): resolve agents by verified owner" (#3168) feat(desktop): redesign agent runtime settings (#3093) fix(mobile): match markContextRead signature in activity test fake (#3158) fix(desktop): use forward slashes for git credential.helper on Windows (#3023) fix(mobile): tapping threaded message in Inbox navigates to top level of channel (#2103) fix(mobile): retry channel-sections startup sync when relay rate-limits cold start (#3004) chore(desktop): add AgentCreationPreview file-size override to unblock main CI (#3154) fix(cli,relay): resolve agents by verified owner (#2615) fix(desktop): make the test loader work on Windows (#2758) fix(desktop): make lint and unit-test gates work on Windows (#2943) feat(mobile): refactor Activity behavior and ui (#2889) feat(desktop): add search to agent emoji picker (#2630) fix(desktop): keep identity key help dialog readable in dark mode (#2854) feat(acp): title agent sessions from the agent and channel name (#3028) feat(mobile): bring message actions to desktop parity (#3070) Signed-off-by: npub122y0pqkertljmedu303rl0aqrj3w8pvu43t6jxm6875lzg6f2pwqegc3xc <5288f082d91aff2de5bc8be23fbfa01ca2e3859cac57a91b7a3fa9f12349505c@buzz.block.builderlab.xyz>
…inks * origin/main: (24 commits) Refine pending message status (#3153) feat(admin): show reported message content in report detail (#3149) fix(desktop): recover full local storage on startup (#3182) Replace mobile reconnect banners with skeleton shimmer (#3143) fix(desktop): keep collapsed table separators out of spoilers (#3169) chore(deps): update plugin org.jetbrains.kotlin.android to v2.2.21 (#3058) resolve findings (#3150) Revert "fix(cli,relay): resolve agents by verified owner" (#3168) feat(desktop): redesign agent runtime settings (#3093) fix(mobile): match markContextRead signature in activity test fake (#3158) fix(desktop): use forward slashes for git credential.helper on Windows (#3023) fix(mobile): tapping threaded message in Inbox navigates to top level of channel (#2103) fix(mobile): retry channel-sections startup sync when relay rate-limits cold start (#3004) chore(desktop): add AgentCreationPreview file-size override to unblock main CI (#3154) fix(cli,relay): resolve agents by verified owner (#2615) fix(desktop): make the test loader work on Windows (#2758) fix(desktop): make lint and unit-test gates work on Windows (#2943) feat(mobile): refactor Activity behavior and ui (#2889) feat(desktop): add search to agent emoji picker (#2630) fix(desktop): keep identity key help dialog readable in dark mode (#2854) ... Signed-off-by: npub1dccv64krpcpse5cmkzfeh998cftungyatw3djt8jwdw6g43f7fyqzzmrf7 <6e30cd56c30e030cd31bb0939b94a7c257c9a09d5ba2d92cf2735da45629f248@buzz.block.builderlab.xyz> Co-authored-by: npub1dccv64krpcpse5cmkzfeh998cftungyatw3djt8jwdw6g43f7fyqzzmrf7 <6e30cd56c30e030cd31bb0939b94a7c257c9a09d5ba2d92cf2735da45629f248@buzz.block.builderlab.xyz>
Category: fix
User Impact: Channel groups created on desktop now reliably appear on Android and iOS on cold start, instead of falling back to the default ungrouped list.
Problem: On mobile cold start, ChannelsNotifier fires ~25 per-channel REQs at once, exhausting the relay's per-connection rate-limit quota.
ChannelSectionsManagerthen gets BOTH its one-shot history fetch and its live subscription rejected withrate-limited: quota exceeded— and both errors were silently swallowed (catch (_)) with no retry, so the manager kept the local (empty/default) store forever. Restarting the app repeats the same storm, so Android reliably lost the race every launch. Captured live on the emulator with instrumentation.Solution: Track whether the startup fetch and the live subscription have each succeeded, and retry
_syncWithRelaywith exponential backoff (2s base, shift-capped, 30s max) until both land. The retry timer is cancelled on dispose, and previously-swallowed errors are now logged.Based directly on
main— independent of #2829 (which fixes the write path: unpublished local edits being clobbered). The analogous retry forChannelSortManagerlives in #2829, since that manager is introduced there.File changes
mobile/lib/features/channels/channel_sections/channel_sections_manager.dart
Extract the startup fetch + live-subscription into
_syncWithRelay, track success of each step, and schedule a backoff retry until both succeed._fetchAndMergeand_startLiveSubscriptionnow report success; swallowed errors are logged; retry timer cancelled on dispose.startupRetryBaseDelayctor param is test-visible.mobile/test/features/channels/channel_sections/channel_sections_manager_test.dart
New regression tests with a rate-limiting relay fake: remote sections are adopted after retries; retry stops once fetch + subscription succeed; dispose cancels pending retries.
Reproduction Steps
fetch FAILED: Exception: rate-limited: quota exceededand the live subscription failing, then silence — the channel list renders the default ungrouped list forever, surviving app restarts.startup sync incomplete; retrying in 2000ms (attempt 1), the retry succeeds, and the desktop-created groups render.Verification
compose_bar_test, 1×channels_page_test) reproduce on unmodifiedmain(74b63e1) — pre-existing, unrelated.flutter analyzeclean on both touched files.Originating thread: Buzz channel ed3994af-0949-447c-be00-29f03965b52e, root 4bf7cbfd48bf.