feat: offline disk overflow + network-aware flush#22
Merged
Conversation
Add offline disk overflow capability to PersistableEventQueue: - Events that overflow from memory while offline are buffered and batched to a separate disk file (offline-overflow.v1.json) - On reconnect, overflow drains directly to network via Dispatcher.sendBatchDirect() without loading into memory queue - Respects maxOfflineDiskEvents cap (default 10000) - EventDiskStore now accepts configurable filename for separate stores - MetaRouterAnalyticsClient wires overflow enable/disable on network transitions and triggers drain on reconnect Tests cover: overflow buffering, batch writes, disk cap enforcement, drain-to-network success/failure, app-kill-relaunch scenario, toggle behavior, sendBatchDirect, and configurable filename.
Test Results 58 files ± 0 58 suites ±0 1m 7s ⏱️ +12s Results for commit 934af10. ± Comparison against base commit a96ed6f. This pull request removes 18 and adds 32 tests. Note that renamed tests count towards both.♻️ This comment has been updated with latest results. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds offline-resilient event delivery to the Android SDK. When the device is offline or the queue is under backpressure, events flush to disk instead of being dropped. On connectivity recovery or after successful online flushes, disk events drain directly to the network.
Core changes
maxQueueEventsORmaxCapacityBytes, the entire memory queue flushes to the disk store and memory resets to 0. Happens regardless of online/offline state — events are never dropped.EventQueueInterface.flushToOfflineStorage().requeueToFrontpreserves events at capacity: On failed-send retry, if adding requeued events would exceed capacity, the current memory queue flushes to disk first so older (requeued) events take priority and newer events follow via drain.queue.v1.jsonfile holds all persisted unsent events (capacity overflow, offline flush, background snapshot). Replaces the original two-file design.onFlushCompletecallback), and at app launch if online, disk events drain directly to the network viaDispatcher.sendBatchDirect()— never loaded back into the memory queue.ResponseCategoryenum: Single source of truth for HTTP status classification, used by both the main dispatcher and the overflow drain. Ensures consistent handling of 413 (batch halving), 401/403/404 (fatal config), 429 (rate limit), 5xx (retry), 4xx (drop).dispatcher.onFatalConfigErrorto stop the main pipeline, matching the main flush path.isDrainingAtomicBoolean prevents duplicate drain coroutines.diskLockmutex synchronizes disk access between the drain pipeline (suspend) and the synchronized queue methods. Drain short-circuits ifdispatcher.isNetworkPaused().hasOverflowDatavolatile flag: Lightweight gate for drain attempts, initialized viaEventDiskStore.exists()(no file parse needed).Drain response handling (mirrors dispatcher)
onFatalConfigError, stopTicket: https://app.shortcut.com/metarouter/story/36909
Test plan
flushToOfflineStoragedrains memory queue to disk when offlinerequeueToFrontflushes memory queue to disk at capacity (older events preserved at front)maxOfflineDiskEventscap enforced across multiple flushesonFatalConfigErroron 401/403/404dispatcher.isNetworkPaused()isDrainingguard)sendBatchDirectreturnsNetworkResponse?for category-aware handling