PE-9103: Add total timeout on data fetch fallback chain - #2143
Merged
Conversation
A single missing/broken tx could block sync for minutes as the fallback chain tried every gateway with CORS timeouts and 504s. Added a 15-second total timeout wrapping the entire fallback chain (primary → GAR gateways → arweave.net). Combined with the existing 5s per-request timeout, worst case for any single tx is now 15s max. This prevents the sync from appearing "stuck" — even if metadata fetches fail, the sync completes within a bounded time and the periodic sync can trigger again. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
Warning Review limit reached
More reviews will be available in 26 seconds. Learn how PR review limits work. Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file). ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits. 🚦 How do rate limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
vilenarios
added a commit
that referenced
this pull request
Jun 26, 2026
* perf: prefetch next snapshot + streaming JSON parse PE-9103
Two optimizations for snapshot loading during sync:
1. Prefetch: start downloading the next snapshot body while the
current one is being parsed and written to DB. Only 1 ahead to
avoid gateway 429s. Overlaps network I/O with CPU work.
2. Streaming parse: instead of jsonDecode on the entire snapshot
(which builds a massive Map with all entries in memory), scan
the JSON string for individual txSnapshot object boundaries
using brace counting and parse each one separately. This:
- Avoids building the full nested Map (lower peak memory)
- Starts yielding transactions immediately
- Still needs the full string downloaded, but parsing is
incremental
For a drive with 3 snapshots of ~15MB, ~10MB, ~5MB:
Before: download1 + parse1 + download2 + parse2 + download3 + parse3
After: download1 + [parse1 || download2] + [parse2 || download3] + parse3
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Revert "perf: prefetch next snapshot + streaming JSON parse PE-9103"
This reverts commit 7833178.
* PE-9103: Bulk-load revision lookups instead of per-entity DB queries (#2142)
* perf: bulk-load revision lookups instead of per-entity DB queries PE-9103
During sync, for each file/folder entity, the code issued individual
SELECT queries to find the latest and oldest revisions. For a drive
with 19k files, that was 38k+ individual DB queries (19k latest +
19k oldest for files, plus similar for folders).
Changes:
- Add 4 bulk SQL queries to drive_queries.drift (latest/oldest ×
files/folders) using GROUP BY with MAX/MIN dateCreated
- Add 4 helper methods to DriveDao returning Map<entityId, Revision>
- Pre-load all revision maps at the start of each transaction chunk
- Pass maps through to _addNewFileEntityRevisions,
_addNewFolderEntityRevisions, _computeRefreshedFileEntriesFromRevisions,
_computeRefreshedFolderEntriesFromRevisions
- Update latestRevisionsCache as new revisions are inserted so
subsequent sub-batches see fresh data
- First-sync shortcut: skip all bulk queries when drive.lastBlockHeight
is 0/null (every entity is new, no previous revisions exist)
Performance: 38,000+ individual SELECTs → 4 bulk SELECTs per chunk.
First sync: 0 queries (all skipped).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* perf: add 5s request timeout + reduce retries on data gateway PE-9103
Entity metadata fetches had NO timeout — a slow gateway could hang
indefinitely. With 2 retries × 5 gateways = 10 attempts with no
timeout, a single failed entity could take minutes.
Changes:
- Add 5-second timeout per HTTP request (metadata is tiny JSON)
- Reduce retries per gateway from 2 to 1 (move on, don't retry slow)
- Reduce GAR fallbacks from 3 to 2
Worst case per entity: 4 attempts × 5s = 20s max (was: 10 attempts,
unlimited time)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add 15s total timeout on data fetch fallback chain PE-9103 (#2143)
A single missing/broken tx could block sync for minutes as the
fallback chain tried every gateway with CORS timeouts and 504s.
Added a 15-second total timeout wrapping the entire fallback chain
(primary → GAR gateways → arweave.net). Combined with the existing
5s per-request timeout, worst case for any single tx is now 15s max.
This prevents the sync from appearing "stuck" — even if metadata
fetches fail, the sync completes within a bounded time and the
periodic sync can trigger again.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* perf: prefetch next snapshot + streaming JSON parse PE-9103 (#2141)
Two optimizations for snapshot loading during sync:
1. Prefetch: start downloading the next snapshot body while the
current one is being parsed and written to DB. Only 1 ahead to
avoid gateway 429s. Overlaps network I/O with CPU work.
2. Streaming parse: instead of jsonDecode on the entire snapshot
(which builds a massive Map with all entries in memory), scan
the JSON string for individual txSnapshot object boundaries
using brace counting and parse each one separately. This:
- Avoids building the full nested Map (lower peak memory)
- Starts yielding transactions immediately
- Still needs the full string downloaded, but parsing is
incremental
For a drive with 3 snapshots of ~15MB, ~10MB, ~5MB:
Before: download1 + parse1 + download2 + parse2 + download3 + parse3
After: download1 + [parse1 || download2] + [parse2 || download3] + parse3
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: retry gateway price fetch and handle AR cost failure gracefully PE-9103
- getPrice now retries 3 times with backoff instead of failing on a
single 502 from the gateway
- AR cost calculation in the upload modal falls back to zero estimate
on failure so the modal still opens with Turbo available instead of
crashing entirely
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: dateCreated null crash in file/folder entry refresh on first sync PE-9103
toEntryCompanion() on file/folder revision companions produces entry
companions with dateCreated = Value.absent() (schema has DEFAULT).
On first sync, oldestRevisionsCache is empty, so the fallback
.dateCreated.value returns null → JSNull crash on web.
Fixed by keeping a separate map of revision dateCreated values and
using those as the fallback instead of the entry companion's field.
Same bug pattern as the drive revision fix (line 1887), but for
files (line 1838) and folders (line 1871).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* perf: add drive owner to tx status and license gql queries PE-9126
Add an optional owners filter to the by-ID GraphQL queries that the
gateway struggles with (large ClickHouse row scans). Scoping by owner
lets the gateway prune its search space. The owner variable is nullable,
so when an owner can't be determined the query behaves exactly as before.
- TransactionStatuses, LicenseComposed, LicenseAssertions: add $owners
- getTransactionConfirmations / getLicenseComposed / getLicenseAssertions:
accept an optional owner and pass it through
- tx status (per-drive): scope to the drive's ownerAddress
- tx status (global): scope to the logged-in wallet's address
- licenses: scope to the best-known tx owner (revision/file owner)
Edge case: data txs of files pinned from other authors are owned by
those authors and won't match the owner filter; treated as best-effort.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: re-query owner-mismatched txs unscoped to avoid false failures PE-9126
Owner-scoped getTransactionConfirmations leaves cross-owner txs (e.g. the
data tx of a file pinned from another author's upload) at -1, which the
sync status logic would turn into 'failed' after the pending timeout.
Add a fallback pass: after the selective owner-scoped query, re-query any
ids still unresolved (-1) without the owner filter to determine their true
status. The residual set is normally tiny, so the selectivity win of the
first pass is preserved while confirmed cross-owner txs are no longer
misclassified as failed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: resolve owner-mismatched txs via pin owner, not unscoped re-query PE-9126
The previous fallback re-queried unresolved (-1) txs without an owner
filter, which can re-trigger the gateway's expensive row scan that the
owner scoping was meant to avoid — and, because the call is wrapped in a
5s timeout, a slow/erroring fallback discards the whole batch's results
and makes no progress, repeating every sync.
Replace it with a selective approach: only re-query unresolved ids whose
real on-chain owner is known locally. Currently that's pinned files,
whose data tx is owned by the original uploader (pinnedDataOwnerAddress).
These are re-queried scoped to that owner, so every query stays selective.
Genuinely missing txs have no override and are left unresolved, handled by
the caller's existing pending/failed logic — no unscoped scan, no retry
storm on the common "not found yet" case.
- add pinnedFileRevisions drift query
- getTransactionConfirmations: ownerOverrides param replaces unscoped pass
- sync repo: build pin owner overrides and pass them through
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: make pinned-owner confirmation recovery best-effort PE-9126
The second (pinned-owner) pass merges into the already-populated first-pass
map, so its failure must never discard first-pass progress. Wrap it in a
try/catch and bound it with its own 3s timeout: even if the pin re-queries
error or stall, the confirmations resolved by the owner-scoped first pass
are still returned.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf: preserve resolved confirmations across a timeout via verified sink PE-9126
getTransactionConfirmations is wrapped in a 5s timeout by the sync status
update; on expiry it previously returned an empty map, discarding every
confirmation already resolved that cycle (the whole 5000-tx page).
Add an optional caller-owned verifiedSink that the method populates with
each resolved confirmation (>= 0 only) as queries complete. The callers pass
one in and, on timeout, fall back to it instead of an empty map — so work
done before the deadline (including a fully-completed first pass when only
the pinned-owner pass is slow) is applied rather than thrown away.
Because the sink holds only positive verifications and never the -1
"not found" placeholders, applying it after a partial/timed-out run only
upgrades txs to confirmed and never marks anything failed off incomplete data.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: bound confirmation fan-out and use type-safe id filtering PE-9126
- Cap the per-chunk GraphQL fan-out in getTransactionConfirmations to
maxConcurrentDataFetches instead of launching every chunk at once, so a
large pending-tx page can't burst into a concurrent-retry storm against
the gateway (matches the throttling on the other gateway-heavy paths).
- Replace `sublist(...) as List<String>?` with `.whereType<String>().toList()`.
The cast threw a TypeError for the pinned-owner pass (whose ids list is
typed List<String?>), which the best-effort catch swallowed — silently
disabling pin recovery. The filter yields a real List<String> regardless
of input type.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* PE-9103: Fix empty explorer after drive attach (#2145)
* fix: empty explorer after drive attach PE-9103
Two bugs caused the explorer to show a permanent spinner after
attaching a drive (data was in DB but UI didn't update):
1. buildWhen guard in drive_detail_page.dart blocked BlocBuilder
rebuilds when SyncCubit was SyncInProgress. If DriveDetailCubit
emitted DriveDetailLoadSuccess during sync, the emission was
permanently skipped with no replay mechanism. Removed the sync
check — DriveDetailCubit already gates emissions via
waitCurrentSync() in the Rx.combineLatest3 callback.
2. startSyncForDrive silently aborted when a sync was in progress.
The .then(selectDrive) still fired, selecting a drive whose
content was never synced. Changed to await waitCurrentSync()
so the single-drive sync runs after the current sync finishes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* perf: faster snapshot validation — skip fallback on 404, reduce timeouts PE-9103
Snapshot validation was slow on localhost (and anywhere Solana RPC is
unavailable): 2 HEAD retries × 10s timeout + GAR fallback via Solana
= 25+ seconds per snapshot. With 16 drives having multiple snapshots,
sync appeared stuck.
Changes:
- 1 HEAD attempt instead of 2 (fail fast, fall back to GQL)
- HEAD timeout 10s → 5s
- GAR list timeout 5s → 3s
- Skip GAR fallback entirely when primary returned 404 (snapshot
doesn't exist, no point trying other gateways via Solana RPC)
- Only try GAR fallback for transient errors (timeout, 5xx)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: guard startSyncForDrive race after waitCurrentSync PE-9103
Two callers could both pass the SyncInProgress guard after
waitCurrentSync() returned, causing concurrent single-drive syncs.
Re-check state after wait — if another sync started, bail out.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: bump arweave-dart to v4.0.2 to fix file download crash
Pulls in the TransactionData null-field fix so downloads no longer crash
on web with "type 'JSNull' is not a subtype of type 'String'" when a
gateway returns null for optional tx fields (e.g. anchor on
turbo-gateway.com). Updates both the dependency and the override.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* chore(version): bump version to 2.84.0
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Ariel Melendez <ariel@ardrive.io>
Co-authored-by: arielmelendez <ariel.l.melendez@gmail.com>
vilenarios
added a commit
that referenced
this pull request
Jul 3, 2026
* perf: prefetch next snapshot + streaming JSON parse PE-9103
Two optimizations for snapshot loading during sync:
1. Prefetch: start downloading the next snapshot body while the
current one is being parsed and written to DB. Only 1 ahead to
avoid gateway 429s. Overlaps network I/O with CPU work.
2. Streaming parse: instead of jsonDecode on the entire snapshot
(which builds a massive Map with all entries in memory), scan
the JSON string for individual txSnapshot object boundaries
using brace counting and parse each one separately. This:
- Avoids building the full nested Map (lower peak memory)
- Starts yielding transactions immediately
- Still needs the full string downloaded, but parsing is
incremental
For a drive with 3 snapshots of ~15MB, ~10MB, ~5MB:
Before: download1 + parse1 + download2 + parse2 + download3 + parse3
After: download1 + [parse1 || download2] + [parse2 || download3] + parse3
* Revert "perf: prefetch next snapshot + streaming JSON parse PE-9103"
This reverts commit 7833178.
* PE-9103: Bulk-load revision lookups instead of per-entity DB queries (#2142)
* perf: bulk-load revision lookups instead of per-entity DB queries PE-9103
During sync, for each file/folder entity, the code issued individual
SELECT queries to find the latest and oldest revisions. For a drive
with 19k files, that was 38k+ individual DB queries (19k latest +
19k oldest for files, plus similar for folders).
Changes:
- Add 4 bulk SQL queries to drive_queries.drift (latest/oldest ×
files/folders) using GROUP BY with MAX/MIN dateCreated
- Add 4 helper methods to DriveDao returning Map<entityId, Revision>
- Pre-load all revision maps at the start of each transaction chunk
- Pass maps through to _addNewFileEntityRevisions,
_addNewFolderEntityRevisions, _computeRefreshedFileEntriesFromRevisions,
_computeRefreshedFolderEntriesFromRevisions
- Update latestRevisionsCache as new revisions are inserted so
subsequent sub-batches see fresh data
- First-sync shortcut: skip all bulk queries when drive.lastBlockHeight
is 0/null (every entity is new, no previous revisions exist)
Performance: 38,000+ individual SELECTs → 4 bulk SELECTs per chunk.
First sync: 0 queries (all skipped).
* perf: add 5s request timeout + reduce retries on data gateway PE-9103
Entity metadata fetches had NO timeout — a slow gateway could hang
indefinitely. With 2 retries × 5 gateways = 10 attempts with no
timeout, a single failed entity could take minutes.
Changes:
- Add 5-second timeout per HTTP request (metadata is tiny JSON)
- Reduce retries per gateway from 2 to 1 (move on, don't retry slow)
- Reduce GAR fallbacks from 3 to 2
Worst case per entity: 4 attempts × 5s = 20s max (was: 10 attempts,
unlimited time)
---------
* fix: add 15s total timeout on data fetch fallback chain PE-9103 (#2143)
A single missing/broken tx could block sync for minutes as the
fallback chain tried every gateway with CORS timeouts and 504s.
Added a 15-second total timeout wrapping the entire fallback chain
(primary → GAR gateways → arweave.net). Combined with the existing
5s per-request timeout, worst case for any single tx is now 15s max.
This prevents the sync from appearing "stuck" — even if metadata
fetches fail, the sync completes within a bounded time and the
periodic sync can trigger again.
* perf: prefetch next snapshot + streaming JSON parse PE-9103 (#2141)
Two optimizations for snapshot loading during sync:
1. Prefetch: start downloading the next snapshot body while the
current one is being parsed and written to DB. Only 1 ahead to
avoid gateway 429s. Overlaps network I/O with CPU work.
2. Streaming parse: instead of jsonDecode on the entire snapshot
(which builds a massive Map with all entries in memory), scan
the JSON string for individual txSnapshot object boundaries
using brace counting and parse each one separately. This:
- Avoids building the full nested Map (lower peak memory)
- Starts yielding transactions immediately
- Still needs the full string downloaded, but parsing is
incremental
For a drive with 3 snapshots of ~15MB, ~10MB, ~5MB:
Before: download1 + parse1 + download2 + parse2 + download3 + parse3
After: download1 + [parse1 || download2] + [parse2 || download3] + parse3
* fix: retry gateway price fetch and handle AR cost failure gracefully PE-9103
- getPrice now retries 3 times with backoff instead of failing on a
single 502 from the gateway
- AR cost calculation in the upload modal falls back to zero estimate
on failure so the modal still opens with Turbo available instead of
crashing entirely
* fix: dateCreated null crash in file/folder entry refresh on first sync PE-9103
toEntryCompanion() on file/folder revision companions produces entry
companions with dateCreated = Value.absent() (schema has DEFAULT).
On first sync, oldestRevisionsCache is empty, so the fallback
.dateCreated.value returns null → JSNull crash on web.
Fixed by keeping a separate map of revision dateCreated values and
using those as the fallback instead of the entry companion's field.
Same bug pattern as the drive revision fix (line 1887), but for
files (line 1838) and folders (line 1871).
* perf: add drive owner to tx status and license gql queries PE-9126
Add an optional owners filter to the by-ID GraphQL queries that the
gateway struggles with (large ClickHouse row scans). Scoping by owner
lets the gateway prune its search space. The owner variable is nullable,
so when an owner can't be determined the query behaves exactly as before.
- TransactionStatuses, LicenseComposed, LicenseAssertions: add $owners
- getTransactionConfirmations / getLicenseComposed / getLicenseAssertions:
accept an optional owner and pass it through
- tx status (per-drive): scope to the drive's ownerAddress
- tx status (global): scope to the logged-in wallet's address
- licenses: scope to the best-known tx owner (revision/file owner)
Edge case: data txs of files pinned from other authors are owned by
those authors and won't match the owner filter; treated as best-effort.
* fix: re-query owner-mismatched txs unscoped to avoid false failures PE-9126
Owner-scoped getTransactionConfirmations leaves cross-owner txs (e.g. the
data tx of a file pinned from another author's upload) at -1, which the
sync status logic would turn into 'failed' after the pending timeout.
Add a fallback pass: after the selective owner-scoped query, re-query any
ids still unresolved (-1) without the owner filter to determine their true
status. The residual set is normally tiny, so the selectivity win of the
first pass is preserved while confirmed cross-owner txs are no longer
misclassified as failed.
* fix: resolve owner-mismatched txs via pin owner, not unscoped re-query PE-9126
The previous fallback re-queried unresolved (-1) txs without an owner
filter, which can re-trigger the gateway's expensive row scan that the
owner scoping was meant to avoid — and, because the call is wrapped in a
5s timeout, a slow/erroring fallback discards the whole batch's results
and makes no progress, repeating every sync.
Replace it with a selective approach: only re-query unresolved ids whose
real on-chain owner is known locally. Currently that's pinned files,
whose data tx is owned by the original uploader (pinnedDataOwnerAddress).
These are re-queried scoped to that owner, so every query stays selective.
Genuinely missing txs have no override and are left unresolved, handled by
the caller's existing pending/failed logic — no unscoped scan, no retry
storm on the common "not found yet" case.
- add pinnedFileRevisions drift query
- getTransactionConfirmations: ownerOverrides param replaces unscoped pass
- sync repo: build pin owner overrides and pass them through
* fix: make pinned-owner confirmation recovery best-effort PE-9126
The second (pinned-owner) pass merges into the already-populated first-pass
map, so its failure must never discard first-pass progress. Wrap it in a
try/catch and bound it with its own 3s timeout: even if the pin re-queries
error or stall, the confirmations resolved by the owner-scoped first pass
are still returned.
* perf: preserve resolved confirmations across a timeout via verified sink PE-9126
getTransactionConfirmations is wrapped in a 5s timeout by the sync status
update; on expiry it previously returned an empty map, discarding every
confirmation already resolved that cycle (the whole 5000-tx page).
Add an optional caller-owned verifiedSink that the method populates with
each resolved confirmation (>= 0 only) as queries complete. The callers pass
one in and, on timeout, fall back to it instead of an empty map — so work
done before the deadline (including a fully-completed first pass when only
the pinned-owner pass is slow) is applied rather than thrown away.
Because the sink holds only positive verifications and never the -1
"not found" placeholders, applying it after a partial/timed-out run only
upgrades txs to confirmed and never marks anything failed off incomplete data.
* fix: bound confirmation fan-out and use type-safe id filtering PE-9126
- Cap the per-chunk GraphQL fan-out in getTransactionConfirmations to
maxConcurrentDataFetches instead of launching every chunk at once, so a
large pending-tx page can't burst into a concurrent-retry storm against
the gateway (matches the throttling on the other gateway-heavy paths).
- Replace `sublist(...) as List<String>?` with `.whereType<String>().toList()`.
The cast threw a TypeError for the pinned-owner pass (whose ids list is
typed List<String?>), which the best-effort catch swallowed — silently
disabling pin recovery. The filter yields a real List<String> regardless
of input type.
* PE-9103: Fix empty explorer after drive attach (#2145)
* fix: empty explorer after drive attach PE-9103
Two bugs caused the explorer to show a permanent spinner after
attaching a drive (data was in DB but UI didn't update):
1. buildWhen guard in drive_detail_page.dart blocked BlocBuilder
rebuilds when SyncCubit was SyncInProgress. If DriveDetailCubit
emitted DriveDetailLoadSuccess during sync, the emission was
permanently skipped with no replay mechanism. Removed the sync
check — DriveDetailCubit already gates emissions via
waitCurrentSync() in the Rx.combineLatest3 callback.
2. startSyncForDrive silently aborted when a sync was in progress.
The .then(selectDrive) still fired, selecting a drive whose
content was never synced. Changed to await waitCurrentSync()
so the single-drive sync runs after the current sync finishes.
* perf: faster snapshot validation — skip fallback on 404, reduce timeouts PE-9103
Snapshot validation was slow on localhost (and anywhere Solana RPC is
unavailable): 2 HEAD retries × 10s timeout + GAR fallback via Solana
= 25+ seconds per snapshot. With 16 drives having multiple snapshots,
sync appeared stuck.
Changes:
- 1 HEAD attempt instead of 2 (fail fast, fall back to GQL)
- HEAD timeout 10s → 5s
- GAR list timeout 5s → 3s
- Skip GAR fallback entirely when primary returned 404 (snapshot
doesn't exist, no point trying other gateways via Solana RPC)
- Only try GAR fallback for transient errors (timeout, 5xx)
* fix: guard startSyncForDrive race after waitCurrentSync PE-9103
Two callers could both pass the SyncInProgress guard after
waitCurrentSync() returned, causing concurrent single-drive syncs.
Re-check state after wait — if another sync started, bail out.
---------
* fix: bump arweave-dart to v4.0.2 to fix file download crash
Pulls in the TransactionData null-field fix so downloads no longer crash
on web with "type 'JSNull' is not a subtype of type 'String'" when a
gateway returns null for optional tx fields (e.g. anchor on
turbo-gateway.com). Updates both the dependency and the override.
* chore(version): bump version to 2.84.0
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Ariel Melendez <ariel@ardrive.io>
Co-authored-by: arielmelendez <ariel.l.melendez@gmail.com>
vilenarios
added a commit
that referenced
this pull request
Jul 3, 2026
* perf: prefetch next snapshot + streaming JSON parse PE-9103
Two optimizations for snapshot loading during sync:
1. Prefetch: start downloading the next snapshot body while the
current one is being parsed and written to DB. Only 1 ahead to
avoid gateway 429s. Overlaps network I/O with CPU work.
2. Streaming parse: instead of jsonDecode on the entire snapshot
(which builds a massive Map with all entries in memory), scan
the JSON string for individual txSnapshot object boundaries
using brace counting and parse each one separately. This:
- Avoids building the full nested Map (lower peak memory)
- Starts yielding transactions immediately
- Still needs the full string downloaded, but parsing is
incremental
For a drive with 3 snapshots of ~15MB, ~10MB, ~5MB:
Before: download1 + parse1 + download2 + parse2 + download3 + parse3
After: download1 + [parse1 || download2] + [parse2 || download3] + parse3
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Revert "perf: prefetch next snapshot + streaming JSON parse PE-9103"
This reverts commit 78331785e351f7dc5c02079ff82d0db6cfc76e6e.
* PE-9103: Bulk-load revision lookups instead of per-entity DB queries (#2142)
* perf: bulk-load revision lookups instead of per-entity DB queries PE-9103
During sync, for each file/folder entity, the code issued individual
SELECT queries to find the latest and oldest revisions. For a drive
with 19k files, that was 38k+ individual DB queries (19k latest +
19k oldest for files, plus similar for folders).
Changes:
- Add 4 bulk SQL queries to drive_queries.drift (latest/oldest ×
files/folders) using GROUP BY with MAX/MIN dateCreated
- Add 4 helper methods to DriveDao returning Map<entityId, Revision>
- Pre-load all revision maps at the start of each transaction chunk
- Pass maps through to _addNewFileEntityRevisions,
_addNewFolderEntityRevisions, _computeRefreshedFileEntriesFromRevisions,
_computeRefreshedFolderEntriesFromRevisions
- Update latestRevisionsCache as new revisions are inserted so
subsequent sub-batches see fresh data
- First-sync shortcut: skip all bulk queries when drive.lastBlockHeight
is 0/null (every entity is new, no previous revisions exist)
Performance: 38,000+ individual SELECTs → 4 bulk SELECTs per chunk.
First sync: 0 queries (all skipped).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* perf: add 5s request timeout + reduce retries on data gateway PE-9103
Entity metadata fetches had NO timeout — a slow gateway could hang
indefinitely. With 2 retries × 5 gateways = 10 attempts with no
timeout, a single failed entity could take minutes.
Changes:
- Add 5-second timeout per HTTP request (metadata is tiny JSON)
- Reduce retries per gateway from 2 to 1 (move on, don't retry slow)
- Reduce GAR fallbacks from 3 to 2
Worst case per entity: 4 attempts × 5s = 20s max (was: 10 attempts,
unlimited time)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add 15s total timeout on data fetch fallback chain PE-9103 (#2143)
A single missing/broken tx could block sync for minutes as the
fallback chain tried every gateway with CORS timeouts and 504s.
Added a 15-second total timeout wrapping the entire fallback chain
(primary → GAR gateways → arweave.net). Combined with the existing
5s per-request timeout, worst case for any single tx is now 15s max.
This prevents the sync from appearing "stuck" — even if metadata
fetches fail, the sync completes within a bounded time and the
periodic sync can trigger again.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* perf: prefetch next snapshot + streaming JSON parse PE-9103 (#2141)
Two optimizations for snapshot loading during sync:
1. Prefetch: start downloading the next snapshot body while the
current one is being parsed and written to DB. Only 1 ahead to
avoid gateway 429s. Overlaps network I/O with CPU work.
2. Streaming parse: instead of jsonDecode on the entire snapshot
(which builds a massive Map with all entries in memory), scan
the JSON string for individual txSnapshot object boundaries
using brace counting and parse each one separately. This:
- Avoids building the full nested Map (lower peak memory)
- Starts yielding transactions immediately
- Still needs the full string downloaded, but parsing is
incremental
For a drive with 3 snapshots of ~15MB, ~10MB, ~5MB:
Before: download1 + parse1 + download2 + parse2 + download3 + parse3
After: download1 + [parse1 || download2] + [parse2 || download3] + parse3
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: retry gateway price fetch and handle AR cost failure gracefully PE-9103
- getPrice now retries 3 times with backoff instead of failing on a
single 502 from the gateway
- AR cost calculation in the upload modal falls back to zero estimate
on failure so the modal still opens with Turbo available instead of
crashing entirely
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: dateCreated null crash in file/folder entry refresh on first sync PE-9103
toEntryCompanion() on file/folder revision companions produces entry
companions with dateCreated = Value.absent() (schema has DEFAULT).
On first sync, oldestRevisionsCache is empty, so the fallback
.dateCreated.value returns null → JSNull crash on web.
Fixed by keeping a separate map of revision dateCreated values and
using those as the fallback instead of the entry companion's field.
Same bug pattern as the drive revision fix (line 1887), but for
files (line 1838) and folders (line 1871).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* perf: add drive owner to tx status and license gql queries PE-9126
Add an optional owners filter to the by-ID GraphQL queries that the
gateway struggles with (large ClickHouse row scans). Scoping by owner
lets the gateway prune its search space. The owner variable is nullable,
so when an owner can't be determined the query behaves exactly as before.
- TransactionStatuses, LicenseComposed, LicenseAssertions: add $owners
- getTransactionConfirmations / getLicenseComposed / getLicenseAssertions:
accept an optional owner and pass it through
- tx status (per-drive): scope to the drive's ownerAddress
- tx status (global): scope to the logged-in wallet's address
- licenses: scope to the best-known tx owner (revision/file owner)
Edge case: data txs of files pinned from other authors are owned by
those authors and won't match the owner filter; treated as best-effort.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: re-query owner-mismatched txs unscoped to avoid false failures PE-9126
Owner-scoped getTransactionConfirmations leaves cross-owner txs (e.g. the
data tx of a file pinned from another author's upload) at -1, which the
sync status logic would turn into 'failed' after the pending timeout.
Add a fallback pass: after the selective owner-scoped query, re-query any
ids still unresolved (-1) without the owner filter to determine their true
status. The residual set is normally tiny, so the selectivity win of the
first pass is preserved while confirmed cross-owner txs are no longer
misclassified as failed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: resolve owner-mismatched txs via pin owner, not unscoped re-query PE-9126
The previous fallback re-queried unresolved (-1) txs without an owner
filter, which can re-trigger the gateway's expensive row scan that the
owner scoping was meant to avoid — and, because the call is wrapped in a
5s timeout, a slow/erroring fallback discards the whole batch's results
and makes no progress, repeating every sync.
Replace it with a selective approach: only re-query unresolved ids whose
real on-chain owner is known locally. Currently that's pinned files,
whose data tx is owned by the original uploader (pinnedDataOwnerAddress).
These are re-queried scoped to that owner, so every query stays selective.
Genuinely missing txs have no override and are left unresolved, handled by
the caller's existing pending/failed logic — no unscoped scan, no retry
storm on the common "not found yet" case.
- add pinnedFileRevisions drift query
- getTransactionConfirmations: ownerOverrides param replaces unscoped pass
- sync repo: build pin owner overrides and pass them through
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: make pinned-owner confirmation recovery best-effort PE-9126
The second (pinned-owner) pass merges into the already-populated first-pass
map, so its failure must never discard first-pass progress. Wrap it in a
try/catch and bound it with its own 3s timeout: even if the pin re-queries
error or stall, the confirmations resolved by the owner-scoped first pass
are still returned.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf: preserve resolved confirmations across a timeout via verified sink PE-9126
getTransactionConfirmations is wrapped in a 5s timeout by the sync status
update; on expiry it previously returned an empty map, discarding every
confirmation already resolved that cycle (the whole 5000-tx page).
Add an optional caller-owned verifiedSink that the method populates with
each resolved confirmation (>= 0 only) as queries complete. The callers pass
one in and, on timeout, fall back to it instead of an empty map — so work
done before the deadline (including a fully-completed first pass when only
the pinned-owner pass is slow) is applied rather than thrown away.
Because the sink holds only positive verifications and never the -1
"not found" placeholders, applying it after a partial/timed-out run only
upgrades txs to confirmed and never marks anything failed off incomplete data.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: bound confirmation fan-out and use type-safe id filtering PE-9126
- Cap the per-chunk GraphQL fan-out in getTransactionConfirmations to
maxConcurrentDataFetches instead of launching every chunk at once, so a
large pending-tx page can't burst into a concurrent-retry storm against
the gateway (matches the throttling on the other gateway-heavy paths).
- Replace `sublist(...) as List<String>?` with `.whereType<String>().toList()`.
The cast threw a TypeError for the pinned-owner pass (whose ids list is
typed List<String?>), which the best-effort catch swallowed — silently
disabling pin recovery. The filter yields a real List<String> regardless
of input type.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* PE-9103: Fix empty explorer after drive attach (#2145)
* fix: empty explorer after drive attach PE-9103
Two bugs caused the explorer to show a permanent spinner after
attaching a drive (data was in DB but UI didn't update):
1. buildWhen guard in drive_detail_page.dart blocked BlocBuilder
rebuilds when SyncCubit was SyncInProgress. If DriveDetailCubit
emitted DriveDetailLoadSuccess during sync, the emission was
permanently skipped with no replay mechanism. Removed the sync
check — DriveDetailCubit already gates emissions via
waitCurrentSync() in the Rx.combineLatest3 callback.
2. startSyncForDrive silently aborted when a sync was in progress.
The .then(selectDrive) still fired, selecting a drive whose
content was never synced. Changed to await waitCurrentSync()
so the single-drive sync runs after the current sync finishes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* perf: faster snapshot validation — skip fallback on 404, reduce timeouts PE-9103
Snapshot validation was slow on localhost (and anywhere Solana RPC is
unavailable): 2 HEAD retries × 10s timeout + GAR fallback via Solana
= 25+ seconds per snapshot. With 16 drives having multiple snapshots,
sync appeared stuck.
Changes:
- 1 HEAD attempt instead of 2 (fail fast, fall back to GQL)
- HEAD timeout 10s → 5s
- GAR list timeout 5s → 3s
- Skip GAR fallback entirely when primary returned 404 (snapshot
doesn't exist, no point trying other gateways via Solana RPC)
- Only try GAR fallback for transient errors (timeout, 5xx)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: guard startSyncForDrive race after waitCurrentSync PE-9103
Two callers could both pass the SyncInProgress guard after
waitCurrentSync() returned, causing concurrent single-drive syncs.
Re-check state after wait — if another sync started, bail out.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: bump arweave-dart to v4.0.2 to fix file download crash
Pulls in the TransactionData null-field fix so downloads no longer crash
on web with "type 'JSNull' is not a subtype of type 'String'" when a
gateway returns null for optional tx fields (e.g. anchor on
turbo-gateway.com). Updates both the dependency and the override.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* chore(version): bump version to 2.84.0
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* PE-9103: Modernize share file and download all modals (#2149)
* ui: modernize share file and download all modals PE-9103
Both modals used the old ArDriveStandardModal with deprecated
typography and color tokens. Updated to match the current design
system used by drive attach, upload, and other modern modals.
Share File modal:
- ArDriveStandardModal → ArDriveStandardModalNew (adds red header bar)
- Old typography (buttonNormalBold, buttonLargeRegular) → semantic
ArDriveTypographyNew (paragraphSmall, paragraphNormal)
- Old colors (themeFgDefault, themeWarningEmphasis) → colorTokens
(textHigh, textMid, textLow, strokeRed)
- ArDriveTextField → ArDriveTextFieldNew
- Warning banner wrapped in styled container with containerL1 bg
Download All Files modal:
- All 3 ArDriveStandardModal instances → ArDriveStandardModalNew
- File list items wrapped in styled containers (containerL1 bg,
rounded corners, file/folder icons, proper spacing)
- Old typography (smallBold, smallRegular) → semantic typography
- Old colors (themeFgSubtle, themeFgMuted) → colorTokens
- Error state text styled with new typography
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address CodeRabbit review issues in share/download modals PE-9103
- add explicit FileShareLoadedPendingFile state branch with localized body text
- add TODO comment for hardcoded pending warning string (needs ARB extraction)
- add close action to fallback modal in multiple file download
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: scope tx-status query per-tx by drive owner, not logged-in wallet PE-9126
The global tx-status update assumed the logged-in wallet owns every pending
tx and scoped the whole batch by walletAddress. That breaks for ATTACHED
drives owned by other wallets: their pending data txs were queried under the
wrong owner, and when no wallet is present (browsing a public/attached drive)
walletAddress is null, so TransactionStatuses went out unscoped — hitting the
gateway's expensive full-scan path and timing out.
Resolve each pending tx's owner from the drive it actually belongs to:
- add pendingDataFileRevisions drift query (read-only; no schema change)
- _buildPendingTxDriveOwners: map pending data tx id -> its drive's ownerAddress
- getTransactionConfirmations: new ownersByTxId map; first pass groups txs by
their resolved per-tx owner (map wins, else the single owner fallback) and
queries each owner once, so a batch spanning multiple drives stays selective.
A tx with no resolvable owner is left unresolved rather than queried unscoped.
- global _updateTransactionStatuses passes the per-tx owner map; walletAddress
remains only as a fallback for unmapped txs.
The per-drive path is unchanged (single drive => single owner). Pinned-owner
recovery (pass 2) still works: pins map to their drive owner in pass 1 (miss)
and are recovered under pinnedDataOwnerAddress in pass 2.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: drop unscopable txs from confirmations instead of returning -1 PE-9126
A pending tx with no resolvable owner (e.g. global path with no wallet and a
tx not mappable to a drive) is never queried, but was left at its pre-seeded
-1 in the result map. The caller reads -1 as "not found" and can age an old
pending tx into failed.
Remove such txs from the returned map so they're absent rather than -1: the
caller skips absent txs (no status change), leaving them pending to be
retried once their owner is known — the same "unknown => skip, never fail off
incomplete data" rule used by the verified-sink path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: raise tx-status timeouts above gateway upstream ceiling PE-9126
The confirmation query intermittently takes ~10s when turbo-gateway's
indexer-core circuit breaker is open (observed 9.87s with UPSTREAM_CIRCUIT_OPEN
/ "timeout of 9500ms exceeded" warnings, returning valid data). The client's
5s per-batch timeout fired first and discarded the whole batch, leaving
long-confirmed txs stuck as pending even though the gateway returned their
confirmations.
Move both timeouts above that ~9.5s ceiling:
- per-batch getTransactionConfirmations: 5s -> 15s
- overall per-drive/global status update: 10s -> 30s
Extracted as named constants with the rationale. Typical responses are ~0.5s,
so this only lengthens waits while the gateway is degraded (when we want to
wait, not drop the batch); the verified sink still preserves partial progress.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: generalize tx-status timeout comments to be backend-agnostic PE-9126
Reword the timeout rationale in terms of general gateway slowness rather than a
specific gateway's internal index/circuit-breaker/warning codes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* PE-9103: Eliminate redundant GraphQL calls in sync pipeline (#2150)
* perf: sync pipeline performance and resilience optimizations PE-9103
- skip unchanged drives via bulk GQL probe (1 query per owner replaces ~40 queries per idle drive)
- add database indexes on file_revisions.dataTxId and network_transactions.status
- bulk pre-load dateCreated for tx status updates (eliminates N+1 per-tx DB queries)
- bulk pre-load folders and drives in ghost creation (eliminates N+1 per-folder DB queries)
- batch transaction status writes using insertNewNetworkTransactions (replaces individual writes)
- replace full file_revisions table load with filtered query for snapshot tx matching
- fix DataGatewayFallback timeout budget (15s→25s) so arweave.net is reachable
- fix 4 GraphQL queries bypassing GraphQLRetry (missing retry + fallback)
- remove 200ms artificial delay between tx status batches
- bump database schema version 28→29
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: don't skip post-sync ops when all drives are unchanged PE-9103
The early return on numberOfDrivesToSync == 0 skipped transaction status
updates, ghost folder creation, and ARNS record updates. Pending
transactions from recent uploads need confirmation checks even when no
drives have new entities.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* perf: skip GraphQL call for public file downloads PE-9103
Public file downloads were calling getTransactionDetails() via GraphQL
before starting the download — entirely unnecessary since the txId is
already known locally. This GQL call was also one of the four that
bypassed GraphQLRetry, making it the likely cause of downloads failing
to start when the gateway returns 429/5xx.
Now only private/encrypted files call GraphQL (to fetch cipher/IV tags
from the data transaction). Public files start downloading immediately
with no network round-trip.
Refactored ArDriveDownloader interface: replaced TransactionCommonMixin
dataTx parameter with String txId + bool verifyDownload, since only
those two values were ever used from the full transaction object.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: hedged gateway requests, download fallback, retry UX, drive attach dedup PE-9103
hedged gateway requests:
- replace serial waterfall with staggered parallel requests in DataGatewayFallback
- fire primary immediately, launch fallbacks every 1.5s if no response
- first 200 response wins, rest ignored — worst case ~5s instead of ~20s
- applies to all metadata fetches automatically (no caller changes)
download resilience:
- add downloadWithFallback() for file downloads with same hedged pattern
- add fetchManifestWithFallback() for manifest downloads (had zero fallback)
- add stall detection: throws DownloadStalledException if no chunk for 60s
- typed exceptions: DownloadFileNotFoundException, DownloadNetworkException,
DownloadRateLimitException, DownloadStalledException
download UX:
- differentiated error dialogs: network error, file not found, rate limited
- retry button on retryable failures (network, rate limit, unknown)
- file not found shows OK only (retry won't help)
- error classification in both personal and shared download cubits
drive attach dedup:
- getDrivePrivacyForId() now returns DrivePrivacyResult with owner + tx node
- drivePrivacyLoader() passes owner to getLatestDriveEntityWithId(), skipping
redundant owner lookup — 4 GQL queries reduced to 2
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: revert metadata fetches to serial fallback, keep hedged for downloads only PE-9103
Hedged (staggered parallel) requests fire extra gateway requests when the
primary is slow but succeeds. During sync with hundreds of metadata fetches,
this wastes bandwidth and could trigger rate limits on GAR gateways.
Now:
- metadata fetches (fetchData): serial waterfall (primary → GAR → arweave.net)
- file downloads (downloadWithFallback): hedged staggered (latency-sensitive, single request)
- manifest downloads (fetchManifestWithFallback): serial waterfall
Also fixes stall detection for empty files — timer only starts after the
first chunk arrives, so 0-byte files don't trigger DownloadStalledException.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* perf: batch snapshot queries and share gateway cache PE-9103
batch snapshot queries:
- SnapshotEntityHistory.graphql now accepts $driveIds array instead of
single $driveId — fetches snapshots for all drives in one paginated query
- syncAllDrives() prefetches snapshots for all drives per owner before
the per-drive sync loop, passes results to _syncDrive()
- reduces N snapshot GQL queries (one per drive) to 1 per unique owner
- also fixes Entity-Type tag syntax: values: "snapshot" → values: ["snapshot"]
share gateway cache:
- DataGatewayFallback.cachedGateways is now public so
SnapshotValidationService can reuse the same gateway list
- syncAllDrives() passes the cache before sync starts
- eliminates 1 duplicate Solana RPC call per sync cycle
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address CodeRabbit review — stall detection + mobile GCM path PE-9103
- forward verifyDownload param to mobile AES-GCM download path
- wrap mobile GCM stream with _withStallDetection (was bypassed)
- cancel upstream subscription when stall timer fires
- guard against adding to closed StreamController in stall detection
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* perf: eliminate redundant GraphQL calls in sync pipeline PE-9103
HAR analysis of a real sync session (16 drives, no changes) revealed
130 GraphQL calls taking 376s. This commit reduces that to ~3 calls
and <1s for incremental syncs with no changes.
Changes:
- fix DriveActivityProbe: partition drives into never-synced and
previously-synced before probing. Never-synced drives (lastBlockHeight=0)
were poisoning the probe's minBlockHeight to 0, causing it to query
from genesis, overflow the page limit, and fall back to syncing ALL
drives. Now only previously-synced drives are probed.
- cache UserDriveEntityTxs in ArweaveService with event-based
invalidation. Auth flow (isExistingUser, _validateUser) and sync
(updateUserDrives) all call getUniqueUserDriveEntityTxs for the same
wallet within seconds. Cache is cleared after sync completion.
- cache updateUserDrives in SyncRepository with event-based flag.
Multiple entry points (syncMetadataOnly, startSync, startSyncForDrive)
call it redundantly. Flag cleared only when sync processes drives.
- skip genesis block [0,0] range in GQLDriveHistory. Snapshot gaps at
block 0 produced phantom ranges causing 3 wasted queries per drive.
- add local DB pre-check for PendingDriveEntities. Only query gateway
when pendingTransactionsForDrive returns local entries. Skip entirely
for non-owned (read-only) drives.
- stop Solana RPC retry spam in DataGatewayFallback and
SnapshotValidationService. Cache empty gateway list on first failure
instead of retrying every fetchData/validation call.
- fix zero-drives-to-sync: when probe skips all drives, return
emptySyncCompleted instead of falling through to "all failed".
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address CodeRabbit review — race condition, cache sharing, probe gaps PE-9103
- fix updateUserDrives race condition: replace boolean flag with Future
so concurrent callers await the in-flight request instead of both firing
- fix snapshot prefetch minBlock: exclude never-synced drives from the min
calculation so they don't drag the batched snapshot query to block 0
- fix gateway cache sharing: pass DataGatewayFallback reference to
SnapshotValidationService instead of copying the list, so both services
share one cache and writes propagate bidirectionally
- add cancellation check in probe loop before each owner group
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* perf: cache entity data, drive signatures, and skip redundant balance refresh PE-9103
- cache raw entity data bytes from getUniqueUserDriveEntities so
getLatestDriveEntityWithId (called during password validation) can
re-parse without re-downloading from the gateway
- cache drive signatures permanently (immutable on-chain) to avoid
redundant GQL + data fetch on every login for v1-signed private drives
- skip refreshBalance after no-op sync (drivesSynced == 0) to avoid
redundant PendingTxFees query when nothing changed
- skip redundant getLatestDriveEntityWithId in drive attach flow when
drivePrivacyLoader already cached the entity
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* perf: use unfiltered GQL strategy for drive history (48 calls → 16) PE-9103
Switch GQLDriveHistory from GetSegmentedTransactionFromDrive-
FilteringByEntityTypeStrategy (3 queries per drive: drive, folder,
file) to the unfiltered strategy (1 query per drive returning all
entity types). The downstream parsing pipeline already separates
entities by type via whereType<DriveEntity/FolderEntity/FileEntity>,
so the per-type filtering at the query level was redundant round trips.
For 16 drives on first sync: 48 → 16 DriveEntityHistory calls.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: clear cached updateUserDrives future on error to allow retry PE-9103
If the updateUserDrives future completes with an error (transient
network issue), subsequent callers would receive the same cached error
without retrying. Now the cached future is cleared on error so the
next caller gets a fresh attempt.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address CodeRabbit review — infinite loop guard, dead code cleanup PE-9103
- add empty-edges guard in getAllSnapshotsForDrives pagination loop to
prevent infinite loop when gateway returns hasNextPage=true with 0 edges
- remove unreachable duplicate cachedDriveEntity check in driveNameLoader
- fix misleading comment in drivePrivacyLoader
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* PE-9103: Snapshot creation progress reporting, caching, and retry (#2153)
* PE-9103: Sync modal UX improvements (#2154)
* ux: sync modal improvements — retry, drive names, elapsed time, probe status PE-9103
- show "Checking for changes..." during drive activity probe phase
instead of misleading "0 of 16 Drives Synced"
- include drive names in sync error messages (e.g., "My Drive: Gateway
timeout (504)") so users know which drive failed
- add "Retry Failed" button in sync error modal to retry only the
drives that failed without re-syncing everything
- show elapsed time (e.g., "45s elapsed") after 5 seconds during sync
so users know the sync is progressing on longer first syncs
- expose syncStartTime getter on SyncCubit for elapsed time widget
- add localization keys for all new strings in 6 locales
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: retry only failed drives, not all drives PE-9103
CodeRabbit correctly identified that retryFailedDrives was calling
startSync(deepSync: true) which resyncs ALL drives. Now iterates
over failed drive IDs and syncs each individually.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: route retry through syncAllDrives with driveIdsToRetry filter PE-9103
Instead of looping startSyncForDrive (which flashes the modal per
drive), add driveIdsToRetry parameter to syncAllDrives that filters
the drives list. Retry runs as a single sync session with one modal,
proper ghost creation, and transaction status updates.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address CodeRabbit review — safe getter, consistent error names PE-9103
- make _initSync non-late (initialized to DateTime.now()) to prevent
LateInitializationError if syncStartTime is read before sync starts
- prefix drive name in syncSingleDrive error messages to match
syncAllDrives format ("Drive Name: error message")
- skip localization finding: statusMessage strings are hardcoded English
as a pre-existing pattern — repository has no BuildContext access
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* PE-9103: Snapshot creation improvements + download UX fixes (#2155)
* perf: sync pipeline performance and resilience optimizations PE-9103
- skip unchanged drives via bulk GQL probe (1 query per owner replaces ~40 queries per idle drive)
- add database indexes on file_revisions.dataTxId and network_transactions.status
- bulk pre-load dateCreated for tx status updates (eliminates N+1 per-tx DB queries)
- bulk pre-load folders and drives in ghost creation (eliminates N+1 per-folder DB queries)
- batch transaction status writes using insertNewNetworkTransactions (replaces individual writes)
- replace full file_revisions table load with filtered query for snapshot tx matching
- fix DataGatewayFallback timeout budget (15s→25s) so arweave.net is reachable
- fix 4 GraphQL queries bypassing GraphQLRetry (missing retry + fallback)
- remove 200ms artificial delay between tx status batches
- bump database schema version 28→29
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: don't skip post-sync ops when all drives are unchanged PE-9103
The early return on numberOfDrivesToSync == 0 skipped transaction status
updates, ghost folder creation, and ARNS record updates. Pending
transactions from recent uploads need confirmation checks even when no
drives have new entities.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* perf: skip GraphQL call for public file downloads PE-9103
Public file downloads were calling getTransactionDetails() via GraphQL
before starting the download — entirely unnecessary since the txId is
already known locally. This GQL call was also one of the four that
bypassed GraphQLRetry, making it the likely cause of downloads failing
to start when the gateway returns 429/5xx.
Now only private/encrypted files call GraphQL (to fetch cipher/IV tags
from the data transaction). Public files start downloading immediately
with no network round-trip.
Refactored ArDriveDownloader interface: replaced TransactionCommonMixin
dataTx parameter with String txId + bool verifyDownload, since only
those two values were ever used from the full transaction object.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: hedged gateway requests, download fallback, retry UX, drive attach dedup PE-9103
hedged gateway requests:
- replace serial waterfall with staggered parallel requests in DataGatewayFallback
- fire primary immediately, launch fallbacks every 1.5s if no response
- first 200 response wins, rest ignored — worst case ~5s instead of ~20s
- applies to all metadata fetches automatically (no caller changes)
download resilience:
- add downloadWithFallback() for file downloads with same hedged pattern
- add fetchManifestWithFallback() for manifest downloads (had zero fallback)
- add stall detection: throws DownloadStalledException if no chunk for 60s
- typed exceptions: DownloadFileNotFoundException, DownloadNetworkException,
DownloadRateLimitException, DownloadStalledException
download UX:
- differentiated error dialogs: network error, file not found, rate limited
- retry button on retryable failures (network, rate limit, unknown)
- file not found shows OK only (retry won't help)
- error classification in both personal and shared download cubits
drive attach dedup:
- getDrivePrivacyForId() now returns DrivePrivacyResult with owner + tx node
- drivePrivacyLoader() passes owner to getLatestDriveEntityWithId(), skipping
redundant owner lookup — 4 GQL queries reduced to 2
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: revert metadata fetches to serial fallback, keep hedged for downloads only PE-9103
Hedged (staggered parallel) requests fire extra gateway requests when the
primary is slow but succeeds. During sync with hundreds of metadata fetches,
this wastes bandwidth and could trigger rate limits on GAR gateways.
Now:
- metadata fetches (fetchData): serial waterfall (primary → GAR → arweave.net)
- file downloads (downloadWithFallback): hedged staggered (latency-sensitive, single request)
- manifest downloads (fetchManifestWithFallback): serial waterfall
Also fixes stall detection for empty files — timer only starts after the
first chunk arrives, so 0-byte files don't trigger DownloadStalledException.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* perf: batch snapshot queries and share gateway cache PE-9103
batch snapshot queries:
- SnapshotEntityHistory.graphql now accepts $driveIds array instead of
single $driveId — fetches snapshots for all drives in one paginated query
- syncAllDrives() prefetches snapshots for all drives per owner before
the per-drive sync loop, passes results to _syncDrive()
- reduces N snapshot GQL queries (one per drive) to 1 per unique owner
- also fixes Entity-Type tag syntax: values: "snapshot" → values: ["snapshot"]
share gateway cache:
- DataGatewayFallback.cachedGateways is now public so
SnapshotValidationService can reuse the same gateway list
- syncAllDrives() passes the cache before sync starts
- eliminates 1 duplicate Solana RPC call per sync cycle
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address CodeRabbit review — stall detection + mobile GCM path PE-9103
- forward verifyDownload param to mobile AES-GCM download path
- wrap mobile GCM stream with _withStallDetection (was bypassed)
- cancel upstream subscription when stall timer fires
- guard against adding to closed StreamController in stall detection
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* perf: eliminate redundant GraphQL calls in sync pipeline PE-9103
HAR analysis of a real sync session (16 drives, no changes) revealed
130 GraphQL calls taking 376s. This commit reduces that to ~3 calls
and <1s for incremental syncs with no changes.
Changes:
- fix DriveActivityProbe: partition drives into never-synced and
previously-synced before probing. Never-synced drives (lastBlockHeight=0)
were poisoning the probe's minBlockHeight to 0, causing it to query
from genesis, overflow the page limit, and fall back to syncing ALL
drives. Now only previously-synced drives are probed.
- cache UserDriveEntityTxs in ArweaveService with event-based
invalidation. Auth flow (isExistingUser, _validateUser) and sync
(updateUserDrives) all call getUniqueUserDriveEntityTxs for the same
wallet within seconds. Cache is cleared after sync completion.
- cache updateUserDrives in SyncRepository with event-based flag.
Multiple entry points (syncMetadataOnly, startSync, startSyncForDrive)
call it redundantly. Flag cleared only when sync processes drives.
- skip genesis block [0,0] range in GQLDriveHistory. Snapshot gaps at
block 0 produced phantom ranges causing 3 wasted queries per drive.
- add local DB pre-check for PendingDriveEntities. Only query gateway
when pendingTransactionsForDrive returns local entries. Skip entirely
for non-owned (read-only) drives.
- stop Solana RPC retry spam in DataGatewayFallback and
SnapshotValidationService. Cache empty gateway list on first failure
instead of retrying every fetchData/validation call.
- fix zero-drives-to-sync: when probe skips all drives, return
emptySyncCompleted instead of falling through to "all failed".
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address CodeRabbit review — race condition, cache sharing, probe gaps PE-9103
- fix updateUserDrives race condition: replace boolean flag with Future
so concurrent callers await the in-flight request instead of both firing
- fix snapshot prefetch minBlock: exclude never-synced drives from the min
calculation so they don't drag the batched snapshot query to block 0
- fix gateway cache sharing: pass DataGatewayFallback reference to
SnapshotValidationService instead of copying the list, so both services
share one cache and writes propagate bidirectionally
- add cancellation check in probe loop before each owner group
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* perf: cache entity data, drive signatures, and skip redundant balance refresh PE-9103
- cache raw entity data bytes from getUniqueUserDriveEntities so
getLatestDriveEntityWithId (called during password validation) can
re-parse without re-downloading from the gateway
- cache drive signatures permanently (immutable on-chain) to avoid
redundant GQL + data fetch on every login for v1-signed private drives
- skip refreshBalance after no-op sync (drivesSynced == 0) to avoid
redundant PendingTxFees query when nothing changed
- skip redundant getLatestDriveEntityWithId in drive attach flow when
drivePrivacyLoader already cached the entity
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* perf: use unfiltered GQL strategy for drive history (48 calls → 16) PE-9103
Switch GQLDriveHistory from GetSegmentedTransactionFromDrive-
FilteringByEntityTypeStrategy (3 queries per drive: drive, folder,
file) to the unfiltered strategy (1 query per drive returning all
entity types). The downstream parsing pipeline already separates
entities by type via whereType<DriveEntity/FolderEntity/FileEntity>,
so the per-type filtering at the query level was redundant round trips.
For 16 drives on first sync: 48 → 16 DriveEntityHistory calls.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: clear cached updateUserDrives future on error to allow retry PE-9103
If the updateUserDrives future completes with an error (transient
network issue), subsequent callers would receive the same cached error
without retrying. Now the cached future is cleared on error so the
next caller gets a fresh attempt.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address CodeRabbit review — infinite loop guard, dead code cleanup PE-9103
- add empty-edges guard in getAllSnapshotsForDrives pagination loop to
prevent infinite loop when gateway returns hasNextPage=true with 0 edges
- remove unreachable duplicate cachedDriveEntity check in driveNameLoader
- fix misleading comment in drivePrivacyLoader
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* perf: snapshot creation progress reporting, caching, and retry PE-9103
progress reporting:
- ComputingSnapshotData state now includes processedTransactions and
totalTransactions (optional, defaults to 0 for backward compat)
- SnapshotItemToBeCreated accepts onProgress callback, fires after each
batch of 100 transactions
- dialog shows "Processing X of Y transactions..." instead of just
"This may take a while"
performance:
- cache drive privacy check once in _reset() instead of querying
driveDao.driveById() per transaction (eliminates N+1 DB queries)
- cache MetadataCache instance once instead of re-creating per transaction
retry without recompute:
- cache computed snapshot data after _getSnapshotData() completes
- on upload failure, "Try Again" reuses cached data (skips 30-120s
recomputation)
- cache cleared on success or drive/range change
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: snapshot creation cache staleness + missing Drive-Id logging PE-9103
- always refresh MetadataCache on _reset() instead of ??= to avoid
stale cache references from prior sessions
- clear _cachedSnapshotData on cancellation to prevent reusing
partially computed data on retry
- log warning when snapshot transaction has no Drive-Id tag during
batched prefetch (silent skip was hiding malformed snapshots)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: guard progress emit against disposed cubit PE-9103
If the user dismisses the snapshot dialog while computation is running,
the progress callback would call emit() on a closed cubit, crashing
with StateError. Now checks isClosed before emitting.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: download success dialog shows filename instead of duplicate title PE-9103
- success dialog was showing "Download Finished" as both title and
description — now shows the filename as description
- check saveResult in onDone handler so cancelled browser save dialogs
don't incorrectly show the success modal
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: turbo payment failure no longer blocks snapshot creation PE-9103
If the Turbo payment service is unreachable (e.g. payment.ardrive.dev
returns 404/500), the entire snapshot flow failed with
ComputeSnapshotDataFailure — even though AR payment would have worked.
Now wraps Turbo cost calculation in try-catch: on failure, Turbo is
marked unavailable and the confirmation dialog shows with AR as the
only payment option.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: lazy-init MetadataCache to fix CI test failures PE-9103
Moving newSharedPreferencesCacheStore() into _reset() broke all 7
create_snapshot_cubit tests in CI — the shared_preferences plugin
isn't available in the test environment (no platform channel).
Now lazily initialized on first use in _jsonMetadataOfTxId() instead
of eagerly in _reset(). Cache is still cleared on reset (set to null)
so stale references are avoided.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* ci: trigger CI run
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore(version): bump version to 2.85.0 (#2156)
* PE-9103: Release v2.84.0 (#2148) (#2158)
* perf: prefetch next snapshot + streaming JSON parse PE-9103
Two optimizations for snapshot loading during sync:
1. Prefetch: start downloading the next snapshot body while the
current one is being parsed and written to DB. Only 1 ahead to
avoid gateway 429s. Overlaps network I/O with CPU work.
2. Streaming parse: instead of jsonDecode on the entire snapshot
(which builds a massive Map with all entries in memory), scan
the JSON string for individual txSnapshot object boundaries
using brace counting and parse each one separately. This:
- Avoids building the full nested Map (lower peak memory)
- Starts yielding transactions immediately
- Still needs the full string downloaded, but parsing is
incremental
For a drive with 3 snapshots of ~15MB, ~10MB, ~5MB:
Before: download1 + parse1 + download2 + parse2 + download3 + parse3
After: download1 + [parse1 || download2] + [parse2 || download3] + parse3
* Revert "perf: prefetch next snapshot + streaming JSON parse PE-9103"
This reverts commit 78331785e351f7dc5c02079ff82d0db6cfc76e6e.
* PE-9103: Bulk-load revision lookups instead of per-entity DB queries (#2142)
* perf: bulk-load revision lookups instead of per-entity DB queries PE-9103
During sync, for each file/folder entity, the code issued individual
SELECT queries to find the latest and oldest revisions. For a drive
with 19k files, that was 38k+ individual DB queries (19k latest +
19k oldest for files, plus similar for folders).
Changes:
- Add 4 bulk SQL queries to drive_queries.drift (latest/oldest ×
files/folders) using GROUP BY with MAX/MIN dateCreated
- Add 4 helper methods to DriveDao returning Map<entityId, Revision>
- Pre-load all revision maps at the start of each transaction chunk
- Pass maps through to _addNewFileEntityRevisions,
_addNewFolderEntityRevisions, _computeRefreshedFileEntriesFromRevisions,
_computeRefreshedFolderEntriesFromRevisions
- Update latestRevisionsCache as new revisions are inserted so
subsequent sub-batches see fresh data
- First-sync shortcut: skip all bulk queries when drive.lastBlockHeight
is 0/null (every entity is new, no previous revisions exist)
Performance: 38,000+ individual SELECTs → 4 bulk SELECTs per chunk.
First sync: 0 queries (all skipped).
* perf: add 5s request timeout + reduce retries on data gateway PE-9103
Entity metadata fetches had NO timeout — a slow gateway could hang
indefinitely. With 2 retries × 5 gateways = 10 attempts with no
timeout, a single failed entity could take minutes.
Changes:
- Add 5-second timeout per HTTP request (metadata is tiny JSON)
- Reduce retries per gateway from 2 to 1 (move on, don't retry slow)
- Reduce GAR fallbacks from 3 to 2
Worst case per entity: 4 attempts × 5s = 20s max (was: 10 attempts,
unlimited time)
---------
* fix: add 15s total timeout on data fetch fallback chain PE-9103 (#2143)
A single missing/broken tx could block sync for minutes as the
fallback chain tried every gateway with CORS timeouts and 504s.
Added a 15-second total timeout wrapping the entire fallback chain
(primary → GAR gateways → arweave.net). Combined with the existing
5s per-request timeout, worst case for any single tx is now 15s max.
This prevents the sync from appearing "stuck" — even if metadata
fetches fail, the sync completes within a bounded time and the
periodic sync can trigger again.
* perf: prefetch next snapshot + streaming JSON parse PE-9103 (#2141)
Two optimizations for snapshot loading during sync:
1. Prefetch: start downloading the next snapshot body while the
current one is being parsed and written to DB. Only 1 ahead to
avoid gateway 429s. Overlaps network I/O with CPU work.
2. Streaming parse: instead of jsonDecode on the entire snapshot
(which builds a massive Map with all entries in memory), scan
the JSON string for individual txSnapshot object boundaries
using brace counting and parse each one separately. This:
- Avoids building the full nested Map (lower peak memory)
- Starts yielding transactions immediately
- Still needs the full string downloaded, but parsing is
incremental
For a drive with 3 snapshots of ~15MB, ~10MB, ~5MB:
Before: download1 + parse1 + download2 + parse2 + download3 + parse3
After: download1 + [parse1 || download2] + [parse2 || download3] + parse3
* fix: retry gateway price fetch and handle AR cost failure gracefully PE-9103
- getPrice now retries 3 times with backoff instead of failing on a
single 502 from the gateway
- AR cost calculation in the upload modal falls back to zero estimate
on failure so the modal still opens with Turbo available instead of
crashing entirely
* fix: dateCreated null crash in file/folder entry refresh on first sync PE-9103
toEntryCompanion() on file/folder revision companions produces entry
companions with dateCreated = Value.absent() (schema has DEFAULT).
On first sync, oldestRevisionsCache is empty, so the fallback
.dateCreated.value returns null → JSNull crash on web.
Fixed by keeping a separate map of revision dateCreated values and
using those as the fallback instead of the entry companion's field.
Same bug pattern as the drive revision fix (line 1887), but for
files (line 1838) and folders (line 1871).
* perf: add drive owner to tx status and license gql queries PE-9126
Add an optional owners filter to the by-ID GraphQL queries that the
gateway struggles with (large ClickHouse row scans). Scoping by owner
lets the gateway prune its search space. The owner variable is nullable,
so when an owner can't be determined the query behaves exactly as before.
- TransactionStatuses, LicenseComposed, LicenseAssertions: add $owners
- getTransactionConfirmations / getLicenseComposed / getLicenseAssertions:
accept an optional owner and pass it through
- tx status (per-drive): scope to the drive's ownerAddress
- tx status (global): scope to the logged-in wallet's address
- licenses: scope to the best-known tx owner (revision/file owner)
Edge case: data txs of files pinned from other authors are owned by
those authors and won't match the owner filter; treated as best-effort.
* fix: re-query owner-mismatched txs unscoped to avoid false failures PE-9126
Owner-scoped getTransactionConfirmations leaves cross-owner txs (e.g. the
data tx of a file pinned from another author's upload) at -1, which the
sync status logic would turn into 'failed' after the pending timeout.
Add a fallback pass: after the selective owner-scoped query, re-query any
ids still unresolved (-1) without the owner filter to determine their true
status. The residual set is normally tiny, so the selectivity win of the
first pass is preserved while confirmed cross-owner txs are no longer
misclassified as failed.
* fix: resolve owner-mismatched txs via pin owner, not unscoped re-query PE-9126
The previous fallback re-queried unresolved (-1) txs without an owner
filter, which can re-trigger the gateway's expensive row scan that the
owner scoping was meant to avoid — and, because the call is wrapped in a
5s timeout, a slow/erroring fallback discards the whole batch's results
and makes no progress, repeating every sync.
Replace it with a selective approach: only re-query unresolved ids whose
real on-chain owner is known locally. Currently that's pinned files,
whose data tx is owned by the original uploader (pinnedDataOwnerAddress).
These are re-queried scoped to that owner, so every query stays selective.
Genuinely missing txs have no override and are left unresolved, handled by
the caller's existing pending/failed logic — no unscoped scan, no retry
storm on the common "not found yet" case.
- add pinnedFileRevisions drift query
- getTransactionConfirmations: ownerOverrides param replaces unscoped pass
- sync repo: build pin owner overrides and pass them through
* fix: make pinned-owner confirmation recovery best-effort PE-9126
The second (pinned-owner) pass merges into the already-populated first-pass
map, so its failure must never discard first-pass progress. Wrap it in a
try/catch and bound it with its own 3s timeout: even if the pin re-queries
error or stall, the confirmations resolved by the owner-scoped first pass
are still returned.
* perf: preserve resolved confirmations across a timeout via verified sink PE-9126
getTransactionConfirmations is wrapped in a 5s timeout by the sync status
update; on expiry it previously returned an empty map, discarding every
confirmation already resolved that cycle (the whole 5000-tx page).
Add an optional caller-owned verifiedSink that the method populates with
each resolved confirmation (>= 0 only) as queries complete. The callers pass
one in and, on timeout, fall back to it instead of an empty map — so work
done before the deadline (including a fully-completed first pass when only
the pinned-owner pass is slow) is applied rather than thrown away.
Because the sink holds only positive verifications and never the -1
"not found" placeholders, applying it after a partial/timed-out run only
upgrades txs to confirmed and never marks anything failed off incomplete data.
* fix: bound confirmation fan-out and use type-safe id filtering PE-9126
- Cap the per-chunk GraphQL fan-out in getTransactionConfirmations to
maxConcurrentDataFetches instead of launching every chunk at once, so a
large pending-tx page can't burst into a concurrent-retry storm against
the gateway (matches the throttling on the other gateway-heavy paths).
- Replace `sublist(...) as List<String>?` with `.whereType<String>().toList()`.
The cast threw a TypeError for the pinned-owner pass (whose ids list is
typed List<String?>), which the best-effort catch swallowed — silently
disabling pin recovery. The filter yields a real List<String> regardless
of input type.
* PE-9103: Fix empty explorer after drive attach (#2145)
* fix: empty explorer after drive attach PE-9103
Two bugs caused the explorer to show a permanent spinner after
attaching a drive (data was in DB but UI didn't update):
1. buildWhen guard in drive_detail_page.dart blocked BlocBuilder
rebuilds when SyncCubit was SyncInProgress. If DriveDetailCubit
emitted DriveDetailLoadSuccess during sync, the emission was
permanently skipped with no replay mechanism. Removed the sync
check — DriveDetailCubit already gates emissions via
waitCurrentSync() in the Rx.combineLatest3 callback.
2. startSyncForDrive silently aborted when a sync was in progress.
The .then(selectDrive) still fired, selecting a drive whose
content was never synced. Changed to await waitCurrentSync()
so the single-drive sync runs after the current sync finishes.
* perf: faster snapshot validation — skip fallback on 404, reduce timeouts PE-9103
Snapshot validation was slow on localhost (and anywhere Solana RPC is
unavailable): 2 HEAD retries × 10s timeout + GAR fallback via Solana
= 25+ seconds per snapshot. With 16 drives having multiple snapshots,
sync appeared stuck.
Changes:
- 1 HEAD attempt instead of 2 (fail fast, fall back to GQL)
- HEAD timeout 10s → 5s
- GAR list timeout 5s → 3s
- Skip GAR fallback entirely when primary returned 404 (snapshot
doesn't exist, no point trying other gateways via Solana RPC)
- Only try GAR fallback for transient errors (timeout, 5xx)
* fix: guard startSyncForDrive race after waitCurrentSync PE-9103
Two callers could both pass the SyncInProgress guard after
waitCurrentSync() returned, causing concurrent single-drive syncs.
Re-check state after wait — if another sync started, bail out.
---------
* fix: bump arweave-dart to v4.0.2 to fix file download crash
Pulls in the TransactionData null-field fix so downloads no longer crash
on web with "type 'JSNull' is not a subtype of type 'String'" when a
gateway returns null for optional tx fields (e.g. anchor on
turbo-gateway.com). Updates both the dependency and the override.
* chore(version): bump version to 2.84.0
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Ariel Melendez <ariel@ardrive.io>
Co-authored-by: arielmelendez <ariel.l.melendez@gmail.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Ariel Melendez <ariel@ardrive.io>
Co-authored-by: arielmelendez <ariel.l.melendez@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A single missing/broken transaction could block sync for minutes as the data fetch fallback chain tried every gateway — each potentially timing out with CORS errors or 504s.
Added a 15-second total timeout wrapping the entire fallback chain (primary → GAR gateways → arweave.net). This caps the worst-case time for any single entity metadata fetch.
How it works
Combined with the existing 5s per-request timeout:
Why the sync appeared "stuck"
The sync state was
SyncInProgresswhile metadata fetches hung. The periodic sync (every 5 min) checkedif (state is SyncInProgress) return;and skipped. The sync wasn't actually stuck — it was waiting for the fallback chain to exhaust all gateways. With the total timeout, it now fails fast and the next periodic sync can proceed.Test plan
flutter analyze— no issues🤖 Generated with Claude Code