fix(#53): serve header backfill from bitswap/gateway, Glif as last resort#67
Merged
Conversation
…sort
The polling header Sync agent and the gossip ingestor backfill were both
hardcoded to a Glif client (8s timeout). Live blocks arrive over gossipsub
fine, but a block landing at head+N (N>1) triggered parent backfill =
Glif FetchBlock calls. When Glif was slow/rate-limited those timed out,
backfillFail climbed, the header store could not advance contiguously, and
the node fell behind live head (tester reports, 2026-06-22).
FetchBlock is content-addressed, so it can be served by the combined
gateway+bitswap fetcher already running for state reads. New
bitswapBackedSource adapter:
- HeadEpoch / TipsetCIDsByHeight stay on Glif (inherently RPC-shaped;
live head also arrives via gossipsub, so rarely the limiter).
- FetchBlock tries the combined fetcher first (CID-verified), falls back
to Glif only on miss/decode-fail. Parent backfill leaves the Glif
critical path.
- Fetcher resolved lazily (getFetcher func) because the combined fetcher
is rebuilt to add the bitswap source after libp2p is up, AFTER the
sync/gossip wiring is constructed — lazy resolve sees the bitswap-
enabled fetcher, not a stale gateway+glif snapshot.
Wired into both the Sync agent and the gossip ingestor backfill source.
Compile-time guards assert the adapter against the REAL hstore.RPCSource +
blockingest.BackfillSource interfaces. Tests: fetcher-hit (glif untouched),
fetcher-miss fallback, decode-fail fallback, nil-fetcher path, RPC-method
delegation. Build+vet+gofmt+hermetic green. Closes #53
ChainGetTipSet only resolved the synthetic current head and returned ErrTipSetNotFound for any other key, even with a populated header store wired. Curio's message/watch.go + deps/apiinfo.go ask for specific recent (non-head) tipset keys -> refused -> chain watcher stalls with 'tipset not in local store (only current head is cached in V1)'. Add Store.GetTipSet(tsk) (resolve a tipset by key from persisted headers, ErrNotFound on any missing block) and fall ChainGetTipSet through to it after the head fast-path. +5 assertions covering head, non-head historical, empty-key, and missing-block cases.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #53 (p1).
Problem
The polling header Sync agent and the gossip ingestor backfill were both hardcoded to a Glif client (8s timeout). Live blocks arrive over gossipsub fine, but a block landing at head+N (N>1) triggered parent backfill = Glif
FetchBlockcalls. When Glif was slow/rate-limited those timed out →backfillFailclimbed → the header store couldn't advance contiguously → the node fell behind live head (tester reports 2026-06-22:context deadline exceeded,client canceled request, dashboard "slow (Ns behind)").Fix
FetchBlockis content-addressed, so it can be served by the combined gateway+bitswap fetcher already running for state reads. NewbitswapBackedSourceadapter:HeadEpoch/TipsetCIDsByHeightstay on Glif (inherently RPC-shaped; live head also arrives via gossipsub, so rarely the limiter).FetchBlocktries the combined fetcher first (CID-verified), falls back to Glif only on miss/decode-fail → parent backfill leaves the Glif critical path.getFetcher func) because the combined fetcher is rebuilt to add the bitswap source after libp2p is up — which is after the sync/gossip wiring is constructed. Lazy resolve sees the bitswap-enabled fetcher, not a stale gateway+glif snapshot.Wired into both the Sync agent and the gossip ingestor backfill source.
Safety / tests
Compile-time guards assert the adapter against the real
hstore.RPCSource+blockingest.BackfillSourceinterfaces (build breaks if either drifts). Unit tests: fetcher-hit (Glif untouched), fetcher-miss fallback, decode-fail fallback, nil-fetcher path, RPC-method delegation. Build + vet + gofmt + hermetic suite green. Live mainnet node untouched.Related: #50 (message-block availability over bitswap), PR #52 (prefetch-on-send).