Repository navigation
Replies: 1 comment
|
Gentle bump — it's been a few days and I wanted to make sure this didn't get buried. Still genuinely curious about your team's take on the UNAVAILABLE + retry cursor gate as a defense against tip-race omissions. If the question isn't a good fit for Discussions, happy to take it elsewhere or close. Thanks either way for the open-source work you all do. |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Hi SubQuery team — a failure mode I hit and documented, and I'd value your perspective since SubQuery's indexing pipeline processes live chain data at scale.
I ran a 24h Polygon OrderFilled listener (2,182 tracked wallets as maker). Its status reported lag_blocks: 0 for the whole run, but independent reconciliation — re-pulling the same events from three public RPCs and comparing per-fill tx + log_index — found 5 blocks silently missing 61 fills (16/12/1/29/3). Consistent with a tip race: the cursor advanced while data for some blocks wasn't usable yet, and those blocks were never revisited. Zero lag means the cursor reached its reported tip; it doesn't prove every expected event was recorded.
My fix: confirmation-depth handling — unusable blocks become UNAVAILABLE, the cursor stops before them and retries on a later poll — plus 13 offline regression tests against mock RPCs (no network access, keys, or funds to reproduce). Evidence and repro: https://github.com/HCTDIP/jevaudit (CASES.md) · https://github.com/HCTDIP/openship-fork
Question: in SubQuery's indexing pipeline, what is the recommended defense against this class of silent omission at the tip — and would a cursor gate like mine belong in the project config, or deeper in the data source layer? Any hole you spot goes into the public regression suite, with credit.
All reactions