docs: correct the head-lag claim in the Serialized audit, add follow-ups - #2264
Merged
Conversation
v1.0 claimed bench 001 references archive nodes and was therefore unaffected by provider clock disagreement. That repeated the spec instead of reading the harness: there is no archive-node reference in aggregator-head-lag at all, and the leaderboard gauge is computed from each provider's own timestamp. The spec says the opposite in three places. Also records the now-conclusive bench 067 result (verified 5 of 19 chains), probe-confirmed negative capabilities, and a verified 5.2x BONK mispricing traced to pool discovery missing the main market. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CpArutAtXuBb1BVNUDXoYA
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The correction
v1.0 of the audit claimed bench 001 references archive nodes and was therefore unaffected by providers disagreeing about on-chain time. That was wrong. It repeated
benchmarks/aggregator-head-lag.ymlinstead of readingharnesses/aggregator-head-lag.There is no archive-node reference in that harness.
grep -rl "archive|eth_getBlockByNumber|getBlockTime|blockTimestamp"over itscmd/returns nothing. The gauge behind the leaderboard is:The spec claims otherwise in three places:
methodology[7], the FAQ, and the per-chain explainers. Measured impact: for the same transaction hash,serialized.atminusmobula.dateis −707 ms on Solana and +1000 ms on Base, so the published numbers partly measure clock placement rather than pipeline speed.Plus
head_lag_monitor.go:219drops negative lags silently, which would discard 100% of Serialized's Base preconfirmed sample.This is a documentation-versus-code defect on a live, publicly cited bench, independent of any provider. Not fixed here, only documented.
New results recorded
registry.go. Re-run verbatim: listed 19, verified 5, 0 errors. Would rank last (CoinStats 127, Mobula 50, Zerion 42, Moralis 15).404on/v1/wallet/nfts,/v1/nft/collection,/v1/nfts,/v1/swap/quote,/v1/quote,/v1/route,/v1/bridge/quote./v1/token/pools: rank 1 is a pool with liquidity 4,339 and the deep Orca pool is absent from the list, while their own ranks 2 and 3 disagree with rank 1 by the same 5x.🤖 Generated with Claude Code
https://claude.ai/code/session_01CpArutAtXuBb1BVNUDXoYA