feat: add Serialized as provider on benches 004 and 008 - #2261
Merged
Conversation
Wires serialized.xyz into metadata-coverage (004) and wallet-labels (008), the two benches where their endpoints map 1:1 to the existing scoring rule. Both providers throttle client-side at ~16 rps: Serialized enforces a hard 40 req/s burst cap and an unthrottled worker pool turns coverage into a rate-limit artifact (measured 77% -> 37%). Adds the registry entry, logo and a full onboarding audit documenting the apples-to-apples numbers and two scoring flaws the tests exposed in our own benches (logo presence vs resolution on 004, name-service strings counted as entity labels on 008). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CpArutAtXuBb1BVNUDXoYA
This was referenced Sep 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wires serialized.xyz into the two benches where their endpoints map 1:1 onto the existing scoring rule.
What lands
harnesses/metadata-coverage/cmd/script/serialized_rest_monitor.go.GET /v1/token/metadata, fields mappediconUrl/description/twitterUrl/websiteUrl. Chain ids are already in the bench's own shape (solana,evm:8453), only the legacysolana:solanaform needs normalising. Chains outside their 19 are skipped, not counted as misses.harnesses/wallet-labels/cmd/script/serialized.go.GET /v1/wallet/profile, same non-generic-name precedence rule every other provider gets.docs/methodology/serialized-onboarding-audit.md— the full pre-onboarding audit.Client-side throttling is not optional
Serialized enforces a hard 40 req/s burst cap per key. An unthrottled 8-worker pool lost 60 of 100 anchors to
429and made their coverage read 37% instead of 77%. Both providers pace at ~16 rps.Apples-to-apples numbers (from
ocb-par-main, same egress as the live monitors)Bench 008, same 100 curated anchors, same hit rule:
Bench 004, same token set both sides, only tokens both resolved:
Two scoring flaws this exposed in our own benches
Neither is fixed here. Both are provider-neutral and should ship separately, ideally before this lands publicly so the fix does not read as a reaction to a new entrant.
metadata.mobula.ioat a deterministic path, so its logo field is 100% by construction (11 of 12 sampled URLs actually resolved). Serialized returns the upstream URL. Suggested fix: HEAD-check the logo..solrecord on a well-known contract scores as a correct entity label. 25% of Serialized's hits are wrong this way; Mobula's ratio is 25.4% on the same sample. Suggested fix: score against the curatedHintthe harness already carries and currently ignores.Details and the full method in the audit doc.
Not included
005/090 (they would rank last on chain count, 19 vs 465/247), 067 (needs curated funded probe addresses per chain first), 001 (needs a Base preconfirmation policy: their flashblocks feed reports negative self-lag while losing the actual wall-clock race 77% of the time), 033/102 (they are not a swap router).
Post-merge
/products/serializedrenders only once Prom carriesprovider="serialized", so the harness containers need adocker build --no-cacheplus stop/rm/run onocb-par-main, then a materialize-worker rebuild.SERIALIZED_API_KEYmust be added to both container envs.🤖 Generated with Claude Code
https://claude.ai/code/session_01CpArutAtXuBb1BVNUDXoYA