Skip to content

feat: add Serialized as provider on benches 004 and 008 - #2261

Merged
Flotapponnier merged 1 commit into
devfrom
feat/serialized-provider
Sep 5, 2026
Merged

feat: add Serialized as provider on benches 004 and 008#2261
Flotapponnier merged 1 commit into
devfrom
feat/serialized-provider

Conversation

@Flotapponnier

Copy link
Copy Markdown
Collaborator

Wires serialized.xyz into the two benches where their endpoints map 1:1 onto the existing scoring rule.

What lands

  • Bench 004 metadata-coverageharnesses/metadata-coverage/cmd/script/serialized_rest_monitor.go. GET /v1/token/metadata, fields mapped iconUrl/description/twitterUrl/websiteUrl. Chain ids are already in the bench's own shape (solana, evm:8453), only the legacy solana:solana form needs normalising. Chains outside their 19 are skipped, not counted as misses.
  • Bench 008 wallet-labels-coverageharnesses/wallet-labels/cmd/script/serialized.go. GET /v1/wallet/profile, same non-generic-name precedence rule every other provider gets.
  • Registry entry, square mark logo, wordmark.
  • docs/methodology/serialized-onboarding-audit.md — the full pre-onboarding audit.

Client-side throttling is not optional

Serialized enforces a hard 40 req/s burst cap per key. An unthrottled 8-worker pool lost 60 of 100 anchors to 429 and made their coverage read 37% instead of 77%. Both providers pace at ~16 rps.

Apples-to-apples numbers (from ocb-par-main, same egress as the live monitors)

Bench 008, same 100 curated anchors, same hit rule:

Provider Coverage Contract EOA p50
Serialized 77.0% 76.3% 78.0% 45 ms
Mobula 59.0% 54.2% 65.9% 34 ms

Bench 004, same token set both sides, only tokens both resolved:

Chain n Serialized Mobula
solana 48 10.4% 34.4%
base 37 19.6% 34.5%
bsc 38 60.5% 80.9%

Two scoring flaws this exposed in our own benches

Neither is fixed here. Both are provider-neutral and should ship separately, ideally before this lands publicly so the fix does not read as a reaction to a new entrant.

  1. 004 logo field measures "field non-empty", not "image resolves". Mobula rewrites every logo onto metadata.mobula.io at a deterministic path, so its logo field is 100% by construction (11 of 12 sampled URLs actually resolved). Serialized returns the upstream URL. Suggested fix: HEAD-check the logo.
  2. 008 hit rule counts any non-generic string, so a personal ENS or .sol record on a well-known contract scores as a correct entity label. 25% of Serialized's hits are wrong this way; Mobula's ratio is 25.4% on the same sample. Suggested fix: score against the curated Hint the harness already carries and currently ignores.

Details and the full method in the audit doc.

Not included

005/090 (they would rank last on chain count, 19 vs 465/247), 067 (needs curated funded probe addresses per chain first), 001 (needs a Base preconfirmation policy: their flashblocks feed reports negative self-lag while losing the actual wall-clock race 77% of the time), 033/102 (they are not a swap router).

Post-merge

/products/serialized renders only once Prom carries provider="serialized", so the harness containers need a docker build --no-cache plus stop/rm/run on ocb-par-main, then a materialize-worker rebuild. SERIALIZED_API_KEY must be added to both container envs.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CpArutAtXuBb1BVNUDXoYA

Wires serialized.xyz into metadata-coverage (004) and wallet-labels (008),
the two benches where their endpoints map 1:1 to the existing scoring rule.

Both providers throttle client-side at ~16 rps: Serialized enforces a hard
40 req/s burst cap and an unthrottled worker pool turns coverage into a
rate-limit artifact (measured 77% -> 37%).

Adds the registry entry, logo and a full onboarding audit documenting the
apples-to-apples numbers and two scoring flaws the tests exposed in our
own benches (logo presence vs resolution on 004, name-service strings
counted as entity labels on 008).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CpArutAtXuBb1BVNUDXoYA
@Flotapponnier
Flotapponnier merged commit 6adc842 into dev Sep 5, 2026
1 check failed
@Flotapponnier
Flotapponnier deleted the feat/serialized-provider branch September 5, 2026 16:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant