fix(hyperliquid-flow): fetch both default dex and xyz builder dex#3077
Conversation
Root cause: Hyperliquid's commodity and FX perps (xyz:CL, xyz:BRENTOIL,
xyz:GOLD, xyz:SILVER, xyz:PLATINUM, xyz:PALLADIUM, xyz:COPPER, xyz:NATGAS,
xyz:EUR, xyz:JPY) live on a separate 'xyz' builder dex, NOT the default
perp dex. The MIT reference repo listed these with xyz: prefixes but
didn't document that they require {type:metaAndAssetCtxs, dex:xyz} as a
separate POST.
Production symptom (Railway bundle logs 2026-04-14 04:10):
[Hyperliquid-Flow] SKIPPED: validation failed (empty data)
The seeder polled the default dex only, matched 4 of 14 whitelisted assets
(BTC/ETH/SOL/PAXG), and validateFn rejected snapshots with <12 assets.
Seed-meta was refreshed on the skipped path so health stayed OK but
market:hyperliquid:flow:v1 was never written.
Fix:
- New fetchAllMetaAndCtxs(): parallel-fetches both dexes and merges
{universe, assetCtxs} by concatenation. xyz entries already carry the
xyz: prefix in their universe names.
- New validateDexPayload(raw, dexLabel, minUniverse): per-dex floor so the
thinner xyz dex (~63 entries) does not false-trip the default floor of
50. Errors include the dex label for debuggability.
- validateUpstream(): back-compat wrapper — accepts either the legacy
single-dex [meta, assetCtxs] tuple (buildSnapshot tests) or the merged
{universe, assetCtxs} shape from fetchAllMetaAndCtxs.
Tests: 37/37 green. New tests cover dual-dex fetch merge, cross-dex error
propagation, xyz floor accept/reject, and merged-shape pass-through.
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
Greptile SummaryThis PR fixes the Hyperliquid seeder introduced in #3074, which was always skipping due to empty data because commodity/FX perps ( Confidence Score: 5/5
Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Entry Point] --> B[fetchAllMetaAndCtxs]
B --> C1[fetchHyperliquidMetaAndCtxs\ndex=undefined]
B --> C2[fetchHyperliquidMetaAndCtxs\ndex='xyz']
C1 & C2 --> D[Promise.all]
D --> E1[validateDexPayload\nlabel='default', min=50]
D --> E2[validateDexPayload\nlabel='xyz', min=30]
E1 -->|pass| F[Merge universes\nand assetCtxs]
E2 -->|pass| F
E1 -->|fail: too small / shape error| G[throw → runSeed\nextends TTL, count=0]
E2 -->|fail: too small / shape error| G
F --> H[merged shape\nuniverse + assetCtxs]
H --> I[buildSnapshot\nvalidateUpstream pass-through]
I --> J[indexBySymbol → Map]
J --> K[Iterate ASSETS whitelist\n14 symbols]
K --> L[validateFn: assets.length >= 12]
L -->|pass| M[Write to Redis\nCANONICAL_KEY]
L -->|fail| G
Reviews (1): Last reviewed commit: "fix(hyperliquid-flow): fetch both defaul..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| * @param {string|undefined} dex | ||
| * @param {typeof fetch} [fetchImpl] | ||
| */ | ||
| export async function fetchHyperliquidMetaAndCtxs(dex = undefined, fetchImpl = fetch) { |
There was a problem hiding this comment.
fetchImpl = fetch deviates from codebase convention
All other seed scripts either call globalThis.fetch(...) directly or define an explicit DEFAULT_FETCH = (...args) => globalThis.fetch(...args) wrapper — see seed-regulatory-actions.mjs for the canonical pattern. AGENTS.md reinforces this: the preferred call site is (...args) => globalThis.fetch(...args), not a bare fetch reference. Using fetch as a default parameter captures the module-scope binding rather than always resolving through globalThis at call time, which can silently fail in environments where fetch hasn't been populated on the global object yet.
| export async function fetchHyperliquidMetaAndCtxs(dex = undefined, fetchImpl = fetch) { | |
| export async function fetchHyperliquidMetaAndCtxs(dex = undefined, fetchImpl = (...args) => globalThis.fetch(...args)) { |
Context Used: AGENTS.md (source)
| * xyz: asset names already carry the `xyz:` prefix in their universe entries, | ||
| * so no rewriting is needed — just concatenate. | ||
| */ | ||
| export async function fetchAllMetaAndCtxs(fetchImpl = fetch) { |
There was a problem hiding this comment.
fetchImpl = fetch default needed here
Same issue as fetchHyperliquidMetaAndCtxs above — the default should follow the (...args) => globalThis.fetch(...args) pattern used by seed-regulatory-actions.mjs and recommended in AGENTS.md.
| export async function fetchAllMetaAndCtxs(fetchImpl = fetch) { | |
| export async function fetchAllMetaAndCtxs(fetchImpl = (...args) => globalThis.fetch(...args)) { |
Context Used: AGENTS.md (source)
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
1 participant
Why this PR?
Railway bundle logs 2026-04-14 04:10 UTC showed the new Hyperliquid seeder from PR #3074 always skipping with empty data:
```
[Hyperliquid-Flow] SKIPPED: validation failed (empty data) — seed-meta refreshed, existing cache TTL extended
[Hyperliquid-Flow] WARNING: 2 key(s) were expired/missing — EXPIRE was a no-op; manual seed required
```
Root cause: Hyperliquid splits commodity/FX perps into a separate "xyz" builder dex — not the default perp dex. The seeder only queried `{type:"metaAndAssetCtxs"}` which returns the 229-entry default universe (BTC, ETH, SOL, PAXG, and ~225 other crypto perps) but NONE of the `xyz:*` targets. Of 14 whitelisted assets, only 4 matched → `validateFn: assets.length >= 12` rejected the snapshot.
The MIT reference repo listed these with `xyz:` prefixes but didn't document the dex-parameter requirement. Verified live:
```bash
curl -sX POST https://api.hyperliquid.xyz/info -H 'Content-Type: application/json' \
-d '{"type":"metaAndAssetCtxs","dex":"xyz"}' | jq '.[0].universe | map(.name) | [.[]|select(startswith("xyz:"))] | length'
→ 63, including xyz:CL, xyz:BRENTOIL, xyz:GOLD, xyz:SILVER, xyz:EUR, xyz:JPY...
```
Fix
New `fetchAllMetaAndCtxs()` — parallel-fetches both dexes via `Promise.all`, validates each payload independently (with per-dex floors), and merges by concatenation. `xyz:` entries already carry the prefix in their universe names, so no rewriting needed.
New `validateDexPayload(raw, dexLabel, minUniverse)` — per-dex floor gate so the thinner xyz dex (~63 entries) doesn't false-trip the default floor of 50. Errors include the dex label for debuggability.
`validateUpstream()` — back-compat wrapper. Accepts either the legacy single-dex `[meta, assetCtxs]` tuple (used by `buildSnapshot` tests and still valid) or the merged `{universe, assetCtxs}` shape from `fetchAllMetaAndCtxs`.
Entry point now calls `fetchAllMetaAndCtxs()` with a comment explaining why both dexes are required.
Files
Testing
Post-Deploy Monitoring & Validation
Related