Skip to content

fix(crypto-quotes): use CoinPaprika as primary, CoinGecko as fallback#3086

Merged
koala73 merged 3 commits into
mainfrom
fix/crypto-quotes-paprika-first
Apr 14, 2026
Merged

fix(crypto-quotes): use CoinPaprika as primary, CoinGecko as fallback#3086
koala73 merged 3 commits into
mainfrom
fix/crypto-quotes-paprika-first

Conversation

@koala73
Copy link
Copy Markdown
Owner

@koala73 koala73 commented Apr 14, 2026

Why this PR?

Railway `seed-bundle-market-backup` log 2026-04-14 07:17:10 UTC reported `failed:1` because the Crypto-Quotes section hit chronic CoinGecko 429s and exhausted the bundle's 120s section timeout inside the retry loop — the existing CoinPaprika fallback never got a chance to run.

```
[Crypto-Quotes] CoinGecko 429 — waiting 10s (attempt 1/5)
[Crypto-Quotes] CoinGecko 429 — waiting 20s (attempt 2/5)
[Crypto-Quotes] CoinGecko 429 — waiting 30s (attempt 3/5)
[Crypto-Quotes] CoinGecko 429 — waiting 40s (attempt 4/5)
[Crypto-Quotes] CoinGecko 429 — waiting 50s (attempt 5/5)
[Crypto-Quotes] Crypto-Quotes failed after 120.0s: timeout
[Bundle:market-backup] Finished in 124.3s, ran:1 skipped:6 failed:1
```

Math: CoinGecko's 5-step back-off budget = 10+20+30+40+50 = 150s. Bundle section timeout = 120s. The child process is killed mid-retry, so the `catch` block that would have called `fetchFromCoinPaprika()` never executes.

Fix

Swap the source order: CoinPaprika is now PRIMARY; CoinGecko is retained as FALLBACK for its `sparkline_in_7d` data (CoinPaprika does not provide sparklines).

```js
async function fetchCryptoQuotes() {
let data;
try {
data = await fetchFromCoinPaprika();
} catch (err) {
console.warn(` [CoinPaprika] Failed: ${err.message} — falling back to CoinGecko`);
data = await fetchFromCoinGecko();
}
}
```

Live probe of CoinPaprika confirms coverage: `/v1/tickers?quotes=USD` returns 2000 entries, all 10 mapped IDs present, no auth required, no documented per-IP rate limit problems.

Trade-off

When CoinPaprika is healthy, sparkline arrays will be empty (CoinPaprika doesn't expose 7d price history in `/tickers`). Acceptable — the panel already handles undefined sparklines, and the alternative (no quotes at all because CoinGecko is rate-limited) is strictly worse.

Files

  • `scripts/seed-crypto-quotes.mjs` — swap source order in `fetchCryptoQuotes`; update `fetchFromCoinPaprika` log line ("Falling back" → "Fetching tickers") since it's now primary.

Testing

  • `node --test tests/crypto-config.test.mjs` → 6/6 pass
  • `npm run typecheck` → clean
  • `npm run typecheck:api` → clean
  • Live CoinPaprika probe: 10/10 mapped crypto IDs returned

Post-Deploy Monitoring & Validation

  • Logs: Next `seed-bundle-market-backup` cron run on Railway — Crypto-Quotes section should log `[CoinPaprika] Fetching tickers...` followed by `Done` (no `failed:1`).
  • Redis: `GET market:crypto:v1` → non-empty `quotes` array, length 10, prices populated.
  • Bundle: `Finished in N.Ns, ran:1 skipped:6 failed:0` — the `failed:1` from the 07:17 cycle should not recur.
  • Failure signal / rollback: if CoinPaprika starts 5xx-ing, the new fallback path will hit CoinGecko (with its existing rate-limit guard). If both fail, the section reports `failed:1` exactly as before — no worse. Revert = git revert this commit.
  • Validation window: 6 cycles (30 min).
  • Owner: @koala73

Related

Railway bundle log 2026-04-14 07:17:10 UTC showed seed-bundle-market-backup
finishing with failed:1. Crypto-Quotes hit CoinGecko 429s on every retry:

  [Crypto-Quotes] CoinGecko 429 — waiting 10s (1/5)
  ... (5 attempts, 10/20/30/40/50s back-off)
  [Crypto-Quotes] Crypto-Quotes failed after 120.0s: timeout

Root cause: CoinGecko 5-step retry budget (10+20+30+40+50 = 150s) exceeds
the bundle 120s section timeout, so the existing CoinPaprika fallback
never runs — the child process is killed mid-retry.

Fix: swap source order. CoinPaprika is now primary; CoinGecko is retained
as fallback for sparkline_in_7d data (CoinPaprika does not provide
sparklines). Probed CoinPaprika live: all 10 mapped crypto IDs present in
/v1/tickers, no auth required.

Trade-off: when CoinPaprika is healthy, sparkline arrays will be empty.
Acceptable — the panel already handles undefined sparklines, and the
alternative (no quotes at all because CoinGecko is rate-limited) is worse.

Tests: crypto-config.test.mjs 6/6, typecheck + typecheck:api clean.
@vercel
Copy link
Copy Markdown

vercel Bot commented Apr 14, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
worldmonitor Ignored Ignored Preview Apr 14, 2026 8:41am

Request Review

…snapshots

Codex review on PR #3086 caught: validate() only required >=1 quote with
positive price. With the new CoinPaprika-primary path, a single dropped or
renamed ticker would silently publish a 9/10 snapshot. Health stays green
while one tracked asset disappears from the panel — exactly the silent
data-loss class we want to avoid on a fixed-cardinality top-10 feed.

Tightened validate() to require:
- quotes.length === CRYPTO_IDS.length (full cardinality)
- every quote has Number.isFinite(price) && price > 0
- every configured symbol is present in the response (defends against
  duplicate IDs masquerading as full coverage)

When the validator rejects, runSeed() takes the skipped path: existing
TTL is extended, seed-meta is bumped with count=0, and the Railway log
will scream which symbol is missing on the next cycle so the broken
CoinPaprika mapping is caught immediately.

Tests: crypto-config.test.mjs 6/6, typecheck clean.
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented Apr 14, 2026

Greptile Summary

This PR fixes a chronic failed:1 in the Railway seed-bundle-market-backup service by promoting CoinPaprika to the primary crypto-quotes source and demoting CoinGecko to a fallback. The root cause was correct: CoinGecko's 5-step back-off budget (150 s) consistently exceeded the 120 s bundle section timeout, preventing the existing fallback from ever firing.

Two minor items worth addressing before or after merging:

  • sourceVersion: 'coingecko-markets' written to Redis health metadata is now stale; it should reflect the new primary source.
  • The CoinGecko fallback still calls fetchWithRateLimitRetry with maxAttempts=5 (up to 150 s sleep), so if both sources fail the original timeout scenario can technically recur; capping the fallback at maxAttempts=2 would harden this path.

Confidence Score: 4/5

  • Safe to merge — the fix is correct and well-reasoned; two P2 cleanup items remain.
  • Score is 4 rather than 5 because the stale sourceVersion metadata is a concrete correctness issue for health monitoring (mislabeled Redis meta key), and the unaddressed CoinGecko fallback timeout budget means the original failure mode can still surface in the fallback path. Both are low-probability in practice but are real issues rather than purely style concerns.
  • scripts/seed-crypto-quotes.mjs — stale sourceVersion label and uncapped CoinGecko fallback retry budget

Important Files Changed

Filename Overview
scripts/seed-crypto-quotes.mjs Swap fetchCryptoQuotes() to use CoinPaprika as primary and CoinGecko as fallback; update log message. Two P2 findings: stale sourceVersion: 'coingecko-markets' metadata, and the CoinGecko fallback still carries the full 150s retry budget.

Sequence Diagram

sequenceDiagram
    participant Bundle as Railway Bundle (120s timeout)
    participant FQ as fetchCryptoQuotes()
    participant CP as fetchFromCoinPaprika()
    participant CG as fetchFromCoinGecko()
    participant Redis as Redis (market:crypto:v1)

    Bundle->>FQ: invoke
    FQ->>CP: try (PRIMARY)
    CP->>CP: "GET /v1/tickers?quotes=USD (15s timeout)"
    alt CoinPaprika succeeds
        CP-->>FQ: "normalized data (sparkline=[])"
        FQ->>Redis: SET market:crypto:v1
        FQ-->>Bundle: success ✅
    else CoinPaprika fails
        CP-->>FQ: throw Error
        FQ->>CG: catch → fallback
        CG->>CG: "fetchWithRateLimitRetry (maxAttempts=5, up to 150s)"
        alt CoinGecko succeeds
            CG-->>FQ: data with sparklines
            FQ->>Redis: SET market:crypto:v1
            FQ-->>Bundle: success ✅
        else CoinGecko 429s / both fail
            CG-->>FQ: throw Error
            FQ-->>Bundle: failed:1 ❌ (same as before)
        end
    end
Loading

Comments Outside Diff (1)

  1. scripts/seed-crypto-quotes.mjs, line 45 (link)

    P2 CoinGecko fallback still carries the full 150s retry budget

    fetchWithRateLimitRetry with maxAttempts=5 can sleep 10+20+30+40+50 = 150 s. In the fallback path (CoinPaprika fails after its 15 s timeout → CoinGecko 429s heavily), only ~105 s remain before the bundle kills the process, so the same timeout that triggered this PR can resurface. Reducing maxAttempts to 2 in the fallback call caps CoinGecko sleep at 30 s and keeps the total well within 120 s even if CoinPaprika hits its full timeout.

    // In fetchFromCoinGecko, or pass maxAttempts via a parameter:
    const resp = await fetchWithRateLimitRetry(url, 2, headers); // 10+20s max wait

    This is a minor hardening suggestion — the PR description explicitly calls this scenario "no worse than before," so it doesn't block merge.

Reviews (1): Last reviewed commit: "fix(crypto-quotes): require full coverag..." | Re-trigger Greptile

data.quotes.some((q) => q.price > 0)
);
if (!Array.isArray(data?.quotes)) return false;
if (data.quotes.length !== CRYPTO_IDS.length) return false;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Stale sourceVersion metadata

sourceVersion is written to seed-meta:market:crypto:v1 in Redis for health monitoring (per AGENTS.md). With CoinPaprika now as the primary source, the value 'coingecko-markets' no longer reflects the live data path, which could mislead dashboards or on-call runbooks that inspect this field.

Suggested change
if (data.quotes.length !== CRYPTO_IDS.length) return false;
sourceVersion: 'coinpaprika-tickers',

…rceVersion

P2-1: CoinGecko fallback was still wired with maxAttempts=5 (10+20+30+40+50
= 150s budget), so when CoinPaprika fails the fallback path could itself
overrun the bundle's 120s section timeout — recreating the exact failure
mode this PR fixes. Capped at maxAttempts=2 (10+20=30s) so the fallback
always finishes well within the bundle window.

P2-2: sourceVersion in seed-meta was still 'coingecko-markets' even though
CoinPaprika is now primary. Changed to 'coinpaprika-tickers+coingecko-fallback'
so health dashboards and on-call runbooks see the real data path.

Tests: crypto-config.test.mjs 6/6, typecheck clean.
@koala73 koala73 merged commit 29d3946 into main Apr 14, 2026
10 checks passed
@koala73 koala73 deleted the fix/crypto-quotes-paprika-first branch April 14, 2026 08:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant