fix(seed-economy): retry proxy/EIA transients; gate stress index on full FRED coverage

[koala73]

Summary

Log review of 16 seed-economy runs (2026-04-14 00:45–04:31 UTC) showed 50% degraded, 2 no write failures, and silent StressIndex degradation. Zero-tolerance fixes:

• fredFetchJson: retry Decodo proxy 3× with jittered backoff on 5xx/522/timeout before falling back direct (was: one direct retry → FRED's datacenter 500 instantly dropped the series).
• New eiaFetchJson helper: 20s timeout (was 10s) + 3× retry on 5xx/timeout. Wired into all 5 EIA call-sites (energy prices, crude, nat-gas, SPR, refinery). Eliminates the lockstep timeout cascade that produced full no write runs at 01:00 and 02:00.
• StressIndex: throw when any FRED-sourced component (T10Y2Y, T10Y3M, VIXCLS, STLFSI4, ICSA) is missing — no more silently-degraded composites (same bug class as PR fix(economy): GSCPI shape mismatch with ais-relay payload #3072). GSCPI (ais-relay) still tolerated as absent. Caught in fetchAll so other secondary writes proceed.

No cadence, TTL, or key changes.

Test plan

• npm run typecheck + typecheck:api
• npm run lint (no new errors)
• npm run test:data — 5174/5174 pass
• node --test tests/edge-functions.test.mjs — 167/167 pass
• npm run lint:md / version:check
• esbuild edge bundle check (all api/*.js)
• Observe next 24h of Railway seed-economy logs for FRED 24/24 rate and absence of StressIndex … missing + silent composite writes

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
worldmonitor	Ready	Preview, Comment	Apr 14, 2026 4:51am

[greptile-apps]

Greptile Summary

This PR hardens seed-economy against the observed 50% degradation rate by adding 3× jittered-backoff proxy retries to fredFetchJson, introducing a new eiaFetchJson helper (20 s timeout, 3× retry) wired into all 5 EIA call-sites, and making computeStressIndex throw on any missing FRED-sourced component instead of silently publishing a partial composite.

• P1 — eiaFetchJson retries 4xx errors: throw err on the resp.status < 500 branch sits inside the try block and is caught by the surrounding catch, which only re-throws on i === attempts. A 403 or 404 on attempt 1 is silently swallowed, the loop sleeps (~900 ms), and the same doomed request is retried — the fast-fail intent is defeated. See inline comment for a structural fix.

Confidence Score: 4/5

• Safe to merge after fixing the 4xx-retry logic bug in eiaFetchJson; all other changes are well-structured and correct.
• A P1 logic error exists in the new eiaFetchJson helper: 4xx responses (e.g. bad API key, wrong endpoint) are retried up to 3 times due to a throw inside a try block being caught by the surrounding catch. In the current deployment the EIA key is valid and endpoints are hardcoded, so this is unlikely to trigger, but the defect is present in the shipped code. All other changes — the fredFetchJson 3× proxy retry loop and the StressIndex throw-on-missing guard — are correct and well-wrapped in fetchAll.
• scripts/seed-economy.mjs — the eiaFetchJson retry logic (lines 48–53)

Important Files Changed

Filename	Overview
scripts/seed-economy.mjs	Adds eiaFetchJson helper with 20 s timeout and 3× retry, wires it into 5 EIA call-sites, and hardens computeStressIndex to throw on missing FRED components. The retry helper has a logic bug: throw err for 4xx responses is inside the try block and caught by the surrounding catch, causing client-error responses to be retried up to 3 times instead of failing fast.
scripts/_seed-utils.mjs	Replaces the single-retry proxy path in fredFetchJson with a 3× jittered-backoff loop that retries only on transient errors (5xx, 522, timeout, ECONNRESET, ETIMEDOUT, EAI_AGAIN), then always falls back to direct. Logic is correct.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[fredFetchJson] -->|proxyAuth set| B{proxy attempt 1–3}
    B -->|success| C[return JSON]
    B -->|transient 5xx/522/timeout\nattempt < 3| D[jittered backoff\n400ms×attempt]
    D --> B
    B -->|non-transient OR\nattempt 3 failed| E[direct fetch\n20s timeout]
    E -->|ok| C
    E -->|fail| F[throw direct error]
    A -->|no proxyAuth| E

    G[eiaFetchJson] --> H{fetch attempt 1–3}
    H -->|resp.ok| I[return JSON]
    H -->|5xx + attempt < 3| J[sleep 500ms×i\nthen retry]
    J --> H
    H -->|4xx OR last attempt| K[throw error]
    H -->|network error + last attempt| K
    H -->|network error + not last| J

    L[computeStressIndex] --> M{FRED component missing?}
    M -->|comp = GSCPI| N[warn + skip\ncontinue]
    M -->|comp ≠ GSCPI| O[throw — refuse\npartial composite]
    O --> P[fetchAll catch:\nconsole.warn, skip write]
    M -->|all present| Q[compute weighted\ncomposite score]
    Q --> R[writeExtraKeyWithMeta]

Loading

_{Reviews (1): Last reviewed commit: "fix(seed-economy): retry proxy/EIA trans..." | Re-trigger Greptile}

[greptile-apps]

4xx errors are silently retried

throw err on line 48 is inside the try block, so it is caught by the catch on line 50. The catch only re-throws when i === attempts, which means a 4xx response (e.g. 401, 403, 404) on attempt 1 or 2 is swallowed, the loop sleeps (~900 ms then ~1.4 s), and the same doomed request is retried — the opposite of "fail fast on client errors." The intent of resp.status < 500 in the condition was to bail immediately on 4xx, but that intent is defeated by the surrounding try/catch.

Fix: separate the network-error catch from the HTTP-status check so that 4xx responses throw directly without entering the catch:

Suggested change

	if (resp.status < 500 || i === attempts) throw err;
	lastErr = err;
	} catch (e) {
	lastErr = e;
	if (i === attempts) throw e;
	}
	if (resp.ok) return await resp.json();
	const err = new Error(`EIA ${label}: HTTP ${resp.status}`);
	if (resp.status < 500 || i === attempts) throw err; // 4xx: always fail fast; 5xx last attempt: give up
	lastErr = err;
	} catch (e) {
	lastErr = e;
	// Re-throw on last attempt OR if this is a non-5xx HTTP error (status attached)
	if (i === attempts) throw e;
	}

A cleaner structural fix that keeps the intent unambiguous is to split the try into a network-only catch and handle HTTP status outside it:

async function eiaFetchJson(url, label, { timeoutMs = 20_000, attempts = 3 } = {}) {
  let lastErr;
  for (let i = 1; i <= attempts; i++) {
    let resp;
    try {
      resp = await fetch(url, {
        headers: { Accept: 'application/json', 'User-Agent': CHROME_UA },
        signal: AbortSignal.timeout(timeoutMs),
      });
    } catch (e) {
      lastErr = e;
      if (i === attempts) throw e;
      await new Promise((r) => setTimeout(r, 500 * i + Math.random() * 400));
      continue;
    }
    if (resp.ok) return await resp.json();
    const err = new Error(`EIA ${label}: HTTP ${resp.status}`);
    if (resp.status < 500 || i === attempts) throw err; // 4xx: immediate; 5xx final: give up
    lastErr = err;
    await new Promise((r) => setTimeout(r, 500 * i + Math.random() * 400));
  }
  throw lastErr;
}