Data Pipeline

Data Pipeline

Everything is fetched by npm run fetch-data (scripts/fetch-data.mjs) into public/data/. Flags: --analytics-only, --validation-only (with --heroes 1,31 and --select-only), --brawl.

file	source	used by
items.json	assets…/v2/items/by-type/upgrade (all 251 upgrade items; shopable/disabled flags kept)	generator, UI
heroes.json	assets…/v2/heroes, filtered to player_selectable && !disabled && !in_development (38 heroes)	generator, UI
abilities.json	assets…/v2/items/by-type/ability for active heroes	generator, UI
analytics/<hero_id>.json	/v1/analytics/item-stats, ability-order-stats (top 400 by matches), item-permutation-stats (pairs, top 600); fetched twice: all ranks, and under top with min_average_badge=90 (Phantom+ lobbies). Under top.styles: one extra item-stats + ability-order-stats population per detected build style, fetched with include_item_ids / exclude_item_ids (see Build Styles)	generator only
validation/<account>-<hero>.json	for each hero, 5 top players picked from /v1/analytics/scoreboards/players (see Held-out Validation); /v1/matches/{id}/metadata for each player's 20 most recent matchmaking games on the hero	validation and scripts/tune.ts only
img/…	webp images for items, heroes, signature abilities	UI (no network needed)

• Analytics window: last 30 days (min_unix_timestamp), recorded in manifest.json. Live data changes daily. node scripts/fetch-data.mjs --analytics-only refreshes just the analytics files, keeping the same window.
• Population: builds are generated from the high-rank population when the hero's most-bought item has ≥ 500 high-rank matches (ability sequences: best sequence ≥ 200 matches), otherwise from all ranks. The board says which. Badge ≥ 90 was chosen as the highest bracket where all three endpoints are still well populated for every hero (≥ 100 leaves Infernus ability sequences with < 100 matches); it was fixed before the resulting agreement was looked at.
• "Real matchmaking" = match_mode 1 (Unranked) or 2 (Ranked) and game_mode 1 (Normal); private lobbies, bot, hero-labs and street-brawl games are dropped.
• Rate limit is 200 req/min; the script sleeps 350 ms between calls and honours retry-after on 429 up to 45 s. Old match metadata sometimes answers 429 with an hour-long retry-after; that match is skipped.
• The catalog has 251 items of which 173 are currently shopable (the other 78 are disabled/legacy). The generator only considers shopable, non-disabled items with a cost.