Store APIs and Images

Store APIs and Images (Dev Reference)

How the pipeline and the web app fetch game data and header images from each supported storefront. Living document; update in the same session as any pipeline change that touches a store API.

Steam

appdetails (metadata + header URL)

• Endpoint: https://store.steampowered.com/api/appdetails?appids=<id>&filters=basic
• No auth required. Rate limited (~200 requests / 5 min from a single IP; pipeline throttles via cache).
• Response: { "<id>": { "success": true, "data": { name, header_image, content_descriptors: {ids, notes}, ... } } }
• Used by: scripts/pipeline/game_images.py, scripts/pipeline/common.py (fetch_steam_title_with_source, fetch_steam_content_descriptors).
• Cache: .cache/steam-title-cache.json (30d TTL); .cache/steam-content-descriptors-cache.json (30d TTL).

Content descriptor IDs (adult filter)

Steam publishes numeric descriptor IDs on each app. The adult filter treats 3 and 4 as adult:

ID	Meaning	Filtered?
1	Some Nudity or Sexual Content	No (too broad — BG3, Cyberpunk 2077, Rust, GTA V)
2	Frequent Violence or Gore	No (most action games)
3	Adult Only Sexual Content	Yes
4	Frequent Nudity or Sexual Content	Yes
5	General Mature Content	No (CS2, DBD, Rust — mainstream M-rated)

Trust Steam's developer self-flagging: genuine adult titles (porn / hentai / VNs) carry ID 3 or 4. Mainstream M-rated games only get 1, 2, or 5.

Constant lives in scripts/pipeline/common.py::ADULT_DESCRIPTOR_IDS. Adding a new ID is a conscious edit; drop the test in tests/test_content_descriptors.py too.

Storefront tags vs content descriptors — do NOT confuse them. The Steam store page's Tags row ("Sexual Content", "Nudity", "Visual Novel") is user-curated community tagging. Content descriptors are a separate developer-set field returned via content_descriptors.ids in the appdetails JSON. Our filter uses only descriptors. A game with the "Sexual Content" tag but no descriptor 3/4 will not be filtered (correctly — mainstream games commonly carry that tag).

filters=basic gotcha — STRIPS content_descriptors

Steam's appdetails endpoint accepts a filters query parameter to trim the response. filters=basic removes content_descriptors from the payload. Any code path that needs the adult flag MUST call appdetails WITHOUT filters=basic:

• ✅ common.py::STEAM_APP_DETAILS_URL (no filter) — used by fetch_steam_content_descriptors. Correct.
• ⚠️ game_images.py::STEAM_APPDETAILS_URL (has filters=basic) — only used for header_image extraction. Do NOT reuse this URL if you need descriptors.

If the pipeline flips this by accident, every game silently ends up with adult: false because content_descriptors is None in the response. Caught this bug in prod on 2026-07-02 for Naughty Chat and 15k+ catalog stubs.

Which entries get descriptor-checked

Not every search-index row hits appdetails. Cost tradeoff:

Entry type	Descriptor check?	Notes
Steam game with local reports	Always	~16k API calls first run; cached 30d after that
Steam catalog stub (no reports) with adult-hint title	Yes	Only ~58 of 15k stubs match the regex (naughty|hentai|nsfw|adult|erotic|sexy|xxx|nude|nudity|lewd|kinky|18\+|sensual|waifu|ecchi|yuri|yaoi|bimbo)
Steam catalog stub without adult-hint title	Skipped	Row still gets adult: false in a 9-column shape so the frontend never sees N/A
GOG / Epic (any)	Skipped	No public no-auth API returns Steam-equivalent descriptors for those stores. Falls back to admin curation (#173)

The hint-regex approach is a heuristic — some genuine adult games with innocuous titles will still slip through. Store-APIs-and-Images.md and common.py must stay in sync when the regex changes. Full-catalog enrichment is the long-term fix, tracked in issue #176.

Header image URL chain

Steam header images live at multiple CDNs. The frontend walks the chain on 404 (see js/app/lib/steam-img.js):

1. https://shared.akamai.steamstatic.com/store_item_assets/steam/apps/<id>/header.jpg — canonical.
2. https://cdn.cloudflare.steamstatic.com/steam/apps/<id>/header.jpg — fallback CDN.
3. Hashed URL from game-images.json — the pipeline finds the current asset hash via appdetails (data.header_image) when the canonical path 404s and stores the hashed URL there. Format: .../steam/apps/<id>/<sha>/header.jpg.
4. Hidden if all three fail.

Web scrape fallback

For apps behind an age gate (18+) where the JSON API returns success: false, common.py::_scrape_steam_store_title fetches https://store.steampowered.com/app/<id> with the wants_mature_content=1 cookie and greps apphub_AppName for the title.

GOG

Catalog list

• Endpoint: https://catalog.gog.com/v1/catalog?limit=48&locale=en-US&page=<n>&order=asc:releaseDate&productType=in:game,pack
• No auth. Paginated (~48/page).
• Response contains products[].id (numeric productId) and products[].slug.
• Used by: scripts/pipeline/gog_catalog.py.
• Cache: .cache/gog-catalog-cache.json. Refreshed on every pipeline run (catalog is small).

Header images

• URL format: https://images.gog-statics.com/<sha256>.png where the SHA is stored per game.
• The pipeline extracts image URLs from catalog product responses and writes them to nonsteam-images.json keyed by gog:<productId>.
• No fallback CDN. If the URL 404s the image is broken.

Epic

GraphQL storefront

• Endpoint: https://store.epicgames.com/graphql
• No auth (public storefront queries), but requires:
  • Origin: https://store.epicgames.com
  • Referer: https://store.epicgames.com/en-US/browse
  • Standard browser User-Agent (Cloudflare gates on this).
• Used by: scripts/pipeline/epic_catalog.py.
• Response includes namespace, title, keyImages[] (with type = "OfferImageTall", "OfferImageWide", etc.).

Header images

• URLs come from keyImages[] in the GraphQL response — Epic hosts on their own CDN with hashed paths.
• Written to nonsteam-images.json keyed by epic:<namespace>.
• No fallback CDN.

Data files summary

File	Written by	Keyed by	Value
game-images.json	game_images.py (pipeline)	Steam appId	Hashed header URL for apps where the canonical path 404s
nonsteam-images.json	game_images.py + catalog scripts	gog:<id> or epic:<namespace>	Store CDN URL for GOG/Epic games
steam-catalog.json	most_played.py	Steam appId	Title for catalog-only apps (in Steam charts but not in the ProtonDB signal export)
.cache/steam-title-cache.json	common.py (local, not deployed)	Steam appId	{title, source, ts} — 30d TTL
.cache/steam-content-descriptors-cache.json	common.py (local, not deployed)	Steam appId	{ids, ts} — 30d TTL
.cache/gog-catalog-cache.json	gog_catalog.py (local, not deployed)	GOG productId	Full catalog snapshot
.cache/epic-catalog-cache.json	epic_catalog.py (local, not deployed)	Epic namespace	Full catalog snapshot

Store URL parsing (client-side)

js/lib/store-url-parser.js (with an inline copy in js/lib/topbar.js — kept in sync manually) parses pasted store URLs and returns { store, appId, canonicalId, slug }.

• Steam URLs yield the canonical numeric appId directly from the path.
• GOG / Epic URLs carry a human slug (not the internal canonical id). Slug → id resolution requires reading the catalog cache; the parser leaves that to the caller.
• Tests: tests/storeUrlParser.test.js.

Admin: Missing Box Art tab

/admin.html -> Missing Box Art loads all three cache files in the browser and lets an admin:

• Probe: HEAD the canonical URL for a row (Steam walks akamai -> cloudflare -> pipeline fallback; GOG/Epic re-probe the cached URL).
• Refetch: for Steam, hit appdetails and return the current header_image URL. For GOG/Epic, there is no public no-auth manifest API, so refetch is a re-probe of the cached URL.

Failed probes surface the HTTP status and error text so an admin can tell CORS, 404, region-lock, or removed-app apart. See js/admin/api/boxart.js and js/admin/components/boxart.js.

Update policy

Any change to a store API URL, response shape, or auth requirement lands in the same commit as the code that uses it. Cache files under .cache/ are gitignored — bumping their TTL constants is a code change and needs a matching update here.