Skip to content

Caching and Cache Busting

Jump to bottom
mdeguzis edited this page Jul 2, 2026 · 6 revisions

Caching and Cache Busting

How the site stops browsers and the GitHub Pages CDN from serving stale CSS and JS, and how to work with it day to day.

Why it exists

The site ships raw ES modules and CSS with no bundler. Browsers and the Pages CDN cache those files by URL, and they hold onto them. Without a cache key, a deploy that changes a file can leave visitors on the old version until their cache expires on its own. The fix is to put a content hash in the query string of every asset reference, so a changed file looks like a brand new URL and gets refetched.

How it works

scripts/cache-bust.sh rewrites two kinds of references. It runs from make build, and make pre-push calls it first.

  1. Relative ES module imports inside JS files, for example import { x } from './foo.js?v=ab12cd34'
  2. src and href references to local css/ and js/ files inside every *.html

The ?v= value is the first 8 hex digits of the md5 of the referenced file. Change the file, the hash changes, every reference to it changes, and the browser and CDN treat it as a new URL.

Transitive invalidation

A JS file's content includes the ?v= of its own imports. So when a leaf module changes, the importer's content changes too, which changes the importer's hash, and so on up the import graph. One edit to a shared helper busts the cache for the whole chain that depends on it, not just the helper.

This works even with circular imports between modules (e.g. router.js and the component renderers in js/app/). The hash for each file is computed from its stripped content (?v= parameters removed before hashing), so a file's hash depends only on its own source code and never on what it imports. The script writes the imports' ?v= values in a single pass and the result is idempotent. Earlier versions hashed raw bytes, which oscillated forever on a cycle; that is fixed (see issue #36).

When it runs

  • make build runs cache-bust on its own.
  • make pre-push runs it first, before Jest and the smoke test.
  • The deploy workflow does not regenerate hashes. It ships whatever is committed. Run make build (or make pre-push) and commit the result before deploying, or the live site references the wrong hashes.

Service worker image cache

Game cover art (Steam, GOG, Epic header images) is served by a service worker, sw.js, so the browse grid paints instantly on repeat visits instead of waiting on dozens of third-party CDN round trips. This is separate from the ?v= scheme above: cache-bust governs the site's own CSS and JS, the service worker governs cover images.

What it touches

Only image GET requests (request.destination === 'image'). Everything else (HTML, JS, CSS, JSON) passes straight through to the network, untouched, so the ?v= cache-bust stays the single source of truth for site assets and nothing the service worker does can make the app code go stale.

Strategy: cache-first with stale-while-revalidate

  1. A cached image is served immediately from Cache Storage. No network round trip, so it paints at once (DevTools shows the request served "from ServiceWorker" at 0 ms).
  2. On serve, the worker checks the entry's age. If it is older than the max-age (7 days), it kicks off a quiet background refetch that updates the cache for next time. The user still sees the cached copy instantly; the refresh never blocks paint.
  3. Fresh entries (younger than 7 days) skip the network entirely, so a cover we just fetched is never re-downloaded.
  4. On a cache miss, the worker fetches, caches, and returns the response.

Cover art almost never changes, so this keeps the paint instant while still healing a changed cover within a week.

Cache shape and eviction

  • Cache name: pp-img-cache-v1. Bumping the version (to v2, etc.) is the manual "flush everything" lever; the activate step deletes every cache whose name is not the current one.
  • Capped at 300 entries. Once it exceeds the cap, the oldest entries are trimmed (FIFO), so heavy browsing keeps a rolling window of the most recent covers.
  • The browser can also evict the cache under storage pressure, or the user can clear site data.

There is no time-based purge beyond the 7-day revalidation. Entries live until trimmed by the 300 cap, wiped by a version bump, or evicted by the browser.

Why timestamps live in IndexedDB

Cover images load no-cors, so their responses are opaque (status 0, headers unreadable). We cannot stamp a "cached at" header on an opaque response, so the per-URL timestamps used by the max-age check live in a small IndexedDB side table (pp-sw-meta, store ts), keyed by request URL. Entries written before timestamps existed read back as undefined and are treated as stale, so they self-heal on first access.

Failure handling

If the network fetch throws (offline, blip), the worker serves any cached copy, otherwise it returns Response.error() so the existing <img> onerror fallback chain (cloudflare -> game-images.json -> hide) still runs. The 404s you may see attributed to the service worker in DevTools are expected: a game whose primary CDN URL does not exist returns 404 through the worker, the image onerror fires, and the fallback resolves the real cover. Steam's CDN sends CORS headers, so those 404s are readable and the res.ok check skips caching them; only good responses are stored.

Registration

Registered from js/lib/topbar.js with a plain script tag (no build step). register('sw.js') resolves relative to the page, so it works at the production root and under the /proton-pulse-web-staging/ subpath alike. Scope is the directory the worker is served from, which controls every page on the site.

Cache hit-rate analytics

The worker counts cache hits and misses for the session. When a page is hidden (visibilitychange -> hidden), topbar.js queries the worker over a MessageChannel, and the worker replies with the counts and resets them so each report is a delta. The page then fires one sw_cache event through window.ppTrack into the site_events table. One aggregate event per session, not one per image.

The admin Analytics tab shows an "Image cache (service worker)" card: overall hit rate, images served from cache, misses, and how many sessions reported. The admin API pulls sw_cache events for the selected range and aggregates them client-side (hit rate plus a per-day series), so no Supabase migration was needed.

Files

Concern File
Worker (cache-first + SWR + max-age + counters) sw.js
Registration + per-session stats report js/lib/topbar.js
Event sink (ppTrack -> site_events) js/lib/analytics.js
Admin aggregation of sw_cache events js/admin/api/analytics.js
Admin "Image cache" card js/admin/components/analytics.js
Deploy listing (root file) gh-pages-manifest.txt

How to flush the image cache

Bump CACHE in sw.js from pp-img-cache-v1 to the next version and deploy. On activate, the worker deletes the old cache for every visitor. Use this only when covers genuinely need to be re-pulled site-wide; normal cover changes self-heal within the 7-day max-age.

Verifying it works

  1. Load app.html, reload, and watch the Network tab show header.jpg rows served by "service worker" at 0 ms.
  2. DevTools > Application > Service Workers shows it active; Cache Storage shows pp-img-cache-v1 filling.
  3. Switch tabs or navigate away to fire the stats report, then open the admin Analytics tab and check the Image cache card (needs a few real sessions to populate).

Tracked in #98 (cache) and #103 (analytics).

Pipeline data files

CSS and JS are hashed at build time by scripts/cache-bust.sh. Pipeline-emitted data files (search-index.json, recent-reports.json, most_played.json, etc.) are hashed at pipeline time by scripts/pipeline/data_versions.py and the result is written to data-versions.json next to the data files.

Why a separate mechanism

Build-time hashing assumes the file lives in the source tree when the build runs. The pipeline data files are produced by a long-running CI step that runs after the build, so the build cannot see them. We hash them in the same step that produces them and emit a tiny manifest the frontend reads at runtime.

Manifest shape

{
  "search-index.json": "a1b2c3d4",
  "recent-reports.json": "deadbeef",
  "most_played.json": "12345678"
}

Keys are filenames the frontend can fetch. Values are the first 8 hex characters of the SHA-256 of the file contents -- same shape as the JS/CSS ?v= hashes for visual consistency. Missing files (e.g. a catalog cache that did not run this time) are skipped silently.

How the frontend uses it

js/lib/data-url.js exports an async dataUrl(name) helper:

import { dataUrl } from '../lib/data-url.js?v=...';
const r = await fetch(await dataUrl('search-index.json'));

The first call fetches data-versions.json with cache: 'no-store' so the manifest is always fresh. Subsequent calls share the in-flight promise. Once resolved, every data fetch returns name?v=<hash>, which the browser and CDN can cache indefinitely. A new pipeline run with new content changes the hash, the URL becomes new, and the browser refetches.

If the manifest is missing or does not have an entry for the file, dataUrl falls back to the bare name -- staging without its own pipeline output and one-off files still work.

Files that route through dataUrl

  • search-index.json -- js/app/components/search.js, js/index/main.js, js/profile/components/my-reports.js
  • recent-reports.json -- js/app/components/home.js
  • most_played.json -- js/app/components/home.js, js/index/main.js
  • stats.json -- js/stats/main.js

js/lib/topbar.js still fetches search-index.json without the helper because it is a classic script and cannot import an ES module synchronously. The topbar dropdown's search results stay cached normally; a small follow-up could wrap it via a global registered from data-url.js.

One round-trip cost

Each page load adds one tiny data-versions.json request before the first data fetch. The manifest is a few hundred bytes -- negligible compared to the multi-megabyte data files it indexes. The pay-off is no more stale data after a deploy.

Files

Concern File
Pipeline manifest writer scripts/pipeline/data_versions.py
Frontend helper js/lib/data-url.js
Tests tests/test_data_versions.py

Tracked in #119.

GitHub Pages CDN cache

After a correct deploy, the Pages CDN still holds the previous copy of a file at its edge for a short window, usually seconds to a couple of minutes. Two things follow from that:

  • A file you just deployed can briefly still serve the old content. That is normal propagation, not a failed deploy.
  • To confirm what actually shipped, bypass the edge cache with a throwaway query string:
curl -s "https://mdeguzis.github.io/proton-pulse-web-staging/admin.html?cb=$(date +%s)"

The server ignores the ?cb= value, but the CDN sees an uncached URL and serves the freshly deployed file.

Mobile Android browser caching

Android Chrome (and other mobile browsers) sometimes cache HTML pages aggressively, ignoring CDN TTLs. The result is the browser loads a stale HTML file that references old ?v= hashes, so none of the new JS or CSS is picked up even after a successful deploy. The about page shows the correct version (it fetches version.json at runtime, bypassing the HTML cache), but the rest of the UI stays on the old build.

The fix applied as of 2026-06-18: every HTML page now includes three meta tags immediately after <meta charset="utf-8">:

<meta http-equiv="Cache-Control" content="no-cache, no-store, must-revalidate">
<meta http-equiv="Pragma" content="no-cache">
<meta http-equiv="Expires" content="0">

These tell the browser the HTML must always be revalidated against the server. JS and CSS are unaffected -- they are still cached normally via their ?v= hashes and only refetched when the hash changes.

If a user on mobile reports seeing an old version after a confirmed deploy:

  1. Ask them to check the about page for the expected version and commit SHA.
  2. If the about page shows the right version but the UI is wrong, it is a page-level HTML cache. A hard reload (long-press refresh in Chrome, or clear site data for the domain) will fix it.
  3. The no-cache meta tags should prevent recurrence for deploys after 2026-06-18.

Quick reference

Task Command
Regenerate all ?v= hashes make build
Regenerate, then test and smoke make pre-push
Verify a deployed file, bypassing the CDN curl ".../file?cb=$(date +%s)"

Last verified: 2026-07-02.

Clone this wiki locally