RFC: cached islands

[ascorbic]

Summary

A new island primitive: cached islands, using a new server:cache directive. A cached island is server rendered once, stored, and replayed into subsequent responses until it expires or is invalidated. The page around it stays fully dynamic and rendered on-demand, while the expensive shared subtrees inside it stop being rendered per request.

This is the inverse of server islands: they are dynamic holes in a static shell, this is durable fragments in a dynamic shell. Slots are per-request, so islands can nest: dynamic holes inside cached fragments inside dynamic pages.

Background & Motivation

Astro provides several ways of speeding up responses, using caching and prerendering. Prerendering and route caching store the whole page, while server islands allow dynamic islands inside a static shell.

Prerendering is all-or-nothing per route: anything that requires personalisation, customisation or anything else dynamic means the whole tree must be rendered on-demand, whether as a server island or as an on-demand route. The new route caching feature solves this at the response level, but a response cache is keyed by URL: a page whose shell varies per user either can't be response-cached. Those are exactly the pages whose expensive subtrees are shared – the product grid, the CMS fragment, the mega-nav.

Server islands can in theory be cached, but this is not ergonomic and will still need to be loaded separately from the shell, meaning that the cached part perversely appears slower than the dynamic one.

Several of the largest Astro deployments (incuding major ecommerce and news sites) already solve this by hand: render a fragment, stash the HTML in a KV or blob store, replay it with set:html, invalidate by TTL or webhook. Everyone rebuilds the same plumbing, and maintains a complex architecture that is only sustainable for the largest organisations. Also, everyone hits the same bugs – because a rendered Astro fragment is not representable as an HTML string. The runtime interleaves markup with render instructions (hydration bootstrap, framework renderer scripts, CSP hash registration) that are deduplicated per response; cache a string and you double-emit or drop them depending on what else is on the page. Cached HTML also embeds per-build asset URLs and per-build encrypted server-island props, so entries that outlive a deploy break silently. And string caching freezes children, so anything with a personalised child cannot be cached.

It is tempting to propose fixing this in userland, such as with a first-class Cacheable component. However these drawbacks are not fixable outside core, because the solution is caching render chunk streams, not HTML strings – markup plus those instructions, persisted symbolically and replayed through the normal per-response machinery. A replayed island is indistinguishable from a rendered one, which is why nested client islands, nested server islands and CSP all just work.

Goals

• Semantics-preserving. server:cache does not change behaviour: identical markup, streaming, hydration and CSP. On a hit the island is just faster than plain SSR.
• Shared API with route caching. The same maxAge / swr / tags vocabulary, the same Astro.cache / context.cache API, and one invalidate() that clears tagged routes and tagged islands together.
• Bounded freshness (maxAge + stale-while-revalidate), tag purge, key purge and automatic invalidation on deploy.
• All existing island types keep working inside a cached island; cached islands nest inside each other, and invalidating a tag handles every layer.
• Slots stay per-request – allowing dynamic holes inside the cached fragment.
• Live collection cacheHints flow into island entries through the same cache.set(entry) call route caching accepts.
• Pluggable storage built on the web Cache API, with adapter defaults so most projects configure nothing.
• Development stays always-fresh (possibly with per-island visibility in the dev toolbar).

Non-Goals

• Changing route caching – this adds the fragment level beneath it.
• Build-time pre-rendering (could be a follow-up).
• Cache variance per-request: everything an island depends on arrives through props (used in cache key) or slots (per-request holes).
• Data/function caching.
• Caching inside prerendered pages (no-op – maybe warning).

Example

---
const user = await Astro.session.get('user');
const currency = Astro.cookie.get('currency');
const category = await getCategory(Astro.params.slug);
---
<Layout user={user}>
  <ProductGrid
    category={category.id}
    currency={currency}
    server:cache={{ maxAge: 600, swr: 21_600, tags: [`category:${category.id}`] }}
  >
    <!-- per-request hole inside the cached fragment -->
    <CartBadge slot="corner" user={user} />
  </ProductGrid>

  <Recommendations user={user} server:defer />
</Layout>

Everything per-request reaches a cached island one of two ways: through props, which become part of the cache key, or through slots, which are rendered fresh on every request and inserted into the stored fragment. Reading request state directly – Astro.request, Astro.cookies, Astro.session etc – inside a cached subtree will throw in the build, with a runtime guard for dynamic access.

The bare boolean form server:cache uses configured defaults and an automatic key, derived from component identity, props, locale and a per-deploy id – optionally extended with a key. User keys can't collide across components, locales or deploys, and every deploy opens a fresh namespace, preventing stale-asset bugs.

Inside a server:cache subtree, Astro.cache is island-scoped with the same semantics as route caching: set({ maxAge, swr, tags }) configures the entry, set(liveDataEntry) uses a live collection's cache hints, and set(false) opts this render out entirely. Components calling cache.set() become scope-agnostic: the same code declares cacheability whether it's the whole response or one island of it.

Invalidation uses the same API as route caching: one call clears both layers:

export const POST: APIRoute = async ({ request, cache }) => {
  const { tags } = await request.json();
  await cache.invalidate({ tags });
  return new Response('ok');
};

Configuration nests inside the existing cache block. Adapters supply platform defaults, so this is optional:

cache: {
  provider: memoryCache(),                    // route level, unchanged
  islands: {
    store: cacheStores.web('astro:islands'),  // native CacheStorage where available
    defaults: { maxAge: 300, swr: 3600 },
  },
},

Storage is the web Cache API (caches.open()) – native on Cloudflare, Deno and Netlify, with pluggable providers everywhere else – with entries regional and tag state global, and an optional tiered mode that backs regional caches with a global store so an expensive fragment renders once worldwide. In development, storage is a noop like route caching: islands render fresh every time and the toolbar shows each island's derived key, tags and freshness instead.

The directive is implemented only in the experimental Rust compiler, so this proposal requires experimental.rustCompiler.

[delucis]

I’ve seen a few people ask about this before (here’s a userland proof-of-concept I coded for someone to do something similar, although more limited than what a built-in option could do). It’s mainly for really high traffic and complex sites, but the API described here sounds like a nice approach for those who do need it.

[matthewp]

Cool idea! One potential issue I guess is that the outer parts need to have shorter cache life than the inner parts or it doesn't do what you want. For example, if the ProductGrid caches for 2 days but the page caches for 3 days, then the ProductGrid only updates after the page does. Am I thinking about this right, or were you thinking there would be something that stitched these things together?

And then I guess related, is this a problem for the CMS use-case? I imagine you are going to want to give a short cache life for the listing and article parts, and longer for the shell. Can you do that with this model?