Skip to content

Only generate responsive image variants for images that use them - #3281

Merged
borland merged 3 commits into
mainfrom
orion/responsive-images
Aug 7, 2026
Merged

Only generate responsive image variants for images that use them#3281
borland merged 3 commits into
mainfrom
orion/responsive-images

Conversation

@borland

@borland borland commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

How to review this PR

Go commit by commit and skip commit 2; there's too many files otherwise

  • Commit 1 is the change to the img.mjs script, so it only generates responsive images for things referenced by :img{ and bannerImage: frontmatter
  • Commit 2 is the cleanup, which deletes about 9,800 files. This isn't realistic to review, so I had claude programatically write a python script to verify that the commit only contains deletions of generated images, which it does.
  • Commit 3 is me just testing that I can add a new responsive image and it works. I changed the main page to use :img{ as there is significant benefit for the index page that everyone starts with - it contains two images, both of which are larger than they should be

Before

image

After

image

Reducing risk

I tested this both locally and on the staging site. All you need to do is browse to pages with images. The main /docs page now uses responsive images to prove that path, and basically any other page has a raw PNG in it


Claude's notes

The problem

public/docs/i/ holds 600/1000/2000px WebP variants plus an x/ fallback for every image in public/docs/img/7,905 files, 335 MB, all committed to the repo and all uploaded to blob storage on every deploy.

Almost none of it is ever requested. The only code that emits /docs/i/ URLs is getImageInfo in custom-markdown.mjs, and it only runs for two things: the :img{} remark directive and bannerImage frontmatter. Across 1,267 content pages, exactly one page (/docs/projects/badges) uses :img{}. Everything else writes plain ![](/docs/img/…) markdown, which serves the original straight out of /docs/img/.

So 7,905 committed binaries exist to serve two images.

I found this by grepping the built package: of 2,697 HTML files, 2 reference /docs/i/ at all.

The tree had also gone stale, because img.mjs only ever added files, never removed them. i/x/blueprint.png was still there long after its source image was deleted — and the site's own index page still pointed at it.

The fix

Rewrite src/themes/octopus/utilities/img.mjs to scan src/**/*.{md,mdx,astro} for the paths getImageInfo actually receives, generate variants only for those, and prune anything that no longer belongs.

That makes the tree self-maintaining: removing the last :img{} reference to an image now removes its generated output on the next npm run dev, and adding one generates just what that image needs. public/docs/i/ goes from 7,905 files to 8.

Three smaller things fall out of it:

  • 1,971 .json metadata sidecars dropped. These are build-time inputs — getImageInfo reads them out of public/ to emit width/height and cap the srcset widths — but because they live under public/ they were also copied into the deployed package and uploaded, where nothing ever fetches them.
  • The updated timestamp is gone from the sidecars. Its only reader is dist-thinner.mjs, which is not wired into the build and never runs; without the field it simply finds nothing to do. (dist-thinner is deleted on the housekeeping branch, which is independent of this one.)
  • The dead bannerImage in src/pages/index.md is removed. It pointed at /docs/img/blueprint.png, which no longer exists, so it produced a broken itemprop="image" — and it was the sole reason a stale variant survived.

I also fixed a latent bug in the generator while rewriting it: the original never awaited its sharp(...).toFile() calls, so writes raced against the process exiting.

Why remove rather than expand

Responsive images are worth having, and the honest alternative is to make every markdown image go through the pipeline — the win there would be large, since /docs/img/ is 214 MB of unresized PNGs. But that is a content-and-rendering change affecting 1,267 pages, not a build cleanup, and it should be its own decision.

What is not defensible is the status quo: paying 335 MB of repo weight and deploy time to serve two images, with no mechanism to keep it in sync. This PR makes the cost match the usage. If we later decide to apply :img{} broadly, the generator now scales to it correctly instead of brute-forcing every file in the tree.

Note that the variants must stay committed: astro build does not run img.mjs (only npm run dev does), so CI copies whatever is in public/. At 8 files that is fine.

Verification

Package size on this branch alone: 758.9 MB / 15,887 files → 483 MB / 6,020 files.

check result
asset references resolved against files on disk 20,298 checked, 1 missing — see below
npm run crawl (linkinator) 5,472 links, 0 broken
npx playwright test 99 passed, 0 failed
astro build clean, 2,697 pages

The one missing reference is /docs/img/icon-example.svg from components.mdx. It is absent from main and from the previously built package too, so it is pre-existing and unrelated — and components.mdx is an internal-only page that the housekeeping branch stops publishing anyway.

I confirmed /docs/projects/badges still renders correctly: the <img> keeps its srcset with all three widths, its x/ fallback, and the width/height attributes derived from the metadata sidecar.

Review notes

  • The diff is 9,870 files, but 9,868 of them are deletions of generated binaries. The reviewable change is src/themes/octopus/utilities/img.mjs plus three frontmatter lines in src/pages/index.md.
  • This branch is independent of Stop shipping build output that nothing can request #3280 (orion/housekeeping) and can merge in either order. The two overlap only in that housekeeping deletes 367 orphaned source images whose sidecars this branch also deletes; git resolves identical deletions without conflict, and img.mjs re-prunes on the next dev run regardless.

🤖 Generated with Claude Code

@team-marketing-branch-protections

Copy link
Copy Markdown

Pull request environment is available at https://stoctodocspr3281.z22.web.core.windows.net.

You can view the ephemeral environment status in Octopus Deploy.

This environment will be automatically deprovisioned when the pull request is closed, or after 7 days of inactivity.

@borland
borland force-pushed the orion/responsive-images branch 3 times, most recently from 0ad0b5f to 08432a7 Compare August 5, 2026 21:19
borland and others added 3 commits August 6, 2026 10:08
`public/docs/i/` holds 600/1000/2000px WebP variants plus an `x/` fallback
for every image in `public/docs/img/` - 7,905 files, 335 MB, all committed
and all uploaded to the CDN on every deploy.

Almost none of it is ever requested. The only code that emits `/docs/i/`
URLs is `getImageInfo` in custom-markdown.mjs, and it only runs for the
`:img{}` remark directive and `bannerImage` frontmatter. Across 1,267
content pages, exactly one page (/docs/projects/badges) uses `:img{}`;
everything else writes plain `![](/docs/img/...)` markdown, which serves the
original straight out of `/docs/img/`. So 7,905 binaries exist to serve two
images.

The tree has also gone stale, because `img.mjs` only ever added files:
`i/x/blueprint.png` is still present long after its source image was
deleted, and the site's own index page still points at it.

Rewrite `img.mjs` to scan `src/**/*.{md,mdx,astro}` for the paths
`getImageInfo` actually receives and generate variants only for those, then
prune anything that no longer belongs. That makes the tree self-maintaining
- removing the last `:img{}` reference to an image now removes its generated
output on the next `npm run dev` - and takes `public/docs/i/` from 7,905
files to 8.

This also drops 1,971 `.json` metadata sidecars. They are build-time inputs
that `getImageInfo` reads out of `public/` to emit width/height, but because
they live under `public/` they were also copied into the deployed package and
uploaded to blob storage, where nothing ever fetches them.

The sidecars no longer carry an `updated` timestamp. Its only reader is
dist-thinner.mjs, which is not wired into the build and never runs; without
the field it simply finds nothing to do. (dist-thinner is removed on the
housekeeping branch, which is independent of this change.)

Finally, drop the dead `bannerImage` from src/pages/index.md. It points at
`/docs/img/blueprint.png`, which no longer exists, so it produces a broken
`itemprop="image"` and is the sole reason a stale variant survives.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@borland
borland force-pushed the orion/responsive-images branch from 08432a7 to f663a69 Compare August 5, 2026 22:08

@enf0rc3 enf0rc3 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@borland
borland merged commit 093397f into main Aug 7, 2026
7 checks passed
@borland
borland deleted the orion/responsive-images branch August 7, 2026 00:25
enf0rc3 added a commit that referenced this pull request Aug 7, 2026
#3281 scoped variant generation to what getImageInfo actually consumed, which
was :img{} and bannerImage - four images. The figure plugin makes markdown
images inside a :::figure consume them too, so the generator now scans for
those, along with the Image component, and the tree covers 1,372 sources
instead of 4.

Two things keep that from undoing the win #3281 banked.

The x fallback is gone. It re-encoded every source at full size purely to
fill src= for browsers without srcset support, duplicating an image tree
that already ships under /img/. getImageInfo now points src at the source
itself: same pixels, same fidelity, one fewer copy. That alone is 115 MB,
and it retires the .jpeg/.jpg naming mismatch, since the generator wrote
.jpg where getDestination asked for .jpeg.

The generator also stops writing candidates the srcset will never name.
getImageInfo caps the list once the source runs out of pixels, so a 500px
image only ever names the 600w entry; generating 1000w and 2000w for it
produced files no markup references. That skips 406 of 4,116 variants.

Package size lands at 542 MB against 483 MB after #3281 and 758.9 MB before
it. The 59 MB buys webp at the right width for 1,372 images rather than 4:
the pages those images sit on drop from 158 MB of source PNG to 43 MB at
the 1000w candidate most desktop readers get.

Reading headers for 1,372 images needs batching - opening them all at once
exhausts the descriptor limit, which sharp reports as a missing input file.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants