Fall back to unoptimized images when Sharp is installed but can't load

[alexmv]

Body

Summary

When sharp is installed but its native binary fails to load, have the default image service warn and pass images through unoptimized, instead of failing the build with MissingSharp.

Background & Motivation

Sharp's prebuilt Linux x64 binaries require an x86-64-v2 CPU. On older hardware, or on VMs using a generic CPU model like kvm64 or qemu64 (common on self-hosted hypervisors), import('sharp') throws Unsupported CPU: Prebuilt binaries for Linux x64 require v2 microarchitecture. Astro turns this into MissingSharp: Could not find Sharp, and the build fails. withastro/astro#18432 addresses that misleading error message, by surfacing the "unsupported CPU" part; this proposal is about not failing at all, and instead falling back to providing unoptimized images.

For projects that build on the deploying machine, like Zulip's self-hosted help center (built with Starlight), this turns a routine upgrade into a hard failure the administrator can't fix. Several administrators were stuck for months -- one gave up and migrated to a new server (pretty much declaring the old hardware haunted), and others decided to patch astro.config.mjs by hand on every upgrade.

Astro already does this kind of fallback in a nearby case. When Sharp can't decode a particular image, the service warns ("Astro could not optimize image … The image will be used unoptimized") and continues. Extending that to also cover the case of "Sharp can't load at all" seems consistent with it. #16451 pretty much used the argument that "a warning and not crashing seems like a better UX" when there was "no real solution on the user side" which this also fits.

We're working around it in Zulip (zulip/zulip#40415) by trying import('sharp') in our config and switching to passthroughImageService() if it fails. That works, but ideally this behavior would be handled by Astro itself, instead of in every project that uses it.

Goals

• A Sharp binary that is successfully installed but can't run shouldn't fail an otherwise-valid build.
• The user should still see clearly that images weren't optimized, and why, including Sharp's own error.
• A project where sharp isn't installed at all should keep failing loudly. That's usually a dependency mistake worth fixing, not something to hide.

Open questions

• Do we need a strict mode which fails instead of allowing unoptimized images? Astro can already leave some images unoptimized -- but that varies by the image, not by the deployment. This would widen the cases that could happen, and make it possible that two installs on different hosts end up with different served images. Should a project be able to opt into "no, fail if you can't optimize everything"? My gut says we don't need that flag until someone asks for it, and this is probably a very niche case.
• Limitations: on CPUs without SSE4.1 (roughly pre-2008 Intel or pre-2011 AMD), Sharp's binary crashes the process with SIGILL instead of throwing, so an in-process fallback can't help there. Checking from a child process could, at the cost of spawning an extra process; this probably isn't worth it.

[delucis]

For cases like this, shouldn’t a project build sharp so that it can run on the target architecture? https://sharp.pixelplumbing.com/install/#building-from-source

[alexmv]

Building from source would avoid the check -- but it needs a C++ toolchain, node-addon-api/node-gyp, and a global libvips of 8.18.7 or later, which no current Debian or Ubuntu LTS ships.

In our case the "project" is Zulip, which builds its help center on self-hosted servers during upgrades from git, so the target is whatever hardware each admin happens to run. We can't reasonably require every admin with an older CPU to build libvips from source, especially for something where the only cost of falling back is slightly larger images.

More generally, Sharp is an optional optimization, and Astro already chooses to warn and pass through when Sharp can't handle a given image (withastro/astro#16451). A Sharp that can't load seems like the same flavor of situation.

[delucis]

I wonder in this case whether you could ship a custom image service which would let you enable the passthrough behaviour if Sharp fails (or even provide e.g. a WASM-backed optimizer in those cases).

Custom services can re-use the default Sharp service (e.g. I have one that reuses the sharp service for various things), so it could match the default and you’d just customize how missing Sharp is handled. (Obviously, Astro doing this for you would be even easier, but I’m still not 100% convinced that’s the right behaviour for most users.)

[alexmv]

I wonder in this case whether you could ship a custom image service which would let you enable the passthrough behaviour if Sharp fails

Yup, that's precisely what zulip/zulip#40415 already does. However, my purpose in proposing this change was to attempt to make it so not every use case of Astro had to write that same behaviour.

but I’m still not 100% convinced that’s the right behaviour for most users

Can you tell me more concretely the "most users" failure mode you're worried about? The proposal is still that Astro fails loudly if Sharp is uninstalled, so we don't mask real breakage in the dependency installation step; it's changing the behaviour if Sharp is installed but inoperable, which is matching the existing case that Astro does for when Sharp can't handle specific images. I feel like most users would prefer to have a working (albeit unoptimized site) and a warning, rather than none at all.

[delucis]

I feel like most users would prefer to have a working (albeit unoptimized site) and a warning, rather than none at all.

The issue with warnings is that they are very easily missed. Especially in build output where very often there are 100s of lines of output. So the end result can easily be people either not noticing things are optimized (and therefore not fixing it although they’d want to) or noticing and assuming it was an Astro bug rather than a configuration question.

If the idea was to still fail by default, but offer an option to ignore the failure instead of fixing it, then maybe that could work.