Request for position: JPEG XL

[saschanaz]

Request for Mozilla Position on an Emerging Web Specification

• Specification Title: JPEG XL Image Coding System
• Specification or proposal URL:
  • ISO standards:
    • Part 1: https://www.iso.org/standard/77977.html
    • Part 2: https://www.iso.org/standard/80617.html
    • Part 3: https://www.iso.org/standard/80618.html
    • Part 4: https://www.iso.org/standard/80619.html
  • Project page: https://github.com/libjxl/libjxl
• Caniuse.com URL (optional): https://caniuse.com/jpegxl
• Bugzilla URL (optional): https://bugzil.la/jpeg-xl
• Mozillians who can provide input (optional):

Other information

This is included in Nightly 90 but there hasn't been an active discussion about whether this is the right way to go.

[hsivonen]

I think JXL is, in a good way, unlike multiple other new potentially-lossy image formats in three key ways:

1. Can recompress JPEG without further visual changes. (I think this is a big deal for migration path, since it allows size wins without human supervision for quality loss. Also, this gives the format a more legitimate claim to a "JPEG something" brand than e.g. JPEG2000. It should be pretty non-controversial that this property can be verified, though I haven't personally done the verification.)
2. Is designed for high-fidelity still photos. (The primary way both JPEG2000 and the video codec-based image formats improve on JPEG is that they have non-awful but still visible artifacts at compression ratios where JPEG has very awful and very visible artifacts. I think it's important to also address the case where you don't want visible artifacts but still want a substantial compression win over JPEG. How well JXL meets its design intent on this point is harder to assess than the previous point. I haven't done this assessment personally, either, but I react favorably to this even being the design intent.)
3. Less generation loss, so better suited for authoring workflow. (I haven't verified this personally)
4. Better progressive decoding story: Does not need a site-provided mechanism for low-res placeholder swapping if the browser implements incremental painting of incremental decode.
5. No tile boundaries based on maximum video frame size. (Unclear if at all relevant at high-fidelity compression ratios.)

Unless a Mozilla expert shows that key marketing claims are wrong, I think point 1 alone is such a big deal that we should take a favorable position on the format on point 1 alone, and, personally, I've been hoping that Firefox would add support.

For point 2, I think the interesting case when adding a high-fidelity still photo format when you already have AVIF is not so much comparing particular photos with tight compression ratios but finding JXL and AVIF settings such that one can be confident that the software will produce a high-fidelity encode of pretty much any photo without human supervision and then compare which one of JXL or AVIF ends up generating smaller files on average for this kind of "unsupervised high-fidelly for sure" configuration. I feel I don't have appropriate data to advocate a position based on point 2.

Point 3 seems like a big deal for being able to use a new format throughout the workflow in the future the way JPEG is used today as opposed to the format being something than the end of the pipeline of a CDN vendor generates for the last hop. Another strong point in favor when considering the format ecosystem-wide and not just for the step of displaying it in a browser.

Point 4 seems a nice additional point in favor.

I'm less enthusiastic about the decoder at hand being a bundle of multithreaded C++ instead of being a Rayon-using Rust crate. I don't want to block either the position on the format or enabling it in Firefox on RIIR, though.

[martinthomson]

Based on the discussion I've seen here and elsewhere, it looks like were resolving on 'worth prototyping' here. @saschanaz, do you think you can create a pull request that says that?

There are a few issues that need to be worked through. @annevk notes that the code does sniffing and should not. I agree. Henri's concerns about implementation are valid, but not something that should factor into the decision here. I also see that this is a real content type (image/jxl) rather than a content coding, which resolves a concern I had in earlier iterations of this. So it seems like this is in a good place generally; more so when Anne's concern is resolved by removing sniffing.

[bholley]

Beyond the specific technical merits of the format, I think our position needs to account for the broader context and facts on the ground. Since AVIF has basically already shipped in Firefox and Chrome, the bar on a second same-generation format is much higher. If there's no clear winner, two formats bifurcate the ecosystem. If there is a clear winner, browsers still have to support the loser indefinitely.

Lossless JPEG re-compression might clear that bar if the lack of such a feature ends up being a significant barrier to AVIF adoption, but that feels pretty speculative at this juncture. As such I recommend we mark this non-harmful for now and potentially reassess down the road based on how AVIF fares in the wild.

[veluca93]

Based on the discussion I've seen here and elsewhere, it looks like were resolving on 'worth prototyping' here. @saschanaz, do you think you can create a pull request that says that?

There are a few issues that need to be worked through. @annevk notes that the code does sniffing and should not. I agree. Henri's concerns about implementation are valid, but not something that should factor into the decision here. I also see that this is a real content type (image/jxl) rather than a content coding, which resolves a concern I had in earlier iterations of this. So it seems like this is in a good place generally; more so when Anne's concern is resolved by removing sniffing.

Without going into the merits of whether one should have sniffing for new image formats or not (my opinion on it is here: w3ctag/design-reviews#633, a tldr is that I don't particularly see how the world gets better without sniffing just for new image formats), I would like to remark that at the moment AVIF does sniffing too. I don't think that the decision of whether to enable sniffing for JPEG XL should be different from the decision for AVIF :)

[annevk]

Thanks @veluca93 for raising that again. I filed AOMediaCodec/av1-avif#149 to track the AVIF issue. I had mistakenly somewhat ignored it as it reuses a media container format, but that doesn't mean no changes are needed or an improvement in delivery cannot be made.

[bholley]

Facebook has some words of support for jxl [1], which I'm linking to here to keep all the inputs discoverable.

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1539075#c18

[hsivonen]

second same-generation format

I think the key question is whether they address the same ecosystem role.

The justification for two formats would be that they address different ecosystem roles. Arguably, WebP and JPEG today address two different roles: WebP addresses the tightly-compressed last-step delivery-to-browser case and JPEG addresses the high-fidelity works-throughout-the-workflow case.

Lossless recompression of JPEG and generation loss resistance seem like points in favor of the notion that JPEG XL would be well suited for the high-fidelity works-throughout-the-workflow case. (Working in browsers is part of "throughout", so if it doesn't work in browsers, the "throughout the workflow" story breaks.)

It's easy to believe that AVIF works well as a replacement for WebP for tightly-compressed last-step delivery. However, AVIF seems to have non-converging generation loss. Is the generation loss slight enough that we should still expect AVIF to be accepted as an intermediate format in photo workflows that use JPEG today?

[bholley]

Lossless recompression of JPEG and generation loss resistance seem like points in favor of the notion that JPEG XL would be well suited for the high-fidelity works-throughout-the-workflow case.

That's certainly plausible, but as I said, very much speculative. And by definition, that story depends on momentum in a bunch of other places in the stack (operating systems, authoring tools, etc). I'm open to supporting JPEG XL in Firefox if that momentum materializes, but I'd like to see evidence of it first.

(Working in browsers is part of "throughout", so if it doesn't work in browsers, the "throughout the workflow" story breaks.)

Call me jaded, but there is a long history (MNG, JPEG 2000, JPEG XR) of this shape of image format advocacy, and while we can't observe the counterfactual, I am skeptical that Firefox support would have made any of these formats successful. JPEG XL may yet succeed, but if it does we will have ample time to take notice and adjust accordingly.

[jonsneyers]

Lossless recompression of JPEG and generation loss resistance seem like points in favor of the notion that JPEG XL would be well suited for the high-fidelity works-throughout-the-workflow case.

That's certainly plausible, but as I said, very much speculative. And by definition, that story depends on momentum in a bunch of other places in the stack (operating systems, authoring tools, etc). I'm open to supporting JPEG XL in Firefox if that momentum materializes, but I'd like to see evidence of it first.

What kind of evidence would you like to see? I would assume that Facebook declaring to be basically ready to deploy it when browsers are ready for it, is a relatively strong evidence, right? What other evidence would be required to establish "momentum materializing"?
(asking because I am genuinely curious, it is not a rhetorical question)

[bholley]

What kind of evidence would you like to see?

Native support in major operation systems and authoring tools, primarily.

I would assume that Facebook declaring to be basically ready to deploy it when browsers are ready for it, is a relatively strong evidence, right?

Not of the "works-throughout-the-workflow" case that we were discussing above, no. I think Facebook is squarely in the "last-step delivery-to-browser" bucket of Henri's taxonomy. That bucket matters too, and I certainly value Facebook's perspective — which is why I linked to it above — but it's not the only site on the Web either.

Taking a step back: our position here is that we don't oppose JPEG XL if it manages to achieve critical velocity, but that we're also not going to put our finger on the scale right now in the way that some of its proponents seem to be hoping for. To that end I'd ask folks to refrain from general advocacy in this thread, though comments of the form "X is deploying JPEG XL for reason Y" are welcome as we continue to monitor the situation.

[jonsneyers]

That makes sense. Note though that in my experience, most major operating systems and authoring tools, at least the proprietary ones, don't tend to move extremely quickly when it comes to adding native support for new codecs. For example, for WebP and AVIF, I don't think it can be said that these are natively supported in most major OSs and authoring tools. Neither of them are natively supported in Windows or in Photoshop, for example.

I do understand the hesitation to add support for a new codec, and I think it is well-justified to be sceptical especially when it comes to new image codecs, where indeed there have been several failed attempts at dethroning the good old JPEG already. But if the (justifiably strict) criteria for adding support are different for WebP and AVIF than they are for JPEG XL, then that does feel a little weird.

Besides momentum and "works throughout the workflow", which I agree are important aspects, I think it would also be useful to consider the technical pros and cons, in terms of web-oriented features, compression density, etc.

Back when WebP was first proposed, if I recall correctly, mozjpeg was created to demonstrate that the old JPEG was actually basically just as good as the (early versions of) WebP. I think that was great. Firefox decided not to adopt WebP, and it was justified based on technical arguments, with benchmark results etc. That in turn lead to encoder improvements in cwebp.

It would be interesting if mozilla could again perform such an independent investigation of the technical merits of all of the codecs currently on the table of the modern web: (moz)jpeg, webp, avif, jxl. That would be very valuable information, which might help make it easier to make solid decisions on where we want to go with not just firefox but with the web platform in general.

[baumanj]

Neither of them are natively supported in Windows or in Photoshop, for example

Not installed by default, but Microsoft does have official AV1 support, it seems: https://www.microsoft.com/en-us/p/av1-video-extension-beta/9mvzqvxjbq9v?activetab=pivot:overviewtab

[veluca93]

Neither of them are natively supported in Windows or in Photoshop, for example

Not installed by default, but Microsoft does have official AV1 support, it seems: https://www.microsoft.com/en-us/p/av1-video-extension-beta/9mvzqvxjbq9v?activetab=pivot:overviewtab

That's not the same thing as AVIF support though :)

[bholley]

I do understand the hesitation to add support for a new codec, and I think it is well-justified to be sceptical especially when it comes to new image codecs, where indeed there have been several failed attempts at dethroning the good old JPEG already. But if the (justifiably strict) criteria for adding support are different for WebP and AVIF than they are for JPEG XL, then that does feel a little weird.

The difference is that the support cost (maintenance, attack surface, code size) for a video-derived codec is much lower, because browsers ship the associated codec already for video decoding. AVIF support in Firefox weighs in at about 1k lines of straightforward glue code. JPEG-XL is roughly 100k lines of multi-threaded C++.

It would be interesting if mozilla could again perform such an independent investigation of the technical merits of all of the codecs currently on the table of the modern web

I think there are enough cooks in this kitchen already. :-)

[baumanj]

That's not the same thing as AVIF support though :)

But they do support AVIF if you have the official plugin installed: https://avif.io/blog/tutorials/use-avif-in-windows/#microsoftsupportsavif

[saschanaz]

Neither of them are natively supported in Windows or in Photoshop, for example.

WebP is supported by File Explorer and Paint, while interestingly not by Photos app, without any extension installed.

Edit: Oh, actually the extension has been installed https://www.microsoft.com/store/productId/9PG2DK419DRG

[jonsneyers]

I do understand the hesitation to add support for a new codec, and I think it is well-justified to be sceptical especially when it comes to new image codecs, where indeed there have been several failed attempts at dethroning the good old JPEG already. But if the (justifiably strict) criteria for adding support are different for WebP and AVIF than they are for JPEG XL, then that does feel a little weird.

The difference is that the support cost (maintenance, attack surface, code size) for a video-derived codec is much lower, because browsers ship the associated codec already for video decoding. AVIF support in Firefox weighs in at about 1k lines of straightforward glue code. JPEG-XL is roughly 100k lines of multi-threaded C++.

That argument cuts both ways though. Yes, image codecs derived from an already-supported video codec are basically 'free', but it also means that retiring support for a video codec becomes harder. In the world of video, there's a new codec every 5 years, codecs come and go, and servers are used to serving multiple codecs. Once a video codec becomes an image format too though, it likely means it will need to stay in the codebase for much longer. Maybe at some point you'll want to retire VP8 and AV1 as video codecs because everyone is using AV2 or AV3 anyway, but it won't be possible because it would break too many images...

[erikandre]

Crossposting an updated version of my post from the other task, since I wanted to clarify some of the reasons why we're investing in JPEG XL support at Facebook and why we're hoping to get wider browser support. Apologies if this increases the noise in this thread!

• Encode performance: JPEG XL encoding performance at speed/effort 6 is comparable to JPEG encoding using MozJpeg with Trellis encoding enabled. This means that it's practical to encode JPEG XL images on the fly and serve to clients. This can be compared with the encoding speed of AVIF which necessitates offline encoding which offers much less flexibility when it comes to delivering dynamically sized and compressed content.
• Decode performance: Depending on the settings used, JPEG XL can also be very fast to decode. Our mobile benchmarks show that especially for larger images we can reach parity with JPEG when using multiple threads to decode. This matches and in some cases surpasses the decoding performance of other new image formats.
• Progressive rendering: The JPEG XL image format supports progressive decoding, offering similar gains in perceived image load performance we are already benefitting from with JPEG.
• Image quality: Perhaps eve more importantly, JPEG XL offers great image quality at very competitive bitrates. Of course, there’s always cases where one image format delivers a better looking image than the other, or cases where one needs more bytes than the other to achieve the same quality but overall we’ve not found any issues with the image quality offered by JPEG XL.

The issue of wider support from operating systems and common photo viewers and editors is definitely a concern. At the moment, for most non-technical users, they will be in a similar position with both AVIF and JPEG XL in the sense that out of the box they wont' be able to open them in anything but their web browser. While that's not ideal, we're hoping that by getting to a critical mass we'll help encourage others to support the new format.

[kornelski]

I'm responsible for image processing products at @cloudflare. We're optimistic about JPEG XL and are interested in supporting it for still images.

• We've found that AVIF (using rav1e) is very slow to encode when used at quality levels beating WebP. Encoding cost is high enough that it makes AVIF difficult to deploy. JPEG XL is much faster.

• Overwhelming majority of images hosted by our customers is in JPEG. JPEG XL's ability to quickly and losslessly transcode JPEG is an advantage here.

• We do not plan to support any animation features of JPEG XL. We don't support animated AVIF either. If we had to pick one, then AV1 payload makes way more sense for animations. We would prefer for "animation format" as a concept to be completely abandoned in favor of regular video formats supported in <img> instead (Safari does it well).

Currently AVIF seems more mature, as it has multiple implementations, including a Rust one. JPEG XL having a sole C++ implementation is a bit of a problem for us, because we'd rather not use C++ any more.

[jyrkialakuijala]

@bholley I try to summarise my personal viewpoints on the situation here for some more food of thought on the web codec situation.

I observe 39 % win in a multi-corpus lossless compression test for JPEG XL vs. AVIF. In the same study I observe JPEG XL as a lossless compression leader of all image formats intended for the web, including WebP lossless, AVIF, and PNG. I observe that some research compressors (BMF and EMMA) can still do better (by 5-15 %), but they would be too slow for web use.

In the standardization experiments at ISO (Core Experiment 8.2 / WG1M90008), JPEG XL's lossless coding was tested against other lossless mage codings using FLIF -N, FLIF, WebP lossless, FUIF -R0, FUIF -R1, JPEG-LS2, ZopfliPNG, PNG-Pingo, JPEG 2000, JPEG-LS, JPEG XS, JPEG LLA, JPEG LL and Lossless JPEG. JPEG XL produced consistently the most dense results in all categories: 8 bit per channel non-photo, 8 bit per channel photo, and the more than 8 bit per channel category. In that experiment JPEG XL produced 45 % smaller files in the 8+ bit lossless category than any currently available web codec in the 8+ bit category (ZopfliPNG was the best currently available option for 8+ bit lossless Web transport -- FLIF is not available in the web, WebP lossless only up to 8 bits). In 8 bit categories, the 2nd and 3rd place in lossless density in the Core Experiment 8.2 went to FLIF and WebP lossless. Authors of ZopfliPNG, FLIF, FUIF, and WebP lossless participate in the development of JPEG XL and openly support JPEG XL as a way forward.

I observe a similar 25-40 % win for psychovisually lossless compression for JPEG XL vs. current AVIF encoders. Often the problems relate to textures and oversmoothing, and removal of small details. Compare MozJpeg, JPEG XL, and AVIF. The AVIF and JPEG XL are courtesy of the WebP/WebM team, Mozjpeg by eclipseo.

I observe a 'it-just-works' approach used in JPEG XL. You can literally just write cjxl input.png output.jxl and be happy with the results. No difficult decisions about colorspaces, no SDR/HDR decisions (same modeling in both cases), yuv444/422/420, no --tune=psnr/vmaf/ssim, no flag exploration.

Professional and prosumer photography workflows need more bit depth than AVIF is giving, preferrably 16 bits. Libjxl gives 24 bits of dynamics, twice that of AVIF's up to 12 bits. In this article the photographer slightly prefers 14 bits over 12 bits for his workflows: "If you want the absolute best quality in the shadows, shoot 14+ bit RAW files (ideally with lossless compression to save space). This is the best choice if you don’t care about larger files and shoot scenes with wide dynamic range (deep shadows)."

I observe higher resulting image quality in medium-to-high-BPP (above 0.5 BPP) for JPEG XL, including both Internet (0.5-2 BPP, histogram from this article) and digital camera (3-5 BPP) use cases.

I observe a that JPEG XL provides online lossless transcoding from JPEG to JPEG XL and back, with 16-22 % savings. This may simplifies transition as webmasters do not need to keep two copies of encoded images. Full JPEG XL encoding can be done with speeds similar to MozJpeg, whereas observed AVIF encoding speeds are slower and some practitioners (1, 2) find AVIF's encoding speed problematic.

JPEG XL provides saliency-ordered updating. After delivering the first scan of 8x8 subsampled rendering, it can give the rest of the image in an encode-time defined order. This means that a portrait can render eyes, nose, and mouth first, and then continue to other areas. When coupled with saliency-prediction ML models, it may be a powerful further improvement in user experience.

I observe JPEG XL's format decisions to be more geared towards high quality: more context modeling and less prediction, filtering to reuse information from adaptive quantization so that appropriate fine details and low contrast textures where sent are not impacted through filtering. Filtering strength control field in JPEG XL is at a relatively high 8x8 pixel resolution. Deblocking filtering "Gaborish" in JPEG XL runs the opposite filtering at encoding time, making it transparent: image features in the original that happen to look like blocking artefacts will not be impacted.

I am not aware of any published usage statistics of AVIF. I believe we are still in a time window where the shift to JPEG XL would cause very little if any confusion and doubled effort in the field. I observe two of the early AVIF supporters (Facebook and Cloudflare, in this thread) shifting their interest from AVIF towards JPEG XL.

I observe JPEG XL to have the lowest header overhead of modern image formats. My profile pic is a 28 byte JPEG XL image. Avoiding the header overhead has resulted in strange systems where the browser uses JavaScript to patch together an existing header with the image body, or spriteing together GUI elements, i.e., adding system complexity on another level.

I observe JPEG XL to have some more tendency to become artefacty before it comes forgetful, and for AVIF to become forgetful first and artefacty later. This may lead into user confusion if cracks and bolts start disappearing rather than user being otherwise informed about bad quality of image transport. For an example of bolts disappearing, see the thick red pole in the center of the image having four bolts in the original: AVIF and JPEG XL images from WebM/WebP team's codec comparison images. Mozjpeg from the older comparison by eclipseo preserves the bolts, but blurs them to some extent.

I consider the browsers are leading this and similar changes, and operating systems such as iOS and Microsoft Windows will come later. The same happened with Brotli and WOFF2, and will likely happen with AVIF and JPEG XL, too.

[jonsneyers]

Regarding the "works-throughout-the-workflow" aspect: Adobe just commented saying "JXL is an great advancement in raster imaging and we look forward to being able to offer support for content authoring and editing in JPEG-XL to our customers." (https://bugzilla.mozilla.org/show_bug.cgi?id=1539075#c19)

Regarding maintenance / code size / attack surface / binary size, I want to point out that potentially a JPEG XL decoder could also be used as a JPEG decoder (libjxl currently doesn't do that yet, but it does contain all the code needed to do that), which means the dependency that browsers have on libjpeg-turbo could potentially be dropped.
If the numbers from android chrome are an indication (see https://chromium-review.googlesource.com/c/chromium/src/+/2932498): libjpeg_turbo weighs in at 151 KiB, libjxl currently weighs in at 191 KiB, but plausibly libjpeg-turbo could be dropped and the jxl decoder could be used instead also for jpeg decoding, which brings the net binary size overhead down to ~40 KiB. So it does come at a cost in maintenance / code size / attack surface / binary size, but arguably the net cost is not huge, e.g. compared to libwebp (165 KiB), and probably quite comparable to the net cost of AVIF (i.e. implementing the HEIF functionality on top of an AV1 decoder that is and presumably will remain 'free' in the sense of 'available anyway').

[jonsneyers]

Nothing has happened in this discussion for two months. Maybe it is time to revive it? If I understand correctly, firefox nightly now has correct color management for jxl, and support for animated jxl and progressive loading of jxl is on its way.

In terms of compression performance, what I've seen in subjective and objective evaluations so far, is that jxl is a generation ahead of avif in the "q60+" quality range, in fact jxl gives a bigger improvement over avif than the improvement avif gives over (moz)jpeg or webp, at least for photographic images (and I consider q60+ photographic images by far the most important category of images on the web). And that's without factoring in encode times, where cjxl remains roughly an order of magnitude faster than avif encoders.

[Pentaphon]

I keep seeing people discussing AVIF but JPEG-XL is the most promising format of the bunch. JPEG-XL should be top-priority right now over AVIF instead of what seems to be the other way around.

[gcp]

I keep seeing people discussing AVIF but JPEG-XL is the most promising format of the bunch. JPEG-XL should be top-priority right now over AVIF instead of what seems to be the other way around.

The reason for this is already explained here: #522 (comment) We need to support AV1 anyway, and AVIF is (comparatively!) a lot less code on top of that. It went out in Firefox 93 - and it's in other browsers - so that particular ship has sailed.

Hence some focus in the discussion as to what JPEG XL offers over AVIF.

[Yay295]

The comment you are replying to was made before Firefox 93 was released.

Regardless, that comment would seem to be even more relevant now, not less. If AVIF has already shipped, then all of the focus should be on JPEG XL now, right? What more discussion do you think is necessary?

[Pentaphon]

What more discussion do you think is necessary?

I think the issue is that there's too many competing image formats right now and everybody is trying to get their favorite one into Firefox. This is probably making the project maintainers nervous about adding too many image libraries to Firefox, bloating the project. Think about it. After AVIF, people are going to talk about JPEG-XL and then WEBP2. After that, there could be yet another format people will demand be supported. I think everybody needs to slow down, look at which image format is actually objectively the best and focus on that one. Right now, I think JPEG-XL is the clear leader in quality and efficiency.

Eventually, we will need to start deprecating image formats as their use becomes more and more limited. I personally do not see any future for HEIC or WEBP on the web. I even think AVIF was riding solely on the hype surrounding the AV1 video format and it being available for use so soon created a rush to adopt it too quickly before anybody could see if AVIF was worth adopting. I predict that people will see that AVIF was simply an overhyped format and it will be forgotten.

We need to focus all our support behind JPEG-XL as a long term solution to replace most existing image formats. Once people start widely using JPEG-XL and see how efficient and versatile it is, it will become clear that the other competing formats are simply not necessary.

[ekr]

Hi folks,

I'd like to remind people that the purpose of standards-positions is to assess whether Mozilla thinks that a proposed specification is a good addition to the Web, not principally whether it should be implemented in Firefox--though of course all other things being equal we would prefer to implement things that we think are good and not implement that we think are not good.

Given that AVIF has now shipped fairly widely, including in Firefox, the question at hand is whether the Web would benefit by also shipping JPEG-XL. Given that a wide proliferation of image formats is generally not a great thing, the test for that is not merely is JPEG-XL better but rather whether it is enough better -- in ways that can't be addressed by tweaks to AVIF -- that it justifies having yet another image format at this time.

[Pentaphon]

Given that AVIF has now shipped fairly widely, including in Firefox, the question at hand is whether the Web would benefit by also shipping JPEG-XL. Given that a wide proliferation of image formats is generally not a great thing, the test for that is not merely is JPEG-XL better but rather whether it is enough better -- in ways that can't be addressed by tweaks to AVIF -- that it justifies having yet another image format at this time.

See, this is what I was talking about before. People are concerned that there's too many image formats, and now they are concerned that too many image formats are being supported by Firefox. I don't get why people rushed shipping AVIF support when there's:

1. Data that clearly shows that JPEG-XL is superior to AVIF and can replace legacy formats like JPEG, PNG and GIF.
2. No pressing need to support AVIF at this time as almost every website has yet to adopt it.

Instead of rushing AVIF support and now questioning whether it's worth having the superior format be added to Firefox, we should have prioritized supporting the obviously superior format instead of rushing the support of an overhyped format like AVIF.

Is this going to happen all over again when Google introduces their WebP2 format? The point of formats like JPEG-XL is to completely replace existing formats and to simplify things for end users, not to complicate things for end users by adopting the latest fad image format as soon as it becomes stable.

Is it so hard to say no to a new image format? Or at least say "let's wait and see what the data says" before we cram yet another image library into our browsers? We're only having this issue because we're allowing it to happen.

[jonsneyers]

Given that AVIF has now shipped fairly widely, including in Firefox, the question at hand is whether the Web would benefit by also shipping JPEG-XL. Given that a wide proliferation of image formats is generally not a great thing, the test for that is not merely is JPEG-XL better but rather whether it is enough better -- in ways that can't be addressed by tweaks to AVIF -- that it justifies having yet another image format at this time.

How much is "enough better"? Is this strictly a matter of compression performance, or also about format features? Who defines this, and what is the methodology used to assess it?

Where can I find the data demonstrating that WebP is "enough better" than JPEG, and that AVIF is "enough better" than WebP?

Also: what is the argumentation for saying that "a wide proliferation of image formats is generally not a great thing"? Browser binary size and security surface? Something else? Would it really be a big problem if all browsers would support both AVIF and JPEG XL?

[kornelski]

WebP wasn't better enough (Mozilla chose to go for MozJPEG instead), which is why non-Blink engines ignored it for as long as they could, until Chrome-only websites serving only WebP became a web-compat problem.

What I'm wondering about is: Is it possible to retire AVIF? Could vendors that shipped AVIF commit to un-shipping it at some later date? (e.g. when JPEG XL becomes an option in browsers, or when AV2 ships making AV1-based AVIF obsolete)

Maybe even "grease" it by not always supporting AVIF, so that websites are forced to always perform content-negotiation and have a non-AVIF fallback?

[Pentaphon]

Is it possible to retire AVIF?

Sure. I don't really see it taking off, especially once JPEG-XL comes out except in the small possibility that it could become popular for web video snippets the way vp8 webm has found significant use by gif sites seeking to reduce gif size with web video.

Generally speaking, the web needs to go in the direction of having less formats instead of introducing more formats.

The same way AV1, for example, was agreed upon by the AOM due to financial pressure, major companies need to agree upon a single format that benefits everybody rather than 1 company.

WebP and WebM, as you rightfully pointed out, were pushed on the rest of us by Google seeking to leverage their browser and thus tighten their stranglehold over the web. The same thing will happen with WebP2 and JPEG-XL will be the best way to resist that.

[jonsneyers]

Forcing content negotiation seems like a really good idea to make it possible to retire formats once they become obsolete. With JPEG, PNG and GIF it will not ever be possible to really retire them (too much would break), but maybe for WebP and AVIF it is not too late to keep the option open of eventually retiring them — after all, at the moment content negotiation is still a necessity for them (even WebP is not really universally supported yet atm).

This is what the Accept header and the picture tag are made for.

If we want to avoid browser bloat, planning how to eventually retire old codecs is more important than blocking new codecs.

Also, accepting any codec/format that is accepted in a video tag also in an img tag would simplify things a lot - there would be no need to define or maintain special wrappers like avif or lossy webp, if you can just use any video container. It might be small compared to the decoder itself, but such containers also contribute to bloat and to potential for bugs (not to mention the potential patent issues with the heif container).

By the way, I don't consider AVIF really shipped in firefox yet at this point, since animation isn't supported yet, which imo is a big reason to use AVIF - it is obviously better than GIF (while it is not always better than JPEG or PNG). It is somewhat annoying that to understand what Accept: image/avif actually means, you also need to look at the user agent...

[Pentaphon]

If we want to avoid browser bloat, planning how to eventually retire old codecs is more important than blocking new codecs.

By saying no to new codecs that simply aren't the very best, you avoid the problem of having to retire old codecs altogether. The root cause of having too many formats and "browser bloat" is that the community just lets formats be thrown at the wall to see what "sticks" and everybody has to clean up the mess down the road.

The entire community needs to come together and demand more from new formats before they can be supported. If your new format isn't the best, it should be consolidated with another format, like how VP9/10, Thor and Daala became AV1 or FLIF and PIK became JPEG-XL, or simply scrapped altogether until a single format is shown to be clearly superior to the rest and the community pledges to stand behind that format for the long run.

Instead of allowing a "format war" there should be a "format contest" and the winner gets to be enjoyed by the web for years to come until the standard has served its purpose and can no longer meet the needs of the current web, at which point a new "format contest" decides upon a successor.

[ekr]

This discussion is rapidly veering off topic. As I said above, from we should assume that AVIF is going to be part of the Web ecosystem going forward. If you have new reasons why Mozilla should come down in favor of also adding JPEG-XL, please make them. However, this is not the place for a generic discussion of the state of image formats on the Web.

[jonsneyers]

Fair enough.

Well, I don't particularly have new reasons why Mozilla should come down in favor of adding JPEG XL, but I think the reasons mentioned above by experts from (amongst others) Facebook and Cloudflare should be convincing enough.

To summarize the main arguments in favor of adding JPEG XL:

• Compression performance: For lossy compression: at the fidelity targets most relevant to the web (q60-q90), jxl is 10-30% better than avif. For (non-photographic) images where lossless compression is more useful (e.g. web comics, icons, emojis, logos), jxl is 10-20% better than webp (while avif is worse than png for such images).
• Deployable right now: encode speed makes on-the-fly jxl encoding practical, while avif encoding necessitates offline encoding (which makes it non-deployable in several important use cases). Also, jxl has an encoder that produces perceptually consistent results (with relatively strong fidelity guarantees), while no such encoder currently exists for avif. This also makes a big operational difference for deployment (basically avif requires manual encoding / encode quality assurance if high fidelity is desired, while jxl encoding can be easily automated).
• Progressive decoding: for above-the-fold images, but also lazy-loaded below-the-fold images, progressive decoding results in a significant UX improvement. In this aspect, other modern codecs (webp, avif) are a regression compared to progressive jpeg, which means their compression improvements need to be weighed against the UX regression of having to wait longer for an image preview to be visible. In contrast, jxl extends the progressive features of jpeg, e.g. by allowing saliency progression, progressive DC, and more flexible progressive scans.
• JPEG recompression: jxl can losslessly recompress existing jpegs, which facilitates the transition of existing images (lossy transcoding is risky and can be ineffective), and also allows web servers to enjoy improved compression without increasing storage (since only the recompressed jxl needs to be stored; they can be converted back to fallback jpegs on demand).
• Ready for HDR: jxl performs better than any other codec on HDR images, and is future-proof in case higher fidelity, wider gamut and higher dynamic range will become desirable. By contrast, webp is limited to 8-bit tv-range YCbCr (so only 7-bit RGB), and while avif can do up to 12-bit in software, it is likely that for interoperability with hardware implementations, it will in practice be restricted to 10-bit 4:2:0 YCbCr (so 9-bit RGB), which could be a limitation.
• "Works throughout the workflow": jxl is designed to work well for end-user image delivery, but it is not specifically designed only for that (unlike e.g. webp). It also has many features intended for image authoring workflows, such as support for named layers, high bit depth, fast lossless modes, CMYK, extra channels that can e.g. store selection masks, etc.
• Generation loss: e.g. memes or other viral images can be shared many times between different platforms, often resulting in significant generation loss since the image gets recompressed many times. The design focus on high fidelity (as well as the JPEG recompression option) makes jxl have better behavior in this respect than other codecs.
• Minimal header overhead: many images on the web are quite small (e.g. icons or emojis). Every byte of header overhead counts when it gets multiplied by trillions of image transfers. No format is as concise as jxl: there are "jxl art" contests where the goal is to make an interesting jxl image in 32 bytes or less (that's header + payload), while the header overhead alone of jpeg and avif is in the order of 200-300 bytes.

[G2G2G2G]

@baumanj

But they do support AVIF if you have the official plugin installed: https://avif.io/blog/tutorials/use-avif-in-windows/#microsoftsupportsavif

it's more of a bug than a feature, it works with SOME avif images in their software (including edge) adding anything that isn't in the video codec (like alpha channels or animated AVIF) breaks it.

also support JPEG XL in firefox already ffs

[gnat]

JPEG XL is a genuine "one true format", and i'll be switching all assets across all of my sites to it the moment support is turned on in Firefox and Chrome. I won't even wait for Webkit to play catchup.

The problems around lossless AVIF, with no resolution on the horizon, makes AVIF a non-starter for many: AOMediaCodec/av1-avif#111

[G2G2G2G]

@gnat Yep, AVIF is a great replacement for... gif and all of the mp4's around the internet that pretend to be gif these days. Nothing else.. and definitely not a replacement for jpeg and png. JPEG XL is though.

[ekr]

I'm locking this conversation as it has passed the new information stage.

[martinthomson]

After a lot of consultation (and time, sorry), we've concluded that we are neutral on JPEG-XL.

There are two competing forces in play here that we need to acknowledge explicitly. Adding new formats comes with a cost, not just to us (adding, securing, and maintaining code is non-trivial), but to the Web as a whole. On the whole, having fewer formats is better for the Web as that limits the complexity of authoring and serving content. We support multiple formats only to the extent that those formats address the needs of people and sites.

We might assess a format based on features that it provides and the overall performance of the format, which includes a range of factors including compression ratios, CPU cost, and image quality. We might also look at how widely a format is used; features and performance advantages matter less for widely used formats. New formats need to justify their inclusion on the basis that they provide some material advantage over what exists.

On the other hand, the Web has a ton of raster image formats already with many of the same problems. Adding yet another format doesn't make the Web significantly worse for it. We acknowledge that JPEG-XL offers some potential advantages, both in terms of features and performance.

Overall, we don't see JPEG-XL performing enough better than its closest competitors (like AVIF) to justify addition on that basis alone. Similarly, its feature advancements don't distinguish it above the collection of formats that are already included in the platform.

So we don't see support for JPEG-XL as either good or bad for the Web. We might find it necessary to support the format if usage becomes more widespread, but that will be a product decision¹.

Footnotes

1. Some reminders regarding the product planning point: Our positions on standards do influence product planning for Firefox, but they are not the only thing we consider when planning what features to build and ship. We also request that people seeking to provide updates on changes in market conditions refrain from doing so. We can re-open a position when there is new information, but updates on market conditions will not typically cause us to change our position here. ↩