AVIF codec support

[LasseRosenow]

Use case

AVIF aims to be the successor of jpg on the web and is growing very fast. AVIF images also compress with much better results then jpg. So it would be very good for flutter applications to support AVIF.

Proposal

Add AVIF codec support to flutter.

[maks]

I'm guessing this will need support in the engine from Skia? If so I think the issue tracking it in chromium seems to indicate that is now at least initially available as support is shipping in Chrome M85.
Also I wonder if the work the @dnfield did for HEIC support be relevant to AVIF support in Flutter too?

[dnfield]

Right now, we're delegating to native platforms (except on Linux and Android) to decode formats we don't recognize - but there's no support for that (yet) for animated images.

[dnfield]

(so you would just get the first frame if the platform can decode at least that much)

[maks]

@dnfield great thanks!
Would you have a pointer to how to handle this on Linux? I'd like to experiment with using AVIF images in a linux desktop person project app - would just using FFI to bind to libavif be the way to go there?

[chinmaygarde]

cc @cbracken

[jamesblasco]

I think it would be a great time to reconsider this as it was already implemented on Skia on April 2021

[jimmyff]

Could we get an update on the status of this please?

[cpboyd]

Even though we may defer to native decoding, iOS 16 currently has some rendering issues with RGB AVIF files: https://www.reddit.com/r/AV1/comments/xd4v8w/avif_in_safari_on_ios_16_vs_google_chrome_on/

Not to mention, Android 12 only constitutes 13.5% of devices according to Android Studio (and I'm not entirely certain that AVIF support isn't dependent on device manufacturer configuration):

If we're stuck with native decoding options, then it'll be years until we can use AVIF on Android without abandoning a significant chunk of users.

[dnfield]

Skia supports this if you ship a codec that can decode it.

Shipping a codec adds non-trivial cost for all customers of Flutter.

I don't have a good sense for how popular AVIF actually is. The strategy Flutter has had for a while at this point is just let the native platforms add support, and if they're not we're not.

I'm not opposed to AVIF in principle (I don't really know enough about it), but I think the following would help to get consensus on whether Flutter should support this specially:

• How popular is AVIF, compared to other formats that Flutter already supports?
• How much binary size gets added by adding support for AVIF?
• Is there a codec implementation that is both high quality and has a suitable license (e.g. BSD, MIT, Apache2) available for usage in Flutter?

[dnfield]

How many browsers support it doesn't really give us a good sense of how popular the format is.

I'm also not clear on how much better it'd be to use AVIF images than say webp images in the same context in Flutter. AVIF appears to support HDR/10bpp images, but Flutter doesn't.

[dnfield]

Looks like Chromium and Skia use libavif, which appears to be AOM's reference implementation (though I might be missing something, haven't looked at this all the way through).

It probably wouldn't be too hard to add an appropriate hash of libavif to the engine's DEPS file, create a build rule (probably based on Chromium's), add the gn argument for skia_use_libavif to let skia know to use it, and do a build to see what the difference is.

It would still help to know either how many customers are really hurting for this (as opposed to one ofthe other formats we already support). HEIC images, for example, are supported from Android P onward from what I recall.

[cpboyd]

How many browsers support it doesn't really give us a good sense of how popular the format is.

It's rapidly being supported by various platforms because of that wide support (since over 71% of users can render it).

If someone's already generating AVIF files to serve to web users, then it'd be nice to also allow iOS and Android users to benefit from the same file size reductions.

Speaking of which, AVIF has a decent improvement over WebP (I believe that's effectively 30% smaller): https://www.ctrl.blog/entry/webp-avif-comparison.html

If you have an app that serves a lot of images, the benefits of AVIF could be significant at scale.

Edit:
Regarding HEIC, there's no browser support: https://caniuse.com/heif
The key advantage of AVIF is that a lot of static site generators and content management systems are starting to support it.

[jimmyff]

Not only are compressed file sizes significantly smaller but the compressed image sharpness is far superior than that of webp (which is much softer).

I'm migrating a large app to GCP + Flutter (~35m JPGs). At the moment I'm using webp but it would be a huge benefit to me to be able to use avif in terms of storage & bandwidth costs, and also for my users which would experience faster & sharper images.

[starsforeveryone]

I would also like to see AVIF supported. Since the AV1 video format is increasing hugely in popularity, it makes only sense to add support for the corresponding AV1 image format (AVIF) better sooner than later.

There is already significant hardware support, with hardware decoders being already present in most devices, hardware encoders coming with the next generation of cpus/gpus that's hitting the market right now (Intel ARC, AMD RDNA3, NVIDIA's latest cards), and webbrowser support enabled (e.g. in latest Firefox version). So the vast majority of end users can use hardware accelerated decoding as of now, and content creators will have access to hardware encoders very soon (read: this christmas).

These new codecs are open, royalty-free and their compression capability is far superior to anything the old JPEG and H.264 (MP4) image and video codecs could do. And let's face it, while nobody wants to remove support for those, with content resolution increasing rapidly (e.g. 4K, 8K videos and higher megapixel cameras) the demands on storage are getting so high that the need for new, open, more efficient codecs is really there. They save bandwith and storage, and therefore money.

Don't want to annoy anyone here, but if somebody could take some time off to implement AV1 & AVIF support in Flutter, this would be greatly appreciated! Thank you very much.

[novomesk]

People are encountering AVIF images more often. I see complains on Twitter that they downloaded an AVIF file from some website and that all their programs are not working with AVIF yet.

Supporting AVIF is not difficult.

In GIMP we use libheif (it has AVIF support too beside HEIC), in KDE and Qt applications (like Telegram Desktop on Windows/Linux) we use libavif.

libavif is now maintained by people who work on libaom AV1 decoder/encoder and who integrated it into Google Chrome browser. libavif and libaom can be easily cross-compiled for armeabi-v7a and they work on Android (tried it myself).

libavif & libaom are fuzzed on oss-fuzz, the code is reliable and well tested.

If you need to know something about AVIF and/or the mentioned libraries, don't hesitate to ask me.

[slightfoot]

@dnfield I have been looking forward to the day that Flutter supports AVIF since I read this article in 2020. I would suggest reading it if you want to get up to speed. But there are newer articles since then. https://www.smashingmagazine.com/2021/09/modern-image-formats-avif-webp/

To answer your previous points.

• How popular is AVIF, compared to other formats that Flutter already supports?

Gaining traction every day since 2020. Now all the major browser vendors ship the decoding library. I can post an AVIF image to this post and chrome will decode it just fine. However Flutter will not. Flutter lead the charge on being at the forefront of mobile and I think its is a case where it should do the same.

• How much binary size gets added by adding support for AVIF?

It would be great if you can build the latest engine with skia avif flag enabled and let us know. However, the increase in size would be insignificant compared to the saving we would get if we could use AVIF in our applications. Apps are getting larger in size and part of this is the assets. We want to ship high quality graphics for multiple pixel densities but this takes up space. AVIF can actually help reduce the overall application deployment sizes.

• Is there a codec implementation that is both high quality and has a suitable license (e.g. BSD, MIT, Apache2) available for usage in Flutter?

As it ships as part of the chromium project (libavif) I assume it has the appropriate licenses. This is: 2-Clause BSD

https://chromium.googlesource.com/chromium/src/+/75fd5322c0a1ce80d0ea2384b570a93c3f4dae1c/third_party/libavif/LICENSE

@kevmoo Since the image decoders will fall back to the underlying platform, I assume this means that Flutter on Web supports AVIF format. And really it makes sense for the other platforms to now adopt this format to better get cross platform support.

For example; I don't really want to ship the same graphic in multiple formats to support web and desktop. Which means we end up supports the lowest common denom aka JPEG instead of the better sized AVIF.

[kennylevinsen]

It probably wouldn't be too hard to add an appropriate hash of libavif to the engine's DEPS file, create a build rule (probably based on Chromium's), add the gn argument for skia_use_libavif to let skia know to use it, and do a build to see what the difference is.

Hmm, it seemed like a small task even for an outsider, but the interactions between gclient and gn seem a little opaque to those not familiar with chromium tooling.

One would have expected the DEPS of skia on libavif as well as skia's third_party/libavif/BUILD.gn file to be sufficient when the skia_use_libavif switch was set in tools/gn, but that does not appear to be the case, nor does adding libavif as a flutter dependency help as it then looks for a BUILD.gn file:

Generating GN files in: out/host_debug_unopt
ERROR at //third_party/skia/BUILD.gn:1127:12: Can't load input file.
  deps = [ "//third_party/libavif" ]
           ^----------------------
Unable to load:
  .../src/third_party/libavif/BUILD.gn
  I also checked in the secondary tree for:
  .../src/build/secondary/third_party/libavif/BUILD.gn

Is flutter supposed to duplicate skia's third_party BUILD.gn files or is something broken?

[slightfoot]

I just found out that Android 12 ships with AVIF support. Can the flutter-engine take advantage of this?
Ref: https://developer.android.com/about/versions/12/features#avif

I can see AVIF being popular with Flutter developers, if the engine supported it, the community could put together supporting tooling.

@tvolkert I see this is a P3, but it looks like its a simple addition and would have a lot of gain. Any chances on understanding size cost to the engine for including it? (Not sure who heads up the engine team, if that's still you or not)

[dnfield]

@slightfoot If Android 12 supports it, thenit should "just work" on Android 12 devices and up. We fall back to platform based decoding for formats we don't ship support for natively.

@kennylevinsen - You need to have libavif in the //third_party directory, probably with some kind of DEPS file entry pointing to it, similar to Skia's here: https://github.com/google/skia/blob/50814d9ca5bb4f2b14d0eb53bcb4f4dca88ebb04/DEPS#L40

Just note that Skia uses //third_party/external for, this, but the engine does not use external typically.

[slightfoot]

@dnfield created a test project and have AVIF working on Android. The issue still remains if you wanted to use AVIF it wouldn't be just for one platform.

[yekeskin]

If anyone wants to use Avif until Flutter supports it, I have created a package you can try. https://github.com/yekeskin/flutter_avif

[SAGARSURI]

This is so critical for my current use case. I have to convince my client that they go with webp because Flutter doesn't support this format right now.