RAW, lossless, lossy and lossily optimised modes supporting multiple image, video and audio codecs

[EmanuelLoos]

It would be great if LibreCamera could have RAW, lossless and lossy modes with multiple formats to choose from.

Some useful additional image formats could be:

• RAW (RAW data as it comes)
• DNG (RAW data in a standardised format)
• QOI (lossless)
• PNG (lossless or lossily optimised)
• WebP (lossy, lossless or lossily optimised)
  based on the free video codec VP8
• AVIF (lossy, lossless or lossily optimised)
  based on the free video codec AV1
• JXL (lossy, lossless or lossily optimised)
  JPEG XL - successor of JPEG
• (FLIF)
• (FUIF)
• (Pik)

Proprietary: HEIC

As for free video codecs:

• VP8 (lossy, lossless or lossily optimised)
• VP9 (lossy, lossless or lossily optimised)
• AV1 (lossy, lossless or lossily optimised)
• HuffYUV (lossless or lossily optimised; simple, fast to encode but but takes up much space)
• FFV1 (lossless or lossily optimised; probably too slow for real time encoding, efficient)
• (Daala)
• (Thor)

Proprietary: h264 (currently used), h265

And free audio codecs:

• Opus
• Vorbis
• FLAC (lossless or lossily optimised)
  Free Lossless Audio Codec
• MP3 (patents expired in 2017)

Proprietary: AAC (currently used)

Containers:

• MP4
• MKV
• WebM

Instead of a "this or that" choice, it might also useful to have the possibility to encode in multiple codecs of the user's choice like RAW+PNG, JPEG+JXL as well as for videos MKV{AV1+MJPEG+Opus+MP3}. Ideally the user could add self-defined combinations of codecs with per codec settings like compression levels etc.

Some of these formats are useful for later editing, some offer smaller file size at better quality for the web. Many of the listed formats are natively supported by Android, some would need to be converted by the app from lossless ones.

In order for the app to remain small, some formats could be installable as extensions in the form of separate apps.

As the compromise is always between file size, quality and compression time, adding a triangle would suit this notion better than the quality bar. could allow for more choice, it could be some kind of advanced option, maybe in some kind of advanced mode.

[iakdis]

Hey Emanuel,

thanks for the ideas. I will add the ability to choose between JPEG/JPG (default), PNG and WebP in the coming version. As for your other ideas:

1. Multiple outputs for the same image: I could add that in the coming versions
2. RAW, lossless and lossy modes with multiple formats to choose from: This would be great, however I am quite limited by the abilities of the Flutter camera. I could imagine to add the RAW format, once it gets implemented (see the Flutter Issue)
3. A triangle would suit this notion better than the quality bar: Because there are not many "big" formats currently supported I will probably leave it as it is right now, as it seems to be simpler for the end user who chooses between quality and file size

Hope you are okay with this. I will leave the issue open as ideas for the future.

[nekohayo]

We can simplify this list quite a bit:

• There is no reason to implement FLIF support anymore because it has been replaced by FUIF and then JPEG XL (all three are from the same main authors).
• Likewise, PNG is as good as useless when JPEG XL can do lossless 30 to 40% more efficiently.
• WebP destroys image quality (doesn't even support proper color spaces etc.) and is marginally better than traditional JPEG in the form of MozJPEG, and is again made obsolete by JPEG XL (or, if you really must, AVIF).

It is no surprise that in #45 I am in favor of JPEG XL as the prime format for camera apps. It just beats everything else out there, probably the only thing it doesn't replace is raw formats (but even then, I'm not entirely sure).

[starsforeveryone]

I would very much like to see the new open-source and royalty-free codecs AV1 (for video) and AVIF (for pictures) supported!

They will most likely become the new industry standard, with hardware acceleration present in modern hardware already.

Would love to see it supported so I can save loads of storage space on my phone!

I've just asked the guys on Flutter to please add support for it.
(Link to issue there: flutter/flutter#61229 (comment))

Would you be so kind as to implement it, too, then?

[iakdis]

@starsforeveryone Sure, I would

[EmanuelLoos]

@nekohayo: I tend to disagree. It is not pointless to also support (some) older standards. To my knowledge AOSP does not even support JPEG XL on the system level yet. Including older codecs as well provides backwards compatibility while keeping the quality as good and the file size as small as possible which makes for a better user experience. PNG is not useless as long as long as quite some software supports PNG and not JPEG XL. LibreOffice for example just added WebP support, no AVIF, no JPEG XL. It does not even allow you to save documents with WebP images yet but converts all embedded WebPs to PNGs upon saving.

[emmatebibyte]

It would be cool to have QOI support, if that were possible.

[G-i-o]

Very much second this comment about AV1 and AVIF formats for video and camera.

Currently, just a few mins of good quality video or a good quality pic occupy a relatively big amount of data space but with the above formats I notice reductions up to 10x without visible quality loss to the layperson.

Looking forward to this.

Otherwise, great work. Keep it up! 👍

[Pantyhose-X]

And free audio codecs:

• WavPack