Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cgifsave: avoid size issue #2628

Merged
merged 15 commits into from
Feb 28, 2022
Merged

cgifsave: avoid size issue #2628

merged 15 commits into from
Feb 28, 2022

Conversation

dloebl
Copy link
Contributor

@dloebl dloebl commented Jan 19, 2022

Avoid size issues when resizing GIF animations with a static background. The trick is to restore the original transparency setting after quantization and dithering.
Background: Keeping many pixels at the transparency index can reduce the size a lot and is the reason why the original animations in (#2576) are quite small in terms of bytes despite their huge width/height. The current resizing and subsequent color quantization method puts the focus on image quality but can make this transparency optimization less effective.

The suggested fix depends on dloebl/cgif#38 and cgif 0.2.0 to be released:

Results:
$ vipsthumbnail --size 200 city.gif[n=-1] -o vips_city.gif
$ vipsthumbnail --size 1400 watch.gif[n=-1] -o vips_watch.gif
$ du -h vips_*
2.2M vips_city.gif
812K vips_watch.gif

The size is still larger than before, as the transparency setting is changed by the resizing. However, the two animations are probably extreme cases and the size problem is mitigated quite a bit with this change.
I've visualized the transparency setting for the resized city GIF and the original (green means that a pixel is identical to the frame before):
Resized:
out
Original:
sorg

One can see some "kernels" (in the resized image). I guess these come from the interpolation method and illustrate where the remaining increase in size comes from. To further improve the size it could be interesting to have a "transparency-friendly" resize method (which keeps identical areas identical). For instance, nearest-neighbor resizing would be "transparency-friendly" but is of course not ideal in terms of quality. An alternative would be to determine transparent areas in a lossy way in cgifsave (I think @jcupitt once mentioned such a low-pass filter). So the problem is finding the right balance between quality and size.

Please let me know if there are changes required on this PR.

avoid size issue by restoring the original transparency setting after quantization/dithering
@dloebl
Copy link
Contributor Author

dloebl commented Feb 9, 2022

Further improvements / additional notes:
Right now, a simple Euclidean distance in the color space is used for the lossy transparency optimization. Say [dr,dg,db] is the color difference between a pixel and the same pixel in the previous frame. If sqrt(dr^2 + dg^2 + db^2) <= maxerror is fulfilled, the pixel is set transparent. One might run into certain ghosting artifacts (examples below):

Command used: vips copy <input.gif>[n=-1] <output.gif>[maxerror=16]

Original Lossy Transparency (maxerror=16)
cc450056a8757c76c7e0df10704d4b39 cc
3419f140bf10126c059cb71d7064c9b9 34

I thought about taking the human perception of colors into account e.g. by weighting the red/green/blue channels differently or getting luminance into play. Such a modification should reduce the visibility of possible artifacts. But maybe that is something more for a follow-up PR, as the quality seems to be quite good already (2576#comment).
@jcupitt: Do you think such a modification would make sense? How would you weight the different color channels?

@jcupitt
Copy link
Member

jcupitt commented Feb 9, 2022

@jcupitt: Do you think such a modification would make sense? How would you weight the different color channels?

Yes, that's a good idea. CIELAB is the simplest perceptually uniform colour space (mostly uniform anyway!) so I would use that.

https://en.wikipedia.org/wiki/CIELAB_color_space#From_CIEXYZ_to_CIELAB

sRGB -> XYZ is this:

https://en.wikipedia.org/wiki/SRGB#From_CIE_XYZ_to_sRGB

You only need 8 bit data and you don't care about small errors, so you can cut a lot of corners. You'll find you end up with something like:

  • Look up table for each of RGB to get linear data
  • 3x3 matmul (so R' = a * R + b * G + c * B etc.)
  • Look up table for cuberoot term in LAB
  • L = 116 * G - 16, a = 500 * (R - G), b = 200 * (G - B)

Not too painful.

libvips has a fairly fast srgb -> lab path, but it'd probably be simpler to code your own. You could use the libvips one to check your implementation, of course.

@dloebl dloebl marked this pull request as ready for review February 24, 2022 19:34
@dloebl
Copy link
Contributor Author

dloebl commented Feb 25, 2022

Thanks @jcupitt for this very detailed explanation!
From my side, this pull request is now ready for review. cgif v0.2.0 has been released.
As the quality vs. size tradeoff with the lossy transparency option is quite good already (2576#comment), I would like to do the improvement that you suggest as a follow-up pull request soon.

Copy link
Member

@jcupitt jcupitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The C programmer in me couldn't resist making some unnecessary optimisation, of course :( gif write is about 10% faster on my PC with the suggested changes. Maybe not a useful amount.

I noticed some small layout and naming things too.

libvips/foreign/cgifsave.c Outdated Show resolved Hide resolved
libvips/foreign/cgifsave.c Show resolved Hide resolved
libvips/foreign/cgifsave.c Outdated Show resolved Hide resolved
libvips/foreign/cgifsave.c Show resolved Hide resolved
@dloebl
Copy link
Contributor Author

dloebl commented Feb 28, 2022

Makes all perfect sense. Thanks!
I added your recommendations accordingly.
Additionally, I switched the lossy transparency off by default (maxerror = 0).
I think it makes sense to be a bit conservative before the pixel compare function has been improved (RGB to CIELAB).

@dloebl
Copy link
Contributor Author

dloebl commented Feb 28, 2022

The PR depends on cgif v0.2.0.
Seems like lovell/cgif-packaging / ppa:lovell/cgif is not yet up-to-date.
Let me have a look..

@jcupitt jcupitt merged commit b0f993e into libvips:master Feb 28, 2022
@jcupitt
Copy link
Member

jcupitt commented Feb 28, 2022

Ok, let's merge! Nice work @dloebl

jcupitt added a commit that referenced this pull request Feb 28, 2022
also note in changelog and revise layout for 80 columns

see #2628
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants