Consider adding support for mango (jpeg/png)

[gunta]

Why:

Mango JPEG decoder is faster than libjpeg-turbo.
This means that we can have a 2-7x improved latency according to these benchmarks.

This would provide huge benefits for Image as a Service use cases where throughput matters but latency also does.

[jcupitt]

Hello @gunta,

That looks very nice, I'd not seen mango before. The benchmarks are impressive.

Let's revisit this when it's been packaged.

[gunta]

Looks its performance for PNG also is even better than libspng and stb.

[jcupitt]

Perhaps, though I'm puzzled by those numbers, libspng should be much quicker than libpng. I wonder if there's a build problem.

There was a build problem; the default build is for DEBUG so the BREW version was debug.. :D

Re: png, did some optimizations this week so it's ~30% faster. Color conversion still needs some tweaking, it's straight C++ code (no SIMD intrinsics) because it's performance effect is relatively small. The changes were: 1. fixed some filtering code to generate better code (especially with gcc). 2. use libdeflate, which is really fast (!) and 3. organized code so that it does more work with same buffers immediately - less passes over the data. A bit more work ahead and hopefully the working data remains in L1 until it is flushed out (that's the goal anyway). Cheers!

[gemini133]

I added vips test based on the mango's jpeg_benchmark.cpp .

here is the result with 19M jpg on i7-9700 with HT on

./test-mango jpg.jpg 
----------------------------------------------
                load         save             
----------------------------------------------
mango:       240.6 ms     195.0 ms 
vips:         1.1 ms     749.3 ms 

time vips copy jpg.jpg copy.jpg[strip=true,Q=100]

real	0m0.729s
user	0m0.964s
sys	0m0.037s

vips was bound with libjpeg-turbo and I enable avx512 for mango.

vips-8.10.4-Tue Dec 15 07:56:59 UTC 2020

ldd $(which vips) | grep jpeg
libjpeg.so.62 => /lib/x86_64-linux-gnu/libjpeg.so.62 (0x00007f4906d6a000)
ls -la /lib/x86_64-linux-gnu/libjpeg.so.62
/lib/x86_64-linux-gnu/libjpeg.so.62 -> /opt/libjpeg-turbo/lib64/libjpeg.so.62

the generated files:
32M t-mango.jpg
49M t-vips.jpg

Here is the code and please check if it's written properly. @jcupitt

Details

/*
    MANGO Multimedia Development Platform
    Copyright (C) 2012-2020 Twilight Finland 3D Oy Ltd. All rights reserved.
*/
#include <mango/mango.hpp>
#include <vips/vips8>

using namespace vips;
using namespace mango;
using namespace mango::filesystem;

// ----------------------------------------------------------------------
// print
// ----------------------------------------------------------------------

void print(const char* name, u64 time0, u64 time1, u64 time2)
{
    printf("%s", name);
    printf("%7d.%d ms ", int((time1 - time0) / 1000), int((time1 - time0) % 10));
    printf("%7d.%d ms ", int((time2 - time1) / 1000), int((time2 - time1) % 10));
    printf("\n");
}

// ----------------------------------------------------------------------
// main()
// ----------------------------------------------------------------------

int main(int argc, const char* argv[])
{
    if (argc < 2)
    {
        printf("Too few arguments. usage: <filename.jpg>\n");
        exit(1);
    }

    if (VIPS_INIT (argv[0]))
      vips_error_exit (NULL);

    printf("----------------------------------------------\n");
    printf("                load         save             \n");
    printf("----------------------------------------------\n");

    u64 time0;
    u64 time1;
    u64 time2;
    // ------------------------------------------------------------------

    time0 = Time::us();

    Bitmap bitmap(argv[1]);

    time1 = Time::us();

    ImageEncodeOptions options;
    options.quality = 1.00f;

    bitmap.save("t-mango.jpg", options);

    time2 = Time::us();
    print("mango:   ", time0, time1, time2);

    // ------------------------------------------------------------------

    time0 = Time::us();

    VImage in = VImage::new_from_file (argv[1], VImage::option()->set ("access", VIPS_ACCESS_SEQUENTIAL));

    time1 = Time::us();

    in.write_to_file ("t-vips.jpg[strip=true,Q=100]");

    time2 = Time::us();

    print("vips:   ", time0, time1, time2);

    vips_shutdown ();

}

[gemini133]

here is the meson.build just for your convenience

project(
    'test mango', 'cpp',
    default_options : [
        'c_std=c11',
        'cpp_std=c++14',
        'default_library=static'
    ]
)
cpp = meson.get_compiler('cpp')
mango_deps = cpp.find_library('mango', required: true)
pthread = cpp.find_library('pthread', required: true)
vips = dependency('vips-cpp')
executable('test-mango', 'test.cpp', dependencies : [mango_deps, pthread, vips])

CC=gcc CXX=g++ meson build; ninja -C build; cd build; ./test-mango jpg.jpg

[jcupitt]

Yes, looks reasonable, though libvips disables chrominance subsampling for Q>=90, so you'd want to do the same for mango for this benchmark.

The encoder is always 4:4:4, not that much options for encoding. More paths can be added but I never needed anything more, and I am mostly writing this code for myself so it's not that easy to use. Gemini suggested versioning the library and I agree that should be done but this is "just" a hobby project and my main programming time goes into other things.

I looked at the vips jpeg decode timings and verified the results on AMD TR, these are with libjpeg-turbo, or some hw accelerated path? I tested libjpeg (-t) through GTK and get roughly same timings I get with calls to libjpeg directly. I noticed some GTK code there so figured maybe they were doing something (only tested 2.0, not 3.0). 240ms and 1.1 ms above, 200x faster, what's the trick? =D

I noticed a nice trick by ARM engineers to detect zeros in the DCT block to speed up the encoder, could do the same for x86.. but that's on encoder side. The approach was two parter: part one, de-zigzag into SIMD registers and then basically use compare to generate masks. Intel has lane-msb-to-mask instruction which should be useful for this.. but I am steering off-topic so no more about that..

TL;DR - 4:4:4 is always used by the encoder, it's very basic. It's not a big feat to add modes, just never needed any. libjpeg-turbo is really good library, I'd just keep using it. I wrote my code inspired by OpenCL feasibility study I did at ATi in 200? when compute support was being added to a GPU silicon we were designing. I used JPEG as one of the test cases so that's why I got into it. This was so one lifetime ago.. but wrote a CPU decoder from scratch for fun as an afterthought and here we are.

Now that you know what kind of crap the code is, you probably are better informed to stay out of it. :D

[jcupitt]

Hi @t0rakka, thanks for the information! Mango looks really nice, and the benchmarks are impressive.

Maybe try Q=89 for the benchmark then? It should give more comparable results. The large difference in output file size should vanish, at least.

libvips decode time is quick, but all it does is setup. Pixel decode happens at the same time as encode -- running the two together lets libvips overlap encode and decode for a 2x speedup.

Does the mango API support read and write of pixel buffers (as opposed to whole images)? libvips would need that to allow the overlapping. libvips usually reads and writes 8 or 16 scanlines at a time.

Has mango been fuzzed? libvips is part of oss-fuzz so an experimental read and write with mango might be a simple way to get mango into the fuzzing programme.

Yes, I have used zzuf for this purpose. A lot of checks in the chunk processing come from fuzzing and reading the specification with a magnifying glass. :D

The weakest part is the huffman decoder which is a bit speed-sensitive so using checks sparingly. It checks that it won't read out of bounds and writes are (mostly) guarded.

That said, there is still some work left in this area since when I run 1,000,000 iterations I get 20 or so failures. Not with all inputs but I have found some that still manage to fail.. but it is much more rare than before fuzzing that's for sure.

My main concern is non-sanitized random inputs, since I wrote a "coder's tool" that I use for checking compressed textures and other image formats. It's kind of neat that I can load one image as background, use other as a layer on top and move it around with mouse, fidding with blending modes and such. If I want to see alpha channel, I can and so on.. the key feature for me is that I can write plugins with GLSL for different parts of the rendering pipe so I can upgrade the tool for any new task that comes along. It's internal tool and so badly documented and crappy coding that I don't plan on releasing sources.. (before anyone asks).. but when some broken image is in a folder I am scanning, it sucks if the tool crashes so I sat down and enforced the code against broken inputs. Has worked fairly alright since then.. if you find any images that won't work, even if they SHOULDN'T, I can take a peek and try to guard. It's not a defence to just say; "this is not a valid jpeg file", since broken and corrupted files exist and we should at least be able to detect it in runtime (when possible).

So I did fuzz just for practical self-interest reasons; I don't like crashes even if they aren't "my fault" (since at end of the day, they are).

Not all formats are fuzzed, it would be a dream situation if fuzzing everything was part of regression tests and had proper CI, but the problem is time and how to best use it.. these are on the TODO list, but the list seems to be growing, so.. :P