Introduce high quality up-sizing/enlarging

[kowpa1990]

...in cjpeg as a option similar to -scale. In comparison to downsampling, upsampling needs more complexity in the calculations to the picture looked good. Why use a magnified images?

• people use super-resolution (SR) for print
• in web pages to avoid users scale from low-res interpolation (see also image-rendering, 'Gecko uses bilinear resampling'; they call it 'high quality' - there is a lot better scalers)
• for 'recover detail from all kinds of degraded sources'
• for pseudo-retina display on web pages (and when they do not have pictures straight from the source - camera - in hi-res)
• to zoom worms and insects
• etc.

NNEDI3 is the best interpolation algorithm available in Avisynth, and in my opinion lend itself for an extra feature for mozjpeg. In the future, when the DC Tune, aggressive optimizations find themselves in mozjpeg, it will be a great complement to the high quality resampler via Retina Revolution, and competitor for adaptive (edge-detecting) algorithm PhotoZoom.

Why use NNEDI3?
It is regarded by the community dedicated to converting video for the best interpolation algorithm but by the fossilization / orthodoxy / conservatism of the community is not available to a potential audience. Adding it to mozjpeg add freshness to the project.

Someone has already done a detailed testcase perfect NNEDI3.

In addition, a few links concerning interpolation.

cons NNEDI3:

• enlarging images only by powers of 2

I made a visual testcase which takes account of popular interpolation algorithms.
Screen
Inside there's more links regarding upscaling.

[kornelski]

The upscaling algorithm looks great, but does it require any integration with the encoder? Can clients apply it to any image/JPEG from any encoder?

[kowpa1990]

Yes, it require integration with the encoder.

Can clients apply it to any image/JPEG from any encoder?"

Yes; in this case, if cjpeg handle the encoded images from other encoders.
(cjpeg support PPM, PGM, BMP...)

In general terms I mean something like this:

1. I have a picture with dimensions of 100x100
2. cjpeg -nnedi 4 -quality 50 file.bmp > file_x4.jpg
3. I get a picture with dimensions of 400x400

(-nnedi 4 is just an example enlarging by 4.)

Each JPEG decoder should read it.

This project's goal is to reduce the size of JPEG files without reducing quality or compatibility with the vast majority of the world's deployed decoders.

djpeg supports upscaling but it is not so sophisticated as NNEDI3.

djpeg -scale 2/1 -pnm zelda.jpg > zelda.pnm

[dwbuiten]

Another useful link: https://github.com/dubhater/vapoursynth-nnedi3

NNEDI3 working port on Linux/Windows/Mac.

[dwbuiten]

I question the usefulness of scaling in cjpeg. Hardly anyone would use cjpeg as-is... with ImageMagick's convert being the most used, which has several different upscaling algorithms already.

[kowpa1990]

Since mozjpeg project focuses on the encoding of images, rather than decoding and even more on their basic photo-related editing your concerns seem justified. However, after the introduction of jpegin in cjpeg and after leaving support of trellis in the transcoder jpegtran we have to look at it from the other side. It seems logical that the main priority of the project was to reduce the file size when creating them, more than their recompression. Now when jpegtran is worthless - cjpeg plays a primary role and has full functionality in reducing the file size. Question: are any functionality of lossless transformations/rearrangements of the image data from jpegtran could find in cjpeg? Question for the new issue.

I have written to apply upscaling to cjpeg, although more reasonable will be join this to jpegtran - a tool designed for this type of operation.

Simple actions such as: cropping, reducing colors, rotating can be made by jpegtran (especially if it is a batch conversion), for the rest of the operations such as noise removal, color balance correction, people will use Photoshop. But none of the common graphics programs do not have an advanced upscaling algorithm.

GIMP - Lanczos.
Photoshop - Bilinear/Bicubic.
IrfanView - Lanczos.

That a picture looked good in larger, users must download a special program for this, or he could use the mozjpeg while saving the image.

I made one more comparison, this time more pictures of various types; 3 commercial products vs 2 popular algorithms vs NNEDI3.
Screen.

[dwbuiten]

@kowpa1990 you have completely missed my point.

Nobody uses cjpeg to encode JPEGs directly. They use ImageMagick's convert, or whatever image editing software they use. Adding support for upscaling in cjpeg is not going to help adoption. It is (as I've done it locally) triviall possible to add support for mozjpeg's special parameters to ImageMagick, and I gather you will see far adoption this way, which, if I am not mistaken, is the goal of mozjpeg (better encoding isn't useful is nobody uses it because the tools suck, or support isn't added to the most-used tool).

The quest to make cjpeg more feature-rich is misguided for this reason, IMO. It's not practical for adoption.

Aside: I am happy to submit a patch to ImageMagick to detect mozjpeg during confgure and build with it's more advanced features, once the a decision is made on the ABI/API.

P.S. Upscaling before encode is a poor fix for retina displays. Better fix is to have browsers use better upscaling algorithms post-decode. Upscaling pre-encude is just throwing bits away for a worse result IMO.

[kowpa1990]

you're right...

maybe I was just very impressed with how nnedi works. I need to learn how to use imagemagick ;)

Now issue is tagged as 'enhancement' so I dont know if anyone will implement this. A similar issue with large hidpi images someone already wrote - something about banding like Avisynth GradFun3().

Actually, in this project a lot of ideas you can take just from the Avisynth filters...

[andrews05]

@dwbuiten have you submitted a patch to ImageMagick? If not, can you quickly tell how to build it with mozjpeg?