Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Progressive jpeg #28

Closed
motioncircus opened this issue Sep 6, 2018 · 52 comments
Closed

Progressive jpeg #28

motioncircus opened this issue Sep 6, 2018 · 52 comments

Comments

@motioncircus
Copy link

More of a question than an issue..
I generate and upload lots of 5K images to a web gallery and would like to present them progressively.
I use imgp and love it's speed but I can't find any reference to progressive mode.
Is it on the road map?
Thanks
Sdack
@jarun
Copy link
Owner

jarun commented Sep 7, 2018

So I assume you want progressive mode support. Is that correct?

@motioncircus
Copy link
Author

motioncircus commented Sep 7, 2018 via email

@jarun
Copy link
Owner

jarun commented Sep 7, 2018 •
edited
Loading

I see some indicative code:

Please try to send in a patch.

The ROI of imgp development is literally 0 while there are users who use it after every trip taken. No contributors, 0 donation. So I am quite reluctant to dedicate my time and effort into it. Help yourself.

@jarun jarun closed this as completed Sep 7, 2018
@motioncircus
Copy link
Author

motioncircus commented Sep 7, 2018 via email

@jarun
Copy link
Owner

jarun commented Sep 7, 2018

I'm not a proficient coder so please explain

I think the simplest way is to add a new argument that says enable progressive mode detection and save progressive images as progressive.
If this argument is passed (True), detect if the image is progressive while loading the image. If the image is progressive transform it and save it as progressive. There's a catch here - it needs to be investigated if the image transformations (like resize and rotate) are supported by the PIL library.

Would you be prepared to do this for a fee

It's GPL so it's free. Which also means anyone can modify the code and contribute back. If you find the utility useful, feel free to donate.

@motioncircus
Copy link
Author

motioncircus commented Sep 7, 2018 via email

@jarun
Copy link
Owner

jarun commented Sep 7, 2018

Let me check the feasibility in that case. I will get back with what I find.

@motioncircus
Copy link
Author

motioncircus commented Sep 7, 2018 via email

@jarun
Copy link
Owner

jarun commented Sep 7, 2018

Thanks a lot for the offer.

@motioncircus
Copy link
Author

motioncircus commented Sep 7, 2018 via email

@jarun
Copy link
Owner

jarun commented Sep 7, 2018

Even I don't. I will check the effort required. There are areas I need to investigate. The most critical part is - if PIL supports the transformations for progressive images.

@motioncircus
Copy link
Author

motioncircus commented Sep 7, 2018 via email

@jarun
Copy link
Owner

jarun commented Sep 7, 2018

Hi @motioncircus,

I did a little research. Here are the tasks I could figure out in pieces:

  • add a new argument to save JPEG images as progressive
  • detect if a JPEG image is progressive and save it as progressive. I think currently we do not retain the progressive aspect of it.
  • if the progressive argument is passed, save all JPEG images as progressive
  • update documents, help and man page with the new argument
  • test and confirm that the output images are indeed progressive. Do you know any quick way to figure it out? Otherwise I would have to look for some utility.

I can work it out in the weekend. Does $50 sound too much?

@jarun jarun reopened this Sep 7, 2018
@motioncircus
Copy link
Author

motioncircus commented Sep 7, 2018 via email

@jarun
Copy link
Owner

jarun commented Sep 8, 2018

$50 is is a bit steep, how about $40?

Done!

@jarun jarun closed this as completed in 1cde5c6 Sep 8, 2018
@jarun
Copy link
Owner

jarun commented Sep 8, 2018

@motioncircus I had added JPEG support to latest master. Can you please confirm if your output JPEG files are progressive as expected? You will have to use the new switch --pr.

@motioncircus
Copy link
Author

motioncircus commented Sep 8, 2018 via email

@motioncircus
Copy link
Author

motioncircus commented Sep 8, 2018 via email

@motioncircus
Copy link
Author

motioncircus commented Sep 8, 2018 via email

@motioncircus
Copy link
Author

motioncircus commented Sep 8, 2018 via email

@jarun
Copy link
Owner

jarun commented Sep 9, 2018

Can you please send me the image?

@motioncircus
Copy link
Author

motioncircus commented Sep 9, 2018 via email

@jarun
Copy link
Owner

jarun commented Sep 9, 2018

works with the -p option

The option for progressive is --pr

I downloaded - http://motioncircus.com/temp/TestProgressive.jpg

I ran:

imgp -x 50 TestProgressive.jpg --pr

When I uploaded the image to https://www.imgonline.com.ua/eng/progressive-or-baseline-jpeg.php it is confirmed as progressive.

@jarun
Copy link
Owner

jarun commented Sep 9, 2018

Does it work correctly with the option --pr?

@motioncircus
Copy link
Author

motioncircus commented Sep 9, 2018 via email

@jarun
Copy link
Owner

jarun commented Sep 9, 2018

I didn't get any mail from you yet. But please try:

imgp -x 100 TestProgressive.jpg

"missing transformation" is thrown when you don't use the resize or rotate option. -x 100 says use 100% resolution.

@jarun
Copy link
Owner

jarun commented Sep 9, 2018

Sorry, the right command is:

imgp -x 100 TestProgressive.jpg --pr

@motioncircus
Copy link
Author

motioncircus commented Sep 9, 2018 via email

@jarun
Copy link
Owner

jarun commented Sep 9, 2018

No problem. I would also make the changes where it works without the -x 100 input in the meantime.

@motioncircus
Copy link
Author

motioncircus commented Sep 9, 2018 via email

@jarun
Copy link
Owner

jarun commented Sep 9, 2018

Hi Nigel,

With commit 5658ca8 in master (take latest), now you can use:

imgp  TestProgressive.jpg --pr --keep

to avoid any transformation.

Or if you want optimization (enabled by default), just run:

imgp  TestProgressive.jpg --pr

@motioncircus
Copy link
Author

motioncircus commented Sep 9, 2018 via email

@jarun
Copy link
Owner

jarun commented Sep 9, 2018 •
edited
Loading

I think the options are simply to choose how many scans the image offers.

I checked the PIL docs for JPEG - https://pillow.readthedocs.io/en/5.1.x/handbook/image-file-formats.html#jpeg

I don't think there's any exposed way to specify the number of scans. So this is the best I could do.

@jarun
Copy link
Owner

jarun commented Sep 9, 2018

Also, I took this image from Google: https://www.google.com/about/datacenters/gallery/images/_2000/IDI_018.jpg
The source is a progressive image.

EXIF data from original and imgp-converted image:

~/G/imgp$ exiftool IDI_018.jpg 
ExifTool Version Number         : 10.10
File Name                       : IDI_018.jpg
Directory                       : .
File Size                       : 1140 kB
File Modification Date/Time     : 2018:09:09 10:29:03+05:30
File Access Date/Time           : 2018:09:09 10:29:17+05:30
File Inode Change Date/Time     : 2018:09:09 10:29:03+05:30
File Permissions                : rw-rw-r--
File Type                       : JPEG
File Type Extension             : jpg
MIME Type                       : image/jpeg
JFIF Version                    : 1.01
Resolution Unit                 : None
X Resolution                    : 1
Y Resolution                    : 1
Image Width                     : 2000
Image Height                    : 1333
Encoding Process                : Progressive DCT, Huffman coding
Bits Per Sample                 : 8
Color Components                : 3
Y Cb Cr Sub Sampling            : YCbCr4:4:4 (1 1)
Image Size                      : 2000x1333
Megapixels                      : 2.7


~/G/imgp$ exiftool IDI_018_IMGP.jpg 
ExifTool Version Number         : 10.10
File Name                       : IDI_018_IMGP.jpg
Directory                       : .
File Size                       : 455 kB
File Modification Date/Time     : 2018:09:09 10:29:18+05:30
File Access Date/Time           : 2018:09:09 10:30:33+05:30
File Inode Change Date/Time     : 2018:09:09 10:29:18+05:30
File Permissions                : rw-rw-r--
File Type                       : JPEG
File Type Extension             : jpg
MIME Type                       : image/jpeg
JFIF Version                    : 1.01
Resolution Unit                 : None
X Resolution                    : 1
Y Resolution                    : 1
Image Width                     : 2000
Image Height                    : 1333
Encoding Process                : Progressive DCT, Huffman coding
Bits Per Sample                 : 8
Color Components                : 3
Y Cb Cr Sub Sampling            : YCbCr4:2:0 (2 2)
Image Size                      : 2000x1333
Megapixels                      : 2.7

As you can see, both the images show:

Encoding Process                : Progressive DCT, Huffman coding

@motioncircus
Copy link
Author

motioncircus commented Sep 9, 2018 via email

@jarun
Copy link
Owner

jarun commented Sep 9, 2018

Hi Nigel,

Thanks! Please use the PayPal donate button in the project page.

@motioncircus
Copy link
Author

motioncircus commented Sep 9, 2018 via email

@jarun
Copy link
Owner

jarun commented Sep 9, 2018 •
edited
Loading

I have checked the PIL source as well. I does call the progressive option of the underlying library when the option is used: https://github.com/python-pillow/Pillow/blob/fdbd719da4c77c7e23e2e9e9b71d0d177f2d3369/src/libImaging/JpegEncode.c#L218

Also, you can get an idea on the type of progression used:

https://www4.cs.fau.de/Services/Doc/graphics/doc/jpeg/libjpeg.html
search the function "jpeg_simple_progression" in this file for a description.

I will raise an issue on the PIL library to handle custom scan scripts if they can support it.

@motioncircus
Copy link
Author

motioncircus commented Sep 9, 2018 via email

@jarun
Copy link
Owner

jarun commented Sep 9, 2018

Thanks for the compliment Nigel!

Do you have a resume of your skills online somewhere?

Nope, nothing of that sort. I guess you can look me up on Google though.

The defect has been accepted as an enhancement: python-pillow/Pillow#3343

@motioncircus
Copy link
Author

motioncircus commented Sep 9, 2018 via email

@jarun
Copy link
Owner

jarun commented Sep 10, 2018

Sure thing!

@kkopachev
Copy link

Simple progression used by libjpeg already has 10 scans.
There is a chance, however, that mobile phone, to save on decoding, waits for the whole image to only show it once it's done. You might want to use jpegtran with -scans option to pass custom scans script to try out if it makes any difference for your tests.

@motioncircus
Copy link
Author

motioncircus commented Sep 11, 2018 via email

@kkopachev
Copy link

Oh, ok. First 2 images both have 10 scans total. They could be different configuration scans, though, but given amount of data each scan brings, I think they are the same.
You might want to try jpegtran with custom scans file and see if what you want is achievable with more scans.

@motioncircus
Copy link
Author

motioncircus commented Sep 11, 2018 via email

@jarun
Copy link
Owner

jarun commented Sep 11, 2018

Oh, ok. First 2 images both have 10 scans total.

@kkopachev which utility are you using to detect the number of scans?

It would be great if I can have the support from the PIL library. I believe with jpegtran it would be enough to write a small script that runs on every file in a directory and changes the number of scans.

@kkopachev
Copy link

kkopachev commented Sep 11, 2018 •
edited
Loading

@jarun exiftool is probably easiest

exiftool -v3 2018_0911_213000.jpg | grep SOS`

I'd be happy to have Pillow support custom progressive scans too. I mentioned jpegtran just to test things and see if custom scans could do something to the problem.

@jarun
Copy link
Owner

jarun commented Sep 11, 2018

Thank you!

@jarun
Copy link
Owner

jarun commented Sep 11, 2018

Yes, I can see the default 10 scans set for an IMGP generated output file:

$ exiftool -v3 TestProgressive_IMGP.jpg |grep SOS
JPEG SOS
JPEG SOS
JPEG SOS
JPEG SOS
JPEG SOS
JPEG SOS
JPEG SOS
JPEG SOS
JPEG SOS
JPEG SOS

@jarun
Copy link
Owner

jarun commented Sep 11, 2018 •
edited
Loading

@motioncircus till this support in PIL arrives, you may want to use jpegtran, you will find more info on scan scripts here: https://github.com/cloudflare/jpegtran/blob/master/wizard.txt

Check for the string scans.

@motioncircus
Copy link
Author

motioncircus commented Sep 11, 2018 via email

@lock lock bot locked and limited conversation to collaborators May 20, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@kkopachev @jarun @motioncircus