Crash when compressing large number of pictures

[jl0476]

Describe the bug

When I try to compress a large number of images in batch, over 8000 images, with 4 threads for parallel compression as shown in the screenshot, the program becomes unresponsive at the beginning and then automatically exits.

Software version
2.5.1

Operating System information

• OS: Windows
• Version 11

To Reproduce
Steps to reproduce the behavior:

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots

Application Log
[2023-11-15 14:48:31.903][I] ---- Starting application ----
[2023-11-15 14:48:31.904][I] UUID: "df98b942-bfa8-40cb-a616-1c4f4b7ba7ee"
[2023-11-15 14:48:32.067][I] Starting UI
[2023-11-15 14:49:07.086][W] "W:/download/00down/0728/miababe/miababe01/miababe01/�� ��ѹ���롿���ϴ廨��̳��С����.jpg" is not supported. Error: Image format is not supported.
[2023-11-15 14:49:09.012][W] "W:/download/00down/0728/sunwall95/sunwall95/�� ��ѹ���롿���ϴ廨��̳��С����.jpg" is not supported. Error: Image format is not supported.
[2023-11-15 14:49:09.013][W] "W:/download/00down/0728/TUTU_June/�� ��ѹ���롿���ϴ廨��̳��С����.jpg" is not supported. Error: Image format is not supported.
[2023-11-15 14:49:09.592][W] "W:/download/00down/0728/TUTU_June/ҧһ������ovo - ���¶��� ΢��������/STM07254.pTPCdMZQ.jpg" is not supported. Error: Image format is not supported.
[2023-11-15 14:49:09.616][W] "W:/download/00down/0728/TUTU_June/ҧһ������ovo - ���¶��� ΢��������/STM07261.BnIRsbmh.jpg" is not supported. Error: Image format is not supported.
[2023-11-15 14:49:09.626][W] "W:/download/00down/0728/TUTU_June/ҧһ������ovo - ���¶��� ΢��������/STM07261.tXaHLkQb.jpg" is not supported. Error: Image format is not supported.
[2023-11-15 14:49:09.636][W] "W:/download/00down/0728/TUTU_June/ҧһ������ovo - ���¶��� ΢��������/STM07262.PQCCAakW.jpg" is not supported. Error: Image format is not supported.
[2023-11-15 14:49:09.639][W] "W:/download/00down/0728/TUTU_June/ҧһ������ovo - ���¶��� ΢��������/STM07262.zlkpxhhe.jpg" is not supported. Error: Image format is not supported.
[2023-11-15 14:49:09.649][W] "W:/download/00down/0728/TUTU_June/ҧһ������ovo - ���¶��� ΢��������/STM07264.VKSjALne.jpg" is not supported. Error: Image format is not supported.
[2023-11-15 14:49:09.651][W] "W:/download/00down/0728/TUTU_June/ҧһ������ovo - ���¶��� ΢��������/STM07264.zmnSjuQM.jpg" is not supported. Error: Image format is not supported.
[2023-11-15 14:49:09.652][W] "W:/download/00down/0728/TUTU_June/ҧһ������ovo - ���¶��� ΢��������/STM07266.aQjpBHBW.jpg" is not supported. Error: Image format is not supported.
[2023-11-15 14:49:09.654][W] "W:/download/00down/0728/TUTU_June/ҧһ������ovo - ���¶��� ΢��������/STM07266.fbqZIYPj.jpg" is not supported. Error: Image format is not supported.
[2023-11-15 14:49:09.665][W] "W:/download/00down/0728/TUTU_June/ҧһ������ovo - ���¶��� ΢��������/STM07267.eXRpFKvi.jpg" is not supported. Error: Image format is not supported.
[2023-11-15 14:49:09.675][W] "W:/download/00down/0728/TUTU_June/ҧһ������ovo - ���¶��� ΢��������/STM07267.vESmStaQ.jpg" is not supported. Error: Image format is not supported.
[2023-11-15 14:49:09.686][W] Corrupt JPEG data: 19377 extraneous bytes before marker 0xd0
[2023-11-15 14:49:09.697][W] "W:/download/00down/0728/TUTU_June/ҧһ������ovo - ���¶��� ΢��������/STM07270.sXSuJYQC.jpg" is not supported. Error: Image format is not supported.
[2023-11-15 14:49:24.084][W] "W:/download/00down/0728/��12��/���ҽ�/�� ��ѹ���롿���ϴ廨��̳��С����.jpg" is not supported. Error: Image format is not supported.
[2023-11-15 14:52:44.655][W] OpenType support missing for "Microsoft YaHei UI", script 11
[2023-11-15 14:52:44.655][W] OpenType support missing for "Tahoma", script 11
[2023-11-15 14:52:44.656][W] OpenType support missing for "SimSun", script 11
[2023-11-15 14:52:44.657][W] OpenType support missing for "Arial", script 11
[2023-11-15 14:52:44.659][W] OpenType support missing for "MS UI Gothic", script 11
[2023-11-15 14:52:44.659][W] OpenType support missing for "Segoe UI Emoji", script 11
[2023-11-15 14:52:44.660][W] OpenType support missing for "Segoe UI Symbol", script 11
[2023-11-15 15:03:57.127][W] Corrupt JPEG data: 19377 extraneous bytes before marker 0xd0

-----over----

[Lymphatus]

Seems it's not detecting some images correctly and possibly crashes. Can you share some of the pictures that in the log seems to be corrupted so I can perform some tests?

[jl0476]

Seems it's not detecting some images correctly and possibly crashes. Can you share some of the pictures that in the log seems to be corrupted so I can perform some tests?

sure, can you give me an email address, i will package the images and send it to you

[Lymphatus]

You can attach a zip file here on Github, no need to share emails. I just need a bunch of them, should not be too big in size either.

[jl0476]

You can attach a zip file here on Github, no need to share emails. I just need a bunch of them, should not be too big in size either.

I tested it and found that whenever I compress this file, the application crashes. This image contains some NSFW content, so please let me know if you mind.

STM07268.zip

[Lymphatus]

This image contains some NSFW content, so please let me know if you mind.

It's ok.
I'll test and let you know. Thanks!

[Lymphatus]

The image it self seems fine and works on my machine. Can you please share the content you see when going to "Edit > Preferences... > Show Usage Data" (编辑 > 选项... > 展示使用数据). It may be something related to the compression options. A screenshot of the Preferences panel could also be helpful.

Also, are you compressing the files on an external device or a network storage?

[jl0476]

The image it self seems fine and works on my machine. Can you please share the content you see when going to "Edit > Preferences... > Show Usage Data" (编辑 > 选项... > 展示使用数据). It may be something related to the compression options. A screenshot of the Preferences panel could also be helpful.

系统数据
{
"appVersion": "2.5.1",
"cpuArchitecture": "x86_64",
"kernelType": "winnt",
"kernelVersion": "10.0.22621",
"productType": "windows",
"productVersion": "11",
"uuid": "df98b942-bfa8-40cb-a616-1c4f4b7ba7ee"
}

压缩数据
{
"compressedSize": "...",
"compressionOptions": {
"compression/jpeg_quality": 80,
"compression/keep_metadata": "true",
"compression/keep_structure": "true",
"compression/lossless": "true",
"compression/max_output_size": 2,
"compression/max_output_size_unit": 2,
"compression/mode": 0,
"compression/png_quality": 80,
"compression/webp_quality": 60,
"output/format": 0,
"output/keep_creation_date": "true",
"output/keep_dates": 2,
"output/keep_last_access_date": "true",
"output/keep_last_modified_date": "true",
"output/move_original_to_trash": "false",
"output/output_folder": "J:/H/CaeOut",
"output/output_suffix": "",
"output/same_folder_as_input": "true",
"output/skip_if_bigger": "true",
"resize/do_not_enlarge": "true",
"resize/fit_to": 4,
"resize/height": 1000,
"resize/keep_aspect_ratio": "true",
"resize/resize": "true",
"resize/size": 3840,
"resize/width": 100
},
"elapsedTime": "...",
"totalFiles": "...",
"uncompressedSize": "...",
"uuid": "df98b942-bfa8-40cb-a616-1c4f4b7ba7ee"
}

Also, are you compressing the files on an external device or a network storage?

firstly it store on a Samba shared disk, but later I copied it to a local disk for compression, and it also crashed.

[jl0476]

HI，
If there is any need for more information, please let me know.

[jl0476]

@Lymphatus Hello, is there any progress? Do you need any additional information from me?

[Lymphatus]

Sorry, been a little busy lately. I'll take a closer look in the next days.

[Lymphatus]

Found the issue. Some of the images, like the one you provided, have some corruption related to the RST marker and causes problem with Caesium while resizing.
The only workaround right now, while I try to fix the bug for the next version, is to disable resizing and compress all the pictures (overwriting the originals). Then you can re-run Caesium with the same set of images using resizing. This works because on the first compression, the app "fixes" the issue and the can resize without problems.

[jl0476]

Found the issue. Some of the images, like the one you provided, have some corruption related to the RST marker and causes problem with Caesium while resizing. The only workaround right now, while I try to fix the bug for the next version, is to disable resizing and compress all the pictures (overwriting the originals). Then you can re-run Caesium with the same set of images using resizing. This works because on the first compression, the app "fixes" the issue and the can resize without problems.

thanks, I will try it

[jl0476]

@Lymphatus
During the compression process, the UI becomes unresponsive, but after a while, it manages to display a completion message.

系统数据
{
"appVersion": "2.5.1",
"cpuArchitecture": "x86_64",
"kernelType": "winnt",
"kernelVersion": "10.0.22621",
"productType": "windows",
"productVersion": "11",
"uuid": "df98b942-bfa8-40cb-a616-1c4f4b7ba7ee"
}

压缩数据
{
"compressedSize": "...",
"compressionOptions": {
"compression/jpeg_quality": 80,
"compression/keep_metadata": "true",
"compression/keep_structure": "true",
"compression/lossless": "true",
"compression/max_output_size": 2,
"compression/max_output_size_unit": 2,
"compression/mode": 0,
"compression/png_quality": 80,
"compression/webp_quality": 60,
"output/format": 0,
"output/keep_creation_date": "true",
"output/keep_dates": 2,
"output/keep_last_access_date": "true",
"output/keep_last_modified_date": "true",
"output/move_original_to_trash": "false",
"output/output_folder": "J:/H/CaeOut",
"output/output_suffix": "",
"output/same_folder_as_input": "true",
"output/skip_if_bigger": "true",
"resize/do_not_enlarge": "true",
"resize/fit_to": 0,
"resize/height": 1000,
"resize/keep_aspect_ratio": "true",
"resize/resize": "false",
"resize/size": 3840,
"resize/width": 100
},
"elapsedTime": "...",
"totalFiles": "...",
"uncompressedSize": "...",
"uuid": "df98b942-bfa8-40cb-a616-1c4f4b7ba7ee"
}

[jl0476]

Found the issue. Some of the images, like the one you provided, have some corruption related to the RST marker and causes problem with Caesium while resizing. The only workaround right now, while I try to fix the bug for the next version, is to disable resizing and compress all the pictures (overwriting the originals). Then you can re-run Caesium with the same set of images using resizing. This works because on the first compression, the app "fixes" the issue and the can resize without problems.

Hi, Do you have any plans for a recent upgrade? Thank you very much.

[Lymphatus]

Sorry for the delay, I was a little bit busy with the online version of the tool. I still don't have a date, but it's not too far away.

[Lymphatus]

Fixed in v2.6.0.
The app is still not really able to compress corrupted JPEGs but it does not crash anymore. I'll try to improve it in the future.

[jl0476]

Fixed in v2.6.0. The app is still not really able to compress corrupted JPEGs but it does not crash anymore. Ill try to improve it in the future.

okay, thank you