-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Limit the number of decoded images in memory #6
Comments
Hi @cozarkd, thanks for the request. My deadlines are pretty rough at moment, so it might be a few weeks before I can look into this. I feel this issue is nonetheless important so it definitely will get looked at when I get a moment or need a break from other work. So I can hit the ground running and reproduce the issue quickly, could you tell me the average size and format of the images you're dealing with? |
My images are 5-15mb and PNG without alpha channel.
If I sort by size and get the smaller ones I can take more each time. With
the bigger ones I can't process more than 20.
16gb RAM, Windows WSL2 (I think it takes like 70% of total RAM)
El mié., 21 feb. 2024 4:46, aggregate1166877 ***@***.***>
escribió:
… Hi @cozarkd <https://github.com/cozarkd>, thanks for the request.
My deadlines are pretty rough at moment, so it might be a few weeks before
I can look into this. I feel this issue is nonetheless important so it
definitely will get looked at when I get a moment or need a break from
other work.
So I can hit the ground running and reproduce the issue quickly, could you
tell me the average size and format of the images you're dealing with?
—
Reply to this email directly, view it on GitHub
<#6 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABRKPG4MP6HB6KTMQDSJJPDYUVUZ5AVCNFSM6AAAAABDSDUHJOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJVHAZTKMJXG4>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Hi @cozarkd - I believe this issue is now fixed. Can you please test with CLI version The application will now only process as many images as you have CPU cores. If you have a crazy amount of cores and run out of memory, I've added a I also noticed that the terminal output gets badly corrupted with a large amount of files. The CLI will still use the fancy output when processing less than 16 images, but anything more than that will now use a basic / boring output that works correctly with any amount of images. TestingI did the following tests using 16 threads, all converting png to webp:
All completed without error. EditThere was a memory leak in |
Is your feature request related to a problem? Please describe.
I'm trying to optimize 500+ images but it eats all the RAM and then crashes (On WSL2, Windows). On my Mac it just gets stuck and blinking. I've found an issue in the original project with a PR.
No idea if this is hard to implement but would be nice. I can optimize in batches of around 30-50 images but it's not ideal since I need to do those 500 images multiple times.
Thanks for your fork.
The text was updated successfully, but these errors were encountered: