Keep track of files that have already been compressed

[LuckyTurtleDev]

If the user has a slow pc or using a high compression, he might can not compress all images at once.
But after aborting it is not possible to continue at the last compressed image anymore.
So I would suggests adding an option to define an archive file (similar to youtube-dl) which store the path of all images, which where already compressed. This would make it possible to continue compressing where you have stopped.

I can implement this, if this feature is wanted.

[andrews05]

Hm, I understand your issue but this seems like an incredibly niche feature which I would say falls out of scope of what oxipng should be providing.

Can you explain your scenario a little more? Like, could you just run oxipng on a subset of your files at a time so you know where you're up to?

[LuckyTurtleDev]

I have run oxipng at my home folder yesterday, but it has take forever and I have abort it. Today I started it again, but it starts from the beginning. Because I have absolute no idea where it has stop I can not continue without starting from scratch. (It would also be a pain to define this per argument).

Some thing like oxipng --archive foo.txt . would allow to save a list of compressed files and to skip them if I execute the same command again.

could you just run oxipng on a subset of your files at a time so you know where you're up to?

I think this should also be possible. But it much more complicated and annoying . You do not know how long this would take and if you abort the process you still don not know where you have stop. You must also note/remember which files you have compress before.

[ace-dent]

could you just run oxipng on a subset of your files

I would say this is how all png optimizers work. Handling of bulk jobs is normally done by a script / coded GUI, etc. ... Not sure it should be a core function here, due to portability and maintaining that extra code...

[andrews05]

Yeah, I hear the frustration but you should be able to script something yourself outside of oxipng. E.g. you could start by constructing a list of all your png files and then making a script to work through them. Or you could try only finding files modified before a given date, so as to exclude ones that were recently optimised (assuming Windows has some sort of find equivalent that can do this?).

I'm afraid I just don't agree that this should be built-in to oxipng. That is just my opinion though - @AlexTMjugador or @shssoichiro may have different ideas.

[edit] Keeping track of a file queue is a feature you might expect from a GUI program such as FileOptimizer. Have you tried using this instead? If you like oxipng, perhaps you could ask the developer to add it as a plugin. [edit2] I forgot, this is blocked on apng support in oxipng, which will be available in the next release.

[AlexTMjugador]

I agree with @andrews05 and @ace-dent that this feature is out of scope for OxiPNG. In my opinion, tracking the progress of a batch optimization job is better suited to a script or higher level application with more context about the job, and I find the alternative approaches suggested by @andrews05 to be spot-on.

I would suggests adding an option to define an archive file (similar to youtube-dl) which store the path of all images, which where already compressed.

This statement made me curious as to why youtube-dl has such an option, however. After some digging, I found out that it was introduced to handle new videos being added to time-varying video sources (i.e., playlists and the like). [1] [2]

I think this is a substantially different scenario than that of OxiPNG: video sources can change at indeterminate times, and there are no good ways to keep track of changes to video sources outside of youtube-dl: youtube-dl is usually the only reasonable way to query those video sources, which often have proprietary interfaces that change all the time. On the contrary, OxiPNG CLI users can easily know when the input dataset changes, because the interface is the filesystem.

All in all, I think this feature would trade complexity on one side for another, in a way that I'm not sure OxiPNG would do better than a custom script. It's also not that hard to process a few PNG files at a time and record their names in a temporary file to resume the job if necessary.

[aaronfranke]

For anyone who wants this feature, feel free to grab and compile my branch: https://github.com/aaronfranke/oxipng/tree/cache-opt-files and then run oxipng with the --cache flag, which will cache the hashes of optimized files in your user cache folder, so you can re-run oxipng on the same files and it will skip already-optimized files.

All in all, I think this feature would trade complexity on one side for another, in a way that I'm not sure OxiPNG would do better than a custom script.

It really isn't much complexity. My branch accomplishes this in under a hundred lines of code (plus a dependency on the dirs crate). To me it seems very much worth it, it's a small amount of code for a huge feature that will save users either hours of execution time or hours of writing a script to keep track of optimized files.

[andrews05]

Just to comment further here:
If we did want to proceed with a feature like this it would require extensive consideration about all the different ways and reasons people might want to use it, including problems that might be encountered and how they could be resolved. We can already see that people have different ideas on how it could/should be done, and what works well for one person may not necessarily work well for someone else.

@aaronfranke It might be good if you could detail your specific use case here, just for reference.

I'll stress again though, that this is not a direction we want to take oxipng in at this time.

[aaronfranke]

Let's say I have a folder of 100 PNG files. I may run oxipng on this folder, which takes a long time. Then, I add 10 more PNG files to this folder so that there are 110 files. If I want to re-optimize the folder, running oxipng on the whole folder again will re-optimize all 110 files. I would like to optimize the folder without repeating the work already done to optimize the first 100 images, so that it only needs to optimize the 10 new images.

[LuckyTurtleDev]

I personal would prefer if the cache file stored at a user definded place. So it can be moved together with the pngs.
For example '--cache-file cache.txt'

[andrews05]

[edit] Keeping track of a file queue is a feature you might expect from a GUI program such as FileOptimizer. Have you tried using this instead? If you like oxipng, perhaps you could ask the developer to add it as a plugin. [edit2] I forgot, this is blocked on apng support in oxipng, which will be available in the next release.

FileOptimizer now supports oxipng 🙂
[edit] ImageOptim has also been updated to oxipng v9.