-
-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Compression Improvements #2
Comments
To elaborate on
here are a few jars which had significant gains from this (using Detonater) (this was after having been processed with
so, there are definitely significant savings to be had here by doing this. the mods that primarily benefit from this change are the ones which bundle many libraries in them. |
Thanks for the details! Your proposal will definitely help improving mod(pack) sizes. I will work on optional directory skips in ZIP/JAR files. This removal will already make the mods smaller. This would make things simpler because the library ( The new file types you mentioned can be easily added for minification or recompression. I was testing mainly on Forge mods so I may overlook file types like The jar-in-jar repacking situation is very tricky. It may need a completely new minifier with customizible options. I will try to make it possible. New separate issues will be made. Thanks again for using MC-Repack! |
the mods I used to generate that list were mainly fabric mods, as I just used the modpacks on my laptop. I have a few more instances on my desktop, so I can re-run it on there when I get back home. also, I don't think that
you could do smth similar to the following: but for the sorting, you could just sort all of them. Also, I'm interested in using this as part of a gradle plugin I'm making (I was originally planning to do all these things myself, and was looking at making an oxipng kotlin/java jni wrapper, but then I found this and this is honestly basically what I need) |
yo, unsure if you saw this or not bc it was a weekend, so imma bump it lol |
Sorry for late response. I would rather not make a library for a specific environment. I will keep maintaining CLI and core library. There is another project I am working on and |
wdym a library for a specific environment? also, no worries, take your time 👍 |
I meant JNI "glue" for JVM. |
yeah, I just mainly want to make a jni interface bc it's a lot more convenient to have this applied as a gradle plugin rather than a cli library |
Hi,
just found your project and this looks rather interesting.
Here's a couple of improvements/changes that would be amazing if they could be added:
Explode nested jars and recursively repack them. however, in those nested jars only everSeparate issue: Jar-in-jar minifier #3STORE
files. then addDEFLATE
compression at the final level.Sort files by their extension. When files of the same type are beside each other in a nested jar file, it will be able to be better compressed. (based on some rudementary testing, this combined with the previous item, can drastically improve the compression at times)Separate issue: Jar-in-jar minifier #3Recompress nbt files (they are just gzip compressed, so uncompress then attempt to recompress at level 9), then STORE them.Separate issue: NBT minifier #4Based on a quick little command I ran, I recorded the frequencies of different file types present in jar files.
The command used was
Here are the results from that:
.class
.json
.png
.nbt
.ogg
.MF
.jar
.mcmeta
.xdelta
(I have no clue what this file format is).md5
.properties
.at
.txt
.accesswidener
.xml
.md
.js
Based on this, I think it might be reasonable to consider adding optimization processes for the following files:
.ogg
files. These can possibly be optimized using ffmpeg. (you could possibly consider decreasing the quality to increase compression. a cli arg for this would be nice.) (there seem to be a couple of crates that provide ffmpeg support, so those could be considered).nbt
files. They are just gzip compressed binary data, so uncompress then recompress with level 9, thenSTORE
them..properties
files. Blank lines can be stripped from these. (it doesn't make up a large amount of the files, but it's an easy addition).accesswidener
files. Blank lines can be stripped from these, as well as any lines starting with a hash (#
) are comments. (it doesn't make up a large amount of the files, but it's an easy addition)It's probably not worth it to consider additional compressors for files not in that table as they appear so infrequently it just won't make much of a difference.
Do note however, that this is not based off of the size of the files in the jar, rather just their count. may make something basic to calculate this using the size later.
The text was updated successfully, but these errors were encountered: