You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I've been reviewing several potential low-level zip libraries and found this compact and very readable for my purposes. Good work! I'm testing this on Android devices, and I found the performance was ~3x slower than Java's ZipInputStream class. This was surprising so I benchmarked the current zip_extract method and found that the bulk of the delay was with the current mz_crc32 method. The file I am testing on has ~600 files in a 350mb uncompressed zip file. The current library was taking around 4.5 seconds to unzip an uncompressed zip file. Java was taking 1.8 seconds to do the same task. To fix this, I tested replacing the current mz_crc32 method with a method from FastCRC32 by Stephen Brumme https://github.com/stbrumme/crc32 That library has several extreme optimizations for calculating a crc32, and the sweet spot from my testing was using the crc32_16bytes_prefetch method with the 256 look ahead as noted in the comments. This reduced the 4.5 second loading to 1.4 seconds. Hope this feedback helps! :)
The text was updated successfully, but these errors were encountered:
Good stuff, Thanks!
I'm not sure if we can use directly https://github.com/stbrumme/crc32 library here (because of licensing), but definitively we can try extend API, so everyone can pass own implementation of CRC32.
I've been reviewing several potential low-level zip libraries and found this compact and very readable for my purposes. Good work! I'm testing this on Android devices, and I found the performance was ~3x slower than Java's ZipInputStream class. This was surprising so I benchmarked the current zip_extract method and found that the bulk of the delay was with the current mz_crc32 method. The file I am testing on has ~600 files in a 350mb uncompressed zip file. The current library was taking around 4.5 seconds to unzip an uncompressed zip file. Java was taking 1.8 seconds to do the same task. To fix this, I tested replacing the current mz_crc32 method with a method from FastCRC32 by Stephen Brumme https://github.com/stbrumme/crc32 That library has several extreme optimizations for calculating a crc32, and the sweet spot from my testing was using the crc32_16bytes_prefetch method with the 256 look ahead as noted in the comments. This reduced the 4.5 second loading to 1.4 seconds. Hope this feedback helps! :)
The text was updated successfully, but these errors were encountered: