Deterministic output

[arcanis]

Does this crate guarantees deterministic outputs for a given input and release? I sometimes need to regenerate archives and compare their checksums, and this can only be performed if we're reasonably certain the input is equivalent regardless of the platform / time of day / other sources of enthropy.

[Byron]

I don't think checksums over archives are stable - there is no guarantee. Using a stable backend like miniz_oxide would certainly help to keep the output stable though.

I still remember how GitHub caused some issues downstream as they took hashes from archives, and then something changed. It would probably better to hash the contents (similar to what Git is doing).

[arcanis]

I can tolerate variance given different input parameters under my control, whether those parameters are library versions etc. I'm mostly worried about entropy-based inputs like race conditions, or machine-specific inputs like gzip's OS field. Is flate2 relying on similar unpredictable inputs?

[fintelia]

Asking whether the crate currently uses unpredictable inputs is different from asking for a guarantee that it never will

[oyvindln]

miniz_oxide and zlib do not use any threads internally or access any os data so I don't see how any race conditions would be possible and they should give the same output given the same input. I haven't really looked much into zlib-rs code but I would assume the same is the case there, relatively sure there aren't any thread calls there either.

The os field in the gzip header is provided by the caller, it's not interpreted based on system currently.

Note that none of the backends give the exact same output as each other when compressing as the implementations differ slightly so you have to fully control what backend is used if you want to have consistent output. zlib-ng, zlib-rs (and cloudflare-zlib) made some tradeoffs for increased performance over compression efficiency over stock zlib while the original miniz and by extension miniz_oxide differs a little in implementation compared to zlib.

Since this is the case it's important to note that that the zlib backends may not be consistent between platforms if you are using a os-provided version of the library as I know at least debian (and likely by extension ubuntu and other debian/ubuntu based linux distros) replaces all zlib-type backends with stock zlib. It also downgrades miniz_oxide to 0.7.4. I belive fedora and possibly some other linux distros also replace stock zlib with zlib-ng in case one is using the zlib-sys backend.

EDIT: that's the current situation - not a guarantee of anything changing in the future of course as others have stated