Add an option to compile against zlib-ng instead of zlib

[dralley]

Zlib-ng has roughly (ymmv) twice the performance of standard zlib.

[dralley]

@praiskup I used that rubygem repo from COPR with 270,000 packages to test.

The end result is a little less impressive than I was hoping. The main issue is that without --skip-stat --recycle-packagelist --update the difference gets swamped by stat calls and reading headers, and without --no-database it gets swamped by the cost of BZ2 compression, which seems to be very inefficient in comparison to even original zlib.

In the best case scenario using all 4 flags together, it does reduce time spent inside the compression library by about 40%, but that only equates to a 10% overall improvement in runtime, which might not be worthwhile relative to the maintenance burden. zstd should be faster than that and result in a 10-25% overall improvement which is easier to justify especially given the file size benefits.

But as far as COPR goes it would be a great idea to disable generating sqlite metadata everywhere except EL6 and EL7 repos because it has a huge impact. Disabling it dropped the runtime from 242 seconds to 106 seconds

DNF doesn't use it so it's safe to drop, but I think "mdapi" does still use it currently, I don't know if that's a blocker for COPR the way it is for Fedora proper https://pagure.io/releng/issue/10745

[dralley]

This is a profile of a --skip-stat --recycle-packagelist --update run on the aforementioned repo with original zlib and including sqlite metadata. You can see that the main issue is bz2 compression (of the sqlite metadata), specifically.

[praiskup]

Thank you for another look at performance. We use --skip-stat
--recycle-packagelist --update for sure, otherwise it would be entirely
unusable. Disabling sqlite data will help a lot, and is tracked here.
Going to zstd seems to be a long-term task (not sure how likely is to get
this to e.g. RHEL9+ at least).

[dralley]

It should be supported by RHEL 8+ unless it's compiled out when building libsolv. It's hard to test without having a repo first.