Migrate to zlib-ng, part 2: consume it in runtime #102403
Conversation
|
To make this easier to review, it would be best to split this into 3 PRs:
|
|
@jkotas yes, I can do that. I don't want you to keep a fire extinguisher next to your overworked processor. |
|
The test binaries like src/native/external/zlib-ng/test/CVE-2002-0059/test.gz are going to pain to deal with for source build. Can we exclude the test subdirectory from the vendored copy? I do not expect that we are going to run the native zlib-ng tests in this repo. |
src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Windows.targets
Show resolved
Hide resolved
This is better than nothing, but it is still pretty clunky experience. FWIW, I am regularly splitting my changes into multiple PRs when they are mix of large mechanical delta and smaller actual code delta. For example, I have done it a few days ago in #102422. I could have folded this mechanical change into #102392 that it is needed for, but I have intentionally split it into multiple PRs to get it signed-off and merged faster. |
|
As requested, I submitted #102520 to only bring in the zlib-ng code into our repo (and its licensing files), but without consuming it anywhere yet. I'll update this PR to only include the code that enables zlib-ng, and will revert the changes that remove zlib and zlib-intel (I'll remove those in a third PR). |
carlossanlop
changed the title
|
Big difference now: 23 files instead of 388. |
src/native/libs/System.IO.Compression.Native/zlib_allocator_win.c
Outdated
Show resolved
Hide resolved
src/native/libs/System.IO.Compression.Native/zlib_allocator_win.c
Outdated
Show resolved
Hide resolved
|
I updated the PR description to include some initial perf comparisons in Linux, Windows and MacOS. |
|
The intention is that we don't use system zlib anymore anywhere right? is that OK with distro maintainers? I wonder if we should set ZLIB_SYMBOL_PREFIX to e.g. |
The linked issue indicates that the repro steps require creating a new iOS project. What that tells me is that the issue was found after merging, and it was caught in servicing (8.0, back in February). I also don't see a PR that fixed the issue, it looks like a workaround was given instead. I'd like to avoid getting at that point to find out about this problem, and also avoid a workaround. Do you know if I can repro this before merging this PR? Maybe even write a test than can confirm this does not happen? |
No, the problem was in fact caused by an iOS update after we shipped so there was no way to catch it beforehand, which is why I'd like to make sure we don't have the same potential conflict. |
|
ARMv6 support in zlib-ng is pretty new and still incomplete... We used to require minimum of ARMv8 as older processors are pretty rare nowadays and don't really benefit much (or any) from any of the currently implemented vector optimizations. |
|
/azp run runtime-community |
This comment was marked as outdated.
This comment was marked as outdated.
|
/azp run runtime-nativeaot-outerloop |
This comment was marked as outdated.
This comment was marked as outdated.
|
/azp run runtime-extra-platforms |
This comment was marked as outdated.
This comment was marked as outdated.
|
/ba-g all failures have been investigated, none are related to this PR |
| #include <unistd.h> | ||
| #ifndef _WIN32 | ||
| #include <unistd.h> | ||
| #endif |
There was a problem hiding this comment.
This file is available on all recent Windows compiler versions, so excluding is no longer strictly necessary as it was a few years ago.
|
This was reverted because it broke the official build. I'll take a look soon. |
…2403)" (dotnet#104414) This reverts commit 5b86dca.
…104454) * Reapply "Migrate to zlib-ng, part 2: consume it in runtime (#102403)" (#104414) * Apply jkotas comment suggestion in configureplatform.cmake * Delete unnecessary comment in zlib-ng.cmake * Fix windows nativeaot failure happening when executing: build.cmd -ci -arch x64 -os windows -s clr.nativeaotlibs+clr.nativeaotruntime+libs+packs -c Release /p:BuildNativeAOTRuntimePack=true /p:SkipLibrariesNativeRuntimePackages=true
Contributes to: #101465
Adjust levels if necessary to compensate any regressions.Performance comparisons
dotnet/performance benchmark results: #102403 (comment)
Comparison of CompressionLevel file sizes and elapsed time: #102403 (comment)