Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2.1.7 backport release #1740

Merged
merged 67 commits into from
Jun 19, 2024
Merged

2.1.7 backport release #1740

merged 67 commits into from
Jun 19, 2024

Conversation

Dead2
Copy link
Member

@Dead2 Dead2 commented May 30, 2024

Due to the high amount of refactoring changes in develop, I have decided to target those to a new version branch 2.2.x.
There is also a lot of fixes and minor improvements, so those will be backported and released as 2.1.7.

To work around the #1708 issue (incompatibility with applications mis-using the zlib zalloc/zfree API), #1710 is merged instead of backporting the much bigger #1713.

Backported Changes

Buildsystem

CI/Test

Cleanup

Refactoring and Optimizations

ARM

Power

RVV

S390x

x86

Misc

nmoinvaz and others added 30 commits May 30, 2024 16:01
We need to turn off support all higher instruction sets in order to hit code
for a lower instruction set.
These were left on my part, the inline copy + checksum is the very
thing the function is doing.
* POWER8/9 feature checks were enabled even if the toolchain didn't support AT_HWCAP2
* Add detection if we need to include <linux/auxvec.h>
* For COMPAT mode, zlib-config.cmake and zlib-config-version.cmake
* For non-COMPAT mode, zlib-ng-config.cmake and zlib-ng-config-version.cmake
madler/zlib#8988e03256e9c80766ac6899e86c3bc57c347efc
madler/zlib#5af7cef45eeef86ddf6ab00b4e363c1eecaf47b6
To avoid warnings when building with -Wmissing-prototypes.

madler/zlib#bd9c329c1055a9265812352655ed2eec93f36e92
madler/zlib#431a9b65eacab7efabf2230ba97ff426c0e07f9d
If it is negative, then the code will enter an infinite loop.

madler/zlib#36e369e1a54b35a978dc584496af69a07ec2d71a
gz_intmax() is noted in zlib.map. This assures it's always there.

madler/zlib#01253ecd7e0a01d311670f2d03c61b82fc12d338
There used to be one, but no more. It is up to the user or vendor
to compile zlib.

madler/zlib#25263462377fa5c569947a40315815bd59e37f4b
The --version-script linker option is not supported by the linker on AIX systems

madler/zlib#ade6825c4996f73d776b0ffa038360762bd318ae
Not all C compilers have a -w option.

madler/zlib#88ec24670e9c484219392def5f2f4b5e3f839db3
madler/zlib#51b7f2abdade71cd9bb0e7a373ef2610ec6f9daf
When exporting a CMake target the `install(TARGETS ${ZLIB_INSTALL_LIBRARIES})` is repeated with an `EXPORT` parameter. On Windows this would install the `.dll` files into the `lib` directory.

Set the `EXPORT_NAME` earlier and unconditionally on the `TARGET`. Then move the `EXPORT` to a single `install(TARGETS)`.
thewtex and others added 20 commits May 30, 2024 16:32
Addresses:

  3.29.0/share/cmake/Modules/CMakeDependentOption.cmake:89 (message):
    Policy CMP0127 is not set: cmake_dependent_option() supports full Condition
    Syntax.  Run "cmake --help-policy CMP0127" for policy details.  Use the
    cmake_policy command to set the policy and suppress this warning.
  Call Stack (most recent call first):
    CMakeLists.txt:107 (cmake_dependent_option)
This warning is for project developers.  Use -Wno-dev to suppress it.
Although Git is able to automatically modify line-endings during checkin
and checkout, this brings a lot of trouble, especially when trying to
use a repository from different platforms (as Windows and Linux). This
is due to the fact that Git consults different local gitconfig settings
and uses problematic defaults if not set.

Therefore, Git should enfoce one type of line-ending (LF) and not
consult the local config, which is what the change from this commit
does.

Signed-off-by: Deniz Bahadir <deniz@code.bahadir.email>
Signed-off-by: Vladislav Shchapov <vladislav@shchapov.ru>
Signed-off-by: Vladislav Shchapov <vladislav@shchapov.ru>
Signed-off-by: Vladislav Shchapov <vladislav@shchapov.ru>
Currently the DFLTCC sanitizer instrumentation is limited to
MSAN-unpoisoning the parameter block. Add ASAN and MSAN checks;
also MSAN-unpoison the window.

Introduce the generic instrument_read(), instrument_write() and
instrument_read_write() macros, that are modeled after the repsective
functions in the Linux kernel.
The benchmarks fail to compile properly when built with ZLIB_COMPAT.
The name of the class cannot have the same name as the function.
- New dockerfile
  - Using native actions-runner instead of relying on qemu.
  - To support s390x, we include patches to actions-runner.
  - Using Almalinux 9 instead of Ubuntu, with functional .Net.
- Update CI workflow.
- Update readme guide.
Disable unneccessary compilation of tests, benchmarks, docs.
Signed-off-by: Vladislav Shchapov <vladislav@shchapov.ru>
The Xeon Phi x200 family of processors (Knights Landing) supports
AVX512 (F, CD, ER, PF) but does not support AVX512 (VL, DQ, BW).

Because of processors like this, the Intel Software Developer's Manual
suggests the bits AVX512 (DQ,BW,VL) are also tested in EBX together with
AVX512F before deciding to run AVX512 (DQ,BW,VL) instructions.

This also adds a new x86 feature called avx512_common that indicates
that AVX512 (F,DQ,BW,VL) are all available and start using this for both
adler32_avx512 and crc32_vpclmulqdq implementations because they are
both built with -mavx512dq -mavx512bw -mavx512vl.

This has been reported downstream as
https://bugzilla.redhat.com/show_bug.cgi?id=2280347 .
Signed-off-by: Vladislav Shchapov <vladislav@shchapov.ru>
Check that calling inflateCopy() twice does not result in memory
corruption.
- Fix error in generating list of reports to upload.
- Clean up the implementation a bit.
Signed-off-by: Vladislav Shchapov <vladislav@shchapov.ru>
Signed-off-by: Vladislav Shchapov <vladislav@shchapov.ru>
Copy link

codecov bot commented May 30, 2024

Codecov Report

Attention: Patch coverage is 71.68142% with 32 lines in your changes missing coverage. Please review.

Project coverage is 82.64%. Comparing base (7425372) to head (9a85aa8).

Files Patch % Lines
arch/x86/x86_features.c 22.22% 6 Missing and 1 partial ⚠️
test/fuzz/fuzzer_minigzip.c 66.66% 3 Missing and 3 partials ⚠️
test/benchmarks/benchmark_compress.cc 70.58% 1 Missing and 4 partials ⚠️
zutil.c 20.00% 3 Missing and 1 partial ⚠️
gzlib.c 0.00% 2 Missing ⚠️
test/benchmarks/benchmark_adler32_copy.cc 0.00% 0 Missing and 2 partials ⚠️
test/benchmarks/benchmark_adler32.cc 0.00% 0 Missing and 1 partial ⚠️
test/benchmarks/benchmark_crc32.cc 0.00% 0 Missing and 1 partial ⚠️
test/minigzip.c 85.71% 1 Missing ⚠️
test/test_adler32.cc 0.00% 0 Missing and 1 partial ⚠️
... and 2 more
Additional details and impacted files
@@            Coverage Diff             @@
##            2.1.x    #1740      +/-   ##
==========================================
- Coverage   83.03%   82.64%   -0.39%     
==========================================
  Files         133      135       +2     
  Lines       10897    10334     -563     
  Branches     2816     2820       +4     
==========================================
- Hits         9048     8541     -507     
+ Misses       1147     1075      -72     
- Partials      702      718      +16     
Flag Coverage Δ
macos_clang ?
macos_gcc ?
ubuntu_clang ?
ubuntu_clang_debug ?
ubuntu_clang_inflate_allow_invalid_dist ?
ubuntu_clang_inflate_strict ?
ubuntu_clang_mmap ?
ubuntu_clang_pigz ?
ubuntu_clang_pigz_no_optim ?
ubuntu_clang_pigz_no_threads ?
ubuntu_clang_reduced_mem ?
ubuntu_clang_toolchain_riscv ?
ubuntu_gcc ?
ubuntu_gcc_aarch64 ?
ubuntu_gcc_aarch64_compat_no_opt ?
ubuntu_gcc_aarch64_no_acle ?
ubuntu_gcc_aarch64_no_neon ?
ubuntu_gcc_armhf ?
ubuntu_gcc_armhf_compat_no_opt ?
ubuntu_gcc_armhf_no_acle ?
ubuntu_gcc_armhf_no_neon ?
ubuntu_gcc_armsf ?
ubuntu_gcc_armsf_compat_no_opt ?
ubuntu_gcc_benchmark ?
ubuntu_gcc_compat_no_opt ?
ubuntu_gcc_compat_sprefix ?
ubuntu_gcc_m32 ?
ubuntu_gcc_mingw_i686 ?
ubuntu_gcc_mingw_x86_64 ?
ubuntu_gcc_mips ?
ubuntu_gcc_mips64 ?
ubuntu_gcc_native_inst ?
ubuntu_gcc_no_avx2 ?
ubuntu_gcc_no_ctz ?
ubuntu_gcc_no_ctzll ?
ubuntu_gcc_no_pclmulqdq ?
ubuntu_gcc_no_sse2 ?
ubuntu_gcc_no_sse42 ?
ubuntu_gcc_o1 ?
ubuntu_gcc_osb ?
ubuntu_gcc_pigz ?
ubuntu_gcc_pigz_aarch64 ?
ubuntu_gcc_ppc ?
ubuntu_gcc_ppc64 ?
ubuntu_gcc_ppc64_power9 ?
ubuntu_gcc_ppc64le ?
ubuntu_gcc_ppc64le_novsx ?
ubuntu_gcc_ppc64le_power9 ?
ubuntu_gcc_ppc_no_power8 ?
ubuntu_gcc_s390x ?
ubuntu_gcc_s390x_dfltcc ?
ubuntu_gcc_s390x_dfltcc_compat ?
ubuntu_gcc_s390x_no_crc32 ?
ubuntu_gcc_sparc64 ?
ubuntu_gcc_sprefix ?
win64_gcc ?
win64_gcc_compat_no_opt ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mtl1979
Copy link
Collaborator

mtl1979 commented Jun 9, 2024

I presume the Codecov failure is partly due to CI not supporting AVX512. I currently don't have anything "new" that I would want to be included in next 2.1.x release, but isn't already included here.

mtl1979 and others added 3 commits June 12, 2024 14:11
Especially when replacing zlib with zlib-ng in old binaries, it is possible that the binary is mixing allocation and
deallocation functions from different libraries. As not all old binaries can be rebuild, we should validate the passed
in pointer and if the pointer doesn't seem to be allocated with alloc_aligned of zlib-ng, we should not try to adjust
the pointer.
@Dead2 Dead2 added the Backport label Jun 12, 2024
@Dead2 Dead2 changed the title Prepare 2.1.7 backport release 2.1.7 backport release Jun 12, 2024
@Dead2 Dead2 merged commit 8d8768a into 2.1.x Jun 19, 2024
276 of 278 checks passed
@Dead2 Dead2 deleted the prep-2.1.7 branch June 19, 2024 12:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet