Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend GZIP conditional #1294

Merged
merged 1 commit into from
Jun 16, 2022
Merged

Extend GZIP conditional #1294

merged 1 commit into from
Jun 16, 2022

Conversation

stoeckmann
Copy link
Contributor

If gzip support has been disabled during compilation then also
consider gzip relevant states as invalid in deflateStateCheck.

Also the gzip state definitions can be removed.

This change leads to failure in test/example, and I am not sure
what the GZIP conditional is trying to achieve. All gzip related
functions are still defined in zlib.h

Alternative approach is to remove the GZIP define.

If gzip support has been disabled during compilation then also
consider gzip relevant states as invalid in deflateStateCheck.

Also the gzip state definitions can be removed.

This change leads to failure in test/example, and I am not sure
what the GZIP conditional is trying to achieve. All gzip related
functions are still defined in zlib.h

Alternative approach is to remove the GZIP define.
Copy link
Member

@nmoinvaz nmoinvaz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may be possible to also wrap these struct fields in #ifdef GZIP:

    uint32_t             gzindex;          /* where in extra, name, or comment */
    PREFIX(gz_headerp)   gzhead;           /* gzip header information to write */

@Dead2
Copy link
Member

Dead2 commented Jun 14, 2022

It may be possible to also wrap these struct fields in #ifdef GZIP:

    uint32_t             gzindex;          /* where in extra, name, or comment */
    PREFIX(gz_headerp)   gzhead;           /* gzip header information to write */

The problem with doing that is that it affects cacheline alignment and grouping for the whole rest of the struct, potentially making adjacent hot variables no longer reside in the same cacheline etc, so probably not worth it unless we do a much bigger job of it and ensure these get placed at the end of the struct where they will do no harm (or replace them with dummy values of the same size, not that that would really make any difference other than potentially for readability).

@codecov
Copy link

codecov bot commented Jun 14, 2022

Codecov Report

Merging #1294 (edf9c45) into develop (a7598c1) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           develop    #1294   +/-   ##
========================================
  Coverage    86.59%   86.59%           
========================================
  Files          124      124           
  Lines        10533    10533           
  Branches      2622     2622           
========================================
  Hits          9121     9121           
  Misses        1056     1056           
  Partials       356      356           
Flag Coverage Δ
macos_clang 33.33% <ø> (ø)
macos_gcc 73.82% <ø> (ø)
ubuntu_clang 85.42% <ø> (ø)
ubuntu_clang_debug 85.06% <ø> (ø)
ubuntu_clang_inflate_allow_invalid_dist 85.13% <ø> (ø)
ubuntu_clang_inflate_strict 85.24% <ø> (-0.14%) ⬇️
ubuntu_clang_mmap 85.56% <ø> (ø)
ubuntu_clang_pigz 40.25% <ø> (ø)
ubuntu_clang_pigz_no_optim 41.70% <ø> (ø)
ubuntu_clang_pigz_no_threads 39.84% <ø> (ø)
ubuntu_clang_reduced_mem 85.81% <ø> (+0.13%) ⬆️
ubuntu_gcc 75.67% <ø> (ø)
ubuntu_gcc_aarch64 77.52% <ø> (ø)
ubuntu_gcc_aarch64_compat_no_opt 75.64% <ø> (ø)
ubuntu_gcc_aarch64_no_acle 76.32% <ø> (ø)
ubuntu_gcc_aarch64_no_neon 76.22% <ø> (ø)
ubuntu_gcc_armhf 77.54% <ø> (ø)
ubuntu_gcc_armhf_compat_no_opt 75.62% <ø> (ø)
ubuntu_gcc_armhf_no_acle 77.54% <ø> (ø)
ubuntu_gcc_armhf_no_neon 77.32% <ø> (ø)
ubuntu_gcc_armsf 77.19% <ø> (ø)
ubuntu_gcc_armsf_compat_no_opt 75.29% <ø> (ø)
ubuntu_gcc_benchmark 74.08% <ø> (ø)
ubuntu_gcc_compat_no_opt 76.85% <ø> (ø)
ubuntu_gcc_compat_sprefix 74.00% <ø> (ø)
ubuntu_gcc_m32 73.62% <ø> (-0.15%) ⬇️
ubuntu_gcc_mingw_i686 0.00% <ø> (ø)
ubuntu_gcc_mingw_x86_64 0.00% <ø> (ø)
ubuntu_gcc_no_avx2 74.82% <ø> (ø)
ubuntu_gcc_no_ctz 74.88% <ø> (ø)
ubuntu_gcc_no_ctzll 74.92% <ø> (ø)
ubuntu_gcc_no_pclmulqdq 74.07% <ø> (ø)
ubuntu_gcc_no_sse2 75.03% <ø> (ø)
ubuntu_gcc_no_sse4 74.72% <ø> (ø)
ubuntu_gcc_o1 74.40% <ø> (ø)
ubuntu_gcc_osb ∅ <ø> (∅)
ubuntu_gcc_pigz 38.13% <ø> (-0.14%) ⬇️
ubuntu_gcc_pigz_aarch64 39.30% <ø> (ø)
ubuntu_gcc_ppc 73.69% <ø> (ø)
ubuntu_gcc_ppc64 74.46% <ø> (ø)
ubuntu_gcc_ppc64le 74.43% <ø> (ø)
ubuntu_gcc_ppc_no_power8 74.64% <ø> (ø)
ubuntu_gcc_s390x 74.85% <ø> (ø)
ubuntu_gcc_s390x_dfltcc 72.23% <ø> (ø)
ubuntu_gcc_s390x_dfltcc_compat 73.77% <ø> (ø)
ubuntu_gcc_s390x_no_crc32 74.63% <ø> (ø)
ubuntu_gcc_sparc64 74.74% <ø> (ø)
ubuntu_gcc_sprefix 73.66% <ø> (ø)
win64_gcc 73.86% <ø> (ø)
win64_gcc_compat_no_opt 74.38% <ø> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
deflate.c 83.29% <ø> (ø)
deflate.h 100.00% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a7598c1...edf9c45. Read the comment docs.

@stoeckmann
Copy link
Contributor Author

Neither the configure script nor the CMakeLists.txt offer a reasonable way to disable GZIP. Since zlib has only the configure script, the statement is true for that project as well. So maybe it wouldn't hurt to remove these defines. It reduces complexity after all.

Or is it desirable to extend these build environments?

@Dead2
Copy link
Member

Dead2 commented Jun 14, 2022

This define is only really useful when building zlib statically into an application that does not need to support gzip at all.

deflate.h has this comment:

/* define NO_GZIP when compiling if you want to disable gzip header and
   trailer creation by deflate().  NO_GZIP would be used to avoid linking in
   the crc code when it is not needed.  For shared libraries, gzip encoding
   should be left enabled. */
#ifndef NO_GZIP
#  define GZIP
#endif

I am not sure how much code space is saved by disabling gzip and crc support, so that would have to be investigated before any real discussion on this could take place. I feel pretty sure that it does not affect performance measurably (except by random cache alignment chance). I also think gzip support can safely be enabled even when an application expects it to be absent.
So removing this support could potentially be done while still remaining compatible with zlib, the question is whether doing so is worth losing the (likely meagre) gains when using it and whether this actually simplifies the code enough to warrant a removal.

@mtl1979
Copy link
Collaborator

mtl1979 commented Jun 14, 2022

Especially the new CRC tables from 1.2.12 are quite large, so the savings can be few kilobytes.

@nmoinvaz
Copy link
Member

@mtl1979 the code size after the new tables were incorporated decreased.

See #1268 (comment)

@nmoinvaz
Copy link
Member

nmoinvaz commented Jun 14, 2022

@mtl1979 the tables look like they might be larger in the header file but actually only a portion of those generated tables are used.

@mtl1979
Copy link
Collaborator

mtl1979 commented Jun 14, 2022

@nmoinvaz Code size might have decreased but const variables aren't considered code. Old CRC code had no support for 64-bit processors, so it had simpler tables. When we separated little-endian and big-endian tables, we practically reduced the size of the tables by 50%.

@Dead2 Dead2 merged commit 3f7b0b4 into zlib-ng:develop Jun 16, 2022
@Dead2 Dead2 mentioned this pull request Dec 27, 2022
Dead2 added a commit that referenced this pull request Mar 7, 2023
Changes since 2.0.6:
- Fix CVE-2022-37434 #1328
- Fix chunkmemset #1196
- Fix deflateBound too small #1236
- Fix Z_SOLO #1263
- Fix ACLE variant of crc32 #1274
- Fix inflateBack #1311
- Fix deflate_quick windowsize #1431
- Fix DFLTCC bugs related to adler32 #1349 and #1390
- Fix warnings #1194 #1312 #1362
- MacOS build fix #1198
- Add invalid windowBits handling #1293
- Support for Force TZCNT #1186
- Support for aligned_alloc() #1360
- Minideflate improvements #1175 #1238
- Dont use unaligned access for memcpy #1309
- Build system #1209 #1233 #1267 #1273 #1278 #1292 #1316 #1318 #1365
- Test improvements #1208 #1227 #1241 #1353
- Cleanup #1266
- Documentation #1205 #1359
- Misc improvements #1294 #1297 #1306 #1344 #1348
- Backported zlib fixes
- Backported CI workflows from Develop branch
Dead2 added a commit that referenced this pull request Mar 17, 2023
Changes since 2.0.6:
- Fix CVE-2022-37434 #1328
- Fix chunkmemset #1196
- Fix deflateBound too small #1236
- Fix Z_SOLO #1263
- Fix ACLE variant of crc32 #1274
- Fix inflateBack #1311
- Fix deflate_quick windowsize #1431
- Fix DFLTCC bugs related to adler32 #1349 and #1390
- Fix warnings #1194 #1312 #1362
- MacOS build fix #1198
- Add invalid windowBits handling #1293
- Support for Force TZCNT #1186
- Support for aligned_alloc() #1360
- Minideflate improvements #1175 #1238
- Dont use unaligned access for memcpy #1309
- Build system #1209 #1233 #1267 #1273 #1278 #1292 #1316 #1318 #1365
- Test improvements #1208 #1227 #1241 #1353
- Cleanup #1266
- Documentation #1205 #1359
- Misc improvements #1294 #1297 #1306 #1344 #1348
- Backported zlib fixes
- Backported CI workflows from Develop branch
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants