Disable dynamic function dispatching for native or arch-specific builds

[phprus]

Another implementation of PR #1645.

Main differences:

• Separate CPU-specific function prototypes and CPU-arch features checker.
• Reducing dependencies between header files.

[codecov]

Codecov Report

Attention: Patch coverage is 97.22222% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 83.03%. Comparing base (af494fc) to head (a25d5a7).

❗ Current head a25d5a7 differs from pull request most recent head 660b36b. Consider uploading reports for the commit 660b36b to get more accurate results

Files	Patch %	Lines
deflate.c	88.88%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##           develop    #1659       +/-   ##
============================================
+ Coverage         0   83.03%   +83.03%     
============================================
  Files            0      134      +134     
  Lines            0    10336    +10336     
  Branches         0     2813     +2813     
============================================
+ Hits             0     8583     +8583     
- Misses           0     1055     +1055     
- Partials         0      698      +698     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[phprus]

All compile time CPU features checks is implemented.

[phprus]

Rebased.

[nmoinvaz]

I like how it is split into cpu_features and cpu_functions.

[Dead2]

This dependency can be removed from win32/ makefiles. And the one below should be added.

[phprus]

This dependency is not direct now, but indirect, through cpu_functions.h.

As I see the dependencies in win32/Makefile.* were broken before this PR.
For example, win32/Makefile.arm has a dependency on x86 file $(SRCDIR)/arch/x86/x86_features.h.

If we need actual nmake support (without cmake), I suggest fix the nmake-dependencies with separate PRs.

[mtl1979]

I can try to fix the nmake dependencies when all the "more important" changes have been merged... It would help a lot if people would create issues when there is something wrong in the nmake Makefiles as I can't track all the changes if pull requests are merged in batches. I try to check most of the new pull requests at least once a week (unless I'm tagged in the pull request), but my time is limited due to other projects that are more important.

[Dead2]

I don't see any of the new header files added to the win32/ makefiles, those should probably go in so the dependency handling works properly.
Actually we should probably make a script that generates those deps, and have a CI job that fails when there is a diff. Those are kind of a mess already, and steadily becoming worse. @mtl1979 what do you think, is it worth it or not?

I think perhaps the cpu_functions.h file should be named arch_functions.h, since it is not so much related to the running cpu (like cpu_features.h does) but rather to the arch-specific functions located in the arch/* directories.

I have only done a quick review so far, but what I have looked at so far looks good. I agree that the separation looks good.

[mtl1979]

I don't see any of the new header files added to the win32/ makefiles, those should probably go in so the dependency handling works properly. Actually we should probably make a script that generates those deps, and have a CI job that fails when there is a diff. Those are kind of a mess already, and steadily becoming worse. @mtl1979 what do you think, is it worth it or not?

I think perhaps the cpu_functions.h file should be named arch_functions.h, since it is not so much related to the running cpu (like cpu_features.h does) but rather to the arch-specific functions located in the arch/* directories.

I have only done a quick review so far, but what I have looked at so far looks good. I agree that the separation looks good.

I have never looked up if cl.exe can generate dependency information for Windows. Even if it can, there's always issue that there is no sed-like tool to reformat the output so it matches the format nmake expects.

[nmoinvaz]

Time to ditch nmake like I've been saying for years. :)

[mtl1979]

Time to ditch nmake like I've been saying for years. :)

nmake is still getting new features... It's just that some people use older versions, for example version that is bundled with Visual C++ 2017.9 ...

[nmoinvaz]

nmake is still getting new features...

I can't locate a nmake change log anywhere..

[phprus]

Rebased.

@Dead2, Renamed cpu_functions.h to arch_functions.h.

[Dead2]

Develop ca0e463 Feb 21

 Tool: minigzip   Levels: 1-9
 Runs: 70         Trim worst: 40

 Level   Comp   Comptime min/avg/max/stddev  Decomptime min/avg/max/stddev  Compressed size
 1     54.166%      0.150/0.154/0.156/0.001        0.034/0.036/0.038/0.001        8,523,732
 2     43.871%      0.244/0.249/0.250/0.001        0.032/0.036/0.038/0.002        6,903,609
 3     42.387%      0.295/0.297/0.299/0.001        0.031/0.034/0.036/0.001        6,670,099
 4     41.647%      0.317/0.324/0.326/0.002        0.030/0.034/0.035/0.001        6,553,723
 5     41.216%      0.344/0.352/0.354/0.002        0.032/0.034/0.035/0.001        6,485,938
 6     41.037%      0.390/0.397/0.400/0.003        0.030/0.033/0.035/0.001        6,457,776
 7     40.778%      0.493/0.498/0.503/0.003        0.030/0.034/0.035/0.001        6,416,919
 8     40.704%      0.605/0.613/0.618/0.004        0.029/0.033/0.034/0.001        6,405,244
 9     40.409%      0.912/0.919/0.925/0.004        0.029/0.033/0.035/0.002        6,358,951

 avg1  42.913%                        0.423                          0.034
 tot                                114.095                          9.195       60,775,991

   text    data     bss     dec     hex filename
 138190    1352       8  139550   2211e libz-ng.so.2

Develop Native=on

Level   Comp   Comptime min/avg/max/stddev  Decomptime min/avg/max/stddev  Compressed size
1     54.166%      0.147/0.151/0.153/0.002        0.028/0.035/0.037/0.002        8,523,732
2     43.871%      0.239/0.242/0.244/0.001        0.031/0.035/0.037/0.002        6,903,609
3     42.387%      0.285/0.289/0.291/0.002        0.031/0.034/0.035/0.001        6,670,099
4     41.647%      0.316/0.318/0.320/0.001        0.030/0.033/0.035/0.001        6,553,723
5     41.216%      0.345/0.351/0.354/0.002        0.030/0.033/0.034/0.001        6,485,938
6     41.037%      0.389/0.397/0.401/0.003        0.029/0.033/0.034/0.002        6,457,776
7     40.778%      0.485/0.495/0.498/0.003        0.030/0.033/0.035/0.001        6,416,919
8     40.704%      0.600/0.607/0.611/0.003        0.027/0.032/0.034/0.002        6,405,244
9     40.409%      0.903/0.909/0.912/0.003        0.028/0.033/0.034/0.002        6,358,951

avg1  42.913%                        0.418                          0.033
tot                                112.795                          9.000       60,775,991

  text    data     bss     dec     hex filename
137222    1352       8  138582   21d56 libz-ng.so.2

PR #1659 Native=off

Level   Comp   Comptime min/avg/max/stddev  Decomptime min/avg/max/stddev  Compressed size
1     54.166%      0.151/0.154/0.155/0.001        0.031/0.035/0.037/0.001        8,523,732
2     43.871%      0.247/0.249/0.250/0.001        0.032/0.036/0.038/0.002        6,903,609
3     42.387%      0.291/0.295/0.297/0.002        0.032/0.035/0.036/0.001        6,670,099
4     41.647%      0.319/0.323/0.326/0.002        0.032/0.034/0.035/0.001        6,553,723
5     41.216%      0.345/0.351/0.354/0.002        0.029/0.034/0.035/0.001        6,485,938
6     41.037%      0.392/0.397/0.400/0.002        0.031/0.033/0.035/0.001        6,457,776
7     40.778%      0.491/0.499/0.503/0.003        0.029/0.033/0.035/0.001        6,416,919
8     40.704%      0.606/0.611/0.616/0.003        0.031/0.033/0.035/0.001        6,405,244
9     40.409%      0.911/0.918/0.924/0.004        0.031/0.033/0.035/0.001        6,358,951

avg1  42.913%                        0.422                          0.034
tot                                113.946                          9.209       60,775,991

  text    data     bss     dec     hex filename
138190    1352       8  139550   2211e libz-ng.so.2

PR #1659 Native=on

 Level   Comp   Comptime min/avg/max/stddev  Decomptime min/avg/max/stddev  Compressed size
 1     54.166%      0.093/0.097/0.099/0.001        0.027/0.034/0.037/0.003        8,523,732
 2     43.871%      0.172/0.176/0.178/0.002        0.032/0.035/0.037/0.001        6,903,609
 3     42.387%      0.224/0.227/0.229/0.001        0.031/0.034/0.035/0.001        6,670,099
 4     41.647%      0.253/0.256/0.259/0.002        0.029/0.033/0.035/0.002        6,553,723
 5     41.216%      0.285/0.290/0.293/0.002        0.029/0.033/0.034/0.002        6,485,938
 6     41.037%      0.328/0.338/0.342/0.003        0.028/0.032/0.034/0.002        6,457,776
 7     40.778%      0.489/0.495/0.499/0.003        0.027/0.033/0.035/0.002        6,416,919
 8     40.704%      0.601/0.608/0.612/0.003        0.028/0.033/0.034/0.002        6,405,244
 9     40.409%      0.903/0.912/0.918/0.004        0.025/0.031/0.033/0.002        6,358,951

 avg1  42.913%                        0.378                          0.033
 tot                                102.026                          8.896       60,775,991

   text    data     bss     dec     hex filename
 133814    1208       8  135030   20f76 libz-ng.so.2

Develop w/native=ON vs PR w/native=ON compression speedup:

• Level 1: 35%
• Level 3: 21%
• Level 6: 15%
• Level 9: 0.3%

PR w/native=ON vs PR #1681 w/native=OFF:

• Level 1: 11%
• Level 3: 8%
• Level 6: 5%
• Level 9: 0.5% slower

Looks like a decent speedup to me.
Ideally this would have been based on top of #1681, as it removes a good portion of the complexity in this PR.

[phprus]

My suggestion:

1. Extract commits:

• Split CPU features checks and CPU-specific function prototypes and reduce include-dependencies.
• Move select for generic functions into generic_functions.h.
• Rename cpu_functions.h to arch_functions.h.

into separate PR.

2. Rebase Remove update_hash and insert_string implementations from functable #1681 onto separated files implementation.

3. Rebase this PR onto new develop branch after previous PRs is merged.

What do your think about this plan?

[Dead2]

What do your think about this plan?

@phprus I think that is a good plan and would probably ensure we get it all merged the fastest. 👍

[phprus]

@Dead2

1. PR Remove duplicated codecov parameter #1684 - small fix for Allow overwrite NATIVEFLAG value. #1662.
2. PR Split cpu features and arch functions #1685 - split headers.

[phprus]

Rebase required.
Waiting PR #1673 (NMake)
Waiting #1681, #1682 (deflate optimizations).

[phprus]

Ready for review.

[Dead2]

LGTM

[phprus]

Rebased onto develop

[nmoinvaz]

Can we use macos-latest here instead? If there is a reason, perhaps it should be commented?

[phprus]

https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners

The macos-latest workflow label currently uses the macOS 12 runner image.

Therefore we need macos-14 (first ARM64 runner).

[nmoinvaz]

I see. Got it.