Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use crc32 from libdeflate, remove bundled crc32 code #6387

Merged
merged 1 commit into from
Mar 13, 2022

Conversation

ThomasWaldmann
Copy link
Member

@ThomasWaldmann ThomasWaldmann commented Mar 2, 2022

No description provided.

@codecov-commenter
Copy link

codecov-commenter commented Mar 12, 2022

Codecov Report

Merging #6387 (07f4d53) into master (d9d1e44) will not change coverage.
The diff coverage is n/a.

❗ Current head 07f4d53 differs from pull request most recent head 98a675f. Consider uploading reports for the commit 98a675f to get more accurate results

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #6387   +/-   ##
=======================================
  Coverage   83.15%   83.15%           
=======================================
  Files          39       39           
  Lines       10457    10457           
  Branches     2057     2057           
=======================================
  Hits         8696     8696           
  Misses       1254     1254           
  Partials      507      507           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d9d1e44...98a675f. Read the comment docs.

@ThomasWaldmann
Copy link
Member Author

ThomasWaldmann commented Mar 13, 2022

Local test on MBA, macOS 12 (M1 cpu):

Name (time in us)                Mean             StdDev                Median                   OPS          
--------------------------------------------------------------------------------------------------------------
test_zlib_crc32              328.8660 (1.0)       9.6358 (1.0)        326.3340 (1.0)      3,040.7518 (1.0)    
test_deflate_crc32         1,261.1159 (3.83)     76.3716 (7.93)     1,236.2500 (3.79)       792.9485 (0.26)   
test_borg_crc32_slice8     4,386.7739 (13.34)    35.7977 (3.72)     4,375.3965 (13.41)      227.9580 (0.07)   
--------------------------------------------------------------------------------------------------------------

On github CI, macOS (x64):

Name (time in ms)            Mean            StdDev            Median                 OPS          
---------------------------------------------------------------------------------------------------
test_zlib_crc32            3.0155 (1.0)      0.2969 (1.02)     2.9466 (1.0)      331.6165 (1.0)    
test_deflate_crc32         3.1616 (1.05)     0.2898 (1.0)      3.0889 (1.05)     316.2978 (0.95)   
test_borg_crc32_slice8     4.5672 (1.51)     0.24.. (1.39)     4.5125 (1.53)     218.9503 (0.66)   
---------------------------------------------------------------------------------------------------

On github CI, Linux (x64):

Name (time in us)                Mean             StdDev                Median                   OPS          
--------------------------------------------------------------------------------------------------------------
test_deflate_crc32           589.7927 (1.0)       5.6325 (1.24)       588.5160 (1.0)      1,695.5110 (1.0)    
test_borg_crc32_clmul        591.3464 (1.00)      4.5526 (1.0)        589.60.. (1.00)     1,691.0562 (1.00)   
test_borg_crc32_slice8     4,550.2839 (7.72)     20.4307 (4.49)     4,555.1760 (7.74)       219.7665 (0.13)   
test_zlib_crc32            8,602.3128 (14.59)     7.8226 (1.72)     8,600.7385 (14.61)      116.2478 (0.07)   
--------------------------------------------------------------------------------------------------------------

@ThomasWaldmann
Copy link
Member Author

OK, so guess this means:

if macOS:
     use zlib.crc32
else:
    use deflate.crc32

And we'll remove our bundled crc32 code, looks like the stuff in (lib)deflate is comparable or even better.

@ThomasWaldmann ThomasWaldmann marked this pull request as ready for review March 13, 2022 18:46
@ThomasWaldmann ThomasWaldmann changed the title crc32 stuff replace bundled crc32 code by libdeflate/zlib code Mar 13, 2022
we use zlib.crc32 (macOS) or libdeflate_crc32 (Linux and all others) now.
@ThomasWaldmann ThomasWaldmann changed the title replace bundled crc32 code by libdeflate/zlib code use crc32 from libdeflate, remove bundled crc32 code Mar 13, 2022
@ThomasWaldmann ThomasWaldmann merged commit a6634c0 into borgbackup:master Mar 13, 2022
@ThomasWaldmann ThomasWaldmann deleted the crc32 branch March 13, 2022 21:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants