Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improves compression ratio for small windowLog #1624

Merged
merged 7 commits into from
Jun 15, 2019
Merged

Improves compression ratio for small windowLog #1624

merged 7 commits into from
Jun 15, 2019

Commits on May 31, 2019

  1. first implementation of small window size for btopt

    noticeably improves compression ratio
    when window size is small (< 18).
    
    enwik7	level 19
    
    windowLog	`dev`	`smallwlog`	improvement
    23	3.577	3.577	0.02%
    22	3.536	3.538	0.06%
    21	3.462	3.467	0.14%
    20	3.364	3.377	0.39%
    19	3.244	3.272	0.86%
    18	3.110	3.166	1.80%
    17	2.843	3.057	7.53%
    16	2.724	2.943	8.04%
    15	2.594	2.822	8.79%
    14	2.456	2.686	9.36%
    13	2.312	2.523	9.13%
    12	2.162	2.361	9.20%
    11	2.003	2.182	8.94%
    Cyan4973 committed May 31, 2019
    Configuration menu
    Copy the full SHA
    bc601bd View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    58adb10 View commit details
    Browse the repository at this point in the history
  3. Made fast strategy compatible with new offset validation strategy

    fast mode does the same thing as before :
    it pre-emptively invalidates any index that could lead to offset > maxDistance.
    It's supposed to help speed.
    
    But this logic is performed inside zstd_fast,
    so that other strategies can select a different behavior.
    Cyan4973 committed May 31, 2019
    Configuration menu
    Copy the full SHA
    a30feba View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    d605f48 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    a968099 View commit details
    Browse the repository at this point in the history

Commits on Jun 1, 2019

  1. Configuration menu
    Copy the full SHA
    fce4df3 View commit details
    Browse the repository at this point in the history
  2. removed UINT32_MAX

    apparently not guaranteed on all platforms,
    replaced by UINT_MAX.
    Cyan4973 committed Jun 1, 2019
    Configuration menu
    Copy the full SHA
    80d6cce View commit details
    Browse the repository at this point in the history