Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add ZSTD support #104

Merged
merged 55 commits into from
Feb 25, 2022
Merged

Add ZSTD support #104

merged 55 commits into from
Feb 25, 2022

Commits on May 5, 2021

  1. Added zstd directories to include paths.

    @james-s-willis
    james-s-willis committed May 5, 2021
    Configuration menu
    Copy the full SHA
    050477e View commit details
    Browse the repository at this point in the history

  2. Updated API with calls to zstd compress/decompress functions.

    @james-s-willis
    james-s-willis committed May 5, 2021
    Configuration menu
    Copy the full SHA
    ddaabbe View commit details
    Browse the repository at this point in the history

  3. Added zstd source v1.4.9.

    @james-s-willis
    james-s-willis committed May 5, 2021
    Configuration menu
    Copy the full SHA
    2774f40 View commit details
    Browse the repository at this point in the history

Commits on May 6, 2021

  1. Added ZSTD compression to H5 filter.

    @james-s-willis
    james-s-willis committed May 6, 2021
    Configuration menu
    Copy the full SHA
    26653a8 View commit details
    Browse the repository at this point in the history

  2. Added ZSTD python interface.

    @james-s-willis
    james-s-willis committed May 6, 2021
    Configuration menu
    Copy the full SHA
    ec9085d View commit details
    Browse the repository at this point in the history

  3. Removed unused ZSTD files.

    @james-s-willis
    james-s-willis committed May 6, 2021
    Configuration menu
    Copy the full SHA
    65670fa View commit details
    Browse the repository at this point in the history

Commits on May 13, 2021

  1. Added compression level as argument to compress_ZSTD.

    @james-s-willis
    james-s-willis committed May 13, 2021
    Configuration menu
    Copy the full SHA
    930a3b3 View commit details
    Browse the repository at this point in the history

  2. Retrieve compression level from cd_values array.

    @james-s-willis
    james-s-willis committed May 13, 2021
    Configuration menu
    Copy the full SHA
    a82fdb5 View commit details
    Browse the repository at this point in the history

Commits on May 17, 2021

  1. Added extra source files for ZSTD.

    @james-s-willis
    james-s-willis committed May 17, 2021
    Configuration menu
    Copy the full SHA
    e933d0f View commit details
    Browse the repository at this point in the history

  2. Include all source files from zstd/common, zstd/compress and zstd/dec… 

    …ompress.
    @james-s-willis
    james-s-willis committed May 17, 2021
    Configuration menu
    Copy the full SHA
    30bbe72 View commit details
    Browse the repository at this point in the history

Commits on May 18, 2021

  1. Added H5_COMPRESS_ZSTD constant for ZSTD compression.

    @james-s-willis
    james-s-willis committed May 18, 2021
    Configuration menu
    Copy the full SHA
    c31023f View commit details
    Browse the repository at this point in the history

  2. Test ZSTD compression with h5filter.

    @james-s-willis
    james-s-willis committed May 18, 2021
    Configuration menu
    Copy the full SHA
    d729fd1 View commit details
    Browse the repository at this point in the history

  3. Fixed check for ZSTD decompression.

    @james-s-willis
    james-s-willis committed May 18, 2021
    Configuration menu
    Copy the full SHA
    c4eb82e View commit details
    Browse the repository at this point in the history

Commits on May 19, 2021

  1. Fix bitshuffle __version__ string. Update bitshuffle version to 0.3.6… 

    … in C header.
    @james-s-willis
    james-s-willis committed May 19, 2021
    Configuration menu
    Copy the full SHA
    40efa63 View commit details
    Browse the repository at this point in the history

  2. Added ZSTD compression to regression tests.

    @james-s-willis
    james-s-willis committed May 19, 2021
    Configuration menu
    Copy the full SHA
    7979cb1 View commit details
    Browse the repository at this point in the history

  3. Removed old test data and replaced with lz4 and zstd specific files.

    @james-s-willis
    james-s-willis committed May 19, 2021
    Configuration menu
    Copy the full SHA
    b65519f View commit details
    Browse the repository at this point in the history

  4. Update documentation and remove debug.

    @james-s-willis
    james-s-willis committed May 19, 2021
    Configuration menu
    Copy the full SHA
    a1dc362 View commit details
    Browse the repository at this point in the history

  5. Re-added old regression test data.

    @james-s-willis
    james-s-willis committed May 19, 2021
    Configuration menu
    Copy the full SHA
    e7220f4 View commit details
    Browse the repository at this point in the history

  6. Removed lz4 and zstd specific test files.

    @james-s-willis
    james-s-willis committed May 19, 2021
    Configuration menu
    Copy the full SHA
    b02bd98 View commit details
    Browse the repository at this point in the history

  7. Add ZSTD compressed data as another dataset to regression test file.

    @james-s-willis
    james-s-willis committed May 19, 2021
    Configuration menu
    Copy the full SHA
    392d234 View commit details
    Browse the repository at this point in the history

  8. New 0.3.6 regression test file.

    @james-s-willis
    james-s-willis committed May 19, 2021
    Configuration menu
    Copy the full SHA
    3838e0a View commit details
    Browse the repository at this point in the history

Commits on May 20, 2021

  1. Update README with ZSTD comments.

    @james-s-willis
    james-s-willis committed May 20, 2021
    Configuration menu
    Copy the full SHA
    1646cbb View commit details
    Browse the repository at this point in the history

Commits on May 26, 2021

  1. Merge branch 'master' into jsw/zstd-support

    @james-s-willis
    james-s-willis committed May 26, 2021
    Configuration menu
    Copy the full SHA
    910eefa View commit details
    Browse the repository at this point in the history

  2. Formatting.

    @james-s-willis
    james-s-willis committed May 26, 2021
    Configuration menu
    Copy the full SHA
    877dab4 View commit details
    Browse the repository at this point in the history

  3. Missed HEAD from merge.

    @james-s-willis
    james-s-willis committed May 26, 2021
    Configuration menu
    Copy the full SHA
    5a65422 View commit details
    Browse the repository at this point in the history

  4. Formatting.

    @james-s-willis
    james-s-willis committed May 26, 2021
    Configuration menu
    Copy the full SHA
    07c77e9 View commit details
    Browse the repository at this point in the history

  5. Formatting.

    @james-s-willis
    james-s-willis committed May 26, 2021
    Configuration menu
    Copy the full SHA
    928a20a View commit details
    Browse the repository at this point in the history

  6. Removed duplicate variable.

    @james-s-willis
    james-s-willis committed May 26, 2021
    Configuration menu
    Copy the full SHA
    7e51067 View commit details
    Browse the repository at this point in the history

  7. Move new regression test data.

    @james-s-willis
    james-s-willis committed May 26, 2021
    Configuration menu
    Copy the full SHA
    eb7e85c View commit details
    Browse the repository at this point in the history

Commits on May 31, 2021

  1. Remove ZSTD source.

    @james-s-willis
    james-s-willis committed May 31, 2021
    Configuration menu
    Copy the full SHA
    1e77673 View commit details
    Browse the repository at this point in the history

  2. Add ZSTD as a submodule.

    @james-s-willis
    james-s-willis committed May 31, 2021
    Configuration menu
    Copy the full SHA
    21467ba View commit details
    Browse the repository at this point in the history

  3. New path for zstd library.

    @james-s-willis
    james-s-willis committed May 31, 2021
    Configuration menu
    Copy the full SHA
    ec69b88 View commit details
    Browse the repository at this point in the history

  4. Fix bitshuffle __version__ string.

    @james-s-willis
    james-s-willis committed May 31, 2021
    Configuration menu
    Copy the full SHA
    18a06ce View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2021

  1. Update src/bshuf_h5filter.c 

    Co-authored-by: Richard Shaw <richard@phas.ubc.ca>
    @james-s-willis @jrs65
    james-s-willis and jrs65 committed Aug 5, 2021
    Configuration menu
    Copy the full SHA
    f4dc23c View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into jsw/zstd-sub

    @james-s-willis
    james-s-willis committed Aug 5, 2021
    Configuration menu
    Copy the full SHA
    4d0a88e View commit details
    Browse the repository at this point in the history

  3. Add nogil to ZSTD functions.

    @james-s-willis
    james-s-willis committed Aug 5, 2021
    Configuration menu
    Copy the full SHA
    d832710 View commit details
    Browse the repository at this point in the history

  4. Updated Bitshuffle minor version to 0.4.0.

    @james-s-willis
    james-s-willis committed Aug 5, 2021
    Configuration menu
    Copy the full SHA
    b370026 View commit details
    Browse the repository at this point in the history

  5. Formatting.

    @james-s-willis
    james-s-willis committed Aug 5, 2021
    Configuration menu
    Copy the full SHA
    2cb8bdb View commit details
    Browse the repository at this point in the history

  6. Pull in ZSTD repo when running CI.

    @james-s-willis
    james-s-willis committed Aug 5, 2021
    Configuration menu
    Copy the full SHA
    83df9d0 View commit details
    Browse the repository at this point in the history

  7. Update regression file version.

    @james-s-willis
    james-s-willis committed Aug 5, 2021
    Configuration menu
    Copy the full SHA
    3282095 View commit details
    Browse the repository at this point in the history

  8. Fixed typo in name of 'origional' dataset inside regression data files.

    @james-s-willis
    james-s-willis committed Aug 5, 2021
    Configuration menu
    Copy the full SHA
    8c245e3 View commit details
    Browse the repository at this point in the history

Commits on Aug 10, 2021

  1. Bump version to 0.4.0 in setup.py

    @james-s-willis
    james-s-willis committed Aug 10, 2021
    Configuration menu
    Copy the full SHA
    00f897b View commit details
    Browse the repository at this point in the history

  2. Add instructions for pulling the ZSTD repo.

    @james-s-willis
    james-s-willis committed Aug 10, 2021
    Configuration menu
    Copy the full SHA
    1314cd7 View commit details
    Browse the repository at this point in the history

  3. Make ZSTD build optional. Enable with --zstd to install command.

    @james-s-willis
    james-s-willis committed Aug 10, 2021
    Configuration menu
    Copy the full SHA
    ef01823 View commit details
    Browse the repository at this point in the history

Commits on Aug 11, 2021

  1. Provide --zstd option when building bitshuffle.

    @james-s-willis
    james-s-willis committed Aug 11, 2021
    Configuration menu
    Copy the full SHA
    7fcb0f1 View commit details
    Browse the repository at this point in the history

  2. Formatting.

    @james-s-willis
    james-s-willis committed Aug 11, 2021
    Configuration menu
    Copy the full SHA
    7e33d74 View commit details
    Browse the repository at this point in the history

Commits on Aug 12, 2021

  1. Only define ZSTD_SUPPORT when --zstd is used.

    @james-s-willis
    james-s-willis committed Aug 12, 2021
    Configuration menu
    Copy the full SHA
    7149779 View commit details
    Browse the repository at this point in the history

  2. Ignore __init__.py when using flake8.

    @james-s-willis
    james-s-willis committed Aug 12, 2021
    Configuration menu
    Copy the full SHA
    6d44c8d View commit details
    Browse the repository at this point in the history

Commits on Aug 13, 2021

  1. Enable ZSTD support when building wheel using environment variable.

    @james-s-willis
    james-s-willis committed Aug 13, 2021
    Configuration menu
    Copy the full SHA
    e865764 View commit details
    Browse the repository at this point in the history

  2. Use inbuilt CIBUILDWHEEL for plugin test instead of user defined CI_B… 

    …UILD_WHEEL. Define ENABLE_ZSTD in CIBW_ENVIRONMENT.
    @james-s-willis
    james-s-willis committed Aug 13, 2021
    Configuration menu
    Copy the full SHA
    c9ab26f View commit details
    Browse the repository at this point in the history

  3. Flake8.

    @james-s-willis
    james-s-willis committed Aug 13, 2021
    Configuration menu
    Copy the full SHA
    a75a165 View commit details
    Browse the repository at this point in the history

  4. Only run certain unit tests when ZSTD support is present.

    @james-s-willis
    james-s-willis committed Aug 13, 2021
    Configuration menu
    Copy the full SHA
    abe0cbb View commit details
    Browse the repository at this point in the history

  5. Build with ZSTD when running unit tests.

    @james-s-willis
    james-s-willis committed Aug 13, 2021
    Configuration menu
    Copy the full SHA
    4a98bf3 View commit details
    Browse the repository at this point in the history

  6. Flake8.

    @james-s-willis
    james-s-willis committed Aug 13, 2021
    Configuration menu
    Copy the full SHA
    f5c6de1 View commit details
    Browse the repository at this point in the history

  7. Flake8.

    @james-s-willis
    james-s-willis committed Aug 13, 2021
    Configuration menu
    Copy the full SHA
    3596158 View commit details
    Browse the repository at this point in the history