Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Zstandard HDF5 plugin #93

Closed
lilastic opened this issue Oct 20, 2020 · 4 comments · Fixed by #106
Closed

Add Zstandard HDF5 plugin #93

lilastic opened this issue Oct 20, 2020 · 4 comments · Fixed by #106
Milestone
3.0.0

Comments

@lilastic
Copy link

lilastic commented Oct 20, 2020 •
edited

Would it be possible to add Zstandard (https://facebook.github.io/zstd/) in the form of an HDF5 plugin into your repository? An already possible integration can be found here: https://github.com/aparamon/HDF5Plugin-Zstandard. Thanks and keep up the good work! :)
@lilastic lilastic changed the title Adding Zstandard plugin Add Zstandard HDF5 plugin Oct 20, 2020
@t20100
Copy link
Member

t20100 commented Oct 20, 2020

Zstd is already available as part of the Blosc filter, see https://github.com/silx-kit/hdf5plugin#blosc-cname-lz4-clevel-5-shuffle-shuffle

But yes, this filter could be added.
Since Zstd is already used by the blosc filter, the sources are already included, so I don't see major issue in adding it here.

If you feel like making a Pull Request for it, you can check https://github.com/silx-kit/hdf5plugin/blob/master/doc/contribute.rst

@lilastic
Copy link
Author

Thanks for the info @t20100!

Not sure if I'm able to do it as I'm not used with the process of building filters for HDF5. Would you mind to do it? Thanks!

@t20100
Copy link
Member

t20100 commented Nov 2, 2020

I don't have time for it right now, but we'll definitely add Zstd to the filter zoo.

In the meantime, if you want to give a try we can provide support.

@vasole
Copy link
Member

vasole commented Dec 15, 2020

@lilastic

#106 adds support for Zstandard but we lack any reference data for testing.

Do you have any data already using that compression filter?

@t20100 t20100 mentioned this issue Dec 16, 2020
Merged
@t20100 t20100 added this to the 3.0.0 milestone Dec 16, 2020
t20100 added a commit to t20100/hdf5plugin that referenced this issue Nov 8, 2022
@t20100
Squashed 'src/H5Z-ZFP/' changes from 5d9a995..cd5422c
a4e000e 
cd5422c Compatibility with Visual Studio 2019 (silx-kit#93)
bcff4d2 CMake: (fix) Missing header file. (silx-kit#90)
59b7f38 updating for 1.0.0 (silx-kit#85)
3802c0f Tests: (fix) Makefile - CFLAGS and clean target (silx-kit#84)
111e5a1 Fix Silo doc section and warnings (silx-kit#83)
9dd4546 Use ZFP's version string (silx-kit#80)
557a4f6 Pin zfp version in windows ci (silx-kit#82)
f51ac59 added parameter to fortran const (silx-kit#76)
35a08e0 [github] add windows build config (silx-kit#72)
1844b5b Fixed detection of CFP availability (silx-kit#74)
c8544d3 [windows] enable compilation in windows with ClangCL (silx-kit#71)
7b34cac Update installation.rst (silx-kit#66)
e153bf9 Feature direct write zfp array (silx-kit#43)
20b0f1f Added missing HDF5 include (CMake) (silx-kit#63)
20b893a Test and fix working with HDF5-1.12 (silx-kit#62)
497e942 CMake: (fix) Autotools build HDF5 (silx-kit#59)
8f63f7d Add missing headers for string functions and fix printfs for cd_nelmts (silx-kit#60)
8de12f7 Added generic interface fortran wrappers with tests (silx-kit#58)
983a187 find zfp and hdf5 in lib64 instead of lib (silx-kit#56)
e6c9c14 Fix typos
f7670c4 Fix second typo in h5repack docs (silx-kit#55)
57a849d handle optional/mandatory flag (silx-kit#54)
48126d3 CMake: (feature) Added CMake build configuration (silx-kit#52)
bee7434 fix link to travis badge (silx-kit#51)
00146c1 update to download from github (silx-kit#49)
259feb8 Remove include for HDF5 header file (silx-kit#48)
3d0b176 adding missing call to h5z_zfp_finaliz() (silx-kit#45)
b642fe8 minor fixes to docs and h5repack utility (silx-kit#42)

git-subtree-dir: src/H5Z-ZFP
git-subtree-split: cd5422c146836e17c7a0380bfb05cf52d0c4467c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
3.0.0
Development

Successfully merging a pull request may close this issue.

Add Zstd to the supported plugin list silx-kit/hdf5plugin
3 participants
@vasole @t20100 @lilastic