Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fletcher32 checksum #34

Open
tedhabermann opened this issue Jun 24, 2021 · 3 comments
Open

fletcher32 checksum #34

tedhabermann opened this issue Jun 24, 2021 · 3 comments

Comments

@tedhabermann
Copy link
Contributor

All datasets with fletcher32 == True are rejected by referenceMaker. This is the checksum supported by HDF5.
Would it be possible to add it to nomcodecs? @ajelenak

@ajelenak
Copy link
Collaborator

I think so. numcodecs already supports Adler32 which is similar to Flether32, I think.

@martindurant
Copy link
Member

The code looks simple enough.

The question is, how is this value included in the target bytes? Is it just in metadata around the bytes? If yes, then we can already ignore the checksum and use the bytes; but implementing the function above would enable the (optional) validation.

@martindurant
Copy link
Member

martindurant commented Jun 25, 2021

Example code of how to fetch the checksums: https://forum.hdfgroup.org/t/accessing-fletcher32-checksum/7299/3

Apparently it's just the last 4 bytes of the given block. So to ignore, we can just truncate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants