fix reading of byte-swapped input files (#95) #101
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When reading a byte-swapped file, the input is grouped to 4-byte words and each of them is swapped individually. When we try to read such a file, we first validate its header using zfp_read_header with the ZFP_HEADER_MAGIC flag. This flag causes it to only validate the first word to be "zfp\x05". If it is not exactly that, it gives up. Unfortunately, this magic word can already be swapped. The actual byte swapping code would only be tried once the full header would fail to read, so automatic byte swapping never worked.
Instead, when encountering a header with bad magic, try swapping it already and only try reading the full header once the magic (normal or swapped) has been read successfully.
Thanks to Mark C. Miller, Peter Lindstrom and Enrico Zini for doing most of the debugging to get here.