Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Foreign b2nd array compatibility #1072

Merged
merged 12 commits into from Oct 19, 2023
Merged

Foreign b2nd array compatibility #1072

merged 12 commits into from Oct 19, 2023

Commits on Oct 17, 2023

  1. Configuration menu
    Copy the full SHA
    5d79cd4 View commit details
    Browse the repository at this point in the history
  2. Mention B2ND in errors when reading chunk slices from the file

    To avoid confusion with the HDF5 array dataset itself.
    ivilata committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    6051655 View commit details
    Browse the repository at this point in the history
  3. Note on why a b2nd-specific case is needed in filter decompression

    Thanks to @martaiborra for the clarification.
    ivilata committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    8cd8559 View commit details
    Browse the repository at this point in the history
  4. Relax filter check for shape of b2nd array used to store a chunk

    Instead of checking that the array's chunk shape matches the dataset's chunk
    shape, check that the array's whole shape matched the dataset's chunk shape.
    PyTables stores one Blosc2 chunk per HDF5 chunk, but it should be able to cope
    with Blosc2 frames containing several chunks (since reading does not really
    operate at the Blosc2 chunk level), as long as the whole Blosc2 array has the
    proper shape.
    
    This may ease having PyTables read b2nd-compressed arrays where dataset chunks
    contain several Blosc2 chunks.
    ivilata committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    44178b3 View commit details
    Browse the repository at this point in the history
  5. Check rank & shape of b2nd array when getting a slice

    These checks were already made by the filter, but still missing here.
    ivilata committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    78e1b39 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    942d2db View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    5d691de View commit details
    Browse the repository at this point in the history
  8. Relax filter checks for b2nd array shape with no aux filter values

    This increases compatibility with datasets written with other tools, esp. if
    they also use b2nd for scalar or unidimensional data, as chunk rank/shape
    filter values used by PyTables for the checks may be missing (and filter set
    function would not set them either since rank < 2).
    ivilata committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    93d61a0 View commit details
    Browse the repository at this point in the history

Commits on Oct 19, 2023

  1. Remove workaround for stack smashing/segmentation fault on GCC+Linux

    The issue seems to have vanished on Guix (GCC 12.3.0), let us see what CI says
    about Ubuntu.
    ivilata committed Oct 19, 2023
    Configuration menu
    Copy the full SHA
    e16edd5 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    c6b47e1 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    3ba4e78 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    d5536db View commit details
    Browse the repository at this point in the history