Skip to content

Conversation

@BrianMichell
Copy link
Collaborator

ZFP is an array to bytes serializer and cannot be treated as a compressor directly. Changes here make that distinction opaque to the user. Also avoids creating an unnecessary default compressor on ZFP compressed stores.

@BrianMichell BrianMichell added the bug Something isn't working label Nov 10, 2025
@codecov
Copy link

codecov bot commented Nov 10, 2025

Codecov Report

❌ Patch coverage is 77.77778% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.30%. Comparing base (c3ba558) to head (b512c36).
⚠️ Report is 217 commits behind head on main.

Files with missing lines Patch % Lines
src/mdio/builder/xarray_builder.py 65.21% 6 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #748      +/-   ##
==========================================
+ Coverage   85.30%   88.30%   +2.99%     
==========================================
  Files          46       90      +44     
  Lines        2219     4952    +2733     
  Branches      306      326      +20     
==========================================
+ Hits         1893     4373    +2480     
- Misses        281      504     +223     
- Partials       45       75      +30     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@BrianMichell
Copy link
Collaborator Author

In order to append additional Variables to the Dataset (e.g. transpose volume) in Zarr V2 I had to add the following snippet right before calling to_mdio.

This should be less of an issue with this PR and something that should be addressed in the MDIO API.

    for var_name in list(ds.data_vars) + list(ds.coords):
        if "_FillValue" in ds[var_name].attrs:
            del ds[var_name].attrs["_FillValue"]

from zarr.codecs.numcodecs import ZFPY as zarr_ZFPY # noqa: N811
except ImportError:
zfpy_ZFPY = None # noqa: N816
zarr_ZFPY = None # noqa: N816

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was looking through test_dataset_serializer.py and I only found imports for zfpy.ZFPY. Are there tests for zarr.codecs.numcodecs.ZFPY?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#747 will help with this. Lossy compression has not been a high priority.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh nice. You're on it already.

@tasansal tasansal merged commit 775f680 into TGSAI:main Nov 11, 2025
9 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants