Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Zarrr inconsitantly returns Nans #925

Open
taras-sereda opened this issue Dec 23, 2021 · 2 comments
Open

Zarrr inconsitantly returns Nans #925

taras-sereda opened this issue Dec 23, 2021 · 2 comments

Comments

@taras-sereda
Copy link
Contributor

Minimal, reproducible code sample, a copy-pastable example if possible

# Your code here
>>> import zarr
>>> import numpy as np
>>> path = 'example.zarr'
>>> group = zarr.open_group(path, 'r')
>>> data.info
Name               : /pred_spectrum_3
Type               : zarr.core.Array
Data type          : float32
Shape              : (80, 269819)
Chunk shape        : (80, 2500)
Order              : C
Read-only          : True
Compressor         : Blosc(cname='lz4', clevel=5, shuffle=SHUFFLE, blocksize=0)
Store type         : zarr.storage.DirectoryStore
No. bytes          : 86342080 (82.3M)
No. bytes stored   : 63233738 (60.3M)
Storage ratio      : 1.4
Chunks initialized : 97/108
>>> data =  group['pred_spectrum_3'][:, 253676 : 254056]
>>> np.argwhere(np.isnan(data))
array([[ 0, 10],
       [ 0, 11],
       [ 0, 12],
       [ 0, 13]])
>>> data.mean()
nan
>>> data_2 =  group['pred_spectrum_3'][:]
>>> data_2[:, 253676 : 254056].mean()
0.0

Problem description

While accessing data Zarr returns Nans inconsistently. The only difference is that data is a slice when data_2 is a whole 2d array, sliced latter. I expect both ways of accessing data should give exactly same results. What might be a reason of such undetermined behavior?

Version and installation information

Please provide the following:

  • Value of zarr.__version__ = 2.10.3
  • Value of numcodecs.__version__ = 0.6.4
  • Version of Python interpreter Python 3.9.7
  • Linux MANJARO
  • How Zarr was installed (e.g., "using pip into virtual environment", or "using conda"):
    Via yay package manager
yay -Qi python-zarr
Name            : python-zarr
Version         : 2.10.3-1
Description     : An implementation of chunked, compressed, N-dimensional arrays for Python
Architecture    : any
URL             : https://github.com/zarr-developers/zarr-python

Also, if you think it might be relevant, please provide the output from pip freeze or
conda env export depending on which was used to install Zarr.

@pudasainimiraj
Copy link

Hi,
I'd like to try on this issue if it's not already solved.
Many thanks :)

@joshmoore
Copy link
Member

That'd be great, @KlokMiraj. It's not solved as far as I know, but I'd definitely suggest starting by reproducing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants