Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Group list members optimization when scanning keys #57

Open
alimanfoo opened this issue Sep 6, 2016 · 3 comments
Open

Group list members optimization when scanning keys #57

alimanfoo opened this issue Sep 6, 2016 · 3 comments
Labels
help wanted Issue could use help from someone with familiarity on the topic performance Potential issues with Zarr performance (I/O, memory, etc.)

Comments

@alimanfoo
Copy link
Member

When a store does not implement listdir and keys need to be scanned, there is a possible optimization for listing members of a group because child arrays and groups could be discovered within the key scan, rather than requiring additional __contains__ tests.

@alimanfoo alimanfoo added the performance Potential issues with Zarr performance (I/O, memory, etc.) label Nov 21, 2017
@joshmoore joshmoore added the help wanted Issue could use help from someone with familiarity on the topic label Sep 22, 2021
@marynaggita
Copy link

marynaggita commented Oct 19, 2022

Hi @joshmoore, I am Mary, an outreachy candidate. Iam interested in this issue.

@marynaggita
Copy link

Kindly provide pointers which i can look at

@joshmoore
Copy link
Member

@marynaggita: the listdir() method that is being referred to can be found here: https://github.com/zarr-developers/zarr-python/blob/main/zarr/storage.py#L214-L220

It uses _listdir_from_keys() method here:
https://github.com/zarr-developers/zarr-python/blob/main/zarr/_storage/store.py#L409

Maybe the first thing to do is to find a test that uses the listdir method.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Issue could use help from someone with familiarity on the topic performance Potential issues with Zarr performance (I/O, memory, etc.)
Projects
None yet
Development

No branches or pull requests

3 participants