Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

N5FSStore #793

Merged
merged 45 commits into from
Sep 19, 2021
Merged

N5FSStore #793

merged 45 commits into from
Sep 19, 2021

Commits on Jun 14, 2021

  1. Drop skip_if_nested_chunks from test_storage.py

    @joshmoore
    joshmoore committed Jun 14, 2021
    Configuration menu
    Copy the full SHA
    d26923a View commit details
    Browse the repository at this point in the history

  2. Add failing nested test

    @joshmoore
    joshmoore committed Jun 14, 2021
    Configuration menu
    Copy the full SHA
    c06476d View commit details
    Browse the repository at this point in the history

  3. Make DirectoryStore dimension_separator aware

    @joshmoore
    joshmoore committed Jun 14, 2021
    Configuration menu
    Copy the full SHA
    ce8b2f0 View commit details
    Browse the repository at this point in the history

  4. Migrate key logic to core rather than storage 

    Previous tests (now commented out) used logic in the store
classes to convert "0/0" keys into "0.0" keys, forcing the
store to be aware of array details. This tries to swap the
logic so that stores are responsible for passing dimension
separator values down to the arrays only. Since arrays can
also get the dimension_separator value from a .zarray file
they are now in charge.
    @joshmoore
    joshmoore committed Jun 14, 2021
    Configuration menu
    Copy the full SHA
    e183566 View commit details
    Browse the repository at this point in the history

  5. Fix linting in new test

    @joshmoore
    joshmoore committed Jun 14, 2021
    Configuration menu
    Copy the full SHA
    449a67f View commit details
    Browse the repository at this point in the history

Commits on Jun 16, 2021

  1. Merge 'origin/master' into fix-dstore

    @joshmoore
    joshmoore committed Jun 16, 2021
    Configuration menu
    Copy the full SHA
    10c874e View commit details
    Browse the repository at this point in the history

Commits on Jun 17, 2021

  1. Extend the test suite for dim_sep

    @joshmoore
    joshmoore committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    2e4f4d7 View commit details
    Browse the repository at this point in the history

Commits on Jul 5, 2021

  1. add n5fsstore and tests

    @d-v-b
    d-v-b committed Jul 5, 2021
    Configuration menu
    Copy the full SHA
    8660fa5 View commit details
    Browse the repository at this point in the history

  2. resolve merge conflicts with main

    @d-v-b
    d-v-b committed Jul 5, 2021
    Configuration menu
    Copy the full SHA
    3b341ed View commit details
    Browse the repository at this point in the history

Commits on Jul 6, 2021

  1. slightly smarter kwarg interception

    @d-v-b
    d-v-b committed Jul 6, 2021
    Configuration menu
    Copy the full SHA
    bb1121c View commit details
    Browse the repository at this point in the history

  2. remove outdated unittest ref and fix the name of a test func

    @d-v-b
    d-v-b committed Jul 6, 2021
    Configuration menu
    Copy the full SHA
    be8f37f View commit details
    Browse the repository at this point in the history

  3. fix massive string block and fix default key_separator kwarg for FSStore

    @d-v-b
    d-v-b committed Jul 6, 2021
    Configuration menu
    Copy the full SHA
    95b2573 View commit details
    Browse the repository at this point in the history

  4. flake8

    @d-v-b
    d-v-b committed Jul 6, 2021
    Configuration menu
    Copy the full SHA
    ceba78d View commit details
    Browse the repository at this point in the history

  5. promote n5store to toplevel import and fix examples in docstring

    @d-v-b
    d-v-b committed Jul 6, 2021
    Configuration menu
    Copy the full SHA
    02ea91c View commit details
    Browse the repository at this point in the history

Commits on Aug 17, 2021

  1. Merge branch 'master' into fix-dstore

    @joshmoore
    joshmoore committed Aug 17, 2021
    Configuration menu
    Copy the full SHA
    cb62c10 View commit details
    Browse the repository at this point in the history

  2. Try fsspec 2021.7 (see zarr-developers#802)

    @joshmoore
    joshmoore committed Aug 17, 2021
    Configuration menu
    Copy the full SHA
    68adca5 View commit details
    Browse the repository at this point in the history

  3. Revert "Try fsspec 2021.7 (see zarr-developers#802)" 

    This reverts commit 68adca5.
    @joshmoore
    joshmoore committed Aug 17, 2021
    Configuration menu
    Copy the full SHA
    f2f75b7 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' into n5fsstore

    @d-v-b
    d-v-b committed Aug 17, 2021
    Configuration menu
    Copy the full SHA
    930a821 View commit details
    Browse the repository at this point in the history

  5. Add missing core tests for N5FSStore, and rchanges required for makin… 

    …g them pass
    @d-v-b
    d-v-b committed Aug 17, 2021
    Configuration menu
    Copy the full SHA
    a57b3bc View commit details
    Browse the repository at this point in the history

  6. Merge branch 'fix-dstore' of https://github.com/joshmoore/zarr-python … 

    …into n5fsstore
    @d-v-b
    d-v-b committed Aug 17, 2021
    Configuration menu
    Copy the full SHA
    9bb058f View commit details
    Browse the repository at this point in the history

Commits on Aug 18, 2021

  1. tmp: debug

    @joshmoore
    joshmoore committed Aug 18, 2021
    Configuration menu
    Copy the full SHA
    ee9cdbc View commit details
    Browse the repository at this point in the history

  2. uncomment N5 chunk ordering test

    @d-v-b
    d-v-b committed Aug 18, 2021
    Configuration menu
    Copy the full SHA
    a853a29 View commit details
    Browse the repository at this point in the history

  3. more commented tests get uncommented

    @d-v-b
    d-v-b committed Aug 18, 2021
    Configuration menu
    Copy the full SHA
    7d3c879 View commit details
    Browse the repository at this point in the history

  4. add dimension_separator to array metadata adaptor

    @d-v-b
    d-v-b committed Aug 18, 2021
    Configuration menu
    Copy the full SHA
    f3ecd79 View commit details
    Browse the repository at this point in the history

Commits on Aug 19, 2021

  1. Merge branch 'fix-dstore' into pr-793+773

    @joshmoore
    joshmoore committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    2d3d286 View commit details
    Browse the repository at this point in the history

  2. Revert "tmp: debug" 

    This reverts commit ee9cdbc.
    @joshmoore
    joshmoore committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    5a105eb View commit details
    Browse the repository at this point in the history

  3. Attempt failed: keeping '.' and switching

    @joshmoore
    joshmoore committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    51b3109 View commit details
    Browse the repository at this point in the history

  4. Revert "Attempt failed: keeping '.' and switching" 

    This reverts commit 51b3109.
    @joshmoore
    joshmoore committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    aa75c98 View commit details
    Browse the repository at this point in the history

  5. regex: attempt failed due to slight diff in files

    @joshmoore
    joshmoore committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    3daea7c View commit details
    Browse the repository at this point in the history

  6. Revert "regex: attempt failed due to slight diff in files" 

    This reverts commit 3daea7c.
    @joshmoore
    joshmoore committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    ce8a79e View commit details
    Browse the repository at this point in the history

  7. N5: use "." internally for dimension separation 

    This allows N5 to detect the split between key and chunks
and pre-process them (re-ordering and changing the separator).

see: zarr-developers#773 zarr-developers#793
    @joshmoore
    joshmoore committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    985c2a4 View commit details
    Browse the repository at this point in the history

  8. move FSSpec import guard

    @d-v-b
    d-v-b committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    51836df View commit details
    Browse the repository at this point in the history

  9. remove os.path.sep concatenation in listdir that was erroring a test,… 

    … and add a mea culpa docstring about the dimension_separator for n5 stores
    @d-v-b
    d-v-b committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    3c5da2f View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2021

  1. resolve merge conflicts in favor of upstream

    @d-v-b
    d-v-b committed Sep 13, 2021
    Configuration menu
    Copy the full SHA
    eea4aaa View commit details
    Browse the repository at this point in the history

  2. resolve merge conflicts in favor of upstream

    @d-v-b
    d-v-b committed Sep 13, 2021
    Configuration menu
    Copy the full SHA
    b8fe803 View commit details
    Browse the repository at this point in the history

Commits on Sep 14, 2021

  1. make listdir implementation for n5fsstore look more like fsstore's li… 

    …stdir, and add crucial lstrip
    @d-v-b
    d-v-b committed Sep 14, 2021
    Configuration menu
    Copy the full SHA
    8fec1d6 View commit details
    Browse the repository at this point in the history

Commits on Sep 15, 2021

  1. Update hexdigest tests for N5Stores to account for the presence of th… 

    …e dimension_separator keyword now present in metadata
    @d-v-b
    d-v-b committed Sep 15, 2021
    Configuration menu
    Copy the full SHA
    46ebb44 View commit details
    Browse the repository at this point in the history

  2. Add tests for dimension_separator in array meta for N5Stores

    @d-v-b
    d-v-b committed Sep 15, 2021
    Configuration menu
    Copy the full SHA
    864773d View commit details
    Browse the repository at this point in the history

Commits on Sep 17, 2021

  1. N5FSStore: try to increase code coverage 

     * Adds a test for the dimension_separator warning
 * uses the parent test_complex for listdir
 * "nocover" the import error since fsspec is ever present
    @joshmoore
    joshmoore committed Sep 17, 2021
    Configuration menu
    Copy the full SHA
    3b56155 View commit details
    Browse the repository at this point in the history

  2. flake8

    @d-v-b
    d-v-b committed Sep 17, 2021
    Configuration menu
    Copy the full SHA
    b0f6d33 View commit details
    Browse the repository at this point in the history

  3. add chunk nesting test to N5FSStore test suite

    @d-v-b
    d-v-b committed Sep 17, 2021
    Configuration menu
    Copy the full SHA
    82ce89f View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' into n5fsstore

    @joshmoore
    joshmoore committed Sep 17, 2021
    Configuration menu
    Copy the full SHA
    267c744 View commit details
    Browse the repository at this point in the history

  5. make array_meta_key, group_meta_key, attrs_key private

    @d-v-b
    d-v-b committed Sep 17, 2021
    Configuration menu
    Copy the full SHA
    2b85410 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'n5fsstore' of https://github.com/d-v-b/zarr-python into… 

    … n5fsstore
    @d-v-b
    d-v-b committed Sep 17, 2021
    Configuration menu
    Copy the full SHA
    8bd6c41 View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2021

  1. N5FSStore: Remove ImportError test 

    FSStore only throws ModuleNotFoundError on initialization
rather than on import. Therefore N5FSStore does the same.
If this *weren't* the case, then the import in zarr/init
would need to test the import as well, which isn't the case.
    @joshmoore
    joshmoore committed Sep 19, 2021
    Configuration menu
    Copy the full SHA
    aa4a723 View commit details
    Browse the repository at this point in the history