-
Notifications
You must be signed in to change notification settings - Fork 36
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix for zarr.ZipStore #66
Conversation
Using fsspec directly does work, although the incantation s a little strange: temp_zip:
description: zarr zipstore
driver: zarr
args:
urlpath: 'zip://'
storage_options:
fo: 'temp_zip.zip'
consolidated: False Just like with |
Hi @martindurant and thanks for the help. I'm trying this out rn and I can make it work for consolidated ZipStore but not un-consolidated. Any idea about what the problem might be:
the error is:
my hunch is that it is blind to the difference between directories and files and expect to find a .zarray under .zattrs which is just a file. what do you think? |
The exact same thing did work for me, unconsolidated. Are you on windows by any chance? |
my config is: OSX Catalina |
Would you mind trying with fsspec master (or indeed the chained_mapper branch)? |
I've tried both and can't get it to work either. weird... can you post your test? |
|
great! I'm gonna call docker to the rescue to make sure it's not due to something in my config. |
Perhaps if you show me how you are calling docker, I can follow it. Obviously, it would be good to test this kind of thing here, but fsspec should be released first. Are you aware of #66 ? That might make things more consistent. |
@martindurant I'm going to create and point you to a github repo with my dockerfiles once I'm done with afternoon meetings. So far fsspec master works with the storage_option but your new branch (fsspec/filesystem_spec#282) does not work with the new syntax. |
This is progress! |
@martindurant docker tests here: https://github.com/raphaeldussin/docker_tests_zipstore.git |
The chained version doesn't seem to install the right version
In my case, to test, I did extra steps to be certain
|
(and it worked) |
@ and # keys are too close for my own good... the change to fsspec is enough for my application. This PR has no reason to be anymore. |
intake-xarray is presently not able to work with zarr.ZipStore. The issue seems to be coming from the fsspec mapping. From what I understood of fsspec mappings, I am not sure if it is able to handle local zip files or if it's desirable. The proposed fix adds an option to bypass the mapper and use the local path to the file.
problem and solution can be tested with: