Connecting with the Julia Community

[jakirkham]

Appears there are Julia users, who are discussing how to get a Zarr implementation going for Julia. Would be good to connect with them and see how we can help them get a Julia implementation going.

[hearnsj]

I will help with testing.

[alimanfoo]

Thank you @hearnsj. FWIW I was thinking it might be useful to have a go at creating a notebook using a Julia kernel that tries to adapt the code examples in the zarr tutorial to Julia, with calls to zarr via pycall. This should throw up any compatibility or usability issues. Another useful activity would be to benchmark some simple read and write operations in both Python and in Julia via pycall, to get a sense of if there are any major performance issues.

[meggart]

This sounds great and I would be highly interested and would like to support this initiative. I have started a very rudimentary wrapper using PyCall which allows me to read from Zarr arrays as Julia arrays, https://github.com/meggart/Zarr.jl . This was basically just for my own reading benchmarks to decide if the format would work for me, so sorry for missing tests/docs. Basically you can just run zopen(path) on any zarr group or variable and then use Julia indexing (supports ranges and colons) to read slices from the dataset.

What is really missing because I did not need it is the creation and writing of datasets. I will see if I can add a few functions soon, so that one can repeat the tutorial.

[alimanfoo]

Fabian, very cool! Look forward to hearing more.

…

On Fri, 10 Aug 2018, 12:24 Fabian Gans, ***@***.***> wrote: This sounds great and I would be highly interested and would like to support this initiative. I have started a very rudimentary wrapper using PyCall which allows me to read from Zarr arrays as Julia arrays, https://github.com/meggart/Zarr.jl . This was basically just for my own reading benchmarks to decide if the format would work for me, so sorry for missing tests/docs. Basically you can just run zopen(path) on any zarr group or variable and then use Julia indexing (supports ranges and colons) to read slices from the dataset. What is really missing because I did not need it is the creation and writing of datasets. I will see if I can add a few functions soon, so that one can repeat the tutorial. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <https://github.com/zarr-developers/zarr/issues/284#issuecomment-412054853>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAq8QraTS-Ig26WPSdhRDh-vwGwI5KAbks5uPW1XgaJpZM4Vo9eR> .

[jakirkham]

Indeed very cool. Please let us know if there is any way we can help.

[rabernat]

Should also point out that @meggart has a newer, julia-native implementation of zarr: https://github.com/meggart/ZarrNative.jl

[rabernat]

I'm pinging this issue to check in on the status of zarr + julia. We now have julia kernels working in pangeo environments (see e.g. https://github.com/pangeo-data/pangeo_ocean_examples). The next step for interoperability is to be able to load data from zarr stores in Google Cloud Storage.

Could someone (probably @meggart) give an update on where the zarr julia packages stand? I checked the repos and didn't see much recent activity. Are there any roadblocks we can help overcome?

[visr]

Regarding ZarrNative.jl, there is a basis there, but it still needs more work. I think the main thing holding it back is simply developer time. Thanks for the offer for help though, I'm sure we will be needing it.

There is no Google Cloud Storage support yet, it's probably best to get to a more solid base, testing with local stores first.

[alimanfoo]

@meggart, @visr, I just wanted to let you know that we've started up a regular conference call as a way of building connections within the zarr and n5 community, it's currently fairly small but includes folks from groups implementing libraries in various languages as well as folks representing user communities. I know bandwidth is short but if you had any time to attend you would be very welcome. Details of the call are over on #315.

[meggart]

Thanks for the info, I have already followed that thread and I think I will join one of the next meetings.

[meggart]

Just to let you know, we have released a first version of Zarr.jl and registered them in the official package channels. It does not yet implement the full Zarr spec, many compressors and backends are missing, but this may come over time as needed. The main focus for now will be on extending documentation, in particular finish translating the Tutorial.

[alimanfoo]

Fantastic news, and perfect timing!

…

On Tue, 9 Jul 2019, 09:55 Fabian Gans, ***@***.***> wrote: Just to let you know, we have released a first version of Zarr.jl <https://github.com/meggart/Zarr.jl> and registered them in the official package channels. It does not yet implement the full Zarr spec, many compressors and backends are missing, but this may come over time as needed. The main focus for now will be on extending documentation, in particular finish translating the Tutorial <https://meggart.github.io/Zarr.jl/latest/tutorial/>. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#17?email_source=notifications&email_token=AAFLYQTYEV2ZGZRAWU32BF3P6RHBLA5CNFSM4H5MBZ52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZPTAWY#issuecomment-509554779>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAFLYQTCBYEXM5ASUQRYQMTP6RHBLANCNFSM4H5MBZ5Q> .