Zarr in the News

[joshmoore]

Similar to the ongoing conversation in "Regular zarr/n5 community conference calls" (https://github.com/zarr-developers/zarr/issues/315), this is an ongoing place to post blogs posts and another announcements about Zarr that everyone is aware of.

[joshmoore]

https://www.unidata.ucar.edu/blogs/news/entry/netcdf-and-native-cloud-storage

[alimanfoo]

Zarr Python 2.3 release on the Zarr blog.

[alimanfoo]

Zarr protocol v3 design update on the Zarr blog.

[jakirkham]

Loading image data with Dask and storing it in Zarr.

ref: https://blog.dask.org/2019/06/20/load-image-data

[joshmoore]

• https://blog.openmicroscopy.org/community/file-formats/2019/06/25/formats/
• https://twitter.com/openmicroscopy/status/1143480183158185985

[alimanfoo]

Article about how pangeo is used at CNES:

• https://medium.com/pangeo/why-and-how-we-use-pangeo-at-cnes-74553c7fb19b
• https://twitter.com/rabernat/status/1151123875922624520

Mention of Zarr:

Outside of Satellite imagery, Ifremer has used Pangeo on our cluster to analyze the outputs of a big ocean simulation (MitGCM LLC4320). One typical thing done on this simulation is the computation of time-frequency kinetic energy spectra, which basically build a big datacube over the globe and reorganize the data so as to have the full temporal axis on a given location in memory in order to perform the scientific computation on it. Xarray helps to do this in very few lines of code, and by using Zarr file format they were able to speed-up the data loading by a factor of ten compared to NetCDF.

[alimanfoo]

Zarr at SciPy 2019: talk, slides.

[alimanfoo]

An older post but still relevant, HDF in the cloud by Matt Rocklin.

[alimanfoo]

@jhamman's article about the CESM Large Ensemble which is now in the AWS Public Dataset Program in Zarr format.

[alimanfoo]

Zarr gets CZI EOSS funding!!!

[alimanfoo]

Zarr/CZI news piece on my institution's website.

[joshmoore]

Cross link between Zarr & Bioimaging EOSS funding

[joshmoore]

Zarr/N5 used to expedite transforming bio-imaging files into pyramidal OME-TIFF

[jhamman]

New climate model data now in Google Public Datasets

[ryan-williams]

@jakirkham presented this short deck at the CZI EOSS Kickoff Mtng today

[ryan-williams]

Loading NumPy arrays from disk: mmap() vs. Zarr/HDF5 is on Hacker News today (h/t @hammer)

[jhamman]

Streaming with Zarr - written by yours truly. Binder ready demo here:

[jakirkham]

That's great Joe! Thanks for sharing! How can we promote this so more people are aware of this use case? 🙂

[jhamman]

How can we promote this so more people are aware of this use case?

I think a zarr-examples repo would be an interesting way to highlight some of these more specialized use-cases. I'm thinking something like dask-examples.

[joshmoore]

Been quite for a while: I'll kick us off with https://xarray.dev/blog/xarray-kvikio 😄