Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add API reference for V3 Implementation in the docs #1345

Merged
merged 8 commits into from Mar 17, 2023
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/api.rst
Expand Up @@ -13,6 +13,7 @@ API reference
api/codecs
api/attrs
api/sync
api/v3

Indices and tables
------------------
Expand Down
77 changes: 77 additions & 0 deletions docs/api/v3.rst
@@ -0,0 +1,77 @@
V3 Specification Implementation(``zarr._storage.v3``)
=====================================================

This module contains the implementation of the `Zarr V3 Specification <https://zarr-specs.readthedocs.io/en/latest/core/v3.0.html#zarr-core-specification-v3-0>`_.
MSanKeys963 marked this conversation as resolved.
Show resolved Hide resolved

.. warning::
Since Zarr Python 2.12 release, this module provides experimental infrastructure for reading and
writing the upcoming V3 spec of the Zarr format. Users wishing to prepare for the migration can set
the environment variable ``ZARR_V3_EXPERIMENTAL_API=1`` to begin experimenting, however data
written with this API should be expected to become stale, as the implementation will still change.

The new ``zarr._store.v3`` package has the necessary classes and functions for evaluating Zarr V3.
Since the design is not finalised, the classes and functions are not automatically imported into
the regular Zarr namespace.

Code snippet for creating Zarr V3 arrays::

>>> import zarr
>>> z = zarr.create((10000, 10000),
>>> chunks=(100, 100),
>>> dtype='f8',
>>> compressor='default',
>>> path='path-where-you-want-zarr-v3-array',
>>> zarr_version=3)

Further, you can use `z.info` to see details about the array you just created::

>>> z.info
Name : path-where-you-want-zarr-v3-array
Type : zarr.core.Array
Data type : float64
Shape : (10000, 10000)
Chunk shape : (100, 100)
Order : C
Read-only : False
Compressor : Blosc(cname='lz4', clevel=5, shuffle=SHUFFLE, blocksize=0)
Store type : zarr._storage.v3.KVStoreV3
No. bytes : 800000000 (762.9M)
No. bytes stored : 557
Storage ratio : 1436265.7
Chunks initialized : 0/10000

You can also check ``Store type`` here (which indicates Zarr V3).

.. module:: zarr._storage.v3

.. autoclass:: RmdirV3
.. autoclass:: KVStoreV3
.. autoclass:: FSStoreV3
.. autoclass:: MemoryStoreV3
.. autoclass:: DirectoryStoreV3
.. autoclass:: ZipStoreV3
.. autoclass:: RedisStoreV3
.. autoclass:: MongoDBStoreV3
.. autoclass:: DBMStoreV3
.. autoclass:: LMDBStoreV3
.. autoclass:: SQLiteStoreV3
.. autoclass:: LRUStoreCacheV3
.. autoclass:: ConsolidatedMetadataStoreV3
MSanKeys963 marked this conversation as resolved.
Show resolved Hide resolved

In v3 `storage transformers <https://zarr-specs.readthedocs.io/en/latest/core/v3.0.html#id19>`_
MSanKeys963 marked this conversation as resolved.
Show resolved Hide resolved
can be set via ``zarr.create(…, storage_transformers=[…])``.
The experimental sharding storage transformer can be tested by setting
the environment variable ``ZARR_V3_SHARDING=1``. Data written with this flag
enabled should be expected to become stale until
`ZEP 2 <https://zarr.dev/zeps/draft/ZEP0002.html>`_ is approved
and fully implemented.

.. module:: zarr._storage.v3_storage_transformers

.. autoclass:: ShardingStorageTransformer

The abstract base class for storage transformers is

.. module:: zarr._storage.store

.. autoclass:: StorageTransformer