Repository navigation
Python
Robin Kooyman edited this page Nov 10, 2026
·
4 revisions
lance_nd stores xarray datasets in a Lance
table, one dataset per row. You can add, read, delete and replace datasets,
and the table keeps each earlier version.
pip install lance_ndimport xarray as xr
import lance_nd
with lance_nd.create("profiles.lance", stats=True) as writer:
writer.write(xr.open_dataset("profile_001.nc"))
table = lance_nd.open("profiles.lance")
for batch in table.scan(prune="TEMP.values > 20"):
ds = batch.to_xarray()Getting started makes your first table, adds datasets and reads them back, in a few minutes.
Each chapter explains one task, with examples that you can run.
| Chapter | Contents |
|---|---|
| Create a table | A new table, stats, compression, overwrite and fragment size. |
| Add datasets | Append datasets, many files, errors, new variables, type casts and dask chunks. |
| Read datasets | Scan, select variables, filter on attributes and values, prune with stats. |
| Delete datasets | Delete one dataset or many datasets with one filter. |
| Replace datasets | Change datasets: delete the old datasets and add the new datasets. |
| Compact a table | Merge small fragments and remove deleted rows. |
| Versions and disk space | Read earlier versions, undo a change and free disk space. |
| Maintain a collection | A routine for a collection that gets new files each day. |
- API reference: each function, option, error and encoding rule.
- Object stores: S3, Google Cloud Storage and Azure.
- Table layout: how a table stores the datasets.
Concepts
Rust
Python