Home

lance-nd stores N-dimensional Arrow arrays in a Lance table, one nd record batch per row. Use it from Rust, or from Python with xarray datasets. The README gives a short start.

Concepts

These pages apply to Rust and to Python.

Page	Contents
Table layout	The nd-arrow data model, the columns of a table, schema evolution, rows and chunks.
Object stores	S3, Google Cloud Storage and Azure, and the S3 tests.

Rust

The lance-nd crate. The API documentation is on docs.rs.

Page	Contents
Writing	The write functions, NdWriter, errors, fragments, casting, widen and column encoding.
Reading and filters	Stats, pruning and the three filter APIs.
Delete, compact and versions	Delete rows, compact fragments, free disk space and restore versions.

Python

The lance_nd package on PyPI.

Page	Contents
Python	Install, a short example and the chapters.
Getting started	Your first table in a few minutes.
Create a table	A new table, stats, compression, overwrite and fragment size.
Add datasets	Append datasets, many files, errors, new variables, type casts and dask chunks.
Read datasets	Scan, select variables, filter on attributes and values, prune with stats.
Delete datasets	Delete one dataset or many datasets with one filter.
Replace datasets	Delete the old datasets and add the new datasets.
Compact a table	Merge small fragments and remove deleted rows.
Versions and disk space	Read earlier versions, undo a change and free disk space.
Maintain a collection	A routine for a collection that gets new files each day.
API reference	Each function, option, error and encoding rule.

Specifications

• FORMAT.md for the lance-nd table.
• The nd-arrow FORMAT.md for the nd-arrow encoding.