adata-cli 0.7.1

A patch release that fixes several ways subset, recast, split and concat could lose or corrupt data while reporting success. Upgrading is recommended if you use subset on Zarr stores or --inplace.

pip install -U pyadata-cli

Fixed

• subset x.zarr -o x.zarr emptied the store (#35). An output that names the input is now refused; use --inplace to replace a store.
• A failed selection left an empty output (#35). It also truncated an existing -o target. The selection is now resolved before anything is written.
• --inplace deleted the source before its replacement was in place (#35). This affected subset and recast: an interrupt at the wrong moment lost the data. The source is now renamed aside and deleted only afterwards.
• A failure part-way through left a partial store that looked finished (#35). This affected subset, recast, split and concat.
  • All four now write beside their output (out.adata-tmp.zarr) and move it into place only once it's complete, as convert already did.
  • split writes its manifest last.
  • If a run is killed outright, the next run refuses to start until the leftover *.adata-tmp* or *.adata-old* is dealt with, rather than deleting what may be the only copy.
• concat wrote Zarr v3 whatever its inputs' version (#36). It now keeps the version its Zarr inputs share. --zarr-format still overrides it.
• subset and concat now check --zarr-format themselves (#37), and six --help examples that still said h5ad are corrected (#38). Both contributed by @iamcipherdev, who opened their first pull requests here: thank you.

Benchmarks

The recast benchmark now narrows a float64/int64 store, and reports file size before and after. Narrowing halves an uncompressed X. On an lzf- or gzip-compressed X it saves only a few percent on disk, because the compressor had already removed the zero bytes; the halving shows up in memory once the matrix is loaded.

Full changelog: CHANGELOG.md

Benchmark vs anndata/scanpy

Results — 0.7.1

Tier	ci, 50,000 obs x 20,000 var, 5% dense CSR
Var-heavy shape	2,000 obs x 36,601 var
Page cache	dropped between runs
Address-space ceiling	12.9 GB per process
Repeats	3 (fastest shown)
Platform	Linux x86_64, Python 3.12.3
Versions	adata-cli 0.7.1, anndata 0.13.4, scanpy 1.12.4
Generated	2026-10-15T14:55:32Z

startup

What does each contender cost before it does any work?

Contender	Wall time	Peak RSS	Output	vs previous
adata-cli	1.30 s	76.1 MB	-	time +108%, RSS +12%
anndata	3.65 s	133.4 MB	-	time +121%
scanpy	5.81 s	260.4 MB	-	time +62%

concat-inner

Concatenate two stores on the obs axis, inner join.

Contender	Wall time	Peak RSS	Output	vs previous
adata-cli	15.37 s	172.4 MB	583.3 MB	time +65%
anndata (concat_on_disk)	8.35 s	449.9 MB	831.9 MB	time +41%
anndata (in memory)	10.20 s	1.8 GB	580.3 MB	time +12%
scanpy	n/a	n/a	n/a	-
	scanpy has no concat of its own; it re-exports anndata's

concat-merge-same

Concatenate carrying var columns forward -- the REQ-71798 case.

Contender	Wall time	Peak RSS	Output	vs previous
adata-cli	1.83 s	193.5 MB	45.7 MB	time +22%
anndata (concat_on_disk)	3.83 s	246.2 MB	63.2 MB	time +71%
anndata (in memory)	3.90 s	289.0 MB	44.6 MB	time +61%

subset-query

Keep the obs rows matching a predicate on an existing column.

Contender	Wall time	Peak RSS	Output	vs previous
adata-cli	5.30 s	193.4 MB	147.1 MB	time +58%
anndata (backed)	6.98 s	674.9 MB	146.6 MB	time +64%
anndata (in memory)	7.77 s	985.1 MB	146.6 MB	time +73%
scanpy (filter_cells)	11.88 s	1.6 GB	291.2 MB	time +54%

h5ad-to-zarr

Convert a store to Zarr.

Contender	Wall time	Peak RSS	Output	vs previous
adata-cli	10.21 s	121.9 MB	151.6 MB in 4717 files	time +66%, RSS -21%
anndata (in memory)	7.97 s	775.5 MB	136.8 MB in 55 files	time +40%
scanpy	n/a	n/a	n/a	-
	no streaming converter; scanpy defers to anndata

inspect

Report what is in the store, without reading the matrix.

Contender	Wall time	Peak RSS	Output	vs previous
adata-cli	1.21 s	69.7 MB	-	time +98%
anndata (read_elem on obs)	3.72 s	160.7 MB	-	time +111%
anndata (full load)	4.68 s	584.0 MB	-	time +47%

export-obs

Write the obs table out as CSV.

Contender	Wall time	Peak RSS	Output	vs previous
adata-cli	1.66 s	83.2 MB	1.8 MB	time +116%
anndata (read_elem on obs)	3.95 s	161.1 MB	1.8 MB	time +122%
anndata (full load)	9.18 s	584.4 MB	1.8 MB	time +185%

split-by-sample

Write one store per distinct value of an obs column.

Contender	Wall time	Peak RSS	Output	vs previous
adata-cli	7.63 s	197.3 MB	302.6 MB in 9 files	time +53%
anndata (hand-written loop)	6.32 s	684.6 MB	301.7 MB in 8 files	no change

ls

List everything in the store.

Contender	Wall time	Peak RSS	Output	vs previous
adata-cli	0.86 s	68.9 MB	-	time +37%
h5py (visit)	1.81 s	136.5 MB	-	no change
h5ls -r	n/a	n/a	n/a	-
	h5ls is not installed

create

Write an empty store with a given obs/var shape.

Contender	Wall time	Peak RSS	Output	vs previous
adata-cli	1.32 s	82.5 MB	3.4 MB	time +101%
anndata	8.44 s	226.1 MB	49.2 MB	no change

export-array

Write a dense element out as .npy.

Contender	Wall time	Peak RSS	Output	vs previous
adata-cli	1.33 s	81.3 MB	10.0 MB	time +101%
anndata (read_elem)	4.05 s	154.8 MB	10.0 MB	time +142%
anndata (full load)	5.90 s	584.4 MB	10.0 MB	time +83%

export-sparse

Write X out as Matrix Market text.

Contender	Wall time	Peak RSS	Output	vs previous
adata-cli	35.18 s	74.6 MB	761.3 MB	time +66%
anndata (sparse_dataset)	12.00 s	968.5 MB	661.6 MB	time +84%
anndata (full load)	14.57 s	773.4 MB	661.6 MB	time +117%

import-dataframe

Replace obs from a CSV.

Contender	Wall time	Peak RSS	Output	vs previous
adata-cli	2.27 s	112.3 MB	296.0 MB	time +16%
anndata	6.91 s	597.0 MB	291.2 MB	time +29%

concat-outer

Concatenate two stores keeping the union of variables.

Contender	Wall time	Peak RSS	Output	vs previous
adata-cli	16.37 s	175.3 MB	583.3 MB	time +79%
anndata (concat_on_disk)	12.00 s	448.9 MB	831.9 MB	time +102%
anndata (in memory)	16.64 s	1.8 GB	580.3 MB	time +83%

recast-narrow

Narrow X from float64 values with int64 indices to float32 and int32, as issue #13 asked. X is uncompressed, as write_h5ad leaves it by default, so the output keeps that and the baseline writes it the same way.

Contender	Wall time	Peak RSS	Input → output	vs previous
adata-cli	6.00 s	117.7 MB	815.6 MB → 415.4 MB (-49%)	-
anndata (in memory)	14.61 s	1.8 GB	815.6 MB → 417.3 MB (-49%)	-
scanpy	n/a	n/a	n/a	-
	no dtype rewrite; scanpy defers to anndata

recast-narrow-lzf

The same narrowing on an lzf-compressed X. Integer counts in float64 are mostly zero bytes, which lzf already removes, so expect a small saving on disk; the halving is in memory, once the matrix is loaded.

Contender	Wall time	Peak RSS	Input → output	vs previous
adata-cli	10.85 s	125.5 MB	302.5 MB → 291.1 MB (-4%)	-
anndata (in memory)	9.40 s	1.8 GB	302.5 MB → 291.2 MB (-4%)	-

convert-layout

Transpose X from CSR to CSC. The output should be the size of the input: 0.7.0 fixed a --layout that wrote it three times larger.

Contender	Wall time	Peak RSS	Input → output	vs previous
adata-cli (streaming)	19.18 s	357.8 MB	291.2 MB → 297.0 MB (+2%)	no change
adata-cli (--in-memory)	10.51 s	2.3 GB	291.2 MB → 297.0 MB (+2%)	time +19%
anndata (in memory)	7.69 s	984.4 MB	291.2 MB → 297.0 MB (+2%)	time +27%

rss-vs-size

Does peak memory track input size, at a fixed --chunk? This is the README's claim, stated as a measurement.

Contender	Wall time	Peak RSS	Output	vs previous
adata-cli	16.12 s	174.3 MB	583.3 MB	time +75%
anndata (in memory)	12.82 s	1.8 GB	580.3 MB	time +43%

---

Rows marked n/a are operations the baseline does not offer; that is a result, not a gap in the measurement. Where a baseline is faster, the row stands as it is -- the trade this tool makes is memory for time, and hiding the cost would make the table useless.

Output size is the file as the filesystem reports it, which for HDF5 includes allocation slack. On a small tier that slack can be several times the stored bytes and the column says more about the writer's allocation strategy than about the data; at ci and above it is noise. Zarr stores report a file count alongside, because a directory of small chunks measures much larger than it holds. Where a case exists to change a file's size, the column reads input → output, with the change.