lstar 0.2.2

A patch release fixing one defect in viewer-store preparation, across all four surfaces (C++ core, Python, R, JS/WASM).

Action required for existing viewer stores

A viewer@0.1 store prepped by an earlier version may carry its count basis in the wrong orientation. Re-run extend_for_viewer() on such a store to repair it. lstar.validate() now reports the problem instead of passing silently, and the JS reader raises instead of returning wrong numbers.

Non-viewer stores, the on-disk format (spec_version 0.1), and every public signature are unchanged.

What was wrong

A viewer@0.1 store carries the count payload in both orientations, because the viewer has two hot paths and neither is a byte-range read away from the other:

• gene-major (csc) — colouring an embedding by a gene reads one column;
• cell-major (csr, counts_cellmajor) — a cluster or lasso selection reads rows.

All three surfaces normalized the basis to CSC in memory only, for the kernels, and never wrote it back. The converter preserves whatever orientation the source had — and a raw AnnData .X is CSR in essentially every h5ad — so nobody owned the on-disk orientation. A store converted from h5ad and then prepped came out with two cell-major copies and no column-readable one: it paid the full 2× payload and delivered one of the two access patterns.

The JS reader's cscColumn then read a well-formed slice of the wrong axis, returning a cell's expression profile as a gene's column — plausible numbers, no error. Genes past nCells read empty locally, or 416 over HTTP.

What changed

• extend_for_viewer() writes the normalized basis back (Python, R, JS) and stamps it provenance.viewer="basis". An already gene-major basis is left untouched; in JS it is restamped metadata-only rather than rewriting a potentially huge array, and chunks the new layout no longer covers are pruned.
• validate() errors on a missing or non-csc count basis — symmetric with the counts_cellmajor check it has always made.
• The JS reader asserts orientation in cscColumn / cscColumns / csrRow / csrRows, as the C++ core and Python's lazy reader always have.
• The write layout keys its "raw, single-chunk" basis branch on the csc encoding rather than the field name, so a cell-major measure can never be given a large uncompressed, un-range-readable layout.
• Basis selection excludes the viewer's own cache-tagged navigators, so re-running the prep to repair a store cannot make it feed on its own output.

Why it was not caught

Three separate tests covered this axis and all passed, because they asserted invariance — "the prep gives the same navigators whether counts arrive CSR or CSC" — which is true, and strictly weaker than "the output basis is gene-major". A green run actively read as "CSR is fine". Meanwhile every fixture that was later read was built CSC, so no test ever composed "prep a store" with "range-read a gene column".

Both gaps are now closed, and docs/parity.md carries the rule ("Invariance is not correctness"). Also added: docs/releasing.md.

Install

pip install --upgrade lstar-sc

lstar 0.2.1

Patch release — JS/WASM only. Fixes a viewer crash on store open in browsers that back the growable WebAssembly heap with a resizable ArrayBuffer.

Fix: viewer crash on open (resizable WASM heap)

The WASM reader is built with growable memory (ALLOW_MEMORY_GROWTH), so its heap is a resizable ArrayBuffer. Emscripten decodes embind std::string returns — including the store manifest read at open (Reader.groupAttrs) — in place over that heap, and browsers that back a growable heap with a resizable ArrayBuffer reject it (TextDecoder … must not be resizable), crashing the viewer on every store open.

The build now copies such bytes off the heap before decoding, in all three WASM modules, and a CI check (textdecoder_resizable) asserts the guard is present in the built glue so a toolchain change can't silently drop it (the crash is browser-only — no headless runtime exercises it). Thanks to pagoda3 for the diagnosis and end-to-end browser verification.

R and Python are unaffected; lstar-sc is republished at 0.2.1 for version coherence.

Install — Python: pip install lstar-sc==0.2.1 · R package lstar 0.2.1.

lstar 0.2.0

Zarr v3 is now the default on-disk format across all surfaces (C++/Python/R/JS). The legacy Zarr v2 layout stays available (format = "v2" / --zarr-format v2), and readers open both transparently — only newly written stores change layout.

Highlights

• Zstd + sharded writes. Zstd (Zarr v3's standard codec) is supported alongside gzip. A shard_elems option packs many inner chunks into fewer store objects while keeping them byte-range-readable, so a many-chunk array can be hosted without a file-per-chunk explosion.
• Compressed, range-readable viewer stores by default. extend_for_viewer() compresses the viewer store per field (zstd), and the reader resolves compressed arrays at chunk granularity — a hosted viewer fetches only the chunks it displays instead of whole arrays. Use compress = FALSE for the previous all-raw layout, or compress_primary = TRUE to trade gene-color latency for a smaller store.
• CLI knobs. convert and viewer gained --zarr-format {auto,v2,v3}, --compression {none,gzip,zstd} (+ --compression-level), --chunk-elems, and --shard-elems; viewer now re-emits in place keeping the store's existing format.

See R/NEWS.md for the full changelog.

Install — Python: pip install lstar-sc==0.2.0 · R package lstar 0.2.0.

v0.1.7

Viewer-prep basis selection — extend_for_viewer() auto-selects the count basis instead of erroring when an object kept only normalized values: it prefers raw counts (log1p), falls back to a log-normalized measure (used as-is, with a warning), and never uses a scaled/z-scored measure. The CLI gains --basis {auto,raw,lognorm} + --counts.

Includes a fix for a name-shortcut hole: a measure literally named counts that is actually scaled is no longer mis-picked as raw and log1p'd — the raw picker now excludes state == "scaled", symmetric with the log-normalized name fallback.

The selection contract is identical across R (.viewer_counts_basis), Python (_select_counts_basis), and JS (selectCountsBasis), enforced by cross-surface parity tests.

Packages: R lstar 0.1.7 · Python lstar-sc 0.1.7 (PyPI).

v0.1.6

Breaking (install requirements): the Python package now uses the zarr-python 3 library and requires zarr>=3.1 and Python ≥3.11 (drops 3.8–3.10). The on-disk format is unchanged (Zarr v2) — stores written by 0.1.6 stay readable by zarr-python 2 and by the C++/R/JS surfaces; zarr-python 3 is the library dependency, not a format change.

Fixes

• Dense primary measure now handled by the viewer prep and the live viewer (e.g. an SCE logcounts assay or a scaled/dense AnnData X) — previously raised NotFoundError and viewer optimization was silently skipped.
• Consolidated .zmetadata is emitted in parent-before-child key order (and read tolerantly of any order), so JS-extended stores open cleanly under zarr-python 3.
• Seurat → extend_for_viewer: a boolean QC column is no longer mistaken for a grouping, and the active identity no longer duplicates the clustering it mirrors.

Internal

• Single-sourced the JS viewer-compute recipe (compute.ts) and the measure-as-CSC read (fieldAsCsc); DRY'd the Python densify idiom (as_csc/as_csr). Behavior-preserving; cross-surface conformance green.

v0.1.5

Patch release. The PyPI-packaged (Python/C++) change over 0.1.4 is the extend_for_viewer(primary=) API; the rest is JS (the @lstar/core reader, consumed from the repo, not the wheel).

Python / R / JS — extend_for_viewer(primary=)

Name the grouping the viewer opens on. It's hoisted to the front of the prepared groupings, so it keys the counts_cellmajor locality reorder and is summarized first, and it composes with auto-detect (primary="cell_type" with no explicit groupings still preps every detected grouping but keys the reorder on cell_type). counts_cellmajor_order records provenance.group; a primary that isn't a grouping over the cell axis is rejected with a clear error. Identical semantics on Python, R, and JS, enforced by a cross-surface conformance leg (conformance/viewer_primary.sh).

JS reader (@lstar/core) — hosted single-file .lstar.zarr.zip + throughput

Not in the PyPI wheel; in the tag for browser/Node consumers:

• ZipStore correctness + performance: tolerates zarrita's leading-slash chunk keys (fixes a silent data-collapse on hosted zips); reads a chunk in one round-trip (local header folded into the data read); and fetches with cache: "no-store" so concurrent same-URL range reads parallelize (avoids the browser HTTP cache lock).
• cscColumns(name, cols[]) — a batched, coalesced multi-column read (the column-major twin of csrRows) for gene panels; and the byte-range fast path now range-reads within a chunk for multi-chunk arrays.
• A field's independent component arrays (labels/values/offsets, codes/categories) are read concurrently, removing an avoidable round-trip from the viewer's first-paint path.

v0.1.4

Single-file .lstar.zarr.zip

A store can now be packaged as one file — a .lstar.zarr.zip, written with every entry STORED (no deflate) so its already-codec-compressed chunks stay byte-range-readable when hosted. Read and write on all four surfaces (Python, R, C++, JS) and the CLI.

• Produce: lstar convert dir.lstar.zarr out.lstar.zarr.zip (repackage a directory store), lstar convert … --viewer out.lstar.zarr.zip (a single-file viewer store), or from a library — lstar.write(ds, "x.lstar.zarr.zip") (Python), lstar_write(ds, "x.lstar.zarr.zip") (R).
• Read: lstar.read(...) / lstar_read(...) / lstar::read(...); in the browser, ZipStore.open(httpZipSource(url)) reads a hosted zip by HTTP Range — one request per chunk, no extraction.
• Writers force STORED (regardless of a chunk compressor) and are ZIP64-aware; a DEFLATE-packed .lstar.zarr.zip is rejected on read with a clear message. The directory form stays the default working format. See docs/format.md §Packaging.

Cross-surface parity is enforced by the conformance suite (conformance/zip*.sh).

v0.1.3

lstar-sc 0.1.3 — the first release since 0.1.2, bundling the macOS packaging fix, the viewer@0.1 cross-language parity work, and the cross-surface parity-audit fixes.

macOS packaging

• Fix a reproducible arm64 SIGSEGV in extend_for_viewer's OpenMP kernels caused by a dual-libomp cross-runtime: the bundled libomp is coalesced to @rpath/libomp.dylib so a host libomp (scikit-learn/numba/scipy) is reused instead of a second copy loading. A bare pip install lstar-sc on Apple Silicon no longer risks the crash.

viewer@0.1 cross-language parity

• extend_for_viewer now produces a field-for-field identical store across C++/Python/R/JS. The cell reorder (viewer_cell_order: cluster-contiguous, then a Hilbert curve over the embedding), the CSR/CSC encoding normalization, the group ordering, and grouping detection are single-sourced on the shared C++ core + a canonical policy, and enforced by cross-surface conformance legs (including a corpus-driven check) and a policy linter. See docs/parity.md.

Cross-surface fidelity (parity audit)

• COO fields normalize to CSC on write (portable to every reader); dense fields carry their shape in the manifest; the C++ core preserves a field's uncertainty; the .h5ad direct backend infers state from content like the native backend.
• R preserves float32 value dtypes and a graph relation's directed/weighted flags across a lstar_read/lstar_write round-trip.
• pseudobulk routes its reduction through the shared col_sum_by_group kernel (was a numpy duplicate).
• extend_for_viewer gains order=/markers= on R and order= on JS (parity with Python).
• New CI guardrails: a binding-parity tripwire (a shared kernel must be bound on all surfaces) and a policy linter (viewer policy constants single-sourced).

lstar-sc 0.1.2

Content-based measure state + state-based viewer basis, consistent across Python/R/JS.

Since 0.1.1:

• read_anndata/read_mudata infer state (raw/lognorm/scaled) from data content, not the source slot/name (fixes a scaled .X reading as None and a log-normalized .raw mislabeled raw); a genuinely-raw .X is named counts.
• extend_for_viewer (Python + R) selects its count basis by state; adds counts=/basis=, a clear error listing present measures when there's no raw basis, and an opt-in basis="lognorm" mode. Fixes: convert_anndata output (scaled .X + lognorm .raw, no counts) could not be viewer-prepped.
• JS extendForViewer + selectCountsBasis bindings — state-based basis selection now consistent across Python/R/JS.
• lstar.view() / lstar::view() soft-delegate to the pagoda3 viewer (optional, one-way).

lstar-sc 0.1.1

Viewer-prep feature release (Python package lstar-sc; R stays 0.1.0 for the CRAN wave).

Since 0.1.0: extend_for_viewer() + the lstar viewer CLI, the viewer@0.1 profile + validate() contract, cache-tag converter hygiene (anndata/seurat/sce drop+record regenerable navigators), and bound core kernels (markers_one_vs_rest, overdispersion).