scx-cli-v0.20.0

What's Changed

• accel: mark every evaluated node stable in Leiden's parallel local moving by @nick-youngblut in #544
• docs: refresh the README feature surface, and correct two stale build-csc claims by @nick-youngblut in #545
• ops: dictionary output from append, merge and merge --sort-by by @nick-youngblut in #546
• ops: dictionary output from scx sort's obs spill path by @nick-youngblut in #547
• accel: five CPU-accelerator numerical-parity and convergence fixes (review §7.3, §7.10, §7.11, §7.15, §7.17) by @nick-youngblut in #548
• gpu: cuVS stream synchronization, reduction determinism, decoder bounds, and VRAM accounting by @nick-youngblut in #549
• cli+mtx: destination overwrite safety, and MTX multimodal / streaming / integer integrity by @nick-youngblut in #550
• format: refuse an unscoped whole-matrix read of a multimodal file, and name the CSR structural checker by @nick-youngblut in #551
• bench: four community analytical benchmarks, gated and documented by @nick-youngblut in #552
• accel: let CSC dispatch serve row-indexed transforms by @nick-youngblut in #553
• accel: serve CSC dispatch on a row-filtered or gene-subset handle by @nick-youngblut in #554
• sparse: one-pass CSC sidecar builder, replacing the per-chunk transpose by @nick-youngblut in #555
• ops: build-csc appends its CSC sidecar in place instead of rewriting the file by @nick-youngblut in #556
• ops+convert: build the CSC sidecar in the same pass, and carry it through rewrite ops by default by @nick-youngblut in #557
• sparse+format-io+engine: parallel CSC build, and csc="auto" as the ingest default by @nick-youngblut in #558
• accel: promote exact-nnz Wilcoxon to the CSC default, and fix the two CSC routes that lost for a reason by @nick-youngblut in #559
• bench: recapture the laptop-test pipeline on the CSC DE route by @nick-youngblut in #560
• docs: split scanpy/api/performance into per-topic directories, and clean up outdated docs by @nick-youngblut in #561
• chore: bump workspace version to 0.20.0 by @nick-youngblut in #562

Full Changelog: pyscx-v0.19.0...scx-cli-v0.20.0

pyscx-v0.20.0

What's Changed

• accel: mark every evaluated node stable in Leiden's parallel local moving by @nick-youngblut in #544
• docs: refresh the README feature surface, and correct two stale build-csc claims by @nick-youngblut in #545
• ops: dictionary output from append, merge and merge --sort-by by @nick-youngblut in #546
• ops: dictionary output from scx sort's obs spill path by @nick-youngblut in #547
• accel: five CPU-accelerator numerical-parity and convergence fixes (review §7.3, §7.10, §7.11, §7.15, §7.17) by @nick-youngblut in #548
• gpu: cuVS stream synchronization, reduction determinism, decoder bounds, and VRAM accounting by @nick-youngblut in #549
• cli+mtx: destination overwrite safety, and MTX multimodal / streaming / integer integrity by @nick-youngblut in #550
• format: refuse an unscoped whole-matrix read of a multimodal file, and name the CSR structural checker by @nick-youngblut in #551
• bench: four community analytical benchmarks, gated and documented by @nick-youngblut in #552
• accel: let CSC dispatch serve row-indexed transforms by @nick-youngblut in #553
• accel: serve CSC dispatch on a row-filtered or gene-subset handle by @nick-youngblut in #554
• sparse: one-pass CSC sidecar builder, replacing the per-chunk transpose by @nick-youngblut in #555
• ops: build-csc appends its CSC sidecar in place instead of rewriting the file by @nick-youngblut in #556
• ops+convert: build the CSC sidecar in the same pass, and carry it through rewrite ops by default by @nick-youngblut in #557
• sparse+format-io+engine: parallel CSC build, and csc="auto" as the ingest default by @nick-youngblut in #558
• accel: promote exact-nnz Wilcoxon to the CSC default, and fix the two CSC routes that lost for a reason by @nick-youngblut in #559
• bench: recapture the laptop-test pipeline on the CSC DE route by @nick-youngblut in #560
• docs: split scanpy/api/performance into per-topic directories, and clean up outdated docs by @nick-youngblut in #561
• chore: bump workspace version to 0.20.0 by @nick-youngblut in #562

Full Changelog: pyscx-v0.19.0...pyscx-v0.20.0

scx-cli-v0.19.0

What's Changed

• loader: tokenisation kernel set v1 (rank, bin, sample, crop) + collate contract v3 by @nick-youngblut in #538
• loader+format-io: neighbourhood plan builder (graph and coordinate driven) by @nick-youngblut in #539
• ops+format-io: sort and compact emit sharded obsm/varm/obsp/varp by @nick-youngblut in #541
• loader+format-io: reuse-signal row-group admission and chunked parallel group decode by @nick-youngblut in #540
• loader+format-io: multi-set batch executor for the cell-set gather by @nick-youngblut in #542
• chore: bump workspace version to 0.19.0 by @nick-youngblut in #543

Full Changelog: pyscx-v0.18.1...scx-cli-v0.19.0

pyscx-v0.19.0

What's Changed

• loader: tokenisation kernel set v1 (rank, bin, sample, crop) + collate contract v3 by @nick-youngblut in #538
• loader+format-io: neighbourhood plan builder (graph and coordinate driven) by @nick-youngblut in #539
• ops+format-io: sort and compact emit sharded obsm/varm/obsp/varp by @nick-youngblut in #541
• loader+format-io: reuse-signal row-group admission and chunked parallel group decode by @nick-youngblut in #540
• loader+format-io: multi-set batch executor for the cell-set gather by @nick-youngblut in #542
• chore: bump workspace version to 0.19.0 by @nick-youngblut in #543

Full Changelog: pyscx-v0.18.1...pyscx-v0.19.0

scx-cli-v0.18.1

What's Changed

• loader: move the multimodal batch into numpy; one shard-index ordering rule (OPT-LOADER-1, -2) by @nick-youngblut in #521
• ops: build-csc decodes each CSR shard once (OPT-OPS-1) by @nick-youngblut in #522
• engine: one shard list per pipeline, one gene-set sort per query, one copy per shard (OPT-ENGINE-1, -9) by @nick-youngblut in #523
• format-io: encode a shard's codec candidates and row groups in parallel (OPT-FORMATIO-3) by @nick-youngblut in #524
• convert+ops: charge the framed encode transient, so a memory budget is a bound (OPT-CONVERT-10, OPT-OPS-7) by @nick-youngblut in #525
• ops: batch the value encode, and encode optimize's shards in parallel (OPT-OPS-6, OPT-OPS-4) by @nick-youngblut in #526
• format: an ObspCsrShard's minor extent is n_obs, not n_vars (OPT-FORMATIO-4) by @nick-youngblut in #527
• format-io+loader: retain decoded row groups in the shard LRU (OPT-FORMATIO-1) by @nick-youngblut in #528
• accel: partition the pseudobulk scatter across the pool, bit-identical to serial (OPT-ACCEL-4) by @nick-youngblut in #529
• convert: stream --from 10x, by default (OPT-CONVERT-9) by @nick-youngblut in #530
• gpu: drop two dead tie launches, the per-shard indptr drain, and a host round-trip (OPT-GPU-1, -4, -10, -13) by @nick-youngblut in #531
• docs: qualify the CSC sidecar's "GPU-fast" claim for rank_genes_groups by @nick-youngblut in #533
• pyscx: make the eager-assembly warning size something, and skip scipy's int64 upcast by @nick-youngblut in #534
• bench: per-regime data-wait gate and scatter-route re-measure by @nick-youngblut in #532
• loader: pre-size the cell-set gather, per-set crop mask, gather(plan), GIL release, batch charge by @nick-youngblut in #535
• loader: bounded reader registry for large manifests (W8) by @nick-youngblut in #536
• chore: bump workspace version to 0.18.1 by @nick-youngblut in #537

Full Changelog: pyscx-v0.18.0...scx-cli-v0.18.1

pyscx-v0.18.1

What's Changed

• loader: move the multimodal batch into numpy; one shard-index ordering rule (OPT-LOADER-1, -2) by @nick-youngblut in #521
• ops: build-csc decodes each CSR shard once (OPT-OPS-1) by @nick-youngblut in #522
• engine: one shard list per pipeline, one gene-set sort per query, one copy per shard (OPT-ENGINE-1, -9) by @nick-youngblut in #523
• format-io: encode a shard's codec candidates and row groups in parallel (OPT-FORMATIO-3) by @nick-youngblut in #524
• convert+ops: charge the framed encode transient, so a memory budget is a bound (OPT-CONVERT-10, OPT-OPS-7) by @nick-youngblut in #525
• ops: batch the value encode, and encode optimize's shards in parallel (OPT-OPS-6, OPT-OPS-4) by @nick-youngblut in #526
• format: an ObspCsrShard's minor extent is n_obs, not n_vars (OPT-FORMATIO-4) by @nick-youngblut in #527
• format-io+loader: retain decoded row groups in the shard LRU (OPT-FORMATIO-1) by @nick-youngblut in #528
• accel: partition the pseudobulk scatter across the pool, bit-identical to serial (OPT-ACCEL-4) by @nick-youngblut in #529
• convert: stream --from 10x, by default (OPT-CONVERT-9) by @nick-youngblut in #530
• gpu: drop two dead tie launches, the per-shard indptr drain, and a host round-trip (OPT-GPU-1, -4, -10, -13) by @nick-youngblut in #531
• docs: qualify the CSC sidecar's "GPU-fast" claim for rank_genes_groups by @nick-youngblut in #533
• pyscx: make the eager-assembly warning size something, and skip scipy's int64 upcast by @nick-youngblut in #534
• bench: per-regime data-wait gate and scatter-route re-measure by @nick-youngblut in #532
• loader: pre-size the cell-set gather, per-set crop mask, gather(plan), GIL release, batch charge by @nick-youngblut in #535
• loader: bounded reader registry for large manifests (W8) by @nick-youngblut in #536
• chore: bump workspace version to 0.18.1 by @nick-youngblut in #537

Full Changelog: pyscx-v0.18.0...pyscx-v0.18.1

scx-cli-v0.18.0

What's Changed

• convert: stream /raw on h5ad export; read shards by catalog entry (OPT-CONVERT-1, -3) by @nick-youngblut in #510
• convert: one traversal for HDF5 string ingest, straight into a pre-sized Arrow builder (OPT-CONVERT-2) by @nick-youngblut in #511
• pyscx: to_anndata obsp= / varp= / varm= slot filters and a raw= opt-out (PR H, REC-8) by @nick-youngblut in #512
• pyscx: eager to_anndata(var_names=) projects while assembling (PR I, REC-9) by @nick-youngblut in #513
• pyscx: one QC kernel for every X, plus score_genes(ctrl_genes=) (PR J, REC-11) by @nick-youngblut in #514
• pyscx: accel docs signature guard, DE backed layer=, and shard-skipping under a row projection (PR J deferred items) by @nick-youngblut in #515
• accel: the streaming DE kernels take the shared prefetch drivers — row-projection skip + decode-prefetch by @nick-youngblut in #516
• ops+pyscx+cli+rscx: attach_var_columns — key-joined per-gene columns, in place (PR K, REC-13) by @nick-youngblut in #517
• engine+pyscx: decode a query's X at the requested dtype — collect(data_dtype=) (REC-15) by @nick-youngblut in #518
• format+pyscx: the eager-layers decode-loss guard folds over the selected layers by @nick-youngblut in #519
• chore: bump workspace version to 0.18.0 by @nick-youngblut in #520

Full Changelog: pyscx-v0.17.0...scx-cli-v0.18.0

pyscx-v0.18.0

What's Changed

• convert: stream /raw on h5ad export; read shards by catalog entry (OPT-CONVERT-1, -3) by @nick-youngblut in #510
• convert: one traversal for HDF5 string ingest, straight into a pre-sized Arrow builder (OPT-CONVERT-2) by @nick-youngblut in #511
• pyscx: to_anndata obsp= / varp= / varm= slot filters and a raw= opt-out (PR H, REC-8) by @nick-youngblut in #512
• pyscx: eager to_anndata(var_names=) projects while assembling (PR I, REC-9) by @nick-youngblut in #513
• pyscx: one QC kernel for every X, plus score_genes(ctrl_genes=) (PR J, REC-11) by @nick-youngblut in #514
• pyscx: accel docs signature guard, DE backed layer=, and shard-skipping under a row projection (PR J deferred items) by @nick-youngblut in #515
• accel: the streaming DE kernels take the shared prefetch drivers — row-projection skip + decode-prefetch by @nick-youngblut in #516
• ops+pyscx+cli+rscx: attach_var_columns — key-joined per-gene columns, in place (PR K, REC-13) by @nick-youngblut in #517
• engine+pyscx: decode a query's X at the requested dtype — collect(data_dtype=) (REC-15) by @nick-youngblut in #518
• format+pyscx: the eager-layers decode-loss guard folds over the selected layers by @nick-youngblut in #519
• chore: bump workspace version to 0.18.0 by @nick-youngblut in #520

Full Changelog: pyscx-v0.17.0...pyscx-v0.18.0

scx-cli-v0.17.0

What's Changed

• test: commit the op output-identity A/B harness (scx-testkit, scx-loader) by @nick-youngblut in #495
• bench: make the peak-RSS metrics real, and give them a fixture worth measuring by @nick-youngblut in #496
• bench: benchmark the three SCX ops that had no instrument at all by @nick-youngblut in #497
• bench: instrument the five SCX ops with no timing, and un-break 16 suppressed floors by @nick-youngblut in #498
• bench: recapture and promote the regression baseline (v0.16.0-opt-instruments) by @nick-youngblut in #499
• uns: exact float round-trip, non-finite scalars, several uns keys per attach, update_uns (PR A) by @nick-youngblut in #500
• ops: categoricals survive every in-place obs writer (PR B, REC-3) by @nick-youngblut in #501
• reader: bounded row gather — handle[rows] and gather_rows_sparse assemble once (PR C, REC-1) by @nick-youngblut in #502
• pyscx: column selectors project without a decode; array raises; stored_dtype, cache_shards, Experiment encoding getters (PR D, REC-7) by @nick-youngblut in #503
• pyscx: zero-row / zero-var files write and read; empty matrices get no CSC or index; merge / append / metadata rewrites tolerate 0 rows (PR E, REC-5) by @nick-youngblut in #504
• pyscx: read_obs() / obs_categorical() return logical rows by default, logical=False for physical; positional attach and modify_metadata(obs=) accept either row space (PR F, REC-6 — breaking, 0.17) by @nick-youngblut in #505
• accel: rank_genes_groups(pts=, groups=, corr_method=) + pct_nz_* in rank_genes_groups_df; pdex_ref(groups=); pseudobulk groupby str|list (PR G, REC-10) by @nick-youngblut in #506
• uns: pandas.DataFrame envelope — rank_genes_groups(pts=) survives a write/read cycle, both h5ad directions (X6) by @nick-youngblut in #507
• accel: singlet-group guard on the default path, and scanpy's "rest" keeps unlabelled cells (PR M+N, X8/X9 — numerical change, 0.17) by @nick-youngblut in #508
• chore: bump workspace version to 0.17.0 by @nick-youngblut in #509

Full Changelog: pyscx-v0.16.0...scx-cli-v0.17.0

pyscx-v0.17.0

What's Changed

• test: commit the op output-identity A/B harness (scx-testkit, scx-loader) by @nick-youngblut in #495
• bench: make the peak-RSS metrics real, and give them a fixture worth measuring by @nick-youngblut in #496
• bench: benchmark the three SCX ops that had no instrument at all by @nick-youngblut in #497
• bench: instrument the five SCX ops with no timing, and un-break 16 suppressed floors by @nick-youngblut in #498
• bench: recapture and promote the regression baseline (v0.16.0-opt-instruments) by @nick-youngblut in #499
• uns: exact float round-trip, non-finite scalars, several uns keys per attach, update_uns (PR A) by @nick-youngblut in #500
• ops: categoricals survive every in-place obs writer (PR B, REC-3) by @nick-youngblut in #501
• reader: bounded row gather — handle[rows] and gather_rows_sparse assemble once (PR C, REC-1) by @nick-youngblut in #502
• pyscx: column selectors project without a decode; array raises; stored_dtype, cache_shards, Experiment encoding getters (PR D, REC-7) by @nick-youngblut in #503
• pyscx: zero-row / zero-var files write and read; empty matrices get no CSC or index; merge / append / metadata rewrites tolerate 0 rows (PR E, REC-5) by @nick-youngblut in #504
• pyscx: read_obs() / obs_categorical() return logical rows by default, logical=False for physical; positional attach and modify_metadata(obs=) accept either row space (PR F, REC-6 — breaking, 0.17) by @nick-youngblut in #505
• accel: rank_genes_groups(pts=, groups=, corr_method=) + pct_nz_* in rank_genes_groups_df; pdex_ref(groups=); pseudobulk groupby str|list (PR G, REC-10) by @nick-youngblut in #506
• uns: pandas.DataFrame envelope — rank_genes_groups(pts=) survives a write/read cycle, both h5ad directions (X6) by @nick-youngblut in #507
• accel: singlet-group guard on the default path, and scanpy's "rest" keeps unlabelled cells (PR M+N, X8/X9 — numerical change, 0.17) by @nick-youngblut in #508
• chore: bump workspace version to 0.17.0 by @nick-youngblut in #509

Full Changelog: pyscx-v0.16.0...pyscx-v0.17.0