perf: "two-pass" seurat hvg via scanpy.get.aggregate

[ilan-gold]

An idea that popped into my head for disk-bound datasets but likely also normal ones. This should, in theory, greatly improve on-disk access and produce speed ups for disk bound data by reducing the amount of i/o in the worst case, unordered scenario (while, I would guess, leaving in-memory datasets untocuhed or maybe improved thanks to memory access + more efficient mean/var).

• Closes #
• Tests included or not required because:

• Release notes not necessary because:

[codecov]

Codecov Report

❌ Patch coverage is 83.87097% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.64%. Comparing base (cb3e6c2) to head (cc0d67e).
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
src/scanpy/preprocessing/_highly_variable_genes.py	83.87%	5 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #4013   +/-   ##
=======================================
  Coverage   78.63%   78.64%           
=======================================
  Files         118      118           
  Lines       12729    12753   +24     
=======================================
+ Hits        10010    10030   +20     
- Misses       2719     2723    +4     

Flag	Coverage Δ
hatch-test.low-vers	77.95% <83.87%> (+0.01%)	⬆️
hatch-test.pre	78.53% <83.87%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
src/scanpy/preprocessing/_highly_variable_genes.py	93.83% <83.87%> (-1.24%)	⬇️

... and 1 file with indirect coverage changes

[scverse-benchmark]

No changes in benchmarks.

Comparison: https://github.com/scverse/scanpy/compare/cb3e6c25ebe209267153ee94636437cbdd1267fd..cc0d67e08f5296b5d139c6baa674e322c4730c87
Last changed: Thu, 16 Apr 2026 18:02:17 +0000

More details: https://github.com/scverse/scanpy/pull/4013/checks?check_run_id=71665970920