Speed up Augur predict

[Zethson]

Speeds up Augur.predict with two results-preserving changes.

Background

Profiling predict() on the bundled dataset showed the time is dominated by scikit-learn per-call overhead across the many small models Augur fits (n_cell_types x n_subsamples x folds), not the random-forest math itself (tree building was ~1.2 s of ~42 s).

Changes

1. NumPy instead of subsample.to_df() in run_cross_validation. The DataFrame carries an arrow-backed string column index, which forces scikit-learn to re-validate pandas dtypes and convert the column labels on every fold and every scorer. Passing the dense matrix (with gene names kept separately for the feature-importance table) avoids that. The values, and therefore the results, are identical. ~14% per cross-validation in isolation.

2. Flattened parallelism. predict now fans every (cell_type, subsample) cross-validation task out through a single joblib pool instead of a per-cell-type pool inside a serial loop, so workers no longer idle at each cell-type boundary. Feature selection and eligibility checks still run serially first; submission order is preserved so results regroup per cell type unchanged. 20-30% on the parallel portion in isolation, growing with the number of cell types.

A threading backend was also evaluated and rejected (~3x slower, GIL-bound), so loky is kept.

Results

Back-to-back on the bundled 3-cell-type dataset (best of 3):

n_threads	before	after
4	17.0 s	14.8 s
8	13.9 s	12.2 s

The end-to-end gain is diluted here by serial fixed costs and only 3 cell types; datasets with more cell types benefit more from change 2.

Testing

• pytest tests/tools/test_augur.py -> 9 passed (the exact-value AUC assertions confirm results are unchanged)
• ruff, ruff-format, mypy pass

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 78.46%. Comparing base (eb35861) to head (f03e3b3).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1043      +/-   ##
==========================================
+ Coverage   77.99%   78.46%   +0.46%     
==========================================
  Files          50       51       +1     
  Lines        6694     7039     +345     
==========================================
+ Hits         5221     5523     +302     
- Misses       1473     1516      +43     

Files with missing lines	Coverage Δ
pertpy/tools/_augur.py	90.03% <100.00%> (+0.39%)	⬆️

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.