Skip to content

Obs filter - #11

Merged
jjschirle merged 3 commits into
mainfrom
obs_filter
Sep 1, 2026
Merged

Obs filter#11
jjschirle merged 3 commits into
mainfrom
obs_filter

Conversation

@jjschirle

Copy link
Copy Markdown
Contributor

Adds optional obs-based cell subsetting to load_and_normalize() via an obs_filter callable.

  • bumped python version pinned to 3.12 as required in pyproject.toml.
  • Adds obs_filter, e.g. obs_filter=lambda obs: obs["condition"] == "control".
  • Applies obs subsetting before cell filtering, gene filtering, depth normalization, and log transformation.
  • Computes gene totals and the gene_threshold cutoff using only the selected cohort.
  • Preserves the existing load_and_normalize() implementation when obs_filter=None.
  • Avoids materializing decoded indices and values for excluded IVCSR rows. The packed byte stream must still be read and scanned because byte offsets for major slices are not currently stored.
  • Adds validation for invalid masks, empty selections, non-callable filters, and files without obs.
  • Adds tests comparing filtered results against preprocessing an equivalent AnnData subset, including interleaved conditions and metadata slicing.

The callable is easily extended:

obs_filter=lambda obs: (
    obs["condition"].isin(["control", "vehicle"])
    & obs["timepoint"].isin(["T1", "T3"])
    & (obs["batch"] != "bad_batch")
)

Passes all rapid_load pytests, ruff, and ty checks.

@jjschirle
jjschirle requested a review from aarmey September 1, 2026 19:32
@jjschirle
jjschirle merged commit f4b9cbe into main Sep 1, 2026
5 checks passed
@jjschirle
jjschirle deleted the obs_filter branch September 1, 2026 20:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants