diff --git a/CLAUDE.md b/CLAUDE.md index 6847c57..e7e5b90 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -113,6 +113,8 @@ backend/ seqfish_reader.py seqFISH / Spatial Genomics GenePS; v2 full, v1 partial. Mixed µm/pixel coordinate handling — see its own section. duck.py Shared DuckDB query helpers used by the spatial readers + metadata_filter.py Categorical-vs-continuous typing + the MetadataFilter + subsetting spec; shared by cells and edges (#35, #45) spatial_cache.py Spatially-sorted parquet cache (build on first access) visium_hd_reader.py Visium HD — bins as square polygons; see its own section merscope_reader.py MERSCOPE implementation (inherits SpatialDatasetReader) @@ -127,7 +129,7 @@ backend/ Dockerfile tests/ golden_snapshot.py Reader regression guard — see Development Workflow - golden_baseline.json Recorded baseline (133 probes / 5 datasets) + golden_baseline.json Recorded baseline (191 probes / 7 datasets) frontend/ src/ @@ -366,7 +368,15 @@ All shared state lives in a single Zustand store. Key sections: - **Layer visibility**: `layers` object — each layer has `visible` + `opacity`; `cellSegments` also has `outlineOpacity` (independent from fill opacity) - **Cell color**: `cellColorEnabled`, `colorBy` (`mode`: off/gene_set/metadata, `field`), - `cellColorPalette`, `cellColorClamp` (squish/oob cutoffs) + `cellColorPalette`, `cellColorClamp` (squish/oob cutoffs). `cellColorType` / + `cellColorCategories` hold the type the backend actually returned, written by + panel 0 — the LayerPanel reads these instead of guessing from the schema dtype. +- **Categorical override**: `categoricalOverrides`, keyed `cell::` / + `edge::` → `true | false`; absent means auto-detect (issue #35). +- **Metadata filter**: `cellFilter` / `edgeFilter`, each + `{ field, values, min, max, includeMissing }` or null (issue #45). `cellFilter` + also governs edges — both endpoints must survive it. Both reset on dataset change + (column names are dataset-specific); `edgeFilter` also resets on edge-file change. - **Transcript gene filter**: `selectedGenes` — `null` = no filter (show all); `Set` = allowlist (show only those genes). Dataset-scoped; resets on dataset change. See Gene Filter section below. @@ -852,6 +862,99 @@ Beyond 20 categories, `geneColor()` provides deterministic hash-based colors. --- +## Metadata Typing and Subsetting (readers/metadata_filter.py) + +Two features share one module because they are the same question asked twice: *what +kind of thing is this column?* Issue #35 asks it to pick a colour scheme, issue #45 +to pick a subset. `metadata_filter.py` answers both, and `base_reader` uses it for +the cells table while `edge_reader` uses it for the edge table, so the two panels +cannot drift apart. + +### Categorical vs continuous (#35) + +`is_categorical(col, forced)`: + +- `forced=None` — auto: strings, objects, bools, pandas categoricals, and **integers + with ≤ 30 distinct values** are categorical. That threshold is what makes Seurat + cluster IDs work, since `fwrite` on a `@meta.data` writes them as ints. +- `forced=True` / `False` — the user's explicit "treat as categorical" choice. + Forcing *continuous* on a text column is ignored: there is no gradient to draw, + and honouring it would paint every unit one colour. + +`sort_categories()` sorts numerically when every label parses as a number, so cluster +10 comes after cluster 2 rather than between 1 and 2. + +**`_color_values_meta` now lives on the base class.** Every reader used to carry a +near-identical copy, and the six copies had already drifted — CosMx filled NaN with +`""`/`0` where the others dropped it, and only some passed `key=str` to `sorted`. +A reader now supplies only `_metadata_frame()`, the cells table it already builds. + +The override travels as `categorical` on `POST /color-values` and +`POST /edge-color-values`, and lives in the store under `categoricalOverrides` +keyed `cell::` / `edge::`. + +**The frontend no longer guesses the type from the schema dtype.** It could not: the +backend's rule also depends on cardinality, which the schema does not carry. The old +guess disagreed for exactly the columns issue #35 is about — an integer cluster column +drew discrete colours on the canvas while the panel showed a viridis bar with two +sliders that did nothing. Panel 0 now records the type the backend actually returned +(`cellColorType` / `cellColorCategories`), and `EdgeSection` asks directly for the +edge side. That also removed the duplicate `color-values` fetch both legends were +making for themselves. + +### Subsetting (#45) + +`MetadataFilter` is either a categorical allowlist (`values`, compared as strings so +it works whatever the dtype) or an inclusive numeric range (`vmin`/`vmax`), plus +`include_missing` — false by default, because a cell with no cluster call is not part +of "cluster 4". + +**Filters are resolved and applied server-side, before sampling.** This is the whole +design constraint. Both the boundary and edge queries sample on the server, so a +client-side filter would leave a fraction of a subset: narrowing to a cluster holding +5% of cells at a 10% sample would draw 0.5% of the tissue. Filtering first means the +subset renders at full density. + +- `SpatialDatasetReader.filter_cell_ids(spec)` resolves against `_metadata_frame()` + and caches per (reader, spec) — the same filter is re-resolved on every pan. + An unknown column raises `ValueError` → HTTP 400, rather than silently rendering + everything while the panel shows an active filter. +- Each reader's `cell_boundaries()` takes `cell_ids` and **must apply it before the + count and the sample**. The five implementations differ too much to share code: + Xenium and CosMx join it into their DuckDB query, MERSCOPE skips non-matching rows + before decoding WKB, Visium HD and seqFISH mask their in-memory frames. +- `EdgeReader.query_grouped()` takes `cell_ids` and `edge_filter`. An edge survives + the cell filter only when **both** endpoints do — the point of "focus on 2–3 cell + types" is the signalling within that subset, and a half-outside edge would run off + to a cell that is not drawn. `edge_filter` becomes a real SQL predicate when the + column is in the parquet, and a semi-join against a registered frame when it comes + from `edge-metadata/`. + +**Large id sets go through `duck.register_ids()`, not `IN (?, ?, …)`.** A filter can +keep hundreds of thousands of cells; binding that many parameters is unworkable and +the SQL text alone reaches megabytes. Registering a one-column frame makes it an +ordinary hash semi-join. + +Two things that bit during implementation and are easy to reintroduce: + +- **Boolean columns need lowering.** The categories the panel offers come from pandas + (`"True"`), while DuckDB's `CAST(BOOLEAN AS VARCHAR)` yields `"true"`, so a literal + comparison silently matches nothing. `edge_filter_sql` lowers both sides for boolean + columns only — doing it for every column would merge genuinely distinct string labels. +- **The auto sample fraction must recalibrate after a filter.** `useCellBoundaries` + picks its fraction from the previous fetch's total, which a filter invalidates, and + nothing else would trigger another fetch — so the layer sat showing a tenth of an + already-small subset until the user happened to pan. It now re-fetches once when the + corrected fraction is >1.2× the one used. The threshold matters: the panel derives + its displayed percentage from the *current* total, so a looser one leaves the readout + advertising a fraction the canvas is not drawing at. + +**Transcripts are deliberately not filtered.** Several platforms ship no +transcript→cell assignment at all (seqFISH v2 dropped the column), so the filter has +nothing to join on and would work on some datasets and not others. + +--- + ## Tile Pyramid The backend uses pyvips when available (fast streaming, handles very large OME-TIFFs @@ -1014,7 +1117,9 @@ which it does not cover at all. to be the active color-by field. `EdgeInfoPanel` does render its annotations generically — the cell panel should be brought in line with it. `sample_data/mouse_ileum_tiny/cell-metadata/example_clusters.csv` now exercises the - feature locally. + feature locally. It carries a `seurat_clusters` column spanning 0–11 specifically so + the demo data reproduces issue #35: twelve integer levels, where a lexicographic sort + would put 10 and 11 between 1 and 2. 3. **Cell expression bar chart** — click panel shows cell metadata but not a sorted gene expression readout. `/spatial/{dataset}/expression/{cell_id}` exists; the UI does not. @@ -1042,12 +1147,15 @@ which it does not cover at all. ### Open GitHub issues -- **#45 — Select cells/edges by metadata.** Let the user restrict the view to a subset - (a sample, or 2–3 cell types) rather than all data at once. -- **#35 — Force-categorical toggle for numeric metadata columns.** Integer-coded - categoricals (Seurat cluster IDs, `*_snn_res.*`, phenotype codes) currently route to a - continuous viridis gradient. Partially mitigated already: `_color_values_meta()` treats - an integer column with ≤ 30 unique values as categorical, and `edge_color_values()` does - the same. Above that threshold users still fall back to renaming values to strings. - The ask is an explicit per-column "treat as categorical" toggle in the color panel, - with numeric sort order preserved in the legend. +Both of the previously open issues (**#35** force-categorical toggle, **#45** select +cells/edges by metadata) are implemented — see the *Metadata Typing and Subsetting* +section. What each issue asked for but this pass did not deliver: + +- **#35** — the choice is per column and per session, but is not persisted across a + reload, and the legend has no per-category visibility checkbox. The filter section + covers the "show only cluster 4" case that checkbox would have served. +- **#45** — filtering is on **one column at a time**. Composing two cell-side + predicates ("cluster 4 *and* sample B") needs a list of filters rather than a single + one; the backend `MetadataFilter` is already a value object, so the change is an + `and`-list in the store and a loop in `filter_cell_ids`. Transcripts are excluded + by design (no cell assignment on several platforms). diff --git a/README.md b/README.md index f60a968..205941a 100644 --- a/README.md +++ b/README.md @@ -24,6 +24,8 @@ Spatial transcriptomics platforms (Xenium, seqFISH, Visium HD, MERSCOPE, CosMx) - **Per-panel rotation** — rotate either panel to any angle to align tissue orientation - **Transcript dot overlay** — per-gene colored dots, filterable by gene species, with hover tooltips - **Cell/spot segmentation** — polygon boundaries with color-by-gene-set or color-by-metadata, and editable per-category colors +- **Metadata filtering** — restrict the view to a subset of cells or edges (a sample, a few cell types, a value range). Applied server-side before sampling, so a rare cluster renders at full density instead of being sampled away +- **Treat-as-categorical toggle** — integer-coded cluster IDs get a discrete editable palette rather than a viridis gradient, with the numeric order preserved in the legend - **Region drawing and measurement tools** — annotate areas, export cell selections, save PNG screenshots - **Supplemental metadata** — drop any CSV or parquet into a `cell-metadata/` folder to add custom color-by columns (clusters, pseudotime, etc.) without touching the original data - **Multi-dataset support** — switch between datasets without restarting; each is auto-detected by platform @@ -160,7 +162,7 @@ Standard R export works out of the box: write.csv(my_metadata, file.path(dataset_dir, "cell-metadata", "metadata.csv")) ``` -Columns appear automatically in the **Cell Color** dropdown. Continuous columns get a gradient; string and low-cardinality integer columns get discrete colors. +Columns appear automatically in the **Cell Color** and **Cell Filter** dropdowns. Continuous columns get a gradient; string and low-cardinality integer columns get discrete colors. Use **treat as categorical** to override that guess either way — a Seurat cluster column with more than 30 levels still gets discrete colors, and a coded column you want as a gradient can have one. ## Supplemental edge metadata @@ -232,10 +234,13 @@ The frontend automatically adapts its layer controls to the capabilities your re ## Roadmap -Tracked in [GitHub issues](https://github.com/RaredonLab/TissuePlex/issues). Currently open: +Tracked in [GitHub issues](https://github.com/RaredonLab/TissuePlex/issues). -- **[#45](https://github.com/RaredonLab/TissuePlex/issues/45)** — select cells and edges by metadata, so you can focus on a sample or a few cell types instead of the whole dataset -- **[#35](https://github.com/RaredonLab/TissuePlex/issues/35)** — a "treat as categorical" toggle for numeric metadata columns, so integer-coded cluster IDs get a discrete editable palette instead of a continuous gradient +[#45](https://github.com/RaredonLab/TissuePlex/issues/45) (select by metadata) and [#35](https://github.com/RaredonLab/TissuePlex/issues/35) (treat-as-categorical toggle) are both implemented. Natural follow-ups, neither yet built: + +- Combining more than one filter at a time — today it is one column, so "cluster 4 *and* sample B" needs two passes +- Persisting the categorical choice and the palette across reloads +- Filtering transcripts, which needs a transcript→cell assignment that several platforms do not ship Large datasets are handled by a spatial index built automatically on first access, alongside the tile pyramid: files over 64 MB are rewritten sorted by a spatial grid, which makes viewport queries roughly 27× faster on a 40M-row transcript file and also converts seqFISH CSV to parquet along the way. Set `SPATIAL_CACHE=0` to disable it. diff --git a/backend/app/main.py b/backend/app/main.py index fb15c48..b393894 100644 --- a/backend/app/main.py +++ b/backend/app/main.py @@ -3,7 +3,7 @@ from app.routers import tiles, spatial, edges, layers -APP_VERSION = "0.7.1" +APP_VERSION = "0.8.0" app = FastAPI(title="TissuePlex API", version=APP_VERSION) diff --git a/backend/app/readers/base_reader.py b/backend/app/readers/base_reader.py index e4f2338..87dfd66 100644 --- a/backend/app/readers/base_reader.py +++ b/backend/app/readers/base_reader.py @@ -12,7 +12,8 @@ import pandas as pd -from app.readers import supplemental +from app.readers import metadata_filter, supplemental +from app.readers.metadata_filter import MetadataFilter _UNSET = object() # sentinel: "not yet loaded" vs "loaded, no data" @@ -44,6 +45,7 @@ class SpatialDatasetReader(ABC): def __init__(self, dataset_path: Path): self.path = dataset_path self._supp_meta_cache = _UNSET + self._filter_id_cache: dict = {} # ── Identity ────────────────────────────────────────────────────────────── @@ -106,10 +108,22 @@ def cell_boundaries( self, bbox: Optional[tuple] = None, fraction: float = 1.0, - ) -> list[dict]: + cell_ids: Optional[set] = None, + ) -> dict: """Boundary vertex records in pixel space. - Required keys: cell_id, vertex_x, vertex_y. - fraction: 0 < f ≤ 1.0 — randomly sample this fraction of cells in viewport.""" + + Returns {"boundaries": list[dict], "total": int}; each record carries + cell_id, vertex_x, vertex_y. + + fraction : 0 < f ≤ 1.0 — randomly sample this fraction of cells in viewport. + cell_ids : when given, restrict to these cell ids (issue #45's metadata + filter, already resolved by `filter_cell_ids`). + + **Apply `cell_ids` before sampling and before computing `total`.** Sampling + first would draw from the whole viewport and leave only the small fraction + of the subset that happened to survive, so filtering to a rare cluster would + empty the canvas rather than isolate it. + """ ... @abstractmethod @@ -129,14 +143,100 @@ def color_values( mode: str, field: Optional[str] = None, genes: Optional[list[str]] = None, + categorical: Optional[bool] = None, ) -> dict: """Per-cell values for coloring. Returns one of: {type:'continuous', values:{cell_id:float}, min:float, max:float} {type:'categorical', values:{cell_id:str}, categories:[str,...]} + + `categorical` is the user's explicit override for a metadata column: + None auto-detects, True/False force the interpretation (issue #35). """ ... + # ── Metadata typing, colouring and filtering (platform-agnostic) ────────── + # + # These sit on the base class because they are pure pandas over whatever frame + # the reader calls its cells table. Every platform previously carried its own + # near-identical copy of `_color_values_meta`, and the copies had already + # drifted — one filled NaN with 0, others dropped it; some sorted categories + # with `key=str` and some without. All a reader supplies now is the frame. + + def _metadata_frame(self) -> Optional[pd.DataFrame]: + """The cells table used for colouring and filtering, including supplemental + columns, or None. Must contain a `cell_id` column. + + Readers override this to point at whatever they already build — usually + `_cells_full()`. The default returns None, which degrades to "no metadata + columns" rather than raising. + """ + return None + + def _color_values_meta(self, field: str, + categorical: Optional[bool] = None) -> dict: + """Per-cell values for one metadata column, typed for the frontend.""" + empty = {"type": "continuous", "values": {}, "min": 0.0, "max": 0.0} + df = self._metadata_frame() + if df is None or df.empty or "cell_id" not in df.columns \ + or field not in df.columns: + return empty + + col = df[field] + ids = df["cell_id"].astype(str).tolist() + has = col.notna().tolist() + + if metadata_filter.is_categorical(col, categorical): + labels = col.astype(str).tolist() + values = {ids[i]: labels[i] for i in range(len(ids)) if has[i]} + return { + "type": "categorical", + "values": values, + "categories": metadata_filter.sort_categories(set(values.values())), + } + + # to_numeric rather than float(): a forced-continuous request can land on a + # column holding stray text, and coercing those rows to NaN drops them the + # same way a genuinely missing value is dropped. + numeric = pd.to_numeric(col, errors="coerce") + valid = numeric.notna().tolist() + vals = numeric.tolist() + values = {ids[i]: float(vals[i]) for i in range(len(ids)) if valid[i]} + if not values: + return empty + finite = [v for v in values.values() if math.isfinite(v)] + if not finite: + return empty + return {"type": "continuous", "values": values, + "min": min(finite), "max": max(finite)} + + def filter_cell_ids(self, spec: Optional[MetadataFilter]) -> Optional[set]: + """Resolve a metadata filter to the set of cell ids it keeps (issue #45). + + Returns None when there is nothing to filter, so callers can pass the + result straight through and treat None as "no restriction". + + Raises ValueError for an unknown column. Silently ignoring it would render + the full dataset while the panel showed an active filter, which reads as + the filter being broken rather than misspelled. + + Cached per (reader instance, filter), since the same filter is re-resolved + on every viewport change while the user pans. + """ + if spec is None: + return None + df = self._metadata_frame() + if df is None or "cell_id" not in df.columns: + raise ValueError("this dataset exposes no cell metadata to filter on") + if spec.field not in df.columns: + raise ValueError(f"unknown cell metadata column '{spec.field}'") + if spec in self._filter_id_cache: + return self._filter_id_cache[spec] + keep = df.loc[spec.mask(df[spec.field]), "cell_id"].astype(str) + ids = set(keep.tolist()) + self._filter_id_cache[spec] = ids + return ids + # ── Platform capabilities ───────────────────────────────────────────────── def capabilities(self) -> dict: diff --git a/backend/app/readers/cosmx_reader.py b/backend/app/readers/cosmx_reader.py index b688c5b..b87f03f 100644 --- a/backend/app/readers/cosmx_reader.py +++ b/backend/app/readers/cosmx_reader.py @@ -301,7 +301,8 @@ def _polygon_file(self) -> Optional[Path]: return None def cell_boundaries(self, bbox: Optional[tuple] = None, - fraction: float = 1.0) -> dict: + fraction: float = 1.0, + cell_ids: Optional[set] = None) -> dict: """Cell outlines in pixel space from the polygons CSV. One row per vertex, already in global pixels, so no FOV offset has to be @@ -342,6 +343,10 @@ def cell_boundaries(self, bbox: Optional[tuple] = None, visible = conn.execute( f"SELECT DISTINCT {key} AS cid FROM {src} WHERE {sql}", params ).df()["cid"].tolist() + # Metadata filter (issue #45) — narrow the visible set before the + # count so `total` and the sample both describe the subset. + if cell_ids is not None: + visible = [c for c in visible if str(c) in cell_ids] if not visible: return {"boundaries": [], "total": 0} total = len(visible) @@ -360,6 +365,8 @@ def cell_boundaries(self, bbox: Optional[tuple] = None, df = conn.execute( f'SELECT {key} AS cell_id, "x_global_px", "y_global_px" ' f"FROM {src}").df() + if cell_ids is not None: + df = df[df["cell_id"].astype(str).isin(cell_ids)] total = df["cell_id"].nunique() fraction = max(0.0001, min(1.0, fraction)) n = round(fraction * total) @@ -404,33 +411,15 @@ def color_values( mode: str, field: Optional[str] = None, genes: Optional[list[str]] = None, + categorical: Optional[bool] = None, ) -> dict: if mode == "gene_set": # Gene-set coloring requires aggregating transcripts per cell — stub return {"type": "continuous", "values": {}, "min": 0.0, "max": 0.0} - return self._color_values_meta(field or "") + return self._color_values_meta(field or "", categorical) - def _color_values_meta(self, field: str) -> dict: - df = self._load_cells() - if df is None or field not in df.columns: - return {"type": "continuous", "values": {}, "min": 0.0, "max": 0.0} - col = df[field] - cell_ids = df["cell_id"].astype(str).tolist() - is_categorical = ( - pd.api.types.is_string_dtype(col) or - pd.api.types.is_object_dtype(col) or - (pd.api.types.is_integer_dtype(col) and col.nunique() <= 30) - ) - if is_categorical: - labels = col.fillna("").astype(str).tolist() - categories = sorted(col.dropna().astype(str).unique().tolist()) - return {"type": "categorical", - "values": {cell_ids[i]: labels[i] for i in range(len(cell_ids))}, - "categories": categories} - filled = col.fillna(0) - return {"type": "continuous", - "values": {cell_ids[i]: float(filled.iloc[i]) for i in range(len(cell_ids))}, - "min": float(filled.min()), "max": float(filled.max())} + def _metadata_frame(self): + return self._load_cells() # ── Internal helpers ────────────────────────────────────────────────────── diff --git a/backend/app/readers/duck.py b/backend/app/readers/duck.py index cbbb75a..18fc7ca 100644 --- a/backend/app/readers/duck.py +++ b/backend/app/readers/duck.py @@ -138,6 +138,24 @@ def in_predicate(col: str, values: list) -> tuple[str, list]: return f'"{col}" IN ({placeholders})', list(values) +def register_ids(conn, ids, name: str = "tp_filter", col: str = "cell_id") -> str: + """Register a set of ids as a relation and return a semi-join predicate on it. + + The metadata filter (issue #45) can keep hundreds of thousands of cells, which + is far past the point where an ``IN (?, ?, …)`` list is workable — DuckDB has + to bind every parameter, and the SQL text itself grows to megabytes. Handing + the ids over as a one-column frame instead makes it an ordinary hash semi-join. + + ``ids`` must not be empty; an empty filter means "nothing matches" and callers + should short-circuit rather than build a query that cannot return rows. + """ + import pandas as pd + + frame = pd.DataFrame({col: [str(i) for i in ids]}) + conn.register(name, frame) + return f"IN (SELECT \"{col}\" FROM {name})" + + # Sampling is seeded so that re-fetching an unchanged viewport returns the same # rows. Without this, every refetch reshuffles which transcripts are drawn and # the layer visibly flickers. diff --git a/backend/app/readers/edge_reader.py b/backend/app/readers/edge_reader.py index d9ff864..6896852 100644 --- a/backend/app/readers/edge_reader.py +++ b/backend/app/readers/edge_reader.py @@ -16,9 +16,11 @@ from typing import Optional import duckdb import pandas as pd +import pyarrow as pa import pyarrow.parquet as pq -from app.readers import supplemental +from app.readers import duck, metadata_filter, supplemental +from app.readers.metadata_filter import MetadataFilter _DUCKDB_MEMORY_LIMIT = os.getenv("DUCKDB_MEMORY_LIMIT", "8GB") @@ -130,13 +132,15 @@ def lrm_catalogue(self) -> list[dict]: return self._lrm_catalogue_cache def edge_color_values(self, mode: str, lrms: list[str] | None = None, - field: str | None = None) -> dict: + field: str | None = None, + categorical: bool | None = None) -> dict: """ Return per-directed-edge color values. mode='lrm_set' — sum score across requested LRMs per edge; continuous mode='metadata' — group by edge, take first value of `field` per edge; - auto-detect categorical vs continuous + categorical vs continuous auto-detected unless the caller + overrides it with `categorical` (issue #35) """ col_names = set(self._parquet_schema().names) @@ -182,28 +186,100 @@ def edge_color_values(self, mode: str, lrms: list[str] | None = None, col = supp.set_index("edge")[field].dropna() if col.empty: return {"type": "continuous", "values": {}, "min": 0, "max": 0} - dtype = str(col.dtype) - n_unique = col.nunique() - is_cat = ( - dtype in ("object", "string", "bool") - or (dtype.startswith("int") and n_unique <= 30) - ) - if is_cat: - categories = sorted(col.dropna().unique().tolist(), key=str) + # Same typing rule as the cell side, from the shared module, so the + # two color panels can never disagree about what is categorical. + if metadata_filter.is_categorical(col, categorical): + labels = col.dropna().astype(str) return { "type": "categorical", - "values": col.to_dict(), - "categories": categories, + "values": labels.to_dict(), + "categories": metadata_filter.sort_categories(labels.unique()), } + numeric = pd.to_numeric(col, errors="coerce").dropna() + if numeric.empty: + return {"type": "continuous", "values": {}, "min": 0, "max": 0} return { "type": "continuous", - "values": col.to_dict(), - "min": float(col.min()), - "max": float(col.max()), + "values": numeric.to_dict(), + "min": float(numeric.min()), + "max": float(numeric.max()), } return {"type": "continuous", "values": {}, "min": 0, "max": 0} + # ── Metadata filtering (issue #45) ──────────────────────────────────────── + + def edge_filter_sql(self, spec: Optional[MetadataFilter], conn) -> tuple[str, list]: + """WHERE fragment restricting the query to edges matching `spec`. + + Two paths, because edge metadata has two sources: + + * A column in ``edges.parquet`` becomes an ordinary SQL predicate, which + DuckDB can push down and use for row-group pruning. + * A column from ``edge-metadata/`` only exists in pandas, so the matching + edge ids are resolved there and registered as a relation to semi-join + against — the same trick the cell filter uses, and for the same reason: + the id list is far too long to bind as parameters. + + Raises ValueError for an unknown column rather than quietly returning the + unfiltered view, which would look like the filter had failed. + """ + if spec is None: + return "", [] + schema = self._parquet_schema() + parquet_cols = set(schema.names) + if spec.field in parquet_cols: + quoted = f'"{spec.field}"' + if spec.values is not None: + # Compare as text so one code path covers int, float and string + # columns. Booleans need lowering: the categories the panel offers + # come from pandas, which writes "True", while DuckDB's cast writes + # "true", so a literal comparison would never match. + cast = f"CAST({quoted} AS VARCHAR)" + vals = list(spec.values) + if pa.types.is_boolean(schema.field(spec.field).type): + cast = f"lower({cast})" + vals = [v.lower() for v in vals] + ph = ", ".join("?" for _ in vals) + sql = f"{cast} IN ({ph})" + params = vals + else: + parts, params = [], [] + if spec.vmin is not None: + parts.append(f"{quoted} >= ?"); params.append(spec.vmin) + if spec.vmax is not None: + parts.append(f"{quoted} <= ?"); params.append(spec.vmax) + sql = " AND ".join(parts) if parts else "" + if spec.include_missing and sql: + sql = f"({sql} OR {quoted} IS NULL)" + return sql, params + + supp = self._supplemental() + if supp is None or spec.field not in supp.columns: + raise ValueError(f"unknown edge metadata column '{spec.field}'") + keep = supp.loc[spec.mask(supp[spec.field]), "edge"].astype(str) + if keep.empty: + return "FALSE", [] + pred = duck.register_ids(conn, keep.tolist(), name="tp_edge_filter", col="edge") + return f'CAST("edge" AS VARCHAR) {pred}', [] + + @staticmethod + def cell_filter_sql(cell_ids: Optional[set], conn) -> tuple[str, list]: + """WHERE fragment keeping only edges whose **both** endpoints survive the + cell filter. + + Both, not either: the point of "focus on 2–3 cell types" is the signalling + *within* that subset. An edge with one endpoint outside would be drawn + running off to a cell that is not on screen. + """ + if cell_ids is None: + return "", [] + if not cell_ids: + return "FALSE", [] + pred = duck.register_ids(conn, cell_ids, name="tp_cell_filter") + return (f'CAST("sending_cell" AS VARCHAR) {pred} ' + f'AND CAST("receiving_cell" AS VARCHAR) {pred}'), [] + def edge_detail(self, edge_id: str) -> dict | None: """Return all LRM rows for a single directed edge, structured for the info panel.""" with self._conn() as conn: @@ -278,6 +354,8 @@ def query_grouped( min_lrm_count: int = 1, density: float = 1.0, max_limit: int = 500_000, + cell_ids: Optional[set] = None, + edge_filter: Optional[MetadataFilter] = None, ) -> list[dict]: """ Return one row per directed edge (GROUP BY edge), pre-aggregated. @@ -294,6 +372,11 @@ def query_grouped( density=1.0 returns all edges in the viewport (up to max_limit). density<1.0 uses bernoulli sampling so each edge is independently included with probability `density` — spatially uniform. + + `cell_ids` and `edge_filter` are the metadata filters from issue #45. Both + go into the WHERE clause, so they run before the GROUP BY and before the + density sample: filtering to a rare cell type keeps that type's edges at + full density rather than sampling them away. """ ps = self.pixel_size schema_names = set(self._parquet_schema().names) @@ -315,8 +398,6 @@ def query_grouped( where_params.extend([xmin_u, xmax_u, ymin_u, ymax_u, xmin_u, xmax_u, ymin_u, ymax_u]) - where = f"WHERE {' AND '.join(where_conditions)}" if where_conditions else "" - # Build SELECT columns agg_cols = ["edge"] for col in ("sending_cell", "receiving_cell", "is_autocrine", @@ -346,18 +427,36 @@ def query_grouped( if density < 1.0 else "" ) - sql = f""" - SELECT * FROM ( - SELECT {select} - FROM {self._from()} - {where} - GROUP BY edge - HAVING lrm_count >= 1 - ) {sample_clause} - LIMIT {max_limit} - """ + # The connection is opened before the WHERE clause is finalised because the + # metadata filters may need to register a relation on it to semi-join + # against. Filter conditions are appended after the bbox so the parameter + # order still matches the order the placeholders appear in the SQL text — + # DuckDB binds positionally by text order, not by clause. + # An edge file without endpoint columns cannot be filtered by cell; the + # tissue graph still draws, it just ignores the cell subset. + if not {"sending_cell", "receiving_cell"} <= schema_names: + cell_ids = None with self._conn() as conn: + for cond, prm in ( + self.cell_filter_sql(cell_ids, conn), + self.edge_filter_sql(edge_filter, conn), + ): + if cond: + where_conditions.append(cond) + where_params.extend(prm) + where = f"WHERE {' AND '.join(where_conditions)}" if where_conditions else "" + + sql = f""" + SELECT * FROM ( + SELECT {select} + FROM {self._from()} + {where} + GROUP BY edge + HAVING lrm_count >= 1 + ) {sample_clause} + LIMIT {max_limit} + """ df = conn.execute(sql, where_params).df() for col in ("x1", "y1", "x2", "y2"): diff --git a/backend/app/readers/merscope_reader.py b/backend/app/readers/merscope_reader.py index 74094c9..61412d4 100644 --- a/backend/app/readers/merscope_reader.py +++ b/backend/app/readers/merscope_reader.py @@ -319,7 +319,8 @@ def _boundary_file(self) -> Optional[Path]: return self._boundary_path_cache # type: ignore[return-value] def cell_boundaries(self, bbox: Optional[tuple] = None, - fraction: float = 1.0) -> dict: + fraction: float = 1.0, + cell_ids: Optional[set] = None) -> dict: """Cell polygons in pixel space, from the geoparquet boundary file. Software v231 and earlier wrote per-FOV HDF5 instead @@ -351,6 +352,10 @@ def cell_boundaries(self, bbox: Optional[tuple] = None, for entity, blob in zip(df["EntityID"].astype(str), df["Geometry"]): if blob is None: continue + # Metadata filter (issue #45): skip before decoding the WKB, which is + # the expensive part, and before `total` so sampling sees the subset. + if cell_ids is not None and entity not in cell_ids: + continue try: rings = _wkb_polygons(bytes(blob)) except Exception: @@ -412,10 +417,14 @@ def cell_expression(self, cell_id: str) -> dict: # ── Colour values ───────────────────────────────────────────────────────── def color_values(self, mode: str, field: Optional[str] = None, - genes: Optional[list[str]] = None) -> dict: + genes: Optional[list[str]] = None, + categorical: Optional[bool] = None) -> dict: if mode == "gene_set": return self._color_values_gene_set(genes or []) - return self._color_values_meta(field or "") + return self._color_values_meta(field or "", categorical) + + def _metadata_frame(self): + return self._cells_full() def _color_values_gene_set(self, genes: list[str]) -> dict: empty = {"type": "continuous", "values": {}, "min": 0.0, "max": 0.0} @@ -443,33 +452,6 @@ def _color_values_gene_set(self, genes: list[str]) -> dict: return {"type": "continuous", "values": vals, "min": 0.0, "max": vmax if vmax > 0 else 1.0} - def _color_values_meta(self, field: str) -> dict: - empty = {"type": "continuous", "values": {}, "min": 0.0, "max": 0.0} - df = self._cells_full() - if df is None or field not in df.columns: - return empty - col = df[field] - ids = df["cell_id"].astype(str).tolist() - has = col.notna() - categorical = ( - pd.api.types.is_string_dtype(col) or pd.api.types.is_object_dtype(col) - or (pd.api.types.is_integer_dtype(col) and col.nunique() <= 30) - ) - if categorical: - return { - "type": "categorical", - "values": {ids[i]: str(col.iloc[i]) for i in range(len(ids)) if has.iloc[i]}, - "categories": sorted(col[has].astype(str).unique().tolist(), key=str), - } - valid = col[has] - if valid.empty: - return empty - return { - "type": "continuous", - "values": {ids[i]: float(col.iloc[i]) for i in range(len(ids)) if has.iloc[i]}, - "min": float(valid.min()), "max": float(valid.max()), - } - def spatial_cache_sorted(reader, path: Path, xcol: str, ycol: str) -> Path: """Spatially-sorted copy of a transcript table, or the original.""" diff --git a/backend/app/readers/metadata_filter.py b/backend/app/readers/metadata_filter.py new file mode 100644 index 0000000..87969ec --- /dev/null +++ b/backend/app/readers/metadata_filter.py @@ -0,0 +1,126 @@ +""" +Metadata typing and subsetting, shared by every reader. + +Two related problems live here because they are the same problem seen twice: + +* **Is this column categorical or continuous?** (issue #35) Seurat writes cluster + IDs as integers, so dtype alone routes them to a viridis gradient when the user + wants twenty distinct colours. The auto-rule below guesses, and an explicit + caller override wins over the guess. + +* **Which units does the user want to see?** (issue #45) The same column, read the + same way, also drives "show me only clusters 4 and 7" — a categorical allowlist + or a numeric range that restricts what gets rendered. + +Both are expressed against a single pandas column so cells and edges behave +identically; `base_reader` uses this for the cells table and `edge_reader` for the +edge table. +""" +from dataclasses import dataclass +from typing import Optional + +import pandas as pd + +# Above this many distinct integers, a column is assumed to be a measurement rather +# than a code. Cluster IDs, phenotype codes and bin assignments sit well below it; +# transcript counts sit well above. Users past the threshold reach for the explicit +# "treat as categorical" toggle, which is exactly what issue #35 asked for. +CATEGORICAL_MAX_UNIQUE = 30 + + +def is_categorical(col: pd.Series, forced: Optional[bool] = None) -> bool: + """Decide how a metadata column should be coloured. + + ``forced`` is the user's explicit choice from the color panel: + * ``None`` — auto-detect (the historical behaviour) + * ``True`` — treat as categorical whatever the dtype + * ``False`` — treat as continuous, but only if the values are actually + numeric; a text column has no gradient to draw, so the request is + ignored rather than silently rendering every cell the same colour. + """ + numeric = pd.api.types.is_numeric_dtype(col) and not pd.api.types.is_bool_dtype(col) + if forced is True: + return True + if forced is False: + return not numeric + return ( + pd.api.types.is_string_dtype(col) + or pd.api.types.is_object_dtype(col) + or pd.api.types.is_bool_dtype(col) + or isinstance(col.dtype, pd.CategoricalDtype) + or (pd.api.types.is_integer_dtype(col) and col.nunique() <= CATEGORICAL_MAX_UNIQUE) + ) + + +def sort_categories(labels) -> list[str]: + """Order category labels, numerically when they are all numbers. + + Labels reach the legend as strings, so a plain sort puts cluster 10 between 1 + and 2. Issue #35 asks for the numeric order to survive, and it costs one parse + attempt: if every label is a number the sort key is that number, otherwise it + falls back to the lexicographic order used before. + """ + labels = [str(v) for v in labels] + try: + return sorted(labels, key=lambda s: (float(s), s)) + except (TypeError, ValueError): + return sorted(labels) + + +@dataclass(frozen=True) +class MetadataFilter: + """A restriction of the view to a subset of units (issue #45). + + Exactly one of the two forms is meaningful: + + * ``values`` — a categorical allowlist, compared against ``str(value)`` so it + works regardless of whether the column arrived as int, float or text. + * ``vmin`` / ``vmax`` — an inclusive numeric range; either end may be open. + + ``include_missing`` decides what happens to rows where the column is NaN. It + defaults to False: a cell with no cluster call is not part of "cluster 4". + """ + + field: str + values: Optional[tuple] = None + vmin: Optional[float] = None + vmax: Optional[float] = None + include_missing: bool = False + + @classmethod + def build(cls, field: Optional[str], values=None, vmin=None, vmax=None, + include_missing: bool = False) -> Optional["MetadataFilter"]: + """Construct from loose router input, or None when nothing is constrained. + + A field name with no values and no bounds is not a filter — it is a column + the user has selected but not yet narrowed — so it returns None and the + caller renders everything. + """ + if not field: + return None + vals = tuple(str(v) for v in values) if values else None + if vals is None and vmin is None and vmax is None: + return None + return cls( + field=field, + values=vals, + vmin=None if vmin is None else float(vmin), + vmax=None if vmax is None else float(vmax), + include_missing=bool(include_missing), + ) + + def mask(self, col: pd.Series) -> pd.Series: + """Boolean mask over ``col`` selecting the rows this filter keeps.""" + present = col.notna() + if self.values is not None: + keep = col.astype(str).isin(set(self.values)) & present + else: + numeric = pd.to_numeric(col, errors="coerce") + keep = numeric.notna() + if self.vmin is not None: + keep &= numeric >= self.vmin + if self.vmax is not None: + keep &= numeric <= self.vmax + if self.include_missing: + keep = keep | ~present + return keep diff --git a/backend/app/readers/seqfish_reader.py b/backend/app/readers/seqfish_reader.py index b573c4e..3048c74 100644 --- a/backend/app/readers/seqfish_reader.py +++ b/backend/app/readers/seqfish_reader.py @@ -388,7 +388,8 @@ def transcripts( # ── Boundaries ──────────────────────────────────────────────────────────── def cell_boundaries(self, bbox: Optional[tuple] = None, - fraction: float = 1.0) -> dict: + fraction: float = 1.0, + cell_ids: Optional[set] = None) -> dict: """Polygon vertices in pixel space, as long-format {cell_id, vertex_x, vertex_y} rows so the frontend needs no seqFISH-specific handling. @@ -430,6 +431,11 @@ def cell_boundaries(self, bbox: Optional[tuple] = None, if pts: polys.append((cid, pts)) + # Metadata filter (issue #45) — applied before the bbox so it also governs + # `total`, and therefore the fraction the frontend asks for next. + if cell_ids is not None: + polys = [(cid, pts) for cid, pts in polys if cid in cell_ids] + if not polys: return {"boundaries": [], "total": 0} @@ -498,10 +504,14 @@ def cell_expression(self, cell_id: str) -> dict: # ── Color values ────────────────────────────────────────────────────────── def color_values(self, mode: str, field: Optional[str] = None, - genes: Optional[list[str]] = None) -> dict: + genes: Optional[list[str]] = None, + categorical: Optional[bool] = None) -> dict: if mode == "gene_set": return self._color_values_gene_set(genes or []) - return self._color_values_meta(field or "") + return self._color_values_meta(field or "", categorical) + + def _metadata_frame(self): + return self._cells_full() def _color_values_gene_set(self, genes: list[str]) -> dict: df = self._cxg() @@ -519,32 +529,3 @@ def _color_values_gene_set(self, genes: list[str]) -> dict: "max": float(summed.max()) if len(summed) and summed.max() > 0 else 1.0, } - def _color_values_meta(self, field: str) -> dict: - df = self._cells_full() - empty = {"type": "continuous", "values": {}, "min": 0.0, "max": 0.0} - if df is None or field not in df.columns: - return empty - col = df[field] - ids = df["cell_id"].astype(str).tolist() - has = col.notna() - # Same rule as the other readers: strings are categorical, and so are - # low-cardinality integers (cluster IDs arrive as ints from Seurat). - categorical = ( - pd.api.types.is_string_dtype(col) or pd.api.types.is_object_dtype(col) - or (pd.api.types.is_integer_dtype(col) and col.nunique() <= 30) - ) - if categorical: - return { - "type": "categorical", - "values": {ids[i]: str(col.iloc[i]) for i in range(len(ids)) if has.iloc[i]}, - "categories": sorted(col[has].astype(str).unique().tolist()), - } - valid = col[has] - if valid.empty: - return empty - return { - "type": "continuous", - "values": {ids[i]: float(col.iloc[i]) for i in range(len(ids)) if has.iloc[i]}, - "min": float(valid.min()), - "max": float(valid.max()), - } diff --git a/backend/app/readers/visium_hd_reader.py b/backend/app/readers/visium_hd_reader.py index b00e1de..9c57b4f 100644 --- a/backend/app/readers/visium_hd_reader.py +++ b/backend/app/readers/visium_hd_reader.py @@ -282,7 +282,8 @@ def cell_detail(self, cell_id: str) -> Optional[dict]: # ── Boundaries: each bin as a square ────────────────────────────────────── def cell_boundaries(self, bbox: Optional[tuple] = None, - fraction: float = 1.0) -> dict: + fraction: float = 1.0, + cell_ids: Optional[set] = None) -> dict: """Bin outlines as square polygons, in pixel space. A bin qualifies if its centroid lies in the bbox, and then all four of its @@ -293,6 +294,10 @@ def cell_boundaries(self, bbox: Optional[tuple] = None, if df is None or df.empty: return {"boundaries": [], "total": 0} + # Metadata filter (issue #45), before the bbox count and the sample. + if cell_ids is not None: + df = df[df["cell_id"].astype(str).isin(cell_ids)] + if bbox: xmin, ymin, xmax, ymax = bbox if None not in (xmin, ymin, xmax, ymax): @@ -397,10 +402,14 @@ def cell_expression(self, cell_id: str) -> dict: # ── Colour values ───────────────────────────────────────────────────────── def color_values(self, mode: str, field: Optional[str] = None, - genes: Optional[list[str]] = None) -> dict: + genes: Optional[list[str]] = None, + categorical: Optional[bool] = None) -> dict: if mode == "gene_set": return self._color_values_gene_set(genes or []) - return self._color_values_meta(field or "") + return self._color_values_meta(field or "", categorical) + + def _metadata_frame(self): + return self._cells_full() def _color_values_gene_set(self, genes: list[str]) -> dict: empty = {"type": "continuous", "values": {}, "min": 0.0, "max": 0.0} @@ -421,30 +430,3 @@ def _color_values_gene_set(self, genes: list[str]) -> dict: "max": vmax, } - def _color_values_meta(self, field: str) -> dict: - empty = {"type": "continuous", "values": {}, "min": 0.0, "max": 0.0} - df = self._cells_full() - if df is None or df.empty or field not in df.columns: - return empty - col = df[field] - ids = df["cell_id"].astype(str).tolist() - has = col.notna() - categorical = ( - pd.api.types.is_string_dtype(col) or pd.api.types.is_object_dtype(col) - or (pd.api.types.is_integer_dtype(col) and col.nunique() <= 30) - ) - if categorical: - return { - "type": "categorical", - "values": {ids[i]: str(col.iloc[i]) for i in range(len(ids)) if has.iloc[i]}, - "categories": sorted(col[has].astype(str).unique().tolist(), key=str), - } - valid = col[has] - if valid.empty: - return empty - return { - "type": "continuous", - "values": {ids[i]: float(col.iloc[i]) for i in range(len(ids)) if has.iloc[i]}, - "min": float(valid.min()), - "max": float(valid.max()), - } diff --git a/backend/app/readers/xenium_reader.py b/backend/app/readers/xenium_reader.py index 00414e6..ebda33c 100644 --- a/backend/app/readers/xenium_reader.py +++ b/backend/app/readers/xenium_reader.py @@ -202,7 +202,8 @@ def cells_schema(self) -> dict: # ── Cell boundaries ─────────────────────────────────────────────────────── - def cell_boundaries(self, bbox: Optional[tuple] = None, fraction: float = 1.0) -> dict: + def cell_boundaries(self, bbox: Optional[tuple] = None, fraction: float = 1.0, + cell_ids: Optional[set] = None) -> dict: """Cell polygon vertices in pixel space for cells visible in the bbox. Selection is per *cell*, not per vertex. A cell qualifies if any one of its @@ -212,6 +213,11 @@ def cell_boundaries(self, bbox: Optional[tuple] = None, fraction: float = 1.0) - torn shapes. Sampling likewise draws whole cells, so a sampled cell is never missing part of its outline. + ``cell_ids`` narrows the query to a metadata-filtered subset (issue #45). + It joins in the same WHERE clause as the bbox, so it applies before both + the count and the sample — filtering to a rare cluster isolates it rather + than thinning it. + ``total`` is the number of distinct cells touching the bbox before sampling — ``useCellBoundaries`` divides its ~5K target by this to pick the next fraction, so it has to stay a pre-sample count. @@ -219,6 +225,8 @@ def cell_boundaries(self, bbox: Optional[tuple] = None, fraction: float = 1.0) - path = self.path / "cell_boundaries.parquet" if not path.exists(): return {"boundaries": [], "total": 0} + if cell_ids is not None and not cell_ids: + return {"boundaries": [], "total": 0} cols = duck.columns(path) x_col = next((c for c in cols if "vertex_x" in c), None) @@ -230,11 +238,14 @@ def cell_boundaries(self, bbox: Optional[tuple] = None, fraction: float = 1.0) - bbox_sql, bbox_params = duck.bbox_predicate( x_col, y_col, self._bbox_to_native(bbox) if bbox else None ) - where = duck.where_clause([bbox_sql]) src = duck.scan(path) select = f'"cell_id", "{x_col}", "{y_col}"' with duck.connect() as conn: + filter_sql = "" + if cell_ids is not None: + filter_sql = f'CAST("cell_id" AS VARCHAR) {duck.register_ids(conn, cell_ids)}' + where = duck.where_clause([bbox_sql, filter_sql]) total = conn.execute( f"SELECT COUNT(DISTINCT cell_id) FROM {src} {where}", bbox_params ).fetchone()[0] @@ -318,10 +329,14 @@ def color_values( mode: str, field: Optional[str] = None, genes: Optional[list[str]] = None, + categorical: Optional[bool] = None, ) -> dict: if mode == "gene_set": return self._color_values_gene_set(genes or []) - return self._color_values_meta(field or "") + return self._color_values_meta(field or "", categorical) + + def _metadata_frame(self): + return self._cells_full() def _color_values_gene_set(self, genes: list[str]) -> dict: h5 = self.path / "cell_feature_matrix.h5" @@ -351,35 +366,6 @@ def _color_values_gene_set(self, genes: list[str]) -> dict: except Exception: return {"type": "continuous", "values": {}, "min": 0.0, "max": 0.0} - def _color_values_meta(self, field: str) -> dict: - df = self._cells_full() - if df is None or field not in df.columns: - return {"type": "continuous", "values": {}, "min": 0.0, "max": 0.0} - col = df[field] - cell_ids = df["cell_id"].astype(str).tolist() - has_value = col.notna() - is_categorical = ( - pd.api.types.is_string_dtype(col) or - pd.api.types.is_object_dtype(col) or - (pd.api.types.is_integer_dtype(col) and col.nunique() <= 30) - ) - if is_categorical: - categories = sorted(col[has_value].astype(str).unique().tolist()) - values = { - cell_ids[i]: str(col.iloc[i]) - for i in range(len(cell_ids)) if has_value.iloc[i] - } - return {"type": "categorical", "values": values, "categories": categories} - valid = col[has_value] - if valid.empty: - return {"type": "continuous", "values": {}, "min": 0.0, "max": 0.0} - values = { - cell_ids[i]: float(col.iloc[i]) - for i in range(len(cell_ids)) if has_value.iloc[i] - } - return {"type": "continuous", "values": values, - "min": float(valid.min()), "max": float(valid.max())} - # ── Supplemental metadata ───────────────────────────────────────────────── # The loader itself lives on SpatialDatasetReader so every platform gets it. # All Xenium contributes is the list of its own root CSVs to ignore. diff --git a/backend/app/routers/edges.py b/backend/app/routers/edges.py index 7b71c0b..58540af 100644 --- a/backend/app/routers/edges.py +++ b/backend/app/routers/edges.py @@ -20,6 +20,7 @@ from typing import Optional, List from app.readers.edge_reader import EdgeReader +from app.readers.metadata_filter import MetadataFilter router = APIRouter() @@ -142,6 +143,25 @@ def list_edge_files(dataset: str): return {"files": files, "default": default} +class MetadataFilterSpec(BaseModel): + """A metadata restriction (issue #45), for either the cell or the edge table. + + `values` is a categorical allowlist; `min`/`max` an inclusive numeric range. + Sending a field with neither is not an error — it means the user has picked a + column but not yet narrowed it, and everything is returned. + """ + field: Optional[str] = None + values: Optional[List[str]] = None + min: Optional[float] = None + max: Optional[float] = None + include_missing: bool = False + + def build(self) -> Optional[MetadataFilter]: + return MetadataFilter.build( + self.field, self.values, self.min, self.max, self.include_missing + ) + + class EdgeGroupedQueryRequest(BaseModel): xmin: Optional[float] = None ymin: Optional[float] = None @@ -149,6 +169,29 @@ class EdgeGroupedQueryRequest(BaseModel): ymax: Optional[float] = None min_strength: Optional[float] = None density: float = 1.0 # fraction of viewport edges to return (0.01–1.0) + # cell_filter restricts by *cell* metadata: an edge survives only if both of + # its endpoints do. edge_filter restricts by a column of the edge table itself + # (or of edge-metadata/). They compose. + cell_filter: Optional[MetadataFilterSpec] = None + edge_filter: Optional[MetadataFilterSpec] = None + + +def _cell_ids_for(dataset: str, spec: Optional[MetadataFilterSpec]) -> Optional[set]: + """Resolve a cell-metadata filter through the *spatial* reader. + + The edge router has no cells table of its own, so it borrows the platform + reader — which is also what keeps the cell-id vocabulary identical on both + sides, the same assumption `edges.parquet` already makes when it stores + barcodes in `sending_cell`. + """ + built = spec.build() if spec else None + if built is None: + return None + from app.routers import spatial + try: + return spatial._reader(dataset).filter_cell_ids(built) + except ValueError as exc: + raise HTTPException(400, str(exc)) @router.post("/{dataset}/query-grouped") @@ -163,10 +206,15 @@ def query_edges_grouped(dataset: str, body: EdgeGroupedQueryRequest, bbox = (body.xmin, body.ymin, body.xmax, body.ymax) \ if body.xmin is not None else None density = max(0.001, min(1.0, body.density)) - return _reader(dataset, edge_file).query_grouped( - bbox=bbox, - density=density, - ) + try: + return _reader(dataset, edge_file).query_grouped( + bbox=bbox, + density=density, + cell_ids=_cell_ids_for(dataset, body.cell_filter), + edge_filter=body.edge_filter.build() if body.edge_filter else None, + ) + except ValueError as exc: + raise HTTPException(400, str(exc)) class EdgeScoreQueryRequest(BaseModel): @@ -208,6 +256,8 @@ class EdgeColorRequest(BaseModel): mode: str # "lrm_set" | "metadata" lrms: Optional[List[str]] = None # for lrm_set: list of "ligand|receptor" strings field: Optional[str] = None # for metadata: column name + # None = auto-detect; True/False force the interpretation (issue #35). + categorical: Optional[bool] = None @router.post("/{dataset}/edge-color-values") @@ -216,9 +266,12 @@ def edge_color_values(dataset: str, body: EdgeColorRequest, """ Return per-directed-edge color values. lrm_set: sum score for the supplied LRM list, one value per edge. - metadata: return first value of `field` per edge (auto-detects cat/continuous). + metadata: return first value of `field` per edge (auto-detects cat/continuous + unless `categorical` overrides it). """ - return _reader(dataset, edge_file).edge_color_values(body.mode, body.lrms, body.field) + return _reader(dataset, edge_file).edge_color_values( + body.mode, body.lrms, body.field, body.categorical + ) @router.get("/{dataset}/edge/{edge_id:path}") diff --git a/backend/app/routers/spatial.py b/backend/app/routers/spatial.py index da7d498..0eed30f 100644 --- a/backend/app/routers/spatial.py +++ b/backend/app/routers/spatial.py @@ -13,6 +13,7 @@ import io import os +from app.readers.metadata_filter import MetadataFilter from app.readers.reader_factory import ReaderFactory router = APIRouter() @@ -202,12 +203,31 @@ def cell_boundaries( xmax: float = Query(None), ymax: float = Query(None), fraction: float = Query(1.0), + filter_field: str = Query(None, description="Metadata column to restrict on"), + filter_values: List[str] = Query(None, description="Categorical allowlist"), + filter_min: float = Query(None, description="Inclusive lower bound"), + filter_max: float = Query(None, description="Inclusive upper bound"), + filter_missing: bool = Query(False, description="Also keep units with no value"), ): - """Cell polygon boundaries filtered by bounding box. - fraction: 0–1 fraction of cells in viewport to return (randomly sampled).""" - return _reader(dataset).cell_boundaries( + """Cell polygon boundaries filtered by bounding box and, optionally, metadata. + + fraction: 0–1 fraction of cells in viewport to return (randomly sampled). + + The metadata filter (issue #45) is resolved to a cell-id set and applied inside + the reader *before* sampling, so restricting to a rare cluster isolates it at + full density instead of thinning it to almost nothing. + """ + reader = _reader(dataset) + try: + cell_ids = reader.filter_cell_ids(MetadataFilter.build( + filter_field, filter_values, filter_min, filter_max, filter_missing, + )) + except ValueError as exc: + raise HTTPException(400, str(exc)) + return reader.cell_boundaries( bbox=(xmin, ymin, xmax, ymax) if xmin is not None else None, fraction=max(0.0001, min(1.0, fraction)), + cell_ids=cell_ids, ) @@ -235,9 +255,14 @@ class ColorValuesRequest(BaseModel): mode: str field: Optional[str] = None genes: Optional[List[str]] = None + # None = auto-detect from dtype and cardinality; True/False force the + # interpretation of a metadata column (issue #35 — integer cluster IDs). + categorical: Optional[bool] = None @router.post("/{dataset}/color-values") def color_values_post(dataset: str, body: ColorValuesRequest): """Per-cell color values for gene_set or metadata coloring.""" - return _reader(dataset).color_values(body.mode, body.field, body.genes) + return _reader(dataset).color_values( + body.mode, body.field, body.genes, body.categorical + ) diff --git a/backend/tests/golden_baseline.json b/backend/tests/golden_baseline.json index 95dfe8d..d362a25 100644 --- a/backend/tests/golden_baseline.json +++ b/backend/tests/golden_baseline.json @@ -73,6 +73,13 @@ "y_centroid": "float64" } }, + "color_forced_cat__fov": { + "categories_digest": "5d0026d09770336c", + "digest": "8e360a57101220c4", + "n": 38996, + "n_categories": 66, + "type": "categorical" + }, "color_gene_set": { "digest": "44136fa355b3678a", "max": 0.0, @@ -201,6 +208,13 @@ "y_centroid": "float64" } }, + "color_forced_cat__volume": { + "categories_digest": "5184d74a79a96f9b", + "digest": "09e3b3fcdf575db2", + "n": 997, + "n_categories": 995, + "type": "categorical" + }, "color_gene_set": { "digest": "0082fee906d39bfe", "max": 137.0, @@ -299,7 +313,7 @@ "has_transcripts": true, "unit_label": "cell" }, - "cell_detail_0": "79d24a3e5b058813", + "cell_detail_0": "9f848959f6710dd8", "cell_expression_0": { "digest": "e4e6708c3bf53c95", "n": 3 @@ -337,6 +351,7 @@ "pseudotime": "float64", "region": "object", "segmentation_method": "object", + "seurat_clusters": "int64", "total_counts": "int64", "transcript_counts": "int64", "unassigned_codeword_counts": "int64", @@ -344,6 +359,20 @@ "y_centroid": "float64" } }, + "color_forced_cat__cell_area": { + "categories_digest": "03d5cb572a029c51", + "digest": "399d4c4dcd1753c0", + "n": 36, + "n_categories": 36, + "type": "categorical" + }, + "color_forced_cont__transcript_counts": { + "categories_digest": null, + "digest": "0fd6f76f73a915d2", + "n": 36, + "n_categories": null, + "type": "continuous" + }, "color_gene_set": { "digest": "dc12ce5b5034d05e", "max": 1.0, @@ -374,6 +403,24 @@ "digest": "c92dc2b2c78706bf" }, "edge__edges.parquet__detail0": "b5b16c6db1ee824b", + "edge__edges.parquet__filter__interaction_class": { + "digest": "868bc3eaf596bf9c", + "keys": [ + "edge", + "is_autocrine", + "lrm_count", + "receiving_cell", + "receiving_type", + "score_sum", + "sending_cell", + "sending_type", + "x1", + "x2", + "y1", + "y2" + ], + "n": 176 + }, "edge__edges.parquet__grouped": { "digest": "74439cac91762e89", "keys": [ @@ -432,6 +479,24 @@ "digest": "519a5ef75304e432" }, "edge__edges__edge.normalized.product.parquet__detail0": "e5a6e867ae7774fa", + "edge__edges__edge.normalized.product.parquet__filter__interaction_class": { + "digest": "7106481dad64d458", + "keys": [ + "edge", + "is_autocrine", + "lrm_count", + "receiving_cell", + "receiving_type", + "score_sum", + "sending_cell", + "sending_type", + "x1", + "x2", + "y1", + "y2" + ], + "n": 176 + }, "edge__edges__edge.normalized.product.parquet__grouped": { "digest": "60404aba71d2d61d", "keys": [ @@ -490,6 +555,24 @@ "digest": "ea178fdb2da7ef01" }, "edge__edges__edge.raw.minimum.parquet__detail0": "1653c7e83d4142bd", + "edge__edges__edge.raw.minimum.parquet__filter__interaction_class": { + "digest": "46e42445db77b41e", + "keys": [ + "edge", + "is_autocrine", + "lrm_count", + "receiving_cell", + "receiving_type", + "score_sum", + "sending_cell", + "sending_type", + "x1", + "x2", + "y1", + "y2" + ], + "n": 113 + }, "edge__edges__edge.raw.minimum.parquet__grouped": { "digest": "ba6b68c6da29e63c", "keys": [ @@ -540,6 +623,11 @@ ], "n": 155 }, + "filter_bounds__transcript_counts": { + "n_cells": 19, + "total": 19 + }, + "filter_ids__transcript_counts": 19, "gene_list": { "digest": "09d44724709d2b24", "n": 5035 @@ -679,6 +767,13 @@ "y_centroid": "float64" } }, + "color_forced_cat__cell_area": { + "categories_digest": "240b60589d931fdd", + "digest": "cf5a6d01674f343a", + "n": 62, + "n_categories": 62, + "type": "categorical" + }, "color_gene_set": { "digest": "90f5d24654cf35e1", "max": 69.0, @@ -802,6 +897,13 @@ "y_centroid": "float64" } }, + "color_forced_cat__cell_area": { + "categories_digest": "f49dc022795723e8", + "digest": "04b252c1bb65e091", + "n": 36, + "n_categories": 36, + "type": "categorical" + }, "color_gene_set": { "digest": "900101042f1887aa", "max": 55.0, @@ -832,6 +934,24 @@ "digest": "147980b074db33d3" }, "edge__edges.parquet__detail0": "78d7e0433da4808e", + "edge__edges.parquet__filter__is_autocrine": { + "digest": "99f18f8571d0f208", + "keys": [ + "edge", + "is_autocrine", + "lrm_count", + "receiving_cell", + "receiving_type", + "score_sum", + "sending_cell", + "sending_type", + "x1", + "x2", + "y1", + "y2" + ], + "n": 214 + }, "edge__edges.parquet__grouped": { "digest": "95d8d03ded9a9ecb", "keys": [ @@ -982,6 +1102,13 @@ "y_centroid": "float64" } }, + "color_forced_cat__array_row": { + "categories_digest": "cb1f070cb9fb1832", + "digest": "e2371587c0c48cbc", + "n": 20830, + "n_categories": 98, + "type": "categorical" + }, "color_gene_set": { "digest": "654b57fb083efe14", "max": 1.0, @@ -1012,6 +1139,24 @@ "digest": "d64c43b4d5cff51c" }, "edge__edges.parquet__detail0": "89d54bdc782c25cb", + "edge__edges.parquet__filter__is_autocrine": { + "digest": "794574a264114c51", + "keys": [ + "edge", + "is_autocrine", + "lrm_count", + "receiving_cell", + "receiving_type", + "score_sum", + "sending_cell", + "sending_type", + "x1", + "x2", + "y1", + "y2" + ], + "n": 62680 + }, "edge__edges.parquet__grouped": { "digest": "8a6fa8e717282f31", "keys": [ @@ -1135,6 +1280,20 @@ "y_centroid": "float64" } }, + "color_forced_cat__transcript_counts": { + "categories_digest": "0790d6f4f3f4066d", + "digest": "1764190c4065b50d", + "n": 7275, + "n_categories": 422, + "type": "categorical" + }, + "color_forced_cont__control_probe_counts": { + "categories_digest": null, + "digest": "4068afac4f77e1b4", + "n": 7275, + "n_categories": null, + "type": "continuous" + }, "color_gene_set": { "digest": "7e62d7c4719e9927", "max": 37.0, @@ -1165,6 +1324,24 @@ "digest": "f454e9ec05ba3d82" }, "edge__edges.parquet__detail0": "502de7ce02b60495", + "edge__edges.parquet__filter__is_autocrine": { + "digest": "cb64d837bdcc7877", + "keys": [ + "edge", + "is_autocrine", + "lrm_count", + "receiving_cell", + "receiving_type", + "score_sum", + "sending_cell", + "sending_type", + "x1", + "x2", + "y1", + "y2" + ], + "n": 43650 + }, "edge__edges.parquet__grouped": { "digest": "2587c77fd6f6a1b3", "keys": [ @@ -1220,6 +1397,24 @@ "digest": "f746905c1d52f0aa" }, "edge__edges__niches_rad30.parquet__detail0": "0635850ff026fa23", + "edge__edges__niches_rad30.parquet__filter__is_autocrine": { + "digest": "1dea970e0700b956", + "keys": [ + "edge", + "is_autocrine", + "lrm_count", + "receiving_cell", + "receiving_type", + "score_sum", + "sending_cell", + "sending_type", + "x1", + "x2", + "y1", + "y2" + ], + "n": 174248 + }, "edge__edges__niches_rad30.parquet__grouped": { "digest": "d2e6d82664d037f3", "keys": [ @@ -1267,6 +1462,11 @@ ], "n": 181523 }, + "filter_bounds__control_probe_counts": { + "n_cells": 7272, + "total": 7272 + }, + "filter_ids__control_probe_counts": 7272, "gene_list": { "digest": "bfd0eef30077d962", "n": 280 diff --git a/backend/tests/golden_snapshot.py b/backend/tests/golden_snapshot.py index e6b33b8..f5f4ddb 100644 --- a/backend/tests/golden_snapshot.py +++ b/backend/tests/golden_snapshot.py @@ -37,6 +37,8 @@ sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) +from app.readers.metadata_filter import MetadataFilter # noqa: E402 + DATA_ROOT = Path(__file__).resolve().parent.parent.parent / "sample_data" BASELINE = Path(__file__).resolve().parent / "golden_baseline.json" @@ -202,6 +204,68 @@ def probe_spatial(reader, p: Probes) -> None: "digest": digest(cvm.get("values")), }) + probe_metadata_features(reader, p) + + +def probe_metadata_features(reader, p: Probes) -> None: + """Cover the force-categorical override (#35) and the metadata filter (#45). + + Both are driven off the same column list, and the column is chosen by rule + rather than hardcoded, so this works on every platform: the first column whose + auto-detected type is categorical exercises the filter and the forced-continuous + path, and the first continuous one exercises forced-categorical. + """ + df = reader._metadata_frame() + if df is None or df.empty or "cell_id" not in df.columns: + return + # Coordinates are numeric but meaningless to colour by, and forcing one + # categorical yields a category per cell — a probe that says nothing and a + # baseline entry tens of thousands of labels long. + skip = {"cell_id", "x_centroid", "y_centroid"} + fields = [c for c in df.columns if c not in skip and df[c].notna().any()] + + cat_field = cont_field = None + for f in fields: + kind = reader.color_values("metadata", f).get("type") + if kind == "categorical" and cat_field is None: + cat_field = f + elif kind == "continuous" and cont_field is None: + cont_field = f + if cat_field and cont_field: + break + + def summarise(cv): + cats = cv.get("categories") + return {"type": cv.get("type"), "n": len(cv.get("values", {})), + # Count plus digest rather than the list: order is the thing under + # test (issue #35's numeric sort), and the digest is order-sensitive. + "n_categories": None if cats is None else len(cats), + "categories_digest": None if cats is None else digest(cats), + "digest": digest(cv.get("values"))} + + if cat_field: + # Forcing a categorical column continuous must produce numbers or nothing — + # never a silent fallback that colours every unit identically. + p.record(f"color_forced_cont__{cat_field}", + lambda: summarise(reader.color_values("metadata", cat_field, + categorical=False))) + cats = reader.color_values("metadata", cat_field).get("categories") or [] + if cats: + spec = MetadataFilter.build(cat_field, values=cats[:2]) + ids = reader.filter_cell_ids(spec) + p.record(f"filter_ids__{cat_field}", lambda: len(ids)) + if reader.capabilities().get("has_boundaries", True): + b = reader.cell_boundaries(fraction=1.0, cell_ids=ids) + p.record(f"filter_bounds__{cat_field}", lambda: { + "total": b.get("total"), + "n_cells": len({r["cell_id"] for r in b["boundaries"]}) + if b["boundaries"] else 0, + }) + if cont_field: + p.record(f"color_forced_cat__{cont_field}", + lambda: summarise(reader.color_values("metadata", cont_field, + categorical=True))) + def probe_edges(dataset_dir: Path, pixel_size: float, p: Probes) -> None: from app.readers.edge_reader import EdgeReader @@ -234,6 +298,21 @@ def probe_edges(dataset_dir: Path, pixel_size: float, p: Probes) -> None: p.record(f"edge__{key}__detail0", lambda er=er, eid=eid: digest(er.edge_detail(eid))) + # Metadata filter on the edge table (#45). The column is picked by rule + # so this covers whatever the dataset happens to carry — parquet columns + # and edge-metadata/ columns take different code paths inside the reader. + for col, dtype in sorted(er.schema()["columns"].items()): + if col in ("edge", "sending_cell", "receiving_cell", "lrm", + "ligand", "receptor", "lrm_id"): + continue + cv = er.edge_color_values("metadata", None, col) + if cv.get("type") != "categorical" or not cv.get("categories"): + continue + spec = MetadataFilter.build(col, values=cv["categories"][:1]) + p.record(f"edge__{key}__filter__{col}", lambda er=er, spec=spec: + Probes.rows(er.query_grouped(density=1.0, edge_filter=spec))) + break + def collect() -> dict: from app.readers.reader_factory import ReaderFactory diff --git a/frontend/package.json b/frontend/package.json index 5797e94..32602f2 100644 --- a/frontend/package.json +++ b/frontend/package.json @@ -1,6 +1,6 @@ { "name": "tissueplex", - "version": "0.7.1", + "version": "0.8.0", "private": true, "scripts": { "dev": "vite", diff --git a/frontend/src/components/LayerPanel.jsx b/frontend/src/components/LayerPanel.jsx index 7a8df77..2be13f0 100644 --- a/frontend/src/components/LayerPanel.jsx +++ b/frontend/src/components/LayerPanel.jsx @@ -186,6 +186,12 @@ export default function LayerPanel() {
{unitTitle} Color
+ {/* Issue #45. Placed under the color section because picking the column to + subset on is the same act as picking the column to colour by, and users + almost always do the two together. */} +
{unitTitle} Filter
+ + {hasTranscripts && ( <>
Transcript Species
@@ -227,6 +233,8 @@ function ColorBySection({ unitLabel = "cell" }) { selectedGenes, cellColorRange, cellColorClamp, setCellColorClamp, + categoricalOverrides, setCategoricalOverride, + cellColorType, cellColorCategories, } = useStore(); const [cellSchema, setCellSchema] = useState(null); @@ -248,10 +256,20 @@ function ColorBySection({ unitLabel = "cell" }) { const { mode, field } = colorBy; const selectedCount = selectedGenes === null ? allGenes.length : selectedGenes.size; - // Determine if the selected metadata column is categorical + // How the selected column is actually being coloured. This comes from the + // backend's response (via the store, written by panel 0) rather than from the + // schema dtype: the backend also treats a low-cardinality integer column as + // categorical, so guessing from dtype used to draw a gradient legend with two + // dead sliders over a canvas that was already showing discrete colours. const fieldDtype = field && cellSchema ? cellSchema.columns[field] : null; - const isCategorical = fieldDtype === "object" || fieldDtype === "string" || - (fieldDtype?.startsWith("int") && false); // int cols treated as continuous unless overridden + const isCategorical = mode === "metadata" && !!field && cellColorType === "categorical"; + + // The override is only meaningful for a numeric column — a text column has no + // gradient to fall back to, so there is nothing to offer. + const isNumericField = !!fieldDtype && + /^(int|uint|float|Int|UInt|Float)/.test(fieldDtype); + const overrideKey = `cell::${field}`; + const override = categoricalOverrides[overrideKey] ?? null; return (
@@ -300,6 +318,37 @@ function ColorBySection({ unitLabel = "cell" }) { ))} + + {/* Issue #35: integer-coded cluster IDs arrive as ints and would + otherwise be drawn as a gradient. Unchecking forces the reverse, + which is how you get a gradient over a column the auto-rule + called categorical. */} + {isNumericField && ( + + )} )} @@ -326,7 +375,7 @@ function ColorBySection({ unitLabel = "cell" }) { clamp={cellColorClamp} setClamp={setCellColorClamp} accentColor="#6cf" /> )} {mode === "metadata" && field && isCategorical && ( - + )} )} @@ -381,7 +430,16 @@ function ClampableLegend({ label, palette, vmin, vmax, clamp, setClamp, accentCo ); } -function CategoricalLegend({ field, apiBase, dataset }) { +/** + * Editable per-category swatches. + * + * `categories` comes from the store, where panel 0 records whatever the backend + * returned for the active column. This component used to re-POST /color-values + * for itself, which duplicated a request the viewer had already made and — once + * the categorical override existed — would have asked without it, so the legend + * could disagree with the canvas it describes. + */ +function CategoricalLegend({ field, categories = [] }) { const { categoryColorOverrides, setCategoryColorOverride, @@ -389,21 +447,8 @@ function CategoricalLegend({ field, apiBase, dataset }) { resetCategoryColorOverrides, } = useStore(); - const [categories, setCategories] = useState([]); const fileInputRef = useRef(null); - useEffect(() => { - if (!field) return; - fetch(`${apiBase}/spatial/${dataset}/color-values`, { - method: "POST", - headers: { "Content-Type": "application/json" }, - body: JSON.stringify({ mode: "metadata", field }), - }) - .then((r) => r.json()) - .then((d) => { if (d.type === "categorical") setCategories(d.categories); }) - .catch(() => {}); - }, [apiBase, dataset, field]); - // Resolve display color for a category: override → QUAL_PALETTE → hash // Must mirror the logic in useCellColors.js so legend stays in sync. function resolveColor(cat, i) { @@ -533,6 +578,216 @@ function CategoricalLegend({ field, apiBase, dataset }) { ); } +// ── Metadata filter (issue #45) ─────────────────────────────────────────────── +/** + * Restrict the view to a subset of units by one metadata column. + * + * One component serves both the cell filter and the edge filter — they differ + * only in which endpoint supplies the column list and the distinct values, so + * those arrive as props. The chosen filter is written to the store and travels + * to the backend, which applies it *before* sampling; doing it client-side would + * leave a sample of a subset rather than the subset. + * + * Two shapes, chosen by what the backend says the column is: + * categorical — checkboxes, one per value (this is the "focus on 2–3 cell + * types" case from the issue) + * continuous — inclusive min/max bounds + */ +function MetadataFilterSection({ + scope, columns, filter, setFilter, fetchValues, unitLabel = "cell", +}) { + const [meta, setMeta] = useState(null); // { type, categories, min, max } + const [loading, setLoading] = useState(false); + const field = filter?.field ?? ""; + + // Load the distinct values / range for the selected column. + useEffect(() => { + if (!field) { setMeta(null); return; } + let cancelled = false; + setLoading(true); + fetchValues(field) + .then((d) => { if (!cancelled && d) setMeta(d); }) + .catch(() => { if (!cancelled) setMeta(null); }) + .finally(() => { if (!cancelled) setLoading(false); }); + return () => { cancelled = true; }; + }, [field, fetchValues]); + + const selected = new Set(filter?.values ?? []); + const active = (filter?.values?.length ?? 0) > 0 || + filter?.min != null || filter?.max != null; + + function chooseField(next) { + // Values and bounds belong to the old column; carrying them over would + // silently filter on labels that do not exist in the new one. + setFilter(next ? { field: next, values: null, min: null, max: null } : null); + } + + function toggle(cat) { + const next = new Set(selected); + if (next.has(cat)) next.delete(cat); else next.add(cat); + setFilter({ ...filter, values: next.size ? [...next] : null }); + } + + return ( +
+ + + {field && loading && ( +
loading values…
+ )} + + {field && !loading && meta?.type === "categorical" && ( +
+
+ {(meta.categories ?? []).map((cat) => ( + + ))} +
+
+ + + + {/* No selection is "show everything", not "show nothing" — an empty + allowlist would blank the canvas the moment a column is picked. */} + {selected.size + ? `${selected.size} of ${(meta.categories ?? []).length} shown` + : "all shown"} + +
+
+ )} + + {field && !loading && meta?.type === "continuous" && ( +
+ min + setFilter({ + ...filter, min: e.target.value === "" ? null : parseFloat(e.target.value), + })} + style={{ ...SELECT_STYLE, marginTop: 0, width: 0, flex: 1 }} + /> + max + setFilter({ + ...filter, max: e.target.value === "" ? null : parseFloat(e.target.value), + })} + style={{ ...SELECT_STYLE, marginTop: 0, width: 0, flex: 1 }} + /> +
+ )} + + {active && ( + + )} +
+ ); +} + +function fmtBound(v) { + if (v == null) return ""; + return Math.abs(v) >= 1000 || (v !== 0 && Math.abs(v) < 0.01) + ? v.toExponential(1) : String(Math.round(v * 1000) / 1000); +} + +function CellFilterSection({ unitLabel = "cell" }) { + const { apiBase, dataset, cellFilter, setCellFilter, categoricalOverrides } = useStore(); + const [columns, setColumns] = useState([]); + + useEffect(() => { + fetch(`${apiBase}/spatial/${dataset}/cells/schema`) + .then((r) => (r.ok ? r.json() : null)) + .then((s) => setColumns(s?.columns ? Object.keys(s.columns) : [])) + .catch(() => setColumns([])); + }, [apiBase, dataset]); + + // Honour the same categorical override the color panel uses, so a column the + // user has declared categorical offers checkboxes here rather than a range. + const fetchValues = React.useCallback((field) => { + const categorical = categoricalOverrides[`cell::${field}`] ?? null; + return fetch(`${apiBase}/spatial/${dataset}/color-values`, { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ mode: "metadata", field, categorical }), + }).then((r) => (r.ok ? r.json() : null)); + }, [apiBase, dataset, categoricalOverrides]); + + return ( + + ); +} + +function EdgeFilterSection() { + const { apiBase, dataset, edgeFile, edgeFilter, setEdgeFilter, categoricalOverrides } = useStore(); + const [columns, setColumns] = useState([]); + const efParam = `?edge_file=${encodeURIComponent(edgeFile)}`; + + useEffect(() => { + fetch(`${apiBase}/edges/${dataset}/schema${efParam}`) + .then((r) => (r.ok ? r.json() : null)) + // Structural and per-LRM columns are not edge attributes to subset on: + // one edge has many LRM rows, so "lrm = X" is a mechanism filter, which + // the LRM checklist below already does properly. + .then((s) => setColumns( + s?.columns + ? Object.keys(s.columns).filter((c) => !EDGE_FILTER_SKIP.has(c)) + : [] + )) + .catch(() => setColumns([])); + }, [apiBase, dataset, efParam]); + + const fetchValues = React.useCallback((field) => { + const categorical = categoricalOverrides[`edge::${field}`] ?? null; + return fetch(`${apiBase}/edges/${dataset}/edge-color-values${efParam}`, { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ mode: "metadata", field, categorical }), + }).then((r) => (r.ok ? r.json() : null)); + }, [apiBase, dataset, efParam, categoricalOverrides]); + + if (!columns.length) return null; + return ( + + ); +} + +const EDGE_FILTER_SKIP = new Set([ + "edge", "sending_cell", "receiving_cell", "x1", "y1", "x2", "y2", + "lrm", "lrm_id", "ligand", "receptor", "score", "score_norm", +]); + // ── Morphology row ──────────────────────────────────────────────────────────── function MorphologyRow() { const { layers, setLayerProp } = useStore(); @@ -1012,6 +1267,7 @@ function EdgeSection() { hiddenLrms, toggleLrm, setAllLrmsVisible, hideAllLrms, edgeColorRange, edgeColorClamp, setEdgeColorClamp, + categoricalOverrides, setCategoricalOverride, } = useStore(); const state = layers.edges ?? { visible: true, opacity: 0.9 }; const [localStrength, setLocalStrength] = useState(edgeMinStrength ?? 0); @@ -1070,9 +1326,31 @@ function EdgeSection() { const { mode, field } = edgeColorBy; const selectedLrmCount = lrmCatalogue.length - hiddenLrms.size; - // Determine if selected metadata column is categorical + // How the selected edge metadata column is typed. Asking the backend rather + // than reading the dtype matters for the same reason it does on the cell side: + // the auto-rule also calls a low-cardinality integer column categorical, and an + // explicit override can flip either way (issue #35). const fieldDtype = field && edgeSchema ? edgeSchema.columns[field] : null; - const isCategorical = fieldDtype === "object" || fieldDtype === "string" || fieldDtype === "bool"; + const isNumericField = !!fieldDtype && /^(int|uint|float|double|Int|UInt|Float)/.test(fieldDtype); + const edgeOverrideKey = `edge::${field}`; + const edgeOverride = categoricalOverrides[edgeOverrideKey] ?? null; + + const [edgeMeta, setEdgeMeta] = useState(null); // { type, categories } + useEffect(() => { + if (mode !== "metadata" || !field) { setEdgeMeta(null); return; } + let cancelled = false; + fetch(`${apiBase}/edges/${dataset}/edge-color-values${efParam}`, { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ mode: "metadata", field, categorical: edgeOverride }), + }) + .then((r) => (r.ok ? r.json() : null)) + .then((d) => { if (!cancelled && d) setEdgeMeta({ type: d.type, categories: d.categories ?? [] }); }) + .catch(() => {}); + return () => { cancelled = true; }; + }, [apiBase, dataset, efParam, mode, field, edgeOverride]); + + const isCategorical = mode === "metadata" && !!field && edgeMeta?.type === "categorical"; return (
@@ -1241,14 +1519,39 @@ function EdgeSection() { )} {mode === "metadata" && ( - + <> + + {field && isNumericField && ( + + )} + )} {/* Palette — only for continuous color modes */} @@ -1274,9 +1577,16 @@ function EdgeSection() { clamp={edgeColorClamp} setClamp={setEdgeColorClamp} accentColor="#f90" /> )} {mode === "metadata" && field && isCategorical && ( - + )} + {/* ── Edge metadata filter (issue #45) ─────────────────────── */} + {/* Distinct from the LRM checklist below: this subsets *edges* by an + attribute of the pair (a curation call, a confidence), whereas the + checklist subsets the mechanisms scored on every edge. */} +
Edge Filter
+ + {/* ── LRM Mechanisms checklist ─────────────────────────────── */} {lrmCatalogue.length > 0 && (
@@ -1333,22 +1643,10 @@ function EdgeSection() { ); } -function EdgeCategoricalLegend({ field, apiBase, dataset, edgeFile = "edges.parquet" }) { - const [categories, setCategories] = useState([]); - - useEffect(() => { - if (!field) return; - const efParam = `?edge_file=${encodeURIComponent(edgeFile)}`; - fetch(`${apiBase}/edges/${dataset}/edge-color-values${efParam}`, { - method: "POST", - headers: { "Content-Type": "application/json" }, - body: JSON.stringify({ mode: "metadata", field }), - }) - .then((r) => r.json()) - .then((d) => { if (d.type === "categorical") setCategories(d.categories); }) - .catch(() => {}); - }, [apiBase, dataset, field, edgeFile]); - +/** Read-only swatch list. Categories come from EdgeSection, which already asked + * the backend for the column's type — one fetch, one answer, no chance of the + * legend describing a different typing decision than the canvas is using. */ +function EdgeCategoricalLegend({ categories = [] }) { if (!categories.length) return null; return (
diff --git a/frontend/src/components/Viewer.jsx b/frontend/src/components/Viewer.jsx index 1a93ba7..3e3ec55 100644 --- a/frontend/src/components/Viewer.jsx +++ b/frontend/src/components/Viewer.jsx @@ -138,6 +138,7 @@ function ViewerPanel({ panelIndex }) { selectedEdge, setSelectedEdge, setCellColorRange, setEdgeColorRange, cellColorClamp, edgeColorClamp, setEdgeColorClamp, + categoricalOverrides, cellFilter, edgeFilter, setCellColorType, annotationMode, pixelSize, setPixelSize, clearZoomMatch, @@ -537,7 +538,8 @@ function ViewerPanel({ panelIndex }) { total: cellBoundaryTotal, loading: cellBoundariesLoading, } = useCellBoundaries( - apiBase, dataset, viewport, imageSize, cellSegmentsVisible && hasBoundaries, cellBoundaryFraction + apiBase, dataset, viewport, imageSize, cellSegmentsVisible && hasBoundaries, + cellBoundaryFraction, cellFilter ); useEffect(() => { cellPolygonsRef.current = cellPolygons; }, [cellPolygons]); @@ -548,20 +550,38 @@ function ViewerPanel({ panelIndex }) { const { edges, loading: edgesLoading } = useEdges( apiBase, dataset, viewport, imageSize, edgesVisible || tissueGraphVisible, - edgeMinStrength, hiddenLrms, lrmCatalogue, edgeDensity, edgeFile + edgeMinStrength, hiddenLrms, lrmCatalogue, edgeDensity, edgeFile, + cellFilter, edgeFilter ); - const { colorValues, vmin: cellVmin, vmax: cellVmax, loading: cellColorsLoading } = useCellColors( - apiBase, dataset, colorBy, allGenes, selectedGenes, cellColorPalette, cellColorEnabled, cellColorClamp, categoryColorOverrides + // Explicit categorical/continuous choice for the active color-by column, or + // null (auto-detect) when the user has not overridden it — issue #35. + const cellCategorical = categoricalOverrides[`cell::${colorBy?.field}`] ?? null; + const edgeCategorical = categoricalOverrides[`edge::${edgeColorBy?.field}`] ?? null; + + const { + colorValues, vmin: cellVmin, vmax: cellVmax, + type: cellType, categories: cellCategories, loading: cellColorsLoading, + } = useCellColors( + apiBase, dataset, colorBy, allGenes, selectedGenes, cellColorPalette, + cellColorEnabled, cellColorClamp, categoryColorOverrides, cellCategorical ); // Only update shared store ranges from panel 0 to avoid redundant updates useEffect(() => { if (panelIndex === 0) setCellColorRange(cellVmin, cellVmax); }, [cellVmin, cellVmax]); // eslint-disable-line + // The backend is the authority on whether a column is categorical, so report + // the type it actually returned rather than letting the panel re-derive it + // from the schema dtype — the two disagreed for low-cardinality integers. + useEffect(() => { + if (panelIndex === 0) setCellColorType(cellType, cellCategories); + }, [cellType, cellCategories]); // eslint-disable-line + const edgeColorEnabled = edgeColorBy.mode !== "default"; const { colorValues: edgeColorValues, vmin: edgeVmin, vmax: edgeVmax, p95: edgeP95, loading: edgeColorsLoading } = useEdgeColors( - apiBase, dataset, edgeColorBy, hiddenLrms, lrmCatalogue, edgeColorPalette, edgeColorEnabled, edgeColorClamp, edges, edgeFile + apiBase, dataset, edgeColorBy, hiddenLrms, lrmCatalogue, edgeColorPalette, + edgeColorEnabled, edgeColorClamp, edges, edgeFile, edgeCategorical ); useEffect(() => { if (panelIndex === 0) setEdgeColorRange(edgeVmin, edgeVmax); diff --git a/frontend/src/hooks/useCellBoundaries.js b/frontend/src/hooks/useCellBoundaries.js index 5d07c7b..84f38b2 100644 --- a/frontend/src/hooks/useCellBoundaries.js +++ b/frontend/src/hooks/useCellBoundaries.js @@ -19,8 +19,30 @@ import { useState, useEffect, useRef } from "react"; const TARGET_CELLS = 5_000; const SEED_TOTAL = 50_000; // conservative first-probe estimate +/** + * Serialise a metadata filter (issue #45) into query params. + * + * Returns "" when there is nothing to constrain, so the URL is byte-identical to + * the pre-filter one and no cached response is missed. The filter is sent to the + * server rather than applied to the response because sampling happens server-side: + * filtering afterwards would leave a fraction of a fraction on screen. + */ +function filterParams(filter) { + if (!filter?.field) return ""; + const p = new URLSearchParams(); + const hasValues = Array.isArray(filter.values) && filter.values.length > 0; + if (!hasValues && filter.min == null && filter.max == null) return ""; + p.set("filter_field", filter.field); + if (hasValues) for (const v of filter.values) p.append("filter_values", v); + if (filter.min != null) p.set("filter_min", filter.min); + if (filter.max != null) p.set("filter_max", filter.max); + if (filter.includeMissing) p.set("filter_missing", "true"); + return `&${p.toString()}`; +} + export function useCellBoundaries( - apiBase, dataset, viewport, imageSize, enabled = true, fraction = null + apiBase, dataset, viewport, imageSize, enabled = true, fraction = null, + filter = null ) { const [cells, setCells] = useState([]); const [total, setTotal] = useState(0); @@ -31,6 +53,23 @@ export function useCellBoundaries( const abortRef = useRef(null); const prevTotalRef = useRef(SEED_TOTAL); // running estimate of cells in viewport + // Serialised once so it can be both spliced into the URL and used as an effect + // dependency — the filter arrives as an object whose identity changes on every + // render, which would otherwise refetch continuously. + const filterQS = filterParams(filter); + + // One-shot recalibration. + // + // In auto mode the fraction is picked from `prevTotalRef`, the total the *last* + // fetch saw. Applying a metadata filter (or switching dataset) changes that + // total out from under the estimate, and nothing else would trigger another + // fetch — so the layer would sit showing a tenth of an already-small subset + // until the user happened to pan. Bumping this counter re-runs the fetch once + // with the corrected fraction; `calibratedRef` keys it to the current request + // so it can converge rather than oscillate. + const [recalibrate, setRecalibrate] = useState(0); + const calibratedRef = useRef(null); + useEffect(() => { if (!enabled || !dataset) { setLoading(false); @@ -62,6 +101,7 @@ export function useCellBoundaries( } else { url += `?${fracParam}`; } + url += filterQS; const res = await fetch(url, { signal: ctrl.signal }); if (!res.ok) { setCells([]); setTotal(0); return; } const data = await res.json(); @@ -74,6 +114,23 @@ export function useCellBoundaries( if (totalCells > 0) prevTotalRef.current = totalCells; setTotal(totalCells); + // If that estimate was badly wrong, correct it now rather than waiting + // for the user to pan. Only in auto mode — an explicit slider value is + // the user's decision, not an estimate. Once per request key. + // The 1.2 threshold is what makes the panel's sample readout honest: the + // panel derives its percentage from the *current* total, so anything + // looser leaves it advertising a fraction the canvas is not drawing at. + // Total is a pre-sample count for a fixed bbox and filter, so the second + // fetch computes the same fraction and the loop settles after one pass. + if (fraction === null && totalCells > 0) { + const better = Math.min(1.0, TARGET_CELLS / totalCells); + const key = `${url}|${totalCells}`; + if (better > eff * 1.2 && calibratedRef.current !== key) { + calibratedRef.current = key; + setRecalibrate((c) => c + 1); + } + } + if (!Array.isArray(rows)) { setCells([]); return; } // Group flat vertex list by cell_id → polygon arrays @@ -94,7 +151,7 @@ export function useCellBoundaries( }, 200); return () => clearTimeout(timerRef.current); - }, [apiBase, dataset, viewport?.xmin, viewport?.ymin, viewport?.xmax, viewport?.ymax, enabled, fraction]); + }, [apiBase, dataset, viewport?.xmin, viewport?.ymin, viewport?.xmax, viewport?.ymax, enabled, fraction, filterQS, recalibrate]); // Abort in-flight request on unmount useEffect(() => { diff --git a/frontend/src/hooks/useCellColors.js b/frontend/src/hooks/useCellColors.js index 07aa2c0..236aec7 100644 --- a/frontend/src/hooks/useCellColors.js +++ b/frontend/src/hooks/useCellColors.js @@ -25,6 +25,7 @@ import { geneColor } from "../utils/geneColor"; * Modes: * gene_set — POST with selected genes; returns continuous sum * metadata — POST with field; backend auto-detects continuous vs. categorical + * unless `categorical` overrides it (issue #35) * * Returns: * colorValues Map or null when disabled @@ -34,7 +35,7 @@ import { geneColor } from "../utils/geneColor"; * categoryColors Map for categorical legend * loading */ -export function useCellColors(apiBase, dataset, colorBy, allGenes, selectedGenes, palette, enabled, clamp, categoryColorOverrides) { +export function useCellColors(apiBase, dataset, colorBy, allGenes, selectedGenes, palette, enabled, clamp, categoryColorOverrides, categorical = null) { const [result, setResult] = useState({ colorValues: null, type: "continuous", vmin: 0, vmax: 0, categories: [], categoryColors: new Map(), @@ -90,7 +91,7 @@ export function useCellColors(apiBase, dataset, colorBy, allGenes, selectedGenes try { const body = mode === "gene_set" ? { mode: "gene_set", genes: genesToSend } - : { mode: "metadata", field }; + : { mode: "metadata", field, categorical }; const res = await fetch(`${apiBase}/spatial/${dataset}/color-values`, { method: "POST", @@ -119,7 +120,7 @@ export function useCellColors(apiBase, dataset, colorBy, allGenes, selectedGenes } }, 400); return () => clearTimeout(timerRef.current); - }, [apiBase, dataset, colorBy?.mode, colorBy?.field, allGenes, selectedGenes, enabled]); // eslint-disable-line + }, [apiBase, dataset, colorBy?.mode, colorBy?.field, allGenes, selectedGenes, enabled, categorical]); // eslint-disable-line // ── Effect 2: apply clamp + palette to continuous data (no fetch, no debounce) ── // Fires immediately when rawCont, clamp, or palette changes so slider drags diff --git a/frontend/src/hooks/useEdgeColors.js b/frontend/src/hooks/useEdgeColors.js index 6e1a849..618fcb4 100644 --- a/frontend/src/hooks/useEdgeColors.js +++ b/frontend/src/hooks/useEdgeColors.js @@ -28,7 +28,8 @@ export function useEdgeColors( apiBase, dataset, edgeColorBy, hiddenLrms, lrmCatalogue, palette, enabled, clamp, edges, // array from useEdges — used for client-side lrm_set coloring - edgeFile = "edges.parquet" // which edge-source parquet the metadata fetch reads + edgeFile = "edges.parquet", // which edge-source parquet the metadata fetch reads + categorical = null // issue #35 override: null = auto-detect, true/false = forced ) { // Appended to the metadata edge-color-values request (lrm_set is client-side only). const efParam = `?edge_file=${encodeURIComponent(edgeFile)}`; @@ -99,7 +100,7 @@ export function useEdgeColors( const res = await fetch(`${apiBase}/edges/${dataset}/edge-color-values${efParam}`, { method: "POST", headers: { "Content-Type": "application/json" }, - body: JSON.stringify({ mode: "metadata", field }), + body: JSON.stringify({ mode: "metadata", field, categorical }), signal: ctrl.signal, }); if (!res.ok) throw new Error(`HTTP ${res.status}`); @@ -131,7 +132,7 @@ export function useEdgeColors( } }, 400); return () => clearTimeout(timerRef.current); - }, [apiBase, dataset, edgeColorBy?.mode, edgeColorBy?.field, enabled, efParam]); // eslint-disable-line + }, [apiBase, dataset, edgeColorBy?.mode, edgeColorBy?.field, enabled, efParam, categorical]); // eslint-disable-line // ── Effect 2: apply clamp + palette to continuous metadata (no fetch, no debounce) ── useEffect(() => { diff --git a/frontend/src/hooks/useEdges.js b/frontend/src/hooks/useEdges.js index 32a9618..a1d0624 100644 --- a/frontend/src/hooks/useEdges.js +++ b/frontend/src/hooks/useEdges.js @@ -27,13 +27,36 @@ import { useState, useEffect, useRef, useMemo } from "react"; const DEBOUNCE_MS = 400; +/** + * Normalise a store filter into the request-body shape the backend expects + * (issue #45), or undefined when nothing is constrained. + */ +function filterBody(filter) { + if (!filter?.field) return undefined; + const hasValues = Array.isArray(filter.values) && filter.values.length > 0; + if (!hasValues && filter.min == null && filter.max == null) return undefined; + return { + field: filter.field, + values: hasValues ? filter.values : null, + min: filter.min ?? null, + max: filter.max ?? null, + include_missing: !!filter.includeMissing, + }; +} + export function useEdges( apiBase, dataset, viewport, imageSize, enabled, minStrength, hiddenLrms, lrmCatalogue, density = 1.0, - edgeFile = "edges.parquet" + edgeFile = "edges.parquet", cellFilter = null, edgeFilter = null ) { // Which edge-source parquet to query; appended to every /edges request. const efParam = `?edge_file=${encodeURIComponent(edgeFile)}`; + + // Serialised so the structural effect can depend on filter *content* rather + // than object identity, which changes on every render. + const cellFilterBody = filterBody(cellFilter); + const edgeFilterBody = filterBody(edgeFilter); + const filterKey = JSON.stringify([cellFilterBody ?? null, edgeFilterBody ?? null]); // ── Structural state ────────────────────────────────────────────────────── const [structuralEdges, setStructuralEdges] = useState([]); const [loadingStructural, setLoadingStructural] = useState(false); @@ -69,6 +92,11 @@ export function useEdges( const { xmin, ymin, xmax, ymax } = viewport; const body = { xmin, ymin, xmax, ymax, density: Math.max(0.01, Math.min(1.0, density)) }; if (minStrength != null && minStrength > 0) body.min_strength = minStrength; + // Metadata filters go to the server so they apply before the density + // sample; filtering the response instead would sample first and leave a + // fraction of the subset. + if (cellFilterBody) body.cell_filter = cellFilterBody; + if (edgeFilterBody) body.edge_filter = edgeFilterBody; try { const res = await fetch(`${apiBase}/edges/${dataset}/query-grouped${efParam}`, { @@ -86,7 +114,7 @@ export function useEdges( }, DEBOUNCE_MS); return () => clearTimeout(structTimerRef.current); - }, [apiBase, dataset, viewport, imageSize, enabled, minStrength, density, efParam]); // eslint-disable-line + }, [apiBase, dataset, viewport, imageSize, enabled, minStrength, density, efParam, filterKey]); // eslint-disable-line // ── Effect 2: score fetch ────────────────────────────────────────────────── // Runs when viewport OR hiddenLrms changes. diff --git a/frontend/src/store.js b/frontend/src/store.js index 78e6016..3387334 100644 --- a/frontend/src/store.js +++ b/frontend/src/store.js @@ -27,6 +27,10 @@ export const useStore = create((set, get) => ({ edgeFile: "edges.parquet", lrmCatalogue: [], hiddenLrms: new Set(), selectedEdge: null, edgeColorRange: { vmin: null, vmax: null }, edgeColorClamp: { low: null, high: null }, + // Column names are dataset-specific, so a categorical override or an active + // filter naming a column the new dataset does not have would either do nothing + // or 400 on every viewport change. + categoricalOverrides: {}, cellFilter: null, edgeFilter: null, }), setActiveImage: (activeImage) => set({ activeImage }), @@ -36,8 +40,47 @@ export const useStore = create((set, get) => ({ setEdgeFile: (edgeFile) => set({ edgeFile, lrmCatalogue: [], hiddenLrms: new Set(), selectedEdge: null, edgeColorRange: { vmin: null, vmax: null }, edgeColorClamp: { low: null, high: null }, + // The edge filter names a column of the edge table, which differs between + // edge sources; the cell filter is unaffected because cells are shared. + edgeFilter: null, }), + // ── Categorical / continuous override (issue #35) ───────────────────────── + // Keyed "cell::" / "edge::" → true | false. Absent means + // auto-detect, which is what the backend does when `categorical` is null. + // Seurat writes cluster IDs as integers, so dtype alone routes them to a + // viridis gradient; this is how the user says "these are twenty categories". + categoricalOverrides: {}, + setCategoricalOverride: (scope, field, value) => set((s) => { + const next = { ...s.categoricalOverrides }; + if (value === null || value === undefined) delete next[`${scope}::${field}`]; + else next[`${scope}::${field}`] = value; + return { categoricalOverrides: next }; + }), + + // ── Metadata subsetting (issue #45) ─────────────────────────────────────── + // A filter is { field, values: string[] | null, min, max, includeMissing }. + // null means no filter. `values` is a categorical allowlist; min/max an + // inclusive numeric range. Applied server-side before sampling, so narrowing + // to a rare cluster shows all of it rather than a sample of a sample. + // + // cellFilter also governs edges: an edge is drawn only when BOTH endpoints + // survive it. edgeFilter is independent and applies to the edge table itself. + cellFilter: null, + setCellFilter: (f) => set({ cellFilter: f }), + edgeFilter: null, + setEdgeFilter: (f) => set({ edgeFilter: f }), + + // Resolved type of the active cell color-by column, reported by panel 0 so the + // LayerPanel can render the matching legend. The panel used to guess from the + // schema dtype, which disagreed with the backend for low-cardinality integers: + // the canvas drew discrete colors while the panel showed a gradient with two + // sliders that did nothing. + cellColorType: "continuous", + cellColorCategories: [], + setCellColorType: (type, categories) => + set({ cellColorType: type, cellColorCategories: categories ?? [] }), + // ── Platform capabilities (fetched from /spatial/{dataset}/info) ────────── // null = not yet loaded; object = { has_morphology, has_transcripts, has_boundaries, unit_label } platformCapabilities: null, diff --git a/sample_data/mouse_ileum_tiny/cell-metadata/example_clusters.csv b/sample_data/mouse_ileum_tiny/cell-metadata/example_clusters.csv index d220439..0b69f01 100644 --- a/sample_data/mouse_ileum_tiny/cell-metadata/example_clusters.csv +++ b/sample_data/mouse_ileum_tiny/cell-metadata/example_clusters.csv @@ -1,37 +1,37 @@ -,cluster,region,pseudotime -aaamobki-1,1,mid,0.4035 -aaclkaod-1,1,mid,0.3969 -bhakoonb-1,1,mid,0.4145 -bpefijoo-1,1,mid,0.3881 -ckfandjp-1,1,mid,0.4027 -dgcpicgh-1,1,mid,0.4054 -djgiipfb-1,1,mid,0.4114 -dkcgpkmh-1,1,mid,0.3911 -egeelggc-1,1,mid,0.3986 -ehjojgpl-1,1,mid,0.4064 -fkhfgimb-1,1,mid,0.4293 -gackndbe-1,1,mid,0.4185 -ghfdfbpc-1,1,mid,0.4345 -gjklkjjk-1,1,mid,0.424 -gpihhicj-1,1,mid,0.4417 -hakcibka-1,0,crypt,0.0452 -hbpoaeic-1,0,crypt,0.0341 -hhmlkgel-1,0,crypt,0.0079 -hkhcepfl-1,0,crypt,0.0 -ibclpflk-1,0,crypt,0.043 -ifjkhhkf-1,3,villus,0.9792 -jdchjdgi-1,3,villus,0.976 -jedneilm-1,3,villus,0.999 -jpbefbck-1,4,villus,1.0 -kecdfjaf-1,3,villus,0.9832 -kfefndgm-1,3,villus,0.9705 -kggmilif-1,3,villus,0.9673 -kkonlcio-1,3,villus,0.9596 -koaolmni-1,0,crypt,0.0416 -lfenejfi-1,0,crypt,0.0441 -niehkpen-1,0,crypt,0.0238 -ohmibdle-1,0,crypt,0.0337 -oinmeidp-1,0,crypt,0.0293 -ojeggnjp-1,3,villus,0.9824 -olbjkpjc-1,3,villus,0.9555 -omjmdimk-1,3,villus,0.9666 +,cluster,region,pseudotime,seurat_clusters +aaamobki-1,1,mid,0.4035,0 +aaclkaod-1,1,mid,0.3969,1 +bhakoonb-1,1,mid,0.4145,2 +bpefijoo-1,1,mid,0.3881,3 +ckfandjp-1,1,mid,0.4027,4 +dgcpicgh-1,1,mid,0.4054,5 +djgiipfb-1,1,mid,0.4114,6 +dkcgpkmh-1,1,mid,0.3911,7 +egeelggc-1,1,mid,0.3986,8 +ehjojgpl-1,1,mid,0.4064,9 +fkhfgimb-1,1,mid,0.4293,10 +gackndbe-1,1,mid,0.4185,11 +ghfdfbpc-1,1,mid,0.4345,0 +gjklkjjk-1,1,mid,0.424,1 +gpihhicj-1,1,mid,0.4417,2 +hakcibka-1,0,crypt,0.0452,3 +hbpoaeic-1,0,crypt,0.0341,4 +hhmlkgel-1,0,crypt,0.0079,5 +hkhcepfl-1,0,crypt,0.0,6 +ibclpflk-1,0,crypt,0.043,7 +ifjkhhkf-1,3,villus,0.9792,8 +jdchjdgi-1,3,villus,0.976,9 +jedneilm-1,3,villus,0.999,10 +jpbefbck-1,4,villus,1.0,11 +kecdfjaf-1,3,villus,0.9832,0 +kfefndgm-1,3,villus,0.9705,1 +kggmilif-1,3,villus,0.9673,2 +kkonlcio-1,3,villus,0.9596,3 +koaolmni-1,0,crypt,0.0416,4 +lfenejfi-1,0,crypt,0.0441,5 +niehkpen-1,0,crypt,0.0238,6 +ohmibdle-1,0,crypt,0.0337,7 +oinmeidp-1,0,crypt,0.0293,8 +ojeggnjp-1,3,villus,0.9824,9 +olbjkpjc-1,3,villus,0.9555,10 +omjmdimk-1,3,villus,0.9666,11