Add support for spatial tiling of vector data to SpatialData #1251
cornhundred
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
We have been thinking about spatial tiling approaches to make SpatialData objects compatible with a visualization scheme that loads tiled data at high zoom levels to enable visual exploration of large datasets (e.g., >1B transcripts), as we do with our Celldega front end (https://broadinstitute.github.io/celldega/). We have also had some initial discussions along these lines in a previous hackathon (2025-03 hackathon owkin > TileJSON for SpatialData/OMENGFF @ 💬), have read about recent work from the scverse (https://osf.io/preprints/biohackrxiv/s6bph_v1), and have reached out on a Zulip chat.
Based on our testing we have a minimal set of proposed changes to the SpatialData specification that appears to work well with our visualization spatial tiling approach. We would like to get feedback from the SpatialData community on whether this approach might make sense generally for SpatialData.
Proposed SpatialData storage/schema changes and Celldega compatibility demo
The SpatialData logical model is unchanged: the store still contains standard images, points, shapes, and an AnnData table and can be opened with
spatialdata.read_zarr. The proposal changes or more narrowly specifies parts of their on-disk representation to support efficient visualization.spatial_tilingmetadata (zarr.json).xandycolumns; no projection-oriented physical column order is prescribed.x,y, andfeature_name) first (for more efficient parquet_wasm column projection/filtering - note this is demonstrated with this unmerged pull request fix kylebarron/parquet-wasm#811).geoarrow.polygoncolumns so Arrow-compatible clients can read them without decoding WKB.Xmatrix and add the equivalent CSC matrix aslayers["X_csc"]for efficient access to individual genes.The column ordering, GeoArrow encoding, and CSC layer use capabilities already available in the underlying formats. The new convention introduced by this POC is the shared spatial tile and Parquet row-group mapping.
Please see this repo for a proof-of-concept example visualization of a modified SpatialData using an experimental branch of Celldega: https://github.com/cornhundred/POC_SpatialData_spatial-tile_viz_with_Celldega
Please see this repo for a Claude-coded proof-of-concept set of modifications to SpatialData-io in order to produce this new specification: scverse/spatialdata-io#423 [note that this functionality would need to be moved into SpatialData in order to provide general support for spatial transcriptomics technologies]
@jaspreetishar @LucaMarconato @keller-mark
All reactions