diff --git a/.gitignore b/.gitignore
index 343c7ae..c626a90 100644
--- a/.gitignore
+++ b/.gitignore
@@ -46,3 +46,5 @@ notebooks/examples/logs/
# others
node_modules/
+notebooks/examples/_latest_run_notebook.ipynb
+notebooks/developers_resources/storage_format/_latest_run_notebook.ipynb
diff --git a/.readthedocs.yaml b/.readthedocs.yaml
index b2471d4..e048903 100644
--- a/.readthedocs.yaml
+++ b/.readthedocs.yaml
@@ -13,3 +13,4 @@ python:
path: .
extra_requirements:
- doc
+ - pre
diff --git a/datasets/README.md b/datasets/README.md
index ad35e20..dec1c6d 100644
--- a/datasets/README.md
+++ b/datasets/README.md
@@ -1,35 +1,28 @@
# Spatial omics datasets
-Here you can find all datasets necessary to run the example notebooks already converted to the ZARR file format.
-
-If you want to convert additional datasets check out the scripts available in the [spatialdata sandbox](https://github.com/giovp/spatialdata-sandbox).
-
-:::{note}
-S3 URLs cannot be opened directly in a web browser. They should be treated as Zarr stores.
-For example, appending `.zgroup` to any of the URLs will allow you to see that file.
-:::
-
-
-| Technology | Sample | File Size | Filename (spatialdata-sandbox) | download data | work with data remotely (**see note below**) | license |
-| :---------------------------------------- | :-------------------------------------------------------- | --------: | :----------------------------- | :---------------------------------------------------------------------------------------------- | :----------------------------------------------------------------------------------------- | :---------------- |
-| Visium HD | Mouse intestin [^2] | 1 GB | visium_hd_3.0.0_id | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/visium_hd_3.0.0_io.zip) | [S3](https://s3.embl.de/spatialdata/spatialdata-sandbox/visium_hd_3.0.0_io.zarr/) | CCA |
-| Visium | Breast cancer [^3] | 1.5 GB | visium_associated_xenium_io | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/visium_associated_xenium_io.zip) | [S3](https://s3.embl.de/spatialdata/spatialdata-sandbox/visium_associated_xenium_io.zarr/) | CCA |
-| Xenium | Breast cancer [^3] | 2.8 GB | xenium_rep1_io | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/xenium_rep1_io.zip) | [S3](https://s3.embl.de/spatialdata/spatialdata-sandbox/xenium_rep1_io.zarr/) | CCA |
-| Xenium | Breast cancer [^3] | 3.7 GB | xenium_rep2_io | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/xenium_rep2_io.zip) | [S3](https://s3.embl.de/spatialdata/spatialdata-sandbox/xenium_rep2_io.zarr/) | CCA |
-| CyCIF (MCMICRO output) | Small lung adenocarcinoma [^4] | 250 MB | mcmicro_io | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/mcmicro_io.zip) | [S3](https://s3.embl.de/spatialdata/spatialdata-sandbox/mcmicro_io.zarr/) | CC BY-NC 4.0 DEED |
-| MERFISH | Mouse brain [^5] | 50 MB | merfish | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/merfish.zip) | [S3](https://s3.embl.de/spatialdata/spatialdata-sandbox/merfish.zarr/) | CC0 1.0 DEED |
-| MIBI-TOF | Colorectal carcinoma [^6] | 25 MB | mibitof | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/mibitof.zip) | [S3](https://s3.embl.de/spatialdata/spatialdata-sandbox/mibitof.zarr/) | CC BY 4.0 DEED |
-| Imaging Mass Cytometry (Steinbock output) | 4 different cancers (SCCHN, BCC, NSCLC, CRC) [^7][^8][^9] | 820 MB | steinbock_io | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/steinbock_io.zip) | [S3](https://s3.embl.de/spatialdata/spatialdata-sandbox/steinbock_io.zarr/) | CC BY 4.0 DEED |
-| Molecular Cartography (SPArrOW output) | Mouse Liver [^10][^11] | 70 MB | MouseLiver | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/mouse_liver.zip) | [S3](https://s3.embl.de/spatialdata/spatialdata-sandbox/mouse_liver.zarr) | CC BY 4.0 DEED |
-| SpaceM | T cells [^12] | 116 MB | spacem_scseahorse1 | [.zarr.zip](https://s3.embl.de/spatialdata/raw_data/20220121_ScSeahorse1.zip) | NA | CC BY 4.0 DEED |
-| SpaceM | Hepa and NIH3T3 cells [^13] | 59 MB | spacem_hepanih3t3 | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/spacem_helanih3t3.zip) | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/spacem_helanih3t3.zarr) | CC BY 4.0 DEED |
-
-For the first 3 datasets, we also provide a version of them in which they are all aligned in a common coordinate system, and where we added the cell-type information, as described in our paper, to annotate the Xenium cells.
-| Technology | Sample | File Size | Filename (spatialdata-sandbox) | download data | work with data remotely (**see note below**) | license |
-| :---------------------------------------- | :-------------------------------------------------------- | --------: | :-------------------------- | :---------------------------------------------------------------------------------------------- | :----------------------------------------------------------------------------------------- | :---------------- |
-| Visium | Breast Cancer [^3] | 1.5 GB | visium_associated_xenium_io | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/visium_associated_xenium_io_aligned.zip) | [S3](https://s3.embl.de/spatialdata/spatialdata-sandbox/visium_associated_xenium_io_aligned.zarr/) | CCA |
-| Xenium | Breast Cancer [^3] | 2.8 GB | xenium_rep1_io | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/xenium_rep1_io_aligned.zip) | [S3](https://s3.embl.de/spatialdata/spatialdata-sandbox/xenium_rep1_io_aligned.zarr/) | CCA |
-| Xenium | Breast Cancer [^3] | 3.7 GB | xenium_rep2_io | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/xenium_rep2_io_aligned.zip) | [S3](https://s3.embl.de/spatialdata/spatialdata-sandbox/xenium_rep2_io_aligned.zarr/) | CCA |
+Here you can find all datasets necessary to run the example notebooks already converted
+to the SpatialData Zarr file format.
+
+Scripts to convert data from several other technologies into SpatialData Zarr are
+available in the [spatialdata sandbox](https://github.com/giovp/spatialdata-sandbox); in
+particular:
+
+- CyCIF (MCMICRO output)
+- Imaging Mass Cytometry, IMC (Steinbock output)
+- seqFISH
+
+| Technology | Sample | File Size | Filename (spatialdata-sandbox) | download data (latest stable release) | license |
+|:------------------------------------------|:----------------------------------------------------------|----------:|:-------------------------------|:------------------------------------------------------------------------------------------------|:------------------|
+| Visium HD | Mouse intestin [^1] | ~2.4 GB | visium_hd_3.0.0_io | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/visium_hd_3.0.0_io.zip) | CCA |
+| Visium | Breast cancer [^2] | ~1.5 GB | visium_associated_xenium_io | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/visium_associated_xenium_io.zip) | CCA |
+| Xenium | Breast cancer [^2] | ~2.8 GB | xenium_rep1_io | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/xenium_rep1_io.zip) | CCA |
+| Xenium | Lung cancer [^3] | ~5.4 GB | xenium_2.0.0_io | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/xenium_2.0.0_io.zip) | CCA |
+| CyCIF (MCMICRO output) | Small lung adenocarcinoma [^4] | ~250 MB | mcmicro_io | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/mcmicro_io.zip) | CC BY-NC 4.0 DEED |
+| MERFISH | Mouse brain [^5] | ~50 MB | merfish | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/merfish.zip) | CC0 1.0 DEED |
+| MIBI-TOF | Colorectal carcinoma [^6] | ~25 MB | mibitof | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/mibitof.zip) | CC BY 4.0 DEED |
+| Imaging Mass Cytometry (Steinbock output) | 4 different cancers (SCCHN, BCC, NSCLC, CRC) [^7][^8][^9] | ~800 MB | steinbock_io | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/steinbock_io.zip) | CC BY 4.0 DEED |
+| Molecular Cartography (SPArrOW output) | Mouse Liver [^10][^11] | ~70 MB | MouseLiver | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/mouse_liver.zip) | CC BY 4.0 DEED |
+| SpaceM | Hepa and NIH3T3 cells [^12] | ~60 MB | spacem_hepanih3t3 | [.zarr.zip](https://s3.embl.de/spatialdata/spatialdata-sandbox/spacem_helanih3t3.zip) | CC BY 4.0 DEED |
## Licenses abbreviations
@@ -40,39 +33,51 @@ For the first 3 datasets, we also provide a version of them in which they are al
The data retains the license of the original published data.
-
-
-
# Artificial datasets
-Also, here you can find [additional datasets and resources for methods developers](https://github.com/scverse/spatialdata-notebooks/blob/main/notebooks/developers_resources/storage_format/).
+Also, here you can
+find [additional datasets and resources for methods developers](https://github.com/scverse/spatialdata-notebooks/blob/main/notebooks/developers_resources/storage_format/).
# References
If you use the datasets please cite the original sources and double-check their license.
-[^2]: From https://www.10xgenomics.com/datasets/visium-hd-cytassist-gene-expression-libraries-of-mouse-intestine
+[^1]:
+From https://www.10xgenomics.com/datasets/visium-hd-cytassist-gene-expression-libraries-of-mouse-intestine
-[^3]: Janesick, A. et al. High resolution mapping of the breast cancer tumor microenvironment using integrated single cell, spatial and in situ analysis of FFPE tissue. bioRxiv 2022.10.06.510405 (2022) doi:10.1101/2022.10.06.510405.
+[^2]: Janesick, A. et al. High resolution mapping of the breast cancer tumor
+microenvironment using integrated single cell, spatial and in situ analysis of FFPE
+tissue. bioRxiv 2022.10.06.510405 (2022) doi:10.1101/2022.10.06.510405.
-[^4]: Schapiro, D. et al. MCMICRO: A scalable, modular image-processing pipeline for multiplexed tissue imaging. Cold Spring Harbor Laboratory 2021.03.15.435473 (2021) doi:10.1101/2021.03.15.435473.
+[^3]:
+From https://www.10xgenomics.com/datasets/preview-data-ffpe-human-lung-cancer-with-xenium-multimodal-cell-segmentation-1-standard
-[^5]: Moffitt, J. R. et al. Molecular, spatial, and functional single-cell profiling of the hypothalamic preoptic region. Science 362, (2018).
+[^4]: Schapiro, D. et al. MCMICRO: A scalable, modular image-processing pipeline for
+multiplexed tissue imaging. Cold Spring Harbor Laboratory 2021.03.15.435473 (2021) doi:
+10.1101/2021.03.15.435473.
-[^6]: Hartmann, F. J. et al. Single-cell metabolic profiling of human cytotoxic T cells. Nat. Biotechnol. (2020) doi:10.1038/s41587-020-0651-8.
+[^5]: Moffitt, J. R. et al. Molecular, spatial, and functional single-cell profiling of
+the hypothalamic preoptic region. Science 362, (2018).
-[^7]: Windhager, J., Bodenmiller, B. & Eling, N. An end-to-end workflow for multiplexed image processing and analysis. bioRxiv 2021.11.12.468357 (2021) doi:10.1101/2021.11.12.468357.
+[^6]: Hartmann, F. J. et al. Single-cell metabolic profiling of human cytotoxic T cells.
+Nat. Biotechnol. (2020) doi:10.1038/s41587-020-0651-8.
-[^8]: Eling, N. & Windhager, J. Example imaging mass cytometry raw data. (2022). doi:10.5281/zenodo.5949116.
+[^7]: Windhager, J., Bodenmiller, B. & Eling, N. An end-to-end workflow for multiplexed
+image processing and analysis. bioRxiv 2021.11.12.468357 (2021) doi:
+10.1101/2021.11.12.468357.
-[^9]: Eling, N. & Windhager, J. steinbock results of IMC example data. (2022). doi:10.5281/zenodo.7412972.
+[^8]: Eling, N. & Windhager, J. Example imaging mass cytometry raw data. (2022). doi:
+10.5281/zenodo.5949116.
-[^10]: Guilliams, Martin, et al. "Spatial proteogenomics reveals distinct and evolutionarily conserved hepatic macrophage niches." Cell 185.2 (2022) doi:10.1016/j.cell2021.12.018
+[^9]: Eling, N. & Windhager, J. steinbock results of IMC example data. (2022). doi:
+10.5281/zenodo.7412972.
-[^11]: Pollaris, Lotte, et al. "SPArrOW: a flexible, interactive and scalable pipeline for spatial transcriptomics analysis." bioRxiv (2024) doi:10.1101/2024.07.04.601829
+[^10]: Guilliams, Martin, et al. "Spatial proteogenomics reveals distinct and
+evolutionarily conserved hepatic macrophage niches." Cell 185.2 (2022) doi:
+10.1016/j.cell2021.12.018
-[^12]: See https://github.com/giovp/spatialdata-sandbox/blob/main/spacem_scseahorse1/README.md
+[^11]: Pollaris, Lotte, et al. "SPArrOW: a flexible, interactive and scalable pipeline
+for spatial transcriptomics analysis." bioRxiv (2024) doi:10.1101/2024.07.04.601829
-[^13]: See https://github.com/giovp/spatialdata-sandbox/blob/main/spacem_helanih3t3/README.md
+[^12]:
+See https://github.com/giovp/spatialdata-sandbox/blob/main/spacem_helanih3t3/README.md
diff --git a/notebooks/developers_resources/storage_format/_latest_run_notebook.ipynb b/notebooks/developers_resources/storage_format/_latest_run_notebook.ipynb
index 3c1418b..9deab97 100644
--- a/notebooks/developers_resources/storage_format/_latest_run_notebook.ipynb
+++ b/notebooks/developers_resources/storage_format/_latest_run_notebook.ipynb
@@ -923,4 +923,4 @@
},
"nbformat": 4,
"nbformat_minor": 5
-}
\ No newline at end of file
+}
diff --git a/notebooks/developers_resources/storage_format/multiple_elements.zarr/.zattrs b/notebooks/developers_resources/storage_format/multiple_elements.zarr/.zattrs
index 21207e5..ccdcbfd 100644
--- a/notebooks/developers_resources/storage_format/multiple_elements.zarr/.zattrs
+++ b/notebooks/developers_resources/storage_format/multiple_elements.zarr/.zattrs
@@ -3,4 +3,4 @@
"spatialdata_software_version": "0.4.1.dev11+gb889b53",
"version": "0.1"
}
-}
\ No newline at end of file
+}
diff --git a/notebooks/developers_resources/storage_format/multiple_elements.zarr/zmetadata b/notebooks/developers_resources/storage_format/multiple_elements.zarr/zmetadata
index 3f86b6a..fa13d9b 100644
--- a/notebooks/developers_resources/storage_format/multiple_elements.zarr/zmetadata
+++ b/notebooks/developers_resources/storage_format/multiple_elements.zarr/zmetadata
@@ -1437,4 +1437,4 @@
}
},
"zarr_consolidated_format": 1
-}
\ No newline at end of file
+}
diff --git a/notebooks/examples/alignment_using_landmarks.ipynb b/notebooks/examples/alignment_using_landmarks.ipynb
index 6fcef5c..20ecb0f 100644
--- a/notebooks/examples/alignment_using_landmarks.ipynb
+++ b/notebooks/examples/alignment_using_landmarks.ipynb
@@ -570,7 +570,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.10.0"
+ "version": "3.11.0"
},
"vscode": {
"interpreter": {
diff --git a/notebooks/examples/alignment_utils.py b/notebooks/examples/alignment_utils.py
new file mode 100644
index 0000000..fb9e1d9
--- /dev/null
+++ b/notebooks/examples/alignment_utils.py
@@ -0,0 +1,33 @@
+from spatialdata.transformations import Affine
+from spatialdata import SpatialData
+from spatialdata.transformations import (
+ BaseTransformation,
+ Sequence,
+ get_transformation,
+ set_transformation,
+)
+
+
+AFFINE_VISIUM_XENIUM = Affine(
+ [
+ [1.61711846e-01, 2.58258090e00, -1.24575040e04],
+ [-2.58258090e00, 1.61711846e-01, 3.98647301e04],
+ [0.0, 0.0, 1.0],
+ ],
+ input_axes=("x", "y"),
+ output_axes=("x", "y"),
+)
+
+
+def postpone_transformation(
+ sdata: SpatialData,
+ transformation: BaseTransformation,
+ source_coordinate_system: str,
+ target_coordinate_system: str,
+):
+ for element_type, element_name, element in sdata._gen_elements():
+ old_transformations = get_transformation(element, get_all=True)
+ if source_coordinate_system in old_transformations:
+ old_transformation = old_transformations[source_coordinate_system]
+ sequence = Sequence([old_transformation, transformation])
+ set_transformation(element, sequence, target_coordinate_system)
diff --git a/notebooks/examples/densenet.ipynb b/notebooks/examples/densenet.ipynb
index 86e7775..1f25feb 100644
--- a/notebooks/examples/densenet.ipynb
+++ b/notebooks/examples/densenet.ipynb
@@ -28,34 +28,7 @@
"metadata": {
"tags": []
},
- "outputs": [
- {
- "data": {
- "text/html": [
- "\n",
- " \n",
- " "
- ],
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- }
- ],
+ "outputs": [],
"source": [
"%load_ext autoreload\n",
"%autoreload 2\n",
@@ -76,7 +49,18 @@
"outputs_hidden": false
}
},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "/opt/miniconda3/envs/ome/lib/python3.11/site-packages/monai/utils/module.py:399: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.\n",
+ " pkg = __import__(module) # top level module\n",
+ "/opt/miniconda3/envs/ome/lib/python3.11/site-packages/dask/dataframe/__init__.py:31: FutureWarning: The legacy Dask DataFrame implementation is deprecated and will be removed in a future version. Set the configuration option `dataframe.query-planning` to `True` or None to enable the new Dask Dataframe implementation and silence this warning.\n",
+ " warnings.warn(\n"
+ ]
+ }
+ ],
"source": [
"import os\n",
"from typing import Dict\n",
@@ -102,14 +86,6 @@
"mp.set_start_method(\"spawn\", force=True)"
]
},
- {
- "cell_type": "code",
- "execution_count": 3,
- "id": "839c39c8-ac77-4003-b366-6f9b191d386b",
- "metadata": {},
- "outputs": [],
- "source": []
- },
{
"cell_type": "markdown",
"id": "88cad743",
@@ -128,17 +104,15 @@
},
{
"cell_type": "markdown",
- "id": "01df1f51-29db-4955-8189-42648e5cd95e",
+ "id": "85bdf657-df9f-485a-81e4-3bb504599c2a",
"metadata": {},
"source": [
- "You can download the processed Visium and Xenium data (already aligned and Xenium with already the celltype information) from here: [Visium dataset](https://s3.embl.de/spatialdata/spatialdata-sandbox/visium_associated_xenium_io_aligned.zip), [Xenium dataset](https://s3.embl.de/spatialdata/spatialdata-sandbox/xenium_rep1_io_aligned.zip). In alternative you can obtain the data by running [this analysis notebook](https://github.com/scverse/spatialdata-notebooks/blob/main/notebooks/paper_reproducibility/00_xenium_and_visium.ipynb).\n",
- "\n",
- "Please rename the files to `visium_aligned.zarr` and `xenium_aligned.zarr` and place them in the same folder as this notebook (or use symlinks to make the data accessible)."
+ "You can download the Visium and Xenium data from here: [Visium dataset](https://s3.embl.de/spatialdata/spatialdata-sandbox/visium_associated_xenium_io.zip), [Xenium dataset](https://s3.embl.de/spatialdata/spatialdata-sandbox/xenium_rep1_io.zip). Please rename the data to `xenium.zarr` and `visium.zarr`, or use symlinks to make the paths below accesible."
]
},
{
"cell_type": "code",
- "execution_count": 4,
+ "execution_count": 3,
"id": "3bfd5433",
"metadata": {
"ExecuteTime": {
@@ -150,24 +124,475 @@
"outputs_hidden": false
}
},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "WARNING:ome_zarr.io:version mismatch: detected: RasterFormatV02, requested: FormatV04\n",
+ "/opt/miniconda3/envs/ome/lib/python3.11/site-packages/zarr/creation.py:610: UserWarning: ignoring keyword argument 'read_only'\n",
+ " compressor, fill_value = _kwargs_compat(compressor, fill_value, kwargs)\n",
+ "WARNING:ome_zarr.io:version mismatch: detected: RasterFormatV02, requested: FormatV04\n",
+ "WARNING:ome_zarr.io:version mismatch: detected: RasterFormatV02, requested: FormatV04\n",
+ "/opt/miniconda3/envs/ome/lib/python3.11/site-packages/zarr/creation.py:610: UserWarning: ignoring keyword argument 'read_only'\n",
+ " compressor, fill_value = _kwargs_compat(compressor, fill_value, kwargs)\n",
+ "/opt/miniconda3/envs/ome/lib/python3.11/site-packages/zarr/creation.py:610: UserWarning: ignoring keyword argument 'read_only'\n",
+ " compressor, fill_value = _kwargs_compat(compressor, fill_value, kwargs)\n",
+ "/opt/miniconda3/envs/ome/lib/python3.11/site-packages/zarr/creation.py:610: UserWarning: ignoring keyword argument 'read_only'\n",
+ " compressor, fill_value = _kwargs_compat(compressor, fill_value, kwargs)\n",
+ "/opt/miniconda3/envs/ome/lib/python3.11/site-packages/zarr/creation.py:610: UserWarning: ignoring keyword argument 'read_only'\n",
+ " compressor, fill_value = _kwargs_compat(compressor, fill_value, kwargs)\n",
+ "/opt/miniconda3/envs/ome/lib/python3.11/site-packages/zarr/creation.py:610: UserWarning: ignoring keyword argument 'read_only'\n",
+ " compressor, fill_value = _kwargs_compat(compressor, fill_value, kwargs)\n",
+ "WARNING:ome_zarr.io:version mismatch: detected: RasterFormatV02, requested: FormatV04\n",
+ "/opt/miniconda3/envs/ome/lib/python3.11/site-packages/zarr/creation.py:610: UserWarning: ignoring keyword argument 'read_only'\n",
+ " compressor, fill_value = _kwargs_compat(compressor, fill_value, kwargs)\n",
+ "WARNING:ome_zarr.io:version mismatch: detected: RasterFormatV02, requested: FormatV04\n",
+ "/opt/miniconda3/envs/ome/lib/python3.11/site-packages/zarr/creation.py:610: UserWarning: ignoring keyword argument 'read_only'\n",
+ " compressor, fill_value = _kwargs_compat(compressor, fill_value, kwargs)\n"
+ ]
+ }
+ ],
"source": [
- "XENIUM_SDATA_PATH = \"xenium_aligned.zarr\"\n",
- "VISIUM_SDATA_PATH = \"visium_aligned.zarr\"\n",
+ "XENIUM_SDATA_PATH = \"xenium.zarr\"\n",
+ "VISIUM_SDATA_PATH = \"visium.zarr\"\n",
"\n",
"assert os.path.isdir(XENIUM_SDATA_PATH)\n",
"assert os.path.isdir(VISIUM_SDATA_PATH)\n",
"\n",
"xenium_sdata = read_zarr(XENIUM_SDATA_PATH)\n",
- "visium_sdata = read_zarr(VISIUM_SDATA_PATH)\n",
+ "visium_sdata = read_zarr(VISIUM_SDATA_PATH)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "64d28b37-d420-4cb5-8ff5-2e25008be1ff",
+ "metadata": {},
+ "source": [
+ "The downloaded data is not aligned (it requires alignment via an affine transformation) and also needs celltype annotations for the Xenium cells. \n",
+ "\n",
+ "The [\"Alignemnt using landmarks\" tutorial](alignment_using_landmarks.ipynb) shows how to achieve the alignment. \n",
+ "[This analysis notebook](https://github.com/scverse/spatialdata-notebooks/blob/main/notebooks/paper_reproducibility/00_xenium_and_visium.ipynb) shows how the cell-types are mapped into the Xenium data. \n",
"\n",
- "assert \"celltype_major\" in xenium_sdata[\"table\"].obs, (\n",
- " \"The Xenium data does not contain the cell types annotation; it seems that it refers to the Xenium \"\n",
- " \"Zarr data that has not been processed with the analysis notebook mentioned in the 'Getting the Zarr\"\n",
- " \"files' section.\"\n",
+ "For the sake of simplifity and to keep the focus of this notebook on training a densenet to predict cell-types from image data, we will adjust the affine transformation with the aid of some utils functions, and we will add the cell-type information to the Xenium data using some precomputed information that we will download and merge with the `SpatialData` object on-the-fly."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "id": "840a6d43-77cf-4df9-9a39-dfe0aea7521b",
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Affine (x, y -> x, y)\n",
+ " [ 1.61711846e-01 2.58258090e+00 -1.24575040e+04]\n",
+ " [-2.58258090e+00 1.61711846e-01 3.98647301e+04]\n",
+ " [0. 0. 1.]\n"
+ ]
+ }
+ ],
+ "source": [
+ "from alignment_utils import AFFINE_VISIUM_XENIUM, postpone_transformation\n",
+ "from spatialdata.transformations import Identity\n",
+ "\n",
+ "print(AFFINE_VISIUM_XENIUM)\n",
+ "\n",
+ "postpone_transformation(\n",
+ " sdata=visium_sdata,\n",
+ " transformation=AFFINE_VISIUM_XENIUM,\n",
+ " source_coordinate_system=\"CytAssist_FFPE_Human_Breast_Cancer\",\n",
+ " target_coordinate_system=\"aligned\",\n",
+ ")\n",
+ "postpone_transformation(\n",
+ " sdata=xenium_sdata,\n",
+ " transformation=Identity(),\n",
+ " source_coordinate_system=\"global\",\n",
+ " target_coordinate_system=\"aligned\",\n",
")"
]
},
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "id": "07eb1e48-71ed-4822-bfd5-8e4bbe2e96fa",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " celltype_major | \n",
+ " cell_id | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " | 0 | \n",
+ " Normal Epithelial | \n",
+ " 1 | \n",
+ "
\n",
+ " \n",
+ " | 1 | \n",
+ " Cancer Epithelial | \n",
+ " 2 | \n",
+ "
\n",
+ " \n",
+ " | 2 | \n",
+ " Endothelial | \n",
+ " 3 | \n",
+ "
\n",
+ " \n",
+ " | 3 | \n",
+ " Plasmablasts | \n",
+ " 4 | \n",
+ "
\n",
+ " \n",
+ " | 4 | \n",
+ " Cancer Epithelial | \n",
+ " 5 | \n",
+ "
\n",
+ " \n",
+ " | ... | \n",
+ " ... | \n",
+ " ... | \n",
+ "
\n",
+ " \n",
+ " | 167775 | \n",
+ " Cancer Epithelial | \n",
+ " 167776 | \n",
+ "
\n",
+ " \n",
+ " | 167776 | \n",
+ " Cancer Epithelial | \n",
+ " 167777 | \n",
+ "
\n",
+ " \n",
+ " | 167777 | \n",
+ " Cancer Epithelial | \n",
+ " 167778 | \n",
+ "
\n",
+ " \n",
+ " | 167778 | \n",
+ " Cancer Epithelial | \n",
+ " 167779 | \n",
+ "
\n",
+ " \n",
+ " | 167779 | \n",
+ " Cancer Epithelial | \n",
+ " 167780 | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
167780 rows × 2 columns
\n",
+ "
\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " cell_id | \n",
+ " transcript_counts | \n",
+ " control_probe_counts | \n",
+ " control_codeword_counts | \n",
+ " total_counts | \n",
+ " cell_area | \n",
+ " nucleus_area | \n",
+ " region | \n",
+ " celltype_major | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " | 0 | \n",
+ " 1 | \n",
+ " 28 | \n",
+ " 1 | \n",
+ " 0 | \n",
+ " 29 | \n",
+ " 58.387031 | \n",
+ " 26.642188 | \n",
+ " cell_circles | \n",
+ " Normal Epithelial | \n",
+ "
\n",
+ " \n",
+ " | 1 | \n",
+ " 2 | \n",
+ " 94 | \n",
+ " 0 | \n",
+ " 0 | \n",
+ " 94 | \n",
+ " 197.016719 | \n",
+ " 42.130781 | \n",
+ " cell_circles | \n",
+ " Cancer Epithelial | \n",
+ "
\n",
+ " \n",
+ " | 2 | \n",
+ " 3 | \n",
+ " 9 | \n",
+ " 0 | \n",
+ " 0 | \n",
+ " 9 | \n",
+ " 16.256250 | \n",
+ " 12.688906 | \n",
+ " cell_circles | \n",
+ " Endothelial | \n",
+ "
\n",
+ " \n",
+ " | 3 | \n",
+ " 4 | \n",
+ " 11 | \n",
+ " 0 | \n",
+ " 0 | \n",
+ " 11 | \n",
+ " 42.311406 | \n",
+ " 10.069844 | \n",
+ " cell_circles | \n",
+ " Plasmablasts | \n",
+ "
\n",
+ " \n",
+ " | 4 | \n",
+ " 5 | \n",
+ " 48 | \n",
+ " 0 | \n",
+ " 0 | \n",
+ " 48 | \n",
+ " 107.652500 | \n",
+ " 37.479688 | \n",
+ " cell_circles | \n",
+ " Cancer Epithelial | \n",
+ "
\n",
+ " \n",
+ " | ... | \n",
+ " ... | \n",
+ " ... | \n",
+ " ... | \n",
+ " ... | \n",
+ " ... | \n",
+ " ... | \n",
+ " ... | \n",
+ " ... | \n",
+ " ... | \n",
+ "
\n",
+ " \n",
+ " | 167775 | \n",
+ " 167776 | \n",
+ " 229 | \n",
+ " 1 | \n",
+ " 0 | \n",
+ " 230 | \n",
+ " 220.452813 | \n",
+ " 60.599688 | \n",
+ " cell_circles | \n",
+ " Cancer Epithelial | \n",
+ "
\n",
+ " \n",
+ " | 167776 | \n",
+ " 167777 | \n",
+ " 79 | \n",
+ " 0 | \n",
+ " 0 | \n",
+ " 79 | \n",
+ " 37.389375 | \n",
+ " 25.242344 | \n",
+ " cell_circles | \n",
+ " Cancer Epithelial | \n",
+ "
\n",
+ " \n",
+ " | 167777 | \n",
+ " 167778 | \n",
+ " 397 | \n",
+ " 0 | \n",
+ " 0 | \n",
+ " 397 | \n",
+ " 287.058281 | \n",
+ " 86.700000 | \n",
+ " cell_circles | \n",
+ " Cancer Epithelial | \n",
+ "
\n",
+ " \n",
+ " | 167778 | \n",
+ " 167779 | \n",
+ " 117 | \n",
+ " 0 | \n",
+ " 0 | \n",
+ " 117 | \n",
+ " 235.354375 | \n",
+ " 25.197188 | \n",
+ " cell_circles | \n",
+ " Cancer Epithelial | \n",
+ "
\n",
+ " \n",
+ " | 167779 | \n",
+ " 167780 | \n",
+ " 378 | \n",
+ " 0 | \n",
+ " 0 | \n",
+ " 378 | \n",
+ " 270.079531 | \n",
+ " 111.806875 | \n",
+ " cell_circles | \n",
+ " Cancer Epithelial | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
167780 rows × 9 columns
\n",
+ "