Lung dataset has no raw counts

[emdann]

Hello,
I was trying to reproduce the Atlas-level integration tutorial. The dataset downloaded via figshare is supposed to store raw counts in adata.layers['counts'], but these values here don't really look like counts.

adata = sc.read(
    "data/lung_atlas.h5ad",
    backup_url="https://figshare.com/ndownloader/files/24539942",
)

adata.layers['counts'].data

array([1.       , 1.       , 1.       , ..., 1.1387753, 2.2072291,
       1.0936011], dtype=float32)

Are these some sort of soup-corrected counts, and does it matter?

[adamgayoso]

Based on page 7 of the supplementary information of the scIB paper in Nature Methods, it looks like one dataset contained in this example had soupX corrected counts only which appear to be by default not counts.

Technically also the NB distribution can handle non-integer counts through the use of gamma functions in the likelihood (instead of factorials)

[emdann]

Thank you for clarifying! The tutorial says

Note that this dataset has the counts already separated in a layer. Here, adata.X contains log transformed scran normalized expression.

Maybe worth specifying it's SoupX corrected counts?

[adamgayoso]

This is a good point, we should clarify that while we generally expect integer UMI counts, technically we can handle continuous values that are intended to represent counts (e.g., soupx). Also with smartseq2 data, we typically normalize the reads per gene to the median gene length. We actually have this info box in the docs site version of the tutorial.

@jjhong922 can you please add into the main overview tutorial this info about pseudocounts etc? There's already an info box in that one too about raw counts.