/
data-overview.Rmd
134 lines (80 loc) · 5.61 KB
/
data-overview.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
---
title: "Data overview"
author: "Joyce Hsiao"
output: workflowr::wflow_html
---
## Data availability
* This document lists the datasets analyzed in the study.
* We stored all datasets as `expressionSets` (require `Biobase` package).
* We provided data in TXT format on the [Gilad lab website](https://giladlab.uchicago.edu/wp-content/uploads/2019/02/Hsiao_et_al_2019.tar.gz) for the 11,093 genes analyzed in the study and for each of the 888 samples: molecule count, sample phenotpyes, gene information, phenotype label descriptiond and FUCCI intensity data.
## Data structure
We collected two types of data for each single cell sample: single-cell RNA-seq using C1 plates and FUCCI image intensity data.
* Raw RNA-seq data: `data/eset-raw.rds`
* Filtered RNA-seq data: `data/eset-filtered.rds`
* FUCCI intensity data: `data/intensity.rds`
* FUCCI intensity data adjusted for batch effect: `output/images-normalize-anova.Rmd/pdata.adj.rds`
* Final data combining filtered intensity and RNA-seq, including 11093 genes and 888 samples: `data/eset-final.rds`
Code used to generate data from `data/eset-raw.rds` to `data/eset-final.rds` is stored in `code/output-raw-2-final.R`.
### FUCCI intensity data
* Combined intensity data are stored in `data/intensity.rds`. These include samples that were identified to have a single nuclei .
* Data generated by [combine-intensity-data.R](data/combine-intensity-data.R). Combining image analysis output stored in `/project2/gilad/fucci-seq/intensities_stats/` into one `data.frame` and computes summary statistics, including background-corrected RFP and GFP intensity measures.
### Sequencing data
* Raw data from each C1 plate are stored separatley in `data/eset/` by experiment (batch) ID.
* Raw data combining C1 plate are stored in `data/eset-raw.rds`.
* Filtered raw data excluding low-quality sequencing samples and genes that are lowly expressed or overly expressed are stored in `data/eset-filtered.rds`.
### Phenotypic data of singleton samples
* Data file 1: all 1536 samples before filtering (`output/data-overview.Rmd/phenotypes_allsamples.txt`)
* Data file 2: 888 samples after filtering (`output/data-overview.Rmd/phenotypes_singletonsamples.txt`)
* Data file 3: phenotype labels (`output/data-overview.Rmd/phenotypes_labels.txt`)
```{r, eval=F}
library(Biobase)
eset_raw <- readRDS("../data/eset-raw.rds")
df <- data.frame(sample_id=rownames(pData(eset_raw)), pData(eset_raw), stringsAsFactors = F)
write.table(df, quote=F, sep="\t",
row.names = F, col.names = T,
file = "../output/data-overview.Rmd/phenotypes_allsamples.txt")
eset_final <- readRDS("../data/eset-final.rds")
df <- data.frame(sample_id=rownames(pData(eset_final)), pData(eset_final), stringsAsFactors = F)
write.table(df, quote=F, sep="\t",
row.names = F, col.names = T,
file = "../output/data-overview.Rmd/phenotypes_singletonsamples.txt")
labels <- data.frame(var_names=rownames(varMetadata(eset_raw)),
labels=varMetadata(eset_raw)$labeDescription, stringsAsFactors = F)
labels <- rbind(labels,
data.frame(var_names=rownames(varMetadata(eset_final)),
labels=varMetadata(eset_final)$labelDescription, stringsAsFactors = F)[45:54,])
write.table(labels, quote=F,
sep="\t", row.names = F, col.names = T,
file = "../output/data-overview.Rmd/phenotypes_labels.txt")
# testing reading files
library(data.table)
df_all <- fread(file = "../output/data-overview.Rmd/phenotypes_allsamples.txt")
df_singles <- fread(file = "../output/data-overview.Rmd/phenotypes_singletonsamples.txt")
df_labels <- fread(file = "../output/data-overview.Rmd/phenotypes_labels.txt")
```
```{r, eval=F}
eset_final <- readRDS("../data/eset-final.rds")
df <- data.frame(sample_id=rownames(pData(eset_raw)), pData(eset_raw), stringsAsFactors = F)
write.table(df, quote=F, sep="\t", quote=F,
row.names = F, col.names = T,
file = "../output/data-overview.Rmd/phenotypes_allsamples.txt")
write.table(data.frame(var_names=rownames(varMetadata(eset_raw)),
labels=varMetadata(eset_raw)$labeDescription, stringsAsFactors = F),
quote=F,
sep="\t", row.names = F, col.names = T,
file = "../output/data-overview.Rmd/phenotypes_allsamples_labels.txt")
# testing reading files
library(data.table)
df_test <- fread(file = "../output/data-overview.Rmd/phenotypes_allsamples.txt")
df_labels_test <- fread(file = "../output/data-overview.Rmd/phenotypes_allsamples_labels.txt")
```
## How to access expressionSets
We store feature-level (gene) read count and molecule count in `expressionSet` (`data/eset`) objects, which also contain sample metadata (e.g., assigned indivdual ID, cDNA concentraion) and quality filtering criteria (e.g., number of reads mapped to FUCCI transgenes, ERCC conversion rate). Data from different C1 plates are stored in separate `eset` objects:
To combine `eset` objects from the different C1 plates:
`eset <- Reduce(combine, Map(readRDS, Sys.glob("data/eset/*.rds")))`
To access data stored in `expressionSet`:
* `exprs(eset)`: access count data, 20,421 features by 1,536 single cell samples.
* `pData(eset)`: access sample metadata. Returns data.frame of 1,536 samples by 43 labels. Use `varMetadata(phenoData(eset))` to view label descriptions.
* `fData(eset)`: access feature metadata. Returns data.frame of 20,421 features by 6 labels. Use `varMetadata(featureData(eset))` to view label descriptions.
* `varMetadata(phenoData(eset))`: view the sample metadata labels.
* `varMetadata(featureData(eset))`: view the feature (gene) metadata labels.