Bias, robustness and scalability in differential expression analysis of single-cell RNA-seq data
This repository contains all the necessary code to perform the evaluation of differential expression analysis methods in single-cell RNA-seq data, available in
- C Soneson & MD Robinson: Bias, robustness and scalability in single-cell differential expression analysis. Nature Methods 15:255-261 (2018).
In this paper, we compare the performance of more than 30 approaches to differential gene expression analysis in the context of single-cell RNA-seq data. The main results can be further browsed in a shiny app.
Note: The purpose of the
conquer_comparison repository is to provide a public record of the exact code that was used for our publication (Soneson & Robinson, Nature Methods 2018). In particular, it is not intended to be a software package or a general pipeline for differential expression analysis of single-cell data. As a consequence, running the code requires the same software and package versions that were used for our analyses (all versions are indicated in the paper). As the analysis involved running a large number of methods on many data sets and over an extended period of time, we cannot guarantee that it will run successfully with new releases of the software, or that exactly the same results will be obtained with newer versions of the packages. While the repository will not be updated to ensure that it runs with every new version of the used packages, the issues can be used to post questions and/or solutions as they arise.
The repository contains the following information:
config/contains configuration files for all the data sets that we considered. The configuration files detail the cell populations that were compared, as well as the number of cells per group used in each comparison.
data/contains some of the raw data that was used for the comparison. All data sets that were used can be downloaded as a bundle from http://imlspenticton.uzh.ch/robinson_lab/conquer_de_comparison/
export_results/contains results for the final figures, in tabular format
scripts/contains all R scripts used for the evaluation
shiny/contains the code for a shiny app built to browse the results (http://imlspenticton.uzh.ch:3838/scrnaseq_de_evaluation)
unit_tests/contains unit tests that were used to check the calculations
Makefileis the master script, which outlines the entire evaluation and calls all scripts in the appropriate order
plot_methods.mkare additional makefiles listing the filter settings, data set and differential expression methods used in the comparison
Running the comparison
Assuming that all prerequisites are available, the comparison can be run by simply typing
from the top directory (note, however, that this will take a significant amount of time!). The Makefile reads the three files include_filterings.mk, include_datasets.mk and include_methods.mk and performs the evaluation using the data sets, methods and filterings defined in these. The file plot_methods.mk detail the methods included in the final summary plots. For the code to execute properly, an .rds file containing a MultiAssayExperiment object for each data set must be provided in the
data/ directory. Such files can be downloaded, e.g., from the
conquer database. The files used for the evaluation are bundled together in an archive that can be downloaded from here
Adding a differential expression method
To add a differential expression method to the evaluation, construct a script in the form of the provided
apply_*.R scripts (in the
scripts/ directory), where
* should be the name of the method. Then add the name of the method to
include_methods.mk. To make it show up in the summary plots, add it to
plot_methods.mk and assign it a color in
Adding a data set
To add a data set, put the .rds file containing the MultiArrayExperiment object in the
data/ folder and construct a script in the form of the provided
generate_config_*.R scripts (in the
scripts/ directory), where
* should be the name of the data set. Then add the name of the dataset to the appropriate variables in
include_datasets.mk. Also, add the data set to the
data/dataset_type.txt file, indicating the type of values in each data set.
A note on the data sets
Most data sets in the published evaluation are obtained from the
conquer repository. The RPM values for the Usoskin dataset was downloaded from http://linnarssonlab.org/drg/ on December 18, 2016. The 10X data set was downloaded from https://support.10xgenomics.com/single-cell-gene-expression/datasets on September 17, 2017.
Cell cycle genes
The list of mouse cell cycle genes was obtained from http://www.sabiosciences.com/rt_pcr_product/HTML/PAMM-020A.html on March 9, 2017.
To run all the unit tests, start
R, load the
testthat package and run
source("scripts/run_unit_tests.R"). Alternatively, to run just the unit tests in a given file, do e.g.
test_file("unit_tests/test_trueperformance.R", reporter = "summary").