Sparse Cooccurrence Network Investigation for Compositional data Pronounced 'scenic'.
NOTE: The most up to date version of the SCNIC repo is here!
SCNIC is a package for the generation and analysis of cooccurrence (positive correlation) networks with compositional data. Data generated by many next gen sequencing experiments is compositional (is a subsampling of the total community) which violates assumptions of typical cooccurence network analysis techniques. 16S sequencing data is often very compositional in nature so methods such as SparCC (https://bitbucket.org/yonatanf/sparcc) have been developed for studying correlations microbiome data. SCNIC is designed with compositional data in mind and so provides multiple correlation measures including SparCC.
Running SCNIC is possible via two different methods. SCNIC is packaged with scripts to allow running it on the command line but also is avaliable as a Qiime2 plugin (https://www.github.com/lozuponelab/q2-SCNIC). Either method is valid but usage of the Qiime2 plugin provides easier access when working within the Qiime2 ecosystem.
The 'within' method takes as input BIOM formatted files and forms cooccurrence networks using a user specified correlation metric.
From the correlation network generated as part of the within step, SCNIC finds modules of cooccurring observations by finding groups of observations which all have a minimum pairwise correlation value. Modules are summarized and a new biom table with observations contained in modules collapsed into single observations are returned. This biom table along with a list of modules and their contents are output. A gml file of the network that can be opened using network visualization tools such as cytoscape is created which contains all observation metadata provided in the input biom file as well as module information. Please be aware that the networks output by this analysis will only include positive correlations as only positive correlations are used in module finding and summarization.
The 'between' method takes two biom tables as input and calculates all pairwise correlations between the tables using a selection of correlation metrics. A gml correlation network is output as well as a file containing statistics and p-values of all correlations.
We recommend using mamba (or conda) and our environment.yaml file to install the full environment in one step. See the mamba documentation for mamba installation steps.
wget https://raw.githubusercontent.com/lozuponelab/SCNIC/master/environment.yml
mamba env create -n SCNIC2 --file environment.yml
# Optional cleanup
rm environment.yml
If not using mamba, you can install using conda env create in place of mamba env create. However, conda will be slower and may struggle to solve the dependencies.
Users with Apple M1/M2 chips or other ARM architecture should pass CONDA_SUBDIR=osx-64 at the beginning of the env create command, as can be seen in the following:
CONDA_SUBDIR=osx-64 mamba env create -n SCNIC3 --file environment.yml
It is recommended to install all of SCNIC's dependencies via conda in a new conda environment. To do this you only need to create a new environment with SCNIC installed. However, since conda has not always accepted the latest version of SCNIC please manually install SCNIC into your conda environment via PIP. On some computers, there are SciPy conflicts when installing via conda, so we recommend installing SciPy via pip.
conda create -n SCNIC python=3 SCNIC
conda activate SCNIC
To download the latest release from PyPI install using this command:
pip install "scipy>=1.9.0,<=1.10.1"
pip install SCNIC
SCNIC depends on a variety of software all of which can be install via conda and most of which can be installed by pip. The recommended installation method is to install via pip but you must also install fastspar and parallel and have them in your path. If using the environment.yaml installation, this should be unnecessary.
To do so you can create a conda environment below, then install both fastspar and parallel.
ex:
$conda install -c bioconda -c conda-forge fastspar
To download the lastest changes to the repository use the following commands:
git clone https://github.com/lozuponelab/SCNIC.git
cd SCNIC/
python setup.py install
NOTE: This latest code may not be functional and should only be used if you want to play around with the code this is based on.
SCNIC_analysis.py within -i example_table.biom -o within_output/ -m sparcc
SCNIC_analysis.py modules -i within_output/correls.txt -o modules_output/ --min_r .35 --table example_table.biom
NOTE: We use a minimum R value of .3 when running SparCC with 16S data as a computationally demanding bootstrapping procedure must be run to determine p-values. We have run SparCC with 1 million bootstraps on a variety of datasets and found that a R value of between .3 and .35 will always return FDR adjusted p-values less than .05 and .1 respectively.
SCNIC_analysis.py between -1 example_table1.biom -2 example_table2.biom -o output_folder/ -m spearman --min_p .05