This is a mixed notebook and data repository for retrieving cancer data for Project Cognoma. Currently, all data is from the TCGA Pan-Cancer collection of the UCSC Xena Browser.
The data acquisition and analysis is executing by running Jupyter notebooks in the following order:
1.TCGA-download.ipynb— download and compress TCGA datasets.
2.TCGA-process.ipynb— convert downloaded TCGA datasets into sample × gene matrixes.
The execute.sh script installs and activates the conda environment and then executes the notebooks in order. Run with the command bash execute.sh from the repository's root directory.
The repository contains the following directories:
download— contains files retrieved from an external location whose content is unmodified. Downloaded files are currently not tracked due to large file size, although associated metadata files are tracked for versioning.data— contains generated datasets. The complete matrix files are not currently tracked due to file size, but randomly-subsetted versions are available for development use (seedata/subset).
The complete datasets created by this repository (data/expression-matrix.tsv.bz2 and data/mutation-matrix.tsv.bz2) are uploaded to figshare. Since this is a manual process, check the figshare REFERENCES section to see which commit these datasets derive from. In other words, the latest version on figshare may lag behind this repository.
This repository uses conda to manage its environment, which is named cognoma-cancer-data. The required packages and versions are listed in environment.yml. If as a developer, you require an additional package, add it to environment.yml.