Training and evaluating a variational autoencoder for pan-cancer gene expression data
Switch branches/tags
Clone or download
Permalink
Failed to load latest commit information.
.github add stale bot yml (#88) Oct 9, 2017
config remove z parameter sweep (#129) Apr 19, 2018
data Quick Fix to Simulation Processing (#118) Mar 23, 2018
figures Remove simulation analysis (#135) Apr 23, 2018
models Add Two Hidden Layer Model (#81) Oct 5, 2017
param_sweep add parameter sweep results Aug 18, 2017
param_sweep_adage add adage param sweep results Aug 30, 2017
param_sweep_twohidden Adding Parameter Sweep for Two Hidden Layer Model / Reorganizing nbco… Oct 3, 2017
param_sweep_twohidden300 300 feature hidden layer parameter sweep (#76) Oct 3, 2017
results add in adage tsne features May 7, 2018
scripts remove ez compression files May 7, 2018
tybalt move assert to after checking test (#143) Oct 21, 2018
.gitattributes Add Simulation Data (#107) Feb 13, 2018
.gitignore ignore prototypes May 7, 2018
LICENSE.md getting with the times Aug 18, 2017
README.md Add git-lfs instructions (#142) Oct 4, 2018
adage_training.ipynb Updating how ADAGE weights are saved (#93) Oct 10, 2017
download_data.sh adding data download instructions Aug 21, 2017
environment.yml add missing colon! (#109) Feb 21, 2018
extract_tybalt_weights.ipynb replacing rnaseq.tsv with rnaseq.tsv.gz where appropriate Oct 4, 2017
gpu-environment.yml Update conda environments (#108) Feb 19, 2018
hgsc_subtypes_tybalt.ipynb replacing rnaseq.tsv with rnaseq.tsv.gz where appropriate Oct 4, 2017
high_weight_genes.ipynb Adding High Weight Gene Analysis (#95) Oct 16, 2017
latent_space_comparison.ipynb update adage feature path in latent space comparison (#94) Oct 11, 2017
param_sweep.sh add parameter sweep PMACS command Aug 18, 2017
parameter_sweep.md Add Parameter Sweep Figures and Logic from 300 feature model (#77) Oct 3, 2017
process_data.ipynb Adding full status matrix (#82) Oct 5, 2017
setup.py add util import to init (#133) Apr 20, 2018
tsne_tybalt_features.ipynb Adding ADAGE model to tsne visualization (#114) Mar 20, 2018
tybalt_twohidden.ipynb Add Two Hidden Layer Model (#81) Oct 5, 2017
tybalt_vae.ipynb Update conda environments (#108) Feb 19, 2018

README.md

Tybalt 😼

A Variational Autoencoder trained on Pan-Cancer Gene Expression

Gregory Way and Casey Greene 2017

DOI

The repository stores scripts to train, evaluate, and extract knowledge from a variational autoencoder (VAE) trained on 33 different cancer-types from The Cancer Genome Atlas (TCGA).

The specific VAE model is named Tybalt after an instigative, cat-like character in Shakespeare's "Romeo and Juliet". Just as the character Tybalt sets off the series of events in the play, the model Tybalt begins the foray of VAE manifold learning in transcriptomics. Also, deep unsupervised learning likes cats.

We discuss the training and evaluation of Tybalt in our PSB paper:

Extracting a Biologically Relevant Latent Space from Cancer Transcriptomes with Variational Autoencoders.

Citation

Way, GP, Greene, CS. 2018. Extracting a biologically relevant latent space from cancer transcriptomes with variational autoencoders. Pacific Symposium on Biocomputing 23:80-91. doi:10.1142/9789813235533_0008

Notes

As discovered by @enricoferrero, the preprint text (section 2.2) states that the top median absolute deviation (MAD) genes were selected for subsetting, when the data processing code (process_data.ipynb) actually outputs the top mean absolute deviation genes. We discuss this discrepancy and its potential impact in issue #99.

git-lfs (https://git-lfs.github.com/) must be installed prior to cloning the repository. If it is not installed, run git lfs install

The Data

TCGA has collected numerous different genomic measurements from over 10,000 different tumors spanning 33 different cancer-types. In this repository, we extract cancer signatures from gene expression data (RNA-seq).

The RNA-seq data serves as a measurement describing the high-dimensional state of each tumor. As a highly heterogeneous disease, cancer exists in several different combination of states. Our goal is to extract these different states using high capacity models capable of identifying common signatures in gene expression data across different cancer-types.

The Model

We present a variational autoencoder (VAE) applied to cancer gene expression data. A VAE is a deep generative model introduced by Kingma and Welling in 2013. The model has two direct benefits of modeling cancer gene expression data.

  1. Automatically engineer non-linear features
  2. Learning the reduced dimension manifold of cancer expression space

As a generative model, the reduced dimension features can be sampled from to simulate data. The manifold can also be interpolated to interrogate trajectories and transitions between states.

VAEs have typically been applied to image data and have demonstrated remarkable generative capacity and modeling flexibility. VAEs are different from deterministic autoencoders because of the added constraint of normally distributed feature activations per sample. This constraint not only regularizes the model, but also provides the interpretable manifold.

Below is a t-SNE visualization of the VAE encoded features (p = 100) for all tumors.

VAE t-SNE

Training

The current model training is explained in this notebook.

Tybalt dependencies are listed in environment.yml. To download and activate this environment run:

# conda version 4.4.10
conda env create --force --file environment.yml

# activate environment
conda activate tybalt

Tybalt is also configured to train on GPUs using gpu-environment.yml. To activate this environment run:

# conda version 4.4.10
conda env create --force --file gpu-environment.yml

# activate environment
conda activate tybalt-gpu

For a complete pipeline with reproducibility instructions, refer to run_pipeline.sh. Note that scripts originally written in Jupyter notebooks were ported to the scripts folder for pipeline purposes with:

jupyter nbconvert --to=script --FilesWriter.build_directory=scripts/nbconverted *.ipynb

Architecture

We select the top 5,000 most variably expressed genes by median absolute deviation. We compress this 5,000 vector of gene expression (for all samples) into two vectors of length 100; one representing the a mean and the other the variance. This vector can be sampled from to generate samples from an approximation of the data generating function. This hidden layer is then reconstructed back to the original dimensions. We use batch normalization and relu activation layers in the compression steps to prevent dead nodes and positive weights. We use a sigmoid activation in the decoder. We use the Keras library with a TensorFlow backend for training.

VAE Architecture

Parameter sweep

In order to select the most optimal parameters for the model, we ran a parameter search over a small grid of parameters. See parameter_sweep.md for more details.

Overall, we selected optimal learning rate = 0.0005, batch size = 50, and epochs = 100. Because training did not improve much between 50 and 100 epochs, we used a 50 epoch model. Training and validation loss across 50 training epochs for the optimal model is shown below.

Training Performance

Model Evaluation

After training with optimal hyper parameters, the unsupervised model can be interpreted. For example, we can observe the distribution of activation patterns for all tumors across specific nodes. The first 10 nodes (of 100) are visualized below.

Node Activation

In this scenario, each node activation pattern contributes uniquely to each tumor and may represent specific gene expression signatures of biological significance. The distribution is heavily right skewed, with some nodes capturing slightly bimodal attributes.