Permalink
Browse files

Add titer model pipeline & docs

  • Loading branch information...
sidneymbell committed Oct 2, 2018
1 parent 1d288b2 commit 2cdfa7750519616cc1b16077c116673e6f078611
Showing with 68,936 additions and 2 deletions.
  1. +1 −2 README.md
  2. +45 −0 titer_model/README.md
  3. +620 −0 titer_model/implementation-nextstrain-augur/LICENSE.txt
  4. +46 −0 titer_model/implementation-nextstrain-augur/README.md
  5. 0 titer_model/implementation-nextstrain-augur/__init__.py
  6. 0 titer_model/implementation-nextstrain-augur/augur/__init__.py
  7. +100 −0 titer_model/implementation-nextstrain-augur/augur/align.py
  8. +217 −0 titer_model/implementation-nextstrain-augur/augur/export.py
  9. +3 −0 titer_model/implementation-nextstrain-augur/augur/filenames.py
  10. +154 −0 titer_model/implementation-nextstrain-augur/augur/filter.py
  11. +79 −0 titer_model/implementation-nextstrain-augur/augur/mask.py
  12. +63 −0 titer_model/implementation-nextstrain-augur/augur/parse.py
  13. +994 −0 titer_model/implementation-nextstrain-augur/augur/titer_model.py
  14. +47 −0 titer_model/implementation-nextstrain-augur/augur/titers.py
  15. +121 −0 titer_model/implementation-nextstrain-augur/augur/traits.py
  16. +311 −0 titer_model/implementation-nextstrain-augur/augur/translate.py
  17. +213 −0 titer_model/implementation-nextstrain-augur/augur/tree.py
  18. +202 −0 titer_model/implementation-nextstrain-augur/augur/treetime_wrapper.py
  19. +332 −0 titer_model/implementation-nextstrain-augur/augur/utils.py
  20. 0 titer_model/implementation-nextstrain-augur/base/__init__.py
  21. BIN titer_model/implementation-nextstrain-augur/base/__init__.pyc
  22. +149 −0 titer_model/implementation-nextstrain-augur/base/auspice_export.py
  23. BIN titer_model/implementation-nextstrain-augur/base/auspice_export.pyc
  24. +120 −0 titer_model/implementation-nextstrain-augur/base/colorLogging.py
  25. +123 −0 titer_model/implementation-nextstrain-augur/base/config.py
  26. BIN titer_model/implementation-nextstrain-augur/base/config.pyc
  27. +53 −0 titer_model/implementation-nextstrain-augur/base/fetch_outgroups.py
  28. +710 −0 titer_model/implementation-nextstrain-augur/base/fitness_model.py
  29. BIN titer_model/implementation-nextstrain-augur/base/fitness_model.pyc
  30. +375 −0 titer_model/implementation-nextstrain-augur/base/fitness_predictors.py
  31. BIN titer_model/implementation-nextstrain-augur/base/fitness_predictors.pyc
  32. +723 −0 titer_model/implementation-nextstrain-augur/base/frequencies.py
  33. BIN titer_model/implementation-nextstrain-augur/base/frequencies.pyc
  34. +184 −0 titer_model/implementation-nextstrain-augur/base/io_util.py
  35. BIN titer_model/implementation-nextstrain-augur/base/io_util.pyc
  36. +27 −0 titer_model/implementation-nextstrain-augur/base/logger.py
  37. BIN titer_model/implementation-nextstrain-augur/base/logger.pyc
  38. +114 −0 titer_model/implementation-nextstrain-augur/base/prediction.py
  39. +226 −0 titer_model/implementation-nextstrain-augur/base/prepare.py
  40. BIN titer_model/implementation-nextstrain-augur/base/prepare.pyc
  41. +765 −0 titer_model/implementation-nextstrain-augur/base/process.py
  42. BIN titer_model/implementation-nextstrain-augur/base/process.pyc
  43. +38 −0 titer_model/implementation-nextstrain-augur/base/seq_util.py
  44. BIN titer_model/implementation-nextstrain-augur/base/seq_util.pyc
  45. +368 −0 titer_model/implementation-nextstrain-augur/base/sequences_prepare.py
  46. BIN titer_model/implementation-nextstrain-augur/base/sequences_prepare.pyc
  47. +375 −0 titer_model/implementation-nextstrain-augur/base/sequences_process.py
  48. BIN titer_model/implementation-nextstrain-augur/base/sequences_process.pyc
  49. +998 −0 titer_model/implementation-nextstrain-augur/base/titer_model.py
  50. BIN titer_model/implementation-nextstrain-augur/base/titer_model.pyc
  51. +628 −0 titer_model/implementation-nextstrain-augur/base/tree.py
  52. BIN titer_model/implementation-nextstrain-augur/base/tree.pyc
  53. +168 −0 titer_model/implementation-nextstrain-augur/base/utils.py
  54. BIN titer_model/implementation-nextstrain-augur/base/utils.pyc
  55. +161 −0 titer_model/implementation-nextstrain-augur/bin/augur
  56. +8 −0 titer_model/implementation-nextstrain-augur/builds/README.md
  57. 0 titer_model/implementation-nextstrain-augur/builds/__init__.py
  58. BIN titer_model/implementation-nextstrain-augur/builds/__init__.pyc
  59. +41 −0 titer_model/implementation-nextstrain-augur/builds/dengue/README.md
  60. +13 −0 titer_model/implementation-nextstrain-augur/builds/dengue/colors.tsv
  61. +161 −0 titer_model/implementation-nextstrain-augur/builds/dengue/dengue.prepare.py
  62. +157 −0 titer_model/implementation-nextstrain-augur/builds/dengue/dengue.process.py
  63. +24 −0 titer_model/implementation-nextstrain-augur/builds/dengue/dengue.process.wrapper.py
  64. +87 −0 titer_model/implementation-nextstrain-augur/builds/dengue/dengue_subsampling.py
  65. BIN titer_model/implementation-nextstrain-augur/builds/dengue/dengue_subsampling.pyc
  66. +179 −0 titer_model/implementation-nextstrain-augur/builds/dengue/dengue_titers.py
  67. BIN titer_model/implementation-nextstrain-augur/builds/dengue/dengue_titers.pyc
  68. +381 −0 titer_model/implementation-nextstrain-augur/builds/dengue/metadata/dengue_denv1_outgroup.gb
  69. +388 −0 titer_model/implementation-nextstrain-augur/builds/dengue/metadata/dengue_denv2_outgroup.gb
  70. +367 −0 titer_model/implementation-nextstrain-augur/builds/dengue/metadata/dengue_denv3_outgroup.gb
  71. +360 −0 titer_model/implementation-nextstrain-augur/builds/dengue/metadata/dengue_denv4_outgroup.gb
  72. +36,978 −0 titer_model/implementation-nextstrain-augur/builds/dengue/prepared/dengue_config.json
  73. +3 −0 titer_model/implementation-nextstrain-augur/pytest.ini
  74. +9 −0 titer_model/implementation-nextstrain-augur/requirements.txt
  75. +20 −0 titer_model/implementation-nextstrain-augur/setup.py
  76. +3 −0 titer_model/implementation-nextstrain-augur/source-data/README.md
  77. +2,040 −0 titer_model/implementation-nextstrain-augur/source-data/geo_lat_long.tsv
  78. 0 titer_model/implementation-nextstrain-augur/tests/__init__.py
  79. +1 −0 titer_model/implementation-nextstrain-augur/tests/fitness_model/H3N2_tree.newick
  80. +7,880 −0 titer_model/implementation-nextstrain-augur/tests/json_tree_to_nexus/flu_h3n2_ha_3y_frequencies.json
  81. +101 −0 titer_model/implementation-nextstrain-augur/tests/json_tree_to_nexus/flu_h3n2_ha_3y_tree.json
  82. +17 −0 titer_model/implementation-nextstrain-augur/tests/json_tree_to_nexus/flu_h3n2_ha_3y_tree.nex
  83. +236 −0 titer_model/implementation-nextstrain-augur/tests/test_fitness_model.py
  84. +189 −0 titer_model/implementation-nextstrain-augur/tests/test_fitness_predictors.py
  85. +12 −0 titer_model/implementation-nextstrain-augur/tests/titer_model/h3n2_titers_subset.tsv
  86. +6 −0 titer_model/implementation-nextstrain-augur/tox.ini
  87. +16 −0 titer_model/implementation-nextstrain-augur/treetime_augur/.travis.yml
  88. +1 −0 titer_model/implementation-nextstrain-augur/treetime_augur/.travis_test.sh
  89. +21 −0 titer_model/implementation-nextstrain-augur/treetime_augur/LICENSE
  90. +327 −0 titer_model/implementation-nextstrain-augur/treetime_augur/README.md
  91. +12 −0 titer_model/implementation-nextstrain-augur/treetime_augur/__init__.py
  92. BIN titer_model/implementation-nextstrain-augur/treetime_augur/__init__.pyc
  93. +123 −0 titer_model/implementation-nextstrain-augur/treetime_augur/ancestral_reconstruction.py
  94. +20 −0 titer_model/implementation-nextstrain-augur/treetime_augur/data/H3N2_NA_allyears_NA.20.metadata.csv
  95. +1 −0 titer_model/implementation-nextstrain-augur/treetime_augur/data/H3N2_NA_allyears_NA.20.nwk
  96. +199 −0 titer_model/implementation-nextstrain-augur/treetime_augur/data/H3N2_NA_allyears_NA.200.metadata.csv
  97. +1 −0 titer_model/implementation-nextstrain-augur/treetime_augur/data/H3N2_NA_allyears_NA.200.nwk
  98. +1 −0 titer_model/implementation-nextstrain-augur/treetime_augur/data/H3N2_NA_allyears_NA.500.nwk
  99. +19 −0 titer_model/implementation-nextstrain-augur/treetime_augur/data/Zika_country_weights.csv
  100. +87 −0 titer_model/implementation-nextstrain-augur/treetime_augur/data/Zika_metadata.csv
  101. +1 −0 titer_model/implementation-nextstrain-augur/treetime_augur/data/Zika_tree.newick
  102. 0 titer_model/implementation-nextstrain-augur/treetime_augur/data/__init__.py
  103. +363 −0 titer_model/implementation-nextstrain-augur/treetime_augur/data/ebola.metadata.csv
  104. +1 −0 titer_model/implementation-nextstrain-augur/treetime_augur/data/ebola.nwk
  105. +230 −0 titer_model/implementation-nextstrain-augur/treetime_augur/doc/Makefile
  106. BIN titer_model/implementation-nextstrain-augur/treetime_augur/doc/flu_200.png
  107. +36 −0 titer_model/implementation-nextstrain-augur/treetime_augur/doc/source/clock_tree.rst
  108. +319 −0 titer_model/implementation-nextstrain-augur/treetime_augur/doc/source/conf.py
  109. +41 −0 titer_model/implementation-nextstrain-augur/treetime_augur/doc/source/gtr.rst
  110. +43 −0 titer_model/implementation-nextstrain-augur/treetime_augur/doc/source/index.rst
  111. +82 −0 titer_model/implementation-nextstrain-augur/treetime_augur/doc/source/treeanc.rst
  112. +36 −0 titer_model/implementation-nextstrain-augur/treetime_augur/doc/source/treetime.rst
  113. +48 −0 titer_model/implementation-nextstrain-augur/treetime_augur/examples/ancestral_inference.py
  114. +82 −0 titer_model/implementation-nextstrain-augur/treetime_augur/examples/basic_clock_tree_diagnostics.py
  115. +66 −0 titer_model/implementation-nextstrain-augur/treetime_augur/examples/basic_tree_time_diagnostics.py
  116. +83 −0 titer_model/implementation-nextstrain-augur/treetime_augur/examples/ebola.py
  117. +68 −0 titer_model/implementation-nextstrain-augur/treetime_augur/examples/relaxed_clock.py
  118. +84 −0 titer_model/implementation-nextstrain-augur/treetime_augur/examples/rerooting_and_timetrees.py
  119. +58 −0 titer_model/implementation-nextstrain-augur/treetime_augur/examples/rerooting_with_multiplicity.py
  120. +213 −0 titer_model/implementation-nextstrain-augur/treetime_augur/homoplasy_scanner.py
  121. +143 −0 titer_model/implementation-nextstrain-augur/treetime_augur/mugration.py
  122. +5 −0 titer_model/implementation-nextstrain-augur/treetime_augur/requirements.txt
  123. +23 −0 titer_model/implementation-nextstrain-augur/treetime_augur/setup.py
  124. +113 −0 titer_model/implementation-nextstrain-augur/treetime_augur/temporal_signal.py
  125. +13 −0 titer_model/implementation-nextstrain-augur/treetime_augur/test/run_tests.py
  126. +233 −0 titer_model/implementation-nextstrain-augur/treetime_augur/test/test_treetime.py
  127. +195 −0 titer_model/implementation-nextstrain-augur/treetime_augur/timetree_inference.py
  128. +6 −0 titer_model/implementation-nextstrain-augur/treetime_augur/treetime/__init__.py
  129. BIN titer_model/implementation-nextstrain-augur/treetime_augur/treetime/__init__.pyc
  130. +110 −0 titer_model/implementation-nextstrain-augur/treetime_augur/treetime/aa_models.py
  131. BIN titer_model/implementation-nextstrain-augur/treetime_augur/treetime/aa_models.pyc
  132. +110 −0 titer_model/implementation-nextstrain-augur/treetime_augur/treetime/branch_len_interpolator.py
  133. BIN titer_model/implementation-nextstrain-augur/treetime_augur/treetime/branch_len_interpolator.pyc
  134. +643 −0 titer_model/implementation-nextstrain-augur/treetime_augur/treetime/clock_tree.py
  135. BIN titer_model/implementation-nextstrain-augur/treetime_augur/treetime/clock_tree.pyc
  136. +25 −0 titer_model/implementation-nextstrain-augur/treetime_augur/treetime/config.py
  137. BIN titer_model/implementation-nextstrain-augur/treetime_augur/treetime/config.pyc
  138. +396 −0 titer_model/implementation-nextstrain-augur/treetime_augur/treetime/distribution.py
  139. BIN titer_model/implementation-nextstrain-augur/treetime_augur/treetime/distribution.pyc
  140. +800 −0 titer_model/implementation-nextstrain-augur/treetime_augur/treetime/gtr.py
  141. BIN titer_model/implementation-nextstrain-augur/treetime_augur/treetime/gtr.pyc
  142. BIN titer_model/implementation-nextstrain-augur/treetime_augur/treetime/io.pyc
  143. +274 −0 titer_model/implementation-nextstrain-augur/treetime_augur/treetime/merger_models.py
  144. BIN titer_model/implementation-nextstrain-augur/treetime_augur/treetime/merger_models.pyc
  145. +280 −0 titer_model/implementation-nextstrain-augur/treetime_augur/treetime/node_interpolator.py
  146. BIN titer_model/implementation-nextstrain-augur/treetime_augur/treetime/node_interpolator.pyc
  147. +238 −0 titer_model/implementation-nextstrain-augur/treetime_augur/treetime/nuc_models.py
  148. BIN titer_model/implementation-nextstrain-augur/treetime_augur/treetime/nuc_models.pyc
  149. +229 −0 titer_model/implementation-nextstrain-augur/treetime_augur/treetime/seq_utils.py
  150. BIN titer_model/implementation-nextstrain-augur/treetime_augur/treetime/seq_utils.pyc
  151. +1,625 −0 titer_model/implementation-nextstrain-augur/treetime_augur/treetime/treeanc.py
  152. BIN titer_model/implementation-nextstrain-augur/treetime_augur/treetime/treeanc.pyc
  153. +1,013 −0 titer_model/implementation-nextstrain-augur/treetime_augur/treetime/treetime.py
  154. BIN titer_model/implementation-nextstrain-augur/treetime_augur/treetime/treetime.pyc
  155. +265 −0 titer_model/implementation-nextstrain-augur/treetime_augur/treetime/utils.py
  156. BIN titer_model/implementation-nextstrain-augur/treetime_augur/treetime/utils.pyc
  157. +1 −0 titer_model/implementation-nextstrain-augur/treetime_augur/treetime/version.py
  158. BIN titer_model/implementation-nextstrain-augur/treetime_augur/treetime/version.pyc
View
@@ -11,7 +11,6 @@ We find that antigenic fitness mediates fluctuations in DENV clade frequencies,
These results provide a more nuanced understanding of dengue antigenic evolution, with important ramifications for vaccine design and epidemic preparedness.
# Analysis outline
1 - [Run the titer model via augur](https://github.com/blab/dengue-antigenic-dynamics/tree/master/augur) (repackaged portion of the [Nextstrain](www.nextstrain.org/dengue) pipeline) to build a viral phylogeny, assign antigenic change to specific branches, and infer clade frequencies.
1 - [Run the titer model via augur](https://github.com/blab/dengue-antigenic-dynamics/tree/master/titer_model/) (repackaged portion of the [Nextstrain](www.nextstrain.org/dengue) pipeline) to build a viral phylogeny, assign antigenic change to specific branches, and infer clade frequencies.
2 - [Run the fitness model](https://github.com/blab/dengue-antigenic-dynamics/tree/master/fitness_model) to quantify population immunity over time, predict clade frequencies, and assess performance.
3 - [Use the visualization notebooks](https://github.com/blab/dengue-antigenic-dynamics/tree/master/figures) to explore results and recreate all the figures from the paper.
View
@@ -0,0 +1,45 @@
# About the titer model
Antigenic distances between pairs of dengue viruses are experimentally measured via neutralization titers. These titer values are prone to noise, and there is a limited amount of available titer data. If the antigenic heterogeneity observed in the raw data is truly the result of an underlying evolutionary process, we expect that changes in antigenic phenotype correspond to underlying changes in viral genotype.
The titer model maps changes in antigenic phenotype (titer drops) to specific branches in the viral phylogeny as described below. This allows us to directly quantify the extent to which observed phenotypic variation is explained by an underlying genetic evolutionary process. We can also use variations of the model formulation to directly compare competing hypotheses about the nature of dengue antigenic evolution.
The titer model pipeline has three main steps.
First, we build a phylogeny of dengue virus sequences to establish the genetic relationships between viruses.
Next, we infer how much antigenic change has occurred along each branch of the phylogeny by mapping titer changes to individual branches.
This assigns each branch $b$ an antigenic distance $d_b$.
With this in hand, we estimate the antigenic distance between all pairs of viruses by tracing the path between them in the phylogeny, summing branch-specific distances $d_b$ as we go.
To learn these values of $d_b$, we first split our dataset into training (random 90% of measurements) and test data (the remaining 10% of values).
We take the training data and fit $d_b$ for each branch in the tree, subject to regularization.
Parsimoniously, we expect that antigenic change is more likely to occur through larger changes on a few branches than through small changes on many branches; correspondingly, our prior expectation of values of $d_b$ is exponentially distributed such that most values of $d_b = 0$.
This is analogous to lasso regression to identify a few parameters with positive weights and set other parameters to 0.
Additionally, some viruses have greater binding avidity, and some sera are more potent than others; these 'row' and 'column' effects, respectively, are normally distributed and are taken into account when estimating titers.
The model uses convex optimization to learn the values of $d_b$ that minimize the sum of squared errors (SSE) between observed and predicted titers in the training data.
# Model variations
![Titer model variation](https://raw.githubusercontent.com/blab/dengue-antigenic-dynamics/master/figures/png/titer_model_performance.png)
**A** The 'interserotype model' only allows branches that lie between serotypes to contribute to antigenic evolution.
All other branches are assigned $d_b = 0$.
**B** The 'full tree model' allows any branch in the phylogeny to contribute to antigenic evolution ($d_b >= 0$).
**C,D** Predictive performance of each model on the test dataset (aggregated from 10-fold cross-validation).
# Running the titer model
This model was originally published in (Neher et al., PNAS 2016) and implemented as part of [Nextstrain](https://github.com/nextstrain/augur). The relevant portions of the repository have been reproduced here under `./implementation-nextstrain-augur`. Documentation, including install instructions, for the full augur pipeline can be found [here](https://github.com/nextstrain/augur/tree/6d9f7088d8792196e5021c67b876d9de1d2a13dd)
### 1 - Prepare input files
`dengue.prepare.py` handles subsampling and other basic dataset config. You can edit the function `make_config` here with any desired changes.
`cd dengue-antigenic-dynamics/titer_model/implementation-nextstrain-augur/builds/dengue/`
`python dengue.prepare.py`
`mv ./prepared/dengue_all.json ./prepared/dengue_config.json`
**N.B.: For this dataset, step 1 has been run for you; see `./dengue_config.json`**
### 2 - Run the titer pipeline (& estimate clade frequencies)
`dengue.process.py` handles the actual analysis; parameter settings, etc. can be changed in the `make_config` function here.
`python dengue.process.py --titer_model full_tree` for the "full tree" model.
`python dengue.process.py --titer_model interserotype` for the "interserotype" model.
**N.B.: For this dataset, step 2 has been run for you; see `./full-tree-model-output/` and `./interserotype-model-output/`**
### 3 - Check out your results
Results are output in JSON format, found in `./processed/`. You can parse and examine results using the notebooks found [here](../figures/).
Oops, something went wrong.

0 comments on commit 2cdfa77

Please sign in to comment.