finalize 2.0.5

SlavovLab · Mar 1, 2023 · bd82db9 · bd82db9
1 parent eeca097
commit bd82db9
Show file tree

Hide file tree

Showing 41 changed files with 1,217 additions and 118 deletions.
diff --git a/.RData b/.RData
diff --git a/do-ms_cmd.R b/do-ms_cmd.R
@@ -485,4 +485,4 @@ generate_report(input, f_data, raw_files, config[['output']], progress_bar=FALSE
 
 # prnt(paste0('Report written to: ', config[['output']]))
 
-prnt('Done!')
+prnt('Done!')
diff --git a/docs/_config.yml b/docs/_config.yml
@@ -26,17 +26,15 @@ search_enabled: true
 
 # Aux links for the upper right navigation
 aux_links:
-  "DO-MS Preprint":
-    - "https://www.biorxiv.org/content/10.1101/512152v1"
-  "JPR Article":
-    - "https://pubs.acs.org/doi/10.1021/acs.jproteome.9b00039"
+  "BioArxiv Preprint":
+    - "https://www.biorxiv.org/content/10.1101/2023.02.02.526809v1"
   "GitHub Repository":
     - "https://github.com/SlavovLab/DO-MS"
   "Slavov Lab":
     - "http://slavovlab.net"
 
 github_link: https://github.com/SlavovLab/DO-MS
-preprint_link: https://www.biorxiv.org/content/10.1101/512152v1
+preprint_link: https://www.biorxiv.org/content/10.1101/2023.02.02.526809v1
 jpr_link: https://pubs.acs.org/doi/10.1021/acs.jproteome.9b00039
 
 # Color scheme currently only supports "dark" or nil (default)

diff --git a/docs/assets/images/dia_input.png b/docs/assets/images/dia_input.png
diff --git a/docs/assets/images/do-ms-dia-example.png b/docs/assets/images/do-ms-dia-example.png
diff --git a/docs/assets/images/do-ms-dia-foldername.png b/docs/assets/images/do-ms-dia-foldername.png
diff --git a/docs/assets/images/do-ms-dia-generate-report.png b/docs/assets/images/do-ms-dia-generate-report.png
diff --git a/docs/assets/images/do-ms-dia-import.png b/docs/assets/images/do-ms-dia-import.png
diff --git a/docs/assets/images/do-ms-dia-info.png b/docs/assets/images/do-ms-dia-info.png
diff --git a/docs/assets/images/do-ms-dia-load.png b/docs/assets/images/do-ms-dia-load.png
diff --git a/docs/assets/images/do-ms-dia-overview.png b/docs/assets/images/do-ms-dia-overview.png
diff --git a/docs/assets/images/do-ms-dia-pathname.png b/docs/assets/images/do-ms-dia-pathname.png
diff --git a/docs/assets/images/do-ms-dia-rename.png b/docs/assets/images/do-ms-dia-rename.png
diff --git a/docs/assets/images/do-ms-dia-results.png b/docs/assets/images/do-ms-dia-results.png
diff --git a/docs/assets/images/do-ms-dia_config.png b/docs/assets/images/do-ms-dia_config.png
diff --git a/docs/assets/images/do-ms-dia_mode.png b/docs/assets/images/do-ms-dia_mode.png
diff --git a/docs/assets/images/do-ms-dia_title.png b/docs/assets/images/do-ms-dia_title.png
diff --git a/docs/assets/images/do-ms-dia_title_v2.png b/docs/assets/images/do-ms-dia_title_v2.png
diff --git a/docs/assets/images/preprocessing_input.png b/docs/assets/images/preprocessing_input.png
diff --git a/docs/assets/images/preprocessing_output.png b/docs/assets/images/preprocessing_output.png
diff --git a/docs/docs/DO-MS_examples.md b/docs/docs/DO-MS_examples.md
@@ -1,7 +1,7 @@
 ---
 layout: default
 title: DO-MS Examples
-nav_order: 3
+nav_order: 5
 permalink: docs/DO-MS_examples
 parent: Getting Started
 ---
@@ -10,6 +10,13 @@ parent: Getting Started
 
 Bellow are links to example DO-MS Reports
 
+## DIA-NN DIA Reports
+1. [MS2 number optimization](https://drive.google.com/uc?id=1mNrJsk6uaI3ljtQEadr0XwR5mlyO4xZg&export=download)
+2. [Different MS2 strategies](https://drive.google.com/uc?id=1eOJXC2Zb0lmVwbsaj7Zlf6wm3SPGrbWl&export=download)
+3. [Gradient length optimization](https://drive.google.com/uc?id=1dQdo3kR-WxWT8zEeOxztxXmJE1rSMsEM&export=download)
+4. [MS1 sampling optimization](https://drive.google.com/uc?id=1YcT3al9ICu36_ornSsLZgHj1eTBBPOxb&export=download)
+5. [Single cell quality control](https://drive.google.com/uc?id=1v62qJ8JVZ_TSkUa3K0i6nWoJDbW9EQHO&export=download)
 
+## MaxQuant DDA Reports
 1. [Isobaric carrier optimization](https://scope2.slavovlab.net/mass-spec/Isobaric-carrier-optimization#do-ms-reports)
 2. [Technical improvements of SCoPE2](https://scope2.slavovlab.net/mass-spec/Increased-accuracy-and-throughput)
diff --git a/docs/docs/getting_started.md b/docs/docs/getting_started.md
@@ -8,4 +8,19 @@ has_children: true
 
 # Getting Started
 
-DO-MS can be run either from the command-line or as interactive application. Follow the links below to get started using the implementation of your choice. For more details on the data display, read the [DO-MS article](https://pubs.acs.org/doi/10.1021/acs.jproteome.9b00039).  
+DO-MS can be run either from the command-line or as interactive application. Follow the links below to get started using the implementation of your choice. For more details on the data display, read the [DO-MS 2.0 article](https://www.biorxiv.org/content/10.1101/2023.02.02.526809v1). 
+
+Before starting DO-MS the first time, the input data type has to be selected. DO-MS can work with both DDA results coming from MaxQuant as well as with DIA results coming from DIA-NN.
+The mode can be set in the config.yaml file. Open the file in R-Studio or your editor of choice.
+![]({{site.baseurl}}/assets/images/do-ms-dia_mode.png){: width="70%" .center-image}
+
+If You wish to analyze MaxQuant DDA results, change the parameter to `max_quant`, otherwise leave it as `dia-nn`. This setting needs to be changed before SO-MS is started or the R environment is initialized. It is also possible to keep both versionas in two separate folders simultanously.
+![]({{site.baseurl}}/assets/images/do-ms-dia_config.png){: width="70%" .center-image}
+
+# Generating DO-MS Reports
+Please read our detailed getting started guides:
+
+* [DIA Preprocessing]({{site.baseurl}}/docs/getting-started-preprocessing)
+* [DIA Reports using the app]({{site.baseurl}}/docs/getting-started-dia-app)
+* [DDA Reports using the app]({{site.baseurl}}/docs/getting-started-application)
+* [DDA Reports using the command line]({{site.baseurl}}/docs/getting-started-application)
diff --git a/docs/docs/getting_started_app.md b/docs/docs/getting_started_app.md
@@ -1,12 +1,12 @@
 ---
 layout: default
-title: Interactive Application
-nav_order: 1
-permalink: docs/getting-started-application
+title: DDA Reports in the App
+nav_order: 3
+permalink: docs/getting-started-dda-app
 parent: Getting Started
 ---
 
-# Getting Started -- Interactive DO-MS Application
+# Getting Started -- DDA Reports in the App
 
 DO-MS is an application to visualize mass-spec data both in an interactive application and static reports generated via. the command-line. In this document we'll walk you through analyzing an example dataset in the interactive application.
 
@@ -31,13 +31,8 @@ The only constraint for data in DO-MS is that it must be from MaxQuant version >
 
 ## Installation
 
-Download the application via. a zip or tar archive from the [GitHub release page](https://github.com/SlavovLab/DO-MS/releases). Unzip the archive, and then open the `DO-MS.Rproj` to load the project into RStudio
-
-![]({{site.baseurl}}/assets/images/do-ms-proj.png){: width="60%" .center-image}
-
-To start the app open `server.R` in RStudio and on the top right corner of the editor click on the "Run App" button. To run the application in your browser (preferred option) rather than in RStudio, click the dropdown menu and select "Run External".
-
-![]({{site.baseurl}}/assets/images/do-ms-run.png){: width="85%" .center-image}
+Please make sure that you installed DO-MS as descibed in the [installation]({{site.baseurl}}/docs/installation) section. 
+<span style="color:red">Note that DO-MS has to be configured for MaxQuant data. [Find out More]({{site.baseurl}}/docs/getting-started)</span> 
 
 ## Data Import
 

diff --git a/docs/docs/getting_started_cmd.md b/docs/docs/getting_started_cmd.md
@@ -1,7 +1,7 @@
 ---
 layout: default
-title: Command Line
-nav_order: 2
+title: DO-MS Command Line
+nav_order: 4
 permalink: docs/getting-started-command-line
 parent: Getting Started
 ---

diff --git a/docs/docs/getting_started_dia_app.md b/docs/docs/getting_started_dia_app.md
@@ -0,0 +1,100 @@
+---
+layout: default
+title: DIA Reports in the App
+nav_order: 2
+permalink: docs/getting-started-dia-app
+parent: Getting Started
+---
+
+# Getting Started -- DIA Reports in the App
+
+DO-MS is an application to visualize mass-spec data both in an interactive application and static reports generated via. the command-line. In this document we'll walk you through analyzing an example dataset in the interactive application.
+
+**Table of Contents**
+
+1. [Example Data](#example-data)
+2. [Installation](#installation)
+3. [Data Import](#data-import)
+    1. [Adding Folders](#adding-folders)
+    2. [Renaming Experiments](#renaming-experiments)
+4. [Interacting with Modules](#interacting-with-modules)
+5. [Generate Report](#generate-report)
+
+
+
+## Example Data
+
+We have provided an example data set online, which contains parts of the MS2 number optimization in the paper. This data can also be obtained by follwoing the guide on the preprocessing or can be downloaded as a zip file here: [https://drive.google.com/file/d/1BzWVKghIThtgYItgGy9vt6M214mHB74q/view?usp=share_link](https://drive.google.com/file/d/1BzWVKghIThtgYItgGy9vt6M214mHB74q/view?usp=share_link).
+
+
+![]({{site.baseurl}}/assets/images/do-ms-dia-example.png){: width="70%" .center-image}
+
+## Installation
+
+Please make sure that you installed DO-MS as descibed in the [installation]({{site.baseurl}}/docs/installation) section. 
+
+
+## Data Import
+<span style="color:red">Make sure DO-MS has been configured for DIA-NN data. [Find out More]({{site.baseurl}}/docs/getting-started)</span> 
+
+![]({{site.baseurl}}/assets/images/do-ms-dia-overview.png){: width="90%" .center-image}
+
+### Adding Folders
+
+DO-MS is designed to load _folders_ of analyses rather than individual files. To allow quick access to your analyses, as well as to allow analyzing multiple searches simultaneously, DO-MS provides a searchable "folder table" for all of your analyses. We will start loading the folder with the example output.
+
+Start by clicking the "Add Folder" button at the top of the table
+
+![]({{site.baseurl}}/assets/images/do-ms-dia-import.png){: width="70%" .center-image}
+
+Then add the path of your folder into the textbox, as shown:
+
+![]({{site.baseurl}}/assets/images/do-ms-dia-foldername.png){: width="70%" .center-image}
+
+A folder path is the folder's absolute location on your machine. On Windows, you can get the folder path by navigating to it in Explorer, clicking on the top file path bar, and copying the resulting text with Ctrl+C. On Mac/OSX, you can get the folder path by right clicking on the folder while presing the 'option' key.
+
+![]({{site.baseurl}}/assets/images/do-ms-dia-pathname.png){: width="70%" .center-image}
+
+Note that in the example above we checked "Add Single Folder" to add just the folder path we pasted in. If for example, you have a folder that contains many MaxQuant searches, you can select "Add Child Folders" to add all subfolders of the path specified, or "Add Recursively" to add _all_ folders that are below the path specified.
+
+Click "Confirm" and now you should see your folder added to the folder table. When you have multiple folders loaded into the table you can select more than one.
+
+![]({{site.baseurl}}/assets/images/do-ms-dia-load.png){: width="70%" .center-image}
+
+Finally, load the files from the selected folders by scrolling down and clicking on the big "Load Data" button
+
+### Renaming Experiments
+
+An important aspect of data visualization is easy-to-read labels for your experiments. DO-MS provides an accessible interface for renaming your raw file names so that generated figures will be easier to interpret.
+
+Once your data is loaded, scroll down to the "Renaming Experiments" section. Here you will find a table of your loaded raw files and their associated "labels" that will be used when plotting. Double click a label and type enter the new labe:
+
+![]({{site.baseurl}}/assets/images/do-ms-dia-rename.png){: width="100%" .center-image}
+
+Please rename all experiments as following:
+
+```
+wGW027 => 4 MS2
+wGW028 => 8 MS2
+wGW029 => 10 MS2
+```
+
+## Interacting with Modules
+
+Each module has a short description that can be accessed by hovering over the question mark icon next to the module title.
+
+![]({{site.baseurl}}/assets/images/do-ms-dia-info.png){: width="70%" .center-image}
+
+You can also download the module plot as a PNG or PDF, by clicking on the download buttons below each module plot. You can also download the underlying data used for the plot by clicking on the "Download Data" button. This tab-delimited file can be imported into many other visualization packages.
+
+## Generate Report
+
+Click on "Generate Report" in the sidebar to access the report generation page.
+
+![]({{site.baseurl}}/assets/images/do-ms-dia-generate-report.png){: width="70%" .center-image}
+
+Here you will find some options to customize your report. While we support PDF reports and PDF images, we strongly recommend that you generate your reports in HTML format with PNG image plots. Other configurations may result in graphical glitches or unwanted behavior. In addition we recommend that you check your `pandoc` installation ([more details here]({{site.baseurl}}/docs/known-issues#pandoc-not-found)) as any issues will prevent the report generation.
+
+Click the "Download Report" button to begin generating the report. This takes a while as all plots have to be remade. A progress bar at the bottom of the page informs you of the progress.
+
+All images in the report are embedded in the markup, so feel free to share this single file to your colleagues/collaborators and don't worry about having to include anything else.
diff --git a/docs/docs/getting_started_preprocessing.md b/docs/docs/getting_started_preprocessing.md
@@ -0,0 +1,130 @@
+---
+layout: default
+title: DIA Preprocesing
+nav_order: 1
+permalink: docs/getting-started-preprocessing
+parent: Getting Started
+---
+
+# Getting Started -- DIA Preprocesing
+
+DO-MS is an application to visualize mass-spec data both in an interactive application and static reports generated via. the command-line. In this document we'll walk you through analyzing an example dataset in the interactive application.
+
+**Table of Contents**
+
+1. [Example Data](#example-data)
+2. [Installation](#installation)
+3. [Processing Raw Data](#processing-raw-data)
+4. [Command Line Interface](#command-line-interface)
+
+## Example Data
+We have provided an example data set online, which contains parts of the MS2 number optimization in the paper. You can download a .zip bundle of it here: [https://drive.google.com/file/d/1bjFzKqTFLk7ECUJOTy8LNxsCOD0Xf96Q/view?usp=share_link](https://drive.google.com/file/d/1bjFzKqTFLk7ECUJOTy8LNxsCOD0Xf96Q/view?usp=share_link). The contents of the archive are the main dia-nn report and the corresponding raw files.
+
+Your folder should look like this:
+![]({{site.baseurl}}/assets/images/preprocessing_input.png){: width="70%" .center-image}
+
+## Installation
+Please make sure that you installed DO-MS as descibed in the [installation]({{site.baseurl}}/docs/installation) section. For using the preprocessing pipeline it is necessary to install the ThermoRawFileParser and the Dinosaur feature detection.
+
+## Processing Raw Data
+Open a terminal and enter the base folder of your DO-MS installation.
+Make sure that your DO-MS environment is set up and activate it.
+```
+conda activate doms
+```
+
+For processing, the piplline module located at `pipeline/processing.py` will be called with the following parameters.
+
+```bash
+python pipeline/processing.py /location/to/example/report_filtered.tsv
+```
+
+the following additional options will be included:
+```bash
+# Activate Mono if using Mac or Linux. Mono is required to run the Thermo Raw File Parser on Linux and OSX.
+-m
+# location of the ThermoRawFileParser executeable
+--raw-parser-location /location/to/ThermoRawFileParser1.4.2/ThermoRawFileParser.exe 
+# location of the Dinosaur .jar file
+--dinosaur-location /Users/georgwallmann/Library/CloudStorage/OneDrive-Personal/Studium/Northeastern/DO-MS-DIA/Dinosaur-1.2.0.free.jar 
+# location of the example raw data
+-r /location/to/example
+```
+
+The full command needs to be a single line and will look like: 
+```bash
+python pipeline/processing.py /location/to/example/report_filtered.tsv -m --raw-parser-location /location/to/ThermoRawFileParser1.4.2/ThermoRawFileParser.exe  --dinosaur-location /Users/georgwallmann/Library/CloudStorage/OneDrive-Personal/Studium/Northeastern/DO-MS-DIA/Dinosaur-1.2.0.free.jar  -r /location/to/example
+```
+
+After processing, the additional files should be part of your folder:
+![]({{site.baseurl}}/assets/images/preprocessing_output.png){: width="70%" .center-image}
+
+Temporary .mzML files can be deleted. 
+
+## Command Line Interface
+The documentation for the various command line options can be found by typing `python pipeline/processing.py -h`
+
+```bash
+usage: processing.py [-h] --raw-parser-location RAW_PARSER_LOCATION
+                     [--dinosaur-location DINOSAUR_LOCATION] [-m] [-d] [-v]
+                     [-t TEMPORARY_FOLDER] [-r RAW_FILE_LOCATION]
+                     [--no-feature-detection] [--no-fill-times] [--no-tic]
+                     [--no-sn] [--no-mzml-generation]
+                     [--mz-bin-size MZ_BIN_SIZE] [--rt-bin-size RT_BIN_SIZE]
+                     [--resolution RESOLUTION] [-p PROCESSES] [--isotopes-sn]
+                     report
+
+Command line tool for feature detection in shotgun MS experiments. Can be used
+together with DIA-NN to provide additional information on the peptide like
+features identified in the MS1 spectra.
+
+positional arguments:
+  report                Location of the report.tsv output from DIA-NN which
+                        should be used for analysis.
+
+options:
+  -h, --help            show this help message and exit
+  --raw-parser-location RAW_PARSER_LOCATION
+                        Path pointing to the ThermoRawFileParser executeable.
+  --dinosaur-location DINOSAUR_LOCATION
+                        Path pointing to the dinosaur jar executeable.
+  -m, --mono            Use mono for ThermoRawFileParser under Linux and OSX.
+  -d, --delete          Delete generated mzML and copied raw files after
+                        successfull feature generation.
+  -v, --verbose         Show verbose output.
+  -t TEMPORARY_FOLDER, --temporary-folder TEMPORARY_FOLDER
+                        Input Raw files will be temporarilly copied to this
+                        folder. Required for use with Google drive.
+  -r RAW_FILE_LOCATION, --raw-file-location RAW_FILE_LOCATION
+                        By default, raw files are loaded based on the
+                        File.Name column in the report.tsv. With this option,
+                        a different folder can be specified.
+  --no-feature-detection
+                        All steps are performed as usual but Dinosaur feature
+                        detection is skipped. No features.tsv file will be
+                        generated.
+  --no-fill-times       All steps are performed as usual but fill times are
+                        not extracted. No fill_times.tsv file will be
+                        generated.
+  --no-tic              All steps are performed as usual but binned TIC is not
+                        extracted. No tic.tsv file will be generated.
+  --no-sn               Signal to Noise ratio is not estimated for precursors
+  --no-mzml-generation  Raw files are not converted to .mzML. Nevertheless,
+                        mzML files are expected in their theoretical output
+                        location and loaded. Should be only be carefully used
+                        for repeated calulcations or debugging
+  --mz-bin-size MZ_BIN_SIZE
+                        Bin size over the mz dimension for TIC binning.
+  --rt-bin-size RT_BIN_SIZE
+                        Bin size over the RT dimension for TIC binning in
+                        minutes. If a bin size of 0 is provided, binning will
+                        not be applied and TIC is given per scan.
+  --resolution RESOLUTION
+                        Set the resolution used for estimating counts from S/N
+                        data
+  -p PROCESSES, --processes PROCESSES
+                        Number of Processes
+  --isotopes-sn         Use all isototopes from the same scan as the highest
+                        intensity datapoint for estimating the SN and copy
+                        number.
+```