Skip to content

oicr-gsi/gsi-wdl-tools

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

43 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

gsi-wdl-tools

A collection of tools for working with WDL.

Installation

gsi-wdl-tools requires Python 3.10

  1. Install pipenv
pip install --user pipenv
  1. Install dependencies
cd gsi-wdl-tools
PIPENV_VENV_IN_PROJECT=1 PIP_IGNORE_INSTALLED=1 pipenv install

# you can see that the project's venv is in the repo
pipenv --venv
  1. Run tests
pipenv run python3 -m pytest

Tools

generate-markdown-readme

Generates a readme for a WDL file that has been described with the following meta sections:

  1. workflow meta
workflow myWorkflow {

  input {
    String input1
    String input2
  }

...

  output {
    File output1
    File? output2
  }

  parameter_meta {
    input1: "What is input1?"
    input2: "What is input2?"
  }

  meta {
    author: "???"
    email: "???"
    description: "What does the workflow do?"
    dependencies: [
      {
        name: "package1/0.1",
        url: "https://url_to_software"
      },
      {
        name: "package2/0.1",
        url: "https://url_to_software"
      }
    ]
    output_meta: {
      output1: What is output1?.",
      output2: "What is output2?."
    }
  }
}
  1. task parameter meta
task myTask {

  input {
    String message
    String? fileName
  }

...

  parameter_meta {
    message: "What is message?"
    fileName: "What is fileName?"
  }
}

Usage

generate-markdown-readme --input-wdl-path [workflow.wdl]

Or with pipenv shell:

pipenv shell
python3 ./scripts/generate_markdown_readme.py --input-wdl-path [workflow.wdl]

Or without using pipenv run:

pipenv run python3 ./scripts/generate_markdown_readme.py --input-wdl-path [workflow.wdl]

dockstore_preprocess.py

Preprocesses a WDL to be used as a subworkflow, either for a UGE-based or dockstore-based wrapper workflow. Main function is converting task-level parameters to workflow-level parameters (pulling). Tested on all workflows in the WGS Pipeline, but might not catch edge case WDL formatting.

Run Arguments

Argument Required? Description
--input-wdl-path True Source WDL path
--tab-size False Number of spaces in a tab
--pull-json False Path to json specifing which task-level parameters to pull. Exclusive with --pull-all
--pull-all False Whether or not to pull all variables. Exclusive with --pull-json
--dockstore False Whether or not to preprocess the WDL for Dockstore. Prereq of --docker-image
--docker-image False Docker image name and tag (or sha digest)
--import-metas False Whether or not to pull parameter_meta section from subworkflows
--output-wdl-path False Custom output WDL path. Defaults to the source WDL path plus a file prefix

Usage

Common combinations:

dockstore_preprocess --docker-image "g3chen/wgspipeline:2.0" --input-wdl-path [workflow.wdl] --pull-all --dockstore --tab-size 4 --output-wdl-path [dockstore_workflow.wdl]
dockstore_preprocess --input-wdl-path [workflow.wdl] --pull-all

Or with pipenv shell:

pipenv shell
python3 dockstore_preprocess.py --input-wdl-path [workflow.wdl]

Or without using pipenv run:

pipenv run python3 ./dockstore_preprocess.py --input-wdl-path [workflow.wdl]

dockstore_preprocess_all.sh

When given a directory containing WDL files and arguments, calls dockstore_preprocess.py on all files using those args. "--input-wdl-path" is omitted.

Run Arguments

Index Argument Description
1 dockstore_preprocess.py Path to dockstore_preprocess.py
2 WDL dir Path to directory containing target WDL files
3+ preprocess args Arguments for dockstore_preprocess.py, minus --input-wdl-path

Usage

dockstore_preprocess_all.sh [dockstore_preprocess.py] [WDL dir] [preprocess args]

For example:

dockstore_preprocess_all.sh dockstore_preprocess.py /.../wgsPipeline/imports/ --pull-all --tab-size 4

About

No description or website provided.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •