A Python implementation of a WDL parser and language bindings.
-For Scala language bindings, use [Cromwell](http://github.com/broadinstitute/cromwell).
-
-<!---toc start-->
-
-* [PyWDL](#pywdl)
-* [Installation](#installation)
-* [Language Bindings](#language-bindings)
- * [Abstract syntax trees (ASTs)](#abstract-syntax-trees-asts)
- * [wdl.parser.Ast](#wdlparserast)
- * [wdl.parser.Terminal](#wdlparserterminal)
- * [wdl.parser.AstList](#wdlparserastlist)
- * [Working with expressions](#working-with-expressions)
-* [Command Line Usage](#command-line-usage)
-
-<!---toc end-->
-
-#Installation
-
-PyWDL works with Python 2 or Python 3. Install via `setup.py`:
-
-```
-$ python setup.py install
-```
-
-Or via pip:
-
-```
-$ pip install wdl
-```
-
-#Language Bindings
-
-The main `wdl` package provides an interface to turn WDL source code into native Python objects. This means that a `workflow {}` block in WDL would become a `Workflow` object in Python and a `task {}` block becomes a `Task` object.
-
-To parse WDL source code into a `WdlDocument` object, import the `wdl` package and load a WDL string with `wdl.loads("wdl code")` or WDL from a file-like object using `wdl.load(fp, resource_name)`.
-
-For example:
-
-```python
-import wdl
-import wdl.values
-
-wdl_code ="""
-task my_task {
- File file
- command {
- ./my_binary --input=${file} > results
- }
- output {
- File results = "results"
- }
-}
-
-workflow my_wf {
- call my_task
-}
-"""
-
-# Use the language bindings to parse WDL into Python objects
-Using the language bindings as shown above is the recommended way to use PyWDL. One can also directly access the parser to parse WDL source code into an abstract syntax tree using the `wdl.parser` package:
-
-```python
-import wdl.parser
-
-wdl_code ="""
-task my_task {
- File file
- command {
- ./my_binary --input=${file} > results
- }
- output {
- File results = "results"
- }
-}
-
-workflow my_wf {
- call my_task
-}
-"""
-
-# Parse source code into abstract syntax tree
-ast = wdl.parser.parse(wdl_code).ast()
-
-# Print out abstract syntax tree
-print(ast.dumps(indent=2))
-
-# Access the first task definition, print out its name
-An AST is the output of the parsing algorithm. It is a tree structure in which the root node is always a `Document` AST
-
-The best way to get started working with ASTs is to visualize them by using the `wdl parse` subcommand to see the AST as text. For example, consider the following WDL file
-
-example.wdl
-```
-task a {
- command {./foo_bin}
-}
-task b {
- command {./bar_bin}
-}
-task c {
- command {./baz_bin}
-}
-workflow w {}
-```
-
-Then, use the command line to parse and output the AST:
-
-```
-$ wdl parse example.wdl
-(Document:
- imports=[],
- definitions=[
- (Task:
- name=<string:1:6 identifier "YQ==">,
- declarations=[],
- sections=[
- (RawCommand:
- parts=[
- <string:2:12 cmd_part "Li9mb29fYmlu">
- ]
- )
- ]
- ),
- (Task:
- name=<string:4:6 identifier "Yg==">,
- declarations=[],
- sections=[
- (RawCommand:
- parts=[
- <string:5:12 cmd_part "Li9iYXJfYmlu">
- ]
- )
- ]
- ),
- (Task:
- name=<string:7:6 identifier "Yw==">,
- declarations=[],
- sections=[
- (RawCommand:
- parts=[
- <string:8:12 cmd_part "Li9iYXpfYmlu">
- ]
- )
- ]
- ),
- (Workflow:
- name=<string:10:10 identifier "dw==">,
- body=[]
- )
- ]
-)
-```
-
-Programmatically, if one wanted to traverse this AST to pull out data:
-
-```python
-import wdl.parser
-import wdl
-
-withopen('example.wdl') as fp:
- ast = wdl.parser.parse(fp.read()).ast()
-
-task_a = ast.attr('definitions')[0]
-task_b = ast.attr('definitions')[1]
-task_c = ast.attr('definitions')[2]
-
-for ast in task_a.attr('sections'):
-if ast.name =='RawCommand':
- task_a_command = ast
-
-for ast in task_a_command.attr('parts'):
-ifisinstance(ast, wdl.parser.Terminal):
-print('command string: '+ ast.source_string)
-else:
-print('command parameter: '+ ast.dumps())
-```
-
-###wdl.parser.Ast
-
-The `Ast` class is a syntax tree with a name and children nodes.
-
-Attributes:
-
-*`name` is a string that refers to the type of AST, (e.g. `Workflow`, `Task`, `Document`, `RawCommand`)
-*`attributes` is a dictionary where the keys are the name of the attribute and the values can be one of three types: `Ast`, `AstList`, `Terminal`.
-
-Methods:
-
-*`def attr(self, name)`. `ast.attr('name')` is the same as `ast.attributes['name']`.
-*`def dumps(self, indent=None, b64_source=True)` - returns a String representation of this AstList. the `indent` parameter takes an integer for the indent level. Omitting this value will cause there to be no new-lines in the resulting string. `b64_source` will be passed to recursive invocations of `dumps`.
-
-###wdl.parser.Terminal
-
-The `wdl.parser.Terminal` object represents a literal piece of the original source code. This always shows up as leaf nodes on `Ast` objects
-
-Attributes:
-
-*`source_string` - String segment from the source code.
-*`line` - Line number where `source_string` was in source code.
-*`col` - Column number where `source_string` was in source code.
-*`resource` - Name of the location for the source code. Usually a file system path or perhaps URI.
-*`id` - Numeric identifier, unique to the top level `Ast`. Used mostly internally.
-*`str` - String identifier of this terminal. Used mostly internally.
-
-Methods:
-
-*`def dumps(self, b64_source=True, **kwargs)` - return a String representation of this terminal. `b64_source` means that the source code will be base64 encoded because sometimes the source contains newlines or special characters that make it difficult to read when a whole AST is string-ified.
-
-###wdl.parser.AstList
-
-`class AstList(list)` represents a sequence of `Ast`, `AstList`, and `Terminal` objects
-
-Methods:
-
-*`def dumps(self, indent=None, b64_source=True)` - returns a String representation of this AstList. the `indent` parameter takes an integer for the indent level. Omitting this value will cause there to be no new-lines in the resulting string. `b64_source` will be passed to recursive invocations of `dumps`.
-
-##Working with expressions
-
-Parsing a WDL file will result in unevaluated expressions. For example:
-
-```
-workflow test {
- Int a = (1 + 2) * 3
- call my_task {
- input: var=a*2, var2="file"+".txt"
- }
-}
-```
-
-This workflow definition has three expressions in it: `(1 + 2) * 3`, `a*2`, and `"file"+".txt"`.
-
-Expressions are stored in `wdl.binding.Expression` object. The AST for the expression is stored in this object.
-
-Expressions can be evaluated with the `eval()` method on the `Expression` class.
-
-```python
-import wdl
-
-# Manually parse expression into wdl.binding.Expression
-expression = wdl.parse_expr("(1 + 2) * 3")
-
-# Evaluate the expression.
-# Returns a WdlValue, specifically a WdlIntegerValue(9)
-evaluated = expression.eval()
-
-# Get the Python value
-print(evaluated.value)
-```
-
-Sometimes expressions contain references to variables or functions. In order for these to be resolved, one must pass a lookup function and an implementation of the functions that you want to support:
-
-```python
-import wdl
-from wdl.values import WdlInteger, WdlUndefined
-
-deftest_lookup(identifier):
-if identifier =='var':
-return WdlInteger(4)
-else:
-return WdlUndefined
-
-deftest_functions():
-defadd_one(parameters):
-# assume at least one parameter exists, for simplicity
-return WdlInteger(parameters[0].value +1)
-defget_function(name):
-if name =='add_one': return add_one
-else: raise EvalException("Function {} not defined".format(name))
scottfrazer
committed
0 comments on commit
a2a98df