Skip to content

Commit

Permalink
feat: allow config files to be processed with YTE (#2269)
Browse files Browse the repository at this point in the history
This is enabled by adding the top-level key

```yaml
__use_yte__ = true
```

to the config file.

### QC
<!-- Make sure that you can tick the boxes below. -->

* [x] The PR contains a test case for the changes or the changes are
already covered by an existing test case.
* [x] The documentation (`docs/`) is updated to reflect the changes or
this is not necessary (e.g. if the change does neither modify the
language nor the behavior or functionalities of Snakemake).
  • Loading branch information
johanneskoester committed May 22, 2023
1 parent 6f602a3 commit 8e1c22f
Show file tree
Hide file tree
Showing 6 changed files with 21 additions and 11 deletions.
3 changes: 3 additions & 0 deletions docs/snakefiles/configuration.rst
Expand Up @@ -21,6 +21,9 @@ A configuration is provided as a JSON or YAML file and can be loaded with:
configfile: "path/to/config.yaml"
The config file can be used to define a dictionary of configuration parameters and their values.
In case of YAML, the file can optionally be processed with `YTE <https://yte-template-engine.github.io>`_.
To activate this, you have to add the top-level key ``__use_yte__ = true`` to the YAML file.

In the workflow, the configuration is accessible via the global variable `config`, e.g.

.. code-block:: python
Expand Down
13 changes: 2 additions & 11 deletions snakemake/io.py
Expand Up @@ -1694,18 +1694,9 @@ def _load_configfile(configpath_or_obj, filetype="Config"):
except ValueError:
f.seek(0) # try again
try:
# From https://stackoverflow.com/a/21912744/84349
class OrderedLoader(yaml.Loader):
pass
import yte

def construct_mapping(loader, node):
loader.flatten_mapping(node)
return collections.OrderedDict(loader.construct_pairs(node))

OrderedLoader.add_constructor(
yaml.resolver.BaseResolver.DEFAULT_MAPPING_TAG, construct_mapping
)
return yaml.load(f, Loader=OrderedLoader)
return yte.process_yaml(f, require_use_yte=True)
except yaml.YAMLError:
raise WorkflowError(
"Config file is not valid JSON or YAML. "
Expand Down
7 changes: 7 additions & 0 deletions tests/test_config_yte/Snakefile
@@ -0,0 +1,7 @@
configfile: "config.yaml"

rule a:
output:
"test.out"
shell:
"echo {config[test]} > {output}"
3 changes: 3 additions & 0 deletions tests/test_config_yte/config.yaml
@@ -0,0 +1,3 @@
__use_yte__: true

test: ?5 + 5
1 change: 1 addition & 0 deletions tests/test_config_yte/expected-results/test.out
@@ -0,0 +1 @@
10
5 changes: 5 additions & 0 deletions tests/tests.py
Expand Up @@ -2058,3 +2058,8 @@ def test_localrule():
@skip_on_windows
def test_module_wildcard_constraints():
run(dpath("test_module_wildcard_constraints"))


@skip_on_windows
def test_config_yte():
run(dpath("test_config_yte"))

0 comments on commit 8e1c22f

Please sign in to comment.