Skip to content

Commit

Permalink
recipe test spec, use tag char whitelist
Browse files Browse the repository at this point in the history
  • Loading branch information
stolarczyk committed Aug 24, 2021
1 parent 1886d13 commit 2477106
Show file tree
Hide file tree
Showing 2 changed files with 28 additions and 4 deletions.
22 changes: 22 additions & 0 deletions docs/recipe_specification.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,7 @@ Within the `command_template_list`, you have access to variables from several so
- files (`Dict[str, Dict[str, str]]`): The _resolved_ input files.
- assets (`Dict[str, Dict[str, str]]`): The _resolved_ input assets.
- custom*properties (`Dict[str, Dict[str, Any]]`): The \_resolved* custom properties.
- test (`Dict[str, Dict[str, str]]`): The recipe testing setup.

This is a dict-like representation of an example namespaces object available to the command templates:

Expand All @@ -135,6 +136,7 @@ namespaces = {
"custom_properties": {
"version": "0.0.1",
"settings": "a=1, b=2"
}
}
```

Expand All @@ -146,3 +148,23 @@ The `default_tag` value can be a string or a Jinja2 template, that combines the
## custom_properties

This section is an object where keys are the names of custom properties and values are shell commands that will be executed in the asset building requirement. This is a way to record any computing enviroment specific information, like the version of a software package used to produce the asset. Please note that the build namespaces are not available to the commands in this section.

## test

This section is used to specify the test setup for the recipe. The test setup is a dictionary of test parameters, which include:

- `genome`: the genome digest to use for derived asset tests. Any input assets will be sourced from this genome namespace.
- `inputs`: the input files to the recipes. The values must be URLs to remote files, so that the recipes are portable. The keys in this section must match the keys in the `inputs.files` section or the recipe. Currently `inputs.params` and `inputs.assets` cannot be changed, default values from the top-level `inputs` section are used.
- `outputs`: the outputs to test the results of running the recipe commands against. There are two ways the outputs can be tested: 1) by comparing the file checksum (`checksum`), or 2) by comparing the value (`value`). The keys in this section must match the `seek_keys` in the asset class definition.

```yaml
test:
genome: 6c5f19c9c2850e62cc3f89b04047fa05eee911662bd77905
inputs:
fasta: /path/to/genome/fasta.fa
outputs:
html:
md5sum: d41d8cd98f00b204e9800998ecf8427e
# OR
value: 100
```
10 changes: 6 additions & 4 deletions refgenie/refgenie.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
CFG_TAG_DESC_KEY,
LOCKED_BUILD_MAP_CFG,
ORI_LOG_NAME_REGEX,
TAG_NAME_BANNED_CHARS,
TAG_NAME_CHAR_WHITELIST,
TEMPLATE_RECIPE_JSON,
TEMPLATE_TARGET,
)
Expand Down Expand Up @@ -662,10 +662,12 @@ def _build_asset(
or recipe.resolve_default_tag(namespaces=build_namespaces)
or rgc.get_default_tag(genome, asset, use_existing=False)
)
if any([c in tag for c in TAG_NAME_BANNED_CHARS]):

if not all([c in TAG_NAME_CHAR_WHITELIST for c in tag]):
raise ValueError(
f"The tag name can't consist of characters: {TAG_NAME_BANNED_CHARS}"
f"The tag name can consist only of these characters: {TAG_NAME_CHAR_WHITELIST}"
)

build_namespaces["tag"] = tag
build_namespaces.update(
{"asset_outfolder": os.path.join(genome_outfolder, asset, tag)}
Expand Down Expand Up @@ -739,7 +741,7 @@ def _build_asset(
os.rename(locked_map_gencfg, map_gencfg)
_LOGGER.info(
f"Asset metadata saved in '{map_gencfg}'. "
f"To make the asset accessible globally run 'refgenie build --reduce'"
f"To make the asset accessible globally run: refgenie build --reduce"
)

return True
Expand Down

0 comments on commit 2477106

Please sign in to comment.