CWL: config file for always-needed toil-cwl-runner options

[mr-c]

Many sites that run toil-cwl-runner, especially HPC systems, need quite a lot of options defined. It would be nice if there was a config file that formed the basis of the command-line options.

See https://github.com/EBI-Metagenomics/emg-viral-pipeline/blob/f367002f0e1e375840e5696323bde65f7accb31f/cwl/virify.sh#L300 for an advanced script from a toil-cwl-runner user

Where? In toil-cwl. + ini or toml or somesuch; under ${XDG_CONFIG_HOME} (which defaults to ${HOME}/.config)

Additional command line options should have priority. The location of the config file should also be overridable with an environment variable.

┆Issue is synchronized with this Jira Story
┆friendlyId: TOIL-1186

[adamnovak]

We have wanted a config file for a while; we should be able to hook into some of the code we have already to pull from either the command line or environment variables.

It might also be nice if it could integrate well with user Toil workflows, so e.g. Cactus or toil-vg could ride the same system.

Lon had the idea of a config generator to show you all the defaults.

@mr-c Do you want to implement this system?

[adamnovak]

@stxue1 Did your new config file stuff in #4569 address this?

[stxue1]

It looks like the config file does not work for toil-cwl-runner. I think configargparse has an issue with nargs=argparse.REMAINDER. It looks like if another nargs/append action is in the mix, the REMAINDER argparse option swallows all config file options.