Syntax error: "(" unexpected in CactusSetupPhase singularity script

[lparsons]

When running using singularity on our slurm cluster (with singularity 3.0.1) I get a syntax error on in the CactusSetupPhase:

=========> Failed job 'CactusSetupPhase' A/y/jobh8kc9l
INFO:toil.worker:---TOIL WORKER OUTPUT LOG---
INFO:toil:Running Toil version 3.14.0-b91dbf9bf6116879952f0a70f9a2fbbcae7e51b6.
WARNING:toil.resource:'JTRES_57583f1446e225c9fde5b17d58ae8b9c' may exist, but is not yet referenced by the worker (KeyError from os.environ[]).
INFO:toil.lib.bioio:Sequences in cactus setup: ['simHuman_chr6', 'simMouse_chr6', 'simRat_chr6', 'simCow_chr6', 'simDog_chr6']
INFO:toil.lib.bioio:Sequences in cactus setup filenames: ['>id=1|simHuman.chr6|0\n', '>id=0|simMouse.chr6\n', '>id=2|simRat.chr6\n', '>id=4|simCow.chr6|0\n', '>id=3|simDog.chr6|0\n']
INFO:cactus.shared.common:Work dirs: set([u'/tmp/toil-e9feab2f-d6c3-451d-b8e3-3a340eaa4356-56d34a5a-a657-4dd7-a772-c841bc6d87d4/tmpn17gUC/537867c6-3575-4d7d-b2dd-9f8098975e0c'])
INFO:cactus.shared.common:Docker work dir: /tmp/toil-e9feab2f-d6c3-451d-b8e3-3a340eaa4356-56d34a5a-a657-4dd7-a772-c841bc6d87d4/tmpn17gUC/537867c6-3575-4d7d-b2dd-9f8098975e0c
INFO:cactus.shared.common:Running the command ['singularity', '--silent', 'run', u'/tigress/lparsons/kocher_lab/bee_genome_alignment/cactus_evolverMammals_jobStorePath/cactus.img', u'cactus_setup', u'--speciesTree', u'((simHuman_chr6:0.144018,(simMouse_chr6:0.084509,simRat_c
hr6:0.091589)mr:0.271974)Anc1:0.020593,(simCow_chr6:0.18908,simDog_chr6:0.16303)Anc2:0.032898)Anc0;', u'--cactusDisk', u'<st_kv_database_conf type="kyoto_tycoon">\n\t\t\t<kyoto_tycoon database_dir="fakepath" host="127.0.0.1" port="6328" />\n\t\t</st_kv_database_conf>\n\t', u
'--logLevel', u'INFO', u'--outgroupEvents', u'simHuman_chr6 simDog_chr6 simCow_chr6', u'tmpIHe4kJ.tmp', u'tmpKPNO0E.tmp', u'tmpVEzmZ8.tmp', u'tmpDwKNI0.tmp', u'tmpGS_J80.tmp']
/.singularity.d/runscript: 1: eval: Syntax error: "(" unexpected
Traceback (most recent call last):
  File "/tigress/lparsons/miniconda3/envs/bee_genome_alignment/lib/python2.7/site-packages/toil/worker.py", line 309, in workerScript
    job._runner(jobGraph=jobGraph, jobStore=jobStore, fileStore=fileStore)
  File "/tigress/lparsons/kocher_lab/bee_genome_alignment/cactus/src/cactus/shared/common.py", line 1094, in _runner
    super(RoundedJob, self)._runner(jobGraph=jobGraph, jobStore=jobStore, fileStore=fileStore)
  File "/tigress/lparsons/miniconda3/envs/bee_genome_alignment/lib/python2.7/site-packages/toil/job.py", line 1328, in _runner
    returnValues = self._run(jobGraph, fileStore)
  File "/tigress/lparsons/miniconda3/envs/bee_genome_alignment/lib/python2.7/site-packages/toil/job.py", line 1273, in _run
    return self.run(fileStore)
  File "/tigress/lparsons/kocher_lab/bee_genome_alignment/cactus/src/cactus/pipeline/cactus_workflow.py", line 641, in run
    makeEventHeadersAlphaNumeric=self.getOptionalPhaseAttrib("makeEventHeadersAlphaNumeric", bool, False))
  File "/tigress/lparsons/kocher_lab/bee_genome_alignment/cactus/src/cactus/shared/common.py", line 220, in runCactusSetup
    parameters=["cactus_setup"] + args + sequences)
  File "/tigress/lparsons/kocher_lab/bee_genome_alignment/cactus/src/cactus/shared/common.py", line 1038, in cactus_call
    raise RuntimeError("Command %s failed with output: %s" % (call, output))
RuntimeError: Command ['singularity', '--silent', 'run', u'/tigress/lparsons/kocher_lab/bee_genome_alignment/cactus_evolverMammals_jobStorePath/cactus.img', u'cactus_setup', u'--speciesTree', u'((simHuman_chr6:0.144018,(simMouse_chr6:0.084509,simRat_chr6:0.091589)mr:0.271974
)Anc1:0.020593,(simCow_chr6:0.18908,simDog_chr6:0.16303)Anc2:0.032898)Anc0;', u'--cactusDisk', u'<st_kv_database_conf type="kyoto_tycoon">\n\t\t\t<kyoto_tycoon database_dir="fakepath" host="127.0.0.1" port="6328" />\n\t\t</st_kv_database_conf>\n\t', u'--logLevel', u'INFO', u
'--outgroupEvents', u'simHuman_chr6 simDog_chr6 simCow_chr6', u'tmpIHe4kJ.tmp', u'tmpKPNO0E.tmp', u'tmpVEzmZ8.tmp', u'tmpDwKNI0.tmp', u'tmpGS_J80.tmp'] failed with output:
ERROR:toil.worker:Exiting the worker because of a failed job on host della-r4c2n8
WARNING:toil.jobGraph:Due to failure we are reducing the remaining retry count of job 'CactusSetupPhase' A/y/jobh8kc9l with ID A/y/jobh8kc9l to 0
<=========

Complete logfile attached:
logfile.log.gz

[plazonic]

Reported as an issue in singularity 3.0.*
sylabs/singularity#2622

[lparsons]

This appears to be fixed in the latest Singularity 3.1.0 release candidates.

[lparsons]

My suggestion would be to report it to the Singularity folks as it appears to be a bug in the automatically generated runscript. A perhaps more stable alternative would be to provide a singularity image instead of having cactus download and convert the Docker image each time. See sylabs/singularity#2622 for more info.