Allow Singularity to run containers #106
Conversation
Test PASSed. |
I'd hold off on that for now, #69 should be merged first or the diffs will be hard to manage. |
} | ||
|
||
(filesToMount, Seq("singularity", | ||
"-q", |
heuermh
Feb 14, 2018
Member
I haven't gone through the Singularity install on my Mac yet (will soon) and the web docs aren't that helpful, what is -q
for?
I haven't gone through the Singularity install on my Mac yet (will soon) and the web docs aren't that helpful, what is -q
for?
heuermh
Feb 20, 2018
Member
Don't see -q
listed at https://github.com/singularityware/singularity/blob/master/libexec/cli/exec.info
Don't see -q
listed at https://github.com/singularityware/singularity/blob/master/libexec/cli/exec.info
(filesToMount, Seq("singularity", | ||
"-q", | ||
"exec", | ||
"-B", |
heuermh
Feb 14, 2018
Member
is there a --long-argument
form for -B
?
is there a --long-argument
form for -B
?
heuermh
Feb 20, 2018
Member
Yes, --bind
Yes, --bind
"-B", | ||
"%s:%s".format(mountpoint, "/mnt/"), | ||
"docker://" + args.dockerImage, | ||
"bwa", |
heuermh
Feb 14, 2018
Member
Does Singularity support --rm
, or similar flag to remove the image from disk when complete?
Does Singularity support --rm
, or similar flag to remove the image from disk when complete?
heuermh
Feb 20, 2018
Member
No, it does this by default
No, it does this by default
"-q", | ||
"exec", | ||
"-B", | ||
"%s:%s".format(mountpoint, "/mnt/"), |
heuermh
Feb 14, 2018
Member
in Nextflow it looks like they use -B mountpoint
(without the target) or -B mountpoint:mountpoint:ro
to mount read-only, might this work?
in Nextflow it looks like they use -B mountpoint
(without the target) or -B mountpoint:mountpoint:ro
to mount read-only, might this work?
@jpdna @fnothaft @akmorrow13 Please take a look at WIP #107, that is the direction I'm currently headed. Usage would look something like val builder = CommandBuilders.create(args.useDocker, args.useSingularity)
.addEnvironment("ENV", "value")
.addMount("/source", "/target")
.addFile("/file/to/add")
.setTimeout(1000, TimeUnit.SECONDS)
.setSudo(true)
.setImage("namespace/image")
.setExecutable("bwa mem")
.addArgument("-t", "1")
.addArgument("-")
genomicRdd.pipe(
builder.build(),
builder.getEnvironment(),
builder.getFiles(),
builder.getOptTimeout(),
builder.getOptFlankSize()) |
Closing in favor of #107. |
So far the
-use_singularity
option has only been added for BWA and tested successfully, but that should serve as a model for the other Cannoli commands.I will implement and test the other commands over the next days.
Overview:
Singularity is a containerized execution environment similar to Docker, and can directly use Docker images.
It is currently common for HPC cluster administrators to forbid the direct use of Docker due to security concerns, but often do support Singularity, making it the only containerized option available to these users.
The primary difference in parameters to Singularity have to do with the volume mapping into the container. The
-B
bind option in Singularity works similarly to the-v
option in Docker, however I was not able to map the same arbitrary path as Docker does inHowever I was able to map an external path to the within container location
/mnt/
as in: