Skip to content

Unusual and unsupported backends

Jump to bottom
Ash O'Farrell edited this page Oct 31, 2025 · 2 revisions

TB-D's requirements are relatively simple in theory -- Docker and some kind of WDL executor -- but we've been asked about some edge cases, so here's all you need to know.

Note

As of 2025, TB-D is almost entirely run and tested on Terra specifically. Any backend that isn't that is supported only in a vague theoretical sense.

OS

  • Mac OS: Yes
  • Basically any Linux distro: Yes
  • Windows (including WSL): Not tested, not supported, but miiiiight work depending on how the WDL executor is set up
  • FreeBSD/OpenBSD: No, due to lack of Docker support

Architecture

  • x86: Yes
  • ARM (including Apple Silicon): Not tested, but should work provided you set up Docker correctly

Backends

  • Amazon: Not tested, but should work if you set it up correctly
  • GCP: Yes
  • HPC: Entirely depends upon the specifics of the HPC
    • Many HPCs do not support Docker, see below for alternatives
    • For a WDL runner that plays nicely with SLURM, Toil might be what you're after, but please note TB-D has never been tested with Toil
  • Terra: Yes (this is GCP under the hood)
  • Your laptop: Yes, with caveats
    • Earlier notes about OS and Architecture still apply
    • Please be realistic about how powerful your laptop actually is
    • Consider downsampling your data (myco has built-in functionality for this)
    • Small batches have successfully run through a 2019 Macbook Pro (6 cores, Intel, 16 GB mem)
    • It is strongly recommended to use miniwdl instead of Cromwell for this use case due to issues with how Cromwell handles system resources in non-cloud environments. If you must use Cromwell, modify the configuration file's concurrent-task-limit to 1 even if you think your system can do better than that.
      • This greatly reduces the likelihood of tasks getting sigkilled
      • This also helps prevent a bug where Cromwell causes Docker to behave strangely and/or be unable to spin up new containers -- if you do get this bug, you will need to restart Docker Engine

WDL Executors

  • miniwdl: Yes, with --copy-input-files enabled
  • Cromwell: Yes, but not recommended for local-on-my-laptop runs
  • Dockstore CLI: Yes (this is Cromwell under the hood)
  • Toil: Not tested

Docker alternatives (Podman, rootless Docker, Singularity, etc)

This isn't recommended nor supported.

But my institute's HPC doesn't allow Docker :-(

Consider the following:

  • Setting up your WDL executor to use Singularity instead of Docker (we know at least one person got myco to work this way)
  • Asking for help in the OpenWDL Slack
  • Running TB-D on a local computer
  • Bribing your sysadmin

Clone this wiki locally