docker desktop may be incompatible with dockstore cli

[denis-yuen]

Describe the bug
Potential incompatibility and permissions issues running workflows with Docker desktop and docker cli on Ubuntu

To Reproduce

1. git clone git@github.com:GA4GH-DREAM/dockstore-workflow-helloworld.git
2. cd dockstore-workflow-helloworld
3. dockstore workflow launch --entry github.com/GA4GH-DREAM/dockstore-workflow-helloworld/dockstore-workflow-helloworld:1.0.0 --json test.cwl.json

Expected behavior
Should just run.

Temporary workaround is to recommend Docker engine, should emphasize on https://docs.dockstore.org/en/stable/getting-started/getting-started-with-docker.html#how-do-i-install-docker

Screenshots

Additional context
https://oicr.slack.com/archives/C16ET3CF4/p1663265312705299

┆Issue is synchronized with this Jira Story
┆Attachments: image-20230405-192146.png
┆Fix Versions: Dockstore 1.14
┆Issue Number: DOCK-2246
┆Sprint: 101 - Cheeks, Sandy
┆Issue Type: Story

[denis-yuen]

install Docker desktop rather than Docker engine as a prerequisite

[aofarrel]

Possible considerations (caveat: I'm lacking the OICR Slack context)

• When you go to the official instructions for installing Docker Engine, you're gently told to install Docker Desktop instead -- if we direct people to this page to install Docker Engine, we need to give them a good reason to ignore this message
• Assuming the "Docker is not running" text in the screenshot is correct, a possible workaround for Docker Desktop users might be to have Docker Desktop run on startup
• Cromwell has been known to cause Docker to stop responding, so if we are pointing people to Docker Engine, we may need to also include a note on how to force quit/force restart Docker via Docker Engine (with Docker Desktop it's very easy)

[denis-yuen]

When you go to the official instructions for installing Docker Engine, you're gently told to install Docker Desktop instead -- if we direct people to this page to install Docker Engine, we need to give them a good reason to ignore this message

That's annoying. FWIW, it may be as simple as "we've only tested successfully with Docker Engine"
We don't know yet why dockstore CLI has issues on Ubuntu with Docker Desktop, plus we may get to the point where we have to pay for Docker Desktop.

Assuming the "Docker is not running" text in the screenshot is correct, a possible workaround for Docker Desktop users might be to have Docker Desktop run on startup

Might be a red-herring, @hyunnaye was able to run Docker containers manually. We think it was the permissions issue.

[denis-yuen]

Ok, did some research on this. While installing Docker Desktop, it initially completely failed which made me suspicious. Turns out Docker Desktop for Linux requires virtualization to be running unlike Docker engine. It does this because it runs Docker desktop inside a VM(!), which kind of defeats the performance boost of being able to run containers without the overhead of a VM in the first place.

i.e. https://raw.githubusercontent.com/collabnix/dockerlabs/master/beginners/docker/images/vm-docker5.png is no longer correct, Docker engine is running inside a VM with Docker desktop!

My reaction was basically https://news.ycombinator.com/item?id=31331672 and https://news.ycombinator.com/item?id=31334581

[denis-yuen]

Anyway, I gave the dockstore CLI a shot and ran into the above problem.
Debugging inside the Docker container, I found this odd

$ docker     run     -ti     <snip>    --workdir=/TDNIsD     --read-only=true     --user=1000:1000    --env=TMPDIR=/tmp     --env=HOME=/TDNIsD     quay.io/ga4gh-dream/dockstore-tool-helloworld:1.0.2     bash
groups: cannot find name for group ID 1000
I have no name!@c3eca79da23c:~$ cat /usr/local/bin/hello_world
#!/usr/bin/env bash
set -euf -o pipefail

TEMPLATE=$(cat ${1})
INPUT=$(cat ${2})

printf "$TEMPLATE" $INPUT > helloworld.txt
I have no name!@c3eca79da23c:~$ touch foo.txt
touch: cannot touch 'foo.txt': Permission denied
I have no name!@c3eca79da23c:~$ pwd
/TDNIsD
$ ls -alhtr
total 8.0K
drwxr-xr-x 1 root root 4.0K Nov 29 21:57 ..
drwx------ 2 root root 4.0K Nov 29 21:59 .

In other words. The directories are owned by root which is weird since in Docker engine, the ownership looks more like

$ docker     run     -i     <snip>     --workdir=/wcZEpR     --read-only=true     --user=1000:1000  --env=TMPDIR=/tmp     --env=HOME=/wcZEpR     quay.io/ga4gh-dream/dockstore-tool-helloworld:1.0.2     bash 
 
ls
helloworld.txt
ls -alhtr
total 12K
-rw-r--r-- 1 1000 1000   13 Nov 29 22:19 helloworld.txt
drwx------ 2 1000 1000 4.0K Nov 29 22:19 .
drwxr-xr-x 1 root root 4.0K Nov 29 22:21 ..

In other words, the user that owns the directories is that id 1000 user that Docker start command is using with --user=1000:1000 whereas in Docker desktop it is incorrectly root.

This looks a bit like docker/desktop-linux#81

In the meantime, I think we just have to warn against Docker desktop for the Dockstore CLI and keep an eye on Docker Desktop issues in the cwltool repo

[unito-bot]

➤ Kathy Tran commented:

PR: dockstore/dockstore-ui2#1652 ( https://github.com/dockstore/dockstore-ui2/pull/1652|smart-link )

Verified on QA that the quick start has a warning about using Docker Desktop with the Dockstore CLI:

!image-20230405-192146.png|width=1278,height=61!

Moving to Done

[Domain] - https://qa.dockstore.org ( https://qa.dockstore.org|smart-link )

Webservice ( https://github.com/dockstore/dockstore/commits/199226f ) - 199226f

UI ( https://github.com/dockstore/dockstore-ui2/commits/274e8dee ) - 2.10.1-112-g274e8dee

Compose Setup ( https://github.com/dockstore/compose_setup/commits/6f68328 ) - 6f68328

Deploy ( https://github.com/dockstore/dockstore-deploy/commits/140c057 ) - 1.13.2-31-g140c057

cwlParsingLambdaVersion: n/a

wdlParsingLambdaVersion: n/a

nextflowParsingLambdaVersion: n/a

galaxyParsingLambdaVersion: 0.0.6

checkUrlLambdaVersion: n/a