docker-compose is unstable using Docker Desktop and WSL2

[FVolral]

Description of the issue

In some conditions, some command like docker-compose up or docker-compose ps leads to the following error :

Traceback (most recent call last):
  File "bin/docker-compose", line 3, in <module>
  File "compose/cli/main.py", line 67, in main
  File "compose/cli/main.py", line 123, in perform_command
  File "compose/cli/command.py", line 69, in project_from_options
  File "compose/cli/command.py", line 125, in get_project
  File "compose/cli/command.py", line 184, in get_project_name
  File "posixpath.py", line 383, in abspath
FileNotFoundError: [Errno 2] No such file or directory
[3512] Failed to execute script docker-compose

Docker Desktop restart but it doesn't solve the problem.

Context information (for bug reports)

docker-compose.yml

version: '3.8'

services:
    mongo:
        image: mongo
        networks:
            mongo_net:
        volumes:
            - "./mongodata:/data/db"
        ports:
            - 27017:27017
        restart: always
        environment:
            MONGO_INITDB_ROOT_USERNAME: root
            MONGO_INITDB_ROOT_PASSWORD: 1234567
        command: mongod --bind_ip 0.0.0.0
    app:
        build:
            context: ./app/
            dockerfile: app.dockerfile
        volumes:
            - "./app:/app"
        networks:
            - mongo_net
        entrypoint: python
        tty: true
        stdin_open: true
        depends_on:
            - mongo
networks:
    mongo_net:
volumes:
    mongodata:

app/app.dockerfile

FROM python:3

WORKDIR /app

COPY requirements.txt /requirements.txt
RUN python -m pip install --upgrade pip
RUN pip install --no-cache-dir -r /requirements.txt

WORKDIR /app

WSL2 / Win10 version

> ver
Microsoft Windows [version 10.0.19041.572]

>wsl -l -v
  NAME                   STATE           VERSION
* Ubuntu                 Running         2
  docker-desktop         Running         2
  docker-desktop-data    Running         2

Output of docker-compose version

docker-compose version 1.27.4, build 40524192
docker-py version: 4.3.1
CPython version: 3.7.7
OpenSSL version: OpenSSL 1.1.0l  10 Sep 2019

Output of docker version

Client:
 Version:           19.03.8
 API version:       1.40
 Go version:        go1.13.8
 Git commit:        afacb8b7f0
 Built:             Wed Oct 14 19:43:43 2020
 OS/Arch:           linux/amd64
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.0-beta1
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       9c15e82
  Built:            Tue Oct 13 18:17:18 2020
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          v1.4.1
  GitCommit:        c623d1b36f09f8ef6536a057bd658b3aa8632828
 runc:
  Version:          1.0.0-rc92
  GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Output of docker-compose config
(Make sure to add the relevant -f and other flags)

Traceback (most recent call last):
  File "bin/docker-compose", line 3, in <module>
  File "compose/cli/main.py", line 67, in main
  File "compose/cli/main.py", line 120, in perform_command
  File "compose/cli/main.py", line 324, in config
  File "compose/cli/command.py", line 99, in get_config_from_options
  File "compose/config/config.py", line 406, in load
  File "compose/config/config.py", line 520, in load_services
  File "compose/config/config.py", line 501, in build_services
  File "compose/config/config.py", line 501, in <listcomp>
  File "compose/config/config.py", line 480, in build_service
  File "compose/config/config.py", line 289, in with_abs_paths
  File "posixpath.py", line 383, in abspath
FileNotFoundError: [Errno 2] No such file or directory
[3611] Failed to execute script docker-compose

Steps to reproduce the issue

I tried many solutions to solve this and now I'm a bit lost. Docker has worked while few months like a charm on an another project. I don't understand what causes that instability. It happens after a docker-compose up. Yesterday, I had this error and tried to reinstall Docker 2.4.0.0 Stable but I had various problem with it (Installer unable to finish its installation, Docker Desktop unstable, Docker service trying to start forever, Stuck when restoring factory default) That was a mess so I removed the previous version of docker and installed the 2.4.2.0 Edge version of Docker Desktop which can be found here.

Docker become more stable after that and I was able to Clean / Purge data then Reset to factory defaults (looks stupid to do that since the installation was fresh but he).

I thought my ordeal was over but this problem come back now.

[carlsonad]

I'm having a very similar behavior. I'm getting the exact same error message as reported as the OP. But I am able to work around this by opening a new command window. Once I've got a new command window open I seem to be able to use docker-compose for a few commands then it will start throwing the reported error. Once I get that error I'll have to open a new command window to continue working. It's not consistent on how many commands I can execute. Sometimes it's 1 or 2. Other times it's 7 or 8 commands. But it always seems to eventually corrupt.

I recently switched to WSL and seems to happen regardless if I'm running on the windows command prompt or on a Linux command prompt. I do not recall this happening before I switched to WSL.

WSL2 / Win10 version

>ver
Microsoft Windows [Version 10.0.18363.1139]

>wsl -l -v
  NAME                   STATE           VERSION
* Ubuntu-20.04           Running         2
  docker-desktop-data    Running         2
  docker-desktop         Running         2

Output of docker-compose version

docker-compose version 1.27.4, build 40524192
docker-py version: 4.3.1
CPython version: 3.7.4
OpenSSL version: OpenSSL 1.1.1c  28 May 2019

Output of docker version

Client: Docker Engine - Community
 Cloud integration: 1.0.2
 Version:           19.03.13
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        4484c46d9d
 Built:             Wed Sep 16 17:00:27 2020
 OS/Arch:           windows/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.13
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       4484c46d9d
  Built:            Wed Sep 16 17:07:04 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.3.7
  GitCommit:        8fba4e9a7d01810a393d5d25a3621dc101981175
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

[chadbrewbaker]

Just happened to me after I updated the Windows side docker to Docker version 19.03.13, build 4484c46d9d, then called a docker-compose down from Ubuntu 20.04 Docker version 19.03.8, build afacb8b7f0

I'm guessing the solution is to upgrade the WSL2 linux land docker client?

[bcheronn]

Description of the issue

Got hit by the same and the workaround above helped

But I am able to work around this by opening a new command window. Once I've got a new command window open I seem to be able to use docker-compose for a few commands then it will start throwing the reported error. Once I get that error I'll have to open a new command window to continue working.

Context information

Output of docker-compose version

docker-compose version 1.27.4, build 40524192
docker-py version: 4.3.1
CPython version: 3.7.7
OpenSSL version: OpenSSL 1.1.0l  10 Sep 2019

Output of docker version

Client: Docker Engine - Community
 Cloud integration: 1.0.2
 Version:           19.03.13
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        4484c46d9d
 Built:             Wed Sep 16 17:02:36 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.13
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       4484c46d9d
  Built:            Wed Sep 16 17:07:04 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.3.7
  GitCommit:        8fba4e9a7d01810a393d5d25a3621dc101981175
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Output of docker-compose config

services:
  composer:
    image: composer
    volumes:
    - /mnt/c/wamp64/www/parisportif1/code:/app:rw
    - /mnt/c/wamp64/www/parisportif1/.git:/.git:rw
  db:
    environment:
      MYSQL_ALLOW_EMPTY_PASSWORD: "true"
    image: mysql
    restart: on-failure
    volumes:
    - db-volume:/var/lib/mysql:rw
  mailhog:
    image: mailhog/mailhog
    labels:
      traefik.http.services.mailhog.loadbalancer.server.port: '8025'
    restart: on-failure
    volumes:
    - mail-volume:/maildir:rw
  php:
    build:
      cache_from:
      - php:7.4-apache
      context: /mnt/c/wamp64/www/parisportif1/docker/php74-xdebug
    depends_on:
      db:
        condition: service_started
      mailhog:
        condition: service_started
    restart: on-failure
    volumes:
    - /mnt/c/wamp64/www/parisportif1/code:/app:rw
  phpmyadmin:
    depends_on:
      db:
        condition: service_started
    environment:
      PMA_ABSOLUTE_URI: http://phpmyadmin-parisportif1.docker.localhost/
    image: phpmyadmin
    restart: on-failure
  traefik:
    depends_on:
      mailhog:
        condition: service_started
      php:
        condition: service_started
      phpmyadmin:
        condition: service_started
    image: traefik
    ports:
    - published: 80
      target: 80
    - published: 8080
      target: 8080
    restart: on-failure
    volumes:
    - /mnt/c/wamp64/www/parisportif1/docker/traefik/conf/traefik.yml:/etc/traefik/traefik.yml:rw
    - /var/run/docker.sock:/var/run/docker.sock:ro
version: '3'
volumes:
  db-volume: {}
  mail-volume: {}

Steps to reproduce the issue

1. Execute any docker-compose command
2. Output of docker-compose run composer update as example below, but I got the same error stack with all docker-composer commands

Observed result

It fails with the stack trace below. Note that I do not get a stack trace if I run a docker-compose without command. I get the expected usage message.

Expected result

Execute the command passed to docker-compose

Stacktrace / full error message

Traceback (most recent call last):
  File "bin/docker-compose", line 3, in <module>
  File "compose/cli/main.py", line 67, in main
  File "compose/cli/main.py", line 123, in perform_command
  File "compose/cli/command.py", line 69, in project_from_options
  File "compose/cli/command.py", line 125, in get_project
  File "compose/cli/command.py", line 184, in get_project_name
  File "posixpath.py", line 383, in abspath
FileNotFoundError: [Errno 2] No such file or directory
[2527] Failed to execute script docker-compose

Additional information

Output of ver

Microsoft Windows [version 10.0.19041.630]

Output of wsl -l -v

  NAME                   STATE           VERSION
* Debian                 Running         2
  docker-desktop         Running         2
  docker-desktop-data    Running         2

docker-compose installed via Docker Desktop Installer.exe

[F21]

Seeing the problem with the latest version of Windows 10 2004 with WSL 2 and Docker Desktop for Windows 2.5.0.1 stable.

I think the problem is because the version of compose that ships with Docker for Windows is located in the Windows filesystem and due to some instability in WSL2 volume mounts, it doesn't work correctly.

A workaround I am using is to just install docker-compose directly into WSL2. If you execute which docker-compose you should see that the copy in WSL2 should shadow/override the one from Docker for WIndows and it should be much more stable.

[pasisavolainen]

Seeing this as well.

Closing docker, wsl --shutdown and restarting docker has cleared this up at least once now.

[jlu1202]

I ran into this problem just now and was able to do a couple of things to have it resolved.

1. Ensure the docker cli version in WSL 2 matches up the one on your Windows 10 (I'm on Version 10.0.19042.685).
2. Ensure the current working directory is NOT a mount point from Windows, e.g. /mnt/c/..., this is especially important when you are running a script or an utility like 'make'.

[Tapiiri]

Encountered the error after forcing an exit on a container.

Closing the PowerShell window I was using, opening a new one and runnin wsl on it again was sufficient to get it back working...but only until the next time exit from the container, this time without forcing.

Sure enough, wsl --shutdown and Docker restart has it working even after exiting a container, but failed exits still seem to bring the problem about.

Edit: My case definitely could've been a problem about using a Windows mount point, haven't seen these problems lately.

[clairmont32]

I was working out of an existing git repo that I created within windows and it was working fine until I started dealing with volume mounts under that path structure. Moving it to the native WSL 2 filesystem fixed this issue for me.

[sebastien-roche-cko]

I'm now experiencing the same issue. It started to happen a few weeks ago. At that time, this behavior was just occasional and random. Restarting the WSL, Docker or clearing all docker related content was fixing the issue.
Today, every time I try to run docker-compose start, I get the error you mentioned:

Traceback (most recent call last):
  File "bin/docker-compose", line 3, in <module>
  File "compose/cli/main.py", line 67, in main
  File "compose/cli/main.py", line 123, in perform_command
  File "compose/cli/command.py", line 69, in project_from_options
  File "compose/cli/command.py", line 125, in get_project
  File "compose/cli/command.py", line 184, in get_project_name
  File "posixpath.py", line 383, in abspath
FileNotFoundError: [Errno 2] No such file or directory
[1250] Failed to execute script docker-compose

At some point, after restarting WSL/resetting/restarting WSL/resetting/purging all data/reopening new terminal windows (quite random), it will start to work again but only for one or two docker-compose start.

I tried

• to update/reinstall everything and start from scratch. I'm running pretty much the latest stable version of everything.
• to install older versions of docker (3.0.0)
• to ensure I'm using the latest WSL Linux kernel
• to disable buildkit (just in case)
• wsl --shutdown / purge all docker data
• disabling our antivirus and related tools

docker-compose version

docker-compose version 1.27.4, build 40524192
docker-py version: 4.3.1
CPython version: 3.7.7
OpenSSL version: OpenSSL 1.1.0l  10 Sep 2019

docker version

Client: Docker Engine - Community
 Version:           20.10.2
 API version:       1.41
 Go version:        go1.13.15
 Git commit:        2291f61
 Built:             Mon Dec 28 16:17:34 2020
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.2
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       8891c58
  Built:            Mon Dec 28 16:15:28 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.3
  GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
 runc:
  Version:          1.0.0-rc92
  GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Running windows 10 with the latest updates. I'm a bit stuck today. I see lots of comments saying we should not run scripts from a folder located under a mount point. However, in my case this was working perfectly before, and running my script outside of a specific folder break them

[ioanok]

The issue comes from wsl2 mounted directories.
I have the same error and what is working for me is workaround from here
cd; cd -
The only thing is the error appears randomly and I am using a 'Makefile' to pack a series of commands.

[sebastien-roche-cko]

Thx for the info. I tried a couple of things after getting the error message:

• Run these commands > the error still shows up
• Run these commands in a new Terminal Windows > the error still shows up
• Run these commands after restarting docker > the error still shows up

None of this works.. It was working few weeks ago and today it's working 1% of the time. A colleague of mine is starting to have the same issue.

[mfloresVicomtech]

Me having this problem from some time now. I've assumed and I usually just cd ${PWD} and it works again.

[michaelyohanes]

Same here, had to restart to make docker-compose works again in wsl2.

60% occurence on following steps:

1. start docker desktop win
2. open WSL2
3. docker-compose build
4. docker-compose up -d

System:
~ win 10 pro 20H2 - build 19042.844
~ docker desktop win : v3.1.0 (51484)
~ WSL2: Ubuntu 20.04 LTS

  File "bin/docker-compose", line 3, in <module>
  File "compose/cli/main.py", line 67, in main
  File "compose/cli/main.py", line 123, in perform_command
  File "compose/cli/command.py", line 69, in project_from_options
  File "compose/cli/command.py", line 125, in get_project
  File "compose/cli/command.py", line 184, in get_project_name
  File "posixpath.py", line 383, in abspath
FileNotFoundError: [Errno 2] No such file or directory`

Please advise.

[ewenkel]

Has there been any further updates on this? I am experiencing this too since upgrading to wsl 2