docker-compose super slow on macOS Catalina #6956

ghost · 2019-10-11T04:58:49Z

when i do docker-compose up -d, it takes a while to start the process, same when i do docker-compose restart.

in other words, there's a delay of few seconds before it begins to restart containers or stopping containers.

glours · 2019-10-11T05:55:16Z

Hello @benlumia007
Can you give more details, for example the output for docker-compose --log-level debug up -d or docker-compose --log-level debug restart -d

ghost · 2019-10-11T06:20:14Z

is not about the output of docker-compose, is when i type the following

docker-compose up -d and press enter, it just delays for 5 - 10 seconds before it shows restarting or stopping

ghost · 2019-10-11T06:47:03Z

let me do some testing, i'll comment later , it just could be my eyes are deceiving me lol

ndeloof · 2019-10-11T08:54:24Z

Are you using https://github.com/docker/compose/blob/master/contrib/completion/bash/docker-compose for command line completion ?

skazancev · 2019-10-11T09:47:55Z

same problem, macos catalina (10.15 19A583), docker 2.1.0.3 (38240), docker-compose 1.24.1

ghost · 2019-10-11T14:35:17Z

so this is what it supposed to do, when you docker-compose up -d or docker-compose restart

but for macOS Catalina there is a delay between

benlumia007@aatrox:~/WordPress$ docker-compose restart

delays 5 seconds, then

Restarting docker-nginx   ... done
Restarting docker-phpfpm  ... done
Restarting docker-mysql   ... done
Restarting docker-mailhog ... done
benlumia007@aatrox:~/WordPress$

glours · 2019-10-11T14:44:12Z

Can you post the config of your compose file (without any sensitive data of course)
docker-compose config
Just to try to reproduce with the same kind of application (number of containers, eventually network config ...)

ghost · 2019-10-11T14:46:01Z

benlumia007@aatrox:~/WordPress$ docker-compose config
services:
  mailhog:
    container_name: docker-mailhog
    image: benlumia007/mailhog
    ports:
    - published: 1025
      target: 1025
    - published: 8025
      target: 8025
    restart: always
  mysql:
    command: --default-authentication-plugin=mysql_native_password
    container_name: docker-mysql
    environment:
      MYSQL_ROOT_PASSWORD: root
    image: benlumia007/mysql:latest
    ports:
    - published: 3306
      target: 3306
    restart: always
    volumes:
    - db_data:/var/lib/mysql:rw
  nginx:
    container_name: docker-nginx
    depends_on:
    - phpfpm
    image: benlumia007/nginx:latest
    ports:
    - published: 80
      target: 80
    - published: 443
      target: 443
    restart: always
    volumes:
    - /home/benlumia007/WordPress/config/nginx:/etc/nginx/conf.d:rw
    - /home/benlumia007/WordPress/certificates:/etc/nginx/certs:rw
    - /home/benlumia007/WordPress/sites:/var/www/html:rw
  phpfpm:
    container_name: docker-phpfpm
    image: benlumia007/php:7.3-fpm
    ports:
    - target: 9000
    restart: always
    volumes:
    - /home/benlumia007/WordPress/sites:/var/www/html:rw
version: '3.5'
volumes:
  db_data: {}

ghost · 2019-10-11T14:46:53Z

it was pretty good, to begin with when using macOS Mojave, it runs fine, then Catalina just delays a few seconds.

I don't think the issue is how it is config, I think its just the docker itself may not be tested fully under macOS Catalina

carlosonunez · 2019-10-11T18:21:41Z

I am experiencing the same issue. docker-compose can take between 5-60+ seconds to do anything, despite docker commands being executed instantaneously. This makes Compose pretty unusable, tbh.

I didn't see anything coming from docker-compose during startup during tcpdump captures except for UDP traffic within Compose. I did see a lot of tccd and XprotectService sprinkled in during fs_usage traces while docker-compose runs, though I'm not sure if other processes are triggering this activity or not. Trying to invoke docker-compose --log-level debug also incurs a delay.

Interestingly, docker-compose runs very quickly after turning off any networking, so it's definitely something network-related...I just can't see it.

One clue is that whenever I SIGHUP Compose, I get errors that seem to be coming from PyInstaller. This makes me think that running Compose directly from within Python would be much faster and that the issue is with PyInstaller doing...things.

carlosonunez · 2019-10-11T23:35:23Z

I was able to work around this by installing docker-compose from Pip instead of using the precompiled one:

$: pip install docker-compose

This should set your default Compose to the Python one, but double-check just to make sure:

$: file $(which docker-compose) # should NOT be a Mach-O executable

ghost · 2019-10-11T23:38:00Z

yeah but it stil doesn't solve the problem whom is using the compiled version. I'll try that too. but this should be a bug for catalina.

carlosonunez · 2019-10-11T23:38:59Z

I think it's a PyInstaller bug as well :(

wodCZ · 2019-10-14T15:07:10Z

Running into same issue. docker commands are instant:

➜  sites time docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
docker ps  0.03s user 0.08s system 49% cpu 0.232 total

Running docker-compose from Docker for Mac is terribly slow, even when running just version command outside of any docker-compose project:

➜  sites time docker-compose-bundled version
docker-compose version 1.24.1, build 4667896b
docker-py version: 3.7.3
CPython version: 3.6.8
OpenSSL version: OpenSSL 1.1.0j  20 Nov 2018
docker-compose-bundled version  0.43s user 0.14s system 24% cpu 2.338 total

Then I installed docker-compose using pip3 (pip3 install docker-compose --user) and the result is a bit better (but still so slow given that I'm only running version command)

➜  sites time docker-compose version
docker-compose version 1.24.1, build 4667896
docker-py version: 3.7.3
CPython version: 3.7.3
OpenSSL version: LibreSSL 2.8.0
docker-compose version  0.60s user 0.14s system 80% cpu 0.931 total

Of course other subcommands are slow too, I thought using version as an example would eliminate an issue with docker-compose.yml file.
I'm running macOS Catalina Version 10.15.

I've tried turning off network, also tried popular /etc/hosts fix from #3419 (comment) - no success in any case.

diogocnunes · 2019-10-15T08:48:24Z

I have a same issue running on macOS Cataina. If i run the docker-compose version, this take some time, but works, like example below:

If I run under a proxy, the commands doesn't work. It's freeze (this image below it's a animated GIF, not a static image.... i wait for 30 seconds before CTRL+C... watch it...)

I try run the same commands, under proxy, on macOS Mojave and it's works fine...

ndeloof · 2019-10-15T09:57:09Z

I was able to reproduce this issue by updating my own OSX machine to Catalina.
I noticed when running any docker-compose command, even unkown ones (which just show the usage help) I get a request sent to https://api.apple-cloudkit.com. If I switch off my Internet connexion this request is skipped and compose run at normal speed. I assume using a proxy will just make this request hang some way and block docker-compose to run.

I've requested help on the PyInstaller mailing list to diagnose this issue.

Internally tracked as https://docker.atlassian.net/browse/COMPOSE-96

ghost · 2019-10-15T14:50:23Z

@diogocnunes: what software do you use the screen capture.

carlosonunez · 2019-10-15T14:52:21Z

@ndeloof How did you see those requests? I didn't see anything like that when I did a tcpdump capture. I used: sudo tcpdump 'port 80 or port 443'

diogocnunes · 2019-10-15T14:52:35Z

@diogocnunes: what software do you use the screen capture.

I use Kap (https://getkap.co/)

diogocnunes · 2019-10-15T14:56:05Z

I was able to reproduce this issue by updating my own OSX machine to Catalina.
I noticed when running any docker-compose command, even unkown ones (which just show the usage help) I get a request sent to https://api.apple-cloudkit.com. If I switch off my Internet connexion this request is skipped and compose run at normal speed. I assume using a proxy will just make this request hang some way and block docker-compose to run.

I've requested help on the PyInstaller mailing list to diagnose this issue.

Internally tracked as https://docker.atlassian.net/browse/COMPOSE-96

Adding https://api.apple-cloudkit.com to /etc/hosts, pointing to localhost, under proxy, docker-compose is working again ...

I don't know what the real impact of this, but it's one less problem right now :)

ndeloof · 2019-10-15T15:16:17Z

How did you see those requests?

I used charles proxy

I reported this issue on https://groups.google.com/forum/#!msg/pyinstaller/2MvcMaDFXyo/GhpVfaJJCQAJ and was able to reproduce using a minimalist "print hello" python application. Hope this will help maintainers to understand the root cause and propose a fix.

justinpage · 2019-10-23T23:08:20Z

When I run the following command, under our company's VPN, I notice the following behavior:

http://localhost:None "POST /v1.38/networks/test_default/connect HTTP/1.1" 200 0
Pending: set()
Pending: set()
Pending: set()
Pending: set()
Pending: set()
etc...
etc...

This goes on for some time until it finally connects

Pending: set()
Pending: set()
Pending: set()
Pending: set()
http://localhost:None "POST /v1.38/containers/cd03d59f314f95ff1b7c6aa38352bd506028ee542dc3d44f1fa1a29a3007f28b/start HTTP/1.
1" 204 0

This slowdown seems to happen in regards to the following command:

https://docs.docker.com/engine/api/v1.38/#operation/NetworkConnect

Not sure if this is related but I wanted to call this out in case this helps anyone.

ghost · 2019-10-29T23:23:36Z

@diogocnunes
I downloaded the getkap, but it won't work, i keep getting unable to start recording. are you on catalina right.

diogocnunes · 2019-10-30T10:38:24Z

@diogocnunes
I downloaded the getkap, but it won't work, i keep getting unable to start recording. are you on catalina right.

Yes. I use Catalina. Did you select the recording area before you started?

mediaessenz · 2019-10-31T16:15:39Z

Maybe this help someone else: Homebrew/homebrew-core#45687 (comment)

kliyes · 2019-11-09T15:35:55Z

I was able to work around this by installing docker-compose from Pip instead of using the precompiled one:
$: pip install docker-compose
This should set your default Compose to the Python one, but double-check just to make sure:
$: file $(which docker-compose) # should NOT be a Mach-O executable

works for me, thanks.
ps. if a virtualenv activated, you should deactivate it first after pip install docker-compose

rfay · 2019-11-19T17:42:13Z

5.8 seconds to do docker-compose --version on Catalina:

$ time docker-compose --version
docker-compose version 1.25.0-rc4, build 8f3c9c58

real	0m5.762s

rfay · 2019-11-19T17:44:12Z

@benlumia007 (or anyone with privs to do so) could you please edit the title to be more descriptive? Something like "docker-compose super slow on macOS Catalina" - thanks!

mayconmfl · 2019-11-20T05:27:51Z

@benlumia007 I have done added api.apple-cloudkit.com to /etc/hosts, pointing to localhost.
Now it's faster.

My /etc/hosts:

...
# Added by Docker Desktop
# To allow the same kube context to work on the host and the container:
127.0.0.1 kubernetes.docker.internal
127.0.0.1 api.apple-cloudkit.com
...

wodCZ · 2019-12-02T16:51:11Z

I have downloaded compose binary 1.25.1-rc1, which should include the potential fix from #7010 and it still runs slow:

➜  ~ time docker-compose --version
docker-compose version 1.25.1-rc1, build d92e9bee
docker-compose --version  0.49s user 0.14s system 20% cpu 3.178 total
➜  ~ time docker-compose --version
docker-compose version 1.25.1-rc1, build d92e9bee
docker-compose --version  0.49s user 0.15s system 19% cpu 3.340 total
➜  ~ time docker-compose --version
docker-compose version 1.25.1-rc1, build d92e9bee
docker-compose --version  0.49s user 0.15s system 22% cpu 2.932 total
➜  ~ time docker-compose --version
docker-compose version 1.25.1-rc1, build d92e9bee
docker-compose --version  0.51s user 0.15s system 20% cpu 3.244 total

Latest version installed using pip as mentioned earlier in this issue is considerably faster (except the first invocation):

➜  ~ time docker-compose --version
docker-compose version 1.25.0, build b42d419
docker-compose --version  0.45s user 0.18s system 40% cpu 1.566 total
➜  ~ time docker-compose --version
docker-compose version 1.25.0, build b42d419
docker-compose --version  0.32s user 0.07s system 96% cpu 0.403 total
➜  ~ time docker-compose --version
docker-compose version 1.25.0, build b42d419
docker-compose --version  0.31s user 0.07s system 97% cpu 0.392 total
➜  ~ time docker-compose --version
docker-compose version 1.25.0, build b42d419
docker-compose --version  0.30s user 0.07s system 98% cpu 0.370 total

ndeloof · 2019-12-02T17:16:20Z

@wodCZ fix is imlemented in the docker-compose-Darwin-x86_64.tgz "folder-style" installation, the single binary is kept for backward compatibility and pre-catalina users who don't need this fix.

zeshan77 · 2019-12-03T04:13:03Z

@benlumia007 I have done added api.apple-cloudkit.com to /etc/hosts, pointing to localhost.
Now it's faster.

My /etc/hosts:
...
# Added by Docker Desktop
# To allow the same kube context to work on the host and the container:
127.0.0.1 kubernetes.docker.internal
127.0.0.1 api.apple-cloudkit.com
...

This reduced waiting time from 18s to 2s for me. Thanks for sharing!

Before:

After:

ndeloof · 2019-12-03T06:00:58Z

@zeshan77 can you give doker-compose 1.25 tar.gz release a try ?
etc/hosts hack can only be considered a temporary workaround

wodCZ · 2019-12-03T08:24:45Z

@ndeloof the .tgz version seems to work much better than the bundled or the binary:

time ./docker-compose --version
docker-compose version 1.25.1-rc1, build d92e9bee
./docker-compose --version  0.50s user 0.07s system 97% cpu 0.587 total
time ./docker-compose --version
docker-compose version 1.25.1-rc1, build d92e9bee
./docker-compose --version  0.50s user 0.08s system 97% cpu 0.593 total
time ./docker-compose --version
docker-compose version 1.25.1-rc1, build d92e9bee
./docker-compose --version  0.48s user 0.07s system 98% cpu 0.567 total
time ./docker-compose --version
docker-compose version 1.25.1-rc1, build d92e9bee
./docker-compose --version  0.48s user 0.07s system 97% cpu 0.568 total

the version installed using pip still seems to be faster though:

time docker-compose --version
docker-compose version 1.25.0, build b42d419
docker-compose --version  0.38s user 0.10s system 97% cpu 0.487 total
time docker-compose --version
docker-compose version 1.25.0, build b42d419
docker-compose --version  0.38s user 0.09s system 97% cpu 0.491 total
time docker-compose --version
docker-compose version 1.25.0, build b42d419
docker-compose --version  0.38s user 0.09s system 97% cpu 0.480 total

BirkhoffLee · 2019-12-04T06:21:20Z

@wodCZ fix is imlemented in the docker-compose-Darwin-x86_64.tgz "folder-style" installation, the single binary is kept for backward compatibility and pre-catalina users who don't need this fix.

Can you elaborate on "folder-style" installation? How do I do that? Running the executable in that archive is same slow (20+ second for me).

ndeloof · 2019-12-04T07:05:41Z

pyinstaller used by docker-compose to bake a binary executable can both create a single-file binary, or a folder with binary + all libraries.
In the former case, unpacking at execution time trigger Catalina security scanning, so slowness. In the later, scan will occur on first run, but then executable is flagged by system as "allready scanned"

so, unpacking the tar.gz and running docker-compose version from there should fix the issue (you can create a symlink to binary in folder)

Yelfive · 2020-01-10T15:25:40Z

I was able to work around this by installing docker-compose from Pip instead of using the precompiled one:
$: pip install docker-compose
This should set your default Compose to the Python one, but double-check just to make sure:
$: file $(which docker-compose) # should NOT be a Mach-O executable

@carlosonunez I installed with pip, but no executable found anywhere, what did I miss please?

ulyssessouza · 2020-01-10T16:18:18Z

@Yelfive As mentioned by @ndeloof in #6956 (comment) we now release in two flavors for macOS. Please refer to the releases page and get the docker-compose-Darwin-x86_64.tgz from the latest one.

Yelfive · 2020-01-11T06:20:53Z

@ulyssessouza Thx for your reply, finally I got it to work. I previously tried all versions of the binary release, and missed the tgz part. Thank you again.

SteffenAnders · 2020-07-02T06:31:08Z

We have solved the problem by installing docker-compose as a container.
See https://docs.docker.com/compose/install/ → "Alternative Install Options".

sudo curl -L --fail https://github.com/docker/compose/releases/download/1.26.0/run.sh -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

docker-compose --version

ghost added the kind/bug label Oct 11, 2019

glours added the status/0-triage label Oct 11, 2019

ndeloof added kind/question and removed kind/bug labels Oct 11, 2019

ndeloof added kind/bug and removed kind/question labels Oct 14, 2019

ndeloof mentioned this issue Oct 14, 2019

Docker-compose doesn't work on MacOS Catalina #6963

Closed

chris-crone added this to the 1.25.0 milestone Oct 31, 2019

ndeloof mentioned this issue Nov 8, 2019

Build OSX binary as a directory #7010

Merged

ulyssessouza modified the milestones: 1.25.0, 1.25.1 Nov 18, 2019

ghost changed the title ~~docker-compose ( for macOS Catalina )~~ docker-compose super slow on macOS Catalina Nov 19, 2019

This was referenced Nov 20, 2019

Diagnose/improve execution speed on macOS Catalina, for #1881 ddev/ddev#1962

Merged

macOS Catalina: ddev command execution seems sluggish ddev/ddev#1881

Closed

ulyssessouza closed this as completed in #7010 Nov 22, 2019

rizplate mentioned this issue Dec 8, 2019

Docker compose very slow on mac catalina #7087

Closed

yqmmm mentioned this issue Jan 18, 2020

borg init --help extremely slow borgbackup/borg#4859

Closed

docker-compose super slow on macOS Catalina #6956

docker-compose super slow on macOS Catalina #6956

Comments

ghost commented Oct 11, 2019

glours commented Oct 11, 2019

ghost commented Oct 11, 2019

ghost commented Oct 11, 2019

ndeloof commented Oct 11, 2019

skazancev commented Oct 11, 2019

ghost commented Oct 11, 2019

glours commented Oct 11, 2019

ghost commented Oct 11, 2019

ghost commented Oct 11, 2019 • edited by ghost

carlosonunez commented Oct 11, 2019 • edited

carlosonunez commented Oct 11, 2019

ghost commented Oct 11, 2019

carlosonunez commented Oct 11, 2019

wodCZ commented Oct 14, 2019 • edited

diogocnunes commented Oct 15, 2019 • edited

ndeloof commented Oct 15, 2019 • edited

ghost commented Oct 15, 2019

carlosonunez commented Oct 15, 2019 • edited

diogocnunes commented Oct 15, 2019

diogocnunes commented Oct 15, 2019

ndeloof commented Oct 15, 2019

justinpage commented Oct 23, 2019

ghost commented Oct 29, 2019

diogocnunes commented Oct 30, 2019

mediaessenz commented Oct 31, 2019

kliyes commented Nov 9, 2019

rfay commented Nov 19, 2019

rfay commented Nov 19, 2019

mayconmfl commented Nov 20, 2019

wodCZ commented Dec 2, 2019

ndeloof commented Dec 2, 2019

zeshan77 commented Dec 3, 2019

ndeloof commented Dec 3, 2019

wodCZ commented Dec 3, 2019

BirkhoffLee commented Dec 4, 2019

ndeloof commented Dec 4, 2019

Yelfive commented Jan 10, 2020 • edited

ulyssessouza commented Jan 10, 2020

Yelfive commented Jan 11, 2020

SteffenAnders commented Jul 2, 2020

ghost commented Oct 11, 2019 •

edited by ghost

carlosonunez commented Oct 11, 2019 •

edited

wodCZ commented Oct 14, 2019 •

edited

diogocnunes commented Oct 15, 2019 •

edited

ndeloof commented Oct 15, 2019 •

edited

carlosonunez commented Oct 15, 2019 •

edited

Yelfive commented Jan 10, 2020 •

edited