Starting multiple services(with Compose) ends in a compose.parallel.feed_queue loop

[maxehmookau]

I'm trying to run docker-compose up on a project I've used hundreds of times before. Except now starting the containers takes up to half an hour just to get running and then the web server inside the containers runs very very slowly when it does work.

Running docker-compose --verbose up gives me a loop of this thousands of times:

compose.parallel.feed_queue: Pending: set([])

However, when I specify a single service inside the config file to start, this works as fast as normal. The issue only occurs when attempting to run multiple containers.

Using docker on its own in the CLI works as before.

Diagnostic ID: 66FA4AD4-2FBC-486C-8FA7-200F103D9222

I'm running Docker 17.09.0-ce-mac35, and docker-compose version 1.16.1, build 6d1ac21 and MacOS High Sierra.

I've attempted to reinstall, rebuild and using docker edge to no avail :(

I initially opened this ticket in the docker-compose repo but was told to repost here.
docker/compose#5257

[padusumilli]

Same thing here, tried reinstall stable/edge version but same sluggish performance to the point where it's barely usable.

Docker on MacOS High Sierra
Version 17.09.0-ce-mac34 (19605)
Channel: edge
c076099f52

[maxehmookau]

I should also add, when starting my development environment using docker-compose if I start each service separately

docker-compose start web
docker-compose start worker
docker-compose start db
etc...

Everything works fine and very quickly. (Something to try @padusumilli while a solution is figured out!)

[maxehmookau]

I've also just reimaged my entire computer and wiped everything and just installed nothing else except Docker. The issue is persisting across all of my projects.

I'm totally lost where to go to solve this, please help!

[maxehmookau]

After some more narrowing down I believe the issue is to do with native docker on mac specifically.
I've moved back to using docker toolbox/virtualbox/docker-sync for development and it's really fast.

[padusumilli]

I should also add, when starting my development environment using docker-compose if I start each service separately

docker-compose start web
docker-compose start worker
docker-compose start db
etc...
Everything works fine and very quickly. (Something to try @padusumilli while a solution is figured out!)

@maxehmookau, thanks this work around works!
Ended up with a script that runs the below command on all the services of the compose file

docker-compose create <service-name> && docker-compose start <service-name>

[maxehmookau]

Hey @jasonbivins has there been any progress on this? What's the usual timeframe of something like this? I think I've managed to rule out anything on my system that might be causing the issue.

[jasonbivins]

HI @maxehmookau would you be able to share your dockerfile? I'd like to test here on my system if possible

[maxehmookau]

Sure, we're not actually using a Dockerfile. All of the images in our projects use library images. It's happening with all projects that have multiple services on compose. This only affects High Sierra users too.

version: '2'
services:
  redis:
    image: redis
  db:
    image: postgres
    volumes:
      - "./.data/db:/var/lib/postgresql"
  web:
    build: .
    command: bundle exec puma -C config/puma.rb
    environment:
      RAILS_ENV: development
      PORT: 3000
    volumes:
      - .:/opt/substrakt/project
      - ~/.ssh:/root/.ssh
    ports:
      - "3000:3000"
    depends_on:
      - db
      - redis
  sidekiq:
    build: .
    volumes:
      - .:/opt/substrakt/project
    links:
      - db
      - redis
    command: bundle exec sidekiq -C config/sidekiq.yml
  clock:
    build: .
    volumes:
      - .:/opt/substrakt/project
    links:
      - db
      - redis
    command: bundle exec clockwork clock.rb
  mailcatcher:
    image: schickling/mailcatcher
    ports:
      - 1025:1025
      - 1080:1080

[maxehmookau]

Sorry, I was wrong, we do use a Dockerfile (clearly!). Too early in the morning!

FROM ruby:2.3.1
MAINTAINER Max Woolf <max@substrakt.com>

WORKDIR /opt/substrakt/project
COPY Gemfile /opt/substrakt/project
COPY Gemfile.lock /opt/substrakt/project
RUN bundle install -j4
COPY . /opt/substrakt/project

[maxehmookau]

Has there been any update on this? I've seen a few people on Twitter and other forums having similar issues.

I've downgraded my iMac to Sierra for now and will keep it there for the forseeable...

[MatthewMi11er]

I am also experiencing this issue:

Using docker-compose up on this docker-compose.yml takes about 1 second:

version: '3.1'
services:
  n1:
    image: nginx:mainline-alpine

Using docker-compose up on this docker-compose.yml takes about 1 minute:

version: '3.1'
services:
  n1:
    image: nginx:mainline-alpine
  n2:
    image: nginx:mainline-alpine

Using docker-compose up on this docker-compose.yml takes about 3 minutes:

version: '3.1'
services:
  n1:
    image: nginx:mainline-alpine
  n2:
    image: nginx:mainline-alpine
  n3:
    image: nginx:mainline-alpine

Docker-compose down also takes an extended time when there are multiple services.

[leonboot]

Found this thread after digging around on the web, Googling for people experiencing this issue. I upgraded to High Sierra today and boy, did I regret it. For what it's worth, here's my two cents:

I've figured out, like @maxehmookau, that starting the containers separately by running docker-compose up [container], the startup is as quick as you'd expect it. However, starting them by simply running docker-compose up, it takes substantially longer. Bringing the containers down is slow as well, while doing a docker-compose ps seems fairly quick. Adding the --verbose flag to the up statement reveals a loop of these lines, like @maxehmookau mentioned:

compose.parallel.feed_queue: Pending: set([])
compose.parallel.feed_queue: Pending: set([<Service: someservice>])

I really hope this bug is squashed soon, downgrading to Sierra doesn't really sound that appealing to me!

Oh, for a quick test, check out my proof-of-concept repo for a project of mine which consists of a simple docker project with three services.

And for what it's worth: my diagnostic ID: 0841F00D-0E6B-4CE1-8F82-AFCECB894AE6

[MatthewMi11er]

For me the issue appears to be network related, but I'm not sure how. Disconnecting the host machine from the network clears the issue.

[leonboot]

@MatthewMi11er Wow, you're right. Yanked out my UTP cable and the docker-compose up was blazing fast. Now there's an interesting workflow. 1) yank out cable, 2) run docker-compose up 3) reconnect.

[MatthewMi11er]

@leonboot also, try a different network.

[MatthewMi11er]

After further digging, my issues appears to be related to docker/compose#3419

Adding 127.0.0.1 localunixsocket to my /etc/hosts also seems to clear the issue.

[aracanid]

Can also confirm that removing my network cable resolved the issue. I Haven't attempted the resolution suggested by MatthewMi11er yet.

[mandeepmehra]

Both worked for me

1. Adding 127.0.0.1 localunixsocket to my /etc/hosts or
2. Removing network cable

[jorihardman]

I've found I have the same loop:

compose.parallel.feed_queue: Pending: set([])
compose.parallel.feed_queue: Pending: set([<Service: someservice>])

when doing docker-compose pull for any image. For me, it seems to just be a problem with parallel pull. Using docker-compose pull --no-parallel downloads the image just fine.

[bcoughlan]

@jorihardman If you're pulling a lot of images it might be related to #5864

[docker-robott]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30d of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

[Sieboldianus]

For me the issue appears to be network related, but I'm not sure how. Disconnecting the host machine from the network clears the issue.

Thanks! For me it was also related to a VPN connection being active in the background..

[aravindravva]

i am stucked with issue any help apperciated

[docker-robott]

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle locked