New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker for Mac + Symfony too slow #2707

Open
antondachauer opened this Issue Mar 20, 2018 · 15 comments

Comments

Projects
None yet
10 participants
@antondachauer
Copy link

antondachauer commented Mar 20, 2018

Issues without logs and details cannot be exploited, and will be closed.

  • I have tried with the latest version of my channel (Stable or Edge)
  • I have submitted Diagnostics
  • I have included the Diagnostics ID in this Issue

Expected behavior

Page loading time < 10sec (Symfony dev env)
Page loading time < 3sec (Symfony prod env)

Actual behavior

Page loading time > 40sec (dev)

Information

  • Full output of the diagnostics from "Diagnose & Feedback" in the menu
Docker for Mac: version: 18.03.0-ce-rc4-mac57 (dd2831d4b7421cf559a0881cc7a5fdebeb8c2b98)
macOS: version 10.13.3 (build: 17D102)
logs: /tmp/6FBF3F6D-C34A-4A3A-B732-251EFAE7E458/20180320-151751.tar.gz
[OK]     vpnkit
[OK]     vmnetd
[OK]     dns
[OK]     driver.amd64-linux
[OK]     virtualization VT-X
[OK]     app
[OK]     moby
[OK]     system
[OK]     moby-syslog
[OK]     kubernetes
[OK]     files
[OK]     env
[OK]     virtualization kern.hv_support
[OK]     osxfs
[OK]     moby-console
[OK]     logs
[OK]     docker-cli
[OK]     disk

Steps to reproduce the behavior

docker-compose.yml

version: '2'

services:
    db:
        image: mysql
        volumes:
            - "./.data/db:/var/lib/mysql"
        environment:
            MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
            MYSQL_DATABASE: ${MYSQL_DATABASE}
            MYSQL_USER: ${MYSQL_USER}
            MYSQL_PASSWORD: ${MYSQL_PASSWORD}
    php:
        build:
            context: php7-fpm
            args:
                TIMEZONE: ${TIMEZONE}
        volumes:
            - ../:/var/www/symfony
            - ../documents:/var/www/symfony/documents
    nginx:
        build: nginx
        ports:
            - 80:80
        volumes_from:
            - php
        volumes:
            - ./logs/nginx/:/var/log/nginx
    elk:
        image: willdurand/elk
        ports:
            - 81:80
        volumes:
            - ./elk/logstash:/etc/logstash
            - ./elk/logstash/patterns:/opt/logstash/patterns
        volumes_from:
            - php
            - nginx

php7-fpm Dockerfile

See https://github.com/docker-library/php/blob/master/7.1/fpm/Dockerfile

FROM php:7.1-fpm
ARG TIMEZONE

MAINTAINER Maxence POUTORD <maxence.poutord@gmail.com>

RUN apt-get update && apt-get install -y \
    openssl \
    git \
    unzip

# Install Composer
RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
RUN composer --version

# Set timezone
RUN ln -snf /usr/share/zoneinfo/${TIMEZONE} /etc/localtime && echo ${TIMEZONE} > /etc/timezone
RUN printf '[PHP]\ndate.timezone = "%s"\n', ${TIMEZONE} > /usr/local/etc/php/conf.d/tzone.ini
RUN "date"

# Type docker-php-ext-install to see available extensions
RUN docker-php-ext-install pdo pdo_mysql

# install xdebug
RUN pecl install xdebug
RUN docker-php-ext-enable xdebug
RUN echo "error_reporting = E_ALL" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini
RUN echo "display_startup_errors = On" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini
RUN echo "display_errors = On" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini
RUN echo "xdebug.remote_enable=1" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini
RUN echo "xdebug.remote_connect_back=1" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini
RUN echo "xdebug.idekey=\"PHPSTORM\"" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini
RUN echo "xdebug.remote_port=9001" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini


RUN echo 'alias sf="php app/console"' >> ~/.bashrc
RUN echo 'alias sf3="php bin/console"' >> ~/.bashrc

WORKDIR /var/www/symfony

nginx Dockerfile

FROM debian:jessie

MAINTAINER Maxence POUTORD <maxence.poutord@gmail.com>

RUN apt-get update && apt-get install -y \
    nginx \
    wkhtmltopdf

ADD nginx.conf /etc/nginx/
ADD symfony.conf /etc/nginx/sites-available/

RUN ln -s /etc/nginx/sites-available/symfony.conf /etc/nginx/sites-enabled/symfony
RUN rm /etc/nginx/sites-enabled/default

RUN echo "upstream php-upstream { server devenv_php_1:9000; }" > /etc/nginx/conf.d/upstream.conf

RUN usermod -u 1000 www-data && groupmod -g 1000 www-data

CMD ["nginx"]

EXPOSE 80
EXPOSE 443
@djs55

This comment has been minimized.

Copy link
Contributor

djs55 commented Mar 20, 2018

@antondachauer thanks for your report.

I tried to reproduce the build but I think the config files are missing:

Removing intermediate container 0b9e67707d26
 ---> 9915b175f1aa
Step 4/12 : ADD nginx.conf /etc/nginx/
ERROR: Service 'nginx' failed to build: ADD failed: stat /var/lib/docker/tmp/docker-builder232539285/nginx.conf: no such file or directory
@antondachauer

This comment has been minimized.

Copy link

antondachauer commented Mar 22, 2018

It needs some nginx config files, please see the Dockerfile part.

@barat

This comment has been minimized.

Copy link

barat commented Mar 27, 2018

Your problem is commonly known more then a year ... it's because bind-mounts are terribly slow.
Make an experiment and instead of volumes: just CP your project files into container via dockerfile and remove volume bind mounts.
There are workarounds like docker-sync, but "technology is not there yet" for native fix ;)

@antondachauer

This comment has been minimized.

Copy link

antondachauer commented Mar 27, 2018

That sounds like docker missed his mission on Mac. Will the problem eventually be solved with a native solution?

@alpharder

This comment has been minimized.

Copy link

alpharder commented Mar 30, 2018

Use the cached or delegated flags for volume mounting. Also, using docker volume create-volumes for your DB persistent data will also give you a boost in speed.

@kkarkus

This comment has been minimized.

Copy link

kkarkus commented Jun 24, 2018

still no native solution? cached or delegated flags for volume mounting aren't even close to native solution on linux

@djs55

This comment has been minimized.

Copy link
Contributor

djs55 commented Jun 25, 2018

I recommend following @alpharder's advice -- first try with :cached or :delegated and if that's not fast enough, use docker volume create to put the persistent data inside the VM for maximum performance.

If docker volume create doesn't work for you because you want to be able to still edit files on the host, we have some experimental prototypes you can try. One of them adds more caching to :delegated https://download-stage.docker.com/mac/pr/24606/Docker.dmg

Please note:

  • since it's an experimental build don't use it for production :-)
  • if it offers an auto-update, don't accept or you could end up on a build without the feature
  • if you share the same directory with and without :delegated, the results won't be coherent-- for example a write a file in a :delegated mount will write to the cache in the VM and the cache won't be flushed to the host for a few seconds

If you try the experimental build I'd love to know

  • whether this makes any difference or not to your use case (and if not, more details of your use case)
  • whether you discover any bugs (and if so, how to reproduce them)
@kkarkus

This comment has been minimized.

Copy link

kkarkus commented Jun 25, 2018

@djs55
I've just tested this edge version and it's seem to work pretty good.
Later I'm going to compare stable version and this edge version.

@darkbluesun

This comment has been minimized.

Copy link

darkbluesun commented Jun 27, 2018

I've tested the edge version (in combination with some docker volumes and some delegated volumes), and it seems to work fine, and perhaps makes a slight improvement to performance, but again, nothing like running the services natively 😞
(almost exactly the same docker-compose setup as op)

@borNfreee

This comment has been minimized.

Copy link

borNfreee commented Jul 2, 2018

if that's not fast enough, use docker volume create to put the persistent data inside the VM for maximum performance.

could you advise any links to read about why it is more performant and about "inside the VM"?

@Blackskyliner

This comment has been minimized.

Copy link

Blackskyliner commented Aug 6, 2018

You can observe the 'why' part. Open up your Activity Monitor. Run your SF Dockerized App and access it. Now watch the OSXFS process consuming most CPU time in conjunction with the hypervisor running your docker machine host. As the Hypervisor will ask OSXFS for the file and OSXFS will look into the filesystem for the file and then send it to the VM - for each and every read - for each and every write - each and every attribute check like ctime or atime.

The writing scenarios are somehow better through delegated and cached behaviors but the real big problem are the reads which - afaik always - go through the OSXFS layer in a bind mount setting, which is the real bottleneck. (Correct me if I am wrong, that are real observed behaviors in huge SF applications running on docker for mac for development)

In SF and Composer times this is multiple thousand files getting accessed through that combination. resulting in page load times of 30+ seconds. (You will see in the dev-toolbar the pre-framework initialisation time is a huge part of that time)

A Volume instead would not have to go through that layer, thus resulting in blazingly fast - almost native - performance. Thats why projects like docker-sync exist which use that mechanism to trickster a nearly native performance setup (with some delay after editing files locally until synced).

PS: Small SF Applications can be made a bit faster by Volume Mount the 'vendor', 'node_modules' and 'var' folders. Thus having at least all vendors and cache not going through the OSXFS layer. However this at some point will not suffice as the reads on 'src' will kill the performance again at a certain project size. (Small Hello World SF Applications will run relatively fast though)

@antondachauer

This comment has been minimized.

Copy link

antondachauer commented Aug 25, 2018

Today I tried again to get Symfony running for development with the current Docker version. It seems to be working quite well now. You can find the configuration here: https://github.com/webworksnbg/sfdocker

grafik

My Docker version is 18.06.0-ce, build 0ffa825

@docker-desktop-robot

This comment has been minimized.

Copy link
Collaborator

docker-desktop-robot commented Nov 23, 2018

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30d of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

@wissem

This comment has been minimized.

Copy link

wissem commented Nov 24, 2018

@antondachauer thanks a lot for sharing :) with that same build docker for mac + symfony are fast now, no need for docker-sync anymore

@Blackskyliner

This comment has been minimized.

Copy link

Blackskyliner commented Jan 17, 2019

Just until the application reaches a complexity threshold and everything is slow again...

I guess if you only got 4GB or RAM for docker (because 8GB installed and IDE takes 2GB and Browser 1GB and rest of the system the last 1GB because mac) you will hit the same degraded performance again, because as far as I get it/would guess the osxfs delegated caching only works "until full". I had not debugged that for now though. But the delegated only fixed the problem for 16GB+ systems with at least 6-8GB for docker as far as I can observe at work. Fresh reboot and sometimes only after a full-reset of the docker environment you will get decent speed in small bootstrapped applications. Load a really complex "live-bussiness-level with hundreds of tests" one and performance will degrade in no time back to multiple seconds of wait time per request.

I may could provide anything (as long as anonymized) on the diagnostic side, if needed to find the culprit. But in some caeses its currently unusable for some of my colleagues to use it in every-day development as it will slow them more down than just using the central sftp development machine running the needed LAMP stack.

/remove-lifecycle stale

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment