Docker for Mac + Symfony too slow

[antondachauer]

Issues without logs and details cannot be exploited, and will be closed.

• I have tried with the latest version of my channel (Stable or Edge)
• I have submitted Diagnostics
• I have included the Diagnostics ID in this Issue

Expected behavior

Page loading time < 10sec (Symfony dev env)
Page loading time < 3sec (Symfony prod env)

Actual behavior

Page loading time > 40sec (dev)

Information

• Full output of the diagnostics from "Diagnose & Feedback" in the menu

Docker for Mac: version: 18.03.0-ce-rc4-mac57 (dd2831d4b7421cf559a0881cc7a5fdebeb8c2b98)
macOS: version 10.13.3 (build: 17D102)
logs: /tmp/6FBF3F6D-C34A-4A3A-B732-251EFAE7E458/20180320-151751.tar.gz
[OK]     vpnkit
[OK]     vmnetd
[OK]     dns
[OK]     driver.amd64-linux
[OK]     virtualization VT-X
[OK]     app
[OK]     moby
[OK]     system
[OK]     moby-syslog
[OK]     kubernetes
[OK]     files
[OK]     env
[OK]     virtualization kern.hv_support
[OK]     osxfs
[OK]     moby-console
[OK]     logs
[OK]     docker-cli
[OK]     disk

Steps to reproduce the behavior

docker-compose.yml

version: '2'

services:
    db:
        image: mysql
        volumes:
            - "./.data/db:/var/lib/mysql"
        environment:
            MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
            MYSQL_DATABASE: ${MYSQL_DATABASE}
            MYSQL_USER: ${MYSQL_USER}
            MYSQL_PASSWORD: ${MYSQL_PASSWORD}
    php:
        build:
            context: php7-fpm
            args:
                TIMEZONE: ${TIMEZONE}
        volumes:
            - ../:/var/www/symfony
            - ../documents:/var/www/symfony/documents
    nginx:
        build: nginx
        ports:
            - 80:80
        volumes_from:
            - php
        volumes:
            - ./logs/nginx/:/var/log/nginx
    elk:
        image: willdurand/elk
        ports:
            - 81:80
        volumes:
            - ./elk/logstash:/etc/logstash
            - ./elk/logstash/patterns:/opt/logstash/patterns
        volumes_from:
            - php
            - nginx

php7-fpm Dockerfile

See https://github.com/docker-library/php/blob/master/7.1/fpm/Dockerfile

FROM php:7.1-fpm
ARG TIMEZONE

MAINTAINER Maxence POUTORD <maxence.poutord@gmail.com>

RUN apt-get update && apt-get install -y \
    openssl \
    git \
    unzip

# Install Composer
RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
RUN composer --version

# Set timezone
RUN ln -snf /usr/share/zoneinfo/${TIMEZONE} /etc/localtime && echo ${TIMEZONE} > /etc/timezone
RUN printf '[PHP]\ndate.timezone = "%s"\n', ${TIMEZONE} > /usr/local/etc/php/conf.d/tzone.ini
RUN "date"

# Type docker-php-ext-install to see available extensions
RUN docker-php-ext-install pdo pdo_mysql

# install xdebug
RUN pecl install xdebug
RUN docker-php-ext-enable xdebug
RUN echo "error_reporting = E_ALL" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini
RUN echo "display_startup_errors = On" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini
RUN echo "display_errors = On" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini
RUN echo "xdebug.remote_enable=1" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini
RUN echo "xdebug.remote_connect_back=1" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini
RUN echo "xdebug.idekey=\"PHPSTORM\"" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini
RUN echo "xdebug.remote_port=9001" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini


RUN echo 'alias sf="php app/console"' >> ~/.bashrc
RUN echo 'alias sf3="php bin/console"' >> ~/.bashrc

WORKDIR /var/www/symfony

nginx Dockerfile

FROM debian:jessie

MAINTAINER Maxence POUTORD <maxence.poutord@gmail.com>

RUN apt-get update && apt-get install -y \
    nginx \
    wkhtmltopdf

ADD nginx.conf /etc/nginx/
ADD symfony.conf /etc/nginx/sites-available/

RUN ln -s /etc/nginx/sites-available/symfony.conf /etc/nginx/sites-enabled/symfony
RUN rm /etc/nginx/sites-enabled/default

RUN echo "upstream php-upstream { server devenv_php_1:9000; }" > /etc/nginx/conf.d/upstream.conf

RUN usermod -u 1000 www-data && groupmod -g 1000 www-data

CMD ["nginx"]

EXPOSE 80
EXPOSE 443

[djs55]

@antondachauer thanks for your report.

I tried to reproduce the build but I think the config files are missing:

Removing intermediate container 0b9e67707d26
 ---> 9915b175f1aa
Step 4/12 : ADD nginx.conf /etc/nginx/
ERROR: Service 'nginx' failed to build: ADD failed: stat /var/lib/docker/tmp/docker-builder232539285/nginx.conf: no such file or directory

[antondachauer]

It needs some nginx config files, please see the Dockerfile part.

[barat]

Your problem is commonly known more then a year ... it's because bind-mounts are terribly slow.
Make an experiment and instead of volumes: just CP your project files into container via dockerfile and remove volume bind mounts.
There are workarounds like docker-sync, but "technology is not there yet" for native fix ;)

[antondachauer]

That sounds like docker missed his mission on Mac. Will the problem eventually be solved with a native solution?

[alpharder]

Use the cached or delegated flags for volume mounting. Also, using docker volume create-volumes for your DB persistent data will also give you a boost in speed.

[kkarkus]

still no native solution? cached or delegated flags for volume mounting aren't even close to native solution on linux

[djs55]

I recommend following @alpharder's advice -- first try with :cached or :delegated and if that's not fast enough, use docker volume create to put the persistent data inside the VM for maximum performance.

If docker volume create doesn't work for you because you want to be able to still edit files on the host, we have some experimental prototypes you can try. One of them adds more caching to :delegated https://download-stage.docker.com/mac/pr/24606/Docker.dmg

Please note:

• since it's an experimental build don't use it for production :-)
• if it offers an auto-update, don't accept or you could end up on a build without the feature
• if you share the same directory with and without :delegated, the results won't be coherent-- for example a write a file in a :delegated mount will write to the cache in the VM and the cache won't be flushed to the host for a few seconds

If you try the experimental build I'd love to know

• whether this makes any difference or not to your use case (and if not, more details of your use case)
• whether you discover any bugs (and if so, how to reproduce them)

[kkarkus]

@djs55
I've just tested this edge version and it's seem to work pretty good.
Later I'm going to compare stable version and this edge version.

[darkbluesun]

I've tested the edge version (in combination with some docker volumes and some delegated volumes), and it seems to work fine, and perhaps makes a slight improvement to performance, but again, nothing like running the services natively 😞
(almost exactly the same docker-compose setup as op)

[borNfreee]

if that's not fast enough, use docker volume create to put the persistent data inside the VM for maximum performance.

could you advise any links to read about why it is more performant and about "inside the VM"?

[Blackskyliner]

You can observe the 'why' part. Open up your Activity Monitor. Run your SF Dockerized App and access it. Now watch the OSXFS process consuming most CPU time in conjunction with the hypervisor running your docker machine host. As the Hypervisor will ask OSXFS for the file and OSXFS will look into the filesystem for the file and then send it to the VM - for each and every read - for each and every write - each and every attribute check like ctime or atime.

The writing scenarios are somehow better through delegated and cached behaviors but the real big problem are the reads which - afaik always - go through the OSXFS layer in a bind mount setting, which is the real bottleneck. (Correct me if I am wrong, that are real observed behaviors in huge SF applications running on docker for mac for development)

In SF and Composer times this is multiple thousand files getting accessed through that combination. resulting in page load times of 30+ seconds. (You will see in the dev-toolbar the pre-framework initialisation time is a huge part of that time)

A Volume instead would not have to go through that layer, thus resulting in blazingly fast - almost native - performance. Thats why projects like docker-sync exist which use that mechanism to trickster a nearly native performance setup (with some delay after editing files locally until synced).

PS: Small SF Applications can be made a bit faster by Volume Mount the 'vendor', 'node_modules' and 'var' folders. Thus having at least all vendors and cache not going through the OSXFS layer. However this at some point will not suffice as the reads on 'src' will kill the performance again at a certain project size. (Small Hello World SF Applications will run relatively fast though)

[antondachauer]

Today I tried again to get Symfony running for development with the current Docker version. It seems to be working quite well now. You can find the configuration here: https://github.com/webworksnbg/sfdocker

My Docker version is 18.06.0-ce, build 0ffa825

[docker-desktop-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30d of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

[wissem]

@antondachauer thanks a lot for sharing :) with that same build docker for mac + symfony are fast now, no need for docker-sync anymore

[Blackskyliner]

Just until the application reaches a complexity threshold and everything is slow again...

I guess if you only got 4GB or RAM for docker (because 8GB installed and IDE takes 2GB and Browser 1GB and rest of the system the last 1GB because mac) you will hit the same degraded performance again, because as far as I get it/would guess the osxfs delegated caching only works "until full". I had not debugged that for now though. But the delegated only fixed the problem for 16GB+ systems with at least 6-8GB for docker as far as I can observe at work. Fresh reboot and sometimes only after a full-reset of the docker environment you will get decent speed in small bootstrapped applications. Load a really complex "live-bussiness-level with hundreds of tests" one and performance will degrade in no time back to multiple seconds of wait time per request.

I may could provide anything (as long as anonymized) on the diagnostic side, if needed to find the culprit. But in some caeses its currently unusable for some of my colleagues to use it in every-day development as it will slow them more down than just using the central sftp development machine running the needed LAMP stack.

/remove-lifecycle stale