Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File access in mounted volumes extremely slow #77

Open
schmunk42 opened this issue Aug 2, 2016 · 212 comments
Open

File access in mounted volumes extremely slow #77

schmunk42 opened this issue Aug 2, 2016 · 212 comments

Comments

@schmunk42
Copy link

@schmunk42 schmunk42 commented Aug 2, 2016

continued from https://forums.docker.com/t/file-access-in-mounted-volumes-extremely-slow-cpu-bound/8076/178

Expected behavior

File-system performance in the container not significantly slower than on the host or docker-machine.

Actual behavior

Running a composer (PHP) update in a container takes much longer than on the host. In the container we usually hit timeouts when updating huge git repos like twbs/bootstrap.

Information & Steps to reproduce the behavior

See link above

@Vanuan
Copy link

@Vanuan Vanuan commented Sep 4, 2016

Same for npm install, bundle install, browserify, etc.

@MartialGeek
Copy link

@MartialGeek MartialGeek commented Sep 5, 2016

Hi

Ok, that's why a Symfony application is very slow in a Docker container!
I load a page in ~ 150 ms when I'm running my local Symfony server (on my host) VS ~ 2000 ms when the config files have been serialized in the cache directory within a Docker container.
It seems the IO in the vendors and the cache / logs directories are very slow...

@lfv89
Copy link

@lfv89 lfv89 commented Sep 6, 2016

Same here with rails and its friends (rails, rake, rspec...)

Please let me know what information I can provide to help you guys out.

@mduller
Copy link

@mduller mduller commented Sep 7, 2016

Same here as well. We're looking into using IntelliJ IDEA and git on the Mac and compiling & running our product in a container, from the shared filesystem. Development cycles are considerably longer and our devs not happy with the proposed switch. Currently evaluating workarounds to still enable the switch to Docker and in particular Docker for Mac.

@wadjeroudi
Copy link

@wadjeroudi wadjeroudi commented Sep 7, 2016

For those who look for a workaround and didn't read the whole thread, here is the solution :
https://github.com/EugenMayer/docker-sync

@ZZromanZZ
Copy link

@ZZromanZZ ZZromanZZ commented Sep 7, 2016

Same with PHP (Symfony, Nette, Composer deps...). I can confirm https://github.com/EugenMayer/docker-sync is usable workaround.

@iwaffles
Copy link

@iwaffles iwaffles commented Sep 8, 2016

We're experiencing this as well. We have our API dockerized and sometimes endpoints time out on our local machines (only in docker for mac). We're using Rails 5.

@MartialGeek
Copy link

@MartialGeek MartialGeek commented Sep 9, 2016

In my team, some developers use a Mac, others use Linux... Docker sync is not the solution for me because the project configuration will not be the same for all environments. I hope that the OSXFS issues will be fixed quickly.

@wadjeroudi
Copy link

@wadjeroudi wadjeroudi commented Sep 9, 2016

@MartialGeek Off topic : https://github.com/EugenMayer/docker-sync/wiki/Keep-you-docker-compose.yml-portable
But you should create a makefile to handle the environment properly.

@MartialGeek
Copy link

@MartialGeek MartialGeek commented Sep 9, 2016

@wadjeroudi Thank you, I will read that ASAP ;)

@petergombos
Copy link

@petergombos petergombos commented Sep 13, 2016

Same issue here, takes +30 sec to run my babel build using a mounted volume vs. 5 sec using container's fs.

@jsamouh
Copy link

@jsamouh jsamouh commented Sep 14, 2016

👍 Same issue with Git in docker, very slow ! Unusable compare to Docker Machine (with NFS)
Docker For Mac released too fast ! Need to be reviews !!!

@jzahedieh
Copy link

@jzahedieh jzahedieh commented Sep 14, 2016

Whole team is having issues with Magento, 3 minute page loads vs 8 seconds using docker-sync

@BenMcH
Copy link

@BenMcH BenMcH commented Sep 15, 2016

I'm also experiencing this issue. Hitting the cache takes no less than 1 second for each file.

@apahne
Copy link

@apahne apahne commented Sep 17, 2016

Will this issue eventually be addressed?

@Vanuan
Copy link

@Vanuan Vanuan commented Sep 17, 2016

Here's the latest reply from Docker Team:
https://forums.docker.com/t/file-access-in-mounted-volumes-extremely-slow-cpu-bound/8076/158

It's been quite a while since then. We all hope that they have something new to share.

@joemewes
Copy link

@joemewes joemewes commented Sep 17, 2016

Just checked on latest Docker for Mac update : Version 1.12.1 (build: 12133)

seems to be still an issue.

@so0k
Copy link

@so0k so0k commented Sep 19, 2016

Latest posts on the alternatives are interesting - https://forums.docker.com/t/alternatives-to-osxfs-performant-shares-under-osx/19565/14 - which points to https://github.com/cweagans/docker-bg-sync (minimal changes to compose file for developer allow a significant speed improvement for mounted source files)

@joemewes
Copy link

@joemewes joemewes commented Sep 19, 2016

Yeah, I've gone the UNISON approach (also mentioned a couple times in the original thread) : see here https://github.com/4AllDigital/DockerDrupal-lite/blob/master/docker-compose.yml#L7

using this container : https://github.com/4AllDigital/drupaldev-unison

if anyone needs an example to copy and want to use UNISON too.

@piotrekkaminski
Copy link

@piotrekkaminski piotrekkaminski commented Sep 21, 2016

Same issue with Magento. Major performance issues. We are moving to Unison now as well.

@BenMcH
Copy link

@BenMcH BenMcH commented Sep 22, 2016

I just got around to testing the bg-sync workaround with our in-house rails app and it worked perfectly! Thanks for the suggestion @so0k

@jeanlaurent
Copy link
Member

@jeanlaurent jeanlaurent commented Apr 13, 2017

@aLkRicha yes, it's called edge now, and only starting on Docker 17.04 CE. See the FAQ entry on Stable and Edge if you want to switch.

@Vanuan
Copy link

@Vanuan Vanuan commented Apr 14, 2017

@barat I think currently it should be like this

  volumes:
    - .:/symfony:ro,cached
    - app_cache_volume:/symfony/app/cache
    - vendor_volume:/symfony/vendor
    - logs_volume:/symfony/app/logs

volumes:
  app_cache_volume:
  vendor_volume:
  logs_volume:

Yes, if you need to look into vendor files in your IDE you should wait for delegated.

For now you can use duplicate vendor in both docker and out of docker with manual synchronization. So that PHPStorm uses local version and docker uses volume in VM.

@norio-nomura
Copy link

@norio-nomura norio-nomura commented Apr 15, 2017

I tried :cached on building Swift toolchain for Linux that source codes are placed in mac volume.
It seems that it certainly got faster, but the memory consumption of osxfs was so great that it affected other applications.
I will continue to use the combination of docker-machine and nfs.

@andrerom
Copy link

@andrerom andrerom commented Apr 18, 2017

@andrerom I understand that this is a new feature, but what about people that want to support development on multiple OSes. RHEL 6 and 7 are going to be around for a long time still, so you're saying people will need to keep multiple configs around for years just so :cached can be specified on their Macs?

@daveisfera Well, yes and no. Docker is despite all the hype pushing for it's use early adopter stuff, it is currently not mature enough for early majority users to use it broadly yet, has some known stability issues and a fragmented market. RHEL 6 is even late majority users, they by nature don't want to test out new things, so don't make them. RHEL 7 however is supported by docker packages, so once they migrate there they can take advantage of this. In other words research usage of docker for future version of your own software, not for older versions. At least not under all possible supported platforms you have there, that won't blend.

@asteinlein
Copy link

@asteinlein asteinlein commented Apr 18, 2017

I got unknown option: cached when using this in a compose file when deploying to a stack (docker stack deploy foo --compose-file=docker-compose.yml (I'm on 17.06.0-ce-rc1-mac8 edge). When will that be supported?

@andrerom
Copy link

@andrerom andrerom commented Apr 18, 2017

@asteinlein did you set version to 3.2?`, at least that is the thing needed on docker-compose.


For PHP/Symfony users posted numbers for :cached with eZ Platform. TL;DR works, but for any heavy REST based app doing parallel requests needed to set SYMFONY_ENV=prod.

@toooni
Copy link

@toooni toooni commented Apr 18, 2017

@andrerom you may just change the cache and log dir in AppKernel.php to a non shareded dir (for dev):

    public function getCacheDir()
    {
        if (in_array($this->environment, ['dev', 'test'])) {
            return '/var/www/symfony/cache/'.$this->environment;
        }

        return parent::getCacheDir();
    }

    public function getLogDir()
    {
        if (in_array($this->environment, ['dev', 'test'])) {
            return '/var/www/symfony/logs';
        }

        return parent::getLogDir();
    }
@andrerom
Copy link

@andrerom andrerom commented Apr 18, 2017

@toooni I did test with /tmp/symfony/ folder for cache and log but got better response times without for some reason, but I'll have a second look when I can.

@asteinlein
Copy link

@asteinlein asteinlein commented Apr 18, 2017

@andrerom Thanks for the tip, but same error about unknown option: cached with the compose file version set to 3.2 when deploying as a stack, unfortunately.

@rivaros
Copy link

@rivaros rivaros commented Apr 18, 2017

@asteinlein maybe still wrong version of docker. here's mine

ross@MAC-ROSS:~/projects/DockerPack $ docker version
Client:
 Version:      17.05.0-ce-rc1
 API version:  1.29
 Go version:   go1.7.5
 Git commit:   2878a85
 Built:        Tue Apr 11 20:55:05 2017
 OS/Arch:      darwin/amd64

Server:
 Version:      17.05.0-ce-rc1
 API version:  1.29 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   2878a85
 Built:        Tue Apr 11 20:55:05 2017
 OS/Arch:      linux/amd64
 Experimental: true

Btw your docker-compose should start like:

version: "3"
@rivaros
Copy link

@rivaros rivaros commented Apr 18, 2017

Personally staying with inotify/rsync solutions because of gulp watchers/builders inside containers
So waiting for "delegate". :)

@MetalArend
Copy link

@MetalArend MetalArend commented Apr 23, 2017

@rivaros could you share that inotify/rsync solution?

@carn1x
Copy link

@carn1x carn1x commented Apr 28, 2017

Is there any high level estimate for when :delegate might start to appear? Just wondering if I should try and get all my devs converted to something like docker-sync which has a lot more initial/ongoing overhead, especially for our devs who are just putting up with our decision to enforce docker rather than enthusiastically embracing it!

@sanbor
Copy link

@sanbor sanbor commented Apr 28, 2017

@bilby91
Copy link

@bilby91 bilby91 commented Apr 28, 2017

Is there any solution for mac that haves a decent speed ? Running docker in vagrant solves the write speed issue ?

@andrerom
Copy link

@andrerom andrerom commented Apr 29, 2017

@sanbor @bilby91 As found earlier up in this thread, there are two main choices if :cached does not solve it for you:

  • nfs either using Docker for Mac via d4m-nfs or with DockerToobox (cleanest, only thing you need to take care of is paths which gets different)
  • docker-sync or similar rsync/event based syncing (probably fastest, but you'll need to change your setup for it, and you might run into sync issues)
@ianks
Copy link

@ianks ianks commented May 3, 2017

Why does the docker for mac engine not just mount NFS?

@aimakun
Copy link

@aimakun aimakun commented May 3, 2017

@ianks There are several reasons for not using NFS. The issue is current one Docker for Mac using still have own problems. Give the team more time to solve that.

As @andrerom mentioned above, you still can use workarounds. I'm still using docker/toolbox and it's working fine.

Even Docker for Windows has long way to go, currently it still only available on Windows 10 pro.

@lox
Copy link

@lox lox commented May 3, 2017

Could mods close this thread to prevent more comments like @ianks? Lots of us want to keep tabs on updates on the issue and the sheer volume of "use nfs or docker-sync" comments makes that very annoying.

@luckydonald
Copy link

@luckydonald luckydonald commented May 3, 2017

@lox, You shouldn't kill discussion however.
Don't be so grumpy 😃

@ianks
Copy link

@ianks ianks commented May 4, 2017

@lox It was a question, not a statement.

@lox
Copy link

@lox lox commented May 4, 2017

Search on this page for NFS @ianks. If that doesn't clarify your question, we can provide more detail.

@EugenMayer
Copy link

@EugenMayer EugenMayer commented May 4, 2017

To add something for the discussion, i made some benchmarks with :cached and the newest edge release today https://github.com/EugenMayer/docker-sync/wiki/Performance-Tests-2017

In the end, :cached will not really help with 17s, thats even far below the fusion based shares with 12s, not to mention when you compare it with native_osxm which has only 0.20s, near hte native speed with 0.19s.

we are talking about :cached beeing 50x slower then a native docker container or alternatives

@lephuongbg
Copy link

@lephuongbg lephuongbg commented May 4, 2017

@EugenMayer, :cached is used for improving read speed, and you're testing write speed. You will have to wait for :delegate for that benchmark.

@Vanuan
Copy link

@Vanuan Vanuan commented May 5, 2017

Looks like cached/delegated wouldn't solve the issue. People wouldn't know when to apply what.

@EugenMayer
Copy link

@EugenMayer EugenMayer commented May 5, 2017

For sure it would not, and not even because this implicates, that the container has the superior over the host - which is right the opposite what you need when you actually develop. The container will hardly ever surprise you with something "new interesting" - and if anything, its generated and reproducible.

Very very different for code changes, which are not reproducible and very hard to write again ( mentally alone ). Thats why docker-sync uses always "hosts wins" while still using the buffer. Losing code is just not funny - losing generated assets (in case it ever happens to be a conflict) is a no care.

so :delegate is just something you do never want to touch - even if i do get the general idea.

@docker docker locked and limited conversation to collaborators May 5, 2017
@yallop
Copy link
Contributor

@yallop yallop commented May 5, 2017

Subscribers to this issue might be interested to read the following blog post:

    User-guided caching in Docker for Mac

which explains how and when to use cached, along with some details about the implementation.

Since there are now significant improvements in the use cases that prompted this issue, we're closing the issue to further conversation. We'll continue to post occasional updates here when further performance improvements are available. There's a new, more focused issue open to track the progress of the caching implementation (i.e. the implementation of delegated, and further improvements to cached):

    File system performance improvements

If you'd like to follow developments closely, we recommend subscribing there. Thanks to everyone for contributing to this conversation!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.