Volume sharing performance issues

[jmreicha]

As of the 1.3 release, volume mounting seems to be working. The issue is that when attempting to share a large git repo (17k files) in to the container on boot2docker I am getting some crazy slow performance. Are there any workarounds or plans to address this issue?

The problem seems to point at VirtualBox Guest Additions when you have lots of small files - http://mitchellh.com/comparing-filesystem-performance-in-virtual-machines

Maybe NFS shared folders could be used somehow? Or maybe VirtualBox people are working on fixing the performance issues?

[Jupiterrr]

Some people are facing the same problem in #64.

[SvenDowideit]

closed as duplicate. @jmreicha yup, I made an experimental PR to add samba, and sshfs (and was going to play with nfs) support in the boot2docker-cli

the problem is that getting access to configuring these things in OSX, Windows and Linux is a pest - and it was decided vbox was more likely to work in time to be released in 1.3.0.

You're welcome to help get the other sharing options over the line - I havn't got time in the next 3 weeks (i think) due to traveling.

[gae123]

In the write-up pointed by @jmreicha I will single out this "Thanks to this data, I’m definitely going to be focusing more on new synced folder implementations in Vagrant that use only the native filesystems (such as rsync, or using the host machine as an NFS client instead of a server)."

Vagrant added support for rsync in 1.5 a bit later after Hashimoto's performance write-up:

https://www.vagrantup.com/blog/feature-preview-vagrant-1-5-rsync.html
https://docs.vagrantup.com/v2/synced-folders/rsync.html

Something similar for boot2docker would be very useful.

[FelikZ]

Googled a bit of the internet and found the issue is probably in vboxsf and people use NFS as a workaround on this:

http://mitchellh.com/comparing-filesystem-performance-in-virtual-machines
https://docs.vagrantup.com/v2/synced-folders/nfs.html

Is it possible to fix this in that way?

[jmreicha]

@FelikZ NFS shared folders via Vagrant is one workaround I have found to work well but insn't perfect since you can't really leverage boot2docker-cli.

[bentruyman]

I've resolved to using Vagrant + Docker provisioning, pointing my Docker CLI to the Vagrant created machine and using NFS shares (as they're quite easy to setup) and the performance issues vanished. Although, setting up NFS with the boot2docker VM isn't too difficult, but the setup for myself and my team is a bit more straightforward with tools like Vagrant right now, in my opinion.

[FelikZ]

@bentruyman have you any public repo with the example of usage NFS via Vagrant?

[SvenDowideit]

you might be interested in boot2docker/boot2docker-cli#247

though it is out of date now.

[clifton]

👍

Docker performance looks solid for our smaller services but development mode on a large app has major performance problems. Is there a way that NFS could be used by default?

[SvenDowideit]

for large apps, you would need to design your storage layout specifically for your needs, this is not new because of Docker / Boot2Docker, its true without too.

[sdmoralesma]

Same issue here (front-end development with boot2docker+grunt+volume is painfully slow). I've found this workaround : https://vagrantcloud.com/yungsang/boxes/boot2docker , is there a simpler way to use NFS+boot2docker without using vagrant?

[STRML]

I've ended up doing the same as @sdmoralesma, using parallels/boot2docker-vagrant-box. It's a bit more work to get going and it can be fiddly for some of our devs, but at least it's not mind-numbingly slow. Would be great if boot2docker.pkg supported NFS, it would really simplify things. IMO with vboxfs it's not suitable for development at all.

[leighmcculloch]

I attempted to use volume mounting to sync a local web project with ~12k small files and it caused the web app to take from 4 seconds to 1 minute to respond to a single request. It made host volumes unusable for development on my project and I'm not sure how anyone could develop with the lag I was seeing.

I'm now using fswatch and unison to do fast automatic two-way sync. I've wrapped this up in a volume container: leighmcculloch/docker-unison to make it plug-and-play, and it allowed me to keep a normal boot2docker setup without using vagrant.

[md5]

@leighmcculloch you might want to have a look at https://github.com/gansbrest/hodor

[lox]

Interesting idea @leighmcculloch, the only bummer is that it eliminates the nice characteristics of having /Users mounted in the docker host.

[leighmcculloch]

@lox agreed. I've actually switched to using docker inside Vagrant with NFS. I started seeing problems with the shared volume link between the two containers disconnecting. Vagrant with NFS is a simpler setup, but the syncing with NFS is not as fast as unison was.

[brikis98]

For folks still struggling with the performance of mounted folders and vboxsf, check out A productive development environment with Docker on OS X.

[STRML]

Thank you! I've been using NFS, but this is very promising as an
alternative!

On 5/19/15 11:59 AM, Yevgeniy Brikman wrote:

For folks still struggling with the performance of mounted folders and
vboxsf, check out A productive development environment with Docker on
OS X http://www.ybrikman.com/writing/2015/05/19/docker-osx-dev/.

—
Reply to this email directly or view it on GitHub
#593 (comment).

[thaJeztah]

@brikis98 Excellent article! You should submit it for the Docker Weekly newsletter (sorry, don't know where to do that, but I can try and find out if you need)

[brikis98]

@thaJeztah: thanks! From a quick search, it looks like you need to tweet at the Docker Twitter account to get into the newsletter. I'll try that, but if anyone has other suggestions, send them my way.

[thaJeztah]

@brikis98 that could well be the way to do that. If I hear another way, I'll let you know 👍

[mhworth]

@brikis98, Thanks! docker-osx-dev works great.

[fyddaben]

i have the problem too, is there any easy way to slove it ?? i want to use it in my team

[pencilcheck]

Still wish for a native solution directly from virtualbox or boot2docker

[jsamouh]

No temporary Stable solution for Windows ?

[fyddaben]

@jordscream u can try sshd

[jsamouh]

@fyddaben Can you be more specific ? :/

Today The fact is:
1/ docker machine runs almost good in windows 7 except the shared vboxsf of Vbox which is very slow. Change vboxsf to NFS, CIFS, SMB is a complicated stuff and generate severals errors (symlinks, difficult setup for designers in Windows...)
2/ Boot2docker Vagrant Box is an alternative to the problem but the setup takes several hours if everything is okay... The final is something which work but with big problems as "cannot enable tty mode"... Crazy errors if you have a docker image displaying something...

The need is to have a docker in windows with shared volumes for developpers (web designers in my case) by following the normal process of docker install (docker toolbox) and have a "correct" shared volume performance...

If you have something easier than 1/ and faster/better than 2/ , I Take :P

And Yeah, I think everybody sould go to Linux or Mac but in big companies, my bad, Windows stays the King...

[fyddaben]

@jordscream u can install a sshd client on windows. and this is the website

sshfs -p 2023 -o allow_other work@$(docker-machine ip dev):/docker/path/  /widows/path

2023 is my ssh open port.

and the speed should be faster than vbox

[jsamouh]

@fyddaben it seems your website supports only OSX, alternative for windows ?
Thanks for your help