Volume sharing performance issues #593

Open
jmreicha opened this Issue Oct 24, 2014 · 29 comments

Projects

None yet
@jmreicha

As of the 1.3 release, volume mounting seems to be working. The issue is that when attempting to share a large git repo (17k files) in to the container on boot2docker I am getting some crazy slow performance. Are there any workarounds or plans to address this issue?

The problem seems to point at VirtualBox Guest Additions when you have lots of small files - http://mitchellh.com/comparing-filesystem-performance-in-virtual-machines

Maybe NFS shared folders could be used somehow? Or maybe VirtualBox people are working on fixing the performance issues?

@Jupiterrr

Some people are facing the same problem in #64.

@SvenDowideit
Member

closed as duplicate. @jmreicha yup, I made an experimental PR to add samba, and sshfs (and was going to play with nfs) support in the boot2docker-cli

the problem is that getting access to configuring these things in OSX, Windows and Linux is a pest - and it was decided vbox was more likely to work in time to be released in 1.3.0.

You're welcome to help get the other sharing options over the line - I havn't got time in the next 3 weeks (i think) due to traveling.

@gae123
gae123 commented Nov 3, 2014

In the write-up pointed by @jmreicha I will single out this "Thanks to this data, I’m definitely going to be focusing more on new synced folder implementations in Vagrant that use only the native filesystems (such as rsync, or using the host machine as an NFS client instead of a server)."

Vagrant added support for rsync in 1.5 a bit later after Hashimoto's performance write-up:

https://www.vagrantup.com/blog/feature-preview-vagrant-1-5-rsync.html
https://docs.vagrantup.com/v2/synced-folders/rsync.html

Something similar for boot2docker would be very useful.

@FelikZ
FelikZ commented Nov 9, 2014

Googled a bit of the internet and found the issue is probably in vboxsf and people use NFS as a workaround on this:

http://mitchellh.com/comparing-filesystem-performance-in-virtual-machines
https://docs.vagrantup.com/v2/synced-folders/nfs.html

Is it possible to fix this in that way?

@jmreicha
jmreicha commented Nov 9, 2014

@FelikZ NFS shared folders via Vagrant is one workaround I have found to work well but insn't perfect since you can't really leverage boot2docker-cli.

@bentruyman

I've resolved to using Vagrant + Docker provisioning, pointing my Docker CLI to the Vagrant created machine and using NFS shares (as they're quite easy to setup) and the performance issues vanished. Although, setting up NFS with the boot2docker VM isn't too difficult, but the setup for myself and my team is a bit more straightforward with tools like Vagrant right now, in my opinion.

@FelikZ
FelikZ commented Nov 10, 2014

@bentruyman have you any public repo with the example of usage NFS via Vagrant?

@SvenDowideit
Member

you might be interested in boot2docker/boot2docker-cli#247

though it is out of date now.

@clifton
clifton commented Nov 11, 2014

👍

Docker performance looks solid for our smaller services but development mode on a large app has major performance problems. Is there a way that NFS could be used by default?

@SvenDowideit
Member

for large apps, you would need to design your storage layout specifically for your needs, this is not new because of Docker / Boot2Docker, its true without too.

@sdmoralesma

Same issue here (front-end development with boot2docker+grunt+volume is painfully slow). I've found this workaround : https://vagrantcloud.com/yungsang/boxes/boot2docker , is there a simpler way to use NFS+boot2docker without using vagrant?

@STRML
STRML commented Dec 12, 2014

I've ended up doing the same as @sdmoralesma, using parallels/boot2docker-vagrant-box. It's a bit more work to get going and it can be fiddly for some of our devs, but at least it's not mind-numbingly slow. Would be great if boot2docker.pkg supported NFS, it would really simplify things. IMO with vboxfs it's not suitable for development at all.

@thaJeztah thaJeztah referenced this issue in boot2docker/boot2docker-cli Dec 25, 2014
Closed

Seems to be very slow for volumes with many (>= 10k) files #329

@leighmcculloch

I attempted to use volume mounting to sync a local web project with ~12k small files and it caused the web app to take from 4 seconds to 1 minute to respond to a single request. It made host volumes unusable for development on my project and I'm not sure how anyone could develop with the lag I was seeing.

I'm now using fswatch and unison to do fast automatic two-way sync. I've wrapped this up in a volume container: leighmcculloch/docker-unison to make it plug-and-play, and it allowed me to keep a normal boot2docker setup without using vagrant.

@md5
Contributor
md5 commented Apr 19, 2015

@leighmcculloch you might want to have a look at https://github.com/gansbrest/hodor

@lox
Contributor
lox commented Apr 21, 2015

Interesting idea @leighmcculloch, the only bummer is that it eliminates the nice characteristics of having /Users mounted in the docker host.

@leighmcculloch

@lox agreed. I've actually switched to using docker inside Vagrant with NFS. I started seeing problems with the shared volume link between the two containers disconnecting. Vagrant with NFS is a simpler setup, but the syncing with NFS is not as fast as unison was.

@brikis98

For folks still struggling with the performance of mounted folders and vboxsf, check out A productive development environment with Docker on OS X.

@STRML
STRML commented May 19, 2015

Thank you! I've been using NFS, but this is very promising as an
alternative!

On 5/19/15 11:59 AM, Yevgeniy Brikman wrote:

For folks still struggling with the performance of mounted folders and
vboxsf, check out A productive development environment with Docker on
OS X http://www.ybrikman.com/writing/2015/05/19/docker-osx-dev/.


Reply to this email directly or view it on GitHub
#593 (comment).

@thaJeztah

@brikis98 Excellent article! You should submit it for the Docker Weekly newsletter (sorry, don't know where to do that, but I can try and find out if you need)

@brikis98

@thaJeztah: thanks! From a quick search, it looks like you need to tweet at the Docker Twitter account to get into the newsletter. I'll try that, but if anyone has other suggestions, send them my way.

@thaJeztah

@brikis98 that could well be the way to do that. If I hear another way, I'll let you know 👍

@mhworth
mhworth commented Jul 2, 2015

@brikis98, Thanks! docker-osx-dev works great.

@fyddaben
fyddaben commented Sep 6, 2015

i have the problem too, is there any easy way to slove it ?? i want to use it in my team

@pencilcheck

Still wish for a native solution directly from virtualbox or boot2docker

@borsboom borsboom referenced this issue in commercialhaskell/stack Oct 8, 2015
Closed

Docker: support using from non-x86_64-linux platforms #194

@jordscream

No temporary Stable solution for Windows ?

@fyddaben
fyddaben commented Nov 5, 2015

@jordscream u can try sshd

@jordscream

@fyddaben Can you be more specific ? :/

Today The fact is:
1/ docker machine runs almost good in windows 7 except the shared vboxsf of Vbox which is very slow. Change vboxsf to NFS, CIFS, SMB is a complicated stuff and generate severals errors (symlinks, difficult setup for designers in Windows...)
2/ Boot2docker Vagrant Box is an alternative to the problem but the setup takes several hours if everything is okay... The final is something which work but with big problems as "cannot enable tty mode"... Crazy errors if you have a docker image displaying something...

The need is to have a docker in windows with shared volumes for developpers (web designers in my case) by following the normal process of docker install (docker toolbox) and have a "correct" shared volume performance...

If you have something easier than 1/ and faster/better than 2/ , I Take :P

And Yeah, I think everybody sould go to Linux or Mac but in big companies, my bad, Windows stays the King...

@jordscream jordscream referenced this issue in docker/machine Nov 5, 2015
Closed

Shared volume performance #2192

@fyddaben
fyddaben commented Nov 9, 2015

@jordscream u can install a sshd client on windows. and this is the website

sshfs -p 2023 -o allow_other work@$(docker-machine ip dev):/docker/path/  /widows/path

2023 is my ssh open port.

and the speed should be faster than vbox

@jordscream

@fyddaben it seems your website supports only OSX, alternative for windows ?
Thanks for your help

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment