New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NFS folders not properly synced on macOS High Sierra #8788

Closed
Yaroon opened this Issue Jul 16, 2017 · 172 comments

Comments

Projects
None yet
@Yaroon

Yaroon commented Jul 16, 2017

Please note that the Vagrant issue tracker is reserved for bug reports and
enhancements. For general usage questions, please use the Vagrant mailing list:
https://groups.google.com/forum/#!forum/vagrant-up. Thank you!

Vagrant version

$ vagrant -v
Vagrant 1.9.7

Host operating system

$ uname -a
Darwin liver.local 17.0.0 Darwin Kernel Version 17.0.0: Thu Jun 29 19:47:09 PDT 2017; root:xnu-4532.0.0.0.1~23/RELEASE_X86_64 x86_64
$ sw_vers -productVersion
10.13

Guest operating system

$ uname -a
Linux vagrant 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1+deb8u3 (2015-08-04) x86_64 GNU/Linux

Vagrantfile

# -*- mode: ruby -*-
# vi: set ft=ruby :

## Check dependencies

# vagrant version
Vagrant.require_version ">= 1.7.0"


# vagrant-triggers plugin
if !Vagrant.has_plugin?("vagrant-triggers")
  puts "The 'vagrant-triggers' plugin is required, install it with the following command:"
  puts " vagrant plugin install vagrant-triggers"
  exit
end
# vagrant-bindfs plugin
if !Vagrant.has_plugin?("vagrant-bindfs")
  puts "The 'vagrant-bindfs' plugin is required, install it with the following command:"
  puts " vagrant plugin install vagrant-bindfs"
  exit
end


Vagrant.configure("2") do |config|

  # General configuration, valid for all defined boxes.

  # Virtualbox tweaks.
  config.vm.provider :virtualbox do |vb|
   # Moar memory
    vb.customize ["modifyvm", :id, "--memory", "2048"]
    # allow symlinks
    vb.customize ["setextradata", :id, "VBoxInternal2/SharedFoldersEnableSymlinksCreate/vagrant", "1"]
  end

  # NFS shares
  config.vm.synced_folder "www", "/var/www", type: "nfs" 
  config.vm.synced_folder "db", "/home/vagrant/db", :nfs => true
  config.vm.synced_folder "scripts", "/home/vagrant/scripts", :nfs => { :mount_options => ["mfsymlinks"] }

  #config.vm.synced_folder "www", "/home/vagrant/www", type: "nfs"
  #config.bindfs.bind_folder "/home/vagrant/www", "/var/www", owner: "vagrant", group: "vagrant", perms: "0777"

  # https/ssl port forwarding
  config.vm.network :forwarded_port, guest: 443, host: 4444

  # SSH
  config.ssh.shell = "BASH_ENV=/etc/profile bash -l"

  ## Provision
  # setup.sh will run to provide additional provisioning, see script for details
  config.vm.provision :shell, :inline => "
    /vagrant/scripts/setup.sh;
  "

  ## Triggers

  # cleanup actions that will run on 'destroy'
  config.trigger.before :destroy do
    for script in Dir.glob("scripts/cleanup/*")
      begin
        answer = ''
        info("Do you want to execute the '#{script}' cleanup script'? [Y/n] ")
        Timeout::timeout(5) { answer = $stdin.gets }
      rescue Timeout::Error
        answer = 'y'
      end
      if answer == 'Y' or answer == 'y'
        run "vagrant ssh -c 'sh #{script}'"
      end
    end
  end

  # the box
  config.vm.define :vagrant do |vagrant|
    vagrant.vm.network :private_network, ip: "192.168.50.2"
    vagrant.vm.hostname = "vagrant.loc"
  end
end

if File.file?('Vagrantfile.box')
  load 'Vagrantfile.box'
end

Please note, if you are using Homestead or a different Vagrantfile format, we
may be unable to assist with your issue. Try to reproduce the issue using a
vanilla Vagrantfile first.

Debug output

https://gist.github.com/Yaroon/dcb20eb984669688746767b0342c7d55

Expected behavior

I should see all folders that in the NFS share.
I shouldn't see folders doubly in the NFS share.

Actual behavior

I do not see the TypedData folder.
I see Field and Breadcrumb folder twice.

Steps to reproduce

  1. You can try installing Drupal 8.3 on your host.
  2. Then set up Vagrant file so that the www folder gets shared to the guest OS's /var/www
  3. Check out /var/www/core/lib/Drupal/Core and you'll see some folders differently from the host OS.

Additionally you can check out a screencast I did that illustrates the problem.
You can see how the folder is an NFS folder at first. You can see Breadcrumb and Field appearing twice on the guest OS while TypedData does not. When I then create a folder "WhateverOtherData" it gets synced between host and guest. However TypedData and Field and Breadcrumb are still there twice. When I then create a folder StringData, all of a sudden TypedData appears on the guest and we only see 1 Breadcrumb and 1 Field folder. So something got triggered to fix it... I just don't know what.

https://www.dropbox.com/s/ykdi1cp5v689sq0/Vagrant%20%2B%20NFS%20%2B%20macOS%20High%20Sierra.mov?dl=0

References

@pascaldevink

This comment has been minimized.

pascaldevink commented Jul 18, 2017

I've also been seeing the same "behaviour" since upgrading to 10.13, but only with some vagrant boxes.

For those reading this coming from google; changing from NFS to rsync is a good workaround for the problem for now, but requires extra actions (such as running vagrant rsync-auto in the background) and has some small drawbacks.

@briancain

This comment has been minimized.

Member

briancain commented Jul 18, 2017

Hello @Yaroon - I attempted to reproduce this locally with my own local Drupal folder being synced with NFS and I did not notice duplicate folders within the mounted folder on /var/www like you mentioned. If you could destroy your current guest (make sure to back up anything important before doing so), restart your host, and bring up a fresh guest and see if your problem still persists.

Thank you!

@Yaroon

This comment has been minimized.

Yaroon commented Jul 19, 2017

I've 'vagrant destroy'ed the box, and rebooted the computer. Still the same, as soon as I remove the "StringData" folder, "TypedData" goes away as well. When I create "StringData" again, it re-appears.

Maybe next week Wednesday there'll be a new macOS High Sierra Public beta. We'll see how it evolves? Until then, I don't know what to do. macOS HS does of course have a new FS... that could be the cause? But I don't know nearly enough about it in order to gain some info that could diagnose this issue.

@kikitux kikitux self-assigned this Jul 19, 2017

@kikitux

This comment has been minimized.

Collaborator

kikitux commented Jul 19, 2017

@Yaroon can you try to reproduce in a more simple setup?

I wasn't able.

vagrant@precise64:~$ touch /var/www/www_from_guest /home/vagrant/db/db_from_guest /home/vagrant/scripts/scripts_from_guest
vagrant@precise64:~$ ls -al /var/www/ /home/vagrant/db /home/vagrant/scripts                   
/home/vagrant/db:
-rw-rw-r-- 1     501 dialout    0 Jul 19 19:25 db_from_guest

/home/vagrant/scripts:
-rw-rw-r-- 1     501 dialout    0 Jul 19 19:25 scripts_from_guest

/var/www/:
-rw-rw-r--  1  501 dialout    0 Jul 19 19:25 www_from_guest
vagrant@precise64:~$ logout


0 kikitux ~/Dropbox/local/kikitux/p64 (master) $ ls db scripts www
db:
db_from_guest

scripts:
scripts_from_guest

www:
www_from_guest
0 kikitux ~/Dropbox/local/kikitux/p64 (master) $ touch db/db_from_host scripts/scripts_from_host www/www_from_host
0 kikitux ~/Dropbox/local/kikitux/p64 (master) $ ls db scripts www
db:
db_from_guest  db_from_host

scripts:
scripts_from_guest  scripts_from_host

www:
www_from_guest  www_from_host
0 kikitux ~/Dropbox/local/kikitux/p64 (master) $ vagrant ssh 


vagrant@precise64:~$ ls -al /var/www/ /home/vagrant/db /home/vagrant/scripts
/home/vagrant/db:
total 8
drwxr-xr-x 4     501 dialout  136 Jul 19 19:26 .
drwxr-xr-x 6 vagrant vagrant 4096 Jul 19 19:23 ..
-rw-rw-r-- 1     501 dialout    0 Jul 19 19:25 db_from_guest
-rw-r--r-- 1     501 dialout    0 Jul 19 19:26 db_from_host

/home/vagrant/scripts:
total 8
drwxr-xr-x 4     501 dialout  136 Jul 19 19:26 .
drwxr-xr-x 6 vagrant vagrant 4096 Jul 19 19:23 ..
-rw-rw-r-- 1     501 dialout    0 Jul 19 19:25 scripts_from_guest
-rw-r--r-- 1     501 dialout    0 Jul 19 19:26 scripts_from_host

/var/www/:
total 8
drwxr-xr-x  4  501 dialout  136 Jul 19 19:26 .
drwxr-xr-x 12 root root    4096 Jul 19 19:23 ..
-rw-rw-r--  1  501 dialout    0 Jul 19 19:25 www_from_guest
-rw-r--r--  1  501 dialout    0 Jul 19 19:26 www_from_host
vagrant@precise64:~$
@kikitux

This comment has been minimized.

Collaborator

kikitux commented Jul 19, 2017

my Vagrantfile:

Vagrant.configure("2") do |config|
  config.vm.box = "hashicorp/precise64"
  config.vm.network "private_network", type: "dhcp"

  # NFS shares
  config.vm.synced_folder "www", "/var/www", type: "nfs" 
  config.vm.synced_folder "db", "/home/vagrant/db", :nfs => true
  config.vm.synced_folder "scripts", "/home/vagrant/scripts", :nfs => { :mount_options => ["mfsymlinks"] }

end
@kikitux

This comment has been minimized.

Collaborator

kikitux commented Jul 19, 2017

@Yaroon for most use cases NFS is fine.

however if you are using a code generator that may use links, symlinks, hardlinks, etc, it may require a full FS and wont' work with virtualboxfs, vmware hgfs or nfs.

we have seen nodejs and php apps having issues due this.

vagrant does his best to setup nfs for you, however, once is setup, will be host + guest troubleshooting in case of issues.

if you think this is a core vagrant issues, please help us to reproduce this on a simple vagrantfile and simple commands.

thanks!

@pascaldevink

This comment has been minimized.

pascaldevink commented Jul 19, 2017

I'm not sure if this is a vagrant issue per se, could be an NFS implementation issue in High Sierra, or be caused by APFS. One thing I do know, is that I'm using the exact same vagrant box and codebase as before I upgraded. And I didn't have any trouble before that.

I've been trying to come up with a simple reproduction vagrantfile, but unfortunately I wasn't able to. It doesn't even occur on all the vagrant boxes that I have, which makes it hard to pinpoint the problem.

Is there any way to get more debugging output from the NFS process?

@Yaroon

This comment has been minimized.

Yaroon commented Jul 20, 2017

I'll also try to come up with something reproducible for you guys, but it'll be tough I think. I'm really hoping for a fix in macOS High Sierra next week. Thing is, I don't know if Apple is aware, and it's hard to write a bug report without a true reproducible for those guys. It won't easily be looked at then.

@kikitux kikitux closed this Jul 20, 2017

@kikitux

This comment has been minimized.

Collaborator

kikitux commented Jul 20, 2017

please reopen if found a way to reproduce

@kikitux kikitux added needs-repro and removed waiting-reply labels Jul 20, 2017

@Yaroon

This comment has been minimized.

Yaroon commented Aug 23, 2017

So I've found a reproducible. Again, I am on macOS High Sierra.

I don't know how to re-open an issue either, so I hope someone sees this :)

I don't quite not know how to go about it, but this is what I did to get it going... I'll include some commands you need to execute on the vagrant box to get to the point where you see some folders not sync.

Vagrant version: Vagrant 1.9.7
VirtualBox version: Version 5.1.26 r117224 (Qt5.6.2)
My Vagrantfile:

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure("2") do |config|
  config.vm.box = "debian/jessie64"
  config.vm.network "private_network", type: "dhcp"
  config.vm.synced_folder "www", "/var/www", type: "nfs"
end

In vagrant I added these to /etc/apt/sources.list

deb http://http.debian.net/debian jessie-backports main contrib non-free
deb-src http://http.debian.net/debian jessie-backports main contrib non-free
deb http://packages.dotdeb.org jessie all
deb-src http://packages.dotdeb.org jessie all

So I could, on the vagrant box:

$ apt update && apt-get install virtualbox-guest-additions-iso build-essentials linux-headers-3.16.0-4
$ mount /usr/share/virtualbox/VBoxGuestAdditions.iso /media/cdrom && /media/cdrom/VBoxLinuxAdditions.run
$ mkdir /var/www

Then on the Host, download Drupal from https://www.drupal.org/project/drupal/releases/8.3.7
Extract the archive and rename the drupal folder to www, as this is the folder that we'll share to the guest's /var/www

Then... after vagrant up, in guest you can go to /var/www/core/lib/Drupal/Core/ and you won't see the TypedData folder, even though it is there on the host.

If you mkdir StringData in /var/www/core/lib/Drupal/Core/ you'll all of a sudden see the TypedData folder appear as well.

Let me know if you need more instructions.

@Yaroon Yaroon changed the title from NFS folders not being properly synced. to NFS folders not being properly synced on macOS High Sierra. Aug 24, 2017

@Yaroon Yaroon changed the title from NFS folders not being properly synced on macOS High Sierra. to NFS folders not properly synced on macOS High Sierra. Aug 24, 2017

@Yaroon Yaroon changed the title from NFS folders not properly synced on macOS High Sierra. to NFS folders not properly synced on macOS High Sierra Aug 24, 2017

@Yaroon

This comment has been minimized.

Yaroon commented Aug 24, 2017

Booting the machine in VirtualBox, not through vagrant, does show the folder...

In VirtualBox: add the www folder as a permanent Machine Folder with Folder Name drupalwww

Add root to the vboxsf group

$ sudo usermod -a -G vboxsf root

Mount the shared folder

$ sudo mount -t vboxsf drupalwww /var/www

List the folder's contents

$ ls /var/www/core/lib/Drupal/Core

This lists the TypedData folder. When booting the machien via vagrant up, using NFS to share the folder, it does not show.

@Yaroon

This comment has been minimized.

Yaroon commented Aug 24, 2017

Or, when not doing it via VirtualBox' shared folders but configuring the NFS share on the guest OS it also works well.

Adding the line to /etc/fstab for my host's IP and www folder lets me mount it and there are no missing files or folders.
127.28.128.1:/Users/jeroen/Projects/VagrantTest/www /var/www nfs defaults 0 0

@Yaroon

This comment has been minimized.

Yaroon commented Sep 2, 2017

It got even simpler, it's probably an APFS bug that causes it to not work well with NFS, so I think this will blow up soon as macOS High Sierra nears release, here's some commands I ran to show the problem. No Drupal install needed, just a vagrant box.

Darwin liver.local 17.0.0 Darwin Kernel Version 17.0.0: Thu Aug 24 22:01:05 PDT 2017; root:xnu-4570.1.46~3/RELEASE_X86_64 x86_64
jeroen@liver:~/Projects/Wunderkraut/VOK_digit$ cat /etc/exports | grep digit/www.*50.2
"/Users/jeroen/Projects/Wunderkraut/VOK_digit/www" 192.168.50.2 -alldirs -mapall=501:20
jeroen@liver:~/Projects/Wunderkraut/VOK_digit$ vagrant ssh

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
You have new mail.
Last login: Sat Sep  2 11:04:22 2017 from 10.0.2.2
vagrant@vagrant:~$ cat /etc/fstab | grep www
192.168.50.1:/Users/jeroen/Projects/Wunderkraut/VOK_digit/www /var/www nfs defaults,sync 0 0
vagrant@vagrant:~$ sudo ifconfig | grep inet | grep 50.2
          inet addr:192.168.50.2  Bcast:192.168.50.255  Mask:255.255.255.0
vagrant@vagrant:~$ cd /var/www
vagrant@vagrant:/var/www$ mkdir test && cd test
vagrant@vagrant:/var/www/test$ touch test{1..1000}
vagrant@vagrant:/var/www/test$ ls | wc -l
1007
vagrant@vagrant:/var/www/test$ ls | uniq -c | grep " 2"
      2 test136
      2 test137
      2 test472
      2 test689
      2 test7
      2 test723
      2 test853
      2 test96
vagrant@vagrant:/var/www/test$ exit
logout
Connection to 127.0.0.1 closed.
jeroen@liver:~/Projects/Wunderkraut/VOK_digit$ cd www/test/ && ls | wc -l
    1000
jeroen@liver:~/Projects/Wunderkraut/VOK_digit/www/test$ ```
@kylebrowning

This comment has been minimized.

kylebrowning commented Sep 18, 2017

@kikitux we cant re-open High Sierra is coming soon.

@briancain briancain reopened this Sep 18, 2017

@briancain

This comment has been minimized.

Member

briancain commented Sep 18, 2017

I've reopened the issue @kylebrowning @Yaroon

@briancain

This comment has been minimized.

Member

briancain commented Sep 20, 2017

@kylebrowning or @Yaroon ...if you could confirm, does this behavior happen if you run the mount command yourself outside of Vagrant? If that's the case then it is probably a bug with NFS on High Sierra. It'd be something like....

mount -o vers=3,udp <ip address of machine>:/Local/Path/to/Vagrant /Remote/Path/To/vagrant
@Yaroon

This comment has been minimized.

Yaroon commented Sep 20, 2017

@briancain I had indeed already booted the guest OS in just VirtualBox with NFS set up in its /etc/fstab. But just for you I unmounted /var/www on the guest and tried it with:

sudo mount -o vers=3,udp 192.168.50.1:/Path/to/Vagrant/www /var/www

No change. When I create 1000 files in a folder and count them with ls | wc -l it prints 1038 instead of 1000.

@deiga

This comment has been minimized.

deiga commented Nov 30, 2017

Locking the issue could be a good idea

@josadeluca

This comment has been minimized.

josadeluca commented Nov 30, 2017

FWIW, the issue does seem to be fixed on my CentOS 7 VM, but I still have a CentOS 6 (32 bit) VM on which it most definitely is not fixed (with 10.13.2 Beta 5). All the same symptoms persist. The mounts on that VM only work when using sshfs.

@ApacheEx

This comment has been minimized.

ApacheEx commented Dec 6, 2017

Finally, 10.13.2 is released. 🎉
https://support.apple.com/en-us/HT208179

@martell

This comment has been minimized.

martell commented Dec 6, 2017

Can we now close this issue :)
Was waiting for that release for too long.

@rogierslag

This comment has been minimized.

rogierslag commented Dec 6, 2017

I can confirm that the final version also solves the issue

@jandubois

This comment has been minimized.

jandubois commented Dec 7, 2017

After all the positive feedback here I decided to try 10.13.2 too, without the intermediate sparsebundle. Unfortunately it doesn't even mount anymore for me:

vagrant-kube:/home/vagrant # mount -o vers=3,udp 192.168.77.1:/Users/jan/suse/scf /home/vagrant/scf
mount.nfs: access denied by server while mounting 192.168.77.1:/Users/jan/suse/scf

(the same error happens during vagrant up, but I can repro it manually after vagrant ssh as well).

I forgot to test directly after installing the 10.13.2 upgrade, so I don't know if it is related to the upgrade, or somehow cause by removing the sparsebundle.

/etc/exports looks fine to me:

$ cat /etc/exports
# VAGRANT-BEGIN: 501 9b1e50a6-bae8-4e32-9a38-81557fcf4c7a
"/Users/jan/suse/scf" "/Users/jan/suse/scf/.fissile/.bosh" 192.168.77.77 -alldirs -mapall=501:20
# VAGRANT-END: 501 9b1e50a6-bae8-4e32-9a38-81557fcf4c7a

Any ideas?

@emileber

This comment has been minimized.

emileber commented Dec 7, 2017

@jandubois you should open a new issue as it looks unrelated to this issue.

@briancain

This comment has been minimized.

Member

briancain commented Dec 7, 2017

@jandubois yeah...like @emileber just said that seems like a separate issue from this one. I recommend opening a new issue here on Github with all the information from the template filled out. Thanks!

@jandubois

This comment has been minimized.

jandubois commented Dec 7, 2017

you should open a new issue as it looks unrelated to this issue.

Agreed. I'll first look into this some more myself though; I have another machine that I will upgrade to 10.13.2 tomorrow and will look more carefully after each step to see when/if that one will break in the same way.

@jfbibeau

This comment has been minimized.

Contributor

jfbibeau commented Dec 7, 2017

I've just updated from the Beta to the release load of 10.13.2 and everything is good to go. NFS mount works perfectly.

@lazaronixon

This comment has been minimized.

lazaronixon commented Dec 7, 2017

Solved here too..

@sweh

This comment has been minimized.

sweh commented Dec 7, 2017

I have the same issue as @jandubois. If I unmount the SparseBundle and put my files directly into the folder where the Bundle was mounted, vagrant is not able to mount that folder into the VM. Switching back to the SparseBundle works fine.

@janssensjelle

This comment has been minimized.

janssensjelle commented Dec 7, 2017

works for me now 👍

@dnl-jst

This comment has been minimized.

dnl-jst commented Dec 7, 2017

@sweh @jandubois Did you move the project without destroying and recreating the box? I had the same issue, vagrant seems to save the project path, and if you move the project, the paths changes, but vagrant tries to mount the old path. All I did was destroying and recreating the box after moving. Didn't do more research if the path can be fixed without destroying the box.

@sweh

This comment has been minimized.

sweh commented Dec 7, 2017

No, I did not move anything. The path to the mounted SparseBundle and the folder in the AFS filesystem respectively are the same.

@BR0kEN-

This comment has been minimized.

Contributor

BR0kEN- commented Dec 7, 2017

@dnl-jst, you can manually edit the ~/.vagrant.d/data/machine-index/index.

You'll have the ~/.vagrant.d part only if https://www.vagrantup.com/docs/other/environmental-variables.html#vagrant_home is not set or has exactly the same value.

@rakauchuk

This comment has been minimized.

rakauchuk commented Dec 7, 2017

10.13.2 solved the problem for me.

@trappar

This comment has been minimized.

trappar commented Dec 8, 2017

Yeah, it would appear that 10.13.2 solves this, but is it just me.. or is it a little slower than NFS on an HFS+ formatted drive/sparsebundle?

@ireznik

This comment has been minimized.

ireznik commented Dec 8, 2017

I upgraded directly to 10.13.2. Everything works as expected, no issues so far with nfs usage in vagrant for my vm!

@shamdsc

This comment has been minimized.

shamdsc commented Dec 8, 2017

@trappar its much slower I agree. And I find when I save a file on the Mac side, within vagrant the file is missing for quite a few seconds before it shows up again with the up to date content. Definitely much slower than it used to be on HFS+

@Oscarteg

This comment has been minimized.

Oscarteg commented Dec 11, 2017

@trappar @shamdsc I'm having the same issue. It's much slower. Takes couple of seconds for it to sync properly. Is this a Vagrant issue or an Apple issue?

@emileber

This comment has been minimized.

emileber commented Dec 11, 2017

@Oscarteg @shamdsc @trappar Please open a new issue since this one is about folders not properly synced and looks to be resolved, it's not about syncing speeds. Additional issues should be taken care of individually in their own thread.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment