Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rsync on Windows fails – lacks /cygdrive prefix #4073

Closed
brianreavis opened this issue Jun 20, 2014 · 50 comments

Comments

Projects
None yet
@brianreavis
Copy link

commented Jun 20, 2014

For more details, see #3230. No one on the vagrant team seems to be noticing it... so that's why I'm intentionally creating a duplicate issue here. Feel free to close / delete this when it is reopened.

@sowebdev

This comment has been minimized.

Copy link

commented Jun 25, 2014

I experienced this problem on Windows 7 with cwrsync. Somehow Vagrant won't execute cygpath command because it depends on cygwin detection which may fail for some reason. I still have to edit vagrant's source manually to make it execute cygpath.

@starrychloe

This comment has been minimized.

Copy link

commented Jun 25, 2014

Version 1.5.4 worked, but version 1.6.3 broke. It treats c:\Users as /c/Users, which doesn't work on Windows. It should use the colon: c:/Users. (Actually should not use colon for rsync command or else it will think that is a remote path. Cygwin rsync needs /cygdrive/c/ in front.)

==> default: Rsyncing folder: /c/Users/Chloe/Documents/ODesk/AIDC/BK-Development/ => /rsync/BK-Development
There was an error when attempting to rsync a synced folder.
Please inspect the error message below for more info.

Host path: /c/Users/Chloe/Documents/ODesk/AIDC/BK-Development/
Guest path: /rsync/BK-Development
Command: rsync --verbose --archive --delete -z --copy-links --chmod=ugo=rwX --no-perms --no-owner --no-group --rsync-path sudo rsync -e ssh -p 2222 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i 'C:/Users/Chloe/.vagrant.d/insecure_private_key' --exclude .vagrant/ /c/Users/Chloe/Documents/ODesk/AIDC/BK-Development/ vagrant@127.0.0.1:/rsync/BK-Development
Error: Warning: Permanently added '[127.0.0.1]:2222' (RSA) to the list of known hosts.
rsync: change_dir "/c/Users/Chloe/Documents/ODesk/AIDC/BK-Development" failed: No such file or directory (2)
FATAL I/O ERROR: dying to avoid a --delete-during issue with a pre-3.0.7 receiver.
rsync error: requested action not supported (code 4) at /home/lapo/package/rsync-3.0.9-1/src/rsync-3.0.9/flist.c(1800) [sender=3.0.9]
@starrychloe

This comment has been minimized.

Copy link

commented Jun 25, 2014

This seems like a possible solution: #3925

@schkovich

This comment has been minimized.

Copy link

commented Aug 21, 2014

You could change cygdrive prefix from default /cygdrive to /. See #3913

@francislavoie

This comment has been minimized.

Copy link

commented Sep 6, 2014

@mitchellh Will this ever get looked at? Has been a bug for a few months, without any attention given. It should be a very simple fix.

@fabiangarga

This comment has been minimized.

Copy link

commented Oct 1, 2014

Hi, I tried to use rsync on a windows machine but failed on each attempt. I installed cygwin with openssh rsync packages. But now when I boot the machine it always boot up with the default vagrant box, and not the one that I was using before using rsync.
My virtual machine is running Ubuntu with a custom Wordpress site that I'm developing, and my host is running windows 8.1 64.
MINGW32 bash allows me to run my Ubuntu box, but it has some missing features that I previously installed like, Apache, PHP, mysql...
Any clue what happened?

@kikitux

This comment has been minimized.

Copy link
Collaborator

commented Oct 1, 2014

hello

if you change your shell, in windows, sometimes the HOME change

in this home, is usally where the virtualbox default folder is created.

Open the virtualbox gui and see if there are any extra vms.

if you want/need recover some data/work from those vms, turn them one using
the vbox gui, and save the data somehow out of the virtual machine

if you changed shell, and on a new vagrant up a new box created, my advise
would be delete all of the vms and create a new one with vagrant up

bear in mind that the vagrant way is vagrant destroy && vagrant up to
fix/solve most of the issues, so after you do some manual box/customization
on your machines, you may want to put that manual work into the vagrant
file, so next time when you recreate the machine those settings are done as
part of the vagrant up

Alvaro

On Wed, Oct 1, 2014 at 7:44 PM, fabiangarga notifications@github.com
wrote:

Hi, I tried to use rsync on a windows machine but failed on each attempt.
I installed cygwin with openssh rsync packages. But now when I boot the
machine it always boot up with the default vagrant box, and not the one
that I was using before using rsync.
My virtual machine is running Ubuntu with a custom Wordpress site that I'm
developing, and my host is running windows 8.1 64.
MINGW32 bash allows me to run my Ubuntu box, but it has some missing
features that I previously installed like, Apache, PHP, mysql...
Any clue what happened?


Reply to this email directly or view it on GitHub
#4073 (comment).

@fabiangarga

This comment has been minimized.

Copy link

commented Oct 1, 2014

Hi Alvaro,

When i changed the shell from mingw to CygWin it created a new vms, i deleted the new vms, but now i don't know how to associate my project folder to that specific vm.

The only way to pull the data from my Virtual machine is to log in and get the data? There's no way to reassociate?

FIX**
Ok I found out this #1755 (comment)
It fixed the problem, the only issue was to know where to run VBoxManage list vms command, and found out it's on the folder of Oracle VB, usually the path is /Program Files/Oracle/VirtualBox

I'm new using vagrant, it was a month of transition coming from xamp c:

@mitchellh

This comment has been minimized.

Copy link
Member

commented Oct 24, 2014

Okay, so here is Vagrant's cygwin detection: https://github.com/mitchellh/vagrant/blob/da91741b7c011bc69668267842c4796cf34ab6e8/lib/vagrant/util/platform.rb#L12

How can this be improved for your platforms?

@matt-richardson

This comment has been minimized.

Copy link

commented Oct 24, 2014

Where does VAGRANT_DETECTED_OS get set? (Had a search for it, but couldn't find)
It seems that its detecting cygwin incorrectly...

Once the cygwin check is improved, then we also need to fix:
https://github.com/mitchellh/vagrant/blob/efd1d5e11bfc5a72c7a1d1eae294b4751d841544/plugins/synced_folders/rsync/helper.rb#L43 which should be a cygwin check, rather than a windows check.

@mitchellh

This comment has been minimized.

Copy link
Member

commented Oct 25, 2014

@matt-richardson Hm, that used to be a cygwin check but I thought that all rsync implementations on Windows expect Cygwin paths?

@matt-richardson

This comment has been minimized.

Copy link

commented Oct 25, 2014

cwrsync doesn't seem to use that - it expects a path in the format c:/blah/blah.
Well, this is from what I recall - its been a while since I was looking into this... I will take a closer look on Monday.

@abilash222

This comment has been minimized.

Copy link

commented Oct 31, 2014

any solution for this issue

@kalabro

This comment has been minimized.

Copy link

commented Oct 31, 2014

@abilash222, I run vagrant commands from cygwin console and it just works. I've installed chere (http://stackoverflow.com/questions/9637601/open-cygwin-at-a-specific-folder) for quick opening cygwin console from any Windows folder.

If you are running Vagrant commands from common Windows CMD you have to patch Vagrant file: #3230 (comment) I personally don't see the reason to do it.

@abilash222

This comment has been minimized.

Copy link

commented Nov 10, 2014

is there any other solution other than cygwin?

@schkovich

This comment has been minimized.

Copy link

commented Nov 10, 2014

Yes, move to Linux if you can.
On Nov 10, 2014 6:45 AM, "abilash222" notifications@github.com wrote:

is there any other solution other than cygwin?


Reply to this email directly or view it on GitHub
#4073 (comment).

@osroot25

This comment has been minimized.

Copy link

commented Nov 11, 2014

There is another way to fix this bug other then to manually edit the source files:
If you need cygwin-paths, include following line into your Vagrantfile
ENV["VAGRANT_DETECTED_OS"] = ENV["VAGRANT_DETECTED_OS"].to_s + " cygwin"
It sets the Vagrant_detected_os manually so the check returns true.

@aeneasr

This comment has been minimized.

Copy link

commented Nov 12, 2014

There is another way to fix this bug other then to manually edit the source files:
If you need cygwin-paths, include following line into your Vagrantfile
ENV["VAGRANT_DETECTED_OS"] = ENV["VAGRANT_DETECTED_OS"].to_s + " cygwin"
It sets the Vagrant_detected_os manually so the check returns true.

This!

@osroot25

This comment has been minimized.

Copy link

commented Dec 10, 2014

Since it is in another project, perhaps someone following here could help/follow there: hashicorp/vagrant-installers#49 ?

@1jerry

This comment has been minimized.

Copy link

commented Jan 16, 2015

From my testing, if you are using cygwin, use the solution by @osroot25 .

If you are using cwRsync and do not have cygwin, there is no workaround using Vagrant except editing the source code (see #3230 (comment)).

My workaround that works for me is to bypass Vagrant altogether and use cwRsync directly. This works for me because I am syncing a folder that hardly ever changes. I might change it quite a few times in one day (so I have to remember step 2 below each time), but I then go weeks (or months) without any changes. Remember that to use cwRsync you have to edit and use the cwrsync.cmd script. Attempting to access the rsync.exe command directly or by adding it to your path will fail. Step 1: I added the following line to the end of cwrsync.cmd (in the installed folder):

rsync -re "ssh -p 2222" /cygdrive/b/VCS/packages/ vagrant@localhost:packages --exclude ".git/"

Step 2: I have a separate cmd window open that I run the cwrsync.cmd using the full path. Then if I need to sync changes onto the VM, I activate that window, up-arrow, return and updating is instant!

Modifying ENV to set cygwin fix by @osroot25 doesn't work with cwRsync because when you force cygwin detection, the "vagrant ssh" command will not work because it requires the cygpath command in cygwin, which you won't have, so you cannot ssh into the VM. Well you can if you use the ssh command directly with all the right options.

the fix

To answer @mitchellh, the code pointed to by @matt-richardson looks correct. cwRsync (and probably MSYS's rsync) require the /cygdrive prefix, (which seems to be a standard for posix commands on Windows). So I think the logic should be: if Platform:windows then add the prefix. But this is not happening. Either the windows flag isn't being set or this code isn't being executed or the cygwin_path() isn't producing the correct result as of Vagrant 1.6.5.

@osroot25

This comment has been minimized.

Copy link

commented Jan 16, 2015

if Platform:windows then add the prefix. But this is not happening. Either the windows flag isn't being set or this code isn't being executed or the cygwin_path() isn't producing the correct result as of Vagrant 1.6.5.

Then the following code fails:

process = Subprocess.execute(
            "bash",
            "--noprofile",
            "--norc",
            "-c", "cd #{Shellwords.escape(path)} && pwd")
          return process.stdout.chomp

(lib/vagrant/util/platform.rb lines71-76 in v 1.6.5)

as I have no idea how this code-fragment works, I can't help here.

@osroot25

This comment has been minimized.

Copy link

commented Jan 16, 2015

haha, could it be, that

"cd #{Shellwords.escape(path)} && pwd"

should be

"'cd #{Shellwords.escape(path)} && pwd'"

because -c expects a string (so the string passed to the process has to contain a string)
??

Question is, if this command can be executed with cwRsync...

@1jerry

This comment has been minimized.

Copy link

commented Jan 16, 2015

Interesting.. I do have a bash command in the path. I get the expected results (no prepended /cygdrive).

I found one one more interesting/annoying issue with rsync on Windows while doing further testing:

  1. the rsync command from cwRsync is version 3.0.6
    1. this version requires the leading /cygdrive and doesn't work with just the drive (/b/some/name)
  2. the rsync command from minGW (MSYS) is version 3.0.8
    1. this version works with just the drive (/b/some/name) and will not work with /cygdrive
    2. unfortunately, this version, with openSSH v5.2p1 in the same path, still doesn't work with Vagrant's "synced_folder" (vague "read error)

I see two easy solutions (for now)

  1. test for "/cygdrive" on the front of hostpath. Assume the user knows what they are doing and do not change hostpath at all.
  2. in cygwin_path(), to get the path string, always try to run cygpath, then try bash, then return unmodified instead of returning nothing.
    1. cwRsync package includes the cygpath.exe command.
      1. That means that modifying ENV as suggested by @osroot25 might just work if you make sure your path includes the cygpath.exe from cwRsync/bin.
    2. MSYS version of rsync might exist without any bash command
      1. Although I could not get this version (3.0.8) to work for me.
@MetaMemoryT

This comment has been minimized.

Copy link

commented Feb 5, 2015

👍 good research @1jerry

@rarkins

This comment has been minimized.

Copy link

commented Mar 1, 2015

Is there any rsync executable that "just works" with vagrant? (apart from a full cygwin install that is)

@jeancasman

This comment has been minimized.

Copy link

commented Jul 28, 2015

@Hunternnm

Got similar problem with win7x64, vagrant 1.7.4, cygwin (rsync + ssh) and Git Bash.
After struggling for hours and reading proposed solutions on GitHub, this is what worked for me:

  • Add ENV["VAGRANT_DETECTED_OS"] = ENV["VAGRANT_DETECTED_OS"].to_s + " cygwin" to Vagrantfile.
  • Add cygwin to PATH (before path to Git).
  • Use windows command line instead of Git Bash.

Hope this can help.

@vteivans

This comment has been minimized.

Copy link

commented Aug 12, 2015

My solution for GitBash + cygwin + Vagrant

After installing cygwin, add to your .bashrc:

# Add cygwin to path for rsync:
export PATH=$PATH:/c/cygwin/bin  # your cygwin path
# Tell vagrant, we have cygwin:
export VAGRANT_DETECTED_OS=cygwin
@thePanz

This comment has been minimized.

Copy link

commented Aug 14, 2015

Fixed with @vteivans comment (I am using just plain Cygwin, no GitBash):

# Tell vagrant, we have cygwin:
export VAGRANT_DETECTED_OS=cygwin
@WRidder

This comment has been minimized.

Copy link

commented Aug 17, 2015

Confirmed working on Windows 10 using cygwin (rsync + openssh); making sure cygwin is before GIT in PATH and following this guide: http://micahasmith.github.io/2015/01/22/coreos-vagrant-windows-file-share/.

@XedinUnknown

This comment has been minimized.

Copy link

commented Sep 16, 2015

I use GitBash without Cygwin. Still not sure what I should do. Any suggestions?

@StalkAlex

This comment has been minimized.

Copy link

commented Nov 16, 2015

The error is still present in version 1.7.4. "/cygdrive" was added, cygwin was installed and rsync, openssh inside. All variables in path exist. Nothing helped. What else should I do? Os: win 10.

@jamesvl

This comment has been minimized.

Copy link

commented Nov 17, 2015

While not directly related to this issue: cygwin rsync (and I think also cwRysnc) will both fail to run on the latest Windows 10 "major update" (from Nov. 12 2015). I had the cygwin rsync working flawlessly on Windows 8.1 and Windows 10 (and latest versions of Vagrant and Virtual Box); then it failed after applying the Windows service pack.

Errors are along the lines of

rsync 6132 C:\Tools_Data\utils\cygwin64\bin\rsync.EXE: *** fatal error in forked process - fork: can't reserve memory for parent stack 0x600000 - 0x800000, (child has 0x400000 - 0x600000), Win32 error 487
    529 [main] rsync 6132 cygwin_exception::open_stackdumpfile: Dumping stack trace to rsync.EXE.stackdump
      1 [main] rsync 4308 fork: child -1 - forked process 6132 died unexpectedly, retry 0, exit code 0x100, errno 11
rsync: fork: Resource temporarily unavailable (11)
rsync error: error in IPC code (code 14) at pipe.c(65) [sender=3.1.1]

I'm investigating how best to report the issue to cygwin.

@mitchellh

This comment has been minimized.

Copy link
Member

commented Nov 19, 2015

OKAY! This should finally be fixed (I hope) with Vagrant 1.8. We now automatically convert all paths for rsync on Windows to the cygwin path format since every implementation of cygwin requires that.

@mitchellh mitchellh closed this Nov 19, 2015

@kurttomlinson

This comment has been minimized.

Copy link

commented Nov 20, 2015

I'm running into the same problem as @jamesvl after updating to Windows 10 Threshold 2.

==> default: Rsyncing folder: /cygdrive/c/Users/Kurt/.../ => /www/...
      4 [main] rsync 14940 C:\cygwin64\bin\rsync.EXE: *** fatal error in forked process - fork: can't reserve memory for parent stack 0x600000 - 0x800000, (child has 0x400000 - 0x600000), Win32 error 487
   3301 [main] rsync 14940 cygwin_exception::open_stackdumpfile: Dumping stack trace to rsync.EXE.stackdump
      0 [main] rsync 4680 fork: child -1 - forked process 14940 died unexpectedly, retry 0, exit code 0x100, errno 11
There was an error when attempting to rsync a synced folder.
Please inspect the error message below for more info.

Host path: /cygdrive/c/Users/Kurt/.../
Guest path: /www/...
Command: rsync --verbose --archive --delete -z --copy-links --chmod=ugo=rwX --no-perms --no-owner --no-group --rsync-path sudo rsync -e ssh -p 2222 -o StrictHostKeyChecking=no -o IdentitiesOnly=true -o UserKnownHostsFile=/dev/null -i 'C:/Users/Kurt/...' --exclude .vagrant/ /cygdrive/c/Users/Kurt/.../ vagrant@127.0.0.1:/www/...
Error:       4 [main] rsync 14940 C:\cygwin64\bin\rsync.EXE: *** fatal error in forked process - fork: can't reserve memory for parent stack 0x600000 - 0x800000, (child has 0x400000 - 0x600000), Win32 error 487
   3301 [main] rsync 14940 cygwin_exception::open_stackdumpfile: Dumping stack trace to rsync.EXE.stackdump
      0 [main] rsync 4680 fork: child -1 - forked process 14940 died unexpectedly, retry 0, exit code 0x100, errno 11
rsync: fork: Resource temporarily unavailable (11)
rsync error: error in IPC code (code 14) at pipe.c(65) [sender=3.1.1]
@kurttomlinson

This comment has been minimized.

Copy link

commented Nov 20, 2015

I fixed my issue, I had the 64-bit version of cygwin installed. I deleted that and installed the 32-bit version instead. Now it's working fine again on Windows 10. FYI @jamesvl

@stanicek

This comment has been minimized.

Copy link

commented Nov 27, 2015

Thanks @kurttomlinson ! Downgrading cygwin from 64-bit to 32-bit version also worked for me on the latest Windows 10 update.

@ghost

This comment has been minimized.

Copy link

commented Dec 1, 2015

Downgrading to 32-bit didnt worked.
Solution for me was downloading cwRsync https://www.itefix.net/content/cwrsync-free-edition
and placed all files into C:\Git\mingw64\bin

@arthuroy

This comment has been minimized.

Copy link

commented Dec 1, 2015

I had the same issue, fork: can't reserve memory for parent stack, using rsync in msys2-x86_64-20150916.exe from http://msys2.github.io/ on Microsoft Windows 10.0.10586.

0 [main] rsync 8900 C:\msys64\usr\bin\rsync.EXE: *** fatal error in forked process - fork: can't reserve memory for parent stack 0x600000 - 0x800000, (child has 0x400000 - 0x600000), Win32 error 487

And I fixed my issue by using the 32-bit msys2-i686-20150916.exe (32-bit) instead.
Now I can see that rsync works as expected:

==> centos-atomic-host-01: Configuring and enabling network interfaces...
==> centos-atomic-host-01: Rsyncing folder: /d/vagrant-docker-ansible/ => /home/vagrant/sync
@firestalk

This comment has been minimized.

Copy link

commented Dec 3, 2015

In case of someone will still have an issue after previous instructions, don't forget to install rsync and openssh in msys2 32bit using pacman.

@arthuroy

This comment has been minimized.

Copy link

commented Dec 3, 2015

Guess the reason why rsync from msys2-x86_64 failed is vagrant is running 32-bit mode. So you have to use 32-bit msys2's rsync.

@firestalk

This comment has been minimized.

Copy link

commented Dec 3, 2015

What weird is that I used msys2 64 bit before and it worked flawlessly. Problems started after windows 10 recent upgrade.

@arthuroy

This comment has been minimized.

Copy link

commented Dec 3, 2015

@firestalk Thanks for your information.
I wanna use msys2 64-bit as well, but... :trollface:
Using msys2 32-bit is a workaround on Windows 10 build 10586 (Threshold 2).

@rekcodocker

This comment has been minimized.

Copy link

commented Dec 16, 2015

Here's a note from 16-12-2015.
The bug is still there.

I am using cygwin 64 bit, I am not looking forward to downgrading.
And the fix at line 74 still works.

It seems like the cygwin-detection of this helper script is faulty and either fails to detect cygwin, or fails to construct the proper path.

This error occurs when using a Debian-box (jesse, 8.2.2), but not with Ubuntu (ubuntu still shares directory with vboxsf).

Please... fix this. This error stems from 2014, it's a year and a half old.

@arthuroy

This comment has been minimized.

Copy link

commented Dec 22, 2015

rsync doesn't work on Vagrant 1.8.0, was ok on 1.7.4.

rsync & openssh from msys2-i686-20150916.exe (32-bit).

    coreos-01: Rsyncing folder: /d/vagrant-lab/ => /vagrant
There was an error when attempting to rsync a synced folder.
Please inspect the error message below for more info.

Host path: /d/vagrant-lab/
Guest path: /vagrant

Command: rsync --verbose --rsync-path='sudo rsync' --archive --delete -zz --chmod=ugo=rwX --no-perms --no-owner --no-group --rsync-path sudo rsync -e ssh -p 2222 -o ControlMaster=auto -o ControlPath=C:/msys32/tmp/ssh.702 -o ControlPersist=10m -o StrictHostKeyChecking=no -o IdentitiesOnly=true -o UserKnownHostsFile=/dev/null -i 'C:/Users/<user_name>/.vagrant.d/insecure_private_key' --exclude .vagrant/ /d/vagrant-lab/ core@127.0.0.1:/vagrant
Error: Warning: Permanently added '[127.0.0.1]:2222' (ED25519) to the list of known hosts.
mm_receive_fd: no message header
process_mux_new_session: failed to receive fd 0 from slave
mux_client_request_session: read from master failed: Connection reset by peer
Failed to connect to new control master
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(226) [sender=3.1.1]

see also #4586

@adiospeds

This comment has been minimized.

Copy link

commented Sep 9, 2016

@jeancasman 's solution worked for me - especially the switch to windows cmd just for vagrant up part 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.