Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rsync doesn't work on Vagrant 1.8.0 #6702

Closed
arthuroy opened this issue Dec 22, 2015 · 87 comments
Closed

rsync doesn't work on Vagrant 1.8.0 #6702

arthuroy opened this issue Dec 22, 2015 · 87 comments

Comments

@arthuroy
Copy link

@arthuroy arthuroy commented Dec 22, 2015

rsync doesn't work on Vagrant 1.8.0, but it was ok on 1.7.4.

  • Microsoft Windows 10 build 10586
  • Vagrant 1.8.0
  • rsync 3.1.1-1 from msys2-i686-20150916.exe (32-bit)
  • openssh 7.1p1-1 from msys2-i686-20150916.exe (32-bit)

From MSYS2 Shell:

    coreos-01: Rsyncing folder: /d/vagrant-lab/ => /vagrant
There was an error when attempting to rsync a synced folder.
Please inspect the error message below for more info.

Host path: /d/vagrant-lab/
Guest path: /vagrant

Command: rsync --verbose --archive --delete -z --copy-links --chmod=ugo=rwX --no-perms --no-owner --no-group --rsync-path sudo rsync -e ssh -p 2222 -o ControlMaster=auto -o ControlPath=C:/msys32/tmp/ssh.740 -o ControlPersist=10m -o StrictHostKeyChecking=no -o IdentitiesOnly=true -o UserKnownHostsFile=/dev/null -i 'C:/Users/<user_name>/.vagrant.d/insecure_private_key' --exclude .vagrant/ /d/vagrant-lab/ core@127.0.0.1:/vagrant

Error: This rsync lacks old-style --compress due to its external zlib.  Try -zz.
Continuing without compression.

Warning: Permanently added '[127.0.0.1]:2222' (ED25519) to the list of known hosts.

mm_receive_fd: no message header
process_mux_new_session: failed to receive fd 0 from slave
mux_client_request_session: read from master failed: Connection reset by peer
Failed to connect to new control master
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(226) [sender=3.1.1]

see also #4586 #4073

@arthuroy
Copy link
Author

@arthuroy arthuroy commented Dec 22, 2015

From Cygwin Terminal:

    coreos-01: Rsyncing folder: /cygdrive/d/vagrant-lab/ => /vagrant
There was an error when attempting to rsync a synced folder.
Please inspect the error message below for more info.

Host path: /cygdrive/d/vagrant-lab/
Guest path: /vagrant

Command: rsync --verbose --archive --delete -z --copy-links --chmod=ugo=rwX --no-perms --no-owner --no-group --rsync-path sudo rsync -e ssh -p 2222 -o ControlMaster=auto -o ControlPath=C:/cygwin/tmp/ssh.625 -o ControlPersist=10m -o StrictHostKeyChecking=no -o IdentitiesOnly=true -o UserKnownHostsFile=/dev/null -i 'C:/Users/<user_name>/.vagrant.d/insecure_private_key' --exclude .vagrant/ /cygdrive/d/vagrant-lab/ core@127.0.0.1:/vagrant

Error: Warning: Permanently added '[127.0.0.1]:2222' (ED25519) to the list of known hosts.

mm_receive_fd: no message header
process_mux_new_session: failed to receive fd 0 from slave
mux_client_request_session: read from master failed: Connection reset by peer
Failed to connect to new control master
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(226) [sender=3.1.1]
@arthuroy
Copy link
Author

@arthuroy arthuroy commented Dec 22, 2015

Another issue introduced in #6568 - commit 6e6172e

WORKAROUND

  • Edit $VAGRANT_HOME\embedded\gems\gems\vagrant-1.8.0\plugins\synced_folders\rsync\helper.rb
  • Remove the following codes (line 77~79):
"-o ControlMaster=auto " +
"-o ControlPath=#{controlpath} " +
"-o ControlPersist=10m " +
@jraviotta
Copy link

@jraviotta jraviotta commented Dec 22, 2015

@arthuroy thank you. Removing lines 77-79 from helper.rb worked for me.

@berlincount
Copy link

@berlincount berlincount commented Dec 25, 2015

I can confirm.

Windows 10 Build 10240
Vagrant 1.8.0

Same issue, fix helps the same way :)

@matevarga
Copy link

@matevarga matevarga commented Dec 26, 2015

Same issue here on 1.8.1. Removed the lines, now it works.

@mariomka
Copy link

@mariomka mariomka commented Dec 27, 2015

Windows 10, Vagrant 1.8.0, fix working <3

@stellis
Copy link

@stellis stellis commented Dec 27, 2015

I was able to reproduce this with 1.8.1 on Windows 10

@Flightfreak
Copy link

@Flightfreak Flightfreak commented Dec 28, 2015

Windows 8.1 64, Vagrant 1.8.1, removed above lines and it works.

@davincho
Copy link

@davincho davincho commented Dec 29, 2015

Thanks for the workaround - Windows 10, Vagrant 1.8.0

@thewebists
Copy link

@thewebists thewebists commented Jan 4, 2016

+1 For the solution.

When Googling this issue I came across mentions that there are problems with using SSH ControlMaster under Cygwin.

Maybe there should be an environment check on this code.

@chuckyz
Copy link

@chuckyz chuckyz commented Jan 4, 2016

Can anyone confirm if this is only on MinGW (Cygwin) versions of OpenSSH?

If so, a simple check like...

if (/mingw/ =~ RUBY_PLATFORM) == nil
  - insert ControlMaster statements
end

Would work.

A way to check in Windows is with irb in a Powershell prompt and just typing RUBY_PLATFORM. Cygwin should install this by default.

Example:

irb(main):001:0> RUBY_PLATFORM
=> "i386-mingw32"
irb(main):002:0>
@jamiejackson
Copy link

@jamiejackson jamiejackson commented Jan 5, 2016

Thanks, #6702 (comment) got me running on Windows 7/Cygwin/Vagrant 1.8.1. (Obviously, my path has a vagrant-1.8.1 node, instead of vagrant-1.8.0, though.)

@huangsam
Copy link

@huangsam huangsam commented Jan 7, 2016

I'm currently using cwRsync 3.1.0 on Windows 7/Git Bash/Vagrant 1.8.1. The previous comment was not enough for my machine as it caused the following error:

Error: Warning: Permanently added '[127.0.0.1]:2222' (ECDSA) to the list of known hosts.
rsync: change_dir "/c/Users/huangsam/git/sample" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1165) [sender=3.1.0]

I used solutions from #3230 to resolve my problem.

@ryanlraines
Copy link

@ryanlraines ryanlraines commented Jan 10, 2016

Win10 Pro
Virtual Box 5.0
Vagrant 1.8.1
rsync 3.1.1
centos/7 box

Same symptoms as first reported. I tried the workaround (remove lines 77-79), but I'm still experiencing the same symptoms.

Any Ideas? Thanks.

@reedolph
Copy link

@reedolph reedolph commented Jan 11, 2016

Had the same issue on Windows 10 / VirtualBox 5 / MSYS2 rsync & openssh (from msys2-i686-20150916.exe) / Vagrant 1.8.1.

Workaround in #6702 (comment) fixed it, thanks for that.

@redcenter
Copy link

@redcenter redcenter commented Jan 20, 2016

Same issue on:

  • Windows 10
  • Vagrant 1.8.1
  • VirtualBox 5.0.14
  • Rsync 3.1.1 (protocol version 31)

Workaround in the comment worked like a charm. Thanks!

@stellis
Copy link

@stellis stellis commented Jan 20, 2016

This issue is now a month old, and the fix seems to be very simple. This, along with a few other baffling issues, has been a disappointing look into quality assurance at HashiCorp.

I hope that this is fixed soon and that we're also given more insight into the testing and release process at HashiCorp (which seems as though it has a lot of room for improvement at the moment).

@hyfrey
Copy link

@hyfrey hyfrey commented Jan 25, 2016

Same problem here

  • Windows 8.1
  • Vagrant 1.8.1
  • VirtualBox 4.3.18
  • cwRsync 5.5.0

Workaround doesn't work.

Error: Warning: Permanently added '[127.0.0.1]:2222' (RSA) to the list of known
hosts.
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(605) [Receive
r=3.0.9]
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(226) [sender=
3.1.2]
@steroyle
Copy link

@steroyle steroyle commented Jan 25, 2016

I have been experiencing the same issue:

  • Windows 8.1
  • Vagrant 1.8.1
  • Virtual Box 4.3.20
  • rsync 3.1.0 (protocol version 31)

rsync error:

Error: Warning: Permanently added '[127.0.0.1]:2222' (ECDSA) to the list of known
hosts.
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(226) [sender=3.1.0]

I removed Vagrant 1.8.1 and installed an older version that I know works on another machine (1.7.4) and it now works fine.

@slavajacobson
Copy link

@slavajacobson slavajacobson commented Jan 27, 2016

Same problem here.

Windows 10
CYGWIN_NT-10.0 DESKTOP-O1A4SV5 2.4.1(0.293/5/3) 2016-01-24 11:26 x86_64 Cygwin
rsync v3.1.1
vagrant 1.8.1

@blah238
Copy link

@blah238 blah238 commented Jan 28, 2016

Same problem here -- workaround here works: #6702 (comment)

  • Windows 7 64-bit SP1
  • Git for Windows 64-bit SDK 1.0.1 (msys2-runtime 2.3.1.33884.84870b7-1)
  • rsync 3.1.2 (protocol version 31)
  • vagrant 1.8.1
  • virtualbox 5.0.14
@swquinn
Copy link

@swquinn swquinn commented Feb 1, 2016

I can confirm that on Windows 10 64-bit, with rsync 3.1.2, Vagrant 1.8.1 and VirtualBox 5.0.12 the workaround noted here: #6702 (comment) resolves issues I've had with rsync.

@davidski
Copy link

@davidski davidski commented Feb 7, 2016

Another 👍 on the fix from @arthuroy working for a Windows 10 x64, Vagrant 1.8.1, VMWare Workstation user. Any word from a Hashicorp org person on getting a fix into a released version? Makes it really hard to promote Vagrant internally when these sorts of basics don't work out of the box.

@sysarcher
Copy link

@sysarcher sysarcher commented Aug 17, 2016

@frimik I don't think Git bash has rsync. I use rsync that comes with Cygwin. It works.

@frimik
Copy link

@frimik frimik commented Aug 17, 2016

@shrmrf indeed, but rsync in turn use ssh, and it uses whatever client is on path. And while in the git Bash shell where I run pretty much all my things, it picks that SSH client.

Removing the ControlPersist option rolls onto the next issue though.

Host path: /cygdrive/c/Users/foo
rsync: change_dir "/cygdrive/c/Users/foo" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at /usr/src/rsync/rsync-3.0.8

$ pwd
/c/Users/foo

my Rsync (mingw) expects /c/Users/... style paths ....

if Vagrant::Util::Platform.windows?
          # rsync for Windows expects cygwin style paths, always.
          hostpath = Vagrant::Util::Platform.cygwin_path(hostpath)
        end

Guess I need to decouple these things :). For now I've hacked both things above and my rsync works at the moment :).

@sysarcher
Copy link

@sysarcher sysarcher commented Aug 17, 2016

I see...
I tested with mingw (paths below):

> which ssh
/c/MinGW/msys/1.0/bin/ssh
> which rsync
/c/MinGW/msys/1.0/bin/rsync

I can confirm that this works too.

I don't think this is the right place to ask.. but do you know if we can use Bash on Ubuntu on Windows feature recently rolled out with Vagrant? Would be awesome!

@eug3n
Copy link

@eug3n eug3n commented Aug 22, 2016

@frimik Thank you for pointing out that this is broken with the SSH client that comes with Git Bash.

Installing openssh and rsync packages with mingw-get (http://www.mingw.org/wiki/getting_started) and copying all files from C:\MinGW\msys\1.0\bin to C:\Program Files\Git\mingw64\bin solved the issue for me (Windows 10, Vagrant 1.8.5).

@dnadle
Copy link

@dnadle dnadle commented Aug 22, 2016

You don't have to overwrite your Git installation, just put the MinGW bin folder earlier in your PATH than then any other rsync binary.

@blah238
Copy link

@blah238 blah238 commented Aug 22, 2016

You can also uninstall the normal Git for Windows and install the Git for Windows SDK which has a package manager that can install rsync.

@czukowski
Copy link

@czukowski czukowski commented Aug 23, 2016

Managed to resolve the issue by installing msys2 using MSYS2 installer, then installing rsync package with pacman -S rsync (I suppose this step will be the same as with Git for Windows SDK) and moving the resulting rsync.exe to the Git's usr/bin folder (sys2 installation can then be removed). Not sure what else is included in GfW SDK, I gave this approach a try to avoid ending up with the stuff I didn't really need.

@czukowski
Copy link

@czukowski czukowski commented Aug 23, 2016

Note: using just cwRsync isn't going to work as of now, because it expects "cygwin-branded" paths (like /cygwin/c/path/to/my/repo/) and Vagrant will generate that only if it detects a full Cygwin installation.

@baderas
Copy link

@baderas baderas commented Sep 26, 2016

This error is still occuring when using msys2 and vagrant:
Versions:
vagrant 1.8.5
rsync version 3.1.2 protocol version 31

I am also using the openstack plugin from https://github.com/ggiamarchi/vagrant-openstack-provider

Error:

==> default: Rsyncing folder: XXXXXXXXXX => /vagrant
There was an error when attempting to rsync a share folder.
Please inspect the error message below for more info.

Host path: XXXXXXXXXX
Guest path: /vagrant
Error: This rsync lacks old-style --compress due to its external zlib.  Try -zz.
Continuing without compression.

Any ideas? Disabling rsync via Vagrantfile did not work.

Edit:
VAGRANT_LOG=info vagrant up --provider=openstack --debug showd me that it is a sys2 related issue: /cygwin/c/... does not exist in msys2.

@sysarcher
Copy link

@sysarcher sysarcher commented Sep 26, 2016

@baderas can you try checking rsync with an ubuntu/trusty64 machine?

It is working for me. Here's my env:

>which rsync
/c/MinGW/msys/1.0/bin/rsync

>which ssh
/c/MinGW/msys/1.0/bin/ssh

>rsync --version
rsync  version 3.0.8  protocol version 30
@baderas
Copy link

@baderas baderas commented Sep 26, 2016

Just for clarification: I'm using msys2, not msys.
I tried to fix it by editing /etc/fstab, but that did not work.
Using ubuntu/trusty64 is working without openstack plugin.
I also tried it with trusty on openstack - same error as above. It seems that https://github.com/ggiamarchi/vagrant-openstack-provider uses hard coded path to /cygwin/c/... which brakes it on msys2.

@baderas
Copy link

@baderas baderas commented Sep 26, 2016

I opened a bug report there: ggiamarchi/vagrant-openstack-provider#304

@taliesins
Copy link
Contributor

@taliesins taliesins commented Oct 6, 2016

I think we should be checking for the existence of cygpath in the path as well. Then as long as your ssh, rsync and cygpath work together you should be all right. No need to install cygwin then.

Here is the monkey patch I would apply to C:\HashiCorp\Vagrant\embedded\gems\gems\vagrant-1.8.6\lib\vagrant\util\platform.rb:

        def which(cmd)
          exts = ENV['PATHEXT'] ? ENV['PATHEXT'].split(';') : ['']
          ENV['PATH'].split(File::PATH_SEPARATOR).each do |path|
            exts.each { |ext|
              exe = File.join(path, "#{cmd}#{ext}")
              return exe if File.executable?(exe) && !File.directory?(exe)
            }
          end
          return nil
        end     

        # This takes any path and converts it from a Windows path to a
        # Cygwin or msys style path.
        #
        # @param [String] path
        # @return [String]
        def cygwin_path(path)       
          if cygwin?
            begin
              # First try the real cygpath
              process = Subprocess.execute("cygpath", "-u", "-a", path.to_s)
              return process.stdout.chomp
            rescue Errors::CommandUnavailableWindows
            end
          end

          if which('cygpath')
            # First try the real cygpath
            process = Subprocess.execute("cygpath", "-u", "-a", path.to_s)
            return process.stdout.chomp
          end

          # Sometimes cygpath isn't available (msys). Instead, do what we
          # can with bash tricks.
          process = Subprocess.execute(
            "bash",
            "--noprofile",
            "--norc",
            "-c", "cd #{Shellwords.escape(path)} && pwd")
          return process.stdout.chomp
        end
@0x63lv
Copy link

@0x63lv 0x63lv commented Nov 2, 2016

One thing to check, if the workarounds do not work and you have git bash installed on Windows, is which ssh and rsync is used. Had the same error, and turned out that was using rsync from cygwin, but ssh from git bash. Changed the sequence in %PATH%, so that cygwin bin dir is before git bin dir, and it worked.

@yoplait
Copy link

@yoplait yoplait commented Nov 18, 2016

Hi, same problem here,
<<<<<
Complete!
Copy iso file C:\Program Files\Oracle\VirtualBox\VBoxGuestAdditions.iso into the box /tmp/VBoxGuestAdditions.iso
mount: /dev/loop0 is write-protected, mounting read-only
Installing Virtualbox Guest Additions 5.1.8 - guest version is unknown
Verifying archive integrity... All good.
Uncompressing VirtualBox 5.1.8 Guest Additions for Linux...........
VirtualBox Guest Additions installer
Copying additional installer modules ...
Installing additional modules ...
vboxadd.sh: Building Guest Additions kernel modules.
vboxadd.sh: Starting the VirtualBox Guest Additions.

Could not find the X.Org or XFree86 Window System, skipping.

==> default: Checking for guest additions in VM...
==> default: Rsyncing folder: /cygdrive/c/vm/centos/ => /vagrant
There was an error when attempting to rsync a synced folder.
Please inspect the error message below for more info.

Host path: /cygdrive/c/vm/centos/
Guest path: /vagrant
Command: "rsync" "--verbose" "--archive" "--delete" "-z" "--copy-links" "--chmod=ugo=rwX" "--no-perms" "--no-owner" "--no-group" "--rsync-path" "sudo rsync" "-e" "ssh -p 2222 -o LogLevel=FATAL -o IdentitiesOnly=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i 'C:/vm/centos/.vagrant/machines/default/virtualbox/private_key'" "--exclude" ".vagrant/" "/cygdrive/c/vm/centos/" "vagrant@127.0.0.1:/vagrant"
Error: rsync: Failed to exec ssh: No such file or directory (2)
rsync error: error in IPC code (code 14) at pipe.c(85) [sender=3.1.2]
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(226) [sender=3.1.2]

C:\vm\centos>vagrant ssh
ssh executable not found in any directories in the %PATH% variable. Is an
SSH client installed? Try installing Cygwin, MinGW or Git, all of which
contain an SSH client. Or use your favorite SSH client with the following
authentication information shown below:

Host: 127.0.0.1
Port: 2222
Username: vagrant
Private key: C:/vm/centos/.vagrant/machines/default/virtualbox/private_key

C:\vm\centos>vagrant --version
Vagrant 1.8.7

C:\vm\centos>ver

Microsoft Windows [Version 10.0.14393]

C:\vm\centos>

@ulkuniem
Copy link

@ulkuniem ulkuniem commented Nov 22, 2016

Had the rsync issue aswell using Git Bash.
Solution:
Installed MinGW with Rsync and OpenSSH with MinGW Installation Manager Gui
C:\MinGW\libexec\mingw-get\guimain.exe

Added:
export PATH=/c/MinGW/msys/1.0/bin:$PATH
at the end of
"C:\Program Files\Git\etc\bash.bashrc"

So both ssh and rsync are used from C:/MinGW/msys/1.0/bin instead of Git Bash Mingw64 installation.

@valoujjal
Copy link

@valoujjal valoujjal commented Nov 22, 2016

@ulkuniem How to export path in windows10

@ulkuniem
Copy link

@ulkuniem ulkuniem commented Nov 22, 2016

@valoujjal
If you want to just export path to current console (and session) there are two ways:
CMD.exe:
set PATH=C:/MinGW/msys/1.0/bin;%PATH%
PowerShell.exe
$env:Path = "C:/MinGW/msys/1.0/bin;$env:Path"

If you want to persistent edit to the PATH environment variable:
-In Search, search for and then select: System (Control Panel)
-Click the Advanced system settings link.
-Click Environment Variables. ...
-In the Edit System Variable (or New System Variable) window, specify the value of the PATH environment variable.

If you mean to export like in bashrc, there are ways.
Powershell has profile.ps1 files.
The location is in $profile varialbe.
It might not exist so create one with
New-Item $profile -ItemType File
Edit with
Notepad.exe $profile
and add the $env:PATH modification.
More on the powershell profiles
technet

and with cmd.exe profiles I'm not that familiar with but fast googling gave me this blog post that might be helpful: cachemiss

@Dannymx
Copy link

@Dannymx Dannymx commented Nov 29, 2016

@ulkuniem Adding the path to the Git bashrc worked for me! (I added the path to cygwin though).

@vladap
Copy link

@vladap vladap commented Apr 7, 2017

It is line 53 on Vagrant 1.9.3

@superhawk610
Copy link

@superhawk610 superhawk610 commented Jan 10, 2018

@ulkuniem Thanks! I only installed rsync via MinGW and that seemed to solve the issue for me, I didn't have to install ssh.

@andilabs
Copy link

@andilabs andilabs commented Jan 19, 2018

Running vagrant 1.9.4 on Mac OS X edited:
/opt/vagrant/embedded/gems/gems/vagrant-1.9.4/plugins/synced_folders/rsync/helper.rb commented out line 84 #control_options = "-o ControlMaster=auto -o ControlPath=#{controlpath} -o ControlPersist=10m " but still no luck

[localhost] local: rsync  --exclude ".git" --exclude "*.pyc" -pthrvz  --rsh='ssh -i /Users/andi/Projects/acorn/shopify-data-fetcher/.vagrant/machines/default/virtualbox/private_key -p 2200  ' temp/shopify-data-fetcher/ vagrant@127.0.0.1:/var/www/shopify_data_fetcher
building file list ... done
rsync: connection unexpectedly closed (8 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at /BuildRoot/Library/Caches/com.apple.xbs/Sources/rsync/rsync-52/rsync/io.c(453) [sender=2.6.9]

Fatal error: local() encountered an error (return code 12) while executing 'rsync  --exclude ".git" --exclude "*.pyc" -pthrvz  --rsh='ssh -i /Users/andi/Projects/acorn/shopify-data-fetcher/.vagrant/machines/default/virtualbox/private_key -p 2200  ' temp/shopify-data-fetcher/ vagrant@127.0.0.1:/var/www/shopify_data_fetcher'
@hashibot
Copy link

@hashibot hashibot bot commented Mar 31, 2020

I'm going to lock this issue because it has been closed for 30 days . This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@hashibot hashibot bot locked and limited conversation to collaborators Mar 31, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet