Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rsync support #1266

Closed
ceztko opened this issue Oct 6, 2018 · 8 comments
Closed

Rsync support #1266

ceztko opened this issue Oct 6, 2018 · 8 comments

Comments

@ceztko
Copy link

ceztko commented Oct 6, 2018

OpenSSH for Windows version
7.7.2.0 (both on client and server)

Server Operating System
Windows Server 2016 Datacenter 1607 14393.2515

Client Operating System
Windows 10 Pro 1803 17134.320

What is failing
One of the most useful use cases when using ssh transport is rsync file synchronization. rysnc is unfortunately one of the very useful utilities that didn't receive a native Win32 port, only cygwin/msys2 ports. These are self-contained in only 3 files:

  • rsync.exe
  • cygwin1.dll (or msys-2.0.dll);
  • cygiconv-2.dll (or msys-iconv-2.dll).

I attached the cygwin port here for convenience, but the files can be easily stripped from a cygwin or msys2 installation:
rsync-cygwin.zip

Rsync must be present on both client and server to allow file synchronization. Maybe unsurprisingly the cygwin/msys2 port of rsync doesn't work together Win32-OpenSSH on client side, but works very well on the server side, making it very interesting experiment!

Expected output

NOTE1: a cygwin/msys2 port of OpenSSH must be installed on client side!
NOTE2: a cygwin/msys2 port of rsync must be present on server side and visible in PATH!

The following works correctly client side when launched inside cygwin shell using cygwin OpenSSH port:

ceztko@laptopbbk ~
$ rsync -avz -e ssh /cygdrive/c/Test/ Administrator@remotehost:/cygdrive/c/Test/
Administrator@remotehost's password:
sending incremental file list
./
Test.txt

sent 147 bytes  received 38 bytes  10.57 bytes/sec
total size is 3  speedup is 0.02

Obviously having cygwin installed you can achieve the same from a regular command prompt:

C:\cygwin64\bin\bash --login -c "rsync -avz -e ssh /cygdrive/c/Test/ Administrator@remotehost:/cygdrive/c/Test/"

Actual output

NOTE: Win32-OpenSSH ssh.exe must be present on PATH

Forcing cygwin/msys2 port of rsync to use Win32-OpenSSH results in the following errors:

C:\Users\ceztko>rsync -avz -e ssh /cygdrive/d/Test/ Administrator@remotehost:/cygdrive/c/Test/
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(226) [Receiver=3.1.2]
rsync: safe_read failed to read 4 bytes [sender]: Connection reset by peer (104)
rsync error: error in rsync protocol data stream (code 12) at io.c(276) [sender=3.1.2]

My guess is that this is happening because IPC communication between cygwin rsync and Win32-OpenSSH is not compatible. Questions:

  • Can this be addressed/fixed?
  • Alternatively, do you have plans to also port rsync natively to Win32?
@manojampalam
Copy link
Contributor

Unfortunately, this is not in this project's scope. I guess however that you could leverage posix library that is part of this project to create a native port of rsync for Windows.

@ceztko
Copy link
Author

ceztko commented Oct 9, 2018

Unfortunately, this is not in this project's scope. I guess however that you could leverage posix library that is part of this project to create a native port of rsync for Windows.

Thanks for the prompt answer. Yes, I also guessed that rsync support could have not been in the scope of the project. I just hoped that Microsoft wanted to support it since it's extremely useful use case.
Please, can you point me more reference to the posix library that you're mentioning? Is it something related to Windows Subsystem for Linux (WSL) or is it something more similar to cygwin/msys2? Is so, how it differs from these approaches?

@mrfrenzy
Copy link

First of all thankyou for your post about extracting files from msys2. It works great as an rsync server with Win32-OpenSSH.

Did you see the old msvcrt port of rsync? Perhaps that could be used for some ideas when reworking with the new libraries.
http://rsyncwin32.sourceforge.net/

@AndrewSav
Copy link

For me, this worked as a work-around: http://www.mls-software.com/opensshd.html It's also cygwin based, but it's smaller and intstalls only needed dependncies. E.g.:

$ssh = "C:\Program Files\OpenSSH\bin\ssh.exe"
$rsync = "C:\Program Files\OpenSSH\bin\rsync.exe"
$sshkeyscan = "C:\Program Files\OpenSSH\bin\ssh-keyscan.exe"

...


"Copying configuration scripts to $master..." | Write-Host
. $rsync -e "'$ssh'" -r ../scripts "$($master):" | Write-Host

@AndrewSav
Copy link

@maertendMSFT can you comment how this was resolved please?

@bagajjal
Copy link
Collaborator

bagajjal commented May 5, 2020

It's out of scope for this project.

@ValentinChirikov
Copy link

Hello, first of all thank's for the Project.
For all who found this message in 2022 - rsync works perfect with latest Win32-OpenSSH release and Chocolatey rsync package

@jhgorse
Copy link

jhgorse commented May 1, 2023

It is true that rsync has been released for windows via cygwin under choco. It is a bit tricky to use.

from remote win to unix

rsync -e "ssh -p 22" user@hostname_or_ip:/cygdrive/c/path_to_src /path_to_dst

from local powershell win to unix

rsync -e 'C:\ProgramData\Chocolatey\lib\rsync\tools\bin\ssh.exe' /cygdrive/c/path_to_src user@hostname_or_ip:/path_to_dst

The other two combinations are left as an exercise for the reader.

Note that the options passed after are use-case specific, but for reference mine were -hvrPt.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

8 participants