Rsync issue on Windows - hostname not same as transfer hostname

[inuwan]

I have cwRsync on Window. In my flightplan I have:

plan.remote('windows', function(remote) {
remote.log('Preparing destination...');
remote.exec('rm -rf ' + remoteWindowsPath);
remote.exec('mkdir ' + remoteWindowsPath);
});

// fly windows:
plan.local('windows', function(local) {
local.log('Copy files to remote hosts');
var curDir = local.exec('chdir').stdout + '';
local.transfer(curDir + 'myfile', remoteWindowsPath + '/build');
})

I get the following results:

myremotehost.com ● Preparing destination...
myremotehost.com $ rm -rf ~/build
myremotehost.com ● ok
myremotehost.com $ mkdir ~/build
myremotehost.com ● ok

localhost ● Copy files to remote hosts
localhost $ chdir
localhost > C:\src
localhost ● ok
localhost $ rsync --files-from C:\Users\Ken\AppData\Local\Temp\e228633a-7454-46df-8f52-bf9ad0c72323 -az --rsh="ssh -p22"
./ me@myremotehost.com:~/build
localhost > --files-from hostname is not the same as the transfer hostname
localhost > rsync error: syntax or usage error (code 1) at main.c(1293) [sender=3.1.1]
localhost ● failed with exit code: 1
✈ Error: rsync --files-from C:\Users\Me\AppData\Local\Temp\e228633a-7454-46df-8f52-bf9ad0c72323 -az --rsh="ssh -p22" ./ me@myremotehost.com:~/build failed on localhost

The file C:\Users\Me\AppData\Local\Temp\e228633a-7454-46df-8f52-bf9ad0c72323 contains:

   C:\src\myfile

I cannot determine how to fix the rysync "--files-from hostname is not the same as the transfer hostname" error. I tried:

rsync myfile -az --rsh="ssh -p22" ./ me@myremotehost.com:~/build

And that transfers the file ok. So it appears the issue is could be resolved by eliminating the --files-from option.

[pstadler]

Please upgrade to v0.4.4, this should fix the problem.

[inuwan]

Installed v0.4.5 and I'm still getting the same error.

[pstadler]

Could you please test if this works. Replace <path-to-tmp-file> with the file created by Flightplan, e.g.
C:\Users\Me\AppData\Local\Temp\e228633a-7454-46df-8f52-bf9ad0c72323

rsync --files-from `cygpath -u '<path-to-tmp-file>'` -az --rsh="ssh -p22" ./ me@myremotehost.com:~/build

[inuwan]

Get this error:

C:\dev\depot\MoneyFinder>rsync --files-from "cygpath -u 'C:\Users\Ken\AppData\Local\Temp\4903901a-552b-493a-9012-d
220dbee3738'" -az --rsh="ssh -p22" ./ me@myremotehost.com:~/build
--files-from hostname is not the same as the transfer hostname
rsync error: syntax or usage error (code 1) at main.c(1293) [sender=3.1.1]

[pstadler]

Use backticks (`) instead of double quotes (") and try again. Just copy the command from the post above if unclear.

[inuwan]

Doesn't like the backticks - also tried double quotes around with the same result

rsync --files-from cygpath -u 'C:\Users\Ken\AppData\Local\Temp\4903901a-552b-493a-9012-d220dbee3738' -az --rsh="ssh -p22" ./ me@myremotehost.com:~/build
rsync version 3.1.1 protocol version 31
Copyright (C) 1996-2014 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
64-bit files, 64-bit inums, 32-bit timestamps, 64-bit long ints,
socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
append, ACLs, no xattrs, iconv, symtimes, prealloc

rsync comes with ABSOLUTELY NO WARRANTY. This is free software, and you
are welcome to redistribute it under certain conditions. See the GNU
General Public Licence for details.

rsync is a file transfer program capable of efficient remote update
via a fast differencing algorithm.

Usage: rsync [OPTION]... SRC [SRC]... DEST
or rsync [OPTION]... SRC [SRC]... [USER@]HOST:DEST
or rsync [OPTION]... SRC [SRC]... [USER@]HOST::DEST
or rsync [OPTION]... SRC [SRC]... rsync://[USER@]HOST[:PORT]/DEST
or rsync [OPTION]... [USER@]HOST:SRC [DEST]
or rsync [OPTION]... [USER@]HOST::SRC [DEST]
or rsync [OPTION]... rsync://[USER@]HOST[:PORT]/SRC [DEST]
The ':' usages connect via remote shell, while '::' & 'rsync://' usages connect
to an rsync daemon, and require SRC or DEST to start with a module name.

[south-fish]

Did you manage to work around it somehow?

[south-fish]

when you run this command
rsync --files-from d4734e87-8382-4914-a331-a9736be26001 -az --rsh="ssh -p22" ./ anton@ubuntu:~/
from local directory, so that --files-from has no path but only file name it works fine. So it i something wrong with paths in Windows

[pstadler]

The problem is that I have to figure out if you're on cygwin and then transform this path. There's this tool cygpath (see my comments above). Can you try this and let me know if a path translated with cygpath works for you?

[inuwan]

Still an issue for me

[south-fish]

@inuwan btw, do you try flightplan with cwRsync or rsync what is coming with cygwin?
@pstadler cygwin, but will try it one more time as you recommended with cygpath

[south-fish]

Just checked:

It doen't work when the path like 'C:\cygwin64\tmp\91393402-b871-4e58-b302-7b723153f9cf'
ie this
rsync --files-from C:\cygwin64\tmp\91393402-b871-4e58-b302-7b723153f9cf -az --rsh="ssh -p22" ./ user@host:dest
doesnt work

but when the path is like '/cygdrive/c/cygwin64/tmp/abbb4566-8b68-4403-96f9-9155203d067d', works fine:
rsync --files-from /cygdrive/c/cygwin64/tmp/abbb4566-8b68-4403-96f9-9155203d067d -az --rsh="ssh -p22" ./ user@host:dest

but yesteday I tested it with cwRsync, today I insalled rsync for cygwin and it works

[pstadler]

To sum it up: rsync (not cwRsync) in combination with cypath works for you?

[south-fish]

this command

rsync --files-from /cygdrive/c/cygwin64/tmp/3704f930-ac80-403c-beb3-c5e0605b988b -az --rsh="ssh -p22 -i D:\path_to_keys\key.pem" ./ ubuntu@host:~/

works fime for me

[pstadler]

thanks man. one more question, what do you get when running these commands?

cygpath -u "C:\cygwin64\tmp\91393402-b871-4e58-b302-7b723153f9cf"

cygpath -u -p "C:\cygwin64\tmp\91393402-b871-4e58-b302-7b723153f9cf"

[south-fish]

Sure

$ cygpath -u "C:\cygwin64\tmp\91393402-b871-4e58-b302-7b723153f9cf"
/tmp/91393402-b871-4e58-b302-7b723153f9cf

$ cygpath -u -p "C:\cygwin64\tmp\91393402-b871-4e58-b302-7b723153f9cf"
/tmp/91393402-b871-4e58-b302-7b723153f9cf

[pstadler]

what happens when you use this form of the path (i.e. without the /cygdrive/c/cygwin64/ prefix) with rsync?

Thanks for your collaboration. I'm about to set up a VM with cygwin installed.

[south-fish]

doesn't work, failed to open file.
would it be an option to use scp instead of rsync? Should not be such issues.

[pstadler]

I'll investigate the possibilities. Thanks again.

[south-fish]

np :)

[south-fish]

hey, pls take a look how it can be fixed for cygwing / Windows

south-fish@aeecad5

[pstadler]

I don't really want Flightplan to write files into your repository, but surely this would be a pretty simple workaround.

[south-fish]

yeah, agree, but this workaround works

[south-fish]

maybe, one option is to try scp, it works fine on Windows or to use symlinks

[pstadler]

I just realized that cygwin is not officialy supported by node.js and it's not really working for me. What's the setup you're using?

[south-fish]

Cygwin with rsync, node.js, npm, Windows 7.
With workaround I posted works fine for me.
But what error do you see?

Node.js fix for cygwin
http://soyuka.me/using-nodejs-with-cygwin-v0-10-25/

yeah, Windows is not the best environment for node development :)

[mluogh]

Ok, so I ran this
"rsync --files-from /cygdrive/c/cygwin64/tmp/3704f930-ac80-403c-beb3-c5e0605b988b -az --rsh="ssh -p22 -i D:\path_to_keys\key.pem" ./ ubuntu@host:~/"

And it copied all the files to my web server. So how do I actually activate the flightplan functionality from there?

[joshunger]

You could re-define your TMPDIR -

if (process.env['TMPDIR'] === 'C:\\cygwin\\tmp') // or whatever logic you want
{
    process.env['TMPDIR'] = '/tmp';
}

var plan = require('flightplan');

[AlbanMinassian]

Hello

Why not use https://www.npmjs.com/package/scp2 !

scp2 is greatly powered by ssh2, implemented the scp in a sftp way.
It is written in pure javascript, and should work on every OS, even Windows. Nodejs (v0.8.7 or newer) is required to make it work.

var client = require('scp2');
...
plan.remote("default", function(remote) {
    var result = remote.waitFor(function(done) {
        client.scp('file.txt', remote.runtime.username + ':' + remote.runtime.password + '@' + remote.runtime.host + ':/var/www/', function(err) {
            if (err) { plan.abort('Severe turbulences over the atlantic ocean!'); }
            done('sent with scp2')
        })
    });
});

Copy a file to the server and rename it:

client.scp('file.txt', 'admin:password@example.com:/home/admin/rename.txt', function(err) {
})

Copy a directory to the server:

client.scp('data/', 'admin:password@example.com:/home/admin/data/', function(err) {
})

Copy via glob pattern:

client.scp('data/*.js', 'admin:password@example.com:/home/admin/data/', function(err) {
})

ami44

[MarcelRobitaille]

I am still having problems. Could someone please help me?

[joshunger]

What's the exception? What is your code?

[MarcelRobitaille]

Here is my flightplan file:

var plan = require('flightplan');

plan.target('production', [{
    host: '50.87.144.81',
    username: 'mjr',
    port: 2222,
    agent: process.env.SSH_AUTH_SOCK
}]);

plan.local(function(local) {
    var filesToCopy = local.exec('git ls-files', {silent: true});
    local.transfer(filesToCopy, 'public_html/foyerecolecsrs.ca/');
});

Here is the error I get:

--files-from hostname is not the same as the transfer hostname
rsync error: syntax or usage error (code 1) at main.c(1293) [sender=3.1.1]

I am running windows with cwRsync.

[pstadler]

This is a known issue. Will try to solve it with the next release.

[pstadler]

0.6.2 should fix this once and for all. Please let me know if there's still a problem. Thanks @gfilardo for suggesting and implementing this fix.

[MarcelRobitaille]

I am still getting an error when I run flightplan:

localhost $ rsync --files-from a34f38ad757af8d477e75eb930a5205b -az --rsh="ssh -p2222" ./ <my user>@<my host>:public_html/
localhost > usage: ssh [-1246AaCfghkNnqsTtVvXxY] [-b bind_address] [-c cipher_spec]
localhost >            [-D port] [-e escape_char] [-F configfile] [-i identity_file]
localhost >            [-L port:host:hostport] [-l login_name] [-m mac_spec] [-o option]
localhost >            [-p port] [-R port:host:hostport] [user@]hostname [command]
localhost > rsync: connection unexpectedly closed (0 bytes received so far) [sender]
localhost > rsync error: error in rsync protocol data stream (code 12) at io.c(235) [sender=3.1.1]
localhost ● failed (12)
● Command exited abnormally on localhost

[gfilardo]

What shell are you using? I have experienced the same problem using GitBash. It never occurred to me using flightplan from cmd or powershell.

[MarcelRobitaille]

I am using cmd.exe

[pstadler]

I wanted to setup a VM with these tools once but I failed with node.js/npm. Do you have a good guide for that?

On Sat, Mar 28, 2015 at 7:20 PM, Marcel Robitaille
notifications@github.com wrote:

I am using cmd.exe

Reply to this email directly or view it on GitHub:
#42 (comment)

[MarcelRobitaille]

I wanted to setup a VM with these tools once but I failed with node.js/npm. Do you have a good guide for that?

Unfortunately, I do not. I have not had very good luck with VMs in the past.

[pstadler]

I actually know how to set up a VM, but not how Windows users use node.js and all the npm fancy on their machines.

[seybsen]

@pstadler I could provide a guide for windows users soon. Setting up flightplan... no... sorry, setting up ssh on windows' powershell or cmd in combination with flightplan was a long journey for me. I ended up using powershell 4.0, scoop, node, openssh, (cw-)rsync, pageant and puttygen.
(P.S.: I did not use .transfer() from flightplan but local.exec('rsync ...'))

plan.target('staging', {
    host: 'example.com',
    username: 'myuser',
    privateKey: process.env.HOMEDRIVE + process.env.HOMEPATH + '\\.ssh\\id_rsa',
    agent: 'pageant'
});

the transfer looks something like this for me:

local.exec('rsync -rz --delete --rsh \'C:\\cwrsync\\bin\\ssh.exe -o StrictHostKeyChecking=no\' ./ ' + host.username + '@' + host.host + ':' + config.projectDir, { silent: true });

@Marcel-Robitaille I also fixed this error once: for me it occured when I tried using cwrsync with the ssh.exe provided by openssh (or git) and not the one that comes with cwrsync. Check your path environment variable and try putting cwrsync before openssh or git.

[pstadler]

@seybsen wow, that sounds like torture. Let me know if you're going to write a guide or something; I've yet to decide whether it's worth the trouble of setting up the whole env on a Windows VM. Anyway, thanks for dropping a couple of lines here.

[MarcelRobitaille]

@seybsen Thanks for the suggestion. I switched to rsync's ssh. Now I get the following error:

localhost > Host key verification failed.
localhost > rsync: connection unexpectedly closed (0 bytes received so far) [sender]
localhost > rsync error: error in rsync protocol data stream (code 12) at io.c(235) [sender=3.1.1]

[seybsen]

@Marcel-Robitaille that's why I added -o StrictHostKeyChecking=no to my custom rsync command. But you can try adding the servers you want to connect to manually to your known_hosts-file (should be in your .ssh/-folder)

[pstadler]

I'd suggest you manually ssh into the remote hosts (if this is even possible, don't know about cwRsync and "their ssh client"), in order to get the dialog asking you to confirm signatures. But yeah, placing StrictHostKeyChecking: no in the right file will do the trick as well.

[MarcelRobitaille]

Ok, I think I am getting close. I have changed my flightplan.js file to this:

local.exec('rsync -rz --rsh \'\"C:\\Program Files\\rsync\\ssh.exe\" -p2222 -o StrictHostKeyChecking=no\' ./ <user>@<host>:<path>');

I am prompted for my password. After I enter it, I get:

rsync: failed to open "<all the files>", continuing: Permission denied (13)