Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Add support for rsync daemon in RSync::Pull and Rsync::Push #349

Closed
wants to merge 1 commit into
from

Conversation

Projects
None yet
5 participants

Set "daemon" option to true in your config to connect to rsync daemon
instead of SSH access.

Example:
sync_with RSync::Pull do |rsync|
rsync.daemon = true
end

Add support for rsync daemon in RSync::Pull and Rsync::Push.
Set "daemon" option to true in your config to connect to rsync daemon
instead of SSH access.

Example:
    sync_with RSync::Pull do |rsync|
      rsync.daemon = true
    end

This pull request passes (merged 3058597 into 5639587).

Fixes issue #304

linko commented Sep 6, 2012

@alexey-chernikov this works for me!

BUT to make this works I changed directories definition from

rsync.directories do |directory|
  directory.add "~/test_backup/"
end

to

rsync.directories do |directory|
  directory.add "user_root/test_backup/"
end

Where user root is a module from config on remote(source) server:

/etc/rsyncd.conf

[user_root]
path = /home/user/
...

Contributor

tomash commented Nov 2, 2012

i'm +1 on this. any suggestions or objections before it gets merged? @linko @alexey-chernikov some documentation would be great!

@ghost

ghost commented Jan 10, 2013

Updating the rsync syncer is something I had hoped to find the time to investigate further by now :)
For instance, I've been running the following patch to my personal copy for some time now, so I can use the authorized_keys on the remote to specify precisely where the key backup is using is allowed from.
i.e. from="<ip address>" ssh-rsa <backup's public key>

diff --git a/lib/backup/syncer/rsync/push.rb b/lib/backup/syncer/rsync/push.rb
index 1440446..1e8b748 100644
--- a/lib/backup/syncer/rsync/push.rb
+++ b/lib/backup/syncer/rsync/push.rb
@@ -10,12 +10,16 @@ module Backup
         attr_accessor :username, :password

         ##
-        # Server IP Address and SSH port
+        # Server IP Address
         attr_accessor :ip

         ##
-        # The SSH port to connect to
-        attr_accessor :port
+        # SSH port
+        attr_accessor :ssh_port
+
+        ##
+        # SSH identity file
+        attr_accessor :ssh_identity_file

         ##
         # Flag for compressing (only compresses for the transfer)
@@ -35,7 +39,7 @@ module Backup
         def initialize(&block)
           super

-          @port               ||= 22
+          @ssh_port           ||= 22
           @compress           ||= false

           instance_eval(&block) if block_given?
@@ -70,7 +74,7 @@ module Backup
         # Returns all the specified Rsync::[Push/Pull] options,
         # concatenated, ready for the CLI
         def options
-          ([archive_option, mirror_option, compress_option, port_option,
+          ([archive_option, mirror_option, compress_option, ssh_options,
             password_option] + additional_options).compact.join("\s")
         end

@@ -82,8 +86,11 @@ module Backup

         ##
         # Returns Rsync syntax for defining a port to connect to
-        def port_option
-          "-e 'ssh -p #{@port}'"
+        def ssh_options
+          opts = []
+          opts << "-p #{ ssh_port }" if ssh_port
+          opts << "-i #{ File.expand_path(ssh_identity_file) }" if ssh_identity_file
+          opts.empty? ? '' : "-e 'ssh #{ opts.join(' ') }'"
         end

I'm not intimately familiar with all of rsync's options, and it's certainly robust. So, if any of you would like to get with me on this, feel free to do so. The main concern is that if changes are made to the configuration, that this be done in such a way as to make any future changes backward compatible. I think this is why I chose to prefix the port with ssh_ above, but I can't remember atm :)
/cc @alexey-chernikov @linko

A big +1 to adding ssh_identity_file. None of my machines have passwords for any users. Would love to see that change make it's way in.

@burns anything i can do to help get your change in? any reason it hasn't been merged already?

@ghost

ghost commented Feb 5, 2013

I looked into this again recently, but decided to hold off for the moment (worked on the Syslog instead).
But I made some notes and have some ideas about how I'd like to see this proceed.
While I have a few other tasks I need to take care of first, this should be the next feature change I'll work on.

rsync supports 3 different modes:

  1. SSH (no daemon)
  2. rsync (direct TCP to a daemon)
  3. SSH with a single-use daemon spawned on the server (SSH + ::)

Right now, backup is setup for option 1.
This pull request is meant to enable option 2.
Should it not also support option 3?

Also, I think :username and :password are confusing.
The :password here is only used to authenticate with a daemon. Which means, it's not even used at present.
The :username I think is being used for SSH at present.
However, as soon as option 2 or 3 is enabled, this will only be used with daemon authentication as well.
It seems these need to be :rsync_user and :rsync_password. (or :daemon_)
Then, an :ssh_user should be added directly to the ssh command, using -e 'ssh -l ssh_user'.

For :rsync_password, we're currently writing this to a temp file and passing this to rsync's
--password-file option. I'd like the user to be able to specify their own file if they prefer.

:port should be ok. If option 2 is enabled, this should be used with rsync's --port option.

:ip should probably be :host.

As for :compress or other options, I'm still undecided. I think options that would be commonly used would be
ok, but I don't want things to get confusing (there's a lot of options).
I like having a way for advanced users to specify additional options.
And useful options can always be documented in the wiki.
We have :additional_options for the rsync command, but I'd like to add an :additional_ssh_options.
And while it may seem redundant for the Local Syncer, it might be worth changing :additional_options to
:additional_rsync_options.

The -i SSH option I mentioned above I believe should be left for the user to place in
:additional_ssh_options. While some would find this useful, it wouldn't normally be needed.
In most cases, the user running Backup (:ssh_user) would be setup to connect to the remote via SSH
(key with no passphrase). Only when the user's default SSH key has a passphrase, or they have no default key, would this be needed.

Given it's current state, these changes could be made while maintaining backward compatibility (with some deprecation warnings). So, I'd like to fix it all at once, get it right, and not have to revisit it for a while :)

Any suggestions are welcome!

@burns cool. maybe i'll take a look at making some of those changes. especially if my recent pull requests go in soon. i was having a bit of trouble figuring out the best way to test my changes since i mainly program in other languages besides ruby. any tips on building the gem or otherwise testing my changes?

@ghost ghost referenced this pull request Feb 19, 2013

Closed

Add support for rsync daemon #304

@ghost

ghost commented Apr 14, 2013

This was done in 3.2.0. Forgot to close this :)

@ghost ghost closed this Apr 14, 2013

This issue was closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment