Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow using rsync for uploading files to the target machine. #359

Closed
wants to merge 1 commit into from
Closed

Allow using rsync for uploading files to the target machine. #359

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
57 changes: 44 additions & 13 deletions lib/kitchen/ssh.rb
View file
Open in desktop
Expand Up @@ -59,15 +59,41 @@ def exec(cmd)
end

def upload!(local, remote, options = {}, &progress)
if progress.nil?
progress = lambda { |ch, name, sent, total|
if sent == total
logger.debug("Uploaded #{name} (#{total} bytes)")
end
}
# Use rsync for speed if available.
start_time = Time.now
upload_done = false
if !@rsync_failed &&
File.directory?(local) && options[:recursive] &&
File.exists?('/usr/bin/rsync') &&
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe being able to load homebrew rsync for newer features might be useful?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good suggestion. However, I would like to hear about what it would take to integrate this into test-kitchen (as a plugin or otherwise) before spending more time on this pull request.

(!@options[:password] || File.exists?('/usr/bin/sshpass')) # so we can specify password
ssh_command = "ssh #{ssh_args.join(' ')}"
if @options[:password]
ssh_command = "/usr/bin/sshpass -p \"#{@options[:password]}\" #{ssh_command}"
end
rsync_cmd = "/usr/bin/rsync -e '#{ssh_command}' -az #{local} #{username_hostname}:#{remote}"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you tested with and without compression to see if it's actually worth it? Such a small amount over low latency connections; the setup time for zlib might be more.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, have not tested without compression, in my case the time rsync takes is negligible anyway (with compression enabled), and for remote connections compression might actually help.

@logger.debug("Running rsync command: #{rsync_cmd}")
if system(rsync_cmd)
upload_done = true
else
@logger.warn("rsync exited with status #{$?.exitstatus}, using Net::SCP instead")
@rsync_failed = true
end
end

unless upload_done
if progress.nil?
progress = lambda { |ch, name, sent, total|
if sent == total
logger.debug("Uploaded #{name} (#{total} bytes)")
end
}
end

session.scp.upload!(local, remote, options, &progress)
end

session.scp.upload!(local, remote, options, &progress)
@logger.info("Time taken to upload #{local} to #{username_hostname}:#{remote}: " +
"%.2f sec" % (Time.now - start_time))
end

def upload_path!(local, remote, options = {}, &progress)
Expand All @@ -90,21 +116,26 @@ def wait
end

def login_command
LoginCommand.new(["ssh", *ssh_args, username_hostname])
end

private

attr_reader :hostname, :username, :options, :logger

def ssh_args
args = %W{ -o UserKnownHostsFile=/dev/null }
args += %W{ -o StrictHostKeyChecking=no }
args += %W{ -o IdentitiesOnly=yes } if options[:keys]
args += %W{ -o LogLevel=#{logger.debug? ? "VERBOSE" : "ERROR"} }
args += %W{ -o ForwardAgent=#{options[:forward_agent] ? "yes" : "no"} } if options.key? :forward_agent
Array(options[:keys]).each { |ssh_key| args += %W{ -i #{ssh_key}} }
args += %W{ -p #{port}}
args += %W{ #{username}@#{hostname}}

LoginCommand.new(["ssh", *args])
end

private

attr_reader :hostname, :username, :options, :logger
def username_hostname
"#{username}@#{hostname}"
end

def session
@session ||= establish_connection
Expand Down