Skip to content
Perl and SSH presentation at YAPC::Europe 2013
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
README.md
perl-and-ssh.md
presentation.html

README.md

Perl and SSH

Salvador Fandiño (sfandino@yahoo.com)

YAPC::Europe 2013


How do you automate SSH tasks?


If you use ...

  • system("ssh $host $cmd @args")
  • or backticks
  • Net::SSH
  • Net::SSH::Perl
  • Expect
  • Net::SSH::Expect

...you are living in the past!


Using system or backticks

  • ok for simple things
  • but very inneficient, opens a new SSH connection every time
  • no password authentication
  • you have to properly quote the arguments
  • sometimes twice
  • or be insecure

Net::SSH

  • simple wrapper for the ssh command
  • provides also sshopen2 and sshopen3
  • has same problems as using system

Net::SSH::Perl

  • huge module
  • at its time it represented a great effort
  • I guess most of it is a direct translation of OpenSSH C code
  • but today, nobody maintains it
  • lots of known bugs
  • very difficult to install (Math::Pari)
  • API good for simple things, bad for advanced things
  • supports SFTP via Net::SFTP, doesn't support SCP

Expect

  • Expect is a great module for automating interactive programs
  • It's a good way to automate ssh password authentication
  • After that, it is (usually) the wrong tool
  • It talks to the remote shell, you don't want that!
  • On some systems (AIX, HP-UX) ttys may drop data

BTW...

!perl
my $pty = IO::Pty->new;
my $expect = Expect->init($pty);
my $pid = open2($in, $out, '-');
unless ($pid) {
  defined $pid or die "unable to fork";
  $pty->make_slave_controlling_terminal;
  do { exec @ssh_cmd };
  exit -1;
}

Net::SSH::Expect

  • Builds on top of Expect
  • On the wrong way
  • It talks to the shell
  • It is not reliable

So, what should I use?


Modern SSH modules

  • Net::SSH2
  • Net::OpenSSH
  • Net::SSH::Any

Well, actually...

  • Net::SSH2
  • Net::OpenSSH
  • Net::SSH::Any
  • Net::SSH::Mechanize
  • Net::OpenSSH::Parallel
  • Net::OpenSSH::Compat
  • Net::SFTP::Foreign
  • ...

Net::SSH2


Net::SSH2

  • wrapper for the libssh2 C library
  • quite portable
  • quite easy to install on Unix/Linux, almost easy to install on Windows, don't known about VMS
  • it is a very thin wrapper
  • efficient, can run several commands over one SSH connection
  • C'ish low level API:
    • very simple things are easy to do
    • not so simple things become quite hard
  • supports SCP
  • very primitive and inneficient support for SFTP
  • project started by David B. Robins, currently being actively maintained by Rafael Kitover

Net::SSH2 - usage

!perl
use Net::SSH2;

my $ssh2 = Net::SSH2->new();

$ssh2->connect('example.com') or die $!;

if ($ssh2->auth_keyboard('fizban')) {
    my $chan = $ssh2->channel();
    $chan->exec('program');
}

Net::SSH2 problems

  • very low level C'ish API: not for lazy people
  • libssh2 is not a mature project yet
  • requires a C compiler

Net::OpenSSH


Net::OpenSSH

  • wrapper around OpenSSH ssh
  • uses its connection multiplexing feature
    • several commands can be run over the same connection
    • efficient
  • Perlish API with lots of belts and whistles
  • easy to use, complex things are almost easy
  • can work asynchronously
  • supports SFTP, SCP, rsync and sshfs
  • automatic argument quoting

Net::OpenSSH - usage

!perl
use Net::OpenSSH;
my $ssh = Net::OpenSSH->new($host, user => $user, password => $password);
$ssh->die_on_error("unable to connect");

my @output = $ssh->capture("cat /etc/passwd");

my ($out, $err) = $ssh->capture2("find /");

$ssh->system({stdin_data => "hello\n"},
             "cat >>log");

my $pid = $ssh->spawn({stderr_to_stdout => 1,
                       stdout_file => "tar.log"},
                      'tar', 'cf', '/tmp/my backup.tar', '/home/me');
waitpid($pid,0);

$ssh->scp_get('/tmp/*.tar', '.');

$ssh->rsync_put({verbose => 1, safe_links => 1},
                "etc", "/etc");

my $sftp = $ssh->sftp; # return a Net::SFTP::Foreign object
my $ls = $sftp->ls;

Net::OpenSSH argument quoting

!perl

my $out = $ssh->capture("ls /etc"); # no quoting
my $out = $ssh->capture(ls => '~/My Documents'); # quoting
my $out = $ssh->capture({quote_args => 0},
                        ls => '~/My Documents'); # no quoting

# selectively quoting arguments:
my $out = $ssh->capture(ls => \'~/My Documents/*.pdf');
        # lets file name wildcards be expanded by the remote shell

my $out = $ssh->capture(ls => \\'~/My Documents/*.pdf'); # wrong!

my $out = $ssh->capture(@cmd1, \\'&&', @cmd2, \\'2>/dev/null');

Net::OpenSSH argument quoting

  • On the stable release, argument quoting expects a POSIX compatible shell on the remote side

  • The development release has support for different quoting backends - POSIX (i.e. ksh, bash) and csh already there - maybe Windows/DOS backend in the future


Net::OpenSSH problems

  • Net::OpenSSH does not work on Windows
    • OpenSSH multiplexing feature does not work on Windows, not even under cygwin
  • Requires the OpenSSH ssh command

Net::SSH::Any


Net::SSH::Any

  • API very similar to Net::OpenSSH

  • works on top of

    • Net::SSH2
    • Net::OpenSSH
    • maybe Net::SSH::Perl in the future
    • maybe simple wrapper around native ssh
  • supports SCP and SFTP for file transfers, efficiently

  • a work in progress

  • though, basic functionality is already stable


Net::SSH::Mechanize


Net::SSH::Mechanize

  • It uses the AnyEvent framework
  • It aims to support sudoing smoothly
  • Limited API
  • It talks to the remote shell, unreliable!

Net::OpenSSH::Parallel


Net::OpenSSH::Parallel

  • Run commands in parallel in remote hosts through SSH
  • Build on top of Net::OpenSSH
  • Declarative API:
    • tell the module all the actions you want to perform on the remote hosts
    • let the module take care of everything and do the tasks
    • handle possible errors

Net::OpenSSH::Parallel - usage

!perl
my $pssh = Net::OpenSSH::Parallel->new;

# tell the object what the remote hosts are:
$pssh->add_host('host1', user => foo, password => $pwd);
$pssh->add_host('host2', user => foo, password => $pwd);

# declare the actions you want to run:
$pssh->push('host1', command => "echo hello from host1");
$pssh->push('host2', command => "echo hello from host2");

# for several host in one call:
$pssh->push('host*', command => "echo hello from some host");

# with variable expansion:
$pssh->push('host*', command => "echo hello from host %HOST%");

# and run it:
$pssh->run;

Net::OpenSSH::Parallel - usage

!perl
# other actions:
$pssh->push('*', scp_get => '/var/log/messages.0', 'logs/messages.0.%HOST%');

$pssh->push('*', rsync_put => '/var/www', '/var/www');

# can pass extra arguments to the underlaying Net::OpenSSH methods
$pssh->push('*', rsync_put => { safe_links => 1,
                                stdout_file => ['>>', 'mylog'],
                                stderr_to_stdout => 1 },
			  '/var/www', '/var/www');

# runs a custom sub on a locally forked process
$pssh->push('*', parsub => \&my_sub);

Net::OpenSSH::Parallel - Expect and sudo

!perl
sub sudo_install {
    my ($label, $ssh, @pkgs) = @_;
    my ($pty) = $ssh->open2pty('sudo', 'apt-get', 'install', @pkgs);
    my $expect = Expect->init($pty);
    $expect->raw_pty(1);
    $expect->expect($timeout, ":");
    $expect->send("$passwd\n");
    $expect->expect($timeout, "\n");
    $expect->raw_pty(0);
    while(<$expect>) { print };
    close $expect;
}

$pssh->push('*', parsub => \&sudo_install, 'scummvm');

Net::OpenSSH::Parallel - dependencies

!perl
$pssh->push('dmz*', scp_get => '/etc/passwd', 'passwd.%HOST%');
$pssh->push('safe', join => 'dmz*');
$pssh->push('safe', command => mkdir, "/var/safe/$date");
$pssh->push('safe', scp_put => 'passwd.*', "/var/safe/$date");
$pssh->run;

Net::OpenSSH::Parallel problems

  • Unable to distribute tasks around a set of hosts
  • Can't run more than one command per host simultaneously
  • stdin/stdout/stderr has to go to the file system, no pipes between tasks

Net::OpenSSH::Compat


Net::OpenSSH::Compat

  • Implements most of Net::SSH::Perl and Net::SSH2 APIs on top of Net::OpenSSH
  • Because sometimes people has problems installing them

Net::SFTP::Foreign


On the works...


Other interesting modules

  • SSH::Batch
  • App::MrShell
  • POE::Component::OpenSSH
  • GRID::Machine
  • IPC::PerlSSH
  • (R)?ex
  • ...

Questions


Thank you!


Links

You can’t perform that action at this time.