Using :proxy can leave zombie processes behind

[adamruzicka]

Preliminary steps

• Try to open a connection through a proxy

Expected behavior

If it fails it reaps all processes it spawned.

Actual behavior

If either the proxy command connection or the "real" connection can't be established it leaves a zombie process behind.

System configuration

• net-ssh version: 4.1.0
• Ruby version: 2.4.1

Example App

In this case it prompts me for my password for the proxy command and for the "real" connection. Making any of the authentications fail triggers the bug.

gem 'net-ssh', '= 4.1.0'
require 'net/ssh'
require 'net/ssh/proxy/command'
puts Net::SSH::Version::CURRENT

begin
  proxy = Net::SSH::Proxy::Command.new('ssh -W %h:%p localhost')
  Net::SSH.start('localhost', ENV['USER'], :verbose => 0, :proxy => proxy) do |ssh|
    puts ssh.exec! 'hostname'
  end
rescue => e
  puts `ps -ef | grep #{Process.pid}`
end

# Just C-d whenever asked for password
$ ruby reproducer.rb
4.1.0
D, [2017-06-14T10:47:06.816664 #7364] DEBUG -- net.ssh.transport.session[2b0d41d8d49c]: establishing connection to localhost:22
aruzicka@localhost's password: 
Permission denied, please try again.
aruzicka@localhost's password: 
Permission denied, please try again.
aruzicka@localhost's password: 
Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
D, [2017-06-14T10:47:09.702275 #7364] DEBUG -- net.ssh.transport.session[2b0d41d8d49c]: connection established
I, [2017-06-14T10:47:09.702541 #7364]  INFO -- net.ssh.transport.server_version[2b0d41d8c8f8]: negotiating protocol version
D, [2017-06-14T10:47:09.702636 #7364] DEBUG -- net.ssh.transport.server_version[2b0d41d8c8f8]: local is `SSH-2.0-Ruby/Net::SSH_4.1.0 x86_64-linux'
aruzicka  7364  1240  7 10:47 pts/18   00:00:00 ruby repro.rb
aruzicka  7592  7364  0 10:47 pts/18   00:00:00 [ssh] <defunct>
aruzicka  7595  7364  0 10:47 pts/18   00:00:00 sh -c ps -ef | grep 7364
aruzicka  7597  7595  0 10:47 pts/18   00:00:00 grep 7364

[mfazekas]

@adamruzicka thanks for the detailed bugreport! 👍

[DimitriosLisenko]

Hey, thanks for your work so far guys!

This is still an issue in case of a timeout in the ProxyCommand - to reproduce I used the following script:

gem 'net-ssh', '= 4.2.0'
require 'net/ssh'
require 'net/ssh/proxy/command'
puts Net::SSH::Version::CURRENT

begin
  proxy = Net::SSH::Proxy::Command.new('ssh -W %h:%p localhost')
  Net::SSH.start('localhost', ENV['USER'], :verbose => 0, :proxy => proxy) do |ssh|
    puts ssh.exec! 'hostname'
  end
rescue => e
  puts `ps -ef | grep #{Process.pid}`
end

I then ran sudo iptables -A INPUT -p tcp --dport 22 -j DROP on the machine to block all traffic to port 22 and then ran load "./reproducer.rb" from the irb console.

Watching the result using watch -n 0.1 'ps -elf --forest', you can see that after the ProxyCommand timeout (60 seconds), the proxy command gets orphaned, and then if you keep watching, it will eventually become a zombie.

F S UID        PID  PPID  C PRI  NI ADDR SZ WCHAN  STIME TTY          TIME CMD
4 S root      1562     0  0  80   0 -  4560 -      11:32 pts/2    00:00:00 /bin/bash
0 S root     11915  1562  0  80   0 - 20120 -      11:39 pts/2    00:00:00  \_ irb
4 S root     15263 11915  0  80   0 - 11197 -      11:42 pts/2    00:00:00      \_ ssh -W localhost:22 localhost

# Some time later
F S UID        PID  PPID  C PRI  NI ADDR SZ WCHAN  STIME TTY          TIME CMD
4 S root      1562     0  0  80   0 -  4560 -      11:32 pts/2    00:00:00 /bin/bash
0 S root     11915  1562  0  80   0 - 20120 -      11:39 pts/2    00:00:00  \_ irb
4 Z root     15263 11915  0  80   0 -     0 -      11:42 pts/2    00:00:00      \_ [ssh] <defunct>

I have created a possible fix at #556 - it involves handling the child process in the Proxy::Command class, since in the timeout/exception case the IO is not returned, and so can't be handled by the Transport::Session class. Another option is to maybe set autoclose to true on the pipe.