wait_select raises exceptions when many files are open

[Emerentius]

Bug reports

When many filedescriptors are open, single.SSHClient.run_command() throws ValueError: filedescriptor out of range in select(). The same happens for any other code path that uses wait_select. The select function can't accept filedescriptors higher than FD_SETSIZE which is hardcoded to 1024 on linux.

This error is also mentioned in #175, but that issue is focused on the file leaking. It's a problem that parallel-ssh uses select in the first place. To avoid this limit, another system should be used, poll for example (gevent.select.poll probably in this case). With that, there is no upper limit on the file descriptors.

In my case, I have a server here that keeps a couple thousand websocket connections open at all times. parallel-ssh breaks almost completely because of this.

Steps to reproduce:

from pssh.clients.native import single
files = [open(f"{i}.deleteme", "w") for i in range(2000)]
client = single.SSHClient("some.address.tld", "user")
client.run_command('ls')

Results in

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/path/to/.venv/lib/python3.6/site-packages/pssh/clients/native/single.py", line 442, in run_command
    channel = self.execute(_command, use_pty=use_pty)
  File "/path/to/.venv/lib/python3.6/site-packages/pssh/clients/native/single.py", line 328, in execute
    self._eagain(channel.execute, cmd)
  File "/path/to/.venv/lib/python3.6/site-packages/pssh/clients/native/single.py", line 381, in _eagain
    wait_select(self.session)
  File "pssh/native/_ssh2.pyx", line 171, in pssh.native._ssh2.wait_select
  File "/path/to/.venv/lib/python3.6/site-packages/gevent/select.py", line 145, in select
    sel_results = _original_select(rlist, wlist, xlist, 0)
ValueError: filedescriptor out of range in select()

Expected behaviour: [What was expected to happen.]
No exception

Actual behaviour: [What actually happened.]
Exception raised when highest filedescriptor exceeds limit (1024 on linux by default)

Additional info: [Include version of ssh2-python and/or paramiko and any other relevant information.]
parallel-ssh: 1.9.1
ssh2-python: 0.18.0.post1

[pkittenis]

Thanks for the interest and report.

Single client should switch to poll/epoll to better handle FD size limits. This is reported as bug in select