New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problems with async expect #347

Closed
selfuryon opened this Issue May 4, 2016 · 3 comments

Comments

Projects
None yet
3 participants
@selfuryon

selfuryon commented May 4, 2016

I'm trying to write simple test script with async expect (Python 3.5.1 and Pexpect from github):

@asyncio.coroutine
def test_ssh_expect_async(num):
    print('Task #{0} start'.format(num))
    p = pexpect.spawn('ssh localhost', encoding='utf8')
    #p.logfile = sys.stdout
    yield from p.expect('password', async=True)
    p.sendline('***')
    yield from p.expect(r'@self-VirtualBox\:', async=True)
    p.sendline('uptime')
    yield from p.expect(r'@self-VirtualBox\:', async=True)
    p.sendline('uname -a')
    yield from p.expect(r'@self-VirtualBox\:', async=True)
    p.sendline('ll')
    yield from p.expect(r'@self-VirtualBox\:', async=True)
    print('Task #{0} end'.format(num))

@asyncio.coroutine
def test_loop():
    tasks = []
    for i in range(1, 5):
        tasks.append(test_ssh_expect_async(i))
    yield from asyncio.wait(tasks)
    print('All Tasks done')


print('--------------Async--------------------')

loop = asyncio.get_event_loop()
loop.run_until_complete(test_loop())

And if I use range (1,3) or less in test_loop() - all will be fine:

self@self-VirtualBox:/media/sf_netdev$ python3 simple-test.py 
--------------Async--------------------
Task #3 running
Task #1 running
Task #2 running
Task #3 closed
Task #1 closed
Task #2 closed
All Tasks done

But if I increase upper limit - script will not work:

self@self-VirtualBox:/media/sf_netdev$ python3 simple-test.py 
--------------Async--------------------
Task #3 running
Task #1 running
Task #4 running
Task #2 running
Exception in callback BaseSelectorEventLoop.add_reader(11, <bound method...d=11 polling>>)
handle: <Handle BaseSelectorEventLoop.add_reader(11, <bound method...d=11 polling>>)>
Traceback (most recent call last):
  File "/usr/lib/python3.5/asyncio/selector_events.py", line 234, in add_reader
    key = self._selector.get_key(fd)
  File "/usr/lib/python3.5/selectors.py", line 191, in get_key
    raise KeyError("{!r} is not registered".format(fileobj)) from None
KeyError: '11 is not registered'

During handling of the above exception, another exception occurred:
...
@gescheit

This comment has been minimized.

Contributor

gescheit commented Sep 7, 2016

Seems this is fixed in #376

@selfuryon

This comment has been minimized.

selfuryon commented Sep 21, 2016

Seems to be, but I have the problems with range (1,20) with another exception :)

self:pexpect-test/ $ python3 test                                    [20:54:35]
--------------Async--------------------
Task #9 start
Task #1 start
Task #4 start
Task #10 start
Task #15 start
Task #5 start
Task #11 start
Task #16 start
Task #2 start
Task #6 start
Task #12 start
Task #17 start
Task #7 start
Task #13 start
Task #18 start
Task #19 start
Task #8 start
Task #14 start
Task #3 start
Task #9 end
Task #1 end
Task #4 end
Task #10 end
Task #15 end
Task #5 end
Task #2 end
Task #6 end
Task #11 end
Task #16 end
Task #17 end
Task #13 end
Task #12 end
Task #18 end
Task #19 end
Task #14 end
Task exception was never retrieved
future: <Task finished coro=<test_ssh_expect_async() done, defined at test:5> exception=EOF('<pexpect.pty_spawn.spawn object at 0x7f0d3d29f9e8>\ncommand: /usr/bin/ssh\nargs: [b\'/usr/bin/ssh\', b\'localhost\']\nbuffer (last 100 chars): \'\'\nbefore (last 100 chars): \'ssh_exchange_identification: Connection closed by remote host\\r\\r\\n\'\nafter: <class \'pexpect.exceptions.EOF\'>\nmatch: None\nmatch_index: None\nexitstatus: None\nflag_eof: True\npid: 7903\nchild_fd: 20\nclosed: False\ntimeout: 30\ndelimiter: <class \'pexpect.exceptions.EOF\'>\nlogfile: None\nlogfile_read: None\nlogfile_send: None\nmaxread: 2000\nignorecase: False\nsearchwindowsize: None\ndelaybeforesend: 0.05\ndelayafterclose: 0.1\ndelayafterterminate: 0.1\nsearcher: searcher_re:\n    0: re.compile("password")',)>
Traceback (most recent call last):
  File "/usr/lib/python3.5/asyncio/tasks.py", line 239, in _step
    result = coro.send(None)
  File "test", line 10, in test_ssh_expect_async
    yield from p.expect('password', async=True)
  File "/usr/local/lib/python3.5/dist-packages/pexpect/async.py", line 20, in expect_async
    return (yield from asyncio.wait_for(pw.fut, timeout))
  File "/usr/lib/python3.5/asyncio/tasks.py", line 392, in wait_for
    return fut.result()
  File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
    raise self._exception
  File "/usr/local/lib/python3.5/dist-packages/pexpect/async.py", line 67, in eof_received
    index = self.expecter.eof()
  File "/usr/local/lib/python3.5/dist-packages/pexpect/expect.py", line 50, in eof
    raise EOF(msg)
pexpect.exceptions.EOF: <pexpect.pty_spawn.spawn object at 0x7f0d3d29f9e8>
command: /usr/bin/ssh
args: [b'/usr/bin/ssh', b'localhost']
buffer (last 100 chars): ''
before (last 100 chars): 'ssh_exchange_identification: Connection closed by remote host\r\r\n'
after: <class 'pexpect.exceptions.EOF'>
match: None
match_index: None
exitstatus: None
flag_eof: True
pid: 7903
child_fd: 20
closed: False
timeout: 30
delimiter: <class 'pexpect.exceptions.EOF'>
logfile: None
logfile_read: None
logfile_send: None
maxread: 2000
ignorecase: False
searchwindowsize: None
delaybeforesend: 0.05
delayafterclose: 0.1
delayafterterminate: 0.1
searcher: searcher_re:
    0: re.compile("password")
Task exception was never retrieved
future: <Task finished coro=<test_ssh_expect_async() done, defined at test:5> exception=EOF('<pexpect.pty_spawn.spawn object at 0x7f0d3c5dab38>\ncommand: /usr/bin/ssh\nargs: [b\'/usr/bin/ssh\', b\'localhost\']\nbuffer (last 100 chars): \'\'\nbefore (last 100 chars): \'ssh_exchange_identification: Connection closed by remote host\\r\\r\\n\'\nafter: <class \'pexpect.exceptions.EOF\'>\nmatch: None\nmatch_index: None\nexitstatus: None\nflag_eof: True\npid: 7916\nchild_fd: 26\nclosed: False\ntimeout: 30\ndelimiter: <class \'pexpect.exceptions.EOF\'>\nlogfile: None\nlogfile_read: None\nlogfile_send: None\nmaxread: 2000\nignorecase: False\nsearchwindowsize: None\ndelaybeforesend: 0.05\ndelayafterclose: 0.1\ndelayafterterminate: 0.1\nsearcher: searcher_re:\n    0: re.compile("password")',)>
Traceback (most recent call last):
  File "/usr/lib/python3.5/asyncio/tasks.py", line 239, in _step
    result = coro.send(None)
  File "test", line 10, in test_ssh_expect_async
    yield from p.expect('password', async=True)
  File "/usr/local/lib/python3.5/dist-packages/pexpect/async.py", line 20, in expect_async
    return (yield from asyncio.wait_for(pw.fut, timeout))
  File "/usr/lib/python3.5/asyncio/tasks.py", line 392, in wait_for
    return fut.result()
  File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
    raise self._exception
  File "/usr/local/lib/python3.5/dist-packages/pexpect/async.py", line 67, in eof_received
    index = self.expecter.eof()
  File "/usr/local/lib/python3.5/dist-packages/pexpect/expect.py", line 50, in eof
    raise EOF(msg)
pexpect.exceptions.EOF: <pexpect.pty_spawn.spawn object at 0x7f0d3c5dab38>
command: /usr/bin/ssh
args: [b'/usr/bin/ssh', b'localhost']
buffer (last 100 chars): ''
before (last 100 chars): 'ssh_exchange_identification: Connection closed by remote host\r\r\n'
after: <class 'pexpect.exceptions.EOF'>
match: None
match_index: None
exitstatus: None
flag_eof: True
pid: 7916
child_fd: 26
closed: False
timeout: 30
delimiter: <class 'pexpect.exceptions.EOF'>
logfile: None
logfile_read: None
logfile_send: None
maxread: 2000
ignorecase: False
searchwindowsize: None
delaybeforesend: 0.05
delayafterclose: 0.1
delayafterterminate: 0.1
searcher: searcher_re:
    0: re.compile("password")
Task exception was never retrieved
future: <Task finished coro=<test_ssh_expect_async() done, defined at test:5> exception=EOF('<pexpect.pty_spawn.spawn object at 0x7f0d3c5da588>\ncommand: /usr/bin/ssh\nargs: [b\'/usr/bin/ssh\', b\'localhost\']\nbuffer (last 100 chars): \'\'\nbefore (last 100 chars): \'ssh_exchange_identification: Connection closed by remote host\\r\\r\\n\'\nafter: <class \'pexpect.exceptions.EOF\'>\nmatch: None\nmatch_index: None\nexitstatus: None\nflag_eof: True\npid: 7911\nchild_fd: 24\nclosed: False\ntimeout: 30\ndelimiter: <class \'pexpect.exceptions.EOF\'>\nlogfile: None\nlogfile_read: None\nlogfile_send: None\nmaxread: 2000\nignorecase: False\nsearchwindowsize: None\ndelaybeforesend: 0.05\ndelayafterclose: 0.1\ndelayafterterminate: 0.1\nsearcher: searcher_re:\n    0: re.compile("password")',)>
Traceback (most recent call last):
  File "/usr/lib/python3.5/asyncio/tasks.py", line 239, in _step
    result = coro.send(None)
  File "test", line 10, in test_ssh_expect_async
    yield from p.expect('password', async=True)
  File "/usr/local/lib/python3.5/dist-packages/pexpect/async.py", line 20, in expect_async
    return (yield from asyncio.wait_for(pw.fut, timeout))
  File "/usr/lib/python3.5/asyncio/tasks.py", line 392, in wait_for
    return fut.result()
  File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
    raise self._exception
  File "/usr/local/lib/python3.5/dist-packages/pexpect/async.py", line 67, in eof_received
    index = self.expecter.eof()
  File "/usr/local/lib/python3.5/dist-packages/pexpect/expect.py", line 50, in eof
    raise EOF(msg)
pexpect.exceptions.EOF: <pexpect.pty_spawn.spawn object at 0x7f0d3c5da588>
command: /usr/bin/ssh
args: [b'/usr/bin/ssh', b'localhost']
buffer (last 100 chars): ''
before (last 100 chars): 'ssh_exchange_identification: Connection closed by remote host\r\r\n'
after: <class 'pexpect.exceptions.EOF'>
match: None
match_index: None
exitstatus: None
flag_eof: True
pid: 7911
child_fd: 24
closed: False
timeout: 30
delimiter: <class 'pexpect.exceptions.EOF'>
logfile: None
logfile_read: None
logfile_send: None
maxread: 2000
ignorecase: False
searchwindowsize: None
delaybeforesend: 0.05
delayafterclose: 0.1
delayafterterminate: 0.1
searcher: searcher_re:
    0: re.compile("password")
All Tasks done
@selfuryon

This comment has been minimized.

selfuryon commented Oct 8, 2016

Sorry, it was a problem with openssh-server in MaxStartups params. Script working fine, I think this issue can be closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment