asyncio integration for expect

[takluyver]

This is a first stab at integrating Pexpect with asyncio, to allow waiting for more than one process at once. Using it looks like this, although the API is almost certainly going to change before release. So far, I've only implemented async behaviour for expecting a pattern in output - we could add async support for writing large amounts of data to the pty, for opening the connection, for wait() and waitnoecho().

Status updated, see below.

[jquast]

Pexpect was mentioned on the tulip mailing list some time back, "Interacting with sys.stdin/sys.stdout/sys.stderr", https://groups.google.com/forum/#!msg/python-tulip/6tRswyRhwE4/90D1_B1NWEcJ

Trollious may allow us to remain py27 compatible.
http://trollius.readthedocs.org/ -- instead of "yield from", it is "yield From(...)".

[takluyver]

My plan for Pexpect 4 is to refactor the core of pexpect so that it's easy to write async wrappers - I think this mostly just means breaking out the core of expect_loop, so that wrappers can go 'there's new data ready - does it match yet?'. Then I'll provide the asyncio wrapper in a separate module, so that you only need asyncio to use those specific features; the rest of Pexpect will go on working without it.

I don't want to provide support in Pexpect for other async frameworks - trollius, tornado, twisted, etc. (is there something about the letter T and async?). But the refactoring should make it practical for other people to integrate Pexpect with those frameworks if people need to.

[takluyver]

OK, this is now taking on the rough shape that I want, if you want to start reviewing it.

In an asyncio coroutine, you can replace:

idx = p.expect('foo')

with this non-blocking version:

idx = yield from p.expect('foo', async=True)

• Tests
• Docs
• Expose the same option for expect_exact

I also plan to allow an asynchronous version of p.wait(), but I'll tackle that in another PR once this is cleared up.

[jquast]

Cool, I plan to try to use pexpect for functional testing with an asyncio telnet project, I'll make an effort to use use pexpect using asyncio patterns.

[takluyver]

OK, I think this is ready for review. The new parameters are documented, and there are some basic tests, which are passing.

[jquast]

will review this weekend, been on vacation recently as well :)

[takluyver]

I think the most important bit for review is the design of the new Expecter class, which separates out the bits that were previously inside expect_loop, so that they can be used by something other than a blocking loop. I thought about attaching those methods to spawn or to the searcher classes, but it didn't seem very neat either way.

I plan to mark Expecter as a 'provisional' API for 4.0, in case it turns out we need to change it. After that it will become a public API that people can use to integrate Pexpect with other async frameworks.

[takluyver]

@jquast, I'm merging this to keep things moving forwards. Feel free to do post-merge review.

[jquast]

sure will. been a bit full with work/life balance, but have a 2-week stay-at-home-vacation coming up this month.