Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
eventlet.monkey_patch() on Python 3.4 makes stdout non-blocking #248
We have a test suite which writes it's output to stdout. We started observing short writes on Python 3.4:
Now, I could put a loop in there with a timeout etc - but that is the is synchronous codepath in subunit, because we're writing to a known quantity.
This behaviour doesn't happen on Python2.6 or 2.7 with eventlet, so its just a bit of a puzzle.
On Python3.4 we get a binary file object by unwrapping stdout - sys.stdout.buffer is the object we're writing to, and we obtain the handle for that before monkeypatching is done (because the test harness is itself not an eventlet app)
Even more odd, piping the suite output through cat prevents the errors being triggered - I have no idea why!
Are you sure that standard streams (stdin, stdout, stderr) are non-blocking? I'm unable to reproduce the issue. Example with Python 3.5:
How can I try to reproduce the issue? Did you check with "strace -f -o trace" to check that fd 0, 1 or 2 is made non-blocking with fcntl()?
By default (with no redirection), all standard streams are the same device:
If you modify one stream, the two other streams are impacted. Example:
If you redirect a stream and you make it non-blocking, it doesn't use the shared device anymore and only this stream will be make non-blocking. Same code but with stdout redirected (ex: python script.py|cat):
stdin and stderr are still blocking, but the stdout (pipe) became non-blocking.
Ok, I found the code.
Part1 (subunit): subunit.run.SubunitTestRunner.run() => SubunitTestRunner._list() => os.fdopen(fileno) where fileno is 1 (stdout) in my case
Part2 (eventlet): os.fdopen(fd, ...) => eventlet.greenio.py3.GreenPipe() => eventlet.greenio.py3._open() => FileIO(fd, ...) => eventlet.greenio.py3.GreenFileIO()
And greenio.py3.GreenFileIO() constructor calls set_nonblocking(self).
Ok, well, why not. An event loop hates blocking things, so it makes sense.
The issue title is "eventlet.monkey_patch() on Python 3.4 makes stdout non-blocking". Ok, but what is the root issue? eventlet does exactly the same on Python 2: it also makes stdout non-blocking...
== Python 2 ==
GreePipe.write() => socket._fileobject.write() => socket._fileobject.flush() => call sendall() in a loop until the write buffer is empty
sendall() is eventlet.greenio.py3._SocketDuckForFd.sendall() which calls os.write() in a loop until all bytes are written. sendall() calls self._trampoline(self, write=True) to wait until the non-blocking file (a pipe in your case) becomes writable.
== Python 3 ==
GreenFileIO.write() calls os.write() once, but it may wait until the non-blocking file (pipe) becomes writable using trampoline(self, write=True).
Hum, it's every different. On Python 2, there are two loops to ensure that all bytes are written. On Python 3, there is no loop to ensure that all bytes are written.
== subunit ==
In your subunit program, to support the current version of eventlet on Python 3, you should accept that write() can be partial and write less than bytes than requested. write() always return the number of written bytes. You may implement your own loop. Example using memoryview to avoid memory copies:
eventlet has a different behaviour on Python 2 and Python 3, but I don't know which behaviour is the "good" behaviour :-)
In subunit/run.py, I see that the code uses "stream = os.fdopen(fileno, 'wb', 0)". So it explicitly asks a unbuffered writter. On Python 3, unbuffered really means unbuffered: FileIO.write() only calls os.write() once. It can returns less bytes than requested, it's part of FileIO specification.
The I/O stack of Python 2 is based on the stdio.h of the standard C library which doesn't give a full access on low-level files. It's less clear if write() calls the write syscall once or more. But you must expect that calling the write(data) method can be a partial write, write method of the object created by os.fdopen(fileno, 'wb', 0).
Ok now to come back to eventlet: it would be a bad idea to modify write() of eventlet on Python 2 to allow partial write, it will likely break a lot of applications in the wild. It would be tricky to debug such issue: partial write don't occur always, it depends on the network speed, on the size of the pipe buffer, etc. And it would take a lot of effort to "fix" all applications to support the new "pedantic" eventlet.
My conclusion is that eventlet must behave the same on Python 2 and Python 3, and so use a loop to write on Python 3 too. Even if the user asked an unbuffered writer.
Yeah, I need to cater for the FileIO case - it was oversight when I added the unbuffered stuff - its a legitimate bug i nsubunit there, I just need to do some gnarly type checking to determine if I have a high or low level method - e.g. can I consult the return value of write or not.