Fix mixed output in parallel mode #965

Closed
wants to merge 1 commit into
from

Projects

None yet

2 participants

@akitada
Contributor
akitada commented Aug 26, 2013

No description provided.

@bitprophet
Member

Feels possibly related to (now pretty old/outdated =/) #508. If that is the same issue I suspect the change in this one here is more solid.

@akitada
Contributor
akitada commented Aug 26, 2013

How to reproduce "mixed output".

First, add host entries to /etc/hosts for parallel test.

mv /etc/hosts /etc/hosts.orig
for i in {0..4}; do
    echo 127.0.0.1       `hostname`$i
done >> /etc/hosts

And run this fabfile by fab ssh_test | grep '\[.*\['

import socket
from fabric.api import *

env.timeout = 60
env.connection_attempts = 100


@parallel
def _ssh_test():
    run('ls /')


@task
def ssh_test():
    env.hosts = ['%s%d' % (socket.gethostname(), i) for i in xrange(5)]
    for _ in xrange(100):
        with hide('running'):
            execute(_ssh_test)
@akitada
Contributor
akitada commented Aug 26, 2013

This patch doesn't always work.
The write size must be PIPE_BUF or less and it must not be a write to a file.
http://pubs.opengroup.org/onlinepubs/9699919799/functions/write.html

@akitada
Contributor
akitada commented Nov 21, 2013

I've been using this patch for 3 months without any issues.

@bitprophet
Member

Confirmed that I can replicate the "double line prefix" issue by using the above snippet, from/to my localhost (MBA 11" mid-2013 model.)

Re: PIPE_BUF I assume that maps to http://docs.python.org/2/library/select.html#select.select.PIPE_BUF ?

I presume the suggested fix is basically "don't call flush() when in linewise mode" (and commutatively, in parallel mode, since parallel forces linewise.) Will test that out in a moment.

@bitprophet
Member

Confirmed, with change applied, I don't see any doubled lines (using grep.)

I'm worried this may cause things to not flush as often as they should in some situations (e.g. fast loops) but willing to merge this anyway in hopes that's not the case :) thanks!

@bitprophet bitprophet added a commit that referenced this pull request Dec 15, 2013
@bitprophet bitprophet Apply @akidata's fix re #965 + changelog 9b58343
@bitprophet
Member

Merging this to master, feels like the kind of bugfix that should come out in a 'feature' release.

@bitprophet bitprophet closed this Dec 15, 2013
@akitada akitada deleted the akitada:fix-mixed-output branch Dec 15, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment