Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix mixed output in parallel mode #965

Closed
wants to merge 1 commit into from

Conversation

@akitada
Copy link

@akitada akitada commented Aug 26, 2013

No description provided.

@bitprophet
Copy link
Member

@bitprophet bitprophet commented Aug 26, 2013

Feels possibly related to (now pretty old/outdated =/) #508. If that is the same issue I suspect the change in this one here is more solid.

@akitada
Copy link
Author

@akitada akitada commented Aug 26, 2013

How to reproduce "mixed output".

First, add host entries to /etc/hosts for parallel test.

mv /etc/hosts /etc/hosts.orig
for i in {0..4}; do
    echo 127.0.0.1       `hostname`$i
done >> /etc/hosts

And run this fabfile by fab ssh_test | grep '\[.*\['

import socket
from fabric.api import *

env.timeout = 60
env.connection_attempts = 100


@parallel
def _ssh_test():
    run('ls /')


@task
def ssh_test():
    env.hosts = ['%s%d' % (socket.gethostname(), i) for i in xrange(5)]
    for _ in xrange(100):
        with hide('running'):
            execute(_ssh_test)
@akitada
Copy link
Author

@akitada akitada commented Aug 26, 2013

This patch doesn't always work.
The write size must be PIPE_BUF or less and it must not be a write to a file.
http://pubs.opengroup.org/onlinepubs/9699919799/functions/write.html

@akitada
Copy link
Author

@akitada akitada commented Nov 21, 2013

I've been using this patch for 3 months without any issues.

@bitprophet
Copy link
Member

@bitprophet bitprophet commented Dec 15, 2013

Confirmed that I can replicate the "double line prefix" issue by using the above snippet, from/to my localhost (MBA 11" mid-2013 model.)

Re: PIPE_BUF I assume that maps to http://docs.python.org/2/library/select.html#select.select.PIPE_BUF ?

I presume the suggested fix is basically "don't call flush() when in linewise mode" (and commutatively, in parallel mode, since parallel forces linewise.) Will test that out in a moment.

@bitprophet
Copy link
Member

@bitprophet bitprophet commented Dec 15, 2013

Confirmed, with change applied, I don't see any doubled lines (using grep.)

I'm worried this may cause things to not flush as often as they should in some situations (e.g. fast loops) but willing to merge this anyway in hopes that's not the case :) thanks!

bitprophet added a commit that referenced this pull request Dec 15, 2013
@bitprophet
Copy link
Member

@bitprophet bitprophet commented Dec 15, 2013

Merging this to master, feels like the kind of bugfix that should come out in a 'feature' release.

@bitprophet bitprophet closed this Dec 15, 2013
@akitada akitada deleted the akitada:fix-mixed-output branch Dec 15, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants