Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

-P should imply --linewise #482

Closed
bitprophet opened this issue Nov 18, 2011 · 12 comments
Closed

-P should imply --linewise #482

bitprophet opened this issue Nov 18, 2011 · 12 comments
Labels
Milestone

Comments

@bitprophet
Copy link
Member

@bitprophet bitprophet commented Nov 18, 2011

Rather, ditto for the env vars -- when linewise is being tested for, have the test be "linewise or parallel".

Realistically, linewise shouldn't even be an exposed option as it's almost never going to be useful except in parallel mode; but now that it's out, it can't hurt to leave it as-is so folks can use it if they want.

The inverse, forcing bytewise parallelism, strikes me as an unusual case that can be safely ignored until somebody specifically asks for it.

@ghost ghost assigned bitprophet Nov 18, 2011
@shaftoe
Copy link

@shaftoe shaftoe commented Sep 11, 2012

Seems that there's been a regression, because env.linewise is False even when @parallel decorator is used. Crf. https://gist.github.com/3697165
We noticed doing a parallel deploy and getting bytewise output. Fabric v.1.4.3 on MacOSX 10.7.4

@bitprophet
Copy link
Member Author

@bitprophet bitprophet commented Sep 11, 2012

I can't replicate this locally -- can you paste your output & the code+invocation used to generate it?

@shaftoe
Copy link

@shaftoe shaftoe commented Sep 12, 2012

There should be everything in the gist I linked:
https://gist.github.com/3697165

@bitprophet
Copy link
Member Author

@bitprophet bitprophet commented Sep 12, 2012

Sorry, I meant I was hoping for an example of the literally garbled output (and the command generating it), because even though linewise is printing out False (which is certainly odd) I can't actually get bytewise output to occur, everything still ends up line by line when I do e.g. a flight of parallel ls or similar.

@bitprophet
Copy link
Member Author

@bitprophet bitprophet commented Sep 12, 2012

Guess I wasn't trying hard enough, adding even more hosts + changing the ls to a directory with even more files gets it to show up.

What's odd is I already traced the environment updating stuff that sets the 'parallel' and 'linewise' flags, and they are definitely firing, so something else must be going on. I'll figure out wtf. Thanks for the report :)

@shaftoe
Copy link

@shaftoe shaftoe commented Sep 12, 2012

No problem! Was going to answer that for an output report I'll need to wait until tomorrow cause it's the very task we use to do deploys and I'll have to wait for a dev to do it ;)

@shaftoe
Copy link

@shaftoe shaftoe commented Sep 14, 2012

This is an example of output we still get mixed up even forcing linewise via decorator. Actually seems more there's a missing line terminator than byte mixing, but maybe it could be of interest too:

[ec2-xxx.eu-west-1.compute.amazonaws.com.] out: [ec2-yyy.eu-west-1.compute.amazonaws.com.] out: INFO: www-redirect SVN update
INFO: www-profiler SVN update

@bitprophet
Copy link
Member Author

@bitprophet bitprophet commented Sep 14, 2012

I am pretty sure I've seen "missing" newlines before and am not 100% sure that's the same issue. Also, I don't believe the decorator will necessarily help -- like I mentioned, the higher level code that tries to ensure the subprocess runs with env.linewise = True is firing correctly, so it must be something lower down that may be ignoring the setting altogether.

@dmcquay
Copy link

@dmcquay dmcquay commented Oct 2, 2012

I'm having this same problem and it's definitely not just a newline issue. My output is very garbled. Here's an example:

y[prod1.athlete.com] out: Cso'p
/[prod3.athlete.com] out: Cngo p'y/iunsgr /'l/ohcoamle//liubb/upnyttuh/oant2h.l7e/tdei_stt-mppa/cpkuabgleisc//djjsa-nsgro/cc/lonitbr/ipbr/oatdomtiynp/es.tjatsi'c
u[prod3.athlete.com] out: Cdompiyni/njgs /'c/alheonmdea/ru.bjusn't
/[prod1.athlete.com] out: Caotphylinegt e'_/tumspr//pluobclailc/l/ijbs/-psyrtch/olni2b./7g/adrismti-np/auctkialg/eUst/idlj-anBgroo/acdocnatsrtiebr/.ajdsm'in

It seems to only happen when the output is really cooking, so that makes it a little harder to reproduce in a small example script. What is really easy to reproduce though is that if I define a task with the parallel decorator and immediately print out env.parallel and env.linewise, they are both False.

Snippet of my fabfile.py:

@task
@parallel
@roles('webapp')
def deploy(tarball=None):
print("parallel: " + repr(env.parallel))
print("linewise: " + repr(env.linewise))
return

And the resulting output:

$ fab production deploy
[prod1.athlete.com] Executing task 'deploy'
[prod2.athlete.com] Executing task 'deploy'
[prod3.athlete.com] Executing task 'deploy'
parallel: False
linewise: False
parallel: False
linewise: False
parallel: False
linewise: False

Done.

$ fab --version
Fabric 1.4.3
ssh (library) 1.7.14

@shaftoe
Copy link

@shaftoe shaftoe commented Oct 9, 2012

@dmcquay how are you working around the problem? Forcing env.linewise directly into the def?

@bitprophet
Copy link
Member Author

@bitprophet bitprophet commented Oct 28, 2012

Hey guys -- please take any further discussion over to #767. I will be trying some fixes ASAP and would love feedback once something gets pushed.

@bitprophet
Copy link
Member Author

@bitprophet bitprophet commented Oct 29, 2012

Think I do have a fix. Again -- please follow #767 and give feedback for the fix there. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.