When rewriting the I/O loop I had it print the line prefix after any line ending, meaning \n OR \r. I believe I did this because even during testing I ran into cases where a \r was present without a \n, though I'm not sure. It may have simply been pre-emptive on the assumption that somewhere, some program might be using \r as a newline terminator, or someone might be cat-ing such a file.
However, because at least some applications -- such as colorized ls output -- appear to use \r as, well, a carriage return without a newline, we end up with spurious blank, prefixed lines. This looks pretty bizarre, if not downright ugly.
Now, given that the captured text shouldn't have this problem (since the only issue here is that we're printing an extra prefix) it may not be a problem other than cosmetically, but even so, investigate whether a carriage return should be considered a valid line terminator on its own for our prefix-printing purposes. I'm thinking it shouldn't be.
Originally submitted by Jeff Forcier (bitprophet) on 2010-06-21 at 10:32pm EDT
Closed as Worksforme on 2010-07-13 at 08:56pm EDT
Jeff Forcier (bitprophet) posted:
I can't seem to replicate this now, and upon reflection I'm not sure why exactly the described setup would even be something we could defend against -- if it's creating an actual newline, doesn't that imply that a \n did exist too? (Or was it that since we printed something -- the prefix -- after the \r, it was unable to redraw the line correctly, giving the appearance of a "real" newline? no real idea here)
Closing for now, if I see it again in my travels I'll reopen.
on 2010-07-13 at 08:53pm EDT
This is definitely a problem: the example I am seeing while testing #19 (currently using ls / on a number of systems) is that, at the very least, ls does use combo \r\n as line separators (even in situations where no colorization is occurring, FWIW.) This results in extra "blank lines", both in bytewise (with a prefix) and the being-implemented linewise output (as a blank line, no prefix, currently.)
Only obvious solutions are to drop the prefixes or to implement a lookahead functionality. Former is not viable in general use; latter doesn't mesh too well with bytewise output, might be more viable with linewise, but either requires some loop/state juggling.
N.B. at least in the bytewise output, the problem is definitely a carriage-return one; when combined with an average-length line prefix and terminal-window-sized output (as is ls's default), this happens (confirmed by screwing with the line prefix and newline-detection code):
This is why both I and others have seen "missing" text (wrapped text is overwritten) and why it seems to appear as spurious newlines.
If corrected, we'll still see semi-ugly line wrapping in this scenario, but at least the reason for it would become obvious and the missing text uncovered.
With an empty-string line prefix, neither problem occurs, because there's no wrapping.
Whether we can handle this correctly in bytewise output is still unclear, but I think it's possible.
Was actually easier than expected; simply consolidate prefix printing to occur before byte-printing (in the place that currently tests whether to print the initial prefix) and examine the tail end of the capture buffer and the current byte. If the buffer ends with a newline character and the current byte is not a newline character, print a prefix before printing the byte.
This results in correct-looking output, with one caveat -- there are still "extra" newlines after each server's chunk of output. However this is because e.g. ls output ends with a trailing \n (actually \r\n, but in this case the \r shouldn't really matter). Capistrano still seems to avoid running into this but it's not clear how (trying to figure it out reminded me why I globbed onto Fab in the first place...)
At this point, then, the newer linewise output in #19 and this fixed bytewise output looks pretty much identical. There are not any actual extra lines compared to the existing bytewise output; it's just that the existing output has the additional line prefixes.
Small consolation though, given that it causes what look like "breaks" between servers -- at least with the final incorrect prefix, scanning the LHS of lines gives a sense of continuity.
Only way to truly fix this probably is a form of lookahead/buffering where we don't print a byte until we've obtained the following byte to figure out if we're at the end, and then to omit printing any final trailing \r\n or \n.
Not 100% sure that's worth it, esp. given it would technically hide actual remote output which might be desired in some situations. There's also the fact that my testing has been limited to ls, using things like git log looks much more natural.
Fix IO bug with spurious line prefixes.