Skip to content

Commit

Permalink
walk: Flush stdout in batches
Browse files Browse the repository at this point in the history
The previous behaviour was designed to mimic the output buffering of
typical UNIX tools: line-buffered if stdout is a TTY, and fully-buffered
otherwise.  More precicely, when printing to a terminal, fd would flush
explicitly after printing any buffered results, then flush after every
single result once streaming mode started.  When not printing to a
terminal, fd never explicitly flushed, so writes would only happen as
the BufWriter filled up.

The new behaviour actually unifies the TTY and non-TTY cases: we flush
after printing the buffered results, then once we start streaming, we
flush after each batch, but *only when the channel is empty*.  This
provides a good balance: if the channel is empty, the receiver thread
might as well flush before it goes to sleep waiting for more results.
If the channel is non-empty, we might as well process those results
before deciding to flush.

For TTYs, this should improve performance by consolidating write() calls
without sacrificing interactivity.  For non-TTYs, we'll be flushing more
often, but only when the receiver would otherwise have nothing to do,
thus improving interactivity without sacrificing performance.  This is
particularly handy when fd is piped into another command (such as head
or grep): with the old behaviour, fd could wait for the whole traversal
to finish before printing anything.  With the new behaviour, fd will
print those results soon after they are received.

Fixes sharkdp#1313.
  • Loading branch information
tavianator committed Dec 13, 2023
1 parent 00b64f3 commit e1eaf77
Showing 1 changed file with 6 additions and 2 deletions.
8 changes: 6 additions & 2 deletions src/walk.rs
Original file line number Diff line number Diff line change
Expand Up @@ -214,7 +214,6 @@ impl<'a, W: Write> ReceiverBuffer<'a, W> {
}
ReceiverMode::Streaming => {
self.print(&dir_entry)?;
self.flush()?;
}
}

Expand All @@ -232,6 +231,11 @@ impl<'a, W: Write> ReceiverBuffer<'a, W> {
}
}
}

// If we don't have another batch ready, flush before waiting
if self.mode == ReceiverMode::Streaming && self.rx.is_empty() {
self.flush()?;
}
}
Err(RecvTimeoutError::Timeout) => {
self.stream()?;
Expand Down Expand Up @@ -285,7 +289,7 @@ impl<'a, W: Write> ReceiverBuffer<'a, W> {

/// Flush stdout if necessary.
fn flush(&mut self) -> Result<(), ExitCode> {
if self.config.interactive_terminal && self.stdout.flush().is_err() {
if self.stdout.flush().is_err() {
// Probably a broken pipe. Exit gracefully.
return Err(ExitCode::GeneralError);
}
Expand Down

0 comments on commit e1eaf77

Please sign in to comment.