`print!` macro should flush stdout

[tanadeau]

As stdout is line-buffered, stdout is not implicitly flushed until a new-line is encountered. This means that the print! macro does not act like a println! without the newline as the documentation suggests. To be equivalent, the user must explicitly flush stdout like the following:

use std::io::prelude::*;                                                           
use std::io;                                                                       

fn main() {
    print!("Type something: ");
    io::stdout().flush().ok().expect("Could not flush stdout");

    // Read stdin, etc.
}

For easy use of the print macros, the user should not need to know about how I/O flushing works. As such, print! should explicitly flush stdout itself.

[sfackler]

As a quick survey, standard output is line buffered in C, C++, Python, and Ruby. Go's Printf buffers output and writes it to an unbuffered stdout. Java doesn't document what it does, but System.out appears to autoflush.

[richo]

I opened #23823. It's not entirely clear to me that the solution isn't to just update the docs to point out that stdout in unbuffered, but I think at the point where you're invoking the print*! machinery you're sufficiently far from the OS that you probably just do want some output to appear in the terminal.

[tanadeau]

@richo The implementation of std::io::Stdout is definitely line-buffered. See

rust/src/libstd/io/stdio.rs

Line 213 in 9f9e251

inner: Arc<Mutex<LineWriter<StdoutRaw>>>,

[richo]

Sure, but that's orthogonal, since the underlying fd is not, and writing to it will not emit anything until it's flushed.

I'm pretty sure that the correct approach is to merge that patch, but figured it was worth pointing out that updating the docs may be a valid approach too.

[alexcrichton]

It's not 100% clear to me that this is the behavior that we want. In #23087 it was explicitly removed due to worries about deadlocking. The specific problem of deadlocking can be papered over, but in general APIs tend to work much better over time if they do precisely what you expect, and it may not always be expected that print! flushes the output buffer.

One downside of flushing on print! would be that any form of "maximal buffering" when redirected to a TTY would be lost. For example C performs much more buffering when stdout is redirected than when it is to a console. This means that simple benchmarks which print information will be much slower in Rust than in comparison with others (due to the frequent flushing).

I would be fine beefing up the documentation in this regard, of course!

[richo]

That's totally reasonable, and mostly what I expected. Do you have a
citation about the c buffering though? That wasn't be belief and I'd love
to read more.

Will update the docs if no one beats me to it.

On Saturday, March 28, 2015, Alex Crichton notifications@github.com wrote:

It's not 100% clear to me that this is the behavior that we want. In
#23087 #23087 it was explicitly
removed due to worries about deadlocking. The specific problem of
deadlocking can be papered over, but in general APIs tend to work much
better over time if they do precisely what you expect, and it may not
always be expected that print! flushes the output buffer.

One downside of flushing on print! would be that any form of "maximal
buffering" when redirected to a TTY would be lost. For example C performs
much more buffering when stdout is redirected than when it is to a console.
This means that simple benchmarks which print information will be much
slower in Rust than in comparison with others (due to the frequent
flushing).

I would be fine beefing up the documentation in this regard, of course!

—
Reply to this email directly or view it on GitHub
#23818 (comment).

[tanadeau]

While making the documentation clearer would certainly be helpful, I think Rust is losing ease of use by making auto-flushing more difficult than need be. C++ has std::unitbuf to making auto-flushing simple in cases like prompts and progress bars. This at least makes it so that users don't need to call flush() explicitly after every call.

[tanadeau]

Could there be new macros for this use case? I'd hate for Rust I/O to be harder to use correctly than C++'s especially in a common learning case. For example, since C++ does flush cout when cin is read, the following works fine there but the equivalent doesn't for Rust:

#include <iostream>

int main() {
    std::cout << "Type something: ";

    std::string input;
    std::cin >> input;

    std::cout << input << std::endl;
} 

Also according to the C standard:

 At program startup, three text streams are predefined and need not be opened explicitly
— standard input (for reading conventional input), standard output (for writing
conventional output), and standard error (for writing diagnostic output). As initially
opened, the standard error stream is not fully buffered; the standard input and standard
output streams are fully buffered if and only if the stream can be determined not to refer
to an interactive device.

[nagisa]

This means that the print! macro does not act like a println! without the newline as the documentation suggests.

println! macro, on the other hand, does not specify anything about its flushing semantics either. Even the std::io::Stdout doesn’t and totally should.

---

I generally really dislike implicit flushing and prefer to flush explicitly. On the other hand, there’s write{,ln}!, which does not and can’t assume anything about buffering, so I’m ambivalent about this issue.

[alexcrichton]

I think Rust is losing ease of use by making auto-flushing more difficult than need be.

The design here is generally mostly about trade-offs. On one hand you have "ease of use" where you don't have to worry about calling flush(), but on the other hand you have worries about deadlocking, and/or surprising behavior happening behind the scenes. On the other hand you have to write a few more flush calls, but you know precisely what is happening and can avoid situations like deadlocking fairly easily.

For example, what should this code do?

let input = io::stdin();
let mut locked = input.lock();

print!("enter input: ");
let mut input = String::new();
try!(locked.read_line(&mut input));

println!("you entered: {}", input);

If print! were to auto-flush, then it would deadlock the current process because the stdin handle was already locked. The alternative here is adding one call to try!(locked.flush()) right above the call to read_line.

[nagisa]

If print! were to auto-flush, then it would deadlock the current process because the stdin handle was already locked. The alternative here is adding one call to try!(locked.flush()) right above the call to read_line.

I think you misunderstood what the issue is about: it is only proposed to flush std_out_ after print! writes to the std_out_. i.e. change

macro_rules! print(…) {
     …
}

to

macro_rules! print(…) {
    …
     stdout().flush()
}

Since print! already takes a lock to write, this change will not introduce any more new cases where it deadlocks. For example snippet below deadlocks currently and will deadlock in all the same cases if we made it to auto-flush.

let output = io::stdout();
let output = output.lock();
print!("stdout locked"); // deadlock.

I’m starting to believe that stabilising Std{in,out,err}::lock was a wrong and hasty move.

[alexcrichton]

I think you misunderstood what the issue is about: it is only proposed to flush stdout after print! writes to the stdout

Hm yes, I think I have misunderstood! There are definitely performance considerations which affect flush-on-all-print semantics as it's not something that other languages tend to do and can greatly hinder various benchmarks.

[richo]

I think the crux of this is basically "What is the intent of print!". My
intuition is that it's not intended to be in the critical section, and is
instead meant to be a helper for "make this string appear on stdout", I
could really take or leave whether or not it flushes. That said,
considering that it already traverses all of the formatting machinery
anyway, if it can't deadlock my intuition is that the flush is just one
extra syscall to make this do something more user friendly/intuitive.

While I don't believe it's actually on the table, I would be a stoic -1 to
any suggestion of making all stdout writes autoflush.

On Mon, Mar 30, 2015 at 1:50 PM, Alex Crichton notifications@github.com
wrote:

I think you misunderstood what the issue is about: it is only proposed to
flush stdout after print! writes to the stdout

Hm yes, I think I have misunderstood! There are definitely performance
considerations which affect flush-on-all-print semantics as it's not
something that other languages tend to do and can greatly hinder various
benchmarks.

—
Reply to this email directly or view it on GitHub
#23818 (comment).

[tanadeau]

To address the performance considerations, could the print! macro only flush stdout if stdout is attached to a TTY?

[alexcrichton]

@tanadeau unfortunately it's a performance concern both attached and not attached to a TTY

[mkpankov]

I'm against this.

Implicit flushing will degrade performance of programs doing specifically print!() and printing long lines out of several components.

What, for example, do you use for actual text printing of your main output in console programs? I used print!() in clone of hexdump, for example.

About buffering: this man mentions the defaults
http://manpages.courier-mta.org/htmlman3/setbuf.3.html

[softprops]

Fwiw Ive run into this more than a few times and each time it was confusing as a user. Its the same kind of confusion you'd have if you pushed a print button in your browser and nothing happens. I think part of a compromised solution lies in a more meaningful name. We have buffered readers and writers for the performance mentioned above and do not cause the same kind of confusion because their names reveal their expected behavior. What about a bufprint macro you can call when you want buffering and have print's default behavior be to... print something.

[BartMassey]

Hey, I just hit this too!

Maybe a sufficient compromise that would make everybody reasonably happy is just a prompt!() macro which behaves like print!() but flushes? I'd be happy to have such a macro, which would solve the most common case. Add it to the Guessing Game tutorial and everybody will notice it.

I'd prefer that print!() and println!() flush when stdout() is a tty. I can't imagine caring about the slight performance hit in this case, and the multi-thread race and [maybe] deadlock conditions are endemic in trying to print!() from multiple threads anyhow. But I'll settle for an extra macro if that's what it has to be.

[letheed]

I feel that some of the confusion comes from the fact that stdout is line buffered. Flushing is entirely up to the buffer and depends on the value of its content at runtime. It is implicit and left out of not only print! and println!'s semantics but also write! and writeln!'s. This, I find, is contrary to Rust's culture of explicitness and orthogonality.

A solution would be to make stdout a normal buffer:

• print! and println! explicitly call flush and do what you expect: paint on screen. Behavior is mostly the same as today, minus the sometimes missing text with print! that surprises newcomers.
• write! and writeln! can be used to build output. Flush must be called manually and we could now buffer multiple lines of text without triggering a flush, which we can't do today without allocating another buffer/string. (We may or may not want to create write_out! and writeln_out! macros that do not bypass LOCAL_STDOUT the way write! and writeln! do.)

Line buffering has its uses, but I feel like it's trying to be smart but it's really just making things more complicated.

[aalub]

To flush or not to flush... That is the question! (C)

Gentlemen, I'm fully aware the #23823 #25555 are closed, but I have a question on the same thing: How can I write simple and consistent code like C's

  char str_in[1024];
  printf("enter some text: ");
  gets(str_in);
  printf("your input is \"%s\"", str_in);

in Rust? Which would let me input some text on the same line with the prompt for it, without using std::io::prelude::*, calling io::stdout().flush() and doing alike fancy things...

I try to call print!("enter some text: ") and then read_line(&mut str_in) (rustc 1.21.0 in Windows 8.1) and I get read_line() call before printing the prompt.

Which seems very strange for an outsider to Rust.

Thanks in advance.

p.s. Flushing the console buffer just before reading it (at any form) seemed a good idea, but it was rejected. I failed to understand why.

[steveklabnik]

How can I

These kinds of questions are better served by https://users.rust-lang.org/ than the issue tracker.

[BartMassey]

The problem here is that a bunch of new Rust users are going to be confused by behavior that isn't well-documented and in my opinion a bit surprising, without being given any warning in any official documentation, and without being given any obvious path to do what is a very common operation: printing a prompt and getting a response.

Python's solution is to have the input() function take an optional prompt argument. C's solution is to have reading from stdin flush stdout. Rust's solution is apparently to have the user instantiate stdout() and invoke flush() on it after print!(). Make sure the Write trait is in scope. As far as I can tell this example, adapted from the "Guessing Game" tutorial in the documentation, is about as simple as it gets.

use std::io::{stdin, stdout, Write};

fn main() {
    print!("prompt: ");
    stdout().flush().unwrap();
    let mut guess = String::new();
    stdin().read_line(&mut guess).unwrap();
    println!("got {}", guess);
}

(You could just use std::io::*; but that doesn't seem to be the favored style in Rust code, and I think it only works because the Write trait happens to be in the same module as stdout.)

I see this level of fiddly complexity for a common operation routinely performed by beginning programmers as a real issue that needs to be addressed directly. Documenting the above example and giving it the page of explanation that it deserves just doesn't seem like enough to me. The fact that the last println!() can be replaced by a print!() that will then implicitly flush (because of the ending newline and default line buffering) doesn't make the explanation easier.

I don't care too much about how this is addressed, but it should be somehow. Regardless of whether consensus can be reached, by all means at least hack up "Guessing Game" to do this so there's a standard example. But I'd strongly prefer a solution that makes "Guessing Game" prettier.

[aalub]

These kinds of questions are better served by https://users.rust-lang.org/ than the issue tracker.

Sorry. But my How can I question arouses out from the decision not to flush stdout neither after print!() nor before read_line() calls discussed here. And I utterly comprehend the former (locking console for flushing any unfinished line would slow down performance without real need), but am still very disappointed with the latter.

Constructing complex output by multiple sequential print!() calls is a common task for text-based programs. Flushing every output is not necessary in this case, it should be done only at the end of such a sequence. No confusion can emerge.

Another common task is interacting with the user, and the Guessing Game is a good example (a less naughty example is Rust's installer for Windows). Such a task can duly interleave the program's output with the user's input and it seems a good practice to grant the programmer a proper tool for it. Using something like print() and read() functions seems to be the solution, but with Rust it is an illusion: the programmer is obliged to bother with flushing unfinished lines of stdout before every read.

So why this flushing should not be seamlessly placed before every read inside Rust library? It seems obvious that before reading the user's input all the program's output ought to be flushed: the user has to see what [s]he asked for. Apropos, if the library flushes stdout at every new line, why it does not before every read?

Or, if there is other proper practice of interacting with the user - which is it? Using strict println!() vividly resembles me old good BASIC times.

Whence my "How can I...?".

I fully agree with @BartMassey's conclusion above except that I emphasize the only place for flush is not after writing but before reading user's input, which is very slow by nature, where there is plenty of time to do anything useful.

@BartMassey, thank you for example and explanations! I feel the issue is still not settled.

[BartMassey]

the only place for flush is not after writing but before reading user's input, which is very slow by nature, where there is plenty of time to do anything useful.

There are other places where some kind of convenient flush would be useful. One example is printing some non-line-terminated string before an expensive computation.

print!("The answer is  (please wait)... ");
println!("{}", expensive_function());

I'm not too dogmatic about what's done about these cases, but I feel like at least one of the solutions proposed above should be implemented.

[dtolnay]

We discussed this with the libs team and our feeling was that none of the suggestions that involve an implicit flush really address the knowledge gap of people having an incorrect (or no) mental model of how buffered i/o and flushing works. These changes would just move the stumbling block to some later point when buffered i/o and flushing inevitably bites them anyway. People writing interactive command-line apps need to understand what flushing means and inserting implicit flushes would not be doing them any favors.

We would welcome documentation improvements that do a better job of teaching the behavior of std::io with respect to flushing.

Also we would love to consider an RFC for improving the ergonomics of explicit flushing in conjunction with print! or other uses of std::io. In particular, we agree that the current approach of use std::io::{self, Write}; io::stdout().flush().expect("Could not flush stdout") is not the best -- but just condensing it into something like flush!() isn't obviously better either. Looking forward to what people come up with!

[BartMassey]

People writing interactive command-line apps need to understand what flushing means and inserting implicit flushes would not be doing them any favors.

Sounds good! Should we remove the implicit flushing in println!() as well?

In the meantime I've started on an RFC for a Python-style input!() macro for prompted line input. I'll post it wherever I'm supposed to when it's ready.

[leovano]

For those who need to prompt the user to enter any information, why not use a crate for that?

[mstagg]

@leovano B/c we dont want this to turn into some javascript monstrosity of poorly stitched together libraries and frameworks. This is base level functionality, having to import a crate is way too heavy handed of a solution.

[MutantOctopus]

I just ran into this issue for the first time, and just to give my two cents: I generally agree with what's been said.

println! flushing is fine, because when you're done with a line, you're done with a line. On the other hand, in my experiments across various languages, I've generally reserved print! and its relatives in other languages for times where I need to assemble a single line to print, but don't necessarily need to have it flush at separate times, in which case yes, not having a flush is largely irrelevant. And if someone is familiarizing themselves with how output streams work, and they run into unexpected behavior, they'll search, look up documentation, find threads like this, and largely come out of it with a better understanding of the situation (which I did).

At the same time, I think it would be beneficial to have a separate print macro which flushes - print_flush! seems like the obvious name - and I'm not sure if I see the shortcoming in having a flush! macro as @dtolnay suggested, but at the same time I'm sure I don't know the details well enough to insist on it.

[BartMassey]

I've been quite busy lately, but I have a prototype implementation of flushing print macros and an RFC mostly written. I'll try to get it out for review in the next few weeks. If someone wants to help get it out sooner, let me know.

[anirudhb]

I believe that println! and print! should be kept alone.
For the people that want C-style buffering, why not create a printf! macro?
This time, the f refers to flush/force, meaning you will see the data immediately.
This has the advantage of being backwards-compatible, as all existing code using print! will be fine.
For the people that are okay with a performance hit and just want unbuffered printing, printf! should do fine.

(Also, having a printlnf! macro is a bad idea, because most people will mistake printf! for the C version. Having them mistake it for the C version is good, because it should be like the C version, anyways.)

[BartMassey]

The macro names I am currently using in my draft RFC are prompt!, eprompt! and input!. Bikeshed away… I'll try to post a draft Real Soon Now, I promise.

[satchm0h]

My guess is that a large percentage of folks will (or should) read the second edition book to get started. I would recommend changing the chapter two exercise code to use print and io::stdout().flush().unwrap() for the guess prompt. This way newbies (like myself) will just grok the rustafarian way to handle line buffered stdin. I'll try to put forth a pull request on the docs if someone has not already tried it.

If I were to make a suggestion for a more elegant solution I would go back to @CleanCut's suggestion to add a new macro. However, I would look for something a bit terser. Maybe fl_print! or flprint!? @BartMassey's suggestion of prompt! is a good alternative, but there may be other use cases for "print & flush" other than prompting for user input and 'prompt' can certainly have other connotations.

[BartMassey]

My current partially-finished draft RFC has an input!() macro for the prompt-and-read situation and a prompt!() macro for the just print-and-flush situation. Plus some other stuff. Plus a demo crate. I really need to get this out soon so that people can bikeshed it and hopefully move it forward. Unfortunately, the Rust class that I'm teaching for the first time this quarter is taking a lot of my Rust-related bandwidth. :-)

[Lucretiel]

+1 for a python-style input! macro, which takes care of the typical write-then-read case.

Sounds good! Should we remove the implicit flushing in println!() as well?

It's worth noting that technically println! doesn't flush. You can see in the source code that it simply calls print! with an appended newline. Rather, it's the current implementation of stdout which unconditionally line buffers, regardless of whether stdout is a tty or not. In fact, it's called about explicitly as a #FIXME in that code that the buffering behavior should be smarter; that change would implicitly change the flush behavior of all the macros that use it (print, println, and anything built on them).

[NealEhardt]

[Making print!() flush] would just move the stumbling block to some later point when buffered i/o and flushing inevitably bites them anyway. People writing interactive command-line apps need to understand what flushing means and inserting implicit flushes would not be doing them any favors.

I only have a shallow understanding of how flushing works, so please forgive the question. How will people be bitten? My guess:

• The application will be slower.
• If a message is split between multiple print!() statements, a partial message will appear on screen, quickly followed by the message tail.

Is there anything else? These issues seem trivial when I weigh them against the panic and frustration of "I'm printing stuff but it doesn't appear on-screen." The people writing performance-intensive command-line apps can use non-flushing io::stdout().write_fmt(format_args!(...)). Beginners and people using print!() for debugging will expect implicit flushing.

[x0a]

What about a printf! macro that prints and then flushes? Without modifying the behavior of the print! macro

[Lucretiel]

I'd be okay with a macro that guarantees a flush, but I'd be concerned about naming it printf, since that's such a ubiquitous identifier in many other languages for "formatted print"

[hyarsan]

How about flprint!, flprintln! macros? Doing printflln! or printlnfl! would look confusing. Is it a problem if it starts with fl cause that might makes it sound like the flushing happens first?