bpo-28647: Update -u documentation after bpo-30404

[berkerpeksag]

https://bugs.python.org/issue28647

[serhiy-storchaka]

Thank you. Please update also the man page.

[berkerpeksag]

@serhiy-storchaka should I add the "stdin is always buffered" part to 77732be#diff-1dc031ac9a0d3915ff7805109a005799R306 too?

[serhiy-storchaka]

I don't know. This option doesn't affect stdin. Does it add clarity or noise if document this explicitly?

Maybe add this to the RST documentation, but remove from the help and manpage? What other core developers think?

[vstinner]

LGTM.

[vstinner]

@serhiy-storchaka should I add the "stdin is always buffered" part to 77732be#diff-1dc031ac9a0d3915ff7805109a005799R306 too?

I like the idea of mentionning that stdin is always buffered in documentation of the "-u" command line option.

You may also update the man page in this PR, or it can be done in a different PR.

[vstinner]

Strange. The docs job of Travis CI failed on "pip --version":

$ pip --version

No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.

Check the details on how to adjust your build configuration on: https://docs.travis-ci.com/user/common-build-problems/#Build-times-out-because-no-output-was-received

The build has been terminated

First time that I see this error.

[berkerpeksag]

I've addressed review comments.

[serhiy-storchaka]

The man page doesn't contain additional information in Python 3.

[berkerpeksag]

Good catch, removed outdated sentence.

[berkerpeksag]

@serhiy-storchaka documented stdin as unbuffered.

[serhiy-storchaka]

LGTM, but wait if there are other comments.

[vstinner]

"stdin is always unbuffered"

Wait, what? It's always buffered, no?

Reading one byte in Python reads 1024 bytes at the C level:

haypo@selma$ strace -o trace ./python -u -c 'import sys; sys.stdin.buffer.read(1)'
abc
haypo@selma$ grep -F 'read(0,' trace
read(0, "abc\n", 1024)                  = 4

If stdin is a not a TTY, we read even more at the C level: 4096 bytes.

haypo@selma$ echo abc|strace -o trace ./python -u -c 'import sys; sys.stdin.buffer.read(1)'
haypo@selma$ grep -F 'read(0,' trace
read(0, "abc\n", 4096)                  = 4

The -u option has an effect on sys.stdin: line_buffering. I don't think that it matters in practice, since line buffering only impacts write() whereas sys.stdin is open for read-only.

haypo@selma$ ./python -c 'import sys; print(sys.stdin.line_buffering)'
True
haypo@selma$ ./python -u -c 'import sys; print(sys.stdin.line_buffering)'
False

[vstinner]

I checked with gdb just to be sure: I confirm that sys.stdin.buffer.read(1) calls read(1024), so reads more than 1 byte, store the following bytes for the following buffer.read() and so is buffered.

[vstinner]

always buffered

[bedevere-bot]

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

[serhiy-storchaka]

It is buffered in other meaning than in Python 2. This buffering doesn't affect behavior (unlike to Python 2). The note about buffering was related to Python 2 buffering. Try the following code in Python 2 and Python 3:

./python -u -c $'import sys\nfor x in sys.stdin: print(repr(x))'

[vstinner]

It is buffered in other meaning than in Python 2. This buffering doesn't affect behavior (unlike to Python 2).

In that case, we should elaborate that in the documentation. I disagree to simply say "unbuffered".

I suggest to not compare Python 3 to Python 2. Python 2 behaviour is weird and cannot be defined :-D

[serhiy-storchaka]

We are fixing the outdated documentation inherited from Python 2. First than keep some statement we should consider what it means in the context of Python 2 and what it means in the context of Python 3.

I suggest to continue the discussion on the tracker.

[vstinner]

For comparison, this is fully unbuffered read:

haypo@selma$ strace -o trace python3 -c 'f = open("/etc/issue", "rb", buffering=0); f.read(1)'
haypo@selma$ cat trace
...
open("/etc/issue", O_RDONLY|O_CLOEXEC)  = 3
...
read(3, "\\", 1)                        = 1
...
close(3)                                = 0
...

Reading a single byte does... read exactly one byte, and not more.

[berkerpeksag]

I've now updated the PR.

[vstinner]

LGTM.

What do you think Serhiy?

[serhiy-storchaka]

I think all this paragraph is related to stdout and stderr. Just replace "standard streams" with "stdout and stdin". No other changes are needed.

[berkerpeksag]

Just replace "standard streams" with "stdout and stdin".

Did you mean "stdout and stderr" instead of "stdout and stdin"?

[serhiy-storchaka]

Yes. Sorry.

[berkerpeksag]

Sure, I can change that. Another question is that shouldn't we still document stdin buffering?