Use with statement throughout the docs

[merwok]

BPO	10461
Nosy	@birkenfeld, @rhettinger, @terryjreedy, @vsajip, @abalkin, @ezio-melotti, @merwok
Files	cmd.rst.diff difflib.rst.diff collections.rst.diff issue10461.atexit.rst.diff issue10461.stdlib2.rst.diff with-in-docs-1.diff logging-cookbook.rst.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/merwok'
closed_at = <Date 2011-03-12.14:58:19.910>
created_at = <Date 2010-11-19.16:04:59.469>
labels = ['docs']
title = 'Use with statement throughout the docs'
updated_at = <Date 2011-03-12.14:58:19.909>
user = 'https://github.com/merwok'

bugs.python.org fields:

activity = <Date 2011-03-12.14:58:19.909>
actor = 'eric.araujo'
assignee = 'eric.araujo'
closed = True
closed_date = <Date 2011-03-12.14:58:19.910>
closer = 'eric.araujo'
components = ['Documentation']
creation = <Date 2010-11-19.16:04:59.469>
creator = 'eric.araujo'
dependencies = []
files = ['19663', '19664', '19665', '19667', '19668', '19765', '20112']
hgrepos = []
issue_num = 10461
keywords = ['patch']
message_count = 36.0
messages = ['121545', '121559', '121565', '121622', '121625', '121627', '121630', '121634', '121635', '121637', '121638', '121639', '121643', '121644', '121647', '121648', '121653', '121654', '121703', '121706', '121708', '121710', '121713', '121730', '122007', '122067', '122089', '122095', '124363', '124367', '124384', '130580', '130589', '130593', '130678', '130679']
nosy_count = 11.0
nosy_names = ['georg.brandl', 'rhettinger', 'terry.reedy', 'vinay.sajip', 'belopolsky', 'ezio.melotti', 'eric.araujo', 'eli.bendersky', 'SilentGhost', 'docs@python', 'python-dev']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue10461'
versions = ['Python 3.2']

[merwok]

The docs contain numerous examples that would trigger resource warnings under 3.2 (for example “open(...).read()”). They should be changed to use (and thus promote) the with statement.

Not adding the “easy” keyword, since grepping for those things is not easy.

Not sure we’ll backport that to 3.1 and 2.7.

[abalkin]

+1

BTW, I've updated examples in Unicode HOWTO to use with.

[terryjreedy]

+1
I have not yet had occasion to use 'with' yet, but in reading the Unicode HOWTO diff, I noticed that I liked replacing 'open,read,close' with 'with open, read' just for reading purposes since it turns 3 steps into 1 compound transaction.

Perhaps something should also be added to the doc style guide (along with using 'attributes' instead of 'members').

[merwok]

with also replaces open-try-do stuff-finally-close, the correct idiom for ensuring file handles are always closes in all VMs.

I don’t think the doc style guide is the right place, since this is a code issue. with is advertised in http://docs.python.org/dev/tutorial/inputoutput#methods-of-file-objects and http://docs.python.org/dev/howto/doanddont#exceptions ; http://docs.python.org/dev/library/functions#open says nothing about closing, and http://docs.python.org/dev/library/io tells about the with statement without recommending it strongly.

[elibendersky]

Éric, although grepping for all such references may be tricky, could you specify the places where you did see them? I guess a few fixed places is better than none at all.

[merwok]

Certainly. Here is my secret grep:
$ cd py3k/Doc
$ grep -n --color=auto -d skip -I --exclude-dir .svn --exclude-dir .hg -R open\(.*\).read *.rst c-api distutils documenting extending faq howto library reference tutorial using

List without false positives:

faq/library.rst
library/pkgutil.rst
library/atexit.rst
library/pipes.rst
library/cmd.rst
library/logging.rst
library/difflib.rst
library/collections.rst
tutorial/interpreter.rst
tutorial/stdlib2.rst

List for open(.*).write:
faq/library.rst
library/ftplib.rst
library/atexit.rst

There is probably a better regex that would catch “open(.*).[valid method name]”, but I hate regexes.

[elibendersky]

Eric, I'm attaching a provisional fix for library/atexit.rst just to be sure this is what you mean.

[merwok]

Yes, that’s it. Please don’t change whitespace in your patches.

[birkenfeld]

FWIW, this doesn't belong in the style guide; it is obvious that the docs should exhibit "best practice" Python code, and using the with statement when opening resources is certainly such a best practice now.

[elibendersky]

Eric, which whitespace change do you refer to. I changed to 4-spaces indentation in the code sample to conform to PEP-8. Shouldn't I have?

[SilentGhost]

Here is the patch for Docs/library/cmd.rst

[SilentGhost]

Here is the patch for Doc/library/difflib.rst

[elibendersky]

I'm trying to port the example in tutorial/stdlib2.rst, but the sample in "working with binary data record layouts" fails (before my porting to 'with'...)

struct.error: unpack requires a bytes argument of length 16

[SilentGhost]

Patch for Doc/library/collections.rst

[elibendersky]

Attaching patches for library/atexit.rst and for tutorial/stdlib2.rst

[elibendersky]

SilentGhost,

Your patches look fine. I have a doubt re collections.rst, however - about the Python prompt. The same issue is in faq/library.rst and I didn't want to touch it because I thought that on the prompt personally I probably wouldn't use 'with' - why write 2 lines when you can write 1? After all, it's just playing on the prompt - I don't care about safe closing of the file, etc.

[SilentGhost]

patch for Doc/library/logging.rst

Also, note the the change of the mode from 'r' to 'rb'. data_to_send is further send through socket and therefore requires to be bytes.

I expressed my opinion in irc, but I can repeat here that I think only the most trivial code such as in Doc/library/pipes.rst isn't worth converting.

[SilentGhost]

patch for Doc/library/logging.rst

Also, note the the change of the mode from 'r' to 'rb'. data_to_send is further send through socket and therefore requires to be bytes.

I expressed my opinion in irc, but I can repeat here that I think only the most trivial code such as in Doc/library/pipes.rst isn't worth converting.

[rhettinger]

Do not change the examples in collections.rst.
That would interfere with the clarify of what
is being demonstrated. For example, the hamlet.txt
example is not about file reading, it about
counting words.

[SilentGhost]

None of the changes are about file reading, they only about using with statement throughout. May be nofix then?

[rhettinger]

I think the other (non collections patches) are fine. The change doesn't break up the flow of the text.

[rhettinger]

Separate note for Éric: the try/finally examples do not need to change. Those are valid python. Users need to learn both try/finally and the with-statement.

[merwok]

Eli, SilentGhost: Please open other bug reports for doc errors like the struct one in stdlib2 or the r/rb one.

SilentGhost: with statement used with open *is* about file reading :)

Raymond: I agree about try/finally. I agree with Alexander about collections.rst.

[elibendersky]

On Sat, Nov 20, 2010 at 20:44, Éric Araujo <report@bugs.python.org> wrote:

Éric Araujo <merwok@netwok.org> added the comment:

Eli, SilentGhost: Please open other bug reports for doc errors like the
struct one in stdlib2 or the r/rb one.

SilentGhost: with statement used with open *is* about file reading :)

Raymond: I agree about try/finally. I agree with Alexander about
collections.rst.

Éric,

It turned out to be a non-issue, never mind.

[rhettinger]

Éric, please apply most of these.

In the atexit patch, the first change is wrong. Change it to a try/except/finally or skip it altogether.

In the collections patch, only include the change for the tail example; the other two should remain unchanged.

Skip the difflib patch entirely. It unnecessarily makes the example confusing and hard to follow.

Change the cmd patch to:
+ with open(arg) as f:
+ self.cmdqueue.extend(f.read().splitlines())

[merwok]

Raymond: I made a single diff for easier review. I’m not 100% sure I should change 'r' to 'rb' in logging: It’s unrelated to with, and the rest of the file has not been checked for similar errors. I prefer to do commits that don’t mix things.

Eli:

which whitespace change do you refer to. I changed to 4-spaces
indentation in the code sample to conform to PEP-8. Shouldn't I have?
PEP-8 only applies to Python code. http://docs.python.org/dev/documenting/style applies to the docs.

[elibendersky]

Éric,

Yes, in a consequent patch I fixed this - kept the formatted code indented at 3 spaces, while adhering to PEP-8 internally. If there's anything else, let me know.

[rhettinger]

I’m not 100% sure I should change 'r' to 'rb' in logging:
It’s unrelated to with, and the rest of the file
has not been checked for similar errors.

Good catch. This should only be a with-statement transformation. I caught other accidental semantic changes, so be continue to exercise care.

[SilentGhost]

following re-organization of the logging docs, I'm attaching updated patch.

[terryjreedy]

Vinay, you should look at the logging-cookbook patch.

[vsajip]

I've already made the change, Terry, just holding off committing it because Georg has frozen the py3k branch until 3.2b2 is released.

There are a few other changes I'm making, will commit these soon after 3.2b2 is released.

[merwok]

Perhaps something should also be added to the doc style guide (along
with using 'attributes' instead of 'members').

Terry, could you open another report for that, or maybe just commit the change directly?

[python-dev]

New changeset c3d28c84a372 by Éric Araujo in branch 'default':
Use with statement where it improves the documentation (closes bpo-10461)
http://hg.python.org/cpython/rev/c3d28c84a372

[SilentGhost]

Éric, I'm not sure about the first atexit.rst hunk. I thought the purpose of that code was to except IOError when no file is found. I'm not entirely sure why in the simplest case infile.read() would raise IOError. I'd think that eli's patch (file19667) would suit better.

[python-dev]

New changeset cebaf1bdaf78 by Éric Araujo in branch 'default':
Fix example in atexit doc: Both open and read could raise the IOError (bpo-10461 follow-up).
http://hg.python.org/cpython/rev/cebaf1bdaf78

[merwok]

Thanks for the catch! I was so used to the idiom where opening the file doesn’t fail but you protect the read or write that I didn’t think about the issue here.