Support of 'with' statement fo TarFile class

[jaimebuelta]

BPO	7232
Nosy	@gustaebel, @ezio-melotti, @merwok, @bitdancer, @briancurtin, @meadori
Files	tarfileWithSupport.patch: tarfile with statement support patch tarfileWithSupportv2.patch issue7232.diff: minimal patch with test and doc issue7232.2.diff issue7232.3.diff issue7232.4.diff: use assertRaises as a context manager issue7232.5.diff issue7232.6.diff issue7232.7.diff issue7232.8.diff issue7232.9.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/gustaebel'
closed_at = <Date 2010-03-03.12:15:01.223>
created_at = <Date 2009-10-29.07:29:33.067>
labels = ['type-feature', 'library']
title = "Support of 'with' statement fo TarFile class"
updated_at = <Date 2010-03-03.12:16:02.794>
user = 'https://bugs.python.org/jaimebuelta'

bugs.python.org fields:

activity = <Date 2010-03-03.12:16:02.794>
actor = 'lars.gustaebel'
assignee = 'lars.gustaebel'
closed = True
closed_date = <Date 2010-03-03.12:15:01.223>
closer = 'lars.gustaebel'
components = ['Library (Lib)']
creation = <Date 2009-10-29.07:29:33.067>
creator = 'jaime.buelta'
dependencies = []
files = ['15220', '16291', '16307', '16367', '16386', '16391', '16394', '16396', '16398', '16400', '16401']
hgrepos = []
issue_num = 7232
keywords = ['patch', 'needs review']
message_count = 18.0
messages = ['94645', '94646', '94647', '99698', '99707', '99761', '100083', '100181', '100193', '100195', '100198', '100199', '100201', '100210', '100216', '100218', '100222', '100339']
nosy_count = 7.0
nosy_names = ['lars.gustaebel', 'ezio.melotti', 'eric.araujo', 'r.david.murray', 'brian.curtin', 'meador.inge', 'jaime.buelta']
pr_nums = []
priority = 'normal'
resolution = 'accepted'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue7232'
versions = ['Python 2.7', 'Python 3.2']

[jaimebuelta]

Currently, the TarFile is not supporting the 'with' statement, which I
think it should for coherence with other file classes.

I've already created a patch including it for consideration.

[gustaebel]

Please clean up the patch, and I take another look at it.

[jaimebuelta]

I've cleaned the patch, I don't now why Eclipse added a lot of garbage,
sorry.

[briancurtin]

Here is a patch which expands on Jaime's patch. I was converting tests for bpo-7944 and looked at test_tarfile, and implemented the same feature that he did.

All places where context managers should be used in the test, they are used. Includes a doc update with a small example.

[bitdancer]

Woops, I accidentally deleted one of the patch files. Adding back.

[briancurtin]

The last patch does more than it should for this issue. Here is a minimal patch with the change, test, and doc updates.

[meadori]

Built on Brian's patch by adding the following items:

• Added a unit test case to cover exceptional conditions.
• Added doc strings on __enter__ and __exit__ (more consistent
  with the surrounding code).
• Spelling error in doc update: s/manaager/manager/.
• Link doc update to context manager type documentation (just in
  case the tarfile user is unfamiliar with context manager types).

[gustaebel]

I have taken the most recent patch by Meador, fixed the documentation example and made the test more verbose. I will check in my patch on time before 2.7 alpha 4 is released if no one of you protests.

[briancurtin]

What about changing the exception test to something like what I did in bpo-7232.4.diff?

[meadori]

What about changing the exception test to something like what I did in > bpo-7232.4.diff?

That is definitely more succinct, but Lars' solution provides more information about _why_ the test fails. IMHO, the descriptiveness is
more important than succinctness. Especially when debugging a failed
test.

[briancurtin]

Good point. How about version 5? It uses base Exception in the context manager, which will allow us to differentiate between no exception being raised, and the wrong one being raised. After the context manager, we check the type of the exception to make sure it's correct.

I changed the exception being raised to an IOError. It could be anything, but given that AssertionError is what gets raised by the assert functions, it seemed better to avoid that one specifically. This is fairly minor, though.

If you change the "raise IOError" to be a pass statement, or another type of exception, you can see that the same level of information is given to you as in patch version 3 by Lars.

[merwok]

Hello

Minor note: I think magic methods shouldn’t have docstrings, because their name is enough doc (or at least enough to go read the doc). At most a one-line comment like “context protocol” can be useful. (The exception is __init__, which doesn’t have a defined set of arguments.)

Cheers

[gustaebel]

Another version of the patch (bpo-7232.6.diff) that checks if the TarFile object is still open in the __enter__() method (plus a test for that). I removed the docstrings as Eric suggested. This is common practice in the standard library.

[meadori]

This is common practice in the standard library.

This doesn't necessarily mean it is a correct practice :-). All
kidding aside, I think the assumption that the standard documentation
on '__enter__' and '__exit__' is sufficient is a bad one. With respect
to how the 'tarfile' versions of these methods behave, that
documentation is not that helpful.

In particular, the special behavior of an 'IOError' potentially being
thrown from '__enter__' and the fact that '__exit__' does not swallow
the exception. These special behaviors should be documented either in
a docstring or the library documentation. I think this is important,
but I may be being a bit pedantic.

Also, the last change to 'test_context_manager_exception' has a bug.
If the call to 'tarfile.open' throws an exception, then the call to
'self.assertRaises' will swallow it. This will cause an undefined
variable reference to 'tar' in 'self.assertTrue(tar.closed, ...)'. I
attached another update that fixes this problem.

[gustaebel]

IMO it is okay for __enter__() and __exit__() not to have docstrings.
I cannot see what's so special about the behaviour of __enter__() and __exit__().

__enter__() raises IOError only if the TarFile object has been already closed. This is exactly the behaviour I would expect, because it is the same every other TarFile method does when the object has been closed. IOW, using a closed TarFile as a context manager is the programmer's mistake, and I don't feel the need to document that case.

The fact that __exit__() only closes the TarFile object and does not swallow exceptions is what everyone expects from a "file object". It is the only logical thing to do, no need to document that either.

The test_context_manager_exception() test is fine. If the call to tarfile.open() really raises an exception then something is so terribly wrong and probably all of the testsuite's 200 tests will fail anyway. We can safely assume here that this will work, no need to double-check.

However, I have changed the docs again to be a bit more specific.

[meadori]

Patch 'bpo-7232.8.diff' looks good to me. Thanks for the explanation.

[gustaebel]

I found an issue that needs to be addressed: if there is an error while the TarFile object is opened for writing, we cannot simply call TarFile.close() in the __exit__() method. close() would try to finalize the archive, i.e. write two zero end-of-archive blocks and a number of padding blocks.

I changed __exit__() to call close() only if everything went fine. If there was an exception only the most basic cleanup is done.

I added more tests and adapted the docs.

[gustaebel]

Okay, it is done, see r78623 (trunk) and r78626 (py3k).

Thanks to all for your work and support!