mailbox.mbox fails to pop two items in a row, flushing in between

[akheron]

BPO	15036
Nosy	@warsaw, @bitdancer, @akheron
Files	issue15036.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2012-06-15.18:05:30.363>
created_at = <Date 2012-06-08.14:18:54.953>
labels = ['type-bug', 'library', 'expert-email']
title = 'mailbox.mbox fails to pop two items in a row, flushing in between'
updated_at = <Date 2012-06-18.07:50:25.734>
user = 'https://github.com/akheron'

bugs.python.org fields:

activity = <Date 2012-06-18.07:50:25.734>
actor = 'petri.lehtinen'
assignee = 'none'
closed = True
closed_date = <Date 2012-06-15.18:05:30.363>
closer = 'petri.lehtinen'
components = ['Library (Lib)', 'email']
creation = <Date 2012-06-08.14:18:54.953>
creator = 'petri.lehtinen'
dependencies = []
files = ['26017']
hgrepos = []
issue_num = 15036
keywords = ['patch']
message_count = 10.0
messages = ['162528', '162529', '162811', '162890', '162926', '162928', '162982', '162986', '163090', '163091']
nosy_count = 4.0
nosy_names = ['barry', 'r.david.murray', 'python-dev', 'petri.lehtinen']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue15036'
versions = ['Python 2.7', 'Python 3.2', 'Python 3.3']

[akheron]

test_mbox is an mbox mailbox with a few messages in it.

>>> import mailbox
>>> inbox = mailbox.mbox('test_mbox')
>>> inbox.lock()
>>> inbox.popitem()
(0, <mailbox.mboxMessage instance at 0x7f78016bc680>)
>>> inbox.flush()
>>> inbox.unlock()
>>> inbox.lock()
>>> inbox.popitem()
(1, <mailbox.mboxMessage instance at 0x7f7801653320>)
>>> inbox.flush()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/mailbox.py", line 633, in flush
    (self._file_length, cur_len))
mailbox.ExternalClashError: Size of mailbox file changed (expected 141289, found 141147)

[akheron]

Actually, you don't even need to unlock() and lock() the mailbox before the second popitem() and flush().

[akheron]

The fix seems to be very simple:

diff --git a/Lib/mailbox.py b/Lib/mailbox.py
index a677729..2be4c83 100644
--- a/Lib/mailbox.py
+++ b/Lib/mailbox.py
@@ -675,6 +675,7 @@ class _singlefileMailbox(Mailbox):
                     new_file.write(buffer)
                 new_toc[key] = (new_start, new_file.tell())
                 self._post_message_hook(new_file)
+            self._file_length = new_file.tell()
         except:
             new_file.close()
             os.remove(new_file.name)

I guess all single-file mailboxes have this issue, not only mbox. I'll still need to add tests when I have time.

[akheron]

As I suspected, all single-file mailboxes(mbox, MMDF, Babyl) have this issue. Attached a patch with tests.

[python-dev]

New changeset 0add70dd3c43 by Petri Lehtinen in branch '2.7':
bpo-15036: Make a repeated changes and flushes work with single-file mailboxes
http://hg.python.org/cpython/rev/0add70dd3c43

New changeset 714b8f91f3d4 by Petri Lehtinen in branch '3.2':
bpo-15036: Make a repeated changes and flushes work with single-file mailboxes
http://hg.python.org/cpython/rev/714b8f91f3d4

New changeset 87d119117560 by Petri Lehtinen in branch 'default':
bpo-15036: Make a repeated changes and flushes work with single-file mailboxes
http://hg.python.org/cpython/rev/87d119117560

[bitdancer]

The news item isn't completely clear. It sounds like the mailbox is now automatically being flushed between pops, but what you really fixed is popping if the *application* does a flush between them, right?

[akheron]

Yes, this is what I tried to say. It's hard for me to find a good
wording, so what would you suggest?

I also noticed now that there's a typo in the commit messages. But
those cannot be fixed anymore.

[bitdancer]

"Mailbox no longer throws an error if a flush is done between operations when removing or changing multiple items in mbox, MMDF, or Babyl mailboxes."

[python-dev]

New changeset 8b38a81ba3bf by Petri Lehtinen in branch '2.7':
Fix NEWS entry for bpo-15036
http://hg.python.org/cpython/rev/8b38a81ba3bf

New changeset 38e2a87c9051 by Petri Lehtinen in branch '3.2':
Fix NEWS entry for bpo-15036
http://hg.python.org/cpython/rev/38e2a87c9051

New changeset 072b08989731 by Petri Lehtinen in branch 'default':
Fix NEWS entry for bpo-15036
http://hg.python.org/cpython/rev/072b08989731

[akheron]

Perfect, fixed.