mailbox / fromline matching

[camield]

BPO	521478
Nosy	@mwhudson, @gvanrossum, @tim-one, @loewis, @warsaw
Files	mailbox.py: mailbox.py diff mailbox.py.diff2: new diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/warsaw'
closed_at = <Date 2004-10-09.21:18:57.000>
created_at = <Date 2002-02-22.14:54:52.000>
labels = ['library']
title = 'mailbox / fromline matching'
updated_at = <Date 2004-10-09.21:18:57.000>
user = 'https://bugs.python.org/camield'

bugs.python.org fields:

activity = <Date 2004-10-09.21:18:57.000>
actor = 'barry'
assignee = 'barry'
closed = True
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2002-02-22.14:54:52.000>
creator = 'camield'
dependencies = []
files = ['4000', '4001']
hgrepos = []
issue_num = 521478
keywords = ['patch']
message_count = 10.0
messages = ['39054', '39055', '39056', '39057', '39058', '39059', '39060', '39061', '39062', '39063']
nosy_count = 6.0
nosy_names = ['mwh', 'gvanrossum', 'tim.peters', 'loewis', 'barry', 'camield']
pr_nums = []
priority = 'normal'
resolution = 'rejected'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue521478'
versions = ['Python 2.2']

[camield]

mailbox.py does not parse this 'From' line correctly:
From camield@sentia.nl Mon Apr 23 18:22:28 2001 +0200
^^^^^
This is because of the trailing timezone information,
that the regex does not account for.

Also, 'From' should match at the beginning of the line.

[gvanrossum]

Logged In: YES
user_id=6380

That From line is simply illegal, or at least nonstandard.

If your system uses this nonstandard format, you can extend
the mailbox parser by overriding the ._isrealfromline
method.

The pattern doesn't need ^ because match() is used, which
only matches at the start of the line.

Rejected.

[camield]

Logged In: YES
user_id=466784

I have tracked this down to Pine, the mailreader.

In imap/src/c-client/mail.c, it has this flag:
static int notimezones = NIL; /* write timezones in
"From " header */

(so timezones are written in the "From" lines by default)

I also found the following comment in imap/docs/FAQ in the
Pine distribution:

"""
So, good mail reading software only considers a line to be a
"From " line if it follows the actual specification for a
"From " line. This means, among other things, that the day
of week is fixed-format: "May 14", but "May 7" (note the
extra space) as opposed to "May 7". ctime() format for the
date is the most common, although POSIX also allows a
numeric timezone after the year.
"""

While I don't consider Pine to be the ultimate mailreader,
its heritage may warrant that the 'From ' lines it creates
are considered 'standard'.

[warsaw]

Logged In: YES
user_id=12800

IMO, Jamie Zawinski (author of the original mail/news reader
in Netscape among other accomplishments), wrote the
definitive answer on From_

http://home.netscape.com/eng/mozilla/2.0/relnotes/demo/content-length.html

As far as Python's support for this in the mailbox module,
for backwards compatibility, the UnixMailbox class has a
strict-ish interpretation of the From_ delimiter, which I
think should not change. It also has a class called
PortableUnixMailbox which recognizes delimiters as specified
in JWZ's document. Personally, if I was trolling over a
real world mbox file I'd only use PortableUnixMailbox (as
long as non-delimiter From_ lines were properly escaped -- I
have some code in Mailman which tries to intelligently "fix"
non-escaped mbox files).

I agree with the Rejected resolution.

[camield]

Logged In: YES
user_id=466784

PortableUnixMailbox is not that useful, because it only
matches '^From '. From-quoting is an even bigger mess then
From-headerlines, so that does not really help.

I submit a new diff that matches '\n\nFrom ' or
'<start-of-file>From ', which makes PortableUnixMailbox
useful for my purposes. It is not that intrusive as the
comment in the mailbox.py suggests.

[warsaw]

Logged In: YES
user_id=12800

Re-opening and assigning to myself. I'll take a look at
your patches asap.

[mwhudson]

Logged In: YES
user_id=6656

Anything going to happen here by Monday?

[tim-one]

Logged In: YES
user_id=31435

Since the Monday in question happened over 2 years ago, the
answer to Michael's question is apparently "no" <wink>.
Barry, we're stretching the conventional meaning of "asap"
here -- can you close this one way or t'other now?

[loewis]

Logged In: YES
user_id=21627

The patch, as it stands, appears to be incorrect. It is
looking for *two* empty lines between messages, whereas
folder conventionally contain only a single empty line; this
is also what Zawinski says.

If that was fixed, I think the patch would be acceptable -
mailbox.py currently does not implement the rule that the
From: line must be preceded with an empty line.

[warsaw]

Logged In: YES
user_id=12800

I see no follow up to Martin's comment of 2004-08-18.
Therefore, closing, however if you come up with a patch that
addresses his comments you can re-open it.