Reimplementation of multifile.py

[dairiki]

BPO	413766
Nosy	@gvanrossum, @loewis, @warsaw
Files	multifile.py: Reimplementation of lib/multifile.py multifile.py: Reimplementation of lib/multifile.py (with one fewer bug)

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/warsaw'
closed_at = <Date 2002-09-22.09:32:39.000>
created_at = <Date 2001-04-04.18:06:38.000>
labels = ['extension-modules']
title = 'Reimplementation of multifile.py'
updated_at = <Date 2002-09-22.09:32:39.000>
user = 'https://bugs.python.org/dairiki'

bugs.python.org fields:

activity = <Date 2002-09-22.09:32:39.000>
actor = 'loewis'
assignee = 'barry'
closed = True
closed_date = None
closer = None
components = ['Extension Modules']
creation = <Date 2001-04-04.18:06:38.000>
creator = 'dairiki'
dependencies = []
files = ['3247', '3248']
hgrepos = []
issue_num = 413766
keywords = ['patch']
message_count = 8.0
messages = ['36277', '36278', '36279', '36280', '36281', '36282', '36283', '36284']
nosy_count = 4.0
nosy_names = ['gvanrossum', 'loewis', 'barry', 'dairiki']
pr_nums = []
priority = 'normal'
resolution = 'rejected'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue413766'
versions = []

[dairiki]

This is a re-implementation of the stock multifile.py

The main changes:

1. Efficiency:

This version supports calling the read() method with an
argument.
(In many cases, I've found that reading a MultiFile
line by line
is just too slow --- remember multipart messages often
contain
large binary attachments.)

This version performs reads on the underlying input
stream in
larger chunks as well, and uses a regular expression
search to
search for separator lines.

2. Buglets fixed

The original version has a buglet regarding its
handling of the
newline which preceeds a separator line. According to
RFC 2046,
section 5.1.1 the newline preceeding a separator is
part of the
separator, not part of the preceeding content. The old
version
of multifile.py treats the newline as part of the
content. Thus,
it introduces a spurious empty line at the end of each
content.

Matching of the separators: RFC 2046, section 5.1.1
also states,
that if the beginning of a line matches the separator,
it is a
separator. The old code ignores only trailing white
space when
looking for a separator line. This code ignores
trailing anything
on the separator line.

[dairiki]

Logged In: YES
user_id=45814

PS. FWIW, This was developed and tested under python 1.5.2.

[gvanrossum]

Logged In: YES
user_id=6380

Thanks. I'll assign this to Barry, who has been working on
another replacement for multifile, so maybe he can review
your contribution.

Barry, please don't sit on this too long -- If you've no
interest, please unassign it.

[warsaw]

Logged In: YES
user_id=12800

I will definitely look at this -- and soon -- but obviously
not in time for the 2.1 release. Geoffrey, have you looked
at mimelib (see url below)? My intent is to replace all the
MIME handling stuff in the standard library with mimelib.
I'm using mimelib extensively in Mailman, but I would love
to get some additional outside feedback about it. E.g. how
do you think your new multifile.py would fit in with
mimelib, and how well do you think mimelib conforms to RFC
2046?

http://barry.wooz.org/software/pyware.html

[dairiki]

Logged In: YES
user_id=45814

Oof. I wish I had found your mimelib a couple of weeks
ago. :-)

You'll notice I've also posted a patch to Mailman
(SF#413752)
which adds an option to filter MIME attachments to plain
text
(delete binary attachments, convert HTML to plain text,
...)
To do that (without defining new interfaces) I subclassed
MimeWriter --- it's a bit messy. Using mimelib probably
would have/will be cleaner.

The Mailman patch includes a text/{richtext,enriched} parser
(same interface as HTMLParser) which you guys might be
interested in.

I'm about to head off for a (long) weekend of skiing, so I
won't
have a chance to look carefully at mimelib until next week.
Do expect to hear from me then, though.

-Jeff

[dairiki]

Logged In: YES
user_id=45814

I just found an obscure but occasionally fatal bug.

I'm attaching a new, fixed, version of multifile.py to this
page.

[gvanrossum]

Logged In: YES
user_id=6380

Barry, is this still relevant given the email package?

[loewis]

Logged In: YES
user_id=21627

Since there has been no action on this, I consider this
irrelevant; users who want advanced email processing should
use the email package; multifile is for backwards
compatibilty only.