email.parser.Parser is inefficient with large strings

[marcio]

BPO	8009
Nosy	@bitdancer
Files	test.py: Simple speed test

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/bitdancer'
closed_at = <Date 2010-12-28.04:24:37.894>
created_at = <Date 2010-02-24.09:44:03.496>
labels = ['library', 'performance']
title = 'email.parser.Parser is inefficient with large strings'
updated_at = <Date 2010-12-28.04:24:37.892>
user = 'https://bugs.python.org/marcio'

bugs.python.org fields:

activity = <Date 2010-12-28.04:24:37.892>
actor = 'r.david.murray'
assignee = 'r.david.murray'
closed = True
closed_date = <Date 2010-12-28.04:24:37.894>
closer = 'r.david.murray'
components = ['Library (Lib)']
creation = <Date 2010-02-24.09:44:03.496>
creator = 'marcio'
dependencies = []
files = ['16355']
hgrepos = []
issue_num = 8009
keywords = []
message_count = 2.0
messages = ['100019', '124760']
nosy_count = 2.0
nosy_names = ['r.david.murray', 'marcio']
pr_nums = []
priority = 'normal'
resolution = 'wont fix'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue8009'
versions = ['Python 2.6']

[marcio]

The email parser class is slow and memory intensive when dealing with sufficiently large strings.

For example, on a Windows 7 64-bit running at 1.60 GHz the attached test file gives the following results (number of seconds it took to parse a 10 MiB string):
Original: 76.6973627829
Modified: 0.231140741387

[bitdancer]

Parser is a legacy API, and message_from_string (which uses it) is just a convenience function. If performance is an issue for your application, call feedparser directly and optimize the feeding to suit your application.