httplib's _read_chunked extremely slow for lots of chunks

[cjw296]

BPO	6838
Nosy	@cjw296
Files	httplib.patch.txt

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/cjw296'
closed_at = <Date 2009-09-04.17:15:57.116>
created_at = <Date 2009-09-04.16:08:39.430>
labels = ['easy']
title = "httplib's _read_chunked extremely slow for lots of chunks"
updated_at = <Date 2009-09-04.17:15:57.115>
user = 'https://github.com/cjw296'

bugs.python.org fields:

activity = <Date 2009-09-04.17:15:57.115>
actor = 'cjw296'
assignee = 'cjw296'
closed = True
closed_date = <Date 2009-09-04.17:15:57.116>
closer = 'cjw296'
components = []
creation = <Date 2009-09-04.16:08:39.430>
creator = 'cjw296'
dependencies = []
files = ['14830']
hgrepos = []
issue_num = 6838
keywords = ['easy']
message_count = 3.0
messages = ['92257', '92258', '92259']
nosy_count = 1.0
nosy_names = ['cjw296']
pr_nums = []
priority = 'high'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue6838'
versions = ['Python 2.6', 'Python 3.1', 'Python 2.7', 'Python 3.2']

[cjw296]

As the comment in this method suggests, accumulating the value by repeated string concatenation is slow. Appending to a list speeds this
up dramatically.

To quantify this, downloading a 110Mb file from Apache take:

~3s using Internet Explorer
2.2s using wget
30mins using the script in http://mail.python.org/pipermail/python-
dev/2009-September/091581.html

With the attached patch applied, this time drops to 2.3s.

[cjw296]

Fix for trunk committed in r74655.

[cjw296]

Fix merged to 2.6, 3.1 and 3.2 branches.