Use MultiDict for headers

[shazow]

Currently httplib coerces headers into a dict, which breaks some things. Some monkeypatching might be required.

See also: Issue 15 @ GoogleCode for more discussion.

[piotr-dobrogost]

Currently httplib coerces headers into a dict, which breaks some things

Well, to be precise the problem is not dict itself but merging headers with the same name into one entry.
It's worth noting it was fixed in Python 3:

2.7.2

>>> import httplib
>>> conn = httplib.HTTPConnection('www.wp.pl')
>>> conn.request('HEAD', '/index.html')
>>> res = conn.getresponse()
>>> print res.getheaders()
[('set-cookie', 'reksticket=1323027255; expires=Tuesday, 06-Dec-11 19:34:15 GMT; path=/; domain=.www.wp.pl,
rekticket=1323027255; expires=Tuesday, 06-Dec-11 19:34:15 GMT; path=/; domain=.wp.pl,
statid=77.255.32.195.20351:1323027255:2355453207:v1; path=/; expires=Wed, 03-Dec-14 19:34:15 GMT,
statid=77.255.32.195.20351:1323027255:2355453207:v1; domain=.wp.pl; path=/; expires=Wed, 03-Dec-14 19:34:15 GMT'),...]

3.2.2

>>> from http import client
>>> conn = client.HTTPConnection('www.wp.pl')
>>> conn.request('HEAD', '/index.html')
>>> res = conn.getresponse()
>>> print(res.getheaders())
[('Set-Cookie', 'reksticket=1323028283; expires=Tuesday, 06-Dec-11 19:51:23 GMT; path=/; domain=.www.wp.pl'), ('Set-Cookie',
'rekticket=1323028283; expires=Tuesday, 06-Dec-11 19:51:23 GMT; path=/; domain=.wp.pl'), ('Set-Cookie',
'statid=77.255.32.195.7252:1323028283:2993994873:v1; path=/; expires=Wed, 03-Dec-14 19:51:23 GMT'), ('Set-Cookie',
'statid=77.255.32.195.7252:1323028283:2993994873:v1; domain=.wp.pl; path=/; expires=Wed, 03-Dec-14 19:51:23 GMT'), ...]

This functionality is provided by (undocumented?) HTTPMessage class and Python issue 4773 HTTPMessage not documented and has inconsistent API across 2.6/3.0 is of interest here. The difference shown above is noted by Joel Verhagen in comment http://bugs.python.org/issue4773#msg154781.

The issue we see was raised as issue 1660009 continuing problem with httplib multiple set-cookie headers Python issue.
John J Lee suggests using getallmatchingheaders() method or headers property of rfc822.Message class to get raw header data.

[shazow]

@piotr-dobrogost Thanks for doing the research and posting an update of the current state. The 3.2.2 thing is very interesting.

Maybe we can punt this issue to "someday when we fork for 3.x" like the rest. :P I'll open a py3 issue to track these things.

[karlcow]

Just FYI. It is not totally an issue. It is subtle.

https://github.com/kennethreitz/requests/issues/741#issuecomment-7293231

[shazow]

We have HTTPHeaderDict now, I think it satisfies this.