Header.decode_header eats up spaces

[mgoutell]

BPO	1467619
Nosy	@warsaw, @birkenfeld, @bitdancer
Superseder	bpo-1079: decode_header does not follow RFC 2047
Files	emailheader.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2012-06-03.16:14:50.396>
created_at = <Date 2006-04-10.10:33:54.000>
labels = ['type-bug', 'expert-email']
title = 'Header.decode_header eats up spaces'
updated_at = <Date 2012-06-03.16:14:50.394>
user = 'https://bugs.python.org/mgoutell'

bugs.python.org fields:

activity = <Date 2012-06-03.16:14:50.394>
actor = 'r.david.murray'
assignee = 'none'
closed = True
closed_date = <Date 2012-06-03.16:14:50.396>
closer = 'r.david.murray'
components = ['email']
creation = <Date 2006-04-10.10:33:54.000>
creator = 'mgoutell'
dependencies = []
files = ['1963']
hgrepos = []
issue_num = 1467619
keywords = []
message_count = 10.0
messages = ['28181', '28182', '28183', '28184', '28185', '114651', '114722', '150459', '150470', '162219']
nosy_count = 7.0
nosy_names = ['barry', 'georg.brandl', 'alexanderweb', 'mgoutell', 'r.david.murray', 'BreamoreBoy', 'runtux']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '1079'
type = 'behavior'
url = 'https://bugs.python.org/issue1467619'
versions = ['Python 3.3']

[mgoutell]

The Header.decode_header function eats up spaces in
non-encoded part of a header.

See the following source:
# -- coding: iso-8859-1 --
from email.Header import Header, decode_header
h = Header('Essai ', None)
h.append('éè', 'iso-8859-1')
print h
print decode_header(h)

This prints:
Essai =?iso-8859-1?q?=E9=E8?=
[('Test', None), ('\xe9\xe8', 'iso-8859-1')]

This should print:
Essai =?iso-8859-1?q?=E9=E8?=
[('Test ', None), ('\xe9\xe8', 'iso-8859-1')]
^ This space disappears

This appears in Python 2.3 but the source code of the
function didn't change in 2.4 so the same problem
should still exist. Bug "[ 1372770 ] email.Header
should preserve original FWS" may be linked to that one
although I'm not sure this is exactly the same.

This patch (not extensively tested though) seems to
solve this problem:

--- /usr/lib/python2.3/email/Header.py  2005-09-05
00:20:03.000000000 +0200
+++ Header.py   2006-04-10 12:27:27.000000000 +0200
@@ -90,7 +90,7 @@
             continue
         parts = ecre.split(line)
         while parts:
-            unenc = parts.pop(0).strip()
+            unenc = parts.pop(0).rstrip()
             if unenc:
                 # Should we continue a long line?
                 if decoded and decoded[-1][1] is None:

[alexanderweb]

Logged In: YES
user_id=254738

I can confirm this bug and have been bitten by it as well.

[mgoutell]

Hello,
Any news about this bug. It seems still there in 2.5 after a one year notice...
Regards,

[birkenfeld]

I propose the attached patch. RFC 2047 specifies to ignore whitespace between encoded-words, but IMHO not between ordinary text and encoded-words.
File Added: emailheader.diff

[warsaw]

IIRC, I tried the OP's patch and it broke too many of the email package's test suite. I made an attempt at fixing the problem to be much more RFC compliant, but couldn't get the test suite to pass completely. This points to a much deeper problem with email package header management. I don't think the problem is a bug, I think it's a design flaw.