header parsing could apply postel's law to encoded words inside quotes

[bitdancer]

BPO	16983
Nosy	@warsaw, @bitdancer

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2014-02-08.18:20:07.044>
created_at = <Date 2013-01-16.21:13:44.314>
labels = ['type-bug', 'expert-email']
title = "header parsing could apply postel's law to encoded words inside quotes"
updated_at = <Date 2014-02-08.18:20:07.043>
user = 'https://github.com/bitdancer'

bugs.python.org fields:

activity = <Date 2014-02-08.18:20:07.043>
actor = 'r.david.murray'
assignee = 'none'
closed = True
closed_date = <Date 2014-02-08.18:20:07.044>
closer = 'r.david.murray'
components = ['email']
creation = <Date 2013-01-16.21:13:44.314>
creator = 'r.david.murray'
dependencies = []
files = []
hgrepos = []
issue_num = 16983
keywords = []
message_count = 3.0
messages = ['180108', '210671', '210673']
nosy_count = 3.0
nosy_names = ['barry', 'r.david.murray', 'python-dev']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue16983'
versions = ['Python 3.3', 'Python 3.4']

[bitdancer]

It has come to my attention that at least some mail agents apply postel's law to addresses like the following:

From: "=?utf-8?Q?not_really_valid?=" <foo@example.com>

Since encountering something that looks like an encoded word but that is not is a very unlikely occurrence, we could go ahead and decode such strings, resulting in

"not really valid" <foo@example.com>

a defect would be registered, and some sort of 'strict' policy mode could refuse to do the decode (as well as several other non-compliant patterns, such as encoded words not separated by whitespace). I think the decoding should be the default, though.

This applies also to other headers where words or phrases can be quoted, such as in filenames. I have also encountered the quoted-encoded-word-as-filename in the wild.

[python-dev]

New changeset 1dcb9d0d53a6 by R David Murray in branch '3.3':
bpo-16983: Apply postel's law to encoded words inside quoted strings.
http://hg.python.org/cpython/rev/1dcb9d0d53a6

New changeset 5f7e626730df by R David Murray in branch 'default':
Merge: bpo-16983: Apply postel's law to encoded words inside quoted strings.
http://hg.python.org/cpython/rev/5f7e626730df

[bitdancer]

The old header parsing code already decodes these, although it gets the spacing wrong if you do the standard str(make_header(decode_header(x))) dance. The fix for the new header parsing code only handles the specific case of only encoded words surrounded by double quotes. That's the only variation I've seen in the wild so far, so I think that may be enough. To extend it to handle mixed regular text and encoded words would require rewriting the qcontent and ptext functions. Possible, but not worth it unless a real use case turns up. (Although, I think there might be a bug in quoted text parsing that may make that rewrite worthwhile later; but it is only a bug if you are actually walking the parse tree, it is not a functional bug.)

Oh, and I decided to treat this as a bug fix, not an enhancement, because the old parser code already did this decoding.