email "default" policy raises exception iterating over unparseable date headers

[rptb1]

BPO	35342
Nosy	@warsaw, @rptb1, @bitdancer, @iritkatriel
Superseder	bpo-30681: email.utils.parsedate_to_datetime() should return None when date cannot be parsed

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-04-16.19:03:21.701>
created_at = <Date 2018-11-28.17:37:09.953>
labels = ['3.8', 'type-bug', '3.7', 'expert-email']
title = 'email "default" policy raises exception iterating over unparseable date headers'
updated_at = <Date 2021-04-16.19:03:21.700>
user = 'https://github.com/rptb1'

bugs.python.org fields:

activity = <Date 2021-04-16.19:03:21.700>
actor = 'iritkatriel'
assignee = 'none'
closed = True
closed_date = <Date 2021-04-16.19:03:21.701>
closer = 'iritkatriel'
components = ['email']
creation = <Date 2018-11-28.17:37:09.953>
creator = 'rptb1'
dependencies = []
files = []
hgrepos = []
issue_num = 35342
keywords = []
message_count = 4.0
messages = ['330621', '330627', '382353', '391249']
nosy_count = 4.0
nosy_names = ['barry', 'rptb1', 'r.david.murray', 'iritkatriel']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '30681'
type = 'behavior'
url = 'https://bugs.python.org/issue35342'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8']

[rptb1]

It is not possible to loop over the headers of a message with an unparseable date field using the "default" policy. This means that a poison email can break email processing.

I expect to be able to process an email with an unparseable date field using the "default" policy.

$ python3 --version
Python 3.6.7
$ python3
Python 3.6.7 (default, Oct 22 2018, 11:32:17) 
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import email
>>> import email.policy
>>> email.message_from_string('Date: not a parseable date', policy=email.policy.default).items()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.6/email/message.py", line 460, in items
    for k, v in self._headers]
  File "/usr/lib/python3.6/email/message.py", line 460, in <listcomp>
    for k, v in self._headers]
  File "/usr/lib/python3.6/email/policy.py", line 162, in header_fetch_parse
    return self.header_factory(name, value)
  File "/usr/lib/python3.6/email/headerregistry.py", line 589, in __call__
    return self[name](name, value)
  File "/usr/lib/python3.6/email/headerregistry.py", line 197, in __new__
    cls.parse(value, kwds)
  File "/usr/lib/python3.6/email/headerregistry.py", line 306, in parse
    value = utils.parsedate_to_datetime(value)
  File "/usr/lib/python3.6/email/utils.py", line 210, in parsedate_to_datetime
    *dtuple, tz = _parsedate_tz(data)
TypeError: 'NoneType' object is not iterable
>>> 

Related: https://docs.python.org/3/library/email.headerregistry.html#email.headerregistry.DateHeader does not specify what happens to the datetime field if a date header cannot be parsed.

Related: https://docs.python.org/3/library/email.utils.html#email.utils.parsedate_to_datetime does not specify what happens if a date cannot be parsed.

Suggested tests: random fuzz testing of the contents of all email headers, especially those with parsers in the header registry.

[bitdancer]

This is effectively a duplicate of bpo-30681, which has a solution, although it is not yet in final form per the last couple of comments on the issue and the PR.

[iritkatriel]

I get this. Was the bug fixed?

>>> email.message_from_string('Date: not a parseable date', policy=email.policy.default).items()
[('Date', 'not a parseable date')]

[iritkatriel]

I get the same result on macOS as on Windows, it seems this was fixed in #22090, there are unit tests there for invalid dates passed to parsedate_to_datetime.