String index out of range in get_group(), email/_header_value_parser.py

[Cacadril]

BPO	33476
Nosy	@warsaw, @bitdancer, @zooba, @corona10, @miss-islington
PRs	bpo-33476: Fix _header_value_parser.py when address group is missing final ';' #7484 [3.7] bpo-33476: Fix _header_value_parser when address group is missing final ';' (GH-7484) #8522 [3.6] bpo-33476: Fix _header_value_parser when address group is missing final ';' (GH-7484) #8524
Files	email.eml: email with a group of recipient but missing ";"

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-07-28.16:45:54.846>
created_at = <Date 2018-05-13.00:50:02.620>
labels = ['easy', '3.8', 'type-bug', '3.7', 'expert-email']
title = 'String index out of range in get_group(), email/_header_value_parser.py'
updated_at = <Date 2018-07-28.16:45:54.845>
user = 'https://bugs.python.org/Cacadril'

bugs.python.org fields:

activity = <Date 2018-07-28.16:45:54.845>
actor = 'steve.dower'
assignee = 'none'
closed = True
closed_date = <Date 2018-07-28.16:45:54.846>
closer = 'steve.dower'
components = ['email']
creation = <Date 2018-05-13.00:50:02.620>
creator = 'Cacadril'
dependencies = []
files = ['47606']
hgrepos = []
issue_num = 33476
keywords = ['patch', 'easy']
message_count = 9.0
messages = ['316440', '316538', '317165', '318049', '319007', '319368', '322554', '322567', '322568']
nosy_count = 6.0
nosy_names = ['barry', 'r.david.murray', 'steve.dower', 'corona10', 'miss-islington', 'Cacadril']
pr_nums = ['7484', '8522', '8524']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue33476'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8']

[Cacadril]

When address group is missing final ';', 'value' will be an empty string. I suggest the following patch

$ diff -u _save_header_value_parser.py _header_value_parser.py
--- _save_header_value_parser.py        2018-03-14 01:07:54.000000000 +0100
+++ _header_value_parser.py     2018-05-13 02:17:13.830053600 +0200
@@ -1876,7 +1876,7 @@
     if not value:
         group.defects.append(errors.InvalidHeaderDefect(
             "end of header in group"))
-    if value[0] != ';':
+    elif value[0] != ';':
         raise errors.HeaderParseError(
             "expected ';' at end of group but found {}".format(value))
     group.append(ValueTerminal(';', 'group-terminator'))

[bitdancer]

The fix is mostly likely correct, so we need a PR for this that includes tests.

[Cacadril]

Unsure how to issue a "PR" (Problem Report?) with a test case.

Here is my best effort:

Create a file "email.eml" in the current directory, as attached.
(The contents were lifted from RFC2822 section A.1.3, but I deleted the ";" at the end of the "To" header. The file has CRLF line endings.)

Then run the following test program (It appears that I can only attach one file to this ).

$ cat test-bug.py
from email.policy import default
import email

with open('email.eml', 'rb') as f:
    msg = email.message_from_binary_file(f, policy=default)
    toheader = msg['To']
    for addr in toheader.addresses:
        print(addr)

#----------------------------------------------------
# Output without the fix:

$ python3.6.5 test-bug.py
Traceback (most recent call last):
  File "test-bug.py", line 6, in <module>
    toheader = msg['To']
  File "C:\Program Files\Python36\lib\email\message.py", line 391, in __getitem__
    return self.get(name)
  File "C:\Program Files\Python36\lib\email\message.py", line 471, in get
    return self.policy.header_fetch_parse(k, v)
  File "C:\Program Files\Python36\lib\email\policy.py", line 162, in header_fetch_parse
    return self.header_factory(name, value)
  File "C:\Program Files\Python36\lib\email\headerregistry.py", line 589, in __call__
    return self[name](name, value)
  File "C:\Program Files\Python36\lib\email\headerregistry.py", line 197, in __new__
    cls.parse(value, kwds)
  File "C:\Program Files\Python36\lib\email\headerregistry.py", line 340, in parse
    kwds['parse_tree'] = address_list = cls.value_parser(value)
  File "C:\Program Files\Python36\lib\email\headerregistry.py", line 331, in value_parser
    address_list, value = parser.get_address_list(value)
  File "C:\Program Files\Python36\lib\email\_header_value_parser.py", line 1931, in get_address_list
    token, value = get_address(value)
  File "C:\Program Files\Python36\lib\email\_header_value_parser.py", line 1908, in get_address
    token, value = get_group(value)
  File "C:\Program Files\Python36\lib\email\_header_value_parser.py", line 1879, in get_group
    if value[0] != ';':
IndexError: string index out of range

#-----------------------------------------------------
# Output with the fix:

$ test-bug.py
Chris Jones <c@a.test>
joe@where.test
John <jdoe@one.test>

[corona10]

@Cacadril

Please send a pull request(PR) through CPython GitHub(https://github.com/python/cpython)

It will be a great experience :)

[corona10]

@r.david.murray

Please take a look PR 7484 :)

[corona10]

@r.david.murray
Can I get a review for PR 7484?

[zooba]

New changeset 8fe9eed by Steve Dower (Dong-hee Na) in branch 'master':
bpo-33476: Fix _header_value_parser when address group is missing final ';' (GH-7484)
8fe9eed

[miss-islington]

New changeset 2be0124 by Miss Islington (bot) in branch '3.7':
bpo-33476: Fix _header_value_parser when address group is missing final ';' (GH-7484)
2be0124

[miss-islington]

New changeset f17e001 by Miss Islington (bot) in branch '3.6':
bpo-33476: Fix _header_value_parser when address group is missing final ';' (GH-7484)
f17e001