Email does not apply policy to multipart messages with defects

[jishac]

BPO	43818
Nosy	@warsaw, @bitdancer, @jishac
Files	email.multipart.test.py: Test script to reproduce the issue

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-04-12.20:17:28.769>
labels = ['3.8', 'expert-email']
title = 'Email does not apply policy to multipart messages with defects'
updated_at = <Date 2022-03-28.19:43:44.147>
user = 'https://github.com/jishac'

bugs.python.org fields:

activity = <Date 2022-03-28.19:43:44.147>
actor = 'mdengler'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['email']
creation = <Date 2021-04-12.20:17:28.769>
creator = 'jishac'
dependencies = []
files = ['49955']
hgrepos = []
issue_num = 43818
keywords = []
message_count = 2.0
messages = ['390897', '416204']
nosy_count = 4.0
nosy_names = ['barry', 'r.david.murray', 'mdengler', 'jishac']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue43818'
versions = ['Python 3.8']

[jishac]

I have noticed an issue (Python 3.8.5) where an email can be read in as bytes, but will not be returned as such with the as_bytes() call if the message is multipart, has a boundary which is not properly quoted, and the message has non-ascii text. It seems to be a result of how multipart messages are treated if the NoBoundaryInMultipartDefect is encountered [See Test #1].

I would argue that attempting to output the test message in the sample script with an 8bit, utf-8 enabled policy should return the original bytes as the 8bit policy should be applied to the "body" portion (any part after the null lines) of the email (as would be the case if it were not multipart) [See Test #4]

Currently it appears that the entire message is treated as headers, applying the strict 7bit, ascii requirement to the entire contents of the message. Furthermore, the msg.preamble is None.

I am also uncertain that attempting to leverage the handle_defect hooks would be helpful as correcting the boundary doesn't seem to work unless you re-parse the message [See Tests #2 and #3]

So the requested change would be to apply the encoding output policy to all portions of a message after the null line ending the headers.

[mdengler]

Possible related to #32137 and https://bugs.python.org/issue43323