Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

non-ascii characters in headers causes TypeError on email.policy.Policy.fold #77705

Closed
rad164 mannequin opened this issue May 15, 2018 · 6 comments
Closed

non-ascii characters in headers causes TypeError on email.policy.Policy.fold #77705

rad164 mannequin opened this issue May 15, 2018 · 6 comments
Labels
3.7 (EOL) end of life 3.8 only security fixes topic-email type-bug An unexpected behavior, bug, or error

Comments

@rad164
Copy link
Mannequin

rad164 mannequin commented May 15, 2018

BPO 33524
Nosy @warsaw, @bitdancer, @matrixise, @csabella, @miss-islington
PRs
  • bpo-33524: Fix the behavior when max_line_length is 0 or None in email.policy #6966
  • bpo-33524: Fix the folding of email header when max_line_length is 0 or None #13391
  • [3.7] bpo-33524: Fix the folding of email header when max_line_length is 0 or None (GH-13391) #13393
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2019-05-17.20:54:43.655>
    created_at = <Date 2018-05-15.16:35:30.339>
    labels = ['3.8', 'type-bug', '3.7', 'expert-email']
    title = 'non-ascii characters in headers causes TypeError on email.policy.Policy.fold'
    updated_at = <Date 2019-05-17.20:54:43.654>
    user = 'https://bugs.python.org/rad164'

    bugs.python.org fields:

    activity = <Date 2019-05-17.20:54:43.654>
    actor = 'cheryl.sabella'
    assignee = 'none'
    closed = True
    closed_date = <Date 2019-05-17.20:54:43.655>
    closer = 'cheryl.sabella'
    components = ['email']
    creation = <Date 2018-05-15.16:35:30.339>
    creator = 'rad164'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 33524
    keywords = ['patch']
    message_count = 6.0
    messages = ['316680', '316681', '335436', '342758', '342763', '342765']
    nosy_count = 7.0
    nosy_names = ['barry', 'r.david.murray', 'matrixise', 'altvod', 'cheryl.sabella', 'miss-islington', 'rad164']
    pr_nums = ['6966', '13391', '13393']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue33524'
    versions = ['Python 3.7', 'Python 3.8']

    @rad164
    Copy link
    Mannequin Author

    rad164 mannequin commented May 15, 2018

    Python 3.6.5 has trouble on folding email messages:

    Traceback (most recent call last):
      File "emailtest.py", line 7, in <module>
        policy.fold("Subject", msg["Subject"])
      File "/usr/lib/python3.6/email/policy.py", line 183, in fold
        return self._fold(name, value, refold_binary=True)
      File "/usr/lib/python3.6/email/policy.py", line 205, in _fold
        return value.fold(policy=self)
      File "/usr/lib/python3.6/email/headerregistry.py", line 258, in fold
        return header.fold(policy=policy)
      File "/usr/lib/python3.6/email/_header_value_parser.py", line 144, in fold
        return _refold_parse_tree(self, policy=policy)
      File "/usr/lib/python3.6/email/_header_value_parser.py", line 2651, in _refold_parse_tree
        part.ew_combine_allowed, charset)
      File "/usr/lib/python3.6/email/_header_value_parser.py", line 2728, in _fold_as_ew
        first_part = to_encode[:text_space]
    TypeError: slice indices must be integers or None or have an __index__ method

    The message has non-ascii characters in header and set max_line_length=0, regardless length of the header. Here is the code to reproduce.

    from email.message import EmailMessage
    from email.policy import default
    
    policy = default.clone(max_line_length=0)
    msg = EmailMessage()
    msg["Subject"] = "á"
    policy.fold("Subject", msg["Subject"])

    I first found this issue on Maildir.add, which saves the message to a file without word wrap.

    @rad164 rad164 mannequin added topic-email type-bug An unexpected behavior, bug, or error labels May 15, 2018
    @bitdancer
    Copy link
    Member

    Oh, interesting.

    I could argue that that's a missing feature in Python's slice handling. The value of text_space at that point is '+inf', and I obviously incorrectly assumed that slicing would treat that as if it were [:]. The counter argument, of course, is that inf is a float not an integer. I suppose we'll have to use sys.maxsize instead.

    @bitdancer bitdancer added 3.7 (EOL) end of life 3.8 only security fixes labels Jul 25, 2018
    @matrixise
    Copy link
    Member

    See this issue as a duplicata of this one:

    https://bugs.python.org/issue35985

    @bitdancer
    Copy link
    Member

    New changeset feac6cd by R. David Murray (Abhilash Raj) in branch 'master':
    bpo-33524: Fix the folding of email header when max_line_length is 0 or None (bpo-13391)
    feac6cd

    @miss-islington
    Copy link
    Contributor

    New changeset 5386aaf by Miss Islington (bot) in branch '3.7':
    bpo-33524: Fix the folding of email header when max_line_length is 0 or None (GH-13391)
    5386aaf

    @csabella
    Copy link
    Contributor

    Thank you, @Licht-T for the original patch and @maxking for the rebase. Also, thank you, @r.david.murray for the review and merge.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life 3.8 only security fixes topic-email type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants