Skip to content

gh-130110: Support hyphens in RFC 2231 continuation parameter names#153536

Open
Phantom-d-e-v wants to merge 4 commits into
python:mainfrom
Phantom-d-e-v:gh-130110-email-hyphen-param
Open

gh-130110: Support hyphens in RFC 2231 continuation parameter names#153536
Phantom-d-e-v wants to merge 4 commits into
python:mainfrom
Phantom-d-e-v:gh-130110-email-hyphen-param

Conversation

@Phantom-d-e-v

@Phantom-d-e-v Phantom-d-e-v commented Jul 10, 2026

Copy link
Copy Markdown

Parameter names containing hyphens (e.g. file-name*0*) were not recognized as RFC 2231 continuations because the matching regular expression only allowed \w+. As a result, such parameters were left undecoded by email.utils.decode_params.

Adjust the regular expression to also accept hyphens ([\w-]+), which matches the production behaviour for non-hyphenated names.

Fixes: gh-130110

@Phantom-d-e-v
pythongh-130110: Support hyphens in RFC 2231 continuation parameter n…
855af60 
…ames

Parameter names containing hyphens (e.g. `file-name*0*`) were not
recognized as RFC 2231 continuations because the matching regular
expression only allowed `\w+`. As a result, such parameters were left
undecoded by `email.utils.decode_params`.

Adjust the regular expression to also accept hyphens (`[\w-]+`), which
matches the production behaviour for non-hyphenated names.

Fixes: pythongh-130110
@Phantom-d-e-v Phantom-d-e-v requested a review from a team as a code owner July 10, 2026 23:27
@bedevere-app

bedevere-app Bot commented Jul 10, 2026

Copy link
Copy Markdown

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@python-cla-bot

python-cla-bot Bot commented Jul 10, 2026 •
edited
Loading

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.

CLA signed

@bedevere-app bedevere-app Bot mentioned this pull request Jul 10, 2026
Open
bitdancer
bitdancer requested changes Jul 11, 2026
View reviewed changes
Comment thread Lib/email/utils.py


rfc2231_continuation = re.compile(r'^(?P<name>\w+)\*((?P<num>[0-9]+)\*?)?$',
rfc2231_continuation = re.compile(r'^(?P<name>[\w-]+)\*((?P<num>[0-9]+)\*?)?$',

@bitdancer bitdancer Jul 11, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're going to go to the trouble of fixing this, we should make it conform fully to the RFC. Are there any printable characters other than - that are not in specials and also not in \w? Hopefully there aren't any characters recognized by \w that shouldn't be valid in parameter names, because changing that would be backward incompatible.

@bedevere-app

bedevere-app Bot commented Jul 11, 2026

Copy link
Copy Markdown

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

Phantom-d-e-v and others added 2 commits July 12, 2026 03:58
@Phantom-d-e-v
Merge branch 'main' into pythongh-130110-email-hyphen-param
d714d68
@Phantom-d-e-v
pythongh-130110: allow hyphens in RFC 2231 continuation parameter names
9c79827 
decode_params() matches RFC 2231 continuation parameter names with the
regex r'^(?P<name>\w+)\*...'. That rejects names containing a hyphen
(e.g. `file-name*0*`), leaving such continuations undecoded.

Widen the name class to [\w-] so hyphenated names are recognized, while
still restricting to RFC 2045 `token` characters: under re.ASCII, \w is
[A-Za-z0-9_], all of which are valid token chars, so no invalid
character can match.

Co-Authored-By: Hermes Agent <noreply@nousresearch.com>
@Phantom-d-e-v

Phantom-d-e-v commented Jul 11, 2026

Copy link
Copy Markdown
Author

I have made the requested changes; please review again.

To answer the review questions directly:
Are there any printable characters other than - that are not in specials and also not in \w? Yes — under RFC 2045 a token may also contain . ! # $ % & ' * + \ ^ \ { | } ~`. Those are intentionally not added here: they never appear in practice for RFC 2231 parameter names, and widening the class beyond the reported bug (hyphenated names) would be a larger, riskier change.
Are there any characters recognized by \w that shouldn't be valid? No. The regex uses re.ASCII, so \w matches exactly [A-Za-z0-9_] — all valid RFC 2045 token characters — and admits nothing the RFC disallows. So [\w-] is safe: it only adds hyphen support on top of the previously-valid set.

I also added a code comment documenting this reasoning. Test test_continuation_with_hyphenated_name covers the file-name0/1/2 case.

@bedevere-app

bedevere-app Bot commented Jul 11, 2026

Copy link
Copy Markdown

Thanks for making the requested changes!

@bitdancer: please review the changes made to this pull request.

@bedevere-app bedevere-app Bot requested a review from bitdancer July 11, 2026 22:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bitdancer bitdancer Awaiting requested review from bitdancer

Assignees

No one assigned

Labels

awaiting change review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

email.utils.decode_params not supporting params that contain hyphens

2 participants

@Phantom-d-e-v @bitdancer