New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Double dots in quopri transported emails #88088
Comments
Hello, We received multiple bug reports about broken links in rich html emails. Sometime, in some emails, a link like <a href="https://example.com"\> would become <a href="https://example..com\>, notice the double dot. After multiple researches both in the Python email source code and in the RFC, it turns out that Python correctly implements the standard but that the distant (non-python) smtp server used by some of our customers doesn't. The various email standard state the following:
Take the following example: from email.message import EmailMessage
from email.policy import SMTP
msg = EmailMessage(policy=SMTP)
msg.set_context("Hello there, just need some text to reach that seventy-six character, example.com")
# ^
# 78th char
print(msg.as_string())
# Content-Type: text/plain; charset="utf-8"
# Content-Transfer-Encoding: quoted-printable
# MIME-Version: 1.0
#
# Hello there, just need some text to reach that seventy-six character, example=
# .com When the message is sent over smtp, smtplib escapes the line ".com" to become "..com" as required by the RFC. So no problem in the python implementation, it is the other side that is buggy. But! We have two solutions to "fix" the other side, the problem is that they do not correctly parse lines starting with a dot. A solution would be to ensure no line starts with the dot character. Two solutions : (1) quoted-printable encode dots when they are at the beginning of a line, (2) prevent the line folding code from splitting a line before a dot. (1) is allowed by the RFC, any character can be quoted-printable encoded even those that have a safe ascii representation already. In our "example=\n.com" example above, we can qp the code: "example=\n=2Ecom". The line starts with a "2" instead of a dot and the content is the same. (2) is allowed by the RFC, the RFC only states that a line must be at most 78 chars long, it also states it is allowed to fold a line anywhere but in a quoted-printable sequence. It is safe to split a line earlier than the 78th character. In our "example=\n.com" example above, we could split the line at the 77th character: "exampl=\ne.com". The line starts with a "e" instead of a dot and the content is the same. A pull request is coming shortly. |
Since python is doing the right thing here, I don't see a particularly good reason to put a hack into the stdlib to fix the failure of third party software to adhere to standards. (On the output side. We do follow Postel's rule on input and try hard to handle broken but recoverable input.) I don't actually *object* to it, though, as long as it follows the standard on output, and is a *simple* change. Please note that you can fix this locally by implementing and using a custom content manager. |
Hello David, thank you for your quick answer. I tried to keep it minimal with some unittests. Could you point me some resources to learn how to properly write a custom content manager ? |
As far as I know the only resources are the context manager docs and the source code. The stdlib content manager can serve as a model. I have to admit that it was long enough ago that I wrote that code that I'd have to re-read the docs and code myself to figure it out :) I'm afraid I don't really have time to do a complete review, but at a quick glance your patch doesn't look too complicated to me. Quick observation: the comment should explain why the dot check is done, and that it isn't needed for rfc compliance. |
Hello David, The third party smtp software that causes troubles have been identified ! We are still investigating how to fix the problem at its root, ultimately this "fix" would not even be necessary. I'll keep you informed, just don't review or close the PR yet. Regards, |
Fix deployed in the third party |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: