Automatically populate the Message-Id header.

[sh4d0v1]

Sending mails with marrow.mailer results in mails without message ids (MID) for my own mailserver and a big one I tested against. (while a third mail server adds its own message id)

The (valid) message id I try to set via message._id doesn't get sent.

Mails without a message id get very much higher spam ratings and is against the standard. I see the code referring to generating a message id if there is none set (which I assume to be default) though, so I am unsure why no message id is received.

[amcgregor]

SMTP servers in general add Message-ID if it is missing. Reference the local_header_rewrite_clients and always_add_missing_headers directives for Postfix. Notably, an ID is generated by Marrow Mailer, albeit it isn't directly used except in logging. I'll admit: that's a bug! However, an easy one to "work around":

Just set the header.

Note that the above works, correctly, because a conformant ID will be generated automatically and stored ("cached") on first access. Assignment of an explicit _id value will also work. But, notably, the _id (or id) attributes are not utilized by the SMTP transport other than for logging.

[amcgregor]

I don't suppose you could give the develop branch a go? It should be entirely compatible with your own fix (test that, too? ;) and render manual application of the Message-Id unnecessary. Building the MIME message (e.g. for SMTP delivery) will automatically add it if missing.

The reasons this is not performed in the constant declaration at the top of prepare_headers is:

• That I do wish to permit manual specification of that (don't break people's fixes for this long-standing issue).
• It is not performed much earlier to ensure the ID generation is truly lazy, waiting until the last moment (delivery) to generate if left unspecified.

(Edited to add: apologies for the commit spam there, it's been a long day. Latest run is all good: https://travis-ci.org/marrow/mailer/builds/580962733)

[sh4d0v1]

Thank you for the fix.
Are you sure this isn't ready for the master branch? Most people don't pull again without expecting breaking changes anyway (worst case here is two message id headers, so not even a breaking change). Yet in develop branch nobody will recognize it but people will continue building on the master branch and possibly keep running into this issue.

Sidenote: Most SMTP servers should add their own message id if there is not one already there, but it is not default config in postfix, which I guess most - especially the big - mail providers use.

[amcgregor]

Are you sure this isn't ready for the master branch?

There's a release process, I follow it.

…but people will continue building on the master branch…

It is generally good practice to utilize stable, published, release versions. I'm, however, asking the reporter of an issue to check to see if the issue initially reported is actually corrected by the "fix". Otherwise it's not a "fix", it's "just more dead weight", and is absolutely pointless to release, breaking or otherwise. ;P If actually fixed, I can consider rolling a release (after writing a regression test): master branch merge + actual Pypi package release.

Edited to add: and you seem to have given no indication on if you have tested it or not, successfully or not. :/

[sh4d0v1]

Sorry, I misunderstood your former comment.

I tested the develop branch and your fix works. A message id is added to the header without having to append it manually.

[amcgregor]

Awesome, thanks! :D

I'll see what else I can complete for this release, with an estimated release time for this patch (4.0.3) scheduled for Friday. There's a much larger breaking change (breaking at the packaging level itself, not just code using the package) I'll also see about finalizing for this week-end (5.0). I'm aiming for the major bump to be API compatible (no major changes there, just in packaging).

Have a great day!

Edited to add: if you depend on this Message-Id fix, I'd strongly recommend pinning your version numbers a la marrow.mongo~=4.0.3. (4.0.x where x>=3)