Occasionally getting "550-5.7.1 Not RFC 5322 Compliant" errors with multiple recipients

[jazzboME]

Our institution uses Google Workspace, and I send emails through a local smarthost that gets routed through Google's SMTP servers and for the last month or so I've started to get occasional downstream bounces from the google server with an error like:

----- Transcript of session follows -----
... while talking to [aspmx.l.google.com](http://aspmx.l.google.com/).:
>>> DATA
<<< 550-5.7.1 [2610:48:100:820::3d] Our system has detected that this message is not
<<< 550-5.7.1 RFC 5322 compliant: duplicate headers. To reduce the amount of spam
<<< 550-5.7.1 sent to Gmail, this message has been blocked. Please review
<<< 550 5.7.1  RFC 5322 specifications for more information. fw4-20020a05622a4a8400b00342f8191b5asi4790446qtb.213 - gsmtp
554 5.0.0 Service unavailable

Strangely it doesn't happen all the time, and sometimes I can just resend and it works (?!!) but I've been able to effectively eliminate it by changing how the To: headers are generated. I can't tell if some Google servers in the pool are "pickier" or after a certain number of emails, it gets flagged. The standard in 3.6.3 of RTF 5322 states:

   The destination fields of a message consist of three possible fields,
   each of the same form: the field name, which is either "To", "Cc", or
   "Bcc", followed by a comma-separated list of one or more addresses
   (either mailbox or group syntax).

and that is probably what we're hitting here: that they need to be comma separated and not multiple headers, even if (most?) SMTP servers don't care.

So right now I've made the following code change on my local fork and the error has disappeared for me for the last couple weeks:

mine.go

       for _, to := range m.toAddrs {
		fmt.Fprintf(w, "To: %s\r\n", to)
	}

to

        toHeader := strings.Join(m.toAddrs, ",")
	fmt.Fprintf(w, "To: %s\r\n", toHeader)

However, this is a bit of a quick kluge in that a) this may produce lines that exceed the 998 limit specified in RFC-5322 2.1.1, and b) I'm thinking we may have to do the same for CC and BCC (and Reply-To),

Anyway thoughts on the best way to approach this?

[domodwyer]

Well that is very interesting! Nice find.

I implemented this as is based on the protocol examples in RFC 5321, but it looks like RFC 5322 supersedes it in several areas. I've also just tested sending to multiple addresses from Gmail and it does indeed separate them with commas.

As for the max line length I wouldn't be surprised if that is still technically the mandated limit, but implementations are now capable of handling longer lines without the spec being updated. I sent a test email with a To containing ~2500 chars from Gmail and it indeed placed them all on a single line without issue.

Email Headers

MIME-Version: 1.0
Date: Wed, 17 Aug 2022 10:58:11 +0200
Message-ID: <CAFNjMy=ksT_iFiJ+h7OkowHYNvbArdavAVAN_CCpQhiqVriK3w@mail.gmail.com>
Subject: 
From: Dom Dwyer <dom@itsallbroken.com>
To: dom+1@itsallbroken.com, dom+2@itsallbroken.com, dom+3@itsallbroken.com, dom+4@itsallbroken.com, dom+5@itsallbroken.com, dom+6@itsallbroken.com, dom+7@itsallbroken.com, dom+8@itsallbroken.com, dom+9@itsallbroken.com, dom+10@itsallbroken.com, dom+11@itsallbroken.com, dom+12@itsallbroken.com, dom+13@itsallbroken.com, dom+14@itsallbroken.com, dom+15@itsallbroken.com, dom+16@itsallbroken.com, dom+17@itsallbroken.com, dom+18@itsallbroken.com, dom+19@itsallbroken.com, dom+20@itsallbroken.com, dom+21@itsallbroken.com, dom+22@itsallbroken.com, dom+23@itsallbroken.com, dom+24@itsallbroken.com, dom+25@itsallbroken.com, dom+26@itsallbroken.com, dom+27@itsallbroken.com, dom+28@itsallbroken.com, dom+29@itsallbroken.com, dom+30@itsallbroken.com, dom+31@itsallbroken.com, dom+32@itsallbroken.com, dom+33@itsallbroken.com, dom+34@itsallbroken.com, dom+35@itsallbroken.com, dom+36@itsallbroken.com, dom+37@itsallbroken.com, dom+38@itsallbroken.com, dom+39@itsallbroken.com, dom+40@itsallbroken.com, dom+41@itsallbroken.com, dom+42@itsallbroken.com, dom+43@itsallbroken.com, dom+44@itsallbroken.com, dom+45@itsallbroken.com, dom+46@itsallbroken.com, dom+47@itsallbroken.com, dom+48@itsallbroken.com, dom+49@itsallbroken.com, dom+50@itsallbroken.com, dom+51@itsallbroken.com, dom+52@itsallbroken.com, dom+53@itsallbroken.com, dom+54@itsallbroken.com, dom+55@itsallbroken.com, dom+56@itsallbroken.com, dom+57@itsallbroken.com, dom+58@itsallbroken.com, dom+59@itsallbroken.com, dom+60@itsallbroken.com, dom+61@itsallbroken.com, dom+62@itsallbroken.com, dom+63@itsallbroken.com, dom+64@itsallbroken.com, dom+65@itsallbroken.com, dom+66@itsallbroken.com, dom+67@itsallbroken.com, dom+68@itsallbroken.com, dom+69@itsallbroken.com, dom+70@itsallbroken.com, dom+71@itsallbroken.com, dom+72@itsallbroken.com, dom+73@itsallbroken.com, dom+74@itsallbroken.com, dom+75@itsallbroken.com, dom+76@itsallbroken.com, dom+77@itsallbroken.com, dom+78@itsallbroken.com, dom+79@itsallbroken.com, dom+80@itsallbroken.com, dom+81@itsallbroken.com, dom+82@itsallbroken.com, dom+83@itsallbroken.com, dom+84@itsallbroken.com, dom+85@itsallbroken.com, dom+86@itsallbroken.com, dom+87@itsallbroken.com, dom+88@itsallbroken.com, dom+89@itsallbroken.com, dom+90@itsallbroken.com, dom+91@itsallbroken.com, dom+92@itsallbroken.com, dom+93@itsallbroken.com, dom+94@itsallbroken.com, dom+95@itsallbroken.com, dom+96@itsallbroken.com, dom+97@itsallbroken.com, dom+98@itsallbroken.com, dom+99@itsallbroken.com, dom+100@itsallbroken.com
Content-Type: multipart/alternative; boundary="00000000000087aeec05e66c1049"

--00000000000087aeec05e66c1049
Content-Type: text/plain; charset="UTF-8"

TEST LONG LINES!

--00000000000087aeec05e66c1049
Content-Type: text/html; charset="UTF-8"

<div dir="ltr">TEST LONG LINES!<br></div>

--00000000000087aeec05e66c1049--

It sounds like you were premature in calling your fix a kludge! Based on the above I would love to merge the changes - are you happy to open a PR? We should probably give Cc and Bcc the same treatment as you suggest.

Thanks for the fantastic write-up - exceptional!

[pequalsnp]

We use Mailtrap to test emails and we noticed this in a different way. For multiple to recipients, Mailtrap was registering all but one of them as a BCC address. This is because it seems to be taking the last value of the To: header and ignoring the others. Addresses appearing in the RCPT list but not in the headers are treated as BCC in most cases.

[jazzboME]

Thanks for opening the pull request! -- it was going to take me a little while to get to it.

[pequalsnp]

There is also workaround for this which is to join the email addresses before passing them to mailyak:

var mail *mailyak.Mail
mail.To(strings.Join(toEmails[:], ","))

Spoke too soon, this doesn't work, it fixes the DATA headers but barfs on RCPT.

[domodwyer]

Thanks @pequalsnp and @jazzboME - top work!

I shall cut a release now 👍