Click tracking docs does not explain plain email behavior

[brynek]

Expected content

A description that eludes the actual click tracking link substitution behavior for plain emails.

After investigating issues we had with bad activation links being sent to new users, we found that plain e-mails were the source of the problem. If link tracking is enabled (link tracking in plain e-mails is included by default), all links in plain e-mails seem to substituted (it looks like your parser is picking up everything with https:// or http:// in plain e-mails). Including a clicktracking=off in the HTML has no effect on the generated plain e-mail, and the substitution is still made.

Furthermore, there is no description of the "enable click tracking in plain emails" toggle that you have made available in the UI. This luckily served as our fix, without having to turn click tracking off entirely.

Actual content

No mention that using clicktracking=off in the HTML has no effect on click tracking in plain e-mails. Right now, the wording makes the reader believe clicktracking=off also would have an effect on the plain e-mail.
No mention of the "enable click tracking in plain emails" on/off toggle in the UI
No mention of how link substitutions are done on plain e-mails

Link to page:
https://sendgrid.com/docs/ui/analytics-and-reporting/click-tracking-html-best-practices/
https://sendgrid.com/docs/ui/account-and-settings/tracking/#click-tracking

Why did this cause an issue for us?

For Sendgrid developers:
Your link replacement process for the plain e-mails seems to html encode the links, which, when we are using multiple query string parameters leaves the redirected link invalid. For instance, the verification link https://foo.bar.com/verify?id=foo&token=bar is substituted with a sendgrid click tracking link redirecting to the html encoded https://foo.bar.com/verify?id=foo&token=bar (which resolves to the invalid https://foo.bar.com/verify?id=foo&amp%3Btoken=bar). This means that anyone who opened the plain-version of our transactional e-mails, only got invalid verification links. Of course we could implement an html decoder in our router, but this behaviour on your side seems like a bug to us. Same with the clicktracking=off clauses in the HTML - this should also lead to non-substituted links in the plain e-mail.

[Whatthefoxsays]

Hi @brynek,

Thanks so much for your feedback! We've added this to our internal docs backlog.

Twilio SendGrid Docs Team

[brynek]

@Whatthefoxsays thank you for picking this up so quickly!

Will any SendGrid developer be available to have a look at the plain e-mail link substitution issue? Although we got a workaround in place with dropping the link substitution, this seems like a rather subtle bug that might have big, negative implications for the user experience of customers of your transactional email product who are not aware of this.

[kylearoberts]

@brynek

I would recommend hitting up the support team with this issue. I attempted to replicate with my own test account and could not create your issue.

I used the following link for testing:

https://www.google.com/search?q=url+encode&oq=url+encode&aqs=chrome..69i57j0l4j69i60.2829j1j7&sourceid=chrome&ie=UTF-8

I sent it in an email that had only plain text and made sure that my plain text links would get encoded by using the Enable click tracking in plain text emails option, but could not reproduce your encoding issue. Is there a part I am missing?

[brynek]

@kylearoberts
Thank you for looking into this.

After further investigation, we realized it was the Handlebars HTML escaping that caused this (forgot to mention this was in the context of transactional templates).
An update just like the one you did to your transactional template demo recently fixed it. Easy to forget those pesky triple brackets. This specific issue is therefore resolved for us.

As a side note of that, https://sendgrid.com/docs/for-developers/sending-email/using-handlebars/#replacement-with-html could be more clear about this in our opinion, as multiple query parameters are rather common among password reset/verification e-mails etc. I'll open another issue for the docs team on this.

As the link substitutions worked in the HTML version while this issue was happening, it seems you are performing some sort of HTML decode on links in the HTML version, which are not performed on the plain version. Can you confirm this? As the HTML version worked we were unaware of this issue for quite a long time.

I have added an example of one HTML-based and one plain click-tracking link, picked up from the same internal test e-mail and the same verification link (merged in by Handlebars using only double brackets). As you see, the link coming from the HTML version has been decoded somewhere along the way, which previously lead us to believe this worked without the triple brackets in Handlebars. The utm parameters are supplied by Sendgrid afterward, so they work well in both versions. In e-mail clients, browsers will, of course, decode the & as part of their HTML rendering process. However, this resolving is not being done as part of the redirection process from click tracking links, which lead to the error we experienced with plain e-mails.
ClickTrackCheck.txt

[brynek]

Similar issue:
#5261