[$300 bounty] Add support for rfc 8058 (One-Click unsubscribe)

[bigon]

Mautic Version

5.0.x series

PHP version

8

What browsers are you seeing the problem on?

No response

What happened?

Would be great if the rfc 8058 (One-Click unsubscribe) was implemented

Isn't that a matter of just adding List-Unsubscribe-Post: List-Unsubscribe=One-Click in the headers?

This seems to be part of the gmail guidelines: https://support.google.com/mail/answer/81126?hl=en that will become a requirement in February 2024

How can we reproduce this issue?

Send a mail to gmail, there is no unsubscribe link on the top next to the sender name (see: https://www.badsender.com/en/2021/04/26/all-knowing-or-almost-all-knowing-about-the-list-unsubscribe/)

Relevant log output

No response

Code of Conduct

• I confirm that I have read and agree to follow this project's Code of Conduct

[bigon]

If I understand correctly rfc 8058, the unsubscription process also need to check what is sent in the POST and only unsubscribe the person if it matches List-Unsubscribe=One-Click?

[MarkerB]

I'm so supportive of this ticket and any effort.

Google's changes are coming very soon, and then Yahoo/AOL/AT&T/etc will be implementing it after that. It seems like this will cause major spam rejections when Mautic sends out over 5000 emails in one shot. I was starting to roll my own solution, because I need to also integrated one-click unsubscribes into my site's transactional emails. However, Mautic needs to get this together fast anyway.

I could work on these changes, but I'm having difficulty understanding how unsubscribes are currently handled in the code. I've frankly never been good at interpreting other people's programming, especially when there are no comments.

[mautibot]

This issue has been mentioned on Mautic Forums. There might be relevant details there:

https://forum.mautic.org/t/changes-coming-to-gmail/29961/2

[RCheesley]

Hi @MarkerB if you'd like to pair up with @mabumusa1 I'm sure we can get you acquainted with how the unsubscribe works and get a PR for this important feature ready to go!

[mautibot]

This issue has been mentioned on Mautic Forums. There might be relevant details there:

https://forum.mautic.org/t/changes-coming-to-gmail/29961/6

[MarkerB]

Hi @MarkerB if you'd like to pair up with @mabumusa1 I'm sure we can get you acquainted with how the unsubscribe works and get a PR for this important feature ready to go!

Ideally. I'm just starting to code my own transactional mailing system to meet the RFC-8058 spec. I need to first wrap my head around how the spec works in practicality.

I'm also a little nervous about taking this on. My workload is insane, and I have almost zero experience working collaboratively in a CVS. If @mabumusa1 spearheaded the actual integration, I would happily be an adjunct.

[mabumusa1]

@MarkerB nervous is the name of the game, we can defuse this and get you started with Mautic. For unsubscribes they are done like this:

1- If the email is transactional there will be no unsubscribe header added.
2- If the email is a marketing email, we add 'List-Unsubscribe header in this file

3- For your implementation I think a good starting point is to edit MailerHelper.php by adding the new header there.

If you have troubles getting started, please ping me on slack

[MarkerB]

@MarkerB nervous is the name of the game, we can defuse this and get you started with Mautic.

You may have misunderstood. I am not using Mautic for transactional emails. Due to the volume of my transactional emails, I need to implement RFC-8058 in my own application's transactional notification system. I'm going to do that first, in an environment I already understand, before trying to figure out how to integrate into Mautic.

Mautic has precious few comments in its code, which I just don't understand.

[RCheesley]

I've tagged this as something we'd want to release in the 5.1 milestone, which should be in February 2024 with the current timelines.

Copying this to @mautic/core-team as multiple vendors are going to need this to be incorporated. Perhaps we can see if we can work collaboratively to get this feature delivered.

[MarkerB]

This 5.1 timeline is troubling for this issue, as February 2024 is also when Gmail's new requirements are supposed to be implemented. After that, those of us sending over 5,000 emails a day might just see massive spam bounces because of noncompliance to RFC-8058. Furthermore, I've heard a case made that one-click subscribes need to be warming up well in advance of Google's deadline.

We also need to consider that many ESPs do not support HTTP webhooks for unsubscribing, but only email-based ones. Mautic supports email-based Unsubscribe Requests in some form, but I'm not sure that system is fully compatible with the List-Unsubscribe mechanism. See this discussion, which sounds hardly conclusive. I think it would be better for Mautic to be RFC-8058 compliant without relying on fumbling around with the generalized custom headers feature.

Due to the timeline, I'm sorry that I can only act as a consultant on this. I'm learning things about how this works that were not obvious before I started trying, and I'd be happy to be involved in further discussions.

My priority is to rush this into production on my own Mautic v4 installation, and it's going to be a big hack because I need to tightly integrate Mautic unsubscribes into my application's transactional unsubscribe system. Mautic doesn't seem to have any hooks for allowing its unsubscribes to be managed by an outside application. Since my application is only using Mautic for marketing mailings, Mautic's native subscription interface is inadequate.

I feel like we all waited much too long to deal with this. I mistook a lack of forum activity about these coming Google/Yahoo requirements to mean Mautic already supported them. That was very silly of me!

[atowne-mautic]

Technically I think Mautic can support this already - but it would mean not using a preference page, which isn't ideal.

There are a couple options I see that could be done.

• Add the list-unsubscribe header for marketing emails. This one is likely going to be easiest and fastest to get done before February - and is probably a good idea even with the second option.
• Support a different token for preference pages, rather than just having a preference page for the {unsubscribe_text} or {unsubscribe_url} if someone turns preferences on for all emails. Using a different token like {preferences} to link to a selected preference page would enable users to both have a preference page AND a one-click unsubscribe.

[MarkerB]

I appreciate you trying to find a stop-gap solution, given the time constraints and the fact that v5 isn't even in stable release yet.

Technically I think Mautic can support this already - but it would mean not using a preference page, which isn't ideal.

There are a couple options I see that could be done.

• Add the list-unsubscribe header for marketing emails. This one is likely going to be easiest and fastest to get done before February - and is probably a good idea even with the second option.

Keep in mind it's not just a "List-Unsubscribe" header needed, which Mautic already includes. We also need the "List-Unsubscribe-Post: one-click" header that is central to RFC-8058.

• Support a different token for preference pages, rather than just having a preference page for the {unsubscribe_text} or {unsubscribe_url} if someone turns preferences on for all emails. Using a different token like {preferences} to link to a selected preference page would enable users to both have a preference page AND a one-click unsubscribe.

Just to clarify...

Mautic currently doesn't pay attention to whether the unsubscribe request is coming via a POST or not, and doesn't inspect the headers coming that would indicate the request is a one-click unsubscribe.

I think this is why you mentioned that becoming compliant with one-click would preclude the availability of a preference page. That is, Mautic can currently either honor one-click unsubscribes, or provide a more complex preferences user interface.

I imagine a bigger issue is that currently, any unsubscribe links in the body of an email would have to visit the same unsubscribe page as the one-click. That means anti-virus bots could accidentally unsubscribe users behind the scenes.

Since CAN-SPAM requires unsubscribe links in the message body, I don't see how Mautic technically could be made to support RFC-8058 without at least some code changes.

[RCheesley]

I'd love to get it in sooner, but we've missed the boat for Mautic 5 which is no longer accepting features or enhancements as we move towards General Availability. The next feature release is 5.1. So that's the soonest we will be able to release any features. The branch is already there and we're already merging PRs, so work can begin right away.

[github-actions]

This issue has a bounty of $300 associated with it. Read the docs on how to work on and claim the funds here.

[RCheesley]

We have added a $300 bounty from the Product Team's fund - this issue needs to be completed, reviewed and merged in time for the 5.1 release at the very latest. Developers should target the 5.1 branch - please read the documentation on how to work on the issue here.

[RCheesley]

@patrykgruszka has started working on this feature and will be sharing a PR imminently which will address this issue. If we can get this tested ASAP it might be possible to squeeze it into the upcoming 5.0 release given the timings we require to address the changes in time for February. @bigon @MarkerB hope we can count on you to help with testing/reviewing! 🚀

[MarkerB]

@bigon @MarkerB hope we can count on you to help with testing/reviewing! 🚀

Sure. Ask me any questions, and I'll also test any running instance you have. I've already implemented RFC-8058 as a slipshod Mautic 4 hack. It wouldn't be generally useful, as it only works using custom unsubscribe links from my internal transactional system.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If this issue is continuing with the lastest stable version of Mautic, please open a new issue that references this one.

[mautibot]

This issue has been mentioned on Mautic Forums. There might be relevant details there:

https://forum.mautic.org/t/list-unsubscribe-post-support/30809/1