RFC: Automatically destroy expired emails

[cmitz]

Proposal

1: Automatically really destroy deleted emails
2: Automatically really destroy ignored (old) emails

Why

Mailgun only holds emails for 3 days, after that, the emails get destroyed. The only emails that are not automatically forwarded in our email system are emails to an alias for a (semi-)closed mail alias.

When a StoredMail is not moderated within 3 days, it expires and should not live in our database anymore.

When a StoredMail is accepted or rejected, it is soft-deleted.

The purpose of soft-deletion is recovery and debugging issues with this feature critical to our association. However, there are privacy-reasons for not keeping them forever, and after 3 days they are not relevant anymore anyway.

How

My suggestion is to execute a CleanupExpiredStoredMailsJob every day, that:

1. really_destroy!s 2-week-old soft-deleted emails
2. destroy 4 days old ignored emails

That way, we reduce the amount of irrelevant (for us) but potential privacy-compromising information in our database, but still allow a grace-period for debugging issues.

I estimate that a 2-week window before really destroying emails is enough for a member to 1) spot a potential email loss and 2) tell us about it – whereas 1 week will be too short.

Extra

I think we also need to inform a user if an email expired. The moderators would get an email like "The email with subject #{something} has expired".

[Matthijsy]

Sound likes a good plan. I would propose a delay of 30 days instead of 2 weeks. This is the same period as stored logs for mailgun. Keeping this period 2 weeks wouldn't matter for privacy since mailgun also keeps the logs 30 days so keeping the same period would be logical I think.

[tcoenraad]

I'd also like 4 weeks over 2 weeks for actual pruning.

Since soft-deletion is used after 4 days, I think that period is just fine.

[Matthijsy]

@cmitz You already implemented some things for this right? Could you already make a WIP PR out of that? Maybe someone else can help you finishing it?

[cmitz]

Yesh, on my old laptop. I'll look it up once I get back next week!

[cmitz]

@Matthijsy zie PR #73