[Feature]: Automatic duplicate mail checking (and deletion)

[eight-vrnd]

Is your feature request related to a problem?

When importing e.g. from local folders/backups (in my case, from Thunderbird) I occasionally had included mails that have already been grabbed previously from the mailbox that it was sent to before sorting into local folders or making a backup.

Describe the solution you'd like

Having an automated way of identifying duplicates based on subject+timestamp, messageID, or headers, would be greatly appreciated and save a lot of time when searching through the archive due to less duplicates listed in the results.

Additional context

This was also (kind of) mentioned as part of #319, but in this case would focus more on the identification and automation parts of this instead of the bulk edit options suggested there.

[datenheim]

I'd call this deduplication of messages (and attachments) which I consider an fundamentally required feature of mail-archives.
Question here is, what (message headers) exactly make a message unique.

For instance, if a single message had two receivers at the same organisation, so it is received by two individual mailboxes. Is this a duplicate or not? Even though most message parts are identical, there are a few headers that are not, maybe only the one header naming the receiving mailbox is different... how to solve that?

[eight-vrnd]

In my case, i'm talking about 100% identical message objects, e.g. due to importing them once directly from the mailserver via IMAP as well as again from a local backup afterwards.

If a mail is received by inboxA and inboxB, I would assume that in an archiving scenario, you would want to keep the message twice, to show that both inboxes successfully received the message.
But since this might vary depending on your use case, the better - and obviously more complex - solution would be to give the user multiple options/deduplication strategies to choose from.

[Silo-Cloud]

I would also love to see a de-duplication feature. However, it should only remove 100% identical message objects as stated bei eight-vrnd.

[huganega]

Good idea, I love deduplication!