Tuning the junk filter

[martinjanda]

Can I tweak the Spam filter by changing the settings? What are the options? How can I improve my spam protection settings?

In the last month, I have had several instances where the same spam (same text from different addresses) has come repeatedly to users' inboxes, even though it has been moved to the Junk folder.

[mjl-]

Hi @martinjanda! You can tweak some settings. They are per-account. See https://www.xmox.nl/config/#cfg-domains-conf-Accounts-x-JunkFilter for the config options. Accounts can change these through the account web interface, see screenshot. There is a caveat: I don't think the junk filter is rebuilt automatically when you change between onegram/twogram/threegram. There is a admin cli tool to rebuild the junk filter, though it make take some time on large mailboxes.

It will be interesting to understand why marking the messages as junk did not block new copies/similar messages. Could you look at the messages that came through, at the X-Mox-Reason header? The header tries to explain why the message was allowed in, including the scores of the words used during bayesian classification. Perhaps it gives a hint about what's wrong. Btw, you can add the header to "Show additional headers" in the Settings popup (button in top-right corner) in the webmail. I have that (and a few more headers) configured to see why messages are making it in.

[martinjanda]

Thank you for your quick response and advice. I'll try.

OK, here is example from my mailbox.

First e-mail:

X-Mox-Reason: no-bad-signals; not using any dmarc result;
	address/dkim/spf/ip-based reputation (inconclusive, no address/spf/dkim/ip
	 reputation);
	suspicious iprev failure;
	content: not junk, spamscore 0.00, threshold 0.25 (stricter due to reverse
	 ip mismatch)(ham words: (non-ascii) 0.010, (non-ascii) 0.010, To:smartus
	 0.010, (non-ascii) 0.010, (non-ascii) 0.010, (non-ascii) 0.010, (non-ascii)
	 0.010, (non-ascii) 0.010, (non-ascii) 0.010, (non-ascii) 0.010), (spam
	 words: );
	no known reputation and no bad signals;
From: <ac6d838@autothijs.nl>
Subject: =?utf-8?B?Vnlzb2NlIHBsYWNlbsOhIHByw6FjZSBuYSBkw6Fsa3UuIFByYWNvdm7DrWNpIG51dG7DqSE=?=
Date: 6 Mar 2025 19:03:42 +0100

I Moved to Junk folder. Then the same email came twice more:

X-Mox-Reason: no-bad-signals; not using any dmarc result;
	address/dkim/spf/ip-based reputation (inconclusive, no address/spf/dkim/ip
	 reputation);
	suspicious iprev failure;
	content: not junk, spamscore 0.00, threshold 0.25 (stricter due to reverse
	 ip mismatch)(ham words: (non-ascii) 0.010, (non-ascii) 0.010, To:smartus
	 0.010, (non-ascii) 0.010, From:info 0.010, (non-ascii) 0.010, (non-ascii)
	 0.010, (non-ascii) 0.010, (non-ascii) 0.010, (non-ascii) 0.010), (spam
	 words: );
	no known reputation and no bad signals;
From: <info@fairy-tailor.nl>
Subject: =?utf-8?B?Vnlzb2NlIHBsYWNlbsOhIHByw6FjZSBuYSBkw6Fsa3UuIFByYWNvdm7DrWNpIG51dG7DqSE=?=
Date: 6 Mar 2025 10:19:30 -0400

X-Mox-Reason: no-bad-signals; not using any dmarc result;
	address/dkim/spf/ip-based reputation (inconclusive, no address/spf/dkim/ip
	 reputation);
	suspicious iprev failure;
	content: not junk, spamscore 0.00, threshold 0.25 (stricter due to reverse
	 ip mismatch)(ham words: (non-ascii) 0.010, (non-ascii) 0.010, To:smartus
	 0.010, (non-ascii) 0.010, (non-ascii) 0.010, (non-ascii) 0.010, (non-ascii)
	 0.010, (non-ascii) 0.010, (non-ascii) 0.010, (non-ascii) 0.010), (spam
	 words: );
	no known reputation and no bad signals;
Date: 6 Mar 2025 11:48:28 -0400
From: <organisatie@hetsportweekend.nl>
Subject: =?utf-8?B?Vnlzb2NlIHBsYWNlbsOhIHByw6FjZSBuYSBkw6Fsa3UuIFByYWNvdm7DrWNpIG51dG7DqSE=?=

[mjl-]

It looks like the junk filter recognizes words from earlier messages, but doesn't see them as junk (spam), but as not-junk (ham). So mox thinks the messages are very legit. We'll have to investigate how that happened.

Btw, the full words don't make it into the x-mox-reason header due to them being non-ascii and the message not already being in smtputf8 mode (mox doesn't want to turn a message into smtputf8 just for its own added x-mox-reason header, but it does limit usefulness for non-ascii scripts, this may need a rethink...).

In the last month, I have had several instances where the same spam (same text from different addresses) has come repeatedly to users' inboxes, even though it has been moved to the Junk folder.

Can you check if these messages have the "$Junk" flag set? And don't have the "$NotJunk" flag? You can see this in the webmail: A message with $Junk shows a "j" in the message list (first few characters on the message item are the flags). A "?" means there is no $Junk and no $NotJunk flag. Neither "j" nor "?" means the message has the $NotJunk flag.

Are you moving the messages to junk with the mox webmail with the "Junk" button or the "q" shortcut? If so, that should certainly move it to the right folder. You also need to have the "Automatic junk flags" feature enabled (which it is for new accounts by default), to automatically set the $Junk flag when moving a message to the junk folder. See https://www.xmox.nl/config/#cfg-domains-conf-Accounts-x-AutomaticJunkFlags. The defaults for these settings for new accounts are:

        account.AutomaticJunkFlags.Enabled = true
        account.AutomaticJunkFlags.JunkMailboxRegexp = "^(junk|spam)"
        account.AutomaticJunkFlags.NeutralMailboxRegexp = "^(inbox|neutral|postmaster|dmarc|tlsrpt|rejects)"

The value for NotJunkMailboxRegexp is empty by default, and that means moving a message to any other mailbox marks it as $NotJunk. That includes the Archive folder, and also the Trash folder (if users move junk messages not to Junk but to Trash, they should include "trash" in the NeutralMailboxRegepx). If you've changed the name of your junk folder, moving messages to it could also be matched by this default rule and marked as not-junk.

I looked at the code just now, and it seems like moving a message to the mailbox with the special-use flag "Junk" does not automatically mark the message as junk. We may have to rethink that, perhaps that should override the AutomaticJunkFlags setting.

Btw, other mail clients often don't help setting the $Junk and $NotJunk flags. Some mail clients use non-standard names for junk flags. I believe Thunderbird uses just "junk" and "notjunk", and those are not used by mox. Most mail clients don't help you mark messages as $NotJunk at all, presumably because they never experienced the need. I think it's not so common for other mail server software to use those flags for classifying incoming messages.

[martinjanda]

Thanks for the explanation.

I use Drag&drop in Thunderbird to move to the Junk folder. My users use all sorts of things: Outlook, Apple Mail, etc.

Emails in the Junk folder are now marked as "J". But I used the mox retrain command. I don't know what the situation was like before.

For all mailboxes, AutomaticJunkFlags is set to true.

I am wondering how to improve the filter learning.

So is it better to periodically run mox retrain manually? Or is it unnecessary?

Is the learning of spam specific for each mailbox separately? Isn't it possible, for example, to create a spam at domain.com email where users could forward spam so that the filter has something to learn from?

Would learning the spam filter help if I copied a spam folder from another email server via imap?

Thanks for your patience

[mjl-]

Emails in the Junk folder are now marked as "J". But I used the mox retrain command. I don't know what the situation was like before.

The "j" is good. Mox retrain doesn't change flags, it only builds up the junk filter database based on the existing message flags. So running mox retrain should only ever help.

So is it better to periodically run mox retrain manually? Or is it unnecessary?

No, you shouldn't have to run it manually normally. Whenever a message flag is changed (over IMAP or the webmail), the junk filter is automatically updated. Updating the junk filter is essentially: splitting the message up in words, and for each word increasing either the "junk" or "nonjunk" counter (or decreasing it for untraining a message).

When you change between 1/2/3-grams, mox doesn't retrain yet, but it should. The other junk filter options are only used for classifying new incoming messages (where the n-gram setting is also relevant when training, when message flags change). Need to make this more clear, making a note.

Is the learning of spam specific for each mailbox separately? Isn't it possible, for example, to create a spam at domain.com email where users could forward spam so that the filter has something to learn from?

The bayesian junk filtering works per account. "Mailbox" means different things to different people, so we have to be careful to understand each other correctly. (: For me it means a folder in an account where mail is delivered, for some it means an address or localpart (username) of an address. In mox, messages are stored in an account, and an account can have multiple addresses. The bayesian junk filter is essentially a database with words where each word has a "occurrences in a spam message" an "occurrences in a ham message" count. So the database is per account. Any junk/not-junk training updates those word counts. This is intentionally per account: What one person thinks is spam, isn't necessarily the same as what someone else things is spam. And if you allow one account (person) to influence the filtering of another account (person), it could lead it problems where one account can cause legitimate mails for an other account to be blocked.
The bayesian junk filter is only used for messages without enough address/domain/spf/dkim/ip address reputation. The idea is that most good messages will be handled by the known-reputation approach. And that the user will initially get some junk messages delivered to their inbox (due to lack of classifications), but that marking messages as junk for an initial period should relatively quickly build up the junk filter database and start rejecting spam.

Would learning the spam filter help if I copied a spam folder from another email server via imap?

Yes! This would certainly help. As long as the messages get properly marked as junk. Likewise, importing non-junk messages (eg Archive) helps for positive classification.

It's still unclear why the words from the messages you marked as junk were seen as "ham" during classification. I think we have to understand how that happened. If spam messages accidentally get marked as ham, adding more messages will not improve the junk filtering. (:

You could inspect the junk filter database for your account, it's at $moxdir/data/accounts/$accountname/junkfilter.db. I recommend making a backup with mox backup tmp/backup and accessing tmp/backup/data/accounts/msg/mox/junkfilter.db, because a db file can be open by one process at a time. It is a bstore database file and you can use the bstore command to export the "wordscore" table from it. Example:

mox backup tmp/backup
cd tmp/backup/data/accounts/mox
GOBIN=$PWD go install github.com/mjl-/bstore/cmd/bstore@latest # compile bstore command
# alternatively, download a binary from https://beta.gobuilds.org/github.com/mjl-/bstore@latest/cmd/bstore/linux-amd64-latest/
./bstore exportcsv junkfilter.db wordscore >wordscore.csv

You'll get a CSV with 3 headers: Word,Ham,Spam. The first entry is for "-", it counts the number of messages marked as spam and ham. The other entries are for actual words. The CSV should show if messages/words and their ham/spam signals have been accounted for properly.

Btw, do messages also get rejected properly? You can see those messages in the Rejects folder. For reference, I recently checked, and I get around 30 rejects per day (not counting retries or duplicates). And 1 spam gets through to my inbox every two days (so 0.5 spam messages per day). I can't remember the last time a ham message was misclassified as spam.

[mwyvr]

Btw, other mail clients often don't help setting the $Junk and $NotJunk flags. Some mail clients use non-standard names for junk flags.

I have wondered about that. I use Evolution on PCs and the Android client on my phone.

On Evolution, I drag messages to Junk from Rejects (or Inbox if they aren't detected by Mox). On Android I use the Report as Spam feature simply because it is easier than moving them specifically to the folder as needed.

If how we move mail to Spam (or out) matters (using client features or dragging manually), a note in docs would be good.

[mjl-]

If how we move mail to Spam (or out) matters (using client features or dragging manually), a note in docs would be good.

Agreed. The account web interface has the header "Automatic junk flags", and hovering the underdotted text shows hints (which are normally the same as in the config file), but I don't think it's elaborate enough. Though we also shouldn't have a huge wall of text by default. Perhaps details in a popup or something. I think it's about time to make the whole accounts page a bit less dense...

Current places where the behaviour is mentioned:

• https://www.xmox.nl/features/#hdr-junk-filtering
• https://www.xmox.nl/config/#cfg-domains-conf-Accounts-x-AutomaticJunkFlags

[martinjanda]

I've been monitoring spam on 3 mailboxes since then. Since then, spam classification of similar emails has been working fine. I haven't encountered the problem again. Thank you