feat: add quota-triggered per-user mailbox cleanup so get an always below-quota relay experience by hpk42 · Pull Request #927 · chatmail/relay

hpk42 · 2026-04-18T16:31:40Z

Inflate the Dovecot-visible quota to 140% of the chatmail-ini configured max_mailbox_size so that Delta Chat clients (which warn at 80% of IMAP-reported quota) never show quota warnings. A quota_warning at 72% of the inflated limit triggers chatmail-quota-expire, which trims the mailbox to 80% of the configured limit. The PR provides the following benefits:

Delta Chat users will never see a quota warning, and will not get quota-full-bounce messages from upgraded relays
Existing over-quota mailboxes start receiving mail again immediately.
quota-tracking is completely done by dovecot which invokes quota-expire, a very small and tested script
operators only need to upgrade, no manual commands

Inflate the Dovecot-visible quota to 140% of the configured max_mailbox_size so that Delta Chat clients (which warn at 80% of IMAP-reported quota) never show quota warnings. A quota_warning at 72% of the inflated limit triggers chatmail-quota-expire, which trims the mailbox to 80% of the configured limit. Existing over-quota mailboxes start receiving mail again immediately after deploy without any manual operator action needed.

DarkCat09 · 2026-04-19T04:42:09Z


+def parse_size_mb(limit):
+    """Parse a size string like ``500M`` or ``2G`` and return megabytes."""
+    value = limit.strip().upper().rstrip("B")


I suggest using .removesuffix("B") here instead of rstrip since MBB is not a valid unit name.

removesuffix was added in Python 3.9 which is already EOL so I suppose we're not trying to support old versions.

Suggested change

value = limit.strip().upper().rstrip("B")

value = limit.strip().upper().removesuffix("B")

link2xt · 2026-04-19T08:08:37Z

+  # Trigger when usage reaches the configured max_mailbox_size
+  # (72% of inflated = ~100% of configured), then expire oldest
+  # messages down to 80% of the configured max_mailbox_size.
+  quota_warning = storage=72%% quota-warning {{ config.max_mailbox_size_mb * 80 // 100 }} {{ config.mailboxes_dir }}/%u


I guess 140 (above, in quota_rule) is chosen arbitrarily to be above 100 / 0.8 = 125. And 72% is 100 / 140. But now that you have inflated actual quota to 140 * max_mailbox_size, you can set target_mb to max_mailbox_size_mb instead of 0.8 of max_mailbox_size_mb.

Currently you have both inflated the quota to the 140% of what the user configured and trim the mailbox to 80% of what the user configured. Should it be one or the other? Otherwise quota is 140 MB, but you truncate to 80 MB which is 60% of actual quota.

I think if user configured max_mailbox_size_mb, this should be the quota, not 140% of what the config says. Admin configured 1 GB, it should allow 1 GB and not 400 MB more.

link2xt · 2026-04-19T08:38:22Z

+  # Inflate the dovecot-visible quota so that Delta Chat clients
+  # (which warn at 80% of the IMAP-reported limit) never see
+  # quota warnings -- expire kicks in well before that point.
+  quota_rule = *:storage={{ config.max_mailbox_size_mb * 140 // 100 }}M


tl;dr for my comment below, i think it is wrong to configure the quota not to the value from the config, and it seems to have already resulted in a mistake

Dovecot allow the last delivered mail to go over quota already with https://doc.dovecot.org/2.3/configuration_manual/quota/#quota-grace You only need quota_grace that is a size of the largest accepted message size. Can maybe set it to multiple of max message size or so, in case the script takes time to remove messages and multiple messages arrive before the script finishes.

So there is no need to set the quota over 100%. It will also look weird in connectivity view that you set it to 100 MB and 140 MB show up in Delta Chat.

Delivered message should make the mailbox over quota, at which point it can trigger the script and bring the mailbox to 80%, it is ok to trigger only on over quota.

tl;dr for my comment below, i think it is wrong to configure the quota not to the value from the config, and it seems to have already resulted in a mistake

from the template chatmail.ini docs:

# maximum mailbox size of a chatmail address max_mailbox_size = 500M

I consider dovecot quota to be an implementation detail, and there is no promise here that this goes directly to dovecot config. Cmdeploy relays are using dovecot in a particular way, and this PR uses a threshold-activated quota_expire action, to ensure that max_mailboxsize is the limit of what is used per mailbox. In the getting-started and setup docs we basically don't mention the word "quota" at all.

I consider dovecot quota to be an implementation detail, and there is no promise here that this goes directly to dovecot config.

It goes to the connectivity view.

In the current state of the PR if you set mailbox size to 500 M, in the connectivity view user will see 700 M limit, the script gets triggered at 504 MB and brings the mailbox down to 400 MB usage.

link2xt · 2026-04-19T08:48:24Z

I'd really rather have a single line explaining where 140% comes from instead of the list of "benefits" repeating the commit changes and citing the changelog.

hpk42 · 2026-04-19T13:23:18Z

I'd really rather have a single line explaining where 140% comes from instead of the list of "benefits" repeating the commit changes and citing the changelog.

If we pass through the chatmil.ini max_mailbox_size setting to dovecot quota unmodified , then the quota_expire script will not be run for existing full or almost-full mailboxes, because the threshold was crossed in the past already. Even if we did some manual cleanup (telling operators to "run xxxx after the install") firing quota-expiry by hand, we could not use a threshold of 98% and bump it back to 85% or so, because that would generate delta chat device-message quota-warning errors for the user. Delta Chat in current releases gives quota warnings in device messages at 80% and 95%, so we need to have something lower than 80%, although it's a question it should do this once automatic expiry is widely deployed. (yes, i know, people like to know it for their classic provider inbox).

Example: max_mailbox_size is 100MB.

dovecot quota is set to 140MB
quota-expire is triggered at 72% which is ~100MB (the actual configured max size), and is well below the 80% delta chat warning
it brings down the storage to 80MB in one run so it can fill up again until about 100MB.

So 140% could be slightly lower but the actual important bit is when quota-expire kicks in and it should be the max_mailbox_size setting.

link2xt · 2026-04-19T18:09:11Z

+    """
+    mbox = Path(mailbox_dir)
+    messages = scan_mailbox_messages(mbox)
+    total_size = sum(m.size for m in messages)


This is the wrong size for quota calculation. In get_file_entry the size is st_size from stat call, just the file size. But for quota Dovecot uses the size stored in the filename:
https://doc.dovecot.org/2.3/configuration_manual/zlib_plugin/#maildir-mbox-format

We use zlib plugin here:

relay/cmdeploy/src/cmdeploy/dovecot/dovecot.conf.j2

Line 52 in 0aa08b7

mail_plugins = zlib quota

By just looking at the file sizes you underestimate quota usage, likely by more than 25% because of ASCII armoring.

link2xt · 2026-04-19T18:14:57Z

+    (mbox / "maildirsize").unlink(missing_ok=True)
+    cache = mbox / "dovecot.index.cache"
+    try:
+        cache_bytes = cache.stat().st_size


And adding to my comment about S= parameter in the filename, this should likely not be counted. I think it does not contribute to quota. For reference there is https://github.com/dovecot/core/blob/2.3.21.1/src/plugins/quota/quota-maildir.c, have not read it thoroughly but i think dovecot only counts S= for maildir, and also writes maildirsize so there is no need to sum things up manually just to check how the quota is doing.

Also need to unlink maildirsize after unlinking the files directly from the maildir, otherwise dovecot will keep adding to outdated quota and user will get over quota and a warning in Delta Chat: https://doc.dovecot.org/2.3/configuration_manual/quota/quota_maildir/

link2xt · 2026-04-19T19:03:34Z

So the goal is not to just avoid getting over quota, but avoiding the warning that triggers at 80%, and end goal is that 60% (from #927 (comment)) is below the triggering threshold and triggering threshold is below 80%.

I think it would be better to have actual quota at max_mailbox_size_mb, then set trigger to 75% or something below 80%, and then pass 0.7 * max_mailbox_size_mb. That would be straightforward and without all the magic numbers of 72%, 140% and arriving at 60% by calculation to check it is below 80%.

There is indeed a problem of already over quota users because quota_rule is only triggering when you cross the threshold, but I see no better way to make this robust than checking maildirsize for all users periodically.

Here is how maildirsize looks like:

maildirsize example

734003200S
82697847 4296
7113 1
30003 1
7155 1
50677 1
5778 1
5864 1
9889 1
9885 1
9893 1
-44044 -4
9841 1
9841 1
84649 1
33893 1
11996 1
10018 1
10018 1
52140 1
8134 1
10038 1
7980 1
64571 1
10056 1
10068 1
10068 1
9440 1
7888 1
10090 1
8053 1
8118 1
8094 1
50104 1
7058822 1
-5146 -1
5524 1
15321 1
14687 1
14695 1
21839 1
15203 1
15319 1
14612 1
14713 1
15232 1
14610 1
14652 1
14687 1
14651 1
14655 1
84948 1
14647 1
14656 1
14807 1
14569 1
14622 1
14602 1
14823 1
14659 1
14644 1
187205 1
8061 1
10030 1
10026 1
14503 1
14519 1
14673 1
57199 1
14770 1
7628 1
10081 1
39206 1
14652 1
9883 1
9891 1
8078 1
10073 1
10028 1
10028 1
10052 1
14561 1
14638 1
14600 1
8256 1
8041 1
8350 1
11991 1
11999 1
9882 1
10084 1
10191 1
9942 1
8057 1
8057 1
8374 1
9950 1
5593 1
5593 1
7562 1
29562 1
5138 1
5199 1
3475 1
3467 1
5260 1
3516 1
5292 1
3508 1
11783 1
11762 1
29562 1
-64417 -9
5581 1
8045 1
8045 1
3580 1
15140 1
9934 1
9938 1
9934 1
9942 1
6206 1
5979 1
7998 1
7982 1
6784 1
6215 1
14691 1
6653 1
3822 1
5772 1
4050 1
3802 1
6581 1

First line is the limit (734003200S is the size limit of 700 M, 700 * 2**20). The rest looks like current size and number of messages, then a line is added each time size is increased/decreased, on each line the size and message count (-9 for removing 9 messages). So the lines need to be summed up.

missytake · 2026-04-20T18:39:19Z

Replaced by #929 :)

hpk42 had a problem deploying to staging.chatmail.at/doc/relay/ April 18, 2026 16:31 — with GitHub Actions Failure

hpk42 force-pushed the add-quota-expire branch from 0ad98d1 to 4512d1e Compare April 18, 2026 21:09

hpk42 deployed to staging.chatmail.at/doc/relay/ April 18, 2026 21:09 — with GitHub Actions View deployment

DarkCat09 reviewed Apr 19, 2026

View reviewed changes

link2xt reviewed Apr 19, 2026

View reviewed changes

hpk42 mentioned this pull request Apr 20, 2026

feat: automatic oldest-first message removal from mailboxes to always stay under max_mailbox_size #929

Open

missytake closed this Apr 20, 2026

	value = limit.strip().upper().rstrip("B")
	value = limit.strip().upper().removesuffix("B")

Uh oh!

Conversation

hpk42 commented Apr 18, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

link2xt commented Apr 19, 2026

Uh oh!

hpk42 commented Apr 19, 2026

Uh oh!

link2xt Apr 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

link2xt commented Apr 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

missytake commented Apr 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

link2xt Apr 19, 2026 •

edited

Loading

link2xt commented Apr 19, 2026 •

edited

Loading