Feature Message body cache

English · Tiếng Việt · 中文

Message body cache

SkimMail keeps the full body of a message on disk the first time you open it, so the second open does not wait on IMAP. Since 1.11.0 that cache is encrypted at rest by default, has a size ceiling and an age limit, and is finally deleted when the mail it belongs to is deleted. In 1.11.0 you configure it with three environment variables — the Settings panel for it is built, but not visible in this release.

What is cached, and when

• The parsed body (HTML and text) of a message, written the first time that message is opened, and by the background prefetch described below.
• It lives in the blob store: a directory under DATA_DIR by default, or your S3 bucket if you switched the storage engine.
• Subjects, senders, dates, flags, snippets and the search index live in the database, not here, and are not affected by anything on this page.
• Attachments are not cached. They are fetched from the mail server every time you open one.

The cache exists because a miss is expensive in a way no storage backend can fix: it is a synchronous IMAP round-trip pulling a whole message inside an HTTP request, roughly a thousand times the cost of reading the body back from disk.

Configuring it

The cache is always on. Three environment variables control it:

Variable	What it does	Default
CACHE_MAX_SIZE_MB	Evict cached bodies above this total; 0 = no ceiling	2048
BODY_TTL_DAYS	Drop cached bodies older than this; 0 = never expire	90
BLOB_ENCRYPT	Encrypt newly written bodies at rest	true

Two of them behave differently from the third, and it matters:

• CACHE_MAX_SIZE_MB and BODY_TTL_DAYS are Plane 2 settings: the environment provides the initial default, and a value saved from the screen — or through PUT /api/settings/cache — is stored in the database and wins from then on. Since 1.11.1 that screen is Settings ▸ Security ▸ Message body cache. On 1.11.0 the same panel shipped inside the hidden Storage tab, so the environment really was the whole story there.
• BLOB_ENCRYPT is boot-only. No UI, no API, no skimmail config; it is read from the environment at startup and nowhere else. It is on unless you explicitly turn it off: only false, 0, no and off disable it (case-insensitive, surrounding spaces ignored). Every other value — 1, TRUE, yes, on, or a typo — leaves encryption on, because a flag that defaults to on has to fail in the safe direction.

1.11.0 got that last rule backwards. It compared the value against the exact word true, so BLOB_ENCRYPT=1, =yes, =on and =TRUE all read as off — encryption stopped at the exact moment an operator believed they had just confirmed it. Fixed in 1.11.1. Bodies cached while it was off are not rewritten in place; they stay unencrypted until they leave the cache. Lowering the size or age limit in Settings ▸ Security evicts them, and they come back encrypted.

See Configuration for what "Plane 2" means and where these sit in the full environment reference.

The settings screen

Settings ▸ Security ▸ Message body cache, from 1.11.1. It shows four things and offers two actions:

Row	What it means
Cached bodies	how many are stored, and how many bytes that is
Awaiting deletion	bodies already condemned but not yet removed from the store; only shown when there are some
Background prefetch	on/off, instance-wide. Since 1.16.0 — see "Prefetching new mail" below
Size limit (MB)	0 = no ceiling. Above it, the least recently read go first
Keep for (days)	0 = never expire

Save applies the new limits immediately, so the usage figures move as soon as it returns. Purge unaccounted is the S3 cleanup described further down.

Every route behind this panel is owner-only, including the read — so for an operator or a viewer the panel simply does not appear, rather than appearing and refusing. That is deliberate: a control you would only be refused is worse than no control.

Encryption at rest

With BLOB_ENCRYPT=true (the default), every newly written body is sealed with AES-256-GCM. The key is derived from your master key, so:

• There is nothing new to back up, lose or rotate. No second secret exists. Losing the master key already leaves the database unreadable, so this adds no way to lose data that you did not already have.
• Upgrading needs no migration and no downtime. Each blob says whether it is sealed, so a store can hold bodies written by 1.10.0 and bodies written by 1.11.0 at the same time. Reads handle both.
• Turning encryption off does not strand the existing cache. Reads always decrypt; the switch only decides how new bodies are written.
• If the key is ever wrong, a body reads as a cache miss and is fetched again from the mail server. The product degrades into being slow, never into losing mail.

What this does not cover: the database. Subjects, addresses, snippets and the search index are plaintext on disk. For those, encrypt the volume — see Security.

A backup archive never carries this key. Bodies go into the archive decrypted (the archive has its own encryption) and are re-sealed with the target instance's own key on restore, so an archive stays restorable on a machine that has never seen this instance's master key.

The size ceiling and the age limit

• When the cache is over CACHE_MAX_SIZE_MB, the least recently read bodies go first, down to 90% of the ceiling rather than exactly to it — the cache should not spend its life one message away from full.
• BODY_TTL_DAYS is measured from when the body was cached, not from when it was last read: a body read every day is exactly as stale as one nobody has opened.
• The age limit is not only housekeeping. A mail server that resets UIDVALIDITY can hand the same UID to a different message, and the cache is keyed by account, mailbox and UID — so without an age limit a stale body could be shown under a new message indefinitely. The TTL bounds that to N days.
• There is no background timer. A pass runs when roughly 64 MiB of new bodies have accumulated, and immediately when new limits are saved through the API. Deletion is batched (500 blobs per pass, plus 50 on each sync poll), so a pass never stalls somebody's request; whatever is left over is picked up by the next one.
• Eviction costs you a re-fetch and nothing else. The cache never holds the only copy of your mail.

Deleting an account now deletes its bodies

Before 1.11.0, deleting an account cleared its mail from the database and left every cached body on disk for ever — invisible, unrecoverable, and still counted against the Community storage limit. As of 1.11.0 those bytes are removed with the account. Moving a message to another mailbox retires the old body too, since the destination gives it a new UID.

With the filesystem blob store, cached bodies live under DATA_DIR and therefore count toward your tier's storage limit. With the S3 engine they live in the bucket and do not.

The one-time cleanup on first start of 1.11.0

An instance upgrading to 1.11.0 has a store full of bodies and no index of them. On the first start, SkimMail derives which blobs should exist from your messages, puts those on the books, and treats the rest as unaccounted. It runs once, in the background, so it never delays the listening port, and it logs:

reconciled body cache store=fs indexed=1843 bytes=284127744 extra=12 deleted=12 needs_purge=false

The two storage engines are treated differently on purpose:

• Filesystem — the blob directory is SkimMail's alone, so unaccounted blobs really are orphans. They are deleted.
• S3 — the bucket may also be your backup destination. Unaccounted objects are counted and left alone; the log line says needs_purge=true. Nothing is deleted, because deleting the wrong object there could destroy the only copy of your mail.

On S3 the gap also closes by itself: a body gets indexed the next time it is read, so an instance in normal use converges without touching the bucket.

If you are certain the bucket holds nothing but SkimMail's cache, there is a manual purge: Settings ▸ Security ▸ Message body cache ▸ Purge unaccounted. It reports how many stored objects it deleted.

The same thing over the API, for a script:

curl -sS -b /tmp/skim.cookies -X POST http://localhost:8080/api/settings/cache/purge

Both are owner-only. See Remote images in mail for how to obtain the session cookie.

Prefetching new mail

After a sync brings new mail, SkimMail warms up to 10 of the newest bodies in the background, so the first click is instant instead of waiting on IMAP.

Three refusals matter more than the feature itself:

• Only when somebody is connected. With no browser open there is no first open to make faster, and on a metered connection the trade is just a bill.
• Never for an account whose sync has been auto-stopped. Speculative work is the last thing a struggling mailbox needs.
• Never counted as a sync failure. A prefetch that fails is dropped silently, because letting optional background work drive the consecutive-failure counter could auto-stop a mailbox that works perfectly.

It also stops when the cache is already at its ceiling — filling a full cache would evict something a human read in order to store something nobody asked for. Fetches are spaced 250 ms apart so a provider does not see a burst.

Since 1.16.0 there is a switch: Settings ▸ Security ▸ Message body cache, right beside the size and age limits — because it is the same question, how much of your mail ends up on this disk and how it got there. It is on by default, and it stays on across an upgrade: the reason it was worth turning off (the \Seen-marking leak below) is already fixed, and quietly changing a performance default under people would be its own kind of surprise. With it off, background prefetch does no work at all — no targets are computed, no cache is measured, no connection is opened — but opening a message still fetches its body on demand, exactly like any other cache miss. The switch is owner-only, the same as the rest of this panel.

Prefetch used to mark your mail read on your own server — fixed in 1.15.0

Up to and including 1.14.0, every message SkimMail fetched was requested with FETCH BODY[...], and that form implicitly sets the \Seen flag on the mail server (RFC 3501 §6.4.5). BODY.PEEK[...] is the same fetch without the side effect, and nothing in SkimMail used it.

Because prefetch runs in the background over the newest messages, mail nobody had opened was being marked read on the real server — visible in webmail and on the phone, not just here. The next sync then read that state back, so everything inside SkimMail stayed self-consistent and nothing looked wrong. Downloading an attachment and using one-click unsubscribe did the same.

Upgrade to 1.15.0. All three fetch paths now peek.

Mail already marked read this way cannot be repaired. It is indistinguishable from mail you genuinely read, so there is nothing for a migration to undo. The fix stops it from here; it cannot reach backwards.

What it does not do

• It is not an archive. An evicted body is re-fetched from the mail server; if the message is gone from the server, it is gone.
• It has no per-account or per-user limits. The ceiling and the age limit are instance-wide.
• It does not cache attachments.
• It does not show you which messages are cached. The panel reports a count and a total size, and the purge acts on unaccounted objects; there is no per-message list and no way to pin one body.

See also

• Configuration — the environment reference and the three configuration planes
• Security — encryption at rest, and what a stolen DATA_DIR exposes
• Troubleshooting — disk usage that does not drop after an upgrade
• Users and roles — why the cache endpoints are owner-only

---

SkimMail · skimmail@base101.app · 2026-09-15 · commit 767741a