Storage in IMAP METADATA

ALPS runs no database. It keeps each account's settings, sender identities, linked accounts, security keys, OpenPGP keyring and own CalDAV and CardDAV connections on the account's own IMAP server as IMAP METADATA (RFC 5464), or in a file in the user's maildir with the maildir provider. This page covers what is stored and what stays elsewhere, what the mail server must offer, what happens on a server without METADATA, and what this means for backups and for moving accounts. The reference is the persistence section of ARCHITECTURE.md.

Why there is no database

ALPS signs in to the mail server as the user and does all its work as that user. Data that belongs to an account is kept with the account, on the same server as its mail. This has three effects:

• There is no separate database to install, secure or back up.
• Every ALPS node and every browser reads the same records from the mail server, so a user's settings and keys are the same on whichever node serves them.
• A restart of ALPS loses only its node-local state: sessions and their caches, uploaded attachments not yet sent or saved, sign-ins waiting for a second factor, and rate-limit counts. The account data stays on the mail server. See What a Restart Loses.

What ALPS keeps on the mail server

Each record is a JSON value in a private server annotation: an entry on mailbox "" (the server as a whole, not a folder) whose name starts with /private/vendor/alps/. A private entry belongs to the account that wrote it. Other users of the server do not see it.

Record	What it holds	Where
Settings	The user's preferences: theme, colour family, layout and density, threading, themed message content, sender avatars, messages per page, HTML or plain-text view, display name, signature, reply-to, Bcc myself, sender identities, check-mail interval, auto-logout, desktop and sound notifications, sort order and criteria, remote content, compose format, mark-read and undo timeouts, language, hour and date formats, week start, folder order, and the folders unsubscribed in ALPS.	/private/vendor/alps/base.settings
OpenPGP keyring	The gpg plugin's public key, and the private key encrypted with the user's passphrase. The passphrase and the unlocked private key never reach the server (GPG.md).	/private/vendor/alps/gpg_keys
Linked accounts	The accounts under Settings > Linked Accounts, each with its password encrypted with [server] login_key. A link is recorded in both accounts.	Under /private/vendor/alps/
Two-factor sign-in	Whether it is on, the registered security keys, what ALPS needs to check the unused recovery codes, and the Trust Linked Accounts choice.	Under /private/vendor/alps/
CalDAV and CardDAV connections	The connection a user saved under Settings > Calendar or Settings > Contacts: a server of their own, or none for the administrator's, and either the mail credentials or a separate username and password. A separate password is encrypted with [server] login_key. There is an entry only once the user saves a connection.	/private/vendor/alps/dav.connection.caldav, /private/vendor/alps/dav.connection.carddav

Some details matter when you plan storage, backups or migrations:

• Sender identities are part of the settings record. They do not have an entry of their own.
• The settings record must fit in one METADATA entry. ALPS therefore allows at most 20 identities, and at most 200 unsubscribed folder names (16 KiB in all). The mail server's size limit for one entry must leave room for the whole record.
• The two-factor record can take up several entries, so that several ALPS instances can update it safely. When you copy or back up an account's ALPS data, take the whole /private/vendor/alps/ subtree, at every depth.
• An account with no settings saved has no settings entry, and ALPS uses the defaults. The first sign-in from a browser where a language was picked saves that language to the account.
• The settings and the CalDAV and CardDAV connections belong to the base plugin. Of the other bundled plugins, only gpg stores data here. managesieve and password keep nothing in METADATA: their servers come from the configuration. A plugin compiled into ALPS can keep its own entries through the session's store, under the same prefix (see Plugins and PLUGINS.md).

What stays elsewhere

Data	Where it lives
Mail, sent copies and saved drafts	The account's IMAP folders, or its maildir
Contacts and calendars	The CardDAV and CalDAV servers (CDAV.md)
Filters and auto-reply	The account's Sieve script on the ManageSieve server (FILTERS.md)
Sessions, their caches, uploaded attachments, pending sign-ins, rate-limit counts	The memory and temp directory of the ALPS node. A restart loses them.
Let's Encrypt certificates, the log file, the BIMI logo cache	The ALPS host, or S3 for certificates (What ALPS Writes to Disk)
A copy of the settings for a fast start, setting changes not yet saved, the composer's unsent content, the sidebar's collapsed state, contact categories that have no contacts yet	The browser's local storage, kept per user. It is not on the server and not part of a server backup.
The theme, language and a few display choices shown before sign-in, and the calendar's show-tasks choice	The browser's local storage, shared by everyone who uses that browser

What the mail server must offer

The IMAP server must list METADATA or METADATA-SERVER in its CAPABILITY answer. ALPS goes only by that list. It does not check at startup: each new IMAP connection looks at the capabilities and picks its store.

To check a server by hand, connect over TLS with an IMAP client that lets you type commands, and sign in as a test account:

a LOGIN test@example.com password
b CAPABILITY
c GETMETADATA (DEPTH infinity) "" "/private/vendor/alps"
d LOGOUT

The answer to b must include METADATA or METADATA-SERVER. The answer to c lists the entries ALPS has written for that account, and nothing until the account first saves something in ALPS. Do not rely on c alone: a server can answer GETMETADATA with OK without listing the capability, and ALPS then treats it as a server without METADATA.

• sora advertises METADATA.
• Dovecot's imap_metadata setting is off by default (checked with Dovecot 2.4.5), and Dovecot then does not list METADATA. See Setup-Dovecot-and-Postfix for a Dovecot setup.
• Mail-Server-Requirements lists the other capabilities ALPS uses.

When several ALPS instances serve the same accounts, they coordinate updates to the two-factor record through METADATA alone. The server must then make every acknowledged write visible to any read that starts after it, and list a subtree completely and consistently. ALPS cannot check this, so the operator must make sure of it. With [cluster] enabled, ALPS offers recovery codes only on a METADATA store. See WEBAUTHN_COORDINATION.md and Clustering.

On a server without METADATA

ALPS still starts, and users still sign in. At the first connection to such a server it logs this line once per process, at info level, through [logging] like every other line:

alps/provider: IMAP server doesn't support the METADATA extension, using transient store instead

Each IMAP connection to such a server keeps its own store in memory:

• A save succeeds, and the change shows for as long as that connection stays open.
• A reconnect or a new sign-in starts again from the defaults. The session's background connection does not see the change either. Only what the browser keeps for itself, such as its theme and language, still shows in that browser.
• The user is not told. The settings page reports each save as done.

So nothing saved lasts: not settings, identities, linked accounts, the keyring or the CalDAV and CardDAV connections, and not security keys. Two-factor sign-in needs METADATA, because a key added without it is forgotten. With [cluster] enabled, recovery codes are switched off on such a server, and their requests answer 503 recovery_unavailable.

Use a server without METADATA only to try ALPS out. On a new deployment, sign in once and check the log for the line above.

The maildir equivalent

With the maildir provider ([provider] type = "maildir", or a multi domain with type = "maildir"), ALPS keeps the same records in one JSON file, alps_store.json, at the root of the user's maildir, beside cur, new and tmp:

• The file has mode 0600 and belongs to the user ALPS runs as. Each change rewrites the whole file through alps_store.json.tmp and a rename, so a reader never sees half a file.
• Changes are serialised only within one ALPS process, so serve each maildir from a single ALPS instance.
• When the user's maildir does not exist yet, the first write of the file creates it, with any missing parent directories, mode 0700. Settings and security keys can therefore be saved before the first delivery.
• Keep the file valid JSON. While it does not parse, the account cannot sign in (503 with code service_unavailable), and its settings cannot be read or saved. Restoring a good copy fixes this straight away.
• With [cluster] enabled, recovery codes are not available to maildir accounts.

Under the multi provider, each login's backend decides. A domain of type imap keeps its records in its IMAP server's METADATA, and a maildir domain keeps them in the user's alps_store.json. The settings, the keyring and the CalDAV and CardDAV connections use the same names in both stores (base.settings, gpg_keys, dav.connection.caldav and dav.connection.carddav). ALPS has no tool that converts records between the two.

See Setup-Maildir and The Settings File.

Backups

• IMAP: back up the mail server together with its METADATA. Where a server keeps METADATA depends on the server, so check that your mail server backup covers it as well as the folders. The ALPS host holds no account data.
• Maildir: back up alps_store.json along with the maildir.
• ALPS configuration: back it up as well, and keep the copy as private as the file itself, since it holds login_key. Restored accounts need the same [server] login_key for their linked accounts and saved CalDAV and CardDAV passwords, and the same [webauthn] rpid for their security keys.

To restore one account's records, do it while the user is signed out, or have the user sign in again afterwards. A signed-in session keeps the copies it has already read until it next connects to the mail server. For the two-factor record, read WEBAUTHN.md before you restore or reset it.

Moving accounts to another mail server

The records follow an account to another IMAP server only if the move takes them along:

1. The new server must advertise METADATA or METADATA-SERVER.
2. Copy the private server annotations under /private/vendor/alps/, at every depth, along with the mail. A migration that copies only folders and messages leaves the settings, identities, keyring, linked accounts, CalDAV and CardDAV connections and security keys behind. Check what your migration tool copies.
3. Copy the values unchanged. They are JSON, and when ALPS reads a settings entry it cannot decode, it removes the entry: the account starts again from the default settings, and the log shows Removed unreadable settings of <user>: <reason>.
4. Serve the moved accounts from an ALPS with the same [server] login_key and the same [webauthn] rpid as before.

If login_key changes, the stored linked-account passwords can no longer be read. Switching to a linked account then fails, and the account menu tells the user to remove the linked account in Settings and link it again with its current password. The separate CalDAV and CardDAV passwords can no longer be read either: those connections fail, without falling back to the mail password, until the user enters the password again under Settings > Calendar or Settings > Contacts. See Changing the Key.

If rpid changes, security keys added under the old RP ID no longer work, because a key is bound to the RP ID it was added under. A user who still has a recovery code signs in with it, adds a new key under Settings > 2FA / WebAuthn, and removes the old one. Without a code, the operator resets the account's second factor as WEBAUTHN.md describes.

Related pages

• Mail-Server-Requirements: everything ALPS needs from the IMAP and SMTP servers
• Sessions-and-Sign-in: login_key, session restore and linked accounts
• Security-Features: two-factor sign-in and recovery codes
• Mail-Providers and Setup-Maildir: the imap, maildir and multi providers
• Clustering: several ALPS nodes on the same accounts
• Config-Server (login_key) and Config-Rate-Limiting-and-WebAuthn (rpid)
• Running-in-Production