-
Notifications
You must be signed in to change notification settings - Fork 3
Architecture
Source of truth: ARCHITECTURE.md, kept in the repository and copied here. This copy was corrected against master 40ae9491d on 8 September 2026 (the optional-listener paragraphs, the
ExternalFetcher/andUtil/rows, the scheduled tasks) and extended on the same date with the diagrams, the thread-pool table and the pipeline walk-throughs, all drawn from the code cited beside them; the repository file has not yet had the same corrections. Items marked "new in 6.2.28" ship for the first time in 6.2.28, published on 8 September 2026.
A map of the codebase, for someone about to change it. It answers "where does this change go" rather than "how does mail work", and it records the constraints that are not obvious from reading any single file — the ones that have actually caused bugs here.
Pair it with Contributing for process and Release Process for the release gates.
Everything below is a detail of this diagram. Paths in the boxes are relative to
hmailserver/source/Server/. Dashed edges are outbound connections this server
makes as a client; solid edges are calls inside the process.
flowchart TB
subgraph peers["Outside"]
MTA["Remote MTAs"]
MUA["Mail clients<br/>Outlook, Thunderbird, phones"]
ADMIN["Control Panel, DBUpdater,<br/>the regression suite, VBScript"]
WEB["Browsers and scrapers<br/>portal, autoconfig, Prometheus"]
end
subgraph listeners["Listeners"]
direction TB
IOS["IOService + TCPServer<br/>Common/TCPIP<br/>SMTP 25/587, POP3 110, IMAP 143"]
HTTPS["HttpServer<br/>Common/Util/HttpServer<br/>own io_context, 4 workers"]
REST["RestApiServer<br/>/api/v1, /portal"]
WSVC["WebServicesServer<br/>MTA-STS policy, autoconfig,<br/>ACME http-01"]
METR["MetricsServer<br/>raw sockets + std::thread"]
MSIEVE["ManageSieveServer<br/>raw sockets + std::thread"]
end
subgraph sessions["Protocol sessions"]
SMTPC["SMTPConnection<br/>SMTP/"]
IMAPC["IMAPConnection<br/>IMAP/"]
POP3C["POP3Connection<br/>POP3/"]
end
subgraph pipeline["Delivery pipeline"]
DQ["SMTPDeliveryManager<br/>+ DeliveryTask<br/>10 threads"]
DELIV["SMTPDeliverer<br/>rules, AV, DKIM sign"]
LOCAL["LocalDelivery<br/>quota, rules, Sieve,<br/>forward, auto-reply"]
EXT["ExternalDelivery<br/>+ ServerTargetResolver<br/>+ TlsPolicy"]
CLIENT["SMTPClientConnection"]
end
subgraph core["Shared core - Common/"]
BO["BO/ business objects"]
CACHE["Cache/"]
PERS["Persistence/"]
SQLL["SQL/ + SQL/Macros"]
AS["AntiSpam/ SPF, DNSBL, SURBL,<br/>greylisting, DKIM, DMARC, ARC"]
AV["AntiVirus/ clamd, clamscan,<br/>external command"]
SIEVE["Sieve/ lexer, parser,<br/>evaluator, storage"]
SCRIPT["Scripting/ VBScript, JScript"]
MIME["Mime/"]
TRACK["Tracking/ change notifications"]
end
subgraph store["Persistent state"]
DB[("hm_* tables<br/>MySQL / MariaDB<br/>MS SQL Server<br/>PostgreSQL<br/>SQL CE")]
DISK["Data directory<br/>message files, Sieve scripts,<br/>logs, ACME certs"]
INI["hMailServer.ini<br/>IniFileSettings"]
end
subgraph sched["Background work - both long-lived tasks<br/>on the main server queue"]
SCH["Scheduler + ScheduledTask<br/>recurring tasks polled once a minute,<br/>RunOnce through the maintenance queue"]
FETCH["ExternalFetchManager<br/>ExternalFetcher/"]
end
COM["COM / IDispatch API<br/>COM/ - the management seam"]
MTA --> IOS
MUA --> IOS
WEB --> HTTPS
WEB --> METR
MUA --> MSIEVE
ADMIN --> COM
IOS --> SMTPC
IOS --> IMAPC
IOS --> POP3C
HTTPS --> REST
HTTPS --> WSVC
SMTPC --> DQ
DQ --> DELIV
DELIV --> LOCAL
DELIV --> EXT
EXT --> CLIENT
CLIENT -.->|"SMTP out: MX, route or smart host"| MTA
SMTPC --> AS
SMTPC --> BO
IMAPC --> BO
POP3C --> BO
DELIV --> AV
DELIV --> SCRIPT
LOCAL --> SIEVE
LOCAL --> BO
LOCAL --> TRACK
TRACK --> IMAPC
REST --> BO
METR --> BO
MSIEVE --> SIEVE
COM --> BO
BO --> CACHE
CACHE --> PERS
BO --> PERS
PERS --> SQLL
SQLL --> DB
LOCAL --> DISK
DELIV --> MIME
BO --> INI
FETCH -.->|"POP3 or IMAP in"| MTA
FETCH --> BO
SCH --> BO
Two things the picture is meant to make obvious. The COM API is not beside the
server — it is on top of the same BO/ objects every protocol uses, which is why
a configuration change made in the Control Panel is visible to the next SMTP
connection without a restart. The REST listener sits beside it rather than on top of
it: it reaches the same business objects directly (it includes no COM/ header), and
its authorisation goes through ACLManager, which build/check-authz-choke-point.py
keeps as the only place folder access is decided. And the delivery pipeline is not on the connection's
thread: SMTPConnection writes a queue row and lets go, and SMTPDeliveryManager
picks it up. Everything in the "Delivery pipeline" box happens after the sender has
been told 250.
hmailserver/
source/
Server/ C++ mail server - almost all feature work happens here
Tools/ C# administration tools and the Control Panel (.NET 10)
Addons/ Standalone sample add-ons
DBScripts/ Schema creation and the upgrade chain, per backend
Translations/ The legacy hMailServer Administrator's INI catalogues -
english.ini and swedish.ini. The Control Panel's own 17
languages are .resx files under
Tools/ControlPanel/Resources/
test/
RegressionTests/ NUnit suite driving a real running server
installation/ Inno Setup installer
docs/ Operator documentation
libraries/ Vendored third-party code
build/ Build, publish, preflight and test scripts
fuzz/ libFuzzer harnesses, dictionaries and the regression corpus
Entry point: source/Server/hMailServer/hMailServer.sln.
Server/
COM/ COM/IDispatch public API - the management seam
Common/ Shared infrastructure used by every protocol
ExternalFetcher/ POP3 and IMAP *clients*: fetch mail from remote accounts
(`ExternalFetchClientBase` turns a downloaded message into a
delivered one; `IMAPClientConnection` also mirrors whole
folders when `FetchAccount.MirrorFolders` is on - 6.2.25/6.2.27)
hMailServer/ Windows service shell (WinMain, service control)
hMailServer.Minidump/ Crash-dump helper
hMailServer.Updater/ The live-update apply helper: runs a verified installer,
waits for the service, rolls back (new in 6.2.28)
zlib/ Vendored zlib 1.3.1 for IMAP COMPRESS=DEFLATE (new in 6.2.28)
IMAP/ IMAP
POP3/ POP3
SMTP/ SMTP, delivery queue, outbound transport security
platform/ The thin layer that keeps the tree compiling off Windows
Server/hMailServer/ is the service shell only — no protocol or business logic.
| Sub-folder | Purpose |
|---|---|
AntiSpam/ |
SPF, SURBL, DNS blacklists, greylisting, score-based filtering, DMARC/, and DKIM/ (RSA and Ed25519 signing/verification, ARC sealing in Arc.{h,cpp}) |
AntiVirus/ |
ClamAV (clamd and clamscan) and arbitrary command-line scanners |
Application/ |
Startup, configuration, scheduling, logging. IniFileSettings holds every hMailServer.ini setting; Application::StartServers starts the optional listeners |
BO/ |
Business objects — the domain model: domains, accounts, aliases, distribution lists, rules |
Cache/ |
In-memory caches in front of the BO layer, to keep hot paths off the database |
Diagnostics/ |
The checks behind the Control Panel's "Run diagnostics" |
LDAP/ |
The directory backend: LdapSettings, synchronisation, the AD account pickers' server side |
Mime/ |
MIME parsing and construction |
Persistence/ |
One class per business object, mapping it to database columns |
Rules/ |
The global and account rule engine that RuleApplier drives |
Scripting/ |
VBScript/JScript event hooks |
Sieve/ |
SieveLexer/SieveParser (AST), SieveEvaluator, SieveStorage (per-account scripts under {DataDir}\Sieve\), and the optional RFC 5804 ManageSieveServer. Evaluated during local delivery |
SQL/ |
Database abstraction: connections, pooling, parameterised queries |
TCPIP/ |
Boost.Asio networking, TLS, DNS. Also DnssecResolver (validating stub resolver), DaneVerifier (TLSA matching), ProxyProtocol and DeflateStreams (IMAP COMPRESS) |
Threading/ |
Thread pools and task queues |
Tracking/ |
Publish/subscribe bus between components |
Util/ |
Utilities, plus the optional listeners: MetricsServer, RestApiServer, WebServicesServer (the last two on HttpServer, the shared Boost.Asio HTTP/1.1 listener), AcmeClient, TlsRptStore, HttpsClient (the update feed and its downloads, JWKS, token introspection - the one place [Settings] HttpProxy is honoured, and the only one: AcmeClient itself, OutboundOAuth2TokenClient and the MTA-STS fetch in SMTP/TlsPolicy open their own connections and go direct) and the OTLP exporters (Otel*), and the live update (UpdateChecker, SigstoreVerifier, UpdateDownloader, UpdateInstaller, UpdateWindow, UpdateApplyToken). HttpServer, the Update* files and the proxy support are new in 6.2.28; Application/ holds their tasks UpdateCheckTask and MetricsHistoryTask
|
The most complex module: reception, relay decisions, the disk-backed delivery queue,
bounces, DKIM signing, and outbound delivery. Outbound transport security lives here
too — TlsPolicy implements MTA-STS discovery/caching and DANE TLSA lookups,
ExternalDelivery applies the per-host requirements, TlsRptReporterTask sends the
daily RFC 8460 reports. The vendored SPF implementation is SPF/RMSPF.cpp.
One command-handler class per IMAP command. Folder hierarchy and message flags are in
the database via Persistence/; message bodies are on disk. POP3 is much simpler and
reads the same storage.
This is the path a message actually takes, from the remote server's TCP SYN to the
last recipient's mailbox. It is drawn from SMTP/SMTPConnection.cpp,
SMTP/SMTPDeliveryManager.cpp, SMTP/SMTPDeliverer.cpp, SMTP/LocalDelivery.cpp and
SMTP/ExternalDelivery.cpp. Note where the thread changes: the 250 is sent from a
worker on the asynchronous task queue, and everything below the dividing line runs
later, on a delivery-queue thread.
sequenceDiagram
autonumber
participant P as Remote MTA
participant C as SMTPConnection<br/>IOCP thread
participant A as Async task queue<br/>15 threads
participant Q as Delivery queue<br/>10 threads
participant D as SMTPDeliverer
participant L as LocalDelivery
participant X as ExternalDelivery
participant R as Next-hop MTA
P->>C: TCP connect
C->>P: 220 banner
P->>C: EHLO peer.example
C->>P: 250 keyword list
Note over C: STARTTLS, AUTH and XCLIENT<br/>all re-shape what follows
P->>C: MAIL FROM
Note over C: SPF, DNSBL and HELO checks<br/>can run here or at RCPT
C->>P: 250 sender OK
P->>C: RCPT TO
Note over C: CheckDeliveryPossibility,<br/>relay permission, auth requirement,<br/>quota, greylisting
C->>P: 250 recipient OK
P->>C: DATA
C->>P: 354 send the message
P->>C: message octets, then CRLF.CRLF
C->>A: hand off finalization
Note over A: SpamAssassin, post-transmission<br/>spam tests, DKIM/DMARC/ARC verify,<br/>write the queue file, insert hm_messages
A->>P: 250 2.0.0 Queued (0.412 seconds)
Note over Q: --- the conversation is over ---
Q->>D: DeliveryTask picks the row up
D->>D: OnDeliveryStart, virus scan,<br/>global rules, OnDeliverMessage,<br/>DKIM sign (first attempt only)
D->>L: local recipients
L->>L: quota, account spam overrides,<br/>auto-reply, account rules, forward,<br/>trace headers, Sieve
L-->>D: delivered, or a DeliveryFailure
D->>X: remaining external recipients
X->>X: ServerTargetResolver picks a route,<br/>smart host or MX. TlsPolicy applies<br/>MTA-STS and DANE
X->>R: SMTP out
R-->>X: 250 / 4xx / 5xx
X-->>D: delivered, deferred or failed
D->>D: bounce report for failures,<br/>delete or reschedule the queue row
| Stage | Where | Can answer | Consequence of a wrong answer |
|---|---|---|---|
| Accept the TCP connection |
TCPServer, security ranges |
drop before the banner | A blocked range that should not be blocked looks to the peer like a dead host, with no log line at the sender's end |
Banner and EHLO
|
SMTPConnection::SendEHLOKeywords_ |
advertise or withhold an extension | Withholding STARTTLS makes every peer fall back to plaintext silently |
MAIL FROM |
ProtocolMAIL_, DoSpamProtection_
|
5xx, 4xx, or continue | An SPF -all refusal here rejects before the recipient is known, so the log names no mailbox |
RCPT TO |
ProtocolRCPT_ |
250, 451, 452, 530, 550, 554
|
The single most consequential decision in the server — see the flowchart below |
End of DATA
|
the finalization task |
250 or 451
|
A 451 here means the peer will resend the whole message; a 250 means this server now owns it |
| Preprocess | SMTPDeliverer::PreprocessMessage_ |
deliver, defer, or abort | An abort at this point silently destroys a message the sender was told was accepted |
| Local delivery | LocalDelivery::LocalDeliveryPreProcess_ |
file, redirect, reject, discard | A Sieve reject produces a DSN; a discard produces nothing at all |
| External delivery | ExternalDelivery::RescheduleDelivery_ |
retry or bounce | Bouncing a transient failure costs somebody their mail — see the standing rule below |
Every branch here is in SMTP/SMTPConnection.cpp ProtocolRCPT_, in the order the
code takes them. The order matters: the cheap refusals come first so a dictionary
attack costs the server as little as possible, and the tarpit is applied before
any answer so that refusals are held too.
flowchart TD
START["RCPT TO received"] --> TARPIT["TarpitRecipient_<br/>hold this answer"]
TARPIT --> TLS{"STARTTLS required<br/>and not yet done?"}
TLS -- yes --> R530A["530 Must issue STARTTLS first"]
TLS -- no --> SENDER{"MAIL FROM seen?"}
SENDER -- no --> R503["503 Must have sender first"]
SENDER -- yes --> SYNTAX{"Parses as<br/>RCPT TO:<addr>?"}
SYNTAX -- no --> R550A["550 Invalid syntax"]
SYNTAX -- yes --> EXT{"Only NOTIFY= and<br/>ORCPT= parameters?"}
EXT -- no --> R501["501 / unsupported extension"]
EXT -- yes --> VALID{"Valid address<br/>after default domain?"}
VALID -- no --> R550B["550 A valid address is required"]
VALID -- yes --> COUNT{"Under 50,000<br/>recipients?"}
COUNT -- no --> R550C["550 Too many recipients"]
COUNT -- yes --> SQ{"Account sending<br/>quota left?"}
SQ -- no --> RQUOTA["refused by quota"]
SQ -- yes --> DP["CheckDeliveryPossibility<br/>inside DatabaseUnavailableMarker::Scope"]
DP --> DBUP{"Database<br/>answered?"}
DBUP -- no --> R451["451 4.3.2 Unable to verify<br/>the recipient at the moment"]
DBUP -- yes --> FULL{"Mailbox full?"}
FULL -- yes --> R452["452 4.2.2 - temporary,<br/>so the sender's queue holds it"]
FULL -- no --> POSSIBLE{"Deliverable<br/>at all?"}
POSSIBLE -- no --> R550D["550 with the reason"]
POSSIBLE -- yes --> RELAY{"Security range allows<br/>this local/remote pairing?"}
RELAY -- no --> R550E["550 Delivery is not allowed<br/>to this address"]
RELAY -- yes --> AUTH{"Range requires auth<br/>for this pairing<br/>and none given?"}
AUTH -- yes --> R530B["530 SMTP authentication is required"]
AUTH -- no --> BIN{"BODY=BINARYMIME<br/>to a remote address<br/>without OutboundChunking?"}
BIN -- yes --> R554["554 5.6.3 Conversion required<br/>but not supported"]
BIN -- no --> SPAM["Pre-transmission spam protection,<br/>then greylisting"]
SPAM --> OK["250 recipient accepted"]
The 451/550 split in the middle of that chart is the DatabaseUnavailableMarker,
and it is the reason that class exists — see "Constraints learned the hard way".
LocalDelivery::Perform loops over the recipients whose LocalAccountID is
non-zero; this is what happens to each one, from LocalDelivery.cpp. The order is
load-bearing in three places, each marked.
flowchart TD
IN["Recipient with a local account"] --> QUOTA{"CheckAccountQuotas_"}
QUOTA -- over --> DSNQ["DeliveryFailure -<br/>a DSN unless NOTIFY excluded it"]
QUOTA -- ok --> COPY["CreateAccountLevelMessage_<br/>own file, or a hard link to<br/>the shared delivery template"]
COPY -- failed --> DSNC["4.3.1 mail system full<br/>+ HM5209 in the error log"]
COPY -- ok --> SPAMO["ApplyAccountSpamOverrides_<br/>FIRST, so everything below sees<br/>this account's own spam verdict"]
SPAMO -- "delete threshold hit" --> DROPS["dropped, and deliberately<br/>no bounce - it would be backscatter"]
SPAMO -- keep --> VAC["Account vacation message"]
VAC --> RULES["Account rules"]
RULES -- "delete action" --> DROPR["dropped"]
RULES --> FWD["SMTPForwarding"]
FWD -- "forward, no local copy" --> DONEF["forwarded"]
FWD --> TRACE["Trace headers<br/>skipped when the copy came<br/>from the traced template"]
TRACE --> SIEVE["EvaluateSieveScript_<br/>redirect, fileinto, flags,<br/>reject, discard, vacation"]
SIEVE --> DOM{"Neither account vacation<br/>nor Sieve vacation?"}
DOM -- "yes, silent" --> DOMREPLY["Domain-wide out-of-office<br/>- at most ONE auto-reply per message, ever"]
DOM -- no --> REJ
DOMREPLY --> REJ{"Sieve reject?"}
REJ -- yes --> DSNR["5.7.1 refusal DSN"]
REJ -- no --> DISC{"Sieve discard<br/>or redirect without keep?"}
DISC -- yes --> DROPD["no local copy, no report"]
DISC -- no --> FLAGS["ApplySieveFlags_<br/>BEFORE the save, or the flags<br/>are simply lost"]
FLAGS --> FOLDER["Folder selection:<br/>global rule, then account rule,<br/>then Sieve fileinto wins"]
FOLDER --> SAVE["SetState Delivered, recompute size,<br/>PersistentMessage::SaveObject"]
SAVE -- failed --> DSNS["file deleted, HM6081 reported,<br/>4.3.0 DSN - never silent loss"]
SAVE -- ok --> NOTIFY["SetFolderNeedsRefresh +<br/>NotificationMessageAdded<br/>so an idle IMAP session sees it"]
Three orderings worth knowing before you change this function:
- The account's own spam overrides run first, so the auto-replies, rules, forwarding and Sieve all see that account's view of the message rather than the shared verdict.
- The domain-wide auto-reply is decided after Sieve, because the Sieve half of "does this account already have a voice" is not known any earlier. One delivered message produces at most one auto-reply.
-
Sieve flags are applied before the save, and were not until 15 August 2026 —
setflagparsed, ran, reported success in the action summary and changed nothing.
ExternalDelivery::Perform groups recipients by resolved server, splits each group
into batches of MaxSMTPRecipientsInBatch, and gives every batch its own copy of the
ServerInfo — because DeliverToSingleDomain_ overwrites the host name with the MX
it selects. Sharing one object made every batch after the first look up the MTA-STS
policy of an MX host instead of the recipient domain, which silently stopped
enforcement.
The four ways a next hop can be chosen are tried in this order, and the order is
deliberate: a route is about where the mail is going and beats everything, a
per-domain relay is about which provider the sending domain bought, the global
smart host is the fallback for mail nothing else has an opinion about, and MX
is what the recipient's own DNS says (ServerTargetResolver.cpp:225-290).
flowchart LR
R["External recipients"] --> STR["ServerTargetResolver"]
STR --> ROUTE{"A route matches the<br/>RECIPIENT domain?<br/>wildcard match"}
ROUTE -- yes --> RT["Route target host and port,<br/>optional relayer credentials"]
ROUTE -- no --> PDR{"The SENDER's domain<br/>has its own relay host?"}
PDR -- yes --> PDRT["Per-domain relay<br/>port 25 if unset"]
PDR -- no --> SH{"Global SMTP relayer set?"}
SH -- yes --> SHT["Smart host<br/>port 25 if unset"]
SH -- no --> MX["MX lookup, preference order"]
RT --> BATCH
PDRT --> BATCH
SHT --> BATCH
MX --> BATCH["Batch by MaxSMTPRecipientsInBatch,<br/>one ServerInfo copy per batch"]
BATCH --> POL["TlsPolicy:<br/>MTA-STS policy, DANE TLSA,<br/>DNSSEC-validated"]
POL --> CONN["SMTPClientConnection"]
CONN --> RES{"Per-recipient result"}
RES -- "delivered" --> DEL["row deleted"]
RES -- "non-fatal 4xx" --> RETRY{"retries left?"}
RES -- "fatal 5xx" --> BOUNCE["DeliveryFailure -> DSN"]
RETRY -- "under QuickRetries" --> QR["SetNextTryTime<br/>QuickRetriesMinutes<br/>+ QueueRandomnessMinutes"]
RETRY -- "normal" --> NR["SetNextTryTime<br/>the configured interval<br/>+ QueueRandomnessMinutes"]
RETRY -- "exhausted" --> BOUNCE
QR --> UNLOCK["UnlockObject - a later<br/>delivery thread may take it"]
NR --> UNLOCK
QuickRetries (default 0) and QuickRetriesMinutes (default 6) exist for
greylisting: the first N failures are retried after a few minutes rather than after
the full interval (IniFileSettings.cpp:301-303). QueueRandomnessMinutes (default
0) adds jitter so a queue that failed together does not retry together.
There is no "one thread per connection" here, and no unbounded pool anywhere. Six pools exist, each created in one place, and every one of them is a ceiling that a stuck dependency will hit.
flowchart TB
subgraph created_in_InitInstance["Created in InitInstance"]
MQ["Maintenance queue<br/>5 threads<br/>Application.cpp:150"]
end
subgraph created_in_StartServers["Created in StartServers, before io_service_"]
AQ["Asynchronous task queue<br/>MaxNumberOfAsynchronousTasks, seeded 15<br/>Application.cpp:405"]
NQ["Name lookup queue<br/>same setting, seeded 15<br/>Application.cpp:413"]
end
subgraph created_after["Created with io_service_"]
IOCP["IOCPQueue<br/>tcpipthreads, seeded 15<br/>IOService.cpp:152"]
MAIN["Main server queue<br/>4 threads<br/>Application.cpp:471"]
DELQ["SMTP delivery queue<br/>maxdelivertythreads, seeded 10<br/>SMTPDeliveryManager.cpp:37"]
FETQ["External fetch queue<br/>MaxNumberOfExternalFetchThreads, 15<br/>ExternalFetchManager.cpp:39"]
end
subgraph own["Their own threads, outside the queues"]
HTTPW["HttpServer workers<br/>4, own io_context<br/>HttpServer.h worker_threads"]
RAW["MetricsServer and<br/>ManageSieveServer<br/>std::thread each"]
end
IOCP -->|"reads, writes, TLS handshakes"| AQ
IOCP --> NQ
MAIN -->|"hosts the long-lived tasks"| DELQ
MAIN --> FETQ
| Pool | Threads | Set by | What runs on it | What it looks like when it saturates |
|---|---|---|---|---|
| IOCPQueue |
tcpipthreads, seeded 15 |
Control Panel → Advanced → Performance | Every socket read, write and TLS handshake for SMTP, POP3 and IMAP | New connections are accepted but nothing progresses; clients time out mid-command |
| Asynchronous task queue |
MaxNumberOfAsynchronousTasks, seeded 15 |
same page | Message finalization — this is the thread that sends the final 250
|
Senders sit after CRLF.CRLF with no reply. This is the exact shape of discussion #18
|
| Name lookup queue | same setting, seeded 15 | same page | Blocking reverse-DNS on behalf of a live session | Reverse names degrade to "Unknown" — deliberately, so a dead reverse zone cannot stall acknowledgements |
| SMTP delivery queue |
maxdelivertythreads, seeded 10 |
Control Panel → Protocols → SMTP → Delivery of e-mail |
DeliveryTask → SMTPDeliverer for one queued message |
The queue grows and mail is late; the sender was already told 250, so nothing is lost |
| External fetch queue |
[Settings] MaxNumberOfExternalFetchThreads, default 15 |
hMailServer.ini |
One ExternalFetchClientBase run per fetch account |
Fetch accounts fall behind their schedule |
| Maintenance queue | 5, fixed | not configurable |
RunOnce scheduled tasks, submitted immediately |
A slow sweep delays other one-shot sweeps |
| Main server queue | 4, fixed | not configurable | The four long-lived tasks: Scheduler, IOService, SMTPDeliveryManager, ExternalFetchManager
|
It cannot saturate in normal operation — each task occupies one slot for the life of the server |
HttpServer workers |
4, HttpLimits::worker_threads
|
not configurable | One REST or web-services request each, and a handler may block | Requests queue; SMTP is unaffected, which is the reason for the separate io_context
|
Only the asynchronous task queue and the name lookup queue are opted into stall
monitoring (WorkQueue::SetMonitorForStalls). The other four are deliberately not:
their tasks are long-lived by design, and monitoring them would report the healthy
state of a stock installation once a minute.
Application::CreateSessionWorkQueues_ carries the longest comment in the file, and
it is worth reading before moving either call. A task on those two queues holds a
shared_ptr to a session, and that session owns a socket built on io_service_'s
io_context. When the queues were created in InitInstance and removed in
ExitInstance, they straddled the io_context rather than nesting inside it: on a
Reinitialize, a reverse-DNS prefetch finished 1.9 seconds after "Destructing IOCP",
dropped the last reference to its SMTPConnection, and faulted inside
~basic_stream_socket writing through a destroyed context. Creating them in
StartServers before io_service_ exists, and removing them in StopServers before
it is reset, makes the ordering an invariant of those two functions instead of
something every new queue has to remember.
An SMTP session is a state machine, and the states are literal: ConnectionState in
SMTP/SMTPConnection.h:289-303. Anything that reads a line has to know which state it
is in, because in DATA and BDATDATA the bytes are message content, and in the four
SASL states they are base64 continuations.
stateDiagram-v2
[*] --> INITIAL: accept, banner sent
INITIAL --> STARTTLS: STARTTLS
STARTTLS --> INITIAL: handshake complete,<br/>state reset, re-EHLO required
INITIAL --> SMTPUSERNAME: AUTH LOGIN
SMTPUSERNAME --> SMTPUPASSWORD: username accepted
SMTPUPASSWORD --> INITIAL: authenticated or refused
INITIAL --> SMTPSCRAMFIRST: AUTH SCRAM-SHA-256
SMTPSCRAMFIRST --> SMTPSCRAMFINAL: server-first sent
SMTPSCRAMFINAL --> SMTPSCRAMACK: server-final sent
SMTPSCRAMACK --> INITIAL: empty ack
INITIAL --> SMTPBEARERRESPONSE: AUTH XOAUTH2 / OAUTHBEARER
SMTPBEARERRESPONSE --> INITIAL: token checked
INITIAL --> SMTPEXTERNALRESPONSE: AUTH EXTERNAL
SMTPEXTERNALRESPONSE --> INITIAL: client certificate identity checked
INITIAL --> DATA: DATA, 354 sent
DATA --> INITIAL: CRLF.CRLF,<br/>ResumeCommandModeAfterData_
INITIAL --> BDATDATA: BDAT n
BDATDATA --> INITIAL: n octets read
INITIAL --> [*]: QUIT, timeout,<br/>session ceiling, or excessive data
Two rules the state machine enforces that are easy to break:
-
Leaving
DATAmust go throughResumeCommandModeAfterData_. It hands back the bytes a pipelining client sent behind the end-of-data marker so they are parsed as the commands they are.SetReceiveBinary(false)plusEnqueueRead()loses a pipelinedQUITor the nextMAIL FROMsilently. -
STARTTLSresets the session. A client that skips the secondEHLOis answered503; the keyword list it was given before the handshake is not the one that applies after it.
IMAP and POP3 have the same shape with different states; IMAP additionally has
COMPRESS=DEFLATE (new in 6.2.28), which inserts a DeflateStreams layer under the
connection and is one-way — once compression is on there is no going back for the life
of the session.
flowchart LR
subgraph callers["Callers"]
PROTO["Protocol handlers"]
COMI["COM interfaces"]
RESTI["REST handlers"]
end
CACHE["Cache/<br/>AccountCache, MessageCache,<br/>InboxIDCache, AccountSizeCache<br/>CacheContainer"]
BOX["BO/<br/>Account, Domain, Message,<br/>Rule, DistributionList, ..."]
PER["Persistence/<br/>PersistentAccount, PersistentMessage,<br/>one class per business object"]
CMD["SQLCommand + SQLParameter<br/>parameterised, always"]
MAC["SQL/Macros<br/>MSSQL, MySQL, PGSQL, SQLCE<br/>expanders"]
POOL["DatabaseConnectionManager<br/>pool + DatabaseUnavailableMarker"]
DAL["DALConnectionFactory"]
MSSQL[("MS SQL Server<br/>ADOConnection")]
MYSQL[("MySQL / MariaDB<br/>MySQLConnection")]
PG[("PostgreSQL<br/>PGConnection")]
CE[("SQL Server Compact 4.0<br/>SQLCEConnection")]
PROTO --> CACHE
PROTO --> BOX
COMI --> BOX
RESTI --> BOX
CACHE --> PER
BOX --> PER
PER --> CMD
CMD --> POOL
MAC -.->|"DDL only"| DAL
POOL --> DAL
DAL --> MSSQL
DAL --> MYSQL
DAL --> PG
DAL --> CE
The four backends are selected by DatabaseSettings::Type* in
SQL/DALConnectionFactory.cpp:53-64. They are not interchangeable in every respect:
| Backend | Connection class | Typical use | Known constraint |
|---|---|---|---|
| MS SQL Server |
ADOConnection (msado28) |
Larger installations already running SQL Server | The .tlh/.tli import files are committed; a rebuilt one must not carry an absolute path |
| MySQL / MariaDB |
MySQLConnection via libmysql.dll
|
The most common backend upstream | Connector/C is copied beside the exe by post-build.bat, plugins included |
| PostgreSQL |
PGConnection via libpq
|
Preferred on new installations | libpq is built from source by libraries/build-pgsql.ps1
|
| SQL Server Compact 4.0 | SQLCEConnection |
The embedded default — no server to install | Row-size and expression limits are real: CASE WHEN EXISTS in a DBUpdater probe crashed the provider and took the service down (#114, fixed in 6.2.26) |
The COM API is the seam. All configuration and management goes through
Server/COM/. The Control Panel, the regression suite and every third-party script
use it. A new configurable feature normally needs a COM property or method — and
because the test suite drives COM, that is also how the feature becomes testable.
Persistence is layered. BO/ → Persistence/ → SQL/, with Cache/ in front
for frequently-read objects. A feature that persists new data touches all of the
first three, and Cache/ if it is on a hot path.
Four database backends, one abstraction. MySQL/MariaDB, MS SQL Server,
PostgreSQL and the embedded SQL CE. Use parameterised queries exclusively — never
build SQL by string concatenation. Backend-specific DDL goes through the macro
expanders in SQL/Macros/, which recognise a deliberately small vocabulary; if you
need something they do not express, that is a design conversation, not a place to
special-case.
The schema is pinned, one way. REQUIRED_DB_VERSION in
Common/Application/Constants.h (6031 today) must equal hm_dbversion.value; the
server refuses to start on an older or newer database (error 5011; 5010 when the
version cannot be read). A schema change is four
DBScripts/Upgrade<from>to<to><backend>.sql files, a new UpgradeScript(from, to) row
in Tools/DBUpdater/formMain.cs, a probe statement DBUpdater runs after the step, and
the bump in Constants.h; build/check-schema-versions.ps1 reconciles them and
build/check-db-scripts.ps1 builds a database from the create script and executes
every probe through the SQL Server Compact provider, with a negative control — a probe
the provider could not run took the service down with it once (#114, 6.2.26).
Four files, one row, one probe, one constant — and a check for each:
flowchart LR
subgraph edits["What a schema change touches"]
S1["Upgrade6030to6031MSSQL.sql"]
S2["...MSSQLCE.sql"]
S3["...MySQL.sql"]
S4["...PGSQL.sql"]
ROW["new UpgradeScript(6030, 6031)<br/>Tools/DBUpdater/formMain.cs"]
PROBE["A probe statement<br/>SchemaVerification.cs"]
CONST["REQUIRED_DB_VERSION<br/>Common/Application/Constants.h"]
CREATE["CreateTables*.sql<br/>fresh installs stamp the new number"]
end
subgraph checks["What proves it"]
C1["build/check-schema-versions.ps1<br/>chain contiguous, forward-only,<br/>all four dialects present"]
C2["build/check-db-scripts.ps1<br/>build a real SQL CE database,<br/>run every probe, negative control"]
end
S1 --> C1
S2 --> C1
S3 --> C1
S4 --> C1
ROW --> C1
CONST --> C1
PROBE --> C2
CREATE --> C2
S2 --> C2
Server-wide optional features are INI settings, not database settings. MTA-STS,
DANE, ARC, TLS-RPT, ACME, the REST API, web services, metrics and JSON logging are
all hMailServer.ini [Settings] keys read by IniFileSettings. The pattern for a
new one: a getter in IniFileSettings.h, the default on the member declaration, a
ReadIniSetting*_ call in IniFileSettings.cpp, and a control in the Control
Panel's FeatureSettingsView. Per-account and per-domain settings go in the database
instead. build/check-ini-coverage.py fails the build for an INI key the server reads
and the Control Panel cannot edit, so the last step is not optional.
Two of the optional listeners are now Boost.Asio, two are not. RestApiServer and
WebServicesServer are hosted on Common/Util/HttpServer — an HTTP/1.1 server on
Boost.Asio with its own io_context (separate from the mail listeners', so a request
storm cannot starve SMTP accept), a bounded worker pool (4 threads), a connection cap
(64) and absolute per-request (30 s) and per-connection (300 s) deadlines; a handler
runs on a worker and may block. This is new in 6.2.28; before it, both were raw
sockets and std::thread like the other two. MetricsServer and
ManageSieveServer remain raw sockets and std::thread, outside the TCPIP/
stack. All four are started from Application::StartServers only when their port
is non-zero. Two things to know, both of which have bitten the raw-socket pair:
-
A thread of their own has no exception barrier by default. An exception
escaping the top of a raw listener thread is
std::terminate— the whole mail server dies. Anything that can throw, including any database call, needs atry/catchinside the thread. (HttpServercatches around each handler and answers 500, and again at the top of each worker.) -
They share the mail listeners' TLS configuration. Each builds a
boost::asio::ssl::contextand hands it toSslContextInitializer::InitServer(MetricsServer.cpp:730, RestApiServer.cpp:618, WebServicesServer.cpp:472, ManageSieveServer.cpp:497), so cipher lists, key-exchange groups and protocol floors applied there reach all four; there is no secondSSL_CTXconfiguration to keep in step. The REST listener alone adds a TLS 1.2 floor of its own afterInitServer, which can only tighten what the shared configuration allows.
From Common/Util/HttpServer.h, struct HttpLimits. Every one is absolute — a
deadline from a fixed moment — rather than an idle timeout a client can keep alive by
sending one byte at a time.
| Limit | Default | Measured from | What a client sees when it trips |
|---|---|---|---|
max_request_bytes |
64 KiB | head + body together | 413, refused before the body is read when Content-Length would exceed it |
max_request_bytes_large / request_seconds_large
|
0 / 0 (off) | granted per request by the large-request filter | A larger body, with a longer clock, only for the method+target the filter approves |
request_seconds |
30 s | the moment the server started waiting for the request — on a kept-alive connection, when the previous response was written. Handler time counts | The connection is closed mid-request |
connection_seconds |
300 s | accept | The connection is closed whatever it is doing |
max_requests_per_connection |
1000 | per connection | Closed after the thousandth response, so a client that never disconnects still pays an accept occasionally |
max_connections |
64 | server-wide | Closed at accept, before any TLS handshake, so a flood costs an accept and a close each |
worker_threads |
4 | — | Requests queue behind the four |
Scheduled work uses BO/ScheduledTask and the Scheduler. RunOnce tasks go
through the maintenance work queue immediately; recurring tasks are polled once a
minute. Application::CreateScheduledTasks_ registers, in order:
| Task | Cadence | Registered only when | Source |
|---|---|---|---|
GreyListCleanerTask |
every GreylistingExpirationInterval minutes |
SMTP is on | Application.cpp |
RemoveExpiredRecords |
every minute | always | expired IP ranges |
TlsRptReporterTask |
hourly | always registered; sends only when TlsRptFromAddress is set |
RFC 8460 |
DmarcRptReporterTask |
hourly | always registered; sends only when its *RptFromAddress is set |
DMARC rua
|
LogRetentionTask |
at start, then every 6 h | always | |
ArchiveRetentionTask |
start + 12 h | always | |
MailboxRetentionTask |
start + 6 h | always | message retention, 6.2.25 |
MetricsHistoryTask |
start + every minute | always | 6.2.25 |
UpdateCheckTask |
start + every 15 min | always registered; a no-op until UpdateCheckEnabled=1
|
new in 6.2.28 |
IMAPExpungeRetentionTask |
start + 12 h | always | |
MessageStoreConsistencyTask |
start + hourly | always | |
DiskSpaceMonitorTask |
start + hourly | always | |
WorkQueueHealthTask |
every minute | always | the stall monitor's reporter |
BackupScheduleTask |
a one-minute tick against the wall clock | only when a schedule is set | |
DirectorySyncScheduleTask |
start + every [LDAP] SyncScheduleMinutes
|
only when that is set | |
AcmeRenewalTask |
start + hourly | only when AcmeEnabled=1
|
A startup-plus-periodic pair is the shape for a sweep that must also run on a server
that is only ever restarted (source: Common/Application/Application.cpp:600-860).
These are the ones that cost real releases. They are not stylistic.
Every wait on a pooled thread needs a ceiling. The server runs work on bounded
pools — a 15-thread async queue that also sends the SMTP 250, a 10-thread delivery
queue. A dependency that stops responding consumes threads until none are left, and
then the server accepts mail and never replies. That was
discussion #18, and
once found the same shape turned up in DNS, virus scanning, event scripts, external
processes and outbound delivery. Every one of those now has a bound, and a new one
must arrive with one.
An idle timeout is not a ceiling. Idle timeouts here re-arm on every byte
received, so a peer that dribbles one byte at a time is never idle and holds the
connection — and anything waiting on it — indefinitely. Absolute session ceilings are
a separate mechanism (ClientSessionCeiling, default 1800 seconds,
IniFileSettings.cpp:378) for exactly this reason.
Distinguish "the answer is no" from "there was no answer." A recipient lookup
that fails because the database did not respond used to be indistinguishable from one
that found nothing, so a database locked by a backup told the sender a valid mailbox
did not exist and the mail was bounced. The fix is the thread-local
DatabaseUnavailableMarker with its RAII Scope, read at both RCPT TO decision
points to answer 451 instead of 550. Two non-obvious constraints if you extend
it: set the marker after releasing the pool lock and before ReportError,
because the error path can run an OnError script that re-enters the pool on the
same thread.
shared_from_this() is invalid in a constructor. It throws bad_weak_ptr. Timers
and anything else needing a shared_ptr to the object must be armed from Start(),
not the constructor.
A new diagnostic must not fire on the shipped default configuration. Reporting a
default as an ErrorManager error puts a Medium entry in every stock install's ERROR
log — and fails the regression fixtures, which assert a clean log. If it describes a
default, it is LOG_APPLICATION.
Prefer deferral to bouncing, always. A temporary failure costs a retry. A
permanent one costs someone their mail. The whole 451/452/4.x.x vocabulary in
the flowcharts above exists to serve this one rule; when in doubt about a new failure
path, look at what class the code already assigns to the nearest neighbour and match
its reasoning, not its number.
There are 503 ErrorManager::ReportError call sites in the server - 500 of them with a
literal code, which is what the census below counts - and choosing the
severity and the number for a new one is not a formality: the severity decides what an
administrator sees, and a wrong choice fails the regression fixtures, which assert a
clean ERROR log.
flowchart TB
SITE["ReportError(severity, HM number, source, description)"] --> NORM["Source and description normalised -<br/>control characters out, so a peer's banner<br/>or a FormatMessage string cannot forge a log line"]
NORM --> LINE["ERROR log line:<br/>Severity: n (name), Code: HMnnnn,<br/>Source: ..., Description: ..."]
LINE --> EVT["Windows Event Log, when<br/>WindowsEventLogEnabled is on"]
LINE --> SCRIPT{"An OnError event<br/>script exists?"}
SCRIPT -- yes --> FIRE["OnError(severity, number, source, description)"]
SCRIPT -- "already inside OnError<br/>on this thread" --> GUARD["NOT fired again, and a line says so.<br/>Recursing here would exhaust the stack"]
LINE --> METRIC["Counted; the ERROR log is what the<br/>regression suite's AssertNoReportedError<br/>and AssertReportedError read"]
| Severity | Value | Means | Where it should be used |
|---|---|---|---|
Critical |
1 | The server cannot continue doing something it is supposed to do | 48 sites. Includes HM6364, the violated HM_ASSERT on an -Asserts build |
High |
2 | A real failure with a consequence for mail | 144 sites. HM6081 — a message could not be saved during local delivery — is the shape |
Medium |
3 | Something went wrong and was handled | 299 sites, the majority |
Low |
4 | Worth recording, no consequence | 9 sites |
Number bands, from the same census: 4xxx is inherited from upstream, 5xxx is the
main body, and 6xxx is where this fork's own codes live — 129 call sites so far. A new
code takes the next free number in the 6xxx band, and Common/Application/Errors.txt
is the historical list rather than a live registry (it stops at 126 lines and does not
track the newer codes).
Three rules that are not style preferences:
-
A new diagnostic must not fire on the shipped default configuration. Reporting a
default as an
ErrorManagererror puts a Medium entry in every stock install's ERROR log and fails the fixtures. If it describes a default, it isLOG_APPLICATION. - Do not report on a per-attempt path. An error reported once per failed authentication is a log an attacker can fill at will. Where a condition is a property of stored state rather than of the attempt, report it once per row per server start — the app-password path does exactly this.
-
Do not report on a successful-authentication path either. An unwritable database
during a password-hash upgrade would otherwise report on every login by every affected
account, and
DALConnectionhas already reported the underlying SQL failure, so the ERROR log would carry two entries per login for one root cause.
test/RegressionTests/ is NUnit driving a real running server over real sockets,
with live SpamAssassin and ClamAV, DMARC against live DNS, and real TLS handshakes.
Nothing is mocked. 2,127 tests at 6.2.28. Two things to know before adding a test:
-
RegressionTests.csprojlists every source file explicitly — there is no glob. A test file nobody adds to it is not merely unrun, it is invisible, and a green suite says nothing about it.build/preflight-tests.ps1now fails on orphans; that check exists because a committed test file went uncompiled for months. -
A test that deliberately provokes a reported error must clear the ERROR log, via
CustomAsserts.AssertReportedError(...)which asserts and deletes.PerformBasicSetupcallsAssertNoReportedError, so a provoked error left behind fails whichever unrelated fixture runs next.
A fix does not ship without a test that fails against the build before it. A test that passes both ways proves nothing — build the pre-fix binary and check.
Setting the bench up is Regression Test Environment.
| Artifact | How |
|---|---|
| Server |
build/build.ps1 -Configuration Release (VS 2026, toolset v145, x64) |
| Server, assertion build |
build/build.ps1 -Configuration Release -Asserts — keeps HM_ASSERT, reports a violation as Critical HM6364 instead of compiling it out. Never the binary that ships |
| Admin tools |
build/build-tools.ps1 (source/Tools/hMailServer Tools.sln) |
| Control Panel |
source/Tools/ControlPanel.sln, published separately into its publish/ folder |
| Tests | build/build-tests.ps1 |
| Installer | ISCC on installation/hMailServer64.iss — never on the development machine |
| Fuzzers |
fuzz/build-fuzz.ps1 (clang-cl, a separate tree — see Fuzzing) |
.NET targets. Everything shipped is net10.0-windows, pinned via global.json.
The regression suite and its companions are .NET Framework 4.8.1 and stay there:
4.8.1 has no end-of-support date, and the suite drives the server over COM interop.
Dormant upstream dev tools under tools/ and the unused performance/stress projects
target 4.8.1 or 3.5 and are built by nothing — do not assume they compile.
The SDK is pinned in global.json. There is one build tree and the service holds the
output binary, so stop the service before linking — and never build while a
regression run is in progress.
| Change | Start at | Also touches |
|---|---|---|
| A new SMTP behaviour |
SMTP/SMTPConnection.cpp (reception) or SMTP/ExternalDelivery.cpp (sending) |
SendEHLOKeywords_ if it is advertised |
| A new IMAP command | a new IMAP/IMAPCommand*.{h,cpp} plus the dispatch table and IMAPCommandCapability.cpp
|
RegressionTests.csproj for the test file |
| A server-wide setting |
Common/Application/IniFileSettings.{h,cpp} and the Control Panel's feature settings |
build/check-ini-coverage.py will fail without the editor |
| A per-account or per-domain setting |
Common/BO/, Common/Persistence/, the COM interface, the schema, and the upgrade chain |
four SQL files, DBUpdater, a probe, Constants.h
|
| A new anti-spam test |
Common/AntiSpam/ and the score pipeline |
the per-account override path in LocalDelivery
|
| Something exposed to scripts |
Common/Scripting/ and the COM layer |
the seven places in the COM/IDL checklist |
| A new REST route or portal page |
Common/Util/RestApiServer.cpp (dispatch, authorisation through ACLManager, the served openapi.json); the transport is Common/Util/HttpServer — new in 6.2.28 |
build/check-authz-choke-point.py keeps folder-access decisions in ACLManager
|
| A new periodic sweep | a ScheduledTask subclass registered in Application::CreateScheduledTasks_, startup-plus-periodic |
|
| A new Control Panel caption | the view, plus L("_Caption") marking |
check-mnemonics.py, check-localisation.py --write, and all 17 catalogues |
If a change spans more than about three of those rows, it is worth discussing in an issue before writing it.
hMailServer 6.3.2 · AGPL-3.0-or-later · Repository · Report a documentation error
Hmail Server — full index
Start here
1. Install and run
- Before You Install
- Installing hMailServer
- Installing on Linux
- Running in a Container
- The Control Panel
- Your First Domain and Mailbox
- Connecting a Mail Client
- DNS for Your Domain
2. Secure it
3. Operate it
- Monitoring and Health
- Backup and Restore
- Troubleshooting
- Diagnosing Stalled Mail
- Relocating an Installation
- Upgrading hMailServer
- Upgrading Guide
- Migrating the Database Backend
- High Availability Runbook
- Warm Standby
- Runbooks Digest
4. Extend it
- Rules and Sieve
- Aliases Lists and Public Folders
- Routes and Relays
- The COM API and Scripting
- The REST API
- APIs Reference
5. Contribute to it
- Project Handbook
- Architecture
- Contributing
- Release Process
- Governance
- Assurance Case
- Regression Test Environment
- Fuzzing
- Regulatory Scope
- Third-Party Binaries
Look it up — from any journey