Repository navigation
Regression Test Environment
The regression suite (hmailserver/test/RegressionTests, roughly 1,800 tests)
does not mock the server. It runs against a real hMailServer service on the
same machine, over SMTP, IMAP and POP3, and through the COM API, and it
creates and deletes domains, accounts and mail as it goes. That makes it worth
far more than a unit suite, and it makes the machine it runs on part of the
test. This document is how to build that machine.
build/preflight-tests.ps1 is the machine-readable version of everything
below. Every check in it corresponds to a failure that has actually cost a run,
and when a step here is missing the pre-flight names it. Run it before every
run; Pre-flight passed - safe to run the suite is the state this document
gets you to.
Never run the suite against a production installation. It authenticates as
Administrator, binds the live ports, and wipes data.
| Piece | What it is for |
|---|---|
| The service, running from the repository's Release build | The code under test. Not an installed copy. |
| SQL Server Compact 4.0 and the bench database | The default backend; the suite upgrades it in place, never recreates it. |
| A short data folder | Long account names overrun MAX_PATH under a deep checkout. |
| The test domain and the four standard ports | What every fixture assumes exists at start. |
ClamAV (clamd on 3310) |
The live-scanner tests. |
SpamAssassin as a Windows service named SpamAssassinJAM
|
The SpamAssassin tests, including the one that stops and restarts it. |
| The NUnit console runner and the built test assembly |
build/build-tests.ps1 produces both. |
- The service runs from the repository build
Build the server (build/build.ps1 -Configuration Release), then run
build/post-build.ps1 -Configuration Release. It needs elevation and will ask
for it. It copies the runtime DLLs beside hMailServer.exe, registers the COM
server and the Windows service, and points the service at
hmailserver\source\Server\hMailServer\x64\Release\hMailServer.exe in your
checkout.
The pre-flight's first check reads the service's PathName and refuses anything
else. The failure it guards against is a stray installer: running the installer
on the development machine re-points the service at C:\Program Files\..., and
from then on every "green" run tests the installed binary, not the one you just
built. If that has happened, post-build.ps1 again, then re-apply the
permission grant in step 7, then delete HKLM\SOFTWARE\hMailServer in both
registry views (reg delete "HKLM\SOFTWARE\hMailServer" /reg:32 /f and the
same with /reg:64) - a leftover InstallLocation value redirects the server's
INI lookup and it comes up with empty configuration while still reporting
"Running". build/make-hmailserver-writable.ps1 deals with the file
permissions the build needs on the output directory.
Provision the output directory as the installed layout expects
(IMPLEMENTATION-NOTES.md, "Test-environment recipe", has the full list):
hMailServer.ini with a [Directories] section and an empty
AdministratorPassword, the Logs, Temp and Events folders, a copy of
DBScripts\ with CRLF line endings, Languages\, dh2048.pem and tlds.txt
from installation\Extras, and a Bin\ folder holding 7za.exe and a second
copy of hMailServer.ini.
- SQL Server Compact and the bench database
Install the 4.0 x64 package from hmailserver\installation\SQLCE (it is the
version SQLCEConnection binds to; the x86 package is neither needed nor
installable on x64). Create the database once, through COM:
Database.CreateInternalDatabase() followed by Reinitialize() on an
hMailServer.Application object authenticated as Administrator.
From then on the database is upgraded, never recreated. REQUIRED_DB_VERSION
in Server/Common/Application/Constants.h moves whenever a release adds a
column, and a bench that nobody upgraded fails in a way that points somewhere
else entirely: the service refuses the database connection, the first pre-flight
check says "could not start", the ERROR-log check finds a stale error, and
neither says the word "database". The pre-flight's fourth check exists to say it.
Move the bench forward with build/upgrade-test-database.ps1 before
building a server whose required version has moved past it.
Because the bench is always upgraded, no test here takes the fresh-install path
that CreateTablesMSSQL.sql provides. build/check-db-scripts.ps1 covers that
separately by creating a throwaway database from the script; RELEASE.md makes
it a release step for exactly that reason.
- A short data folder
Point the data directory at something short, C:\HMTest\Data is the
convention the pre-flight knows. The suite creates accounts with deliberately
long names, and under a deeply nested checkout path the resulting file names
exceed MAX_PATH. The symptom is not a clear error but Windows error 206
cascading into hundreds of clean-log assertion failures in tests that never
mention paths.
- The test domain and the standard ports
The suite expects exactly one domain and exactly four TCP/IP ports at start (SMTP 25, POP3 110, IMAP 143 and the SMTP submission port). The pre-flight reads both counts from the running server. A port count of sixteen is a signature, not a mystery: the TLS fixtures register twelve extra ports and remove them in teardown, so a run killed while one of them was in flight leaves them behind. Remove the non-standard ones in the Control Panel, or re-add the four if there are fewer.
The suite authenticates over COM as Administrator with the password
testar, falling back to a blank password. Set it to testar. It is a bench
credential on a machine that must never hold real mail, which is why it appears
in the scripts in plain text and why it is not a secret.
- ClamAV
Install ClamAV (winget install Cisco.ClamAV works) and copy the installation
to C:\clamav - the suite's CustomAsserts hard-codes C:\clamav\clamd.exe.
A minimal clamd.conf is enough: TCPSocket 3310, TCPAddr 127.0.0.1,
DatabaseDirectory C:\clamav\database. Run freshclam once, then start
clamd.exe. It runs as a bare process, not a Windows service, so the pre-flight
checks for a listener on 3310 rather than for a service.
Warm-up matters: clamd accepts connections before its signatures have
finished loading, and hMailServer fails open when the scanner gives no verdict.
Before trusting the anti-virus fixtures on a freshly started clamd, send one
EICAR message and confirm "Virus detected" in the log.
- SpamAssassin
The JAM Software x64 build works
(https://downloads.jam-software.de/spamassassin/SpamAssassinForWindows-x64.zip,
rules bundled under share\). Extract it to C:\SpamAssassin and wrap
spamd.exe -i 127.0.0.1 -A 127.0.0.1 -p 783 as a Windows service named
SpamAssassinJAM - WinSW does this well, and the child process keeps the
name spamd, which both the process-gate check and the fixture that stops and
restarts the service rely on.
- Let the unelevated test runner control the services
One fixture stops and restarts SpamAssassinJAM; rebuilding the server stops
and starts hMailServer. Neither should need an elevated console. Grant
Authenticated Users start/stop on both services with sc sdset (the exact
descriptor is in IMPLEMENTATION-NOTES.md). The grant is silently dropped when
a service is re-registered, which is why "Access denied" starting the service
after a post-build.ps1 is the pre-flight's own diagnosis for that failure.
- Build the tests
build/build-tests.ps1 restores the NuGet packages into hmailserver\test\packages
- where the project's hint paths point, which a bare
dotnet restoreon the project does not do - and buildsRegressionTests.dllfor x64. Always x64: an AnyCPU build of the same project also runs, against stale assemblies, and passes tests it never actually executed.
RegressionTests.csproj is a legacy project that lists every source file
explicitly. A new test file that is not added to it is not merely unrun, it is
invisible, and a green suite says nothing about it. The pre-flight compares the
directory against the project and names any orphan.
- Run it
build/preflight-tests.ps1, then build/run-tests.ps1. The runner is
nunit3-console; subsets take -Where "class =~ /RegressionTests.IMAP.Binary/".
NUnit writes its failure details only at the end of the run, so a run that
looks quiet for fifty minutes is normal.
Do not dotnet test this project. It exits 0 having run nothing.
Never build, run static analysis, or start other heavy work while the suite is running. A rebuild restarts the service under the tests, the fixtures' restart detector then fails every test that follows, and the run can neither convict nor exonerate the code - thirty minutes are simply thrown away.
A run that is stopped part-way leaves state behind, and the next run fails in
ways that point at the wrong thing. RELEASE.md step 4 says never abort a run for
this reason; when it happens anyway, build/preflight-tests.ps1 -Clean removes
what it can and names the rest:
- A stale ERROR log. One fixture writes a deliberate scanner error and removes it in teardown. Left behind, it fails every fixture's setup in the next run - the observed shape is 100% of tests failing before doing anything.
-
Test-only settings left in
hMailServer.ini. The suite points the resolver at its own fake DNS server on 127.0.0.1 for the whole run (Shared/SuiteDns.cs, a[SetUpFixture]: NODATA for every name a fixture has not added, the SURBL test point and localhost seeded) and putsDNSServerback when the run ends; fixtures that enable the quarantine, set a POP3 login delay, a password policy, PROXY-protocol trust, or an OTLP endpoint put their key back in teardown. Any of them surviving fails tests that never mention the setting: every DNS lookup waits for the query timeout; every anti-spam refusal becomes an acceptance; every connection from the trusted address is dropped before the banner. The server caches the INI at start, so after-Cleanrestart the service. - Orphan domain directories in the data folder. Several persistence fixtures rename the test domain, and the rename moves its directory. Killed mid-rename, the directory survives with no domain row, and every later run of that fixture fails with "already exists" about a directory it never named.
- The twelve extra TLS ports described in step 4.
Each of these has produced a red run that looked like a code regression.
-
Anti-virus products that proxy localhost mail traffic in-process. ESET,
for one, answers an IMAP literal continuation itself with
+ just send itand rewrites raw SMTP framing. A failure whose text contains that string is the proxy, not the server, and it returns after a reboot. Disable mail and SSL protocol filtering for localhost. - VPN clients that rewrite loopback. ProtonVPN and WireGuard adapters have broken address selection in fixtures that bind a specific local address. The pre-flight warns when one is up.
-
Line endings. The repository checks out CRLF on purpose (
.gitattributes): the database scripts are split on\r\n\r\n, the DKIM-signed.emlresources have body hashes that change with the line ending, and raw-message tests send file content verbatim and are refused for bare LF. A tool that "normalises" the tree to LF breaks all three at once. - Mark-of-the-Web. Files downloaded rather than cloned carry a zone identifier that the service refuses to load. Strip it from the whole tree.
- "Passes alone, fails in a group." This points at shared state between fixtures - an account, a setting, a port left behind by the fixture that ran before - and not at the network, however network-shaped the failure message is. Look at what the preceding fixture in the run order changes.
- A build number that names a tree that no longer exists. Not an environment trap exactly, but the same lesson: the suite's verdict is about the binary the service is running, which is why the pre-flight checks the path and why RELEASE.md voids a gate the moment anything changes after it.
hMailServer 6.3.8, the latest release (2 November 2026, schema 6108) · AGPL-3.0-or-later · Repository · Mirror on GitHub · Report a documentation error (a GitLab issue naming the page)
Home — full index
Start here
1. Install and run
- Before You Install
- Installing hMailServer
- Installing on Linux
- Running in a Container
- Profiles and Environment
- The Control Panel
- The Control Deck
- Your First Domain and Mailbox
- Connecting a Mail Client
- The Webmail
- DNS for Your Domain
- Migration
2. Secure it
- Encryption and Certificates
- Database TLS
- Encryption for People
- Stopping Spam
- Stopping Viruses
- Security Hardening
- Secure Defaults
- The Service Account
- Secrets at Rest
- Sign-in and Identity
- Security Policy
3. Operate it
- Monitoring and Health
- The SNMP Agent
- Backup and Restore
- Message Storage
- Compliance and eDiscovery
- Troubleshooting
- Diagnosing Stalled Mail
- Relocating an Installation
- Upgrading hMailServer
- Upgrading Guide
- The Update Feed
- Migrating the Database Backend
- High Availability Runbook
- Warm Standby
- Runbooks Digest
4. Extend it
- Rules and Sieve
- Aliases Lists and Public Folders
- Rooms and Equipment
- Mailing Lists
- Routes and Relays
- Mail In to a Webhook
- Outbound Shaping
- The COM API and Scripting
- Event Scripts
- The REST API
- Command Line
- APIs Reference
5. Contribute to it
- Project Handbook
- Architecture
- Contributing
- Release Process
- Governance
- Assurance Case
- Regression Test Environment
- Fuzzing
- Regulatory Scope
- Third-Party Binaries
Look it up — from any journey
- Settings Reference
- Settings Overview
- Capabilities and Configuration
- Control Panel Reference
- Control Panel Pages
- Ports Reference
- Changes-Since-6210
- Version-History-to-6210
- Glossary
About this wiki
- hMailServer 6.3.8 is the latest release (2 November 2026, schema 6108)
- Pages say since 6.3.4 where a feature is new in it
- AGPL-3.0-or-later
- Repository, the one place the project is worked on
- Its mirror on GitHub, since 18 October 2026, with no Issues
- Questions: a GitLab issue with the Question template, GitHub Discussions Q&A, or the forum
- Report a documentation error: a GitLab issue naming the page