Skip to content

Stopping Viruses

chrisholloway5 edited this page Sep 8, 2026 · 3 revisions

Stopping Viruses

This page began as a chapter of the 6.2.10 manual and has been corrected against the code of 6.2.28. The Control Panel's pages are grouped differently now, so the paths below use today's groups; the TLS ports 465, 993 and 995 exist only after you create them on the TCP/IP ports page (a fresh install seeds 25, 587, 110 and 143); and everything added since 6.2.10 is in Changes-Since-6210. Where a value here disagrees with the Settings Reference, which is generated from the code, the reference is right.

hMailServer does not scan for viruses itself. It hands each message to an engine you install — ClamAV, ClamWin, or any command-line scanner — and acts on the answer. Nothing here is enabled on a fresh install.

Open Spam & virus filtering → Anti-virus settings:

The Anti-virus settings page, General tab, showing the action for an infected message, the sender and receiver notification switches, the maximum size to scan and the attachment-blocking switch

Read this first if you already have it configured. Spam & virus filtering → Virus scanning overview is a read-only page that answers one question the settings page cannot: will an infected message actually be caught here? It exists because the failure mode is a configuration that looks enabled and is inert — a scanner switched on with a wrong host or a missing executable does not refuse mail and does not disable itself. VirusScanner::ScanFile_ reports the error and moves on, and when every scanner has errored the verdict is NoVirusFound, which from the message's point of view is identical to having been examined and found clean (Tools/ControlPanel/Views/VirusOverviewView.cs).


11.1 What runs, and when

Scanning happens on the delivery side, not during the SMTP conversation — the message has already been accepted and spooled by the time any scanner sees it (Server/SMTP/SMTPDeliverer.cpp).

flowchart TD
    A["Message spooled for delivery"] --> B{"Attachment blocking enabled?"}
    B -- yes --> B2["Matching attachments renamed to .txt<br/>and their content replaced"]
    B -- no --> C
    B2 --> C{"Was this message flagged<br/>for virus scanning?"}
    C -- no --> DELIVER["Deliver"]
    C -- yes --> D{"Any scanner enabled?"}
    D -- no --> DELIVER
    D -- yes --> E{"Message larger than<br/>the maximum size to scan?"}
    E -- yes --> DELIVER
    E -- no --> F["Take one of 10 scanner slots<br/>waiting up to 60 seconds"]
    F --> G["Scan the whole spool file with<br/>every enabled scanner in turn"]
    G --> H{"Virus found?"}
    H -- yes --> ACT["Act on it"]
    H -- no --> I["Write each MIME attachment to a<br/>temporary file and scan that too"]
    I --> J{"Virus found?"}
    J -- yes --> ACT
    J -- no --> K{"Did any enabled scanner<br/>report an error?"}
    K -- no --> DELIVER
    K -- yes --> L{"AVFailAction"}
    L -- "0, the default" --> DELIVER
    L -- 1 --> HOLD["Hold the message and retry later"]
    ACT --> M{"Action on virus found"}
    M -- "delete entire e-mail" --> DEL["Message destroyed,<br/>optional notifications sent"]
    M -- "delete infected attachments" --> STRIP["Message rebuilt from its<br/>first text part and delivered"]
Loading

Which messages are flagged for scanning. For SMTP, the flag is set when the connecting IP's matching security range has the virus-protection option — both default ranges ("Internet" and "My computer") have it. For mail this server fetches over POP3 or IMAP, the flag comes from the external account's own "use anti-virus" setting.

Two passes, deliberately. The whole spool file is scanned first; then every MIME attachment is written to a temporary file in the temp directory and scanned separately. The second pass is what matters for a scanner that does not decode MIME itself. Two failures on that path are reported rather than made fatal:

Code When Consequence
HM6001 The message could not be parsed for the per-attachment pass Only the whole file was scanned; anything the scanner cannot decode itself was not examined separately
HM6002 One attachment could not be written to the temp directory That attachment was not examined; the rest of the message was

Concurrency. At most 10 scans run at once (MaxRunningScanners). A thread that cannot get a slot waits up to 60 seconds and then scans anyway, over the cap, writing Scanning message without an available virus scanner slot. to the debug log. That is a deliberate choice: bouncing or holding mail because the scanners are busy is worse than briefly exceeding the cap.


11.2 The three scanners

All three can be enabled at once. VirusScanner::ScanFile_ runs them in this order — ClamWin, custom, ClamAV — and stops at the first that reports a virus. A scanner that errors does not stop the others; it sets a flag that section 11.5 acts on.

ClamAV (clamd) ClamWin (clamscan.exe) Custom scanner
Switch AntiVirus.ClamAVEnabled AntiVirus.ClamWinEnabled AntiVirus.CustomScannerEnabled
Default off off off
How it is reached TCP socket, INSTREAM one process per file one process per file
Host / path setting AntiVirus.ClamAVHost (localhost), AntiVirus.ClamAVPort (3310) AntiVirus.ClamWinExecutable, AntiVirus.ClamWinDBFolder AntiVirus.CustomScannerExecutable
"Infected" is a reply matching stream…: <name> FOUND exit code 1 exit code equal to AntiVirus.CustomScannerReturnValue
Timeout ClamMinTimeout 15 s to ClamMaxTimeout 90 s, moving with load ExternalProcessTimeout, 300 s ExternalProcessTimeout, 300 s
Virus name reported the name clamd returned always Unknown always Unknown
Test button Test ClamAV connection Test ClamWin scanner and Auto-detect ClamWin Test custom scanner

Shared settings, on the General tab:

Setting Default What it does What it looks like when it is wrong
AntiVirus.Action Delete entire e-mail See 11.4
AntiVirus.NotifySender off Sends the VIRUS_NOTIFICATION server message to the envelope sender On: you generate backscatter. Leave it off
AntiVirus.NotifyReceiver off Sends the same notification to each recipient On: users are told about mail they will never see
AntiVirus.MaximumMessageSize 0 (no limit) Messages above this many KB are not scanned at all, silently A generous-looking limit is a hole: a virus in a 30 MB archive is delivered unscanned with no log line

The custom scanner's return value has a misspelled name in the database. The hm_settings row is customviursscannerreturnvalue, not custom*virus*… (Constants.h:50, and the seed rows in every CreateTables*.sql). It matters only if you edit the table by hand.

ClamAV: the conversation

sequenceDiagram
    participant H as hMailServer
    participant C as clamd
    H->>C: nINSTREAM, terminated by a newline
    loop 4096-byte chunks of the file
        H->>C: 4-byte length, big-endian
        H->>C: chunk bytes
    end
    H->>C: 4 zero bytes, end of stream
    C-->>H: stream: Eicar-Signature FOUND
    Note over H: The reply is matched against a<br/>regular expression ending in FOUND
Loading

Install ClamAV, run freshclam to download signatures, and run clamd listening on TCP 3310. Then set the host and port and press Test ClamAV connection. That button does three things in order (Server/Common/AntiVirus/VirusScannerTester.cpp):

  1. sends nPING and requires PONG — so "nothing is listening" is told apart from "something else is listening";
  2. sends nVERSION and reports what answered, so you know which clamd you reached;
  3. streams a harmless sample and then the EICAR test string, both from memory, and requires the first to come back clean and the second to come back FOUND.

Streaming from memory rather than from a file is deliberate: a host antivirus watching the data directory deletes an EICAR file the instant it is written, and the test then reported a missing file instead of anything about clamd. The EICAR string is even held reversed in the binary so that hMailServer.exe does not itself carry it.

A success looks like:

ClamAV 1.4.2/27700/Mon Sep  8 09:12:11 2026. Result: stream: Eicar-Signature FOUND

A failure names the step that failed, for example Unable to connect to ClamAV server at localhost:3310. or clamd at localhost:3310 answered PING with 'HTTP/1.1 400 Bad Request' rather than PONG.

The custom scanner

Any program that exits with a distinguishable code will do. Put %FILE% where the file to scan belongs; without it, the file name is appended as the single quoted argument. Both the executable and the path are quoted, so spaces in either are safe.

The Control Panel ships presets that fill in the command line and the exit code — adjust the path and the code for your version:

Preset Command line Exit code meaning "infected"
Microsoft Defender "C:\Program Files\Windows Defender\MpCmdRun.exe" -Scan -ScanType 3 -File "%FILE%" -DisableRemediation 2
Sophos "C:\Program Files\Sophos\Sophos Anti-Virus\savscan.exe" -ss -archive "%FILE%" 3
ESET "C:\Program Files\ESET\ESET Security\ecls.exe" "%FILE%" 50
Bitdefender "C:\Program Files\Bitdefender\Endpoint Security\bdscan.exe" "%FILE%" 1
Kaspersky "C:\Program Files (x86)\Kaspersky Lab\Kaspersky Endpoint Security\avp.com" SCAN "%FILE%" 2

For an engine with no command-line interface — an HTTP or API scanner, a DLP product, a SIEM — use an OnAcceptMessage event script instead (The COM API and Scripting), or the HTTP filter hook described in Stopping Spam §10.9.

Whatever you configure, the service account has to be able to run it. The service runs as LocalSystem by default; if you have moved it to a least-privilege account (Security Hardening §17.3) the scanner executable, its database directory and the temp directory all have to be reachable by that account. The symptom of getting this wrong is Unable to launch the virus scanner… reported as HM5406, once per message.


11.3 Testing it end to end

The Control Panel's three test buttons check the scanner. To check the pipeline, send yourself a message with the EICAR test file attached and watch the application log:

Virus found in message from someone@example.com. Taking actions
SMTPDeliverer - Message 4711: Message will be deleted (contained virus Eicar-Signature).

Two things routinely spoil that test on Windows:

  • The host antivirus eats the EICAR file before hMailServer sees it. Exclude the mail store and the temp directory from real-time scanning — the scanner you configure here replaces real-time scanning for mail.
  • Your own outbound mail is not scanned by this server if it never comes back in through SMTP from an unauthenticated range. Send it from an external address.

11.4 What happens when a virus is found

Action on virus found (AntiVirus.Action, default Delete entire e-mail):

Action What actually happens
Delete entire e-mail (0) The message is not delivered to anybody. If Notify receiver is on, each recipient gets the VIRUS_NOTIFICATION server message; if Notify sender is on, so does the envelope sender. Logged as Message will be deleted (contained virus X).
Delete infected attachments only (1) The message is rebuilt, not edited: the first text part is kept, everything else is discarded, the VIRUS_ATTACHMENT_REMOVED server message is prepended to the body, and the subject is prefixed with the VIRUS_FOUND message ("Virus found"). Logged as Message attachments stripped (contained virus X).

The rebuild is worth understanding before you choose it. MessageAttachmentStripper::Strip keeps one text part and drops the rest of the MIME structure, so an HTML alternative, inline images and every clean attachment go with the infected one. If the message cannot be parsed at all it is left exactly as it was, with a line in the application log saying so.

All three notification texts are editable under Mail flow & delivery → Server messages (VIRUS_FOUND, VIRUS_ATTACHMENT_REMOVED, VIRUS_NOTIFICATION).

Do not configure a bounce for virus mail. The sender address on a virus is virtually always forged, so a notification just sends junk to an innocent third party — "backscatter", and a good way to get blacklisted. Both notification switches are off by default for exactly this reason.


11.5 When a scanner cannot run

This is the question the settings page cannot answer, and until AVFailAction existed the answer was "deliver it, with one line in the error log" — indistinguishable from "scanned and found clean".

The three settings are on Anti-virus settings → General, card When a scanner cannot run, and stored in hMailServer.INI:

Setting Default Meaning
AVFailAction 0 0 = deliver the message anyway (the historical behaviour). 1 = never deliver unscanned mail
AVFailRetryMinutes 15 Minutes between scan attempts while a message is held
AVFailMaxHolds 16 How many holds before the message is returned to its sender. 0 = never queue unscanned mail at all — tell the sender immediately

At the defaults, 1 gives a message about four hours to be scanned before its sender is told.

stateDiagram-v2
    [*] --> Scanning
    Scanning --> Delivered: a scanner answered, no virus
    Scanning --> Acted: a scanner answered, virus found
    Scanning --> Unscannable: every enabled scanner errored
    Unscannable --> Delivered: AVFailAction is 0
    Unscannable --> Held: AVFailAction is 1 and holds remain
    Held --> Scanning: retried after AVFailRetryMinutes
    Held --> Returned: AVFailMaxHolds reached
    Acted --> [*]
    Delivered --> [*]
    Returned --> [*]
Loading

Three properties of the hold counter that change how you should configure it:

  • It is not the SMTP retry budget, the message's age, or the message's own try counter. It is a counter this policy keeps for itself, because incrementing the try counter would make the pass that finally scans the message skip DKIM signing — and a message that goes out unsigned fails DMARC at every receiver that checks.
  • It lives in memory. A service restart gives every held message a fresh budget. At the shipped defaults (16 × 15 minutes) a nightly reboot cannot reach the budget anyway; a much longer hold window on a frequently restarted host can mean a sender is never told.
  • It is bounded at 100,000 tracked messages. Beyond that, messages are still held — never delivered unscanned — but their holds are not counted, and error HM6195 is reported repeatedly while that lasts.

A hold is logged, with the message id and the hold number:

SMTPDeliverer - Message 4711: The virus scanner could not examine this message, and
AVFailAction is 1, so it has NOT been delivered. Holding it for another attempt in 15
minute(s); this was hold 3 of 16 before it is returned to the sender.

Every scanner error itself is reported as HM5406, with the failing scanner's own text — which distinguishes a timeout ("did not finish within the maximum time (ExternalProcessTimeout) and was terminated") from a launch failure ("Check that the executable path is correct and runnable by the service account") from an unreachable clamd.


11.6 Blocked attachments

Separate from virus scanning, older than it, and worth using regardless: Spam & virus filtering → Blocked attachments refuses files by extension. The master switch is AntiVirus.EnableAttachmentBlocking on the anti-virus page, default off; the list itself is on its own page.

The Blocked attachments page, one wildcard pattern per row with a description

A sound starting list — no legitimate correspondent emails you an .exe:

*.exe   *.scr   *.pif   *.bat   *.cmd   *.com   *.vbs
*.js    *.jse   *.wsf   *.wsh   *.hta   *.jar   *.msi
*.msp   *.cpl   *.lnk   *.reg   *.ps1   *.iso   *.img

Two facts about how it behaves that are not obvious from the page:

  • The attachment is not deleted. Its file name has .txt appended and its content is replaced by the ATTACHMENT_REMOVED server message, which names the original file via the %MACRO_FILE% macro. The recipient therefore receives invoice.exe.txt containing an explanation, which is usually what you want — the mail still arrives, and the user can see what was removed and ask for it another way.
  • Matching is a case-insensitive wildcard over the declared file name. It does not look inside archives, and it cannot see the real type of a file whose extension lies. It is a policy control, not a scanner — which is why it is worth running alongside one rather than instead.

11.7 Error codes for this page

Code Source Meaning
HM5406 VirusScanner::ReportScanningError_ An enabled scanner errored. The detail text names the scanner and the failure
HM6001 VirusScanner::Scan The message could not be parsed, so only the whole file was scanned
HM6002 VirusScanner::Scan An attachment could not be written to the temp directory for scanning
HM6195 SMTPDeliverer::HandleUnscannableMessage_ 100,000 messages are already held for want of a scan; further holds are not counted

See also: Stopping Spam · Security Hardening · Monitoring and Health · Settings Reference · Diagnosing Stalled Mail · The COM API and Scripting


Clone this wiki locally