[BUG] Symbol or soft reject when clamd unix socket is down

[stephdl]

Prerequisites

• [x ] Put an X between the brackets on this line if you have done all of the following:
  • Read about bug reporting in general: https://rspamd.com/doc/faq.html#how-to-report-bugs-found-in-rspamd
  • Enabled relevant debugging logs: https://rspamd.com/doc/faq.html#how-to-debug-some-module-in-rspamd
  • Checked the FAQs about Core files in case of fatal crash: https://rspamd.com/doc/faq.html#how-to-figure-out-why-rspamd-process-crashed
  • Checked that your issue isn't already filed: https://github.com/issues?utf8=%E2%9C%93&q=is%3Aissue+user%3Arspamd
  • Checked that there is not already an experimental package or master branch

Describe the bug
Nethserver (CentOS clone), uses when it is possible the Unix socket to communicate with the services rather a TCP socket. We have an issue when the Unix socket of Clamav is not existing anymore, Rspamd announces the failed status in maillog about the dead unix socket, but the Clamav lua script scanner does not add the symbol CLAM_VIRUS_FAIL.

Steps to Reproduce

1. [First Step]
   set clamd to create a unix socket in /etc/clamd.d/rspamd.conf
LocalSocket /var/run/clamd@rspamd/clamav

Use clamd with a TCP socket in /etc/rspamd/local.d/antivirus.conf
servers = "/var/run/clamd@rspamd/clamav";

2. [Second Step]
   stop clamd manually
   systemctl stop clamd@rspamd

3. [third step]
   Send you an email with an attachment that it will be scanned, you have the full log transaction

May 14 15:11:31 prometheus rspamd[5810]: <>; ; rspamd_inet_address_connect: connect unix:/var/run/clamd@rspamd/clamav failed: 2, 'No such file or directory'
May 14 15:11:31 prometheus rspamd[5810]: <603dcf>; proxy; lua_tcp_make_connection: cannot connect to /var/run/clamd@rspamd/clamav: No such file or directory

The Clamav lua script is not called I suppose because Rspamd failed to connect to the unix socket of clamd

Expected behavior
When you use a TCP socket, Rspamd expects that the service could not be reachable, then if the Clamd service does not answer, the SYMBOL CLAM_VIRUS_FAIL is added (this symbol can be used to do a force_action after). But when Rspamd cannot connect to the Unix socket of Clamd, apart a maillog warning, nothing is done.

We would like to make a soft reject of emails if Rspamd cannot contact Clamd, this is our security mail policy.

The simple workaround is to use now a TCP socket with Clamd, but we wonder if it is not an issue to fix, for example if Postfix cannot use the Unix socket of Rspamd, the policy is to soft reject all received emails.

Versions

CENTOS_7.6
RSPAMD 1.9.2-1

Additional Information

• clamd settings

[root@prometheus ~]# cat /etc/clamd.d/rspamd.conf 
# Use system logger.
LogSyslog yes

# Specify the type of syslog messages - please refer to 'man syslog'
# for facility names.
LogFacility LOG_MAIL

# This option allows you to save a process identifier of the listening
# daemon (main thread).
PidFile /var/run/clamd@rspamd/clamav.pid

# Remove stale socket after unclean shutdown.
# Default: disabled
FixStaleSocket yes

# Run as a selected user (clamd must be started by root).
User _rspamd

# Path to a local socket file the daemon will listen on.
LocalSocket /var/run/clamd@rspamd/clamav

#restrict permission
LocalSocketMode 770

• rspamd antivirus.conf

[root@prometheus ~]# cat /etc/rspamd/local.d/antivirus.conf 
# ================= DO NOT MODIFY THIS FILE =================
# 
# Manual changes will be lost when this file is regenerated.
#
# Please read the developer's guide, which is available
# at NethServer official site: https://www.nethserver.org
#
# 

#Enable or disable the module 
enabled = true

# multiple scanners could be checked, for each we create a configuration block with an arbitrary name
clamav {
  # If set force this action if any virus is found (default unset: no action is forced, 'rewrite_subject' to tag as spam)
  action = "reject";

  # if `true` only messages with non-image attachments will be checked (default true)
  scan_mime_parts = false;

  # If `max_size` is set, messages > n bytes in size are not scanned
  max_size = 20000000;

  # type of scanner: "clamav", "fprot", "sophos" or "savapi"
  type = "clamav";

  # If set true, log message is emitted for clean messages
  #log_clean = false;

  # servers to query (if port is unspecified, scanner-specific default is used)
  # can be specified multiple times to pool servers
  # can be set to a path to a unix socket
  servers = "/var/run/clamd@rspamd/clamav";

  # if `patterns` is specified virus name will be matched against provided regexes and the related
  # symbol will be yielded if a match is found. If no match is found, default symbol is yielded.
  patterns {
    # symbol_name = "pattern";
    CLAMAV_VIRUS = "^Eicar-Test-Signature$";
  }

  # In version 1.7.0+ patterns could be a list for ordered matching
  #patterns = [{SANE_MAL = "Sanesecurity.Malware.*"}, {CLAM_UNOFFICIAL = "UNOFFICIAL$"}];

  # `whitelist` points to a map of IP addresses. Mail from these addresses is not scanned.
  whitelist = "/etc/rspamd/antivirus.wl";
}

[stephdl]

hi @vstakhov

We wonder when the fix will be released, thank in advance

[vstakhov]

In Rspamd 2.0. ETA ~2 months, might be more.

[stephdl]

For the record what do you plan to make, add a symbol when the Linux socket is down or reject softly the emails that we cannot scan ?

The symbol should be the most common way I think, thank again for your inputs.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stephdl]

please don't close until rspamd 2.0

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stephdl]

need to check if V2 solves this issue...really thank you

[stephdl]

this seems to be fixed with rspamd v2.1 a big really thank you

Nov 21 14:55:24 prometheus postfix/smtpd[27064]: connect from mail-eopbgr20059.outbound.protection.outlook.com[40.107.2.59]
Nov 21 14:55:24 prometheus rspamd[27012]: <8d58fc>; proxy; proxy_accept_socket: accepted milter connection from /var/run/rspamd/worker-proxy port 0
Nov 21 14:55:24 prometheus postfix/smtpd[27064]: 7DDF919053062: client=mail-eopbgr20059.outbound.protection.outlook.com[40.107.2.59]
Nov 21 14:55:24 prometheus rspamd[27012]: <8d58fc>; milter; rspamd_milter_process_command: got connection from 40.107.2.59:32773
Nov 21 14:55:24 prometheus postfix/cleanup[26514]: 7DDF919053062: message-id=<359e5f84-3382-9771-292a-987b30b51db4@chubbfrance.com>
Nov 21 14:55:27 prometheus rspamd[27012]: <8d58fc>; proxy; rspamd_message_parse: loaded message; id: <359e5f84-3382-9771-292a-987b30b51db4@chubbfrance.com>; queue-id: <7DDF919053062>; size: 126875; checksum: <48057c6549e4ef988c32693c81d9ce9a>
Nov 21 14:55:28 prometheus rspamd[27012]: <8d58fc>; proxy; rspamd_mime_part_detect_language: detected part language: fr
Nov 21 14:55:28 prometheus rspamd[27012]: <8d58fc>; proxy; dkim_module_key_handler: stored DKIM key for selector2._domainkey.Chubbfrance.com in LRU cache for 3393 seconds, 1/2000 elements in the cache
Nov 21 14:55:28 prometheus rspamd[27012]: <>; ; rspamd_inet_address_connect: connect unix:/var/run/clamd@rspamd/clamav failed: 2, 'No such file or directory'
Nov 21 14:55:28 prometheus rspamd[27012]: <8d58fc>; proxy; lua_tcp_make_connection: cannot connect to /var/run/clamd@rspamd/clamav (/var/run/clamd@rspamd/clamav): No such file or directory
Nov 21 14:55:28 prometheus rspamd[27012]: <>; ; rspamd_inet_address_connect: connect unix:/var/run/clamd@rspamd/clamav failed: 2, 'No such file or directory'
Nov 21 14:55:28 prometheus rspamd[27012]: <8d58fc>; proxy; lua_tcp_make_connection: cannot connect to /var/run/clamd@rspamd/clamav (/var/run/clamd@rspamd/clamav): No such file or directory
Nov 21 14:55:28 prometheus rspamd[27012]: <>; ; rspamd_inet_address_connect: connect unix:/var/run/clamd@rspamd/clamav failed: 2, 'No such file or directory'
Nov 21 14:55:28 prometheus rspamd[27012]: <8d58fc>; proxy; lua_tcp_make_connection: cannot connect to /var/run/clamd@rspamd/clamav (/var/run/clamd@rspamd/clamav): No such file or directory
Nov 21 14:55:28 prometheus rspamd[27012]: <8d58fc>; lua; clamav.lua:119: clamav: failed to scan, maximum retransmits exceed
Nov 21 14:55:28 prometheus rspamd[27012]: <8d58fc>; lua; common.lua:108: clamav: result - FAILED with error: "failed to scan and retransmits exceed - score: 0"
Nov 21 14:55:28 prometheus rspamd[27012]: <8d58fc>; proxy; spf_plugin_callback: stored record for Chubbfrance.com (0x713e9c57ca5059be) in LRU cache for 1593 seconds, 1/2000 elements in the cache
Nov 21 14:55:28 prometheus rspamd[27012]: <8d58fc>; proxy; rspamd_redis_finalize_process: cannot retreive stat tokens from Redis: skip obtaining bayes tokens for BAYES_SPAM: not enough learns 50; 200 required
Nov 21 14:55:28 prometheus rspamd[27012]: <8d58fc>; proxy; rspamd_redis_finalize_process: cannot retreive stat tokens from Redis: skip obtaining bayes tokens for BAYES_HAM: not enough learns 7; 200 required
Nov 21 14:55:28 prometheus rspamd[27012]: <8d58fc>; lua; greylist.lua:298: Score too low - skip greylisting

...

(default: F (no action): [-0.71/19.90] [DMARC_POLICY_ALLOW_WITH_FAILURES(-0.50){},R_DKIM_ALLOW(-0.20){Chubbfrance.com:s=selector2;},MIME_BASE64_TEXT(0.10){},MIME_GOOD(-0.10){multipart/mixed;text/plain;},MX_GOOD(-0.01){cached: Chubbfrance-com.mail.protection.outlook.com;},ASN(0.00){asn:8075, ipnet:40.64.0.0/10, country:US;},CLAM_VIRUS_FAIL(0.00){failed to scan and retransmits exceed;},DKIM_TRACE(0.00){Chubbfrance.com:+;},DMARC_POLICY_ALLOW(0.00){Chubbfrance.com;none;},FROM_EQ_ENVFROM(0.00){},FROM_HAS_DN(0.00){},HAS_ATTACHMENT(0.00){},HAS_XOIP(0.00){},IP_REPUTATION_HAM(0.00){asn: 8075(-0.22), country: US(-0.00), ip: 40.107.2.59(0.00);},MID_RHS_MATCH_FROM(0.00){},MIME_TRACE(0.00){0:+;1:+;2:~;},RCPT_COUNT_ONE(0.00){1;},RCVD_COUNT_THREE(0.00){3;},RCVD_IN_DNSWL_NONE(0.00){40.107.2.59:from;},RCVD_TLS_LAST(0.00){},R_SPF_SOFTFAIL(0.00){~all;},TO_DN_EQ_ADDR_ALL(0.00){},TO_MATCH_ENVRCPT_ALL(0.00){}]),

[stephdl]

set closed