clamonacc only uses 1 thread if --fdpass is given or tcp is used #394
Comments
arch-user-france1
changed the title
|
nobody answered. WHY! |
|
Hi @debian-user-france1 sorry to hear you're seeing crashes with https://github.com/Cisco-Talos/clamav/blob/rel/0.103/NEWS.md#01034 |
|
Didn't I say that I have had one 104 too that had the same? I'll check it anyways. I wasn't sure if it's the same.. |
|
Same here using clamad/clamonacc on Debian Buster and Bullseye, which is 0.103.5 based. Seems to use only one thread. |
|
There's nothing mentioned in the Changelog too, performance optimisations and other security stuff is there. Manjaro Linux uses 104.3 and it's still happening. I think it's a client sided issue (since the client clamomacc doesn't have to make threads). |
|
I try to summarize that (applies to rel/0.103): It's currently not possible to do multithreaded on access scanning in a safe manner, because:
Is this correct? |
|
And tcp is not working too, not able to instruct clamd to scan parallel. Seems to be due to an unfinished code. It works but slow - clamonacc doesn't create new threads only uses one thread to pass files to clamd IF USING FDPASS OR TCP but the socket works. |
|
Running clamd as root is not a solution if tcp is used - for example: I've got a clamd dedicated server and want to use that with my computer - it won't work good because there's only one thread created - and one thread is slow as hell. Only clamonacc is affected. |
|
@sezuan any news? |
|
@debian-user-france1 I'm sorry that you're frustrated. Does the crash happen every now and then? Can you cause the crash to happen consistently by doing something specific in the monitored directories? Can you share the stdout/stderr output from |
This is not a report about a crash. The problem is that clamonacc is only using one clamd thread if it connects to a remote machine using tcp. The same seems to happen if the argument --fdpass is given. This bug makes clamonacc unusable because the scanning, which the remote machine (clamd server) does, only happens using one thread, and clamonacc just adds scanns to the queue instead of asking the remote machine for a new thread. Now, especially if I would instruct clamonacc to block unscanned files, the system would get stuck. Remember that the same happens if --fdpass is used, and so it is not possible to run clamd under non-root privileges. Oh and clammonacc --verbose just shows that it's putting unscanned files to queue while waiting until clamd scans a file using one thread. If there are 1024 files in queue it just doesn't put new files to queue and the system could break (in case of on-access blocking). |
Sorry, my mistake. I understand, now. I wonder if |
|
No, clamdscan works fine, it's only clamonacc.
|
|
How to reproduce:
|
|
Hello????? |
|
I also run into this problem on Fedora 36. I made some modifications to the command line flags used to start clamd and clamonacc, compared to the official Fedora 36 packages -- cf. the systemd unit files quoted below. The effect of clamonacc not sending multiple FDs to clamd (and thus queueing them and giving clamd a chance to scan multiple files in parallel) is that the whole system comes to a crawl when a larger application starts (e.g. starting IntelliJ IDEA blocks the system for several minutes) and it becomes even impossible to run other applications (like a terminal, for figuring out what is going on) until the first application is done, because this "one file at a time" is global in the system, across all applications, not just per application. I first tried to solve this by de-nicing clamd (to -10, to give it higher priority -- cf. the systemd unit files quoted below) in order to prevent other processes from blocking clamd from scanning files, but it turned out fights over CPU resources are not the problem -- the system is mostly idle during the time it is blocked. Looking at Side question: It would be interesting to know whether this starvation is counted (or could be counted, using some configurable parameters) negatively towards the I/O budget of the process that originally opened the file, so that the kernel's I/O scheduler would give other processes precedence and the system could more quickly resume to being usable. # systemctl cat clamav-clamonacc.service
# /etc/systemd/system/clamav-clamonacc.service
# Adapted from https://github.com/Cisco-Talos/clamav/blob/8b6e53a08a31ea7c792fac2f2fb7a1775a6dc7e5/clamonacc/clamav-clamonacc.service.in
[Unit]
Description=ClamAV On-Access Scanner
Documentation=man:clamonacc(8) man:clamd.conf(5) https://docs.clamav.net/
Requires=clamav-daemon.service
After=clamav-daemon.service syslog.target network.target
[Service]
Type=simple
User=root
ExecStartPre=/bin/bash -c "while [ ! -S /run/clamav/clamd.ctl ]; do sleep 1; done"
ExecStart=/usr/sbin/clamonacc --foreground=true --fdpass=true --config-file=/etc/clamd.d/scan.conf
[Install]
WantedBy=multi-user.target# systemctl cat clamav-daemon.service
# /etc/systemd/system/clamav-daemon.service
# Adapted from https://github.com/Cisco-Talos/clamav/blob/8b6e53a08a31ea7c792fac2f2fb7a1775a6dc7e5/clamd/clamav-daemon.service.in
[Unit]
Description=Clam AntiVirus userspace daemon
Documentation=man:clamd(8) man:clamd.conf(5) https://docs.clamav.net/
Requires=clamav-daemon.socket
# From Fedora's /usr/lib/systemd/system/clamd@.service:
After = syslog.target nss-lookup.target network.target
# Check for database existence
ConditionPathExistsGlob=/var/lib/clamav/main.{c[vl]d,inc}
ConditionPathExistsGlob=/var/lib/clamav/daily.{c[vl]d,inc}
[Service]
# Partially from Fedora's /usr/lib/systemd/system/clamd@.service:
ExecStart=/usr/sbin/clamd --foreground=true --config-file=/etc/clamd.d/scan.conf
# From Fedora's /usr/lib/systemd/system/clamd@.service:
Restart = on-failure
# Reload the database
ExecReload=/bin/kill -USR2 $MAINPID
TimeoutStartSec=420
# clamd regularly blocks other processes because it cannot keep up with the amount of work shoved its way by clamonacc:
Nice=-10
[Install]
WantedBy=multi-user.target
Also=clamav-daemon.socket# systemctl cat clamav-daemon.socket
# /etc/systemd/system/clamav-daemon.socket
# Adapted from https://github.com/Cisco-Talos/clamav/blob/8b6e53a08a31ea7c792fac2f2fb7a1775a6dc7e5/clamd/clamav-daemon.socket.in
[Unit]
Description=Socket for Clam AntiVirus userspace daemon
Documentation=man:clamd(8) man:clamd.conf(5) https://docs.clamav.net/
# Check for database existence
ConditionPathExistsGlob=/var/lib/clamav/main.{c[vl]d,inc}
ConditionPathExistsGlob=/var/lib/clamav/daily.{c[vl]d,inc}
[Socket]
ListenStream=/run/clamav/clamd.ctl
#ListenStream=1024
SocketUser=clamscan
SocketGroup=virusgroup
RemoveOnStop=True
[Install]
WantedBy=sockets.target |
|
hmm could this explain the performance issues that are showing when clamonacc is used? no matter which version, 0.103.7 or 0.105.7 -> once clamonacc is used with --fdpass (clamd runs as user clamav, clamonacc as root with --foreground and --fdpass) i constantly get these errors when the system is active IO wise: clamav-clamonacc[1756008]: ERROR: ClamClient: Connection to clamd failed, Timeout was reached. also, it seems 0.105.1 often does not free up space on filesystems it watches, interestingly enough lsof/fuser do not show the files as deleted or at all -> restarting clamav-daemon will free up the space are there any settings or compilation opptions that can be used to make clamonacc usable? |
|
Honestly, clamonacc is trash and the team has no interest in fixing it at all. But now at least I know why I had to restart the system to get the space ^^ If you have the required knowledge, you could rewrite clamonacc or fork it; but I don't think that's your intention. Not even Windows works that way. Sure, it scans files as they are downloaded (created), but it doesn't block access to them I believe, and isn't that the intention of clamonacc? I guess you should not care about AV systems on Linux, Viruses are mostly too fresh to be in the database of clamav. And clamav doesn't do machine learning. |
|
Hmhm I digged a bit deeper now and it seems that at least my issue with was caused by (stracing clamonacc): -> greatly increased the max open files in the system unit file: While the open files increases up to ~22k it seems to remain stable now. I am using rkhunter on / to "simulate" aggressive load. Also using: Also increased the threads/maxthreads in clamd.conf and it does use more threads? though clamdtop does not reflect this 🤔 ? I am confused. sadly, there dont seem (non cloud based) onaccess scanners for linux (anymore), or at least none that i am aware about. |
|
First: there is no antivirus which is open source, not stealing you, other than clamav. At least no other popular. And the remaining stuff |
|
I do not have the required skills to fix it, otherwise I would attempt todo so. But it seems, at least from pstree viewpoint, that it does use threads up to the defined number in clamd.conf. but yeah, clamdtop says otherwise... |
Describe the bug
clamonacc (parameters --verbose -F --fdpass) is using only one thread (look at clamdtop) and may even crash the system if prevention is on and is busy with a large file
How to reproduce the problem
put this in your clamd config:
start clamonacc by clamonacc
--fdpass -F --verboseReplace this text with specific steps needed to reproduce the issue.
Replace this text with the output from the ClamAV command:
Checking configuration files in /etc/clamav
Config file: clamd.conf
PreludeAnalyzerName = "ClamAV"
LogFile = "/var/log/clamav/clamav.log"
LogFileMaxSize = "4294967295"
LogTime = "yes"
LogRotate = "yes"
ExtendedDetectionInfo = "yes"
LocalSocket = "/var/run/clamav/clamd.ctl"
LocalSocketGroup = "clamav"
LocalSocketMode = "666"
MaxConnectionQueueLength = "50"
MaxThreads = "12"
ReadTimeout = "240"
SendBufTimeout = "200"
MaxQueue = "128"
MaxDirectoryRecursion disabled
SelfCheck = "3600"
User = "clamav"
BytecodeTimeout = "60000"
MaxScanTime = "120000"
MaxScanSize = "314572800"
MaxFileSize = "262144000"
MaxRecursion disabled
PCREMatchLimit = "10000"
PCRERecMatchLimit = "5000"
OnAccessIncludePath = "/home/programmieren"
OnAccessExcludePath = "/proc", "/run", "/dev", "/sys"
OnAccessExcludeUname = "root"
OnAccessMaxFileSize = "104857600"
OnAccessPrevention = "yes"
OnAccessExtraScanning = "yes"
Config file: freshclam.conf
LogFileMaxSize = "4294967295"
LogTime = "yes"
LogRotate = "yes"
UpdateLogFile = "/var/log/clamav/freshclam.log"
Checks = "24"
DatabaseMirror = "db.local.clamav.net", "database.clamav.net"
MaxAttempts = "5"
*** SafeBrowsing is DEPRECATED ***
Config file: clamav-milter.conf
LogFile = "/var/log/clamav/clamav-milter.log"
LogTime = "yes"
LogRotate = "yes"
PidFile = "/var/run/clamav/clamav-milter.pid"
TemporaryDirectory = "/tmp"
User = "clamav"
ClamdSocket = "unix:/var/run/clamav/clamd.ctl"
MilterSocket = "/var/run/clamav/clamav-milter.ctl"
MilterSocketGroup = "clamav"
MilterSocketMode = "666"
AddHeader = "Replace"
LogInfected = "Off"
LogClean = "Off"
Software settings
Version: 0.103.2
Optional features supported: MEMPOOL IPv6 FRESHCLAM_DNS_FIX AUTOIT_EA06 BZIP2 LIBXML2 PCRE2 ICONV JSON
Database information
Database directory: /var/lib/clamav
[3rd Party] winnow_bad_cw.hdb: 1 sig
[3rd Party] phishtank.ndb: 12985 sigs
[3rd Party] junk.ndb: 55802 sigs
[3rd Party] bofhland_malware_URL.ndb: 4 sigs
[3rd Party] doppelstern.hdb: 1 sig
[3rd Party] crdfam.clamav.hdb: 1 sig
[3rd Party] winnow.attachments.hdb: 182 sigs
[3rd Party] bofhland_phishing_URL.ndb: 72 sigs
[3rd Party] winnow_malware_links.ndb: 133 sigs
[3rd Party] jurlbl.ndb: 2839 sigs
[3rd Party] bofhland_cracked_URL.ndb: 40 sigs
[3rd Party] sanesecurity.ftm: 170 sigs
bytecode.cvd: version 333, sigs: 92, built on Mon Mar 8 16:21:51 2021
daily.cld: version 26369, sigs: 1948295, built on Tue Nov 30 10:18:45 2021
[3rd Party] bofhland_malware_attach.hdb: 1836 sigs
[3rd Party] sigwhitelist.ign2: 12 sigs
[3rd Party] blurl.ndb: 842 sigs
[3rd Party] scam.ndb: 12750 sigs
main.cvd: version 62, sigs: 6647427, built on Thu Sep 16 14:32:42 2021
[3rd Party] rogue.hdb: 575 sigs
[3rd Party] porcupine.ndb: 6449 sigs
[3rd Party] phish.ndb: 28057 sigs
[3rd Party] winnow_extended_malware.hdb: 245 sigs
[3rd Party] spamimg.hdb: 200 sigs
[3rd Party] spamattach.hdb: 14 sigs
[3rd Party] winnow_malware.hdb: 293 sigs
Total number of signatures: 8719317
Platform information
uname: Linux 5.11.0-40-generic #44~20.04.2-Ubuntu SMP Tue Oct 26 18:07:44 UTC 2021 x86_64
OS: linux-gnu, ARCH: x86_64, CPU: x86_64
Full OS version: Ubuntu 20.04.3 LTS
zlib version: 1.2.11 (1.2.11), compile flags: a9
platform id: 0x0a217b7b0800000000090300
Build information
GNU C: 9.3.0 (9.3.0)
CPPFLAGS: -Wdate-time -D_FORTIFY_SOURCE=2
CFLAGS: -g -O2 -fdebug-prefix-map=/build/clamav-TW4JTf/clamav-0.103.2+dfsg=. -fstack-protector-strong -Wformat -Werror=format-security -Wall -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64
CXXFLAGS: -g -O2 -fdebug-prefix-map=/build/clamav-TW4JTf/clamav-0.103.2+dfsg=. -fstack-protector-strong -Wformat -Werror=format-security -Wall -D_FILE_OFFSET_BITS=64
LDFLAGS: -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -Wl,--as-needed
Configure: '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--runstatedir=/run' '--disable-maintainer-mode' '--disable-dependency-tracking' 'CFLAGS=-g -O2 -fdebug-prefix-map=/build/clamav-TW4JTf/clamav-0.103.2+dfsg=. -fstack-protector-strong -Wformat -Werror=format-security -Wall -D_FILE_OFFSET_BITS=64' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' 'CXXFLAGS=-g -O2 -fdebug-prefix-map=/build/clamav-TW4JTf/clamav-0.103.2+dfsg=. -fstack-protector-strong -Wformat -Werror=format-security -Wall -D_FILE_OFFSET_BITS=64' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -Wl,--as-needed' '--with-dbdir=/var/lib/clamav' '--sysconfdir=/etc/clamav' '--disable-clamav' '--disable-unrar' '--enable-milter' '--enable-dns-fix' '--with-libjson' '--with-system-libmspack' '--with-libcurl=/usr' '--with-gnu-ld' '--with-systemdsystemunitdir=/lib/systemd/system' 'build_alias=x86_64-linux-gnu' 'OBJCFLAGS=-g -O2 -fdebug-prefix-map=/build/clamav-TW4JTf/clamav-0.103.2+dfsg=. -fstack-protector-strong -Wformat -Werror=format-security'
sizeof(void*) = 8
Engine flevel: 123, dconf: 123
Attachments
currently nothing available
Please note:
clamonacc works if no --fdpass is given
clamonacc doesn't work with tcp connections - the same happens
Bug
clamav/clamonacc doesn't create new threads when fdpass is given
The text was updated successfully, but these errors were encountered: