Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Antivirus: improve memory usage #5803

Closed
gsanchietti opened this issue Aug 9, 2019 · 16 comments
Closed

Antivirus: improve memory usage #5803

gsanchietti opened this issue Aug 9, 2019 · 16 comments
Labels
verified All test cases were verified successfully
Projects
NethServer 7

Comments

@gsanchietti
Copy link
Member

Clamav default signatures (main.cvd, bytecode.cvd, daily.cvd) contain a lot of old and not effective rules which can even cause some false positive.
Also, such signatures have a huge impact on memory usage.

Proposed solution

Starting from NS 7.7 the antivirus will use only unofficial signatures by default.
The sysadmin will be able to change such behavior using a documented property.
@gsanchietti gsanchietti added this to ✋ Needs review in NethServer 7 via automation Aug 9, 2019
This was referenced Aug 9, 2019
Merged
Merged
Merged
@DavidePrincipi DavidePrincipi moved this from ✋ Needs review to ⚙ Developing in NethServer 7 Aug 21, 2019
gsanchietti added a commit to NethServer/nethserver-squidclamav that referenced this issue Aug 28, 2019
@gsanchietti
Merge pull request #7 from gsanchietti/no_officialsig
8f2e78d 
- Restart clamd instance on nethserver-antivirus-update
- Remove signatures selection

NethServer/dev#5803
@nethbot
Copy link
Member

nethbot commented Aug 28, 2019

in 7.6.1810/testing:

gsanchietti added a commit to NethServer/nethserver-mail that referenced this issue Aug 28, 2019
@gsanchietti
Merge pull request #137 from gsanchietti/no_officialsig
53875ca 
Restart clamd rspamd instance on nethserver-antivirus-update

NethServer/dev#5803
gsanchietti added a commit to NethServer/nethserver-antivirus that referenced this issue Aug 28, 2019
@gsanchietti
Merge pull request #4 from gsanchietti/no_official
8698430 
- Disable clamav official signatures on request
- Use new clamav unofficial config file

NethServer/dev#5803
@nethbot
Copy link
Member

nethbot commented Aug 28, 2019

in 7.6.1810/testing:

@nethbot
Copy link
Member

nethbot commented Aug 28, 2019

in 7.6.1810/testing:

@gsanchietti
Copy link
Member Author

gsanchietti commented Aug 28, 2019 •
edited by stephdl

Test case 1

  • Install on a clean machine
  • Verify the unofficial signature script downloads the low risk rules
  • Verify freshclam cron is enabled and clamav default signatures are available after first freshclam run

Test case 2

  • Update an existing machine where antivirus is enabled for both mail and web proxy filter
  • Verify that clamd@rspamd is running
  • Verify that clamd@squidclamav is running

Test case 3

  • After test case 2
  • Disable antivirus for mail filter
  • Verify that clamd@rspamd is not running

Test case 4

  • After test case 2
  • Disable antivirus for web proxy
  • Verify that clamd@squidclamav is not running

Test case 5

  • Disable official signatures:
config setprop clamd OfficialSignatures disabled
signal-event nethserver-antivirus-update
  • Verify that freshclam cron has been disabled 
    grep FRESHCLAM_DELAY /etc/sysconfig/freshclam
  • Daily and main rule files has been deleted from /var/lib/clamav directory
  • Both clamd instances have been restarted

Test case 6

  • Change unofficial signatures rating:
config setprop clamd UnofficialSignaturesRating medium
signal-event nethserver-antivirus-update
  • Verify the modifications has been reflected inside /etc/clamav-unofficial-sigs/user.conf: 
    grep default_dbs_rating /etc/clamav-unofficial-sigs/user.conf

@gsanchietti gsanchietti added the testing Packages are available from testing repositories label Aug 28, 2019
@stephdl stephdl self-assigned this Aug 28, 2019
@stephdl
Copy link

stephdl commented Aug 28, 2019 •
edited

QA
version tested

[root@ns7loc14 ~]# rpm -qa nethserver-antivirus nethserver-mail-common nethserver-mail-filter nethserver-mail-server nethserver-squidclamav
nethserver-mail-common-2.6.7-1.6.g53875ca.ns7.noarch
nethserver-squidclamav-3.0.0-1.4.g8f2e78d.ns7.noarch
nethserver-mail-server-2.6.7-1.6.g53875ca.ns7.noarch
nethserver-antivirus-1.2.2-1.5.g8698430.ns7.noarch
nethserver-mail-filter-2.6.7-1.6.g53875ca.ns7.noarch
  • case 1
    after the installation on a clean machine

Verify the unofficial signature script downloads the low risk rules

[root@ns7loc14 ~]# grep -srn 'LOW' /etc/clamav-unofficial-sigs/user.conf 
40:# valid rating: LOW, MEDIUM, HIGH
41:default_dbs_rating="LOW"

please @gsanchietti , valid the answer

Verify freshclam cron is enabled and clamav default signatures are available after first freshclam run

[root@ns7loc14 ~]# grep -srni freshclam /etc/cron*
/etc/cron.d/clamav-update:4:## It is ok to execute it as root; freshclam drops privileges and becomes
/etc/cron.d/clamav-update:6:0  */3 * * * root /usr/share/clamav/freshclam-sleep

[root@ns7loc14 ~]# freshclam 
ClamAV update process started at Wed Aug 28 16:14:22 2019
Downloading main.cvd [100%]
main.cvd updated (version: 58, sigs: 4566249, f-level: 60, builder: sigmgr)
Downloading daily.cvd [100%]
daily.cvd updated (version: 25555, sigs: 1739106, f-level: 63, builder: raynman)
Downloading bytecode.cvd [100%]
bytecode.cvd updated (version: 330, sigs: 94, f-level: 63, builder: neo)
Database updated (6305449 signatures) from database.clamav.net (IP: 104.16.219.84)

test case 1 must be validated

  • case 2 FAILED
--> Running transaction check
---> Package nethserver-antivirus.noarch 0:1.2.2-1.ns7 will be updated
---> Package nethserver-antivirus.noarch 0:1.2.2-1.5.g8698430.ns7 will be an update
---> Package nethserver-mail-common.noarch 0:2.6.7-1.ns7 will be updated
---> Package nethserver-mail-common.noarch 0:2.6.7-1.6.g53875ca.ns7 will be an update
---> Package nethserver-mail-filter.noarch 0:2.6.7-1.ns7 will be updated
---> Package nethserver-mail-filter.noarch 0:2.6.7-1.6.g53875ca.ns7 will be an update
---> Package nethserver-mail-server.noarch 0:2.6.7-1.ns7 will be updated
---> Package nethserver-mail-server.noarch 0:2.6.7-1.6.g53875ca.ns7 will be an update
---> Package nethserver-squidclamav.noarch 0:3.0.0-1.ns7 will be updated
---> Package nethserver-squidclamav.noarch 0:3.0.0-1.4.g8f2e78d.ns7 will be an update

systemctl status clamd@squidclamav clamd@rspamd

● clamd@squidclamav.service - clamd scanner (squidclamav) daemon
   Loaded: loaded (/usr/lib/systemd/system/clamd@.service; static; vendor preset: disabled)
  Drop-In: /etc/systemd/system/clamd@squidclamav.service.d
           └─c-icap.conf
   Active: active (running) since Wed 2019-08-28 16:52:39 CEST; 50s ago

● clamd@rspamd.service - clamd scanner (rspamd) daemon
   Loaded: loaded (/usr/lib/systemd/system/clamd@.service; static; vendor preset: disabled)
  Drop-In: /usr/lib/systemd/system/clamd@rspamd.service.d
           └─nethserver.conf
   Active: active (running) since Wed 2019-08-28 16:52:23 CEST; 1min 6s ago

the services are running before and after the update, but when you restart the server, the service clamd@rspamd.service is down

see https://github.com/NethServer/nethserver-mail/pull/137/files#diff-55213ff073b5012a5d6164561c45583aR3

  • case 3 FAILED

even with the configuration change, the service is not stopped

Aug 28 17:38:02 ns7loc14 /sbin/e-smith/db[3913]: /var/lib/nethserver/db/configuration: NEW rspamd=service|BlockAttachmentClassList|Exec|BlockAttachmentCustomList|doc,odt|BlockAttachmentCustomStatus|disabled|BlockAttachmentStatus|enabled|Password|aVbTi_Q_J4le01W5|RecipientWhiteList||SenderBlackList||SenderWhiteList||SpamCheckStatus|enabled|SpamGreyLevel||SpamKillLevel|20|SpamSubjectPrefixStatus|disabled|SpamSubjectPrefixString|***SPAM***|SpamTag2Level|6|VirusAction|reject|VirusCheckStatus|disabled|VirusScanOnlyAttachment|false|VirusScanSize|20000000|status|enabled

● clamd@rspamd.service - clamd scanner (rspamd) daemon
   Loaded: loaded (/usr/lib/systemd/system/clamd@.service; static; vendor preset: disabled)
  Drop-In: /usr/lib/systemd/system/clamd@rspamd.service.d
           └─nethserver.conf
   Active: active (running) since Wed 2019-08-28 17:38:04 CEST; 20s ago
  • case 4 OK

squid/antivirus is well stopped by the configuration change

● clamd@squidclamav.service - clamd scanner (squidclamav) daemon
   Loaded: loaded (/usr/lib/systemd/system/clamd@.service; static; vendor preset: disabled)
  Drop-In: /etc/systemd/system/clamd@squidclamav.service.d
           └─c-icap.conf
   Active: inactive (dead)
     Docs: man:clamd(8)
           man:clamd.conf(5)
           https://www.clamav.net/documents/
  • case 5 OK
    before the test
[root@ns7loc14 ~]# ll /var/lib/clamav
total 160900
-rw-r--r-- 1 clamupdate clamupdate    207879 Aug 28 15:57 bytecode.cvd
-rw-r--r-- 1 clamupdate clamupdate  46646043 Aug 28 15:56 daily.cvd
-rw-r--r-- 1 clamupdate clamupdate       162 Aug 28  2019 eicar.ndb
-rw-r--r-- 1 clamupdate clamupdate 117892267 Aug 28 15:56 main.cvd
-rw------- 1 clamupdate clamupdate        64 Aug 28 15:57 mirrors.dat
[root@ns7loc14 ~]# grep FRESHCLAM_DELAY /etc/sysconfig/freshclam
#FRESHCLAM_DELAY=

After the services are restarted as expected and the databases have been cleaned

[root@ns7loc14 ~]# ll /var/lib/clamav
total 8
-rw-r--r-- 1 clamupdate clamupdate 162 Aug 28  2019 eicar.ndb
-rw------- 1 clamupdate clamupdate  64 Aug 28 15:57 mirrors.dat
[root@ns7loc14 ~]# grep FRESHCLAM_DELAY /etc/sysconfig/freshclam
FRESHCLAM_DELAY=disabled
  • case 6 OK
[root@ns7loc14 ~]# grep default_dbs_rating /etc/clamav-unofficial-sigs/user.conf
default_dbs_rating="MEDIUM"

@gsanchietti gsanchietti removed the testing Packages are available from testing repositories label Aug 28, 2019
@gsanchietti gsanchietti assigned gsanchietti and unassigned stephdl Aug 28, 2019
This was referenced Aug 29, 2019
Merged
Merged
Merged
gsanchietti added a commit to NethServer/nethserver-antivirus that referenced this issue Aug 29, 2019
@gsanchietti
Merge pull request #5 from gsanchietti/reload_instances
0a8f544 
unofficial sigs: reload all clamd instances

The default configuration of clamav-unofficial-sigs only restarts clamd@scan instance which is not used by NethServer
All clamd instances are now restarted by nethserver-antivirus event and reloaded by /etc/clamav-unofficial-sigs/user.conf

NethServer/dev#5803
@nethbot
Copy link
Member

nethbot commented Aug 29, 2019

in 7.6.1810/testing:

gsanchietti added a commit to NethServer/nethserver-squidclamav that referenced this issue Aug 29, 2019
@gsanchietti
Revert "Restart clamd instance on nethserver-antivirus-update" (#8)
9aac3a4 
This reverts commit 2ad6658.

Restart logic is now implemented inside NethServer/nethserver-antivirus#5

NethServer/dev#5803
gsanchietti added a commit to NethServer/nethserver-mail that referenced this issue Aug 29, 2019
@gsanchietti
Revert "Restart clamd rspamd instance on nethserver-antivirus-update" (…
12fa6e8 
…#140)

Reverts #137

Restart logic is now implemented inside NethServer/nethserver-antivirus#5
Also the clamd@rspamd instance can't be started/stopped without some ugly hack.

NethServer/dev#5803
@nethbot
Copy link
Member

nethbot commented Aug 29, 2019

in 7.6.1810/testing:

@nethbot
Copy link
Member

nethbot commented Aug 29, 2019

in 7.6.1810/testing:

@gsanchietti
Copy link
Member Author

Test case 1

  • Install on a clean machine
  • Verify the unofficial signature script downloads the low risk rules
  • Verify freshclam cron is enabled and clamav default signatures are available after first freshclam run

Test case 2

  • Update an existing machine where antivirus is enabled for both mail and web proxy filter
  • Verify that clamd@rspamd is running
  • Verify that clamd@squidclamav is running

Test case 3

  • After test case 2
  • Disable antivirus for web proxy
  • Verify that clamd@squidclamav is not running

Test case 4

  • Disable official signatures:
config setprop clamd OfficialSignatures disabled
signal-event nethserver-antivirus-update
  • Verify that freshclam cron has been disabled 
    grep FRESHCLAM_DELAY /etc/sysconfig/freshclam
  • Daily and main rule files has been deleted from /var/lib/clamav directory
  • Both clamd instances must have been reload

Test case 5

  • Change unofficial signatures rating:
config setprop clamd UnofficialSignaturesRating medium
signal-event nethserver-antivirus-update
  • Verify the modifications has been reflected inside /etc/clamav-unofficial-sigs/user.conf: 
    grep default_dbs_rating /etc/clamav-unofficial-sigs/user.conf

@gsanchietti gsanchietti removed their assignment Aug 29, 2019
@gsanchietti gsanchietti added the testing Packages are available from testing repositories label Aug 29, 2019
@stephdl stephdl self-assigned this Aug 29, 2019
@stephdl
Copy link

stephdl commented Aug 29, 2019 •
edited

QA

  • case 1 OK (same as QA above)
  • case 2 OK
    After the installation or after the reboot the two services are running
  • case 3 OK (same as QA above)
  • case 4 OK (same as QA above)
  • case 5 OK (same as QA above)

set verified

@stephdl stephdl assigned stephdl and unassigned stephdl Aug 29, 2019
@stephdl stephdl added the verified All test cases were verified successfully label Aug 29, 2019
@gsanchietti gsanchietti removed the testing Packages are available from testing repositories label Aug 29, 2019
@filippocarletti
Copy link
Member

Side note on memory usage as of yesterday.
Low unofficial - VmSize: 259192 kB
Medium unofficial - VmSize: 282832 kB
Medium+official - VmSize: 1187068 kB

cat /proc/$(cat /var/run/clamd\@rspamd/clamav.pid )/status | grep VmSize

@nethbot
Copy link
Member

nethbot commented Aug 29, 2019

in 7.6.1810/testing:

@nethbot
Copy link
Member

nethbot commented Aug 30, 2019

in 7.6.1810/updates:

@nethbot
Copy link
Member

nethbot commented Aug 30, 2019

in 7.6.1810/updates:

@nethbot
Copy link
Member

nethbot commented Aug 30, 2019

in 7.6.1810/updates:

NethServer 7 automation moved this from ⚙ Developing to 🗑 Done Aug 30, 2019
@stephdl
Copy link

stephdl commented Aug 30, 2019 •
edited

Instead of reverting the QA I think it was needed only to provide a systemd unit service like this

https://gist.github.com/stephdl/d07af66a9790ecf1b669e22ba841e161

the failed QA 2 and 3 could be fixed @gsanchietti

gsanchietti added a commit to NethServer/nethserver-antivirus that referenced this issue Sep 12, 2019
@gsanchietti
Increate start timeout to 10 minutes
e822107 
On very slow machines, clamd could take lobger than 90 seconds to start,
thus systemd restart the service generating a loop.

If 10 minutes are not enough, the only solution is disabling the
official signatures.

NethServer/dev#5803
@gsanchietti gsanchietti mentioned this issue Sep 27, 2019
Merged
gsanchietti added a commit to NethServer/nethserver-box that referenced this issue Sep 27, 2019
@gsanchietti
Remove freshclam hack (#5)
596106e 
Force standard clamd OfficialSignatures property to disabled.

NethServer/dev#5803
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
verified All test cases were verified successfully
Projects
No open projects
NethServer 7
🗑 Done
Milestone
No milestone
Development

No branches or pull requests
4 participants
@filippocarletti @gsanchietti @stephdl @nethbot