freshclam not updating daily.cvd

[andybkay]

There appears to be an issue with updating on our installation. The cdiffs download, and it appears to generate the daily.cld file, but it doesn't create the cvd and upload it. I have dropped -v into the freshclam command, and it's returning Exitcode 1, being the database is up to date, where my daily.cvd hasn't been updated since the 30th November.

Is there something I'm missing here? There was an instance last week where the cdiff section failed, so it downloaded the latest daily.cvd automatically, but it's only had one instance of that in over a month.

Full output with -v attached.

cloudwatch.output.txt

[ocasta]

I'm seeing the same behaviour

[erecica]

Same here

[Lacan82]

return code 1 means the virus database is up to date. I believe this was finally corrected in 101.0...

https://blog.clamav.net/2018/12/clamav-01010-has-been-released.html

[andybkay]

While that may be the case, it’s giving the wrong return code. The DB is definitely not up to date. I can provide logs if required.

[Lacan82]

Just to be clear. The S3 bucket that is holding the cvd are not updating?

[andybkay]

Hi,

Yes that’s the case. The verbose output shows the downloading of the cdiff files, I would have thought it’d check the current DB before doing this to ensure an update was needed.

FYI, I have a different function currently performing the update for me as a work around. So I’d have to turn that off and wait several hours before being able to help with diagnosis

For some reason, when it starts the upload process, the daily.cvd file isn’t present in /tmp

[Lacan82]

Then makes me think this is related to issue. #49 I will dig this weekend and see if I can resolve it.

[Lacan82]

It would be interesting to invoke a call to dump tmp on excution, and see if that resolves.

[andybkay]

I have debug that performs os.listdir() several times throughout the update script, I’ll dig out the logs for you when I get a chance.

[andybkay]

As promised, sorry it took so long. As you can see I dropped quite a few debug lines in (I'm pretty new to Python, sorry for the crude methods). I have the original code for the function if you need to see where each bit runs. Thanks for the help by the way, it's greatly appreciated:

22:09:02
START RequestId: e0382c35-f67e-11e8-be4e-017e70e671cc Version: $LATEST

22:09:02
Script starting at 2018/12/02 22:09:02 UTC

22:09:02
Attempting to create directiory /tmp/clamav_defs.

22:09:02
Downloading definition file main.cvd from s3://ex-plor-definitions/clamav_defs

22:09:07
Downloading definition file daily.cvd from s3://ex-plor-definitions/clamav_defs

22:09:08
Downloading definition file bytecode.cvd from s3://ex-plor-definitions/clamav_defs

22:09:09
Path is /tmp/clamav_defs

22:09:09
1: ['bytecode.cvd', 'main.cvd', 'daily.cvd']

22:09:09
Starting freshclam with defs in /tmp/clamav_defs.

22:09:09
2: ['bytecode.cvd', 'main.cvd', 'daily.cvd']

22:09:09
3: ['bytecode.cvd', 'main.cvd', 'daily.cvd']

22:09:39
freshclam output:

22:09:39
ClamAV update process started at Sun Dec 2 22:09:09 2018

22:09:39
main.cvd is up to date (version: 58, sigs: 4566249, f-level: 60, builder: sigmgr)

22:09:39
Downloading daily-25167.cdiff [100%]

22:09:39
Downloading daily-25168.cdiff [100%]

22:09:39
Downloading daily-25169.cdiff [100%]

22:09:39
Downloading daily-25170.cdiff [100%]

22:09:39
Downloading daily-25171.cdiff [100%]

22:09:39
Downloading daily-25172.cdiff [100%]

22:09:39
Downloading daily-25173.cdiff [100%]

22:09:39
daily.cld updated (version: 25173, sigs: 2167842, f-level: 63, builder: neo)

22:09:39
bytecode.cvd is up to date (version: 327, sigs: 91, f-level: 63, builder: neo)

22:09:39
Database updated (6734182 signatures) from database.clamav.net (IP: 104.16.188.138)

22:09:39
0: ['mirrors.dat', 'bytecode.cvd', 'daily.cld', 'main.cvd']

22:09:39
Current Working Dir /var/task

22:09:39
Local file path: /tmp/clamav_defs/main.cvd

22:09:39
Getting MD5 Hash of file /tmp/clamav_defs/main.cvd

22:09:40
Remote file path clamav_defs/main.cvd

22:09:40
a22e1b59c5e8b8eff166271b08b4ad72

22:09:40
a22e1b59c5e8b8eff166271b08b4ad72

22:09:40
Not uploading main.cvd because md5 on remote matches local.

22:09:40
0: ['mirrors.dat', 'bytecode.cvd', 'daily.cld', 'main.cvd']

22:09:40
0: ['mirrors.dat', 'bytecode.cvd', 'daily.cld', 'main.cvd']

22:09:40
0: ['mirrors.dat', 'bytecode.cvd', 'daily.cld', 'main.cvd']

22:09:40
Current Working Dir /var/task

22:09:40
Local file path: /tmp/clamav_defs/bytecode.cvd

22:09:40
Getting MD5 Hash of file /tmp/clamav_defs/bytecode.cvd

22:09:40
Remote file path clamav_defs/bytecode.cvd

22:09:40
b73a1dc748b5e3b9cd79de0fbbdecb42

22:09:40
b73a1dc748b5e3b9cd79de0fbbdecb42

22:09:40
Not uploading bytecode.cvd because md5 on remote matches local.

22:09:40
0: ['mirrors.dat', 'bytecode.cvd', 'daily.cld', 'main.cvd']

22:09:40
Script finished at 2018/12/02 22:09:40 UTC

22:09:40
END RequestId: e0382c35-f67e-11e8-be4e-017e70e671cc

22:09:40
REPORT RequestId: e0382c35-f67e-11e8-be4e-017e70e671cc Duration: 37491.60 ms Billed Duration: 37500 ms Memory Size: 512 MB Max Memory Used: 512 MB