ClamAV is unusable on FIPS-enabled Linux systems due to MD5 use

[qralston]

Describe the bug

ClamAV is completely unusable on certain Linux distributions when FIPS mode is enabled.

The Linux system affected include at least RHEL8 and RHEL9 (beta).

How to reproduce the problem

Build and install ClamAV on a RHEL8 or RHEL9 beta system without FIPS mode enabled. Observe that all ClamAV functionality works.

Reboot the host into FIPS mode (by passing fips=1 to the kernel).

Observe that any attempt to load any of the databases fails:

$ clamscan /usr/share/dict/words
LibClamAV Error: Can't load /var/lib/clamav/main.cvd: Can't allocate memory
LibClamAV Error: cli_loaddbdir(): error loading database /var/lib/clamav/main.cvd
ERROR: Can't allocate memory
…

In this case, the Can't allocate memory error is somewhat of a red herring. The true issue is that when FIPS mode is active, non–FIPS-approved hashing algorithms are disabled, and that includes MD5, which ClamAV uses extensively internally.

This was a known issue, and was already reported back on 2019-10-29 in BZ#12424:

When FIPS mode is active -- it disables features not approved for cryptographic use. This includes use of the MD5 hashing algorithm, which ClamAV uses for both verdict caching in clamd as well as for many, if not most, hash-based signatures.

You're observing use of MD5 failing when freshclam test-loads a database containing MD5 signatures. I suspect if you downloaded the databases manually and ran clamd, it would also fail pretty fast when it collects an MD5 hash to use in caching, whitelisting, or for comparison with MD5 signatures.

We have a branch where we've experimented with switching to use SHA256 hashes. It it is a rather large change and requires more investigation before I can really make any promises about a specific direction or timeline for this effort.

This bug was not carried over to GitHub, so this issue is a continuation of the Bugzilla bug.

The bottom line: in 2022, no software should be using anything other than the SHA2 (SHA-256, SHA-384, SHA-512) and SHA3 family of hashing algorithms. Any use of MD5 and/or SHA-1 should be replaced with SHA2 or later, as both MD5 and SHA-1 are considered by cryptographers to be broken.

An increasing number of security hardening standards (e.g., DISA STIGs) require FIPS mode, so this will only affect more ClamAV users over time.

[micahsnyder]

@qralston thanks for bringing up the issue again here since you're right it still is a concern, and for linking with the old bugzilla issue.

We haven't made progress on the switch from MD5 to SHA256 for scan verdict caching but it is an issue in our Jira that was prioritized for investigation for our next release.

I believe the plan is for MD5 to continue to be supported for signatures for compatibility with existing MD5 signatures. I'll talk with the team about either replacing as many MD5 signatures with SHA256 signatures as possible and dropping the rest, or else consider having the clamav build detect fips-mode and use conditional compilation to remove MD5 support for fips-mode builds.

[jshoovie]

Having the same issue.
It would appear ClamAV will not run on RHEL9 with FIPS enabled.

When I disable FIPS ClamAV will start.

uname -a
Linux rhel9test 5.14.0-70.26.1.el9_0.x86_64 #1 SMP PREEMPT Fri Sep 2 16:07:40 EDT 2022 x86_64 x86_64 x86_64 GNU/Linux

rpm -qa|grep clam
clamav-filesystem-0.103.7-1.el9.noarch
clamav-data-0.103.7-1.el9.noarch
clamav-lib-0.103.7-1.el9.x86_64
clamav-0.103.7-1.el9.x86_64
clamd-0.103.7-1.el9.x86_64

from /var/log/messages:
Oct 24 12:07:44 rhel9test systemd[1]: Starting clamd scanner daemon...
Oct 24 12:07:45 rhel9test clamd[46661]: LibClamAV Error: Can't load /var/lib/clamav/daily.cvd: Can't allocate memory
Oct 24 12:07:45 rhel9test clamd[46661]: LibClamAV Error: cli_loaddbdir(): error loading database /var/lib/clamav/daily.cvd
Oct 24 12:07:45 rhel9test clamd[46661]: ERROR: Can't allocate memory
Oct 24 12:07:45 rhel9test systemd[1]: clamd@service.service: Control process exited, code=exited, status=1/FAILURE
Oct 24 12:07:45 rhel9test systemd[1]: clamd@service.service: Failed with result 'exit-code'.
Oct 24 12:07:45 rhel9test systemd[1]: Failed to start clamd scanner daemon.

[qralston]

@micahsnyder another possibility would be to add one of the available public domain implementations of MD5 (e.g. Solar Designer’s implementation) to ClamAV and call it instead of the OpenSSL implementation if the system is running in FIPS mode.

The problem isn’t using MD5 per se; the problem is asking OpenSSL to perform MD5 when the system is in FIPS mode. ClamAV doesn’t use MD5 in a cryptographic context, but OpenSSL has no way to know that: it must deny all uses of MD5 if the system is in FIPS mode, in order to guarantee that no caller is using MD5 in a cryptographic context.

[PaulSD]

@micahsnyder: It appears to me that you may be misunderstanding the issue here, possibly due to conflicting uses of the term "signature".

Please correct me if I'm wrong, but it sounds like you are using the word "signature" to refer to checksums that are used to compare scanned files against a list of known safe/malicious files.

FIPS does NOT prohibit the use of MD5 hashes for that purpose. Those "signatures" do NOT need to be replaced in order to resolve this issue.

I'm not confident that I have a full picture of how the scan verdict caching works and is used, however at first glance it looks to me like MD5 should be OK here too. I do NOT think these checksums need to be replaced in order to resolve this issue either.

The real problem is the MD5 checksum and the associated RSA digital signature that are used to verify the integrity and publishing source of the CVD file itself: https://github.com/Cisco-Talos/clamav/blob/main/libclamav/cvd.c#L571
(The term "signature" as it relates to FIPS would refer to something like this RSA digital signature, hence why this "signature" is relevant but the above "signatures" are not.)

Both because of the security context here (depending on the use case, this may be the sole mechanism for verifying that the CVD file has not been tampered with during distribution through insecure distribution channels), and because of the use of an RSA digital signature (which falls under the purview of FIPS), this MD5 checksum does need to be replaced in order to resolve this issue.
It looks like the CVD file format will need to be modified or extended to support a SHA256 checksum and signature in the CVD file header for verification of the CVD file itself.

Are there any plans to address this specific checksum and digital signature in the CVD file header?

@qralston: OpenSSL does have a way to know whether you are using MD5 in a non-cryptographic context: the EVP_MD_CTX_FLAG_NON_FIPS_ALLOW flag. There is no need to replace the MD5 implementation with a different library, just set that flag whenever you are using MD5 for a non-crypto purpose.

clamav is already doing this in some places, like here: https://github.com/Cisco-Talos/clamav/blob/main/libclamav/crypto.c#L191

The problem is that it would NOT be appropriate to set this flag when validating the CVD file itself.

Note that clamav already appears to be setting this flag in the code paths for the file "signature" checks and scan verdict caching, which reinforces my statements above that those uses of MD5 do not need to be replaced. However, since it (appropriately) does not set this flag when validating the CVD file itself, freshclam/clamd/clamav currently all fail when loading/verifying the CVD file.

[micahsnyder]

@PaulSD Thank you for the detailed explanation. I was indeed confused. This helps! I will talk with the team about it.

[qralston]

@PaulSD wrote:

OpenSSL does have a way to know whether you are using MD5 in a non-cryptographic context: the EVP_MD_CTX_FLAG_NON_FIPS_ALLOW flag. There is no need to replace the MD5 implementation with a different library, just set that flag whenever you are using MD5 for a non-crypto purpose.

clamav is already doing this in some places, like here: https://github.com/Cisco-Talos/clamav/blob/main/libclamav/crypto.c#L191

The EVP_MD_CTX_FLAG_NON_FIPS_ALLOW flag was removed in OpenSSL 3.0; see line 218 of OpenSSL’s include/openssl/evp.h. So this work-around will fail with OpenSSL 3.0 and greater.

If the system was not booted in FIPS mode, even if the system’s default OpenSSL 3.0 configuration only loads the fips provider, you should be able to explicitly load the default and/or legacy providers.

But per my experience with attempting to connect to a Wi-Fi network on a RHEL9 / OpenSSL 3.0 laptop booted in FIPS mode, attempts to call cryptographic functions that are not FIPS-validated will fail when the system has been booted in FIPS mode, regardless of what providers you explicitly load. Because that is the entire point of booting a Linux system in FIPS mode: to ensure that all cryptographic providers refuse to provide cryptographic functions that are not FIPS-validated.

The bottom line is that in 2023, you must assume that requesting any of (OpenSSL, GnuTLS, NSS, the Linux kernel) to perform legacy cryptographic functions (MD5, SHA-1, et. al.) will fail, with no possibility of avoiding that failure. Migrating clamav to using SHA-2 or (preferably) SHA-3 hash functions, and removing any/all uses of MD5 or SHA-1, should be a priority for clamav development.

But as per the hostap mailing list thread (linked above), if you cannot immediately rewrite your code so that it no longer needs to call legacy cryptographic functions, the best stopgap measure is to provide internal implementations of those functions, a la hostap.

Finally, I agree with @PaulSD that it is inappropriate to use MD5 to validate the digital signature of a CVD file. MD5 is so hopelessly broken that an attacker with even moderate resources (e.g., an AI/ML system) would likely be able to forge CVD signatures.

[PaulSD]

The EVP_MD_CTX_FLAG_NON_FIPS_ALLOW flag was removed in OpenSSL 3.0

Good point; A lot has changed in OpenSSL 3, and ClamAV has not yet been updated to work around FIPS with OpenSSL 3. In order to resolve this issue on systems using OpenSSL 3, some code updates will also be needed, although I think those will be relatively easy/minor (some details are below).

However, I have to disagree with a couple of your other statements.

But first, let me start with some additional details on OpenSSL 3:

• A prerequisite to using MD5 in OpenSSL 3 is that the default provider must be loaded.
  • On RHEL9 systems, the default provider is loaded by default regardless of whether FIPS mode is enabled or not. This is configured in /etc/pki/tls/openssl.cnf. (The user can disable the default provider entirely by editing or overriding openssl.cnf, but RHEL9 has it enabled by default.)
  • When FIPS mode is enabled (if either the OPENSSL_FORCE_FIPS_MODE env var or /proc/sys/crypto/fips_enabled is set), RHEL9 systems automatically load the base and fips providers (in addition to any providers configured in /etc/pki/tls/openssl.cnf). This enables FIPS mode to be enabled/disabled globally without making any changes to openssl.cnf files.
• EVP_MD_CTX_FLAG_NON_FIPS_ALLOW technically still exists in OpenSSL 3 (#ifdef will find it). However, the flag is ignored in OpenSSL 3, so setting it does not permit you to use MD5 in FIPS mode.
• OpenSSL 3 supports using "properties" to select/filter algorithms, and this is used to control the FIPS restrictions on RHEL9 (replacing EVP_MD_CTX_FLAG_NON_FIPS_ALLOW among other things).
  • Users can configure default_properties in openssl.cnf. Applications can then override those default properties if needed.
  • A provider=fips property may be used to limit access to only the algorithms provided by the fips provider. Alternatively, fips=yes may be used to limit access to algorithms in any provider that are marked as "FIPS algorithms" (the docs state that there are some non-crypto FIPS algorithms in the default and base providers that match fips=yes but not provider=fips).
  • When FIPS mode is enabled (determined as described above), RHEL9 automatically adds fips=yes to default_properties. Again, this enables FIPS mode to be enabled/disabled globally without making any changes to openssl.cnf files.
  • ClamAV can override this default property (when appropriate) by calling EVP_MD_fetch(NULL, "MD5", "-fips") instead of EVP_get_digestbyname("MD5").

To show this in action:

$ cat /etc/redhat-release 
Red Hat Enterprise Linux release 9.1 (Plow)
$ fips-mode-setup --check
FIPS mode is enabled.
$ echo | openssl dgst -md5
Error setting digest
801B786FE57F0000:error:0308010C:digital envelope routines:inner_evp_generic_fetch:unsupported:crypto/evp/evp_fetch.c:349:Global default library context, Algorithm (MD5 : 97), Properties ()
801B786FE57F0000:error:03000086:digital envelope routines:evp_md_init_internal:initialization error:crypto/evp/digest.c:237:
$ echo | openssl dgst -md5 -propquery '-fips'
MD5(stdin)= 68b329da9893e34099c7d8ad5cb9c940
$

To dispute some statements:

attempts to call cryptographic functions that are not FIPS-validated will fail when the system has been booted in FIPS mode, regardless of what providers you explicitly load

As described above, this is due to the fips=yes "property", and it can still be overridden (when appropriate) at the application level, just using a different method/flag than was used in OpenSSL <3.

That said, I do agree with this statement: that is the entire point of booting a Linux system in FIPS mode: to ensure that all cryptographic providers refuse to provide cryptographic functions that are not FIPS-validated.
Applications MUST be careful to override FIPS ONLY when performing operations that are not security-sensitive, otherwise they would be defeating the point of FIPS mode, at which point it would be better to just let FIPS mode break things so that users are forced to acknowledge that the software is not FIPS compliant and deal with it accordingly (eg. by disabling FIPS mode, or finding alternative solutions or mitigations, depending on the user's use case and risk profile).

the best stopgap measure is to provide internal implementations of those functions, a la hostap

I would argue that implementing your own crypto (even legacy crypto) is never a good idea, and internalizing or statically linking a random non-FIPS-compliant library/implementation just to work around FIPS issues isn't much better.
My understanding is that hostap provides its own internal implementations primarily to support unit testing, not for use in production systems.
I really think that the better stopgap is to jump through whatever hoops are necessary to allow legacy algorithms in a mainstream library to continue to be used.

Migrating clamav to using SHA-2 or (preferably) SHA-3 hash functions, and removing any/all uses of MD5 or SHA-1, should be a priority for clamav development.

I agree that modern hash functions are always preferable to obsolete ones, and it would be nice if all uses of MD5 in ClamAV could be replaced. However, let's not make perfect the enemy of good.
For example, I think using MD5 to match scanned files against a list of known bad files is perfectly reasonable. This is clearly out of the scope of FIPS, and the worst case is ... what? Maybe someone manages to generate a non-malicious file that matches as a false positive?
Conversely, using MD5 to match scanned files against a list of known good files is less safe. That is still clearly out of the scope of FIPS, but the worst case is someone manages to generate a malicious file that matches as a false negative.
However, even in that case, if I had to choose between the ClamAV team spending a lot of time replacing this use of MD5, or spending the same amount of time significantly improving/extending other aspects of the ClamAV scanning capabilities, I might choose the latter.
For now, I think it's worth focusing on the parts of ClamAV that are clearly both security sensitive and non-FIPS-compliant.

[tdewolff]

This is preventing me from using ClamAV. What is a workaround? Can I disable signature checking in freshclam?