FreeRDP crashes on system in FIPS mode

[ondrejholy]

I enabled FIPS mode on Red Hat Enterprise Linux [1] and configured Microsoft Window Server to require FIPS-compliant encryption [2]. Consequently, xfreerdp crashes with the following error, when I try to connect to the server regardless of used security protocol:
md4_dgst.c(74): OpenSSL internal error, assertion failed: Digest MD4 forbidden in FIPS mode!

I see several occurrences of MD4, MD5, RC4 algorithms in the FreeRDP codes. Those algorithms are not FIPS-compliant. However, I suppose that those algorithms should not be used if FIPS-compliant encryption is used...

[1] https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Security_Guide/chap-Federal_Standards_and_Regulations.html
[2] https://technet.microsoft.com/en-us/library/cc770833.aspx

[hardening]

I guess you have done your tests with master ?

[ondrejholy]

Yes, I did the tests with master and freerdp-1.0, both versions fail in the same way...

[bmiklautz]

@ondrejholy i might be mistaken but I think when you enable FIPS compliant security this applies only to the transport itself. MD4, MD5, .. might still be used/required in different other parts like for generating ntlm credentials.

[ondrejholy]

As far as I know, FIPS regulations applies on all used crypto algorithms. FreeRDP works correctly with server in FIPS mode, however doesn't work itself on a client system with enabled FIPS mode. NLA security can't be used with FIPS mode probably, because NTLM is not FIPS compliant, because it requires RC4 (see section 5.1 of [MS-NLMP]), and Kerberos is not implemented in FreeRDP. However TLS/RDP security should work with FIPS mode. Triple DES should be used for bulk encryption, and SHA-1 for any hashing operations (e.g. MAC generation). See sections 5.3.5/5.3.6 from [MS-RDPBCGR] for RDP security and section 5.4.1 for Enhanced RDP security. MD5/RC4 is still used in some cases, but I am convinced that it should be replaced by SHA-1 if FIPS encryption is used. Just a note that native Remote Desktop Connection client works correctly on system with enabled FIPS mode.

[kfiresmith]

Hello,
We too are working on testing FIPS 140-2 compliance and came across this on RHEL 7 (freerdp-1.0.2-6.el7_2.1).

From what @bmiklautz posted, does this mean we are just out of luck due to Windows RDP server services requiring bad cryptos?

[ondrejholy]

Hmm, FreeRDP seems to work for me on systems in FIPS mode with the patches I added in my fips branch. It can be used only with TLS security, because NLA is based on NTLM as I already mentioned and RDP security seems to use proprietary certificates which also needs MD5 (section 5.3.3 from [MS-RDPBCGR]). The first patch (63c6717) seems to be correct, however the second one (3c2a8fd) is just workaround to make it work. I am not really sure what to do with it, because the code (skipped by workaround) is needed by Platform Challenge messages. Those messages were not used when I was testing it, but I am afraid that they are used in some cases. Unfortunately [MS-RDPELE] doesn't contain notes regarding this :-(

[hardening]

AFAICT FIPS mode is only in RDP security mode or I'm missing something ?

[kfiresmith]

Thanks for the reply @ondrejholy (though I'm not sure I fully grok it).

I'll be tracking the issue via my Red Hat customer support ticket and I'll be on the lookout for a patched / QA package from them to test with. Our environment is moving toward full Windows 10 / WS2012+ and RHEL 7 so that'll be the client/server connection I'll be targeting for a fix and won't be making much effort to fix it on RHEL 6.

[ondrejholy]

I've tried wfreerdp on Windows with enabled "System cryptography: Use FIPS compliant algorithms for encryption, hashing, and signing" policy and it also works correctly, so it seems to me that this policy is something else than fips_mode on Linux... Maybe I am really trying to achieve something impossible.

[bjcollins]

I have done some research here and have some comments/opinions on workarounds/solutions:

1. I think the licensing code that @ondrejholy is working around in the commits referenced here will be needed as the code will be hit whenever there is a need for a new/updated license. The problem is that freerdp relies on RC4/MD5 from openssl and these algorithms are disallowed in openssl on RHEL6/7 with fips=1 set as a kernel argument. So, we would need to either override openssl to allow these algorithms(not sure this is possible), provide internal implementations of these in freerdp for use here, or use some other crypto library for these algorithms in this case. I think the important thing here is that the licensing packet itself is already encrypted so the use of RC4/MD5 is not providing any cryptographic security but instead is probably just a relic of a part of the protocol that was not changed to use better algorithms because it was not a necessary change.

2. I agree without kerberos that NLA should not be used in FIPS mode since NTLM is not cryptographically secure. I think the only way to use NLA with FIPS would be to support/force kerberos.

3. I think RDP security might still be able to be used when set freerdp to only allow the FIPS encrpytion method. If the only issue is the server proprietary certificate validation then we could potentially just not do it at all. The validation here in freerdp just logs a warning if the md5 hash comparison fails, the reason being that we are not really protecting against anything there. It looks like the key for the proprietary server certificate is well-known and documented in section 5.3.3.1.1 of MS-RDPBCGR. So, I think it could be reasoned to not bother trying to validation at all. Another would be to provide the same workaround as my (1) above since we are not relying on MD5 for security here.

I think the big things here are missing kerberos support for NLA in freerdp and the RDP protocol continues to use bad cryptos in some case even when set to FIPS, and I believe the reason Microsoft has not bothered changing this is because these cases are not relied upon for security. Of course, this causes problems for freerdp since it relies on openssl for all crypto since openssl adheres strictly to fips.

Any feedback or hole-punching in my opinions are welcome.

[ondrejholy]

Sorry, I had to stop working on that due to other issues, however, I will add a quick note. Yes, it is possible to use RC4/MD5 from OpenSSL even with fips=1 and I agree that it is ok for licensing code, because it is not relevant to security, and probably also the same for the certificate validation. It can be done over EVP_ functions (which needs to be used due to OpenSSL 1.1.0 anyway) thanks to the EVP_*_FLAG_NON_FIPS_ALLOW flags. Kerberos support is also on my todo...

[bjcollins]

@ondrejholy Thanks for the info about the openssl override. It seems like overriding the particular cases deemed acceptable from a security standpoint would be the way to go. I would think freerdp should have a fips specific command-line argument to make sure openssl is in fips mode, force "/encryption-method:FIPS", force "-sec-nla" for now, and override crypto algorithms if absolutely necessary and not a security risk. Right now I am gathering up information on what we need to be done to get xfreerdp to work with FIPS, but am planning to look into doing some actual changes to that end in the near future time permitting.

[ondrejholy]

See #3904 which will be merged hopefully soon.

[DarkFenX]

What kind of input needed to have it merged & released anytime soon?

Launching full-blown windows install just to RDP to a few windows servers is quite a pita.

[hardening]

@DarkFenX well #3904 is merged from quite a long time now...

[DarkFenX]

My bad. Got to wait for release with this fix then.

[akallabeth]

Closing as resolved, please reopen if still an issue.