Unable to use HMAC/PRF via openssl plugin when used in FIPS-mode with strongSwan 5.8.2 and newer

[tobiasbrunner]

Discussed in #515

^{Originally posted by tobiasbrunner Aug 16, 2021}
The openssl plugin announces support for PRF_HMAC_SHA2_256. However, creating an instance fails. Since 5.8.2, to ensure HMAC implementations are actually functional, HMAC_Init_ex() is called from the constructor by setting a key. However, that key is the empty string. So this consistently fails due to the length limit that's apparently enforced in FIPS-mode (I wasn't aware of that, is that documented anywhere or configurable?).

Strangely, that patch was done because someone used OpenSSL in FIPS mode and instantiating HMAC-MD5 worked only to fail later when actually using it, so it was not possible to fallback to the hmac plugin during construction (the use case there was RADIUS, which requires HMAC-MD5).

I guess we could use a longer key for the initialization, I pushed such a change to the 515-openssl-hmac-fips branch.