pdnsutil hash-password doesn't work on RHEL8

[v1shnya]

• Program: Authoritative
• Issue type: Bug report

Short description

'pdnsutil hash-password' gives - Error while hashing the supplied password: Hashing a password requires scrypt support in OpenSSL, and it is not available

Environment

• Operating system: RHEL 8
• Software version: PDNS Auth 4.7.2
• Software source: powerdns site - pdns-4.7.2-1pdns.el8.x86_64.rpm

Steps to reproduce

$ pdnsutil hash-password
Error while hashing the supplied password: Hashing a password requires scrypt support in OpenSSL, and it is not available

Other information

Team, first of all - thank you for this great product, I use it - I like it!

I believe that the issue could be related to the fact that during compilation time some macro left part of the code aside

https://github.com/PowerDNS/pdns/blob/auth-4.7.2/pdns/credentials.cc#L166

std::string hashPassword(const std::string& password, uint64_t workFactor, uint64_t parallelFactor, uint64_t blockSize)
{
#if !defined(DISABLE_HASHED_CREDENTIALS) && defined(HAVE_EVP_PKEY_CTX_SET1_SCRYPT_SALT)
<-Hashing code->
#else
throw std::runtime_error("Hashing a password requires scrypt support in OpenSSL, and it is not available");
#endif
}

Checked all the libraries like openssl.x86_64 , libsodium.x86_64, openssl-devel.x86_64 are installed on PowerDNS server

[rgacogne]

In the logs of our el-8 build I see:

checking whether EVP_PKEY_CTX_set1_scrypt_salt is declared... no

so either el-8's OpenSSL is lacking that function or our detection is broken.

[rgacogne]

According to the same logs el-8's OpenSSL version is 1:1.1.1k-9.el8_7 which in theory should have that function.

[rgacogne]

Ah, it's a function-like macro in OpenSSL 1.1.1 and a real function in 3.x, that might be a clue.

[rgacogne]

Hmm, we use AC_CHECK_DECL which in theory should deal with function-like macros, so at the moment I don't know what happens.

[Habbie]

https://gist.github.com/Habbie/e77ef088deada5672ef5e83d9aecbf29#file-build_el-8_4-8-0-beta1-196-sdistalpine318-g621fe9ca2-dirty_20230616-174523-log-L4325 is what happens

[rgacogne]

OK, I'm lost.. What I see in the log is:

In file included from conftest.cpp:69:
#39 55.13 /usr/include/openssl/kdf.h:33:46: error: 'va_list' has not been declared
#39 55.13  int EVP_KDF_vctrl(EVP_KDF_CTX *ctx, int cmd, va_list args);
#39 55.13                                               ^~~~~~~
#39 55.13 /usr/include/openssl/kdf.h:35:1: error: 'size_t' does not name a type
#39 55.13  size_t EVP_KDF_size(EVP_KDF_CTX *ctx);
#39 55.13  ^~~~~~
#39 55.13 /usr/include/openssl/kdf.h:35:1: note: 'size_t' is defined in header '<cstddef>'; did you forget to '#include <cstddef>'?
#39 55.13 /usr/include/openssl/kdf.h:15:1:
#39 55.13  # ifdef __cplusplus
#39 55.13 /usr/include/openssl/kdf.h:35:1:
#39 55.13  size_t EVP_KDF_size(EVP_KDF_CTX *ctx);
#39 55.13  ^~~~~~
#39 55.13 /usr/include/openssl/kdf.h:36:58: error: 'size_t' has not been declared
#39 55.13  int EVP_KDF_derive(EVP_KDF_CTX *ctx, unsigned char *key, size_t keylen);
#39 55.13                                                           ^~~~~~

As far as I can tell this does not match any released version of OpenSSL..
EVP_KDF_vctrl was added in openssl/openssl@5a285ad before openssl-3.0.0-alpha1 but the header issue was quickly fixed by openssl/openssl@0cf5c6a (also before 3.0.0-alpha1) and the function was completely removed by openssl/openssl@fb9e6dd (still before 3.0.0-alpha1). But the log is talking about openssl-1.1.1k-9.el8_7.x86_64.rpm..

[mnordhoff]

openssl-1.1.1-evp-kdf.patch in that package adds EVP_KDF_vctrl.

[Habbie]

I filed a ticket at RH: https://bugzilla.redhat.com/show_bug.cgi?id=2215856

[rgacogne]

RH is not going to fix it, unfortunately, apparently shipping a broken header by half-assing the backport of a new to a stable branch is not a big enough problem:

The change is introduced in openssl-1.1.1-evp-kdf.patch

As it's the only complaint during several years, it probably would be better to fix it in PowerDNS downstream. Unfortunately, we don't plan any changes in OpenSSL in 8.x beyond the CVE fixes.

Sorry for inconvinience

[rgacogne]

I'll see what I can do since we apparently cannot rely on Red Hat's expensive support.