PK callbacks: add build option to give full data to TLS 1.3 RSA-PSS sign callback instead of hash

[cconlon]

Description

This PR adds a new define (TLS13_RSA_PSS_SIGN_CB_NO_PREHASH) which changes the behavior of the PK callbacks RSA-PSS sign callback only for TLS 1.3. Instead of passing the callback the pre-hashed data, it passes the data itself and lets the callback do the hashing.

Some external sign implementations which users may be calling from this callback expect to hash the data themselves prior to the RSA-PSS sign operation.

In TLS 1.3 the "data" that is normally hashed internally by wolfSSL is created in src/tls13.c inside CreateSigData(), and is a concatenation of Prefix | Label | Handshake Hash. In TLS versions prior to TLS 1.3, this data is just the handshake hash. This define only changes the behavior for TLS 1.3, and the RSA-PSS sign callback when called from other TLS protocol versions will still pre-hash the data, since it is a running hash of the handshake messages calculated internal to wolfSSL as the handshake progresses.

ZD 18349 - User confirmed this worked (9/4/24)

Testing

Tested using the example client/server:

./configure --enable-pkcallbacks CFLAGS="-DTEST_PK_PRIVKEY -DDEBUG_PK_CB -DTLS13_RSA_PSS_SIGN_CB_NO_PREHASH"
make

./examples/server/server -P -v 4
./examples/client/client -P -v 4

Checklist

• added tests
• updated/added doxygen
• updated appropriate READMEs
• Updated manual and documentation