EVP_DigestSign fails when called second time in 3.2.0 #23075
Comments
|
Looks like this was introduced as part of commit 3fc2b7d , in which EVP_MD_CTX's were marked as finalized to prevent them from being reused without re-initalization. The proper call flow would be to insert a new call to EVP_DigestSignInit between the two EVP_DigestSign calls. This may constitute a regression though |
nhorman
added
severity: regression
mattcaswell
added
triaged: bug
|
Hmmm. This was introduced as a deliberate decision to force incorrect reuse of a ctx without re-initialising them. It's unclear though whether this is really a regression or incorrect use of the API. Adding an OTC hold to discuss this. OTC Question: Is it a regression that we now require the |
|
Did it produce correct signatures? I.e. the same as if DigestSignInit() was called in between? |
|
@t8m , it didn't get that far. The output message from the second call is correct, but only because it was correct from the first call and unmodified in the second. The second call errored out before making any modifications, due to the first call setting EVP_MD_CTX_FLAG_FINALISED in the ctx flags |
|
What's a rationale to reinitialize the context for each signature if the key is not changing? |
|
@nhorman I mean not with the 3.2.0 but with previous versions where there was no check for that flag. |
The expected sequence with hash-then-sign algorithms which allow streaming is EVP_DigestSignInit->EVP_DigestSignUpdate->...->EVP_DigestSignFinal The expected sequence with algorithms which do not allow streaming is EVP_DigestSignInit->EVP_DigestSign However this sequence can be used with streaming algorithms as well and it leaves the digest in the finalized state. It cannot get into the initialized state without calling EVP_DigestSignInit again. That it worked before to call EVP_DigestSign multiple times was just by chance. |
|
I was under impression that EVP_DigestSign is just a single call for Init/Update/Final, e.g. EVP_DigestSignInit should be called inside EVP_DigestSign if necessary. |
|
@t8m ah, I can try this afternoon, but I expect it would, though seemingly only by happenstance, given that the input buffers are unchanged, and the output buffer is re-written |
|
Second call with different input causes wrong signature, that's how we have found the problem. |
|
@orignal so you're saying that in 3.1, it was still broken, just in a different way? I.e. it completed successfully, but produced incorrect outputs? If so, then this seems like this isn't a regression, just a correction to properly report the error |
No, everything was fine in 3.1. The problem exists in 3.2 only. Even if you don't check the return code of EVP_DigestSign, it still produces wrong signature being called second time. |
|
It does not produce any signature in 3.2.0, it just returns failure. |
Add version constraint openssl-dev<3.2.0 due to openssl/openssl#23075
|
OTC: We need to investigate whether the original code produced valid signatures in the subsequent EVP_DigestSign() calls. |
|
The documentation for EVP_DigestSign() has this sentence:
Doesn't this suggest that if there was no error, it can be called again without reinitializing? Note: further down in the documentation, there is this:
It's documented that EVP_DigestSign() essentially finishes off the same way as EVP_DigestSignFinal(), so the docs leave it a bit unclear that there's now a need to call EVP_DigestSignInit() again. |
|
Removing the OTC hold until the investigation is done. |
|
I performed these Tests: For X25519 multiple calls to EVP_DigestSign() resulted in the same signature output being produced. Debugging the 3.1 code: In 3.1. the code for EVP_DigestSign() has 2 pathways for non legacy For ECDSA it takes the EVP_DigestSignUpdate() and EVP_DigestSignFinal() path X25519 is a bit 'special' because it handles it digests internally (since it is a hardwired name) And this is why it handles multiple calls in this case -since the hash_ctx is created each time. In later versions the following code was added to the EVP_DigestSign() which obviously makes this break with multiple calls. I think because of this 'undefined behavior' it would be better to leave this alone and require the DigestInit to happen again. This new check was added in #20375. |
|
@slontis Thank you for this thorough analysis. Given the different behavior for different algos on 3.1 we should IMO keep the new behavior as is - i.e., fail if EVP_DigestSignInit is not called. We should probably document it explicitly. |
|
OTC: What is the preferred resolution given the analysis above? |
|
@slontis Yeah I am of the opinion that initialization should always be required. Although OpenSSL internal default provider can cope with reuse in some cases, that is not a given for all providers (and some will probably not be able to), nor for all signature schemes that could be added in the future. I do not believe the Init() function to be such an overhead that would warrant keeping an ambiguous behavior, if performance is an issue I would rather focus on optimizing the Init() function so that it has as low re-init overhead as possible within specific provider implementations rather than leaving ambiguous and potentially unsafe behavior available. I guess the semantics of EVP_DigestSign could be change to transparently call a re-init internally, I am not sure it is worth it though, it will complicate the code and cause double calls to provider init functions in the default case, which may fail to work with current providers. |
|
It's not an overhead, but it might break the functionality of existing project, since this requirement was not mentioned in the specs before. |
Understood, but it was totally undefined behavior, so a hard choice should be made in this case. |
|
OTC: It is correct that EVP_DigestSign fails when called a second time. This behaviour should be documented. |
mattcaswell
added
triaged: documentation
Fixes openssl#23075 In OpenSSL 3.2 EVP_DigestSign and EVP_DigestVerify were changed so that a flag is set once these functions do a one-shot sign or verify operation. This PR updates the documentation to match the behaviour. Investigations showed that prior to 3.2 different key type behaved differently if multiple calls were done. By accident X25519 and X448 would produce the same signature, but ECDSA and RSA remembered the digest state between calls, so the signature was different when multiple calls were done. Because of this undefined behaviour something needed to be done, so keeping the 'only allow it to be called once' behaviour seems a reasonable approach.
Fixes #23075 In OpenSSL 3.2 EVP_DigestSign and EVP_DigestVerify were changed so that a flag is set once these functions do a one-shot sign or verify operation. This PR updates the documentation to match the behaviour. Investigations showed that prior to 3.2 different key type behaved differently if multiple calls were done. By accident X25519 and X448 would produce the same signature, but ECDSA and RSA remembered the digest state between calls, so the signature was different when multiple calls were done. Because of this undefined behaviour something needed to be done, so keeping the 'only allow it to be called once' behaviour seems a reasonable approach. Reviewed-by: Richard Levitte <levitte@openssl.org> Reviewed-by: Neil Horman <nhorman@openssl.org> Reviewed-by: Tomas Mraz <tomas@openssl.org> (Merged from #23834) (cherry picked from commit 5e908e6)
Fixes #23075 In OpenSSL 3.2 EVP_DigestSign and EVP_DigestVerify were changed so that a flag is set once these functions do a one-shot sign or verify operation. This PR updates the documentation to match the behaviour. Investigations showed that prior to 3.2 different key type behaved differently if multiple calls were done. By accident X25519 and X448 would produce the same signature, but ECDSA and RSA remembered the digest state between calls, so the signature was different when multiple calls were done. Because of this undefined behaviour something needed to be done, so keeping the 'only allow it to be called once' behaviour seems a reasonable approach. Reviewed-by: Richard Levitte <levitte@openssl.org> Reviewed-by: Neil Horman <nhorman@openssl.org> Reviewed-by: Tomas Mraz <tomas@openssl.org> (Merged from #23834) (cherry picked from commit 5e908e6)
Fixes openssl#23075 In OpenSSL 3.2 EVP_DigestSign and EVP_DigestVerify were changed so that a flag is set once these functions do a one-shot sign or verify operation. This PR updates the documentation to match the behaviour. Investigations showed that prior to 3.2 different key type behaved differently if multiple calls were done. By accident X25519 and X448 would produce the same signature, but ECDSA and RSA remembered the digest state between calls, so the signature was different when multiple calls were done. Because of this undefined behaviour something needed to be done, so keeping the 'only allow it to be called once' behaviour seems a reasonable approach. Reviewed-by: Richard Levitte <levitte@openssl.org> Reviewed-by: Neil Horman <nhorman@openssl.org> Reviewed-by: Tomas Mraz <tomas@openssl.org> (Merged from openssl#23834)
I need sign series of message with EdDSA.
I create EVP_PKEY from a private key, then initialize EVP_MD_CTX using EVP_DigestSignInit and then call EVP_DigestSign multiple times for each message.
It worked fine in 3.1.0 and EVP_DigestSign fails when called second time in 3.2.0.
See the code below, I'm trying to sign and verify result using a test vector from RFC 8032.
It's successive after the first call and fails after the second call.
The text was updated successfully, but these errors were encountered: