PreHash SLH-DSA

[PranavSenthilnathan]

Adds PreHash mode for SLH-DSA. In pre-hash mode OpenSSL requires the caller to encode the message, so we have to build the pre-hash message (see one of the Helpers.cs files for implementation).

Some open questions/design choices:

• Section 10.3 (and 10.2) allow construction of the message outside the module where signing/verification happens. But it specifies that the hash or XOF function must be from a FIPS validated module. Because we take the hash as input, we can't guarantee this. Technically, we are not even implementing hash_slh_sign because we don't do the hash/XOF ourselves. Do we want to support another method on SlhDsa that would fully implement hash_slh_sign?
  • Similar to external mu it can use incremental hashing internally
• HashAlgorithmName for "SHAKE128" - this can represent the arbitrary output length XOF or the fixed 256-bit output restriction defined in RFC 8702. It's going be context specific so we should just be aware of this when we write helper methods.
• HashAlgorithmName - OpenSSL does not even look at the hash OID in the encoded message so technically anything is allowed. Windows likely will only accept known algorithms. We need to decide how we want to validate this at the base class level. In this PR we are restrictive and this allows also validating hash length.

Contributes to #113506

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-security, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

[vcsjones]

Do we want to support another method on SlhDsa that would fully implement hash_slh_sign?

Similar to external mu it can use incremental hashing internally

My 2c... I don't think so. It makes sense for mu because constructing it is not straight-forward. Mu requires SHAKE, glueing together lengths and the context... etc. PreHashML-DSA is just a hash and OID.

must be from a FIPS validated module. Because we take the hash as input, we can't guarantee this

We can't iron-clad guarantee it with OpenSSL either if someone is doing something weird with Providers, either. I think though it's generally okay to expect anyone calling PreHash, with FIPS concerns in mind, to understand that they need to use a FIPS module. In that sense I don't see it any different than RSA.SignHash.

[bartonjs]

The tricky thing here is we want to guess "if OSSL adds primary support for Pre-HashSLH-DSA in the future, what will their API look like?". It's not this.

We can leave the function here as SlhDsaSignPreFormatted if we don't feel that we can adequately anticipate if they'd want a string OID, a binary OID, a string algorithm name, a NID, or whatever... but if we had a sense that they were adding something in OSSL 3.6 then we'd want to go ahead and make the shape be right here so we just needed to patch the body to say "if you know how to call this yourself, do it; otherwise we'll call it the preformatted way".

[PranavSenthilnathan]

I am not really sure what OpenSSL will do for this, but the docs say:

Currently OpenSSL does not support the Pre Hash variant as this does not sit well with the OpenSSL API's. The user could do the encoding themselves and then set the settable to not encode the passed in message.

So SlhDsaSignPreEncoded might be appropriate. Their comment also makes it sound like pre-hash might require API changes on their side so it's hard to predict what the final shape will be.

[bartonjs]

One of these days we'll get around to exposing AsnEncoder to get the non-allocating version of turning the OID into bytes 😄.