signature: change `DigestSigner`/`DigestVerifier` to be ML-DSA "external mu" compatible?

[tarcieri]

Originally signature had DigestSigner/DigestVerifier traits then added hazmat::{PrehashSigner, PrehashVerifier} trait. The original goal of the DigestSigner/DigestVerifier traits was to leverage type safety to ensure that the input is always a hash. We later added the prehash traits out of necessity, especially observing people utilizing hacks like impl'ing pseudo-Digests that emit the hash to work around the constraints of the API in environments where prehashes were being used but computed significantly earlier than the signature function.

The two traits end up being quite similar to the point I've thought about removing DigestSigner/DigestVerifier due to the overlap, which would decouple signature from the digest crate entirely. But instead of that, I think we can do a slight tweak to the API which would make it possible to impl these traits (but not hazmat::{PrehashSigner, PrehashVerifier}), in an ML-DSA external mu-compatible way. Here's a simplified version:

pub trait DigestSigner<D: Digest, S> {
    fn sign_digest<F: Fn(&mut D)>(&self, f: F) -> S;
}

...so instead of DigestSigner and DigestVerifier taking a user-supplied digest instance as they do today, the signer/verifier type which impls this trait instead initializes the provided Digest type themselves, then supply it to a callback function which is expected to perform a series of updates. This means the initialization can include hashing a leading message prefix, which is what's needed to implement an IUF-like API for external mu.

Such an API should work for all existing users of the signature::Digest* traits as well as ML-DSA.

cc @daxpedda

[tarcieri]

In #1880, I noted this shape could even work for vanilla EdDSA, so long as calling f repeatedly ensures the same sequence of updates are made to the digest.

[daxpedda]

The current DigestSigner/DigestVerifier do still make the most sense to me, even if they are apparently unused. While a method taking Fn(&mut D) would be more flexible, I still think it belongs into hazmat, as we can't really prevent misuse, unless we don't plan to use it with EdDSA.

I also think that in many cases this wouldn't offer enough flexibility. E.g. not being able to return a Result, so it would need to be something like Fn(&mut D) -> Result<(), ?>.

In cases like this, I always tend to turn around the API and let users handle this how they want externally, instead of traits and closures. E.g. we could do something like fn sign_digest_prepare(&self) -> PreparedHash<'_, D> and fn sign_digest_finish(&self, prepared: PreparedHash<'_, D>). Where PreparedHash can implement Digest and is just a wrapper around D and a reference to the SigningKey to verify it was produced by this specific key its passed to afterwards.

That way users can do whatever they like with PreparedHash, call it with io::Read, async, its very flexible.

But yeah, if we don't want to go with that API, I can't think of something better then Fn(&mut D) as well.

[tarcieri]

I also think that in many cases this wouldn't offer enough flexibility. E.g. not being able to return a Result, so it would need to be something like Fn(&mut D) -> Result<(), ?>.

That's done easily enough. However, your proposal is less flexible in that it can't cover the EdDSA case. Not that I'm sure either one actually should.

While a method taking Fn(&mut D) would be more flexible, I still think it belongs into hazmat, as we can't really prevent misuse

Assuming you're talking about EdDSA here, it could just be impl'd on a newtype which is in hazmat.

[daxpedda]

I also think that in many cases this wouldn't offer enough flexibility. E.g. not being able to return a Result, so it would need to be something like Fn(&mut D) -> Result<(), ?>.

That's done easily enough.

Might be easier addressed in the future with try_trait_v2, which is what Rust is trying to apply to all the new try_* methods (and why they are all stuck on nightly).

However, your proposal is less flexible in that it can't cover the EdDSA case.

Why would it not? Its just returning a Result, which on Err the implementation should just interrupt.

In any case, I was just pointing out that some users need a try version of the current proposal, it doesn't have to replace it.

[tarcieri]

I agree, it is not. Maybe it might be easier solved in the future with rust-lang/rust#84277, which is what Rust is trying to apply to all the new try_* methods (and why they are all stuck on nightly).

All of the signature traits already offer fallible and infallible APIs, so try_sign_digest can take a fallible Fn which returns a Result, and sign_digest an infallible one.

Why would it not? Its just returning a Result, which on Err the implementation should just interrupt.

I meant the prepare/finish API. I don't understand how something like that could implement a 2-pass algorithm like EdDSA. The callback is nice in that regard because it can be called twice.

[daxpedda]

So the plan is:

• Change DigestSigner/DigestVerifier to take a Fn(&mut D).
• Implement for ML-DSA via external mu.
• Open an issue at at ed25519-dalek to discuss exactly how we want to add this API?

Let me know and I'm happy to do the implementation.

[tarcieri]

@daxpedda yes, though *DigestSigner is a whole family of traits. We'd need AsyncDigestSigner to be e.g.:

pub trait AsyncDigestSigner<D, S> {
    async fn sign_digest_async<F: AsyncFn(&mut D)>(&self, f: F) -> S {
        self.try_sign_digest_async(|digest| f(); Ok()).await.expect("signing operation failed")
    }

    async fn try_sign_digest_async<F: AsyncFn(&mut D) -> Result<()>>(&self, f: F) -> Result<S>;
}

(though that said, I think that covers the async use case you mentioned earlier)