proposal: crypto/x509: add VerifyOptions.UnknownAlgorithmVerifier

[bwesterb]

Proposal Details

Context

It's unclear when and which post-quantum signatures Go should support, cf. #64537. There are use cases where migration is more urgent, and generally it's good to be able to start testing early. Presently the only way to support new algorithms in certificates in Go is to either fork Go, or to fork the x509 package. It'd be convenient if upstream Go allows verifying to it unknown algorithms using a callback.

API

Add a new field to VerifyOptions:

// UnknownAlgorithmVerifier specifies a callback to use to verify
// a signature with an unknown AlgorithmIdentifier.
UnknownAlgorithmVerifier func(alg pkix.AlgorithmIdentifier, signed, signature, pk []byte) error

When during Certificate.Verify an unknown AlgorithmIdentifier is encountered and the UnknownAlgorithmVerifier field is set, Certificate.Verify will call UnknownAlgorithmVerifier to verify the signature.

Notes:

1. We need access to the public key. Presently Certificate.PublicKey is not set when the AlgorithmIdentifier is unknown during parsing. Instead, we can set Certificate.PublicKey to &publicKeyInfo{...}. This might trip up users that assume PublicKey is nil in case of an unknown algorithm. (Are there any?)
2. Technically, the public key is an asn1.BitString. I do not know of any post-quantum signature scheme whose public keys aren't simply byte strings, and I do not expect them in the future. For simplicity, I'd say we'd only support byte strings.
3. Similarly, all post-quantum signature schemes so far do not use parameters, but only an OID. We could simplify matters a bit more by passing an asn1.ObjectIdentifier instead of a pkix.AlgorithmIdentifier.

Example usage:

package main

import (
        "crypto/x509"
        "crypto/x509/pkix"
        "encoding/asn1"
        "encoding/pem"
        "errors"
        "fmt"
        "github.com/cloudflare/circl/sign/schemes"
        "os"
)

func loadCert(path string) (*x509.Certificate, error) {
        raw, err := os.ReadFile(path)
        if err != nil {
                return nil, fmt.Errorf("ReadFile(%s): %w", path, err)
        }
        block, _ := pem.Decode(raw)
        if block == nil {
                return nil, fmt.Errorf("pem.Decode(%s) failed", path)
        }
        return x509.ParseCertificate(block.Bytes)
}

func main() {
        dsaCert, err := loadCert("ML-DSA-44.crt")
        if err != nil {
                panic(err)
        }
        kemCert, err := loadCert("ML-KEM-512.crt")
        if err != nil {
                panic(err)
        }

        roots := x509.NewCertPool()
        roots.AddCert(dsaCert)
        _, err = kemCert.Verify(x509.VerifyOptions{
                Roots: roots,
                UnknownAlgorithmVerifier: func(alg pkix.AlgorithmIdentifier,
                        signed, signature, pk []byte) error {
                        if !alg.Algorithm.Equal(asn1.ObjectIdentifier{2, 16, 840, 1, 101, 3, 4, 3, 17}) {
                                return errors.New("unsupported scheme")
                        }
                        scheme := schemes.ByName("ML-DSA-44")
                        ppk, err := scheme.UnmarshalBinaryPublicKey(pk)
                        if err != nil {
                                return err
                        }
                        if !scheme.Verify(ppk, signed, signature, nil) {
                                return errors.New("invalid signature")
                        }
                        return nil
                },
        })
        if err != nil {
                panic(err)
        }
}

[gabyhelp]

Related Code Changes

• crypto/x509: Add VerifyOptions.UnknownAlgorithmVerifier
• crypto/x509: hint that algo was not compiled in

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[mateusz834]

When should happen when UnknownAlgorithmVerifier is set and VerifyOptions.Roots is nil (System CAs). I am thinking about platform verifiers.

[mateusz834]

What if crypto/x509 at some point adds support for some unsupported algorithm, with this API i think we can consider that as a breaking change (UnknownAlgorithmVerifier no longer called, because it is now supported).

[mateusz834]

We need access to the public key. Presently Certificate.PublicKey is not set when the AlgorithmIdentifier is unknown during parsing. Instead, we can set Certificate.PublicKey to &publicKeyInfo{...}. This might trip up users that assume PublicKey is nil in case of an unknown algorithm. (Are there any?)

Maybe it should use RawSubjectPublicKeyInfo directly and parse the key from DER?

[seankhliao]

I think from #64537 (comment) it's quite clear that pluggable crypto is a non goal of the standard library.
A fork is the best choice if you wish to experiment.

[FiloSottile]

We discussed this with @golang/security while planning for Go 1.26, and we'd like to consider it as one of the options for allowing some post-quantum signatures before we land mainline support, which might be too soon for.

Generally, pluggable crypto itself is indeed a non goal. Here it's in conflict with enabling early PQ adoption, and with not implementing algorithms or modes that might not be the right long-term solution yet.

Not saying this is definitely what we'll go for, but let's discuss it. In particular, I would like feedback on whether this addresses an actual need, or if implementations will always also want TLS handshake integration, which has a much much higher complexity cost.

[bwesterb]

@FiloSottile wrote

In particular, I would like feedback on whether this addresses an actual need, or if implementations will always also want TLS handshake integration, which has a much much higher complexity cost.

We'd like both, but just this change is already helpful. One example is our certificate pipeline, which needs to check/process chains, but doesn't terminate our TLS connections.

@mateusz834 wrote

When should happen when UnknownAlgorithmVerifier is set and VerifyOptions.Roots is nil (System CAs). I am thinking about platform verifiers.

I propose we keep it simple, and ignore UnknownAlgorithmVerifier when using platform verifier.

What if crypto/x509 at some point adds support for some unsupported algorithm, with this API i think we can consider that as a breaking change (UnknownAlgorithmVerifier no longer called, because it is now supported).

Good point. Adding a new algorithm will also break users that assumed Go would error on not supporting an algorithm. This isn't generally considered a breaking change. That isn't quite the same as this. Can we call out this potential breakage with a warning and leave it at that?

Maybe it should use RawSubjectPublicKeyInfo directly and parse the key from DER?

Can do.

[mateusz834]

Good point. Adding a new algorithm will also break users that assumed Go would error on not supporting an algorithm. This isn't generally considered a breaking change. That isn't quite the same as this. Can we call out this potential breakage with a warning and leave it at that?

There is also an option to add something like:

AlgorithmVerifier func(alg pkix.AlgorithmIdentifier, signed, signature, pk []byte) error

That overrides entirely the signature checking, then we would not have such problem.

or even with a bool return:

AlgorithmVerifier func(alg pkix.AlgorithmIdentifier, signed, signature, pk []byte) (bool, error)

and when AlgorithmVerifier returns false, then the default crypto/x509 logic is used.