Using a provider for signing

[jjeffroberts]

libp11 is a PKCS#11 wrapper library. One thing it does is to forward operations using a private key to a smartcard -- the private key never leaves the card but the card provides encrypt/decrypt/sign operations.

In the current implementation, it uses RSA_set_method() to insert custom encrypt and decrypt methods.

Since RSA_set_method() is deprecated in OpenSSL 3.0 I'm trying to instead use a provider.

I have a rough-but-working provider that implements a "p11_rsa" cipher and I use EVP_CIPHER_fetch(), EVP_CipherInit() EVP_CipherUpdate() and EVP_CipherFinal() to invoke the encrypt and decrypt methods (which call back into libp11 to forward the operations to the card).

(I borrowed a lot from the "vigenere" example)

I'm stuck on how to replace RSA_sign() (also deprecated) with code that invokes the provider, however.

I think from reading this:

https://www.openssl.org/docs/manmaster/man7/provider-signature.html

That I probably need to implement these in my provider:

/* Signing */
int OSSL_FUNC_signature_sign_init(void *ctx, void *provkey,
                                  const OSSL_PARAM params[]);
int OSSL_FUNC_signature_sign(void *ctx, unsigned char *sig, size_t *siglen,
                             size_t sigsize, const unsigned char *tbs, size_t tbslen);

But I'm stuck on what is probably the easy part - how to call that code.

In simpler situations I've used EVP_PKEY_sign_init(), EVP_PKEY_CTX_set_signature_md(), EVP_PKEY_set_rsa_padding(), EVP_PKEY_sign.

Is there a way to use that flow but select my provider?

While googling and grepping I've come across EVP_SIGNATURE_fetch() and I'm going to go look for examples using that?

If anyone can nudge me in the right direction I would appreciate it.

[mattcaswell]

For signatures you need to:

1. Implement a key manager in the provider

2. Implement the relevant signature functions

See the following man page for information about writing a key manager:

https://www.openssl.org/docs/man3.1/man7/provider-keymgmt.html

This page gives information on the relevant "signing" functions:

https://www.openssl.org/docs/man3.1/man7/provider-signature.html

The function call EVP_PKEY_sign_init will be passed to the provider's OSSL_FUNC_signature_sign_init function, passing a reference to the key established via the key manager.

You might like to take some inspiration from the "fake-rsa" provider in our test code here:

https://github.com/openssl/openssl/blob/master/test/fake_rsaprov.c

[jjeffroberts]

I'm letting unit tests tell me what key management functions I need to add and I've gotten past encrypt and decrypt in cases where "my" keys stays in the same process as libp11.

Now I've found a place that wants to call i2d_PublicKey on my key and pass it to another process - is that going to work out for me?

I see a path in i2d_PublicKey() involving evp_pkey_is_provided() and i2d_provided() - would my provider need to implement an encoder or something to support that?

[mattcaswell]

You should not need to implement your own encoder as long as the key is exportable from your provider, and you have the encoders available from the base or default providers.

See OSSL_FUNC_keymgmt_export and OSSL_FUNC_keymgmt_export_types on the keymgmt provider man page I linked to above.

[jjeffroberts]

I have this almost working but I found a wrinkle that took me back a step.

I've realized/remembered that although libp11 sets custom RSA methods for encrypt and decrypt, it doesn't set a custom signing method.

What it does is call RSA_sign() with an RSA key that has a custom encrypt method, so it's doing part of what RSA_sign() does (encode the input with an internal encode_pkcs1() method) and then calling the custom encrypt method.

I'm not sure if there was a requirement that the interface to the card only use encrypt/decrypt or why that design choice was made.

I've temporarily borrowed the code for encode_pkcs1() just to show that calling that and then calling the custom encrypt method via the provider gives the same result as before.

I had asked this as a separate question before but I'm not sure I was very clear. Is there a 3.0-compliant way to replicate what encode_pkcs1() does without calling RSA_sign() ?