Finalize private key deserialization/serialization

[briansmith]

When we generate an RSA, ECDSA, or Ed25519 key, we need to be able to serialize it into some binary format for storage. There are a myriad of formats to choose from, but supporting a myriad of formats is counter to the goals and design of ring. I propose, instead, that we support only PKCS#8 format for all keys, and document how users can convert their keys to PKCS#8 format, specifically the version specified in RFC5958.

There is a simple unencrypted form and a more complex encrypted form. I propose that we start by supporting the simple unencrypted form and then add the encrypted form later.

References

• RFC5958 is the IETF Standards Track version of PKCS#8.
• RFC5208 was the earlier version of the IETF spec, which has only Informational status. But, I suspect most software was written against this spec or the original RSA version, so it might be more interoperable.
• draft-ietf-curdle-pkix-newcurves-00 is the work on adding Ed25519/X25519 support to PKCS#8 and other formats. There is currently debate about the encoding and it may change substantially.
• The command below are taken from the documentation at https://www.openssl.org/docs/manmaster/apps/pkcs8.html, modified to output the binary (not PEM) form.
• The format of ECPrivateKey: https://tools.ietf.org/html/rfc5915#section-3

Unencrypted

This is OneAsymmetricKey, or PrivateKeyInfo in the older version of the spec.

To convert a traditional (PKCS#5?) private key to the supported form using a newish version of OpenSSL:

openssl pkcs8 -in key.pem -topk8 -nocrypt -out enckey.pk8 -outform der

Encrypted

This is EncryptedPrivateKeyInfo and also EncryptedPrivateKeyInfo in the older version of the spec.

To convert from the traditional (PKCS#5?) format to this format using a newish version of OpenSSL, using the default parameters of AES-256 (which mode?) and PBKDF2-HMAC-SHA256 (what parameters?):

openssl pkcs8 -in key.pem -topk8 -out enckey.pk8 -outform der -v2 aes256 -v2prf hmacWithSHA256

The design of this API needs to take the acceptable key stretching algorithms (PBKDF2-HMAC-SHA256, PBKDF2-HMAC-SHA384, etc.) and the acceptable encryption/authentication modes (which ring might not even support yet!) as parameters to avoid hard-linking any algorithms.

Unanswered Questions

• Is the format specified in RFC5958 compatible with enough software, in particular not-the-newest OpenSSL? Or, should we support the original(-ish) PKCS#8 in RFC5208?
• Besides the OpenSSL commands mentioned above, what other commands (for other toolkits/libraries) will convert a key to the right format?
• Will Ed25519 users accept the use of ASN.1 for storing Ed25519 keys? It has the advantage of being self-describing, in particular being able to note the difference between prehash and non-prehash keys. It has the disadvantage of being ASN.1, which people who like Ed25519 mostly hate.
• Do things that generate PKCS#8 actually generate DER, or do they use non-DER BER features? Do we have to care about those things that don't output DER? Perhaps we should have a separate tool to deal with those kinds of keys, so we don't have to add BER support into the core library.
• Can we get away with not supporting the PEM (Base64-shrouded) format, and only supporting the binary DER format?
• If we build a tool, then that tool could also do things like normalize RSA keys so that p > q and recover the CRT parameters if they are missing, and convert the OpenSSH bcrypt-based format to PKCS#8.

[briansmith]

Regarding OpenSSL support:

According to openssl/openssl@8fc06e8, the default for OpenSSL's pkcs8 command was changed to AES-256-CBC and PBKDF2-SHA-256 very recently. Previously, the defaults were "DES and MD5" (not even 3DES, but original DES) and PBKDF2-HMAC-SHA1.

The -v2prf option was added very recently, in openssl/openssl@5693a30. It might be the case that anything that supports PBKDF2-HMAC-SHA256 at all already has it as the default. But, adding the -v2prf option to the documentation might help people more easily see that they're using an old version of the tool that doesn't support PBKDF2-HMAC-SHA256.

Support for the -v2 option was added a long time ago, in openssl/openssl@8eb57af. So, it seems like OpenSSL should pretty much always be able to generate things encrypted in AES-CBC mode.

ECC support was added a while back, it seems: openssl/openssl@0fbffe7.

Not sure how long ago AES support was added.

[briansmith]

Based on the above investigation, we may want to add support for AES-{128,256}-CBC encryption and PBKDF2-HMAC-SHA1 for the purpose of being able to import OpenSSL encrypted keys. I'm not sure how valuable it is to support such importing though. I suspect most web server keys stored in files are unencrypted, as the web server typically needs to automatically load the key so a password rarely makes sense.

[briansmith]

One issue BoringSSL ran into was that OpenSSL, until very recently, generated EC private keys using the specifiedCurve form, not the namedCurve form. RFC5915, which seems to be the ECC companion to RFC5958, only specifies the namedCurve choice. This makes it more likely that we'd have to write our own normalization code and/or support the older PKCS#8 spec, at least for EC keys.

[briansmith]

evp_extra_test.cc in BoringSSL contains some test data that will probably be worth looking at for this.

[aidanhs]

What are your feelings about PKCS#12?

I'm looking through what it would hypothetically take to add rustls as an option to native-tls and one of the bits missing seems to be PKCS#12 support which is part of the common interface exposed by native-tls regardless of which tls implementation you're using.

It likely wouldn't need to be a complete implementation (I understand the full spec is quite complex), one that can handle retrieving a private key + chain of certificates (as well as passwords) would be sufficient (which openssl and boringssl both do).

Two comments up it sounds like you're saying that the PKCS#12 code in boringssl may have issues with some private keys generated by openssl so it's not quite as simple as re-adding crypto/pkcs8/. Since I'm more interested in a prototype for now rather than something fully functional, I might get the relevant code from boringssl and expose it in a temporary crate in a git repo.

[briansmith]

What are your feelings about PKCS#12?

PKCS#12 is about certificates, which is something at a higher layer than what ring does. I don't think it makes sense to do PKCS#12 in ring, but it may make sense to do it in webpki. As far as ring is concerned, the main issue is whether ring's API allows a PKCS#12 implementation to be written on top of it. For RSA, we support the bare RSAPrivateKey serialization, which is embedded (IIRC) in the PKCS#12 serialization in some way. So, one should be able to make a PKCS#12 parser that then uses RSAKeyPair::from_der() to create private key pairs in ring.

I'm looking through what it would hypothetically take to add rustls as an option to native-tls and one of the bits missing seems to be PKCS#12 support which is part of the common interface exposed by native-tls regardless of which tls implementation you're using.

It would be helpful to know whether @ctz wants to support PKCS#12 in Rustls. If so then we could consider adding it to webpki, in which case we should discuss it in a new issue in the webpki repo.

FWIW, I'll be working on adding PKCS#8 support to ring sometime before March 31st.

[aidanhs]

It would be helpful to know whether @ctz wants to support PKCS#12 in Rustls. If so then we could consider adding it to webpki, in which case we should discuss it in a new issue in the webpki repo.

I don't think it's necessary to put it in rustls - if I can use PKCS#12 deserialisation from webpki, I can then pass the chain and private key to rustls so it doesn't need to concern itself with where they've come from. If other crates then pop up with alternative certificate format deserialising then in theory rustls doesn't need to be changed.

Raised briansmith/webpki#41

FWIW, I'll be working on adding PKCS#8 support to ring sometime before March 31st.

That sounds good.

[ian-p-cooke]

has there been any interest in adding support for encrypted PKCS#8? I was looking for this feature as I want to generate and store private keys for an end-user application. I have to store the key in the user's home dir and don't want some other application to be able to use it.

[briansmith]

has there been any interest in adding support for encrypted PKCS#8? I was looking for this feature as I want to generate and store private keys for an end-user application. I have to store the key in the user's home dir and don't want some other application to be able to use it.

I don't anticipate working on encrypted PKCS#8 unless/until somebody sponsors it. I think there are much better alternatives to encrypted PKCS#8 and I'd rather implement those instead.

[briansmith]

As of now, all signing keys support serialization and deserialization via PKCS#8, and that's good enough to close this. Feel free to file new, specific, issues, regarding any deficiencies or missing functionality here.

[shi-yan]

how can I do serialization for agreement::EphemeralPrivateKey ? I wasn't able to find an API?