Slice out KeyManager into 2 parts: KeyRing, CertificateManager

[CMCDragonkai]

Specification

The KeyManager is doing 3 duties:

1. Managing the root key pair and symmetric DB key that encrypts the entire PK database.
2. Managing the root x509 certificate chain
3. Managing arbitrary subkeys that are used for arbitrary purposes

KeyRing

The first duty should be separated out into a KeyRing object. This object is a singleton that handles just the root key pair and symmetric DB key. It can be considered an Key Encapsulation Mechanism that enables PK to be a "hybrid cryptosystem". Many downstream domains that depend on the KeyManager can instead depend on the KeyRing, because they only require knowledge about the current node's NodeId. With the change to ed25519, the 32-byte/256-bit public key is now the NodeId, there's no need to hash the public key to get the NodeId. This has many advantages, such as the fact that our NodeGraph contains all the public keys we want to contact, and reduces the amount of chatter required to acquire the public keys of other nodes.

Furthermore now that the KeyRing is a smaller interface, the main goal is that we can pass it around more freely like a POJO. However just passing around KeyRing isn't going to change the behaviour of downstream objects that much. Therefore in the spirit of POLA/POLP, we are going to explore how to have the power of KeyRing object while being able to "extract" read-only dynamic/observable properties out of the KeyRing that can be easily synthetically constructed during mock-testing.

We find it difficult during prototyping time that one must asynchronously create the KeyRing before passing it in to downstream objects during mock-testing. This looks like:

const keyRing = await KeyRing.createKeyRing({ keysPath: './path/to/keys' ,password: 'abc123' });
await Domain.createDomain({ keyRing });

If we had a POJO instead, it would be easy to create synthetic key rings for any purpose.

// Note that with libsodium, there's also a `secretKey` property
const keyPair = { publicKey: ..., privateKey: ..., secretKey: ...,  };
await Domain.createDomain({ keyPair });

But as we know with #386 and #444, the keyPair is a dynamic object. It could be considered a ReadOnly interface at this point, but the downstream domains should be "reacting" to changes to the keyring. So suppose we could do this:

const keyRing = await KeyRing.createKeyRing({ keysPath: './path/to/keys' ,password: 'abc123' });
const nodeId = keyRing.nodeId.observe();
// In fact the Domains only require the `nodeId` and not the original key pair
await Domain.createDomain({ nodeId });

So the idea, is that we make use of Observable from rxjs and allow us the ability to "slice" out observable properties from the main singleton object of KeyRing, and pass just that observable property to downstream domains.

The rxdb interface shows ways of doing this.

At this point we would still have the KeyRing singleton object, and it will have the API methods of:

KeyRing.checkPassword
KeyRing.changePassword
KeyRing.rotateKeyPair
KeyRing.encrypt
KeyRing.decrypt
KeyRing.sign
KeyRing.verify

And dynamic subproperties like the keyPair or publicKey or privateKey or nodeId could preserve API methods on them, or we can expect downstream to use utilities on them, and expect the observable streams to only be passing data around instead of passing objects around.

The KeyRing doesn't require the database, everything it stores is on the filesystem. This means it has to manage the on-disk encryption/decryption directly with encapsulation/decapsulation and wrap/unwrap methods. At minimum there will always be 1 asymmetric keypair and 1 symmetric key, we can think of these as the "root keys".

CertificateManager

The CertificateManager is the remaining methods on the KeyManager once the key ring functionality is sliced out. It has to manage all the root identity certificate chains just liek before. There was some initial exploration of the similarities between the sigchain and the x509 certificate chain. We still need this x509 certificate chain separate from sigchain because TLS still relies on x509.

The certificates now rely on CertificateId which does need to be stored in the database, this means CertificateManager is a domain reliant on the DB unlike KeyRing.

The methods that are relevant here are:

CertificateManager.renewKeyPair
CertificateManager.resetKeyPair
CertificateManager.resetCert
CertificateManager.garbageCollectRootCerts

There's a couple problems to solve now that the KeyRing is separated out. The KeyRing can expose a general key rotation function KeyRing.rotate, doing this rotates the key pair, but it isn't aware of any of the certificates anymore. This means we have a couple of ways to do this:

1. Make CertificateManager depend on KeyRing, and expose the above methods that end up calling KeyRing.rotate.
2. Make CertificateManager depend on an observable of KeyRing key pair, and it would therefore react to change to the root key pair during rotation.
3. Compose KeyRing and CertificateManager together at a composition point, and expose the ability to rotate the key pair and regenerate certificates at a higher level.

Solution 2. doesn't work because there's no way to control whether we want to renew the certificate with the new key pair, reset the certificate with the new keypair, or just reset the certificate with the existing key pair. Although renewing the certificate with the new key pair is the most common operation, and the latter 2 operations are not actually used in the normal operating flow of PK.

Solution 3. would allow a nice separation of duties, but it does mean a key functionality is now done at the RPC handler level. We talked about in the past whether our key functionality is in the handlers or done at the level of domain objects. Doing this would be different from our existing design which currently expects our domain objects to work independently of the RPC handlers. Essentially our RPC handlers are thin controllers that rely mostly on the domain objects. They just marshal RPC request and responses to domain methods and streams. This was intended to make it easier for us to test logic without introducing RPC request/response overhead all over our tests.

It appears solution 1. is the only practical way of doing this. The CertificateManager represents a higher abstraction level, that builds on top of root keys to present an "identity" for the PK node. The methods though can be renamed more appropriately.

CertificateManager.renewWithNewKeyPair - renew the cert with new key pair
CertificateManager.resetWithNewKeyPair - reset the cert with new key pair
CertificateManager.refreshWithExistingKeypair - refresh the current cert

Note that symmetric key does not get rotated, doing so would require a new issue to address it as it requires encryption evolution in the DB.

KeyManager

After removing root key pair and removing the certificate duties. What's left for KeyManager.

It is only to manage arbitrary sub "keys". These subkeys range from derived subkeys which could be asymmetric key pairs or symmetric keys, or they can be newly generated keypairs or keys. These keys can be used for any purpose by other domains.

This doesn't have a much usage right now. Only when we start to tackle secure computation will these become more useful. It is possible to then to generate keys for arbitrary signing, verification, encryption and decryption.

These keys unlike KeyRing is stored in the database. So KeyManager depends on the DB.

Using this would mean that downstream domains that currently create a key like vaultKey or sessionKey would instead depend on the KeyManager to generate and manage these keys. This adds a bit of overhead to the domains relying on the creation of the KeyManager instead of just having the DB. But it can be useful once the KeyManager has the ability to also manage rotations automatically or provide observables over these keys when they have TTLs.

This new class would be low priority since we don't really need this kind of KeyManager until we start on secure computation.

Additional context

• Updating remote node information after root keypair and NodeID has changed. #386
• Integrate Observables to coordinate push-based Dataflow (with reactive properties) #444
• Seed Agent Cluster Auto-Configuration #403
• Smart Tokens #385

Tasks

1. ...
2. ...
3. ...

[CMCDragonkai]

I've already got KeyRing separated out of KeyManager. However pending the revisiting of the keys utilities to libsodium for #168, the KeyRing may need to be reviewed. So there is at least 1 more day to work on getting KeyRing ready.

There's no work yet done on CertificateManager, and as saw the remaining KeyManager is low priority, so we can eliminate that entirely from this issue for now. It's only relevant to a larger issue for secure computation #385.

[CMCDragonkai]

Experiments with Observable hasn't started. This is unknown how well this will work or whether it will work at all. At the moment, we can put Observable integration to a new issue, because this issue can be resolved by just having KeyRing and CertificateManager working, while downstream domains change from depending on KeyManager to depending on KeyRing when they just need the NodeId.

Observable experiments will be for: #444

[CMCDragonkai]

I suspect 1 point left for KeyRing. 2 points left for CertificateManager. 1 point left for testing and refactoring the tests and integration to other domains.

So 4 points in total.

[CMCDragonkai]

Arbitrary subkey management is being factored out entirely to KeyManager. However currently we don't have a need to have this feature, so it's just entirely removed. I've renamed this issue to not bother with a KeyManager atm.