[BC/CWC] Wallet private key updates [1]

[MichaelAJay]

bitcore-client

Encryption

• adds encryptBuffer - enables normalized buffer input
• adds decryptToBuffer - enables improved security by not decrypting directly to string

Storage

• adds addKeysSafe - incoming keys' private key is encrypted, so not immediately available to be used to retrieve pubkey, which should be available anyway. Throws if missing a pubkey

Wallet

• Adds version: 2 & version checks for backwards compatibility (note: to create an old Wallet would require version: 0 - so 1 is skipped - looking for feedback on numbering/versioning).
• create encrypts HDPrivateKey xprivkey and privateKey so they can be decrypted as buffers instead of serializing the whole masterKey and THEN encrypting
• importKeys does the same for signing keys
• signTx decrypts to buffers then uses deriver for chain-aware decoding. In the next phase, this decoding should be made unnecessary by passing the buffer all the way through to use, if possible

Tests

• update prior tests to include versioning
• introduce new tests for version 2 specifics

crypto-wallet-core

Derivation

• Add two passthrough methods on DeriverProxy to buffer & serialize private keys
• Adds privateKeyToBuffer and privateKeyBufferToNativePrivateKey to IDeriver & implement for each Deriver - no need to override for any of the extending classes (UTXOs to AbstractBitcoreLibDeriver & EVM to EthDeriver)

[kajoseph]

I think a better approach may be to auto migrate wallets with version < 2 to v2. That way we can avoid carve-outs while actively push better security.

loadWallet() {
  const wallet = read wallet file;
  if (wallet.version < 2) {
    convert to v2 wallet;
    backup wallet file to .bak;
    overwrite wallet file with v2;
  }
}

[kajoseph]

Functionally, this seems to work well. Nice job!

In addition to the minor comments below, we should add test cases for the migration as well as tests for critical functions with the new version wallets (i.e. deriving, sending, sign message)

[kajoseph]

There seems to be an issue with maintaining the addressIndex through the migration. To reproduce:

1. Pre-migration (i.e. on master), create a BTC wallet
2. Generate some addresses bin/wallet derive --name <name> --gap 1
   • Run that a few times or change gap to the number of addresses you want to create
3. Running bin/wallet check --name <name> will show your latest address
4. Switch to this branch
5. Run the check command again and it'll tell you to derive an address
6. Run the derive command again and it'll start with m/0/0

Note, that bin/wallet balance --name <name> will still show the correct balance and send still works. It just seems like the addressIndex gets unset.

[kajoseph]

I'm still testing but wanted to give you this feedback before I forget it:

• I'm unable to send funds. There seems to be an issue with the expected data structure when decrypting the address private keys

Address keys that are migrated look like this:

{
  "_id": "69cffcc8cdaf237593eaa863",
  "name": "prTest",
  "address": "tb1qgysfj2jvgjf09lrc7uxg346xtduly4u2f5glzq",
  "data": "{\"encKey\":\"970dae4c30e6cb4bac805cd8dd0258cf323dfe06fae6c4a05b58d5c3e9ec85b527b508ee96d19fd223adbde87004b5eed4cf529fb8dde598b61998e687b5964540655cb53b69a5ba868906165a6ed02afc946e043e30299e6f869437a74afbe67553cfc91f1c84a2c22969b481e1c9574b55a973467afe76388ca6575db8b9105ae96354517494550e5dd35da8bbb7cbfbe01b0802eedf113ca4eb2939d1f38107cf09acbf2f6cd423b32a299409f7ede6d43afa1a38e4c2591eb2303b07d1be9bd1d50f1f7a7fae849d922ed7c75d7e3bc031ccab5e4c952415cdc6d6175ee4e78ced1bdb98c83a9de9f7ad3fc34fe7\",\"pubKey\":\"020c6c09b7266172253e95b1a96bf6b2ede71de2d2fa74980d46fa23a26fb2c643\",\"path\":\"m/0/0\"}"
}

However, any new keys post-migration look like the pre-migration data structure (and it looks like this is the form that the send is looking for):

{
  "_id": "69d002ff85f889998b86efbe",
  "name": "prTest",
  "address": "tb1qhmu3lg06j89zrcktttxy62qqp56xwzzr06z8p5",
  "data": "{\"key\":\"{\\\"address\\\":\\\"tb1qhmu3lg06j89zrcktttxy62qqp56xwzzr06z8p5\\\",\\\"privKey\\\":\\\"c4d06d905824d2feaa1034121feae47f6af16a1fbcb53c7fb18a1ecd2fe9969804030da17557a371ffb508aa3a179546\\\",\\\"pubKey\\\":\\\"02b9336cf6b32cf4ecb59de07ed0f913815e69c83abde7f0a6cf61f66b4dfd19ae\\\",\\\"path\\\":\\\"m/1/3\\\"}\",\"pubKey\":\"02b9336cf6b32cf4ecb59de07ed0f913815e69c83abde7f0a6cf61f66b4dfd19ae\",\"path\":\"m/1/3\"}"
}

STEPS TO REPRODUCE:

1. Create and fund a wallet on master
2. Checkout this branch and migrate the wallet
3. Try to send funds (note, this can be combined with step 2 since a migration is done when unlocking the wallet during a send)

We should add a couple of test cases:

1. sending pre-migration funds post-migration
2. new addresses/keys for a v2 wallet have the new data structure

---

• The address migration for Mongo wallets seems broken. It looks like it's nesting duplicate data in the data field. Nevermind - I think I was just looking at the pre-migration data format which has pubKey and path on both data and data.key

[kajoseph]

We need to keep this in here - otherwise the user has no way of backing up their seed phrase. To prevent converting to string, we can pass the buffer directly to process.stdout:

if (!xpriv) {
  process.stdout.write(Buffer.from(mnemonic.phrase));
  process.stdout.write(os.EOL);
} else {
  process.stdout.write(hdPrivKey.toBuffer());
  process.stdout.write(os.EOL);
}

[kajoseph]

tACK - Nice work!