New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[0.13] Create a new HD seed after encrypting the wallet #8389

Merged
merged 2 commits into from Jul 28, 2016

Conversation

Projects
None yet
7 participants
@jonasschnelli
Member

jonasschnelli commented Jul 21, 2016

Reported by @dooglus.

Fixes #8383

@@ -2081,7 +2081,7 @@ UniValue encryptwallet(const UniValue& params, bool fHelp)
// slack space in .dat files; that is bad if the old data is
// unencrypted private keys. So:
StartShutdown();
return "wallet encrypted; Bitcoin server stopping, restart to run with encrypted wallet. The keypool has been flushed, you need to make a new backup.";
return "wallet encrypted; Bitcoin server stopping, restart to run with encrypted wallet. The keypool has been flushed and a new HD seed was generated (if you are using HD). You need to make a new backup.";

This comment has been minimized.

@luke-jr

luke-jr Jul 21, 2016

Member

Maybe the code should just check if HD is being used?

@luke-jr

luke-jr Jul 21, 2016

Member

Maybe the code should just check if HD is being used?

This comment has been minimized.

@jonasschnelli

jonasschnelli Jul 21, 2016

Member

I have though about that but I think it's not worth adding another check here.

@jonasschnelli

jonasschnelli Jul 21, 2016

Member

I have though about that but I think it's not worth adding another check here.

This comment has been minimized.

@laanwj

laanwj Jul 26, 2016

Member

I don't think that's worth it either, just for the message

@laanwj

laanwj Jul 26, 2016

Member

I don't think that's worth it either, just for the message

Show outdated Hide outdated src/wallet/wallet.cpp
@luke-jr

This comment has been minimized.

Show comment
Hide comment
@luke-jr

luke-jr Jul 21, 2016

Member

Will this destroy the old HD seed? Seems like it'd be valuable to keep it around somewhere...

Member

luke-jr commented Jul 21, 2016

Will this destroy the old HD seed? Seems like it'd be valuable to keep it around somewhere...

@MarcoFalke

This comment has been minimized.

Show comment
Hide comment
@MarcoFalke

MarcoFalke Jul 21, 2016

Member

@jonasschnelli Could you open this against master, please? (The backport can be done after merge and discussion/nits etc.)

Member

MarcoFalke commented Jul 21, 2016

@jonasschnelli Could you open this against master, please? (The backport can be done after merge and discussion/nits etc.)

@jonasschnelli

This comment has been minimized.

Show comment
Hide comment
@jonasschnelli

jonasschnelli Jul 21, 2016

Member

@luke-jr The old HD seed is still available over dumpwallet. You can get the CKeyID from the used masterkey when calling validateaddress.

Member

jonasschnelli commented Jul 21, 2016

@luke-jr The old HD seed is still available over dumpwallet. You can get the CKeyID from the used masterkey when calling validateaddress.

@jonasschnelli

This comment has been minimized.

Show comment
Hide comment
@jonasschnelli

jonasschnelli Jul 21, 2016

Member

Fixed the nits.
Sorry for opening this against 0.13 in the first place.

Member

jonasschnelli commented Jul 21, 2016

Fixed the nits.
Sorry for opening this against 0.13 in the first place.

@MarcoFalke

This comment has been minimized.

Show comment
Hide comment
@MarcoFalke

MarcoFalke Jul 21, 2016

Member

So this makes it impossible to encrypt a seed which is already in use and is planned to be used in the future, but I guess for 0.13 this is fair enough.

Concept ACK

Member

MarcoFalke commented Jul 21, 2016

So this makes it impossible to encrypt a seed which is already in use and is planned to be used in the future, but I guess for 0.13 this is fair enough.

Concept ACK

Show outdated Hide outdated src/wallet/wallet.cpp
@pstratem

This comment has been minimized.

Show comment
Hide comment
@pstratem

pstratem Jul 21, 2016

Contributor

@jonasschnelli Does BDB guarantee the order in which keys are iterated? If not then this isn't guaranteed to work.

Contributor

pstratem commented Jul 21, 2016

@jonasschnelli Does BDB guarantee the order in which keys are iterated? If not then this isn't guaranteed to work.

@pstratem

This comment has been minimized.

Show comment
Hide comment
@pstratem

pstratem Jul 21, 2016

Contributor

@jonasschnelli Indeed you're going to need to use the key metadata timestamp to select which hdchain to use.

Contributor

pstratem commented Jul 21, 2016

@jonasschnelli Indeed you're going to need to use the key metadata timestamp to select which hdchain to use.

@dooglus

This comment has been minimized.

Show comment
Hide comment
@dooglus

dooglus Jul 22, 2016

Contributor

@pstratem It reads to me as if the old hdchain key is overwritten by the new one:

bool CWalletDB::WriteHDChain(const CHDChain& chain)
{
    nWalletDBUpdated++;
    return Write(std::string("hdchain"), chain);
}

This WriteHDChain() function is used each time a new address is generated:

    // update the chain model in the database
    if (!CWalletDB(strWalletFile).WriteHDChain(hdChain))

The original chain could be stored under a different database key if we need to keep it around somewhere.

Contributor

dooglus commented Jul 22, 2016

@pstratem It reads to me as if the old hdchain key is overwritten by the new one:

bool CWalletDB::WriteHDChain(const CHDChain& chain)
{
    nWalletDBUpdated++;
    return Write(std::string("hdchain"), chain);
}

This WriteHDChain() function is used each time a new address is generated:

    // update the chain model in the database
    if (!CWalletDB(strWalletFile).WriteHDChain(hdChain))

The original chain could be stored under a different database key if we need to keep it around somewhere.

@pstratem

This comment has been minimized.

Show comment
Hide comment
@pstratem

pstratem Jul 22, 2016

Contributor

@dooglus indeed you're right... I'm not sure that is any better

certainly the old hd seed needs to be saved, possibly just with a prefix to make it not conflict or be used.

Contributor

pstratem commented Jul 22, 2016

@dooglus indeed you're right... I'm not sure that is any better

certainly the old hd seed needs to be saved, possibly just with a prefix to make it not conflict or be used.

@jonasschnelli

This comment has been minimized.

Show comment
Hide comment
@jonasschnelli

jonasschnelli Jul 22, 2016

Member

@pstratem: CWalletDB and BDB is a key value storage as far as I know. Writing the same key at later stage should replace the current value. It may be possible to have the old K/V still stored in the database, though, CWalletDB/BDB needs to make sure the later write operation will result in the laster read/mem-map operation during load wallet.

But I agree we need to keep track of the older/replaced seed. Maybe we should add something to the CKeyMetadata in that case (or at least CAddressBook).

Member

jonasschnelli commented Jul 22, 2016

@pstratem: CWalletDB and BDB is a key value storage as far as I know. Writing the same key at later stage should replace the current value. It may be possible to have the old K/V still stored in the database, though, CWalletDB/BDB needs to make sure the later write operation will result in the laster read/mem-map operation during load wallet.

But I agree we need to keep track of the older/replaced seed. Maybe we should add something to the CKeyMetadata in that case (or at least CAddressBook).

@jonasschnelli

This comment has been minimized.

Show comment
Hide comment
@jonasschnelli

jonasschnelli Jul 22, 2016

Member

Pushed a commit that...

  1. ...factors out the creation of a new HD Master key (will be called now during init creation of the wallet as well as during encryption of the wallet)
  2. ...stores a CKeyMetadata record together with a HD master key where we set a hdkeypath="m" for later identifying this key as hd master key

https://github.com/bitcoin/bitcoin/pull/8206/files would make use of this. It would show old/rewritten master keys in the dumped wallet.

Member

jonasschnelli commented Jul 22, 2016

Pushed a commit that...

  1. ...factors out the creation of a new HD Master key (will be called now during init creation of the wallet as well as during encryption of the wallet)
  2. ...stores a CKeyMetadata record together with a HD master key where we set a hdkeypath="m" for later identifying this key as hd master key

https://github.com/bitcoin/bitcoin/pull/8206/files would make use of this. It would show old/rewritten master keys in the dumped wallet.

Show outdated Hide outdated src/wallet/wallet.h
CKey key;
CPubKey masterPubKey = GenerateNewHDMasterKey();
if (!SetHDMasterKey(masterPubKey))
return false;

This comment has been minimized.

@jonasschnelli

jonasschnelli Jul 22, 2016

Member

I think this error detection case is extremely unlikely and probably on the same level then an exception during NewKeyPool().
I don't see a better way of handling this case.

@jonasschnelli

jonasschnelli Jul 22, 2016

Member

I think this error detection case is extremely unlikely and probably on the same level then an exception during NewKeyPool().
I don't see a better way of handling this case.

@jonasschnelli

This comment has been minimized.

Show comment
Hide comment
@jonasschnelli
Member

jonasschnelli commented Jul 22, 2016

Fixed @instagibbs nits.

@pstratem

This comment has been minimized.

Show comment
Hide comment
@pstratem

pstratem Jul 23, 2016

Contributor

@jonasschnelli needs to be one commit

Contributor

pstratem commented Jul 23, 2016

@jonasschnelli needs to be one commit

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Jul 25, 2016

Member

Is this only aimed at 0.13 on purpose?

Member

laanwj commented Jul 25, 2016

Is this only aimed at 0.13 on purpose?

@jonasschnelli

This comment has been minimized.

Show comment
Hide comment
@jonasschnelli

jonasschnelli Jul 25, 2016

Member

No. I should have opened this PR against master. I'll open a PR against master as soon as this one is "final" (maybe it cleanly applies to master).

Member

jonasschnelli commented Jul 25, 2016

No. I should have opened this PR against master. I'll open a PR against master as soon as this one is "final" (maybe it cleanly applies to master).

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Jul 27, 2016

Member

Tested ACK 77c912d
Sorry, wrong issue - will test this one later

Member

laanwj commented Jul 27, 2016

Tested ACK 77c912d
Sorry, wrong issue - will test this one later

@instagibbs

This comment has been minimized.

Show comment
Hide comment
@instagibbs
Member

instagibbs commented Jul 27, 2016

utACK 77c912d

jonasschnelli added some commits Jul 21, 2016

@jonasschnelli

This comment has been minimized.

Show comment
Hide comment
@jonasschnelli

jonasschnelli Jul 27, 2016

Member

Rebased (against 0.13).

Member

jonasschnelli commented Jul 27, 2016

Rebased (against 0.13).

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Jul 28, 2016

Member

code-review ACK de45c06, going to test

Member

laanwj commented Jul 28, 2016

code-review ACK de45c06, going to test

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Jul 28, 2016

Member

I did this:

$ rm ~/.bitcoin/regtest/wallet.dat
$ src/bitcoind -regtest -rpcport=1234 -daemon -usehd=1
Bitcoin server starting
$ src/bitcoin-cli  -regtest -rpcport=1234 dumpwallet "/tmp/dump_pre"
$ src/bitcoin-cli  -regtest -rpcport=1234 encryptwallet "pass"
wallet encrypted; Bitcoin server stopping, restart to run with encrypted wallet. The keypool has been flushed, you need to make a new backup.
$ src/bitcoind -regtest -rpcport=1234 -daemon -usehd=1
Bitcoin server starting
$ src/bitcoin-cli  -regtest -rpcport=1234 walletpassphrase "pass" 12345
$ src/bitcoin-cli  -regtest -rpcport=1234 dumpwallet "/tmp/dump_post"
$ src/bitcoin-cli  -regtest -rpcport=1234 stop
$ diff  -du /tmp/dump_pre /tmp/dump_post

Result: http://www.hastebin.com/ajuhahofan.diff

Looks OK to me. The old reserve keys have been demoted to "change", and there are new reserve keys in the keypool. These keys are different, suggesting a new master key was used to generate them.

The old master key was kept around as "oldhdmaster":

-cSoMD7tEsoJNfSS3cL6N3E9gXpuQh7wgs79RDUBE6NjTEWS6Ptda 2016-07-28T10:48:28Z hdmaster=1 # addr=msqHNkZcD6ybT7xdCkquN4esLPLmoRyk8b hdkeypath=m
+cSoMD7tEsoJNfSS3cL6N3E9gXpuQh7wgs79RDUBE6NjTEWS6Ptda 2016-07-28T10:48:28Z inactivehdmaster=1 # addr=msqHNkZcD6ybT7xdCkquN4esLPLmoRyk8b hdkeypath=m

And a new one was added

+cRrbkixFuTz1esCbJTz3QqNGyEcPjFCjVyHRTy1JDxKDbHYXn9Qd 2016-07-28T10:48:43Z hdmaster=1 # addr=n3phnt6MDc2xUvKXrf8iW6ZiwtL2i7Rusm hdkeypath=m

The single key that is automatically generated (with empty label) at initialization stayed the same

 cRj4c74AFYdH9W1vbM3UgnaCeGbwztu3qb11NgDReiudtS2Q1ZAr 2016-07-28T10:48:28Z label= # addr=mwR8SR9sfKk7Q8z99xpdcCUNtccq81iRup hdkeypath=m/0'/0'/0'

For the new reserve keys it starts counting again at m/0'/0'/0'

+cNRrj6BtSxcXySqU6WgWgGEHMARifd1VXCVECdsXx4ervHE7iS4t 2016-07-28T10:48:43Z reserve=1 # addr=moBwgfzRf8H4ZjweCbQcckuJbMC16jwtD1 hdkeypath=m/0'/0'/0'

I think everything is as expected. Tested ACK.

Member

laanwj commented Jul 28, 2016

I did this:

$ rm ~/.bitcoin/regtest/wallet.dat
$ src/bitcoind -regtest -rpcport=1234 -daemon -usehd=1
Bitcoin server starting
$ src/bitcoin-cli  -regtest -rpcport=1234 dumpwallet "/tmp/dump_pre"
$ src/bitcoin-cli  -regtest -rpcport=1234 encryptwallet "pass"
wallet encrypted; Bitcoin server stopping, restart to run with encrypted wallet. The keypool has been flushed, you need to make a new backup.
$ src/bitcoind -regtest -rpcport=1234 -daemon -usehd=1
Bitcoin server starting
$ src/bitcoin-cli  -regtest -rpcport=1234 walletpassphrase "pass" 12345
$ src/bitcoin-cli  -regtest -rpcport=1234 dumpwallet "/tmp/dump_post"
$ src/bitcoin-cli  -regtest -rpcport=1234 stop
$ diff  -du /tmp/dump_pre /tmp/dump_post

Result: http://www.hastebin.com/ajuhahofan.diff

Looks OK to me. The old reserve keys have been demoted to "change", and there are new reserve keys in the keypool. These keys are different, suggesting a new master key was used to generate them.

The old master key was kept around as "oldhdmaster":

-cSoMD7tEsoJNfSS3cL6N3E9gXpuQh7wgs79RDUBE6NjTEWS6Ptda 2016-07-28T10:48:28Z hdmaster=1 # addr=msqHNkZcD6ybT7xdCkquN4esLPLmoRyk8b hdkeypath=m
+cSoMD7tEsoJNfSS3cL6N3E9gXpuQh7wgs79RDUBE6NjTEWS6Ptda 2016-07-28T10:48:28Z inactivehdmaster=1 # addr=msqHNkZcD6ybT7xdCkquN4esLPLmoRyk8b hdkeypath=m

And a new one was added

+cRrbkixFuTz1esCbJTz3QqNGyEcPjFCjVyHRTy1JDxKDbHYXn9Qd 2016-07-28T10:48:43Z hdmaster=1 # addr=n3phnt6MDc2xUvKXrf8iW6ZiwtL2i7Rusm hdkeypath=m

The single key that is automatically generated (with empty label) at initialization stayed the same

 cRj4c74AFYdH9W1vbM3UgnaCeGbwztu3qb11NgDReiudtS2Q1ZAr 2016-07-28T10:48:28Z label= # addr=mwR8SR9sfKk7Q8z99xpdcCUNtccq81iRup hdkeypath=m/0'/0'/0'

For the new reserve keys it starts counting again at m/0'/0'/0'

+cNRrj6BtSxcXySqU6WgWgGEHMARifd1VXCVECdsXx4ervHE7iS4t 2016-07-28T10:48:43Z reserve=1 # addr=moBwgfzRf8H4ZjweCbQcckuJbMC16jwtD1 hdkeypath=m/0'/0'/0'

I think everything is as expected. Tested ACK.

@laanwj laanwj merged commit de45c06 into bitcoin:0.13 Jul 28, 2016

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

laanwj added a commit that referenced this pull request Jul 28, 2016

Merge #8389: [0.13] Create a new HD seed after encrypting the wallet
de45c06 [Wallet] Add CKeyMetadata record for HDMasterKey(s), factor out HD key generation (Jonas Schnelli)
f142c11 [0.13] Create a new HD seed after encrypting the wallet (Jonas Schnelli)

laanwj added a commit that referenced this pull request Jul 28, 2016

Port from 0.13: Create a new HD seed after encrypting the wallet
Forward-ports two commits from 0.13:
- [0.13] Create a new HD seed after encrypting the wallet
- [Wallet] Add CKeyMetadata record for HDMasterKey(s), factor out HD key generation

Github-Pull: #8389
Rebased-From: f142c11 de45c06
@MarcoFalke

This comment has been minimized.

Show comment
Hide comment
@MarcoFalke

MarcoFalke Jul 31, 2016

Member

Removed "needs backport"

Member

MarcoFalke commented Jul 31, 2016

Removed "needs backport"

// if we are using HD, replace the HD master key (seed) with a new one
if (!hdChain.masterKeyID.IsNull()) {
CKey key;

This comment has been minimized.

@dooglus

dooglus Jul 31, 2016

Contributor

Unused variable.

@dooglus

dooglus Jul 31, 2016

Contributor

Unused variable.

// write the key&metadata to the database
if (!AddKeyPubKey(key, pubkey))
throw std::runtime_error("CWallet::GenerateNewKey(): AddKey failed");

This comment has been minimized.

@MarcoFalke

MarcoFalke Jul 31, 2016

Member

Nit: Wrong name

@MarcoFalke

MarcoFalke Jul 31, 2016

Member

Nit: Wrong name

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment