[wallet] Reopen CDBEnv after encryption instead of shutting down

[achow101]

This is the replacement for #11678 which implements @ryanofsky's suggestion.

Shutting down the software was to prevent the BDB environment from writing unencrypted private keys to disk in the database log files, as was noted here. This PR replaces the shutdown behavior with a CDBEnv flush, close, and reopen which achieves the same effect: everything is cleanly flushed and closed, the log files are removed, and then the environment reopened to continue normal operation.

To ensure that no unencrypted private keys are in the log files after encrypting the wallet, I wrote this script to pull private keys from the original wallet file and searches for these keys in the log files (note that you will have to change your file paths to make it work on your own machine).

As for concerns about private keys being written to slack space or being kept in memory, these behaviors no longer exist after the original wallet encryption PR and the shutting down solution from 2011.

cc @ryanofsky

[ryanofsky]

This is a great start! I don't think think it is thread safe yet, though, because in parallel with this thread, a block connected notification could be coming in, or another another RPC could be being made that is using one of the Db* or DbEnv* pointers that this closes.

I think all you need to do to make this thread safe is wait for the mapFileUseCount entries to go to zero. You could do this with a condition variable. For example, if you added a std::condition_variable m_cv_in_use; member to CDBEnv you could trigger it in CDB::Close:

m_cv_in_use.notify_all()

and wait for it at the top of CDBEnv::ReloadDbEnv:

WAIT_LOCK(cs_db, lock);
m_cv_in_use.wait(lock, [this](){
    for (count : mapFileUseCount) {
        if (count.second > 0) return false;
    }
    return true;
});

This is one possible approach. Other approaches may be simpler or better. One drawback of this approach is that if there are a lot of background writes happening in different wallets, ReloadDbEnv could get starved out waiting for all wallets to be simultaneously not in use.

[achow101]

@ryanofsky I've implemented something which is basically what you described. It seems to work, although I'm not sure how to test the thread safe-ness of it. Let me know what you think.

[ryanofsky]

It seems to work, although I'm not sure how to test the thread safe-ness of it.

It'd be a little tricky to test the threadsafeness in a unit test. I think you'd need to insert hooks that would allow the test to block threads at particular points. If you want to test in a more ad-hoc way, though, I think you could do this by inserting a sleep in CDB::Write and starting bitcoin with two wallets loaded. You could then call an RPC that triggers a write in one wallet and hits the sleep. Then call an RPC on the other wallet to trigger ReloadDbEnv. If the condition variable is working correctly, the ReloadDbEnv call in the second wallet should get stuck until the sleep is over and the first wallet completes its write.

[ryanofsky]

In commit "Replace cs_db with a recursive mutex use with a conditional_variable_any"

cs_db is actually already a recursive mutex (it inherits from std::recursive_mutex). I think you could just leave it unchanged, and continue using the LOCK macro instead of std::lock_guard everywhere. This would make the commit smaller & simpler.

[achow101]

I tried that, but it kept giving me a compiler error.

[ryanofsky]

#12493 (comment)

I tried that, but it kept giving me a compiler error.

What is the compiler error? The following compiles for me: c271de2 (fetchable with git fetch https://github.com/ryanofsky/bitcoin pr/lreload).

I didn't do any real testing with it but it passes python & unit tests.

[achow101]

Ah, I was doing something with the macros and CCriticalSection. It was an error about not having an unlock method.

I'll test this out, but from testing I just did with the current code, it looks like it does work, so this should work too.

[achow101]

@ryanofsky I've used your commit.

[ryanofsky]

In commit "Replace cs_db with a recursive mutex use with a conditional_variable_any"

Would be good to add an AssertLockNotHeld(cs_db); above this, since it would be a bug if the mutex were held recursively when this was called (wait could hang if the lock wasn't released).

[achow101]

Done

[ryanofsky]

utACK 437d2dd56f81a6258243aca6ed137cbca9255b32

[ryanofsky]

In commit "After encrypting the wallet, reload the database environment"

Delayed flushing seems more like a legitimate design tradeoff than a bad habit. Maybe just say something like "flush and reload the database environment here to clear out any data in memory that might be left behind after the rewrite above."

[achow101]

The comment was just copied over from rpcwallet.cpp.

[ryanofsky]

In commit "After encrypting the wallet, reload the database environment"

Looking at the CDB::Rewrite implementation I noticed that it already has a while loop that sleeps until mapFileUseCount (for just one wallet) is 0. You could take advantage of this if you wanted by tweaking the while loop there to wait for all use counts in the environment to be 0 and then calling ReloadDbEnv inside CDB::Rewrite. This would be a simplification since it would let you get rid of the new condition variable, and it would also make the new waiting code more consistent with previous code.

This is just a suggestion, though. Your current implementation seems fine, too.

[achow101]

I don't want to do anything to CDB::Rewrite since it is used in places other than encryptwallet.

[ryanofsky]

#12493 (comment)

I don't want to do anything to CDB::Rewrite since it is used in places other than encryptwallet

I didn't realize Rewrite was called other places, and the current approach does seem fine. But if you did want to unify the UseCount waiting logic, I think you could do it in a pretty clean way by adding a bool reload_env or similar option to CDB::Rewrite. I think in terms code clarity, having the option would actually be an improvement over always reloading.

[achow101]

I think I'll leave it as it is now.

[promag]

Tested ACK 437d2dd.

Tested with multiple wallets and with different -walletdir.

[promag]

Remove .arg() since %1 was removed.

[achow101]

Done

[promag]

QMessageBox::information?

[achow101]

I don't think it needs to change.

[promag]

Remove filenames and just iterate once, something like:

while (!mapDb.empty()) {
  std::string filename = mapDb.begin().first;
  ...
}

[achow101]

This doesn't work (causes test failures with encryptwallet). I think it's becauese mapDb is modified by CloseDb too.

[promag]

Nit, space after if.

[achow101]

Done

[achow101]

Rebased

[achow101]

Fixed travis and rebased again.

[achow101]

Rebased

[achow101]

Review beg

[achow101]

Review Beg?

[sipa]

It would be slightly cleaner to use std::unique_lock<CCriticalSection> here.

Also, please squash this into the first commit - currently your PR introduces a bug and then fixes it later.

We should probably also work on making the sync.h primitives more C++11 compatible, but let's leave that for later.

[achow101]

Done

[sipa]

utACK 22c1c367062c960a28ca9b3b463b6ab6d96ebd02, just nits.

[sipa]

Performance nit: notifying while not holding the lock is slightly faster (currently you'll wake the waiting thread, which then immediately needs to sleep again because the lock is still held).

[achow101]

Done

[sipa]

Nit: const std::string& filename.

[achow101]

Done

[sipa]

utACK 06f17267ddafa089f093cfffc181d8c834d346e7

[achow101]

Rebased and added a commit to fix the silent merge conflict (test failure).

[sipa]

utACK bdad9c40595c1e4959ebce1b55894060b2d7c146

[instagibbs]

utACK

provided the locking behavior is as I expect, where the lock will be acquired without fail by the .wait. The documentation isn't super clear and I'm definitely not an expert.