/bitcoin

[WIP] [wallet] dynamic loading/unloading of wallets #10740

Open
wants to merge 19 commits into
from
+507 −280

Conversation

Reviewers

@jonasschnelli jonasschnelli

Assignees
No one assigned
Labels
Wallet
Projects
None yet
Milestone
No milestone
4 participants
@jnewbery @promag @jonasschnelli @fanquake
@jnewbery
Member

jnewbery commented Jul 4, 2017 • edited

Work in progress. Builds on top of #10767 and #10604. Not yet ready for code review. Currently looking for concept/API feedback.

Adds loadwallet and unloadwallet RPCs. This allows wallets to be loaded and unloaded dynamically during runtime without having to stop-start the node with new -wallet params.

Main motivation for this was the fact that several wallet parameters are actions for individual wallet load/creation, rather than properties of the wallet component. Examples are -salvagewallet, -rescan, -usehd and -upgradewallet. Continuing with that config/loading model is difficult in a multi-wallet world - how can users run those actions on individual wallets when loading multiple wallets? This PR offers a solution: individual wallets can be loaded at run-time, and RPC parameters can be passed in to carry out the various wallet-loading actions. Note that none of those load actions are yet implemented in this PR, but can easily be added.

A lot of the work in this PR was disentangling wallet initialization and destruction. Hopefully the result is that the interface between init.cpp and the wallet component is now cleaner. All wallet component initialization/destruction functions are now in their own walletinit.cpp translation unit and are no longer static functions on the CWallet class. The bitcoin_server also no longer has any knowledge that there are multiple wallets in vpwallet. Hopefully this is also useful work towards #7965 and the removal of circular server<->wallet dependencies. That part of this PR (commits "Add walletinit.cpp" through "remove early call to FlushWallets()") can be separated into a different PR if that makes reviewing easier.

Includes functional test.

jnewbery added some commits Jun 15, 2017

@jnewbery jnewbery [wallet] [tests] multiwallet test
a952c04
@jnewbery jnewbery [wallet] [rpc] print wallet name in getwalletinfo 5d18ee0
@jnewbery jnewbery [wallet] fix comment for CWallet::Verify() 754434a
@jnewbery jnewbery [wallet] [rpc] Add listwallets RPC 
This commit adds a listwallets RPC, which lists the names of the
currently loaded wallets. This command intentionally shows no
information about the wallet other then the name. Information on
individual wallets can be obtained using the getwalletinfo RPC.
 74882ca
@jnewbery jnewbery [wallet] Add walletinit.cpp 
This commit adds a new walletinit translation unit and moves
GetWalletHelpString() to walletinit from a static member function of
CWallet.
 851cf9f
@jnewbery jnewbery [wallet] move WalletParameterInteraction() to walletinit.cpp a863dc3
@jnewbery jnewbery [wallet] move VerifyWallets() to walletinit.cpp 3954d79
@jnewbery jnewbery [wallet] Allow individual wallets to be verified 4ba8d57
@jnewbery jnewbery [wallet] move InitLoadWallets() to walletinit.cpp 533eb5e
@jnewbery jnewbery fixup with Allow individual wallets to be verified 34d0690

fanquake added the Wallet label Jul 5, 2017

jnewbery added some commits Jul 3, 2017

@jnewbery jnewbery [wallet] move calls to postInitProcess() to walletinit.cpp 4c62008
@jnewbery jnewbery [wallet] move FlushWallets and DeleteWallets to walletinit.cpp 
This completes the movement of wallet initialization functions to
walletinit.cpp. init.cpp now has an interface to the wallet consisting
of the following functions:

- GetWalletHelpString()
- RegisterWalletRPCCommands()
- WalletParameterInteraction()
- VerifyWallets()
- InitLoadWallets()
- WalletCompleteStartup()
- FlushWallets()
- DeleteWallets()
 10e781b
@jnewbery jnewbery [wallet] add CDBEnv::Shutdown() function 0efbd06
@jnewbery jnewbery [wallet] call CDBEnv.shutdown() from FlushWallets() 
This commit removes the call to CDBEnv.Shutdown() from CDBEnv.Flush()
and calls CDBEnv.Shutdown directly from ShutdownWallets() in
walletinit.cpp.

This also changes the interface to CDBEnv.Flush() to allow an individual
wallet to be flushed.
 d1a1218
@jnewbery jnewbery [wallet] remove early call to FlushWallets() 
This commit removes the early call to FlushWallets() in Shutdown() and
renames the function in walletinit.cpp to ShutdownWallets().
 267a74e
@jnewbery jnewbery [wallet] add loadwallet RPC 
This commit adds an RPC to load wallet .dat files dynamically at
runtime.
 d4f6a0f
@jnewbery jnewbery [wallet] add unloadwallet rpc 8d32ecc
@jnewbery jnewbery [wallet] add cs_wallets to protect vpwallets 17549a4
@jnewbery jnewbery [wallet] [test] update multiwallet.py to use loadwallet and unloadwallet
72c3745
src/init.cpp
- delete pwallet;
- }
- vpwallets.clear();
+ DeleteWallets();
@jonasschnelli

jonasschnelli Jul 5, 2017

Member

It'd probably call this deallocWallets()

@jnewbery

jnewbery Jul 7, 2017

Member

Yes, DeleteWallets() is a bad name. DeallocWallets() is good, or perhaps DetachWallets()

src/wallet/db.cpp
@@ -227,40 +227,12 @@ bool CDB::Recover(const std::string& filename, void *callbackDataIn, bool (*reco
bool CDB::VerifyEnvironment(const std::string& walletFile, const fs::path& dataDir, std::string& errorStr)
{
- LogPrintf("Using BerkeleyDB version %s\n", DbEnv::version(0, 0, 0));
@jonasschnelli

jonasschnelli Jul 5, 2017

Member

What's the reasons for keeping this function (that now always returns true)?

@jnewbery

jnewbery Jul 7, 2017

Member

You're right. This (and CWalletDB::VerifyEnvironment()) are now entirely vestigial and are not called. Both can be removed.

src/wallet/wallet.cpp
@@ -35,6 +35,8 @@
#include <boost/algorithm/string/replace.hpp>
#include <boost/thread.hpp>
+/** Protects the vpwallets vector */
+CCriticalSection cs_wallets;
std::vector<CWalletRef> vpwallets;
@jonasschnelli

jonasschnelli Jul 5, 2017

Member

I guess this belongs to walletinit.h now?

@jnewbery

jnewbery Jul 7, 2017

Member

Why? vpwallets is accessed by functions in several of the wallet source files.

@jonasschnelli

jonasschnelli Jul 9, 2017

Member

I had the idea that walletinit.h/.c is kind of the multiwallet interface (wallet manager). The functions we have there (DeleteWallets(), VerifyWallets(), ShutdownWallets();, etc.) sounded for me after vpwallets belongs there...

IMO, the multiwallet map should ideally not sit within the objects (CWallet) implementation.

@jnewbery
Member

jnewbery commented Jul 7, 2017

This overlaps significantly with #10762. Not sure how I should ask for review. Perhaps find out which one people think is the priority? Or I could split the walletinit.cpp commits into their own PR.
src/wallet/rpcwallet.cpp
+ return obj;
+}
+
+UniValue loadwallet(const JSONRPCRequest& request)
@jnewbery

jnewbery Jul 7, 2017

Member

On further thought, I think I prefer the names attachwallet and detachwallet. That gives a bit more distinction from the dump/import RPCs and makes a stronger implication that the new .dat file is being loaded and attached as a separate wallet.

@promag
Contributor

promag commented Jul 7, 2017

@jnewbery IMO this one should rebase on the other.
@jnewbery
Member

jnewbery commented Jul 7, 2017

IMO this one should rebase on the other.

Makes logical sense. Downside is that the other is a more significant change, so may take longer to get merged.
@promag
Contributor

promag commented Jul 7, 2017

If this really depends on that work then there's no other way than waiting. The other downside is that the other commits will show up here too.

This was referenced Jul 7, 2017

Open
Open
@jnewbery
Member

jnewbery commented Jul 7, 2017

I've changed the PR sequence, so this will now depend on #10767. If I get positive concept feedbac on #10767 I'll rebase this on top of it.

For now, I'm still just looking for concept feedback.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment