Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rpc: introduce 'label' API for wallet #7729

Closed
wants to merge 2 commits into from

Conversation

@laanwj
Copy link
Member

commented Mar 21, 2016

Add label API to wallet RPC.

This is one step towards #3816 ("Remove bolt-on account system") although it doesn't actually remove anything yet (that would be a follow-up pull).

These initially mirror the account functions, with the following differences:

  • These functions aren't DEPRECATED in the help
  • Help mentions 'label' instead of accounts. In the language used, labels are associated with addresses, instead of addresses associated with labels. (in contrast to accounts.)
  • Labels have no balance
    • No getreceivedbylabel
    • No balances in listlabels
    • listreceivedbylabel can show received transactions to addresses with a label, not use the account tally (currently it is removed, but according to discussion that goes too far as this doesn't inherently have to do with balance)
    • listlabels has no minconf or watchonly argument
    • Remove move
  • Like in the GUI, labels can be set on any address, not just receiving addreses
  • Unlike accounts, labels can be deleted. Being unable to delete them is a common annoyance (see #1231). Currently only by reassigning all addresses using setlabel, but an explicit call deletelabel which assigns all address to the default label may make sense.
  • These calls stay the same, with account argument renamed to label:
    • importaddress
    • importprivkey
    • importpubkey

API

Short description of every RPC call: for detailed information check RPC help. The general idea is to offer the same functionality as the GUI label system. Labels are simply a name for an address, or a group of addresses.

Do not use the deprecated account system and the label system with the same wallet at the same time. These APIs use the same underlying data in the database for (slightly) different purposes, using them interchangeably will give unexpected results. (Just like using the GUI labels and account system at the same time. Using the GUI labels and the label API at the same time, however, is no problem)

  • getlabel: returns the label (and other address book data) associated with an address
    • This exposes the fields in the CAddressBookData structure, which is currently the 'purpose' (sending address, receiving address) and 'destdata' (used for storing payment requests IIRC)
  • getaddressesbylabel: get addresses labelled with one label
  • listlabels: list all labels (or labels with a certain purpose, such as receive/send)
  • setlabel: assign a label to an address
  • getlabeladdress: get the 'label address' for the specified label. This gets an unused address with the label, creating one if necessary should be removed according to discussion

These calls have a deprecated account parameter, which can be turned into a label-parameter as is:

  • listtransactions

Open questions

  • Should there be such a thing as a 'label address'? My initial feeling about this was
    'no', labels are just a name for one or more addresses, intuitively there is no "default address",
    and it also isn't a GUI feature (->no)
@jonasschnelli

This comment has been minimized.

Copy link
Member

commented Mar 21, 2016

Concept ACK.
I just wonder if this makes the wallet code more complex (add another layer).

My idea was it to duplicate the current wallet implementation (cp src/wallet src/newwallet-approach) and add such things there (after removing the accounts-related code). Also the Bip32 and @CodeShark multi-wallet PR could be added there.

The second wallet could come without API stableness (for the first two releases or so) and could be marked as experimental.

@laanwj

This comment has been minimized.

Copy link
Member Author

commented Mar 21, 2016

I just wonder if this makes the wallet code more complex (add another layer).

I disagree:

  • All of this functionality is required anyway to support the GUI.
  • Yes, there is some intentional duplication, but only until the account calls are ripped out, which should be one of the next steps.
  • There are only added RPC calls in rpcwallet.cpp. The CWallet class is not complicated by this.

The point here is to give a non-deprecated equivalent to the 'label' system as used in the GUI, so the subset of the 'account system' that people are still allowed to use. This is a required, but up to now missing part of deprecating the account system.

I'm not trying to rule out any other work that is being done such as multi-wallet support. I think this is pretty much orthogonal. As for alternative wallets, they've been proposed since at least 2012 - but none have materialized yet. And none of this change rules them out.

@jonasschnelli

This comment has been minimized.

Copy link
Member

commented Mar 21, 2016

As for alternative wallets, they've been proposed since at least 2012 - but none have materialized yet. And none of this change rules them out.

Yes. I agree.
This PR has a clear value.

@luke-jr

This comment has been minimized.

Copy link
Member

commented Mar 21, 2016

Note getaccountaddress does not presently get a "default" address, it gets an unused address with the label, creating one if necessary. This seems useful only for mining, since no other context can guarantee an address hasn't been "used" but not sent to yet. I can't think of a good way to deprecate this, however.

@laanwj

This comment has been minimized.

Copy link
Member Author

commented Mar 22, 2016

it gets an unused address with the label, creating one if necessary.

I wonder if we can find a better (or at least simpler, the GetAccountAddress function is pretty terrible) way to do this, now that we're creating a new API anyway. Need to think about this a bit.

@laanwj

This comment has been minimized.

Copy link
Member Author

commented Mar 23, 2016

@luke-jr

I wonder if we can find a better (or at least simpler, the GetAccountAddress function is pretty terrible) way to do this, now that we're creating a new API anyway. Need to think about this a bit.

I thought of the following: you could use two labels, one for the 'active' address, one for the 'normal'. Say mining_active and mining.

When the miner needs an address it will:

  • addr = getaddressesbylabel mining_active (if no address returned, go to "if so" directly below and skip the first)
  • getreceivedbyaddress - check with wallet that addr has been used before (or maybe add a more convenient RPC call for checking a single address)
  • If so:
    • setlabel <addr> mining - move current address to normal label
    • addr = getnewaddress mining_active - generate new address in mining_active label
    • Use addr for mining to
  • If not:
    • Use addr for mining to

This is a little bit more involved at the user side, but it avoids special administration (needing to keep around CAccount structure per label) at the server side.

@luke-jr

This comment has been minimized.

Copy link
Member

commented Apr 7, 2016

That looks like a lot of overhead, and this is a rather time-sensitive call, as the miner is working on stale work until it's done.

Also, why are there no getreceivedbylabel/listreceivedbylabel? These don't have anything to do with balances.

@laanwj

This comment has been minimized.

Copy link
Member Author

commented Apr 7, 2016

That looks like a lot of overhead, and this is a rather time-sensitive call, as the miner is working on stale work until it's done.

I'd suggest to try it. It shouldn't be much slower.

Also, why are there no getreceivedbylabel/listreceivedbylabel? These don't have anything to do with balances.

Looks like you're right. getreceivedbyaccount doesn't actually return the account balance, but the total number of coins sent to the addresses that make up the account?

listreceivedbyaccount on the other hand goes over the account tally. But I agree if it only were to show actual transactions sent to addresses belonging to a label it'd be ok.

@sipa

This comment has been minimized.

Copy link
Member

commented Apr 7, 2016

@luke-jr Wouldn't it be feasible to instead generate a sequence of deterministic addresses for mining, for example using BIP32 derivation with the block height as index?

@luke-jr

This comment has been minimized.

Copy link
Member

commented Apr 7, 2016

I'd suggest to try it. It shouldn't be much slower.

@laanwj getreceivedbyaddress at least would loop over all the wtx... and then there's the additional latency from the back and forth of multiple calls. I haven't tried it yet, though.

Wouldn't it be feasible to instead generate a sequence of deterministic addresses for mining, for example using BIP32 derivation with the block height as index?

@sipa Perhaps, if the wallet had a way to do this. Using the height seems incompatible with gap limits, though?

@sipa

This comment has been minimized.

Copy link
Member

commented Apr 7, 2016

@luke-jr I mean the mining software can do derivation, and import the keys into the wallet when a block is found.

@MarcoFalke

This comment has been minimized.

Copy link
Member

commented May 22, 2016

The functional test coverage for accounts is minimal or not existent, I think we should move forward with this pull.

Needs rebase.

@jonasschnelli

This comment has been minimized.

Copy link
Member

commented May 22, 2016

Re-Concept ACK.
I think this solution makes more sense than the closed #7830.
Needs tests, rebase and release-note mentioning.

@wallclockbuilder

This comment has been minimized.

Copy link

commented May 22, 2016

Needs tests.

@laanwj

This comment has been minimized.

Copy link
Member Author

commented Jun 2, 2016

@wallclockbuilder No shit, have you seen the TODOs at the bottom of the opening post?

@laanwj

This comment has been minimized.

Copy link
Member Author

commented Jun 2, 2016

To be clear I posted this to get comments on the API, is there anything left to be done there? I'm going to write tests when it is clear that this is what we want at all.

@laanwj laanwj added this to the Future milestone Jun 13, 2016

@MarcoFalke

This comment has been minimized.

Copy link
Member

commented Jul 14, 2016

Do not use the deprecated account system and the label system with the same wallet at the same time
[...]
optional: a flag in the wallet to prevent use of both the account and label API

I would not consider this optional. User will always do what you not want them to do.

@sipa

This comment has been minimized.

Copy link
Member

commented Aug 25, 2016

Concept ACK

@andrewbaine

This comment has been minimized.

Copy link

commented Sep 24, 2016

listtransactions has an "account" argument where you now you would pass "*" if you need to supply non-default args for count, from, and includeWatchOnly. Will there be a way to query for transactions affecting any address with a given label? Could we tack on a "label" argument to listtransactions?

@laanwj

This comment has been minimized.

Copy link
Member Author

commented Sep 24, 2016

I think the account argument of listtransactions could simply be re-used as a label argument. As listing transactions to a label has nothing to do with per-label balances there is no need to drop that particular functionality,

@andrewbaine

This comment has been minimized.

Copy link

commented Oct 3, 2016

repurposing the "account" argument to be "label" makes sense to me

ryanofsky added a commit to ryanofsky/bitcoin that referenced this pull request Mar 19, 2018

Rename account to label where appropriate
This change only updates strings and adds RPC aliases, but should simplify the
implementation of address labels in
bitcoin#7729, by getting renaming out of the
way and letting it focus on semantics.

The difference between accounts and labels is that labels apply only to
addresses, while accounts apply to both addresses and transactions
(transactions have "from" and "to" accounts). The code associating accounts
with transactions is clumsy and unreliable so we would like get rid of it.

ryanofsky added a commit to ryanofsky/bitcoin that referenced this pull request Mar 19, 2018

Rename account to label where appropriate
This change only updates strings and adds RPC aliases, but should simplify the
implementation of address labels in
bitcoin#7729, by getting renaming out of the
way and letting it focus on semantics.

The difference between accounts and labels is that labels apply only to
addresses, while accounts apply to both addresses and transactions
(transactions have "from" and "to" accounts). The code associating accounts
with transactions is clumsy and unreliable so we would like get rid of it.

ryanofsky pushed a commit to ryanofsky/bitcoin that referenced this pull request Mar 19, 2018

[wallet] Make setlabel idempotent
Prevent setlabel calls on an address that already has the same label, and which
is the already the default "label address" of that label, from generating a
new default address.

This way calling setlabel on an address that already has the specified label is
a no-op.

Change suggested by Russell Yanofsky <russ@yanofsky.org> in
bitcoin#7729 (comment)

laanwj and others added some commits Mar 21, 2016

rpc: introduce 'label' API for wallet
Add label API to wallet RPC.

This is one step towards #3816 ("Remove bolt-on account system") although it doesn't
actually remove anything yet.

These initially mirror the account functions, with the following differences:

- These functions aren't DEPRECATED in the help
- Help mentions 'label' instead of accounts. In the language used, labels are
  associated with addresses, instead of addresses associated with labels. (unlike
  with accounts.)
- Labels have no balance
  - No balances in `listlabels`
  - `listlabels` has no minconf or watchonly argument
- Like in the GUI, labels can be set on any address, not just receiving addreses
- Unlike accounts, labels can be deleted.
  Being unable to delete them is a common annoyance (see #1231).
  Currently only by reassigning all addresses using `setlabel`, but an explicit
  call `deletelabel` which assigns all address to the default label may make
  sense.
[wallet] Make setlabel idempotent
Prevent setlabel calls on an address that already has the same label, and which
is the already the default "label address" of that label, from generating a
new default address.

This way calling setlabel on an address that already has the specified label is
a no-op.

Change suggested by Russell Yanofsky <russ@yanofsky.org> in
#7729 (comment)

laanwj added a commit that referenced this pull request Mar 22, 2018

Merge #11536: Rename account to label where appropriate
d2527bd Rename wallet_accounts.py test (Russell Yanofsky)
045eeb8 Rename account to label where appropriate (Russell Yanofsky)

Pull request description:

  Rename account to label where appropriate

  This change only updates strings and adds RPC aliases, but should simplify the implementation of address labels in #7729, by getting renaming out of the way and letting that change focus on semantics.

  The difference between accounts and labels is that labels apply only to addresses, while accounts apply to both addresses and transactions (transactions have "from" and "to" accounts). The code associating accounts with transactions is clumsy and unreliable so we would like get rid of it.

  ---

  There is a rebased version of #7729 atop this PR at https://github.com/ryanofsky/bitcoin/commits/pr/label, see #7729 (comment).

Tree-SHA512: b3f934e612922d6290f50137f8ba71ddfaea4485713c7d97e89400a8b73b09b254f9186dffa462c77f5847721f5af9852b5572ade5443d8ee95dd150b3edb7ff
@ryanofsky

This comment has been minimized.

Copy link
Contributor

commented Mar 23, 2018

Rebased PR at https://github.com/ryanofsky/bitcoin/commits/pr/label:

  • 6a0d274 rpc: introduce 'label' API for wallet
  • fef4178 [wallet] Make setlabel idempotent

@laanwj laanwj force-pushed the laanwj:2016_03_wallet_label_api branch from 0e3ef24 to fef4178 Mar 23, 2018

@laanwj

This comment has been minimized.

Copy link
Member Author

commented Mar 23, 2018

@ryanofsky Thank you! Replaced the branch with that one.

"\nExamples:\n"
+ HelpExampleCli("setlabel", "\"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XX\" \"tabby\"")
+ HelpExampleRpc("setlabel", "\"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XX\", \"tabby\"")
+ HelpExampleCli("setlabel", "\"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XZ\" \"tabby\"")

This comment has been minimized.

Copy link
@MarcoFalke

MarcoFalke Mar 23, 2018

Member

Unrelated change?

If you really wanted to change that DummyAddress(Params()) would be a better choice.

@Sjors

Sjors approved these changes Mar 28, 2018

Copy link
Member

left a comment

tACK fef4178.

I just noticed that changing or removing a label in the console live updates the Receiving Addresses UI, nice! For a future PR we should figure out what to do with the Receive and Transaction tab "labels", as these seem to use an independent mechanism.

"\nArguments:\n"
"1. \"label\" (string, required) The label name for the address. It can also be set to the empty string \"\" to represent the default label. The label does not need to exist, it will be created and a new address created if there is no label by the given name.\n"

This comment has been minimized.

Copy link
@Sjors

Sjors Mar 28, 2018

Member

I'm fine with removing this remark, but note that the behavior of creating a new address still exists (though without an address type param). It might make sense to deprecate that behavior in a followup PR, and require getnewaddress if getlabeladdress doesn't return anything. That also makes the choice of potentially reusing an address more explicit.

@laanwj

This comment has been minimized.

Copy link
Member Author

commented Mar 28, 2018

For a future PR we should figure out what to do with the Receive and Transaction tab "labels", as these seem to use an independent mechanism.

Transaction tab labels use the same mechanism so it would be a matter of listening to a notification and repainting. Receive requests are stored separately, because they have extra metadata besides a label.

@Sjors

This comment has been minimized.

Copy link
Member

commented Mar 28, 2018

Ah yes, I see it. Restarting QT doesn't change or remove a label from Requested payments history for me, as you point out. Switching tabs is enough to update labels in the transactions view.

@jnewbery
Copy link
Member

left a comment

Comments inline.

Feel free to squash my second commit into the first.

{ "wallet", "getaccountaddress", &getlabeladdress, {"account"} },
{ "wallet", "getaccount", &getaccount, {"address"} },
{ "wallet", "getaddressesbyaccount", &getaddressesbyaccount, {"account"} },
{ "wallet", "getaddressinfo", &getaddressinfo, {"address"} },

This comment has been minimized.

Copy link
@jnewbery

jnewbery Mar 28, 2018

Member

I don't think getaddressinfo should be deprecated. It's not an account rpc

This comment has been minimized.

Copy link
@ryanofsky

ryanofsky Mar 28, 2018

Contributor

I don't think getaddressinfo should be deprecated. It's not an account rpc

Good catch. getaddressinfo should remain where it was above "getbalance". It's my fault for accidentally moving it into this section during a rebase.

@@ -290,14 +295,14 @@ UniValue setlabel(const JSONRPCRequest& request)

if (request.fHelp || request.params.size() < 1 || request.params.size() > 2)
throw std::runtime_error(
"setlabel \"address\" \"label\"\n"
"setlabel \"bitcoinaddress\" \"label\"\n"

This comment has been minimized.

Copy link
@jnewbery

jnewbery Mar 28, 2018

Member

Why rename the arguments/return values to bitcoinaddress everywhere? Seems like a gratuitous API break.

address is used in many RPCs for a bitcoin address. Why not continue that convention? (and if you really must change this, current style guidelines call for snake_case for args)

This comment has been minimized.

Copy link
@ryanofsky

ryanofsky Mar 28, 2018

Contributor

Why rename the arguments/return values to bitcoinaddress everywhere?

Agree it would be better to stick to address, but just as a historical note, this wasn't a "gratuitious" API break when it was originally written in 8571fee, because it preceded #11536, so the setlabel RPC was brand new.

"\nResult:\n"
" { (json object with information about address)\n"
" \"name\": \"labelname\" (string) The label\n"
" \"purpose\": \"string\" (string) Purpose of address (\"send\" for sending address, \"receive\" for receiving address)\n"

This comment has been minimized.

Copy link
@jnewbery

jnewbery Mar 28, 2018

Member

Comment: https://github.com/bitcoin/bitcoin/pull/7729/files#r121986789 not addressed. Result also includes an array of destdata. Please update help text.

EDIT: I think we should just drop the destdata from the response. It wasn't available in the old account RPCs, and it appears to me that the only place we add to destdata is in saveReceiveRequest().

We can always add destdata to the response in a later PR if required.

"\nArguments:\n"
"1. \"bitcoinaddress\" (string, required) The bitcoin address for label lookup.\n"
"\nResult:\n"
" { (json object with information about address)\n"

This comment has been minimized.

Copy link
@jnewbery

jnewbery Mar 28, 2018

Member

Can we make this RPC return an array? We may want to be able to attach multiple labels to addresses in the future (#7729 (comment)), and making this RPC return an array will allow us to do that without a breaking API change

This comment has been minimized.

Copy link
@ryanofsky

ryanofsky Mar 28, 2018

Contributor

Can we make this RPC return an array?

It might be better to not add a getlabel RPC at all but instead just return this information in getaddressinfo (recently added in #10583).

}
}
UniValue ret(UniValue::VARR);
for (const std::string &name : setLabels) {

This comment has been minimized.

Copy link
@jnewbery

jnewbery Mar 28, 2018

Member

Should we sort the label names before returning?

if (item.second.name == strLabel) {
ret.push_back(Pair(EncodeDestination(item.first), AddressBookDataToJSON(item.second, false)));
}
}

This comment has been minimized.

Copy link
@jnewbery

jnewbery Mar 28, 2018

Member

Should this throw an error if the label doesn't exist? Currently it returns an empty object.

@@ -208,11 +213,11 @@ UniValue getlabeladdress(const JSONRPCRequest& request)
if (request.fHelp || request.params.size() != 1)
throw std::runtime_error(
"getlabeladdress \"label\"\n"
"\nReturns the current Bitcoin address for receiving payments to this label.\n"
"\nReturns the current 'label address' for this label.\n"

This comment has been minimized.

Copy link
@jnewbery

jnewbery Mar 28, 2018

Member

The behaviour for this RPC is weird. If called for a label that doesn't exist, it creates a new label, and then adds a new address as the 'label address' for that label. That's not very intuitive, and I think it's a bad experience (for example if a user typos an existing label name). Can we change this so that the rpc returns an error if called with a non-existent label name?

@@ -15,6 +15,7 @@

from test_framework.test_framework import BitcoinTestFramework
from test_framework.util import assert_equal
from collections import defaultdict

This comment has been minimized.

Copy link
@jnewbery

jnewbery Mar 28, 2018

Member

nit: standard library imports before project imports please!

@jnewbery

This comment has been minimized.

Copy link
Member

commented Apr 4, 2018

@laanwj - it seems like you're struggling to find time to maintain this PR.

I'd like to push this forwards and make sure it gets in for v0.17. Would you object if I took ownership?

buddilla added a commit to buddilla/bitcoin that referenced this pull request Apr 5, 2018

doc: add qrencode to brew install instructions (#1)
* [gitian] Re-order keys by owner first name alphabetic order

* [gitian] Add kallewoof key for signing

* Simplify Base32 and Base64 conversions

* CheckMinimalPush comments are prescriptive

* [rpc] createrawtransaction: Accept sorted outputs

* qa: Cache only chain and wallet for regtest datadir

* [wallet] Get rid of CWalletTx default constructor

No change in behavior in the normal case. But buggy mapWallet lookups with
invalid txids will now throw exceptions instead of inserting dummy entries into
the map, and potentially causing segfaults and other failures.

This also makes it a compiler error to use the mapWallet[hash] syntax which
could create dummy entries.

* [wallet] Construct CWalletTx objects in CommitTransaction

Construct CWalletTx objects in CWallet::CommitTransaction, instead of having
callers do it. This ensures CWalletTx objects are constructed in a uniform way
and all fields are set.

This also makes it possible to avoid confusing and wasteful CWalletTx copies in
bitcoin#9381

There is no change in behavior.

* bitcoin-cli: Provide a better error message when bitcoind is not running

Before this patch:

```
$ bitcoin-cli -testnet echo 'hello world'
error: Could not locate RPC credentials. No authentication cookie could be found, and RPC password is not set.  See -rpcpassword and -stdinrpcpass.  Configuration file: (/root/.bitcoin/bitcoin.conf)
```

After this patch:

```
$ bitcoin-cli -testnet echo 'hello world'
error: Could not connect to the server 127.0.0.1:18332

Make sure the bitcoind server is running and that you are connecting to the correct RPC port.
```

* Format timestamps using ISO 8601 formatting (e.g. "2018-02-28T12:34:56Z")

* Z is the zone designator for the zero UTC offset.
* T is the delimiter used to separate date and time.

This makes it clear for the end-user that the date/time logged is
specified in UTC and not in the local time zone.

* scripted-diff: Convert 11 enums into scoped enums (C++11)

-BEGIN VERIFY SCRIPT-

sed -i 's/enum DBErrors/enum class DBErrors/g' src/wallet/walletdb.h
git grep -l DB_ | xargs sed -i 's/DB_\(LOAD_OK\|CORRUPT\|NONCRITICAL_ERROR\|TOO_NEW\|LOAD_FAIL\|NEED_REWRITE\)/DBErrors::\1/g'
sed -i 's/^    DBErrors::/    /g' src/wallet/walletdb.h

sed -i 's/enum VerifyResult/enum class VerifyResult/g' src/wallet/db.h
sed -i 's/\(VERIFY_OK\|RECOVER_OK\|RECOVER_FAIL\)/VerifyResult::\1/g' src/wallet/db.cpp

sed -i 's/enum ThresholdState/enum class ThresholdState/g' src/versionbits.h
git grep -l THRESHOLD_ | xargs sed -i 's/THRESHOLD_\(DEFINED\|STARTED\|LOCKED_IN\|ACTIVE\|FAILED\)/ThresholdState::\1/g'
sed -i 's/^    ThresholdState::/    /g' src/versionbits.h

sed -i 's/enum SigVersion/enum class SigVersion/g' src/script/interpreter.h
git grep -l SIGVERSION_ | xargs sed -i 's/SIGVERSION_\(BASE\|WITNESS_V0\)/SigVersion::\1/g'
sed -i 's/^    SigVersion::/    /g' src/script/interpreter.h

sed -i 's/enum RetFormat {/enum class RetFormat {/g' src/rest.cpp
sed -i 's/RF_\(UNDEF\|BINARY\|HEX\|JSON\)/RetFormat::\1/g' src/rest.cpp
sed -i 's/^    RetFormat::/    /g' src/rest.cpp

sed -i 's/enum HelpMessageMode {/enum class HelpMessageMode {/g' src/init.h
git grep -l HMM_ | xargs sed -i 's/HMM_BITCOIN/HelpMessageMode::BITCOIN/g'
sed -i 's/^    HelpMessageMode::/    /g' src/init.h

sed -i 's/enum FeeEstimateHorizon/enum class FeeEstimateHorizon/g' src/policy/fees.h

sed -i 's/enum RBFTransactionState/enum class RBFTransactionState/g' src/policy/rbf.h
git grep -l RBF_ | xargs sed -i 's/RBF_TRANSACTIONSTATE_\(UNKNOWN\|REPLACEABLE_BIP125\|FINAL\)/RBFTransactionState::\1/g'
sed -i 's/^    RBFTransactionState::/    /g' src/policy/rbf.h

sed -i 's/enum BlockSource {/enum class BlockSource {/g' src/qt/clientmodel.h
git grep -l BLOCK_SOURCE_ | xargs sed -i 's/BLOCK_SOURCE_\(NONE\|REINDEX\|DISK\|NETWORK\)/BlockSource::\1/g'
sed -i 's/^    BlockSource::/    /g' src/qt/clientmodel.h

sed -i 's/enum FlushStateMode {/enum class FlushStateMode {/g' src/validation.cpp
sed -i 's/FLUSH_STATE_\(NONE\|IF_NEEDED\|PERIODIC\|ALWAYS\)/FlushStateMode::\1/g' src/validation.cpp
sed -i 's/^    FlushStateMode::/    /g' src/validation.cpp

sed -i 's/enum WitnessMode {/enum class WitnessMode {/g' src/test/script_tests.cpp
sed -i 's/WITNESS_\(NONE\|PKH\|SH\)/WitnessMode::\1/g' src/test/script_tests.cpp
sed -i 's/^    WitnessMode::/    /g' src/test/script_tests.cpp

-END VERIFY SCRIPT-

* Split up and sanitize CWalletTx serialization

* Calculate and store the number of bytes required to spend an input

* Store effective value, fee, and long term fee in CInputCoin

Have CInputCOin store effective value information. This includes the effective
value itself, the fee, and the long term fee for the input

* Log fatal LevelDB errors more verbosely

* configure: UniValue 1.0.4 is required for pushKV(, bool)

* Allow to optional specify the directory for the blocks storage

* Ubuntu xenial first dependencies

Add update and upgrade commands to enable the installation of the first dependencies on ubuntu xenial. If those are not executed some packages can not be found.

* Split up and sanitize CAccountingEntry serialization

* rpc: Update createrawtransaction examples

* qt: Avoid querying unnecessary model data when filtering transactions

* [qa] util: Remove unused sync_chain

* Implement Branch and Bound coin selection in a new file

Create a new file for coin selection logic and implement the BnB algorithm in it.

* Add a GetMinimumFeeRate function which is wrapped by GetMinimumFee

* Remove coinselection.h -> wallet.h circular dependency

Changes CInputCoin to coinselection and to use CTransactionRef in
order to avoid a circular dependency. Also moves other coin selection
specific variables out of wallet.h to coinselectoin.h

* Move output eligibility to a separate function

* Move current coin selection algorithm to coinselection.{cpp,h}

Moves the current coin selection algorithm out of SelectCoinsMinConf
and puts it in coinselection.{cpp,h}. The new function, KnapsackSolver,
instead of taking a vector of COutputs, will take a vector of CInputCoins
that is prepared by SelectCoinsMinConf.

* Use a struct for output eligibility

Instead of specifying 3 parameters, use a struct for those parameters
in order to reduce the number of arguments to SelectCoinsMinConf.

* Add tests for the Branch and Bound algorithm

* Move original knapsack solver tests to coinselector_tests.cpp

* Have SelectCoinsMinConf and SelectCoins use BnB or Knapsack and use it

Allows SelectCoinsMinConf and SelectCoins be able to switch between
using BnB or Knapsack for choosing coins.

Has SelectCoinsMinConf do the preprocessing necessary to support either
BnB or Knapsack. This includes calculating the filtering the effective
values for each input.

Uses BnB in CreateTransaction to find an exact match for the output.
If BnB fails, it will fallback to the Knapsack solver.

* Benchmark BnB in the worst case where it exhausts

* Add a test to make sure that negative effective values are filtered

* travis: Clone depth 1 unless $CHECK_DOC

* Fix ComputeTimeSmart test failure with -DDEBUG_LOCKORDER

Failure looks like:

    Entering test case "ComputeTimeSmart"
    test_bitcoin: sync.cpp💯 void potential_deadlock_detected(const std::pair<void*, void*>&, const LockStack&, const LockStack&): Assertion `false' failed.
    unknown location(0): fatal error in "ComputeTimeSmart": signal: SIGABRT (application abort requested)
    wallet/test/wallet_tests.cpp(566): last checkpoint

Reproducible with:

    ./configure --enable-debug
    make -C src test/test_bitcoin && src/test/test_bitcoin --log_level=test_suite --run_test=wallet_tests/ComputeTimeSmart

Happens due to "92fabcd443 Add LookupBlockIndex function" which acquires
cs_main from inside CWallet::ComputeTimeSmart.

* Merge READWRITEMANY into READWRITE

* Support deserializing into temporaries

Currently, the READWRITE macro cannot be passed any non-const temporaries, as
the SerReadWrite function only accepts lvalue references.

Deserializing into a temporary is very common, however. See for example
things like 's >> VARINT(n)'. The VARINT macro produces a temporary wrapper
that holds a reference to n.

Fix this by accepting non-const rvalue references instead of lvalue references.
We don't propagate the rvalue-ness down, as there are no useful optimizations
that only apply to temporaries.

Then use this new functionality to get rid of many (but not all) uses of the
'REF' macro (which casts away constness).

* Apply hardening measurements in bitcoind systemd service file

Adds typical systemd hardening measurements for network services.

* Polish interfaces around PeerLogicValidation

* Make PeerLogicValidation final to prevent deriving from it [1]
* Prevent deletions of NetEventsInterface and CValidationInterface
  objects via a base class pointer

[1] silences the following compiler warning (from Clang 7.0.0):

/usr/include/c++/v1/memory:2285:5: error: delete called on non-final 'PeerLogicValidation' that has
      virtual functions but non-virtual destructor [-Werror,-Wdelete-non-virtual-dtor]
    delete __ptr;
    ^
/usr/include/c++/v1/memory:2598:7: note: in instantiation of member function
      'std::__1::default_delete<PeerLogicValidation>::operator()' requested here
      __ptr_.second()(__tmp);
      ^
init.cpp:201:15: note: in instantiation of member function 'std::__1::unique_ptr<PeerLogicValidation,
      std::__1::default_delete<PeerLogicValidation> >::reset' requested here
    peerLogic.reset();
                  ^

* Provide relevant error message if datadir is not writable.

* Remove unused variable in SortForBlock

* Actually disable BnB when there are preset inputs

We don't want to use BnB when there are preset inputs because there
is some weirdness with making that work with using the KnapsackSolver
as the fallback. Currently we say that we haven't used bnb when
there are preset inputs, but we don't actually disable BnB. This fixes
that.

* Do not check for main() in libminiupnpc

main() { main(); } causes "infinite recursion" compilation warning
which with -Werror fails the check.

* ax_boost_{chrono,unit_test_framework}.m4: take changes from upstream

Apply changes to
build-aux/m4/ax_boost_chrono.m4 and
build-aux/m4/ax_boost_unit_test_framework.m4
from upstream: https://github.com/peti/autoconf-archive

* Remove redundant checks for MSG_* from configure.ac

It is redundant to check for the presence of MSG_NOSIGNAL macro in
configure.ac, define HAVE_MSG_NOSIGNAL and then check whether the later
is defined in the source code. Instead we can check directly whether
MSG_NOSIGNAL is defined. Same for MSG_DONTWAIT.

In addition to that, the checks we had in configure.ac produce a
compiler warning about unused variable and thus could fail if
-Werror is present and erroneously proclaim that the macros are
not available.

* Test that BnB is not used when there are preset inputs

* Document RPC method aliasing

Suggested by Sjors Provoost <sjors@sprovoost.nl> in
bitcoin#11536 (comment)

* Add static_assert to prevent VARINT(<signed value>)

Using VARINT with signed types is dangerous because negative values will appear
to serialize correctly, but then deserialize as positive values mod 128.

This commit changes the VARINT macro to trigger an error by default if called
with an signed value, and updates broken uses of VARINT to pass a special flag
that lets them keep working with no change in behavior.

* test: Use wait_until in tests where time was used for polling

* test: Use os.path.join consistently in feature_pruning tests

* [Trivial] Simplify if-else blocks and more descriptive variable naming

* Append scripts to new test_list array to fix bad assignment

* wallet: Change output type globals to members

* QA: Add -blocksdir test

* [Tests] Move assert_start_raises_init_error method to TestNode

* [Tests] Require exact match in assert_start_raises_init_eror()

* Avoiding 'file' function name from python2 with more descriptive variable naming

* Qt: remove "new" button during receive-mode in addressbook

* [tests] Fix flake8 warnings in feature_block.py

* [tests] Tidy up feature_block.py

- move all helper methods to the end
- remove block, create_tx and create_and_sign_tx shortcuts
- remove --runbarelyexpensive option, since it defaults to True and it's
unlikely that anyone ever runs the test with this option set to false.

* [tests] Add logging to feature_block.py

* Remove unreachable help conditions

* Rename account to label where appropriate

This change only updates strings and adds RPC aliases, but should simplify the
implementation of address labels in
bitcoin#7729, by getting renaming out of the
way and letting it focus on semantics.

The difference between accounts and labels is that labels apply only to
addresses, while accounts apply to both addresses and transactions
(transactions have "from" and "to" accounts). The code associating accounts
with transactions is clumsy and unreliable so we would like get rid of it.

* Rename wallet_accounts.py test

This is a separate commit because changing the test at the same time as
renaming it breaks git (and github) rename detection.

* qa: Use node.datadir instead of tmpdir in test framework

* [tests] Change feature_block.py to use BitcoinTestFramework

* [tests] Improve assert message when wait_until() fails

* scripted-diff: rename TestNode to TestP2PConn in tests

Several test scripts define a subclass of P2PInterface called TestNode.
This commit renames those to TestP2PConn since we already have a
TestNode class in the test framework.

-BEGIN VERIFY SCRIPT-
sed -i s/TestNode/TestP2PConn/ test/functional/*py test/functional/test_framework/comptool.py
_END VERIFY SCRIPT-

* qa: Allow for partial_match when checking init error

This allows the tests to pass on different platforms

* tests: Test connecting to a non-existing server

* tests: Test connecting with non-existing RPC cookie file

* -blocksdir: keep blockindex leveldb database in datadir

* Remove unnecessary NONNEGATIVE_SIGNED

Switch to unsigned encoding, which is backwards compatible and avoids MSVC
error reported bitcoin#12732

* use base58 map instead of strchr()

* Make FastRandomContext support standard C++11 RNG interface

This makes it possible to plug it into the various standard C++11 random
distribution algorithms and other functions like std::shuffle.

* Fix typos

* tests: Remove unused argument max_invalid from check_estimates(...)

* Fix typos

* Qt: Warn users about invalid-BIP21 URI bitcoin://

* shuffle selected coins before transaction finalization

* Replace boost::call_once with std::call_once

* Add native support for serializing char arrays without FLATDATA

Support is added to serialize arrays of type char or unsigned char directly,
without any wrappers. All invocations of the FLATDATA wrappers that are
obsoleted by this are removed.

This includes a patch by Russell Yanofsky to make char casting type safe.

The serialization of CSubNet is changed to serialize a bool directly rather
than though FLATDATA. This makes the serialization independent of the size
of the bool type (and will use 1 byte everywhere).

* Move compressor utility functions out of class

* [config] Remove blockmaxsize option

The blockmaxsize option was marked as deprecated in V0.15.1, and code
was added to convert provided blockmaxsize into blockmaxweight. However,
this code was incorrectly implemented, and blockmaxsize was silently
ignored.

No users have complained about blockmaxsize being ignored, so just
remove it in V0.17.

* Docs: Improve documentation on standard communication channels

More information about connection on IRC will hopefully help new
contributors.

* shuffle sendmany recipients ordering to shuffle tx outputs

* add release note for sendmany output shuffling

* [CI]: bump travis timeout for make check to 50m

* Inline CKeyStore::AddKey(const CKey &) in CBasicKeyStore

* Move CKeyStore::cs_KeyStore to CBasicKeyStore

* Make CTxMemPool::isSpent() const

* [REST] Handle UTXO retrieval when ignoring the mempool

Current REST API always returns empty UTXO when invoked without `/checkmempool/` URL part.

After the fix:
```
$ curl -s http://localhost:8332/rest/getutxos/0e3e2357e806b6cdb1f70b54c3a3a17b6714ee1f0e68bebb44a74b1efd512098-0.json | jq
{
  "chainHeight": 514109,
  "chaintipHash": "0000000000000000001fe76d1445e8a6432fd2de04261dc9c5915311dc7ad6de",
  "bitmap": "1",
  "utxos": [
    {
      "height": 1,
      "value": 50,
      "scriptPubKey": {
        "asm": "0496b538e853519c726a2c91e61ec11600ae1390813a627c66fb8be7947be63c52da7589379515d4e0a604f8141781e62294721166bf621e73a82cbf2342c858ee OP_CHECKSIG",
        "hex": "410496b538e853519c726a2c91e61ec11600ae1390813a627c66fb8be7947be63c52da7589379515d4e0a604f8141781e62294721166bf621e73a82cbf2342c858eeac",
        "reqSigs": 1,
        "type": "pubkey",
        "addresses": [
          "12c6DSiU4Rq3P4ZxziKxzrL5LmMBrzjrJX"
        ]
      }
    }
  ]
}
```

Before the fix:
```
$ curl -s http://localhost:8332/rest/getutxos/0e3e2357e806b6cdb1f70b54c3a3a17b6714ee1f0e68bebb44a74b1efd512098-0.json | jq
{
  "chainHeight": 514109,
  "chaintipHash": "0000000000000000001fe76d1445e8a6432fd2de04261dc9c5915311dc7ad6de",
  "bitmap": "0",
  "utxos": []
}
```

* Add username and ip logging for RPC method requests

* Qt: Remove unused method setupAmountWidget(...)

* Introduce interface for signing providers

CKeyStore is a rich interface that provides many features, including knowledge
of scripts and pubkeys for solving, private keys for signing, in addition to
watch-only keys and scripts, and distinguishing lack of keys from them just
being encrypted.

The signing logic in script/sign does not actually need most of these features.
Here we introduce a simpler interface (SigningProvider) which *only* provides
keys and scripts. This is actually sufficient for signing.

In addtion, we swap the dependency between keystore and script/sign
(keystore now depends on script/script with CKeyStore deriving from
SigningProvider, rather than CKeyStore being the interface that signing
relies on).

* Reduce variable scopes

* Qt: Add wallet selector to debug console

* Qt: QComboBox::setVisible doesn't work in toolbars, so defer adding it at all until needed

* Qt: Get wallet name from WalletModel rather than passing it around

* Qt: When multiple wallets are used, include in notifications the name

* GUI: RPCConsole: Log wallet changes

* Qt: show wallet name in send confirmation dlg in case of multiwallet

* Qt: show wallet name in request dlg in case of multiwallet

* Qt: hide RPCConsole wallet selector when no wallets are present

* rpc: Adjust ifdef to avoid unreachable code

* [Tests] Use blockmaxweight where tests previously had blockmaxsize

* [wallet] Move wallet init functions into WalletInit class.

* [wallet] Create wallet init interface.

* qt: Avoid resetting on resetguisettigs=0

* do not truncate .dat extension for wallets in gui

* Change all python files to use Python3

* init: Fix help message for checkblockindex

* doc: Refer to witness reserved value as spec. in the BIP

* doc: Add note about our preference for scoped enumerations ("enum class")

* [wallet] Use global g_wallet_init_interface to init/destroy the wallet.

This commit creates a global g_wallet_init_interface, which is created
in bitcoind and bitcoin-qt. g_wallet_init_interface is used to init
and destroy the wallet.

This removes the dependency from init.cpp on the wallet library.

* [wallet] Add dummy wallet init class

* Fix error in memory usage calculation (unintended integer division)

* qa: Fix function names in feature_blocksdir

* Add additional tests for GetBoolArg()

This is meant to be an intermediate commit to prove that the next does not
introduce any changes in the semantics of boolean option parsing.

* Track negated arguments in the argument paser.

This commit adds tracking for negated arguments. This change will be used in a
future commit that allows disabling the debug.log file using -nodebuglogfile.

* test: Make summary row bold-red if any test failed

Make the summary row of the test runner bold red if any test fails.
This helps visibility if something fails.

* Make base58 python contrib code work with python3

* test: List any failed tests at the end of test_runner output

Change sorting output to put failed tests at the end of test_runner
output.

* [Tests] fix a typo in TestNode.assert_start_raises_init_error()

Also, use specific exception for testing TestNode initialization failure.

* contrib: Fix check-doc script regexes

* contrib: Remove unused import string

* init: Remove help text for non-existent -fuzzmessagestest arg

* Revert "test: Update trust git root".

This reverts commit 7deba93.

This is neither a "test" change, nor should the trusted-git-root
have been updated - there is a process for expired PGP keys.

* Add Marco-expired-key-signed-commits to allow-revsig-commits

* Improve formatting of developer notes

Summary of changes:

 * Add a TOC to the page
 * Make tips and tricks section use h3 headings
 * Reformat and clarify some sections

* Add --with-sanitizers option to configure

This enables the use of different compiler sanitizers, coresponding to
the -fsanitize option in GCC and Clang.

* [tests] Fix intermittent rpc_net.py failure.

rpc_net.py would intermittently fail on Travis, probably
due to assuming that two consecutive RPC calls were atomic.
Fix this by only testing that amounts are bounded above and
below rather than equal.

* [contrib] fixup security-check.py Python3 support

* [contrib] fixup symbol-check.py Python3 support

* Bugfix: RPC: savemempool: Don't save until LoadMempool() is finished

* Increase LevelDB max_open_files unless on 32-bit Unix.

This change significantly increases IBD performance by increasing the
amount of the UTXO index that can remain in memory. To ensure this
doesn't cause problems in the future, a static_assert on the LevelDB
version has been added, which must be updated by anyone upgrading
LevelDB.

* [verify-commits] Add some additional useful documentation.

* test: Remove travis checkout depth

Tests on branches of non-head commits are failing, because the depth of
1 doesn't allow checking them out.

Remove `depth` as was the case before fa44af5,
so that Travis can determine the minimum depth to check out.

* Fixes Missing QRCode Build

Build was missing qr code(qrencode libs) support and brew team no longer supports anything less than macOS 10.11

* Fixed notes

Fixed notes to reflect that 10.8 is still supported via gitian

* Rolled Back Tested on

Below is a suggestion for future release after EOL is done

* Works with macOS 10.11 through 10.13 on 64-bit Intel processors only.

* macOS 10.8 and higher is still supported on [gitian](/contrib) builds

* Update build-osx.md

Below is a suggestion for future release after EOL is done

* Works with macOS 10.11 through 10.13 on 64-bit Intel processors only.

* macOS 10.8 and higher is still supported on [gitian](/contrib) builds
@GSPP

This comment has been minimized.

Copy link

commented Apr 5, 2018

Maybe there should be a call returning all addresses and all labels (a bulk call). If you want to dump all address-label pairs this would involve many RPC calls right now. This bulk call should even (optionally?) include internal addresses such as pool addresses. Simply all data. I have had the need many times to reflect over absolutely all wallet contents and such an API call would have been very welcome.

@jnewbery

This comment has been minimized.

Copy link
Member

commented Apr 5, 2018

I've opened #12892 to supersede this PR. Thanks for all your work @laanwj !

@MarcoFalke

This comment has been minimized.

Copy link
Member

commented Apr 5, 2018

Indeed thanks to @laanwj for initiating it and coming up with the original version of the code. And thx to @ryanofsky and @jnewbery for picking it up.

@MarcoFalke MarcoFalke closed this Apr 5, 2018

@jnewbery jnewbery referenced this pull request Apr 11, 2018

Closed

Remove wallet 'account' API #12952

6 of 6 tasks complete

HashUnlimited pushed a commit to HashUnlimited/chaincoin that referenced this pull request Jul 31, 2018

Rename account to label where appropriate
This change only updates strings and adds RPC aliases, but should simplify the
implementation of address labels in
bitcoin#7729, by getting renaming out of the
way and letting it focus on semantics.

The difference between accounts and labels is that labels apply only to
addresses, while accounts apply to both addresses and transactions
(transactions have "from" and "to" accounts). The code associating accounts
with transactions is clumsy and unreliable so we would like get rid of it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.