listreceivedbyaddress Filter Address #9503

JeremyRubin · 2017-01-10T18:26:46Z

This gives listreceivedbyaddress the ability to filter for a single address.

This functionality is useful for users such as TumbleBit who need to filter by address.

jnewbery · 2017-01-10T22:22:35Z

A couple of general comments:

How large are the results from the listreceivedbyaddress rpc expected to be? If the rpc only ever returns a small number of addresses, it should be easy enough for the client to receive the full list of balances and then filter the list itself?
if there's a chance that this functionality needs to be extended further to filter on a list of addresses rather than a single address, it'd be better to include that now. Since rpc: Add support for JSON-RPC named arguments #8811 , the arguments to the RPCs are part of the API, so changing them later becomes more troublesome. Perhaps change the only_address string to a filter_addresses array?

I've also added a few nits.

jnewbery · 2017-01-10T22:12:07Z

qa/rpc-tests/receivedby.py

+
+        #Not on addr
+        other_addr = self.nodes[0].getnewaddress() # note on node[0]! just a random addr
+        res = self.nodes[1].listreceivedbyaddress(5, True, True, other_addr)


Why set minconf to 5 here? Can you just set it to 0 to match the call to listreceivedbyaddress() above?

jnewbery · 2017-01-10T22:14:40Z

src/wallet/rpcwallet.cpp

@@ -1133,6 +1133,9 @@ UniValue ListReceived(const UniValue& params, bool fByAccounts)
    if (params.size() > 0)
        nMinDepth = params[0].get_int();

+    bool fFilterAddress = !fByAccounts && params.size() > 3;
+    const CBitcoinAddress filterAddress = fFilterAddress ?  CBitcoinAddress(params[3].get_str()) : CBitcoinAddress{};


nit: I think it's clearer to move these two lines below the if(params.size() > 2) block, so the params are being tested in order. I also have a slight preference to change this to a if(params.size() > 3) block to match the other tests.

I agree that it's important for the style to be consistent. I opted to convert the other argument parsing in ListReceived to match this style, at the expense of a larger diff, for the benefit of declaring the input parameters const.

My personal preference is to keep the if statements rather than converting them to ternary conditionals, as I think that's clearer (and consistent with all the other RPCs). I'm not that concerned about having the parameters const.

jnewbery · 2017-01-10T22:14:42Z

src/wallet/rpcwallet.cpp

@@ -1178,10 +1185,22 @@ UniValue ListReceived(const UniValue& params, bool fByAccounts)
    // Reply
    UniValue ret(UniValue::VARR);
    map<string, tallyitem> mapAccountTally;
-    BOOST_FOREACH(const PAIRTYPE(CBitcoinAddress, CAddressBookData)& item, pwalletMain->mapAddressBook)
+
+


nit: prefer not to have two blank lines in the middle of a function. Perhaps you can move one of them to between the if block and for block below.

JeremyRubin · 2017-01-13T16:38:49Z

@jnewbery addressed nits.

I think it could be quite large? And then you end up sending a bunch of extra crap over the network.
I don't think it would need to be extended as such, but I'll let others chime in.

instagibbs · 2017-01-13T18:03:04Z

I think (2) isn't really needed since if you need more than a small number you can just do un-filtered or a few repeated calls. I'm no expert on this use-case though.

JeremyRubin · 2017-01-13T19:18:18Z

@EthanHeilman thoughts?

Another optimization would be to allow for caching of this table on construction (maybe keep_cache/clear_cache parameters). This could reduce the O(n*m) complexity for making m repeated calls to O(n + m).

NicolasDorier · 2017-01-13T19:41:10Z

@JeremyRubin The reason why I am interested into that is that here is the code I am using for querying the transactions of a scriptPubKey: Using listtransactions in tumblebit.

As far as I see this PR would be able to replace my listtransactions nicely. Will review.

NicolasDorier · 2017-01-13T19:52:05Z

src/wallet/rpcwallet.cpp

+    const isminefilter filter =  ISMINE_SPENDABLE | (fIncludeWatchOnly ? ISMINE_WATCH_ONLY : 0);
+
+    const bool fFilterAddress = !fByAccounts && params.size() > 3;
+    const CBitcoinAddress filterAddress = fFilterAddress ?  CBitcoinAddress(params[3].get_str()) : CBitcoinAddress{};


CBitcoinAddress{} ? shouldn't it be CBitcoinAddress() ? (I guess not, as it build, but it is the first time I see that)

should be equivalent; it's c++11 list initializer syntax

EthanHeilman · 2017-01-13T20:09:52Z

@jnewbery

How large are the results from the listreceivedbyaddress rpc expected to be? If the rpc only ever returns a small number of addresses, it should be easy enough for the client to receive the full list of balances and then filter the list itself?

To perform a 800 user mix with TumbleBit we would need a watch list of 1600 addresses. However we there are times in which we only want to learn the status of a single address.

Sorting received transactions by address is a common enough usecase to have an RPC call. It seems likely that people are calling it and then writing filters to select the addresses they want (for instance this stackexchange question or this reddit post). It is an natural addition to the RPC API and one that would make our project and other projects more performant and cleaner.

jnewbery · 2017-01-13T21:23:12Z

@EthanHeilman - thanks. Sounds like there's widespread demand for this functionality.

concept ACK

luke-jr

utACK, with some nits.

luke-jr · 2017-01-20T20:32:28Z

src/wallet/rpcwallet.cpp

-    int nMinDepth = 1;
-    if (params.size() > 0)
-        nMinDepth = params[0].get_int();
+    const int nMinDepth = params.size() == 0 ? 1 : params[0].get_int();


Probably better to leave these alone. If params[0] is null, we really should silently use the default value... The longer version is also more readable/obvious.

luke-jr · 2017-01-20T20:35:59Z

src/wallet/rpcwallet.cpp

+    const bool fIncludeWatchOnly = params.size() > 2 && params[2].get_bool();
+    const isminefilter filter =  ISMINE_SPENDABLE | (fIncludeWatchOnly ? ISMINE_WATCH_ONLY : 0);
+
+    const bool fFilterAddress = !fByAccounts && params.size() > 3;


Should be false if params[3] is null.

luke-jr · 2017-01-20T20:37:04Z

src/wallet/rpcwallet.cpp

@@ -1162,6 +1159,10 @@ UniValue ListReceived(const UniValue& params, bool fByAccounts)
            if (!ExtractDestination(txout.scriptPubKey, address))
                continue;

+            if (fFilterAddress && !(filterAddress.Get() == address)) {


Maybe we should be storing filterAddress.Get() above rather than a CBitcoinAddress?

luke-jr · 2017-01-20T20:40:06Z

src/wallet/rpcwallet.cpp

        throw runtime_error(
-            "listreceivedbyaddress ( minconf include_empty include_watchonly)\n"
+            "listreceivedbyaddress ( minconf include_empty include_watchonly only_address)\n"


Prefer replacing all the params with an options Object, but perhaps that is better done in a separate PR.

yeah. Separate PR...

JeremyRubin · 2017-01-20T23:12:13Z

Addressed feedback, and squashed.

@luke-jr it now errors if the passed in address was not a valid address.

kallewoof · 2017-01-30T11:11:48Z

src/wallet/rpcwallet.cpp

        throw runtime_error(
-            "listreceivedbyaddress ( minconf include_empty include_watchonly)\n"
+            "listreceivedbyaddress ( minconf include_empty include_watchonly only_address)\n"


Nit: space before end paren at 'only_address )' to match space after start paren at '( minconf'

Not sure if I should fix that -- that's how it was before my PR.

Yea, best to fix, I think.

luke-jr

utACK

NicolasDorier · 2017-02-03T04:32:16Z

src/wallet/rpcwallet.cpp

@@ -1145,6 +1145,17 @@ UniValue ListReceived(const UniValue& params, bool fByAccounts)
        if(params[2].get_bool())
            filter = filter | ISMINE_WATCH_ONLY;

+    bool fFilterAddress = false;
+    CTxDestination filterAddress =  CBitcoinAddress{}.Get();


can you use CNoDestination(); here ? just to be coherent with nulladdress. (and the fact that I never saw this syntax before)

Yes, please.

NicolasDorier · 2017-02-06T07:03:25Z

tested and integrated in NTumbleBit (NTumbleBit/NTumbleBit@cd7c2f4). This replace listtransaction * as I needed.

Outside of my nit, ACK a96fbed

TheBlueMatt

utACK a96fbed aside from the style nits.

TheBlueMatt · 2017-02-07T23:11:38Z

src/wallet/rpcwallet.cpp

        throw runtime_error(
-            "listreceivedbyaddress ( minconf include_empty include_watchonly)\n"
+            "listreceivedbyaddress ( minconf include_empty include_watchonly only_address)\n"


Yea, best to fix, I think.

TheBlueMatt · 2017-02-07T23:11:46Z

src/wallet/rpcwallet.cpp

@@ -1145,6 +1145,17 @@ UniValue ListReceived(const UniValue& params, bool fByAccounts)
        if(params[2].get_bool())
            filter = filter | ISMINE_WATCH_ONLY;

+    bool fFilterAddress = false;
+    CTxDestination filterAddress =  CBitcoinAddress{}.Get();


Yes, please.

JeremyRubin · 2017-02-16T06:55:21Z

Nits addressed and squashed. Preserved the pre-squash here https://github.com/JeremyRubin/bitcoin/tree/listreceivedbyaddress-filtered-a96fbed73ed24591dd42845088639da0afaa436a

NicolasDorier · 2017-02-16T07:30:13Z

ACK e6f053a

kallewoof

utACK e6f053a

kallewoof · 2017-02-28T22:20:00Z

qa/rpc-tests/receivedby.py

+        res = self.nodes[1].listreceivedbyaddress(0, True, True, addr)
+        assert_array_result(res, {"address":addr}, expected)
+        if len(res) != 1:
+            raise AssertionError("listreceivedbyaddress expected only 1 result")


Couldn't you do

assert_equal(len(res), 1)

instead, here? That way the resulting len(res) would be visible in the output.

kallewoof · 2017-02-28T23:12:35Z

qa/rpc-tests/receivedby.py

+        other_addr = self.nodes[0].getnewaddress() # note on node[0]! just a random addr
+        res = self.nodes[1].listreceivedbyaddress(0, True, True, other_addr)
+        if res != []:
+            raise AssertionError("Should not have listed any transactions, got\n%s"%res)


I believe you can use assert_equal here too.

kallewoof · 2017-02-28T23:55:28Z

src/wallet/rpcwallet.cpp

@@ -1164,6 +1175,10 @@ UniValue ListReceived(const UniValue& params, bool fByAccounts)
            if (!ExtractDestination(txout.scriptPubKey, address))
                continue;

+            if (fFilterAddress && !(filterAddress == address)) {


Why not fFilterAddress && filterAddress != address?

(a != b) is not the same as !(a == b). I don't think != is defined here iirc.

Not the same only in the sense that the != operator may not be defined, right? Semantically they're identical. And yeah, I see now. CNoDestination has no operator!=.

I wonder if it would be worth adding the one liner

friend bool operator!=(const CNoDestination &a, const CNoDestination &b) { return false; }

to make this line look less confusing.

Except it's not a one liner, it needs to be added for all sorts of classes.

It's not a CNoDestination, it's a CTxDestination, which is a boost variant. Some supported versions of boost do not support operator!= for variants.

I thought if all variants accepted the operator the variant would accept it, in which case adding it to CNoDestination would be enough, but maybe I'm off on that one.
Edit: re-read your response; okay, didn't know that. Gotcha.

kallewoof · 2017-02-28T23:58:56Z

src/wallet/rpcwallet.cpp

        throw runtime_error(
-            "listreceivedbyaddress ( minconf include_empty include_watchonly)\n"
+            "listreceivedbyaddress (minconf include_empty include_watchonly only_address)\n"


Nit: standard elsewhere is to show optionals as "( opt1 opt2 ... )", not "(opt1 opt2 ...)" (i.e. instead of removing starting space, add ending space)

kallewoof · 2017-02-28T23:59:41Z

src/wallet/rpcwallet.cpp

@@ -2940,7 +2969,7 @@ static const CRPCCommand commands[] =
    { "wallet",             "listaddressgroupings",     &listaddressgroupings,     false,  {} },
    { "wallet",             "listlockunspent",          &listlockunspent,          false,  {} },
    { "wallet",             "listreceivedbyaccount",    &listreceivedbyaccount,    false,  {"minconf","include_empty","include_watchonly"} },
-    { "wallet",             "listreceivedbyaddress",    &listreceivedbyaddress,    false,  {"minconf","include_empty","include_watchonly"} },
+    { "wallet",             "listreceivedbyaddress",    &listreceivedbyaddress,    false,  {"minconf","include_empty","include_watchonly", "only_address"} },


Nit: abide by surrounding style of no space after comma

NicolasDorier · 2017-03-13T06:17:28Z

@JeremyRubin you might need to add a line in client.cpp (#9982)

NicolasDorier · 2017-03-14T05:05:37Z

@JeremyRubin Let me know if you are a bit busy and prefer I take care of this PR.

JeremyRubin · 2017-03-14T09:03:10Z

@NicolasDorier Go for it, sorry for the hold up!

maflcko · 2017-03-14T14:11:31Z

Discussion continues at #9991

@JeremyRubin

f087613 Add tests of listreceivedbyaddress address filtering (Jeremy Rubin) 8ee0812 Add address filtering to listreceivedbyaddress (Jeremy Rubin) Pull request description: Supersede #9503 created by @JeremyRubin , I will maintain it. Tree-SHA512: 2accaed493b7e1c2eb5cb5270180f100f8c718b6585b9574f294191c318dc622a79e42ac185300f291f82d3b2a6f1c00850b6b17e4ff2dbab94d71df695acbfe

@JeremyRubin

f087613 Add tests of listreceivedbyaddress address filtering (Jeremy Rubin) 8ee0812 Add address filtering to listreceivedbyaddress (Jeremy Rubin) Pull request description: Supersede bitcoin#9503 created by @JeremyRubin , I will maintain it. Tree-SHA512: 2accaed493b7e1c2eb5cb5270180f100f8c718b6585b9574f294191c318dc622a79e42ac185300f291f82d3b2a6f1c00850b6b17e4ff2dbab94d71df695acbfe

@JeremyRubin

f087613 Add tests of listreceivedbyaddress address filtering (Jeremy Rubin) 8ee0812 Add address filtering to listreceivedbyaddress (Jeremy Rubin) Pull request description: Supersede bitcoin#9503 created by @JeremyRubin , I will maintain it. Tree-SHA512: 2accaed493b7e1c2eb5cb5270180f100f8c718b6585b9574f294191c318dc622a79e42ac185300f291f82d3b2a6f1c00850b6b17e4ff2dbab94d71df695acbfe

@JeremyRubin

f087613 Add tests of listreceivedbyaddress address filtering (Jeremy Rubin) 8ee0812 Add address filtering to listreceivedbyaddress (Jeremy Rubin) Pull request description: Supersede bitcoin#9503 created by @JeremyRubin , I will maintain it. Tree-SHA512: 2accaed493b7e1c2eb5cb5270180f100f8c718b6585b9574f294191c318dc622a79e42ac185300f291f82d3b2a6f1c00850b6b17e4ff2dbab94d71df695acbfe

@JeremyRubin

f087613 Add tests of listreceivedbyaddress address filtering (Jeremy Rubin) 8ee0812 Add address filtering to listreceivedbyaddress (Jeremy Rubin) Pull request description: Supersede bitcoin#9503 created by @JeremyRubin , I will maintain it. Tree-SHA512: 2accaed493b7e1c2eb5cb5270180f100f8c718b6585b9574f294191c318dc622a79e42ac185300f291f82d3b2a6f1c00850b6b17e4ff2dbab94d71df695acbfe

jnewbery reviewed Jan 10, 2017

View reviewed changes

fanquake added the Wallet label Jan 10, 2017

JeremyRubin force-pushed the listreceivedbyaddress-filtered branch 3 times, most recently from 0ab4c34 to c0792a1 Compare January 13, 2017 15:49

JeremyRubin closed this Jan 13, 2017

JeremyRubin reopened this Jan 13, 2017

NicolasDorier reviewed Jan 13, 2017

View reviewed changes

luke-jr approved these changes Jan 20, 2017

View reviewed changes

JeremyRubin force-pushed the listreceivedbyaddress-filtered branch 2 times, most recently from b7d7953 to a96fbed Compare January 20, 2017 23:10

kallewoof reviewed Jan 30, 2017

View reviewed changes

luke-jr approved these changes Feb 2, 2017

View reviewed changes

NicolasDorier reviewed Feb 3, 2017

View reviewed changes

NicolasDorier added a commit to NTumbleBit/NTumbleBit that referenced this pull request Feb 6, 2017

Support NTumbleBit with Rubin bitcoin/bitcoin#9503 ListReceivedByAddress

cd7c2f4

TheBlueMatt reviewed Feb 7, 2017

View reviewed changes

JeremyRubin added 2 commits February 16, 2017 01:54

Add address filtering to listreceivedbyaddress

c60de5d

Add tests of listreceivedbyaddress address filtering

e6f053a

JeremyRubin force-pushed the listreceivedbyaddress-filtered branch from a96fbed to e6f053a Compare February 16, 2017 06:54

kallewoof reviewed Mar 1, 2017

View reviewed changes

NicolasDorier mentioned this pull request Mar 14, 2017

listreceivedbyaddress Filter Address #9991

Merged

maflcko closed this Mar 14, 2017

DanGould pushed a commit to DanGould/NTumbleBit that referenced this pull request Mar 29, 2017

Support NTumbleBit with Rubin bitcoin/bitcoin#9503 ListReceivedByAddress

6c7a475

bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021

listreceivedbyaddress Filter Address #9503

listreceivedbyaddress Filter Address #9503

Conversation

JeremyRubin commented Jan 10, 2017

jnewbery commented Jan 10, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JeremyRubin commented Jan 13, 2017

instagibbs commented Jan 13, 2017

JeremyRubin commented Jan 13, 2017

NicolasDorier commented Jan 13, 2017 • edited

NicolasDorier Jan 13, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

EthanHeilman commented Jan 13, 2017 • edited

jnewbery commented Jan 13, 2017

luke-jr left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JeremyRubin commented Jan 20, 2017

Choose a reason for hiding this comment

JeremyRubin Jan 31, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

luke-jr left a comment

Choose a reason for hiding this comment

NicolasDorier Feb 3, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NicolasDorier commented Feb 6, 2017 • edited

TheBlueMatt left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JeremyRubin commented Feb 16, 2017

NicolasDorier commented Feb 16, 2017

kallewoof left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kallewoof Mar 1, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kallewoof Mar 1, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NicolasDorier commented Mar 13, 2017

NicolasDorier commented Mar 14, 2017

JeremyRubin commented Mar 14, 2017

maflcko commented Mar 14, 2017

NicolasDorier commented Jan 13, 2017 •

edited

NicolasDorier Jan 13, 2017 •

edited

EthanHeilman commented Jan 13, 2017 •

edited

JeremyRubin Jan 31, 2017 •

edited

NicolasDorier Feb 3, 2017 •

edited

NicolasDorier commented Feb 6, 2017 •

edited

kallewoof Mar 1, 2017 •

edited

kallewoof Mar 1, 2017 •

edited