New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RPC: Add parameter to addmultisigaddress / createmultisig to sort public keys #8751

Open
wants to merge 5 commits into
base: master
from

Conversation

Projects
None yet
9 participants
@afk11
Contributor

afk11 commented Sep 17, 2016

I figured it may be useful for these RPC methods to allow sorting public keys (BIP67) The PR adds a new boolean to createmultisig / addmultisigaddress at the end of their parameter list. By default, this is set to false to avoid a BC break.

I added a RPC test file sort_multisig.py for testing createmultisig. Tests for addmultisigaddress went in wallet-accounts.py.

Note: Code to check whether sorting is desired had to be replicated in both RPC methods (not in _createmultisig_redeemScript) because addmultisigaddress already takes a parameter at position 3.

@dcousens

This comment has been minimized.

Show comment
Hide comment
@dcousens

dcousens Sep 18, 2016

Contributor

concept ACK

Contributor

dcousens commented Sep 18, 2016

concept ACK

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Sep 22, 2016

Member

Concept ACK, although I really don't like multiple-optional-positional-boolean APIs. Wish we switched to named arguments any day.

One nit: the RPC help should mention BIP67 by name.

Member

laanwj commented Sep 22, 2016

Concept ACK, although I really don't like multiple-optional-positional-boolean APIs. Wish we switched to named arguments any day.

One nit: the RPC help should mention BIP67 by name.

@MarcoFalke

This comment has been minimized.

Show comment
Hide comment
@MarcoFalke

MarcoFalke Nov 8, 2016

Member

Needs rebase

Member

MarcoFalke commented Nov 8, 2016

Needs rebase

@afk11

This comment has been minimized.

Show comment
Hide comment
@afk11

afk11 Nov 8, 2016

Contributor

@MarcoFalke thanks, done. @laanwj I should have mentioned, nits addressed.

One travis run failed due to the compactblocks RPC test.

Contributor

afk11 commented Nov 8, 2016

@MarcoFalke thanks, done. @laanwj I should have mentioned, nits addressed.

One travis run failed due to the compactblocks RPC test.

@ryanofsky

This comment has been minimized.

Show comment
Hide comment
@ryanofsky

ryanofsky Nov 8, 2016

Contributor

I can't see anything on travis right now (503 errors), but the compactblocks error is probably just the spurious #8842 / #9058 failures.

Contributor

ryanofsky commented Nov 8, 2016

I can't see anything on travis right now (503 errors), but the compactblocks error is probably just the spurious #8842 / #9058 failures.

Show outdated Hide outdated qa/rpc-tests/sort_multisig.py
class SortMultisigTest(BitcoinTestFramework):
def __init__(self):
super().__init__()
self.num_nodes = 4

This comment has been minimized.

@MarcoFalke

MarcoFalke Nov 8, 2016

Member

Nit: A single node should be enough?

@MarcoFalke

MarcoFalke Nov 8, 2016

Member

Nit: A single node should be enough?

@MarcoFalke

This comment has been minimized.

Show comment
Hide comment
@MarcoFalke

MarcoFalke Nov 8, 2016

Member

Concept ACK 7d7a647

Member

MarcoFalke commented Nov 8, 2016

Concept ACK 7d7a647

Show outdated Hide outdated src/rpc/misc.cpp
@@ -293,6 +294,7 @@ UniValue createmultisig(const JSONRPCRequest& request)
" \"key\" (string) bitcoin address or hex-encoded public key\n"
" ,...\n"
" ]\n"
"3. \"fSort\" (bool, optional) Whether to sort public keys according to BIP67. Default setting is false.\n"

This comment has been minimized.

@luke-jr

luke-jr Nov 24, 2016

Member

Is it a string or a boolean?

@luke-jr

luke-jr Nov 24, 2016

Member

Is it a string or a boolean?

This comment has been minimized.

@afk11

afk11 Nov 24, 2016

Contributor

It should be a boolean. Just observed they aren't usually quoted in RPC output, fixing now.

@afk11

afk11 Nov 24, 2016

Contributor

It should be a boolean. Just observed they aren't usually quoted in RPC output, fixing now.

@afk11

This comment has been minimized.

Show comment
Hide comment
@afk11

afk11 Nov 24, 2016

Contributor

I probably shouldn't have squashed @MarcoFalke, I'm sorry for rebasing out the commit you reviewed. The only thing to change this time was the removal of "'s from the RPC help message.

Contributor

afk11 commented Nov 24, 2016

I probably shouldn't have squashed @MarcoFalke, I'm sorry for rebasing out the commit you reviewed. The only thing to change this time was the removal of "'s from the RPC help message.

@luke-jr

Code looks reasonably correct, just a few nits. Did not verify tests.

Show outdated Hide outdated src/rpc/misc.cpp
"2. \"keys\" (string, required) A json array of keys which are bitcoin addresses or hex-encoded public keys\n"
" [\n"
" \"key\" (string) bitcoin address or hex-encoded public key\n"
" ,...\n"
" ]\n"
"3. fSort (bool, optional) Whether to sort public keys according to BIP67. Default setting is false.\n"

This comment has been minimized.

@luke-jr

luke-jr Nov 25, 2016

Member

Rather have a named-options Object interface here.

@luke-jr

luke-jr Nov 25, 2016

Member

Rather have a named-options Object interface here.

Show outdated Hide outdated src/script/standard.cpp
vEncoded.resize(keys.size());
BOOST_FOREACH(const CPubKey& key, keys) {
vEncoded[nEncoded++] = ToByteVector(key);
}

This comment has been minimized.

@luke-jr

luke-jr Nov 25, 2016

Member

Seems like the loop would be better as:

for (size_t n = 0; n < keys.size(); ++n) {
    vEncoded[n] = ToByteVector(keys[n]);
}
@luke-jr

luke-jr Nov 25, 2016

Member

Seems like the loop would be better as:

for (size_t n = 0; n < keys.size(); ++n) {
    vEncoded[n] = ToByteVector(keys[n]);
}
Show outdated Hide outdated src/script/standard.cpp
CScript script;
int nEncoded = 0;
std::vector<std::vector<unsigned char>> vEncoded;
vEncoded.resize(keys.size());

This comment has been minimized.

@luke-jr

luke-jr Nov 25, 2016

Member

Wouldn't it be better to reserve and then emplace_back?

@luke-jr

luke-jr Nov 25, 2016

Member

Wouldn't it be better to reserve and then emplace_back?

Show outdated Hide outdated src/script/standard.cpp
}
if (fSorted) {
std::sort(vEncoded.begin(), vEncoded.end());

This comment has been minimized.

@luke-jr

luke-jr Nov 25, 2016

Member

I think this should do what BIP 67 requires, but someone more familiar with C++ and its locale support (or lack thereof) should probably confirm.

@luke-jr

luke-jr Nov 25, 2016

Member

I think this should do what BIP 67 requires, but someone more familiar with C++ and its locale support (or lack thereof) should probably confirm.

Show outdated Hide outdated src/script/standard.h
@@ -78,7 +78,7 @@ bool ExtractDestinations(const CScript& scriptPubKey, txnouttype& typeRet, std::
CScript GetScriptForDestination(const CTxDestination& dest);
CScript GetScriptForRawPubKey(const CPubKey& pubkey);
CScript GetScriptForMultisig(int nRequired, const std::vector<CPubKey>& keys);
CScript GetScriptForMultisig(int nRequired, const std::vector<CPubKey>& keys, bool fSorted);

This comment has been minimized.

@luke-jr

luke-jr Nov 25, 2016

Member

Maybe default fSorted to false here rather than modify all the tests?

@luke-jr

luke-jr Nov 25, 2016

Member

Maybe default fSorted to false here rather than modify all the tests?

Show outdated Hide outdated src/wallet/rpcwallet.cpp
"2. \"keysobject\" (string, required) A json array of bitcoin addresses or hex-encoded public keys\n"
" [\n"
" \"address\" (string) bitcoin address or hex-encoded public key\n"
" ...,\n"
" ]\n"
"3. \"account\" (string, optional) DEPRECATED. An account to assign the addresses to.\n"
"4. fSort (bool, optional) Whether to sort public keys according to BIP67. Default setting is false.\n"

This comment has been minimized.

@luke-jr

luke-jr Nov 25, 2016

Member

As before, rather turn param 3 into an options Object.

@luke-jr

luke-jr Nov 25, 2016

Member

As before, rather turn param 3 into an options Object.

luke-jr added a commit to bitcoinknots/bitcoin that referenced this pull request Dec 21, 2016

RPC: addmultisigaddress / createmultisig: parameterize _createmultisi…
…g_redeemScript to allow sorting of public keys (BIP67)

addmultisig/createmultisig RPC documentation: Remove stray quotes from fSort parameter

Github-Pull: #8751
Rebased-From: 7439562
@luke-jr

This comment has been minimized.

Show comment
Hide comment
@luke-jr

luke-jr Dec 21, 2016

Member

Should update doc/bips.md also.

Member

luke-jr commented Dec 21, 2016

Should update doc/bips.md also.

@luke-jr

This comment has been minimized.

Show comment
Hide comment
@luke-jr

luke-jr Feb 4, 2017

Member

I rebased and addressed all the nits; pushed this to luke-jr/sort-multisigs

@afk11 Are you still maintaining this? Can you pull my changes?

git checkout sort-multisigs
git fetch git://github.com/luke-jr/bitcoin sort-multisigs
git reset --hard FETCH_HEAD
git push ...
Member

luke-jr commented Feb 4, 2017

I rebased and addressed all the nits; pushed this to luke-jr/sort-multisigs

@afk11 Are you still maintaining this? Can you pull my changes?

git checkout sort-multisigs
git fetch git://github.com/luke-jr/bitcoin sort-multisigs
git reset --hard FETCH_HEAD
git push ...
@afk11

This comment has been minimized.

Show comment
Hide comment
@afk11

afk11 Feb 6, 2017

Contributor

Sorry, yep I can pull these!

I wanted to wait until named parameters was merged before hand, so I could avoid adding a positional parameter before the accounts parameters were changed

I'll look at this in the next day or so (away from internet atm) wanna finish this up

Contributor

afk11 commented Feb 6, 2017

Sorry, yep I can pull these!

I wanted to wait until named parameters was merged before hand, so I could avoid adding a positional parameter before the accounts parameters were changed

I'll look at this in the next day or so (away from internet atm) wanna finish this up

@afk11

This comment has been minimized.

Show comment
Hide comment
@afk11

afk11 Mar 8, 2017

Contributor

Merged commits and rebased. Apologies for the delay!

Contributor

afk11 commented Mar 8, 2017

Merged commits and rebased. Apologies for the delay!

@afk11

This comment has been minimized.

Show comment
Hide comment
@afk11

afk11 Mar 8, 2017

Contributor

The Apple build failed because the job time exceeded the maximum :/

Contributor

afk11 commented Mar 8, 2017

The Apple build failed because the job time exceeded the maximum :/

@afk11

This comment has been minimized.

Show comment
Hide comment
@afk11

afk11 Mar 14, 2017

Contributor

Rebased

Contributor

afk11 commented Mar 14, 2017

Rebased

@afk11

This comment has been minimized.

Show comment
Hide comment
@afk11

afk11 May 9, 2017

Contributor

Rebased

Contributor

afk11 commented May 9, 2017

Rebased

@luke-jr

This comment has been minimized.

Show comment
Hide comment
@luke-jr

luke-jr Jun 6, 2017

Member

The description for d70583a is no longer correct.

Member

luke-jr commented Jun 6, 2017

The description for d70583a is no longer correct.

@afk11

This comment has been minimized.

Show comment
Hide comment
@afk11

afk11 Jun 6, 2017

Contributor

@luke-jr Oops, yeah the rebase mightn't have been the easiest. So re-add the contents of setup_network?

Contributor

afk11 commented Jun 6, 2017

@luke-jr Oops, yeah the rebase mightn't have been the easiest. So re-add the contents of setup_network?

luke-jr added a commit to bitcoinknots/bitcoin that referenced this pull request Jun 15, 2017

luke-jr added a commit to bitcoinknots/bitcoin that referenced this pull request Jun 15, 2017

@jnewbery

This comment has been minimized.

Show comment
Hide comment
@jnewbery

jnewbery Jun 15, 2017

Member

Perhaps I'm missing something, but I don't see the need for this. The addmultisigaddress RPC creates the multisig script with the keys in the order provided. Why not just have the client provide keys in sorted order if you want the script to be BIP-67 compliant?

It doesn't look like this PR is enforcing that the provided keys are compressed, so even with this PR, there are still expectations placed on the client.

Member

jnewbery commented Jun 15, 2017

Perhaps I'm missing something, but I don't see the need for this. The addmultisigaddress RPC creates the multisig script with the keys in the order provided. Why not just have the client provide keys in sorted order if you want the script to be BIP-67 compliant?

It doesn't look like this PR is enforcing that the provided keys are compressed, so even with this PR, there are still expectations placed on the client.

@afk11

This comment has been minimized.

Show comment
Hide comment
@afk11

afk11 Jun 16, 2017

Contributor

I think if developers are already committing to using the RPC to make a multisig script, making it easier to produce the same representation is more important than not.

You are correct the PR as it stands doesn't validate it.. fixing this now.

Contributor

afk11 commented Jun 16, 2017

I think if developers are already committing to using the RPC to make a multisig script, making it easier to produce the same representation is more important than not.

You are correct the PR as it stands doesn't validate it.. fixing this now.

@jnewbery

This comment has been minimized.

Show comment
Hide comment
@jnewbery

jnewbery Jun 16, 2017

Member

I'm still a weak concept NACK for this. I don't agree that we should add complexity to the server when the same outcome can be achieved by simply running a sort() function on the client before calling the RPC. Sometimes there's good reason to add that complexity to the server - see for example #9991 which adds a filter to save significantly on bandwidth and server resources. In this case I don't see the benefit.

Sorry if that sounds negative - I think there needs to be some bar for adding new RPCs and arguments to avoid bloat.

However, if I'm wrong and there's widespread consensus that this is useful functionality and should be merged, can I at least ask that you use named arguments instead of an Options object? There's really no need for Options objects in RPC calls since named args were added in #8811.

Member

jnewbery commented Jun 16, 2017

I'm still a weak concept NACK for this. I don't agree that we should add complexity to the server when the same outcome can be achieved by simply running a sort() function on the client before calling the RPC. Sometimes there's good reason to add that complexity to the server - see for example #9991 which adds a filter to save significantly on bandwidth and server resources. In this case I don't see the benefit.

Sorry if that sounds negative - I think there needs to be some bar for adding new RPCs and arguments to avoid bloat.

However, if I'm wrong and there's widespread consensus that this is useful functionality and should be merged, can I at least ask that you use named arguments instead of an Options object? There's really no need for Options objects in RPC calls since named args were added in #8811.

@afk11

This comment has been minimized.

Show comment
Hide comment
@afk11

afk11 Jun 19, 2017

Contributor

Both RPC methods take an options object for this (sorry, the PR description wasn't updated with this)
https://github.com/bitcoin/bitcoin/pull/8751/files#diff-ad6efdc354b57bd1fa29fc3abb6e2872R353
https://github.com/bitcoin/bitcoin/pull/8751/files#diff-df7d84ff2f53fcb2a0dc15a3a51e55ceR1050

I appreciate where you are coming from and agree that most people can probably sort themselves, but they could also build a multisig script out of the keys and m/n. It's been a while since I've even used the RPC, but remember well the time when I didn't have a bitcoin library to do it all.

I think it's worth including since once they continue using the flag, requests which mistakenly use the wrong order will reproduce the same redeem script (instead of always having a stateful order of public keys), and likewise with libraries that support it.

Contributor

afk11 commented Jun 19, 2017

Both RPC methods take an options object for this (sorry, the PR description wasn't updated with this)
https://github.com/bitcoin/bitcoin/pull/8751/files#diff-ad6efdc354b57bd1fa29fc3abb6e2872R353
https://github.com/bitcoin/bitcoin/pull/8751/files#diff-df7d84ff2f53fcb2a0dc15a3a51e55ceR1050

I appreciate where you are coming from and agree that most people can probably sort themselves, but they could also build a multisig script out of the keys and m/n. It's been a while since I've even used the RPC, but remember well the time when I didn't have a bitcoin library to do it all.

I think it's worth including since once they continue using the flag, requests which mistakenly use the wrong order will reproduce the same redeem script (instead of always having a stateful order of public keys), and likewise with libraries that support it.

Show outdated Hide outdated src/script/standard.cpp
CScript script;
std::vector<std::vector<unsigned char>> vEncoded;
vEncoded.reserve(keys.size());
BOOST_FOREACH(const CPubKey& key, keys) {

This comment has been minimized.

@luke-jr

luke-jr Aug 18, 2017

Member

We're not using BOOST_FOREACH anymore I think. (Also below)

@luke-jr

luke-jr Aug 18, 2017

Member

We're not using BOOST_FOREACH anymore I think. (Also below)

@luke-jr

This comment has been minimized.

Show comment
Hide comment
@luke-jr

luke-jr Aug 21, 2017

Member

Rebased and squashed a bit.

git checkout sort-multisigs && git fetch git://github.com/luke-jr/bitcoin sort-multisigs && git reset --hard FETCH_HEAD && git push ...
Member

luke-jr commented Aug 21, 2017

Rebased and squashed a bit.

git checkout sort-multisigs && git fetch git://github.com/luke-jr/bitcoin sort-multisigs && git reset --hard FETCH_HEAD && git push ...
@TheBlueMatt

This comment has been minimized.

Show comment
Hide comment
@TheBlueMatt

TheBlueMatt Sep 29, 2017

Contributor

Concept ACK. Care to rebase?

Contributor

TheBlueMatt commented Sep 29, 2017

Concept ACK. Care to rebase?

@afk11

This comment has been minimized.

Show comment
Hide comment
@afk11

afk11 Sep 30, 2017

Contributor

Rebased, sorry for the delay. Updated to check that keys are compressed before allowing sorting, and added more tests for this.

Updated the docs/bips.md document to mention 0.15.1 instead of 0.15.0 (let me know whatever's best for this)

Contributor

afk11 commented Sep 30, 2017

Rebased, sorry for the delay. Updated to check that keys are compressed before allowing sorting, and added more tests for this.

Updated the docs/bips.md document to mention 0.15.1 instead of 0.15.0 (let me know whatever's best for this)

@TheBlueMatt

This comment has been minimized.

Show comment
Hide comment
@TheBlueMatt

TheBlueMatt Oct 2, 2017

Contributor

Hmm, hate to reopen it, but now that we do actually have named arguments, could you rever to just adding a new boolean argument? options objects are just redundant now, and having options alias account in addmultisigaddress is just gross. Everything else seems fine at first glance.

@jnewbery I'd generally agree with you, but, at least in principal, I think BIP67 is worth it.

Contributor

TheBlueMatt commented Oct 2, 2017

Hmm, hate to reopen it, but now that we do actually have named arguments, could you rever to just adding a new boolean argument? options objects are just redundant now, and having options alias account in addmultisigaddress is just gross. Everything else seems fine at first glance.

@jnewbery I'd generally agree with you, but, at least in principal, I think BIP67 is worth it.

luke-jr added a commit to bitcoinknots/bitcoin that referenced this pull request Nov 6, 2017

RPC: Use options object rather than adding a "sort" boolean for multi…
…sig methods

Also add accounts parameter to vRPCConvertParams (required by RPC mappings test)

Github-Pull: #8751
Rebased-From: 4833935

luke-jr added a commit to bitcoinknots/bitcoin that referenced this pull request Nov 6, 2017

luke-jr added a commit to bitcoinknots/bitcoin that referenced this pull request Nov 6, 2017

Add more tests to sort_multisigs.py / wallet-accounts.py
sort_multisig test: check uncompressed keys are disallowed
sort_multisig: add test demonstrating sorting
wallet-accounts: test addmultisigaddress fails if sort=true and (wallet) address is uncompressed

Github-Pull: #8751
Rebased-From: 50e2ff5

afk11 and others added some commits Nov 8, 2016

RPC: addmultisigaddress / createmultisig: parameterize _createmultisi…
…g_redeemScript to allow sorting of public keys (BIP67)

addmultisig/createmultisig RPC documentation: Remove stray quotes from fSort parameter
Add more tests to sort_multisigs.py / wallet-accounts.py
sort_multisig test: check uncompressed keys are disallowed
sort_multisig: add test demonstrating sorting
wallet-accounts: test addmultisigaddress fails if sort=true and (wallet) address is uncompressed
@afk11

This comment has been minimized.

Show comment
Hide comment
@afk11

afk11 Dec 2, 2017

Contributor

@TheBlueMatt that's fine, revised the PR now.

I missed the boat again for v0.15.1, suggestions for a release to mention in bips.md?

Contributor

afk11 commented Dec 2, 2017

@TheBlueMatt that's fine, revised the PR now.

I missed the boat again for v0.15.1, suggestions for a release to mention in bips.md?

@luke-jr

This comment has been minimized.

Show comment
Hide comment
@luke-jr

luke-jr Mar 1, 2018

Member

Hmm, hate to reopen it, but now that we do actually have named arguments, could you rever to just adding a new boolean argument? options objects are just redundant now, and having options alias account in addmultisigaddress is just gross. Everything else seems fine at first glance.

Strongly disagree. Named arguments is not a reason to have a terrible positional arguments API. Uncommon options should go through an options argument when positional arguments are used.

The account alias is merely for backward compatibility.

Member

luke-jr commented Mar 1, 2018

Hmm, hate to reopen it, but now that we do actually have named arguments, could you rever to just adding a new boolean argument? options objects are just redundant now, and having options alias account in addmultisigaddress is just gross. Everything else seems fine at first glance.

Strongly disagree. Named arguments is not a reason to have a terrible positional arguments API. Uncommon options should go through an options argument when positional arguments are used.

The account alias is merely for backward compatibility.

{
std::string msg = "addmultisigaddress nrequired [\"key\",...] ( \"account\" )\n"
std::string msg = "addmultisigaddress nrequired [\"key\",...] ( \"account\" ) ( sort )\n"

This comment has been minimized.

@luke-jr

luke-jr Mar 2, 2018

Member

Please switch back to an options object for this.

@luke-jr

luke-jr Mar 2, 2018

Member

Please switch back to an options object for this.

"\nAdd a nrequired-to-sign multisignature address to the wallet. Requires a new wallet backup.\n"

This comment has been minimized.

@luke-jr

luke-jr Mar 2, 2018

Member

This blank line won't be in the actual help, so has no purpose here.

@luke-jr

luke-jr Mar 2, 2018

Member

This blank line won't be in the actual help, so has no purpose here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment