Adds deriveaddresses rpc #1162

Vasu-08 · 2023-07-09T17:40:11Z

This pr implements deriveaddresses rpc.

codecov · 2023-07-09T17:47:28Z

Codecov Report

Patch coverage: 93.16% and project coverage change: +0.13% 🎉

Comparison is base (014a104) 70.29% compared to head (82f8c8d) 70.42%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1162      +/-   ##
==========================================
+ Coverage   70.29%   70.42%   +0.13%     
==========================================
  Files         174      174              
  Lines       27367    27515     +148     
==========================================
+ Hits        19237    19377     +140     
- Misses       8130     8138       +8

Files Changed	Coverage Δ
lib/node/rpc.js	`43.49% <83.78%> (+1.05%)`	⬆️
lib/descriptor/keyprovider.js	`93.83% <86.66%> (-0.19%)`	⬇️
lib/descriptor/abstractdescriptor.js	`92.68% <94.73%> (+0.61%)`	⬆️
lib/descriptor/type/addr.js	`91.42% <100.00%> (+0.51%)`	⬆️
lib/descriptor/type/combo.js	`93.02% <100.00%> (+3.36%)`	⬆️
lib/descriptor/type/multisig.js	`94.93% <100.00%> (+0.73%)`	⬆️
lib/descriptor/type/pk.js	`93.54% <100.00%> (+1.24%)`	⬆️
lib/descriptor/type/pkh.js	`94.28% <100.00%> (+1.42%)`	⬆️
lib/descriptor/type/raw.js	`94.28% <100.00%> (+0.16%)`	⬆️
lib/descriptor/type/sh.js	`93.10% <100.00%> (+0.79%)`	⬆️
... and 4 more

... and 2 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

lib/descriptor/keyprovider.js

lib/descriptor/abstractdescriptor.js

theanmolsharma · 2023-07-18T11:38:48Z

lib/descriptor/abstractdescriptor.js

+
+    for (const script of scripts) {
+      const address = script.getAddress();
+      assert(address, 'Descriptor does not have a corresponding address');


in what case would a descriptor not have an address?

In few cases of raw descriptor and for multisig descriptor.

Is this because raw() and multi() get interpreted as raw output script at the top level? but if they are wrapped by sh() then they could have addresses

lib/descriptor/abstractdescriptor.js

lib/descriptor/keyprovider.js

lib/descriptor/type/pk.js

lib/descriptor/type/pkh.js

lib/descriptor/type/sh.js

lib/descriptor/type/wpkh.js

lib/descriptor/type/wsh.js

theanmolsharma

Address comments and mark the PR as ready for review. Great work so far!!

lib/script/script.js

lib/descriptor/abstractdescriptor.js

lib/descriptor/type/addr.js

lib/node/rpc.js

theanmolsharma · 2023-07-21T14:11:32Z

lib/node/rpc.js

+    if (!Number.isInteger(low)) {
+      throw new RPCError(
+        errs.INVALID_PARAMETER, 'Range begin must be an integer'
+      );
+    }
+
+    if (!Number.isInteger(high)) {
+      throw new RPCError(
+        errs.INVALID_PARAMETER, 'Range end must be an integer'
+      );
+    }


I believe this check is redundant. Validator class will ensure it is a u32int

This check is to ensure the entries in range array are valid. I think its required there.

lib/node/rpc.js

lib/script/script.js

pinheadmz

excellent work! did a first pass review, this is definitely on track

pinheadmz · 2023-07-28T13:31:28Z

lib/descriptor/abstractdescriptor.js

+
+    for (const provider of this.keyProviders) {
+      const pubkey = provider.getPublicKey(pos);
+      assert(pubkey, 'Cannot derive script without private keys');


not even from an xpub ?

pinheadmz · 2023-07-28T13:39:36Z

lib/descriptor/abstractdescriptor.js

+
+    for (const script of scripts) {
+      const address = script.getAddress();
+      assert(address, 'Descriptor does not have a corresponding address');


Is this because raw() and multi() get interpreted as raw output script at the top level? but if they are wrapped by sh() then they could have addresses

pinheadmz · 2023-07-28T13:45:38Z

lib/descriptor/type/multisig.js

@@ -4,6 +4,8 @@
 * https://github.com/bcoin-org/bcoin
 */

+// script.fromMultisig(m, n, pubkeys, )


pinheadmz · 2023-07-28T13:49:51Z

lib/node/rpc.js

@@ -2346,10 +2347,95 @@ class RPC extends RPCBase {
    return result;
  }

+  async deriveAddresses(args, help) {
+    if (help || args.length > 2 || args.length === 0)
+      throw new RPCError(errs.MISC_ERROR, 'deriveaddresses "descriptor"');


Is range an optional param? I think it should be,

Suggested change

throw new RPCError(errs.MISC_ERROR, 'deriveaddresses "descriptor"');

throw new RPCError(errs.MISC_ERROR, 'deriveaddresses "descriptor"' (range));

pinheadmz · 2023-07-28T13:51:34Z

lib/node/rpc.js

+    const desc = parseDescriptor(valid.str(0, ''), this.network, true);
+
+    if (desc.isRange() && args.length === 1) {


once you call new Validator() I don't think you need to use args any more. Take a look at other RPCs with optional params. I think you still extract the array argument with valid.array(1) and then can check if its null on L2358

pinheadmz · 2023-07-28T13:59:41Z

lib/script/script.js

   */

-  fromMultisig(m, n, keys) {
+  fromMultisig(m, n, keys, isSorted = true, isWitness = false) {


fromMultisig() is called from several places i.e. in account.js and node/rpc.js. Let's make sure that the new options are set correctly from all contexts and cover with a test (i.e. fail to create a 16 multisig with legacy, succeed with witness, then fail to create a 21 multisig with witness... from the existing wallet code without descriptors)

theanmolsharma · 2023-07-30T17:14:13Z

test/script-test.js

+  for (const isSorted of [true, false]) {
+    for (const isWitness of [true, false]) {
+      for (let n = 1; n <= 20; n++) {
+        for (let m = 1; m <= n; m++) {
+          it(`should handle script generation for ${n} keys. (${m} of ${n}) (isSorted = ${isSorted}) (isWitness = ${isWitness})`, () => {
+            const keys = [];
+
+            for (let i = 0; i < n; i++) {
+              keys.push(HDPrivateKey.generate().publicKey);
+            }
+
+            let script;
+            let error;
+
+            try {
+              script = Script.fromMultisig(m, n, keys, isSorted, isWitness).toRaw().toString('hex');
+            } catch (e) {
+              error = e;
+            }
+
+            if (n > 15 && !isWitness) {
+              assert(error);
+              assert(error.message === `${n} keys not allowed inside script. Max allowed: ${isWitness ? 20 : 15}`);
+            } else {
+              if (isSorted) {
+                keys.sort((a, b) => a.compare(b));
+              }
+
+              const expectedScript = Opcode.fromInt(m).toRaw().toString('hex') // threshold
+                + keys.map(key => '21' + key.toString('hex')).join('') // keys
+                + Opcode.fromInt(n).toRaw().toString('hex') // number of keys
+                + 'ae'; // OP_CHECKMULTISIG
+
+              assert(script === expectedScript);
+            }
+          });
+        }
+      }
+    }
+  }


@pinheadmz
I wrote this test today. Wanted to know how you feel about these tests.
This is probably an overkill but it runs quickly so there is no harm in having it.

theanmolsharma

Code review ACK 21bc9c0
Nice work!!

theanmolsharma · 2023-07-31T04:21:50Z

test/script-test.js

+
+  for (const isSorted of [true, false]) {
+    for (const isWitness of [true, false]) {
+      for (let n = 1; n <= 22; n++) {


Suggested change

for (let n = 1; n <= 22; n++) {

for (let n = 1; n <= 21; n++) {

I think 21 would suffice

pinheadmz · 2023-08-03T22:27:34Z

test/wallet-test.js

@@ -1501,6 +1501,43 @@ describe('Wallet', function() {
    });
  }

+  for (const witness of [true, false]) {
+    it('should create witness multisig account for more than 15 keys', async () => {
+      const wallet = await wdb.create({


I expect this to throw right away for legacy. Kinda surprised we don't already check excessive n size in Account.fromOptions(). Do you think we should? Maybe right after this:

bcoin/lib/wallet/account.js

Lines 157 to 158 in 014a104

if (this.m < 1 || this.m > this.n)

throw new Error('m ranges between 1 and n');

Oh yes I think we can add a check here as well

…sSorted and isWitness flag

pinheadmz

ACK 82f8c8d

Show Signature

pinheadmz's public key is on keybase

theanmolsharma

Approved!!
Address comments before merge!!
Great work!!

theanmolsharma · 2023-08-05T13:33:32Z

lib/script/script.js

+    const script = this.compile();
+
+    if (!isWitness) {
+      assert(script.getSize() <= MAX_SCRIPT_PUSH, 'Script size is too large');


I think the check should be this because it will eventually get wrapped in sh

Suggested change

assert(script.getSize() <= MAX_SCRIPT_PUSH, 'Script size is too large');

assert(script.getSize() + 3 <= MAX_SCRIPT_PUSH,, 'Script size is too large');

nvm, my bad

Vasu-08 force-pushed the descriptor-3 branch from 3794e06 to aa1539e Compare July 9, 2023 17:44

Vasu-08 force-pushed the descriptor-3 branch 9 times, most recently from 3e26117 to 24da42c Compare July 16, 2023 13:33

Vasu-08 force-pushed the descriptor-3 branch 2 times, most recently from 57e8c2f to 6fe06c8 Compare July 17, 2023 20:55

theanmolsharma suggested changes Jul 18, 2023

View reviewed changes

Vasu-08 force-pushed the descriptor-3 branch from 6fe06c8 to e69cb9f Compare July 18, 2023 06:58

theanmolsharma suggested changes Jul 18, 2023

View reviewed changes

Vasu-08 force-pushed the descriptor-3 branch 2 times, most recently from 58cf024 to bbdb682 Compare July 19, 2023 07:51

theanmolsharma suggested changes Jul 21, 2023

View reviewed changes

Vasu-08 force-pushed the descriptor-3 branch 2 times, most recently from 9d24806 to 64f0741 Compare July 26, 2023 09:22

Vasu-08 marked this pull request as ready for review July 27, 2023 15:21

Vasu-08 requested a review from theanmolsharma July 27, 2023 15:22

pinheadmz reviewed Jul 28, 2023

View reviewed changes

feat<descriptor>: added key derivation functions in KeyProvider class

ca83bdd

Vasu-08 force-pushed the descriptor-3 branch 3 times, most recently from b79f190 to 21bc9c0 Compare July 30, 2023 17:57

Vasu-08 requested a review from pinheadmz July 30, 2023 22:02

theanmolsharma reviewed Jul 31, 2023

View reviewed changes

theanmolsharma approved these changes Jul 31, 2023

View reviewed changes

Vasu-08 force-pushed the descriptor-3 branch from 21bc9c0 to c495a7e Compare August 3, 2023 08:03

pinheadmz reviewed Aug 3, 2023

View reviewed changes

Vasu-08 force-pushed the descriptor-3 branch from c495a7e to 48b43c8 Compare August 4, 2023 15:02

Vasu-08 added 4 commits August 5, 2023 00:07

fix(script): allow multisig scripts generation upto 20 keys and add i…

f3184af

…sSorted and isWitness flag

feat<descriptor>: added functions for generating scripts and addresses

32e5940

test: added tests for deriving addresses from descriptor

f7dca8c

feat<node>: added deriveaddresses rpc

82f8c8d

Vasu-08 force-pushed the descriptor-3 branch from 48b43c8 to 82f8c8d Compare August 4, 2023 18:40

pinheadmz approved these changes Aug 5, 2023

View reviewed changes

theanmolsharma approved these changes Aug 5, 2023

View reviewed changes

pinheadmz merged commit 0c18028 into bcoin-org:master Aug 5, 2023
6 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds deriveaddresses rpc #1162

Adds deriveaddresses rpc #1162

Vasu-08 commented Jul 9, 2023 •

edited

Loading

codecov bot commented Jul 9, 2023 •

edited

Loading

theanmolsharma Jul 18, 2023

Vasu-08 Jul 18, 2023

theanmolsharma Jul 19, 2023

pinheadmz Jul 28, 2023

theanmolsharma left a comment

theanmolsharma Jul 21, 2023

Vasu-08 Jul 21, 2023

pinheadmz left a comment

pinheadmz Jul 28, 2023

pinheadmz Jul 28, 2023

pinheadmz Jul 28, 2023

pinheadmz Jul 28, 2023

pinheadmz Jul 28, 2023

pinheadmz Jul 28, 2023

theanmolsharma Jul 30, 2023

theanmolsharma left a comment

theanmolsharma Jul 31, 2023

pinheadmz Aug 3, 2023

Vasu-08 Aug 3, 2023

pinheadmz left a comment

theanmolsharma left a comment

theanmolsharma Aug 5, 2023

theanmolsharma Aug 5, 2023

	throw new RPCError(errs.MISC_ERROR, 'deriveaddresses "descriptor"');
	throw new RPCError(errs.MISC_ERROR, 'deriveaddresses "descriptor"' (range));

		const desc = parseDescriptor(valid.str(0, ''), this.network, true);

		if (desc.isRange() && args.length === 1) {

	for (let n = 1; n <= 22; n++) {
	for (let n = 1; n <= 21; n++) {

	if (this.m < 1 \|\| this.m > this.n)
	throw new Error('m ranges between 1 and n');

	assert(script.getSize() <= MAX_SCRIPT_PUSH, 'Script size is too large');
	assert(script.getSize() + 3 <= MAX_SCRIPT_PUSH,, 'Script size is too large');

Adds deriveaddresses rpc #1162

Adds deriveaddresses rpc #1162

Conversation

Vasu-08 commented Jul 9, 2023 • edited Loading

codecov bot commented Jul 9, 2023 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

theanmolsharma left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pinheadmz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

theanmolsharma left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pinheadmz left a comment

Choose a reason for hiding this comment

theanmolsharma left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Vasu-08 commented Jul 9, 2023 •

edited

Loading

codecov bot commented Jul 9, 2023 •

edited

Loading