test: Run rpc_generateblock.py even with wallet disabled #20808

nginocchio · 2020-12-31T00:11:59Z

This PR enables one more of the non-wallet functional tests (rpc_generateblock.py) to be run even with the Bitcoin Core wallet disabled by using the new MiniWallet instead, as proposed in #20078.

stackman27

ACK 4e8803eCode review and built it with and without wallet

michaelfolkson · 2021-01-03T20:39:28Z

Not a huge deal @stackman27 but we ACK the commit hash of the HEAD commit. In this case it is 4e8803e

When giving an ACK, specify the commits reviewed by appending the commit hash of the HEAD commit.

For more details on the review process see this @jonatack doc: https://jonatack.github.io/articles/how-to-review-pull-requests-in-bitcoin-core

stackman27 · 2021-01-03T22:06:07Z

Not a huge deal @stackman27 but we ACK the commit hash of the HEAD commit. In this case it is 4e8803e

When giving an ACK, specify the commits reviewed by appending the commit hash of the HEAD commit.

For more details on the review process see this @jonatack doc: https://jonatack.github.io/articles/how-to-review-pull-requests-in-bitcoin-core

oooh i see. Thank you!

michaelfolkson

ACK 4e8803e

Ran test with wallet disabled and reviewed code.

test/functional/rpc_generateblock.py

mjdietzx

Concept ACK

taurus228

.

test/functional/rpc_generateblock.py

maflcko

Concept ACK.
Make sure to squash your commits according to https://github.com/bitcoin/bitcoin/blob/master/CONTRIBUTING.md#squashing-commits after addressing the feedback

glozow

Concept ACK, this is a good start. You are (I think unintentionally) changing the test behavior.

test/functional/rpc_generateblock.py

mjdietzx · 2021-01-10T18:50:26Z

test/functional/rpc_generateblock.py

+        tx = block['tx'][0]
+        utxo = {'txid': tx['txid'], 'vout': 0, 'value': tx['vout'][0]['value']}
+        utxos.append(utxo)


You should be able to remove this if you do hash = miniwallet.generate(1)[0]["hash"] instead

If you still want the utxo in your list, you can then just do utxos.append(miniwallet.get_utxo())

mjdietzx · 2021-01-10T19:03:58Z

test/functional/rpc_generateblock.py

+        address = ADDRESS_BCRT1_P2WSH_OP_TRUE
+        scriptPubKey = hex_str_to_bytes(node.validateaddress(ADDRESS_BCRT1_P2WSH_OP_TRUE)['scriptPubKey'])


I know it's frowned upon, but thoughts on doing:

address = miniwallet._address scriptPubKey = miniwallet._scriptPubKey

Otherwise we're just duplicating that logic, and we can get rid of a bunch of imports. Maybe it's worth considering adding a getter for address and scriptPubKey to MiniWallet (although I don't see anything wrong just accessing the private vars)

test/functional/rpc_generateblock.py

DrahtBot · 2021-01-12T08:13:03Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

test: run mempool_reorg.py even with wallet disabled #21178 (test: run mempool_reorg.py even with wallet disabled by DariusParvin)
test: Run mempool_accept.py even with wallet disabled #21014 (test: Run mempool_accept.py even with wallet disabled by stackman27)
test: ensure any failing method in MiniWallet leaves no side-effects #20889 (test: ensure any failing method in MiniWallet leaves no side-effects by mjdietzx)
test: Run mempool_limit.py even with wallet disabled #20874 (test: Run mempool_limit.py even with wallet disabled by stackman27)
rpc: deprecate addresses and reqSigs from rpc outputs #20286 (rpc: deprecate addresses and reqSigs from rpc outputs by mjdietzx)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

mjdietzx

This is looking great! Two minor comments

mjdietzx · 2021-01-12T18:47:41Z

test/functional/rpc_generateblock.py

+        tx = block['tx'][0]
+        utxo = {'txid': tx['txid'], 'vout': 0, 'value': tx['vout'][0]['value']}
+        miniwallet._utxos.append(utxo)


Is this necessary? I don't see why we need this Coinbase in our utxos. I don't see us specifically spending it, and if you remove this it gets rid of the potential problem of appending to mini wallets private _utxos

10 blocks isn't enough for the test because the 10 generated by MiniWallet are sent to sit in the mempool not to be mined. So I can up the generated blocks to 11 and remove that code, or do this approach.

Is there any reason you need this specific utxo for this test? If you need 11, I'd say generate 11 blocks where you need them.

but the 10 generated by MiniWallet are definitely mined. ie if you synced another node, you'd see those blocks with the coinbases (utxos) you are spending. they aren't just in the mempool

if I'm missing something please lmk, definitely possible

I do not need that specific utxo for the test. I guess I should have elaborated more on my point. What I meant was that those 10 blocks are generated via generate and then are sent via send_self_transfer, however, those transactions are never confirmed. So while those new utxos are present within miniwallet generating a confirmed transaction based on them (because they are unconfirmed) isn't possible. Please correct me if I am wrong. Its probably just a better approach to generate 11.

Ah yeah, so the transactions you send with send_self_transfer are in the mempool. However, you could still spend them even though they are only in the mempool. And you could also generate a block, and all the transactions in the mempool would then be mined/confirmed (although I don't think this is necessary).

I'd recommend generating 11 blocks for miniwallet if you need to send 11 utxos. Otherwise the 11th will pick one of the previous send_self_transfer utxos to spend (ie one of the transactions in the mempool is the parent of that new "child" txn)

mjdietzx · 2021-01-12T18:53:41Z

test/functional/rpc_generateblock.py

    def run_test(self):
        node = self.nodes[0]
+        miniwallet = MiniWallet(node)
+        address = ADDRESS_BCRT1_P2WSH_OP_TRUE


nit: I'm not sure if this really matters, but would it be better to use miniwallet._address? I know they are hard-coded to the same value rn

@MarcoFalke what's you're opinion on this, would it be beneficial to add a getter to MiniWallet?

@property def address(self): return self._address

I think it is fine to just make _address public (remove the underscore)

stackman27

I added few comments on wallet.py to make sure we're both on the same track, since we're both modifying it for the same reason :)

test/functional/test_framework/wallet.py

maflcko · 2021-02-16T16:58:31Z

test/functional/rpc_generateblock.py

-        node.generatetoaddress(110, address)
+        # Generate some blocks to spend
+        miniwallet.generate(10)
+        node.generate(100)


Generating blocks takes a few seconds in valgrind, so I am thinking if this test may benefit from using miniwallet.scan_blocks (to be introduced in #21200)

DrahtBot · 2021-03-29T14:22:14Z

🐙 This pull request conflicts with the target branch and needs rebase.

_{Want to unsubscribe from rebase notifications on this pull request? Just convert this pull request to a "draft".}

maflcko · 2021-07-20T07:32:11Z

Are you still working on this?

jsarenik · 2021-11-18T19:28:47Z

Are you still working on this?

I know someone who is working on this and hopefully they will write here soon. (Just wanted to leave a trace in the PR by this message, sorry for spam.)

DariusParvin · 2021-11-19T17:33:01Z

Thanks @jsarenik :) I am working on this issue. I am mostly finished but will go through it a few more times (reviewing the comments here) before submitting a PR. Work in progress is here

1e86ff7 test: run rpc-generateblock.py even with wallet disabled (Darius Parvin) Pull request description: Run rpc_generateblock.py test even when the wallet is disabled, as discussed in #20078. This PR picked up from where #20808 left off, started by @ nginocchio. Since then, there have been many improvements to `MiniWallet`, making this PR more straightforward. L23 makes use of `MiniWallet.rescan_utxos()` to add the pre-mined block utxos (when `self.setup_clean_chain` is not set to `True`), rather than generating new blocks during the test. ACKs for top commit: mjdietzx: Tested ACK 1e86ff7 Tree-SHA512: 4285f61516dd53a08004eeea26d58f45b4c1c67f5ca4c94ff1bc9fc7e50f486de2e033a8b4aaf67cb4c33d73aa929362e18dc75d5c7951cbf58120b5fb1de555

fanquake added the Tests label Dec 31, 2020

stackman27 reviewed Jan 3, 2021

View reviewed changes

michaelfolkson reviewed Jan 4, 2021

View reviewed changes

test/functional/rpc_generateblock.py Show resolved Hide resolved

mjdietzx reviewed Jan 4, 2021

View reviewed changes

test/functional/rpc_generateblock.py Show resolved Hide resolved

mjdietzx reviewed Jan 4, 2021

View reviewed changes

taurus228 reviewed Jan 4, 2021

View reviewed changes

maflcko reviewed Jan 5, 2021

View reviewed changes

test/functional/rpc_generateblock.py Outdated Show resolved Hide resolved

maflcko reviewed Jan 5, 2021

View reviewed changes

test/functional/rpc_generateblock.py Outdated Show resolved Hide resolved

maflcko reviewed Jan 5, 2021

View reviewed changes

glozow reviewed Jan 5, 2021

View reviewed changes

test/functional/rpc_generateblock.py Show resolved Hide resolved

test/functional/rpc_generateblock.py Outdated Show resolved Hide resolved

mjdietzx reviewed Jan 10, 2021

View reviewed changes

mjdietzx mentioned this pull request Jan 10, 2021

test: Run mempool_limit.py even with wallet disabled #20874

Closed

DrahtBot mentioned this pull request Jan 12, 2021

test: ensure any failing method in MiniWallet leaves no side-effects #20889

Closed

mjdietzx reviewed Jan 12, 2021

View reviewed changes

stackman27 reviewed Jan 12, 2021

View reviewed changes

test/functional/test_framework/wallet.py Show resolved Hide resolved

test/functional/test_framework/wallet.py Show resolved Hide resolved

DrahtBot mentioned this pull request Jan 27, 2021

test: Run mempool_accept.py even with wallet disabled #21014

Closed

This was referenced Feb 11, 2021

test: convert feature_bip68_sequence.py to use MiniWallet #21144

Closed

test: run mempool_reorg.py even with wallet disabled #21178

Merged

maflcko reviewed Feb 16, 2021

View reviewed changes

nginocchio added 3 commits March 1, 2021 12:51

test: Run rpc_generateblock.py even with wallet disabled

4a6bb45

add submit transaction to network boolean in MiniWallet

f40a01a

make address public

c4fb414

DrahtBot mentioned this pull request Mar 2, 2021

rpc: deprecate addresses and reqSigs from rpc outputs #20286

Merged

DrahtBot added the Needs rebase label Mar 29, 2021

maflcko added the Up for grabs label Aug 27, 2021

maflcko closed this Aug 27, 2021

maflcko mentioned this pull request Aug 27, 2021

test: Convert non-wallet tests to use our python MiniWallet #20078

Closed

DariusParvin mentioned this pull request Nov 19, 2021

test: run rpc-generateblock.py even with wallet disabled #23558

Merged

maflcko removed Up for grabs Needs rebase labels Nov 20, 2021

bitcoin locked and limited conversation to collaborators Nov 20, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test: Run rpc_generateblock.py even with wallet disabled #20808

test: Run rpc_generateblock.py even with wallet disabled #20808

nginocchio commented Dec 31, 2020

stackman27 left a comment •

edited

Loading

michaelfolkson commented Jan 3, 2021

stackman27 commented Jan 3, 2021

michaelfolkson left a comment

mjdietzx left a comment

taurus228 left a comment •

edited by maflcko

Loading

maflcko left a comment

glozow left a comment

mjdietzx Jan 10, 2021

mjdietzx Jan 10, 2021

mjdietzx Jan 10, 2021

DrahtBot commented Jan 12, 2021 •

edited

Loading

mjdietzx left a comment

mjdietzx Jan 12, 2021

nginocchio Jan 12, 2021

mjdietzx Jan 12, 2021

nginocchio Jan 12, 2021 •

edited

Loading

mjdietzx Jan 12, 2021

mjdietzx Jan 12, 2021

maflcko Feb 16, 2021

stackman27 left a comment

maflcko Feb 16, 2021

DrahtBot commented Mar 29, 2021

maflcko commented Jul 20, 2021

jsarenik commented Nov 18, 2021

DariusParvin commented Nov 19, 2021 •

edited

Loading

		address = ADDRESS_BCRT1_P2WSH_OP_TRUE
		scriptPubKey = hex_str_to_bytes(node.validateaddress(ADDRESS_BCRT1_P2WSH_OP_TRUE)['scriptPubKey'])

test: Run rpc_generateblock.py even with wallet disabled #20808

test: Run rpc_generateblock.py even with wallet disabled #20808

Conversation

nginocchio commented Dec 31, 2020

stackman27 left a comment • edited Loading

Choose a reason for hiding this comment

michaelfolkson commented Jan 3, 2021

stackman27 commented Jan 3, 2021

michaelfolkson left a comment

Choose a reason for hiding this comment

mjdietzx left a comment

Choose a reason for hiding this comment

taurus228 left a comment • edited by maflcko Loading

Choose a reason for hiding this comment

maflcko left a comment

Choose a reason for hiding this comment

glozow left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DrahtBot commented Jan 12, 2021 • edited Loading

Conflicts

mjdietzx left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nginocchio Jan 12, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stackman27 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DrahtBot commented Mar 29, 2021

maflcko commented Jul 20, 2021

jsarenik commented Nov 18, 2021

DariusParvin commented Nov 19, 2021 • edited Loading

stackman27 left a comment •

edited

Loading

taurus228 left a comment •

edited by maflcko

Loading

DrahtBot commented Jan 12, 2021 •

edited

Loading

nginocchio Jan 12, 2021 •

edited

Loading

DariusParvin commented Nov 19, 2021 •

edited

Loading