Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: Run mempool_limit.py even with wallet disabled #20874

Closed
wants to merge 2 commits into from

Conversation

stackman27
Copy link
Contributor

@stackman27 stackman27 commented Jan 7, 2021

This is a PR proposed in #20078.

This PR enables one more of the non-wallet functional tests by running mempool_limit.py even when the wallet is disabled. While restructuring the code, I added an extra method, send_multi_self_transfer in MiniWallet class. This method allows the creation of large transactions (by appending txouts to tx.vout) which will be useful in other files like mining_prioritisetransaction.py as well.

While my approach may not be the cleanest or the most efficient, I totally appreciate any suggestion to improve it. Thank you very much!

@DrahtBot
Copy link
Contributor

DrahtBot commented Jan 7, 2021

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

Copy link
Contributor

@mjdietzx mjdietzx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took a look at doing this refactoring a week or so ago, and figured I'd do a few easier ones bs this one was so tricky. But looks like you figured it out! Anyways I left a bunch of comments on things I noticed

test/functional/mempool_limit.py Outdated Show resolved Hide resolved
test/functional/mempool_limit.py Show resolved Hide resolved
test/functional/test_framework/wallet.py Outdated Show resolved Hide resolved
test/functional/test_framework/wallet.py Outdated Show resolved Hide resolved
test/functional/test_framework/wallet.py Outdated Show resolved Hide resolved
test/functional/test_framework/wallet.py Outdated Show resolved Hide resolved
test/functional/test_framework/wallet.py Outdated Show resolved Hide resolved
test/functional/mempool_limit.py Outdated Show resolved Hide resolved
test/functional/mempool_limit.py Outdated Show resolved Hide resolved
test/functional/test_framework/wallet.py Outdated Show resolved Hide resolved
@stackman27 stackman27 force-pushed the diswallet_mempoollimit branch 4 times, most recently from 423e609 to 421c3ac Compare January 9, 2021 00:27
@stackman27
Copy link
Contributor Author

stackman27 commented Jan 9, 2021

@mjdietzx Thank you very much for the review. I just pushed an update shortening the methods in miniwallet class by adding prepare_tx and also resolved most of your comments

test/functional/mempool_limit.py Outdated Show resolved Hide resolved
test/functional/mempool_limit.py Show resolved Hide resolved
test/functional/test_framework/wallet.py Outdated Show resolved Hide resolved
test/functional/test_framework/wallet.py Outdated Show resolved Hide resolved
test/functional/test_framework/wallet.py Outdated Show resolved Hide resolved
test/functional/mempool_limit.py Outdated Show resolved Hide resolved
@stackman27 stackman27 force-pushed the diswallet_mempoollimit branch 3 times, most recently from 47282a0 to b8344b1 Compare January 11, 2021 00:42
@stackman27
Copy link
Contributor Author

@mjdietzx Thank you very much for the review. Pushed a new update highlighting all the comments and also updated MiniWallet by adding just-create-don't-send method as proposed by @glozow in #20876. Also, I believe the new create_and_sign_rawtx method should resolve @nginocchio's issues as proposed in #20808

Copy link
Contributor

@mjdietzx mjdietzx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, I think this is close! If you look at my comment, I think you can do this PR in a way that wallet.py is not changed by bringing in the submit_tx=False change for miniwallet.send_self_transfer

test/functional/mempool_limit.py Outdated Show resolved Hide resolved
test/functional/mempool_limit.py Outdated Show resolved Hide resolved
Copy link
Member

@glozow glozow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK. I actually think a cleaner approach would be to make create_lots_of_big_transactions and the other util.py transaction not need wallet. Could reduce the amount of refactoring needed in each of the functional tests?

test/functional/mempool_limit.py Outdated Show resolved Hide resolved
test/functional/mempool_limit.py Outdated Show resolved Hide resolved
test/functional/test_framework/wallet.py Outdated Show resolved Hide resolved
@stackman27 stackman27 force-pushed the diswallet_mempoollimit branch 2 times, most recently from 5873d94 to 42e5251 Compare January 12, 2021 22:15
@stackman27 stackman27 force-pushed the diswallet_mempoollimit branch 3 times, most recently from 8d7d57b to a478bf7 Compare February 25, 2021 15:19
@stackman27
Copy link
Contributor Author

Concept ACK. I actually think a cleaner approach would be to make create_lots_of_big_transactions and the other util.py transaction not need wallet. Could reduce the amount of refactoring needed in each of the functional tests?

@glozow Thank you very much for the review!
Are you suggesting an alternative way to create_lots_of_big_transactions or just simply getting rid of it?

Copy link
Contributor

@DariusParvin DariusParvin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK! Just some suggestions that might make it more readable. Also, now that Miniwallet.scan_blocks has been merged, I think you can use it at the start.

test/functional/mempool_limit.py Show resolved Hide resolved
test/functional/mempool_limit.py Outdated Show resolved Hide resolved
assert_raises_rpc_error(-26, "mempool min fee not met", self.nodes[0].sendrawtransaction, txFS['hex'])
assert_raises_rpc_error(-26, "mempool min fee not met", miniwallet.send_self_transfer, from_node=node, fee_rate=relayfee)

def create_large_transactions(self, node, txouts, miniwallet, num, fee, mempool_min_fee):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if send_large_transactions would be a better name since it is also broadcasting them? It would be more consistent with send_self_transfer.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm just following the naming convention used here as it's just a copy of that code. But happy to change :)

tx.vout.extend(txouts)
tx_hex = tx.serialize().hex()
txids = node.sendrawtransaction(tx_hex, 0)
if node.getmempoolinfo()['mempoolminfee'] == mempool_min_fee:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the purpose of having this condition in here? As a result, it means that there are transactions which get broadcast and enter the mempool, but are not added to large_txids. I'll add a comment to where it's used to suggest what I think would be a better alternative.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay! will add a comment but the basic logic is that i'm making sure that the fees have changed and only include those txs whose fees have gone up. This also supports the assert logic at like 48

test/functional/mempool_limit.py Outdated Show resolved Hide resolved
@stackman27
Copy link
Contributor Author

Thank you @DariusParvin for the review. Added changes as suggested


def create_large_transactions(self, node, txouts, miniwallet, num, fee, mempool_min_fee):
large_txids = []
for j in range(num):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: since j is unused, use _ instead?

for i in range (3):
txids.append([])
txids[i] = create_lots_of_big_transactions(self.nodes[0], txouts, utxos[30*i:30*i+30], 30, (i+1)*base_fee)
txids[i] = self.create_large_transactions(node, txouts, miniwallet, 30, (i+1)*base_fee, mempool_min_fee)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here num is set to 30 but the test also works with 22 (with only the original tx getting kicked from the mempool). My feeling is that it's more beneficial to have the test run faster than to have some extra leeway with excess txs.

Suggested change
txids[i] = self.create_large_transactions(node, txouts, miniwallet, 30, (i+1)*base_fee, mempool_min_fee)
txids[i] = self.create_large_transactions(node, txouts, miniwallet, 22, (i+1)*base_fee, mempool_min_fee)

@DrahtBot
Copy link
Contributor

DrahtBot commented Jun 1, 2021

🐙 This pull request conflicts with the target branch and needs rebase.

Want to unsubscribe from rebase notifications on this pull request? Just convert this pull request to a "draft".

@maflcko
Copy link
Member

maflcko commented Jul 20, 2021

Are you still working on this?

@stackman27
Copy link
Contributor Author

Are you still working on this?

Not at the moment, have been busy with work and classes

@fanquake fanquake closed this Jul 21, 2021
maflcko pushed a commit that referenced this pull request Sep 14, 2021
08634e8 fix typos in logging messages (ShubhamPalriwala)
d447ded replace: self.nodes[0] with node (ShubhamPalriwala)
dddca38 test: use MiniWallet in mempool_limit.py (ShubhamPalriwala)

Pull request description:

  This is a PR proposed in #20078

  This PR enables running another non-wallet functional test even when the wallet is disabled thanks to the MiniWallet, i.e. it can be run even when bitcoin-core is compiled with --disable-wallet.

  It also includes changes in wallet.py in the form of a new method, `create_large_transactions()` for the MiniWallet to create large transactions.

  Efforts for this feature started in #20874 but were not continued and that PR was closed hence I picked this up.

  To test this PR locally, compile and build bitcoin-core without the wallet and run:
  ```
  $ test/functional/mempool_limit.py
  ```

ACKs for top commit:
  amitiuttarwar:
    ACK 08634e8, only git changes since last push (and one new line).
  Zero-1729:
    ACK 08634e8 🧉

Tree-SHA512: 0f744ad26bf7a5a784aac1ed5077b59c95a36d1ff3ad0087ffd10ac8d5979f7362c63c20c2ce2bfa650fda02dfbcd60b1fceee049a2465c8d221cce51c20369f
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Aug 18, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants