Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: fix failing test interface_usdt_utxocache.py #25486

Merged

Conversation

theStack
Copy link
Contributor

The from_node argument doesn't exist anymore for MiniWallet.create_self_transfer since PR #25435 (commit fa8421b), leading to an error on master:

$ sudo ./test/functional/interface_usdt_utxocache.py
2022-06-27T17:45:35.585000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_7s1djjo1
2022-06-27T17:45:36.515000Z TestFramework (INFO): testing the utxocache:uncache tracepoint API
2022-06-27T17:45:36.517000Z TestFramework (ERROR): Unexpected exception caught during testing
Traceback (most recent call last):
  File "/home/honeybadger/bitcoin/test/functional/test_framework/test_framework.py", line 133, in main
    self.run_test()
  File "/home/honeybadger/bitcoin/./test/functional/interface_usdt_utxocache.py", line 149, in run_test
    self.test_uncache()
  File "/home/honeybadger/bitcoin/./test/functional/interface_usdt_utxocache.py", line 172, in test_uncache
    invalid_tx = self.wallet.create_self_transfer(
TypeError: create_self_transfer() got an unexpected keyword argument 'from_node'
2022-06-27T17:45:36.568000Z TestFramework (INFO): Stopping nodes
[...]

Fix this by removing the argument. (Unfortunately, the USDT tests don't seem to run on any CI target, I guess that's due to missing permissions to hook into the kernel.)

The `from_node` argument doesn't exist anymore for
`MiniWallet.create_self_transfer` since PR bitcoin#25435 (commit
fa8421b).
@DrahtBot DrahtBot added the Tests label Jun 27, 2022
@maflcko maflcko requested a review from 0xB10C June 28, 2022 06:30
@maflcko
Copy link
Member

maflcko commented Jun 28, 2022

Yeah, doesn't look like this is run in CI.

cr ACK f665c6e

@maflcko maflcko merged commit 78957e7 into bitcoin:master Jun 28, 2022
@ayush933
Copy link
Contributor

crACK f665c6e

@0xB10C
Copy link
Contributor

0xB10C commented Jun 28, 2022

Thanks for fixing!

I'll give #24782 another shot.

@theStack theStack deleted the 202206-test-fix_interface_usdt_utxocache branch June 28, 2022 08:29
sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Jun 28, 2022
….py`

f665c6e test: fix failing test interface_usdt_utxocache.py (Sebastian Falbesoner)

Pull request description:

  The `from_node` argument doesn't exist anymore for `MiniWallet.create_self_transfer` since PR bitcoin#25435 (commit fa8421b), leading to an error on master:

  ```
  $ sudo ./test/functional/interface_usdt_utxocache.py
  2022-06-27T17:45:35.585000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_7s1djjo1
  2022-06-27T17:45:36.515000Z TestFramework (INFO): testing the utxocache:uncache tracepoint API
  2022-06-27T17:45:36.517000Z TestFramework (ERROR): Unexpected exception caught during testing
  Traceback (most recent call last):
    File "/home/honeybadger/bitcoin/test/functional/test_framework/test_framework.py", line 133, in main
      self.run_test()
    File "/home/honeybadger/bitcoin/./test/functional/interface_usdt_utxocache.py", line 149, in run_test
      self.test_uncache()
    File "/home/honeybadger/bitcoin/./test/functional/interface_usdt_utxocache.py", line 172, in test_uncache
      invalid_tx = self.wallet.create_self_transfer(
  TypeError: create_self_transfer() got an unexpected keyword argument 'from_node'
  2022-06-27T17:45:36.568000Z TestFramework (INFO): Stopping nodes
  [...]
  ```
  Fix this by removing the argument. (Unfortunately, the USDT tests don't seem to run on any CI target, I guess that's due to missing permissions to hook into the kernel.)

ACKs for top commit:
  MarcoFalke:
    cr ACK f665c6e

Tree-SHA512: 74f8e398739a25ab5518ff71b998d03d4e529a786ba5b424509de81a511ad3e2e1cd38a5b7bb9f1f5a21340391d6807f4951ff39fa3a2ad65a3b11b989eebea6
@0xB10C
Copy link
Contributor

0xB10C commented Jun 30, 2022

@theStack did you test that this fixed the failing test? It's still failing for me though with a different error.

Bisecting yields fa13375 which makes sense as it adds a call to rescan_utxos() which calls the scantxoutset RPC which internally flushes the utxo set cache. The expected numbers of flushes doesn't match anymore. It seems to be more subtle than that.

I'm working on CI tests for the interface tests and will include a fix there.

@theStack
Copy link
Contributor Author

theStack commented Jul 1, 2022

@theStack did you test that this fixed the failing test? It's still failing for me though with a different error.

Oops, seems I didn't, apparently I ran one of the other USDT tests after the change 🤔

I'm working on CI tests for the interface tests and will include a fix there.

Feel free to tag me in this PR, happy to test (won't have access to a local machine where I can run USDT stuff for the next few days, but will follow up when I have again).

0xB10C added a commit to 0xB10C/bitcoin that referenced this pull request Jul 2, 2022
The USDT interface exposes process internals via the tracepoints. This
means, the USDT interface tests somewhat awardly depend on these
internals. If internals change, the tests have to adopt to that change.
Previously, the USDT interface tests weren't run in the CI so changes
could break the USDT interface tests without being noticed (e.g.
bitcoin#25486).

In fa13375 a 'self.rescan_utxos()' call
was added in the 'generate()' function of the test framework.
'rescan_utxos()' causes the UTXO cache to be flushed. In the USDT
interface tests for the 'utxocache:flush' trancepoint, 'generate()' is
used. As the utxo cache is now flushed more often, the number of flushes
the tests expectes need to be adopted. Also, the utxo cache has now a
different size when being flushed.

The utxocache tracepoint is tested by shutting the node down and
pruning blocks, to test the 'for_prune' argument.

Changes:
- A list 'expected_flushes' is now used which contains 'mode',
  'for_prune', and 'size' for each expected flush.
- When a flush happens, the expected-flush is removed from the list.
  This list is checked to be empty (unchanged).
- Previously, shutting down caused these two flushes:
    UTXOCacheFlush(duration=*, mode=ALWAYS, size=104, memory=*, for_prune=False)
    UTXOCacheFlush(duration=*, mode=ALWAYS, size=0, memory=*, for_prune=False)
  now it causes these flushes:
    UTXOCacheFlush(duration=*, mode=ALWAYS, size=2, memory=*, for_prune=False)
    UTXOCacheFlush(duration=*, mode=ALWAYS, size=0, memory=*, for_prune=False)
  The 104 UTXOs flushed previously were mainly coinbase UTXOs generated
  in previous tests and the test setup. These are now already flushed.
- In the 'for_prune' test we previously hooked into the tracepoint
  before mining blocks. This changed to only get notified about the
  tracepoint being triggered for the prune. Here, the utxo cache is
  empty already as it has just been flushed in 'generate()'.
  old:
    UTXOCacheFlush(duration=*, mode=NONE, size=350, memory=*, for_prune=True)
  new:
    UTXOCacheFlush(duration=*, mode=NONE, size=0, memory=*, for_prune=True)
0xB10C added a commit to 0xB10C/bitcoin that referenced this pull request Jul 2, 2022
The USDT interface exposes process internals via the tracepoints. This
means, the USDT interface tests somewhat awardly depend on these
internals. If internals change, the tests have to adopt to that change.
Previously, the USDT interface tests weren't run in the CI so changes
could break the USDT interface tests without being noticed (e.g.
bitcoin#25486).

In fa13375 a 'self.rescan_utxos()' call
was added in the 'generate()' function of the test framework.
'rescan_utxos()' causes the UTXO cache to be flushed. In the USDT
interface tests for the 'utxocache:flush' trancepoint, 'generate()' is
used. As the utxo cache is now flushed more often, the number of flushes
the tests expectes need to be adopted. Also, the utxo cache has now a
different size when being flushed.

The utxocache tracepoint is tested by shutting the node down and
pruning blocks, to test the 'for_prune' argument.

Changes:
- A list 'expected_flushes' is now used which contains 'mode',
  'for_prune', and 'size' for each expected flush.
- When a flush happens, the expected-flush is removed from the list.
  This list is checked to be empty (unchanged).
- Previously, shutting down caused these two flushes:
    UTXOCacheFlush(duration=*, mode=ALWAYS, size=104, memory=*, for_prune=False)
    UTXOCacheFlush(duration=*, mode=ALWAYS, size=0, memory=*, for_prune=False)
  now it causes these flushes:
    UTXOCacheFlush(duration=*, mode=ALWAYS, size=2, memory=*, for_prune=False)
    UTXOCacheFlush(duration=*, mode=ALWAYS, size=0, memory=*, for_prune=False)
  The 104 UTXOs flushed previously were mainly coinbase UTXOs generated
  in previous tests and the test setup. These are now already flushed.
- In the 'for_prune' test we previously hooked into the tracepoint
  before mining blocks. This changed to only get notified about the
  tracepoint being triggered for the prune. Here, the utxo cache is
  empty already as it has just been flushed in 'generate()'.
  old:
    UTXOCacheFlush(duration=*, mode=NONE, size=350, memory=*, for_prune=True)
  new:
    UTXOCacheFlush(duration=*, mode=NONE, size=0, memory=*, for_prune=True)
0xB10C added a commit to 0xB10C/bitcoin that referenced this pull request Jul 29, 2022
The USDT interface exposes process internals via the tracepoints. This
means, the USDT interface tests somewhat awardly depend on these
internals. If internals change, the tests have to adopt to that change.
Previously, the USDT interface tests weren't run in the CI so changes
could break the USDT interface tests without being noticed (e.g.
bitcoin#25486).

In fa13375 a 'self.rescan_utxos()' call
was added in the 'generate()' function of the test framework.
'rescan_utxos()' causes the UTXO cache to be flushed. In the USDT
interface tests for the 'utxocache:flush' trancepoint, 'generate()' is
used. As the utxo cache is now flushed more often, the number of flushes
the tests expectes need to be adopted. Also, the utxo cache has now a
different size when being flushed.

The utxocache tracepoint is tested by shutting the node down and
pruning blocks, to test the 'for_prune' argument.

Changes:
- A list 'expected_flushes' is now used which contains 'mode',
  'for_prune', and 'size' for each expected flush.
- When a flush happens, the expected-flush is removed from the list.
  This list is checked to be empty (unchanged).
- Previously, shutting down caused these two flushes:
    UTXOCacheFlush(duration=*, mode=ALWAYS, size=104, memory=*, for_prune=False)
    UTXOCacheFlush(duration=*, mode=ALWAYS, size=0, memory=*, for_prune=False)
  now it causes these flushes:
    UTXOCacheFlush(duration=*, mode=ALWAYS, size=2, memory=*, for_prune=False)
    UTXOCacheFlush(duration=*, mode=ALWAYS, size=0, memory=*, for_prune=False)
  The 104 UTXOs flushed previously were mainly coinbase UTXOs generated
  in previous tests and the test setup. These are now already flushed.
- In the 'for_prune' test we previously hooked into the tracepoint
  before mining blocks. This changed to only get notified about the
  tracepoint being triggered for the prune. Here, the utxo cache is
  empty already as it has just been flushed in 'generate()'.
  old:
    UTXOCacheFlush(duration=*, mode=NONE, size=350, memory=*, for_prune=True)
  new:
    UTXOCacheFlush(duration=*, mode=NONE, size=0, memory=*, for_prune=True)
stickies-v pushed a commit to stickies-v/bitcoin that referenced this pull request Aug 2, 2022
The USDT interface exposes process internals via the tracepoints. This
means, the USDT interface tests somewhat awardly depend on these
internals. If internals change, the tests have to adopt to that change.
Previously, the USDT interface tests weren't run in the CI so changes
could break the USDT interface tests without being noticed (e.g.
bitcoin#25486).

In fa13375 a 'self.rescan_utxos()' call
was added in the 'generate()' function of the test framework.
'rescan_utxos()' causes the UTXO cache to be flushed. In the USDT
interface tests for the 'utxocache:flush' trancepoint, 'generate()' is
used. As the utxo cache is now flushed more often, the number of flushes
the tests expectes need to be adopted. Also, the utxo cache has now a
different size when being flushed.

The utxocache tracepoint is tested by shutting the node down and
pruning blocks, to test the 'for_prune' argument.

Changes:
- A list 'expected_flushes' is now used which contains 'mode',
  'for_prune', and 'size' for each expected flush.
- When a flush happens, the expected-flush is removed from the list.
  This list is checked to be empty (unchanged).
- Previously, shutting down caused these two flushes:
    UTXOCacheFlush(duration=*, mode=ALWAYS, size=104, memory=*, for_prune=False)
    UTXOCacheFlush(duration=*, mode=ALWAYS, size=0, memory=*, for_prune=False)
  now it causes these flushes:
    UTXOCacheFlush(duration=*, mode=ALWAYS, size=2, memory=*, for_prune=False)
    UTXOCacheFlush(duration=*, mode=ALWAYS, size=0, memory=*, for_prune=False)
  The 104 UTXOs flushed previously were mainly coinbase UTXOs generated
  in previous tests and the test setup. These are now already flushed.
- In the 'for_prune' test we previously hooked into the tracepoint
  before mining blocks. This changed to only get notified about the
  tracepoint being triggered for the prune. Here, the utxo cache is
  empty already as it has just been flushed in 'generate()'.
  old:
    UTXOCacheFlush(duration=*, mode=NONE, size=350, memory=*, for_prune=True)
  new:
    UTXOCacheFlush(duration=*, mode=NONE, size=0, memory=*, for_prune=True)
Rspigler pushed a commit to Rspigler/bitcoin that referenced this pull request Aug 21, 2022
The USDT interface exposes process internals via the tracepoints. This
means, the USDT interface tests somewhat awardly depend on these
internals. If internals change, the tests have to adopt to that change.
Previously, the USDT interface tests weren't run in the CI so changes
could break the USDT interface tests without being noticed (e.g.
bitcoin#25486).

In fa13375 a 'self.rescan_utxos()' call
was added in the 'generate()' function of the test framework.
'rescan_utxos()' causes the UTXO cache to be flushed. In the USDT
interface tests for the 'utxocache:flush' trancepoint, 'generate()' is
used. As the utxo cache is now flushed more often, the number of flushes
the tests expectes need to be adopted. Also, the utxo cache has now a
different size when being flushed.

The utxocache tracepoint is tested by shutting the node down and
pruning blocks, to test the 'for_prune' argument.

Changes:
- A list 'expected_flushes' is now used which contains 'mode',
  'for_prune', and 'size' for each expected flush.
- When a flush happens, the expected-flush is removed from the list.
  This list is checked to be empty (unchanged).
- Previously, shutting down caused these two flushes:
    UTXOCacheFlush(duration=*, mode=ALWAYS, size=104, memory=*, for_prune=False)
    UTXOCacheFlush(duration=*, mode=ALWAYS, size=0, memory=*, for_prune=False)
  now it causes these flushes:
    UTXOCacheFlush(duration=*, mode=ALWAYS, size=2, memory=*, for_prune=False)
    UTXOCacheFlush(duration=*, mode=ALWAYS, size=0, memory=*, for_prune=False)
  The 104 UTXOs flushed previously were mainly coinbase UTXOs generated
  in previous tests and the test setup. These are now already flushed.
- In the 'for_prune' test we previously hooked into the tracepoint
  before mining blocks. This changed to only get notified about the
  tracepoint being triggered for the prune. Here, the utxo cache is
  empty already as it has just been flushed in 'generate()'.
  old:
    UTXOCacheFlush(duration=*, mode=NONE, size=350, memory=*, for_prune=True)
  new:
    UTXOCacheFlush(duration=*, mode=NONE, size=0, memory=*, for_prune=True)
janus pushed a commit to BitgesellOfficial/bitgesell that referenced this pull request Jan 20, 2023
The USDT interface exposes process internals via the tracepoints. This
means, the USDT interface tests somewhat awardly depend on these
internals. If internals change, the tests have to adopt to that change.
Previously, the USDT interface tests weren't run in the CI so changes
could break the USDT interface tests without being noticed (e.g.
bitcoin/bitcoin#25486).

In fa13375aa3fcb4fd5b9e0d4c69ac31cf66c3209a a 'self.rescan_utxos()' call
was added in the 'generate()' function of the test framework.
'rescan_utxos()' causes the UTXO cache to be flushed. In the USDT
interface tests for the 'utxocache:flush' trancepoint, 'generate()' is
used. As the utxo cache is now flushed more often, the number of flushes
the tests expectes need to be adopted. Also, the utxo cache has now a
different size when being flushed.

The utxocache tracepoint is tested by shutting the node down and
pruning blocks, to test the 'for_prune' argument.

Changes:
- A list 'expected_flushes' is now used which contains 'mode',
  'for_prune', and 'size' for each expected flush.
- When a flush happens, the expected-flush is removed from the list.
  This list is checked to be empty (unchanged).
- Previously, shutting down caused these two flushes:
    UTXOCacheFlush(duration=*, mode=ALWAYS, size=104, memory=*, for_prune=False)
    UTXOCacheFlush(duration=*, mode=ALWAYS, size=0, memory=*, for_prune=False)
  now it causes these flushes:
    UTXOCacheFlush(duration=*, mode=ALWAYS, size=2, memory=*, for_prune=False)
    UTXOCacheFlush(duration=*, mode=ALWAYS, size=0, memory=*, for_prune=False)
  The 104 UTXOs flushed previously were mainly coinbase UTXOs generated
  in previous tests and the test setup. These are now already flushed.
- In the 'for_prune' test we previously hooked into the tracepoint
  before mining blocks. This changed to only get notified about the
  tracepoint being triggered for the prune. Here, the utxo cache is
  empty already as it has just been flushed in 'generate()'.
  old:
    UTXOCacheFlush(duration=*, mode=NONE, size=350, memory=*, for_prune=True)
  new:
    UTXOCacheFlush(duration=*, mode=NONE, size=0, memory=*, for_prune=True)
@bitcoin bitcoin locked and limited conversation to collaborators Jul 1, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants