New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test: Fix intermittent issue in mining_getblocktemplate_longpoll.py #27941
Conversation
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ReviewsSee the guideline for information on the review process.
If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approach NACK. Longpolling is supposed to work regardless of the timing here. If anything, we should explicitly trigger this race.
Can you explain this? How is longpoll supposed to work when the longpoll thread hasn't started? The thread must be initiated by the remote first, before it can be used or assumed to be working. |
To clarify, there are several threads, the |
The longpoll request is supposed to return immediately if the conditions have been met. The token includes a reference to the transaction update state as a counter. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Github ACK fa748c6
Seems like a commonsense fix? Can't test the RPC method if the Bitcoin Core HTTP handler hasn't booted.
Thanks, I'll take another look later. |
@luke-jr I presume if this was a bug on the RPC side, it could be fixed with something like: diff --git a/src/rpc/mining.cpp b/src/rpc/mining.cpp
index 074cecadd2..ca7d1acda8 100644
--- a/src/rpc/mining.cpp
+++ b/src/rpc/mining.cpp
@@ -742,12 +742,12 @@ static RPCHelpMan getblocktemplate()
WAIT_LOCK(g_best_block_mutex, lock);
while (g_best_block == hashWatchedChain && IsRPCRunning())
{
+ // Check transactions for update
+ // without holding the mempool lock to avoid deadlocks
+ if (mempool.GetTransactionsUpdated() != nTransactionsUpdatedLastLP)
+ break;
if (g_best_block_cv.wait_until(lock, checktxtime) == std::cv_status::timeout)
{
- // Timeout: Check transactions for update
- // without holding the mempool lock to avoid deadlocks
- if (mempool.GetTransactionsUpdated() != nTransactionsUpdatedLastLP)
- break;
checktxtime += std::chrono::seconds(10);
}
} However, I couldn't find any documentation about |
In any case, if the Bitcoin Core behavior is changed, it should be accompanied with deterministic tests. I don't think non-deterministic tests are useful, especially if they lead to a failure every time they hit the "other side" of the race. Going to reopen for now to fix the intermittent issue, and make the tests deterministic. Happy to review a future or conflicting pull that instead changes the non-test code, or if there is documentation I missed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
…emplate_longpoll.py fa748c6 test: Fix intermittent issue in mining_getblocktemplate_longpoll.py (MarcoFalke) Pull request description: Fixes bitcoin#26962 Wait for the thread to have started and the RPC to have reached the node before continuing. Otherwise the test may run into a race. For example: ``` test 2023-06-23T13:10:29.245000Z TestFramework (INFO): Test that introducing a new transaction into the mempool will terminate the longpoll node0 2023-06-23T13:10:29.245712Z [http] [httpserver.cpp:254] [http_request_cb] [http] Received a POST request for / from 127.0.0.1:43568 node0 2023-06-23T13:10:29.245915Z [httpworker.3] [rpc/request.cpp:181] [parse] [rpc] ThreadRPCServer method=getblocktemplate user=__cookie__ node0 2023-06-23T13:10:29.252594Z [http] [httpserver.cpp:254] [http_request_cb] [http] Received a POST request for / from 127.0.0.1:43568 node0 2023-06-23T13:10:29.254545Z [httpworker.2] [rpc/request.cpp:181] [parse] [rpc] ThreadRPCServer method=getblockchaininfo user=__cookie__ node0 2023-06-23T13:10:29.256530Z [http] [httpserver.cpp:254] [http_request_cb] [http] Received a POST request for / from 127.0.0.1:43568 node0 2023-06-23T13:10:29.256741Z [httpworker.1] [rpc/request.cpp:181] [parse] [rpc] ThreadRPCServer method=sendrawtransaction user=__cookie__ node0 2023-06-23T13:10:29.258033Z [httpworker.1] [validationinterface.cpp:213] [TransactionAddedToMempool] [validation] Enqueuing TransactionAddedToMempool: txid=38335600f2465c0f8bb2b86d5830a34851d86fa879800c0e1434ddfc78c42898 wtxid=c033cd3efd301c369d66cf759769159609471bd4f9efb3ee30e7209e57b74778 node0 2023-06-23T13:10:29.258263Z [httpworker.1] [txmempool.cpp:660] [check] [mempool] Checking mempool with 1 transactions and 1 inputs node0 2023-06-23T13:10:29.258542Z [scheduler] [validationinterface.cpp:213] [operator()] [validation] TransactionAddedToMempool: txid=38335600f2465c0f8bb2b86d5830a34851d86fa879800c0e1434ddfc78c42898 wtxid=c033cd3efd301c369d66cf759769159609471bd4f9efb3ee30e7209e57b74778 node0 2023-06-23T13:10:29.259549Z [http] [httpserver.cpp:254] [http_request_cb] [http] Received a POST request for / from 127.0.0.1:43568 node0 2023-06-23T13:10:29.259745Z [httpworker.0] [rpc/request.cpp:181] [parse] [rpc] ThreadRPCServer method=decoderawtransaction user=__cookie__ node0 2023-06-23T13:10:29.261066Z [http] [httpserver.cpp:254] [http_request_cb] [http] Received a POST request for / from 127.0.0.1:52690 node0 2023-06-23T13:10:29.261803Z [http] [httpserver.cpp:254] [http_request_cb] [http] Received a POST request for / from 127.0.0.1:43568 node0 2023-06-23T13:10:29.262770Z [httpworker.2] [rpc/request.cpp:181] [parse] [rpc] ThreadRPCServer method=getblocktemplate user=__cookie__ ``` (`sendrawtransaction` is called before `getblocktemplate`) ACKs for top commit: jamesob: Github ACK bitcoin@fa748c6 theStack: ACK fa748c6 Tree-SHA512: c67d9ec7c56e8a22c1a26a3c3d4d4a4bcc17e4282cad0d66561ba2abd6e92240cb028369b4edc6077ea34e8736c0294f6066381979aee22a6166580cea43729a
Fixes #26962
Wait for the thread to have started and the RPC to have reached the node before continuing. Otherwise the test may run into a race.
For example:
(
sendrawtransaction
is called beforegetblocktemplate
)