-
Notifications
You must be signed in to change notification settings - Fork 35.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fuzz: Consolidate fuzzing TestingSetup initialization #20946
Conversation
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
Concept ACK Thanks for improving the fuzzing code! Very happy to the number of active contributors to
I don't think that was intentional on my part. We want |
Can someone help me out with this fuzzer message? Is this a real error that we caught or something benign? https://cirrus-ci.com/task/5473797673320448?command=ci#L3550 |
Looks like an issue with your code. Could it mean that you destroyed some object twice? |
827b5df
to
b4a5ccc
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ACK b4a5ccc
src/test/fuzz/util.h
Outdated
@@ -338,9 +339,17 @@ inline void FillNode(FuzzedDataProvider& fuzzed_data_provider, CNode& node, cons | |||
} | |||
} | |||
|
|||
inline void InitializeFuzzingContext(const std::string& chain_name = CBaseChainParams::REGTEST) | |||
template <class T = const BasicTestingSetup> | |||
inline std::unique_ptr<T> MakeFuzzingContext(const std::string& chain_name = CBaseChainParams::REGTEST, const std::vector<const char*>& extra_args = {}) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
style nit: templates are already inline, no need to specify twice
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, it looks like returning T
is just fine here and should be sufficient to show that the caller owns the object that is returned:
diff --git a/src/test/fuzz/process_message.cpp b/src/test/fuzz/process_message.cpp
index e9c5cb5954..19523d6951 100644
--- a/src/test/fuzz/process_message.cpp
+++ b/src/test/fuzz/process_message.cpp
@@ -39,7 +39,7 @@ const TestingSetup* g_setup;
void initialize_process_message()
{
static const auto testing_setup = MakeFuzzingContext<const TestingSetup>();
- g_setup = testing_setup.get();
+ g_setup = &testing_setup;
for (int i = 0; i < 2 * COINBASE_MATURITY; i++) {
MineBlock(g_setup->m_node, CScript() << OP_TRUE);
}
diff --git a/src/test/fuzz/process_messages.cpp b/src/test/fuzz/process_messages.cpp
index 291d5df191..d8fa1028e5 100644
--- a/src/test/fuzz/process_messages.cpp
+++ b/src/test/fuzz/process_messages.cpp
@@ -24,7 +24,7 @@ const TestingSetup* g_setup;
void initialize_process_messages()
{
static const auto testing_setup = MakeFuzzingContext<const TestingSetup>();
- g_setup = testing_setup.get();
+ g_setup = &testing_setup;
for (int i = 0; i < 2 * COINBASE_MATURITY; i++) {
MineBlock(g_setup->m_node, CScript() << OP_TRUE);
}
diff --git a/src/test/fuzz/util.h b/src/test/fuzz/util.h
index 9f1dfa44a1..68c6838a36 100644
--- a/src/test/fuzz/util.h
+++ b/src/test/fuzz/util.h
@@ -339,8 +339,8 @@ inline void FillNode(FuzzedDataProvider& fuzzed_data_provider, CNode& node, cons
}
}
-template <class T = const BasicTestingSetup>
-inline std::unique_ptr<T> MakeFuzzingContext(const std::string& chain_name = CBaseChainParams::REGTEST, const std::vector<const char*>& extra_args = {})
+template <class T = BasicTestingSetup>
+T MakeFuzzingContext(const std::string& chain_name = CBaseChainParams::REGTEST, const std::vector<const char*>& extra_args = {})
{
// Prepend default arguments for fuzzing
const std::vector<const char*> arguments = Cat(
@@ -349,7 +349,7 @@ inline std::unique_ptr<T> MakeFuzzingContext(const std::string& chain_name = CBa
},
extra_args);
- return MakeUnique<T>(chain_name, arguments);
+ return T{chain_name, arguments};
}
class FuzzedFileProvider
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
w/re returning T
, I think I'm going to keep the unique_ptr
just because in the future we might do something in MakeFuzzingContext
that doesn't trigger RVO and I'm not sure about the copy semantics of our *TestingSetup
classes...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As you are using auto
, changing from/to unique_ptr is a small diff, so can be done later easily, if needed
Previously, the {Basic,}TestingSetup for fuzzers were set up in many ways: 1. Calling InitializeFuzzingContext, which implicitly constructs a static const BasicTestingSetup 2. Directly constructing a static const BasicTestingSetup in the initialize_* function 3. Directly constructing a static TestingSetup and reproducing the initialization arguments (I'm assuming because InitializeFuzzingContext only initializes a BasicTestingSetup) The new, relatively-simple MakeFuzzingContext function allows us to consolidate these methods of initialization by being flexible enough to be used in all situations. It: 1. Is templated so that we can choose to initialize any of the *TestingSetup classes 2. Has sane defaults which are often used in fuzzers but are also easily overridable 3. Returns a unique_ptr, explicitly transferring ownership to the caller to deal with according to its situation
A full TestingSetup is required for both coins_view and load_external_block_file as they interact with the active chainstate.
b4a5ccc
to
abb6fa7
Compare
ACK abb6fa7 |
}, | ||
extra_args); | ||
|
||
return MakeUnique<T>(chain_name, arguments); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: new code can use
return MakeUnique<T>(chain_name, arguments); | |
return std::make_unique<T>(chain_name, arguments); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed in #21003
…ation abb6fa7 fuzz: Initialize a full TestingSetup where appropriate (Carl Dong) 713314a fuzz: Consolidate fuzzing TestingSetup initialization (Carl Dong) Pull request description: ``` Previously, the {Basic,}TestingSetup for fuzzers were set up in many ways: 1. Calling InitializeFuzzingContext, which implicitly constructs a static const BasicTestingSetup 2. Directly constructing a static const BasicTestingSetup in the initialize_* function 3. Directly constructing a static TestingSetup and reproducing the initialization arguments (I'm assuming because InitializeFuzzingContext only initializes a BasicTestingSetup) The new, relatively-simple MakeFuzzingContext function allows us to consolidate these methods of initialization by being flexible enough to be used in all situations. It: 1. Is templated so that we can choose to initialize any of the *TestingSetup classes 2. Has sane defaults which are often used in fuzzers but are also easily overridable 3. Returns a unique_ptr, explicitly transferring ownership to the caller to deal with according to its situation ``` ~~Question for fuzzing people: was it intentional that `src/test/fuzz/net.cpp` would directly instantiate the `BasicTestingSetup` and thus omit the `"-nodebuglogfile"` flag?~~ [Answered](bitcoin#20946 (comment)) ACKs for top commit: MarcoFalke: ACK abb6fa7 Tree-SHA512: 96a5ca6f4cd5ea0e9483b60165b31ae3e9003918c700a7f6ade48010f419f2a6312e10b816b3187f1d263798827571866e4c4ac0bbfb2e0c79dfad254cda68e7
…e it in bench fa576b4 Move MakeNoLogFileContext to common libtest_util, and use it in bench (MarcoFalke) Pull request description: To avoid verbose code duplication, which may lead to accidental mishaps https://github.com/bitcoin/bitcoin/pull/20998/files#r563624041. Also fix a nit I found in bitcoin/bitcoin#20946 (comment). ACKs for top commit: dongcarl: Light Code-Review ACK fa576b4 fanquake: ACK fa576b4 Tree-SHA512: d39ac9c0957813ebb20ed13bd25a8ef8469377ce2651245638bd761c796feac2a17e30dd16f1e5c8db57737fb918c81d56a3d784c33258275e426a1b11e69eb2
… and use it in bench fa576b4 Move MakeNoLogFileContext to common libtest_util, and use it in bench (MarcoFalke) Pull request description: To avoid verbose code duplication, which may lead to accidental mishaps https://github.com/bitcoin/bitcoin/pull/20998/files#r563624041. Also fix a nit I found in bitcoin#20946 (comment). ACKs for top commit: dongcarl: Light Code-Review ACK fa576b4 fanquake: ACK fa576b4 Tree-SHA512: d39ac9c0957813ebb20ed13bd25a8ef8469377ce2651245638bd761c796feac2a17e30dd16f1e5c8db57737fb918c81d56a3d784c33258275e426a1b11e69eb2
1a6323b doc: update developer notes for removal of MakeUnique (fanquake) 3ba2840 scripted-diff: remove MakeUnique<T>() (fanquake) Pull request description: Since requiring C++17, this is just pointless abstraction. I think we should just "tear the band-aid off" and remove it. Similar to the changes happening in #21366. Also, having a comment saying this is deprecated doesn't prevent it's usage in new code. i.e : bitcoin/bitcoin#20946 (comment). The repository is fairly quiet at the moment, so any potential complaints about having to rebase should be minimal. Might as well get this over and done with. ACKs for top commit: jnewbery: utACK 1a6323b practicalswift: cr ACK 1a6323b: patch looks correct ajtowns: ACK 1a6323b -- code review only glozow: ACK bitcoin/bitcoin@1a6323b looks correct Tree-SHA512: 4a14b9611b60b9b3026b54d6f5a2dce4c5d9b63a7b93d7de1307512df736503ed84bac66e7b93372c76e3117f49bf9f29cd473d3a47cb41fb2775bc10234736f
Summary: Previously, the {Basic,}TestingSetup for fuzzers were set up in many ways: 1. Calling InitializeFuzzingContext, which implicitly constructs a static const BasicTestingSetup 2. Directly constructing a static const BasicTestingSetup in the initialize_* function 3. Directly constructing a static TestingSetup and reproducing the initialization arguments (I'm assuming because InitializeFuzzingContext only initializes a BasicTestingSetup) The new, relatively-simple MakeFuzzingContext function allows us to consolidate these methods of initialization by being flexible enough to be used in all situations. It: 1. Is templated so that we can choose to initialize any of the *TestingSetup classes 2. Has sane defaults which are often used in fuzzers but are also easily overridable 3. Returns a unique_ptr, explicitly transferring ownership to the caller to deal with according to its situation This is a backport of [[bitcoin/bitcoin#20946 | core#20946]] [1/2] bitcoin/bitcoin@713314a Note: this backport is a dependency for [[bitcoin/bitcoin#21866 | core#21866]] (removal of global Chainstate) Test Plan: `ninja bitcoin-fuzzers` Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Differential Revision: https://reviews.bitcoinabc.org/D11678
Summary: A full TestingSetup is required for both coins_view and load_external_block_file as they interact with the active chainstate. This is a backport of [[bitcoin/bitcoin#20946 | core#20946]] [2/2] bitcoin/bitcoin@abb6fa7 Depends on D11678 Test Plan: `ninja bitcoin-fuzzers` Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Differential Revision: https://reviews.bitcoinabc.org/D11679
Question for fuzzing people: was it intentional thatAnsweredsrc/test/fuzz/net.cpp
would directly instantiate theBasicTestingSetup
and thus omit the"-nodebuglogfile"
flag?