Use given in __setup__ function

[Pet3ris]

As a StarkNet smart contract developer, I want to use given in the __setup__ function to run a suite of tests on a contract that's been initiated by fuzzed parameters.

In particular, I need to initiate random contract state using fuzzing, however, this initialization is by far the most expensive action in my contract suite. I would like to ideally run it in advance of tests rather than run it separately for each test where I need to use fuzzing.

Is there a way to hack this or do I have to just rely on individual setup functions to use fuzzing?

[mkaput]

Hey @Pet3ris!

We deliberately disabled usage of given and example cheatcodes in __setup__ functions, because we believe that such possibility would lead users to write fuzz tests in a suboptimal way. Fuzzing itself is a pretty heavy and slow process and developers should try not to write many of them, this feature is more suited towards one-off experimentation, and therefore users shouldn't really have a need to share the exact same parameter sets and their strategies across many fuzzed test cases.

But we are open for discussion if you believe our line of thought is not correct 😉

But, the use case you described (having a long initialization step involving contract deployments etc. for several fuzzing test cases) is totally valid, and what's more: it's doable with current Protostar's feature set. Nothing prevents you from using both __setup__ and setup_* hooks! You can do this long initialization in __setup__ and then add separate setup_* functions which only call given, example and max_examples cheatcodes. In order to share values between each other, you can use context as usual.

Example:

@external
func __setup__() {
    %{
        context.contract = deploy(...)
        context.parameter = 0xdeedbeef
    %}
    return;
}

@external
func setup_foo() {
    %{ given(x = strategy.integers(min_value=context.parameter)) %}
    return;
}

@external
func test_foo(x: felt) {
    ...
}

@external
func setup_bar() {
    %{ given(y = strategy.integers(max_value=context.parameter)) %}
    return;
}

@external
func test_bar(y: felt) {
    ...
}

---

I am closing this issue as for now, because I hope this response solves your problem. Feel free to continue discussion if you have any questions or objections! 😃

[Pet3ris]

@mkaput I completely emphatize with the approach you've taken :).

My main challenge is that the code I need to run with fuzzing inputs is sufficiently computationally complex that I don't want to repeat it for every test case (I need to kick off transactions).

Actually this feature helps improve performance rather than decrease it, let me explain it a little better.

• If I have 20 different test cases that need to be initialized the same way
• Let's assume I create 20 more setup_ cases that all contain the same fuzzing initialization code
• Now, effectively the same code will be re-run 20 times to check all 20 test cases
• If the initialization code could go in __setup__, it would only run once and save a lot of time.

In fact, I think this pattern encourages users to use fuzzing more carefully, sharing computation as much as possible.

Alternatively, can I define a custom setup_X function and reuse it across multiple test cases?

[mkaput]

What prevents you from kicking off transactions in __setup__ and calling given in setup_*?

FYI Mind that given only configures the fuzzer. This cheatcode is insta-fast. Initialization of the fuzzer itself happens under the hood when Protostar enters test_* function.

[Pet3ris]

I need to fire off transactions based on the variables in given that are test-independent. It's not the random number generation that is the issue but the need to use those random numbers to pre-configure some contract state that I want to use across multiple test cases.

[Pet3ris]

To give more context, this is what I'm trying to do: https://github.com/a16z/erc4626-tests/blob/main/ERC4626.test.sol.

Note, in Foundry, one can re-use the same fuzzing inputs but cannot do computations after them (hence setUpVault in every test case). Unfortunately in Cairo I think this is much too slow, resulting in a lot of repetitive transactions.

[mkaput]

Oh, now I understand what's going on. Unfortunately, this idea is quite contradictory to the way how Protostar's test runner works under the hood (fuzzer is spun for each test_* function separately, and it wraps single function, setups are running in their own context which is different from test cases ones). Therefore, I don't think we will ever support such use case in Protostar, at least not in this shape of the codebase (the only solution that comes to my mind is adding something like sub-test-cases, which sounds strange?).

As a workaround, I wonder if it wouldn't work for you if you stick with a single test case and eventually chop it into multiple subroutines. Yes, I know this is suboptimal, because you have single test, which runs sequentially and failure in one subroutine stops executing others, but it should still be helpful in successful case.

[Pet3ris]

Thanks @mkaput - yeah I'll consider that, no worries. Completely understand the limitation.