feat: add a --no-fuzz or --skip-fuzz flag to skip fuzz tests

[mds1]

Component

Forge

Describe the feature you would like

This is useful as you don't always need to run fuzz tests, e.g. when iterating on something, and this can significantly speed up tests. Especially if the fuzz takes make RPC calls, for example.

Right now we always pass in true for the value of include_fuzz_tests, so this should be simple to expose with a flag / config item:

foundry/cli/src/cmd/forge/test.rs

Line 324 in 309cfe7

custom_run(self, true)

foundry/cli/src/cmd/forge/test.rs

Line 460 in 309cfe7

pub fn custom_run(args: TestArgs, include_fuzz_tests: bool) -> eyre::Result<TestOutcome> {

Workarounds:

• Test naming conventions like function test_* and function testFuzz_* so you can use the match flags
• Set the number of fuzz runs to 1 (or even 0, if that works, I haven't tested it)

Additional context

No response

[mattsse]

that makes sense

this would also be a good time refactoring the TestFilter type a bit

there was a recent issue #1908 that highlighted a problem with the Filter.

ideally, everything should be merged from the Config Config -> Filter.

so it would be preferable if we could move that type to config entirely

[onbjerg]

This is most likely a breaking change since the default behavior for forge test is to include fuzz tests, and the default behavior for forge snapshot (that piggybacks off of forge test) is to exclude fuzz tests. I'm not sure how to best unify given Clap doesn't give us a way to override defaults of flattened args :/

The non-breaking path would be to just ignore --no-fuzz/--skip-fuzz in forge snapshot and set those values to !include_fuzz_tests

[onbjerg]

Re @mattsse's comment: ref #1088

[onbjerg]

This is easier to do now that we don't have --include-fuzz-tests on forge snapshot

[mds1]

FWIW I don't feel too strongly about this feature and am ok with closing this as "not planned". Setting fuzz runs to zero does seem to work as shown below, so I'm not sure this is worth the API clutter

$ FOUNDRY_FUZZ_RUNS=0 forge test

[⠰] Compiling...
No files changed, compilation skipped

Running 1 test for test/MyContract.t.sol:ContractTest
[PASS] testSomething(address,uint224) (runs: 0, μ: 0, ~: 0)
Test result: ok. 1 passed; 0 failed; finished in 833.35µs

[mds1]

Closing since this can be achieved with zero fuzz runs, or with a good test naming convention and skipping tests with fuzz in the name