Skip to content

feat(invariant): allow user to exit forge gracefully during invariant testing #12269

@rappie

Description

@rappie

Component

Forge

Describe the feature you would like

This is especially useful while shrinking. More on this here (Why only seeing fully shrunk sequences can miss bugs).

Currently, forge does not support closing early and exits without any final report when using CTRL+C

drips-fuzzing on  main [!?]
❯ forge test -vv --show-progress --match-test invariant
[⠊] Compiling...
No files changed, compilation skipped
----------------------------------------       0/1       completed (with 32 threads)
⠐ src/echidna/FoundryFuzzing.sol:FoundryFuzzing
    ↪ invariant_worker1: [1015/5000] Runs  Shrink                                                   
^C

drips-fuzzing on  main [!?] took 8s
❯

Ideally, it should show the same report as when exiting after runs and shrink_run_limit are reached. Like this:

src/echidna/FoundryFuzzing.sol:FoundryFuzzing
  ↪ Suite result: FAILED. 0 passed; 1 failed; 0 skipped; finished in 9.60s (9.60s CPU time)

Ran 1 test for src/echidna/FoundryFuzzing.sol:FoundryFuzzing
[FAIL: panic: assertion failed (0x01)]
        [Sequence] (original: 92, shrunk: 4)
                vm.prank(0x0000000000000000000000000000000000010000);
                FoundryFuzzing(0x7FA9385bE102ac3EAc297483Dd6233D62b3e1496).testBalanceAtInFuture(12, 252, 2451);
                vm.prank(0x0000000000000000000000000000000000030000);
                FoundryFuzzing(0x7FA9385bE102ac3EAc297483Dd6233D62b3e1496).setStreamsWithClamping(171, 218, 96286156910612527355223573063248, 14308, 24159, -170141183460469231731687303715884098556);
                vm.prank(0x0000000000000000000000000000000000020000);
                FoundryFuzzing(0x7FA9385bE102ac3EAc297483Dd6233D62b3e1496).addStreamImmediatelySqueezable(3, 0, 1);
                vm.prank(0x0000000000000000000000000000000000040000);
                FoundryFuzzing(0x7FA9385bE102ac3EAc297483Dd6233D62b3e1496).testSetStreamBalanceShouldNotRevert(129, 170141183460469231731687303715884105629);
 invariant_worker1() (runs: 0, calls: 0, reverts: 7)

╭----------------+----------------------------------------------+-------+---------+----------╮
| Contract       | Selector                                     | Calls | Reverts | Discards |
+============================================================================================+
| FoundryFuzzing | addSplitsReceiverWithClamping                | 1     | 0       | 0        |
|----------------+----------------------------------------------+-------+---------+----------|
<snip>
|----------------+----------------------------------------------+-------+---------+----------|
| FoundryFuzzing | testSqueezeViewVsActual                      | 2     | 0       | 0        |
|----------------+----------------------------------------------+-------+---------+----------|
| FoundryFuzzing | testSqueezeWithDefaultHistoryShouldNotRevert | 2     | 0       | 0        |
|----------------+----------------------------------------------+-------+---------+----------|
| FoundryFuzzing | testSqueezeWithFullyHashedHistory            | 3     | 0       | 0        |
|----------------+----------------------------------------------+-------+---------+----------|
| FoundryFuzzing | testSqueezeWithFuzzedHistoryShouldNotRevert  | 2     | 0       | 0        |
╰----------------+----------------------------------------------+-------+---------+----------╯

Suite result: FAILED. 0 passed; 1 failed; 0 skipped; finished in 9.60s (9.60s CPU time)

Ran 1 test suite in 9.60s (9.60s CPU time): 0 tests passed, 1 failed, 0 skipped (1 total tests)

Failing tests:
Encountered 1 failing test in src/echidna/FoundryFuzzing.sol:FoundryFuzzing
[FAIL: panic: assertion failed (0x01)]
        [Sequence] (original: 92, shrunk: 4)
                vm.prank(0x0000000000000000000000000000000000010000);
                FoundryFuzzing(0x7FA9385bE102ac3EAc297483Dd6233D62b3e1496).testBalanceAtInFuture(12, 252, 2451);
                vm.prank(0x0000000000000000000000000000000000030000);
                FoundryFuzzing(0x7FA9385bE102ac3EAc297483Dd6233D62b3e1496).setStreamsWithClamping(171, 218, 96286156910612527355223573063248, 14308, 24159, -170141183460469231731687303715884098556);
                vm.prank(0x0000000000000000000000000000000000020000);
                FoundryFuzzing(0x7FA9385bE102ac3EAc297483Dd6233D62b3e1496).addStreamImmediatelySqueezable(3, 0, 1);
                vm.prank(0x0000000000000000000000000000000000040000);
                FoundryFuzzing(0x7FA9385bE102ac3EAc297483Dd6233D62b3e1496).testSetStreamBalanceShouldNotRevert(129, 170141183460469231731687303715884105629);
 invariant_worker1() (runs: 0, calls: 0, reverts: 7)

Encountered a total of 1 failing tests, 0 tests succeeded

Tip: Run `forge test --rerun` to retry only the 1 failed test

Additional context

No response

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

Status

Completed

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions