Forge vs. Echidna testing

Note Failing CI is intended here.

Sample contract to test Forge vs. Echidna's fuzzing mechanisms for @transmissions11.

The objective is to find two inputs equal to the magic number 0x69 to supply to Foo's foo(uint256) and bar(uint256) functions. If both foo and bar are called with an input of 0x69, revertIfCracked will fail the test.

Both fuzzers are able to find the revert case consistently for Foo:

• Forge finds the revert case consistently with an invariant_* test.
  • Forge fails to do so roughly ~80-90% of the time in fuzz mode. (Note, for the case in this repo, an invariant style test is better suited anyways)
• Echidna finds the revert case consistently.

However, an interesting observation is that when using bytes32 rather than uint256 as inputs for foo and bar (see: FooBytes.sol), forge is significantly more reliable at finding the revert case than Echidna.

Versions

Forge: 18791dfa2f8c5ea7df99cc7bc6e00e1d20c02499

Echidna: 2.0.4

Foo cases:

Forge

Invariants:

Fuzzing:

Echidna

Cmd: echidna-test ./test/EchidnaFoo.sol --contract EchidnaFoo --test-mode property

FooBytes cases:

Forge

Echidna

Echidna fails to detect the revert case for the FooBytes version of this test case: