Skip to content

feat(invariant): show real-time results while shrinking #12268

@rappie

Description

@rappie

Component

Forge

Describe the feature you would like

Seeing shrinking happen in realtime has multiple advantage:

  1. Seeing how the sequence is shrunk can help give you intuition of what is going on behind the scenes
  2. Knowing when you can cancel shrinking
  3. Spotting potential bugs that are missed when fully shrunk (more on that below)

Showing the number of transactions in the sequence while shrinking helps, but ideally you'll want to see the entire sequence in realtime as it is being shrunk. See below for a screenshot of what it looks like in Echidna.

Why only seeing fully shrunk sequences can miss bugs

Let's use a (slightly oversimplified) example to illustrate this. We have a deposit function that can revert in two ways:

  1. When depositing a larger amount, it overflows (the real bug)
  2. When depositing 0, it reverts because zero deposits are not accepted

For our invariant we are testing "deposit does not revert".

When the fuzzer breaks this invariant with a large number, it will trigger bug 1. While shrinking, it will keep trying smaller amounts and eventually reach 0, which still breaks the invariant but for a different reason (bug 2). If we would only ever see the fully shrunk result, we would miss bug 1.

Additional context

Image

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    Status

    Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions