Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multicore #963

Merged
merged 13 commits into from
May 5, 2023
Merged

Multicore #963

merged 13 commits into from
May 5, 2023

Conversation

arcz
Copy link
Member

@arcz arcz commented Feb 24, 2023

To test, add --workers N to echidna command or workers: N to config.
This PR also adds --timeout <seconds> to CLI which was available only from the yaml config.

Notable code changes:

  • Campaign type is now WorkerState with less data. Tests, coverage and corpus are now IORefs that live in Env and the data is shared between all workers. The contention on this data is minimal as modifications rarely happen besides the very beginning when coverage grows rapidly.
  • There is a shared event queue that uses Chan. Workers push events when something interesting happens. Those events drive the UIs.
  • UI module has been reworked to work with multiple workers that can stop at different times.
  • Open tests don't count calls, it was the same for all open tests and was moved to WorkerState. The call number is now displayed at the top of TUI.
  • The number of functions that use IO has grown because it's easier to grab data straight from Env. This is something that can be improved in the future.

Things to think about:

  • At the moment, all workers replay the corpus but this is redundant work. Either split corpus across workers to replay or replay it first and then start workers. Consider calls to external contracts that might not be yet deployed. Relevant Show a proper error message when an invalid corpus is used #1008 Corpus is now chunked and distributed across workers to replay.
  • There is a lot of contention while shrinking because all workers try to shrink. This can be improved by marking Large test with a workerId that falsified the test and only this worker shrinks the test.

Closes #476

@arcz arcz marked this pull request as draft February 24, 2023 21:40
@arcz arcz added this to the Echidna 2.2.0 milestone Mar 31, 2023
@arcz arcz force-pushed the multicore branch 2 times, most recently from bc482a0 to 56ca072 Compare April 11, 2023 16:35
@arcz arcz force-pushed the multicore branch 7 times, most recently from bab2f7f to f09c7da Compare April 19, 2023 11:46
@arcz arcz force-pushed the multicore branch 10 times, most recently from 6435ec6 to f184321 Compare April 26, 2023 19:23
@arcz arcz force-pushed the multicore branch 3 times, most recently from 937b210 to 736a26c Compare April 27, 2023 23:12
@arcz arcz changed the title Multicore WIP Multicore Apr 28, 2023
@arcz arcz marked this pull request as ready for review April 28, 2023 11:45
@arcz arcz requested a review from ggrieco-tob as a code owner April 28, 2023 11:45
@arcz arcz merged commit 87449e9 into master May 5, 2023
16 checks passed
@arcz arcz deleted the multicore branch May 5, 2023 17:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

multicore fuzzing configurable with jobs count
1 participant