Fuzzing and Auditing roadmap

[zah]

Our fuzzing efforts have to take into consideration the possible target dates for shipping a production-ready ETH2 phase 0 client and an ETH1 client. All components must first go through continuous fuzzing and later, shortly before shipping, through a security audit by an external agency.

Since the external security audit will limit our ability to refactor the code significantly and to pursue further optimizations, we must strive to create a pipeline where we'll be able to test, optimize, fuzz and audit the individual smaller components, gradually combining them into larger and larger pieces.

This document aims to provide a suitable order for carrying out the optimization and fuzzing efforts. To determine the best order, we must first profile the code to determine which components will need to go through significant optimization.

• Create and document a framework for writing fuzzing tests.
• Extract our fuzzing framework into a reusable package (Extract the fuzz-testing tool from nim-eth and move it here nim-testutils#3)
• Implement the APIs needed by Sigma Prime's differential Eth2 fuzzer.
• Select and engage potential partners for the security audit.

WIP Order:

• Nimcrypto (select components)
• Faststreams
• SSZ
• BLS
• Shuffling
• State transition
• Attestation pool
• Block pool
• Fork-choice

The Sigma Prime team has a significant fuzzing expertise and they've spent time refactoring their beacon node implementation to make it more suitable for fuzzing. It may be worthwhile to study their approaches and to optimize our codebase for these purposes as well.

[arnetheduck]

Fuzzing of the spec parts will be part of a generalized client fuzzing framework being developed by sigma prime - in general the idea is that each client provides a client library with a set of C functions exported for the fuzzer to call - before developing a nim-fuzzer library we should probably evaluate what we can reuse from their approach.

[zah]

nim-fuzzer is a very specific tool that takes care of calling Nim in the right way in order to compile and run the fuzzing tests written in the framework prepared by @kdeme. Currently, it exists as a set of nims scripts in the nim-eth repo, but this leaves some open questions regarding reusing it in other projects. Nevertheless, we might still postpone creating a separate tool and instead we can rely on imports featuring paths such as "vendor/nim-eth".

[arnetheduck]

https://github.com/sigp/eth2.0-fuzzing is where the tool is being developed

[zah]

One of our goals for June would be to formulate a specific audit plan and then document it here.

[mratsim]

I think https://github.com/status-im/nim-testutils and the ongoing audit closes this issue.