Evaluate Gitlab CI Fuzz testing

[tamirms]

Gitlab CI has introduced support for fuzz testing as a service. Their service integrates with go-fuzz which is the library we are using to fuzz test Horizon.

Gitlab supports fuzzing jobs which execute for a user defined timeout and they also support continuous fuzzing which run for a much longer duration. It seems that short fuzzing jobs would be appropriate to run against PRs and continuous fuzzing would be useful to run against master or Horizon release branches.

One useful aspect of Gitlab's fuzzing service is that whenever you run a fuzzing job Gitlab will extend the fuzz corpus with any new interesting cases that are found during the test run. The corpus is shared between multiple test runs and each test run can possibly improve the corpus.

If we were to use Gitlab fuzzing it seems that we would need to have CI job defined for every fuzzing target. This is an important point to consider because we plan on having fuzz tests for every ingestion processor. But we may run into issues by having so many fuzz jobs running for each PR.

To evaluate Gitlab's fuzzing service we should verify that it's possible to run Gitlab CI on GitHub repos (according to https://about.gitlab.com/solutions/github/ it should be possible). Then we can start by having a fuzzing job which invokes the claim predicate fuzz test

[tamirms]

My initial tests with Gitlab's fuzzing service have been promising. I was able to get a fairly substantial prototype up and running.

There are still some questions we need to figure out which I have listed in the checklist below. We need to check off all the high priority and medium priority tasks before we can close the issue and move on to writing fuzz targets for Horizon ingestion processors.

• Trigger email notifications whenever a fuzz job finds a crash (high priority)
• Make the visibility of security issues discovered by fuzz jobs private to only Horizon team members (high priority)
  • You can either make the Gitlab repo private or you can go to Settings > General and set the visibility of the Pipelines and Security & Compliance pages to only project members.
• Discuss creating an SDF Gitlab account with Ops team (medium priority)
• Streamline defining gitlab CI jobs for each fuzz target (optional / low priority)

[naveensrinivasan]

https://github.com/google/oss-fuzz provides the fuzzing service which will provide all the things that Gitlab provides.

It is easy to integrate.

[naveensrinivasan]

I can help with this.