From 01dd39abe97120aa164f8445b831fdf5fcb54501 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ren=C3=A9=20Fischer?= Date: Sat, 25 Feb 2023 15:18:00 +0100 Subject: [PATCH 1/5] Add parallel-fuzzing option to docs --- docs/running_clusterfuzzlite.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/docs/running_clusterfuzzlite.md b/docs/running_clusterfuzzlite.md index 0771c6e..e350895 100644 --- a/docs/running_clusterfuzzlite.md +++ b/docs/running_clusterfuzzlite.md @@ -153,6 +153,13 @@ specific CI system. coverage report generation). The default is `'address'`. See [Sanitizers] for more information. +- `parallel-fuzzing`: Whether to use all available CPU cores for fuzzing. + The default value is `false`, which only uses a single CPU core. + When set to `true`, ClusterFuzzLite runs multiple fuzzer processes + in parallel with a shared corpus directory. New inputs found by one + fuzzer process will be available to the other fuzzer processes. + The number of cores available depends on your specific CI system. + - `mode`: The mode for ClusterFuzzLite to execute. `code-change` by default. See [ClusterFuzzLite modes] for more details on how to run different modes. From 32c747250730d718557193ace7bbe69a7ecdb12d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ren=C3=A9=20Fischer?= Date: Sat, 25 Feb 2023 15:41:46 +0100 Subject: [PATCH 2/5] Document more options --- docs/running_clusterfuzzlite.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/docs/running_clusterfuzzlite.md b/docs/running_clusterfuzzlite.md index e350895..2674179 100644 --- a/docs/running_clusterfuzzlite.md +++ b/docs/running_clusterfuzzlite.md @@ -160,6 +160,19 @@ specific CI system. fuzzer process will be available to the other fuzzer processes. The number of cores available depends on your specific CI system. +- `report-unreproducible-crashes`: Whether to report unreproducible crashes. + The default value is `false`. ClusterFuzzLite will always attempt to reproduce + new crashes. Sometimes crashes can not be reproduced reliably, e.g., + because the fuzz target is in a different state. Read more on + non-reproducable bugs in + [this section](https://google.github.io/oss-fuzz/advanced-topics/bug-fixing-guidance#non-reproducible-bugs). + When set to `true`, even non-reproducable bugs will be reported a + as failure. + +- `minimize-crashes`: If `true`, reportable crashes will be minimized. + The default value is `false`. Minimizing crashes reduces fuzzing time + in batch fuzzing. + - `mode`: The mode for ClusterFuzzLite to execute. `code-change` by default. See [ClusterFuzzLite modes] for more details on how to run different modes. From eefb8c025136b74c122e4d6ec818e394eae0e065 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ren=C3=A9=20Fischer?= Date: Sat, 25 Feb 2023 15:54:13 +0100 Subject: [PATCH 3/5] Add options to build integrations --- docs/running-clusterfuzzlite/github_actions.md | 2 ++ docs/running-clusterfuzzlite/gitlab.md | 1 + 2 files changed, 3 insertions(+) diff --git a/docs/running-clusterfuzzlite/github_actions.md b/docs/running-clusterfuzzlite/github_actions.md index 08587a5..996eb02 100644 --- a/docs/running-clusterfuzzlite/github_actions.md +++ b/docs/running-clusterfuzzlite/github_actions.md @@ -102,6 +102,7 @@ Optionally, edit the following fields to customize your settings: - `language` Change to the language of your target code. - `sanitizers` Change or enable more sanitizers. - `fuzz-seconds` Change the amount of time spent fuzzing. +- `parallel-fuzzing`: Use all available CPU cores for fuzzing. - `storage-repo`, `storage-repo-branch`, `storage-repo-branch-coverage` Enable a [storage repo] (not necessary for initial runs, but a useful feature discussed [later on]). @@ -168,6 +169,7 @@ Optionally, edit the following fields to customize your settings: - `cron` Change how frequently batch fuzzing is run. See [GitHub's documentation] on this. - `sanitizers` Change or enable more sanitizers. - `fuzz-seconds` Change the amount of time spent fuzzing. +- `parallel-fuzzing`: Use all available CPU cores for fuzzing. - `storage-repo`, `storage-repo-branch`, `storage-repo-branch-coverage` Enable a [storage repo]. diff --git a/docs/running-clusterfuzzlite/gitlab.md b/docs/running-clusterfuzzlite/gitlab.md index e93a9d9..fab33b7 100644 --- a/docs/running-clusterfuzzlite/gitlab.md +++ b/docs/running-clusterfuzzlite/gitlab.md @@ -87,6 +87,7 @@ Optionally, edit the following variables to customize your settings: - `CFL_BRANCH` Branch to fuzz, default is `CI_DEFAULT_BRANCH`. - `FILESTORE` storage for files: builds, corpus, coverage and crashes. - `FUZZ_SECONDS` Change the amount of time spent fuzzing. +- `PARALLEL_FUZZING` Use all available cores when fuzzing. - `CFL_ARTIFACTS_DIR` To save your artifacts in a different directory than `artifacts` ### Batch fuzzing and corpus pruning From 3f93aee9d4850052b6abe174b4ca16f6624ac128 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ren=C3=A9=20Fischer?= Date: Sat, 25 Feb 2023 18:25:50 +0100 Subject: [PATCH 4/5] Small fixes to gitlab.md --- docs/running-clusterfuzzlite/gitlab.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/running-clusterfuzzlite/gitlab.md b/docs/running-clusterfuzzlite/gitlab.md index fab33b7..017f06a 100644 --- a/docs/running-clusterfuzzlite/gitlab.md +++ b/docs/running-clusterfuzzlite/gitlab.md @@ -137,8 +137,8 @@ keyword to avoid duplicating most of the common parameters between the different The continuous build task causes a build to be triggered and uploaded whenever a new push is done to main/default branches. -Continuous builds are used when a crash is found during PR fuzzing to determine whether the crash was newly introduced. -If the crash was not newly introduced, PR fuzzing will not report it. +Continuous builds are used when a crash is found during MR fuzzing to determine whether the crash was newly introduced. +If the crash was not newly introduced, MR fuzzing will not report it. This means that there will be fewer unrelated failures when running code change fuzzing. @@ -213,7 +213,7 @@ From a performance point of view, it is recommended to use a `docker` gitlab run See this [doc](https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#use-docker-socket-binding) for more information. -To do so, if you have such a runner ready, you simply need to remove the following lines from the configuration : +To do so, if you have such a runner ready, you simply need to remove the following lines from the configuration: {% raw %} ```yaml variables: @@ -250,7 +250,7 @@ For continuous builds, you need to use a [cache](https://docs.gitlab.com/ee/ci/c - cfl-cache/ ``` {% endraw %} -The cache directory needs to defined as `CFL_CACHE_DIR` to be used by ClusterFuzzLite. +The cache directory needs to be defined as `CFL_CACHE_DIR` to be used by ClusterFuzzLite. If it is not defined, the default value is `cache`. You should ensure that the runners share the access to the cache. @@ -264,7 +264,7 @@ project access token, due to your Gitlab license. ![gitlab-project-token] -And this token should be used from the fuzzed repository as a CI/CD variable. +Add the token as a CI/CD variable to your GitLab project. You can name this variable as you like, in the following example it is named `CFL_TOKEN`. This variable should be defined as masked to avoid leaks. It should not be protected if you need it on unprotected branches. From c633c4a1e94dda3f556528ba9ebf37f28bed0f5d Mon Sep 17 00:00:00 2001 From: jonathanmetzman <31354670+jonathanmetzman@users.noreply.github.com> Date: Mon, 27 Feb 2023 12:10:10 -0500 Subject: [PATCH 5/5] Update running_clusterfuzzlite.md --- docs/running_clusterfuzzlite.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/running_clusterfuzzlite.md b/docs/running_clusterfuzzlite.md index 2674179..928cd8b 100644 --- a/docs/running_clusterfuzzlite.md +++ b/docs/running_clusterfuzzlite.md @@ -164,7 +164,7 @@ specific CI system. The default value is `false`. ClusterFuzzLite will always attempt to reproduce new crashes. Sometimes crashes can not be reproduced reliably, e.g., because the fuzz target is in a different state. Read more on - non-reproducable bugs in + non-reproducible bugs in [this section](https://google.github.io/oss-fuzz/advanced-topics/bug-fixing-guidance#non-reproducible-bugs). When set to `true`, even non-reproducable bugs will be reported a as failure.