diff --git a/docs/Gemfile.lock b/docs/Gemfile.lock index 2056105..a0cf830 100644 --- a/docs/Gemfile.lock +++ b/docs/Gemfile.lock @@ -219,17 +219,17 @@ GEM rb-fsevent (~> 0.10, >= 0.10.3) rb-inotify (~> 0.9, >= 0.9.10) mercenary (0.3.6) - mini_portile2 (2.6.1) + mini_portile2 (2.8.1) minima (2.5.1) jekyll (>= 3.5, < 5.0) jekyll-feed (~> 0.9) jekyll-seo-tag (~> 2.1) minitest (5.14.4) multipart-post (2.1.1) - nokogiri (1.12.2) - mini_portile2 (~> 2.6.1) + nokogiri (1.14.0) + mini_portile2 (~> 2.8.0) racc (~> 1.4) - nokogiri (1.12.2-x86_64-linux) + nokogiri (1.14.0-x86_64-linux) racc (~> 1.4) octokit (4.21.0) faraday (>= 0.9) @@ -237,7 +237,7 @@ GEM pathutil (0.16.2) forwardable-extended (~> 2.6) public_suffix (4.0.6) - racc (1.5.2) + racc (1.6.2) rb-fsevent (0.11.0) rb-inotify (0.10.1) ffi (~> 1.0) @@ -263,7 +263,7 @@ GEM thread_safe (0.3.6) typhoeus (1.4.0) ethon (>= 0.9.0) - tzinfo (1.2.9) + tzinfo (1.2.10) thread_safe (~> 0.1) unf (0.1.4) unf_ext diff --git a/docs/running-clusterfuzzlite/gitlab.md b/docs/running-clusterfuzzlite/gitlab.md index 4cda1a3..e93a9d9 100644 --- a/docs/running-clusterfuzzlite/gitlab.md +++ b/docs/running-clusterfuzzlite/gitlab.md @@ -18,10 +18,8 @@ To get the most of this page, you should have already set up your [build integration] and read the more [high-level document on running ClusterFuzzLite]. -## Gitlab runner -The following examples use a `docker` gitlab runner running sibling containers: -See this [doc](https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#use-docker-socket-binding) -for more information. +The following documentation is primarily meant for Gitlab.com with the use of shared runners. +It works also for self-managed Gitlab instances. ## .gitlab-ci.yml For basic ClusterFuzzLite functionality, all you need is a single job @@ -38,7 +36,7 @@ To enable more features, we recommend having different jobs for: To add a fuzzing job that fuzzes all merge requests to your repo, add the following default configurations to `.gitlab-ci.yml`: -This configuration requires at least GitLab 13.3 to be run. +For self-managed Gitlab instances, this configuration requires at least GitLab 13.3 to be run. With older versions, the `parallel` keywords does not exist, but you can define `SANITIZER` as a Gitlab CI variable. {% raw %} @@ -46,8 +44,8 @@ With older versions, the `parallel` keywords does not exist, but you can define variables: SANITIZER: address CFL_PLATFORM: gitlab - DOCKER_HOST: "tcp://docker:2375" - DOCKER_IN_DOCKER: "true" + DOCKER_HOST: "tcp://docker:2375" # may be removed in self-managed Gitlab instances + DOCKER_IN_DOCKER: "true" # may be removed in self-managed Gitlab instances clusterfuzzlite: @@ -55,7 +53,7 @@ clusterfuzzlite: name: gcr.io/oss-fuzz-base/clusterfuzzlite-run-fuzzers:v1 entrypoint: [""] services: - - docker:dind + - docker:dind # may be removed in self-managed Gitlab instances stage: test parallel: @@ -80,7 +78,8 @@ clusterfuzzlite: ``` {% endraw %} -You may also wish to set [tags](https://docs.gitlab.com/runner/#tags) to select a relevant runner. +For self-managed Gitlab instances, you may also wish to set [tags](https://docs.gitlab.com/runner/#tags) +to select a relevant runner. Optionally, edit the following variables to customize your settings: - `SANITIZER` Select sanitizer(s) @@ -106,7 +105,7 @@ clusterfuzzlite-corpus: name: gcr.io/oss-fuzz-base/clusterfuzzlite-run-fuzzers:v1 entrypoint: [""] services: - - docker:dind + - docker:dind # may be removed in self-managed Gitlab instances stage: test rules: - if: $MODE == "prune" @@ -127,6 +126,9 @@ In one, you should set the variable `MODE` to `batch` to run the actual batch fu In the other, you should set the variable `MODE` to `prune` for corpus pruning once a day. These schedules should target the main/default/`CFL_BRANCH` branch. +Note that you can also use gitlab-ci.yml [extends](https://docs.gitlab.com/ee/ci/yaml/#extends) +keyword to avoid duplicating most of the common parameters between the different type of jobs. + ![gitlab-schedule-mode] ### Continuous builds @@ -148,7 +150,7 @@ clusterfuzzlite-build: name: gcr.io/oss-fuzz-base/clusterfuzzlite-run-fuzzers:v1 entrypoint: [""] services: - - docker:dind + - docker:dind # may be removed in self-managed Gitlab instances stage: test rules: # Use $CI_DEFAULT_BRANCH or $CFL_BRANCH. @@ -179,7 +181,7 @@ clusterfuzzlite-coverage: name: gcr.io/oss-fuzz-base/clusterfuzzlite-run-fuzzers:v1 entrypoint: [""] services: - - docker:dind + - docker:dind # may be removed in self-managed Gitlab instances stage: test variables: SANITIZER: "coverage" @@ -202,6 +204,30 @@ This schedule should target the main/default/`CFL_BRANCH` branch. ## Extra configuration +### Gitlab runners on self-managed Gitlab instances + +The previous examples used [Docker in docker](https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#use-docker-in-docker) + +From a performance point of view, it is recommended to use a `docker` gitlab runner running sibling containers: +See this [doc](https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#use-docker-socket-binding) +for more information. + +To do so, if you have such a runner ready, you simply need to remove the following lines from the configuration : +{% raw %} +```yaml +variables: + DOCKER_HOST: "tcp://docker:2375" # may be removed in self-managed Gitlab instances + DOCKER_IN_DOCKER: "true" # may be removed in self-managed Gitlab instances + + services: + - docker:dind # may be removed in self-managed Gitlab instances +``` + +Note that it should be possible to achieve the same functionality using a shell +executor, though this is unsupported. +In that case, the `.gitlab-ci.yml` will different. For one you must explicitly +call the `docker` commands on ClusterFuzzLite images. + ### Gitlab filestore You can use the variable `FILESTORE: gitlab` to use Gitlab artifacts for storing @@ -232,11 +258,15 @@ git repository for storage. You need to create a project access token for this repository, with `read_repository` and `write_repository` rights. +You can also use a personal access token if you do not have access to +project access token, due to your Gitlab license. + ![gitlab-project-token] And this token should be used from the fuzzed repository as a CI/CD variable. You can name this variable as you like, in the following example it is named `CFL_TOKEN`. This variable should be defined as masked to avoid leaks. +It should not be protected if you need it on unprotected branches. ![gitlab-variable-token] @@ -267,12 +297,6 @@ This job will build a static web site with everything which is in the `public` d You may then access the site at `https://baseurl/coverage/latest/report/linux/report.html` where `baseurl` is the domain you configured for your GitLab pages. -## Shell executor -Note that it should be possible to achieve the same functionality using a shell -executor, though this is unsupported. -In that case, the `.gitlab-ci.yml` will different. For one you must explicitly -call the `docker` commands on ClusterFuzzLite images. - [GitLab]: https://about.gitlab.com/ [build integration]: {{ site.baseurl }}/build-integration/ [high-level document on running ClusterFuzzLite]: {{ site.baseurl }}/running-clusterfuzzlite/