Speedup your CI

There is an article in Russian about speeding up building and testing in CI. This page contains a list of tools that helps with speedup.

CI/CD Pipelines are the fundamental building blocks for CI/CD. Making pipelines more efficient helps you save developer time, which:

Speeds up your DevOps processes
Reduces costs
Shortens the development feedback loop

It’s common that new teams or projects start with slow and inefficient pipelines, and improve their configuration over time through trial and error. A better process is to use pipeline features that improve efficiency right away, and get a faster software development lifecycle earlier.

Source code

Code layout Good structure of files in a project helps to make proper dependencies. The rule of thumb is putting tests on the same level with source code that these tests covers. See recommendations for CMake in How to structure your project.
Replace asynchronous synchronization (sleep()) with synchronous one in tests and production source code.

Fast source code checkout

Use --depth N in Git.
Enable parallel mode in Git:
- git config fetch.parallel 0
- git config submodule.fetchJobs 0

Hints with CI setup

Profile your CI to eliminate bottlenecks:
- github-actions-profile - is a profiler for GitHub Actions workflow.
Analyze the performance of your pipeline to find ways to improve efficiency.
- Spendesk/github-actions-exporter - github-actions-exporter for Prometheus.
- mvisonneau/gitlab-ci-pipelines-exporter - Prometheus / OpenMetrics exporter for GitLab CI pipelines insights.
- cpanato/github_actions_exporter - Prometheus exporter exposing GitHub Actions metrics.
- Gitlab CI helps with pipeline efficiency.
Ignoring paths with non-relevant files
- Github CI: paths-ignore (Documentation)
- Gitlab CI: rules:changes (Documentation)
Use caching dependencies
- Github Actions (Documentation)

Hints with OS setup

Disable unnecessary triggers in packages (Processing triggers for man-db (2.9.4-2) ...).
- Debian - https://wiki.debian.org/DpkgTriggers

Compilation and linking

Profile build system to eliminate bottlenecks:
- ninjatracing - converts .ninja_log files to chrome's about:tracing format.
- buildbloat converts ninja build logs to webtreemap JSON files.
- Debugging Build Performance
Use parallel mode.
- Make - option -j
- Ninja - option -j
- Meson - option -j
- CMake - https://www.kitware.com//cmake-building-with-all-your-cores/
Use fast build system. In some projects, Ninja could be faster than Make. CMake supports both build systems, so you could compare build time for both and choose the fastest. See build time comparison.
Replace default linker with Mold. Faster in 17 times in comparison with GNU gold and 3-5 times in comparison with LLVM lld.
Include what you use - is a tool for use with clang to analyze #includes in C and C++ source files.
Use cache: -ccache - is a compiler cache. ccache could speed up your build in 30 (!) times. See performance results.
- sccache - is ccache with cloud storage. Supported C, C++ and Rust.
- firebuild - is an automatic build accelerator. It works by caching the outputs of executed commands and replaying the results when the same commands are executed with the same parameters within the same environment.
Use distributed compilation (boost 2-4 times)
- Goma (C/C++) - is a distributed compiler service for open-source project such as Chromium and Android. It's some kind of replacement of distcc+ccache. Used by Google.
- nocc (C/C++) - is distributed C++ compiler. Used by VK.
- distcc is a fast, free distributed C/C++ compiler. See performance results.
- icecream - is a distributed compiler with a central scheduler to share build load. Created by SUSE.

Running unit tests

Use distributed running
- gg - the Stanford Builder. Uses lambdas for running self-contained binaries, see a paper From Laptop to Lambda: Outsourcing Everyday Jobs to Thousands of Transient Functional Containers.
- llama - is a tool for running UNIX commands inside of AWS Lambda. Its goal is to make it easy to outsource compute-heavy tasks to Lambda, with its enormous available parallelism, from your shell.