For an overview about the ROS build farm including how to deploy the necessary machines see the ROS wiki page.
The ROS build farm performs several different jobs. For each job type you will find a detailed description what they do and how they work.
- release jobs generate binary packages
- devel jobs build and test ROS packages within a single repository
- CI jobs build and test ROS packages across repositories with the option of using artifacts from other CI jobs to speed up the build
- doc jobs generate the API documentation of packages and extract information from the manifests
- miscellaneous jobs perform maintenance tasks and generate informational data to visualize the status of the build farm and its generated artifacts
The configuration is stored in a separate repository (e.g. ros_buildfarm_config for the official ROS build farm).
Every ROS build farm requires a custom configuration to specify URLs, credentials, and notification settings. Please see the list of the basic configuration options which need to be modified.
Beside the use case focused descriptions below a reference listing all configuration options is available.
The configuration options to change:
- which packages / repositories to process
- which platforms and architectures to target
are described here.
In the case when you want to add packages and repositories to the rosdistro database and / or modify the version number of released packages (e.g. lock the version number of certain packages) you must maintain your own rosdistro database.
Please see the documentation about using a custom rosdistro database for the necessary configuration options and administrative tasks.
The YAML loader used to load the configuration implements a special constructor used to load the YAML content from another file. This mechanism can be used to include configuration elements from a separate file as if it were embedded into another file. The special directive takes a single URL argument, which may either be relative to the current URL or absolute.
For example: large_block: !include other_file.yaml
After creating a custom configuration repository you can deploy the configuration which includes:
- generating the administrative Jenkins jobs
- triggering a few jobs to perform necessary initialization steps
- generate all release / devel / doc jobs
After the initial deployment the administrator should continuously monitor the build farm and perform some manual task in regular intervals.
Some of the job types can be easily run locally. This allows to reproduce the behavior of the build farm locally, eases debugging, and shortens the round-trip time for testing. It also means that you can run any of these jobs anywhere, e.g. on Travis.
- release jobs#run-the-release-job-locally generate binary packages
- devel jobs#run-the-devel-job-locally build and test ROS repositories
- devel jobs#run-the-devel-job-on-travis build and test ROS repositories using Travis
- doc jobs#run-the-doc-job-locally generate the documentation for ROS repositories
Another job type can be used locally which is not offered on the build farm.
- prerelease jobs build and test ROS repositories as well as build and test released ROS packages depending on them
If you are going to be running one or more jobs on any machine we recommend using squid-in-a-can to cache downloads. It can greatly speed up download times and saves a lot of bandwidth. It's used by all our developers as well as on all the build machines.
Beyond the administrative requirements for generating jobs, the only noteworth software for executing jobs whether locally or via Jenkins is a container engine compatible with the Docker client CLI. Currently, the only engines tested with ros_buildfarm are the Docker CE or rootless Podman.