REP: 153 Title: ROS distribution files Author: Dirk Thomas <dthomas@openrobotics.org> Status: Final Type: Standards Track Content-Type: text/x-rst Created: 27-Oct-2018 Post-History: 09-Nov-2018, 01-Oct-2019, 03-Dec-2019
This REP updates the specification of the ROS distribution files facilitated in the building, packaging, testing and documenting process.
The intention is to annotate a ROS distribution with more information, such as whether the distribution should use ROS 1 or ROS 2 semantics, to avoid the need to externally define and update these metadata.
Additionally a flag to enable checking for API/ABI compatibility is introduced.
This REP is an extension to the file format defined in REP 143[1]_. It currently does not repeat the content of REP 143 but only states the differences.
For the following use cases additional metadata is currently necessary:
- The ROS buildfarm has to distinguish ROS 1 distributions from ROS 2
distributions because they require different dependencies to build a
workspace.
E.g. for ROS 1 workspaces
catkin
needs to be installed - even for plain CMake packages - since it provides the workspace level setup files. For ROS 2 the packageros_workspace
is needed since it provides the setup files for the workspace and therefore the environment necessary to find resources. - The ROS wiki shows the list of currently active ROS distributions.
Since the
rosdistro
contains all ROS distros - EOLed ones as well as upcoming distributions before their release date. - A ROS distribution uses either Python 2 or Python 3. When a package is bloomed the rosdep keys are mapped to Debian package names and depending on the targeted Python version the conditional dependencies need to be evaluated.
In all cases the missing metadata is currently being hard coded in the source
code and needs to be updated with every new ROS release.
Therefore this REP aims to add the necessary metadata about a ROS distribution
into the rosdistro
instead.
The stability of the different ROS distributions with respect to API/ABI stability has been part of the work done by ROS maintainers. Until now the changes are manually reviewed and their impact on the API or ABI is being judged by the reviewer with no automated tool to assist with this task.
The motivation of this REP is to ensure that undesired changes in terms of source/binary stability are caught in the review process and maintainers can make decisions with more information about the effects of changes.
The inclusion of an option for testing the API/ABI changes in ROS packages can cover at least two scenarios:
- Run an API/ABI analysis together with changes coming from pull requests. Currently the option to use CI in the pull request is covered by the test_pull_requests (as described in REP 143[1]_)
- Run an API/ABI analysis of changes that appear in the source code of ROS packages. This would be part of the CI cycle present in the devel scripts.
The original target to compare both use cases and to be taken as the stable API/ABI are the latest stable released packages for each ROS distribution under testing.
The new flag to check for ABI compatibility follows the same rationale as the two existing flags to run CI on every commit and/or pull request.
As of this REP unknown keys in the yaml
files should be ignored (instead of
resulting in an error).
This will allow future additions in a backward compatible way without the
need to bump the format version of each file.
- distributions
- distribution:
- distribution_status: an optional string describing the status of a ROS
distribution.
For the use case 2. the semantic of the following values is defined:
prerelease
: An upcoming distribution which hasn't been released yetactive
: A distribution which has been released and is actively being supportedend-of-life
: A distribution which has reached its end of liferolling
: A rolling distribution as described by REP 2002 [8]
- distribution_type: an optional string describing the type of the ROS
distribution.
For use case 1. the values
ros1
andros2
will be used to distinguish the major ROS version. - python_version: an optional integer describing the major version of Python of the ROS distribution.
- distribution_status: an optional string describing the status of a ROS
distribution.
For the use case 2. the semantic of the following values is defined:
- distribution:
- version: version number, this REP describes version 4 (instead of version 3 described in REP 143 [1], version 2 described in REP 141 [2] and version 1 described in REP 137 [3])
- repositories
- source
test_abi
: a boolean flag used to enable API/ABI analysis of the code against the latest released packages. Whentest_commits
is enabled the analysis is performed as part of the devel job. Whentest_pull_requests
is enabled the analysis is performed as part of the pull request job.
- source
- version: version number, this REP still describes version 2 (same as REP 143 [1]). See the compatibility considerations below why a version bump was avoided.
The index file changes have been implemented in version 0.7 of the Python package rosdistro. It serves as a reference implementation for this REP. The draft implementation can be found in [4].
The distribution file changes are implemented in version 0.8 of the Python package rosdistro. The draft implementation can be found in [7].
The draft implementation of rosdistro is able to parse multiple index format versions: 2, 3 as well as 4.
If the version of the existing index.yaml
file would be bumped that would
require every user to update to the latest version of the Python package
rosdistro
.
While generating some friction it is especially a problem on some systems (e.g.
Debian) which might not provide a newer version in stable distributions.
Therefore the existing index file isn't being changed but a sibling file named
index-v4.yaml
is being added.
The file has the same content except that is uses version 4 as specified in
this document and includes the additional metadata fields.
To make use of the new index file the new version of the Python package will update the default URL to point to the v4 file.
rosdistro
version 0.7.5 or newer is necessary to access the
python_version
key.
Older versions of rosdistro
will simply ignore the key in the yaml file and
not expose it through the API.
This provides a smooth transition for all users: users using the old version of
the Python package can continue to use it as is, users updating to the newer
version will benefit from the additional metadata.
Python code using the rosdistro
API can easily check if the metadata is
present and if yes use it.
If desired other Python packages can explicily depend on the newer version to
ensure the v4 index is being used.
The new test_abi
could be added in a new format version 3 of the
distribution file.
That would ensure that implementations of this specification won't break if
they decided to result in an error when unknown keys are found.
The downside of bumping the distribution version would be that existing clients
which haven't updated to the latest version of the rosdistro
Python package
would fail trying to parse that unknown version.
With several EOL ROS distributions not receiving new releases of the Python
package rosdistro
anymore but still being used by many users this breakage
should be avoided.
Therefore the new flag was added to the existing format version 2 instead. The reference implementation is already tolerant to unknown keys and simply ignores them so the newly added flag won't affect existing users.
rosdistro
version 0.8.0 or newer is necessary to access the
test_abi
key.
Older versions of rosdistro
will simply ignore the key in the yaml file and
not expose it through the API.
Beside using the rosdistro
API bloom
also includes an explicit check
for the version of the index file [5].
Therefore a new patch release of bloom
is required to also support the new
version 4 [6].
[1] | (1, 2) REP 143: http://www.ros.org/reps/rep-0143.html |
[2] | REP 141: http://www.ros.org/reps/rep-0141.html |
[3] | REP 137: http://www.ros.org/reps/rep-0137.html |
[4] | Patch to python-rosdistro: ros-infrastructure/rosdistro#124 |
[5] | bloom asserting the index file version: https://github.com/ros-infrastructure/bloom/blob/d8be9d1d3469f00f936ad6e4869b847c5a6f8962/bloom/commands/release.py#L221-L223 |
[6] | bloom PR to support v4 ros-infrastructure/bloom#493 |
[7] | Second patch to python-rosdistro: ros-infrastructure/rosdistro#147 |
[8] | REP 2002: http://www.ros.org/reps/rep-2002.html |
This document has been placed in the public domain.