Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update rmw_fastrtps_cpp and rmw_fastrtps_shared_cpp QDs to QL2. #456

Merged
merged 2 commits into from
Sep 30, 2020
Merged

Update rmw_fastrtps_cpp and rmw_fastrtps_shared_cpp QDs to QL2. #456

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
60b1495
Update rmw_fastrtps_cpp and rmw_fastrtps_shared_cpp QDs to QL2.
hidmic Sep 30, 2020
f035c81
Address peer review comments.
hidmic Sep 30, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 15 additions & 5 deletions rmw_fastrtps_cpp/QUALITY_DECLARATION.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@ This document is a declaration of software quality for the `rmw_fastrtps_cpp` pa

# `rmw_fastrtps_cpp` Quality Declaration

The package `rmw_fastrtps_cpp` claims to be in the **Quality Level 3** category.
The package `rmw_fastrtps_cpp` claims to be in the **Quality Level 2** category.
chapulina marked this conversation as resolved.
Show resolved Hide resolved

Below are the rationales, notes, and caveats for this claim, organized by each requirement listed in the [Package Requirements for Quality Level 3 in REP-2004](https://www.ros.org/reps/rep-2004.html).
Below are the rationales, notes, and caveats for this claim, organized by each requirement listed in the [Package Requirements for Quality Level 2 in REP-2004](https://www.ros.org/reps/rep-2004.html).

## Version Policy [1]

Expand Down Expand Up @@ -86,11 +86,18 @@ The results of the test can be found [here](https://ci.ros2.org/view/nightly/job

### Feature Testing [4.i]

All `rmw_fastrtps_cpp` public features are ROS middleware features. Integration and system tests up the stack, such as those found in [`test_rclcpp`](https://github.com/ros2/system_tests/tree/master/test_rclcpp) and [`test_communication`](https://github.com/ros2/system_tests/tree/master/test_communication) packages, provide coverage. Nightly CI jobs in [`ci.ros2.org`](https://ci.ros2.org/) and [`build.ros2.org`](https://build.ros2.org/), where `rmw_fastrtps_cpp` is the default `rmw` implementation, thoroughly exercise this middleware.
All `rmw_fastrtps_cpp` public features are ROS middleware features.

Unit, integration, and system tests higher up in the stack, such as those found in [`test_rmw_implementation`](https://github.com/ros2/rmw_implementation/tree/master/test_rmw_implementation), [`test_rclcpp`](https://github.com/ros2/system_tests/tree/master/test_rclcpp), and [`test_communication`](https://github.com/ros2/system_tests/tree/master/test_communication) packages, provide feature coverage. Nightly CI jobs in [`ci.ros2.org`](https://ci.ros2.org/) and [`build.ros2.org`](https://build.ros2.org/), where `rmw_fastrtps_cpp` is the default `rmw` implementation, thoroughly exercise this middleware.

### Public API Testing [4.ii]

There is no API testing in `rmw_fastrtps_cpp`.
`rmw_fastrtps_cpp` implements the ROS middleware public API, but also provides public API of its own.

Unit tests located in the [`test`](https://github.com/ros2/rmw_fastrtps/tree/master/rmw_fastrtps_cpp/test) directory provide coverage for public but `rmw_fastrtps_cpp` specific API.
New additions or changes to this API require tests before being added.

Unit, integration, and system tests higher up in the stack, such as those found in [`test_rmw_implementation`](https://github.com/ros2/rmw_implementation/tree/master/test_rmw_implementation), [`test_rclcpp`](https://github.com/ros2/system_tests/tree/master/test_rclcpp), and [`test_communication`](https://github.com/ros2/system_tests/tree/master/test_communication) packages, ensure compliance with the ROS middleware API specification (see [`rmw`](https://github.com/ros2/rmw) package) and further extend coverage.

### Coverage [4.iii]

Expand All @@ -104,7 +111,10 @@ This includes:

Changes are required to make a best effort to keep or increase coverage before being accepted, but decreases are allowed if properly justified and accepted by reviewers.

Current coverage statistics can be viewed [here](https://ci.ros2.org/job/ci_linux_coverage/lastSuccessfulBuild/cobertura/src_ros2_rmw_fastrtps_rmw_fastrtps_cpp_src/). This package does not yet meet the 95% coverage guideline.
Current coverage statistics can be viewed [here](https://ci.ros2.org/job/ci_linux_coverage/lastSuccessfulBuild/cobertura/src_ros2_rmw_fastrtps_rmw_fastrtps_cpp_src/).

This package claims to meet the coverage requirements for the current quality level, even though it doesn't have 95% line coverage.
The justification is that the only uncovered lines have to do with system resource exhaustion and Fast-DDS internal failure.

A summary of how these statistics are calculated can be found in the [ROS 2 On-boarding guide](https://index.ros.org/doc/ros2/Contributing/ROS-2-On-boarding-Guide/#note-on-coverage-runs).

Expand Down
20 changes: 14 additions & 6 deletions rmw_fastrtps_shared_cpp/QUALITY_DECLARATION.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@ This document is a declaration of software quality for the `rmw_fastrtps_shared_

# `rmw_fastrtps_shared_cpp` Quality Declaration

The package `rmw_fastrtps_shared_cpp` claims to be in the **Quality Level 3** category.
The package `rmw_fastrtps_shared_cpp` claims to be in the **Quality Level 2** category.

Below are the rationales, notes, and caveats for this claim, organized by each requirement listed in the [Package Requirements for Quality Level 3 in REP-2004](https://www.ros.org/reps/rep-2004.html).
Below are the rationales, notes, and caveats for this claim, organized by each requirement listed in the [Package Requirements for Quality Level 2 in REP-2004](https://www.ros.org/reps/rep-2004.html).

## Version Policy [1]

Expand Down Expand Up @@ -86,12 +86,18 @@ The results of the test can be found [here](https://ci.ros2.org/view/nightly/job

### Feature Testing [4.i]

Many of the features have testing in `rmw_fastrtps_shared_cpp`. Since `rmw_fastrtps_shared_cpp` supports `rmw_fastrtps_cpp` and `rmw_fastrtps_dynamic_cpp`, integration and system tests for ROS middlewares up the stack, such as those found in [`test_rclcpp`](https://github.com/ros2/system_tests/tree/master/test_rclcpp) and [`test_communication`](https://github.com/ros2/system_tests/tree/master/test_communication) packages, provide coverage.
All `rmw_fastrtps_shared_cpp` public features support ROS middleware feature implementations in `rmw_fastrtps_cpp` and `rmw_fastrtps_dynamic_cpp`.

Unit, integration, and system tests for ROS middlewares higher up in the stack, such as those found in [`test_rmw_implementation`](https://github.com/ros2/rmw_implementation/tree/master/test_rmw_implementation), [`test_rclcpp`](https://github.com/ros2/system_tests/tree/master/test_rclcpp), and [`test_communication`](https://github.com/ros2/system_tests/tree/master/test_communication) packages, provide coverage.

### Public API Testing [4.ii]

Many parts of the public API have testing in `rmw_fastrtps_shared_cpp`.
The latest nightly test results can be seen [here](https://ci.ros2.org/view/nightly/job/nightly_linux_release/lastBuild/testReport/rmw_fastrtps_shared_cpp/).
All `rmw_fastrtps_shared_cpp` public API supports API implementations in in `rmw_fastrtps_cpp` and `rmw_fastrtps_dynamic_cpp`.

Unit tests for many parts of this API provide coverage, and they are located in the [`test`](https://github.com/ros2/rmw_fastrtps/tree/master/rmw_fastrtps_shared_cpp/test) directory.
New additions or changes to this API require tests before being added.

Unit, integration, and system tests for `rmw_fastrtps_cpp` and `rmw_fastrtps_dynamic_cpp` higher up in the stack, such as those found in [`test_rmw_implementation`](https://github.com/ros2/rmw_implementation/tree/master/test_rmw_implementation), [`test_rclcpp`](https://github.com/ros2/system_tests/tree/master/test_rclcpp), and [`test_communication`](https://github.com/ros2/system_tests/tree/master/test_communication) packages, further extend coverage.

### Coverage [4.iii]

Expand All @@ -110,7 +116,9 @@ Current coverage statistics can be viewed
[here](https://ci.ros2.org/job/ci_linux_coverage/lastSuccessfulBuild/cobertura/src_ros2_rmw_fastrtps_rmw_fastrtps_shared_cpp_include_rmw_fastrtps_shared_cpp/),
[here](https://ci.ros2.org/job/ci_linux_coverage/lastSuccessfulBuild/cobertura/src_ros2_rmw_fastrtps_rmw_fastrtps_shared_cpp_src/), and
[here](https://ci.ros2.org/job/ci_linux_coverage/lastSuccessfulBuild/cobertura/src_ros2_rmw_fastrtps_rmw_fastrtps_shared_cpp_src_types/).
This package does not yet meet the 95% coverage guideline.

This package claims to meet the coverage requirements for the current quality level, even though it doesn't have 95% line coverage.
The justification is that the only uncovered lines have to do with system resource exhaustion and Fast-DDS internal failure.

A summary of how these statistics are calculated can be found in the [ROS 2 On-boarding guide](https://index.ros.org/doc/ros2/Contributing/ROS-2-On-boarding-Guide/#note-on-coverage-runs).

Expand Down