Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add GitHub actions for linting and source-build CI #178

Merged
merged 8 commits into from Apr 1, 2020

Conversation

@mikaelarguedas
Copy link
Member

mikaelarguedas commented Mar 16, 2020

Currently Ubuntu only and no xmllint test for sros2 due to schema hosting issue.

Looking for feedback on the following:

  • the test workflow runs daily
  • the lint workflow runs on push and PR events only
  • log upload: usefulness of uploading colcon logs?
    Some repositories using the ROS github actions upload the colcon logs (e.g. rosbag2) but they upload them only on job success, which doesn't seem very useful. I opted to not add that upload step until we come across cases where we need the logs.

Examples of jobs using these parameters:
Build + test: https://github.com/mikaelarguedas/sros2/actions/runs/56872161
Lint: https://github.com/mikaelarguedas/sros2/actions/runs/56872157


Failing on Macos for now, haven't tried windows:

E   RuntimeError: Failed to initialize init_options: failed to find shared library of rmw implementation. Searched rmw_fastrtps_cpp, at /Users/runner/runners/2.165.2/work/sros2/sros2/ros_ws/src/ros2/rmw_implementation/rmw_implementation/src/functions.cpp:61, at /Users/runner/runners/2.165.2/work/sros2/sros2/ros_ws/src/ros2/rcl/rcl/src/rcl/init_options.c:55

Possible reason: environment may be sanitized (maybe SIP is enabled) so the relevant libs are not on the DYLD_LIBRARY_PATH: upstream ticket ros-tooling/action-ros-ci#81

@kyrofa did you ever succeed to run some ROS nodes on MacOS using this action ?

@kyrofa

This comment has been minimized.

Copy link
Member

kyrofa commented Mar 16, 2020

MacOS and Windows have been a bit flaky. We're still running them in nodl, and MacOS passes, but Windows consistently fails due to Fast RTPS not building. We only require Linux to pass given the flakiness.

@mikaelarguedas

This comment has been minimized.

Copy link
Member Author

mikaelarguedas commented Mar 16, 2020

We're still running them in nodl, and MacOS passes

Oh nice. Do your tests run any ROS Nodes or are they just pure python tests not relying on a running ROS system ?

@kyrofa

This comment has been minimized.

Copy link
Member

kyrofa commented Mar 16, 2020

I believe they're just pure Python at the moment, which probably explains the difference.

@mikaelarguedas mikaelarguedas force-pushed the mikaelarguedas:github-actions branch 2 times, most recently from 311a7da to 42ff12a Mar 18, 2020
@mikaelarguedas mikaelarguedas requested review from kyrofa and ruffsl Mar 18, 2020
@mikaelarguedas

This comment has been minimized.

Copy link
Member Author

mikaelarguedas commented Mar 18, 2020

@mikaelarguedas mikaelarguedas requested a review from artivis Mar 19, 2020
@mikaelarguedas

This comment has been minimized.

Copy link
Member Author

mikaelarguedas commented Mar 19, 2020

I kinda prefer just having one, but what do you think?

That's a good point.

I default to multiple files because one issue I have with single workflow is that it means same triggers for all jobs in the workflow. You can do some conditionals on triggers but it quickly becomes limited.This is why I use different ones in e.g. https://github.com/osrf/docker_images#table-of-contents.

In this case it doesn't make much of a difference, merging them into the same workflow means triggering linter jobs along the nightly builds and test, which is totally fine.

Copy link
Member

kyrofa left a comment

Looking good @mikaelarguedas, thank you! I have a few inline comments.

Also, note that having separate test files means you also have separate pass/fail badges:

Linter: Lint sros2 repo
Tests: Test sros2

I kinda prefer just having one, but what do you think?

.github/workflows/lint.yml Outdated Show resolved Hide resolved
.github/workflows/test.yml Outdated Show resolved Hide resolved
.github/workflows/test.yml Outdated Show resolved Hide resolved
@kyrofa

This comment has been minimized.

Copy link
Member

kyrofa commented Mar 19, 2020

I default to multiple files because one issue I have with single workflow is that it means same triggers for all jobs in the workflow.

Good point, that makes sense.

In this case it doesn't make much of a difference, merging them into the same workflow means triggering linter jobs along the nightly builds and test, which is totally fine.

Sounds like neither of us feel too strongly about this. Let's ask this, then: do we want to show that badge somewhere? In the README, maybe? Do we care about them at all? If we do, let's make it one job so we only have one to show. If we don't, let's leave it as two.

Personally I think it's more helpful to get a code quality/health impression from the readme as opposed to "are the tests passing right at this moment" (because note that the cron job updates the badges as well). So maybe we revisit that when we have coverage setup.

@mikaelarguedas mikaelarguedas force-pushed the mikaelarguedas:github-actions branch from 43f40ac to e5fb4d7 Mar 19, 2020
@mikaelarguedas

This comment has been minimized.

Copy link
Member Author

mikaelarguedas commented Mar 19, 2020

Sounds like neither of us feel too strongly about this. Let's ask this, then: do we want to show that badge somewhere? In the README, maybe? Do we care about them at all? If we do, let's make it one job so we only have one to show. If we don't, let's leave it as two..

Sounds good to me, it's now in the README.

.github/workflows/ci.yml Outdated Show resolved Hide resolved
.github/workflows/ci.yml Outdated Show resolved Hide resolved
@ros-discourse

This comment has been minimized.

Copy link

ros-discourse commented Mar 20, 2020

This pull request has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/ros-2-tsc-meeting-minutes-2020-03-18/13313/1

@mikaelarguedas mikaelarguedas requested review from artivis and kyrofa Mar 20, 2020
Copy link
Member

kyrofa left a comment

I have one last nit/question, but I'm really +1 either way.

.github/workflows/ci.yml Outdated Show resolved Hide resolved
@mikaelarguedas

This comment has been minimized.

Copy link
Member Author

mikaelarguedas commented Mar 24, 2020

I left this PR hanging the last couple days as ros2 builfarm switched to 20.04 over the weekend. This puts a bigger question mark on the relevance of this set of actions.
I'm exploring alternatives e.g. https://github.com/mikaelarguedas/sros2/actions/runs/61362217 that allow to run faster and match closely the buildfarm because based on the nightly builds.

Leaving this one on hold for now

- One source build to test against latest master
- One build on top of nightly image to give quick feedback

Signed-off-by: Mikael Arguedas <mikael.arguedas@gmail.com>
Signed-off-by: Mikael Arguedas <mikael.arguedas@gmail.com>
Signed-off-by: Mikael Arguedas <mikael.arguedas@gmail.com>
Signed-off-by: Mikael Arguedas <mikael.arguedas@gmail.com>
Signed-off-by: Mikael Arguedas <mikael.arguedas@gmail.com>
Signed-off-by: Mikael Arguedas <mikael.arguedas@gmail.com>
Signed-off-by: Mikael Arguedas <mikael.arguedas@gmail.com>
@mikaelarguedas

This comment has been minimized.

Copy link
Member Author

mikaelarguedas commented Mar 27, 2020

New version of this PR:

Workflow with 2 jobs:

  • job building on top of osrf/ros2:nightly, provides quick feedback for failures such as linters (~2min)
  • job building from source on top of osrf/ros2:devel, takes longer and provides more accurate representation of the code if PR was merged now (~20min)
    • provides coverage information with feedback on the PR
    • runs integration tests (test_security package, though disabled for now as they are not passing on Focal)

Example of repo/PR using this: mikaelarguedas#2

Prerequisites for merging:

  • enable codecov.io for this repository

Related work (non-blocking):


Future work:
There is still custom logic in this repo. Ideally we could modify the ros-tooling actions to meet our need and limit custom code to a minimum, some examples of improvements

  • switch back to use upstream action-ros-ci
  • improve action-ros-ci (non exhaustive):
    • standalone: action should install its own required dependencies
    • more control to users: not hardcoding colcon arguments or allow users to override any argument
    • consistent failure modes (colcon lcov-result --initial fails but doesn't fail the build but colcon lcov-result does o_0)
    • allow to specify rosdep arguments
    • mode using current build dir and not pulling any repos file
    • print test result summary
  • cache docker images daily to avoid pulling (80% of the time of the nightly job)
@mikaelarguedas mikaelarguedas force-pushed the mikaelarguedas:github-actions branch from 3cb4573 to 3963861 Mar 27, 2020
[run]
omit =
# omit test directory
test/*

This comment has been minimized.

Copy link
@mikaelarguedas

mikaelarguedas Mar 27, 2020

Author Member

I did this to get accurate coverage numbers. Code in test is usually pretty well covered and skewing the numbers

@@ -0,0 +1,2 @@
fixes:
- "ros_ws/src/sros2/::"

This comment has been minimized.

Copy link
@mikaelarguedas

mikaelarguedas Mar 27, 2020

Author Member

This rewrites the path of the files in the report so that we can browse them directly on the codecov.io interface and see which lines/conditionals are covered and which are not

Signed-off-by: Mikael Arguedas <mikael.arguedas@gmail.com>
@mikaelarguedas mikaelarguedas requested a review from ros2/security-reviewers Mar 31, 2020
@kyrofa
kyrofa approved these changes Mar 31, 2020
@Arnatious

This comment has been minimized.

Copy link

Arnatious commented Mar 31, 2020

LGTM

@artivis

This comment has been minimized.

Copy link

artivis commented Mar 31, 2020

About the prerequisite

enable codecov.io for this repository

Who's responsibility is this? I don't know codecov.io very well, can we create groups or whatnot so that the admin work is shared?

@kyrofa

This comment has been minimized.

Copy link
Member

kyrofa commented Mar 31, 2020

Who's responsibility is this?

Mine, this is already done. We think 😛 .

@artivis

This comment has been minimized.

Copy link

artivis commented Mar 31, 2020

Mine, this is already done.

Ok, good to know.
We'll have to make sure that this can be managed by other WG members in case it is necessary.

@kyrofa

This comment has been minimized.

Copy link
Member

kyrofa commented Mar 31, 2020

It was actually already enabled and (at least we think) actions don't need a token, so there's really nothing to do there.

README.md Show resolved Hide resolved
@artivis

This comment has been minimized.

Copy link

artivis commented Mar 31, 2020

Left a last small comment/question.
LGTM 👍

@kyrofa

This comment has been minimized.

Copy link
Member

kyrofa commented Mar 31, 2020

Alright, tests for sros2:

  • Linux Build Status
  • Linux-aarch64 Build Status
  • macOS Build Status
  • Windows Build Status
@kyrofa kyrofa merged commit 9e1d105 into ros2:master Apr 1, 2020
@mikaelarguedas mikaelarguedas deleted the mikaelarguedas:github-actions branch Apr 1, 2020
@mikaelarguedas

This comment has been minimized.

Copy link
Member Author

mikaelarguedas commented Apr 1, 2020

Mine, this is already done. We think stuck_out_tongue .

So apparently it was not done ^^
it does upload reports to codecov, but it does not process and post back progress on the PRs.

2 ways to do it:

  • integrate the codecov github app for ros2/sros2
  • add a webhook on ros2/sros2 for codecov to use

App integration seems the way to go these days but may require permission at the org level asd the app doesnt seem to be installed on the ros2 org.

Is there a preference from @ros2/team or @ros2/security-members for one approach or the other ?

Edit: the webhook allows to post comment automatically on PR. But so far didnt allow to browse files on the codecov.io interface

@kyrofa

This comment has been minimized.

Copy link
Member

kyrofa commented Apr 1, 2020

Huh, maybe that's why it looked like it was already enabled, the default settings didn't need the app installed. I prefer that, of course, but it's up to the org admins.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

7 participants
You can’t perform that action at this time.