Migrates legacy launch API tests #167

hidmic · 2019-01-29T20:13:53Z

Connected to #159. This pull request adds the bare minimum functionality to migrate all tests in ROS 2 core packages that make use of the legacy launch API. May eventually face deprecation depending on the path taken after #165.

Some launch.legacy tests still remain to be ported, unclear if these should in the first place as such additions may as well fall under the improving test coverage for launch umbrella.

Signed-off-by: Michel Hidalgo <michel@ekumenlabs.com>

hidmic · 2019-01-31T19:57:01Z

I think it makes sense to get rid of the legacy API regardless of what comes out of the launch based testing framework work. Unrelated functionality I had to add I think could still be useful outside testing, do tell if you think otherwise.

wjwwood

This looks pretty good to me so far, though I had a some questions during review.

launch/launch/events/__init__.py

launch_testing/launch_testing/__init__.py

launch/launch/action.py

launch_testing/launch_testing/__init__.py

wjwwood · 2019-02-01T00:08:13Z

launch_testing/launch_testing/__init__.py

+        if all(status == 'succeeded' for status in self.__tests.values()):
+            return EmitEvent(event=Shutdown(reason='all tests finished'))
+
+    def add_fixture_action(self, launch_description, action, required=False):


Passing the launch_description in here was a bit off-putting to me at first, but I see the reason for it now, though I wish it were more transparent what was happening. If you look just at the calling code (like at one of the converted tests), you don't get a since as to what is being done to the launch description, you can assume it's adding the fixture/test/output_test or whatever, but I hadn't imagined it would add more actions until reading the code for it here.

A possible alternative would be to make this function simply generate and return a list of actions to be added, rather than adding them directly. Though I'm not sure if the launch description is required within this function in any other cases. For example, it might look like:

my_test_exec_action = ExecuteProcess(...) fixture_action = lts.create_fixture_action(my_test_exec_action) ld.add_action(fixture_action)

In that example fixture_action might be just the original action or a GroupAction that consists of several things.

I'm not 100% sold on my approach either, as I wouldn't be able to know without an example or very good docs if I should add the original action or just the result of create_fixture_action() instead.

Passing the launch_description in here was a bit off-putting to me at first...

It sort of is to me as well, but as I said before, I was trying to minimize the amount of code changes (because it's not at all unlikely we end up ditching this in the near future) and thus I kept the API the way it was.

A possible alternative would be to make this function simply generate and return a list of actions to be added, rather than adding them directly.

That is an option, sure. Another easy one (easy to use I mean) would be to have a LaunchTestDescription along with the LaunchTestService and make them wrap the LaunchDescription and LaunchService respectively. Not sure if it makes sense to spend time polishing this API though.

Co-Authored-By: hidmic <michel@ekumenlabs.com>

- Emit ExecutionComplete events on demand. - Shutdown OpaqueCoroutine gracefully. - Fix failing test cases. Signed-off-by: Michel Hidalgo <michel@ekumenlabs.com>

hidmic · 2019-02-04T18:00:38Z

Running CI for all related PRs using pinned FastRTPS:

Linux
Linux-aarch64
Linux packaging
macOS
Windows

To cope with more test cases. Signed-off-by: Michel Hidalgo <michel@ekumenlabs.com>

Signed-off-by: Michel Hidalgo <michel@ekumenlabs.com>

hidmic · 2019-02-12T18:38:41Z

@mjcarroll @sloretz @wjwwood I'm a bit unsure on how to handle bad exit codes upon shutdown in a true cross platform way, thus I'm blatantly ignoring them. Not very proud, but I can't figure out how to do better than that.

Signed-off-by: Michel Hidalgo <michel@ekumenlabs.com>

hidmic · 2019-02-20T14:15:44Z

So, we have a problem with FastRTPS talking to Opensplice and Connext on Windows. Looks like we have an issue tracking it (and someone suggesting that FastRTPS v.1.7.1 may be the cure). But since this predates these changes (which have only revealed more issues), in the interest of time I'd say: let's disable the offending test just as we did before, and follow up that issue. We can then come back and re-enable all tests.

@mjcarroll @sloretz @ivanpauno thoughts?

Signed-off-by: Michel Hidalgo <michel@ekumenlabs.com>

hidmic · 2019-02-25T13:17:37Z

CI is green! @mjcarroll mind to take a shot at this? We can re-run CI on all platforms right after.

mjcarroll

Just some tiny documentation nits, but otherwise LGTM.

mjcarroll · 2019-02-25T14:50:25Z

launch/launch/actions/opaque_coroutine.py

+    .. code-block:: python
+
+        async def coroutine(
+            context: LaunchContext,  # iff ignore_context is False


I understand what iff is in this context, but it may be helpful to be a little bit more explicit. It may also be worth updating the docs for OpaqueFunction so that they are consistent in this regard.

That makes sense. I've updated the documentation, but I'm not sure what you mean about changing OpaqueFunction docs.

mjcarroll · 2019-02-25T14:50:48Z

launch/launch/actions/opaque_coroutine.py

+        super().__init__(**left_over_kwargs)
+        if not asyncio.iscoroutinefunction(coroutine):
+            raise TypeError(
+                "OpaqueCoroutine expected a couroutine for 'couroutine', got '{}'".format(


couroutine -> coroutine

Oops, good catch!

mjcarroll · 2019-02-25T14:52:43Z

launch/launch/event_handlers/on_execution_complete.py

+    """
+    Convenience class for handling an action completion event.
+
+    It may be configured to only handle the completion of a specific action,


Do we have an example or test for handling specific vs all? I can't seem to find it.

Good point, we do not. Test added.

- Improved OpaqueCoroutine documentation. - Added test for launch.event_handlers.OnExecutionComplete. Signed-off-by: Michel Hidalgo <michel@ekumenlabs.com>

hidmic · 2019-02-25T20:19:35Z

Grrr something changed on actions and sros2, tests are failing again.

hidmic · 2019-02-26T16:06:28Z

@mjcarroll CI got unstable because (a) I had to rebase ros2/system_tests branch after ros2/system_tests#330 and (b) there's a test failure coming from sros2 and it's not specific to this set of PRs (see https://ci.ros2.org/job/ci_osx/5150/ or https://ci.ros2.org/job/ci_linux/6268/). So if we run CI again, it'll go yellow again.

We can wait till (b) gets fixed, but I'm afraid more things will get in the way after that. It's fine by me, but that means more delay for these PRs.

hidmic · 2019-03-06T16:19:03Z

@mjcarroll could you take a look at the dependency fixing PRs?

mjcarroll · 2019-03-06T16:25:31Z

Done!

wjwwood · 2019-03-27T03:50:06Z

There was a bug fix in this pr that was unrelated to the original change. @stonier opened a pr for just this change here:

#220

I declined it so that we could put it through our typical patch release process.

The change is small, @nuclearsandwich FYI.

nuclearsandwich · 2019-03-27T13:21:30Z

I declined it so that we could put it through our typical patch release process.

The change is small, @nuclearsandwich FYI.

Thanks for the ping. I don't mind if the original PR gets re-opened or if we open a new PR explicitly preserving the original commit authors as long as we run Crystal CI on it before merging and releasing.

hidmic added 7 commits January 29, 2019 12:31

Emits events.ExecutionComplete on Action completion.

2e954ce

Signed-off-by: Michel Hidalgo <michel@ekumenlabs.com>

Makes LaunchTestService handle non-process actions.

9398aba

Signed-off-by: Michel Hidalgo <michel@ekumenlabs.com>

Makes LaunchTestService handle fixture process actions.

b9f3bae

Signed-off-by: Michel Hidalgo <michel@ekumenlabs.com>

Adds output checks to LaunchTestService.

f4faf91

Signed-off-by: Michel Hidalgo <michel@ekumenlabs.com>

Updates launch_testing package tests.

78a567d

Signed-off-by: Michel Hidalgo <michel@ekumenlabs.com>

Applies fixes after launch_testing refactor.

3a65f65

Signed-off-by: Michel Hidalgo <michel@ekumenlabs.com>

Adds OpaqueCoroutine action subclass.

a1f7fb7

Signed-off-by: Michel Hidalgo <michel@ekumenlabs.com>

tfoote added the in progress Actively being worked on (Kanban column) label Jan 29, 2019

hidmic added in review Waiting for review (Kanban column) and removed in progress Actively being worked on (Kanban column) labels Jan 30, 2019

hidmic requested review from wjwwood, mjcarroll and sloretz January 31, 2019 19:55

wjwwood reviewed Feb 1, 2019

View reviewed changes

wjwwood and others added 2 commits February 4, 2019 10:50

Apply suggestions from code review

7a0ac0a

Co-Authored-By: hidmic <michel@ekumenlabs.com>

Addresses peer review comments.

33d15c2

- Emit ExecutionComplete events on demand. - Shutdown OpaqueCoroutine gracefully. - Fix failing test cases. Signed-off-by: Michel Hidalgo <michel@ekumenlabs.com>

cottsay assigned hidmic Feb 7, 2019

ivanpauno mentioned this pull request Feb 7, 2019

Add test actions #178

Merged

Refactors launch_testing API a bit.

07d55f8

To cope with more test cases. Signed-off-by: Michel Hidalgo <michel@ekumenlabs.com>

hidmic force-pushed the hidmic/drop-legacy-launch branch from 62d58ac to 07d55f8 Compare February 11, 2019 13:49

hidmic added 2 commits February 12, 2019 12:10

Applies style fixes to launch_testing.

34da363

Signed-off-by: Michel Hidalgo <michel@ekumenlabs.com>

Deal with non zero exit on shutdown.

6d04a9f

Signed-off-by: Michel Hidalgo <michel@ekumenlabs.com>

Avoids output tests' races.

aec02bf

Signed-off-by: Michel Hidalgo <michel@ekumenlabs.com>

hidmic force-pushed the hidmic/drop-legacy-launch branch 2 times, most recently from d26d890 to ea034a2 Compare February 18, 2019 21:00

hidmic added 2 commits February 18, 2019 20:57

Applies misc fixes after Windows triaging.

ea034a2

Signed-off-by: Michel Hidalgo <michel@ekumenlabs.com>

Applies more fixes after Windows triaging.

39f1234

Signed-off-by: Michel Hidalgo <michel@ekumenlabs.com>

cottsay requested review from mjcarroll and removed request for mjcarroll February 22, 2019 19:37

Fixes linter issue.

2d9305a

Signed-off-by: Michel Hidalgo <michel@ekumenlabs.com>

mjcarroll approved these changes Feb 25, 2019

View reviewed changes

Addresses peer review comments.

d05379a

- Improved OpaqueCoroutine documentation. - Added test for launch.event_handlers.OnExecutionComplete. Signed-off-by: Michel Hidalgo <michel@ekumenlabs.com>

mjcarroll merged commit 632ef1f into master Feb 27, 2019

mjcarroll removed the in review Waiting for review (Kanban column) label Feb 27, 2019

hidmic mentioned this pull request Feb 28, 2019

Drop legacy launch package #191

Merged

sloretz mentioned this pull request Feb 28, 2019

Increase in test failures on windows debug with actions tests ros2/build_farmer#167

Closed

hidmic deleted the hidmic/drop-legacy-launch branch March 6, 2019 17:04

crdelsey mentioned this pull request Mar 6, 2019

add_test_action no longer triggers shutdown on completion #194

Closed

jacobperron mentioned this pull request Mar 6, 2019

Support Python 3.5.2 (Xenial)? #195

Closed

This was referenced Mar 7, 2019

Launch-based Testing #161

Closed

Fix event loop closed bug #198

Merged

wjwwood mentioned this pull request Mar 27, 2019

Crystal Patch Release 4 ros2/ros2#668

Closed

8 tasks

hidmic mentioned this pull request Apr 29, 2019

Migrate launch tests to new launch_testing features & API ros2/demos#318

Merged

ivanpauno mentioned this pull request May 9, 2019

Corrected matches_action usage in lifecycle_pub_sub example ros2/launch_ros#21

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrates legacy launch API tests #167

Migrates legacy launch API tests #167

hidmic commented Jan 29, 2019

hidmic commented Jan 31, 2019

wjwwood left a comment

wjwwood Feb 1, 2019

hidmic Feb 4, 2019 •

edited

Loading

hidmic commented Feb 4, 2019 •

edited

Loading

hidmic commented Feb 12, 2019

hidmic commented Feb 20, 2019 •

edited

Loading

hidmic commented Feb 25, 2019 •

edited

Loading

mjcarroll left a comment

mjcarroll Feb 25, 2019

hidmic Feb 25, 2019

mjcarroll Feb 25, 2019

hidmic Feb 25, 2019

mjcarroll Feb 25, 2019

hidmic Feb 25, 2019

hidmic commented Feb 25, 2019

hidmic commented Feb 26, 2019

hidmic commented Mar 6, 2019

mjcarroll commented Mar 6, 2019

wjwwood commented Mar 27, 2019

nuclearsandwich commented Mar 27, 2019

Migrates legacy launch API tests #167

Migrates legacy launch API tests #167

Conversation

hidmic commented Jan 29, 2019

hidmic commented Jan 31, 2019

wjwwood left a comment

Choose a reason for hiding this comment

wjwwood Feb 1, 2019

Choose a reason for hiding this comment

hidmic Feb 4, 2019 • edited Loading

Choose a reason for hiding this comment

hidmic commented Feb 4, 2019 • edited Loading

hidmic commented Feb 12, 2019

hidmic commented Feb 20, 2019 • edited Loading

hidmic commented Feb 25, 2019 • edited Loading

mjcarroll left a comment

Choose a reason for hiding this comment

mjcarroll Feb 25, 2019

Choose a reason for hiding this comment

hidmic Feb 25, 2019

Choose a reason for hiding this comment

mjcarroll Feb 25, 2019

Choose a reason for hiding this comment

hidmic Feb 25, 2019

Choose a reason for hiding this comment

mjcarroll Feb 25, 2019

Choose a reason for hiding this comment

hidmic Feb 25, 2019

Choose a reason for hiding this comment

hidmic commented Feb 25, 2019

hidmic commented Feb 26, 2019

hidmic commented Mar 6, 2019

mjcarroll commented Mar 6, 2019

wjwwood commented Mar 27, 2019

nuclearsandwich commented Mar 27, 2019

hidmic Feb 4, 2019 •

edited

Loading

hidmic commented Feb 4, 2019 •

edited

Loading

hidmic commented Feb 20, 2019 •

edited

Loading

hidmic commented Feb 25, 2019 •

edited

Loading