Drop PyKDL dependency in tf2_geometry_msgs

[Flova]

Currently, the python tf2_geometry_msgsare excluded because of the PyKDL/orocos issue.
This pull request drops the dependency all together and replaces the transformations with ones done in pure NumPy.

[clalancette]

I'm torn on whether we should take this. On the plus side, this gets rid of the PyKDL dependency which has been nagging this package for a long time, and makes this package available to users. On the negative side, we are hoping to get PyKDL into Humble (via ros2/orocos_kdl_vendor#1), and this ends up dropping the public API transform_to_kdl.

Here are some options I can see for this:

1. Wait for Add vendor package files for orocos_kdl orocos_kdl_vendor#1 to land, and then just keep the existing code as-is. This assumes we get PyKDL in before Humble.
2. Take this patch as it stands (after review, etc), knowing that we lose transform_to_kdl. We can always add transform_to_kdl back in later by re-introducing a dependency on PyKDL.
3. Wait to see if we actually get PyKDL into Humble. If we do, then close this PR and just use PyKDL. If we don't, then take this PR and follow up later when we do get PyKDL.

There may be other options.

@ros2/team @Flova Thoughts?

[Flova]

On the negative side, we are hoping to get PyKDL into Humble (via ros2/orocos_kdl_vendor#1), and this ends up dropping the public API transform_to_kdl.

I understand the implications and that this is a tough choice. I also appreciate the efforts regarding PyKDL/orocos_kdl which I am following for some time now. As all introduced methods are private, I would see no issue except the administrative overhead, in releasing this fix as it is for now (with your feedback + some testing) and replacing it with the PyKDL implementation later on if it is available again. I am not that worried about the transform_to_kdl method as it is currently not available like the rest of this module, so making a subset of the features available for now would still help. I personally think that this package is quite essential for many people who migrate their python tf2 code to ros2. I therefore would prefer the second option you proposed.

I would suggest opening a new PR to merge things like the improved typing/docstrings of the existing functions as well as the fixes applied to the tests, if we choose to close this one.

[Flova]

Any other thought on this?

[clalancette]

I think we can go ahead here and review this. We can always add back the transform_to_kdl API later on, when we know for sure that we have PyKDL available. I'll go ahead and do a review.

[Flova]

Nice, thank you :)

[clalancette]

I've left a few minor things to change. Looks good overall; once those things are fixed, I'll fire up CI on it.

[Flova]

I think I fixed everything. Let me know if there is something left I overlooked or if I should do something different.

[Flova]

This PR closes #360, right?

[clalancette]

This PR closes #360, right?

Yeah, it should. I think it will also close one or two open issues, but I'll have to check on that.

[Flova]

I also tested in 20.04, but I guess my NumPy version was replaced/overlaid with a newer one by installing some pip package that depended on it. https://packages.ubuntu.com/focal/python3-numpy shows that the current version is 1.17 and Typing was introduced in 1.20. Replacing the more general npt.ArrayLike with np.ndarray should be no problem for this package, as it is only used internally with objects of type np.ndarray.
This also effects #507.

[Flova]

I will address these flake8 errors, but my flake8 checks passed. Is there a config I don't know about for ROS2 and flake8 which is not applied automatically? Or is it because of the first point you mentioned?

[Flova]

I fixed the first and third issue you observed @clalancette, but the flake8 one seems kind of weird. The first point regarding the order of the imported elements from geometry_msgs.msg makes sense, but the second one suggests that the imports should be ordered like this if I understand it correctly:

import numpy as np
from geometry_msgs.msg import (PointStamped, Pose, PoseStamped,
                               PoseWithCovarianceStamped, TransformStamped,
                               Vector3Stamped)
import tf2_ros

This seems of as normally the from * imports come after it. It is also strange, that I get this output from the tests regarding flake8 for both issues you described:

6: Test timeout computed to be: 60
6: -- run_test.py: invoking following command in '/home/florian/colcon_ws/src/geometry2-1/tf2_geometry_msgs':
6:  - /opt/ros/rolling/bin/ament_flake8 --xunit-file /home/florian/colcon_ws/build/tf2_geometry_msgs/test_results/tf2_geometry_msgs/flake8.xunit.xml
6:                                              
6: 3 files checked
6: No problems found
6: 
6: Checked files:
6: 
6: * ./src/tf2_geometry_msgs/tf2_geometry_msgs.py
6: * ./src/tf2_geometry_msgs/__init__.py
6: * ./test/test_tf2_geometry_msgs.py
6: -- run_test.py: return code 0
6: -- run_test.py: verify result file '/home/florian/colcon_ws/build/tf2_geometry_msgs/test_results/tf2_geometry_msgs/flake8.xunit.xml'
 6/10 Test  #6: flake8 ...........................   Passed    0.76 sec

Do you use a special flake8 config? From the output, it seems like you have built the ament_flake8 package yourself (Idk about your setup, so this could be normal).

[clalancette]

Unfortunately flake8 is a bit annoying. Basically it runs with the flake8 plugins you currently have installed locally. The practical upshot of this is that you happen to have one missing, it will just silently ignore it.

As for the weird import order, yeah, I don't always agree with how flake8 orders things. But it's easier to just conform to it. I ended up fixing it with the latest commit.

While testing, I found a small bug in tf2_ros_py, where it isn't depending on sensor_msgs properly. That's now fixed in here as well. With all of that done, I'm happy with this so I'll do one more pass and then run CI on it.

[clalancette]

CI:

• Linux
• Linux-aarch64
• Windows

[Flova]

Nice, the CI seem to look good.

[dawsonc]

Hi!

Is there a timeline for getting these changes into a release? Currently rosdep does not install the python bindings for tf2_geometry_msgs in foxy.

Thanks!

[clalancette]

Is there a timeline for getting these changes into a release? Currently rosdep does not install the python bindings for tf2_geometry_msgs in foxy.

It is currently available in Humble and Rolling. It's probably possible to backport to Foxy, though it might be a bit tricky; we had to do a couple of additional patches on top of this one to get it working.

What I'll do here is to ask the bot to try to backport it. That said, I'm unlikely to have time myself to fix it up, so it would be appreciated if you can fix any conflicts that arise, test it out, and generally make sure it will work in Foxy.

[clalancette]

@Mergifyio backport foxy galactic

[mergify]

backport foxy galactic

✅ Backports have been created

• #530 Drop PyKDL dependency in tf2_geometry_msgs (backport #509) has been created for branch foxy
• #531 Drop PyKDL dependency in tf2_geometry_msgs (backport #509) has been created for branch galactic

[dawsonc]

@clalancette Thank you! I'll take a look at testing the foxy backport and move discussion over to #530 #532 for foxy and #533 for galactic (since I don't think I have write access to the branch created by mergify).