Port pointcloud creation to numpy.

[Flova]

This pull request ports the PointCloud creation to NumPy. It should massively increase the performance for large point clouds as no iteration over the points in python is needed and the bytes are taken directly from NumPy.
Still needs some work regarding other field types. Feel free to comment on my implementation. Suggestions are welcome.
Cheers

[aprotyas]

Repeating my comments from ros2/geometry2#507:

CI is failing here because it couldn't resolve the python-numpy rosdep key for Ubuntu Jammy. I believe the dependency needs to be on python3-numpy instead.

Also, it looks like your commits aren't signed-off. Do you mind signing them off?
https://docs.ros.org/en/rolling/Contributing/Developer-Guide.html#change-control-process

[Flova]

Why is the DCO failing? Github shows my commits as correctly signed.

[aprotyas]

Why is the DCO failing? Github shows my commits as correctly signed.

It looks like you signed the commits with your GPG key, but you need to add a Signed-off-by line in your commit message.

Check here: https://docs.ros.org/en/rolling/Contributing/Developer-Guide.html#change-control-process

[Flova]

Duplicate of #155, but this PR has a bit more features and is activly worked on.

[Flova]

While the CI still fails with some error related to python3.9 vs python3.10 in Jammy all tests are passing locally on my Ubuntu 20.04 ROS rolling machine.

[Flova]

Some performance testing:

As data, I used an unstructured NumPy array containing 480000 points.

Serialization using the old code:
700ms
Serialization using the updated code:
10ms
Deserialization using the old code:
140ms
Deserialization using the new code (read_points_numpy aka with cast to unstructured array):
143 µs (this is the correct unit)
Deserialization using the new code (read_points aka as structured array):
51.7 µs

[clalancette]

While the CI still fails with some error related to python3.9 vs python3.10 in Jammy all tests are passing locally on my Ubuntu 20.04 ROS rolling machine.

Yeah, that's fine. This is currently broken across the board (we are waiting for some fixes in Jammy itself). Once we are closer to having this approved, we'll run CI on https://ci.ros2.org to see the status.

[Flova]

I plan to write some more tests regarding organized point clouds as well as ones with mixed dtypes.

[Flova]

Any other comments regarding this PR? I think all mayor issues are addressed or moved upstream without affecting this PR.

[gbiggs]

LGTM pending passing CI.

[gbiggs]

CI:

• Linux
• Linux-aarch64
• Windows

[gbiggs]

CI has picked up a small pile of flake8 linting errors.

08:12:16 _________________________________ test_flake8 __________________________________
08:12:16 test/test_flake8.py:38: in test_flake8
08:12:16     assert rc == 0, \
08:12:16 E   AssertionError: Found 20 code style errors / warnings:
08:12:16 E     ./sensor_msgs_py/point_cloud2.py:42:1: I100 Import statements are in the wrong order. 'from collections import namedtuple' should be before 'import sys'
08:12:16 E     ./sensor_msgs_py/point_cloud2.py:61:22: Q000 Double quotes found but single quotes preferred
08:12:16 E     ./sensor_msgs_py/point_cloud2.py:95:13: Q000 Double quotes found but single quotes preferred
08:12:16 E     ./sensor_msgs_py/point_cloud2.py:100:30: Q000 Double quotes found but single quotes preferred
08:12:16 E     ./sensor_msgs_py/point_cloud2.py:154:9: Q000 Double quotes found but single quotes preferred
08:12:16 E     ./sensor_msgs_py/point_cloud2.py:208:26: Q000 Double quotes found but single quotes preferred
08:12:16 E     ./sensor_msgs_py/point_cloud2.py:209:20: Q000 Double quotes found but single quotes preferred
08:12:16 E     ./sensor_msgs_py/point_cloud2.py:218:33: Q000 Double quotes found but single quotes preferred
08:12:16 E     ./sensor_msgs_py/point_cloud2.py:221:54: Q000 Double quotes found but single quotes preferred
08:12:16 E     ./sensor_msgs_py/point_cloud2.py:256:17: Q000 Double quotes found but single quotes preferred
08:12:16 E     ./sensor_msgs_py/point_cloud2.py:264:17: Q000 Double quotes found but single quotes preferred
08:12:16 E     ./sensor_msgs_py/point_cloud2.py:275:9: Q000 Double quotes found but single quotes preferred
08:12:16 E     ./sensor_msgs_py/point_cloud2.py:285:31: Q000 Double quotes found but single quotes preferred
08:12:16 E     ./sensor_msgs_py/point_cloud2.py:286:31: Q000 Double quotes found but single quotes preferred
08:12:16 E     ./test/test_point_cloud2.py:107:10: Q000 Double quotes found but single quotes preferred
08:12:16 E     ./test/test_point_cloud2.py:108:10: Q000 Double quotes found but single quotes preferred
08:12:16 E     ./test/test_point_cloud2.py:109:10: Q000 Double quotes found but single quotes preferred
08:12:16 E     ./test/test_point_cloud2.py:228:51: Q000 Double quotes found but single quotes preferred
08:12:16 E     ./test/test_point_cloud2.py:228:58: Q000 Double quotes found but single quotes preferred
08:12:16 E     ./test/test_point_cloud2.py:228:65: Q000 Double quotes found but single quotes preferred
08:12:16 E   assert 1 == 0

[Flova]

@gbiggs I hopefully fixed all flake8 errors

[gbiggs]

Thanks! Another CI run:

• Linux
• Linux-aarch64
• Windows

[Flova]

I'll definitely need to investigate why my flake8 config is so different to the one used by the CI. But I am now more or less confident that the imports are ordered correctly.

[gbiggs]

CI again:

• Linux
• Linux-aarch64
• Windows

[Flova]

Finally ^^

[gbiggs]

Thanks for the contribution!