Enable to publish JPEG images without image decompression on the host

[shuntaraw]

Enable to publish JPEG image at rgb/image_raw/compressed so that clients can subscribe to the compressed image at rgb/image_raw via compressed_image_transport. This feature is useful when the images are received/consumed by remote clients and the local host is not powerful enough to decompress the JPEG images.

Fixes #20

Description of the changes:

This PR implements the technique described in the following site. http://wiki.ros.org/compressed_image_transport#Publishing_compressed_images_directly

• Add color_format ROS param to enable publishing JPEG images without image decompression on the host computer.
• If color_format is jpeg,
  • JPEG images are published at /rgb/image_raw/compressed
  • /rgb/image_raw is not advertised (to avoid confusion).
  • /rgb_to_depth/image_raw does not publish.
  • rgb_point_cloud ROS param must be false.

Before submitting a Pull Request:

• I reviewed CONTRIBUTING.md
• I built my changes locally
• I tested my changes with a device

I tested changes on:

• Windows
• Linux

manual/ad-hoc testing

[RoseFlunder]

I don't have a device here at the moment, but did you check if everything works with jpeg color format and rgb point cloud?
Because on the first glance I see a potential problem at the rgb point cloud there:

Azure_Kinect_ROS_Driver/src/k4a_ros_device.cpp

Line 586 in 7789226

const size_t point_count = k4a_bgra_frame.get_size() / sizeof(BgraPixel);

This check will fail with color format jpeg I think.
Later on at the point cloud generation you need distinct values for Red, Green, Blue.
Thats difficult with jpeg color format I guess. Because of that I turned on conversion to BGRA when using recording files.

[shuntaraw]

@RoseFlunder Nice catch! Please take a look at the updated PR. I confirmed that the following topics work as expected with color_format=jpeg. The default setting color_format=bgra does not change the behavior.

/depth/camera_info
/depth/image_raw
/depth_to_rgb/camera_info
/depth_to_rgb/image_raw
/imu
/ir/camera_info
/ir/image_raw
/joint_states
/points2
/rgb/camera_info
/rgb/image_raw/compressed

[RoseFlunder]

Another questions is how to handle the parameter when opening a recording file?
Whats the expected behaviour?
At the moment we always enable conversion to bgra.

We could use this setting to decide wether to enable color conversion to bgra or not.
If set to bgra we enable color conversion.
If set to jpeg and the recordings color Format is jpeg, we do not enable color conversion.

This is the logic that needs to be changed:
https://github.com/shuntaraw/Azure_Kinect_ROS_Driver/blob/f7f1cf98adb4c1168615b6c1a4ea02742fd487ba/src/k4a_ros_device.cpp

[shuntaraw]

@RoseFlunder Thanks for pointing this out. I will address this issue shortly.

[shuntaraw]

Apparently, the color format in a recorded file is either MJPG, NV12, or YUV2. Since the SDK does not support the conversion to MJPG, the logic implemented here is as follows:

• when JPEG is requested
  • if recorded image is JPEG, it is published.
  • otherwise the image is converted to BGRA and a warning is printed.
• when BGRA is requested
  • the image is converted to BGRA and then published.

I've tested this PR with the files recorded in MJPG, NV12, and YUV2.

[RoseFlunder]

Looks fine to me.
I more question:
Whats about the rectified color image when using jpeg?
I see that there is a rectified version for the bgra option in the code but not for the jpeg version.
Is it not possible?

[shuntaraw]

@RoseFlunder As mentioned in the "description of the changes", /rgb_to_depth/image_raw is advertised but does not publish anything. I believe it is impossible to rectify the image without having an RGB pixel array.

[shuntaraw]

@skalldri will you merge this?

[ooeygui]

@shuntaraw I'm doing a quick local validation and will merge.

[ooeygui]

Confirmed on Windows