Converting ROS message with "8UC1" encoding to "mono8" fails.

[NikolausDemmel]

I have ROS image messages with "8UC1" encoding (recorded from an Intel RealSense ZR300 with their driver http://wiki.ros.org/realsense_camera). I want to calibrate the camera using the Kalibr Toolbox, but I face an exception that traces back to the cv_bridge conversion.

Is the following expected to fail?

In [16]: CVB.imgmsg_to_cv2(I, desired_encoding="mono8")
---------------------------------------------------------------------------
CvBridgeError                             Traceback (most recent call last)
<ipython-input-16-8414acdc7ad8> in <module>()
----> 1 CVB.imgmsg_to_cv2(I, desired_encoding="mono8")

/opt/ros/kinetic/lib/python2.7/dist-packages/cv_bridge/core.pyc in imgmsg_to_cv2(self, img_msg, desired_encoding)
    182             res = cvtColor2(im, img_msg.encoding, desired_encoding)
    183         except RuntimeError as e:
--> 184             raise CvBridgeError(e)
    185 
    186         return res

CvBridgeError: [8UC1] is not a color format. but [mono8] is. The conversion does not make sense

In [17]: I.encoding
Out[17]: '8UC1'

I would have thought that "8UC1" is a valid encoding (according to http://docs.ros.org/api/sensor_msgs/html/msg/Image.html and http://docs.ros.org/kinetic/api/sensor_msgs/html/image__encodings_8h_source.html it is). Also, the Kalibr implementation seems to expect that this is working (see https://github.com/ethz-asl/kalibr/blob/a43a7b29bc43410b8a39a7f89d4c333d9cdad1f1/aslam_offline_calibration/kalibr/python/kalibr_common/ImageDatasetReader.py#L135-L137).

Is this something that maybe used to work?

I'm using the latest released version on ROS Kinetic / Ubuntu 16.04.

[wkentaro]

This is because 8UC and mono8 are not recognized as the same encoding.
ros/common_msgs#107 and #180 will fix this issue.

[NikolausDemmel]

Thanks for the quick investigation and PRs!

[vrabaud]

I'll explain the error format: 8UC1 is not a color format, it just says that the image has one channel of size 8bit.
Mono8 is a color format: it is monochromatic with an 8bit channel.

Think about it this way: it does not make sense to convert 8UC3 to RGB as you don't know what you store in your 3 channels (could be BGR, YUV whatever). Same here, maybe your 1 channel is grayscale, maybe intensity, maybe it's not even color, it's a depth image.

[NikolausDemmel]

@vrabaud, that makes sense, thanks. But does that mean you are suggesting to not support this conversion, or was it purely informational on top of @wkentaro's fix? I would think that interpreting 8UC1 as intensity when converting to mono8 is a reasonable assumption, no?

[vrabaud]

I would not take that risk: typing is what makes ROS strong so you should make sure your driver returns the proper type.
Just waiting for feedback on ros/common_msgs#107 to close this one.

[NikolausDemmel]

@vrabaud, I see your point. I can ofc try to push for a change in the driver upstream.

However, sometimes you don't have control over your data source. I would also argue that it still can make sense to interpret something like a depth image encoded as 8UC1 as a greyscale image and do operations on that, so how would you implement that as opposed to https://github.com/ethz-asl/kalibr/blob/a43a7b29bc43410b8a39a7f89d4c333d9cdad1f1/aslam_offline_calibration/kalibr/python/kalibr_common/ImageDatasetReader.py#L135-L137? Does OpenCV distinguish between different single channel formats?

[NikolausDemmel]

To answer my own question, it looks like converting from mono8 or 8UC without desired encoding results in the same cv image, right?

So in the linked example, one could write instead of

elif data.encoding == "8UC1" or data.encoding == "mono8":
      img_data = np.array(self.CVB.imgmsg_to_cv2(data, desired_encoding="mono8"))

simply

elif data.encoding == "8UC1" or data.encoding == "mono8":
      img_data = np.array(self.CVB.imgmsg_to_cv2(data))

right?

[tfoote]

Indeed the data representation is the same. The operation is more like downcasting. Where you can declare the datatype to be mono8 which is a subset of 8UC with specific semantic meaning. A conversion method should be relatively easy to implement, which simply changes the declared datatype on an assertion from the developer that says it's grayscale. Which would be useful if the datasource cannot be updated, or you want to play fast and loose with the data (but you're being explicit in your code, and the tools can enforce correctness).

[vrabaud]

@NikolausDemmel , right, in your case, that would work. But your driver should really return a "mono8"

[martinakos]

what's the official solution to this?
I have the same problem using realsense2_camera node and camera_calibration.
Is there a way to remap the message type from 8UC1 to momo8 without having to change the code in realsense2_camera?