MJPEG decoder initialization failed

[duanyongli]

I sometimes enconter the problem, not everytime.
comand: roslaunch azure_kinect_ros_driver rectify_test.launch

Logs
[2019-08-01 16:49:34.768] [error] [t=6690] /home/username/Software/Azure-Kinect-Sensor-SDK/src/color/uvc_camerareader.cpp (1274): DecodeMJPEGtoBGRA32(). MJPEG decoder initialization failed

[2019-08-01 16:49:34.769] [error] [t=6691] /home/username/Software/Azure-Kinect-Sensor-SDK/src/sdk/k4a.c (271): capturesync_get_capture(device->capturesync, capture_handle, timeout_in_ms) returned failure in k4a_device_get_capture()
terminate called after throwing an instance of 'k4a::error'
what(): Failed to get capture from device!

Desktop (please complete the following information):

• OS: [Ubuntu16.04]
• ROS Version [Kinectic]

[tomerlevi444]

I'm having the same problem. Ubuntu 18.04 with ROS Melodic.

[eric-schleicher]

ditto me too...

[skalldri]

I've occasionally seen this issue as well, which is why I openned Issue #20 to stop using the MJPEG decoder. This won't fix all cases though, since we need to convert to BGRA to publish the colorized point cloud.

This feels like a bug in the Linux implementation of the SDK, but it only seems to repro when running the nodelet. I'll try to dig into this a bit more on Monday, and open a bug on the Azure Kinect SDK if I can get a reliable repro.

[eric-schleicher]

bump. this seems to happening a lot, without a deliberate workaround.

[skalldri]

Still investigating this: the MJPEG decoder on Linux is very opaque, so getting logs out of it is more challenging. A potential workaround might be to use the node version of the driver instead of the nodelet: I don't think I've seen the bug hit in the node.

[skalldri]

This is some issue initializing libjpeg. Unfortunately, the SDK isn't providing the error message from libjpeg for us to analyze. I've opened an issue on the SDK to see if they have insight / can help us out.

[skalldri]

I added a line to print the contents of tjGetErrorStr() and I'm getting this as the output:

[2019-08-23 11:31:25.386] [error] [t=7529] ../src/color/uvc_camerareader.cpp (1268): DecodeMJPEGtoBGRA32(). tjGetErrorStr(): Wrong JPEG library version: library is 80, caller expects 62

Not sure what that means... digging further.

[skalldri]

Potential hypothesis: maybe there are multiple instances of a libjpeg shared object being loaded into the nodelet manager, causing symbol collision?

[skalldri]

Also potentially possible: ROS might be adding a different version of libjpeg to LD_LIBRARY_PATH that's causing the Azure Kinect SDK to fail.

Similar to this issue: torch/image#171

[RoseFlunder]

@skalldri
This error does not occur when you start the nodelet version without image_proc (which uses opencv, which does also use linjpeg/libjpeg-turbo as third party lib in their repository).
Maybe this helps?
Could it happen that OpenCV version image_proc uses loads another version of libjpeg/libjpeg-turbo which isn't the same that the azure kinect sdk is compiled against?

In general opencv should work with the sdk because they also have an example in their repository which uses opencv.

[skalldri]

OpenCV does work with the Azure Kinect SDK, but there are no guarantees that any random version of OpenCV will work with any random version of the Kinect SDK. The version compatibility matrix is too complex to manage them all, and it looks like we've run into an issue.

It seems likely that a symbol conflict is responsible for this issue. I'll dig into it a bit more today, and if I don't find a good solution I'll move the image_proc phases into a different nodelet manager to ensure they're separated from the SDK. That process-level isolation should fix the issue, at the cost of message serialization/deserialization between the Kinect node and the image_proc nodelets.

[skalldri]

It looks like a version of libjpeg is being pulled in by compressed_image_transport, to do JPEG decompression. It's really tricky to debug this issue, since it only seems to happen when launching the nodelet through the launch file. If I gdb into the nodelet manager, then manually load the image_proc/Rectify and Azure Kinect Nodelets, it doesn't repro.

The best workaround for now might be to switch to the node... it's going to be tricky to untangle this.

[mokoker]

having the same issue,
[2019-09-03 23:09:59.062] [error] [t=6629] /home/vsts/work/1/s/extern/Azure-Kinect-Sensor-SDK/src/color/uvc_camerareader.cpp (1267): DecodeMJPEGtoBGRA32(). MJPEG decoder initialization failed

[2019-09-03 23:09:59.062] [error] [t=6630] /home/vsts/work/1/s/extern/Azure-Kinect-Sensor-SDK/src/sdk/k4a.c (271): capturesync_get_capture(device->capturesync, capture_handle, timeout_in_ms) returned failure in k4a_device_get_capture()
terminate called after throwing an instance of 'k4a::error'
what(): Failed to get capture from device!

[skalldri]

Root caused to an issue in the sensor SDK. This bug will be fixed when microsoft/Azure-Kinect-Sensor-SDK#672 is fixed.

[ooeygui]

The dependent issue in the SDK has been fixed. Resolving.

[AswinkarthikeyenAK]

Hi,
I am having the same issue while using the kinect ros driver in jetson nano device
Using ROS Melodic and Jetson Jetpack 4.3 with OpenCV 4.1.1

Can you kindly help me resolve the issue, please?

I cannot find a solution to fix this.

Please find the error below:

[ WARN] [1597785952.661227805]: Waiting for datum... [ WARN] [1597785953.198828374]: Initializing or re-initializing the device to realtime offset: 1597785947952913948 ns [ WARN] [1597785953.390871803]: Initializing or re-initializing the device to realtime offset: 1597785947941036877 ns [ WARN] [1597785953.985412322]: Initializing or re-initializing the device to realtime offset: 1597785947957346240 ns [ WARN] [1597785954.181852822]: Initializing or re-initializing the device to realtime offset: 1597785947931051835 ns [ WARN] [1597785956.457204583]: Initializing or re-initializing the device to realtime offset: 1597785947958165008 ns [ WARN] [1597785956.657073799]: Initializing or re-initializing the device to realtime offset: 1597785947929930090 ns [ WARN] [1597785956.857069112]: Initializing or re-initializing the device to realtime offset: 1597785947941786590 ns [ WARN] [1597785960.395351689]: Image processing thread is running behind. Expected max loop time: 0.200000000 Actual loop time: 0.215918908 [ WARN] [1597785960.674889873]: Initializing or re-initializing the device to realtime offset: 1597785947948941089 ns [2020-08-18 22:26:00.833] [error] [t=20111] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/color/uvc_camerareader.cpp (1300): DecodeMJPEGtoBGRA32(). MJPEG decode failed: -1 [2020-08-18 22:26:00.942] [error] [t=20112] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/queue/queue.c (112): queue_pop(). Queue "Queue_capture" was popped in a disabled state. [2020-08-18 22:26:00.943] [error] [t=20112] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/sdk/k4a.c (269): capturesync_get_capture(device->capturesync, capture_handle, timeout_in_ms) returned failure in k4a_device_get_capture() [2020-08-18 22:26:01.022] [error] [t=20111] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/queue/queue.c (112): queue_pop(). Queue "Queue_depth" was popped in a disabled state. terminate called after throwing an instance of 'k4a::error' what(): Failed to get capture from device! [ WARN] [1597785962.666794816]: Waiting for datum... [azure_kinect_ros_driver-7] process has died [pid 20042, exit code -6, cmd /home/ak/catkin_ws/devel/lib/azure_kinect_ros_driver/node __name:=azure_kinect_ros_driver __log:=/home/ak/.ros/log/5c0280fa-e199-11ea-9f31-00044be73986/azure_kinect_ros_driver-7.log]. log file: /home/ak/.ros/log/5c0280fa-e199-11ea-9f31-00044be73986/azure_kinect_ros_driver-7*.log

Thanks for your support in advance

[ooeygui]

@AswinkarthikeyenAK, Are you able to capture using the k4aviewer?

[AswinkarthikeyenAK]

@ooeygui,
Yes I am able to capture using k4aviewer.
Can you possibly let me know the next steps, please?
Thanks

[AswinkarthikeyenAK]

@ooeygui,
I have fixed this issues for now with my device.
These were the following I tried,

1. To fix the issues with __log:=/home/ak/.ros/log/5c0280fa-e199-11ea-9f31-00044be73986/azure_kinect_ros_driver-7.log]. log file: /home/ak/.ros/log/5c0280fa-e199-11ea-9f31-00044be73986/azure_kinect_ros_driver-7*.log,
   I did the catkin_make as suggested in the issue "process has died" when roslaunch driver.launch #70 .

2. I reduced the resolution of the camera to 720p and changed the depth parameters to _2X2BINNED with frame rate as 15 fps. These changes were made in the driver.launch files.

This fixed the issues I had initially reported here and the node doesn't die after sometime thereby continuously streaming the data.

Thanks

[ooeygui]

@AswinkarthikeyenAK thank you for posting back. The converting the output from the kinect to ROS friendly representations is a time consuming, which negatively impacts performance. it would be a good enhancement to identify that this is occurring and notify the user; and perhaps have a property allow downshifting the performance of the camera.