Depth camera cannot be loaded

[vinesmsuic]

Describe the bug

I downloaded latest SDK (1.4.1) and opened the Azure Kinect Viewer through command sudo k4aviewer. The first time the k4aviewer works fine, but in the second time when I connected the device, opened the k4aviewer and clicking start with default options, the error message pops up: Failed to start device! If you unplugged the device, you must close and reopen the device. Then the camera outputs are not shown. I tried re-plugging the usb cables and ensure the back indicator is solid white, and restarting the computer. I found if I disable the depth camera and clicking start again the RGB camera will function properly.

To Reproduce

1. opened the Azure Kinect Viewer through command sudo k4aviewer
2. click on 'open device' and 'start' with default options
3. error message pops up: Failed to start device! If you unplugged the device, you must close and reopen the device.
4. after clicking dismiss, no camera output shown

Expected behavior

k4aviewer showing the camera outputs

Logs

[ error    ] : dynlib_create(). Failed to load shared object libdepthengine.so.2.0 with error: libdepthengine.so.2.0: cannot open shared object file: No such file or directory
[ error    ] : deloader_init_once(). Failed to Load Depth Engine Plugin (depthengine). Depth functionality will not work
[ error    ] : deloader_init_once(). Make sure the depth engine plugin is in your loaders path
[ error    ] : deloader_depth_engine_create_and_initialize(). Failed to load depth engine plugin
[ error    ] : depth_engine_start_helper(). Depth engine create and initialize failed with error code: 108.
[ error    ] : deresult == K4A_DEPTH_ENGINE_RESULT_SUCCEEDED returned failure in depth_engine_start_helper()
[ error    ] : depth_engine_start_helper(dewrapper, dewrapper->fps, dewrapper->depth_mode, &depth_engine_max_compute_time_ms, &depth_engine_output_buffer_size) returned failure in depth_engine_thread()
[ warning  ] : capturesync_add_capture(). Capture Error Detected, Depth 
[ error    ] : dewrapper_start(). Depth Engine thread failed to start
[ error    ] : dewrapper_start(depth->dewrapper, config, depth->calibration_memory, depth->calibration_memory_size) returned failure in depth_start()
[ error    ] : cmd_status == CMD_STATUS_PASS returned failure in depthmcu_depth_stop_streaming()
[ error    ] : depthmcu_depth_stop_streaming(). ERROR: cmd_status=0x00000063
[ error    ] : depth_start(device->depth, config) returned failure in k4a_device_start_cameras()

Screenshots

Error message pops up: Failed to start device! If you unplugged the device, you must close and reopen the device.

After disabling the depth camera and clicking start again, the RGB camera will function properly.

Desktop (please complete the following information):

• OS with Version: Linux Ubuntu 18.04
• SDK Version: 1.4.1
• Firmware version: 1.6.110079014

Additional context

My computer is dual boot, so I swtiched to windows and check my device manager and I saw the Depth camera is not found in the device manager. I tried replugging the cables and uninstalling the 4K camera and microphone array several times but still the depth camera is not shown.

[vinesmsuic]

Running enumerate_devices in Azure-Kinect-Sensor-SDK/build/bin folder:

Found 1 connected devices:
0: Device "001030302712"
[2020-12-24 20:36:33.446] [error] [t=3098] ../src/dynlib/dynlib_linux.c (82): dynlib_create(). Failed to load shared object libdepthengine.so.2.0 with error: libdepthengine.so.2.0: cannot open shared object file: No such file or directory
[2020-12-24 20:36:33.446] [error] [t=3098] ../src/deloader/deloader.cpp (75): deloader_init_once(). Failed to Load Depth Engine Plugin (depthengine). Depth functionality will not work
[2020-12-24 20:36:33.446] [error] [t=3098] ../src/deloader/deloader.cpp (76): deloader_init_once(). Make sure the depth engine plugin is in your loaders path

[RoseFlunder]

1. The SDK doens't need to be compiled by yourself. You can instead just install it via apt if you add the microsoft package repository.
2. The depth engine is closed source and comes with the apt package and is not included in this repo, thats why I recommend to use the apt package instead of cloning this repository.
   https://github.com/microsoft/Azure-Kinect-Sensor-SDK/blob/develop/docs/depthengine.md
3. You don't need sudo if you copy the rules file from the scripts folder in this repo to the folder described in the linux setup guide:
   https://github.com/microsoft/Azure-Kinect-Sensor-SDK/blob/develop/docs/usage.md
4. Have you tried different USB ports? The azure kinect is very sensible regarding specific usb controllers. Only a few work.

[vinesmsuic]

1. The SDK doens't need to be compiled by yourself. You can instead just install it via apt if you add the microsoft package repository.
2. The depth engine is closed source and comes with the apt package and is not included in this repo, thats why I recommend to use the apt package instead of cloning this repository.
   https://github.com/microsoft/Azure-Kinect-Sensor-SDK/blob/develop/docs/depthengine.md
3. You don't need sudo if you copy the rules file from the scripts folder in this repo to the folder described in the linux setup guide:
   https://github.com/microsoft/Azure-Kinect-Sensor-SDK/blob/develop/docs/usage.md
4. Have you tried different USB ports? The azure kinect is very sensible regarding specific usb controllers. Only a few work.

1. for installation, I first configured Microsoft's Package Repository and then install the k4atools through sudo apt install k4a-tools.
2. I did copy the rules file from the scripts folder and both sudo k4aviewer and k4aviewer will come out same error message.
3. My laptop only have 2 USB 3.0 ports and I have tried both of them. They yield the same result.
4. The k4aviewer has been working fine for a few times, but ever since I tried to install OpenGL (the only obvious thing I remember) and the k4aviewer has come to this situation (i.e. depth camera cannot be loaded)

[RoseFlunder]

I also ment installing the sdk via apt although it might be just a dependency of k4a-tools and though already installed.
Just looked like you also tried to compile the sdk yourself because you mention a sdk build folder once and in the log file is stated that the depth engine is missing (first line of your log file).
Thats usually a symptom for users that compile the sdk instead of getting it via apt.
In this case the rgb camera will work but the depth camera not because depth engine is only in binaries

[vinesmsuic]

Just looked like you also tried to compile the sdk yourself because you mention a sdk build folder once and in the log file is stated that the depth engine is missing (first line of your log file).
Thats usually a symptom for users that compile the sdk instead of getting it via apt.
In this case the rgb camera will work but the depth camera not because depth engine is only in binaries

This is what I did in setup:

curl -sSL https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -
sudo apt-add-repository https://packages.microsoft.com/ubuntu/18.04/prod
sudo apt-get update
sudo apt install k4a-tools

git clone https://github.com/microsoft/Azure-Kinect-Sensor-SDK.git
cd Azure-Kinect-Sensor-SDK
mkdir build && cd build
cmake .. -GNinja
ninja
sudo ninja install

How can I get the sdk via apt?

[RoseFlunder]

I think you could have just skipped the second step with cloning and building the repository.
The SDK should come with k4a-tools I think and if not you can install it with:
sudo apt install libk4a1.4
and if you also need the header files for developing own applications:
sudo apt install libk4a1.4-dev

You can check for installed k4a packages with:
apt list --installed | grep k4a

But if you are planning on using the body tracking sdk on Ubuntu there is a drawback.
Its only compatible with version 1.3 of the body tracking sdk on ubuntu at the moment.
In that case you have to remove k4a-tools and libk4a1.4 again and install them in version 1.3 again.
sudo apt intsall k4a-tools=1.3.0
sudo apt install libk4a1.3
sudo apt install libk4abt1.0

In this case its good to hold the k4a-tools version so that auto update doesnt break the body tracking sdk:
sudo apt-mark hold k4a-tools

Edit:
In your case I would just try to reinstall the SDK via apt.
I don't know if the "sudo ninja install" might have replaced things like k4aviewer and sdk on your system with a new built version that lacks the depthengine.
If you still get the message that depth engine is missing, I don't really know whats wrong, sorry.
But enjoy the christmas days anyways :)

[vinesmsuic]

You can check for installed k4a packages with:
apt list --installed | grep k4a

apt list --installed | grep k4a shown that I have all the all the dependency already.

k4a-tools/bionic,now 1.4.1 amd64 [installed]
libk4a1.4/bionic,now 1.4.1 amd64 [installed]
libk4a1.4-dev/bionic,now 1.4.1 amd64 [installed]

Isn't sudo apt install k4a-tools will automatically install libk4a1.4 and libk4a1.4-dev as well?

At first I only did sudo apt install k4a-tools and the k4aviewer works fine, and I found the library cannot be located in vscode (e.g. k4a is supposed to locate at /usr/local/include). Then I decided to compile the SDK again and now the k4a is found in /usr/local/include . I didn't notice doing this will crash k4aviewer.
I think it is possible that compiling the SDK has replaced something on my system.
https://github.com/microsoft/Azure-Kinect-Sensor-SDK/blob/develop/docs/depthengine.md here said If I am building the sensor SDK from source, you will need to obtain a copy from a binary release to read data from a device.
I unpacked both libk4a1.4 and libk4a1.4-dev from https://packages.microsoft.com/ubuntu/18.04/prod/pool/main/libk/ but the k4aviewer still yield the same error.

I am not planning to use the body tracking sdk at the moment but thanks for the info. Merry Christmas :)

[vinesmsuic]

I noticed since I built the k4a-tools via both apt and source, I have 2 k4aviewer in the system.

whereis k4aviewer
k4aviewer: /usr/bin/k4aviewer /usr/local/bin/k4aviewer

then I just tried to purge the k4a-tools and libk4a1.4 and libk4a1.4-dev, then reinstall libk4a1.3 and libk4a1.3-dev.

The depth engine is needed as well. The depth engine (DE) is a closed source binary shipped with the Linux Debian package. As an example, run apt install libk4a1.3 to install the Azure Kinect 1.3 and get the depth engine. See using the depth engine for information about versioning and adding the Microsoft's Package Repository to your machine. NOTE This step is not need for building, but is required running the SDK

sudo apt-get purge k4a-tools
sudo apt-get purge libk4a1.4
sudo apt-get purge libk4a1.4-dev
sudo apt install libk4a1.3
sudo apt install libk4a1.3-dev

Then now the k4aviewer should work fine. The depth camera is finally back.