Body tracking not working well with dark suit jacket in large room

[adamiq]

Describe the bug
Body tracking seems to have a lot of issues keeping track of my arms when wearing a dark suit jacket in an open room. After I remove my jacket the tracking gets much better. This issue also happens when wearing a gray sweat jacket.

To Reproduce
Point the Kinect out into an empty area and try to use when wearing a dark jacket.

Expected behavior
Tracking should work similarly between wearing a jacking and not wearing a jacket like previous Kinect.

Screenshots
With Suit Jacket:

Without Suit Jacket:

Desktop (please complete the following information):

• OS with Version: Windows 1809
• SDK Version: 1.3.0
• RGB Camera Firmware: 1.6.102
• Depth Camera Firmware: 1.6.75
• Depth Config Firmware: 6109.7

[qm13]

@adamiq Is this recording using the included body tracking viewer or your implementation? The reason I ask is that the SDK always returns all 32 joints and consequently I would not expect the arms of the skeleton to disappear for a couple of frames in SDK viewer (they can be widely wrong of course).

[adamiq]

@qm13 this recording is from the body tracking viewer included with the body tracking SDK. both gifs above are from a single video. Would it help to send you the full video? If so, what's the best way to send that to you?

[qm13]

@adamiq you don't mention but I assume you are using the latest body tracking SDK (v0.9.4). We added joint confidence in this release and updated the viewer to not color LOW confidence joints. You are seeing a couple of frames where the joint confidence is LOW most likely as a result of the dark jacket you mentioned. Some fabrics (usually dark but not always) have very low IR reflectivity resulting in the Azure Kinect not seeing them. It is a limitation of IR based time of flight cameras.

[adamiq]

Yes, I am using the latest body tracking SDK. Attached are four more gifs, where I'm trying the same movement with the Kinect v1 and v2. Even though the v1 and v2 have more trouble with tracking joints in my suit, they still manage to keep track of my hands. I understand that the Azure body tracking is currently in preview but it seems like it's an issue that the Azure currently has lower quality tracking than older Kinects.

Kinect v1 With Suit Jacket:

Kinect v1 Without Suit Jacket:

Kinect v2 With Suit Jacket:

Kinect v2 Without Suit Jacket:

[adamiq]

Just for some quick clarification: the two issues we've been noticing most with the Azure tracking in a suit:

1. Hand/wrist tracking staying sort of on target but jumping around (as seen with my left arm in the Azure tracking gif with suit)

2. The arm joints snapping down to my side and then jumping back to my actual arm (as seen with my right arm in the Azure tracking gif with suit)

[billpottle]

@adamiq - I noticed something very similar to what you are seeing, but with legs instead of arms in #738 That issue was closed as fixed in 0.9.3, but i'm still having the issue with my application.

[qm13]

@adamiq we would like to get a recording that demonstrates the problems you are reporting. We require a signed data release agreement along with the recording. Could you provide me with contact details so I can provide you both the agreement and upload instructions?

[adamiq]

@qm13 Feel free to contact me at adam@iqfoundry.com

[adamiq]

@qm13 Have there been any updates? Have you been able to confirm the issue that we're running into? Thanks!