Drift

[dcanelhas]

Experiment:

• Start InfiniTAM
• Reconstruct small environment,
• Place camera on table, pointed at static part of scene,
• Wait for a couple of minutes;
• Use f to see the scene from a different viewpoint

Expected result:
slight surface degradation due to quantization and continuous integration without motion
Actual result:

Possible reasons why it happens:

• There is some offset somewhere in the image alignment (maybe a missing +0.5 on a pixel coordinate somewhere)?
• The TSDF values are not being returned correctly
• Factory calibration is not enough?

[olafkaehler]

Before I try reproducing it, I assume you are using the depth tracker (ICP) and not the colour tracker? If so, then only the intrinsic calibration of the depth camera is used and maybe the OpenNI internal conversion of disparities to depth values. Since this calibration of the depth sensor is used both for reconstruction and for tracking, it should not cause any drift, regardless of how wrong it is.

Also, we are not using OpenGL- or CUDA-related images anywhere, which are notorious for the +0.5 coordinate bugs. I doubt there is any such issue in the current implementation, though.

That leaves the last of the three ideas, the TSDF values. These values are not actually returned anywhere for tracking, but they are used for raycasting, and it might be that a small bias in the raycasting leads to the observed drift. This would be a nightmare to track down. I'd assume in this case the direction of drift would vary depending on the scene context. Could you try and verify this?

Of course it might also be something completely different. I might find a bit of time to look for the problem myself...

[dcanelhas]

I'm using the standard depth tracker (from a clean git clone/build) and OpenNI2 with an asus xtion pro.

I'm suspecting the third option, since I ran into something similar earlier when trying to port my own sdf-tracker to the InfTAM framework. I kept getting slight offsets on the values I was reading out from it when fusing/evaluating into/from the tsdf without applying any transformations and not moving the camera. I didn't keep probing because I assumed I was doing something wrong (some transposed index or something) since, as far as i understand, the Ren tracker uses the TSDF values directly, too. Maybe the robust loss function it uses has been set to throw away the offset points as outliers though? (just brainstorming here).

A way to verify it would be perhaps to:

• Get depth frame - d,
• fuse into volume
• convert d to vertex map p,
• query volume at p
  the result should be zero within [numeric representation] precision.

If you do this, an example code snippet for how to do this would be helpful :)

[dcanelhas]

Tried a couple of different scenes
It seems to "prefer" drifting to the right, forward and down

Here I started with the camera rotated ca 90 degrees about the view direction to see if the result would be different.

but it still seems to go right, forward and down so I'm guessing it's something in image-space

[dcanelhas]

Not sure if anything has changed to address this issue specifically, but it doesn't appear to be happening in the current version so I will close this issue.

[lyxLeo-0728]

Hello, I have encountered the same problem. My problem is that the disparity parameters in calib.txt has not been replaced with a new sensor after the Azure kinect DK has been replaced. hope it can help you