Repository navigation
|
Hi, I'm using D455 camera and I'm using it with the following configuration As i have understand it's the best configuration to good depth reading and also use full Fov of rgb camera. But I think that due to different sensors fov the RGB and Depth frames are not aligned. So it's neccesary to align them. My pipeline is like the following. I have observed that in Reading the Whitepaper: https://dev.realsenseai.com/docs/depth-post-processing-for-intel-realsense-depth-camera-d400-series/ I have noticed that the filters have to be applied on Disparity so it's necessary to apply Depth to disparity, them do postprocess and finally viceversa (Disparity to Depth).
My doubt is about alignment. What is the correct way of doing alignment? Do as I was doing as first step or do after all post-processing. Doing before postprocessing Doing after postprocessing So doing after postprocessing will generate bigger depth images with the consecuent memory usage... So, to conclude I need to get as much as possible the best depth image with less noise and also with good precission (I have to compare frames with Opencv and if I can reduce as much noise as possible I can do better comparissions with less false positive) Thanks for your help |
Replies: 1 comment 5 replies
|
Hi! You are asking the right questions—improper filter ordering and performing alignment too early are the two main reasons why depth data in API code looks noisier and performs worse than in the RealSense Viewer. Here is the short answer: All depth post-processing filters MUST be applied BEFORE the 1. When should the Align filter be applied? (Before vs. After)Always AFTER post-processing. If you apply alignment first:
2. When to apply the Threshold filter?The Threshold filter operates on the Depth domain (distance in meters/millimeters).
3. Best Recommended Pipeline (Order of Execution)To match the clean, stable depth output of the RealSense Viewer while maximizing performance for OpenCV processing, run your pipeline in this exact sequence:
Code Structure Example (C#)private void FrameSetPostprocess(FrameSet frames, out VideoFrame colorFrame, out DepthFrame depthFrame)
{
depthFrame = frames.DepthFrame;
// 1. Post-process RAW Depth FIRST
if (_camConfig.DepthPipelineCfg.DepthPostProcessing)
{
// Decimation & Threshold (Depth Domain)
depthFrame = _depthFilterDecimation.Process<DepthFrame>(depthFrame).DisposeWith(frames);
depthFrame = _depthFilterThreshold.Process<DepthFrame>(depthFrame).DisposeWith(frames);
// Transform to Disparity Domain
depthFrame = _depthToDisparity.Process<DepthFrame>(depthFrame).DisposeWith(frames);
// Temporal / Spatial filtering (Disparity Domain)
// depthFrame = _depthFilterSpatial.Process<DepthFrame>(depthFrame).DisposeWith(frames);
depthFrame = _depthFilterTemporal.Process<DepthFrame>(depthFrame).DisposeWith(frames);
// Transform back to Depth Domain
depthFrame = _disparityToDepth.Process<DepthFrame>(depthFrame).DisposeWith(frames);
}
// 2. NOW Align the clean, filtered depth to the Color frame
using (var processedFrameset = frames.ReplaceFrame(depthFrame))
{
var alignedFrames = _alignFilter.Process<FrameSet>(processedFrameset).DisposeWith(frames);
colorFrame = alignedFrames.ColorFrame.DisposeWith(frames);
depthFrame = alignedFrames.DepthFrame.DisposeWith(frames);
}
}By filtering at the native Hope this helps! I would be happy if you marked my answer as correct if it resolves your issue. |
Hi!
Glad that helped clarify things! You're asking exact, practical questions. Here is the breakdown for your setup:
1. Does lower depth resolution affect Alignment?
Not at all. The
Alignfilter doesn't do a basic 2D pixel resize—it projects pixels in 3D space using the camera's intrinsic and extrinsic calibration parameters.Applying alignment last on a downsampled depth map (e.g., 424x240) gives you the exact same geometric alignment to the 1280x800 RGB frame, but saves a massive amount of CPU processing.
2. Threshold filter placement & out-of-range pixels
You hit on a very sharp detail: spatial and temporal filters (especially hole-filling options) can occasionally create interpolated …