diff --git a/docs/source/_static/images/components/depth_diagram.jpeg b/docs/source/_static/images/components/depth_diagram.jpeg deleted file mode 100644 index 535da88b6..000000000 Binary files a/docs/source/_static/images/components/depth_diagram.jpeg and /dev/null differ diff --git a/docs/source/_static/images/components/depth_diagram.png b/docs/source/_static/images/components/depth_diagram.png new file mode 100644 index 000000000..a45405433 Binary files /dev/null and b/docs/source/_static/images/components/depth_diagram.png differ diff --git a/docs/source/_static/images/examples/depth_comparison.png b/docs/source/_static/images/examples/depth_comparison.png new file mode 100644 index 000000000..33ef4ba61 Binary files /dev/null and b/docs/source/_static/images/examples/depth_comparison.png differ diff --git a/docs/source/components/nodes/stereo_depth.rst b/docs/source/components/nodes/stereo_depth.rst index 0da516d51..cb3f0275e 100644 --- a/docs/source/components/nodes/stereo_depth.rst +++ b/docs/source/components/nodes/stereo_depth.rst @@ -73,7 +73,7 @@ Inputs and Outputs Internal block diagram of StereoDepth node ########################################## -.. image:: /_static/images/components/depth_diagram.jpeg +.. image:: /_static/images/components/depth_diagram.png :target: https://whimsical.com/stereo-node-EKcfcXGjGpNL6cwRPV6NPv On the diagram, red rectangle are firmware settings that are configurable via the API. Gray rectangles are settings that that are not yet @@ -126,9 +126,15 @@ Currently configurable blocks For comparison of normal disparity vs. subpixel disparity images, click `here `__. - .. tab:: Mesh file / Homography matrix + .. tab:: Depth Filters - Mesh files are generated using the camera intrinsics, distortion coeffs, and rectification rotations. + **Depth Filtering** / **Depth Post-Processing** is performed at the end of the depth pipeline. It helps with noise reduction and overall depth quality. + + .. include:: ../../includes/depth-filters.rst + + .. tab:: Mesh files + + Mesh files (homography matrix) are generated using the camera intrinsics, distortion coeffs, and rectification rotations. These files helps in overcoming the distortions in the camera increasing the accuracy and also help in when `wide FOV `__ lens are used. .. note:: @@ -148,7 +154,7 @@ Currently configurable blocks .. tab:: Confidence Threshold - - **Confidence threshold**: Stereo depth algorithm searches for the matching feature from right camera point to the left image (along the 96 disparity levels). During this process it computes the cost for each disparity level and choses the minimal cost between two disparities and uses it to compute the confidence at each pixel. Stereo node will output disparity/depth pixels only where depth confidence is below the **confidence threshold** (lower the confidence value means better depth accuracy). Note: This threshold only applies to Normal stereo mode as of now. + - **Confidence threshold**: Stereo depth algorithm searches for the matching feature from right camera point to the left image (along the 96 disparity levels). During this process it computes the cost for each disparity level and chooses the minimal cost between two disparities and uses it to compute the confidence at each pixel. Stereo node will output disparity/depth pixels only where depth confidence is below the **confidence threshold** (lower the confidence value means better depth accuracy). - **LR check threshold**: Disparity is considered for the output when the difference between LR and RL disparities is smaller than the LR check threshold. .. doxygenfunction:: dai::StereoDepthConfig::setConfidenceThreshold @@ -159,8 +165,8 @@ Currently configurable blocks :project: depthai-core :no-link: -Current limitations -################### +Limitations +########### - Median filtering is disabled when subpixel mode is set to 4 or 5 bits. diff --git a/docs/source/includes/depth-filters.rst b/docs/source/includes/depth-filters.rst new file mode 100644 index 000000000..1d5c79645 --- /dev/null +++ b/docs/source/includes/depth-filters.rst @@ -0,0 +1,65 @@ +.. tabs:: + + .. tab:: Median + + This is a non-edge preserving `Median filter `__, which can be used + to reduce noise and smoothen the depth map. Median filter is implemented in hardware, so it's the fastest filter. + + .. doxygenenum:: dai::MedianFilter + :project: depthai-core + :no-link: + + .. tab:: Speckle + + **Speckle Filter** is used to reduce the speckle noise. Speckle noise is a region with huge + variance between neighboring disparity/depth pixels, and speckle filter tries to filter this region. + + .. doxygenstruct:: dai::RawStereoDepthConfig::PostProcessing::SpeckleFilter + :project: depthai-core + :no-link: + :members: + + .. tab:: Temporal + + **Temporal Filter** is intended to improve the depth data persistency by manipulating per-pixel + values based on previous frames. The filter performs a single pass on the data, adjusting the depth + values while also updating the tracking history. In cases where the pixel data is missing or invalid, + the filter uses a user-defined persistency mode to decide whether the missing value should be + rectified with stored data. Note that due to its reliance on historic data the filter may introduce + visible blurring/smearing artifacts, and therefore is best-suited for static scenes. + + .. doxygenstruct:: dai::RawStereoDepthConfig::PostProcessing::TemporalFilter + :project: depthai-core + :no-link: + :members: + + .. tab:: Spatial + + **Spatial Edge-Preserving Filter** will fill invalid depth pixels with valid neighboring depth pixels. + It performs a series of 1D horizontal and vertical passes or iterations, to enhance the smoothness of + the reconstructed data. It is based on `this research paper `__. + + .. doxygenstruct:: dai::RawStereoDepthConfig::PostProcessing::SpatialFilter + :project: depthai-core + :no-link: + :members: + + .. tab:: Threshold + + **Threshold Filter** filters out all disparity/depth pixels outside the configured min/max + threshold values. + + .. autoclass:: depthai.RawStereoDepthConfig.PostProcessing.ThresholdFilter + :members: + :inherited-members: + :noindex: + + .. tab:: Decimation + + **Decimation Filter** will sub-samples the depth map, which means it reduces the depth scene complexity and allows + other filters to run faster. Setting :code:`decimationFactor` to 2 will downscale 1280x800 depth map to 640x400. + + .. doxygenstruct:: dai::RawStereoDepthConfig::PostProcessing::DecimationFilter + :members: + :project: depthai-core + :no-link: \ No newline at end of file diff --git a/docs/source/samples/StereoDepth/depth_post_processing.rst b/docs/source/samples/StereoDepth/depth_post_processing.rst new file mode 100644 index 000000000..37b4eb6eb --- /dev/null +++ b/docs/source/samples/StereoDepth/depth_post_processing.rst @@ -0,0 +1,49 @@ +Depth Post-Processing +===================== + +This example shows how you can run depth post-processing filters on the device itself to reduce noise, +smooth the depth map and overall improve the depth map quality. Post-processing can be added to :ref:`StereoDepth` node. + + +Demo +#### + +.. image:: /_static/images/examples/depth_comparison.png + +Depth filters +############# + +.. include:: /includes/depth-filters.rst + +.. rubric:: Similar samples: + +- :ref:`Depth Preview` +- :ref:`Stereo Depth from host` + +Setup +##### + +.. include:: /includes/install_from_pypi.rst + +Source code +########### + +.. tabs:: + + .. tab:: Python + + Also `available on GitHub `__ + + .. literalinclude:: ../../../../examples/StereoDepth/depth_post_processing.py + :language: python + :linenos: + + .. tab:: C++ + + Also `available on GitHub `__ + + .. literalinclude:: ../../../../depthai-core/examples/StereoDepth/depth_post_processing.cpp + :language: cpp + :linenos: + +.. include:: /includes/footer-short.rst diff --git a/docs/source/tutorials/code_samples.rst b/docs/source/tutorials/code_samples.rst index 13c8afcd8..29e82a814 100644 --- a/docs/source/tutorials/code_samples.rst +++ b/docs/source/tutorials/code_samples.rst @@ -122,6 +122,7 @@ are presented with code. .. rubric:: StereoDepth - :ref:`Depth Crop Control` - Demonstrates how to control cropping of depth frames from the host +- :ref:`Depth Post-Processing` - Depth post-processing filters - :ref:`Depth Preview` - Displays colorized stereo disparity - :ref:`Stereo Depth from host` - Generates stereo depth frame from a set of mono images from the host - :ref:`Stereo Depth Video` - An extended version of **Depth Preview**