Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 51 additions & 18 deletions docs/source/components/nodes/stereo_depth.rst
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
StereoDepth
===========
###########

Stereo depth node calculates the disparity/depth from two :ref:`mono cameras <MonoCamera>`.
StereoDepth node calculates the disparity/depth from the stereo camera pair (2x :ref:`MonoCamera <MonoCamera>`).

How to place it
###############
===============

.. tabs::

Expand All @@ -20,7 +20,7 @@ How to place it


Inputs and Outputs
##################
==================

.. code-block::

Expand Down Expand Up @@ -71,7 +71,7 @@ Inputs and Outputs
- :code:`debugDispCostDump` - :ref:`ImgFrame`

Internal block diagram of StereoDepth node
##########################################
==========================================

.. image:: /_static/images/components/depth_diagram.png
:target: https://whimsical.com/stereo-node-EKcfcXGjGpNL6cwRPV6NPv
Expand All @@ -83,7 +83,7 @@ configurable sooner.
If you click on the image, you will be redirected to the webapp. Some blocks have notes that provide additional technical information.

Currently configurable blocks
*****************************
-----------------------------

.. tabs::

Expand Down Expand Up @@ -165,12 +165,12 @@ Currently configurable blocks
:no-link:

Limitations
###########
===========

- Median filtering is disabled when subpixel mode is set to 4 or 5 bits.

Stereo depth FPS
################
================

.. list-table::
:header-rows: 1
Expand Down Expand Up @@ -201,7 +201,7 @@ All stereo modes were measured for :code:`depth` output with **5x5 median filter
to **60 FPS** and for 400P mono cameras were set to **110 FPS**.

Usage
#####
=====

.. tabs::

Expand Down Expand Up @@ -238,15 +238,15 @@ Usage
right->out.link(stereo->right);

Examples of functionality
#########################
=========================

- :ref:`Depth Preview`
- :ref:`RGB Depth alignment`
- :ref:`Mono & MobilenetSSD & Depth`
- :ref:`RGB & MobilenetSSD with spatial data`

Reference
#########
=========

.. tabs::

Expand All @@ -266,7 +266,7 @@ Reference
:undoc-members:

Disparity
#########
=========

Disparity refers to the distance between two corresponding points in the left and right image of a stereo pair.
By looking at the image below, it can be seen that point :code:`X` gets projected to :code:`XL = (u, v)` in the :code:`Left view` and :code:`XR = (p, q)` in the :code:`Right view`.
Expand All @@ -290,7 +290,7 @@ For the final disparity map, a filtering is applied based on the confidence thre
the threshold get invalidated, i.e. their disparity value is set to zero. You can set the confidence threshold with :code:`stereo.initialConfig.setConfidenceThreshold()`.

Calculate depth using disparity map
###################################
===================================

Disparity and depth are inversely related. As disparity decreases, depth increases exponentially depending on baseline and focal length. Meaning, if the disparity value is close to zero, then a small change in disparity generates a large change in depth. Similarly, if the disparity value is big, then large changes in disparity do not lead to a large change in depth.

Expand Down Expand Up @@ -339,7 +339,7 @@ Examples for calculating the depth value, using the OAK-D (7.5cm baseline):
Note the value of disparity depth data is stored in :code:`uint16`, where 0 is a special value, meaning that distance is unknown.

Min stereo depth distance
#########################
=========================

If the depth results for close-in objects look weird, this is likely because they are below the minimum depth-perception distance of the device.

Expand Down Expand Up @@ -375,8 +375,41 @@ or roughly 35cm.

See `these examples <https://github.com/luxonis/depthai-experiments/tree/master/gen2-camera-demo#real-time-depth-from-depthai-stereo-pair>`__ for how to enable Extended Disparity.

Disparity shift to lower min depth perception
---------------------------------------------

Another option to perceive closer depth range is to use disparity shift. Disparity shift will shift the starting point
of the disparity search, which will significantly decrease max depth (MazZ) perception, but it will also decrease min depth (MinZ) perception.
Disparity shift can be combined with extended/subpixel/LR-check modes.

.. image:: https://user-images.githubusercontent.com/18037362/189375017-2fa137d2-ad6b-46de-8899-6304bbc6c9d7.png

**Left graph** shows min and max disparity and depth for OAK-D (7.5cm baseline, 800P resolution, ~70° HFOV) by default (disparity shift=0). See :ref:`Calculate depth using disparity map`.
Since hardware (stereo block) has a fixed 95 pixel disparity search, DepthAI will search from 0 pixels (depth=INF) to 95 pixels (depth=71cm).

**Right graph** shows the same, but at disparity shift set to 30 pixels. This means that disparity search will be from 30 pixels (depth=2.2m) to 125 pixels (depth=50cm).
This also means that depth will be very accurate at the short range (**theoretically** below 5mm depth error).

**Limitations**:

- Because of the inverse relationship between disparity and depth, MaxZ will decrease much faster than MinZ as the disparity shift is increased. Therefore, it is **advised not to use a larger than necessary disparity shift**.
- Tradeoff in reducing the MinZ this way is that objects at **distances farther away than MaxZ will not be seen**.
- Because of the point above, **we only recommend using disparity shift when MaxZ is known**, such as having a depth camera mounted above a table pointing down at the table surface.
- Output disparity map is not expanded, only the depth map. So if disparity shift is set to 50, and disparity value obtained is 90, the real disparity is 140.

**Compared to Extended disparity**, disparity shift:

- **(+)** Is faster, as it doesn't require an extra computation, which means there's also no extra latency
- **(-)** Reduces the MaxZ (significantly), while extended disparity only reduces MinZ.

Disparity shift can be combined with extended disparity.

.. doxygenfunction:: dai::StereoDepthConfig::setDisparityShift
:project: depthai-core
:no-link:

Max stereo depth distance
#########################
=========================

The maximum depth perception distance depends on the :ref:`accuracy of the depth perception <Depth perception accuracy>`. The formula used to calculate this distance is an approximation, but is as follows:

Expand All @@ -397,7 +430,7 @@ So using this formula for existing models the *theoretical* max distance is:
If greater precision for long range measurements is required, consider enabling Subpixel Disparity or using a larger baseline distance between mono cameras. For a custom baseline, you could consider using `OAK-FFC <https://docs.luxonis.com/projects/hardware/en/latest/pages/DM1090.html>`__ device or design your own baseboard PCB with required baseline. For more information see Subpixel Disparity under the Stereo Mode tab in :ref:`this table <Currently configurable blocks>`.

Depth perception accuracy
#########################
=========================

Disparity depth works by matching features from one image to the other and its accuracy is based on multiple parameters:

Expand All @@ -415,7 +448,7 @@ Lower baseline enables us to detect the depth at a closer distance as long as th
So the common norm is to adjust the baseline according to how far/close we want to be able to detect objects.

Limitation
##########
==========

Since depth is calculated from disparity, which requires the pixels to overlap, there is inherently a vertical
band on the left side of the left mono camera and on the right side of the right mono camera, where depth
Expand Down Expand Up @@ -472,7 +505,7 @@ forum post.
`here <https://github.com/luxonis/depthai-hardware/issues/114>`__.

Measuring real-world object dimensions
######################################
======================================

Because the depth map contains the Z distance, objects in parallel with the camera are measured accurately standard. For objects not in parallel, the Euclidean distance calculation can be used. Please refer to the below:

Expand Down