From b5490325c09d70b371f4670004be15031af75128 Mon Sep 17 00:00:00 2001 From: Erol444 Date: Tue, 6 Sep 2022 22:45:58 +0200 Subject: [PATCH 1/3] Added disparity shift docs --- docs/source/components/nodes/stereo_depth.rst | 46 +++++++++++-------- 1 file changed, 28 insertions(+), 18 deletions(-) diff --git a/docs/source/components/nodes/stereo_depth.rst b/docs/source/components/nodes/stereo_depth.rst index 535a77302..87817aac2 100644 --- a/docs/source/components/nodes/stereo_depth.rst +++ b/docs/source/components/nodes/stereo_depth.rst @@ -1,10 +1,10 @@ StereoDepth -=========== +########### -Stereo depth node calculates the disparity/depth from two :ref:`mono cameras `. +StereoDepth node calculates the disparity/depth from the stereo camera pair (2x :ref:`MonoCamera `). How to place it -############### +=============== .. tabs:: @@ -20,7 +20,7 @@ How to place it Inputs and Outputs -################## +================== .. code-block:: @@ -71,7 +71,7 @@ Inputs and Outputs - :code:`debugDispCostDump` - :ref:`ImgFrame` Internal block diagram of StereoDepth node -########################################## +========================================== .. image:: /_static/images/components/depth_diagram.png :target: https://whimsical.com/stereo-node-EKcfcXGjGpNL6cwRPV6NPv @@ -83,7 +83,7 @@ configurable sooner. If you click on the image, you will be redirected to the webapp. Some blocks have notes that provide additional technical information. Currently configurable blocks -***************************** +----------------------------- .. tabs:: @@ -165,12 +165,12 @@ Currently configurable blocks :no-link: Limitations -########### +=========== - Median filtering is disabled when subpixel mode is set to 4 or 5 bits. Stereo depth FPS -################ +================ .. list-table:: :header-rows: 1 @@ -201,7 +201,7 @@ All stereo modes were measured for :code:`depth` output with **5x5 median filter to **60 FPS** and for 400P mono cameras were set to **110 FPS**. Usage -##### +===== .. tabs:: @@ -238,7 +238,7 @@ Usage right->out.link(stereo->right); Examples of functionality -######################### +========================= - :ref:`Depth Preview` - :ref:`RGB Depth alignment` @@ -246,7 +246,7 @@ Examples of functionality - :ref:`RGB & MobilenetSSD with spatial data` Reference -######### +========= .. tabs:: @@ -266,7 +266,7 @@ Reference :undoc-members: Disparity -######### +========= Disparity refers to the distance between two corresponding points in the left and right image of a stereo pair. By looking at the image below, it can be seen that point :code:`X` gets projected to :code:`XL = (u, v)` in the :code:`Left view` and :code:`XR = (p, q)` in the :code:`Right view`. @@ -290,7 +290,7 @@ For the final disparity map, a filtering is applied based on the confidence thre the threshold get invalidated, i.e. their disparity value is set to zero. You can set the confidence threshold with :code:`stereo.initialConfig.setConfidenceThreshold()`. Calculate depth using disparity map -################################### +=================================== Disparity and depth are inversely related. As disparity decreases, depth increases exponentially depending on baseline and focal length. Meaning, if the disparity value is close to zero, then a small change in disparity generates a large change in depth. Similarly, if the disparity value is big, then large changes in disparity do not lead to a large change in depth. @@ -339,7 +339,7 @@ Examples for calculating the depth value, using the OAK-D (7.5cm baseline): Note the value of disparity depth data is stored in :code:`uint16`, where 0 is a special value, meaning that distance is unknown. Min stereo depth distance -######################### +========================= If the depth results for close-in objects look weird, this is likely because they are below the minimum depth-perception distance of the device. @@ -375,8 +375,18 @@ or roughly 35cm. See `these examples `__ for how to enable Extended Disparity. +Disparity shift to lower min depth perception +--------------------------------------------- + +Another option to perceive closer depth range is to use disparity shift. Disparity shift will shift the starting point +of the disparity search, which will significantly decrease max depth perception, but it will also decrease min depth perception. + +.. doxygenfunction:: dai::StereoDepthConfig::setDisparityShift + :project: depthai-core + :no-link: + Max stereo depth distance -######################### +========================= The maximum depth perception distance depends on the :ref:`accuracy of the depth perception `. The formula used to calculate this distance is an approximation, but is as follows: @@ -397,7 +407,7 @@ So using this formula for existing models the *theoretical* max distance is: If greater precision for long range measurements is required, consider enabling Subpixel Disparity or using a larger baseline distance between mono cameras. For a custom baseline, you could consider using `OAK-FFC `__ device or design your own baseboard PCB with required baseline. For more information see Subpixel Disparity under the Stereo Mode tab in :ref:`this table `. Depth perception accuracy -######################### +========================= Disparity depth works by matching features from one image to the other and its accuracy is based on multiple parameters: @@ -415,7 +425,7 @@ Lower baseline enables us to detect the depth at a closer distance as long as th So the common norm is to adjust the baseline according to how far/close we want to be able to detect objects. Limitation -########## +========== Since depth is calculated from disparity, which requires the pixels to overlap, there is inherently a vertical band on the left side of the left mono camera and on the right side of the right mono camera, where depth @@ -472,7 +482,7 @@ forum post. `here `__. Measuring real-world object dimensions -###################################### +====================================== Because the depth map contains the Z distance, objects in parallel with the camera are measured accurately standard. For objects not in parallel, the Euclidean distance calculation can be used. Please refer to the below: From 5be75edeab65ade63308933de0efcc4e19b29e3a Mon Sep 17 00:00:00 2001 From: Erol444 Date: Fri, 9 Sep 2022 16:58:39 +0200 Subject: [PATCH 2/3] Updated disparity shift docs --- docs/source/components/nodes/stereo_depth.rst | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/source/components/nodes/stereo_depth.rst b/docs/source/components/nodes/stereo_depth.rst index 87817aac2..36c0b7b0a 100644 --- a/docs/source/components/nodes/stereo_depth.rst +++ b/docs/source/components/nodes/stereo_depth.rst @@ -381,6 +381,14 @@ Disparity shift to lower min depth perception Another option to perceive closer depth range is to use disparity shift. Disparity shift will shift the starting point of the disparity search, which will significantly decrease max depth perception, but it will also decrease min depth perception. +.. image:: https://user-images.githubusercontent.com/18037362/189375017-2fa137d2-ad6b-46de-8899-6304bbc6c9d7.png + +**Left graph** shows min and max disparity and depth for OAK-D (7.5cm baseline, 800P resolution, ~70° HFOV) by default (disparity shift=0). Since hardware (stereo +block) has a fixed 95 pixel disparity search, DepthAI will search from 0 pixels (depth=INF) to 95 pixels (depth=71cm). + +**Right graph** shows the same, but at disparity shift set to 30 pixels. This means that disparity search will be from 30 pixels (depth=2.2m) to 125 pixels (depth=50cm). +This also means that depth will be very accurate at the short range (**theoretically** below 5mm depth error). + .. doxygenfunction:: dai::StereoDepthConfig::setDisparityShift :project: depthai-core :no-link: From b83bd0dd0f88621b6bc5f79097a9849a7bb223ae Mon Sep 17 00:00:00 2001 From: Erol444 Date: Wed, 14 Sep 2022 20:03:29 +0200 Subject: [PATCH 3/3] Updated disparity shift docs --- docs/source/components/nodes/stereo_depth.rst | 21 ++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-) diff --git a/docs/source/components/nodes/stereo_depth.rst b/docs/source/components/nodes/stereo_depth.rst index 36c0b7b0a..6ae99ed25 100644 --- a/docs/source/components/nodes/stereo_depth.rst +++ b/docs/source/components/nodes/stereo_depth.rst @@ -379,16 +379,31 @@ Disparity shift to lower min depth perception --------------------------------------------- Another option to perceive closer depth range is to use disparity shift. Disparity shift will shift the starting point -of the disparity search, which will significantly decrease max depth perception, but it will also decrease min depth perception. +of the disparity search, which will significantly decrease max depth (MazZ) perception, but it will also decrease min depth (MinZ) perception. +Disparity shift can be combined with extended/subpixel/LR-check modes. .. image:: https://user-images.githubusercontent.com/18037362/189375017-2fa137d2-ad6b-46de-8899-6304bbc6c9d7.png -**Left graph** shows min and max disparity and depth for OAK-D (7.5cm baseline, 800P resolution, ~70° HFOV) by default (disparity shift=0). Since hardware (stereo -block) has a fixed 95 pixel disparity search, DepthAI will search from 0 pixels (depth=INF) to 95 pixels (depth=71cm). +**Left graph** shows min and max disparity and depth for OAK-D (7.5cm baseline, 800P resolution, ~70° HFOV) by default (disparity shift=0). See :ref:`Calculate depth using disparity map`. +Since hardware (stereo block) has a fixed 95 pixel disparity search, DepthAI will search from 0 pixels (depth=INF) to 95 pixels (depth=71cm). **Right graph** shows the same, but at disparity shift set to 30 pixels. This means that disparity search will be from 30 pixels (depth=2.2m) to 125 pixels (depth=50cm). This also means that depth will be very accurate at the short range (**theoretically** below 5mm depth error). +**Limitations**: + +- Because of the inverse relationship between disparity and depth, MaxZ will decrease much faster than MinZ as the disparity shift is increased. Therefore, it is **advised not to use a larger than necessary disparity shift**. +- Tradeoff in reducing the MinZ this way is that objects at **distances farther away than MaxZ will not be seen**. +- Because of the point above, **we only recommend using disparity shift when MaxZ is known**, such as having a depth camera mounted above a table pointing down at the table surface. +- Output disparity map is not expanded, only the depth map. So if disparity shift is set to 50, and disparity value obtained is 90, the real disparity is 140. + +**Compared to Extended disparity**, disparity shift: + +- **(+)** Is faster, as it doesn't require an extra computation, which means there's also no extra latency +- **(-)** Reduces the MaxZ (significantly), while extended disparity only reduces MinZ. + +Disparity shift can be combined with extended disparity. + .. doxygenfunction:: dai::StereoDepthConfig::setDisparityShift :project: depthai-core :no-link: