Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 29 additions & 16 deletions docs/source/components/nodes/color_camera.rst
Original file line number Diff line number Diff line change
Expand Up @@ -24,27 +24,40 @@ Inputs and Outputs

.. code-block::

┌───────────────────┐ still
│ ├───────────►
inputConfig │ │
──────────────►│ │ preview
│ ColorCamera ├───────────►
inputControl │ │
──────────────►│ │ video
│ ├───────────►
└───────────────────┘
ColorCamera node
┌──────────────────────────────┐
│ ┌─────────────┐ │
│ │ Image │ raw │ raw
│ │ Sensor │---┬--------├────────►
│ └────▲────────┘ | │
│ │ ┌--------┘ │
│ ┌─┴───▼─┐ │ isp
inputControl │ │ │-------┬-------├────────►
──────────────►│------│ ISP │ ┌─────▼────┐ │ video
│ │ │ | |--├────────►
│ └───────┘ │ Image │ │ still
inputConfig │ │ Post- │--├────────►
──────────────►│----------------|Processing│ │ preview
│ │ │--├────────►
│ └──────────┘ │
└──────────────────────────────┘

**Message types**

- :code:`inputConfig` - :ref:`ImageManipConfig`
- :code:`inputControl` - :ref:`CameraControl`
- :code:`still` - :ref:`ImgFrame`
- :code:`preview` - :ref:`ImgFrame`
- :code:`video` - :ref:`ImgFrame`

:code:`Preview` is RGB (or BGR planar/interleaved if configured) and is mostly suited for small size previews and to feed the image
into :ref:`NeuralNetwork`. :code:`video` and :code:`still` are both NV12, so are suitable for bigger sizes. :code:`still` image gets created when
a capture event is sent to the ColorCamera, so it's like taking a photo.
- :code:`raw` - :ref:`ImgFrame` - RAW10 bayer data. Demo code for unpacking `here <https://github.com/luxonis/depthai-experiments/blob/3f1b2b2/gen2-color-isp-raw/main.py#L13-L32>`__
- :code:`isp` - :ref:`ImgFrame` - YUV420 planar (same as YU12/IYUV/I420)
- :code:`still` - :ref:`ImgFrame` - NV12, suitable for bigger size frames. The image gets created when a capture event is sent to the ColorCamera, so it's like taking a photo
- :code:`preview` - :ref:`ImgFrame` - RGB (or BGR planar/interleaved if configured), mostly suited for small size previews and to feed the image into :ref:`NeuralNetwork`
- :code:`video` - :ref:`ImgFrame` - NV12, suitable for bigger size frames

**ISP** (image signal processor) is used for bayer transformation, demosaicing, noise reduction, and other image enhancements.
It interacts with the 3A algorithms: **auto-focus**, **auto-exposure**, and **auto-white-balance**, which are handling image sensor
adjustments such as exposure time, sensitivity (ISO), and lens position (if the camera module has a motorized lens) at runtime.
Click `here <https://en.wikipedia.org/wiki/Image_processor>`__ for more information.

**Image Post-Processing** converts YUV420 planar frames from the **ISP** into :code:`video`/:code:`preview`/:code:`still` frames.

Usage
#####
Expand Down
22 changes: 11 additions & 11 deletions docs/source/components/nodes/stereo_depth.rst
Original file line number Diff line number Diff line change
Expand Up @@ -53,26 +53,26 @@ Inputs and Outputs
Disparity
#########

When calculating the disparity, each pixel in the disparity map gets assigned a confidence value 0..255 by the stereo matching algorithm, as:
- 0 - maximum confidence that it holds a valid value
- 255 - minimum confidence, so there are chances the value is incorrect
When calculating the disparity, each pixel in the disparity map gets assigned a confidence value :code:`0..255` by the stereo matching algorithm,
as:

- :code:`0` - maximum confidence that it holds a valid value
- :code:`255` - minimum confidence, so there is more chance that the value is incorrect

(this confidence score is kind-of inverted, if say comparing with NN)

For the final disparity map, a filtering is applied based on the confidence threshold value: the pixels that have their confidence score larger than
the threshold get invalidated, i.e. their disparity value is set to zero.
the threshold get invalidated, i.e. their disparity value is set to zero. You can set the confidence threshold with :code:`stereo.setConfidenceThreshold()`.

Current limitations
###################

If one or more of the additional depth modes (lrcheck, extended, subpixel) are enabled, then:

- :code:`depth` output is FP16.
- median filtering is disabled on device.
- with subpixel, either depth or disparity has valid data.
If one or more of the additional depth modes (:code:`lrcheck`, :code:`extended`, :code:`subpixel`) are enabled, then:

Otherwise, depth output is U16 (in milimeters) and median is functional.
- median filtering is disabled on device
- with subpixel, if both :code:`depth` and :code:`disparity` are used, only :code:`depth` will have valid output

Like on Gen1, either :code:`depth` or :code:`disparity` has valid data.
Otherwise, :code:`depth` output is **U16** (in millimeters) and median is functional.

Usage
#####
Expand Down