diff --git a/docs/source/components/nodes/color_camera.rst b/docs/source/components/nodes/color_camera.rst index c995f0bc8..daf4dcd8c 100644 --- a/docs/source/components/nodes/color_camera.rst +++ b/docs/source/components/nodes/color_camera.rst @@ -24,27 +24,40 @@ Inputs and Outputs .. code-block:: - ┌───────────────────┐ still - │ ├───────────► - inputConfig │ │ - ──────────────►│ │ preview - │ ColorCamera ├───────────► - inputControl │ │ - ──────────────►│ │ video - │ ├───────────► - └───────────────────┘ + ColorCamera node + ┌──────────────────────────────┐ + │ ┌─────────────┐ │ + │ │ Image │ raw │ raw + │ │ Sensor │---┬--------├────────► + │ └────▲────────┘ | │ + │ │ ┌--------┘ │ + │ ┌─┴───▼─┐ │ isp + inputControl │ │ │-------┬-------├────────► + ──────────────►│------│ ISP │ ┌─────▼────┐ │ video + │ │ │ | |--├────────► + │ └───────┘ │ Image │ │ still + inputConfig │ │ Post- │--├────────► + ──────────────►│----------------|Processing│ │ preview + │ │ │--├────────► + │ └──────────┘ │ + └──────────────────────────────┘ **Message types** - :code:`inputConfig` - :ref:`ImageManipConfig` - :code:`inputControl` - :ref:`CameraControl` -- :code:`still` - :ref:`ImgFrame` -- :code:`preview` - :ref:`ImgFrame` -- :code:`video` - :ref:`ImgFrame` - -:code:`Preview` is RGB (or BGR planar/interleaved if configured) and is mostly suited for small size previews and to feed the image -into :ref:`NeuralNetwork`. :code:`video` and :code:`still` are both NV12, so are suitable for bigger sizes. :code:`still` image gets created when -a capture event is sent to the ColorCamera, so it's like taking a photo. +- :code:`raw` - :ref:`ImgFrame` - RAW10 bayer data. Demo code for unpacking `here `__ +- :code:`isp` - :ref:`ImgFrame` - YUV420 planar (same as YU12/IYUV/I420) +- :code:`still` - :ref:`ImgFrame` - NV12, suitable for bigger size frames. The image gets created when a capture event is sent to the ColorCamera, so it's like taking a photo +- :code:`preview` - :ref:`ImgFrame` - RGB (or BGR planar/interleaved if configured), mostly suited for small size previews and to feed the image into :ref:`NeuralNetwork` +- :code:`video` - :ref:`ImgFrame` - NV12, suitable for bigger size frames + +**ISP** (image signal processor) is used for bayer transformation, demosaicing, noise reduction, and other image enhancements. +It interacts with the 3A algorithms: **auto-focus**, **auto-exposure**, and **auto-white-balance**, which are handling image sensor +adjustments such as exposure time, sensitivity (ISO), and lens position (if the camera module has a motorized lens) at runtime. +Click `here `__ for more information. + +**Image Post-Processing** converts YUV420 planar frames from the **ISP** into :code:`video`/:code:`preview`/:code:`still` frames. Usage ##### diff --git a/docs/source/components/nodes/stereo_depth.rst b/docs/source/components/nodes/stereo_depth.rst index f7ecdffef..306eaffd8 100644 --- a/docs/source/components/nodes/stereo_depth.rst +++ b/docs/source/components/nodes/stereo_depth.rst @@ -53,26 +53,26 @@ Inputs and Outputs Disparity ######### -When calculating the disparity, each pixel in the disparity map gets assigned a confidence value 0..255 by the stereo matching algorithm, as: -- 0 - maximum confidence that it holds a valid value -- 255 - minimum confidence, so there are chances the value is incorrect +When calculating the disparity, each pixel in the disparity map gets assigned a confidence value :code:`0..255` by the stereo matching algorithm, +as: + +- :code:`0` - maximum confidence that it holds a valid value +- :code:`255` - minimum confidence, so there is more chance that the value is incorrect + (this confidence score is kind-of inverted, if say comparing with NN) For the final disparity map, a filtering is applied based on the confidence threshold value: the pixels that have their confidence score larger than -the threshold get invalidated, i.e. their disparity value is set to zero. +the threshold get invalidated, i.e. their disparity value is set to zero. You can set the confidence threshold with :code:`stereo.setConfidenceThreshold()`. Current limitations ################### -If one or more of the additional depth modes (lrcheck, extended, subpixel) are enabled, then: - -- :code:`depth` output is FP16. -- median filtering is disabled on device. -- with subpixel, either depth or disparity has valid data. +If one or more of the additional depth modes (:code:`lrcheck`, :code:`extended`, :code:`subpixel`) are enabled, then: -Otherwise, depth output is U16 (in milimeters) and median is functional. +- median filtering is disabled on device +- with subpixel, if both :code:`depth` and :code:`disparity` are used, only :code:`depth` will have valid output -Like on Gen1, either :code:`depth` or :code:`disparity` has valid data. +Otherwise, :code:`depth` output is **U16** (in millimeters) and median is functional. Usage #####