From c635459b8ca642e8fe23895b6b58c4ff24d63df2 Mon Sep 17 00:00:00 2001 From: MOj0 Date: Tue, 9 Nov 2021 20:51:34 +0100 Subject: [PATCH 1/5] Added "How to preserve RAM" --- docs/source/index.rst | 1 + docs/source/tutorials/preserve_ram.rst | 18 ++++++++++++++++++ 2 files changed, 19 insertions(+) create mode 100644 docs/source/tutorials/preserve_ram.rst diff --git a/docs/source/index.rst b/docs/source/index.rst index e90dcc83a..783e6de57 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -78,6 +78,7 @@ Now, pick a tutorial or code sample and start utilizing Gen2 capabilities tutorials/hello_world.rst tutorials/multiple.rst tutorials/maximize_fov.rst + tutorials/preserve_ram.rst .. toctree:: :maxdepth: 1 diff --git a/docs/source/tutorials/preserve_ram.rst b/docs/source/tutorials/preserve_ram.rst new file mode 100644 index 000000000..6c0a41b77 --- /dev/null +++ b/docs/source/tutorials/preserve_ram.rst @@ -0,0 +1,18 @@ +How to preserve RAM +=================== + +The amount of memory consumed can be measured in debug mode with the following lines of code. + +.. code-block:: python + + device.setLogLevel(dai.LogLevel.DEBUG) + device.setLogOutputLevel(dai.LogLevel.DEBUG) + +There might be some memory issues when creating a larger number (around 10) of image manip, config and manip output nodes. + +A simple way to reduce the memory consumed is to reduce the video resolution to 1080P, however there are methods to reduce it further. + +Each ImageManip instance creates an output pool of 4 frames by default, this number can be controller with :code:`setNumFramesPool`. Each frame has 1 MiB of size, but that can also be controlled using the :code:`setMaxOutputFrameSize`, eg. if the output is 300x300 RGB, only 300x300x3=270000 bytes are needed. + +Furthermore, you can lower the data size for the XLinkIn connections using the :code:`setMaxDataSize` to set the number of bytes. +If XLinkIn is used to send config message, then 5*1024 is more than enough, if it's used to send image data then it should be the number of bytes that the image takes, e.g. for 1920x1080 RGB it would be 1920*1080*3. From d6b3827b835804e03c471111b4d3f6f9c9032d9b Mon Sep 17 00:00:00 2001 From: MOj0 Date: Thu, 11 Nov 2021 10:17:15 +0100 Subject: [PATCH 2/5] Updated according to PR comments --- docs/source/tutorials/preserve_ram.rst | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/source/tutorials/preserve_ram.rst b/docs/source/tutorials/preserve_ram.rst index 6c0a41b77..33fc46299 100644 --- a/docs/source/tutorials/preserve_ram.rst +++ b/docs/source/tutorials/preserve_ram.rst @@ -8,11 +8,13 @@ The amount of memory consumed can be measured in debug mode with the following l device.setLogLevel(dai.LogLevel.DEBUG) device.setLogOutputLevel(dai.LogLevel.DEBUG) -There might be some memory issues when creating a larger number (around 10) of image manip, config and manip output nodes. +There might be some memory issues when creating a larger number (around 10) of :ref:`ImageManip`, :ref:`XLinkIn` and other nodes. A simple way to reduce the memory consumed is to reduce the video resolution to 1080P, however there are methods to reduce it further. -Each ImageManip instance creates an output pool of 4 frames by default, this number can be controller with :code:`setNumFramesPool`. Each frame has 1 MiB of size, but that can also be controlled using the :code:`setMaxOutputFrameSize`, eg. if the output is 300x300 RGB, only 300x300x3=270000 bytes are needed. +Each ImageManip instance creates an output pool of 4 frames by default, this number can be controlled with :code:`setNumFramesPool`. Each frame has 1 MiB of size, but that can also be controlled using the :code:`setMaxOutputFrameSize`, eg. if the output is 300x300 RGB, only 300x300x3=270000 bytes are needed. Furthermore, you can lower the data size for the XLinkIn connections using the :code:`setMaxDataSize` to set the number of bytes. If XLinkIn is used to send config message, then 5*1024 is more than enough, if it's used to send image data then it should be the number of bytes that the image takes, e.g. for 1920x1080 RGB it would be 1920*1080*3. + +All devices have 0.5GB (4Gbit) RAM, which is used for firmware (about 15MB), assets (a few KB up to 100MB+, eg. Neural networks), and the rest of it is used by resources (node pools where messages are stored). \ No newline at end of file From df3a83a33958ffb356212f9b6ae15416eba7d03a Mon Sep 17 00:00:00 2001 From: Erol444 Date: Fri, 19 Nov 2021 14:31:07 +0100 Subject: [PATCH 3/5] Fixing RAM Usage docs --- docs/source/index.rst | 4 +- docs/source/tutorials/debugging.rst | 2 + docs/source/tutorials/preserve_ram.rst | 20 ---------- docs/source/tutorials/ram_usage.rst | 53 ++++++++++++++++++++++++++ 4 files changed, 57 insertions(+), 22 deletions(-) delete mode 100644 docs/source/tutorials/preserve_ram.rst create mode 100644 docs/source/tutorials/ram_usage.rst diff --git a/docs/source/index.rst b/docs/source/index.rst index 1cad2846c..48b615f33 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -78,9 +78,9 @@ Now, pick a tutorial or code sample and start utilizing Gen2 capabilities tutorials/hello_world.rst tutorials/multiple.rst tutorials/maximize_fov.rst - tutorials/preserve_ram.rst - tutorials/dispaying_detections.rst tutorials/debugging.rst + tutorials/ram_usage.rst + tutorials/dispaying_detections.rst .. toctree:: :maxdepth: 1 diff --git a/docs/source/tutorials/debugging.rst b/docs/source/tutorials/debugging.rst index 4b6016eef..4b6a36002 100644 --- a/docs/source/tutorials/debugging.rst +++ b/docs/source/tutorials/debugging.rst @@ -4,6 +4,8 @@ Debugging DepthAI pipeline Currently, tools for debugging the DepthAI pipeline are limited. We plan on creating a software that would track all messages and queues, which would allow users to debug a "frozen" pipeline much easier, which is usually caused by a filled up :ref:`blocking queue `. +.. _depthai_logging: + DepthAI debugging level ======================= diff --git a/docs/source/tutorials/preserve_ram.rst b/docs/source/tutorials/preserve_ram.rst deleted file mode 100644 index 33fc46299..000000000 --- a/docs/source/tutorials/preserve_ram.rst +++ /dev/null @@ -1,20 +0,0 @@ -How to preserve RAM -=================== - -The amount of memory consumed can be measured in debug mode with the following lines of code. - -.. code-block:: python - - device.setLogLevel(dai.LogLevel.DEBUG) - device.setLogOutputLevel(dai.LogLevel.DEBUG) - -There might be some memory issues when creating a larger number (around 10) of :ref:`ImageManip`, :ref:`XLinkIn` and other nodes. - -A simple way to reduce the memory consumed is to reduce the video resolution to 1080P, however there are methods to reduce it further. - -Each ImageManip instance creates an output pool of 4 frames by default, this number can be controlled with :code:`setNumFramesPool`. Each frame has 1 MiB of size, but that can also be controlled using the :code:`setMaxOutputFrameSize`, eg. if the output is 300x300 RGB, only 300x300x3=270000 bytes are needed. - -Furthermore, you can lower the data size for the XLinkIn connections using the :code:`setMaxDataSize` to set the number of bytes. -If XLinkIn is used to send config message, then 5*1024 is more than enough, if it's used to send image data then it should be the number of bytes that the image takes, e.g. for 1920x1080 RGB it would be 1920*1080*3. - -All devices have 0.5GB (4Gbit) RAM, which is used for firmware (about 15MB), assets (a few KB up to 100MB+, eg. Neural networks), and the rest of it is used by resources (node pools where messages are stored). \ No newline at end of file diff --git a/docs/source/tutorials/ram_usage.rst b/docs/source/tutorials/ram_usage.rst new file mode 100644 index 000000000..fb6812c9d --- /dev/null +++ b/docs/source/tutorials/ram_usage.rst @@ -0,0 +1,53 @@ +RAM usage +========= + +All devices have 0.5GB (4Gbit) RAM, which is used for firmware (about 15MB), assets (a few KB up to 100MB+, eg. NN models), and the rest of it +is used by resources, such as node pools where messages are stored. + +If you enable :code:`info` :ref:`logging `, you will see how RAM is used: + +.. code-block:: bash + + [info] Memory Usage - DDR: 41.23 / 358.82 MiB, CMX: 2.17 / 2.50 MiB, LeonOS Heap: 20.70 / 78.63 MiB, LeonRT Heap: 3.51 / 23.84 MiB + +As you can see, RAM is split between the two LEON (CPU) cores, :code:`CMX` (used for image manipulation), and :code:`DDR` (everything else). +If :code:`DDR` usage is close to the max (356MiB), you might get an error such as + +.. code-block:: bash + + [error] Neural network executor '0' out of '2' error: OUT_OF_MEMORY + +This means you should decrease RAM consumption, and we will take a look at a few ways on how to do this. + +Decreasing RAM consumption +########################## + +- **Large frames** + + If we change resolution from 1080P to 4K in the :ref:`RGB video` example, DDR usage will increase from 41 MiB to 161 MiB. That's because + 4K takes 4x more space compared to 1080P. So easy way to decrease RAM consumption is to use lower resolution / smaller frames. + +- **VideoEncoder** + + :ref:`VideoEncoder` nodes can consume a lot of RAM, especially at high resolutions. For example, :ref:`RGB Encoding` example consumes + 259 MiB. If we change resolution from 4K to 1080P, we decrease DDR consumption to only 65 MiB. + +- **ImageManip** + + Each :ref:`ImageManip` node will have it's own (output) pool of 4 frames (by default), so having multiple ImageManips that are manipulating + high resolution frames will consume a lot of DDR RAM. You can change the pool size with :code:`manip.setNumFramesPool(4)`. By default, + each pool "spot" will consume 1 MiB, even if it's a small 300x300 RGB frame (which is 270kB). Specifying the output frame size + can therefore decrease the RAM as well. Example for 300x300 RGB frame: :code:`manip.setMaxOutputFrameSize(270000)`. + +- **XLinkIn** + + Just like :ref:`ImageManip`, each :ref:`XLinkIn` node has it's own message pool as well. By default, each XLinkIn will consume 40 MiB, as + each pool "spot" has 5 MiB reserved, and there are 8 "spots" in the pool. If you are sending 300x300 RGB frames from the host to the device, + you can set :code:`xin.setMaxDataSize(270000)`, and also limit number of messages per pool :code:`xin.setNumFrames(4)`. This will decrease + DDR RAM consumption from 40 MiB to about 1 MiB. + + If you are just sending control/config from the host, you can + set :code:`xin.setMaxDataSize(1)`, as :ref:`CameraControl` and :ref:`ImageManipConfig` don't have any extra data + (like :ref:`NNData`/:ref:`ImgFrame`/:ref:`Buffer`). + +.. include:: ../includes/footer-short.rst \ No newline at end of file From 7a54c41ce96ac0a4dbf443ade1d3817d2f22b6fc Mon Sep 17 00:00:00 2001 From: Erol444 Date: Fri, 19 Nov 2021 14:41:20 +0100 Subject: [PATCH 4/5] Fix typos --- docs/source/tutorials/ram_usage.rst | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/docs/source/tutorials/ram_usage.rst b/docs/source/tutorials/ram_usage.rst index fb6812c9d..c1f74dc51 100644 --- a/docs/source/tutorials/ram_usage.rst +++ b/docs/source/tutorials/ram_usage.rst @@ -1,17 +1,18 @@ RAM usage ========= -All devices have 0.5GB (4Gbit) RAM, which is used for firmware (about 15MB), assets (a few KB up to 100MB+, eg. NN models), and the rest of it -is used by resources, such as node pools where messages are stored. +All devices have 0.5GB (4Gbit) on-board RAM, which is used for firmware (about 15MB), assets (a few KB up to 100MB+, eg. NN models), and other +resources, such as message pools where messages are stored. If you enable :code:`info` :ref:`logging `, you will see how RAM is used: .. code-block:: bash - [info] Memory Usage - DDR: 41.23 / 358.82 MiB, CMX: 2.17 / 2.50 MiB, LeonOS Heap: 20.70 / 78.63 MiB, LeonRT Heap: 3.51 / 23.84 MiB + [info] Memory Usage - DDR: 41.23 / 358.82 MiB, CMX: 2.17 / 2.50 MiB, + LeonOS Heap: 20.70 / 78.63 MiB, LeonRT Heap: 3.51 / 23.84 MiB As you can see, RAM is split between the two LEON (CPU) cores, :code:`CMX` (used for image manipulation), and :code:`DDR` (everything else). -If :code:`DDR` usage is close to the max (356MiB), you might get an error such as +If :code:`DDR` usage is close to the max (358 MiB), you might get an error such as: .. code-block:: bash @@ -24,20 +25,20 @@ Decreasing RAM consumption - **Large frames** - If we change resolution from 1080P to 4K in the :ref:`RGB video` example, DDR usage will increase from 41 MiB to 161 MiB. That's because - 4K takes 4x more space compared to 1080P. So easy way to decrease RAM consumption is to use lower resolution / smaller frames. + If we change the resolution from 1080P to 4K in the :ref:`RGB video` example, DDR usage will increase from 41 MiB to 161 MiB. That's because + 4K uses 4x more RAM compared to 1080P frame. An easy way to decrease RAM consumption is to use lower resolution / smaller frames. - **VideoEncoder** :ref:`VideoEncoder` nodes can consume a lot of RAM, especially at high resolutions. For example, :ref:`RGB Encoding` example consumes - 259 MiB. If we change resolution from 4K to 1080P, we decrease DDR consumption to only 65 MiB. + 259 MiB. If we change the resolution from 4K to 1080P, we decrease DDR consumption to only 65 MiB. - **ImageManip** Each :ref:`ImageManip` node will have it's own (output) pool of 4 frames (by default), so having multiple ImageManips that are manipulating high resolution frames will consume a lot of DDR RAM. You can change the pool size with :code:`manip.setNumFramesPool(4)`. By default, each pool "spot" will consume 1 MiB, even if it's a small 300x300 RGB frame (which is 270kB). Specifying the output frame size - can therefore decrease the RAM as well. Example for 300x300 RGB frame: :code:`manip.setMaxOutputFrameSize(270000)`. + can therefore decrease the RAM as well, eg. for a 300x300 RGB frame, you can set :code:`manip.setMaxOutputFrameSize(270000)`. - **XLinkIn** From 069c551d3bc20c0ed80771b7b1007401be7aae5c Mon Sep 17 00:00:00 2001 From: Erol444 Date: Fri, 19 Nov 2021 15:09:52 +0100 Subject: [PATCH 5/5] Fixed according to change requests --- docs/source/tutorials/ram_usage.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/tutorials/ram_usage.rst b/docs/source/tutorials/ram_usage.rst index c1f74dc51..445851061 100644 --- a/docs/source/tutorials/ram_usage.rst +++ b/docs/source/tutorials/ram_usage.rst @@ -1,7 +1,7 @@ RAM usage ========= -All devices have 0.5GB (4Gbit) on-board RAM, which is used for firmware (about 15MB), assets (a few KB up to 100MB+, eg. NN models), and other +All devices have 512 MiB (4 Gbit) on-board RAM, which is used for firmware (about 15MB), assets (a few KB up to 100MB+, eg. NN models), and other resources, such as message pools where messages are stored. If you enable :code:`info` :ref:`logging `, you will see how RAM is used: @@ -12,7 +12,7 @@ If you enable :code:`info` :ref:`logging `, you will see how RA LeonOS Heap: 20.70 / 78.63 MiB, LeonRT Heap: 3.51 / 23.84 MiB As you can see, RAM is split between the two LEON (CPU) cores, :code:`CMX` (used for image manipulation), and :code:`DDR` (everything else). -If :code:`DDR` usage is close to the max (358 MiB), you might get an error such as: +If :code:`DDR` usage is close to the max (in this example, 358 MiB), you might get an error such as: .. code-block:: bash