From 44ab86bfcb537b2a36d138ab4374ca448482fcf7 Mon Sep 17 00:00:00 2001 From: MarcA711 <40744649+MarcA711@users.noreply.github.com> Date: Fri, 24 May 2024 09:44:36 +0000 Subject: [PATCH 1/5] improve docs for rockchip --- .../configuration/hardware_acceleration.md | 30 +---------- docs/docs/configuration/object_detectors.md | 37 ++------------ docs/docs/frigate/hardware.md | 12 +++++ docs/docs/frigate/installation.md | 50 +++++++++++++++++++ 4 files changed, 67 insertions(+), 62 deletions(-) diff --git a/docs/docs/configuration/hardware_acceleration.md b/docs/docs/configuration/hardware_acceleration.md index 030142adeb..93ee247a6a 100644 --- a/docs/docs/configuration/hardware_acceleration.md +++ b/docs/docs/configuration/hardware_acceleration.md @@ -366,35 +366,7 @@ Hardware accelerated video de-/encoding is supported on all Rockchip SoCs using ### Prerequisites -Make sure that you use a linux distribution that comes with the rockchip BSP kernel 5.10 or 6.1 and rkvdec2 driver. To check, enter the following commands: - -``` -$ uname -r -5.10.xxx-rockchip # or 6.1.xxx; the -rockchip suffix is important -$ ls /dev/dri -by-path card0 card1 renderD128 renderD129 # should list renderD128 -``` - -I recommend [Joshua Riek's Ubuntu for Rockchip](https://github.com/Joshua-Riek/ubuntu-rockchip), if your board is supported. - -### Setup - -Follow Frigate's default installation instructions, but use a docker image with `-rk` suffix for example `ghcr.io/blakeblackshear/frigate:stable-rk`. - -Next, you need to grant docker permissions to access your hardware: -- During the configuration process, you should run docker in privileged mode to avoid any errors due to insufficient permissions. To do so, add `privileged: true` to your `docker-compose.yml` file or the `--privileged` flag to your docker run command. -- After everything works, you should only grant necessary permissions to increase security. Add the lines below to your `docker-compose.yml` file or the following options to your docker run command: `--security-opt systempaths=unconfined --security-opt apparmor=unconfined --device /dev/dri:/dev/dri --device /dev/dma_heap:/dev/dma_heap --device /dev/rga:/dev/rga --device /dev/mpp_service:/dev/mpp_service`: - -```yaml - security_opt: - - apparmor=unconfined - - systempaths=unconfined - devices: - - /dev/dri:/dev/dri - - /dev/dma_heap:/dev/dma_heap - - /dev/rga:/dev/rga - - /dev/mpp_service:/dev/mpp_service -``` +Make sure to follow the [Rockchip specific installation instrucitions](/frigate/installation#rockchip-platform). ### Configuration diff --git a/docs/docs/configuration/object_detectors.md b/docs/docs/configuration/object_detectors.md index ed38fb2148..f73ce29827 100644 --- a/docs/docs/configuration/object_detectors.md +++ b/docs/docs/configuration/object_detectors.md @@ -313,39 +313,11 @@ Hardware accelerated object detection is supported on the following SoCs: - RK3576 - RK3588 -This implementation uses the [Rockchip's RKNN-Toolkit2](https://github.com/airockchip/rknn-toolkit2/) Currently, only [Yolo-NAS](https://github.com/Deci-AI/super-gradients/blob/master/YOLONAS.md) is supported as object detection model. +This implementation uses the [Rockchip's RKNN-Toolkit2](https://github.com/airockchip/rknn-toolkit2/). Currently, only [Yolo-NAS](https://github.com/Deci-AI/super-gradients/blob/master/YOLONAS.md) is supported as object detection model. ### Prerequisites -Make sure that you use a linux distribution that comes with the rockchip BSP kernel 5.10 or 6.1 and rknpu driver. To check, enter the following commands: - -``` -$ uname -r -5.10.xxx-rockchip # or 6.1.xxx; the -rockchip suffix is important -$ ls /dev/dri -by-path card0 card1 renderD128 renderD129 # should list renderD129 -$ sudo cat /sys/kernel/debug/rknpu/version -RKNPU driver: v0.9.2 # or later version -``` - -I recommend [Joshua Riek's Ubuntu for Rockchip](https://github.com/Joshua-Riek/ubuntu-rockchip), if your board is supported. - -### Setup - -Follow Frigate's default installation instructions, but use a docker image with `-rk` suffix for example `ghcr.io/blakeblackshear/frigate:stable-rk`. - -Next, you need to grant docker permissions to access your hardware: - -- During the configuration process, you should run docker in privileged mode to avoid any errors due to insufficient permissions. To do so, add `privileged: true` to your `docker-compose.yml` file or the `--privileged` flag to your docker run command. -- After everything works, you should only grant necessary permissions to increase security. Add the lines below to your `docker-compose.yml` file or the following options to your docker run command: `--security-opt systempaths=unconfined --security-opt apparmor=unconfined --device /dev/dri:/dev/dri`: - -```yaml -security_opt: - - apparmor=unconfined - - systempaths=unconfined -devices: - - /dev/dri:/dev/dri -``` +Make sure to follow the [Rockchip specific installation instrucitions](/frigate/installation#rockchip-platform). ### Configuration @@ -405,6 +377,5 @@ $ cat /sys/kernel/debug/rknpu/load ::: -- By default the rknn detector uses the yolonas_s model (`model: path: default-fp16-yolonas_s`). This model comes with the image, so no further steps than those mentioned above are necessary and no download happens. -- The other choices are automatically downloaded and stored in the folder `config/model_cache/rknn_cache`. After upgrading Frigate, you should remove older models to free up space. -- Finally, you can also provide your own `.rknn` model. You should not save your own models in the `rknn_cache` folder, store them directly in the `model_cache` folder or another subfolder. To convert a model to `.rknn` format see the `rknn-toolkit2` (requires a x86 machine). Note, that there is only post-processing for the supported models. +- All models are automatically downloaded and stored in the folder `config/model_cache/rknn_cache`. After upgrading Frigate, you should remove older models to free up space. +- You can also provide your own `.rknn` model. You should not save your own models in the `rknn_cache` folder, store them directly in the `model_cache` folder or another subfolder. To convert a model to `.rknn` format see the `rknn-toolkit2` (requires a x86 machine). Note, that there is only post-processing for the supported models. diff --git a/docs/docs/frigate/hardware.md b/docs/docs/frigate/hardware.md index 47e0e4ce17..1c805978bc 100644 --- a/docs/docs/frigate/hardware.md +++ b/docs/docs/frigate/hardware.md @@ -95,6 +95,18 @@ Frigate supports all Jetson boards, from the inexpensive Jetson Nano to the powe Inference speed will vary depending on the YOLO model, jetson platform and jetson nvpmodel (GPU/DLA/EMC clock speed). It is typically 20-40 ms for most models. The DLA is more efficient than the GPU, but not faster, so using the DLA will reduce power consumption but will slightly increase inference time. +#### Rockchip platform + +Frigate support hardware video processing on all Rockchip boards. However, hardware object detection is only supported on these boards: + +- RK3562 +- RK3566 +- RK3568 +- RK3576 +- RK3588 + +The inference time of an rk3588 with all 3 cores enabled is typically 25-30 ms for yolo-nas s. + ## What does Frigate use the CPU for and what does it use a detector for? (ELI5 Version) This is taken from a [user question on reddit](https://www.reddit.com/r/homeassistant/comments/q8mgau/comment/hgqbxh5/?utm_source=share&utm_medium=web2x&context=3). Modified slightly for clarity. diff --git a/docs/docs/frigate/installation.md b/docs/docs/frigate/installation.md index 0ff4c464f9..d9003bd582 100644 --- a/docs/docs/frigate/installation.md +++ b/docs/docs/frigate/installation.md @@ -95,6 +95,56 @@ By default, the Raspberry Pi limits the amount of memory available to the GPU. I Additionally, the USB Coral draws a considerable amount of power. If using any other USB devices such as an SSD, you will experience instability due to the Pi not providing enough power to USB devices. You will need to purchase an external USB hub with it's own power supply. Some have reported success with this (affiliate link). +### Rockchip platform + +Make sure that you use a linux distribution that comes with the rockchip BSP kernel 5.10 or 6.1 and necessary drivers (especially rkvdec2 and rknpu). To check, enter the following commands: + +``` +$ uname -r +5.10.xxx-rockchip # or 6.1.xxx; the -rockchip suffix is important +$ ls /dev/dri +by-path card0 card1 renderD128 renderD129 # should list renderD128 (VPU) and renderD129 (NPU) +$ sudo cat /sys/kernel/debug/rknpu/version +RKNPU driver: v0.9.2 # or later version +``` + +I recommend [Joshua Riek's Ubuntu for Rockchip](https://github.com/Joshua-Riek/ubuntu-rockchip), if your board is supported. + +#### Setup + +Follow Frigate's default installation instructions, but use a docker image with `-rk` suffix for example `ghcr.io/blakeblackshear/frigate:stable-rk`. + +Next, you need to grant docker permissions to access your hardware: + +- During the configuration process, you should run docker in privileged mode to avoid any errors due to insufficient permissions. To do so, add `privileged: true` to your `docker-compose.yml` file or the `--privileged` flag to your docker run command. +- After everything works, you should only grant necessary permissions to increase security. Disable the privileged mode and add the lines below to your `docker-compose.yml` file: + +```yaml +security_opt: + - apparmor=unconfined + - systempaths=unconfined +devices: + - /dev/dri + - /dev/dma_heap + - /dev/rga + - /dev/mpp_service +``` + +or add these options to your `docker run` command: + +``` +--security-opt systempaths=unconfined \ +--security-opt apparmor=unconfined \ +--device /dev/dri \ +--device /dev/dma_heap \ +--device /dev/rga \ +--device /dev/mpp_service +``` + +#### Configuration + +Next, you should configure [hardware object detection](/configuration/object_detectors#rockchip-platform) and [hardware video processing](/configuration/hardware_acceleration#rockchip-platform). + ## Docker Running in Docker with compose is the recommended install method. From cd84f696db855a4cb0f6867a7c0f7312d3c8db8c Mon Sep 17 00:00:00 2001 From: MarcA711 <40744649+MarcA711@users.noreply.github.com> Date: Fri, 24 May 2024 09:49:05 +0000 Subject: [PATCH 2/5] update version info --- docs/docs/configuration/hardware_acceleration.md | 2 +- docs/docs/configuration/object_detectors.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/docs/configuration/hardware_acceleration.md b/docs/docs/configuration/hardware_acceleration.md index 93ee247a6a..712960567c 100644 --- a/docs/docs/configuration/hardware_acceleration.md +++ b/docs/docs/configuration/hardware_acceleration.md @@ -362,7 +362,7 @@ that NVDEC/NVDEC1 are in use. ## Rockchip platform -Hardware accelerated video de-/encoding is supported on all Rockchip SoCs using [Nyanmisaka's FFmpeg Fork](https://github.com/nyanmisaka/ffmpeg-rockchip) based on [Rockchip's mpp library](https://github.com/rockchip-linux/mpp). +Hardware accelerated video de-/encoding is supported on all Rockchip SoCs using [Nyanmisaka's FFmpeg 6.1 Fork](https://github.com/nyanmisaka/ffmpeg-rockchip) based on [Rockchip's mpp library](https://github.com/rockchip-linux/mpp). ### Prerequisites diff --git a/docs/docs/configuration/object_detectors.md b/docs/docs/configuration/object_detectors.md index f73ce29827..9c4dc9893a 100644 --- a/docs/docs/configuration/object_detectors.md +++ b/docs/docs/configuration/object_detectors.md @@ -313,7 +313,7 @@ Hardware accelerated object detection is supported on the following SoCs: - RK3576 - RK3588 -This implementation uses the [Rockchip's RKNN-Toolkit2](https://github.com/airockchip/rknn-toolkit2/). Currently, only [Yolo-NAS](https://github.com/Deci-AI/super-gradients/blob/master/YOLONAS.md) is supported as object detection model. +This implementation uses the [Rockchip's RKNN-Toolkit2](https://github.com/airockchip/rknn-toolkit2/), version v2.0.0.beta0. Currently, only [Yolo-NAS](https://github.com/Deci-AI/super-gradients/blob/master/YOLONAS.md) is supported as object detection model. ### Prerequisites From f5fa4f93fdb047a589456d60862f3c7c46c1a4ee Mon Sep 17 00:00:00 2001 From: Marc Altmann <40744649+MarcA711@users.noreply.github.com> Date: Fri, 24 May 2024 12:04:36 +0200 Subject: [PATCH 3/5] fix typo --- docs/docs/frigate/hardware.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/frigate/hardware.md b/docs/docs/frigate/hardware.md index 1c805978bc..3bad43401c 100644 --- a/docs/docs/frigate/hardware.md +++ b/docs/docs/frigate/hardware.md @@ -105,7 +105,7 @@ Frigate support hardware video processing on all Rockchip boards. However, hardw - RK3576 - RK3588 -The inference time of an rk3588 with all 3 cores enabled is typically 25-30 ms for yolo-nas s. +The inference time of a rk3588 with all 3 cores enabled is typically 25-30 ms for yolo-nas s. ## What does Frigate use the CPU for and what does it use a detector for? (ELI5 Version) From 27f8963e5df40221f899f807158b97a83b411e1b Mon Sep 17 00:00:00 2001 From: Marc Altmann <40744649+MarcA711@users.noreply.github.com> Date: Fri, 24 May 2024 16:26:06 +0200 Subject: [PATCH 4/5] fix typo Co-authored-by: Nicolas Mowen --- docs/docs/configuration/hardware_acceleration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/configuration/hardware_acceleration.md b/docs/docs/configuration/hardware_acceleration.md index 712960567c..1a29220dde 100644 --- a/docs/docs/configuration/hardware_acceleration.md +++ b/docs/docs/configuration/hardware_acceleration.md @@ -366,7 +366,7 @@ Hardware accelerated video de-/encoding is supported on all Rockchip SoCs using ### Prerequisites -Make sure to follow the [Rockchip specific installation instrucitions](/frigate/installation#rockchip-platform). +Make sure to follow the [Rockchip specific installation instructions](/frigate/installation#rockchip-platform). ### Configuration From d611762c29e32d35617959317130e125d3e80207 Mon Sep 17 00:00:00 2001 From: Marc Altmann <40744649+MarcA711@users.noreply.github.com> Date: Fri, 24 May 2024 16:26:17 +0200 Subject: [PATCH 5/5] fix typo Co-authored-by: Nicolas Mowen --- docs/docs/frigate/hardware.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/frigate/hardware.md b/docs/docs/frigate/hardware.md index 3bad43401c..602cc906f0 100644 --- a/docs/docs/frigate/hardware.md +++ b/docs/docs/frigate/hardware.md @@ -97,7 +97,7 @@ Inference speed will vary depending on the YOLO model, jetson platform and jetso #### Rockchip platform -Frigate support hardware video processing on all Rockchip boards. However, hardware object detection is only supported on these boards: +Frigate supports hardware video processing on all Rockchip boards. However, hardware object detection is only supported on these boards: - RK3562 - RK3566