From f7651168063ec034ed87a63adb25cfb533afcd5f Mon Sep 17 00:00:00 2001 From: Wu MinLong <726485702@qq.com> Date: Tue, 21 Oct 2025 20:51:36 +0800 Subject: [PATCH 1/2] fix: translate cubie npu use docs --- .../current/common/ai/_cubie_acuity_usage.mdx | 224 ++++++++++-------- .../current/common/ai/_cubie_lenet.mdx | 177 +++++++++----- .../current/common/ai/_cubie_resnet50.mdx | 154 ++++++++---- .../current/common/ai/_cubie_vpm_run.mdx | 184 ++++++++------ .../current/common/ai/_cubie_yolact.mdx | 92 ++++--- .../current/common/ai/_cubie_yolov5.mdx | 194 +++++++++------ 6 files changed, 633 insertions(+), 392 deletions(-) diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_acuity_usage.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_acuity_usage.mdx index c0d42fa9b..eac0bcb28 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_acuity_usage.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_acuity_usage.mdx @@ -1,13 +1,14 @@ -This example uses a MobileNetV2 target recognition model in Keras format, and uses Acuity Toolkit to parse, quantize, and compile the model, generating project code, and using Vivante IDE for simulation, finally inferring on the VIP9000 series NPU. +This guide uses a MobileNetV2 object detection model in Keras format as an example. We will use the Acuity Toolkit to parse, quantize, and compile the model, generate project code, perform simulation with Vivante IDE, and finally run inference on the VIP9000 series NPU. -## NPU Version Table +## NPU Version Comparison Table -| Product | NPU | SoC | NPU Version | NPU Software Version | -| --------- | ------ | ---- | ----------- | -------------------- | -| Radxa A5E | 2 Tops | T527 | v2 | v1.13 | -| Radxa A7A | 3 Tops | A733 | v3 | v2.0 | +| Product | TOPS | Platform | NPU Version | NPU Software Version | +| --------- | ------ | -------- | ----------- | -------------------- | +| Radxa A5E | 2 Tops | T527 | v2 | v1.13 | +| Radxa A7A | 3 Tops | A733 | v3 | v2.0 | +| Radxa A7Z | 3 Tops | A733 | v3 | v2.0 | -{/* ## Configure environment variables */} +{/* ## Configure Environment Variables */} {/* - Initialize ACUITY pegasus.py script */} {/* ```bash */} {/* python3 ~/work/Acuity/toolkit/bin/pegasus.py completion */} @@ -33,11 +34,11 @@ This example uses a MobileNetV2 target recognition model in Keras format, and us {/* export IDE=~/VeriSilicon/VivanteIDE5.10.0/cmdtools */} {/* ``` */} -## Download example repository +## Download Example Repository -In [ACUITY Docker container](./cubie_acuity_env#创建-docker-容器) download example repository +In the [ACUITY Docker container](./cubie_acuity_env#create-docker-container), download the example repository. -Download repository +Download the repository: @@ -47,25 +48,42 @@ git clone https://github.com/ZIFENG278/ai-sdk.git -Configure model compilation script +Configure the model compilation script: - + + -```bash -cd ai-sdk/models -source env.sh v3 # NPU_VERSION -cp ../scripts/* . -``` + + ```bash + cd ai-sdk/models + source env.sh v3 # NPU_VERSION + cp ../scripts/* . + ``` + - + + + + + + ```bash + cd ai-sdk/models + source env.sh v2 # NPU_VERSION + cp ../scripts/* . + ``` + + + + + :::tip -Specify NPU_VERSION, A733 specify `v3`, T527 specify `v1.13`. For NPU version selection, please refer to [NPU version comparison table](#npu-version-comparison-table) +Specify NPU_VERSION: Use `v3` for A733 and `v2` for T527. Refer to the [NPU Version Comparison Table](#npu-version-comparison-table) for details. ::: -## Model parsing +## Model Parsing -Enter the MobileNetV2_Imagenet model example directory in the Docker container of the example repository. +Navigate to the ai-sdk/models/MobileNetV2_Imagenet example directory in the Docker container. @@ -77,11 +95,11 @@ cd ai-sdk/models/MobileNetV2_Imagenet This directory contains the following files: -- MobilNetV2_Imagenet.h5 original model file (required). -- channel_mean_value.txt model input mean and scale value file. -- dataset.txt model quantization calibration dataset file. -- space_shuttle_224x224.jpg test input image and calibration image in calibration dataset. -- input_outputs.txt model input output node file. +- `MobilNetV2_Imagenet.h5`: The original model file (required). +- `channel_mean_value.txt`: File containing mean and scale values for model input. +- `dataset.txt`: Calibration dataset file for model quantization. +- `space_shuttle_224x224.jpg`: Test input image and calibration image included in the dataset. +- `inputs_outputs.txt`: File containing model input and output nodes (set output nodes if necessary to avoid quantization failure). ```bash . @@ -94,14 +112,14 @@ This directory contains the following files: 0 directories, 5 files ``` -### Import Model +### Importing the Model -`pegasus_import.sh` model import script, can parse the model structure and weights of various AI frameworks, and output the parsed model files: +The `pegasus_import.sh` script can parse model structures and weights from various AI frameworks and output the parsed files: - Model architecture will be saved in MODEL_DIR.json - Model weights will be saved in MODEL_DIR.data -- Automatically generate model input file template MODEL_DIR_inputmeta.yml -- Automatically generate model postprocessing file template MODEL_DIR_postprocess_file.yml +- Automatically generates model input file template MODEL_DIR_inputmeta.yml +- Automatically generates model post-processing file template MODEL_DIR_postprocess_file.yml @@ -114,12 +132,12 @@ This directory contains the following files: Parameters: -- MODEL_DIR directory containing source model files +- MODEL_DIR: Folder containing the source model files ### Manually Modify Model Input File -Here, we need to manually set the mean and scale in `MobileNetV2_Imagenet_inputmeta.yml` based on the model input preprocessing mean and scale. -Here, we use MobileNetV2_ImageNet as an example, because MobileNetV2 outputs (1,224,224,3) RGB three channels, according to the model preprocessing formula: +You need to manually set the mean and scale values in `MobileNetV2_Imagenet_inputmeta.yml` according to the model's input preprocessing requirements. +Using MobileNetV2_ImageNet as an example, since the MobileNet output is (1,224,224,3) RGB three-channel, the model preprocessing formula is: ```bash x1 = (x - mean) / std @@ -127,8 +145,8 @@ x1 = (x - mean) * scale scale = 1 / std ``` -Because the training dataset is ImageNet, here the ImageNet training set's normalized mean is [0.485, 0.456, 0.406], std is [0.229, 0.224, 0.225], -here needs to perform reverse normalization calculation. Normalization data reference [pytorch documentation](https://docs.pytorch.org/vision/main/models/generated/torchvision.models.mobilenet_v2.html) +Since the training dataset is ImageNet, the normalization mean for the ImageNet training set is [0.485, 0.456, 0.406] and std is [0.229, 0.224, 0.225]. +We need to perform denormalization calculations. Normalization data reference: [PyTorch Documentation](https://docs.pytorch.org/vision/main/models/generated/torchvision.models.mobilenet_v2.html) ```bash # mean @@ -144,7 +162,7 @@ here needs to perform reverse normalization calculation. Normalization data refe 1 / (0.225 * 255) = 0.01743 ``` -Here, we manually set the mean and scale in `MobileNetV2_Imagenet_inputmeta.yml` based on the calculated mean and scale: +Based on the calculations above, modify the mean and scale values in `MobileNetV2_Imagenet_inputmeta.yml` as follows: ```vim mean: @@ -157,14 +175,14 @@ scale: - 0.01743 ``` -## Quantize Model +## Model Quantization -Before conversion, models can be quantized in different types, ACUITY supports uint8 / int16 / bf16 / pcq (int8 per-channel quantized) multiple quantization types, -if using float, it means no quantization. +Before model conversion, different types of quantization can be applied to the model. ACUITY supports various quantization methods including uint8/int16/bf16/pcq (int8 per-channel quantized). +Using float means no quantization will be applied. -Use `pegasus_quantize.sh` script to quantize the model to the specified type. +The `pegasus_quantize.sh` script can be used to quantize the model with specified type. :::tip -If the source model is already a quantized model, no quantization is needed here, otherwise it will report an error. +If the source model is already quantized, no additional quantization is needed here, otherwise it will cause an error. ::: @@ -176,7 +194,7 @@ pegasus_quantize.sh MobileNetV2_Imagenet int16 10 -Quantization will generate the quantized file MODEL_DIR_QUANTIZED.quantize corresponding to the quantization method. +The quantization process will generate a quantized file MODEL_DIR_QUANTIZED.quantize corresponding to the quantization method used. | QUANTIZED | TYPE | QUANTIZER | | --------- | ----- | --------------------------- | @@ -185,33 +203,33 @@ Quantization will generate the quantized file MODEL_DIR_QUANTIZED.quantize corre | pcq | int8 | perchannel_symmetric_affine | | bf16 | bf16 | qbfloat16 | -{/* ### Perform pcq int8 quantization */} +{/* ### Perform pcq int8 Quantization */} {/* ```bash */} {/* # pegasus_quantize.sh MODEL_DIR QUANTIZED ITERATION */} {/* pegasus_quantize.sh MobileNetV2_Imagenet pcq 10 */} {/* ``` */} -{/* ### Perform int16 quantization */} +{/* ### Perform int16 Quantization */} {/* ```bash */} {/* # pegasus_quantize.sh MODEL_DIR QUANTIZED ITERATION */} {/* pegasus_quantize.sh MobileNetV2_Imagenet int16 10 */} {/* ``` */} -{/* ### Perform bf16 quantization */} +{/* ### Perform bf16 Quantization */} {/* ```bash */} {/* # pegasus_quantize.sh MODEL_DIR QUANTIZED ITERATION */} {/* pegasus_quantize.sh MobileNetV2_Imagenet bf16 10 */} {/* ``` */} -## Inference quantized model +## Inference on Quantized Models -After quantization, the performance will be improved to different degrees, but the precision will be slightly reduced. The quantized model can be inferred through `pegasus_inference.sh` to verify whether the quantized model meets the precision requirements. -Test inference input is the first picture in dataset.txt. +After quantization, the model's performance will improve to varying degrees, but the accuracy may slightly decrease. The quantized model can be inferred using `pegasus_inference.sh` to verify if the accuracy meets requirements after quantization. +The input for test inference is the first image in dataset.txt. ![space_shuttle](/img/e/e54c/rs400_npu_3.webp) -### Inference float model +### Inference on Float Model -Infer the unquantized float model to get the result as the reference for the quantized model. +Inference on the non-quantized float model to get reference results for comparison with quantized models @@ -222,7 +240,7 @@ pegasus_inference.sh MobileNetV2_Imagenet/ float -Inference result output is +The inference output is: ```bash I 07:01:06 Iter(0), top(5), tensor(@attach_Logits/Softmax/out0_0:out0) : @@ -233,13 +251,13 @@ I 07:01:06 864: 6.59249781165272e-05 I 07:01:06 536: 2.808812860166654e-05 ``` -The top5 confidence is 812, corresponding to the label `space shuttle`, which matches the actual input image type, indicating that the model input preprocessing mean and scale are set correctly. +The top5 confidence score is highest for class 812, which corresponds to the label `space shuttle`. This matches the actual input image type, indicating that the mean and scale settings for the model input preprocessing are correct. -The tensor of inference is saved in the MODEL_DIR/inf/MODEL_DIR_QUANTIZED folder at the same time. +The inference tensors are saved in the MODEL_DIR/inf/MODEL_DIR_QUANTIZED directory: -- iter_0_input_1_158_out0_1_224_224_3.qnt.tensor is the original image tensor -- iter_0_input_1_158_out0_1_224_224_3.tensor is the model input tensor after preprocessing -- iter_0_attach_Logits_Softmax_out0_0_out0_1_1000.tensor is the model output tensor +- iter_0_input_1_158_out0_1_224_224_3.qnt.tensor: The original image tensor +- iter_0_input_1_158_out0_1_224_224_3.tensor: The preprocessed model input tensor +- iter_0_attach_Logits_Softmax_out0_0_out0_1_1000.tensor: The model's output tensor ```bash . @@ -250,7 +268,7 @@ The tensor of inference is saved in the MODEL_DIR/inf/MODEL_DIR_QUANTIZED folder 0 directories, 3 files ``` -### Inference uint8 quantized model +### Inference on uint8 Quantized Model @@ -261,7 +279,7 @@ pegasus_inference.sh MobileNetV2_Imagenet/ uint8 -Inference result output is +The inference output is: ```bash I 07:02:20 Iter(0), top(5), tensor(@attach_Logits/Softmax/out0_0:out0) : @@ -273,10 +291,10 @@ I 07:02:20 466: 0.005068646278232336 ``` :::warning -The top5 confidence is 904, this needs to be noted that 904 corresponds to the label `wig`, which does not match the actual input image type, and does not match the inference result of the float type, which means that the uint8 quantization has caused precision loss. At this time, higher precision quantization models can be applied, such as **pcq** or **int16**. For methods to improve model precision, please refer to [**Hybrid Quantization**](cubie_quant_acc_improve#hybrid-quantization) +The top5 confidence score is highest for class 904, which corresponds to the label `wig`. This does not match the input image result and is inconsistent with the float model's inference result. This indicates precision loss after uint8 quantization. In this case, you can apply higher precision quantization models such as **pcq** or **int16**. For methods to improve model accuracy, please refer to [**Mixed Quantization**](cubie_quant_acc_improve#mixed-quantization). ::: -### Inference pcq quantized model +### Inference on PCQ Quantized Model @@ -287,7 +305,7 @@ pegasus_inference.sh MobileNetV2_Imagenet/ pcq -The inference result output is +The inference results are: ```bash I 03:36:41 Iter(0), top(5), tensor(@attach_Logits/Softmax/out0_0:out0) : @@ -298,9 +316,9 @@ I 03:36:41 833: 9.26952125155367e-05 I 03:36:41 576: 6.784773722756654e-05 ``` -The top5 confidence is 812, corresponding to the label `space shuttle`, which matches the actual input image type, and matches the inference result of the float type, indicating that the pcq quantization precision is correct. +The highest confidence score in the top5 results is 812, corresponding to the label `space shuttle`, which matches the actual input image type and is consistent with the float type inference results. This indicates that the accuracy is correct with pcq quantization. -### Inference int16 quantized model +### Inference on int16 Quantized Model @@ -311,7 +329,7 @@ pegasus_inference.sh MobileNetV2_Imagenet/ int16 -The inference result output is +The inference results are: ```bash I 06:54:23 Iter(0), top(5), tensor(@attach_Logits/Softmax/out0_0:out0) : @@ -322,13 +340,13 @@ I 06:54:23 864: 6.788487371522933e-05 I 06:54:23 536: 3.0241633794503286e-05 ``` -The top5 confidence is 812, corresponding to the label `space shuttle`, which matches the actual input image type, and matches the inference result of the float type, indicating that the int16 quantization precision is correct. +The highest confidence score in the top5 results is 812, corresponding to the label `space shuttle`, which matches the actual input image type and is consistent with the float type inference results. This indicates that the accuracy is correct with int16 quantization. -## Model compilation and export +## Model Compilation and Export `pegasus_export_ovx.sh` can export the model files and project code required for NPU inference. -Here we take the INT16 quantized model as an example +Here we use the INT16 quantized model as an example: @@ -339,10 +357,10 @@ pegasus_export_ovx.sh MobileNetV2_Imagenet int16 -Generated OpenVX project and NBG project paths: +Generates OpenVX and NBG project paths: -- MODEL_DIR/wksp/MODEL_DIR_QUANTIZED : Cross-platform OpenVX project, requires hardware just-in-time (JIT) compilation for model initialization. -- MODEL_DIR/wksp/MODEL_DIR_QUANTIZED_nbg_unify: NBG format, pre-compiled machine code format, low overhead, fast initialization. +- `MODEL_DIR/wksp/MODEL_DIR_QUANTIZED`: Cross-platform OpenVX project, requires hardware just-in-time (JIT) compilation for model initialization. +- `MODEL_DIR/wksp/MODEL_DIR_QUANTIZED_nbg_unify`: NBG format, pre-compiled machine code format, low overhead, fast initialization. ```bash (.venv) root@focal-v4l2:~/work/Acuity/acuity_examples/models/MobileNetV2_Imagenet/wksp$ ls @@ -350,14 +368,14 @@ MobileNetV2_Imagenet_int16 MobileNetV2_Imagenet_int16_nbg_unify ``` :::tip -In the NBG project, you can get the `network_binary.nb` model file. The compiled model can be copied to the board for inference using vpm_run or awnn API. +In the NBG project, you can obtain the `network_binary.nb` model file. The compiled model can be copied to the board for on-device inference using vpm_run or awnn API. ::: -## Use Vivante IDE to simulate running inference +## Simulating Inference with Vivante IDE -Use Vivante IDE to verify the generated target model and OpenVX project on X86 PC in ACUITY Docker. +Vivante IDE can be used to verify the generated target model and OpenVX project in the ACUITY Docker on an X86 PC. -### Import Vivante IDE required environment variables +### Import Required Environment Variables for Vivante IDE @@ -370,9 +388,9 @@ unset VSI_USE_IMAGE_PROCESS -### Simulate running cross-platform OpenVX project +### Simulating Cross-platform OpenVX Project -#### Compile executable file +#### Compile the Executable @@ -383,9 +401,9 @@ make -f makefile.linux -The generated target file is the binary executable file of MODEL_DIR_QUANTIZED. +The generated target file is a binary executable of MODEL_DIR_QUANTIZED. -#### Run executable file +#### Run the Executable @@ -396,7 +414,7 @@ The generated target file is the binary executable file of MODEL_DIR_QUANTIZED. -The inference result output is: +Execution Result: ```bash Create Neural Network: 11ms or 11426us @@ -415,13 +433,13 @@ Average 229309.53ms or 229309536.00us 0: 0.000000 ``` -### Simulate running NBG project +### Simulate Running NBG Project :::tip -Using Vivante IDE to run NBG projects will increase the time. +Running NBG projects in Vivante IDE will increase the execution time. ::: -#### Compile executable file +#### Compile the Executable @@ -432,9 +450,9 @@ make -f makefile.linux -The generated target file is the binary executable file of MODEL_DIR_QUANTIZED. +The generated target file is a binary executable of MODEL_DIR_QUANTIZED. -#### Run executable file +#### Run the Executable @@ -445,7 +463,7 @@ The generated target file is the binary executable file of MODEL_DIR_QUANTIZED. -The inference result output is: +Execution Result: ```bash Create Neural Network: 4ms or 4368us @@ -464,10 +482,10 @@ Average 229388.52ms or 229388512.00us 0: 0.000000 ``` -{/* ## Using the NPU inference model on the board */} -{/* After verifying the availability of the OpenVX or NBG project through simulated inference, you can use cross-compilation to compile the target project into an executable file for execution on the board. */} +{/* ## On-board NPU Inference Model */} +{/* After confirming the OpenVX project or NBG project is available through simulation inference, you can use cross-compilation to compile the target project into an executable file for on-board execution. */} -{/* ### Importing environment variables required for cross-compilation */} +{/* ### Import Environment Variables for Cross-Compilation */} {/* ```bash */} {/* unset USE_IDE_LIB */} {/* unset VIVANTE_SDK_DIR */} @@ -475,42 +493,42 @@ Average 229388.52ms or 229388512.00us {/* export AQROOT=~/work/library/acuity-root-dir */} {/* ``` */} -{/* ### Compile the project */} -{/* #### Compile the cross-platform OpenVX project */} +{/* ### Compile the Project */} +{/* #### Compile Cross-Platform OpenVX Project */} {/* ```bash */} {/* cd MobileNetV2_Imagenet/wksp/MobileNetV2_Imagenet_int16 */} {/* make -f makefile.linux */} {/* ``` */} -{/* Generate the target folder `bin_r`, copy the generated `bin_r` folder to the board, and copy the MobileNetV2_Imagenet_int16.export.data and test images to the board. */} +{/* The target folder `bin_r` will be generated. Copy the generated `bin_r` folder to the target board, and also copy MobileNetV2_Imagenet_int16.export.data and test images to the board. */} -{/* #### Compile the NBG project */} +{/* #### Compile NBG Project */} {/* ```bash */} {/* cd MobileNetV2_Imagenet/wksp/MobileNetV2_Imagenet_int16_nbg_unify */} {/* make -f makefile.linux */} {/* ``` */} -{/* Generate the target folder `bin_r`, copy the generated `bin_r` folder to the board, and copy the network_binary.nb NBG model and test images to the board. */} +{/* The target folder `bin_r` will be generated. Copy the generated `bin_r` folder to the target board, and also copy the network_binary.nb NBG model and test images to the board. */} -## Board-side NPU inference +## On-board NPU Inference -Board-side inference of NBG format models can be performed using the `vpm_run` tool for inference testing. +For on-board NPU inference of NBG format models, you can use the `vpm_run` tool for inference testing. -`vpm_run` installation and usage please refer to [**vpm_run model testing tool**](cubie_vpm_run) +For `vpm_run` installation and usage, please refer to [**vpm_run Model Testing Tool**](cubie_vpm_run) -{/* ### Board-side execution */} +{/* ### On-board Execution */} -{/* #### Copy ACUITY dynamic library */} -{/* Copy the following ACUITY ARM64 dynamic library folder path to the board */} +{/* #### Copy ACUITY Dynamic Libraries */} +{/* Copy the following ACUITY ARM64 dynamic library folder to the target board */} {/* ```bash */} {/* library/acuity-root-dir/lib/arm64/1619b */} {/* ``` */} -{/* #### Import dynamic library path variable */} -{/* Please modify `ACUITY_LIB_PATH` value to the copied `1619b` path */} +{/* #### Import Dynamic Library Path Variable */} +{/* Please modify the `ACUITY_LIB_PATH` to the copied `1619b` path */} {/* ```bash */} {/* export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ACUITY_LIB_PATH */} {/* ``` */} -{/* #### Run cross-platform OpenVX project */} +{/* #### Run Cross-platform OpenVX Project */} {/* ```bash */} {/* root@Stark:/home/nas/MobileNetV2/openvx_bin_r# ./mobilenetv2imagenetint16 ./MobileNetV2_Imagenet_int16.export.data ../space_shuttle_224x224.jpg */} {/* Create Neural Network: 112ms or 112315us */} @@ -528,7 +546,7 @@ Board-side inference of NBG format models can be performed using the `vpm_run` t {/* 864: 0.000089 */} {/* 0: 0.000000 */} {/* ``` */} -{/* #### Run NBG project */} +{/* #### Run NBG Project */} {/* ```bash */} {/* root@Stark:/home/nas/MobileNetV2/NBG_bin_r# ./mobilenetv2imagenetint16 ./network_binary.nb ../space_shuttle_224x224.jpg */} {/* Create Neural Network: 19ms or 19022us */} diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_lenet.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_lenet.mdx index a30d828ba..3714d1c83 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_lenet.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_lenet.mdx @@ -1,13 +1,13 @@ :::tip -This document demonstrates how to run the LeNet handwritten digit classification model on Allwinner T527/A733 series chips. +This document demonstrates how to run on-board inference of the LeNet handwritten digit classification model on Allwinner T527/A733 series chips. ::: Deploying LeNet on the board requires two steps: -- Use the [ACUITY Toolkit](./cubie_acuity_sdk#acuity-toolkit) on the PC to convert models from different frameworks into NBG format. -- Use the awnn API on the board to perform inference with the model. +- On the PC side, use [ACUITY Toolkit](./cubie_acuity_sdk#acuity-toolkit) to convert models from different frameworks to NBG format +- On the board side, use the awnn API for model inference -## Download the ai-sdk Example Repository +## Download ai-sdk Example Repository @@ -17,50 +17,67 @@ git clone https://github.com/ZIFENG278/ai-sdk.git -## Model Conversion on PC +## PC-Side Model Conversion :::tip -Radxa provides a pre-converted `lenet.nb` model. Users can directly refer to [**LeNet Inference on the Board**](#lenet-inference-on-the-board) and skip the PC model conversion section. +Radxa provides a pre-converted `lenet.nb` model. Users can directly refer to [**On-board LeNet Inference** ](#on-board-lenet-inference) to skip the PC-side model conversion chapter. ::: :::tip -The files used in the LeNet example are already included in the [ai-sdk example repository](https://github.com/ZIFENG278/ai-sdk.git) under `models/lenet`. +The files used in the LeNet example are already included in the `models/lenet` directory of the [ai-sdk example repository](https://github.com/ZIFENG278/ai-sdk.git) ::: -- Enter the ACUITY Toolkit Docker container. +- Enter the ACUITY Toolkit Docker Container - For ACUITY Toolkit Docker environment setup, refer to [ACUITY Toolkit Environment Setup](./cubie_acuity_env). + For ACUITY Toolkit Docker environment setup, please refer to [ACUITY Toolkit Environment Configuration](./cubie_acuity_env) - Configure environment variables: + Configure Environment Variables - + - ```bash - cd ai-sdk/models - source env.sh v3 # NPU_VERSION - ``` + - + + ```bash + cd ai-sdk/models + source env.sh v3 # NPU_VERSION + cp ../scripts/* . + ``` + + + - For A733, choose `v3`; for T527, choose `v2`. + + + + ```bash + cd ai-sdk/models + source env.sh v2 # NPU_VERSION + cp ../scripts/* . + ``` + + + + + :::tip - Refer to the [NPU Version Comparison Table](./cubie_acuity_usage#npu-version-comparison-table) for NPU version selection. + Specify NPU_VERSION: use `v3` for A733 and `v2` for T527. For reference, see [**NPU Version Comparison Table**](cubie_acuity_usage#npu-version-comparison) ::: -- Navigate to the LeNet model directory: +- Enter the LeNet Model Directory - + ```bash cd ai-sdk/models/lenet ``` - + -- Create a quantization calibration dataset. +- Create Quantization Calibration Dataset - Use a set of images for quantization calibration. Save the image paths in `dataset.txt`. + Use an appropriate number of images to create a quantization calibration dataset, with image paths saved in `dataset.txt` @@ -83,7 +100,7 @@ The files used in the LeNet example are already included in the [ai-sdk example ./input_image/9.jpg 9 ``` -- Directory structure: +- Directory contains files ```bash . @@ -104,25 +121,25 @@ The files used in the LeNet example are already included in the [ai-sdk example `-- lenet.prototxt ``` -- Parse the model. +- Parse the Model :::tip - The `pegasus` script is located in `ai-sdk/scripts` and can be copied to the `models` directory. + pegasus script is in ai-sdk/scripts, copy it to models directory ::: - Use `pegasus_import.sh` to parse the model into an intermediate representation (IR). This generates `lenet.json` (model structure) and `lenet.data` (model weights). + Use `pegasus_import.sh` to parse the model into IR expression, will get `lenet.json` containing model structure and `lenet.data` containing model weights - + ```bash ./pegasus_import.sh lenet/ ``` - + -- Modify the `lenet_inputmeta.yml` file. +- Modify lenet_inputmeta.yml file - Update the `scale` value based on the formula `scale = 1 / std`. + Here modify scale value, according to the formula scale = 1 / std ```bash scale = 1 / 255 @@ -179,21 +196,21 @@ The files used in the LeNet example are already included in the [ai-sdk example redirect_to_output: false ``` -- Quantize the model. +- Quantize the Model - Use `pegasus_quantize.sh` to quantize the model into `uint8` format. + Use `pegasus_quantize.sh` to quantize the model to uint8 type - + ```bash ./pegasus_quantize.sh lenet/ uint8 10 ``` - + -- Compile the model. +- Compile the Model - Use `pegasus_export_ovx.sh` to compile the model into NBG format. + Use `./pegasus_export_ovx.sh` to compile the model into NBG model format @@ -203,11 +220,11 @@ The files used in the LeNet example are already included in the [ai-sdk example - The NBG model is saved in `lenet/wksp/lenet_uint8_nbg_unify/network_binary.nb`. + NBG model is saved in `lenet/wksp/lenet_uint8_nbg_unify/network_binary.nb` -## LeNet Inference on the Board +## On-board LeNet Inference -Navigate to the LeNet example code directory. +Enter the LeNet example code file directory path @@ -217,44 +234,80 @@ cd ai-sdk/examples/lenet -### Compile the Example +### Compile Example - + + -```bash -make AI_SDK_PLATFORM=a733 -make install AI_SDK_PLATFORM=a733 INSTALL_PREFIX=./ -``` + - + ```bash + make AI_SDK_PLATFORM=a733 + make install AI_SDK_PLATFORM=a733 INSTALL_PREFIX=./ + ``` -Parameter explanation: + -- `AI_SDK_PLATFORM`: Specify the SoC, options are **`a733`** or **`t527`**. -- `INSTALL_PREFIX`: Specify the installation path. + -### Run the Example + -Set environment variables. + + ```bash + make AI_SDK_PLATFORM=t527 + make install AI_SDK_PLATFORM=t527 INSTALL_PREFIX=./ + ``` + - + -```bash -export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/rock/ai-sdk/viplite-tina/lib/aarch64-none-linux-gnu/NPU_VERSION # NPU_SW_VERSION -``` + - +Parameter Analysis: + +`AI_SDK_PLATFORM`: Specify SoC, optional values: **`a733`**, **`t527`** + +`INSTALL_PREFIX`: Specify installation path + +### Run Example + +Import environment variables + + + + + + + ```bash + export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/rock/ai-sdk/viplite-tina/lib/aarch64-none-linux-gnu/v2.0 # NPU_SW_VERSION + ``` + + + + + + + + + ```bash + export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/rock/ai-sdk/viplite-tina/lib/aarch64-none-linux-gnu/v1.13 # NPU_SW_VERSION + ``` + + + + + :::tip -Specify `NPU_SW_VERSION`. For A733, choose `v2.0`; for T527, choose `v1.13`. Refer to the [NPU Version Comparison Table](./cubie_acuity_usage#npu-version-comparison-table) for details. +Specify NPU_SW_VERSION, A733 select `v2.0`, T527 select `v1.13`, NPU information对照请参考 [NPU Version Comparison Table](./cubie_acuity_usage#npu-version-comparison) ::: -Navigate to the example installation directory. +Enter the example installation directory ```bash -cd INSTALL_PREFIX/etc/npu/lenet +cd $INSTALL_PREFIX/etc/npu/lenet # ./lenet nbg_model input_picture ./lenet ./model/lenet.nb ./input_data/lenet.dat ``` @@ -262,11 +315,11 @@ cd INSTALL_PREFIX/etc/npu/lenet :::tip -The example automatically installs the `lenet.nb` model provided by Radxa. You can manually specify the path to your converted NBG model. +The example will automatically install the lenet.nb model provided by Radxa. Here you can manually specify the path to the user-converted NBG model. :::
- + lenet input image
diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_resnet50.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_resnet50.mdx index 79b2b556f..a3fc8ea74 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_resnet50.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_resnet50.mdx @@ -1,13 +1,13 @@ :::tip -This document demonstrates how to run the ResNet50 object classification model on Allwinner T527/A733 series chips. +This document demonstrates how to run on-board inference of the ResNet50 object classification model on Allwinner T527/A733 series chips. ::: -This example uses the pre-trained ONNX format model from [resnet50-v2-7.onnx](https://github.com/onnx/models/blob/main/validated/vision/classification/resnet/model/resnet50-v2-7.onnx) as an example to demonstrate the complete process of converting the model for inference on the board. +This example uses a pre-trained ONNX format model from [resnet50-v2-7.onnx](https://github.com/onnx/models/blob/main/validated/vision/classification/resnet/model/resnet50-v2-7.onnx) as an example to demonstrate the complete process from model conversion to on-board inference. -The deployment of ResNet50 on the board requires two steps +Deploying ResNet50 on the board requires two steps: -- On the PC, use the [ACUITY Toolkit](./cubie_acuity_sdk#acuity-toolkit) to convert models from different frameworks into NBG format. -- On the board, use the awnn API to infer the model. +- On the PC side, use [ACUITY Toolkit](./cubie_acuity_sdk#acuity-toolkit) to convert models from different frameworks to NBG format +- On the board side, use the awnn API for model inference ## Download ai-sdk Example Repository @@ -19,33 +19,51 @@ git clone https://github.com/ZIFENG278/ai-sdk.git -## PC Model Conversion +## PC-Side Model Conversion :::tip -Radxa provides a pre-converted `resnet50.nb` model. Users can directly refer to [**Board-side inference ResNet50** ](#board-side-inference-resnet50) and skip the PC model conversion section. +Radxa provides a pre-converted `resnet50.nb` model. Users can directly refer to [**On-board ResNet50 Inference** ](#on-board-resnet50-inference) to skip the PC-side model conversion chapter. ::: -- Enter the ACUITY Toolkit Docker container +- Enter the ACUITY Toolkit Docker Container - ACUTIY Toolkit Docker environment preparation please refer to [ACUITY Toolkit environment configuration](./cubie_acuity_env) + For ACUITY Toolkit Docker environment setup, please refer to [ACUITY Toolkit Environment Configuration](./cubie_acuity_env) - Configure environment variables + Configure Environment Variables - + - ```bash - cd ai-sdk/models - source env.sh v3 #NPU_VERSION - ``` + + + + ```bash + cd ai-sdk/models + source env.sh v3 # NPU_VERSION + cp ../scripts/* . + ``` + + + + + + + + ```bash + cd ai-sdk/models + source env.sh v2 # NPU_VERSION + cp ../scripts/* . + ``` + + + - + - A733 select `v3`, T527 select `v2` :::tip - NPU version selection please refer to [NPU version comparison table](./cubie_acuity_usage#npu-version-comparison-table) + Specify NPU_VERSION: use `v3` for A733 and `v2` for T527. For reference, see [**NPU Version Comparison Table**](cubie_acuity_usage#npu-version-comparison) ::: -- Download resnet50 onnx model +- Download the ResNet50 ONNX Model @@ -56,9 +74,9 @@ Radxa provides a pre-converted `resnet50.nb` model. Users can directly refer to -- Fixed input +- Fix Input Dimensions - Use NPU inference only accepts fixed input size, here use onnxsim to fix input size + NPU inference only accepts fixed input dimensions. Here we use onnxsim to fix the input dimensions @@ -69,9 +87,9 @@ Radxa provides a pre-converted `resnet50.nb` model. Users can directly refer to -- Make quantization calibration set +- Create Quantization Calibration Dataset - Use an appropriate amount of images as a quantitative calibration set, and save the quantized images in the form of image paths in `dataset.txt` + Use an appropriate number of images to create a quantization calibration dataset. The image paths should be saved in `dataset.txt` @@ -89,7 +107,7 @@ Radxa provides a pre-converted `resnet50.nb` model. Users can directly refer to - Make model input output file - You can use [netron](https://netron.app/) to confirm the input and output names of the onnx model + You can use [netron](https://netron.app/) to confirm the names of the model's input and output @@ -106,8 +124,8 @@ Radxa provides a pre-converted `resnet50.nb` model. Users can directly refer to {" "}
- - resnet50 in/output name + + ResNet50 input/output names
- Directory contains files @@ -121,13 +139,13 @@ Radxa provides a pre-converted `resnet50.nb` model. Users can directly refer to `-- space_shuttle_224x224.jpg ``` -- Parse model +- Parse the Model :::tip - pegasus script is in ai-sdk/scripts, you can copy it to models directory + pegasus script is in ai-sdk/scripts, copy it to models directory ::: - Use `pegasus_import.sh` to parse the model into an intermediate representation (IR). This generates `resnet50-sim.json` (model structure) and `resnet50-sim.data` (model weights). + Use `pegasus_import.sh` to parse the model into IR expression, will get `resnet50-sim.json` containing model structure and `resnet50-sim.data` containing model weights @@ -139,8 +157,7 @@ Radxa provides a pre-converted `resnet50.nb` model. Users can directly refer to - Modify resnet50-sim_inputmeta.yml file - Because the training dataset is ImageNet, here the ImageNet training set's normalized mean is [0.485, 0.456, 0.406], std is [0.229, 0.224, 0.225], - here needs to perform reverse normalization calculation. Normalization data reference [pytorch documentation](https://docs.pytorch.org/vision/main/models/generated/torchvision.models.mobilenet_v2.html) + Since the training dataset is ImageNet, the normalization mean for the ImageNet training set is [0.485, 0.456, 0.406] and std is [0.229, 0.224, 0.225]. Denormalization calculation is required here. For normalization data, refer to [PyTorch documentation](https://docs.pytorch.org/vision/main/models/generated/torchvision.models.mobilenet_v2.html) ```bash # mean @@ -156,7 +173,7 @@ Radxa provides a pre-converted `resnet50.nb` model. Users can directly refer to 1 / (0.225 * 255) = 0.01743 ``` - Here, based on the calculated mean and scale, modify the mean and scale values in `resnet50-sim_inputmeta.yml`: + Modify the mean and scale values in `resnet50-sim_inputmeta.yml` according to the calculated mean and scale: ```vim mean: @@ -249,7 +266,7 @@ Radxa provides a pre-converted `resnet50.nb` model. Users can directly refer to NBG model saved in `resnet50-sim/wksp/resnet50-sim_uint8_nbg_unify/network_binary.nb` -## Board-side inference ResNet50 +### Board-side ResNet50 Inference Enter the resnet50 example code file directory path @@ -261,37 +278,72 @@ cd ai-sdk/examples/resnet50 -### Compile example +### Compile Example - + + -```bash -make AI_SDK_PLATFORM=a733 -make install AI_SDK_PLATFORM=a733 INSTALL_PREFIX=./ -``` + - + ```bash + make AI_SDK_PLATFORM=a733 + make install AI_SDK_PLATFORM=a733 INSTALL_PREFIX=./ + ``` + + + + + + -Parameter parsing: + + ```bash + make AI_SDK_PLATFORM=t527 + make install AI_SDK_PLATFORM=t527 INSTALL_PREFIX=./ + ``` + + + + + + +Parameter description: `AI_SDK_PLATFORM`: Specify SoC, optional **`a733`**, **`t527`** `INSTALL_PREFIX`: Specify installation path -### Run example +### Run Example Import environment variables - + + -```bash -export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/rock/ai-sdk/viplite-tina/lib/aarch64-none-linux-gnu/NPU_VERSION # NPU_SW_VERSION -``` + - + ```bash + export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/rock/ai-sdk/viplite-tina/lib/aarch64-none-linux-gnu/v2.0 # NPU_SW_VERSION + ``` + + + + + + + + + ```bash + export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/rock/ai-sdk/viplite-tina/lib/aarch64-none-linux-gnu/v1.13 # NPU_SW_VERSION + ``` + + + + + :::tip -Specify NPU_SW_VERSION, A733 select `v2.0`, T527 select `v1.13`, NPU version对照请参考 [NPU 版本对照表](./cubie_acuity_usage#npu-版本对照表) +Specify NPU_SW_VERSION: select `v2.0` for A733 and `v1.13` for T527. For NPU information, refer to [NPU Version Comparison Table](./cubie_acuity_usage#npu-version-comparison) ::: Enter the example installation directory @@ -299,7 +351,7 @@ Enter the example installation directory ```bash -cd INSTALL_PREFIX/etc/npu/resnet50 +cd $INSTALL_PREFIX/etc/npu/resnet50 # ./resnet50 nbg_model input_picture ./resnet50 model/resnet50.nb ./input_data/dog_224_224.jpg ``` @@ -307,11 +359,11 @@ cd INSTALL_PREFIX/etc/npu/resnet50 :::tip -The example will automatically install the radxa provided resnet50.nb model, here you can manually specify the NBG model path converted by the user. +The example will automatically install the resnet50.nb model provided by Radxa. Here you can manually specify the path to the user-converted NBG model. :::
- + resnet50 demo input image
diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_vpm_run.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_vpm_run.mdx index 83090a68f..8fc89b830 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_vpm_run.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_vpm_run.mdx @@ -2,13 +2,13 @@ This document demonstrates how to use the model testing tool `vpm_run` on Allwinner T527/A733 series chips. ::: -`vpm_run` is an edge-side program based on the VIPLite driver, primarily used for quickly validating and testing model files. +vpm_run is an edge-side program based on the VIPLite driver, primarily used for quick validation and testing of model files. -- `vpm_run` can serve as a general-purpose model runtime environment capable of running any correctly deployed NBG model file. -- `vpm_run` is based on the VIPLite network layer API. -- `vpm_run` includes a default post-processing program for TOP5 results. +- vpm_run serves as a universal model runtime environment that can run any properly deployed NBG model file. +- vpm_run is built on the VIPLite network layer API. +- vpm_run includes a default TOP5 post-processing program. -## Download the ai-sdk Example Repository +## Download ai-sdk Example Repository @@ -20,7 +20,7 @@ git clone https://github.com/ZIFENG278/ai-sdk.git ## Using vpm_run on the Board -Navigate to the directory containing the `vpm_run` example code. +Navigate to the vpm_run example code file directory path @@ -32,20 +32,38 @@ cd ai-sdk/examples/vpm_run ### Compile the Example - + + -```bash -make AI_SDK_PLATFORM=a733 -make install AI_SDK_PLATFORM=a733 INSTALL_PREFIX=./ -``` + - + ```bash + make AI_SDK_PLATFORM=a733 + make install AI_SDK_PLATFORM=a733 INSTALL_PREFIX=./ + ``` + + + + + + + + + ```bash + make AI_SDK_PLATFORM=t527 + make install AI_SDK_PLATFORM=t527 INSTALL_PREFIX=./ + ``` + -Parameter explanation: + -`AI_SDK_PLATFORM`: Specify the SoC, options are **`a733`**, **`t527`**. + -`INSTALL_PREFIX`: Specify the installation path. +Parameter description: + +`AI_SDK_PLATFORM`: Specify SoC, optional **`a733`**, **`t527`** + +`INSTALL_PREFIX`: Specify installation path Usage @@ -72,34 +90,51 @@ example: ./vpm_run -s sample.txt -l 10 -d 1 specify the NBG runs 10 times on dev - Import environment variables - + + -```bash -export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/rock/ai-sdk/viplite-tina/lib/aarch64-none-linux-gnu/NPU_VERSION # NPU_SW_VERSION -``` + - + ```bash + export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/rock/ai-sdk/viplite-tina/lib/aarch64-none-linux-gnu/v2.0 # NPU_SW_VERSION + ``` + + + + + + + + + ```bash + export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/rock/ai-sdk/viplite-tina/lib/aarch64-none-linux-gnu/v1.13 # NPU_SW_VERSION + ``` + + + + + :::tip -Specify NPU_SW_VERSION, A733 select `v2.0`, T527 select `v1.13`, NPU information对照请参考 [NPU 版本对照表](./cubie_acuity_usage#npu-版本对照表) +Specify NPU_SW_VERSION: use `v2.0` for A733 and `v1.13` for T527. Please refer to the [NPU Version Comparison Table](./cubie_acuity_usage#npu-version-comparison-table) for NPU information. ::: -- Make configuration file +- Create Configuration File - This example configuration file runs two NBG models, the first is a multi-input model, saving output files, and comparing with golden, the second is a single-input model, not saving output files. + This example configuration file executes two NGB models. The first one is a multi-input model that saves output files and compares them with golden data, while the second one is a single-input model that does not save output files. ```vim [network] ./network_binary.nb [input] - #Multi-input model, configure multiple input files; single-input model, configure 1 input file. + # For multi-input models, configure multiple input files; for single-input models, configure one input file. ./input_0.dat ./input_1.dat [golden] - #Optional + # Optional ./int8_output_003.dat [output] - #Optional + # Optional ./output_0.dat [network] @@ -108,59 +143,54 @@ Specify NPU_SW_VERSION, A733 select `v2.0`, T527 select `v1.13`, NPU information ./input1.tensor ``` - sample.txt file example as above, main label introduction as follows: + The sample.txt file is shown above, with the main tags described as follows: - - **[network]**：NBG file path. - - **[input]**：Input data path. Input data format supports two types: Acuity Toolkits tool inference stage generated input.tensor file; IDE tool simulation stage generated input.dat file. - - **[golden]**：Optional golden data path. If the golden tag exists, it will be used as golden data and vpm_run run output tensor for binary comparison. If the comparison is consistent, it will print pass, and if the comparison is inconsistent, vpm_run will exit. - - **[output]**：Optional. Output data save path. + - **[network]**: Path to the NBG file. + - **[input]**: Path to the input data. Two formats are supported: input.tensor files generated during the Acuity Toolkits inference phase, or input.dat files generated during IDE tool simulation. + - **[golden]**: Optional path to golden data. If the golden tag exists, it will be used for binary comparison with the tensor output from vpm_run. If they match, it will print 'pass'; if not, vpm_run will exit. + - **[output]**: Optional. Path to save the output data. -- Enter the example installation directory - - - -```bash -cd INSTALL_PREFIX/etc/npu/vpm_run -# ./vpm_run nbg_model input_picture -./vpm_run ./model/network_binary.nb ./input_data/input_image.jpg -``` +- Navigate to the example installation directory - + -:::tip -The example automatically installs a sample NBG model provided by Radxa. You can manually specify the path to your own NBG model. -::: + ```bash + cd $INSTALL_PREFIX/etc/npu/vpm_run + ./vpm_run -s sample.txt -l 1 + ``` -```bash -(.venv) rock@radxa-cubie-a7a:~/ai-sdk/examples/vpm_run/etc/npu/vpm_run$ ./vpm_run -s sample.txt -l 1 -loop_count=1, device_index=0, core_index=-1, file_name=sample.txt, time_out=0x0, bypass=1 -enable_npd=0, preload=0 -show_top50, save_txt=0 -init vip lite, driver version=0x00020003... -VIPLite driver software version 2.0.3.2-AW-2024-08-30 -vip lite init OK. - -cid=0x1000003b, device_count=1 - device[0] core_count=1 -config file read network count=1 -init test resources, task_count: 1 ... -create/prepare networks ... -task i=0, binary name: ./network_binary.nb -nbg name=./network_binary.nb -create network 0: 12354 us. -input 0 dim 3 224 224 1, data_format=5, quant_format=1, name=input/output[0], dfp=13 -output 0 dim 1000 1 0 0, data_format=1, name=uid_2_sub_uid_1_out_0, none-quant -memory pool size=1073536byte -network core count=1 -prepare network 0: 1409 us. -golden file count=0 -input 0 name: ./input_0.dat -read input and golden 0: 541 us. -task: 0, loop count: 1 -start to run network=./network_binary.nb -run time for this network 0: 3160 us. -run network done... -profile inference time=2945us, cycle=2916338 -destroy test resource task_count=1 -vpm run ret=0 -``` + + + ```bash + (.venv) rock@radxa-cubie-a7a:~/ai-sdk/examples/vpm_run/etc/npu/vpm_run$ ./vpm_run -s sample.txt -l 1 + loop_count=1, device_index=0, core_index=-1, file_name=sample.txt, time_out=0x0, bypass=1 + enable_npd=0, preload=0 + show_top50, save_txt=0 + init vip lite, driver version=0x00020003... + VIPLite driver software version 2.0.3.2-AW-2024-08-30 + vip lite init OK. + + cid=0x1000003b, device_count=1 + device[0] core_count=1 + config file read network count=1 + init test resources, task_count: 1 ... + create/prepare networks ... + task i=0, binary name: ./network_binary.nb + nbg name=./network_binary.nb + create network 0: 12354 us. + input 0 dim 3 224 224 1, data_format=5, quant_format=1, name=input/output[0], dfp=13 + output 0 dim 1000 1 0 0, data_format=1, name=uid_2_sub_uid_1_out_0, none-quant + memory pool size=1073536byte + network core count=1 + prepare network 0: 1409 us. + golden file count=0 + input 0 name: ./input_0.dat + read input and golden 0: 541 us. + task: 0, loop count: 1 + start to run network=./network_binary.nb + run time for this network 0: 3160 us. + run network done... + profile inference time=2945us, cycle=2916338 + destroy test resource task_count=1 + vpm run ret=0 + ``` diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_yolact.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_yolact.mdx index 7e120e685..f09428995 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_yolact.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_yolact.mdx @@ -1,13 +1,13 @@ :::tip -This document demonstrates how to run the YOLACT object segmentation model on Allwinner T527/A733 series chips. +This document demonstrates how to run YOLACT instance segmentation model inference on Allwinner T527/A733 series chips. ::: -Deploying YOLACT on the board requires two steps: +Deploying YOLACT on the development board involves two main steps: -- Use the [ACUITY Toolkit](./cubie_acuity_sdk#acuity-toolkit) on the PC to convert models from different frameworks into NBG format. -- Use the awnn API on the board to perform inference with the model. +- On the PC side, use [ACUITY Toolkit](./cubie_acuity_sdk#acuity-toolkit) to convert models from different frameworks to NBG format +- On the device side, use awnn API for model inference -## Download the ai-sdk Example Repository +## Download ai-sdk Example Repository @@ -17,9 +17,9 @@ git clone https://github.com/ZIFENG278/ai-sdk.git -## YOLACT Inference on the Board +## Board-side YOLACT Inference -Navigate to the YOLACT example code directory. +Navigate to the YOLACT example code file directory path @@ -31,42 +31,78 @@ cd ai-sdk/examples/yolact ### Compile the Example - + + -```bash -make AI_SDK_PLATFORM=a733 -make install AI_SDK_PLATFORM=a733 INSTALL_PREFIX=./ -``` + - + ```bash + make AI_SDK_PLATFORM=a733 + make install AI_SDK_PLATFORM=a733 INSTALL_PREFIX=./ + ``` -Parameter explanation: + -- `AI_SDK_PLATFORM`: Specify the SoC, options are **`a733`**, **`t527`**. -- `INSTALL_PREFIX`: Specify the installation path. + -### Run the Example + -- Set environment variables. + + ```bash + make AI_SDK_PLATFORM=t527 + make install AI_SDK_PLATFORM=t527 INSTALL_PREFIX=./ + ``` + - + -```bash -export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/rock/ai-sdk/viplite-tina/lib/aarch64-none-linux-gnu/NPU_VERSION # NPU_SW_VERSION -``` + - +Parameter description: + +`AI_SDK_PLATFORM`: Specify SoC, optional **`a733`**, **`t527`** + +`INSTALL_PREFIX`: Specify installation path + +### Run Example + +Import environment variables + + + + + + + ```bash + export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/rock/ai-sdk/viplite-tina/lib/aarch64-none-linux-gnu/v2.0 # NPU_SW_VERSION + ``` + + + + + + + + + ```bash + export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/rock/ai-sdk/viplite-tina/lib/aarch64-none-linux-gnu/v1.13 # NPU_SW_VERSION + ``` + + + + + :::tip -Specify `NPU_SW_VERSION`. For A733, choose `v2.0`; for T527, choose `v1.13`. Refer to the [NPU Version Comparison Table](./cubie_acuity_usage#npu-version-comparison-table) for details. +Specify NPU_SW_VERSION: use `v2.0` for A733 and `v1.13` for T527. Please refer to the [NPU Version Comparison Table](./cubie_acuity_usage#npu-version-comparison-table) for NPU information. ::: -Navigate to the example installation directory. +Navigate to the example installation directory ```bash -cd INSTALL_PREFIX/etc/npu/yolact +cd $INSTALL_PREFIX/etc/npu/yolact # ./yolact nbg_model input_picture ./yolact ./model/yolact.nb ./input_data/dog_550_550.jpg ``` @@ -74,7 +110,7 @@ cd INSTALL_PREFIX/etc/npu/yolact :::tip -The example automatically installs the `yolact.nb` model provided by Radxa. You can manually specify the path to your converted NBG model. +The example will automatically install the yolact.nb model provided by Radxa. You can manually specify the path to your own converted NBG model here. ::: ```bash @@ -124,6 +160,6 @@ awnn_uninit total: 0.71 ms. ```
- + yolact demo result
diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_yolov5.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_yolov5.mdx index c4451577a..cb5f256ce 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_yolov5.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_yolov5.mdx @@ -1,15 +1,15 @@ :::tip -This document demonstrates how to run the YOLOv5 object detection model on Allwinner T527/A733 series chips. +This document demonstrates how to run YOLOv5 object detection model inference on Allwinner T527/A733 series chips. ::: -This example uses the pre-trained ONNX format model from [ultralytics/yolov5](https://github.com/ultralytics/yolov5/releases/tag/v6.0) as an example to demonstrate the complete process of converting the model for inference on the board. +This example uses a pre-trained ONNX format model from [ultralytics/yolov5](https://github.com/ultralytics/yolov5/releases/tag/v6.0) to demonstrate the complete process from model conversion to on-device inference. -Deploying YOLOv5 on the board requires two steps: +Deploying YOLOv5 on the development board involves two main steps -- Use the [ACUITY Toolkit](./cubie_acuity_sdk#acuity-toolkit) on the PC to convert models from different frameworks into NBG format. -- Use the awnn API on the board to perform inference with the model. +- On the PC side, use [ACUITY Toolkit](./cubie_acuity_sdk#acuity-toolkit) to convert models from different frameworks to NBG format +- On the device side, use awnn API for model inference -## Download the ai-sdk Example Repository +## Download ai-sdk Example Repository @@ -19,38 +19,55 @@ git clone https://github.com/ZIFENG278/ai-sdk.git -## Model Conversion on PC +## PC-side Model Conversion :::tip -Radxa provides a pre-converted `yolov5.nb` model. Users can directly refer to [**YOLOv5 Inference on the Board**](#yolov5-inference-on-the-board) and skip the PC model conversion section. +Radxa provides a pre-converted `yolov5.nb` model. Users can directly refer to [**On-device YOLOv5 Inference**](#on-device-yolov5-inference) to skip the PC-side model conversion section. ::: :::tip -The files used in the YOLOv5 example are already included in the [ai-sdk example repository](https://github.com/ZIFENG278/ai-sdk.git) under `models/yolov5s-sim`. +The files used in the YOLOv5 example are already included in the `models/yolov5s-sim` directory of the [ai-sdk example repository](https://github.com/ZIFENG278/ai-sdk.git) ::: -- Enter the ACUITY Toolkit Docker container. +- Enter the ACUITY Toolkit Docker container - For ACUTIY Toolkit Docker environment preparation, please refer to [ACUITY Toolkit Environment Configuration](./cubie_acuity_env) + For ACUITY Toolkit Docker environment setup, please refer to [ACUITY Toolkit Environment Configuration](./cubie_acuity_env) - Configure environment variables: + Configure environment variables - + - ```bash - cd ai-sdk/models - source env.sh v3 # NPU_VERSION - ``` + - + + ```bash + cd ai-sdk/models + source env.sh v3 # NPU_VERSION + cp ../scripts/* . + ``` + + + - For A733, choose `v3`; for T527, choose `v2`. + + + + ```bash + cd ai-sdk/models + source env.sh v2 # NPU_VERSION + cp ../scripts/* . + ``` + + + + + :::tip - Refer to the [NPU Version Comparison Table](./cubie_acuity_usage#npu-version-comparison-table) for NPU version selection. + Specify NPU_VERSION: use `v3` for A733 and `v2` for T527. Please refer to the [**NPU Version Comparison Table**](cubie_acuity_usage#npu-version-comparison-table) for reference. ::: -- Download the YOLOv5s ONNX model. +- Download yolov5s ONNX model @@ -61,9 +78,9 @@ The files used in the YOLOv5 example are already included in the [ai-sdk example -- Fix the input size. +- Fix input dimensions - NPU inference only accepts fixed input sizes. Use `onnxsim` to fix the input size. + NPU inference only accepts fixed input dimensions. Here we use onnxsim to fix the input dimensions @@ -74,9 +91,9 @@ The files used in the YOLOv5 example are already included in the [ai-sdk example -- Create a quantization calibration dataset. +- Create quantization calibration dataset - Use a set of images for quantization calibration. Save the image paths in `dataset.txt`. + Use an appropriate number of images to create a quantization calibration dataset. The image paths should be saved in `dataset.txt` @@ -95,17 +112,17 @@ The files used in the YOLOv5 example are already included in the [ai-sdk example images/dog.jpg ``` -- Create model input/output files. +- Create model input/output files - Use [Netron](https://netron.app/) to confirm the input/output names of the ONNX model. + You can use [netron](https://netron.app/) to confirm the input/output names of the ONNX model - + ```bash vim inputs_outputs.txt ``` - + ```vim --inputs images --input-size-list '3,640,640' --outputs '350 498 646' @@ -114,11 +131,11 @@ The files used in the YOLOv5 example are already included in the [ai-sdk example {" "}
- + yolov5s in/output name
-- Directory structure: +- Directory contains files ```bash . @@ -134,25 +151,25 @@ The files used in the YOLOv5 example are already included in the [ai-sdk example |-- yolov5s-sim.onnx ``` -- Parse the model. +- Parse the model :::tip - The `pegasus` script is located in `ai-sdk/scripts` and can be copied to the `models` directory. + The pegasus script is located in ai-sdk/scripts and can be copied to the models directory ::: - Use `pegasus_import.sh` to parse the model into an intermediate representation (IR). This generates `yolov5s-sim.json` (model structure) and `yolov5s-sim.data` (model weights). + Use `pegasus_import.sh` to parse the model into IR expressions, which will generate `yolov5s-sim.json` containing the model structure and `yolov5s-sim.data` containing the model weights - + ```bash ./pegasus_import.sh yolov5s-sim/ ``` - + -- Modify the `yolov5s-sim_inputmeta.yml` file. +- Modify yolov5s-sim_inputmeta.yml file - Update the `scale` value based on the formula `scale = 1 / std`. + Here we modify the scale value according to the formula: scale = 1 / std ```bash scale = 1 / 255 @@ -213,21 +230,21 @@ The files used in the YOLOv5 example are already included in the [ai-sdk example redirect_to_output: false ``` -- Quantize the model. +- Quantize the model - Use `pegasus_quantize.sh` to quantize the model into uint8 format. + Use `pegasus_quantize.sh` to quantize the model into uint8 type - + ```bash ./pegasus_quantize.sh yolov5s-sim/ uint8 10 ``` - + -- Compile the model. +- Compile the model - Use `pegasus_export_ovx.sh` to compile the model into NBG format. + Use `./pegasus_export_ovx.sh` to compile the model into NBG model format @@ -237,11 +254,11 @@ The files used in the YOLOv5 example are already included in the [ai-sdk example - The NBG model is saved in `yolov5s-sim/wksp/yolov5s-sim_uint8_nbg_unify/network_binary.nb`. + The NBG model is saved in `yolov5s-sim/wksp/yolov5s-sim_uint8_nbg_unify/network_binary.nb` -## YOLOv5 Inference on the Board +## Board-side YOLOv5 Inference -Navigate to the YOLOv5 example code directory. +Navigate to the yolov5 example code file directory path @@ -253,43 +270,78 @@ cd ai-sdk/examples/yolov5 ### Compile the Example - + + -```bash -make AI_SDK_PLATFORM=a733 -make install AI_SDK_PLATFORM=a733 INSTALL_PREFIX=./ -``` + - + ```bash + make AI_SDK_PLATFORM=a733 + make install AI_SDK_PLATFORM=a733 INSTALL_PREFIX=./ + ``` -Parameter explanation: + -- `AI_SDK_PLATFORM`: Specify the SoC, options are **`a733`**, **`t527`**. -- `INSTALL_PREFIX`: Specify the installation path. + -### Run the Example + -Set environment variables. + + ```bash + make AI_SDK_PLATFORM=t527 + make install AI_SDK_PLATFORM=t527 INSTALL_PREFIX=./ + ``` + - + -```bash -export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/rock/ai-sdk/viplite-tina/lib/aarch64-none-linux-gnu/NPU_VERSION # NPU_SW_VERSION -``` + - +Parameter description: + +`AI_SDK_PLATFORM`: Specify SoC, optional **`a733`**, **`t527`** + +`INSTALL_PREFIX`: Specify installation path + +### Run Example + +Import environment variables + + + + + + + ```bash + export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/rock/ai-sdk/viplite-tina/lib/aarch64-none-linux-gnu/v2.0 # NPU_SW_VERSION + ``` + + + + + + + + + ```bash + export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/rock/ai-sdk/viplite-tina/lib/aarch64-none-linux-gnu/v1.13 # NPU_SW_VERSION + ``` + + + + + :::tip -Specify `NPU_SW_VERSION`. For A733, choose `v2.0`; for T527, choose `v1.13`. Refer to the [NPU Version Comparison Table](./cubie_acuity_usage#npu-version-comparison-table) for details. +Specify NPU_SW_VERSION: use `v2.0` for A733 and `v1.13` for T527. Please refer to the [**NPU Version Comparison Table**](./cubie_acuity_usage#npu-version-comparison-table) for NPU information. ::: -Navigate to the example installation directory. +Navigate to the example installation directory ```bash -export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/rock/ai-sdk/viplite-tina/lib/aarch64-none-linux-gnu/NPU_VERSION # NPU_SW_VERSION -cd INSTALL_PREFIX/etc/npu/yolov5 +cd $INSTALL_PREFIX/etc/npu/yolov5 # ./yolov5 nbg_model input_picture ./yolov5 ./model/yolov5.nb ./input_data/dog.jpg ``` @@ -297,7 +349,7 @@ cd INSTALL_PREFIX/etc/npu/yolov5 :::tip -The example automatically installs the `yolov5.nb` model provided by Radxa. You can manually specify the path to your converted NBG model. +The example will automatically install the yolov5.nb model provided by Radxa. You can manually specify the path to your own converted NBG model here. ::: ```bash @@ -343,9 +395,9 @@ awnn_destroy total: 1.95 ms. awnn_uninit total: 0.66 ms. ``` -The inference result is saved in `result.png`. +The inference results are saved in result.png
- - YOLOv5s demo output + + yolov5s demo output
From bb57caa9d75f49d7c2a1dba108b97cd55b385363 Mon Sep 17 00:00:00 2001 From: Wu MinLong <726485702@qq.com> Date: Wed, 22 Oct 2025 09:41:07 +0800 Subject: [PATCH 2/2] fix: docs index error --- .../current/common/ai/_cubie_lenet.mdx | 2 +- .../current/common/ai/_cubie_resnet50.mdx | 4 ++-- .../current/common/ai/_cubie_yolov5.mdx | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_lenet.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_lenet.mdx index 3714d1c83..0ea0cefe1 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_lenet.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_lenet.mdx @@ -299,7 +299,7 @@ Import environment variables :::tip -Specify NPU_SW_VERSION, A733 select `v2.0`, T527 select `v1.13`, NPU information对照请参考 [NPU Version Comparison Table](./cubie_acuity_usage#npu-version-comparison) +Specify NPU_SW_VERSION: Select v2.0 for A733, select v1.13 for T527. For NPU information, please refer to the [NPU Version Comparison Table](./cubie_acuity_usage#npu-version-comparison) ::: Enter the example installation directory diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_resnet50.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_resnet50.mdx index a3fc8ea74..d21975042 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_resnet50.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_resnet50.mdx @@ -22,7 +22,7 @@ git clone https://github.com/ZIFENG278/ai-sdk.git ## PC-Side Model Conversion :::tip -Radxa provides a pre-converted `resnet50.nb` model. Users can directly refer to [**On-board ResNet50 Inference** ](#on-board-resnet50-inference) to skip the PC-side model conversion chapter. +Radxa provides a pre-converted `resnet50.nb` model. Users can directly refer to [**On-board ResNet50 Inference** ](#board-side-resnet50-inference) to skip the PC-side model conversion chapter. ::: - Enter the ACUITY Toolkit Docker Container @@ -60,7 +60,7 @@ Radxa provides a pre-converted `resnet50.nb` model. Users can directly refer to :::tip - Specify NPU_VERSION: use `v3` for A733 and `v2` for T527. For reference, see [**NPU Version Comparison Table**](cubie_acuity_usage#npu-version-comparison) + Specify NPU_VERSION: use `v3` for A733 and `v2` for T527. For reference, see [**NPU Version Comparison Table**](cubie_acuity_usage#npu-version-comparison-table) ::: - Download the ResNet50 ONNX Model diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_yolov5.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_yolov5.mdx index cb5f256ce..49353520f 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_yolov5.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/_cubie_yolov5.mdx @@ -22,7 +22,7 @@ git clone https://github.com/ZIFENG278/ai-sdk.git ## PC-side Model Conversion :::tip -Radxa provides a pre-converted `yolov5.nb` model. Users can directly refer to [**On-device YOLOv5 Inference**](#on-device-yolov5-inference) to skip the PC-side model conversion section. +Radxa provides a pre-converted `yolov5.nb` model. Users can directly refer to [**On-device YOLOv5 Inference**](#board-side-yolov5-inference) to skip the PC-side model conversion section. ::: :::tip