【Hackthon_4th 180】Support HORIZON BPU Backend for FastDeploy #1822

seyosum · 2023-04-14T06:57:54Z

PR types(PR类型)

Add Horizon backend

Description

1、Add Horizon backend
2、Clas model inference
3、PPYOLOE inference

seyosum · 2023-04-17T12:54:17Z

@DefTruth 有空可以帮review下嘛

DefTruth · 2023-04-18T02:20:40Z

@DefTruth 有空可以帮review下嘛

好的，这两天会整体review一下哈，感谢您的贡献~

DefTruth · 2023-04-19T13:14:24Z

FastDeploy.cmake.in

@@ -167,6 +168,23 @@ if(ENABLE_RKNPU2_BACKEND)
  list(APPEND FASTDEPLOY_LIBS ${RKNPU2_LIB})
 endif()

+if(ENABLE_HORIZON_BACKEND)
+  set(DNN_PATH "~/.horizon/ddk/xj3_aarch64/dnn/")


这个路径不能是绝对路径，可以参考paddlelite.cmake的处理。不属于系统so的，需要从云端下载，并安装到用户的编译环境。可以打包一下成tgz提供给我们哈

系统的package和编译器都打包好了，我怎么发你呢

@seyosum 贴个下载链接到这个评论即可

DefTruth · 2023-04-19T13:14:59Z

FastDeploy.cmake.in

@@ -167,6 +168,23 @@ if(ENABLE_RKNPU2_BACKEND)
  list(APPEND FASTDEPLOY_LIBS ${RKNPU2_LIB})
 endif()

+if(ENABLE_HORIZON_BACKEND)
+  set(DNN_PATH "~/.horizon/ddk/xj3_aarch64/dnn/")
+  set(APPSDK_PATH "~/.horizon/ddk/xj3_aarch64/appsdk/appuser/")


DefTruth · 2023-04-19T13:16:45Z

cmake/horizon.cmake

@@ -0,0 +1,12 @@
+set(DNN_PATH "~/.horizon/ddk/xj3_aarch64/dnn/")


同上，不能是绝对路径，请参考paddlelite.cmake的处理

DefTruth · 2023-04-19T13:17:49Z

cmake/horizon.cmake

+set(APPSDK_LIB_PATH ${APPSDK_PATH}/lib/hbbpu)
+set(BPU_libs dnn cnn_intf hbrt_bernoulli_aarch64)
+
+include_directories(${DNN_PATH}/include


这里需要使用 ExternalProject_Add 的方式进行下载和配置

这块目前我还没搞清楚ExternalProject_Add这个的用法，之前没用过😂😂😂，我了解下之后更新吧

DefTruth · 2023-04-19T13:22:25Z

examples/vision/classification/paddleclas/horizon/cpp/CMakeLists.txt

+set(CMAKE_CXX_STANDARD 14)
+
+# 指定下载解压后的fastdeploy库路径
+set(FASTDEPLOY_INSTALL_DIR "/home/xuhao/Hackathon/FastDeploy/build/fastdeploy-0.0.0")


这个路径不能hardcode，参考其他examples的写法，可以让用户设置

DefTruth · 2023-04-19T13:29:13Z

fastdeploy/runtime/enum_variables.cc

@@ -110,6 +118,9 @@ std::vector<Backend> GetAvailableBackends() {
 #ifdef ENABLE_RKNPU2_BACKEND
  backends.push_back(Backend::RKNPU2);
 #endif
+#ifdef ENABLE_HORIZON_BACKEND


这个宏也需要在fastdeploy/core/config.h.in中增加下，可参考已有写法。

DefTruth · 2023-04-19T13:32:13Z

fastdeploy/runtime/backends/horizon/horizon_backend.cc

+}
+bool HorizonBackend::Init(const RuntimeOption& runtime_option){
+    // Init model from file
+    if (!this->LoadModel((char*)runtime_option.model_file.data())) {


FastDeploy统一不使用this关键字

DefTruth · 2023-04-19T13:33:56Z

fastdeploy/runtime/backends/horizon/horizon_backend.cc

+    }
+
+    int ret = -1;
+    if(!this->infer_init){


类私有成员使用下划线结尾

DefTruth · 2023-04-19T13:35:06Z

fastdeploy/runtime/backends/horizon/horizon_backend.cc

+        input_mems_ = (hbDNNTensor*)malloc(sizeof(hbDNNTensor) * input_count);
+        output_mems_ = (hbDNNTensor*)malloc(sizeof(hbDNNTensor) * output_count);
+
+        for(uint32_t i = 0; i < input_count; i++){


可以参考Paddle Lite后端中的写法，增加推理benchmark宏，以支持benchmark功能，对推理块进行包裹。

FastDeploy/fastdeploy/runtime/backends/lite/lite_backend.cc

Line 202 in b30f62a

RUNTIME_PROFILE_LOOP_BEGIN(1)

更详细的说明，还可以参考这个PR：

[Backend] support bechmark mode for runtime and backend #1201

编译带benchmark功能的SDK，需要-DENABLE_BENCHMARK=ON

DefTruth · 2023-04-19T13:39:12Z

fastdeploy/runtime/runtime_option.cc

@@ -65,6 +65,10 @@ void RuntimeOption::UseRKNPU2(fastdeploy::rknpu2::CpuName rknpu2_name,
  device = Device::RKNPU;
 }

+void RuntimeOption::UseHorizon(){


UseHorizon()是指硬件的含义？是否应该提供UseHorizonNPUBackend() API设置backend=Backend::HORIZONNPU？可以参考Sophgo/RKNPU2的写法。

UseHorizon()这块指的是使用地平线backends，我理解之后会有不同的芯片，目前还没有改这部分，之后如果有新增的话可以考虑加参数。

如果是使用具体的推理后端，需要提供类似UseHorizonNPUBackend()来明确语义，Backend强调的是推理后端，比如UseCpu(), UseGpu(), UseKunlunXin()这些则是强调硬件（CPU/GPU/昆仑芯XPU等），强调硬件的接口会执行一些和硬件相关的初始化配置，比如UseGpu(), UseKunlunXin()需要指定device_id，UseKunlunXin()还需要指定L3 Cache size等。带Backend后缀的接口，则强调的是使用哪个推理后端，比如UsePaddleInferBackend(), UsePaddleLiteBackend()等。硬件配置 + 后端配置，形成一个完整的推理条件组合。举个例子：

// CPU + Paddle Inference option.UseCpu(); option.UsePaddleInferBackend(); // CPU + OpenVINO option.UseCpu(); option.UseOpenVINOBackend(); // GPU + Paddle Inference option.UseGpu(); // device_id默认为0 option.UsePaddleInferBackend();

因此建议把硬件API和推理后端API区分开来，在地平线：

option.UseHorizon(); // 一般理解这个指定是硬件 option.UseHorizonNPUBackend(); // 指的是使用哪个推理后端

这块我大致理解了，增加了这部分内容，infer demo也同步改啦

@seyosum LGTM ~

…develop

DefTruth · 2023-04-27T09:10:56Z

FastDeploy.cmake.in

+  set(APPSDK_LIB_PATH ${APPSDK_PATH}/lib/hbbpu)
+  set(BPU_libs dnn cnn_intf hbrt_bernoulli_aarch64)
+
+  include_directories(${DNN_PATH}/include


这个include路径，在编译FD SDK后使用，是必须的吗？如果是必须的需要添加到FASTDEPLOY_INCS中，不是单独include_directories，参考：

list(APPEND FASTDEPLOY_INCS ${LITE_DIR}/include)

这个是非必须的，我改下这块

DefTruth · 2023-04-27T09:13:22Z

examples/vision/classification/paddleclas/horizon/cpp/infer.cc

+  model.GetPreprocessor().DisableNormalize();
+
+
+


代码风格，需要先安装pre-commit check一下。

cd FastDeploy sh ./commit-prepare.sh # then commit your codes

DefTruth · 2023-04-27T09:13:38Z

examples/vision/classification/paddleclas/horizon/cpp/infer.cc

+    return;
+  }
+  // print res
+  // std::cout << res.Str() << std::endl;


删除无用代码

DefTruth · 2023-04-27T09:14:11Z

examples/vision/detection/paddledetection/horizon/cpp/infer_ppyoloe_demo.cc

+  model.GetPostprocessor().ApplyNMS();
+  fastdeploy::vision::DetectionResult res;
+  auto im = cv::imread(image_file);
+  for(int i = 0; i < 100; i++){


案例中不需要保留测速代码

DefTruth · 2023-04-27T09:15:05Z

fastdeploy/fastdeploy_model.cc

-  } else if (runtime_option.device == Device::TIMVX) {
+  } else if (runtime_option.device == Device::SUNRISENPU){
+    return CreateHorizonBackend();
+  }else if (runtime_option.device == Device::TIMVX) {


代码风格，else if前需要空格

add horizon backend and PPYOLOE examples

3a61686

seyosum mentioned this pull request Apr 14, 2023

【PaddlePaddle Hackathon 第四期】任务总览 PaddlePaddle/Paddle#50629

Closed

DefTruth self-requested a review April 14, 2023 07:49

seyosum added 3 commits April 17, 2023 14:07

更改horizon头文件编码规范

f150771

更改horizon头文件编码规范

4b22b9c

更改horizon头文件编码规范

308e26b

Ligoml mentioned this pull request Apr 17, 2023

【PaddlePaddle Hackathon 第四期】任务总览 PaddlePaddle/Paddle#51281

Closed

DefTruth added 2 commits April 18, 2023 10:20

Merge branch 'develop' into develop

904274d

Merge branch 'develop' into develop

48b7937

DefTruth requested changes Apr 19, 2023

View reviewed changes

DefTruth and others added 7 commits April 20, 2023 12:53

Merge branch 'develop' into develop

634d7ca

Merge branch 'develop' into develop

0434a38

增加horizon packages下载及自动安装

98e316f

Merge branch 'develop' of https://github.com/seyosum/FastDeploy into …

eded85a

…develop

Add UseHorizonNPUBackend Method

44aada9

Merge branch 'develop' into develop

50af24d

Merge branch 'develop' into develop

0200b64

DefTruth requested changes Apr 27, 2023

View reviewed changes

seyosum and others added 4 commits April 27, 2023 19:56

删除编译FD SDK后多余的头文件,同时更改部分规范

dd0d9fc

Merge branch 'develop' into develop

07d8805

Update horizon.md

e7ee652

Update horizon.md

c8dfa41

DefTruth approved these changes May 6, 2023

View reviewed changes

DefTruth changed the title ~~【Hackthon_4th 180】在FastDeploy中集成地平线推理引擎~~ 【Hackthon_4th 180】Support BPU Backend for FastDeploy May 6, 2023

DefTruth changed the title ~~【Hackthon_4th 180】Support BPU Backend for FastDeploy~~ 【Hackthon_4th 180】Support HORIZON BPU Backend for FastDeploy May 6, 2023

DefTruth merged commit df8dd3e into PaddlePaddle:develop May 6, 2023
12 of 13 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

【Hackthon_4th 180】Support HORIZON BPU Backend for FastDeploy #1822

【Hackthon_4th 180】Support HORIZON BPU Backend for FastDeploy #1822

seyosum commented Apr 14, 2023

seyosum commented Apr 17, 2023

DefTruth commented Apr 18, 2023

DefTruth Apr 19, 2023

seyosum Apr 21, 2023

DefTruth Apr 21, 2023 •

edited

DefTruth Apr 19, 2023

DefTruth Apr 19, 2023

DefTruth Apr 19, 2023

seyosum Apr 21, 2023

DefTruth Apr 19, 2023

DefTruth Apr 19, 2023

DefTruth Apr 19, 2023

DefTruth Apr 19, 2023

DefTruth Apr 19, 2023

DefTruth Apr 19, 2023

seyosum Apr 21, 2023

DefTruth Apr 23, 2023

seyosum Apr 23, 2023

DefTruth Apr 23, 2023

DefTruth Apr 27, 2023

seyosum Apr 27, 2023

DefTruth Apr 27, 2023

DefTruth Apr 27, 2023

DefTruth Apr 27, 2023

DefTruth Apr 27, 2023

		@@ -0,0 +1,12 @@
		set(DNN_PATH "~/.horizon/ddk/xj3_aarch64/dnn/")

【Hackthon_4th 180】Support HORIZON BPU Backend for FastDeploy #1822

【Hackthon_4th 180】Support HORIZON BPU Backend for FastDeploy #1822

Conversation

seyosum commented Apr 14, 2023

PR types(PR类型)

Description

seyosum commented Apr 17, 2023

DefTruth commented Apr 18, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DefTruth Apr 21, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DefTruth Apr 21, 2023 •

edited