Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewrite cmake/external/protobuf.cmake and add proto_library to cmake/generic.cmake #2569

Closed
wants to merge 8 commits into from

Conversation

wangkuiyi
Copy link
Collaborator

@wangkuiyi wangkuiyi commented Jun 22, 2017

Fixes #2570
Fixes #2568
Fixes #2567

# protobuf_generate_cpp. This find_package invocation might fail if
# the host doesn't have protobuf install. In that case, it overwrites
# some variables like PROTOBUF_LIBRARIES. Because we don't really use
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the reason I did not modify this code is too hard to do that, because its also include how to build protobuf on arm, it's better if yiqun can review this. we only need to import the generate protobuf libs as name protobuf

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the reminder. I see. I am basically rewriting this protobuf.cmake from scratch, in hope that we can rebuild those functions step-by-step.

@wangkuiyi wangkuiyi requested a review from Xreki June 22, 2017 23:31
@wangkuiyi wangkuiyi requested a review from helinwang June 23, 2017 03:01
Copy link
Contributor

@Xreki Xreki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

简而言之,protobuf.cmake的功能是:

  • 交叉编译:
    • 编译host版protobuf,使用protoc
    • 编译target版protobuf,使用include和libprotobuf.a
  • 普通编译:
    • 通过PROTOBUF_ROOT寻找protobuf库,找到则退出
    • 没找到:通过find_package寻找3.1版本以上的protobuf库,找到则退出
    • 没找到:编译target版protobuf

这个PR修改之前,protobuf.cmake的工作流程是:

  1. line 29-44,在用户手动配置的PROTOBUF_ROOT路径下面寻找protobuf,若找到则调用line 17-23定义的marco PROMPT_PROTOBUF_LIB,其中打印出protoclibprotobuf.a的位置信息,以及protobuf的版本信息,然后退出整个protobuf.cmake。这个改动是由@reyoungAdd user can define PROTOBUF_ROOT. #2281Set protobuf version when PROTOBUF_ROOT set #2297 中引入,原因是一些场景下,并不要求3.1以上的版本。可通过cmake -DPROTOBUF_ROOT=xxx ..来配置。
  2. 若用户没有配置PROTOBUF_ROOT,或者是在PROTOBUF_ROOT下没有找到完整的protobuf库,cmake流程会走到line 104,后面的流程分是否交叉编译两种情况,修改在Add cross-compiling toolchain files for Android and Raspberry Pi. #1973 中引入:
if 交叉编译:
    调用find_package(Prorobuf ${PROTOBUF_VERSION}),并且要求版本在3.1以上。没找到或者版本过低,都会设置PROTOBUF_FOUND为OFF。对应line 106-113
else:
    编译host版本的protobuf即build_protobuf(protobuf_host TRUE),调用line 45-102定义的function build_protobuf。
    设置PROTOBUF_PROTOC_EXECUTABLE为host版本的protoc
    注意,这个时候没有设置PROTOBUF_FOUND的值。对应代码line 115-119

if PROTOBUF_FOUND没有设置或者为OFF:
    编译target版本的protobuf即build_protobuf(protobuf FALSE),仍然调用line 45-102定义的function build_protobuf。
    设置PROTOBUF_INCLUDE_DIR为target版本的include
    if 不是交叉编译:
        设置PROTOBUF_PROTOC_EXECUTABLE为target版本的protoc
    设置相关的LIBIRARY路径为target protobuf的lib路径

调用line 17-23定义的macro PROMPT_PROTOBUF_LIB,打印protobuf库的信息

所以,我建议对这个文件的修改是:

  • line 123:build_protobuf(protobuf FALSE) -> build_protobuf(extern_protobuf FALSE),并将line 124-134中的protobuf都替换成extern_protobuf
  • 无论哪个分支,都是调用macro(PROMPT_PROTOBUF_LIB)总结并退出的,因此建议在line17-23 macro(PROMPT_PROTOBUF_LIB)中添加add_library内容,还需要个参数控制是否存在extern_protobuf依赖:
ADD_LIBRARY(protobuf STATIC IMPORTED GLOBAL)
SET_PROPERTY(TARGET protobuf PROPERTY IMPORTED_LOCATION ${PROTOBUF_LIBRARIES})
IF(some_flag)
    ADD_DEPENDENCIES(protobuf extern_protobuf)

ADD_LIBRARY(protobuf_lite STATIC IMPORTED GLOBAL)
SET_PROPERTY(TARGET protobuf_lite PROPERTY IMPORTED_LOCATION ${PROTOBUF_LITE_LIBRARIES})
IF(some_flag)
    ADD_DEPENDENCIES(protobuf_lite extern_protobuf)

ADD_LIBRARY(protoc STATIC IMPORTED GLOBAL)
SET_PROPERTY(TARGET protoc PROPERTY IMPORTED_LOCATION ${PROTOC_LIBRARIES})
IF(some_flag)
    ADD_DEPENDENCIES(protoc extern_protobuf)

@wangkuiyi
Copy link
Collaborator Author

编译host版protobuf,使用protoc

这个和

普通编译

有啥区别呢?为什么前者的结果要叫做 protobuf_host 呢?

@Xreki
Copy link
Contributor

Xreki commented Jun 23, 2017

@wangkuiyi 我们在进行交叉编译的时候,比如交叉编译android库的时候,会配置CMAKE_C_COMPILER=arm-linux-androideabi-gccCMAKE_C_FLAGS="-marm -mfpv=neon",使用arm-linux-androideabi-gcc编译出的可执行程序是不能在host机器上执行的。
在编译Paddle的时候,需要在编译机器上运行protoc --cpp_out XXX a.proto以生成a.pb.ha.pb.cc,因此protoc必须是host机器上的可执行程序,需要使用host上的gcc来编译。
protobufprotobuf_host的区别是使用的编译器和编译选项不一样,具体见line 64-77。

@wangkuiyi
Copy link
Collaborator Author

This PR includes too many changes, and there had been a better implementation for the change of making protobuf dependable by cc_library. I (or someone else) will try add proto_libarary to generic.cmake later.

@wangkuiyi wangkuiyi closed this Jun 25, 2017
heavengate pushed a commit to heavengate/Paddle that referenced this pull request Aug 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants