Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

【第三方库离线编译】add submodule pocketfft and xbyak #54344

Merged
merged 3 commits into from
Jun 7, 2023

Conversation

RedContritio
Copy link
Contributor

@RedContritio RedContritio commented Jun 5, 2023

PR types

Others

PR changes

Others

Description

将 pocketfft 和 xbyak 作为子模块引入,以支持离线编译。

@paddle-bot
Copy link

paddle-bot bot commented Jun 5, 2023

你的PR提交成功,感谢你对开源项目的贡献!
请关注后续CI自动化测试结果,详情请参考Paddle-CI手册
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@paddle-bot paddle-bot bot added contributor External developers status: proposed labels Jun 5, 2023
@RedContritio RedContritio force-pushed the offline_cpx branch 3 times, most recently from 02d7afc to 7d74285 Compare June 5, 2023 09:37
@luotao1 luotao1 added the HappyOpenSource 快乐开源活动issue与PR label Jun 5, 2023
@RedContritio RedContritio force-pushed the offline_cpx branch 3 times, most recently from f5455b1 to 426412c Compare June 5, 2023 12:18
@KimBioInfoStudio
Copy link
Contributor

KimBioInfoStudio commented Jun 6, 2023

related issue https://github.com/PaddlePaddle/Paddle/issues/54350, hi @risemeup1 @RedContritio, 我们能先保持cryptopp不动, 讨论下用 https://github.com/abdes/cryptopp-cmake 这个库替代的可行性吗? 貌似 https://github.com/noloader/cryptopp-cmake 已经 archive 不再维护, 原作者也在 readme 中推荐使用 https://github.com/abdes/cryptopp-cmake

@risemeup1
Copy link
Contributor

related issue https://github.com/PaddlePaddle/Paddle/issues/54350, hi @risemeup1 @RedContritio, 我们能先保持cryptopp不动, 讨论下用 https://github.com/abdes/cryptopp-cmake 这个库替代的可行性吗? 貌似 https://github.com/noloader/cryptopp-cmake 已经 archive 不再维护, 原作者也在 readme 中推荐使用 https://github.com/abdes/cryptopp-cmake

ok,好的,那就暂时不动cryptopp

@RedContritio
Copy link
Contributor Author

related issue https://github.com/PaddlePaddle/Paddle/issues/54350, hi @risemeup1 @RedContritio, 我们能先保持cryptopp不动, 讨论下用 https://github.com/abdes/cryptopp-cmake 这个库替代的可行性吗? 貌似 https://github.com/noloader/cryptopp-cmake 已经 archive 不再维护, 原作者也在 readme 中推荐使用 https://github.com/abdes/cryptopp-cmake

好的了解,我看看

@RedContritio RedContritio force-pushed the offline_cpx branch 2 times, most recently from cfb70ab to 03a3ae9 Compare June 6, 2023 04:19
@RedContritio RedContritio changed the title 【第三方库离线编译】add submodule crypotopp, pocketfft and xbyak 【第三方库离线编译】add submodule pocketfft and xbyak Jun 6, 2023
@RedContritio
Copy link
Contributor Author

@risemeup1 待 review

@risemeup1
Copy link
Contributor

@risemeup1 待 review

好的,明天给你合入

COMMAND ${CMAKE_COMMAND} -E remove_directory ${POCKETFFT_SOURCE_DIR}
COMMAND ${CMAKE_COMMAND} -E make_directory ${POCKETFFT_SOURCE_DIR}
COMMAND ${CMAKE_COMMAND} -E copy_directory ${SOURCE_DIR}
${POCKETFFT_SOURCE_DIR}
INSTALL_COMMAND ""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?为什么要copy目录??

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

因为现存代码里有对 extern_pocketfft 目录的引用:

#if defined(PADDLE_WITH_ONEMKL)
#include "paddle/phi/kernels/funcs/mkl_fft_utils.h"
#elif defined(PADDLE_WITH_POCKETFFT)
#define POCKETFFT_CACHE_SIZE 16
#include "extern_pocketfft/pocketfft_hdronly.h"
#endif

为了保留外部代码的一致性,所以需要复制一份到 ${THIRD_PARTY}/pocketfft/src/extern_pocketfft,并且将 ${THIRD_PARTY}/pocketfft/src 作为 include_directories

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

如果不复制的话,就需要将 ${PADDLE_SOURCE_DIR}/third_party 作为 include_directories 了,我觉得暴露这个目录不太优雅。

一个可选的思路的将保留 extern_pocketfft 这个目录,但是仅复制需要的 "extern_pocketfft/pocketfft_hdronly.h"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

因为现存代码里有对 extern_pocketfft 目录的引用:

#if defined(PADDLE_WITH_ONEMKL)
#include "paddle/phi/kernels/funcs/mkl_fft_utils.h"
#elif defined(PADDLE_WITH_POCKETFFT)
#define POCKETFFT_CACHE_SIZE 16
#include "extern_pocketfft/pocketfft_hdronly.h"
#endif

为了保留外部代码的一致性,所以需要复制一份到 ${THIRD_PARTY}/pocketfft/src/extern_pocketfft,并且将 ${THIRD_PARTY}/pocketfft/src 作为 include_directories

#include "extern_pocketfft/pocketfft_hdronly.h" 改为 #include "pocketfft_hdronly.h" 应该也可以吧,然后include_directories(SOURCE_DIR)?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

如果不复制的话,就需要将 ${PADDLE_SOURCE_DIR}/third_party 作为 include_directories 了,我觉得暴露这个目录不太优雅。

一个可选的思路的将保留 extern_pocketfft 这个目录,但是仅复制需要的 "extern_pocketfft/pocketfft_hdronly.h"

复制也是一种方法,但是系统本身有一份再进行复制是不是冗余操作?这样暴露不太优雅?指的是?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

如果不复制的话,就需要将 ${PADDLE_SOURCE_DIR}/third_party 作为 include_directories 了,我觉得暴露这个目录不太优雅。
一个可选的思路的将保留 extern_pocketfft 这个目录,但是仅复制需要的 "extern_pocketfft/pocketfft_hdronly.h"

复制也是一种方法,但是系统本身有一份再进行复制是不是冗余操作?这样暴露不太优雅?指的是?

是这样的,参考 pybind 的引用,我是倾向于 #include "<third_part_name>/<header_filename>" 这种形式。

之所以之前没有改成 #include "pocketfft_hdronly.h" ,是因为感觉不能直观看出是第三方库。

pybind 那边的目录结构是 pybind/include/pybind11/xxx.h,所以能够直接将 ${SOURCE_DIR}/include 作为 include 目录;

不优雅指的是,不同于 pybindpocketfft 目录是展开的,如果想保留 <库名>/<头文件名> 的方式,就可能导致 ${PADDLE_SOURCE_DIR}/third_party 目录暴露,担心后续可能会出现 include 冲突的情况,"${PADDLE_SOURCE_DIR}/third_party/xxx" 和其他位置的 xxx 冲突;

如果修改 INCLUDE_DIRSOURCE_DIR,会导致在 #include 的时候看不到库名。

出于这个考虑,所以直接复制了整个目录到原来的 extern_pocketfft,来保证外部引用的一致。

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

可以,那就复制吧,的确,肉眼第一时间看出来某个第三方库的头文件是比较好的。good job

@risemeup1 risemeup1 merged commit f774780 into PaddlePaddle:develop Jun 7, 2023
@RedContritio RedContritio deleted the offline_cpx branch June 7, 2023 14:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contributor External developers HappyOpenSource 快乐开源活动issue与PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants