Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor the generation process of _swig_paddle.so #1083

Closed
gangliao opened this issue Jan 6, 2017 · 6 comments
Closed

Refactor the generation process of _swig_paddle.so #1083

gangliao opened this issue Jan 6, 2017 · 6 comments
Assignees

Comments

@gangliao
Copy link
Contributor

gangliao commented Jan 6, 2017

来源:https://cmake.org/cmake/help/v3.2/module/UseSWIG.html

当前Paddle与swig的编译过程时写在paddle/api/paddle_ld_flags.py, setup.py.inpaddle/api/paddle_api_config.py.in里面,跨平台效果并不好,非常不方便进行维护。

具体做法:去掉paddle_api_config.py.in和paddle_ld_flags.py, 简化setup.py。

简单的说,完成这个过程我们只需要在CMake里面调两个函数就行了:

SWIG_ADD_MODULE(example python example.i example.cxx)
SWIG_LINK_LIBRARIES(example ${PYTHON_LIBRARIES})

官方的说法:

CMake knows how to build shared libraries and loadable modules on many different operating systems. This allows easy cross platform SWIG development. It also can generate the custom commands necessary for driving SWIG from IDE's and makefiles. All of this can be done from a single cross platform input file. 
# This is a CMake example for Python
FIND_PACKAGE(SWIG REQUIRED)
INCLUDE(${SWIG_USE_FILE})

FIND_PACKAGE(PythonLibs)
INCLUDE_DIRECTORIES(${PYTHON_INCLUDE_PATH})

INCLUDE_DIRECTORIES(${CMAKE_CURRENT_SOURCE_DIR})

SET(CMAKE_SWIG_FLAGS "")

SET_SOURCE_FILES_PROPERTIES(example.i PROPERTIES CPLUSPLUS ON)
SET_SOURCE_FILES_PROPERTIES(example.i PROPERTIES SWIG_FLAGS "-includeall")
SWIG_ADD_MODULE(example python example.i example.cxx)
SWIG_LINK_LIBRARIES(example ${PYTHON_LIBRARIES})

The above example will generate native build files such as makefiles, nmake files and Visual Studio projects which will invoke SWIG and compile the generated C++ files into _example.so (UNIX) or _example.pyd (Windows). For other target languages on Windows a dll, instead of a .pyd file, is usually generated.

@gangliao
Copy link
Contributor Author

gangliao commented Jan 6, 2017

improve #1001 #1017

@backyes
Copy link
Contributor

backyes commented Jan 6, 2017

@gangliao

貌似应该也能解决 #883 问题吧。

如果可以, 也fix 这个issue吧, 价值非常大,能极大改善内部用户一键安转的体验,另外,其实目前的swig_paddle使用的编译器不是CMAKE控制的,而是系统环境变量控制的,所以基于cmake 来控制swig的编译理论上也能减少很多复杂编译场景下的bug。

@reyoung
Copy link
Collaborator

reyoung commented Jan 6, 2017

@gangliao 问题不在于build出来那个so,而在于打包成wheel包。

我之前看wheel包似乎只能通过setup.py打包。不太清楚怎么把已经编译好的shared library加到setup.py里面。

@gangliao
Copy link
Contributor Author

gangliao commented Jan 6, 2017

@backyes #1017 这个merge之后,就已经没有#883 什么问题了。https://github.com/PaddlePaddle/Paddle/pull/1017/files#diff-21ac5135e25d5aca9c7bcb46e118118dR158

@gangliao
Copy link
Contributor Author

gangliao commented Jan 6, 2017

@reyoung

setup(
...
package_data=[`_swig_paddle.so`]
)

@gangliao
Copy link
Contributor Author

we already merge this feature into PaddlePaddle. So I closed this PR.

wangxicoding pushed a commit to wangxicoding/Paddle that referenced this issue Dec 9, 2021
lizexu123 pushed a commit to lizexu123/Paddle that referenced this issue Feb 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants