CURA-6187 Fixes for building in docker #94

LipuFei · 2019-02-22T10:47:31Z

This commit should work in both docker (using centos7 with cmake 3.13) and the current build/CI servers (with cmake <3.12).

Requires CMake 3.6+
Cleanups and Fixes for FindSIP
Fix find Python to prefer Python 3 found in CMAKE_PREFIX_PATH

LipuFei · 2019-04-02T07:02:41Z

CURA-6188 Also added GitLab CI files.

thopiekar · 2019-04-04T18:27:29Z

(Regarding your question whether I can test the PR 😉)

Sure! Excuse that I was not replying recently.
I got very busy with my (main) job again.

thopiekar · 2019-04-04T19:03:35Z

Up and running: https://thopiekar.eu:5443/cura/cdk/pipelines/314
Short explanation:

My fork works in 3 stages:
- Build of all build tools
- Build of all development tools (to do tests)
- Build of Ultimaker software

However, I never came to the point, where software distributions are done, eg. AppImage for Linux, NSIS for Windows, etc.
So my fork is basically a mix of cura-build and -environment with extra software. Also can cache download, etc.
If you are interested in any further I can give you more details.

rburema · 2019-04-09T14:34:51Z

CMakeLists.txt

+
+    # FIXME: Remove the code for CMake <3.12 once we have switched over completely.
+    # FindPython3 is a new module since CMake 3.12. It deprecates FindPythonInterp and FindPythonLibs. The FindPython3
+    # module is copied from the CMake repository here so in CMake <3.12 we can still use it.


I(/we) assume this is still 'fixme' because we don't want to rely on the copy once we're over to 3.12+?

Our current (old) systems are still using CMake < 3.12, but in the docker image, it's CMake >= 3.12. This FIXME is to keep compatibility for now. Once the old systems are retired, we can remove this FIXME.

The comments are not entirely correct. I've updated them in e15df83.

hroncok · 2019-06-25T10:02:41Z

Any chance I can force the Python module to be installed to /usr/lib64/python3.7/site-packages instead of /usr/lib/python3.7/site-packages? It used to work automatically, but now it stopped.

I've tried with:

-DPYTHON_SITE_PACKAGES_DIR=/usr/lib64/python3.7/site-packages
-DPython3_SITELIB=/usr/lib64/python3.7/site-packages
-DPython3_SITEARCH=/usr/lib64/python3.7/site-packages

But it always ends up in /usr/lib/python3.7/site-packages

LipuFei · 2019-06-25T10:48:23Z

@hroncok You can try CMAKE_INSTALL_LIBDIR=lib (see https://cmake.org/cmake/help/v3.12/module/GNUInstallDirs.html)

EDIT: CMAKE_INSTALL_LIBDIR=lib64

hroncok · 2019-06-25T11:42:28Z

Doesn't help:

-DCMAKE_INSTALL_LIBDIR=lib64

-- Installing: /builddir/build/BUILDROOT/libarcus-4.1.0-1.fc31.x86_64/usr/lib/python3.7/site-packages/Arcus.so
-- Installing: /builddir/build/BUILDROOT/libarcus-4.1.0-1.fc31.x86_64/usr/lib64/libArcus.so.1.1.0
-- Installing: /builddir/build/BUILDROOT/libarcus-4.1.0-1.fc31.x86_64/usr/lib64/libArcus.so.3
-- Installing: /builddir/build/BUILDROOT/libarcus-4.1.0-1.fc31.x86_64/usr/lib64/libArcus.so
-- Installing: /builddir/build/BUILDROOT/libarcus-4.1.0-1.fc31.x86_64/usr/include/Arcus/Socket.h
-- Installing: /builddir/build/BUILDROOT/libarcus-4.1.0-1.fc31.x86_64/usr/include/Arcus/SocketListener.h
-- Installing: /builddir/build/BUILDROOT/libarcus-4.1.0-1.fc31.x86_64/usr/include/Arcus/Types.h
-- Installing: /builddir/build/BUILDROOT/libarcus-4.1.0-1.fc31.x86_64/usr/include/Arcus/ArcusExport.h
-- Installing: /builddir/build/BUILDROOT/libarcus-4.1.0-1.fc31.x86_64/usr/include/Arcus/MessageTypeStore.h
-- Installing: /builddir/build/BUILDROOT/libarcus-4.1.0-1.fc31.x86_64/usr/include/Arcus/Error.h
-- Installing: /builddir/build/BUILDROOT/libarcus-4.1.0-1.fc31.x86_64/usr/lib64/cmake/Arcus/Arcus-targets.cmake
-- Installing: /builddir/build/BUILDROOT/libarcus-4.1.0-1.fc31.x86_64/usr/lib64/cmake/Arcus/Arcus-targets-noconfig.cmake
-- Installing: /builddir/build/BUILDROOT/libarcus-4.1.0-1.fc31.x86_64/usr/lib64/cmake/Arcus/ArcusConfig.cmake
-- Installing: /builddir/build/BUILDROOT/libarcus-4.1.0-1.fc31.x86_64/usr/lib64/cmake/Arcus/ArcusConfigVersion.cmake

LipuFei · 2019-06-25T12:04:57Z

@hroncok I think this has to do with where the python3.7 libraries are installed. Because the python modules are installed in usr/lib/..., it simply uses that path to install the Arcus python modules. Maybe you can try a trick to create a lib64 symbolic link pointing to lib or the other way around.

hroncok · 2019-06-25T12:46:53Z

I'm afraid I don't understand. This is an extension module and hence need to be installed in sitearch. sitearch is determined by distutils.sysconfig.get_python_lib(plat_specific=1), instead it is installed into sitelib (plat_specific=0).

The /builddir/build/BUILDROOT/libarcus-4.1.0-1.fc31.x86_64/usr/directory is empty before I do make install DESTDIR=/builddir/build/BUILDROOT/libarcus-4.1.0-1.fc31.x86_64.

Maybe you can try a trick to create a lib64 symbolic link pointing to lib or the other way around.

How does that help? I can "trick" it by mving that file after make install, I'm just looking for a less hackish solution.

Ghostkeeper · 2019-06-27T12:22:44Z

I think the bug in this case lies in the FindPython script that we're using. This detects the installation folder that you need changed. On Cura's side we can only make hacks around that script, but the real bug is in CMake itself here.

hroncok · 2019-06-27T12:24:27Z

That macro should give you proper Python_SITEARCH. Where do you instruct cmake to install the Python extension? I don't really know cmake, but maybe you need to tell it it belongs to Python_SITEARCH?

Ghostkeeper · 2019-06-27T12:33:50Z

The code that causes the plug-in to be installed is this line:

libArcus/CMakeLists.txt

Line 98 in 63f3f00

add_sip_python_module(Arcus python/Socket.sip Arcus)

This calls upon a CMake function in the Sip import of CMake. The only documentation of this function that I found online was here: https://api.kde.org/cmake/modules.html (search for add_sip_python_module).

hroncok · 2019-06-27T12:41:25Z

This looks suspicious:

libArcus/cmake/SIPMacros.cmake

Line 127 in 63f3f00

    
               INSTALL(TARGETS ${_logical_name} DESTINATION "${Python3_SITELIB}/${_parent_module_path}")

hroncok · 2019-06-27T12:46:28Z

Changing that to Python3_SITEARCH fixes the problem. I'd report that to the upstream of those macros, but I have no idea where is that.

Ghostkeeper · 2019-06-27T13:27:29Z

Aha, good find. I thought that add_sip_python_module came from the find_package(SIP REQUIRED) line, but it's not from there apparently.

The upstream of those macros is @sedwards2009 who was since hired by Ultimaker (by sheer coincidence)... But I doubt that he's still involved in this. What exact change did you make, and perhaps you'd like to get recognition of your change by making a PR? 😄

hroncok · 2019-06-27T13:29:36Z

I can definitively send a PR that just changes it here (and possible in libsavitar), I just assumed those are vendored/bundled/copied from somewhere else.

The sitearch directory is used for compiled code, such as this extension module. See Ultimaker#94 (comment)

hroncok · 2019-06-27T13:34:51Z

🎉 #99 🎉

The sitearch directory is used for compiled code, such as this extension module. See Ultimaker/libArcus#94 (comment)

hroncok · 2019-06-27T14:22:34Z

And Ultimaker/libSavitar#18

LipuFei added 5 commits February 22, 2019 10:52

Change to require CMake 3.6

64b5eb1

Move find_package Python to CMakeLists.txt

c8b8c90

FindSIP should prefer CMAKE_PREFIX_PATH

195f8cd

Prefer python3 in CMAKE_PREFIX_PATH

7873a37

FindSIP should fail if it fails to get version

eabd935

LipuFei force-pushed the WIP_fix_docker branch from 4621109 to eabd935 Compare February 22, 2019 10:52

LipuFei mentioned this pull request Feb 22, 2019

CURA-6187 Fixes for building in docker Ultimaker/libSavitar#13

Merged

LipuFei changed the title ~~Fixes for building in docker~~ CURA-6187 Fixes for building in docker Feb 22, 2019

Ghostkeeper approved these changes Feb 28, 2019

View reviewed changes

LipuFei added 2 commits April 1, 2019 13:45

Fix CMake compatibility problem

724d8f6

Use GitLab CI

d34543c

LipuFei mentioned this pull request Apr 2, 2019

[WIP] CMake: Find Python3 package and make use of new variables #90

Closed

rburema reviewed Apr 9, 2019

View reviewed changes

Fix comments and find_package(Python3 3.4)

e15df83

Ghostkeeper merged commit e15df83 into master Apr 17, 2019

Ghostkeeper deleted the WIP_fix_docker branch April 17, 2019 14:53

hroncok added a commit to hroncok/libArcus that referenced this pull request Jun 27, 2019

Install the Python extension module to Python sitearch, not sitelib

c3b0028

The sitearch directory is used for compiled code, such as this extension module. See Ultimaker#94 (comment)

hroncok mentioned this pull request Jun 27, 2019

Install the Python extension module to Python sitearch, not sitelib #99

Merged

hroncok added a commit to hroncok/libSavitar that referenced this pull request Jun 27, 2019

Install the Python extension module to Python sitearch, not sitelib

e567eb0

The sitearch directory is used for compiled code, such as this extension module. See Ultimaker/libArcus#94 (comment)

hroncok mentioned this pull request Jun 27, 2019

Install the Python extension module to Python sitearch, not sitelib Ultimaker/libSavitar#18

Merged

Ghostkeeper mentioned this pull request May 13, 2021

Remove the Debian hack from CMakeLists.txt #81

Closed

CURA-6187 Fixes for building in docker #94

CURA-6187 Fixes for building in docker #94

Uh oh!

Conversation

LipuFei commented Feb 22, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

LipuFei commented Apr 2, 2019

Uh oh!

thopiekar commented Apr 4, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thopiekar commented Apr 4, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rburema Apr 9, 2019

Choose a reason for hiding this comment

Uh oh!

LipuFei Apr 10, 2019

Choose a reason for hiding this comment

Uh oh!

LipuFei Apr 10, 2019

Choose a reason for hiding this comment

Uh oh!

hroncok commented Jun 25, 2019

Uh oh!

LipuFei commented Jun 25, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hroncok commented Jun 25, 2019

Uh oh!

LipuFei commented Jun 25, 2019

Uh oh!

hroncok commented Jun 25, 2019

Uh oh!

Ghostkeeper commented Jun 27, 2019

Uh oh!

hroncok commented Jun 27, 2019

Uh oh!

Ghostkeeper commented Jun 27, 2019

Uh oh!

hroncok commented Jun 27, 2019

Uh oh!

hroncok commented Jun 27, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Ghostkeeper commented Jun 27, 2019

Uh oh!

hroncok commented Jun 27, 2019

Uh oh!

hroncok commented Jun 27, 2019

Uh oh!

hroncok commented Jun 27, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

LipuFei commented Feb 22, 2019 •

edited

Loading

thopiekar commented Apr 4, 2019 •

edited

Loading

thopiekar commented Apr 4, 2019 •

edited

Loading

LipuFei commented Jun 25, 2019 •

edited

Loading

hroncok commented Jun 27, 2019 •

edited

Loading