Resolve -pthread error that pops up in some (most?) environments

[jeremiahpslewis]

No description provided.

[joka921]

Please tell us and the code, in which circumstances this helps,
and we are ready to merge.

[joka921]

Can you add a short comment, on what this does, and on which environments it is necessary?

[joka921]

Hi,
Thank you very much. On which environments does this make a difference?
Are you still trying to make cross-platform builds work? How far are you with that?
In general, QLever should run on a POSIX system, that supports GCC11 or clang13, where you can (and have to) link
against libstdc++ (libc++ currently doesn't support some C++20 features, which we already use), and where you have a boost implementation, that also links against the libstdc++.

In practice, this normally excludes building on MacOS, unless you manually compile the required boost libraries and link them against libstdc++ (Not yet, but we will soon use boost::program_options which is not header only, then this problem will arise.

Also, since you have been suggesting improvements for quite some time now, may I ask you, what your use case for QLever is,
And why native cross-platform builds for non-linux systems are so important for you.

We are always interested in knowing, where our software is used, and are always willing to help.

[jeremiahpslewis]

Hey! This fix resolves the following build warning (which appears in your builds as well as mine):

From your builds:
https://pipelines.actions.githubusercontent.com/ZGRXQ8YmerkMortb8B7CrpYTGFb53zQYrlq2gRXdk7gt3Wm18o/_apis/pipelines/1/runs/3138/signedlogcontent/2?urlExpires=2022-01-27T16%3A55%3A58.2199126Z&urlSigningMethod=HMACV1&urlSignature=H61bs%2Bfsl0ahkw4KP%2BdVl%2BwCWvbk%2Fhcb2LecCh2%2F8x4%3D

2022-01-27T14:23:18.9430942Z -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
2022-01-27T14:23:18.9431826Z -- Looking for pthread_create in pthreads
2022-01-27T14:23:18.9971133Z -- Looking for pthread_create in pthreads - not found
2022-01-27T14:23:18.9972615Z -- Looking for pthread_create in pthread
2022-01-27T14:23:19.0614685Z -- Looking for pthread_create in pthread - found
2022-01-27T14:23:19.0629925Z -- Found Threads: TRUE  

With this PR, both the 'Failed' and 'not found' should be resolved.

[jeremiahpslewis]

I'm still working on cross-compiling qlever and while I've learned a fair bit about C++ and binaries along the way, I'm still running into the error I posted a while back with the coroutine error log output.

E.g.:

[16:54:45] `_ZN16sparqlExpression6detail85_ZN16sparqlExpression6detail15resultGeneratorIdEEN7cppcoro9gener
atorIT_EES4_m.destroyEPZNS0_15resultGeneratorIdEEN7cppcoro9generatorIT_EES4_mE83_ZN16sparqlExpression6deta
il15resultGeneratorIdEEN7cppcoro9generatorIT_EES4_m.frame' referenced in section `.rodata.cst8' of lib/lib
parser.a(SparqlParserHelpers.cpp.o): defined in discarded section `.text._ZN16sparqlExpression6detail85_ZN
16sparqlExpression6detail15resultGeneratorIdEEN7cppcoro9generatorIT_EES4_m.destroyEPZNS0_15resultGenerator
IdEEN7cppcoro9generatorIT_EES4_mE83_ZN16sparqlExpression6detail15resultGeneratorIdEEN7cppcoro9generatorIT_
EES4_m.frame[_ZN16sparqlExpression6detail15resultGeneratorIdEEN7cppcoro9generatorIT_EES4_m]' of lib/libpar
ser.a(SparqlParserHelpers.cpp.o)
[16:54:45] `_ZN16sparqlExpression6detail85_ZN16sparqlExpression6detail15resultGeneratorIlEEN7cppcoro9gener
atorIT_EES4_m.destroyEPZNS0_15resultGeneratorIlEEN7cppcoro9generatorIT_EES4_mE83_ZN16sparqlExpression6deta
il15resultGeneratorIlEEN7cppcoro9generatorIT_EES4_m.frame' referenced in section `.rodata.cst8' of lib/lib
parser.a(SparqlParserHelpers.cpp.o): defined in discarded section `.text._ZN16sparqlExpression6detail85_ZN
16sparqlExpression6detail15resultGeneratorIlEEN7cppcoro9generatorIT_EES4_m.destroyEPZNS0_15resultGenerator
IlEEN7cppcoro9generatorIT_EES4_mE83_ZN16sparqlExpression6detail15resultGeneratorIlEEN7cppcoro9generatorIT_
EES4_m.frame[_ZN16sparqlExpression6detail15resultGeneratorIlEEN7cppcoro9generatorIT_EES4_m]' of lib/libpar
ser.a(SparqlParserHelpers.cpp.o)
[16:54:45] `_ZN16sparqlExpression6detail93_ZN16sparqlExpression6detail15resultGeneratorINS_4BoolEEEN7cppco
ro9generatorIT_EES5_m.destroyEPZNS0_15resultGeneratorINS_4BoolEEEN7cppcoro9generatorIT_EES5_mE91_ZN16sparq
lExpression6detail15resultGeneratorINS_4BoolEEEN7cppcoro9generatorIT_EES5_m.frame' referenced in section `
.rodata.cst8' of lib/libparser.a(SparqlParserHelpers.cpp.o): defined in discarded section `.text._ZN16spar
qlExpression6detail93_ZN16sparqlExpression6detail15resultGeneratorINS_4BoolEEEN7cppcoro9generatorIT_EES5_m
.destroyEPZNS0_15resultGeneratorINS_4BoolEEEN7cppcoro9generatorIT_EES5_mE91_ZN16sparqlExpression6detail15r
esultGeneratorINS_4BoolEEEN7cppcoro9generatorIT_EES5_m.frame[_ZN16sparqlExpression6detail15resultGenerator
INS_4BoolEEEN7cppcoro9generatorIT_EES5_m]' of lib/libparser.a(SparqlParserHelpers.cpp.o)

I think that qlever is a quite intriguing piece of software which makes it possible to process Wikidata-scale linked data sets on relatively modest hardware; this is relevant for research that I am working on in the accounting field (https://www.accounting-for-transparency.de) as a student research assistant.

Ideally, I would like to build qlever into a Julia-based data processing pipeline which takes open accounting data and generates datasets which can be used for further research. But in order to do this, I need the tech to 'get out of the way', which is where Yggdrasil and prebuilt-binaries come it; cross-compilation comes by way of Julia's Yggdrasil approach.

I want to be able to allow other researchers to run the pipeline at scale and without significant technical knowledge; ideally the more platforms I can support (at a reasonable effort), the better, but linux is often sufficient.

[jeremiahpslewis]

@joka921 If you had time to look at the error I keep running into together with me, I'd love to chat, I just sent you an email so we can discuss. :)

[joka921]

Ok,
The
Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
Still appears. But this doesn't seem to make a difference, the build still works, because finally CMake finds a suitable threading library. Is there a platform on which CMAKE actually fails?

[jeremiahpslewis]

I guess one last point on the topic of cross-compilation: having looked at various SPARQL options out there, it seems like qlever is unique and promising; if you're able to make it more broadly available (in terms of platforms) (and free it from docker), I would expect there might be broad interest, particularly given the quantities of reliable linked open data now available.

[jeremiahpslewis]

Hopefully qlever + Julia stack are complementary, qlever doesn't offer a story about how one loads data into the db and processes it and I think Julia would be a reasonable choice for the glue that does the data prep work.

[joka921]

I think we have to talk, why you want to "free it from docker". The reason why we often use Docker is exactly your story:
Somebody with some machine wants to install some software, and somewhere during the installation of the dependecies, something goes wrong, and the developers of the software cannot really help, because they do not have a machine with the exact setup to reproduce the error.

What in your opinion, or in the domain you are working in, speaks against docker? Or, if you have licensing issues with docker, some similar runtime like podman?

I think the only way of producing builds for not really new operating systems would be to statically link against libstdc++.
Our current philosophy is "Always use the newest features, and use Docker to make them portable" but this clashes with your philosophy of "I want to build this natively on any machine without a lot of dependencies"

[joka921]

@jeremiahpslewis
Do we already know, if the concrete fix that is proposed in this PR makes a difference
(reduces a hard error, not a warning) in any of your builds, or if this is a "red herring"?

[jeremiahpslewis]

Think we can close this.