A better build system?

[erikd]

We are using this library at my work place and would like to provide you with a better, more standard build/install system.

I have previous experience with:

• autoconf/automake/libtool
• Cmake

I am also willing to look into other well known standard build systems like Meson if you would prefer.

Using a more common/standard build system improves the chances of your library being packaged by all the various Linux distributions as well as package management systems for Mac (like brew) and Windows (like chocolatey).

[dot-asm]

packaged by all the various Linux distributions

Well, it makes little sense, because blst is not really a general-purpose library. Besides, you're obviously talking about the C part, while the library value is rather in its bindings and those are handled within the corresponding build frameworks. Well, not all of them really, and that's where contributions would be most appreciated;-) For example, Python bindings are not "wired" into Python infrastructure. But I'm not suggesting that you jump into it! As we all have own things to do:-) So maybe instead of speaking in hypotheticals, just tell what is it that you're actually missing at work. If your project relies on the "naked" C (why?), and it resides in git, what prevents you from creating a git reference to blst and simply call blst/build.sh as part of your build procedure? It doesn't have any system dependencies and doesn't require any autoconfiguration...

[erikd]

We use blst from Haskell. Haskell has a whole bunch of machinery to handle C libraries via the pkg-config mechanism (even for Mac and Windows). This is for all the C libraries we currently use.

If the existing blst build system provided a pkg-config file I would probably not be asking this question.

[dot-asm]

If the existing blst build system provided a pkg-config file I would probably not be asking this question.

But it's not something we can just add to the repository, right? I mean the file is supposed to be in a specific place on the target system and has to have an absolute path. Unless you're accustomed to set and extend PKG_CONFIG_PATH to your liking. But even then it doesn't tell you anything about how to build. What are the discovery mechanisms there?

[dot-asm]

We use blst from Haskell.

It's getting weirder and weirder :-) But on a serious note. Out of curiosity (and for future reference), do you have to mirror structure declarations or are they "imported" straight from blst.h? Or maybe you ended up querying sizes and using correspondingly sized arrays of 64-bit integers as an opaque storage... Like in Java and C#...

[dot-asm]

What are the discovery mechanisms there?

Even if there are none on the Haskell side, you (not you personally, but the organization) ought to have some procedure to accommodate custom packages into your environment. Again, what prevents you from using git references? Even if git references is not an option, you can create a Makefile (or an equivalent of your choice) that does git clone blst, executes its build.sh and generate pkg-config...

[erikd]

But it's not something we can just add to the repository, right?

No, it requires dynamic system dependent content.

Out of curiosity (and for future reference), do you have to mirror structure declarations or are they "imported" straight from blst.h?

Basically the later. Haskell has very good interoperability with C.

Even if there are none on the Haskell side

Our preference would be for all our C library dependencies use the same mechanism, ie pkg-config. We would like to minimize the number of special snowflakes that have their own special method. At the moment we do git clone blst but then we have special snowflake build code to hook it into the rest of our system, but blst is the only C library we use that has special requirements.

[dot-asm]

But it's not something we can just add to the repository, right?

No, it requires dynamic system dependent content.

"No" as in "not right" or "no" as in "not something we can add"? If former, then elaborate.

snowflake build

It doesn't really answer the question about what are the discovery mechanism. So let me rephrase, if given an opportunity to add one artefact to the blst repository, what would it be? On the other hand, even if there an artefact you would still have to git clone, and you would still classify it as a "snowflake build." :-) :-) :-) In which case no amount of artefacts would help...

[erikd]

"No" as in "not right" or "no" as in "not something we can add"? If former, then elaborate.

No, you can't just add a static file to the repo. The file changes depending on installation location.

So let me rephrase, if given an opportunity to add one artefact to the blst repository, what would it be?

I would have blst provide a pkg-config file.

Having to git clone is less than idea, but not as big a deal as having to MacGyver up a pkg-config file.

[dot-asm]

I would have blst provide a pkg-config file.

Now I'm utterly confused. How do you reconcile "no, you can't" and "I would do it"? As for MacGyver-ing one. Is it really that much of a challenge to have the script that does git clone and calls build.sh generate a few lines?

[eli-schwartz]

So let me rephrase, if given an opportunity to add one artefact to the blst repository, what would it be?

If that artefact was a file called meson.build then it could internally generate a spec-compliant pkg-config file containing paths correct for the installation location, using the single line import('pkgconfig').generate(the_library) after building a library.

(Neither cmake nor autotools have pkgconfig generators, by the way.)

BTW: meson could also handle the python bindings natively, including handling the correct versioned filename as well as installing it to the system site-packages, which would greatly simplify the current extremely ad-hoc approach. Meson also can generate emscripten projects as well as compile jars / JNI and compile rust projects if you manually wrap Cargo.toml dependencies using meson (that's probably more work than you're interested in, though it's on the roadmap to generate this on the fly).

You can do a lot better than build.sh, basically. :)

[dot-asm]

You can do a lot better than build.sh, basically. :)

Problem is that there are too many people that would define "do" differently, let alone "better":-) Anyway, the intention behind build.sh is to provide a baseline if you wish. Something people can use as a starting point in a build system of their choice. I'm suggesting to use build.sh, because if it works, then it's done. But if there is an aversion to using it, it's not really a secret that it takes just two files to compile, no configuration needed.

As for Python. The preference is to have something that works with minimum extra requirements. I mean if Python has an "official" way to build modules that requires only Python itself, then it would be preferred over anything else. As already mentioned, I didn't actually dig into it, expecting that users would fill in:-) The run.me-s are really about testing.

But instead of just talking, care to concoct an example?

[dot-asm]

Please see #196.