Switching to CMake

[unixod]

What do you think about switching re2c to using CMake build system?

Although the codebase of re2c is cross-platform, the currently used build system breaks portability of the project by imposing hard dependency on GNU toolchain. Switching to CMake may free the project from this dependency and make it possible to build re2c using native compilers on supported OSes. In other words, with CMake the build system in it self will become cross-platform and more clear.

A few days ago, I partially ported re2c to CMake. I call it partially because I didn't enable unit tests and I used sources from bootstrap directory. The only obstacle I encountered during the poring was the necessity to get rid of c99 stdint.h file, because this file relies on config.h. I simply removed all content in c99 stdint.h and left only #include <stdint.h> which seems admissible solution.

So, if you want I may try to contribute to this direction.

[skvadrik]

Eh, people are asking about CMake build system from time to time... I can't say anything for sure, but let's give it a try.

The difficulty is in the details. Bootstrap is requires custom rules, but I think it can be done. The c99_stdint.h header is important for pre-C++11 compatibility (re2c is sometimes used in strange or old environments), so we need to keep it. There is a number of other checks, e.g. for compiler flags, that I think will require custom modules. There are tests and docs. As of recent, there is also libre2c that needs to be built into static and shared library. Finally, there are tricky interdependencies between various autogenerated files that I'm not sure I can port to CMake that easily.

If you create a pull request against master, I will pull it and see how it goes.

Thanks for the porting effort!

[unixod]

Ulya, thanks for your willingness to switch build system to CMake.

I can prepare a PR, but I still don't know how to preserve all content of c99_stdint.h. If I find some admissible workaround how to preserve this file I'll prepare PR.

By the way, if you find some way to omit that file, or realize that the file actually isn't used by clients, let me know, I'll try to prepare PR.

[skvadrik]

I still don't know how to preserve all content of c99_stdint.h

The problem is the dependency of c99_stdint.h on config.h, which is generated by configure script and contains various defines such as HAVE_STDINT_H, PACKAGE_VERSION, etc. To port this stuff on CMake, we need to find equivalent functionality in CMake (ability to to test for various headers, programs, patch files, etc. when configuring the package). Re2c does not need much, but that what is used in configure.ac should be ported.

I have some further questions:

• Current build system uses bash scripts for testing and building documentation. As I understand, with CMake on linux we can continue using these scripts, but on windows there is no bash, so neither tests nor documentation would work. What is the canonical approach here, rewrite all scripts in (a portable subset of) python?

• Are there CMake analogues to make distcheck (command that builds and tests the release tarball)?

• Cross-compilation currently it is as simple as configure --host i686-w64-mingw32, can that be done in CMake?

• There is a bunch of scripts __build_*.sh that build re2c in various configurations by passing parameters to configure. Can they be ported to CMake?

[unixod]

Hi Ulya,

Sorry for the late response)

The problem is the dependency of c99_stdint.h on config.h, which is generated by configure script and contains various defines such as HAVE_STDINT_H, PACKAGE_VERSION, etc.

Yes, CMake provides facilities to test and extract certain properties of the target environment (such as size of types, existence of headers, availability of compiler flags, etc.). If I remember correctly, when I porting this part of re2c's build system, I stuck in determing size of oi8 (because I simply didn't recognize this type). If you help me to clarify what is this type, I may try to implement the generating of config.h in CMake.

By answering to your questions:

• As I understand, re2c's build system uses rst2man to generate documentation. CMake allows running external programs in crossplatform maner, hence in CMake to run rst2man and generate documentation we don't need to use Bash.
  With regard to testing the documentation. I haven't yet delved into this portion of re2c's build system, therefore before answering to this question, I need to understand how documentation is tested. I would appreciate if you help me to clarify this process.

• CMake allows creating packages with sources and with binaries in many different formats (zip, tgz, ..., deb, rpm, msi, etc.). This part of CMake is called CPack. With regard to testing of release tarball, hmm... perhaps it is possible, but first I need to understand the nature of this process. Isn't this as same as testing of sources from which the tarball is generated?

• Crosscompiling is possible in CMake. For that purpose, we need to write toolchain file with a few variables. You may see some examples here: https://cmake.org/cmake/help/v3.6/manual/cmake-toolchains.7.html#cross-compiling

• Yes, there are many ways to port them to CMake.

[skvadrik]

Sorry for the late response)

It's ok! I'll just reopen the issue for now. If you have some time to work on it, great, but discussion is also useful.

I [got] stuck in determing size of oi8 (because I simply didn't recognize this type). If you help me to clarify what is this type, I may try to implement the generating of config.h in CMake.

It's not oi8, it is 0i8: zero literal with suffix i8 to disambiguate size. This suffix is non-standard and probably comes from MSVC, together with __int64. It may be needed on some old MSVC version that predates the existence of stdint.h and long long, and where size of long is 4 bytes. On such platform the only 64-bit signed integer type would be __int64, and the corresponding suffix would be i8.

I expect that CMake, like Autoconf, would be able to check for the existence of any type X, not just some set of predefined types (configure does this by trying to compile a small program that uses X).

As I understand, re2c's build system uses rst2man to generate documentation. CMake allows running external programs in crossplatform maner, hence in CMake to run rst2man and generate documentation we don't need to use Bash.

This is, unfortunately, not so trivial: we also have a shell script genhelp.sh that generates source file help.cc from manpage. This is needed to avoid manual copy-pasting and to keep manpage and help message in sync (and also other docs in gh-pages-gen branch).

[skvadrik]

I need to understand how documentation is tested.

It's not tested in any automatic way.

Usually it doesn't change at all, unless you provide --enable-docs option to configure. If you do, and if you touch the contents of doc folder, manpage doc/re2c.1 and source file help.cc will be regenerated using rst2man and local script genhelp.sh, and their bootstrap versions will be updated.

We only enable doc regeneration conditionally with --enable-docs because we want re2c to build on platforms that don't have rst2man.

With regard to testing of release tarball, hmm... perhaps it is possible, but first I need to understand the nature of this process. Isn't this as same as testing of sources from which the tarball is generated?

Not quite: it also checks that the tarball is correct. There is absolutely no guarantee that we package what we test: e.g. a common mistake is when people forget to package some file (it builds fine on their system, because the file is already in place, but it breaks when testing a freshly unpacked tarball). make distcheck also takes other sanity precautions like making source files read-only to ensure default build doesn't mangle them.

[unixod]

With regard to 0i8, now I see) Thanks for clarification!

I expect that CMake, like Autoconf, would be able to check for the existence of any type X

Yes, sure.

As I see for now the main complexity is concentrated around the logic which is responsible for building docs and boostrap target, but I think this can be solved. I'll try to see what I can to do.

[nightlark]

Here's a partial CMakeLists.txt from a few years ago: https://github.com/nightlark/re2c/blob/master/re2c/CMakeLists.txt

I don't entirely remember where it's at (doesn't do tests/docs or use yacc/bison), but it has at least some of the checks for generating the config.h file, and the bootstrap process used in the existing Makefiles.

[skvadrik]

@nightlark Thanks for sharing. In two years re2c has changed, but it's good to have a starting point.

[ligfx]

Took a crack at this in #275 . It generates config.h, can update docs, does the bootstrap cycle correctly (I'm pretty sure), and builds on Unix and Windows. I couldn't come up with an answer to "make distcheck": CMake doesn't really support this, since all you really need is a Git checkout/tarball.

[skvadrik]

@ligfx Thanks! Please see the comments on #275.