-
Notifications
You must be signed in to change notification settings - Fork 182
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a CMake build system #275
Conversation
Thanks for your effort! I will do my best to integrate it, but there are a few problems that need to be solved before merging. (And I may discover some more as I try various build scenarios and configurations.) If you don't feel like working on this, many thanks for your work anyway. Here is what I found so far:
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -344,6 +344,6 @@ if (RE2C_BUILD_LIBS)
include(CheckIncludeFileCXX)
check_include_file_cxx("re2/re2.h" HAVE_RE2_H)
if(HAVE_RE2_H)
- target_link_libraries(bench_libre2c re2c)
+ target_link_libraries(bench_libre2c libre2c)
endif()
endif()
--- a/cmake/Re2cBootstrapLexer.cmake
+++ b/cmake/Re2cBootstrapLexer.cmake
@@ -32,8 +32,10 @@ function(re2c_bootstrap_lexer re_file cc_file)
if(NOT TARGET re2c)
message(FATAL_ERROR "'re2c' is not a valid target")
endif()
+ get_filename_component(parent_dir "${cc_file}" DIRECTORY)
add_custom_command(
OUTPUT ${outputs}
+ COMMAND ${CMAKE_COMMAND} -E make_directory ${parent_dir}
COMMAND
cmake -Dre2c="$<TARGET_FILE:re2c>" -Dsource_dir="${CMAKE_CURRENT_SOURCE_DIR}"
-Dre_file="${re_file}" -Dcc_file="${cc_file}" -Dh_file="${h_file}" --- a/cmake/Re2cBootstrapParser.cmake
+++ b/cmake/Re2cBootstrapParser.cmake
@@ -1,7 +1,9 @@
function(re2c_bootstrap_parser ypp_file cc_file h_file)
+ get_filename_component(parser_dir "${cc_file}" DIRECTORY)
if (BISON_EXECUTABLE)
add_custom_command(
OUTPUT ${cc_file} ${h_file}
+ COMMAND cmake -E make_directory "${parser_dir}"
COMMAND "${BISON_EXECUTABLE}" --defines="${h_file}" -o "${cc_file}" "${CMAKE_CURRENT_SOURCE_DIR}/${ypp_file}"
COMMAND cmake -E copy_if_different "${cc_file}" "${CMAKE_CURRENT_SOURCE_DIR}/bootstrap/${cc_file}"
COMMAND cmake -E copy_if_different "${h_file}" "${CMAKE_CURRENT_SOURCE_DIR}/bootstrap/${h_file}"
@@ -10,9 +12,9 @@ function(re2c_bootstrap_parser ypp_file cc_file h_file)
else()
add_custom_command(
OUTPUT ${cc_file} ${h_file}
- COMMAND cmake -E make_directory "src/parse"
- COMMAND cmake -E copy "${CMAKE_CURRENT_SOURCE_DIR}/bootstrap/${cc_file}" "src/parse"
- COMMAND cmake -E copy "${CMAKE_CURRENT_SOURCE_DIR}/bootstrap/${h_file}" "src/parse"
+ COMMAND cmake -E make_directory "${parser_dir}"
+ COMMAND cmake -E copy "${CMAKE_CURRENT_SOURCE_DIR}/bootstrap/${cc_file}" "${parser_dir}"
+ COMMAND cmake -E copy "${CMAKE_CURRENT_SOURCE_DIR}/bootstrap/${h_file}" "${parser_dir}"
DEPENDS
"${CMAKE_CURRENT_SOURCE_DIR}/bootstrap/${cc_file}"
"${CMAKE_CURRENT_SOURCE_DIR}/bootstrap/${h_file}"
|
@skvadrik , thanks for the thoughtful review comments.
|
Great, thank you! Very good going! Meanwhile I found a few more things:
To answer your question:
Sure, I by no means meant testing all of them, just a few different configurations that are often used:
It's good that you keep separate commits while working on the patch set, but before merging I think all commits should be squashed in one with a descriptive commit message that lists a few of the above configure/build scenarios (what you wrote above, "The general way to invoke CMake ...", is also very useful information). Not that it can't be found elsewhere (e.g. in re2c release notes), but some people may read the commits instead of release notes. Thanks again for what you've done already. |
|
@@ -0,0 +1,16 @@ | |||
include(CheckCXXCompilerFlag) | |||
# Iff C++ compiler recognizes 'flag', append 'flag' and 'implied-flags' to CXXFLAGSDEFAULT |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is 'Iff' here means 'if and only if'? Don't you think this is bit nerdy? :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@sergeyklay It comes from the configure.ac, which was added by me back in the days when I was an undergrad student and thought the world could work on 1st order predicate logic. :D
@ligfx just did a good job at porting the comment as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually, that was not so long ago: 4c7a3c8
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@skvadrik @sergeyklay I mean it's not wrong, and you could always go more nerdy and start using the symbols like ⇔ !
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not until re2c is rewriten in haskell. ;)
Almost there! I have two things to fix (and for both I have suggestions how to fix them), and then I can merge.
# optionally build shared libraries
if (RE2C_BUILD_LIBS_SHARED)
add_library(libre2c_shared SHARED ${libre2c_sources})
if (UNIX)
install(TARGETS libre2c_shared)
endif()
set_target_properties(libre2c_shared PROPERTIES OUTPUT_NAME "re2c")
endif()
# build static libraries
add_library(libre2c_static STATIC ${libre2c_sources})
if (UNIX)
install(TARGETS libre2c_static)
endif()
set_target_properties(libre2c_static PROPERTIES OUTPUT_NAME "re2c") To be used like
...
function(try_cxxflag flag)
set(varname "cxxflag_${flag}")
string(REPLACE "=" "_" varname "${varname}")
check_cxx_compiler_flag("${flag}" ${varname})
if(${varname})
set(CMAKE_CXX_FLAGS "${flag} ${CMAKE_CXX_FLAGS}" PARENT_SCOPE)
foreach(implied_flag IN LISTS ARGN)
set(CMAKE_CXX_FLAGS "${implied_flag} ${CMAKE_CXX_FLAGS}" PARENT_SCOPE)
endforeach()
endif()
endfunction() This way default flags are prepended, not appended. Later on this can be changed to avoid reversing the list of default flags.
I thought it over, and I agree with you reasoning. This was probably needed before we enforced the ordering requirement that all generated sources must be built before any compilation starts.
Thanks, works for me now. What did you fix?
I found a way to do build mingw-compiled re2c and libre2c and test them under wine, so this one is solved. I took me a bit of time because linker flags
Yes, maybe even until the next release cycle to give maintainers more time. But as a paranoid maintainer I don't want to take responsibility for anything that I cannot maintain, and I certainly want cmake build system to used, tested and covered by CI, and I plan to use it myself and eventually deprecate autotools. Not because I don't like autotools --- quite the contrary, I think for Unix systems they are more elegant than cmake. But living with two build systems just won't work in the long run.
Your commit message is is quite perfect (and helped me a few times already). Just make sure you update it if you do any last-minute changes. |
0563c56
to
0ee8182
Compare
|
I had two reasons to change the code: first, I wanted to simplify the logic, and second, I wanted to get rid of the mingw linking error in the default configuration.
Thanks for keeping close to Autotools. Your reasoning indeed makes sense. This is the final variant that I ended up with. It behaves like your original code: build both shared and static libraries by default, but it is possible to choose either variant exclusively with # build shared libraries
if ((NOT DEFINED BUILD_SHARED_LIBS) OR BUILD_SHARED_LIBS)
add_library(libre2c_shared SHARED ${libre2c_sources})
set_target_properties(libre2c_shared PROPERTIES OUTPUT_NAME "re2c")
if (UNIX)
install(TARGETS libre2c_shared)
endif()
endif()
# build static libraries
if ((NOT DEFINED BUILD_SHARED_LIBS) OR (NOT BUILD_SHARED_LIBS))
add_library(libre2c_static STATIC ${libre2c_sources})
set_target_properties(libre2c_static PROPERTIES OUTPUT_NAME "re2c")
if (UNIX)
install(TARGETS libre2c_static)
endif()
endif()
# define top-level aliases to either static or shared libraries (default is static)
if (BUILD_SHARED_LIBS)
add_library(libre2c ALIAS libre2c_shared)
else()
add_library(libre2c ALIAS libre2c_static)
endif()
...
target_link_libraries(test_libre2c libre2c)
...
target_link_libraries(bench_libre2c libre2c)
Yes, I know that cmake appends some flags on its own when a certain build type is used. This should not be a problem, because it is possible to configure without
Ah! I was grepping for
I was experimenting with various combinations of At first I didn't realize that I was trying to link against shared My understanding is that linking with static libraries
No, let's not add a check --- it is best to keep such workarounds out of the build system, preferably in helper scripts. Every bit of extra logic in the build system makes it harder to read and understand. This complexity is ok in cases like bootstrap, but for simple or rare cases it is better not to clutter the build rules with extra logic. One minor fix that is needed is to do
Let's do it in a follow-up commit, and let's keep the autotools scripts and write a new set of scripts for cmake. I wanted to make sure that it is possible to cover all these scenarios in principle. |
Invoke it like this on Unix: mkdir build cd build cmake .. \ [-DCMAKE_C_FLAGS=...] [-DCMAKE_CXX_FLAGS=...] \ [-DCMAKE_EXE_LINKER_FLAGS=...] \ [-DCMAKE_SHARED_LINKER_FLAGS=...] [-DCMAKE_STATIC_LINKER_FLAGS=...] \ [-DCMAKE_C_COMPILER=...] [-DCMAKE_CXX_COMPILER=...] \ [-DCMAKE_BUILD_TYPE=Debug|Release|RelWithDebInfo] \ [-DRE2C_REBUILD_DOCS=YES] \ [-DRE2C_BUILD_LIBS=YES [-DBUILD_SHARED_LIBS=YES|NO]] \ [-G Ninja] make -j # or ninja, if "-G Ninja" was used Or on Windows, install Visual Studio's CMake tools and then open this folder.
|
Hooray \o/ I've merged this. Thanks @ligfx for all your work! One question regarding static libraries, why do you have to add A few things are still needed before we can switch to this as default build system (like |
Windows shared libraries generate a .dll file (the shared object) and a .lib file (the static object to interface with the shared object.. or something like that). Static libraries just generate a .lib file. If a shared and static library have the same name, their .lib files will conflict |
Ah, got it. I should add a comment in CMakeLists.txt to remember this. |
2.0.3 (2020-08-22) ~~~~~~~~~~~~~~~~~~ - Fix issues when building re2c as a CMake subproject (`#302 <https://github.com/skvadrik/re2c/pull/302>`_: - Final corrections in the SIMPA article "RE2C: A lexer generator based on lookahead-TDFA", https://doi.org/10.1016/j.simpa.2020.100027 2.0.2 (2020-08-08) ~~~~~~~~~~~~~~~~~~ - Enable re2go building by default. - Package CMake files into release tarball. 2.0.1 (2020-07-29) ~~~~~~~~~~~~~~~~~~ - Updated version for CMake build system (forgotten in release 2.0). - Added a short article about re2c for the Software Impacts journal. 2.0 (2020-07-20) ~~~~~~~~~~~~~~~~ - Added new code generation backend for Go and a new ``re2go`` program (`#272 <https://github.com/skvadrik/re2c/issues/272>`_: Go support). Added option ``--lang <c | go>``. - Added CMake build system as an alternative to Autotools (`#275 <https://github.com/skvadrik/re2c/pull/275>`_: Add a CMake build system (thanks to ligfx), `#244 <https://github.com/skvadrik/re2c/issues/244>`_: Switching to CMake). - Changes in generic API: + Removed primitives ``YYSTAGPD`` and ``YYMTAGPD``. + Added primitives ``YYSHIFT``, ``YYSHIFTSTAG``, ``YYSHIFTMTAG`` that allow to express fixed tags in terms of generic API. + Added configurations ``re2c:api:style`` and ``re2c:api:sigil``. + Added named placeholders in interpolated configuration strings. - Changes in reuse mode (``-r, --reuse`` option): + Do not reset API-related configurations in each `use:re2c` block (`#291 <https://github.com/skvadrik/re2c/issues/291>`_: Defines in rules block are not propagated to use blocks). + Use block-local options instead of last block options. + Do not accumulate options from rules/reuse blocks in whole-program options. + Generate non-overlapping YYFILL labels for reuse blocks. + Generate start label for each reuse block in storable state mode. - Changes in start-conditions mode (``-c, --start-conditions`` option): + Allow to use normal (non-conditional) blocks in `-c` mode (`#263 <https://github.com/skvadrik/re2c/issues/263>`_: allow mixing conditional and non-conditional blocks with -c, `#296 <https://github.com/skvadrik/re2c/issues/296>`_: Conditions required for all lexers when using '-c' option). + Generate condition switch in every re2c block (`#295 <https://github.com/skvadrik/re2c/issues/295>`_: Condition switch generated for only one lexer per file). - Changes in the generated labels: + Use ``yyeof`` label prefix instead of ``yyeofrule``. + Use ``yyfill`` label prefix instead of ``yyFillLabel``. + Decouple start label and initial label (affects label numbering). - Removed undocumented configuration ``re2c:flags:o``, ``re2c:flags:output``. - Changes in ``re2c:flags:t``, ``re2c:flags:type-header`` configuration: filename is now relative to the output file directory. - Added option ``--case-ranges`` and configuration ``re2c:flags:case-ranges``. - Extended fixed tags optimization for the case of fixed-counter repetition. - Fixed bugs related to EOF rule: + `#276 <https://github.com/skvadrik/re2c/issues/276>`_: Example 01_fill.re in docs is broken + `#280 <https://github.com/skvadrik/re2c/issues/280>`_: EOF rules with multiple blocks + `#284 <https://github.com/skvadrik/re2c/issues/284>`_: mismatched YYBACKUP and YYRESTORE (Add missing fallback states with EOF rule) - Fixed miscellaneous bugs: + `#286 <https://github.com/skvadrik/re2c/issues/286>`_: Incorrect submatch values with fixed-length trailing context. + `#297 <https://github.com/skvadrik/re2c/issues/297>`_: configure error on ubuntu 18.04 / cmake 3.10 - Changed bootstrap process (require explicit configuration flags and a path to re2c executable to regenerate the lexers). - Added internal options ``--posix-prectable <naive | complex>``. - Added debug option ``--dump-dfa-tree``. - Major revision of the paper "Efficient POSIX submatch extraction on NFA". ---- 1.3x ---- 1.3 (2019-12-14) ~~~~~~~~~~~~~~~~ - Added option: ``--stadfa``. - Added warning: ``-Wsentinel-in-midrule``. - Added generic API primitives: + ``YYSTAGPD`` + ``YYMTAGPD`` - Added configurations: + ``re2c:sentinel = 0;`` + ``re2c:define:YYSTAGPD = "YYSTAGPD";`` + ``re2c:define:YYMTAGPD = "YYMTAGPD";`` - Worked on reproducible builds (`#258 <https://github.com/skvadrik/re2c/pull/258>`_: Make the build reproducible). ---- 1.2x ---- 1.2.1 (2019-08-11) ~~~~~~~~~~~~~~~~~~ - Fixed bug `#253 <https://github.com/skvadrik/re2c/issues/253>`_: re2c should install unicode_categories.re somewhere. - Fixed bug `#254 <https://github.com/skvadrik/re2c/issues/254>`_: Turn off re2c:eof = 0. 1.2 (2019-08-02) ~~~~~~~~~~~~~~~~ - Added EOF rule ``$`` and configuration ``re2c:eof``. - Added ``/*!include:re2c ... */`` directive and ``-I`` option. - Added ``/*!header:re2c:on*/`` and ``/*!header:re2c:off*/`` directives. - Added ``--input-encoding <ascii | utf8>`` option. + `#237 <https://github.com/skvadrik/re2c/issues/237>`_: Handle non-ASCII encoded characters in regular expressions + `#250 <https://github.com/skvadrik/re2c/issues/250>`_ UTF8 enoding - Added include file with a list of definitions for Unicode character classes. + `#235 <https://github.com/skvadrik/re2c/issues/235>`_: Unicode character classes - Added ``--location-format <gnu | msvc>`` option. + `#195 <https://github.com/skvadrik/re2c/issues/195>`_: Please consider using Gnu format for error messages - Added ``--verbose`` option that prints "success" message if re2c exits without errors. - Added configurations for options: + ``-o --output`` (specify output file) + ``-t --type-header`` (specify header file) - Removed configurations for internal/debug options. - Extended ``-r`` option: allow to mix multiple ``/*!rules:re2c*/``, ``/*!use:re2c*/`` and ``/*!re2c*/`` blocks. + `#55 <https://github.com/skvadrik/re2c/issues/55>`_: allow standard re2c blocks in reuse mode - Fixed ``-F --flex-support`` option: parsing and operator precedence. + `#229 <https://github.com/skvadrik/re2c/issues/229>`_: re2c option -F (flex syntax) broken + `#242 <https://github.com/skvadrik/re2c/issues/242>`_: Operator precedence with --flex-syntax is broken - Changed difference operator ``/`` to apply before encoding expansion of operands. + `#236 <https://github.com/skvadrik/re2c/issues/236>`_: Support range difference with variable-length encodings - Changed output generation of output file to be atomic. + `#245 <https://github.com/skvadrik/re2c/issues/245>`_: re2c output is not atomic - Authored research paper "Efficient POSIX Submatch Extraction on NFA" together with Dr Angelo Borsotti. - Added experimental libre2c library (``--enable-libs`` configure option) with the following algorithms: + TDFA with leftmost-greedy disambiguation + TDFA with POSIX disambiguation (Okui-Suzuki algorithm) + TNFA with leftmost-greedy disambiguation + TNFA with POSIX disambiguation (Okui-Suzuki algorithm) + TNFA with lazy POSIX disambiguation (Okui-Suzuki algorithm) + TNFA with POSIX disambiguation (Kuklewicz algorithm) + TNFA with POSIX disambiguation (Cox algorithm) - Added debug subsystem (``--enable-debug`` configure option) and new debug options: + ``-dump-cfg`` (dump control flow graph of tag variables) + ``-dump-interf`` (dump interference table of tag variables) + ``-dump-closure-stats`` (dump epsilon-closure statistics) - Added internal options: + ``--posix-closure <gor1 | gtop>`` (switch between shortest-path algorithms used for the construction of POSIX closure) - Fixed a number of crashes found by American Fuzzy Lop fuzzer: + `#226 <https://github.com/skvadrik/re2c/issues/226>`_, `#227 <https://github.com/skvadrik/re2c/issues/227>`_, `#228 <https://github.com/skvadrik/re2c/issues/228>`_, `#231 <https://github.com/skvadrik/re2c/issues/231>`_, `#232 <https://github.com/skvadrik/re2c/issues/232>`_, `#233 <https://github.com/skvadrik/re2c/issues/233>`_, `#234 <https://github.com/skvadrik/re2c/issues/234>`_, `#238 <https://github.com/skvadrik/re2c/issues/238>`_ - Fixed handling of newlines: + correctly parse multi-character newlines CR LF in ``#line`` directives + consistently convert all newlines in the generated file to Unix-style LF - Changed default tarball format from .gz to .xz. + `#221 <https://github.com/skvadrik/re2c/issues/221>`_: big source tarball - Fixed a number of other bugs and resolved issues: + `#2 <https://github.com/skvadrik/re2c/issues/2>`_: abort + `#6 <https://github.com/skvadrik/re2c/issues/6>`_: segfault + `#10 <https://github.com/skvadrik/re2c/issues/10>`_: lessons/002_upn_calculator/calc_002 doesn't produce a useful example program + `#44 <https://github.com/skvadrik/re2c/issues/44>`_: Access violation when translating the attached file + `#49 <https://github.com/skvadrik/re2c/issues/49>`_: wildcard state \000 rules makes lexer behave weard + `#98 <https://github.com/skvadrik/re2c/issues/98>`_: Transparent handling of #line directives in input files + `#104 <https://github.com/skvadrik/re2c/issues/104>`_: Improve const-correctness + `#105 <https://github.com/skvadrik/re2c/issues/105>`_: Conversion of pointer parameters into references + `#114 <https://github.com/skvadrik/re2c/issues/114>`_: Possibility of fixing bug 2535084 + `#120 <https://github.com/skvadrik/re2c/issues/120>`_: condition consisting of default rule only is ignored + `#167 <https://github.com/skvadrik/re2c/issues/167>`_: Add word boundary support + `#168 <https://github.com/skvadrik/re2c/issues/168>`_: Wikipedia's article on re2c + `#180 <https://github.com/skvadrik/re2c/issues/180>`_: Comment syntax? + `#182 <https://github.com/skvadrik/re2c/issues/182>`_: yych being set by YYPEEK () and then not used + `#196 <https://github.com/skvadrik/re2c/issues/196>`_: Implicit type conversion warnings + `#198 <https://github.com/skvadrik/re2c/issues/198>`_: no match for ‘operator!=’ in ‘i != std::vector<_Tp, _Alloc>::rend() [with _Tp = re2c::bitmap_t, _Alloc = std::allocator<re2c::bitmap_t>]()’ + `#210 <https://github.com/skvadrik/re2c/issues/210>`_: How to build re2c in windows? + `#215 <https://github.com/skvadrik/re2c/issues/215>`_: A memory read overrun issue in s_to_n32_unsafe.cc + `#220 <https://github.com/skvadrik/re2c/issues/220>`_: src/dfa/dfa.h: simplify constructor to avoid g++-3.4 bug + `#223 <https://github.com/skvadrik/re2c/issues/223>`_: Fix typo + `#224 <https://github.com/skvadrik/re2c/issues/224>`_: src/dfa/closure_posix.cc: pack() tweaks + `#225 <https://github.com/skvadrik/re2c/issues/225>`_: Documentation link is broken in libre2c/README + `#230 <https://github.com/skvadrik/re2c/issues/230>`_: Changes for upcoming Travis' infra migration + `#239 <https://github.com/skvadrik/re2c/issues/239>`_: Push model example has wrong re2c invocation, breaks guide + `#241 <https://github.com/skvadrik/re2c/issues/241>`_: Guidance on how to use re2c for full-duplex command & response protocol + `#243 <https://github.com/skvadrik/re2c/issues/243>`_: A code generated for period (.) requires 4 bytes + `#246 <https://github.com/skvadrik/re2c/issues/246>`_: Please add a license to this repo + `#247 <https://github.com/skvadrik/re2c/issues/247>`_: Build failure on current Cygwin, probably caused by force-fed c++98 mode + `#248 <https://github.com/skvadrik/re2c/issues/248>`_: distcheck still looks for README + `#251 <https://github.com/skvadrik/re2c/issues/251>`_: Including what you use is find, but not without inclusion guards - Updated documentation and website.
Attempt at #244
Built successfully on macOS, and on Windows with Visual Studio. Works great for embedding in other projects that use re2c to generate lexers!
I tried to copy the autotools build system as much as possible, including writing CMake functions to mimic autoconf, and using CMake script mode to do the bootstrap vs. use-built-re2c cycle.