Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using CMake add_subdirectory fails #302

Closed
hanak opened this issue Aug 19, 2020 · 9 comments
Closed

Using CMake add_subdirectory fails #302

hanak opened this issue Aug 19, 2020 · 9 comments

Comments

@hanak
Copy link

hanak commented Aug 19, 2020

Before I get to the description, I appreciate the fact that you have added support for CMake. It helped me with the multi-platform environment a lot. Thank you for doing so. Yet, let us get back to the behaviour I observed.

When I try to use add_subdirectory("<path to re2c source code outside source tree>" "<path to binary dir>"), CMake 3.13.x (the version I have to use) fails with:

CMake Error at external/re2c-2.0.2/CMakeLists.txt:127 (add_library):
  Cannot find source file:

    <binary dir on my machine>/src/msg/help.cc

  Tried extensions .c .C .c++ .cc .cpp .cxx .cu .m .M .mm .h .hh .h++ .hm
  .hpp .hxx .in .txx


CMake Error at external/re2c-2.0.2/CMakeLists.txt:127 (add_library):
  No SOURCES given to target: re2c_objects_autogen

I noticed that re2c/CMakeLists.txt contains at the line 234 following:
if (CMAKE_SOURCE_DIR STREQUAL PROJECT_SOURCE_DIR)

When I disable this condition (e.g. if (CMAKE_SOURCE_DIR STREQUAL PROJECT_SOURCE_DIR)), build works. Why is the condition there?

Also, I have noticed that the line 311 excludes non-UNIX machines from installation through if(UNIX). This means, if I want to deploy the build output somewhere using Windows machine I have to do it on my own while on Linux (and cross-compiling for Windows on Linux), re2c's CMakeLists.txt will install it. It this intentional? I am sorry for adding another issue into this one. I can separate it if there is no straight answer.

Context:
I am trying to build re2c as a prerequisite of another project. The build has to run automatically on a build machine in order to allow CI/CD pipeline and it should run on a developer's machine, too. Developer's machine might be either Linux or Windows.

@skvadrik
Copy link
Owner

skvadrik commented Aug 19, 2020

Hi @hanak ! think both if (CMAKE_SOURCE_DIR STREQUAL PROJECT_SOURCE_DIR) and if(UNIX) conditions were added by @ligfx in the initial pull request. Perhaps @ligfx can explain why they are needed (if they are indeed needed). I think @ligfx intended to make it possible to build re2c as a subproject.

Meanwhile, can you provide a minimal example of your top-level CMakeLists.txt in which you try to add re2c as a subproject?

@ligfx
Copy link
Contributor

ligfx commented Aug 19, 2020

Looks like the logic to generate help.cc was accidentally moved inside the if (CMAKE_SOURCE_DIR STREQUAL PROJECT_SOURCE_DIR) block in be688fd , it seems that rule should be at top level instead.

Non-UNIX machines are excluded from installation because there's no standard installation path— CMake makes you explicitly specify one. What installation path would you use for a command-line tool? I'm not sure what you're trying to do, but you shouldn't need to install re2c in order to use it in your toplevel build.

@skvadrik
Copy link
Owner

skvadrik commented Aug 19, 2020

Thanks @ligfx for a very fast reply and for pointing at the faulty commit! I'll upload a fix for it shortly. Am I right in assuming that you added the if (CMAKE_SOURCE_DIR STREQUAL PROJECT_SOURCE_DIR) condition to disable test and install targets when re2c is built as a sub-project of another project?

@hanak I use the following CMakeLists.txt to experiment:

cmake_minimum_required(VERSION 3.13)
project(other)
add_subdirectory(../re2c re2c-build)

And the following directory layout:

├── other
│   ├── build
│   └── CMakeLists.txt
└── re2c

@ligfx
Copy link
Contributor

ligfx commented Aug 19, 2020

Am I right in assuming that you added the if (CMAKE_SOURCE_DIR STREQUAL PROJECT_SOURCE_DIR) condition to disable test and install targets when re2c is built as a sub-project of another project?

Yep, exactly

skvadrik added a commit that referenced this issue Aug 19, 2020
These rules are always needed (e.g. the file help.cc generated by these
commands is one of the source files that are required for the minimal
build). The rules were accidentally moved under conditional branch in
commit be688fd, and were enabled only
if re2c is the root project. This caused a build error "Cannot find
source file" when trying to build re2c as a subproject:

  ├── other
  │   ├── build
  │   └── CMakeLists.txt
  └── re2c

With the following other/CMakeLists.txt:

  project(other)
  add_subdirectory(../re2c re2c-build)

This is a partial fix for #302 "Using CMake add_subdirectory fails".
@skvadrik
Copy link
Owner

skvadrik commented Aug 19, 2020

@hanak The above example builds after commit e626849, so the first problem should be solved. What remains is to sort out install rule. Please refer to the comment by ligfx:

Non-UNIX machines are excluded from installation because there's no standard installation path— CMake makes you explicitly specify one. What installation path would you use for a command-line tool? I'm not sure what you're trying to do, but you shouldn't need to install re2c in order to use it in your toplevel build.

I added a small example of building re2c as a subproject: c1b246a. In this example, if you run build.sh it will build re2c and put the binary in .build/re2c-build/re2c. At the build stage you can refer to the re2c binary as $<TARGET_FILE:re2c>.

@hanak
Copy link
Author

hanak commented Aug 20, 2020

Thank you for all responses and thank you for the quick update. The example works (Windows machine). And, I tried it within the project and it built successfully, too. Good work. Is there going to be a new release?

Regarding the installation. @ligfx commented about uncertain file structure on non-UNIX systems. And, I agree. As he has suggested, I am using the re2c from the build. I might add my thoughts about it in the following paragraph but my question was motivated by curiosity. I am thankful there is a working CMakeList script in re2c.

My question around installation was motivated by my observation that running cmake --build .build --target install works in Linux but fails on Windows/NMAKE Makefiles/CMake 3.13 (error: don't know how to make 'install'). In my case I use the target "install" to package the build output on the build machine. Also, a Linux machine will add re2c binaries to ${CMAKE_INSTALL_PREFIX} which does not happen on Windows. Still, this does not break anything.

@ligfx asked earlier about potential location. What I would do is to ask whether the installed files can be mapped to the file types considered by CMake (ARCHIVE, RUNTIME, LIBRARY, PUBLIC_HEADER, ...). If so, I would use related rules. If not and assuming the files can be used (by re2c?) when CMAKE_INSTALL_PREFIX changes, I would assume UNIX file structure allowing CMAKE_INSTALL_PREFIX to redirect the "root" of the structure. Yet, these are just my thoughts.

If you feel all is right, this ticket can be closed I think. I am not sure whether I should close it (as the creator of the ticket) or your should close it (as maintainers of the repository).

@skvadrik
Copy link
Owner

skvadrik commented Aug 20, 2020

My question around installation ...

Reading about CMAKE_INSTALL_PREFIX used by install() command for CMake-3.12 (the minimal version required by re2c), it says that there is a default path c:/Program Files/${PROJECT_NAME} on Windows as well: https://cmake.org/cmake/help/v3.12/variable/CMAKE_INSTALL_PREFIX.html. Is it not suitable for some reason?

If the default path cannot be used, one possible approach is to have a variable RE2C_INSTALL_PREFIX that, if defined, overrides CMAKE_INSTALL_PREFIX. And change the condition if (UNIX) to if (UNIX or RE2C_INSTALL_PREFIX). Then, if one wants to enable install target on Windows, they should define RE2C_INSTALL_PREFIX explicitly.

What I would do is to ask whether the installed files can be mapped to the file types considered by CMake ...

But they are already mapped, e.g. install(TARGETS re2c RUNTIME DESTINATION bin) is mapped to RUNTIME.

Is there going to be a new release?

I can make a minor bugfix release 2.0.3 in a few days.

If you feel all is right, this ticket can be closed I think. I am not sure whether I should close it (as the creator of the ticket) or your should close it (as maintainers of the repository).

No worries, I can close it later. I usually leave the bugs open for a while in case people want to follow up, and before the next release I make a raid and close them all. The size of the project allows for this lazy workflow. Thank you for reporting the problem.

@hanak
Copy link
Author

hanak commented Aug 21, 2020

My question around installation ...
Reading about CMAKE_INSTALL_PREFIX used by install()... Is it (the default) not suitable for some reason?

I am not sure whether I understand the question or whether this is a question for me. Thus, my response might just miss the spot. I am sorry for this. Still, C:\Program Files is area which is protected by the OS to some extent (I tried to find a reference on MSDN but I was not successful. Thus, I am using my past experience here.). I think a regular user will not be allowed to make a change there. So, a developer (which is a regular user) might want to redirect installation somewhere else especially if that installation is meant to be used as a support tool/library.

@skvadrik
Copy link
Owner

skvadrik commented Aug 22, 2020

I am not sure whether I understand the question or whether this is a question for me. Thus, my response might just miss the spot.

You are always welcome to participate! By the way, I released re2c-2.0.3 with the fix.

C:\Program Files is area which is protected by the OS

I see. But it's not much different on Unix systems, where the default is /usr/local. Regular users do not have write permissions, one has to be root. Anyways, let's close this bug since it works for you. If someone wants to enable install rules on Windows, they can open a new bug.

netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this issue Sep 20, 2020
2.0.3 (2020-08-22)
~~~~~~~~~~~~~~~~~~

- Fix issues when building re2c as a CMake subproject
  (`#302 <https://github.com/skvadrik/re2c/pull/302>`_:

- Final corrections in the SIMPA article "RE2C: A lexer generator based on
  lookahead-TDFA", https://doi.org/10.1016/j.simpa.2020.100027

2.0.2 (2020-08-08)
~~~~~~~~~~~~~~~~~~

- Enable re2go building by default.

- Package CMake files into release tarball.

2.0.1 (2020-07-29)
~~~~~~~~~~~~~~~~~~

- Updated version for CMake build system (forgotten in release 2.0).

- Added a short article about re2c for the Software Impacts journal.

2.0 (2020-07-20)
~~~~~~~~~~~~~~~~

- Added new code generation backend for Go and a new ``re2go`` program
  (`#272 <https://github.com/skvadrik/re2c/issues/272>`_: Go support).
  Added option ``--lang <c | go>``.

- Added CMake build system as an alternative to Autotools
  (`#275 <https://github.com/skvadrik/re2c/pull/275>`_:
  Add a CMake build system (thanks to ligfx),
  `#244 <https://github.com/skvadrik/re2c/issues/244>`_: Switching to CMake).

- Changes in generic API:

  + Removed primitives ``YYSTAGPD`` and ``YYMTAGPD``.
  + Added primitives ``YYSHIFT``, ``YYSHIFTSTAG``, ``YYSHIFTMTAG``
    that allow to express fixed tags in terms of generic API.
  + Added configurations ``re2c:api:style`` and ``re2c:api:sigil``.
  + Added named placeholders in interpolated configuration strings.

- Changes in reuse mode (``-r, --reuse`` option):

  + Do not reset API-related configurations in each `use:re2c` block
    (`#291 <https://github.com/skvadrik/re2c/issues/291>`_:
    Defines in rules block are not propagated to use blocks).
  + Use block-local options instead of last block options.
  + Do not accumulate options from rules/reuse blocks in whole-program options.
  + Generate non-overlapping YYFILL labels for reuse blocks.
  + Generate start label for each reuse block in storable state mode.

- Changes in start-conditions mode (``-c, --start-conditions`` option):

  + Allow to use normal (non-conditional) blocks in `-c` mode
    (`#263 <https://github.com/skvadrik/re2c/issues/263>`_:
    allow mixing conditional and non-conditional blocks with -c,
    `#296 <https://github.com/skvadrik/re2c/issues/296>`_:
    Conditions required for all lexers when using '-c' option).
  + Generate condition switch in every re2c block
    (`#295 <https://github.com/skvadrik/re2c/issues/295>`_:
    Condition switch generated for only one lexer per file).

- Changes in the generated labels:

  + Use ``yyeof`` label prefix instead of ``yyeofrule``.
  + Use ``yyfill`` label prefix instead of ``yyFillLabel``.
  + Decouple start label and initial label (affects label numbering).

- Removed undocumented configuration ``re2c🎏o``, ``re2c🎏output``.

- Changes in ``re2c🎏t``, ``re2c🎏type-header`` configuration:
  filename is now relative to the output file directory.

- Added option ``--case-ranges`` and configuration ``re2c🎏case-ranges``.

- Extended fixed tags optimization for the case of fixed-counter repetition.

- Fixed bugs related to EOF rule:

  + `#276 <https://github.com/skvadrik/re2c/issues/276>`_:
    Example 01_fill.re in docs is broken
  + `#280 <https://github.com/skvadrik/re2c/issues/280>`_:
    EOF rules with multiple blocks
  + `#284 <https://github.com/skvadrik/re2c/issues/284>`_:
    mismatched YYBACKUP and YYRESTORE
    (Add missing fallback states with EOF rule)

- Fixed miscellaneous bugs:

  + `#286 <https://github.com/skvadrik/re2c/issues/286>`_:
    Incorrect submatch values with fixed-length trailing context.
  + `#297 <https://github.com/skvadrik/re2c/issues/297>`_:
    configure error on ubuntu 18.04 / cmake 3.10

- Changed bootstrap process (require explicit configuration flags and a path to
  re2c executable to regenerate the lexers).

- Added internal options ``--posix-prectable <naive | complex>``.

- Added debug option ``--dump-dfa-tree``.

- Major revision of the paper "Efficient POSIX submatch extraction on NFA".

----
1.3x
----

1.3 (2019-12-14)
~~~~~~~~~~~~~~~~

- Added option: ``--stadfa``.

- Added warning: ``-Wsentinel-in-midrule``.

- Added generic API primitives:

  + ``YYSTAGPD``
  + ``YYMTAGPD``

- Added configurations:

  + ``re2c:sentinel = 0;``
  + ``re2c:define:YYSTAGPD = "YYSTAGPD";``
  + ``re2c:define:YYMTAGPD = "YYMTAGPD";``

- Worked on reproducible builds
  (`#258 <https://github.com/skvadrik/re2c/pull/258>`_:
  Make the build reproducible).

----
1.2x
----

1.2.1 (2019-08-11)
~~~~~~~~~~~~~~~~~~

- Fixed bug `#253 <https://github.com/skvadrik/re2c/issues/253>`_:
  re2c should install unicode_categories.re somewhere.

- Fixed bug `#254 <https://github.com/skvadrik/re2c/issues/254>`_:
  Turn off re2c:eof = 0.

1.2 (2019-08-02)
~~~~~~~~~~~~~~~~

- Added EOF rule ``$`` and configuration ``re2c:eof``.

- Added ``/*!include:re2c ... */`` directive and ``-I`` option.

- Added ``/*!header:re2c:on*/`` and ``/*!header:re2c:off*/`` directives.

- Added ``--input-encoding <ascii | utf8>`` option.

  + `#237 <https://github.com/skvadrik/re2c/issues/237>`_:
    Handle non-ASCII encoded characters in regular expressions
  + `#250 <https://github.com/skvadrik/re2c/issues/250>`_
    UTF8 enoding

- Added include file with a list of definitions for Unicode character classes.

  + `#235 <https://github.com/skvadrik/re2c/issues/235>`_:
    Unicode character classes

- Added ``--location-format <gnu | msvc>`` option.

  + `#195 <https://github.com/skvadrik/re2c/issues/195>`_:
    Please consider using Gnu format for error messages

- Added ``--verbose`` option that prints "success" message if re2c exits
  without errors.

- Added configurations for options:

  + ``-o --output`` (specify output file)
  + ``-t --type-header`` (specify header file)

- Removed configurations for internal/debug options.

- Extended ``-r`` option: allow to mix multiple ``/*!rules:re2c*/``,
  ``/*!use:re2c*/`` and ``/*!re2c*/`` blocks.

  + `#55 <https://github.com/skvadrik/re2c/issues/55>`_:
    allow standard re2c blocks in reuse mode

- Fixed ``-F --flex-support`` option: parsing and operator precedence.

  + `#229 <https://github.com/skvadrik/re2c/issues/229>`_:
    re2c option -F (flex syntax) broken
  + `#242 <https://github.com/skvadrik/re2c/issues/242>`_:
    Operator precedence with --flex-syntax is broken

- Changed difference operator ``/`` to apply before encoding expansion of
  operands.

  + `#236 <https://github.com/skvadrik/re2c/issues/236>`_:
    Support range difference with variable-length encodings

- Changed output generation of output file to be atomic.

  + `#245 <https://github.com/skvadrik/re2c/issues/245>`_:
    re2c output is not atomic

- Authored research paper "Efficient POSIX Submatch Extraction on NFA"
  together with Dr Angelo Borsotti.

- Added experimental libre2c library (``--enable-libs`` configure option) with
  the following algorithms:

  + TDFA with leftmost-greedy disambiguation
  + TDFA with POSIX disambiguation (Okui-Suzuki algorithm)
  + TNFA with leftmost-greedy disambiguation
  + TNFA with POSIX disambiguation (Okui-Suzuki algorithm)
  + TNFA with lazy POSIX disambiguation (Okui-Suzuki algorithm)
  + TNFA with POSIX disambiguation (Kuklewicz algorithm)
  + TNFA with POSIX disambiguation (Cox algorithm)

- Added debug subsystem (``--enable-debug`` configure option) and new debug
  options:

  + ``-dump-cfg`` (dump control flow graph of tag variables)
  + ``-dump-interf`` (dump interference table of tag variables)
  + ``-dump-closure-stats`` (dump epsilon-closure statistics)

- Added internal options:

  + ``--posix-closure <gor1 | gtop>`` (switch between shortest-path algorithms
    used for the construction of POSIX closure)

- Fixed a number of crashes found by American Fuzzy Lop fuzzer:

  + `#226 <https://github.com/skvadrik/re2c/issues/226>`_,
    `#227 <https://github.com/skvadrik/re2c/issues/227>`_,
    `#228 <https://github.com/skvadrik/re2c/issues/228>`_,
    `#231 <https://github.com/skvadrik/re2c/issues/231>`_,
    `#232 <https://github.com/skvadrik/re2c/issues/232>`_,
    `#233 <https://github.com/skvadrik/re2c/issues/233>`_,
    `#234 <https://github.com/skvadrik/re2c/issues/234>`_,
    `#238 <https://github.com/skvadrik/re2c/issues/238>`_

- Fixed handling of newlines:

  + correctly parse multi-character newlines CR LF in ``#line`` directives
  + consistently convert all newlines in the generated file to Unix-style LF

- Changed default tarball format from .gz to .xz.

  + `#221 <https://github.com/skvadrik/re2c/issues/221>`_:
    big source tarball

- Fixed a number of other bugs and resolved issues:

  + `#2 <https://github.com/skvadrik/re2c/issues/2>`_: abort
  + `#6 <https://github.com/skvadrik/re2c/issues/6>`_: segfault
  + `#10 <https://github.com/skvadrik/re2c/issues/10>`_:
    lessons/002_upn_calculator/calc_002 doesn't produce a useful example program
  + `#44 <https://github.com/skvadrik/re2c/issues/44>`_:
    Access violation when translating the attached file
  + `#49 <https://github.com/skvadrik/re2c/issues/49>`_:
    wildcard state \000 rules makes lexer behave weard
  + `#98 <https://github.com/skvadrik/re2c/issues/98>`_:
    Transparent handling of #line directives in input files
  + `#104 <https://github.com/skvadrik/re2c/issues/104>`_:
    Improve const-correctness
  + `#105 <https://github.com/skvadrik/re2c/issues/105>`_:
    Conversion of pointer parameters into references
  + `#114 <https://github.com/skvadrik/re2c/issues/114>`_:
    Possibility of fixing bug 2535084
  + `#120 <https://github.com/skvadrik/re2c/issues/120>`_:
    condition consisting of default rule only is ignored
  + `#167 <https://github.com/skvadrik/re2c/issues/167>`_:
    Add word boundary support
  + `#168 <https://github.com/skvadrik/re2c/issues/168>`_:
    Wikipedia's article on re2c
  + `#180 <https://github.com/skvadrik/re2c/issues/180>`_:
    Comment syntax?
  + `#182 <https://github.com/skvadrik/re2c/issues/182>`_:
    yych being set by YYPEEK () and then not used
  + `#196 <https://github.com/skvadrik/re2c/issues/196>`_:
    Implicit type conversion warnings
  + `#198 <https://github.com/skvadrik/re2c/issues/198>`_:
    no match for ‘operator!=’ in ‘i != std::vector<_Tp, _Alloc>::rend() [with _Tp = re2c::bitmap_t, _Alloc = std::allocator<re2c::bitmap_t>]()’
  + `#210 <https://github.com/skvadrik/re2c/issues/210>`_:
    How to build re2c in windows?
  + `#215 <https://github.com/skvadrik/re2c/issues/215>`_:
    A memory read overrun issue in s_to_n32_unsafe.cc
  + `#220 <https://github.com/skvadrik/re2c/issues/220>`_:
    src/dfa/dfa.h: simplify constructor to avoid g++-3.4 bug
  + `#223 <https://github.com/skvadrik/re2c/issues/223>`_:
    Fix typo
  + `#224 <https://github.com/skvadrik/re2c/issues/224>`_:
    src/dfa/closure_posix.cc: pack() tweaks
  + `#225 <https://github.com/skvadrik/re2c/issues/225>`_:
    Documentation link is broken in libre2c/README
  + `#230 <https://github.com/skvadrik/re2c/issues/230>`_:
    Changes for upcoming Travis' infra migration
  + `#239 <https://github.com/skvadrik/re2c/issues/239>`_:
    Push model example has wrong re2c invocation, breaks guide
  + `#241 <https://github.com/skvadrik/re2c/issues/241>`_:
    Guidance on how to use re2c for full-duplex command & response protocol
  + `#243 <https://github.com/skvadrik/re2c/issues/243>`_:
    A code generated for period (.) requires 4 bytes
  + `#246 <https://github.com/skvadrik/re2c/issues/246>`_:
    Please add a license to this repo
  + `#247 <https://github.com/skvadrik/re2c/issues/247>`_:
    Build failure on current Cygwin, probably caused by force-fed c++98 mode
  + `#248 <https://github.com/skvadrik/re2c/issues/248>`_:
    distcheck still looks for README
  + `#251 <https://github.com/skvadrik/re2c/issues/251>`_:
    Including what you use is find, but not without inclusion guards

- Updated documentation and website.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants