Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to build re2c in windows? #210

Closed
chjhaijiang opened this issue Jul 4, 2018 · 15 comments
Closed

How to build re2c in windows? #210

chjhaijiang opened this issue Jul 4, 2018 · 15 comments

Comments

@chjhaijiang
Copy link

@chjhaijiang chjhaijiang commented Jul 4, 2018

Do we have any specific request for GCC version to build Re2c on Windows?

Follow the steps:
6. Build for windows using mingw:
$ ../configure --host i686-w64-mingw32 [--prefix=]
$ make

I got below errors:
c:\hchen\mingw\lib\gcc\mingw32\6.3.0\include\c++\cwchar:164:11: error: '::vfwscanf' has not been declared
using ::vfwscanf;
^~~~~~~~
c:\hchen\mingw\lib\gcc\mingw32\6.3.0\include\c++\cwchar:170:11: error: '::vswscanf' has not been declared
using ::vswscanf;
^~~~~~~~
c:\hchen\mingw\lib\gcc\mingw32\6.3.0\include\c++\cwchar:174:11: error: '::vwscanf' has not been declared
using ::vwscanf;
^~~~~~~
c:\hchen\mingw\lib\gcc\mingw32\6.3.0\include\c++\cwchar:191:11: error: '::wcstof' has not been declared
using ::wcstof;
^~~~~~
In file included from c:\hchen\mingw\lib\gcc\mingw32\6.3.0\include\c++\bits\locale_facets.h:39:0,
from c:\hchen\mingw\lib\gcc\mingw32\6.3.0\include\c++\bits\basic_ios.h:37,
from c:\hchen\mingw\lib\gcc\mingw32\6.3.0\include\c++\ios:44,
from c:\hchen\mingw\lib\gcc\mingw32\6.3.0\include\c++\ostream:38,
from c:\hchen\mingw\lib\gcc\mingw32\6.3.0\include\c++\iostream:39,
from ../src/code/go.h:4,
from ../src/code/bitmap.cc:5:
c:\hchen\mingw\lib\gcc\mingw32\6.3.0\include\c++\cwctype:89:11: error: '::iswblank' has not been declared
using ::iswblank;
^~~~~~~~

@sergeyklay
Copy link
Collaborator

@sergeyklay sergeyklay commented Jul 4, 2018

This can be related to the compiler configuration issues. I'm not pretty sure but it's probably an issue related to the older MinGW compiler. Just for test try to use mingw-w64.

P.S. As you can see it's not platform specific issue sass/libsass#1894

Loading

@skvadrik
Copy link
Owner

@skvadrik skvadrik commented Jul 4, 2018

@chjhaijiang, are you able to build a minimal hello-world C++ program with you MinGW toolchain?

Loading

@chjhaijiang
Copy link
Author

@chjhaijiang chjhaijiang commented Jul 5, 2018

@skvadrik, Yes. The simple hello-world example below can be run.

#include <iostream>
int main()
{
  std::cout<<"hello, world!"<<std::endl;  
  return 0;
}

Loading

@chjhaijiang
Copy link
Author

@chjhaijiang chjhaijiang commented Jul 6, 2018

Alternatively, Do we have a 1.0.3 release binary for windows 64-bit to use?

Loading

@skvadrik
Copy link
Owner

@skvadrik skvadrik commented Jul 6, 2018

I've uploaded a windows binary built on linux (built with MinGW version 7.3.0): https://github.com/skvadrik/re2c/releases/download/1.0.3/re2c-1.0.3.exe.zip.

I'm also trying to build in a windows VM with MinGW to see if I can reproduce the failure, but so far I have problems setting up a working environment.

Loading

@chjhaijiang
Copy link
Author

@chjhaijiang chjhaijiang commented Jul 9, 2018

@skvadrik, Build re2c.exe with cygwin64 seems worked for me.
But if I build with ../configure --host i686-w64-mingw32, then If I run re2c.exe, It pop up a error that libgcc_s_sjlj-1.dll is missed.

Is it because re2c is dynamic linked with other dlls? Do we have a static link configure option to be used with?
Thanks

Loading

@skvadrik
Copy link
Owner

@skvadrik skvadrik commented Jul 9, 2018

See https://stackoverflow.com/a/4703059/815525.
I think you get the error because 1) re2c.exe is dynamically linked with this libgcc_s_sjlj-1.dll, and 2) when you run re2c.exe, the dynamic linker cannot find this library. Item 1 can be fixed with -static-libgcc -static-libstdc++, and item 2 can be fixed by adding the path to the library to PATH environment variable. Fixing either 1 or 2 should fix the problem.

Loading

@skvadrik
Copy link
Owner

@skvadrik skvadrik commented Jul 9, 2018

Note that you can add the options to CXXFLAGS as follows:

$ configure CXXFLAGS="-static-libgcc -static-libstdc++" --host i686-w64-mingw32

If you then run make V=1, you'll see that the options are passed to the C++ compiler. Finally, on Linux (and perhaps in Cygwin) you can see all the dynamic dependencies of your re2c.exe executable with ldd. For example, on my system I have the following, when running configure without options:

$ ldd re2c
        linux-vdso.so.1 (0x00007ffcacfb3000)
        libstdc++.so.6 => /usr/lib/gcc/x86_64-pc-linux-gnu/8.1.0/libstdc++.so.6 (0x00007fcf787f6000)
        libm.so.6 => /lib64/libm.so.6 (0x00007fcf784a4000)
        libgcc_s.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/8.1.0/libgcc_s.so.1 (0x00007fcf7828c000)
        libc.so.6 => /lib64/libc.so.6 (0x00007fcf77eca000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fcf78e6b000

And if I configure with -static-libgcc -static-libstdc++, the dependency list shrinks to this:

$ ldd re2c
        linux-vdso.so.1 (0x00007ffdcf1de000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f251f4f4000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f251f132000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f251fc09000)

Loading

@chjhaijiang
Copy link
Author

@chjhaijiang chjhaijiang commented Jul 9, 2018

@skvadrik, Thank you very much. I will get it confirmed and give another comment here.

Loading

@TomUdale
Copy link

@TomUdale TomUdale commented Nov 27, 2018

Greetings, I was wondering if this had been resolved. I am seeing the exact same error messages as the original post while trying to compile re2c v 1.1.1 using brand new mingw (updated Nov 16, 2018) on Windows 7 64 bit.

I agree with @sergeyklay that is very much feels like a compiler configuration issue but I am way out of my depth with mingw/gcc. Any thoughts on what to try? I am happy to provide screenshots of my mingw configuration or whatever else might be helpful.

Loading

@chjhaijiang
Copy link
Author

@chjhaijiang chjhaijiang commented Nov 27, 2018

Yes, this issue can be closed. I switched to use cygwin for building. It was working to me.

Loading

@skvadrik
Copy link
Owner

@skvadrik skvadrik commented Nov 27, 2018

@TomUdale It seems to be a mingw issue (googling shows lots of similar errors in other projects). You can try reporting the bug upstream (former sf issue tracker https://sourceforge.net/p/mingw/bugs/ now points to osdn tracker https://osdn.net/projects/mingw/ticket/). Some suggest using -std=gnu++11 compiler flag as a workaround (RE2C use -std=c++98 by default).

Loading

@TomUdale
Copy link

@TomUdale TomUdale commented Nov 28, 2018

@skvadrik I edited the config.ac file to change the -std setting to gnu++11 as you suggested (not sure if this is the "correct" way - I expect there is some switch to configure that is more appropriate) and it worked in my hands. I did get two integer conversion warnings in parser.cc - they do not look dramatic.

I ran the resulting re2c.exe (both in the build location and in another location) in the mingw environment and a regular cmd prompt and it told me "error no source file" which was true. So it appears that the linking is fine and if there is any dynamic linking going on, it is all handled correctly by default and regardless of path.

I will actually try it with a real input later and see how that goes.

Let me know if you want to see the output of the build warning or anything.

@chjhaijiang thanks for the information on cygwin. I was going there next if mingw failed me.

Loading

@skvadrik
Copy link
Owner

@skvadrik skvadrik commented Nov 28, 2018

@TomUdale Good to know that it worked!

The usual way to override CXXFLAGS defined in configure.ac is to pass CXXFLAGS="-std=gnu++11" to configure.

You can validate the build by running RE2C test suite: make test, but that runs a bash script, so it won't work on windows unless you use cygwin or WSL.

Loading

@skvadrik
Copy link
Owner

@skvadrik skvadrik commented Nov 28, 2018

@TomUdale About the warnings in parser.cc, they come from the code autogenerated by bison, and normally people don't see them because they don't set -Wconversion -Wsign-conversion compiler options. They are relativey harmless, but they can only be fixed in bison (and bison devs are busy doing more useful stuff).

Loading

@skvadrik skvadrik closed this Mar 19, 2019
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this issue Sep 20, 2020
2.0.3 (2020-08-22)
~~~~~~~~~~~~~~~~~~

- Fix issues when building re2c as a CMake subproject
  (`#302 <https://github.com/skvadrik/re2c/pull/302>`_:

- Final corrections in the SIMPA article "RE2C: A lexer generator based on
  lookahead-TDFA", https://doi.org/10.1016/j.simpa.2020.100027

2.0.2 (2020-08-08)
~~~~~~~~~~~~~~~~~~

- Enable re2go building by default.

- Package CMake files into release tarball.

2.0.1 (2020-07-29)
~~~~~~~~~~~~~~~~~~

- Updated version for CMake build system (forgotten in release 2.0).

- Added a short article about re2c for the Software Impacts journal.

2.0 (2020-07-20)
~~~~~~~~~~~~~~~~

- Added new code generation backend for Go and a new ``re2go`` program
  (`#272 <https://github.com/skvadrik/re2c/issues/272>`_: Go support).
  Added option ``--lang <c | go>``.

- Added CMake build system as an alternative to Autotools
  (`#275 <https://github.com/skvadrik/re2c/pull/275>`_:
  Add a CMake build system (thanks to ligfx),
  `#244 <https://github.com/skvadrik/re2c/issues/244>`_: Switching to CMake).

- Changes in generic API:

  + Removed primitives ``YYSTAGPD`` and ``YYMTAGPD``.
  + Added primitives ``YYSHIFT``, ``YYSHIFTSTAG``, ``YYSHIFTMTAG``
    that allow to express fixed tags in terms of generic API.
  + Added configurations ``re2c:api:style`` and ``re2c:api:sigil``.
  + Added named placeholders in interpolated configuration strings.

- Changes in reuse mode (``-r, --reuse`` option):

  + Do not reset API-related configurations in each `use:re2c` block
    (`#291 <https://github.com/skvadrik/re2c/issues/291>`_:
    Defines in rules block are not propagated to use blocks).
  + Use block-local options instead of last block options.
  + Do not accumulate options from rules/reuse blocks in whole-program options.
  + Generate non-overlapping YYFILL labels for reuse blocks.
  + Generate start label for each reuse block in storable state mode.

- Changes in start-conditions mode (``-c, --start-conditions`` option):

  + Allow to use normal (non-conditional) blocks in `-c` mode
    (`#263 <https://github.com/skvadrik/re2c/issues/263>`_:
    allow mixing conditional and non-conditional blocks with -c,
    `#296 <https://github.com/skvadrik/re2c/issues/296>`_:
    Conditions required for all lexers when using '-c' option).
  + Generate condition switch in every re2c block
    (`#295 <https://github.com/skvadrik/re2c/issues/295>`_:
    Condition switch generated for only one lexer per file).

- Changes in the generated labels:

  + Use ``yyeof`` label prefix instead of ``yyeofrule``.
  + Use ``yyfill`` label prefix instead of ``yyFillLabel``.
  + Decouple start label and initial label (affects label numbering).

- Removed undocumented configuration ``re2c🎏o``, ``re2c🎏output``.

- Changes in ``re2c🎏t``, ``re2c🎏type-header`` configuration:
  filename is now relative to the output file directory.

- Added option ``--case-ranges`` and configuration ``re2c🎏case-ranges``.

- Extended fixed tags optimization for the case of fixed-counter repetition.

- Fixed bugs related to EOF rule:

  + `#276 <https://github.com/skvadrik/re2c/issues/276>`_:
    Example 01_fill.re in docs is broken
  + `#280 <https://github.com/skvadrik/re2c/issues/280>`_:
    EOF rules with multiple blocks
  + `#284 <https://github.com/skvadrik/re2c/issues/284>`_:
    mismatched YYBACKUP and YYRESTORE
    (Add missing fallback states with EOF rule)

- Fixed miscellaneous bugs:

  + `#286 <https://github.com/skvadrik/re2c/issues/286>`_:
    Incorrect submatch values with fixed-length trailing context.
  + `#297 <https://github.com/skvadrik/re2c/issues/297>`_:
    configure error on ubuntu 18.04 / cmake 3.10

- Changed bootstrap process (require explicit configuration flags and a path to
  re2c executable to regenerate the lexers).

- Added internal options ``--posix-prectable <naive | complex>``.

- Added debug option ``--dump-dfa-tree``.

- Major revision of the paper "Efficient POSIX submatch extraction on NFA".

----
1.3x
----

1.3 (2019-12-14)
~~~~~~~~~~~~~~~~

- Added option: ``--stadfa``.

- Added warning: ``-Wsentinel-in-midrule``.

- Added generic API primitives:

  + ``YYSTAGPD``
  + ``YYMTAGPD``

- Added configurations:

  + ``re2c:sentinel = 0;``
  + ``re2c:define:YYSTAGPD = "YYSTAGPD";``
  + ``re2c:define:YYMTAGPD = "YYMTAGPD";``

- Worked on reproducible builds
  (`#258 <https://github.com/skvadrik/re2c/pull/258>`_:
  Make the build reproducible).

----
1.2x
----

1.2.1 (2019-08-11)
~~~~~~~~~~~~~~~~~~

- Fixed bug `#253 <https://github.com/skvadrik/re2c/issues/253>`_:
  re2c should install unicode_categories.re somewhere.

- Fixed bug `#254 <https://github.com/skvadrik/re2c/issues/254>`_:
  Turn off re2c:eof = 0.

1.2 (2019-08-02)
~~~~~~~~~~~~~~~~

- Added EOF rule ``$`` and configuration ``re2c:eof``.

- Added ``/*!include:re2c ... */`` directive and ``-I`` option.

- Added ``/*!header:re2c:on*/`` and ``/*!header:re2c:off*/`` directives.

- Added ``--input-encoding <ascii | utf8>`` option.

  + `#237 <https://github.com/skvadrik/re2c/issues/237>`_:
    Handle non-ASCII encoded characters in regular expressions
  + `#250 <https://github.com/skvadrik/re2c/issues/250>`_
    UTF8 enoding

- Added include file with a list of definitions for Unicode character classes.

  + `#235 <https://github.com/skvadrik/re2c/issues/235>`_:
    Unicode character classes

- Added ``--location-format <gnu | msvc>`` option.

  + `#195 <https://github.com/skvadrik/re2c/issues/195>`_:
    Please consider using Gnu format for error messages

- Added ``--verbose`` option that prints "success" message if re2c exits
  without errors.

- Added configurations for options:

  + ``-o --output`` (specify output file)
  + ``-t --type-header`` (specify header file)

- Removed configurations for internal/debug options.

- Extended ``-r`` option: allow to mix multiple ``/*!rules:re2c*/``,
  ``/*!use:re2c*/`` and ``/*!re2c*/`` blocks.

  + `#55 <https://github.com/skvadrik/re2c/issues/55>`_:
    allow standard re2c blocks in reuse mode

- Fixed ``-F --flex-support`` option: parsing and operator precedence.

  + `#229 <https://github.com/skvadrik/re2c/issues/229>`_:
    re2c option -F (flex syntax) broken
  + `#242 <https://github.com/skvadrik/re2c/issues/242>`_:
    Operator precedence with --flex-syntax is broken

- Changed difference operator ``/`` to apply before encoding expansion of
  operands.

  + `#236 <https://github.com/skvadrik/re2c/issues/236>`_:
    Support range difference with variable-length encodings

- Changed output generation of output file to be atomic.

  + `#245 <https://github.com/skvadrik/re2c/issues/245>`_:
    re2c output is not atomic

- Authored research paper "Efficient POSIX Submatch Extraction on NFA"
  together with Dr Angelo Borsotti.

- Added experimental libre2c library (``--enable-libs`` configure option) with
  the following algorithms:

  + TDFA with leftmost-greedy disambiguation
  + TDFA with POSIX disambiguation (Okui-Suzuki algorithm)
  + TNFA with leftmost-greedy disambiguation
  + TNFA with POSIX disambiguation (Okui-Suzuki algorithm)
  + TNFA with lazy POSIX disambiguation (Okui-Suzuki algorithm)
  + TNFA with POSIX disambiguation (Kuklewicz algorithm)
  + TNFA with POSIX disambiguation (Cox algorithm)

- Added debug subsystem (``--enable-debug`` configure option) and new debug
  options:

  + ``-dump-cfg`` (dump control flow graph of tag variables)
  + ``-dump-interf`` (dump interference table of tag variables)
  + ``-dump-closure-stats`` (dump epsilon-closure statistics)

- Added internal options:

  + ``--posix-closure <gor1 | gtop>`` (switch between shortest-path algorithms
    used for the construction of POSIX closure)

- Fixed a number of crashes found by American Fuzzy Lop fuzzer:

  + `#226 <https://github.com/skvadrik/re2c/issues/226>`_,
    `#227 <https://github.com/skvadrik/re2c/issues/227>`_,
    `#228 <https://github.com/skvadrik/re2c/issues/228>`_,
    `#231 <https://github.com/skvadrik/re2c/issues/231>`_,
    `#232 <https://github.com/skvadrik/re2c/issues/232>`_,
    `#233 <https://github.com/skvadrik/re2c/issues/233>`_,
    `#234 <https://github.com/skvadrik/re2c/issues/234>`_,
    `#238 <https://github.com/skvadrik/re2c/issues/238>`_

- Fixed handling of newlines:

  + correctly parse multi-character newlines CR LF in ``#line`` directives
  + consistently convert all newlines in the generated file to Unix-style LF

- Changed default tarball format from .gz to .xz.

  + `#221 <https://github.com/skvadrik/re2c/issues/221>`_:
    big source tarball

- Fixed a number of other bugs and resolved issues:

  + `#2 <https://github.com/skvadrik/re2c/issues/2>`_: abort
  + `#6 <https://github.com/skvadrik/re2c/issues/6>`_: segfault
  + `#10 <https://github.com/skvadrik/re2c/issues/10>`_:
    lessons/002_upn_calculator/calc_002 doesn't produce a useful example program
  + `#44 <https://github.com/skvadrik/re2c/issues/44>`_:
    Access violation when translating the attached file
  + `#49 <https://github.com/skvadrik/re2c/issues/49>`_:
    wildcard state \000 rules makes lexer behave weard
  + `#98 <https://github.com/skvadrik/re2c/issues/98>`_:
    Transparent handling of #line directives in input files
  + `#104 <https://github.com/skvadrik/re2c/issues/104>`_:
    Improve const-correctness
  + `#105 <https://github.com/skvadrik/re2c/issues/105>`_:
    Conversion of pointer parameters into references
  + `#114 <https://github.com/skvadrik/re2c/issues/114>`_:
    Possibility of fixing bug 2535084
  + `#120 <https://github.com/skvadrik/re2c/issues/120>`_:
    condition consisting of default rule only is ignored
  + `#167 <https://github.com/skvadrik/re2c/issues/167>`_:
    Add word boundary support
  + `#168 <https://github.com/skvadrik/re2c/issues/168>`_:
    Wikipedia's article on re2c
  + `#180 <https://github.com/skvadrik/re2c/issues/180>`_:
    Comment syntax?
  + `#182 <https://github.com/skvadrik/re2c/issues/182>`_:
    yych being set by YYPEEK () and then not used
  + `#196 <https://github.com/skvadrik/re2c/issues/196>`_:
    Implicit type conversion warnings
  + `#198 <https://github.com/skvadrik/re2c/issues/198>`_:
    no match for ‘operator!=’ in ‘i != std::vector<_Tp, _Alloc>::rend() [with _Tp = re2c::bitmap_t, _Alloc = std::allocator<re2c::bitmap_t>]()’
  + `#210 <https://github.com/skvadrik/re2c/issues/210>`_:
    How to build re2c in windows?
  + `#215 <https://github.com/skvadrik/re2c/issues/215>`_:
    A memory read overrun issue in s_to_n32_unsafe.cc
  + `#220 <https://github.com/skvadrik/re2c/issues/220>`_:
    src/dfa/dfa.h: simplify constructor to avoid g++-3.4 bug
  + `#223 <https://github.com/skvadrik/re2c/issues/223>`_:
    Fix typo
  + `#224 <https://github.com/skvadrik/re2c/issues/224>`_:
    src/dfa/closure_posix.cc: pack() tweaks
  + `#225 <https://github.com/skvadrik/re2c/issues/225>`_:
    Documentation link is broken in libre2c/README
  + `#230 <https://github.com/skvadrik/re2c/issues/230>`_:
    Changes for upcoming Travis' infra migration
  + `#239 <https://github.com/skvadrik/re2c/issues/239>`_:
    Push model example has wrong re2c invocation, breaks guide
  + `#241 <https://github.com/skvadrik/re2c/issues/241>`_:
    Guidance on how to use re2c for full-duplex command & response protocol
  + `#243 <https://github.com/skvadrik/re2c/issues/243>`_:
    A code generated for period (.) requires 4 bytes
  + `#246 <https://github.com/skvadrik/re2c/issues/246>`_:
    Please add a license to this repo
  + `#247 <https://github.com/skvadrik/re2c/issues/247>`_:
    Build failure on current Cygwin, probably caused by force-fed c++98 mode
  + `#248 <https://github.com/skvadrik/re2c/issues/248>`_:
    distcheck still looks for README
  + `#251 <https://github.com/skvadrik/re2c/issues/251>`_:
    Including what you use is find, but not without inclusion guards

- Updated documentation and website.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants