Update bison to 3.4.1

[pal1000]

http://lists.gnu.org/archive/html/bison-announce/2019-05/msg00002.html.

[lexxmark]

This release kinda painful for adoption.
Starting from bison 3.4.0 these is a new dependency. Here is a quote from bison news:

** New features

*** Colored diagnostics

As an experimental feature, diagnostics are now colored, controlled by the
new options --color and --style.

To use them, install the libtextstyle library before configuring Bison.
It is available from

https://alpha.gnu.org/gnu/gettext/

for instance

https://alpha.gnu.org/gnu/gettext/libtextstyle-0.8.tar.gz

So we need to incorporate libtextstyle lib into winflexbison somehow.
There are some solutions I see:

1. Try to merge source code of libtextstyle into winflexbison - pos: no external dependencies, contra: a lot of work for adaptation
2. Use precompiled windows dll from https://github.com/vslavik/gettext-tools-windows. pos: less adaptation, contra: one external file in the package
3. Don't merge text style changes from bison into winflexbison - pos: no external dependencies; contra: need some adoption; no text style feature in winflexbison; more complex adoption bison in the future.

This needs more investigation.
I need your opinions @pal1000, @GitMensch before decide what to do.

[GitMensch]

I suggest to use vcpkg to install the gettext "port". This way you end up with a "native" solution (same compiler + target) without an external binary dependency and this should also allow providing translated messages.

Hm, libtextstyle may not be part of the port yet, you may want to ask the port's authors to add it either to this port or add a completely new port for it (or create it on your own).

[lexxmark]

OK, will need more time for investigation

[GitMensch]

A quick glance over the configure script leads to the point that it is an optional dependency. In the used config.h (or compilation commands) define HAVE_LIBTEXTSTYLE=no (and possibly a bunch of other defines to nothing) and everything should be fine.
I'd still recommend to have this added, but it seems reasonable to allow a build with/without and (depending on the size of libtextstyle and its dependencies) possibly provide a release with/without it.

[lexxmark]

That's interesting. Will try to avoid libtextstyle library (and add it later).
Unfortunately I ruined my laptop and have to reinstall windows. It will take a while to return to winflexbison stuff.

[lexxmark]

HAVE_LIBTEXTSTYLE doesn't help. There is no option to disable textstyle library in gnu bison.
We have to incorporate libtextstyle into winflexbison as static lib.

I've merged code from gnu bison v3.4.1 in the branch bison3.4.1. It doesn't compile because of missing "textstyle.h" header.

Also there are changes in generated files: "scan-code.c" is missing now and replaced with "scan-code-c.c" with the following content:
#include <config.h> #include "system.h" #include "src/scan-code.c"
The question is how to obtain "src/scan-code.c" source code?
The same thin happened with "scan-gram.c" and "scan-skel.c" files.

[GitMensch]

Anyone hacking bison should very likely regenerated the .l files instead of using pre-generated C sources - can you adjust this (remove generated C files from the repo, invoke win_flex to generate them in the CI build)?

Concerning textstyle - it is used in Bison as a gnulib optional module - this means that if it isn't locally available a local textstyle.h.in gets copied to textstyle.h and leads to "nothing happens" instead of "check for the header + additional add checks in the code via HAVE_LIBTEXTSTYLE", this was adjusted in 4d34b06fb3a38eb050439084476a6b3e292c5680, see the gnulib module definition and textstyle.h.in which you can use as long as you don't want to link against textstyle.

[lexxmark]

can you adjust this (remove generated C files from the repo, invoke win_flex to generate them in the CI build)?

Unfortunately I'm not familiar with autoconf, can you help me with such automation?

Concerning textstyle - it is used in Bison as a gnulib optional module - this means that if it isn't locally available a local textstyle.h.in gets copied to textstyle.h and leads to "nothing happens" instead of "check for the header + additional add checks in the code via HAVE_LIBTEXTSTYLE", this was adjusted in 4d34b06fb3a38eb050439084476a6b3e292c5680, see the gnulib module definition and textstyle.h.in which you can use as long as you don't want to link against textstyle.

The same thing for textstyle. As I understand all these machinery happens during configuration that my project totally miss. I see two solutions here: make configuration one time and use generated files or reproduce configuration stuff in VS project files or CMake configs.

[GitMensch]

can you adjust this (remove generated C files from the repo, invoke win_flex to generate them in the CI build)?

Unfortunately I'm not familiar with autoconf, can you help me with such automation?

Just invoking win_flex to generate the C files from the scanner? It may be integrated as a pre-built-step in the project used for building win_bison (which would work in any environment that uses the project/solution to build) or as a Makefile using nmake (which has the benefit that the files will only be re-generated if necessary [= the scanner source changed]).

The same thing for textstyle. As I understand all these machinery happens during configuration that my project totally miss.

Yes.

I see two solutions here: make configuration one time and use generated files or reproduce configuration stuff in VS project files or CMake configs.

I'd suggest not to use the first one, either can be done in the project files/CMake (or nmake) or - for the part of textstyle: just copy the linked textstyle.h.in into your include directory as textstyle.h for now. As a second step you may use two different configurations in the project file, one that uses a special include directory "lib/textstyle" and one that expects to find the file in the normal places and links against libtextstyle.

In any case I highly suggest to split this:

• bison update = generate the scanner C files ones on your machine, including them in the repo (instead of the ones from Akim), do the same for textstyle.h.in as textstyle.h
• get the scanner files to be automatically rebuilt either if missing, or better: if out-of-date; maybe an additional issue to create, being able to close this
• support for libtextstyle - should be moved to a different issue (still very useful for development that includes grammar-files, but not necessary to generate sources from these [which is done much more often])

[lexxmark]

Sorry I'm totally lost.
Where can I find textstyle.h.in file?
Where can I find which flex commands (with parameters) should I invoke for *.l files?
The same for bison command for parse-gram.y file.

[GitMensch]

Where can I find textstyle.h.in file?

textstyle.h.in

Where can I find which flex commands (with parameters) should I invoke for *.l files?

The main Makefile.am includes src/local.mk where the sources are defined.
No special definitions of commands are used, so I guess it is the simple invocation without special flags... rechecked with a simple build on bash on windows, this is what is done:

/bin/bash ../build-aux/ylwrap ../src/parse-gram.y y.tab.c ../src/parse-gram.c y.tab.h ../src/parse-gram.h y.output ../src/parse-gram.output -- ./tests/bison -o y.tab.c --defines -Werror -Wall --report=all --no-lines
/bin/bash ../build-aux/ylwrap ../src/scan-code.l lex.yy.c ../src/scan-code.c -- flex
/bin/bash ../build-aux/ylwrap ../src/scan-gram.l lex.yy.c ../src/scan-gram.c -- flex
/bin/bash ../build-aux/ylwrap ../src/scan-skel.l lex.yy.c ../src/scan-skel.c -- flex

ylwrap's usage is

Usage: ylwrap [--help|--version] INPUT [OUTPUT DESIRED]... -- PROGRAM [ARGS]...

So flex is invoked without arguments, bison is invoked with -o y.tab.c --defines -Werror -Wall --report=all --no-lines (and later on the output file is renamed and internally adjusted so the output name should be different in your case).

[lexxmark]

It was my mistake to grab bison source code from github repo. It lacks some sources.

I finally managed to upgrade win_bison to 3.4.1 version (without textstyle).