This is a very simple example of how to generate a lexer with JFlex:
- for a very simple grammar of the Toy programmimg language described in the user manual.
- without integration to a parser. As a result, the program does nothing really useful, because there is no parser.
The generated lexer has the default name Yylex because the flex
specification doesn't define a name with the
The project comes with a test class for the lexer:
- runs the lexer in debug mode on
- collects the output of JFlex by redirecting
- and verifies that the verbose logs of JFlex corresponds to
the expected content of
src/main/flex/simple.flex: the simple grammar specification
src/test/data/test.txt: sample input
src/test/data/output.good: golden file, i.e. expected output corresponding to the sample input from
src/test/java/YylexTest.java: jUnit integration test that running the lexer on the sample input produces the same output as the golden file.
Compile and test
Tip JFlex comes with the Maven wrapper (mvnw).
This guide uses the wrapper located in
..\..\..\mvnw.bat if your are on Microsoft Windows.
You can also use
mvn if Maven is installed on your system.
Generate the lexer
The jflex-maven-plugin reads the grammar
and generates a Java scanner Yylex.
This is defined by the following section
<build> <plugins> <plugin> <groupId>de.jflex</groupId> <artifactId>jflex-maven-plugin</artifactId> <version>1.8.0</version> <executions> <execution> <goals> <goal>generate</goal> </goals> </execution> </executions> </plugin> </plugins> </build>
N.B. By default, the jflex-maven-plugin generates a lexer (scanner) for every file
The compile phase will generate the sources, and build all Java classes, including those generated automatically.
- Java compiled class files in
Tip In fact, you don't have to invoke
mvn generate-sources explicitly,
the compile phase will do it automatically.
The test phase does everything above and executes the test in
There is only one test in
In this test, the scanner is run with the input file
By default, the scanner outputs debugging information about each returned token to
until the end of file is reached, or an error occurs.
But in the test, the output is redirected into an in-memory output stream.
Then, test opens the golden file
Finally, the test iterates:
- Reads one line from the golden file
- Reads one line from the in-memory actual output
- Fails if the lines aren't equal
- Stops if one file reaches the end of file
The package phase does everything above and packages the jar archive of the Java classes.
N.B. You need to install ant with ivy.
Run on sample file
TODO #429 This currently fails
Please see bazel_rules/examples.