Skip to content
Branch: master
Find file History
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
..
Failed to load latest commit information.
src
.gitignore
BUILD
README.md
build.xml
ivy.xml
pom.xml

README.md

Simple usage

This is a very simple example of how to generate a lexer with JFlex:

  • for a very simple grammar of the Toy programmimg language described in the user manual.
  • without integration to a parser. As a result, the program does nothing really useful, because there is no parser.

The generated lexer has the default name Yylex because the flex specification doesn't define a name with the %class.

The project comes with a test class for the lexer: YylexTest.

The test:

  1. runs the lexer in debug mode on test.txt
  2. collects the output of JFlex by redirecting System.out
  3. and verifies that the verbose logs of JFlex corresponds to the expected content of output.good.

Files

  • src/main/flex/simple.flex: the simple grammar specification
  • src/test/data/test.txt: sample input
  • src/test/data/output.good: golden file, i.e. expected output corresponding to the sample input from test.txt
  • src/test/java/YylexTest.java: jUnit integration test that running the lexer on the sample input produces the same output as the golden file.

Compile and test

Using Maven

Tip JFlex comes with the Maven wrapper (mvnw). This guide uses the wrapper located in ../../../mvnw. Use ..\..\..\mvnw.bat if your are on Microsoft Windows. You can also use mvn if Maven is installed on your system.

Generate the lexer

../../../mvnw generate-sources

The jflex-maven-plugin reads the grammar src/main/jflex/simple.jflex and generates a Java scanner Yylex.

Expected output:

  • target/generated-sources/flex/Yylex.java.

This is defined by the following section

  <build>
    <plugins>
      <plugin>
        <groupId>de.jflex</groupId>
        <artifactId>jflex-maven-plugin</artifactId>
        <version>1.8.0</version>
        <executions>
          <execution>
            <goals>
              <goal>generate</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>

N.B. By default, the jflex-maven-plugin generates a lexer (scanner) for every file in src/main/jflex/.

Build

../../../mvnw compile

The compile phase will generate the sources, and build all Java classes, including those generated automatically.

Expected output:

  • Java compiled class files in target/classes.

Tip In fact, you don't have to invoke mvn generate-sources explicitly, the compile phase will do it automatically.

Test

../../../mvnw test

The test phase does everything above and executes the test in src/test/java.

There is only one test in src/test/java/YylexTest.java. In this test, the scanner is run with the input file src/test/data/test.txt.

By default, the scanner outputs debugging information about each returned token to System.out until the end of file is reached, or an error occurs. But in the test, the output is redirected into an in-memory output stream.

Then, test opens the golden file src/test/data/output.good.

Finally, the test iterates:

  1. Reads one line from the golden file
  2. Reads one line from the in-memory actual output
  3. Fails if the lines aren't equal
  4. Stops if one file reaches the end of file

Package

../../../mvnw package

The package phase does everything above and packages the jar archive of the Java classes.

Using ant

N.B. You need to install ant with ivy.

Build

ant build

Run on sample file

ant run

Test

TODO #429 This currently fails

ant run

Using Bazel

Please see bazel_rules/examples.

You can’t perform that action at this time.