Skip to content
C++14 version of lexertl
C++ Other
  1. C++ 98.9%
  2. Other 1.1%
Branch: master
Clone or download

Latest commit

Fetching latest commit…
Cannot retrieve the latest commit at this time.

Files

Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
examples
extras
include/lexertl
test
CMakeLists.txt
README.md
lexertlConfig.cmake

README.md

lexertl14: The Modular Lexical Analyser Generator

lexertl is a header-only library for writing lexical analysers. With lexertl you can:

  • Build lexical analysers at runtime
  • Scan Unicode and ASCII input
  • Scan from files or memory
  • Generate C++ code or even write your own code generator

Construct a Lexer and Tokenise input

#include <lexertl/generator.hpp>
#include <lexertl/lookup.hpp>
#include <iostream>

int main()
{
    lexertl::rules rules;
    lexertl::state_machine sm;

    rules.push("[0-9]+", 1);
    rules.push("[a-z]+", 2);
    lexertl::generator::build(rules, sm);

    std::string input("abc012Ad3e4");
    lexertl::smatch results(input.begin(), input.end());

    // Read ahead
    lexertl::lookup(sm, results);

    while (results.id != 0)
    {
        std::cout << "Id: " << results.id << ", Token: '" <<
            results.str () << "'\n";
        lexertl::lookup(sm, results);
    }

    return 0;
}

The same thing using lexertl::iterator:

#include <lexertl/generator.hpp>
#include <lexertl/iterator.hpp>
#include <iostream>

int main()
{
    lexertl::rules rules;
    lexertl::state_machine sm;

    rules.push("[0-9]+", 1);
    rules.push("[a-z]+", 2);
    lexertl::generator::build(rules, sm);

    std::string input("abc012Ad3e4");
    lexertl::siterator iter(input.begin(), input.end(), sm);
    lexertl::siterator end;

    for (; iter != end; ++iter)
    {
        std::cout << "Id: " << iter->id << ", Token: '" <<
            iter->str() << "'\n";
    }

    return 0;
}

More examples and documentation

See http://www.benhanson.net/lexertl.html for full documentation and more usage examples.

You can’t perform that action at this time.