Skip to content
[UNMAINTAINED] Yet another lexical analyser library written in pure Ruby
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
lib
spec
.gitignore
.rspec
.rubocop.yml
.travis.yml
CHANGELOG.md
Gemfile
LICENSE.txt
README.md
Rakefile
rhetor.gemspec

README.md

Rhetor

GitHub RubyGems.org Documentation License

Dependency Status Coverage Status Build Status Code Climate Gem Version

Rhetor is yet another lexical analyser library written in pure Ruby with no runtime dependencies. It is properly documented and easy-to-use.

Installation

Add this line to your application's Gemfile:

gem 'rhetor'

And then execute:

$ bundle

Or install it yourself as:

$ gem install rhetor

Usage

# Define behaviour
lexer = Rhetor::LexicalAnalyser.new do
  # Within this block you can define your own rules, your own patterns
  # to recognize. You can call these methods out of this block
  # like this: lexer.rule(...), it also works.
  rule '+', :plus
  rule '-', :minus
  rule '*', :asterisk
  rule '/', :solidus
  rule '(', :left_parenthesis
  rule ')', :right_parenthesis

  # This rule will return tokens with floating-point values.
  # Actually, you can make Rhetor to return tokens with any types of values,
  # you only have to provide an evaluator - a block which receives a matched
  # substring and returns the desired value. Like here:
  rule(/[-+]?[0-9]*\.?[0-9]+/, :number) { |string| string.to_f }

  # Here we make Rhetor to ignore whitespaces via the regular expression:
  ignore(/\s+/)
  # We could also do it with a string: ignore ' ',
  # but regular expressions are more powerful.
end

# Analyse method returns an array of tokens. You can provide a block
# and it will be run for each of encountered tokens.
tokens = lexer.analyse('2 + 2 * 2 - (25 / (3 + 70 / 4))') do |token|
  puts token
end
# It produces:
# (number: 2.0 [0,1])
# (plus: "+" [2,1])
# (number: 2.0 [4,1])
# (asterisk: "*" [6,1])
# (number: 2.0 [8,1])
# (minus: "-" [10,1])
# (left_parenthesis: "(" [12,1])
# (number: 25.0 [13,2])
# (solidus: "/" [16,1])
# (left_parenthesis: "(" [18,1])
# (number: 3.0 [19,1])
# (plus: "+" [21,1])
# (number: 70.0 [23,2])
# (solidus: "/" [26,1])
# (number: 4.0 [28,1])
# (right_parenthesis: ")" [29,1])
# (right_parenthesis: ")" [30,1])

# Now get information about any of the tokens:
token = tokens.first # => (number: 2.0 [0,1])
token.value          # => 2.0
token.name           # => :number
token.position       # => 0
token.length         # => 1

Contributing

  1. Fork it (https://github.com/edubenetskiy/rhetor)
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create a new Pull Request

License

Rhetor © 2015 by Egor Dubenetskiy. Rhetor is licensed under the MIT license. Please see LICENSE.txt for further details.

You can’t perform that action at this time.