Skip to content
Browse files

Parslet accelerators

Accelerators allow to transform a parser into another parser, using
pattern matching. This works in a similar fashion as parslet
transformers do. See example/optimized_erb for sample code.
  • Loading branch information...
1 parent 172fe26 commit 66e242841ea27554a64f42d52093c1f41bc56e1e @kschiess committed Aug 16, 2013
View
4 HISTORY.txt
@@ -5,6 +5,10 @@
= 1.6 / ??
+ + Parslet accelerators permit replacing parts of your parser with optimized
+ atoms using pattern matching. Look at examples/optimized_erb.rb or the
+ introduction to the feature in qed/accelerators.md.
+
+ infix_expression permits to declare an infix expression parser (think
calculator) directly. This will solve many of the problems we have
more elegantly.
View
73 example/big.erb
@@ -0,0 +1,73 @@
+Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
+tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
+quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
+consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
+cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
+proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
+Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
+tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
+quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
+consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
+cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
+proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
+Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
+tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
+quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
+consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
+cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
+proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
+Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
+tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
+quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
+consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
+cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
+proident, sunt in culpa qui officia deserunt mollit anim id est laborum.Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
+tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
+quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
+consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
+cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
+proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
+Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
+tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
+quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
+consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
+cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
+proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
+
+<%= erb tag %>
+
+Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
+tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
+quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
+consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
+cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
+proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
+Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
+tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
+quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
+consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
+cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
+proident, sunt in culpa qui officia deserunt mollit anim id est laborum.Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
+tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
+quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
+consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
+cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
+proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
+Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
+tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
+quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
+consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
+cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
+proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
+Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
+tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
+quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
+consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
+cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
+proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
+Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
+tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
+quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
+consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
+cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
+proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
View
42 example/optimized_erb.rb
@@ -0,0 +1,42 @@
+# Please also look at the more naive 'erb.rb'. This shows how to optimize an
+# ERB like parser using parslet.
+
+$:.unshift File.dirname(__FILE__) + "/lib"
+
+require 'parslet'
+require './qed/applique/gobbleup'
+require 'parslet/accelerator'
+
+class ErbParser < Parslet::Parser
+ rule(:ruby) { (str('%>').absent? >> any).repeat.as(:ruby) }
+
+ rule(:expression) { (str('=') >> ruby).as(:expression) }
+ rule(:comment) { (str('#') >> ruby).as(:comment) }
+ rule(:code) { ruby.as(:code) }
+ rule(:erb) { expression | comment | code }
+
+ rule(:erb_with_tags) { str('<%') >> erb >> str('%>') }
+ rule(:text) { (str('<%').absent? >> any).repeat(1) }
+
+ rule(:text_with_ruby) { (text.as(:text) | erb_with_tags).repeat.as(:text) }
+ root(:text_with_ruby)
+end
+
+parser = ErbParser.new
+
+A = Parslet::Accelerator
+optimized = A.apply(parser,
+ A.rule((A.str(:x).absent? >> A.any).repeat(1)) { GobbleUp.new(x, 1) },
+ A.rule((A.str(:x).absent? >> A.any).repeat(0)) { GobbleUp.new(x, 0) })
+
+input = File.read(File.dirname(__FILE__) + "/big.erb")
+
+# Remove the comment marks here to see what difference the optimisation makes.
+# Commented out for the acceptance tests to run.
+#
+# require 'benchmark'
+# Benchmark.bm(7) do |bm|
+# bm.report('original') { parser.parse(input) }
+# bm.report('gobble') { optimized.parse(input) }
+# end
+p optimized.parse(input)
View
1 example/output/optimized_erb.out
@@ -0,0 +1 @@
+{:text=>[{:text=>"Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod\ntempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,\nquis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo\nconsequat. Duis aute irure dolor in reprehenderit in voluptate velit esse\ncillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non\nproident, sunt in culpa qui officia deserunt mollit anim id est laborum.\nLorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod\ntempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,\nquis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo\nconsequat. Duis aute irure dolor in reprehenderit in voluptate velit esse\ncillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non\nproident, sunt in culpa qui officia deserunt mollit anim id est laborum.\nLorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod\ntempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,\nquis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo\nconsequat. Duis aute irure dolor in reprehenderit in voluptate velit esse\ncillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non\nproident, sunt in culpa qui officia deserunt mollit anim id est laborum.\nLorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod\ntempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,\nquis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo\nconsequat. Duis aute irure dolor in reprehenderit in voluptate velit esse\ncillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non\nproident, sunt in culpa qui officia deserunt mollit anim id est laborum.Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod\ntempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,\nquis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo\nconsequat. Duis aute irure dolor in reprehenderit in voluptate velit esse\ncillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non\nproident, sunt in culpa qui officia deserunt mollit anim id est laborum.\nLorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod\ntempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,\nquis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo\nconsequat. Duis aute irure dolor in reprehenderit in voluptate velit esse\ncillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non\nproident, sunt in culpa qui officia deserunt mollit anim id est laborum.\n\n"@0}, {:expression=>{:ruby=>" erb tag "@2685}}, {:text=>"\n\nLorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod\ntempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,\nquis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo\nconsequat. Duis aute irure dolor in reprehenderit in voluptate velit esse\ncillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non\nproident, sunt in culpa qui officia deserunt mollit anim id est laborum.\nLorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod\ntempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,\nquis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo\nconsequat. Duis aute irure dolor in reprehenderit in voluptate velit esse\ncillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non\nproident, sunt in culpa qui officia deserunt mollit anim id est laborum.Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod\ntempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,\nquis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo\nconsequat. Duis aute irure dolor in reprehenderit in voluptate velit esse\ncillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non\nproident, sunt in culpa qui officia deserunt mollit anim id est laborum.\nLorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod\ntempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,\nquis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo\nconsequat. Duis aute irure dolor in reprehenderit in voluptate velit esse\ncillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non\nproident, sunt in culpa qui officia deserunt mollit anim id est laborum.\nLorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod\ntempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,\nquis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo\nconsequat. Duis aute irure dolor in reprehenderit in voluptate velit esse\ncillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non\nproident, sunt in culpa qui officia deserunt mollit anim id est laborum.\nLorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod\ntempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,\nquis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo\nconsequat. Duis aute irure dolor in reprehenderit in voluptate velit esse\ncillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non\nproident, sunt in culpa qui officia deserunt mollit anim id est laborum.\n"@2696}]}
View
102 lib/parslet/accelerator.rb
@@ -1,5 +1,39 @@
+
+
+# Optimizes the parsers by pattern matching on the parser atoms and replacing
+# matches with better versions. See the file qed/accelerators.md for a more
+# in-depth description.
+#
+# Example:
+# quote = str('"')
+# parser = quote >> (quote.absent? >> any).repeat >> quote
+#
+# A = Accelerator # for making what follows a bit shorter
+# optimized_parser = A.apply(parser,
+# A.rule( (A.str(:x).absent? >> A.any).repeat ) { GobbleUp.new(x) })
+#
+# optimized_parser.parse('"Parsing is now fully optimized! (tm)"')
+#
module Parslet::Accelerator
+ # An expression to match against a tree of parser atoms. Normally, an
+ # expression is produced by Parslet::Accelerator.any,
+ # Parslet::Accelerator.str or Parslet::Accelerator.re.
+ #
+ # Expressions can be chained much like parslet atoms can be:
+ #
+ # expr.repeat(1) # matching repetition
+ # expr.absent? # matching absent?
+ # expr.present? # matching present?
+ # expr1 >> expr2 # matching a sequence
+ # expr1 | expr2 # matching an alternation
+ #
+ # @see Parslet::Accelerator.str
+ # @see Parslet::Accelerator.re
+ # @see Parslet::Accelerator.any
+ #
+ # @see Parslet::Accelerator
+ #
class Expression
attr_reader :type
attr_reader :args
@@ -9,25 +43,32 @@ def initialize(type, *args)
@args = args
end
+ # @return [Expression]
def >> other_expr
join_or_new :seq, other_expr
end
+ # @return [Expression]
def | other_expr
join_or_new :alt, other_expr
end
+ # @return [Expression]
def absent?
Expression.new(:absent, self)
end
+ # @return [Expression]
def present?
Expression.new(:present, self)
end
+ # @return [Expression]
def repeat min=0, max=nil
Expression.new(:rep, min, max, self)
end
+ # @api private
+ # @return [Expression]
def join_or_new tag, other_expr
if type == tag
@args << other_expr
@@ -38,19 +79,78 @@ def join_or_new tag, other_expr
end
module_function
+ # Returns a match expression that will match `str` parslet atoms.
+ #
+ # @return [Parslet::Accelerator::Expression]
+ #
def str variable, *constraints
Expression.new(:str, variable, *constraints)
end
+ # Returns a match expression that will match `match` parslet atoms.
+ #
+ # @return [Parslet::Accelerator::Expression]
+ #
def re variable, *constraints
Expression.new(:re, variable, *constraints)
end
+ # Returns a match expression that will match `any` parslet atoms.
+ #
+ # @return [Parslet::Accelerator::Expression]
+ #
+ def any
+ Expression.new(:re, ".")
+ end
+
+ # Given a parslet atom and an expression, will determine if the expression
+ # matches the atom. If successful, returns the bindings into the pattern
+ # that were made. If no bindings had to be made to make the match successful,
+ # the empty hash is returned.
+ #
+ # @param atom [Parslet::Atoms::Base] parslet atom to match against
+ # @param expr [Parslet::Accelerator::Expression] expression to match
+ # @return [nil, Hash] bindings for the match, nil on failure
+ #
def match atom, expr
engine = Engine.new
return engine.bindings if engine.match(atom, expr)
end
+
+ # Constructs an accelerator rule. A rule is a matching expression and the
+ # code that should be executed once the expression could be bound to a
+ # parser.
+ #
+ # Example:
+ # Accelerator.rule(Accelerator.any) { Parslet.match('.') }
+ #
+ def rule expression, &action
+ [expression, action]
+ end
+
+ # Given a parslet atom and a set of rules, tries to match the rules
+ # recursively through the parslet atom. Once a rule could be matched,
+ # its action block will be called.
+ #
+ # Example:
+ # quote = str('"')
+ # parser = quote >> (quote.absent? >> any).repeat >> quote
+ #
+ # A = Accelerator # for making what follows a bit shorter
+ # optimized_parser = A.apply(parser,
+ # A.rule( (A.str(:x).absent? >> A.any).repeat ) { GobbleUp.new(x) })
+ #
+ # optimized_parser.parse('"Parsing is now fully optimized! (tm)"')
+ #
+ # @param atom [Parslet::Atoms::Base] a parser to optimize
+ # @param *rules [Parslet::Accelerator::Rule] rules produced by .rule
+ # @return [Parslet::Atoms::Base] optimized parser
+ #
+ def apply atom, *rules
+ Application.new(atom, rules).call
+ end
end
-require 'parslet/accelerator/engine'
+require 'parslet/accelerator/engine'
+require 'parslet/accelerator/application'
View
62 lib/parslet/accelerator/application.rb
@@ -0,0 +1,62 @@
+
+# @api private
+module Parslet::Accelerator
+ class Application
+ def initialize atom, rules
+ @atom = atom
+ @rules = rules
+ end
+
+ def call
+ @atom.accept(self)
+ end
+
+ def visit_parser(root)
+ transform root.accept(self)
+ end
+ def visit_entity(name, block)
+ transform Parslet::Atoms::Entity.new(name) { block.call.accept(self) }
+ end
+ def visit_named(name, atom)
+ transform Parslet::Atoms::Named.new(atom.accept(self), name)
+ end
+ def visit_repetition(tag, min, max, atom)
+ transform Parslet::Atoms::Repetition.new(atom.accept(self), min, max, tag)
+ end
+ def visit_alternative(alternatives)
+ transform Parslet::Atoms::Alternative.new(
+ *alternatives.map { |atom| atom.accept(self) })
+ end
+ def visit_sequence(sequence)
+ transform Parslet::Atoms::Sequence.new(
+ *sequence.map { |atom| atom.accept(self) })
+ end
+ def visit_lookahead(positive, atom)
+ transform Parslet::Atoms::Lookahead.new(atom, positive)
+ end
+ def visit_re(regexp)
+ transform Parslet::Atoms::Re.new(regexp)
+ end
+ def visit_str(str)
+ transform Parslet::Atoms::Str.new(str)
+ end
+
+ def transform atom
+ @rules.each do |expr, action|
+ # Try and match each rule in turn
+ binding = Parslet::Accelerator.match(atom, expr)
+ if binding
+ # On a successful match, allow the rule action to transform the
+ # parslet into something new.
+ ctx = Parslet::Context.new(binding)
+ return ctx.instance_eval(&action)
+ end
+ end # rules.each
+
+ # If no rule matches, this is the fallback - a clean new parslet atom.
+ return atom
+ end
+ end
+end
+
+require 'parslet/context'
View
2 lib/parslet/accelerator/engine.rb
@@ -2,6 +2,7 @@
require 'parslet/atoms/visitor'
module Parslet::Accelerator
+ # @api private
class Apply
def initialize(engine, expr)
@engine = engine
@@ -69,6 +70,7 @@ def match(type_tag)
end
end
+ # @api private
class Engine
attr_reader :bindings
View
3 lib/parslet/transform/context.rb → lib/parslet/context.rb
@@ -10,7 +10,8 @@
# a # => :b
# end
#
-class Parslet::Transform::Context < BlankSlate
+# @api private
+class Parslet::Context < BlankSlate
reveal :methods
reveal :respond_to?
reveal :inspect
View
12 lib/parslet/source.rb
@@ -53,6 +53,18 @@ def consume(n)
def chars_left
@str.rest_size
end
+
+ # Returns how many chars there are between current position and the
+ # string given. If the string given doesn't occur in the source, then
+ # the remaining chars (#chars_left) are returned.
+ #
+ # @return [Fixnum] count of chars until str or #chars_left
+ #
+ def chars_until str
+ slice_str = @str.check_until(Regexp.new(Regexp.escape(str)))
+ return chars_left unless slice_str
+ return slice_str.size - str.size
+ end
# Position of the parse as a character offset into the original string.
# @note: Encodings...
View
6 lib/parslet/transform.rb
@@ -116,8 +116,6 @@ class Parslet::Transform
# context?
include Parslet
- autoload :Context, 'parslet/transform/context'
-
class << self
# FIXME: Only do this for subclasses?
include Parslet
@@ -233,4 +231,6 @@ def recurse_hash(hsh, ctx)
def recurse_array(ary, ctx)
ary.map { |elt| apply(elt, ctx) }
end
-end
+end
+
+require 'parslet/context'
View
10 qed/accelerators.md
@@ -71,8 +71,8 @@ And all is fine, right? We don't think so. You've chosen to use parslet, so you
parser = quote >> (quote.absent? >> any).repeat >> quote
A = Accelerator # for making what follows a bit shorter
- optimized_parser = A.apply(
- A.rule( (A.str(:x).absent? >> any).repeat ) { GobbleUp.new(x) })
+ optimized_parser = A.apply(parser,
+ A.rule( (A.str(:x).absent? >> A.any).repeat ) { GobbleUp.new(x) })
optimized_parser.parse('"Parsing is now fully optimized! (tm)"')
@@ -115,6 +115,11 @@ Note how the internal regular expression value used for the match atom is really
Also the Accelerator matcher for `Parslet.match` is called `Accelerator.re` - the reason for this should be obvious from what stands above.
+Just to make this complete, a special case for the `match(...)` atom is the `any` atom.
+
+ binding = Accelerator.match(any, Accelerator.any)
+ binding.assert != nil
+
## Composite Parsers
Let's start assembling these simple parsers into more complex patterns and match those. As our first pattern, we'll consider sequences.
@@ -171,7 +176,6 @@ Another property should be that literal strings passed to the pattern should be
The binding is empty here, since no variables were given. But lets also implement constrained variable bindings, that seems useful. The way this works is that you specify a variable you want to bind to first, and then a list of constraints that are matched by `#===`.
- A = Accelerator
A.match(str('abc'), A.str(:x, /c/))[:x].assert == 'abc'
A.match(str('abc'), A.str(:x, /d/)).assert == nil
View
26 qed/applique/gobbleup.rb
@@ -0,0 +1,26 @@
+
+require 'parslet'
+
+# This is where the famous GobbleUp atom is defined. This parslet atom consumes
+# all chars until a given string (absent) is encountered.
+#
+class GobbleUp < Parslet::Atoms::Base
+ def initialize absent, min_chars=0
+ @absent = absent
+ @min_chars = min_chars
+ end
+
+ def try(source, context, consume_all)
+ excluding_length = source.chars_until(@absent)
+
+ if excluding_length >= @min_chars
+ return succ(source.consume(excluding_length))
+ else
+ return context.err(self, source, "No such string in input: #{@absent.inspect}.")
+ end
+ end
+
+ def to_s_inner(prec)
+ "until('#{@absent}')"
+ end
+end
View
2 spec/acceptance/examples_spec.rb
@@ -11,7 +11,7 @@ def product_path(str, ext)
gsub('example/','example/output/')
end
- it "runs successfully", :ruby => 1.9 do
+ it "runs successfully" do
stdin, stdout, stderr = Open3.popen3("ruby #{example}")
handle_map = {
View
2 spec/acceptance/regression_spec.rb
@@ -184,7 +184,7 @@ class UnicodeSentenceLanguage < Parslet::Parser
rule(:sentences) { sentence.repeat }
root(:sentences)
end
- describe UnicodeSentenceLanguage, :ruby => 1.9 do
+ describe UnicodeSentenceLanguage do
let(:string) {
"RubyKaigi2009のテーマは、「変わる/変える」です。 前回の" +
"RubyKaigi2008のテーマであった「多様性」の言葉の通り、 " +
View
7 spec/parslet/source_spec.rb
@@ -52,6 +52,11 @@
end
end
end
+ describe '#chars_until' do
+ it 'should return 100 chars before line end' do
+ source.chars_until("\n").should == 100
+ end
+ end
describe "<- #column & #line" do
subject { source.line_and_column }
@@ -134,7 +139,7 @@
end
- describe "reading encoded input", :ruby => 1.9 do
+ describe "reading encoded input" do
let(:source) { described_class.new("éö変わる") }
def r str
View
2 spec/parslet/transform/context_spec.rb
@@ -1,6 +1,6 @@
require 'spec_helper'
-describe Parslet::Transform::Context do
+describe Parslet::Context do
def context(*args)
described_class.new(*args)
end

0 comments on commit 66e2428

Please sign in to comment.
Something went wrong with that request. Please try again.