Browse files

[README] modifications; added STATUS and COOLTHINGS

  • Loading branch information...
masak committed Dec 27, 2009
1 parent d431fc5 commit 81bf4d7f70db90cf2fb99515e0ade5034e623d20
Showing with 159 additions and 8 deletions.
  1. +63 −0 COOLTHINGS
  2. +12 −8 README
  3. +84 −0 STATUS
@@ -0,0 +1,63 @@
+One of the really cool things about Perl 6 is that it contains the means
+of parsing its own source code into an abstract syntax tree, which can
+then be inspected and modified in various ways from inside the language
+That dream still hasn't come true, though. GGE is being built with the
+explicit purpose of showing what can be done when Perl 6 can read and
+modify Perl 6 on the syntax tree level. Like proto, it's more of a
+proof-of-concept than a long-term solution.
+The purpose of this file is to share a bit of the enthusiasm, by listing
+the set of possible applications that become possible with a fully
+functioning GGE.
+In order of increasing ambitiousness:
+= A CLI grammar debugger
+Basically, a clone of the debugging feature in Damian Conway's
+Regexp::Grammars. This is possible because GGE can easily be made to contain
+hooks in the right places for debugging.
+= A grammar debugger
+An interactive web form where people modify their grammar and get feedback
+on a given input in real time, seeing what happens when the grammar tries
+to match the input text. Specifically, the user will see where the match
+*fails*, which is something that is missing from the current set of tools.
+The applications below also require that we have a fully operational Perl
+6 parser on top of GGE.
+= A Perl 6 macro preprocessor
+A simple filter which parses macros and applies them to the source code,
+generating a new source code with the macros applied -- and which even a
+Rakudo not implementing macros can run. A sort of a 'have the cake and
+eat it' scenario.
+= A static code analyzer
+This is one of my favorites. I have a list of things I'd like to check
+statically in Perl 6 code, such as mis-use of some features, possible
+(or certain!) value errors, unreachable code etc. Sort of a Perl::Critic
+for Perl 6.
+= A runtime
+Running Perl 6 on top of Perl 6? The thought is too attainable to resist.
+= A coverage tool
+As long as we have a runtime, we might as well construct it so that it
+keeps track of coverage information.
+= A time-travelling debugger
+As long as we have a runtime, we might as well construct it so that it
+can be rewinded and fast-forwarded at will. This is a goal only reached
+a few times in computer history? Wondering why that variable got the
+value it did? No need to re-run the program; just rewind it to its last
+assignment. Changed a value in the interactive prompt? No prob; we'll
+just re-run things from that point onwards.
@@ -1,11 +1,11 @@
== Glacial Grammar Engine -- like PGE, but slower
Ok, so there was some guy called Patrick who implemented a thing called
-PGE, right? It was this grammar engine written in PIR, the assembly
-language of Parrot. PGE was the first thing in the known universe that
-could parse Perl 6 regexes. It was then plugged into PCT, a toolkit for
-building HLL (Higher-Level Language) compilers for Parrot. Rakudo, a Perl 6
-compiler, was then built on top of PCT.
+PGE, right? It was a grammar engine written in PIR, the assembly language of
+Parrot. PGE was the first thing in the known universe that could parse Perl 6
+regexes. It was then plugged into PCT, a toolkit for building HLL (Higher-Level
+Language) compilers for Parrot. Rakudo, a Perl 6 compiler, was built on
+top of PCT.
In short, a whole lot of abstractions, and in the bottom of it all, driving
the parser, sits PGE. Humming along, it does all the daily parsing we now
@@ -27,9 +27,9 @@ slower. Much slower. Hence the name.
The "albeit not too hard" reservation to the implementation-independence
stems from the fact that some PIR tricks used by PGE are unavailable in
Perl 6. When faced with a choice of (a) giving up, and (b) cheating by
-introducing some PIR-dependent workaround, the implementor of GGE will
-choose the latter road. That said, implementation-independence stands as
-a goal whenever possible, just as with all honest Perl 6 code.
+introducing some PIR-dependent workaround, the latter road will be chosen.
+That said, implementation-independence stands as a goal whenever possible,
+just as with all honest Perl 6 code.
The "though perhaps not too convincingly" reservation to GGE attempting to
be a drop-in replacement for PGE stems from the fact that now that regular
@@ -40,6 +40,10 @@ Perl 6 -- one not using any regexes at all, directly or indirectly -- but
that is not the kind of fun optimized for in this case. Future teams of
developers are welcome to optimize for that kind of fun, should they wish.
+For an overview of the status of the project, see STATUS. For an overview
+of the cool things that could be built with the project as a dependency,
== License
Code coming from the Parrot repository is Artistic License 2.0. See
@@ -0,0 +1,84 @@
+PGE contains three regex engines: a Perl 6 one, a Perl 5 one, and one for
+file globbing. The current focus on GGE is to port the Perl 6 regex engine
+enough to be able to parse arbitrary Perl 6 grammars (to the extent that
+PGE can).
+The implementation work proceeds in order of the PGE test files. The
+rx_quantifiers was the first one to be attacked, because quantifiers seemed
+like great fun. A few tests have been marked TODO in that file, pending
+re-entrant regex parsing which stops on a specified precedence level.
+The OPTable tests were next. Only one test was marked TODO in there; it
+depends on the :parsed argument accepting a GGE::Perl6Regex object as its
+value. (No real obstacle to that one; just haven't found it necessary.)
+All tests pass in rx_metachars. Currently, work is ongoing in rx_charclass,
+with no real obstacles in sight.
+There are two features I can see that need to be implemented for the rest
+of the tests to pass. The first is some kind of stacking of TreeSpider
+objects. This is needed both for lookarounds and for subrule calls, both
+with form a sort of 'regex in a regex', both statically and dynamically.
+The second is a mechanism for building real Match objects, replete with
+numbered and named captures.
+name tests status
+rx_quantifiers 179 DONE (a few todo tests remain)
+t/03-optable.t 42 DONE (one todo test remains)
+rx_metachars 238 DONE
+rx_charclass 63 in progress
+rx_modifiers 102 to be done
+rx_captures 48 to be done
+rx_subrules 60 to be done
+rx_backtrack 23 to be done
+rx_lookarounds 33 to be done
+rx_goal 11 to be done
+rx_syntax 33 to be done
+Here's a copy of PGE's STATUS, adapted for GGE:
+Perl 6 rule features implemented
+* standard regular expression constructs
+* quantifiers (*, +, ?, *?, +?, ??, **{m..n}?)
+* alternation (|)
+* conjunction (&)
+* non-capturing groups
+* enumerated character lists (<[abcdef]>, <-[a..f]>)
+* character classes (., \d, \w, \s, \n, \e, \f, \r, \t, \x[hhh], \o[nnn])
+* anchors (^, $, ^^, $$, \b, \b)
+* \c[...] and \C[...]
+Perl 6 rule features to-do
+* capturing groups
+* modifiers :i, :ignorecase, :s, :sigspace (with synonyms :w and :words)
+* composed character classes (<+alpha-[aeiou]>)
+* negated character classes
+* capturing subrules (<expr>)
+* non-capturing subrules (<.expr>)
+* negated subrules (<!expr>)
+* lookahead and lookbehind (<before> and <after>)
+* built-in rules: <null>, <fail>, <alpha>, <digit>, <alnum>,
+ <upper>, <lower>, <xdigit>, <space>, <print>, <graph>,
+ <blank>, <cntrl>, <punct>, <sp>, <lt>, <gt>, <dot>, <ws>,
+ <p6rule>, <p5regexp>
+* backreferences ($0, $1, $<foo>)
+* scalar subrule and subpattern aliases ($<foo>:=(...), $0:=(...))
+* balanced delimited text (<GGE::Text::bracketed>)
+* embedded closures ({{ puts "hello" }})
+* :lang() modifier
+* firstchar/nextchar optimizations
+* cut operators (:, ::, :::, <commit>)
+* lookbehind according to S05 definition
+* <cut> assertion
+* versions of <commit> and cuts that release backtracking state
+* :perl5 modifier
+* literal assertions (<'literal'>)
+* interpolated assertions (<"literal">)
+* array and hash aliasing (@<foo>:=(...), %<foo>:=(...))
+* don't generate unneeded internal data structures (e.g., gpad)

0 comments on commit 81bf4d7

Please sign in to comment.