Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Comparing changes

Choose two branches to see what's changed or to start a new pull request. If you need to, you can also compare across forks.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also compare across forks.
base fork: junniest/bach_test_repo
base: efe1c4ef2c
...
head fork: junniest/bach_test_repo
compare: 4dbe06c2a7
Checking mergeability… Don't worry, you can still create the pull request.
  • 2 commits
  • 4 files changed
  • 0 commit comments
  • 1 contributor
View
2  paper/Makefile
@@ -3,7 +3,7 @@ DOC := paper
PKG := $(DOC)-$(shell date +%F).tar.gz
all: paper.pdf
-paper.pdf: paper.tex intro.tex parser-model.tex dynamic-extensions.tex
+paper.pdf: paper.tex intro.tex parser-model.tex dynamic-extensions.tex paper.bib
pdflatex $(DOC).tex
bibtex $(DOC)
pdflatex $(DOC).tex
View
12 paper/paper.bib
@@ -31,7 +31,13 @@ @misc{mli
HowPublished = {http://www.ml1.org.uk/}
}
-@article{veldhuizen,
- Author = {Todd L. Veldhuizen},
- Title = {C++ Templates are Turing Complete}
+@techreport{veldhuizen,
+ abstract = {{We sketch a proof of a well-known folk theorem that C++
+templates are Turing complete. The absence of a formal
+semantics for C++ template instantiation makes a rigorous
+proof unlikely.}},
+ author = {Veldhuizen, Todd L.},
+ keywords = {c, computational-models, example, fun, template},
+ title = {{C++ Templates are Turing Complete}},
+ year = {2003}
}
View
5 paper/paper.tex
@@ -4,7 +4,8 @@
\usepackage[utf8x]{inputenc}
\usepackage[russian, english]{babel}
\usepackage{color}
-\usepackage[colorlinks,bookmarksopen,bookmarksnumbered,citecolor=red,urlcolor=red]{hyperref}
+\usepackage[colorlinks,bookmarksopen,bookmarksnumbered,%
+ citecolor=red,urlcolor=red]{hyperref}
\usepackage{graphicx}
\usepackage{listings}
@@ -12,7 +13,7 @@
\newcommand{\codesize}{\fontsize{8}{8}\selectfont}
\lstset{% general command to set parameter(s)
language=C,
- basicstyle=\codesize, % print whole listing small
+ basicstyle=\codesize, % print whole listing small
%keywordstyle=\color{black}\bfseries,
% underlined bold black keywords
identifierstyle=, % nothing happens
View
100 paper/parser-model.tex
@@ -1,19 +1,24 @@
\section{\label{sec:parser}Parser model}
-Our work is concerned with a dynamic grammar modification on the
-fly, and as a base of our approach we are going to consider an
-LL(k) recursive descent parser with a certain properties.
+The parser which is a basis for building a preprocessor on is
+based on recursive descent LL(k) or LL(*) algorithm. Recursive
+descent is a natural human approach to write parsers, and in
+case if $k$ is small, the efficiency of parsing is linear with
+respect to the number of tokens on the input stream.
-As a running example in this paper we are going to use an imaginary
-language with a C-like syntax. Consider a grammar of the language.
+As a running example in this paper we are going to use a
+grammar of a language with C-like syntax described in
+Fig.~\ref{fig:grammar}.
+\begin{figure}[h!]
+\centering
\begin{verbatim}
program ::= ( function ) * ;
function ::= type-id '(' arg-list ')' stmt-block ;
arg-list ::= ( type-id id ) * ;
stmt-block ::= '{' ( expr | return ';' ) * '}' ;
expr ::= fun-call | assign | cond-expr ;
-fun-call ::= id '(' ( expr ) * ')' ;
+fun-call ::= id '('expr (',' expr ) * ')' ;
assign ::= id '=' expr ;
cond-expr ::= bin-expr '?' cond-expr ':' expr ;
bin-expr ::= bin-expr binop primary-expr
@@ -21,33 +26,66 @@ \section{\label{sec:parser}Parser model}
binop ::= '&&' | '||' | '==' | '!=' ... ;
prefix-op ::= '-' | '+' | '!' | '~' ;
\end{verbatim}
+\caption{\label{fig:grammar}A grammar of a C-like language.}
+\end{figure}
-% These are the requirements.
-First of all we ask, that every production is represented as a function
-with a signature \verb/Parser -> (AST|Error)/, i.e. function gets a
-parser-object on input and returns either an AST node or an error.
-We would call those functions handle-functions. We require that
-handle-functions structure mimic a formulation of the grammar,
-i.e. if a production A depends on a production B, we require
-function handle-A to call function handle-B.
-
-Each handle-function implements error recovery (if needed) and takes
-care about disambiguating productions according to the language
-specification, resolving operation priorities, syntax ambiguities
-and so on. Each handle function has an access to the parser, which
-keeps has an internal state, which changes when a handle-function
-is applied. In a some sense an application of a handle-function
-is a reduce step of a shift-reducer.
-
-Each handle-function is paired with a predicate function which checks
-whether a sequence of tokens pointed by a parser-state matches a
-given rule. This type of functions we will call is-functions.
-Application of an is-function does not modify the state of
-the parser. Is-functions may require unbounded look-ahead from
-the parser, which also happens to be a requirement. We assume that in
-order to resolve complicated ambiguities unbounded look-ahead is needed
-anyways, as language expressions normally allow unbounded nesting.
+The preprocessor is build as an extension to the parser, so
+it expect a certain behaviour from the parser.
+\begin{description}
+ \item[Token stream] The parser should conceptually represent
+ a stream of tokens as a doubly linked list, which is possible
+ to traverse in either direction and perform a substitution of
+ a token group with another token group, and restarting a
+ stream from an arbitrary position. The implementation details
+ are left to the creators of the parser.
+
+ \item[Pseudo tokens] The parser normally reduces the grammar rule
+ by reading tokens from the input stream. We introduce a notion
+ of pseudo-token, which conceptually is an atomic element of the
+ input stream, but which represents a reduced grammar rule. The
+ implementation details are left to the parser creator. The most
+ straight forward and inefficient way would be to convert the
+ pseudo-token into the token stream and parse again.
+
+ \item[Handle functions] First of all we ask, that every production
+ is represented as a function\footnote{Note, that these functions
+ have side-effects, so the order of calling is important.} with a
+ signature \verb/Parser -> (AST|Error)/, i.e. function gets a
+ parser-object as an input and returns either an Abstract Syntax Tree
+ node or an error. We call those functions handle-functions. We
+ require that handle-functions structure mimic a formulation of the
+ grammar, i.e. if a production A depends on a production B, we
+ require function handle-A to call function handle-B.
+
+ Each handle-function implements error recovery (if needed) and takes
+ care about disambiguating productions according to the language
+ specification, resolving operation priorities, syntax ambiguities
+ and so on. Each handle function has an access to the parser, which
+ keeps has an internal state, which changes when a handle-function is
+ applied.
+ %In a some sense an application of a handle-function
+ %is a reduce step of a shift-reducer.
+
+ \item[Is functions] Each handle-function is paired with a predicate
+ function which checks whether a sequence of tokens pointed by a
+ parser-state matches a given rule. This type of functions we will
+ call is-functions. Application of an is-function does not modify
+ the state of the parser. Is-functions may require unbounded
+ look-ahead in general case, however we leave the implementation
+ decision to the parser creator. One can always reuse matched AST
+ nodes to perform subsequent matches.
+
+ \item[Match function] In the beginning of each handle-function, each
+ production calls a function called \verb|match| with a signature
+ \verb/(Parser, Production) -> Parser/. A match-function is an
+ interface to the preprocessor which checks if a stream of tokens
+ pointed by the parser has a valid substitution in a given
+ production; if it does it performs a substitution and makes sure
+ that the parser token stream starts with that substitution. In
+ case \verb|match| simply returns a parser, then the preprocessor\
+ would not perform any substitutions.
+\end{description}
Assuming that all the requirements are met, the grammar $G = (N, T, P,
S)$ provides a full information required to build a support for

No commit comments for this range

Something went wrong with that request. Please try again.