tastr: Type AST for R

tastr is a library that implements a grammar for R types. It provides functions to parse types from input streams (files, strings, etc.) to C++ objects. It also provides the reverse functionality to unparse these objects. The Type AST is modeled using C++ classes. The design relies heavily on unique ownership, i.e., an AST node holds unique references to its child nodes. The grammar is implemented using flex and bison.

Installation

To obtain the project source, run:

$ git clone https://github.com/aviralg/tastr.git

To build the project, run:

$ make

This will build the following artifacts:

• Static library: build/lib/libtastr.a
• Shared library: build/lib/libtastr.so
• Executable: build/bin/tastr

To clean the build artifacts, run:

$ make clean

To execute tests, run:

$ make test

For building and testing in parallel, invoke make with -j flag.

Usage

The tastr executable (build/bin/tastr) parses type declarations from different sources and unparses them back.

> build/bin/tastr --help
Parse type declarations.
Usage:
tastr [OPTION...]

-h, --help    Print usage
-l, --lexer   Debug lexer
-p, --parser  Debug parser
-a, --ast     Show AST structure
-s, --style   Style output

Render input

Pass the filename to render the input as is (retains spaces, newlines and comments):

Pass --style flag to render with styling.

To read the input from stdin pass - argument (press Ctrl-D to simulate end of input).

Input can also be directly parsed from the argument if it is quoted using " or '.

View AST structure

Use --ast flag to view the ast structure without any styling.

Use --style flag along with --ast to view the ast structure with styling.

Type Grammar

<integer>: "integer"
         | "int"
         | "i"

<double>: "double"
        | "dbl"
        | "d"

<complex>: "complex"
         | "clx"
         | "x"

<character>: "character"
           | "chr"
           | "s"

<logical>: "logical"
         | "lgl"
         | "l"

<raw>: "raw"
     | "r"

<ascalar>: <integer>
         | <double>
         | <complex>
         | <character>
         | <logical>

<nascalar>:  "^" <ascalar>

<scalar>: <ascalar>
        | <raw>
        | <nascalar>

<environment>: "environment"
             | "env"
             | "t"

<expression>: "expression"
            | "exr"
            | "e"

<language>: "language"
          | "lng"
          | "g"

<symbol>: "symbol"
        | "sym"
        | "y"

<externalptr>: "externalptr"
             | "ept"
             | "p"

<bytecode>: "bytecode"
          | "bcd"
          | "b"

<pairlist>: "pairlist"
          | "plt"

<s4>: "s4"

<weakref>: "weakref"
         | "wrf"
         | "w"

<dataframe>: "dataframe"
           | "df"

<any>: "any"
     | "*"

<unknown> : "???"

<vector>: <scalar> "[" "]"

<list>: "list" "<" <type> ">"
      | "lst" "<" <type> ">"

<struct>: "struct" "<" <namedtypeseq> ">"
        | "srt" "<" <namedtypeseq> ">"
        | "struct" "<" ">"
        | "srt" "<" ">"

<tuple>: "tuple" "<" <typeseq> ">"
       | "tpl" "<" <typeseq> ">"
       | "tuple" "<" ">"
       | "tpl" "<" ">"

<typeseq>: <type>
         | <typeseq> "," <type>

<namedtype>: <identifier> ":" <type>
           | "^" ":" <type>

<namedtypeseq> : <namedtype>
               | <namedtypeseq> "," <namedtype>

<param>: <type>
       | "..."

<paramseq>: <param>
          | <paramseq> "," <param>

<params>: "<" <paramseq> ">"
        | "<" ">"
        | "any"

<function>: <params> "=>" <type>

<group>: "(" <type> ")"

<nonunion>: <scalar>
          | <environment>
          | <expression>
          | <language>
          | <symbol>
          | <externalptr>
          | <bytecode>
          | <pairlist>
          | <s4>
          | <weakref>
          | <dataframe>
          | <vector>
          | <function>
          | <struct>
          | <list>
          | <tuple>
          | <group>
          | <unknown>

<union>: <nonunion>
       | <union> "|" <nonunion>

<null>: "?"
      | "?" <union>

<type>: <union>
      | <null>
      | <any>

<decl>: "type" <identifier> <type> ";"

<declseq>: <decl>
         | <declseq> <decl>

Useful Resources

Bison and Flex

• Flex Token Order
• UTK Bison Notes
• Parsing rule for empty files
• Dynamic file loading in Flex
• Redefine YY_INPUT in Flex
• C++ istream with Flex
• Bison/Flex Example
• Read from C++ streams in Flex
• Lex and Yacc Tutorial
• Parsing strings in Flex
• Bison/Flex Example
• Unique Pointer as a Semantic Value in Bison
• Resolving Grammar Conflicts
• Bison/Flex Example
• Bison/Flex Example
• Bison/Flex Example
• Bison/Flex Example
• Bison/Flex Example

C++: Unique Pointers and Polymorphism

• Arguments and Unique Pointers
• Smart Pointer Parameters
• Move Constructors and Multiple Inheritance
• Copy Constructors and Unique Pointer
• Rules for Smart Pointers
• Using Smart Pointers as Class Members
• Cloning, Covariant Return and Smart Pointer
• How I Declare my Class and Why
• Polymorphic Cloning and CRTP
• Delegating Constructors
• == operator and Polymorphism
• Move constructors and inheritance
• Move assignment operator and inheritance
• Move assignment on self
• C++ final keyword

Make and Code Compilation

• Compiling a static library in linux
• Compiling C/C++ software

R

• Various uses of quoting in R

Miscellaneous

• Unicode Characters in the 'Punctuation, Open' Category
• http://www.fileformat.info/info/unicode/block/control_pictures/images.htm
• https://www.htmlsymbols.xyz/arrow-symbols

TODO

1. Check clone_impl, it should be empty for abstract base classes and defined for concrete classes.

   \u[0-9A-Fa-f]{1,4} { get_identifier().push_back(yytext); } \U[0-9A-Fa-f]{1,8} { get_identifier().push_back(yytext); } \u{LBRACE}[0-9A-Fa-f]{1,4}{RBRACE} { get_identifier().push_back(yytext); } \U{LBRACE}[0-9A-Fa-f]{1,8}{RBRACE} { get_identifier().push_back(yytext); }

   \[0-7]{2} { get_identifier().append("0"); get_identifier().push_back(yytext[1]); get_identifier().push_back(yytext[2]); } \[0-7]{3} { get_identifier().append(yytext); }