Browse files

More README love.

  • Loading branch information...
1 parent b75f59f commit bf34df68dc1b480ef232807410be84c8b1b6fc61 @vic committed Dec 24, 2010
Showing with 110 additions and 3 deletions.
  1. +5 −1
  2. +11 −2 lib/typhon/
  3. +83 −0 lib/typhon/compiler/
  4. +11 −0 lib/typhon/environment/
@@ -43,9 +43,13 @@ Maybe Rubinius VM does what Parrot was originally intended to.
# Try -C --print-all on your python script.
# This will most likely blow up and show you
- # a hint of what is needed to be implemented.
+ # a hint of what is needed to be implemented.
$ ./bin/typhon -C --print-all
+ # You can use the --print-all switches with
+ # the REPL like this:
+ $ ./bin/typhon --print-all
## Status
@@ -1,4 +1,13 @@
-This directory contains a Python compiler implemented in ruby
-using Rubinius compiler chain.
+Typhon implementation
+This directory contains all ruby source files implementing the Typhon
+compiler and runtime environment.
+How Typhon works
+Everything start with a very basic `bin/typhon` ruby script. It loads
+the typhon `compiler.rb`, runtime `environment` and executes the command line
+interface `main.rb`
@@ -0,0 +1,83 @@
+Typhon Compiler
+This directory contains a Python compiler implemented in ruby
+using Rubinius compiler chain.
+Compiling Python
+The main interface to the Typhon compiler is by using the methods implemented
+in `compiler.rb`. `Typhon::Compiler` exposes functions to compile .py
+files used as modules and python code string intended to be
+evaled. The compiler is implemented in stages.
+Compiling is made in stages, implemented in `stages.rb`.
+## Parsing ##
+The first step is to turn your python program (an string representing
+valid python code) into a machine-representable format. This is called
+_parsing_, and in current Typhon it's implemented by Python's own
+parsing module. The `bin/` script is a tiny Python program
+that simply outputs a _sexp_ representation of the source code. The
+_sexp_ is made of only basic ruby literals, that is arrays, symbols,
+strings, numbers and nil.
+`parser.rb` exposes a simple `Typhon::Parser` module that just opens a
+pipe to the `` program and returns the evaluated _sexp_.
+This stage is implemented by `PyFile` and `PyCode` that essentially
+just use `Typhon::Parser`.
+## AST ##
+The next step is turn the _sexp_ (an array of ruby literals) into
+actual AST Nodes. An AST Node is a more logical representation of
+what's being done in your program, for example, there's an AST Node
+for every string literal, for import statements, for method
+invocation, etc.
+Once the AST (Abstract Syntax Tree) is complitely built, we are ready
+to proceed with compilation.
+This stage is implemented by `PyAST`
+## AST Transformations ##
+Next stage is doing AST transformations, that is, adapting the AST
+tree for better suitting the purposes of python semantics.
+All python programs are represented by a module object in python, and
+because of this, the result of parsing any valid python code is a tree
+having a root with type ModuleNode. However, in some cases, like
+evaling a python expession with the _eval_ builtin function, we just
+dont need nor want to create a new module for the code being
+evaluated, instead, we want the code to be evaled on the current
+context. For this end, Typhon has a `EvalExpr` stage that performs a
+very simple transformation for an AST tree intended for eval.
+In case of an eval, the `EvalExpr` stage simply removes the top
+`ModuleNode` and makes sure the last expression returns a value
+(unwraps any final `DiscardNode`).
+If the AST tree was not intended for evaling, this stage does nothing.
+## Bytecode Generator ##
+This stage is where actual compilation takes place, it's implemented
+in the `Generator` class. All it does is call the `bytecode` method on
+the top of the AST tree. Every AST node is responsible for producing
+the Rubinius bytecode for it's own.
+See the _ast_ directory to find all AST nodes.
+## Rest of Rubinius compiler chain ##
+After the `Generator` stage, all we do is feed the produced generator
+to Rubinius Encoder. So we rely on Rubinius doing bytecode analysis
+and validation, also on writing compiled .rbc files to the file system
+or producing an EvalExpression object for evals.
@@ -0,0 +1,11 @@
+Runtime Environment
+A Python to Rubinius-bytecode would be pretty much useless without a
+way of running that as a valid Python program. To this end, Typhon
+provides a full Python runtime environment, that is, core modules,
+builtin function, everything you need to run your Python program on
+# TODO: document how python object model is implemented in Typhon.
+# Add links to design/algorithm docs if needed.

0 comments on commit bf34df6

Please sign in to comment.