Code from my series on writing a Ruby compiler in Ruby
Ruby Gherkin Other
Failed to load latest commit information.
examples Example used for register allocation article Nov 5, 2013
features Yet more parsing edge cases that needs a cleanup (as does this code..… Nov 10, 2016
lib Remove special handling of empty range for Array#[] Jan 28, 2017
test Added self-test case that breaks in clean_method_name when compiled w… Nov 3, 2016
.gitignore Additional ignore patterns Sep 5, 2015
AUTHORS added my email adress to AUTHORS file Apr 15, 2009
Dockerfile Update Dockerfile to compile with MRI 2.1 Aug 26, 2016
Gemfile Make bundler stop whining about ssl Jul 15, 2014
Gemfile.lock Make bundler stop whining about ssl Jul 15, 2014
Makefile Update 'clean' target; update 'compile' target to reflect that the 'c… Aug 26, 2016 Expanded the README with details on problem areas Sep 11, 2016
Rakefile Updated outdated Rakefile Mar 6, 2014
arrayoutput.rb Slight refactoring Sep 5, 2015
ast.rb Workaround for variable rewrite bug Aug 29, 2016
atom.rb Refactoring of tokenizer Sep 25, 2014
classcope.rb For selfhosting, as compiler doesn't handle ||= very well yet Aug 29, 2016
compile Modified 'compile' script to spawn within a Docker container; updated… Aug 21, 2016
compile_arithmetic.rb Implement s-exp 'mod' keyword (modulus) in terms of div May 17, 2015
compile_calls.rb More aggressive invalidation of 'self' register allocation Aug 29, 2016
compile_class.rb Handle "break" in bare blocks and proc's, as well as in control Sep 1, 2016
compile_comparisons.rb Remove hardcoded register one more place Sep 15, 2013
compile_control.rb Basic support for 'next' Oct 30, 2016
compile_include.rb Adding constant resolution for modules. Started preparing to refactor… Sep 8, 2015
compiler.rb Rewriting Compiler#clean_method_name for a couple of reasons: Nov 12, 2016
controlscope.rb Basic support for 'next' Oct 30, 2016
cucumber Run cucumber on a given feature file in the docker build container Nov 10, 2016
debugscope.rb Adding constant resolution for modules. Started preparing to refactor… Sep 8, 2015
driver.rb Add --dumsymtabs option to dump symbol tables, and improve :deref dia… Nov 23, 2014
emitter.rb Basic support for 'next' Oct 30, 2016
extensions.rb Temporarily inlined 'rest' into Array since we don't yet support 'inc… Nov 12, 2016
funcscope.rb Fixhandling of multi-level class scopes Feb 3, 2015
function.rb Handle "break" in bare blocks and proc's, as well as in control Sep 1, 2016
globals.rb Add munging of global function names on re-definition, and avoid re-c… Oct 9, 2014
globalscope.rb Stub out start of support for , $:, bash, ARGV Sep 11, 2016
iooutput.rb Another temporary workaround for a compiler bug Nov 12, 2016
localvarscope.rb Refactored *Scope classes into separate files Sep 16, 2014
operators.rb Added more selftests; Drastically lowered priority of splat operator.… Nov 3, 2016
output_functions.rb Make output_functions *actually* ensure stack space for default varia… Sep 20, 2015
parser.rb Basic support for 'next' Oct 30, 2016
parserbase.rb Using Parserbase#expect to require a keyword breaks because it is not… Aug 29, 2016
print_sexp.rb Improve s-expression output Sep 15, 2015
quoted.rb Support %w(words split on whitespace) Aug 31, 2015
regalloc.rb Add checks for minimum/maximum arguments to a method; fixes register … Apr 16, 2014
register.rb Dependency on this accidentally made it into a commit; since I want t… Nov 8, 2013
saveregs.rb Fix return type for saveregs Nov 19, 2014
scanner.rb Hacking workaround of a bug for self-hosting purposes Jan 28, 2017
scope.rb Basic support for 'next' Oct 30, 2016
sexp.rb Handle any valid method name in %s() syntax Jun 6, 2013
sexpscope.rb Missing inheritance from new Scope base class Sep 25, 2014
shunting.rb Handling the finer (uglier) aspects of Ruby whitespace and parentheses Sep 1, 2016
splat.rb Fixed a stupid off by one regression Feb 3, 2015
stackfence.rb Fixed typo Aug 28, 2013
sym.rb Parsing ':[]=' didn't 'get' the final '=', causing parsing errors Sep 8, 2016
tokenizeradapter.rb Call back into tokenizer to correctly parser '%' as distinct prefix/i… May 25, 2015
tokens.rb Basic support for 'next' Oct 30, 2016
trace.rb Allow debug methods to force trace() to output something for programs… Aug 1, 2013
transform.rb Handle cases of methods with both a splat and a named block argument,… Nov 3, 2016
treeoutput.rb Yet more parsing edge cases that needs a cleanup (as does this code..… Nov 10, 2016
utils.rb reformatted the code to have a somewhat more homogeneous feel. Apr 13, 2009
value.rb Trivial class to hold typed value Jun 27, 2014
vtableoffsets.rb Refactored *Scope classes into separate files Sep 16, 2014

Writing a (Ruby) compiler in Ruby

Source for my series on writing a compiler in Ruby.


NOTE This is still wildly incomplete.


This section covers caveats about compiled Ruby vs. MRI, not generally missing pieces or bugs in the current state of the compiler (of which there are many).


Presently, "require" is evaluated statically at compile time.

This makes certain Ruby patterns hard or impossible to support. E.g. reading the contents of a directory and caling "require" for each .rb file found will not presently work, and may never work, as it is not clear in the context of compilation whether or not the intent is to load this file at compile time or runtime.

Ruby allows the argument to "require" to be dynamically generated. E.g. "require File.dirname(FILE) + '/blah'". To facilitate compatibility, limited forms of this pattern may eventually be supported.

On MRI, "require" is generally overridden by a custom version for rubygems or bundler. This is not likely to ever be supported. "require" is likely to be treated as a keyword, rather than as an overrideable method.


While $0 will at some point be initialized with the name of the file compilation is triggered for, certain patterns of Ruby, such as conditionally executing code based on whether a given file is executed directly are conceptually different, given that $0 gets bound at compile time.


The load path is malleable in MRI, and this is very frequently used alongside certain methods to modify which files may be loaded. Currently this is not supported.

It is likely that for compatibility a limited subset of Ruby will be interpreted at compile time to support some forms of this pattern. See also "require"