This is a compiler for the Jack language, the high-level language of the Hack platform, an exercise of the book The Elements of Computing Systems, by Nisan and Schocken.
The Jack language is introduced on Chapter 9, and Chapters 10 and 11 are dedicated to building a simple compiler for such language, which is what this project is.
This compiler was completely written in Common Lisp.
This software is beta quality, and is currently on its road to perform full compilations of Jack programs for Chapter 11 compliance.
This project uses qlot for both development and unit testing, so one should be able to use this tool to either run or develop the compiler.
Qlot assumes that you have Roswell installed, preferably running SBCL. To install and configure qlot, simply run:
ros install qlot cd /path/to/cl-jackc qlot install
To use qlot along with SLIME on Emacs, see qlot’s repository documentation.
This project depends on a number of Common Lisp systems, which are specified below. All of them can are fetched from Quicklisp’s main repository.
Compilation and usage:
This project takes a different approach when compared to other projects of the book (cl-hackasm, cl-hackvmtr), with respect to its organization. The book talks about compiler modules, and so the project is organized in modules as well, though not strictly as suggested.
Each one of the project’s packages correspond to a certain module of the compiler. When doing this, instead of having separate binaries, we just make sure that the packages are minimalistic and serve their purpose explicitly. The user shall only interface with a single package, which depends on the other modules and works as a frontend.
Default interface for compiler. Exports procedures so the user can interact with the compiler.
Utilities and miscellaneous structures for all other compiler modules, usually related to language extension.
Definitions for compiler-related conditions.
Capable of reading specific tokens and handling a file stream, while also holding the language grammar specification.
Relates to the JackTokenizer module, proposed in the book, except that the generation of the parsing tree was passed to the analyzer.
Takes a single file stream, and matches it with the language grammar, generating a syntax tree. Redirects the output to the parser.
Relates mostly to the JackAnalyzer module, proposed in the book.
Takes the output of a file analysis, and effectively compiles it to one of the desired outputs (XML, VM or SEXP), outputting it to console.
Relates to the CompilationEngine module, proposed in the book.
Takes the console output of the parser, and writes it to the desired (
Takes compilation arguments and configuration, passed by the user. For each given file, generates a file stream and passes it to the analyzer.
There are three ways to use the compiler:
roswell/jackc.ros, the default compiler script, which can be used from command line with Roswell;
run-jackc.sh, which directly invokes the compiler script for command line, using a Qlot context;
cl-jackcpackage, which works as an interface to the compiler.
Please notice that this software is still beta quality and is not ready for usage.
Installing from Roswell
It is possible to install this program directly from Roswell, though it is not recommended while the software is at beta stage.
ros install luksamuk/cl-jackc
After the required operations, the script
jackc should be available for
use from command line:
jackc /path/to/files/and/dirs [args...]
jackc, any separate given path is treated as a single,
different project. Each file of a project can be separately
compiled. However, if a directory name is passed instead of the path
to a Jack code file,
jackc will attempt to find all Jack files on the
first level of such directory and compile each one of them. Each
compiled file will be written in the same directory as the code file.
As for extra arguments,
--sexp, which indicate
specific syntax analysis steps. When both are informed,
precedence. More information can be seen on the compiler’s help
jackc does not demand any order of arguments, so one can safely
interleave arguments and paths when using the script.
Passing no arguments shows a usage and help text.
UPDATE 2019-08-25: At this date, the newest version of
broken on Quicklisp repositories and could not be properly
loaded. This is why the unit testing now uses fixed versions for some
systems. I cannot guarantee that these libraries will not break if you
install them using plain Roswell; should anything happen, plese refer
to the bundled script.
Using the bundled script
Given that Roswell and Qlot are installed, and that Qlot is correctly
configured in the project’s directory, the compiler can be invoked
from command line using the
./run-jackc.sh /path/to/files/and/dirs [args...]
More information on script usage can be found in the previous subsection (Installing from Roswell).
Using from REPL
cl-jackc from a REPL (SLIME, Roswell, Qlot or any other
means), provided that it has access to the
cl-jackc system (loadable
using Quicklisp), just load and use the default interface:
(ql:quickload :cl-jackc) (cl-jackc:compile-exec "/path/to/file/or/dir" :analyze case)
COMPILE-EXEC takes a single path, which can point to a single file or
directory. The key argument
:ANALYZE specifies whether the file should
be analyzed; if so, the user may pass one of the
:ANALYZE can also be ignored. If so, it defaults to
indicates that a full compilation of the file(s) should be done, and
not just syntax analysis.
Notice that, while it is already possible to invoke a full compilation for a path, the compiler will fail unless a syntax analysis case is specified, due to compilation progress (this will change shortly).
Any compilation or syntax analysis files generated are saved with the same name of its Jack file, except for its extension, on the same folder of the source code.
This project uses rove for unit testing. For using it, a system
cl-jackc/test is provided, which includes a number of test
suites for parts of the compiler.
The test system is comprised of the following files:
parser-test, which contains tests for the whole analyzer, and compares the exported ASTs;
analyzer-test, which contains tests for checking whether the compilation runs without problems on valid files;
tokenizer-test, which tests for the ability of the tokenizer head to find specific and rule-based tokens on a character stream.
Tests also assume that you have
Running tests automatically
run-tests.sh script is included on the repository root, and will
automatically run all tests when invoked.
Running tests manually, from Slime
If you are hacking the project’s files, and you have a Slime REPL open
qlot; for that, see
qlot’s documentation), just invoke
the testing system:
(ql:quickload :cl-jackc/test) (rove:run :cl-jackc/test)
Running tests manually, from Bash
Assuming that qlot is installed and configured, navigate to the project’s root directory and open the REPL:
When the Lisp REPL is opened, load the test system and use rove to run the unit tests:
(ql:quickload :cl-jackc/test) (rove:run :cl-jackc/test)
For a brief documentation, check the generated documentation file.
Below are links to more information related to this project.
- [Portuguese] Integrando testes automáticos para Common Lisp com Docker e Travis CI
- [English] A grammar language based on S-expressions (Org / TeX / PDF)
This project is distributed under the MIT License.
Copyright (c) 2019 Lucas Vieira.