Compiler

Matt Bierner edited this page May 16, 2015 · 9 revisions
# Compile file to stdout
$ khepri file.kep

# Get help
$ khepri --help

# Compile file to output file
$ khepri file.kep -o build/file.js

# Generate node package code in output
$ khepri file.kep -o build/file.js --package_manager=node

# Compile all *.kep files in a directory to an output directory
$ khepri lib -o dist

Input and Output

stdin and stdout

The default behavior of the Khepri compiler is to read khepri input from stdin and write ECMAScript output to stdout. This allows the Khepri compiler to be used in standard pipelines.

$ echo "\x -> x;" | khepri --package_manager=node | wc

Input File

An Khepri input file may be provided as the first argument to the compiler:

# read my_file.kep and write to stdout
$ khepri my_file.kep

If the input file is a directory, then all sub *.kep files in that directory will be compiled:

# read all kep files recursively in lib, writing to stdout
$ khepri lib/

Output File

The -o flag specifies an output file

# read my_file.kep and write to my_file.js
$ khepri my_file.kep -o my_file.js

When the input file is a directory and the output file is a directory, then all input file is written to files in the output directory relative to the input file.

# Consider this directory structure
lib/
    file1.kep
    non_kep_file.js
    file2.kep
    other_dir/
        file3.kep
$ khepri lib -o dist
# Will generate the following structure for output files.
dist/
    file1.js
    file2.js
    other_dir/
        file3.js

File Watching

Instead of directly compiling input to output, the Khepri compiler can watch a file or directory for changes and auto compile changed files with the -w flag.

# Auto compile `my_file.kep` to `my_file.js` whenever `my_file.kep` changes
$ khepri -w my_file.kep -o my_file.js

All the standard flags and options will work, including watching an input directory and compiling to an output directory

# Auto compile any changed file in `lib` to an output file in `dist`
$ khepri -w lib -o dist

Flags

--help

$ khepri --help

Display help message with command line options overview

--version

$ khepri --version

Print the version of Khepri used.

--package_manager=(node|amd)

$ khepri lib -o dist --package_manager=amd

Specify which package manager Khepri should generate output for.

--prune

$ khepri lib -o dist --prune

Prune unreachable symbols. Defaults to false.

Why is the Compiler So Damn Slow?

When I wrote the Khepri compiler, I asked myself, "Is it possible to implement a language transformer in Javascript using only functional programming techniques"? A fully functional implementation turns out not only to be possible, but also a fairly elegant solution for many problems. But it does have a serious performance tradeoff.

A few interesting points about the compiler:

  • The AST is never mutated. Instead, the compiler uses Neith to zipper the AST and transform it immutably.
  • In fact, mutation is not used internally by the compiler at all.
    • Even data structures such as maps are stored in using an immutable HAMTs.
  • The compiler uses Akh monads to simply implementation. Again, these monadic computations do not mutate anything, and also require a huge number of function calls to evaluate (specifically the state monad).
    • Normally all these calls would blow up the stack, so Akh has to use a trampoline that can invoke thunks in fixed stack space.

Additionally, Khepri is parsed and lexed using Bennu parser combinators. These also require a huge number of function calls for continuation passing. Especially for lexing, using parser combinations may be a bit silly.

The end result is a fully functional compiler. It does not perform too well, but has a rather interesting implementation.

Performance was never goal of this project. If anyone is interested enough in the language and concerned about performance, please consider forking the existing compiler code and making use of more standard Javascript conventions.

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.