Lilac is an addon for Emacs Org mode to make Literate Programming (LP) using <a href=”glossary: Noweb”>Noweb-style [cite:@ramsey_1994] references easy. It comes in two parts:
lilac.el: to be loaded and used when /weaving/ [cite:@knuth_1984 98] and /tangling/ [cite:@knuth_1984 98] from your own Org mode files (which use Noweb references), andlilac.theme: to be sourced by your Org mode files to use Lilac’s CSS and JavaScript on top of the HTML export.
This document itself reuses the lilac.theme file, which gives you a preview
(if you are viewing the HTML output) of what weaving with Lilac looks like.
NOTE: Do not view this page using GitHub’s default Org file viewer, because many sections will appear broken. Instead, please view this page by visiting https://funloop.org/lilac, or by cloning the repository and pointing your browser to the top-level index.html file directly.
Lilac sacrifices some degree of customization in Org mode in order to make it easier to write literate programs using Noweb-style references. Lilac also focuses on HTML export for weaving.
The following are some examples of how things are done in the default Org mode settings, and how Lilac does things differently.
By default, source code blocks do not get any captions in the HTML output. This can be an annoyance for LP because you have to manually explain the name of each block to keep track of their usage in other blocks (when referring to them with Noweb style references).
Lilac frees you from having to write a custom #+caption: ... text for every
source code block, because it generates them based on the name of the Noweb
reference. Lilac links child blocks and parent blocks together so that you can
click on code block names to navigate around this parent/child hierarchy.
While Org mode allows source code blocks to refer to other blocks (Noweb references), it does not do any kind of linking. So you can write blocks like
#+name: example-parent-block
#+begin_src bash
echo "Hello from the parent block"
<<some-other-block>>
#+end_src
...
#+name: some-child-block
#+begin_src bash
echo "I am child 1"
#+end_src
but when this is exported to HTML neither example-parent-block nor
some-child-block have any links to each other. This is frustrating because it
makes navigating between them difficult.
Org mode does not create link anchors for headings, which makes referring to them from other external documents slightly annoying. Org mode does not add link anchors to source code blocks.
Lilac automatically links child blocks and parent blocks together so that you can click on code block names to navigate around this parent/child hierarchy.
Also, link anchors are added to all source code blocks as well as headlines, and they are saved in your browser’s history as you click around, such that you can go back/forward and be taken back to these intra-document locations. This makes intra-document navigation more structured to help prevent you from getting lost.
Org mode generates random link ID’s on every export. That is, even if you change nothing in your Org mode file, if you export to HTML there will be a diff because all of the linked objects will get a new link ID.
Lilac makes HTML generation deterministic. This makes it VCS and CI-friendly because the output changes if and only if the input changes. Speaking of CI-friendliness, Lilac was designed to be run from outside of Emacs by default (and not interactively).
When exporting to HTML, the Table of Contents (TOC) is at the top of the document.
Lilac puts the TOC in a sidebar on the left. It also uses some JavaScript to check where the current mouse position is to update the current location in the TOC. That is, the TOC follows you as you scroll down the document.
Side notes are used to insert notes on the right-hand-side margin, and are inspired by the ones found in Donald Knuth’s books.
They are not supported in Org mode.
Lilac supports the #+begin_sidenote and #+end_sidenote delimiters. Text
inside such delimiters are shown on the right hand margin.
First create an Org mode file that uses LP. Then either clone this repo or add
it as a submodule to your project. The point is to get the lilac.el (and its
dependencies, themselves submodules) and lilac.theme (and associated
CSS/JavaScript files) available locally. Then for the Org mode file you are
using, you can load lilac.el and run (lilac-publish) to generate the HTML
file, or run (org-babel-tangle) to generate the source code from it. See the
Makefile that this project uses as a reference.
The main thing that you need to keep in mind when writing Org mode files for
consumption by Lilac is that the source code blocks must use Noweb references
using __NREF__ as a prefix. See the discussion below.
In Org mode, Noweb-style references by default must be enclosed in double angle
brackets << and >>. While this works, it’s problematic because it can mean
different things syntactically based on the source code language. Your source
code block’s language might think that the angle brackets are operators or
keywords and colorize them differently, for example.
Instead, Lilac expects Noweb-style references in the form __NREF__foo (where
the “NREF” stands for “Noweb reference”). Then you are free to name your child
block with this same __NREF__foo name. This is better because now you can
search for this word __NREF__foo in your raw Org mode document and you’ll
instantly be able to see where it is used. Contrast this with the default Org
mode behavior where you’ll have to search for <<foo>> and foo separately
(because searching for just foo may collide with other names that are not
source code block names).
In the example below, the parent-block refers to 2 other child blocks for its
definition.
#+name: parent-block
#+begin_src bash
echo "Hello from the parent block"
__NREF__child-block-1
__NREF__child-block-2
#+end_src
...
#+name: __NREF__child-block-1
#+begin_src bash
echo "I am child 1"
#+end_src
...
#+header: :noweb-ref __NREF__child-block-2
#+begin_src bash
echo -n "I am "
#+end_src
#+header: :noweb-ref __NREF__child-block-2
#+begin_src bash
echo "child 2"
#+end_src
This example illustrates the two ways to define a child block: either as a
single code block with #+name: __NREF__foo, or as multiple blocks with
#+header: :noweb-ref __NREF__foo. Lilac calls them monoblocks and polyblocks
respectively. Polyblocks are concatenated together in the order they appear in
the overall Org file; this final concatenated version is what gets inserted into
the Noweb reference in the parent block.
Below is an example of the usage described above. Notice how the child blocks referenced in the parent block are linked to their definitions.
echo "Hello from the parent block"
__NREF__example-child-block-foo
echo ""
__NREF__example-child-block-barBelow are the child blocks. The first is a monoblock. Every child block’s name links back up to the parent block where it is referenced.
echo "I am child 1"The blocks below are polyblocks.
echo -n "I am "Note the fraction after the name which denotes the position of the block in the overall polyblock “chain”.
echo "child 2"Sometimes a piece of code will be reused in different source code blocks. You can think of it as constants in a programming language, but in LP the concept extends to any arbitrary piece of text.
Below is an example where we define a child block once…
echo "Hello from child block baz"…but where we also reuse it (reference it) in different parent blocks.
echo "Hello from parent 1"
__NREF__example-child-block-bazecho "Hello from parent 2"
__NREF__example-child-block-bazNotice that the child block has an additional link named “2”, which points to the second parent that references this block (the first parent is linked by the child block name itself, as in the previous example). If it is referenced in additional parent blocks, they will show up as links “3”, “4”, etc.
Here we have some examples of how certain Org mode features look when exported
to HTML with Lilac. Each section has an orgmode input and HTML output. The input
is what you type into your *.org (plaintext) file, and the output is what you
get in HTML after it is rendered with Lilac’s publishing function,
(lilac-publish).
Orgmode input:
#+begin_src python
def foo():
return 42
#+end_src
Rendered HTML output:
def foo():
return 42Orgmode input:
#+caption: A parent block.
#+begin_src python
def foo():
__NREF__ex_monoblock
return 42
#+end_src
#+name: __NREF__ex_monoblock
#+begin_src python
print("hello world")
#+end_src
Rendered HTML output:
def foo():
__NREF__ex_monoblock
return 42print("hello world")Sometimes it’s useful to evaluate code and to use the resulting value. This is in contrast to always inserting the contents of the source code blocks literally.
However you must make sure that the referenced source code block’s programming
language is added to the list of org-babel-do-load-languages. Otherwise the
evaluation will be forbidden.
Orgmode input:
#+name: __NREF__ex_evaluate_me
#+begin_src python :exports none
return 4 * 10 + 2
#+end_src
#+caption: This uses an evaluation result.
#+begin_src python :noweb yes
print("The answer is __NREF__ex_evaluate_me()")
#+end_src
Rendered HTML output:
print("The answer is __NREF__ex_evaluate_me()")Here we have an example of passing in an argument to a source code block.
Orgmode input:
#+name: __NREF__ex_take_argument
#+begin_src python :exports none :var my_var="foo"
return "Hello, my name is " + my_var
#+end_src
#+caption: Passing string args.
#+begin_src python :noweb yes
print("Call without explicit argument: __NREF__ex_take_argument()")
print("Call with explicit argument: __NREF__ex_take_argument(my_var="bar")")
#+end_src
Rendered HTML output:
print("Call without explicit argument: __NREF__ex_take_argument()")
print("Call with explicit argument: __NREF__ex_take_argument(my_var="bar")")Orgmode input:
#+caption: A parent block.
#+begin_src python
def foo():
__NREF__ex_polyblock
return 42
#+end_src
#+header: :noweb-ref __NREF__ex_polyblock
#+begin_src python
print("hello world")
#+end_src
Some intervening text.
#+header: :noweb-ref __NREF__ex_polyblock
#+begin_src python
print("abcdefg")
#+end_src
Rendered HTML output:
def foo():
__NREF__ex_polyblock
return 42print("hello world")Some intervening text.
print("abcdefg")Orgmode input:
#+begin_src python -r -l "#ref:%s"
print("foo")
print("hi") #ref:embedded-link
print("bar")
#+end_src
The line at [[(embedded-link)][=embedded-link=]] prints "hi".
Rendered HTML output:
print("foo")
print("hi") #ref:embedded-link
print("bar")The line at =embedded-link= prints “hi”.
Orgmode input:
#+begin_example
Hello world.
Foo bar.
#+end_example
Rendered HTML output:
Hello world. Foo bar.
Orgmode input:
#+begin_quote
Hello world.
Foo bar.
#+end_quote
Rendered HTML output:
Hello world. Foo bar.
Note that Lilac only adds some special CSS styling to side notes. Orgmode is
generous with source code blocks and accepts any #+begin_foo and #+end_foo
lines.
Orgmode input:
#+begin_sidenote
Hello world.
Foo bar.
#+end_sidenote
Rendered HTML output (see right hand margin):
This is not a Lilac feature (it’s just part of Org mode), but it’s worth demonstrating how it looks.
Orgmode input:
The equation \( E = mc^2 \) is probably the most famous equation in the world,
perhaps more famous than the Pythagorean theorem
\[
a^2 + b^2 = c^2.
\]
The equation
\[
\frac{\pi^2}{6} = \sum_{n=1}^\infty \frac{1}{n^2} = \
\frac{1}{1^2} + \frac{1}{2^2} + \frac{1}{3^2} + \cdots
\]
was first demonstrated by Leonhard Euler in 1734. Here's another equation,
#+name: eqn:1
\begin{equation}
\pi = \sum_{k = 0}^{\infty}\left[\frac{1}{16^k} \
\left(\frac{4}{8k+1}-\frac{2}{8k+4}-\frac{1}{8k + 5}-\frac{1}{8k+6}\right)\right]
\end{equation}
which is known as the Bailey-Borwein-Plouffe formula and can compute arbitrary
digits of \(\pi\). A variant of this equation is as follows
#+name: eqn:2
\begin{equation}
\pi = \frac1{2^6} \sum_{n=0}^\infty \frac{(-1)^n}{2^{10n}} \
\left(-\frac{2^5}{4n+1} - \frac1{4n+3} + \frac{2^8}{10n+1} - \
\frac{2^6}{10n+3} - \frac{2^2}{10n+5} - \frac{2^2}{10n+7} + \
\frac1{10n+9} \right)
\end{equation}
and is known as the Bellard's formula. Equation [[eqn:2]] is too wide so
MathJax breaks it over multiple lines. Below is a version where we do the line
breaking ourselves.
\begin{equation}
\pi = \frac1{2^6} \sum_{n=0}^\infty \frac{(-1)^n}{2^{10n}} \
\left(-\frac{2^5}{4n+1} - \frac1{4n+3} + \frac{2^8}{10n+1} - \\
\frac{2^6}{10n+3} - \frac{2^2}{10n+5} - \frac{2^2}{10n+7} + \
\frac1{10n+9} \right)
\end{equation}
Rendered HTML output:
The equation \( E = mc^2 \) is probably the most famous equation in the world, perhaps more famous than the Pythagorean theorem
\[ a^2 + b^2 = c^2. \]
The equation
\[ \frac{π^2}{6} = ∑n=1^∞ \frac{1}{n^2} = \ \frac{1}{1^2} + \frac{1}{2^2} + \frac{1}{3^2} + \cdots \]
was first demonstrated by Leonhard Euler in 1734. Here’s another equation,
\begin{equation} π = ∑k = 0∞\left[\frac{1}{16^k} \ \left(\frac{4}{8k+1}-\frac{2}{8k+4}-\frac{1}{8k + 5}-\frac{1}{8k+6}\right)\right] \end{equation}
which is known as the Bailey-Borwein-Plouffe formula and can compute arbitrary digits of \(π\). A variant of this equation is as follows
\begin{equation} π = \frac1{2^6} ∑n=0^∞ \frac{(-1)^n}{210n} \ \left(-\frac{2^5}{4n+1} - \frac1{4n+3} + \frac{2^8}{10n+1} - \ \frac{2^6}{10n+3} - \frac{2^2}{10n+5} - \frac{2^2}{10n+7} + \ \frac1{10n+9} \right) \end{equation}
and is known as the Bellard’s formula. Equation eqn:2 is too wide, though. Below is a version where we insert manual line breaks.
\begin{equation}
π = \frac1{2^6} ∑n=0^∞ \frac{(-1)^n}{210n} \
\left(-\frac{2^5}{4n+1} - \frac1{4n+3} + \frac{2^8}{10n+1} -
\frac{2^6}{10n+3} - \frac{2^2}{10n+5} - \frac{2^2}{10n+7} + \
\frac1{10n+9} \right)
\end{equation}
These limitations stem from the fact that they are not natively supported by Org mode.
- Source code blocks must reference other blocks in the same file
- If you’re using source code blocks, and they refer to other blocks, those referenced blocks must be defined in the same Org file. In practice this isn’t a big problem.
If you are interested in contributing back to this project, or if you are curious to see how everything works, have a look at the Developer Guide.
- <<glossary: literate document>> literate document
- A file or collection of
files that include both source code and prose to explain it. Well-known
formats include Noweb files (
*.nw) and Org mode files (*.org). - <<glossary: monoblock>> monoblock
- an Org mode source code block with a
#+name: ...field. This block is an independent block and there are no other blocks with the same name. - <<glossary: Noweb>> Noweb
- A literate programming tool from 1989 that still works and from which Org mode borrows heavily using Noweb-style references. See Wikipedia.
- <<glossary: noweb-ref>> noweb-ref
- aka “Noweb-style reference”. A Noweb-style reference is just a name (string) that refers to a monoblock or polyblock. See the Org manual.
- <<glossary: Org mode>> Org mode
- An Emacs major mode for
*.orgfiles, where “major mode” means that it provides things like syntax highlighting and keyboard shortcuts for*.orgtext files if you are using Emacs. For Lilac, the important thing is that we use Org mode as a literate programming tool. See Org mode. - <<glossary: polyblock>> polyblock
- an Org mode source code block without a
#+name: ...field, but which has a#+header: :noweb-ref ...field. Other blocks with the same Noweb-ref name are concatenated together when they are tangled. Polyblocks are used in cases where we would like to break up a single block into smaller pieces for explanatory purposes. In all other cases, monoblocks are preferable, unless the source code block is not to be tangled and is only for explanatory purposes in the woven output. - <<glossary: source code block>> source code block
- An Org mode facility
that allows you to enclose a multiline text (typically source code) with
#+begin_src ...and#+end_srclines. They are enclosed in a separate background color in the HTML output, and are often used for illustrating source code listings. The format is#+begin_src LANGUAGE_NAMEwhereLANGUAGE_NAMEis the name of the programming language used for the listing. If the name is a recognized name, it will get syntax highlighting in the output automatically. - <<glossary: tangling>> tangling
- The act of extracting source code from a raw literate document.
- <<glossary: weaving>> weaving
- The act of converting a raw literate document to a richer format such as PDF or HTML. This allows fancier output, such as for mathematical formulas, which are easier to read versus the original literate document.