mlp is a tool for extracting text from the code blocks in markdown files. It allows programmers to write in a literate programming style using markdown as the source language.
mlp is a self-contained Go program written using the LP paradigm. The source is committed alongside the markdown source of this repository for bootstrapping purposes.
You require the Go language if you don't already have it.
To build the tool:
git clone https://github.com/driusan/mlp
cd mlp
go buildThis will build the binary named mlp for your platform in the current
directory. You can use the -o $path argument to go build to build
the binary in a different location. (i.e. go build -o ~/bin/ to put the
binary in ~/bin/.)
This repo also comes with a shell.nix file. While an existing version is included, like mlp itself, this is mainly for bootstrapping purposes. To compile it, use mlp Nix-Shell.md
To observe mlp at work, put this file in an empty directory, cd to that
directory, and run mlp README.md. Now look in the directory and you'll see
files extracted from the code blocks alongside this markdown file. In
literate programming lingo, this extraction is (somewhat counterintuitively)
called "tangling." Generating documentation from the source is called
"weaving", and mlp leaves that to existing markdown renderers (such as
the GitHub frontend.)
mlp is language agnostic. The below demonstration of features is written
in (very trivial) C++ to demonstrate using other languages.
The markup for the code block below starts with ```cpp > hello.cpp:
<<<copyright>>>
<<<includes>>>
int main() {
<<<body of main>>>
}The header says 3 things:
cpp: the code block is written in C++. In the rendered markdown output, that affects syntax highlighting, to mlp it means that language-appropriate pragma directives will be added so that when debugging the extracted code, your debugger will show you the line in the original markdown source file. (If you don't want this effect, you can just use an unrecognized language name likecxx).> hello.cpp: The code block will be written to the filehello.cpp.
The files are generated by mlp in the current directory by default. The -o option
may be used to generate the files in another directory. For example
mlp -o /tmp/test README.mdgenerates the files in /tmp/test.
When no name is specified after the > in the block code, mlp uses the
input file name with the language as extension to generate the output file.
For example ```py >
def foo():
print("foo")generates a file README.md.py. This is especially convenient when the
document is used to generate a test that checks the validity of the code
in the document.
The <<<string>>> sequences in the body of the code block are called
"macro references." An mlp "macro" is just a variable whose value can be
extracted from one or more code blocks, and will be substituted wherever
its name appears in triple angle brackets on a line. There are no arguments
to mlp macros.
If we were to run mlp on this file at this point, we would get the warnings:
Warning: Block named copyright referenced but not defined.
Warning: Block named includes referenced but not defined.
Warning: Block named body of main referenced but not defined.
This allows us to stub in a macro reference whenever we want in our code,
and only later define them in whatever order best fits our prose. When there
are no more warnings, the hello.cpp file should build (assuming we didn't
include any syntax or other compiler errors.)
The markup for the code block below starts with ```cpp "body of main"
std::cout << "Hello, world!" << std::endl;The double quotes around body of main mean that the code block will be
extracted into a macro of that name. You can see where its value will be
injected into hello.cpp via <<<body of main>>>,
above.
mlp uses quotation marks to differentiate between macros and file
destinations. If a name is encased in quotes, it's a macro, if not, it's
a file.
mlp parses each file passed on the command line in order. The last
definition of a macro will be used for all references to that macro in
other code blocks (including blocks which preceded it in the source.)
There are two includes in this program. The second is appended to the first.
#include <iostream>followed by
#include <numeric>The cpp includes macro value is now:
#include <iostream>
#include <numeric>(the code block above is not being tangled).
Hidden content
The raw markdown in this file contains a comment containing a code block with a copyright notice. It looks a bit like this one:
<!--
```cpp "copyright"
// Copyright 42 BCE not the actual copyright
```
-->
If you're reading the rendered markdown in your browser, you can't see the
actual comment, but it still gets tangled into the copyright macro, which is
substituted into hello.cpp by the <<<copyright>>> macro reference. This
technique lets you tangle content that you don't want showing up in the
documentation.
We can tangle into a random data file (```csv > data.csv)
foo, bar, baz,
qix, qux, quux,You need to specify both a language and a destination (macro or file) if you want the code block tangled:
No destination, but includes syntax highlighting (```cpp)
auto x = "This doesn't get tangled anywhere";But any language string and filename (```arbitrary > foo.txt) will do
This gets tangled
into foo.txt.
Running mlp on this file at this point should generate the files data.csv
and foo.txt with the expected contents and produce no warnings.
mlp is a fork of tlm that mostly changes the syntax and the default append behavior.
Some more minor changes are also being implemented when the need comes.
tlm is primarily authored by Dave MacFarlane (@driusan). Bryan Allred (@bmallred) improved the
parsing code to include the metadata in the code block header rather than a
rendered markdown header. @mek-apelsin
wrote the patch to include pragmas for line numbers, and Dave Abrahams
(@dabrahams) wrote the demo of features in
this README, making it more user-focused (it previously dove straight into
implementation.)