docs-tools -- Buildsystem Components for MongoDB Documentation Projects
docs-tools holds all common build system components for MongoDB
documenting projects (e.g. the MongoDB Manual, MongoDB Ecosystem
Documentation, etc.) These tools include: Sphinx extensions, the
meta-build system, themes, deployment and orchestration scripts, and
scripts that generate common restructured text elements.
The goal of
docs-tools is to totally remove all programs from the
documentation source trees to facilitate maintenance of these
components in the context of a multi-branch and multi-repository
This document explains the componets and operations of the build system, and serves as a guide for anyone who wants to maintain a MongoDB Documentation-like project.
fabfile (orchestration and scripts)
fabric is a deployment orchestration system
that provides simple and convenient Python interfaces for shell
scripting and task running.
docs-tools uses fabric tasks for two
- to deploy build products to testing and production environments.
- to bundle groups of related operations in a consistent and simple interface.
Historically, these operations were a mixture of multi-step makefile targets, shell scripts, and Python programs. Combining all of these operations into a single interface increases portability, consistency, and overall reliability of the build systems.
makecloth (meta-build system)
Makecloth is a
meta-build tool that generates build specifications
ninja.build files.) Makecloth makes it
possible to take advantage of the dependency resolution and
concurrency features of a robust and stable buildsystem while mostly
writing Python rather than a build tool specific specification.
docs-tools implementation, Makecloth scripts generate
buildscripts based on project-specific data supplied with the source
data, so that many projects can use common logic. See the build data
for the MongoDB Manual
for an example.
rstcloth (rst generation)
rstcloth use the model of
makecloth but generates reStructuredText: In some situations, it
makes sense to generate
rst programatically rather than maintains
that content manually.
- to make tables, it often makes more sense to generate the table based on more generic data than to build the table directly in restructured text, given the flexibility of the table syntax.
- to keep multiple related pieces of content in sync, it often makes sense to generate when two pieces of content derive from the same source but need different reStructuredText forms.
- to enforce common formatting requirements centrally for forms that appear throughout the documentation.
This approach allows minimally invasive generation of output for multiple formats and reduces the amount of custom template logic.
This directory contains conventional Sphinx extensions, required for producing and referencing MongoDB documentation projects. This includes the extensions for the MongoDB Domain to cross reference MongoDB objects (database commands, shell methods, database output, etc.) as well as custom admonition and directive types.
These are standard Sphinx themes customized for MongoDB documentation projects, centralized here for ease of use across projects.
dtf is a consistency testing framework for documentation projects to provide ways of enforcing conventions in documentation. The content of these scripts just include the test logic, and not the testing running architecture, or data used to define testable situations.
The scripts in the
bin directory include common components and
logic shared between bootstrapping operations, fabric scripts, and
is an example bootstrapping script that intializes a
Integration and Getting Started
See the mongodb manual repository
for an example of an integrated repository using
docs-tools. In particular, note the build_conf.yaml
file: this is the only strict requirement for
Copy the bootstrap.py
script in the top level of your repository and create a
docs_meta.py file in the
bin/ directory of your
repository. You can now run the following command to bootstrap your
To begin using the meta-build system, specify a generated makefile in
build.system.files list in the
docs_meta.py file. This
will correspond to one of the scripts in the
docs-tools. Then in your
makefile, include the following
output = build tools = $(output)/docs-tools .PHONY:$(output)/makefile.meta -include $(output)/makefile.meta build/makefile.meta:$(tools)/makecloth/meta.py @mkdir -p $(output) @python $< $@
tools variable if you downloaded
docs-tools to an
alternate location. You may also include the following bootstrapping
bootstrap fabfile build/docs-tools: @python bootstrap.py @echo "[bootstrap]: configured build environment."
Some of the makecloth files require a corresponding
yaml file in
the directory specified in the
build.paths.builddata field of
To generate the run
make without any arguments.
Extension and Development
There are several ways to add functionality to the build system:
- add additional tasks to the
fabfilemodules according to standard fabric development practices.
- add additional rstcloth generated files. These scripts typically take inputs via arguments on the command line or a file passed in on the command line. For integration, add corresponding makecloth scripts to generate build instructions.
- add additional makecloth files for additional processing and build
products. Add generated makecloth files to the
docs_meta.yamlto generate a new makefile.
- improve documentation of each makecloth's purpose and use.
- expand documentation of makecloth data forms and schema.
- improve symlink handling throughout makecloth (for windows compatibility.)
- stabilize makecloth and rstcloth and move them out of the tree.
- build helpers for working with development virtualenvs.
- make build system fully Python 3 compatible.