Browse files

Update for expanded manubot package (#48)

* Update for expanded manubot package
* Create distinct caching directory
* Deployment: references branch now output
* Rewrite documentation
* Rename to

Refs manubot/manubot#2
  • Loading branch information...
dhimmel committed Aug 9, 2017
1 parent 64f3ddb commit 084e30d97cdc9997953c2099d8d81062bc48fb1c
@@ -1,12 +1,11 @@
# Generated manuscript output files

# Generated reference files

# Manubot cache directory

# Python
@@ -20,7 +20,7 @@ script:
- sh build/
- references/generated
- ci/cache
- test $TRAVIS_BRANCH = "master" &&
test $TRAVIS_PULL_REQUEST = "false" &&

This file was deleted.

Oops, something went wrong.
@@ -15,10 +15,36 @@ To see what's incoming, check the [open pull requests](
Instructions for using Manubot Rootstock for your own manuscript are still evolving.
The recommended approach is to clone this repository, as detailed [here](

## Source
## Repository directories & files

The manuscript source is located in [`content`](content).
Text should be written in markdown files, with a digit prefix for ordering (e.g. `01.`, `02.`, etc.) and a `.md` suffix.
See [``]( for documentation on how to create or contribute to a manuscript.

The directories are as follows:

+ [`content`](content) contains the manuscript source, which includes markdown files as well as inputs for citations and references.
+ [`output`](output) contains the outputs (generated files) from the manubot including the resulting manuscripts.
You should not edit these files manually, because they will get overwritten.
+ [`webpage`](webpage) is a directory meant to be rendered as a static webpage for viewing the HTML manuscript.
+ [`build`](build) contains commands and tools for building the manuscript.
+ [`ci`](ci) contains files necessary for deployment via continuous integration.
For the CI configuration, see [`.travis.yml`](.travis.yml).

## Local execution

To run the Manubot locally, install the [conda]( environment as described in [`build`](build).
Then, you can build the manuscript on POSIX systems by running the following commands.

# Activate the manubot conda environment
source activate manubot
# Build the manuscript
sh build/
# View the manuscript locally at http://localhost:8000/
cd webpage
python -m http.server

## Continuous Integration

@@ -28,7 +54,7 @@ When you make a pull request, Travis CI will test whether your changes break the
The build process aims to detect common errors, such as invalid references.
If your build fails, see the Travis CI logs for the cause of failure and revise your pull request accordingly.

When a pull request is merged, Travis CI performs the build and writes the results to the [`gh-pages`]( and [`references`]( branches.
When a pull request is merged, Travis CI performs the build and writes the results to the [`gh-pages`]( and [`output`]( branches.
The `gh-pages` branch hosts the following URLs:

+ **HTML manuscript** at
@@ -49,7 +75,7 @@ All files matched by the following glob patterns are dual licensed under CC BY 4

+ `*.sh`
+ `*.py`
+ `*.yml`
+ `*.yml` / `*.yaml`
+ `*.json`
+ `*.bib`
+ `*.tsv`
@@ -60,5 +86,6 @@ All other files are only available under CC BY 4.0, including:
+ `*.md`
+ `*.html`
+ `*.pdf`
+ `*.docx`

Please open [an issue]( for any question related to licensing.
@@ -0,0 +1,134 @@
# Manubot usage guidelines

This repository uses the [Manubot]( to automatically produce a manuscript from its source.

## Manubot markdown

Manuscript text should be written in markdown files, which should be located in [`content`](content) with a digit prefix for ordering (e.g. `01.`, `02.`, etc.) and a `.md` extension.

For basic formatting, check out the [CommonMark Help]( page for an introduction to the formatting options provided by standard markdown.
In addition, manubot supports an extended version of markdown, tailored for scholarly writing.

### Tables

Manubot supports [markdown tables]( "GitHub Help: Organizing information with tables").

| Column 1 | Column 2 | Column 3 |
| value_a | 1 | 47 |
| value_b | 2 | 56 |

Table: Caption for this example table. {#tbl:example-id}

Support for table numbering and citation is provided by [`pandoc-tablenos`](
Above, `{#tbl:example-id}` sets the table ID, which creates an HTML anchor and allows citing the table like `@tbl:example-id`.
For easy creation of markdown tables, check out the [Tables Generator]( webapp.

### Figures

Figures can be included with the following markdown:

![Caption for the example figure.](url_or_path_to_figure){#fig:example-id}

Support for figure numbering and citation is provided by [`pandoc-fignos`](
This figure can be cited in the text using `@fig:example-id`.
In context, a figure citation may look like: `Figure {@fig:example-id}B shows …`.

For images created by the manuscript authors that are hosted elsewhere on GitHub, we recommend using a [versioned]( GitHub URL to embed figures, thereby preserving exact image provenance.
When embedding SVG images hosted on GitHub, passing the URL through [RawGit]( is necessary.
An example of a URL that has been passed through RawGit is:


Figures placed in the [`content/images`](content/images) directory can be embedded using their relative path.
For example, we embed an [ORCID]( icon inline using:

![ORCID icon](images/orcid.svg){height="13px"}

The bracketed text following the image declaration is interpreted by Pandoc's [`link_attributes`]( extension.
For example, the following will override the figure number to be "S1" and set the image width to 5 inches:

{#fig:supplement tag="S1" width="5in"}

### Citations

Manubot supports Pandoc [citations]( via `pandoc-citeproc`.
However, Manubot performs automated citation processing and metadata retrieval on raw citations.
Therefore, citations must be of the following form: `@source:identifier`, where `source` is one of the options described below.
When choosing which source to use for a citation, we recommend the following order:

1. DOI (Digital Object Identifier), cite like `@doi:10.15363/thinklab.4`.
2. PubMed ID, cite like `@pmid:26158728`.
3. _arXiv_ ID, cite like `@arxiv:1508.06576v2`.
4. URL / webpage, cite like `@url:`.

Cite multiple items at once like:

Here is a sentence with several citations [@doi:10.15363/thinklab.4; @pmid:26158728; @arxiv:1508.06576].

Note that multiple citations must be semicolon separated.

#### Citation tags

The system also supports citation tags, which are recommended for the following applications:

1. A citation's identifier contains forbidden characters, such as `;` or ending with a non-alphanumeric character other than `/`.
In these instances, you must use a tag.
2. A single reference is cited many times.
Therefore, it might make sense to define a tag, so if the citation updates (e.g. a newer version becomes available), only a single change is required.

Tags should be defined in [`content/citation-tags.tsv`](content/citation-tags.tsv).
If `citation-tags.tsv` defines the tag `study-x`, then this study can be cited like `@tag:study-x`.

## Reference metadata

The Manubot workflow requires the bibliographic details for references (the set of all cited works) as CSL (Citation Style Language) Items (also known as [citeproc JSON](
The Manubot attempts to automatically retrieve metadata and generate valid citeproc JSON for references, which is exported to `output/references.json`.
However, in some cases the Manubot fails to retrieve metadata or generates incorrect or incomplete citeproc metadata.
Errors are most common for `url` references.
For these references, you can manually specify the correct citeproc in [`content/manual-references.json`](content/manual-references.json), which will override the automatically generated reference data.
To do so, create a new citeproc record that contains the field `"standard_citation"` with the appropriate reference identifier as its value.
The identifier can be obtained from the `standard_citation` column of `citations.tsv`, which is located in the `output` branch or in the `output` subdirectory of local builds.
As an example, `manual-references.json` contains:

"standard_citation": "url:"

For guidance on what CSL JSON should be like for different document types, refer to [these examples](

## Manuscript metadata

[`content/metadata.yaml`](content/metadata.yaml) contains manuscript metadata that gets passed through to Pandoc, via a [`yaml_metadata_block`](
`metadata.yaml` should contain the manuscript `title`, `authors` list, and `keywords`.
Additional metadata, such as `date`, will automatically be created by the Manubot.

We recommend authors add themselves to `metadata.yaml` via pull request (when requested by a maintainer), thereby signaling that they've read and approved the manuscript.
The following YAML shows the supported key–value pairs for an author:

github: dhimmel # strongly suggested
name: Daniel S. Himmelstein # mandatory
initials: DSH # strongly suggested
orcid: 0000-0002-3012-7446 # mandatory
twitter: dhimmel # optional
email: # suggested
affiliations: Department of Systems Pharmacology and Translational Therapeutics, University of Pennsylvania # strongly suggested
funders: GBMF4552 # optional

## Manubot feedback

If you experience any issues with the Manubot or would like to contribute to its source code, please visit [`greenelab/manubot`]( or [`greenelab/manubot-rootstock`](
File renamed without changes.
@@ -5,13 +5,17 @@ export LC_ALL=en_US.UTF-8

# Generate reference information
echo "Retrieving and processing reference metadata"
(cd build && python
manubot \
--content-directory=content \
--output-directory=output \
--cache-directory=ci/cache \

# pandoc settings

# Make output directory
mkdir -p output
@@ -32,7 +36,7 @@ pandoc --verbose \
--css=github-pandoc.css \
--include-in-header=build/assets/analytics.js \
--include-after-body=build/assets/anchors.js \
--output=output/index.html \
--output=output/manuscript.html \

# Create PDF output
@@ -44,13 +48,14 @@ wkhtmltopdf \
--margin-bottom 17 \
--margin-left 0 \
--margin-right 0 \
output/index.html \
webpage/index.html \

# Create DOCX output when user specifies to do so
if [ "$BUILD_DOCX" = "true" ];
echo "Exporting Word Docx manuscript"
ln --symbolic content/images images
pandoc --verbose \
--smart \
--from=markdown \
@@ -63,4 +68,5 @@ then
--reference-docx=$DOCX_PATH \
--output=output/manuscript.docx \
rm --recursive images
Oops, something went wrong.

0 comments on commit 084e30d

Please sign in to comment.