spec-publish-asciidoc

Purpose

This repository contains a model publishing environment showing how technical specifications can be published in HTML, PDF and DocBook, based on authored content merged with UML models from a tool. The method replaces openEHR.org’s previous method of publishing based on Adobe FrameMaker documents.

The type of technical specifications for which this approach is useful is standard-style documentation of formal artefacts, including:

• object models

• languages

• other formalisms.

Description

The publishing environment takes various sources of input to create a logical specification:

• manually authored text (masterxxx.adoc documents) and diagrams (any tool, as long as .png or .svg is generated);

• a formal UML model, represented in the MagicDraw UML tool;

• other resources, e.g. Antlr3 or Antlr4 grammar files.

We use Asciidoc as the source document syntax, the MagicDraw UML tool for the models and Asciidoctor as the master publishing tool.

The general concept is that the overall specification is considered to be a manually authored artefact, which may include headings, complex formatting, tables, diagrams, source code, and any other features that would normally be expected in a sophisticated authoring toolset. Within this master document are 'include' statements for documents and diagrams generated from a formal model, such as a UML tool or programming environment.^[1]

Model elements are extracted from the UML tool and serialised as source documents and diagrams that are included in the master Asciidoc document using include statements.

Other content, e.g. parse files, are pulled into the Asciidoctor file by include statements.

The outputs are generated using asciidoctor tool.

Status

Alpha

Just Looking?

We have converted two large openEHR specifications to the new form - the AOM and the ADL specifications. You can see:

• AOM specification: source form here; HTML view;

  • features to look for: UML tool generated diagrams and specification tables;

• ADL specification: source form here; HTML view;

  • features to look for: machine highlighting using a newly developed ADL mode for Pygments;

The generated PDF is very rough and contains various errors that will be fixed with control over the PDF output stage.

Workflow

The approach to building specification documentation and web resources is shown graphically here.

To publish the specifications for one openEHR 'component' requires the Git repo of that component, e.g. specifiations-AM and also this spec-publish-asciidoc repository, which supplies the resource files (CSS, PDF theme, boilerplate text etc).

Benefits

Once a toolchain like this is established, we can get the following benefits:

• model-based documentation is always up to date, as long as the models are maintained;

• specifications built like software, under continuous build on server;

• easier to report and fix errors, and to update documents, due to easy-ish source form;

• more flexibility to create new output formats.

Installation

Publishing tools

To get the environment going you need quite a few tools. If you already have a working Ruby/gem environment, just skip to the Asciidoctor installation bit below.

Warning

if you have never set up Ruby before, take some care. Experts suggest to install rvm first, and use that as a version manager. It’s not strictly necessary, but may help what can otherwise become a versioning nightmare.

Rvm

A ruby version manager. Install here.

Ruby

See install Ruby. The scripts and commands in the publishing environment in this repository assume a unix-style environment, since the ultimate aim is to have it working under a Linux continuous build server. It can be set up on any normal machine; if you are on a Windows machine, using cygwin to run everything is probably advisable.

Rubygems

On Linux see here; Cygwin, use the normal installer.

Asciidoctor

The guidance on this page pretty much covers it.

Pygments

This supports code highlighting in the generated output. See here.

Asciidoctor-stylesheet-factory

You don’t need to install this unless you actually want to change the openehr.css or build new stylesheets. See openEHR fork of asciidoctor repo. The openEHR .scss files are in sass and sass/settings. Use compass compile on command line at root of repo to regenerate.

Pygments adjustments

The Pygments tool is a bit tricky to manage. The pygments.rb installation contains a static copy of an old snapshot of the main pygments Python project in its vendor directory. To achieve this, you will need to clone the original project. The best way to do this right now is to clone his Bitbucket fork of it, which contains the Archetype language additions (enabling colourising of adl, cadl, odin syntax), and then copy the contents of that into the vendor area of the pygments.rb install (which will be wherever your Ruby gems are being installed).

Note that the Bitbucket Pygments repository is a Mercury repository, not Git, so you have to use commands like hg clone etc. Any online Mercury tutorial will provide a guide to these commands. You will most likely only need clone, status, ci (check-in), push and pull.

Then you will need to go to the root directory of the pygments.rb install area, and run cache-lexers.rb to update the lexers cache file..

This is not ideal and hopefully, the pygments.rb project (which is just a Ruby wrapper around Pygments, which is in Python) will be updated to a better method or replaced within Asciidoctor proper.

MagicDraw and UML Extractor

If you have MagicDraw, you can install the UML extractor plugin (files OpenEhrModelExporter.jar and plugin.xml in /uml_extract/OpenEhrModelExporter in this repository) in the MagicDraw install area directory on your machine, under /plugin/org.openehr.docs.magicdraw.

The extract can now be done manually, which is the normal method for this repository, or by using a command line invocation, which is the way it can most easily be done in the production component repositories (specifications-RM, etc).

Manual method: start MagicDraw and open the project openEHR_UML-AM.mdzip. You will see an 'openEHR' menu option, with one option to extract documentation. Choose this, and choose source/AOM2 as the output directory. The generated outputs will appear in the subdirectories uml_diagrams and classes.

CLI method: go to the root of the repository you are working in (e.g. a component one such as specifications-RM) and do:

$ ./do_uml_generate-spec-RM.sh # or do_uml_generate-spec-AM.sh etc

The Process

Conversion of FrameMaker and other source documents

For conversion from Frame, various approachs are used including the following:

• in Frame, for each chapter document, including 'Front.fm', but not TOC or end.fm, set HTML mappings for Heading types, List types and main Font types

• save each chapter as HTML

• hand-edit each file to fix <pre></pre> blocks, and heading level errors

• use pandoc to convert each HTML file xxxx.html to asciidoc, as follows

$ pandoc -f html -t asciidoc -o xxxx.adoc --no-wrap --atx-headers xxxx.html

Other methods like cut and paste from PDFs are usable, although a bit painful.

Generating the publishing outputs

The following commands can be used to generate the HTML and PDF outputs:

$ ./publish.sh -r  # use the standard remote CSS files

$ ./publish.sh     # use repo-local copy of CSS files

$ ./publish.sh -p  # generate PDF as well - takes time

$ ./publish.sh -pt # generate PDF with tracing on, for debug purposes

Have a look at this script to get an idea of how we are invoking asciidoctor.

Status of PDF

Currently, the PDF output is not usable, but Asciidoctor-pdf is being worked on heavily, and the next release will include a lot of fixes. We should be able to generate decent PDFs for openEHR after that.

Other output formats

There is an epub extension for Asciidoctor. This has not yet been integrated to the environment here, but could be, and should in theory enable us to generate .epub files.

Resources, styles and themes for HTML and PDF

We are using a stylesheet generated using the asciidoctor-stylesheet-factory; it is in ./resources/css/. An early theme file for PDF, built according to the Asciidoctor-pdf theming guide is in ./resources as well.

TODO

Many things…​

• control PDF publishing properly

• sort out continuous build

• Add AQL syntax to Archetype Pygments lexer file.

---

1. There is an alternative theory that says specifications are completely represented inside a tool such as a UML tool, and it is responsible for generating the whole output. We don’t believe this theory, mainly because UML tools are not good at dealing with arbitrary complex text, and also because UML isn’t the only possible formalism that might be included in the output document. What is clearly needed is a standardised publishing environment that can deal with specifications about anything.