Docs HOWTO

Conditions of use

This documentation build process is provided to the public purely for the purpose of testing documentation changes before submitting pull requests to the appropriate Elasticsearch repository.

The documents produced by this build process may be published only on http://www.elastic.co. They may not be published in any other form or on any other website without explicit permission.

Getting started

Requirements

First, you need to make sure that you have the following installed:

perl — system perl is fine
python — system python is fine
xsltproc — if not already installed, see Installing a Processor

Tip	Windows users are recommended to use Cygwin (install `asciidoc` package and `libxslt`).

Cloning the repository

Clone the docs repository with:

git clone git@github.com:elastic/docs.git

Building this README

You can test that everything is working correctly by building this README as follows:

cd docs/
./build_docs.pl --doc README.asciidoc --open

This should convert README.asciidoc into HTML and open it in your browser.

Building documentation

For a local repo

When you are making changes to documentation in a locally checked out repository, and you want to check that they are building correctly, use build_docs.pl with the --doc parameter to generate the HTML version of the docs:

cd path/to/your/repo
build_docs.pl --doc path/to/index.asciidoc

Specifying a different output dir

By default, the HTML docs are generated in ./html_docs. You can change the output directory with the --out parameter:

build_docs.pl --doc path/to/index.asciidoc --out output/dir/

Warning

The output/dir/ will be deleted and recreated, so don’t point it at a directory with contents that you want to keep.

Viewing the docs

To view the generated docs in your web browser immediately after the build has finished, use the --open parameter:

build_docs.pl --doc path/to/index.asciidoc --open

Single- or multi-page

By default, the build process generates an HTML file per part/chapter/section. To generate all of the docs in a single file instead, use the --single parameter:

build_docs.pl --doc path/to/index.asciidoc --single

And if you want a table of contents added at the beginning of the single page output, add the --toc parameter:

build_docs.pl --doc path/to/index.asciidoc --single --toc

Note	The multi-page output always contains tables-of-content where appropriate.

Chunking in the right place

Asciidoc combines all of the source files into one big file, which it converts to a single XML file for DocBook to process. Chunking is the process of breaking that XML file up into multiple HTML pages.

By default, each part (= Part Title) and chapter (== Chapter Title) is `chunked'' into a separate HTML file. However, for bigger books, such as the Elasticsearch reference docs, this results in enormous pages. These bigger books are also chunked at the first section level (=== Section One Title`). This behaviour is specified in the conf.yaml file, but can also be specified on the command line when building the docs locally:

build_docs.pl --doc path/to/index.asciidoc --chunk 1

Correcting errors

Asciidoc is converted to DocBook, which is then converted to HTML. DocBook is strict, which means that you have to follow the rules (see Asciidoc Guide) otherwise the build process will throw an error.

The most common errors are:

Asciidoc syntax error, eg =Title instead of = Title or unclosed block delimiters, ie ====== or ------
incorrect order of elements, eg = Title followed by a === Level 3 header
Linking to an non-existent ID

Errors may be thrown either by Asciidoc or by DocBook. Asciidoc errors refer to the actual .asciidoc page where the error occurred, but DocBook errors only refer to a line in the generated XML file, and look something like this:

path/to/index.xml:1518: element xref: validity error : XXXXX

The easiest way to debug these is to open the XML file (index.xml in the example above) and navigate to the specified line number (1518 in the example above). This marks the end of the section containing the error.

Building part of a book

Building a big book is time consuming. Unfortunately, in order to test that all links exist, the entire book needs to be built in one go. However, while you’re writing docs it is useful to be able to build just a single .asciidoc page.

You can do this by telling build_docs.pl to ignore errors with the --lenient parameter:

build_docs.pl --doc path/to/some/page.asciidoc --lenient --single --open

If the page you are building contains links to content in other pages, the above command will output warnings like:

ERROR: xref linking to relevance-intro has no generated link text.

Note	You should still build the whole book without the `--lenient` parameter before commiting, to be sure that you haven’t broken any links.

For the website

Usually you don’t need to build all the docs that will be uploaded to the website, but if you are linking between documents (e.g. between the Java API docs and the main reference documentation), then building all of the docs will report any missing links.

You can build all the docs with:

build_docs.pl --all

The first time you run this will be slow as it needs to:

clone each repository
build the docs for each branch

Subsequent runs will pull any changes to the repos and only build the branches that have changed.

Checking your changes

Because the docs are built from the remote repositories, you will need to push your changes to the main repo before running build_docs.pl --all.

Assuming you have already checked that your docs build correctly using the local build process, the only other errors that might occur at this stage are bad cross-document links.

Once the docs build correctly, you don’t need to do anything more. The changes that you have pushed to your repository will be picked up automatically by the docs build service.

Adding new docs or new branches

The documentation that appears on the http://www.elastic.co/guide website is controlled by the conf.yaml file in the docs repo.

You can add a new repository under the repos section, if it doesn’t already exist, and you can add a new `book'' under the `contents section.

The repos.$NAME.branches[] key lists all of the branches which should be built — all of these branches will be available on the website — while repos.$NAME.current lists the branch which should be used as the default version on the site.

Note	The `branches` and `current` settings can be overridden in the config for each book. For instance, the `Community Clients` docs are built only from the `master` branch.

When you release a new version of your code, you need to add a new branch to the config and to update the current branch for your project. Commit the change to conf.yaml and push to the remote docs repo.

Asciidoc Guide

Basic book structure

Asciidocs can be built as a book, article, manpage etc. All our docs are built as a book, and thus follow the layout for books. The most basic structure is as follows:

= Book title                # level 0

== Chapter title            # level 1

=== Section title           # level 2

==== Section title          # level 3

===== Section title         # level 4

Usually this structure will be sufficient for most of your documentation needs. More complicated ``books'', such as the Elasticsearch reference docs, however, require a few additional elements, described on the following pages.

Filenames

By default, each chapter will generate a new chunk or HTML file. You can control the name of the file by giving the header an ID, as follows:

[[intro-to-xyz]]
== Intro to XYZ

This chapter would then be written to a file called intro-to-xyz.html. If no ID is provided, then a filename will be auto-generated. See Controlling chunking for more.

These IDs are also used to link to sections within each book. See Linking.

Multi-part books

Books may also be divided into multiple parts, which are indicated with level 0 headers:

= Book title                # level 0

= Part title                # level 0

== Chapter title            # level 1

=== Section title           # level 2

... etc ...

Each part also creates a new chunk or HTML file.

Part intro

A part may include text before the first chapter, but it must be marked with [partintro] in order to be valid:

= Book title                # level 0

= Part one                  # level 0

[partintro]
A paragraph introducing this Part

== Chapter title            # level 1

... etc ...

Longer partintro blocks should be wrapped in an open block which starts and ends with two dashes: --:

= Part two                  # level 0

[partintro]
.A partintro title
-- (1)
This section may contain multiple paragraphs.

[float]
== A header should use `[float]`

Everything up to the closing -- marker
will be considered part of the partintro.
-- (1)

== Chapter title           # level 2

... etc ...

The open block delimiters

Optional sections

Books may include other sections such as a preamble, a preface, a glossary or appendices.

Preamble

= Book title                # level 0

.Optional preamble title
Preamble text...

Preface and Appendix

[preface]
= Preface title             # level 0

=== Preface header          # level 2 (1)

= Part one                  # level 0

and

[appendix]
= Appendix title            # level 0

=== Appendix header         # level 2 (1)

Any headers in the appendix or in the preface start out-of-sequence at level 2, not at level 1.

Glossary

[glossary]
= Glossary title            # level 0

[glossary]
Term one::
    Defn for term one

Term two::
    Defn for term two

Note	The two `[glossary]` elements above have different purposes: The first marks this section of the document as a glossary, to be included in the table of contents The second marks the definitions list as type `glossary`

Also see

If you need to use some of these more advanced structural elements, have a look at the example of a multi-part book included in this repo in resources/asciidoc-8.6.8/doc/book-multi.txt.

Paragraphs

A paragraph consists of multiple lines of text which start in the left hand column:

This is a paragraph
even though it contains
line breaks.

This is a second paragraph.

Paragraph titles

Like most elements, a paragraph can have a title:

Example 1. Paragraph with a title

.Paragraph title
Text of my paragraph

Paragraph title

Text of my paragraph

Admonition paragraphs

A paragraph which starts with TIP:, NOTE:, IMPORTANT:, WARNING: or CAUTION: is rendered as an admonition paragraph, eg:

NOTE: Compare admonition paragraphs with <<admon-blocks>>.

This renders as:

Note	Compare admonition paragraphs with Admonition blocks.

Literal paragraphs

Literal paragraphs, which are rendered as <pre> blocks without any source highlighting, must be indented:

Example 2. A literal paragraph

.Optional title

    This para must
    be indented

Optional title

This para must
be indented

See also Code blocks for blocks with syntax highlighting.

Inline text

Inline text can be formatted as follows:

`emphasis`	emphasis
`bold`	bold
\`mono`	`mono`
\`\`double quoted''	``double quoted''
\`single quoted'	`single quoted'
`^superscript`	^superscript
`_subscript`	_subscript

These formatting characters expect to adjoin whitespace or common punctuation characters. To combine bold with emphasis, double up the quotes (ie use __ and **):

Example 3. Combining bold and emphasis

This example co__mb**in**es__ bold and emphasis

This example combines bold and emphasis.

Unwanted quotes can be escaped with a \ character.

Replacement characters

Certain runs of ASCII characters are replaced as follows:

`--`	— (em dash)
`…`	…
`→`	→
`←`	←
`⇒`	⇒
`⇐`	⇐
`©`	©
`™`	™
`®`	®

Linking

You can link to any block in the document that has an ID — an identifier before the block which is wrapped in double square brackets:

[[para-id]]
This paragraph can be linked to using the ID `para-id`.

When you need to combine an ID with a style, you can either specify each on a separate line:

[[note-id]]
[NOTE]
===============================
This note can be linked to using the ID `note-id`.
===============================

or in one line:

["NOTE",id="note-id"] (1)
===============================
This note can be linked to using the ID `note-id`.
===============================

In the one line format, the NOTE must be enclosed in double quotes.

Both of the above render as:

Note	This note can be linked to using the ID `note-id`.

The ID is added to the HTML document as an <a> anchor and, as explained in Controlling chunking, the ID is used as the filename for sections which are chunked — written to separate HTML files.

Internal links

You can link to any ID within a document using double angle brackets:

Example 4. Links with default link text

* <<setup>>
* <<structure>>
* <<note-id>>

Getting started
Basic book structure
[note-id]

It will use the title associated with each ID as the link text. In the example above, note-id is not associated with any title, which is why the text is rendered as "Note".

Alternative link text can be provided as a second parameter inside the angle brackets:

Example 5. Links with custom link text

See the <<note-id,note about IDs>>.

See the note about IDs.

External links

Links to external websites can just be added as normal inline text, optionally with custom link text in square brackets:

Example 6. External links

See http://github.com/elasticsearch
or  http://github.com/elasticsearch/docs[this repository]

See http://github.com/elasticsearch or this repository

The existence of external links is not confirmed by the build process.

Cross document links

Links to other Elasticsearch docs are essentially the same as external links. However, for conciseness and maintainability, you should use an attribute to represent the absolute URL of the docs.

Attributes can be added to the beginning of the docs, under the book title:

Example 7. Using attributes for cross-document linking

= My Book Title
:ref:  http://www.elastic.co/guide/elasticsearch/reference/current

Here is a link to the {ref}/search.html[search page]

Here is a link to the search page

The main benefit of using attributes for cross document links is that, when the docs for an old version contain links that no longer exist in the current branch, you can update all the links in the document to point to the older version, by just updating a single attribute.

Cross document links are checked when build_docs.pl is run with the --all parameter. See For the website.

Lists

Bullet points

Bullet point lists are written using asterisks:

Example 8. Bullet points

.Optional title
* Point
* Point
** Sub-point
*** Sub-sub-point
* A point can have multiple paragraphs
+
But use a `+` instead of an empty line between paras.

An empty line signifies the end of the list.

Optional title

Point
Point
- Sub-point
- Sub-point
A point can have multiple paragraphs

But use a + instead of an empty line between paras

An empty line signifies the end of the list.

Ordered lists

Ordered lists use . instead of *, and will alternate between numbers and letters automatically:

Example 9. An ordered list

.Optional title
. foo
.. bar
... baz
.... balloo

Optional title

foo
1. bar
  1. baz
    
    balloo

Alternatively, you can control whether it uses a number or a letter as follows:

Example 10. Controlling the counters

a. Start with a letter
b. Another letter
  1. Now numbers
  2.  And more numbers

Start with a letter
Another letter
1. Now numbers
2. And more numbers

Definition lists

Definition lists are used to define terms. The term must be followed by a double colon :: eg:

Example 11. A vertical definition list

term one::      Definition for term one
term two::
                Can start on the next line
term three::    A definition can have multiple
+
paragraphs, but use `+` to separate them

term four:::    Definitions can be nested
                by adding more colons
term five::     A definition can even include
                lists:
                * point one
                * point two

term one

Definition for term one

term two

Can start on the next line

term three

A definition can have multiple

paragraphs, but use + to separate them

term four: Definitions can be nested by adding more colons

term five

A definition can even include lists:

point one
point two

Horizontal definition lists

Often definition lists are better rendered horizontally, eg:

Example 12. A horizontal definition list

[horizontal]
term one::      Definition for term one
term two::
                Can start on the next line
term three::    A definition can have multiple
+
paragraphs, but use `+` to separate them

term four:::    Definitions can be nested
                by adding more colons
term five::     A definition can even include
                lists:
                * point one
                * point two

term one	Definition for term one
term two	Can start on the next line
term three	A definition can have multiple paragraphs, but use `+` to separate them term four Definitions can be nested by adding more colons
term five	A definition can even include lists: point one point two

Blocks

Blocks are used for special blocks of content, such as Code blocks, Example blocks, Sidebars and Admonition blocks.

Blocks are delimited with a start and end line which uses the same characters, like =====.

Code blocks

Code blocks are rendered as <pre> blocks, and use syntax highlighting, eg:

Example 13. A code block

.Optional title
[source,js]
----------------------------------
{
    "query": "foo bar"
}
----------------------------------

Optional title

{
    "query": "foo bar"
}

Important

The language to use for source highlighting — eg js above — must be specified, otherwise Asciidoc emits invalid DocBook.

Callouts

Code blocks can use callouts to add an explanatory footnote to a particular line of code:

Example 14. Code block with callouts

[source,js]
----------------------------------
{
    "query": "foo bar" <1>
}
----------------------------------

\<1> Here's the explanation

{
    "query": "foo bar" (1)
}

Here’s the explanation

View in Sense

Code blocks can be followed by a `View in Sense'' link which, when clicked, will open the code snippet in Sense. The snippet can either be taken directly from the code block (`AUTOSENSE), or be a link to a custom snippet.

Example 15. Code block with AUTOSENSE link

[source,js]
----------------------------------
GET /_search
{
    "query": "foo bar" <1>
}
----------------------------------
// AUTOSENSE (1)

\<1> Here's the explanation

The // AUTOSENSE line must follow immediately after the code block, before any callouts.

GET /_search
{
    "query": "foo bar" (1)
}

Here’s the explanation

Note

The ``View in Sense'' links will only work if the docs are viewed in web-browser mode, as follows:

build_docs.pl -d my_doc.asciidoc --open <1> --web (2)

The --open option usually opens the docs in the web browser, served directly from the file system.
The --web option will start a local web browser to serve the docs, which allows the Sense links to work correctly.

The local web browser can be stopped with Ctrl-C.

Custom Sense snippets

Sometimes you will want to show a small amount of code in the code block, but to provide a full recreation in the Sense snippet. In this case, you need to:

Save the snippet file in the ./snippets/ directory in the root docs directory.
Under the code block, specify the name of the snippet file with
```
// SENSE: path/to/snippet.json
```

For instance, to add a custom snippet to the file ./one/two/three.asciidoc, save the snippet to ./snippets/one/two/three/example_1.json, then add the SENSE link below the code block:

Example 16. Code block with custom SENSE link

[source,js]
----------------------------------
GET /_search
{
    "query": "foo bar" <1>
}
----------------------------------
// SENSE:one/two/three/example_1.json (1)

\<1> Here's the explanation

The path should not contain the initial snippets directory

Admonition blocks

Admonition blocks are much the same as Admonition paragraphs, except that they can be longer and contain more than just a paragraph. For instance:

[NOTE]
=========================
This note contains a list:

* foo
* bar
* baz

and some code

[source,js]
----------------------------------
{ "query": "foo bar"}
----------------------------------
=========================

This renders as:

Note	This note contains a list: foo bar baz and some code { "query": "foo bar"}

Sidebars

Sidebars are used to highlight a block of content that is outside the usual flow of text:

.Optional title
**********************************
So why does the `bulk` API have such a
funny format?  Sit down and I'll tell you
all about it!
**********************************

Optional title

So why does the bulk API have such a funny format? Sit down and I’ll tell you all about it!

Example blocks

Example blocks contain normal text which is used as an example. The title, if any, is labelled as an example and numbered:

.My first example
========================================
Text explaining the first example.
========================================

.My second example
========================================
Text explaining the second example.
========================================

This renders as:

Example 17. My first example

Text explaining the first example.

Example 18. My second example

Text explaining the second example.

Caution

The === and --- delimiters can sometimes be confused with a header, resulting in an error. To resolve this, add newlines between the delimiter and the content before and after it.

Including files

For long documentation, you probably want to break up the Asciidoc files into smaller units, and just include them where appropriate:

include::myfolder/mydoc.asciidoc[]

Paths are relative to the file which contains the include statement.

Additions and deprecations

Documentation is built for various branches, eg 0.90, 1.00, master. However, we release versions 0.90.0, 0.90.1, etc, which are all based on the 0.90 branch.

When adding new functionality to a branch, or deprecating existing functionality, you can mark the change as added, coming or deprecated. Use coming when the addition is in an as yet unreleased version of the current branch, and added when the functionality is already released.

The update_versions.pl script can be used to change coming notices to added notices when doing a new release, and can also be used to remove added, coming and deprecated notices completely.

Inline notifications

Use inline notifications for small changes, such as the addition or deprecation of individual parameters.

[horizontal]
`foo.bar`::   Does XYZ. added[0.90.4]
`foo.bar`::   Does XYZ. coming[0.90.4]
`foo.baz`::   Does XYZ. deprecated[0.90.4]

`foo.bar`	Does XYZ. added[0.90.4]
`foo.bar`	Does XYZ. coming[0.90.4]
`foo.baz`	Does XYZ. deprecated[0.90.4]

You can also include details about additional notes in the notifications which show up when the user hovers over it:

[horizontal]
`foo.bar`::   Does XYZ. added[0.90.4,Replaces `foo.baz`]
`foo.bar`::   Does XYZ. coming[0.90.4,Replaces `foo.baz`]
`foo.baz`::   Does XYZ. deprecated[0.90.4,Replaced by `foo.bar`]

`foo.bar`	Does XYZ. added[0.90.4,Replaces `foo.baz`]
`foo.bar`	Does XYZ. coming[0.90.4,Replaces `foo.baz`]
`foo.baz`	Does XYZ. deprecated[0.90.4,Replaced by `foo.bar`]

Section notifications

Use section notifications to mark an entire chapter or section as added/deleted. Notifications can just refer to the version in which the change was made:

==== New section

added[0.90.4]

Text about new functionality...

==== New section not yet released

coming[0.90.9]

Text about new functionality...

==== Old section

deprecated[0.90.4]

Text about old functionality...

New section

added[0.90.4]

Text about new functionality…

New section not yet released

coming[0.90.9]

Text about new functionality…

Old section

deprecated[0.90.4]

Text about old functionality…

With details…

Or they can include extra text, including more Asciidoc markup:

[[new-section]]
==== New section

added[0.90.4,Replaces `foo.bar`. See <<old-section>>]

Text about new functionality...

[[coming-section]]
==== New section not yet released

coming[0.90.9,Replaces `foo.bar`. See <<old-section>>]

Text about new functionality...

[[old-section]]
==== Old section

deprecated[0.90.4,Replace by `foo.baz`. See <<new-section>>]

Text about old functionality...

New section

added[0.90.4,Replaces foo.bar. See Old section]

Text about new functionality…

Old section

deprecated[0.90.4,Replace by foo.baz. See New section]

Text about old functionality…

Experimental

APIs or parameters that are experimental can be marked as such, using markup similar to that used in Additions and deprecations. For instance:

[[new-feature]]
=== New experimental feature

experimental[]

experimental[Custom text goes here]

Text about new feature...

[[old-feature]]
=== Established feature

This feature has been around for a while, but we're adding
a new experimental parameter:

[horizontal]
`established_param`::  This param has been around for ages and won't change.
`experimental_param`:: experimental[] This param is experimental and may change in the future.
`experimental_param`:: experimental[Custom text goes here] This param is experimental and may change in the future.

New experimental feature

experimental[]

experimental[Custom text goes here]

Text about new feature…

Established feature

This feature has been around for a while, but we’re adding a new experimental parameter:

`established_param`	This param has been around for ages and won’t change.
`experimental_param`	experimental[] This param is experimental and may change in the future.
`experimental_param`	experimental[Custom text goes here] This param is experimental and may change in the future.

Images

Any images you want to include should be saved in a folder in your repo, and included using a path relative to the document where the image:: statement appears.

[[cat]]
.A scaredy cat
image::resources/cat.jpg[Alt text]

A link to <<cat>>

Figure 1. A scaredy cat

A link to A scaredy cat

Width and height

The width and/or height of the image can be specified in pixels or as a percentage:

image::resources/cat.jpg["Alt text",width=50]
image::resources/cat.jpg["Alt text",width="20%"]

Alignment

Images are left-aligned by default, but they can be centred or right-aligned:

image::resources/cat.jpg["Alt text",width=100,align="left"]
image::resources/cat.jpg["Alt text",width=100,align="right"]
image::resources/cat.jpg["Alt text",width=100,align="center"]

Tables

In general, tables are frowned upon in DocBook as they don’t display well in formats other than HTML, eg PDF, ePub, etc.

It’s almost always better to use Horizontal definition lists instead, but if you really want to use tables, you can read about them here.

Controlling chunking

In Basic book structure, we said that each part or chapter generates a new chunk or HTML file. For more complex documentation, you may want the first level of sections to also generate new chunks.

For instance, in the ES reference docs, we have:

= Search APIs               # part

== Request body search      # chapter

=== Query                   # section level 1

=== From/Size               # section level 1

... etc ...

There are too many parameters for ``Request body search'' to list them all on one page. In this case, it is worth turning on chunking for top level sections.

Enabling section chunking

To enable section chunking when building docs in a local repository, pass the --chunk parameter:

build_docs.pl --doc path/to/index.asciidoc --chunk 1

To enable section chunking when building docs for the website, add chunk: 1 to the conf.yaml file in the docs repo.

contents:
    -
        title:      Elasticsearch reference
        prefix:     elasticsearch/reference
        repo:       elasticsearch
        index:      docs/reference/index.asciidoc
        chunk:      1 (1)

Chunking is enabled for this book

Chunking selected sections

If you enable session chunking, you will probably find that you have a few short sections which you want to keep on the same page.

To do this, you can use the [float] marker before a section header, to tell Asciidoc that what follows isn’t a ``real'' header:

[[chapter-one]]
== chapter               # new chunk

[[section-one]]
=== Section one         # new chunk

[[section-two]]
[float]
=== Section two         # same chunk

[[section-three]]
=== Section three       # new chunk

The above would produce three HTML files, named for their IDs:

chapter-one.html
section-one.html which would also contain ``Section two''
section-three.html

To link to `Section two'' from an external document, you would use the URL: `section-one.html#section-two

madboxkr/docs

ErrorLooks like something went wrong!

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Languages

Error
Looks like something went wrong!

Packages