No description or website provided.
HTML XSLT JavaScript CSS Perl Python
Latest commit 1a4e612 Jan 24, 2017 Docs User Updated docs

README.asciidoc

Docs HOWTO

Conditions of use

This documentation build process is provided to the public purely for the purpose of testing documentation changes before submitting pull requests to the appropriate Elasticsearch repository.

The documents produced by this build process may be published only on http://www.elastic.co. They may not be published in any other form or on any other website without explicit permission.

Getting started

Requirements

First, you need to make sure that you have the following installed:

  • perl  — system perl is fine

  • python  — system python is fine

  • xsltproc — if not already installed, see Installing a Processor

Tip
Windows users are recommended to use Cygwin (install asciidoc package and libxslt).

Cloning the repository

Clone the docs repository with:

git clone git@github.com:elastic/docs.git

Building this README

You can test that everything is working correctly by building this README as follows:

cd docs/
./build_docs.pl --doc README.asciidoc --open

This should convert README.asciidoc into HTML and open it in your browser.

Building documentation

For a local repo

When you are making changes to documentation in a locally checked out repository, and you want to check that they are building correctly, use build_docs.pl with the --doc parameter to generate the HTML version of the docs:

cd path/to/your/repo
build_docs.pl --doc path/to/index.asciidoc

Specifying a different output dir

By default, the HTML docs are generated in ./html_docs. You can change the output directory with the --out parameter:

build_docs.pl --doc path/to/index.asciidoc --out output/dir/
Warning
The output/dir/ will be deleted and recreated, so don’t point it at a directory with contents that you want to keep.

Viewing the docs

To view the generated docs in your web browser immediately after the build has finished, use the --open parameter:

build_docs.pl --doc path/to/index.asciidoc --open

Single- or multi-page

By default, the build process generates an HTML file per part/chapter/section. To generate all of the docs in a single file instead, use the --single parameter:

build_docs.pl --doc path/to/index.asciidoc --single

And if you want a table of contents added at the beginning of the single page output, add the --toc parameter:

build_docs.pl --doc path/to/index.asciidoc --single --toc
Note
The multi-page output always contains tables-of-content where appropriate.

Chunking in the right place

Asciidoc combines all of the source files into one big file, which it converts to a single XML file for DocBook to process. Chunking is the process of breaking that XML file up into multiple HTML pages.

By default, each part (= Part Title) and chapter (== Chapter Title) is `chunked'' into a separate HTML file. However, for bigger books, such as the Elasticsearch reference docs, this results in enormous pages. These bigger books are also chunked at the first section level (=== Section One Title`). This behaviour is specified in the conf.yaml file, but can also be specified on the command line when building the docs locally:

build_docs.pl --doc path/to/index.asciidoc --chunk 1

Correcting errors

Asciidoc is converted to DocBook, which is then converted to HTML. DocBook is strict, which means that you have to follow the rules (see Asciidoc Guide) otherwise the build process will throw an error.

The most common errors are:

  • Asciidoc syntax error, eg =Title instead of = Title or unclosed block delimiters, ie ====== or ------

  • incorrect order of elements, eg = Title followed by a === Level 3 header

  • Linking to an non-existent ID

Errors may be thrown either by Asciidoc or by DocBook. Asciidoc errors refer to the actual .asciidoc page where the error occurred, but DocBook errors only refer to a line in the generated XML file, and look something like this:

path/to/index.xml:1518: element xref: validity error : XXXXX

The easiest way to debug these is to open the XML file (index.xml in the example above) and navigate to the specified line number (1518 in the example above). This marks the end of the section containing the error.

Building part of a book

Building a big book is time consuming. Unfortunately, in order to test that all links exist, the entire book needs to be built in one go. However, while you’re writing docs it is useful to be able to build just a single .asciidoc page.

You can do this by telling build_docs.pl to ignore errors with the --lenient parameter:

build_docs.pl --doc path/to/some/page.asciidoc --lenient --single --open

If the page you are building contains links to content in other pages, the above command will output warnings like:

ERROR: xref linking to relevance-intro has no generated link text.
Note
You should still build the whole book without the --lenient parameter before commiting, to be sure that you haven’t broken any links.

For the website

Usually you don’t need to build all the docs that will be uploaded to the website, but if you are linking between documents (e.g. between the Java API docs and the main reference documentation), then building all of the docs will report any missing links.

You can build all the docs with:

build_docs.pl --all

The first time you run this will be slow as it needs to:

  • clone each repository

  • build the docs for each branch

Subsequent runs will pull any changes to the repos and only build the branches that have changed.

Checking your changes

Because the docs are built from the remote repositories, you will need to push your changes to the main repo before running build_docs.pl --all.

Assuming you have already checked that your docs build correctly using the local build process, the only other errors that might occur at this stage are bad cross-document links.

Once the docs build correctly, you don’t need to do anything more. The changes that you have pushed to your repository will be picked up automatically by the docs build service.

Adding new docs or new branches

The documentation that appears on the http://www.elastic.co/guide website is controlled by the conf.yaml file in the docs repo.

You can add a new repository under the repos section, if it doesn’t already exist, and you can add a new `book'' under the `contents section.

The repos.$NAME.branches[] key lists all of the branches which should be built — all of these branches will be available on the website — while repos.$NAME.current lists the branch which should be used as the default version on the site.

Note
The branches and current settings can be overridden in the config for each book. For instance, the Community Clients docs are built only from the master branch.

When you release a new version of your code, you need to add a new branch to the config and to update the current branch for your project. Commit the change to conf.yaml and push to the remote docs repo.

Asciidoc Guide

Basic book structure

Asciidocs can be built as a book, article, manpage etc. All our docs are built as a book, and thus follow the layout for books. The most basic structure is as follows:

= Book title                # level 0

== Chapter title            # level 1

=== Section title           # level 2

==== Section title          # level 3

===== Section title         # level 4

Usually this structure will be sufficient for most of your documentation needs. More complicated ``books'', such as the Elasticsearch reference docs, however, require a few additional elements, described on the following pages.

Filenames

By default, each chapter will generate a new chunk or HTML file. You can control the name of the file by giving the header an ID, as follows:

[[intro-to-xyz]]
== Intro to XYZ

This chapter would then be written to a file called intro-to-xyz.html. If no ID is provided, then a filename will be auto-generated. See Controlling chunking for more.

These IDs are also used to link to sections within each book. See Linking.

Multi-part books

Books may also be divided into multiple parts, which are indicated with level 0 headers:

= Book title                # level 0

= Part title                # level 0

== Chapter title            # level 1

=== Section title           # level 2

... etc ...

Each part also creates a new chunk or HTML file.

Part intro

A part may include text before the first chapter, but it must be marked with [partintro] in order to be valid:

= Book title                # level 0

= Part one                  # level 0

[partintro]
A paragraph introducing this Part

== Chapter title            # level 1

... etc ...

Longer partintro blocks should be wrapped in an open block which starts and ends with two dashes: --:

= Part two                  # level 0

[partintro]
.A partintro title
(1)
This section may contain multiple paragraphs.

[float]
== A header should use `[float]`

Everything up to the closing -- marker
will be considered part of the partintro.
(1)

== Chapter title           # level 2

... etc ...
  1. The open block delimiters

Optional sections

Books may include other sections such as a preamble, a preface, a glossary or appendices.

Preamble

= Book title                # level 0

.Optional preamble title
Preamble text...

Preface and Appendix

[preface]
= Preface title             # level 0

=== Preface header          # level 2 (1)

= Part one                  # level 0

and

[appendix]
= Appendix title            # level 0

=== Appendix header         # level 2 (1)
  1. Any headers in the appendix or in the preface start out-of-sequence at level 2, not at level 1.

Glossary

[glossary]
= Glossary title            # level 0

[glossary]
Term one::
    Defn for term one

Term two::
    Defn for term two
Note

The two [glossary] elements above have different purposes:

  • The first marks this section of the document as a glossary, to be included in the table of contents

  • The second marks the definitions list as type glossary

Also see

If you need to use some of these more advanced structural elements, have a look at the example of a multi-part book included in this repo in resources/asciidoc-8.6.8/doc/book-multi.txt.

Paragraphs

A paragraph consists of multiple lines of text which start in the left hand column:

This is a paragraph
even though it contains
line breaks.

This is a second paragraph.

Paragraph titles

Like most elements, a paragraph can have a title:

Example 1. Paragraph with a title
.Paragraph title
Text of my paragraph
Paragraph title

Text of my paragraph

Admonition paragraphs

A paragraph which starts with TIP:, NOTE:, IMPORTANT:, WARNING: or CAUTION: is rendered as an admonition paragraph, eg:

NOTE: Compare admonition paragraphs with <<admon-blocks>>.

This renders as:

Note
Compare admonition paragraphs with Admonition blocks.

Literal paragraphs

Literal paragraphs, which are rendered as <pre> blocks without any source highlighting, must be indented:

Example 2. A literal paragraph
.Optional title

    This para must
    be indented
Optional title
This para must
be indented

See also Code blocks for blocks with syntax highlighting.

Inline text

Inline text can be formatted as follows:

emphasis

emphasis

bold

bold

\`mono`

mono

\`\`double quoted''

``double quoted''

\`single quoted'

`single quoted'

superscript

superscript

subscript

subscript

These formatting characters expect to adjoin whitespace or common punctuation characters. To combine bold with emphasis, double up the quotes (ie use __ and **):

Example 3. Combining bold and emphasis
This example co__mb**in**es__ bold and emphasis

This example combines bold and emphasis.

Unwanted quotes can be escaped with a \ character.

Replacement characters

Certain runs of ASCII characters are replaced as follows:

--

 — (em dash)

…​

…​

©

©

®

®

Linking

You can link to any block in the document that has an ID — an identifier before the block which is wrapped in double square brackets:

[[para-id]]
This paragraph can be linked to using the ID `para-id`.

When you need to combine an ID with a style, you can either specify each on a separate line:

[[note-id]]
[NOTE]
===============================
This note can be linked to using the ID `note-id`.
===============================

or in one line:

["NOTE",id="note-id"] (1)
===============================
This note can be linked to using the ID `note-id`.
===============================
  1. In the one line format, the NOTE must be enclosed in double quotes.

Both of the above render as:

Note

This note can be linked to using the ID note-id.

The ID is added to the HTML document as an <a> anchor and, as explained in Controlling chunking, the ID is used as the filename for sections which are chunked — written to separate HTML files.

You can link to any ID within a document using double angle brackets:

Example 4. Links with default link text
* <<setup>>
* <<structure>>
* <<note-id>>

It will use the title associated with each ID as the link text. In the example above, note-id is not associated with any title, which is why the text is rendered as "Note".

Alternative link text can be provided as a second parameter inside the angle brackets:

Example 5. Links with custom link text
See the <<note-id,note about IDs>>.

See the note about IDs.

Links to external websites can just be added as normal inline text, optionally with custom link text in square brackets:

Example 6. External links
See http://github.com/elasticsearch
or  http://github.com/elasticsearch/docs[this repository]

The existence of external links is not confirmed by the build process.

Links to other Elasticsearch docs are essentially the same as external links. However, for conciseness and maintainability, you should use an attribute to represent the absolute URL of the docs.

Attributes can be added to the beginning of the docs, under the book title:

Example 7. Using attributes for cross-document linking
= My Book Title
:ref:  http://www.elastic.co/guide/elasticsearch/reference/current

Here is a link to the {ref}/search.html[search page]

Here is a link to the search page

The main benefit of using attributes for cross document links is that, when the docs for an old version contain links that no longer exist in the current branch, you can update all the links in the document to point to the older version, by just updating a single attribute.

Cross document links are checked when build_docs.pl is run with the --all parameter. See For the website.

Lists

Bullet points

Bullet point lists are written using asterisks:

Example 8. Bullet points
.Optional title
* Point
* Point
** Sub-point
*** Sub-sub-point
* A point can have multiple paragraphs
+
But use a `+` instead of an empty line between paras.

An empty line signifies the end of the list.
Optional title
  • Point

  • Point

    • Sub-point

    • Sub-point

  • A point can have multiple paragraphs

    But use a + instead of an empty line between paras

An empty line signifies the end of the list.

Ordered lists

Ordered lists use . instead of *, and will alternate between numbers and letters automatically:

Example 9. An ordered list
.Optional title
. foo
.. bar
... baz
.... balloo
Optional title
  1. foo

    1. bar

      1. baz

        1. balloo

Alternatively, you can control whether it uses a number or a letter as follows:

Example 10. Controlling the counters
a. Start with a letter
b. Another letter
  1. Now numbers
  2.  And more numbers
  1. Start with a letter

  2. Another letter

    1. Now numbers

    2. And more numbers

Definition lists

Definition lists are used to define terms. The term must be followed by a double colon :: eg:

Example 11. A vertical definition list
term one::      Definition for term one
term two::
                Can start on the next line
term three::    A definition can have multiple
+
paragraphs, but use `+` to separate them

term four:::    Definitions can be nested
                by adding more colons
term five::     A definition can even include
                lists:
                * point one
                * point two
term one

Definition for term one

term two

Can start on the next line

term three

A definition can have multiple

paragraphs, but use + to separate them

term four

Definitions can be nested by adding more colons

term five

A definition can even include lists:

  • point one

  • point two

Horizontal definition lists

Often definition lists are better rendered horizontally, eg:

Example 12. A horizontal definition list
[horizontal]
term one::      Definition for term one
term two::
                Can start on the next line
term three::    A definition can have multiple
+
paragraphs, but use `+` to separate them

term four:::    Definitions can be nested
                by adding more colons
term five::     A definition can even include
                lists:
                * point one
                * point two
term one

Definition for term one

term two

Can start on the next line

term three

A definition can have multiple

paragraphs, but use + to separate them

term four

Definitions can be nested by adding more colons

term five

A definition can even include lists:

  • point one

  • point two

Blocks

Blocks are used for special blocks of content, such as Code blocks, Example blocks, Sidebars and Admonition blocks.

Blocks are delimited with a start and end line which uses the same characters, like =====.

Code blocks

Code blocks are rendered as <pre> blocks, and use syntax highlighting, eg:

Example 13. A code block
.Optional title
[source,js]
----------------------------------
{
    "query": "foo bar"
}
----------------------------------
Optional title
{
    "query": "foo bar"
}
Important
The language to use for source highlighting — eg js above — must be specified, otherwise Asciidoc emits invalid DocBook.

Callouts

Code blocks can use callouts to add an explanatory footnote to a particular line of code:

Example 14. Code block with callouts
[source,js]
----------------------------------
{
    "query": "foo bar" <1>
}
----------------------------------

\<1> Here's the explanation
{
    "query": "foo bar" (1)
}
  1. Here’s the explanation

View in Sense

Code blocks can be followed by a `View in Sense'' link which, when clicked, will open the code snippet in Sense. The snippet can either be taken directly from the code block (`AUTOSENSE), or be a link to a custom snippet.

Example 15. Code block with AUTOSENSE link
[source,js]
----------------------------------
GET /_search
{
    "query": "foo bar" <1>
}
----------------------------------
// AUTOSENSE (1)

\<1> Here's the explanation
  1. The // AUTOSENSE line must follow immediately after the code block, before any callouts.

GET /_search
{
    "query": "foo bar" (1)
}
  1. Here’s the explanation

Note

The ``View in Sense'' links will only work if the docs are viewed in web-browser mode, as follows:

build_docs.pl -d my_doc.asciidoc --open <1> --web (2)
  1. The --open option usually opens the docs in the web browser, served directly from the file system.

  2. The --web option will start a local web browser to serve the docs, which allows the Sense links to work correctly.

The local web browser can be stopped with Ctrl-C.

Custom Sense snippets

Sometimes you will want to show a small amount of code in the code block, but to provide a full recreation in the Sense snippet. In this case, you need to:

  • Save the snippet file in the ./snippets/ directory in the root docs directory.

  • Under the code block, specify the name of the snippet file with

    // SENSE: path/to/snippet.json

For instance, to add a custom snippet to the file ./one/two/three.asciidoc, save the snippet to ./snippets/one/two/three/example_1.json, then add the SENSE link below the code block:

Example 16. Code block with custom SENSE link
[source,js]
----------------------------------
GET /_search
{
    "query": "foo bar" <1>
}
----------------------------------
// SENSE:one/two/three/example_1.json (1)

\<1> Here's the explanation
  1. The path should not contain the initial snippets directory

Admonition blocks

Admonition blocks are much the same as Admonition paragraphs, except that they can be longer and contain more than just a paragraph. For instance:

[NOTE]
=========================
This note contains a list:

* foo
* bar
* baz

and some code

[source,js]
----------------------------------
{ "query": "foo bar"}
----------------------------------
=========================

This renders as:

Note

This note contains a list:

  • foo

  • bar

  • baz

and some code

{ "query": "foo bar"}

Sidebars

Sidebars are used to highlight a block of content that is outside the usual flow of text:

.Optional title
**********************************
So why does the `bulk` API have such a
funny format?  Sit down and I'll tell you
all about it!
**********************************
Optional title

So why does the bulk API have such a funny format? Sit down and I’ll tell you all about it!

Example blocks

Example blocks contain normal text which is used as an example. The title, if any, is labelled as an example and numbered:

.My first example
========================================
Text explaining the first example.
========================================

.My second example
========================================
Text explaining the second example.
========================================

This renders as:

Example 17. My first example

Text explaining the first example.

Example 18. My second example

Text explaining the second example.

Caution
The === and --- delimiters can sometimes be confused with a header, resulting in an error. To resolve this, add newlines between the delimiter and the content before and after it.

Including files

For long documentation, you probably want to break up the Asciidoc files into smaller units, and just include them where appropriate:

include::myfolder/mydoc.asciidoc[]

Paths are relative to the file which contains the include statement.

Additions and deprecations

Documentation is built for various branches, eg 0.90, 1.00, master. However, we release versions 0.90.0, 0.90.1, etc, which are all based on the 0.90 branch.

When adding new functionality to a branch, or deprecating existing functionality, you can mark the change as added, coming or deprecated. Use coming when the addition is in an as yet unreleased version of the current branch, and added when the functionality is already released.

The update_versions.pl script can be used to change coming notices to added notices when doing a new release, and can also be used to remove added, coming and deprecated notices completely.

Inline notifications

Use inline notifications for small changes, such as the addition or deprecation of individual parameters.

[horizontal]
`foo.bar`::   Does XYZ. added[0.90.4]
`foo.bar`::   Does XYZ. coming[0.90.4]
`foo.baz`::   Does XYZ. deprecated[0.90.4]
foo.bar

Does XYZ. added[0.90.4]

foo.bar

Does XYZ. coming[0.90.4]

foo.baz

Does XYZ. deprecated[0.90.4]

You can also include details about additional notes in the notifications which show up when the user hovers over it:

[horizontal]
`foo.bar`::   Does XYZ. added[0.90.4,Replaces `foo.baz`]
`foo.bar`::   Does XYZ. coming[0.90.4,Replaces `foo.baz`]
`foo.baz`::   Does XYZ. deprecated[0.90.4,Replaced by `foo.bar`]
foo.bar

Does XYZ. added[0.90.4,Replaces foo.baz]

foo.bar

Does XYZ. coming[0.90.4,Replaces foo.baz]

foo.baz

Does XYZ. deprecated[0.90.4,Replaced by foo.bar]

Section notifications

Use section notifications to mark an entire chapter or section as added/deleted. Notifications can just refer to the version in which the change was made:

==== New section

added[0.90.4]

Text about new functionality...

==== New section not yet released

coming[0.90.9]

Text about new functionality...

==== Old section

deprecated[0.90.4]

Text about old functionality...

New section

added[0.90.4]

Text about new functionality…​

New section not yet released

coming[0.90.9]

Text about new functionality…​

Old section

deprecated[0.90.4]

Text about old functionality…​

With details…​

Or they can include extra text, including more Asciidoc markup:

[[new-section]]
==== New section

added[0.90.4,Replaces `foo.bar`. See <<old-section>>]

Text about new functionality...

[[coming-section]]
==== New section not yet released

coming[0.90.9,Replaces `foo.bar`. See <<old-section>>]

Text about new functionality...

[[old-section]]
==== Old section

deprecated[0.90.4,Replace by `foo.baz`. See <<new-section>>]

Text about old functionality...

New section

added[0.90.4,Replaces foo.bar. See Old section]

Text about new functionality…​

Old section

deprecated[0.90.4,Replace by foo.baz. See New section]

Text about old functionality…​

Experimental and Beta

APIs or parameters that are experimental or in beta can be marked as such, using markup similar to that used in Additions and deprecations. For instance:

[[new-feature]]
=== New experimental feature

experimental[]

experimental[Custom text goes here]

Text about new feature...

[[old-feature]]
=== Established feature

This feature has been around for a while, but we're adding
a new experimental parameter:

[horizontal]
`established_param`::  This param has been around for ages and won't change.
`experimental_param`:: experimental[] This param is experimental and may change in the future.
`experimental_param`:: experimental[Custom text goes here] This param is experimental and may change in the future.

New experimental feature

experimental[]

experimental[Custom text goes here]

Text about new feature…​

Established feature

This feature has been around for a while, but we’re adding a new experimental parameter:

established_param

This param has been around for ages and won’t change.

experimental_param

experimental[] This param is experimental and may change in the future.

experimental_param

experimental[Custom text goes here] This param is experimental and may change in the future.

[[new-beta-feature]]
=== New beta feature

beta[]

beta[Custom text goes here]

Text about new feature...

[[old-beta-feature]]
=== Established feature

This feature has been around for a while, but we're adding
a new beta parameter:

[horizontal]
`established_param`::  This param has been around for ages and won't change.
`beta_param`:: beta[] This param is beta and may change in the future.
`beta_param`:: beta[Custom text goes here] This param is beta and may change in the future.

New beta feature

beta[]

beta[Custom text goes here]

Text about new feature…​

Established feature

This feature has been around for a while, but we’re adding a new beta parameter:

established_param

This param has been around for ages and won’t change.

beta_param

beta[] This param is experimental and may change in the future.

beta_param

beta[Custom text goes here] This param is beta and may change in the future.

Images

Any images you want to include should be saved in a folder in your repo, and included using a path relative to the document where the image:: statement appears.

[[cat]]
.A scaredy cat
image::resources/cat.jpg[Alt text]

A link to <<cat>>
Alt text
Figure 1. A scaredy cat

A link to A scaredy cat

Width and height

The width and/or height of the image can be specified in pixels or as a percentage:

image::resources/cat.jpg["Alt text",width=50]
image::resources/cat.jpg["Alt text",width="20%"]
Alt text
Alt text

Alignment

Images are left-aligned by default, but they can be centred or right-aligned:

image::resources/cat.jpg["Alt text",width=100,align="left"]
image::resources/cat.jpg["Alt text",width=100,align="right"]
image::resources/cat.jpg["Alt text",width=100,align="center"]
Alt text
Alt text
Alt text

Tables

In general, tables are frowned upon in DocBook as they don’t display well in formats other than HTML, eg PDF, ePub, etc.

It’s almost always better to use Horizontal definition lists instead, but if you really want to use tables, you can read about them here.

Controlling chunking

In Basic book structure, we said that each part or chapter generates a new chunk or HTML file. For more complex documentation, you may want the first level of sections to also generate new chunks.

For instance, in the ES reference docs, we have:

= Search APIs               # part

== Request body search      # chapter

=== Query                   # section level 1

=== From/Size               # section level 1

... etc ...

There are too many parameters for ``Request body search'' to list them all on one page. In this case, it is worth turning on chunking for top level sections.

Enabling section chunking

To enable section chunking when building docs in a local repository, pass the --chunk parameter:

build_docs.pl --doc path/to/index.asciidoc --chunk 1

To enable section chunking when building docs for the website, add chunk: 1 to the conf.yaml file in the docs repo.

contents:
    -
        title:      Elasticsearch reference
        prefix:     elasticsearch/reference
        repo:       elasticsearch
        index:      docs/reference/index.asciidoc
        chunk:      1 (1)
  1. Chunking is enabled for this book

Chunking selected sections

If you enable session chunking, you will probably find that you have a few short sections which you want to keep on the same page.

To do this, you can use the [float] marker before a section header, to tell Asciidoc that what follows isn’t a ``real'' header:

[[chapter-one]]
== chapter               # new chunk

[[section-one]]
=== Section one         # new chunk

[[section-two]]
[float]
=== Section two         # same chunk

[[section-three]]
=== Section three       # new chunk

The above would produce three HTML files, named for their IDs:

  • chapter-one.html

  • section-one.html which would also contain ``Section two''

  • section-three.html

To link to `Section two'' from an external document, you would use the URL: `section-one.html#section-two