Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Style end edit changes to collections doc #59658

Merged
merged 2 commits into from
Jul 26, 2019
Merged

Style end edit changes to collections doc #59658

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
bd31ec7
copy-edit of file
samccann Jul 26, 2019
0fb0183
Apply suggestions from code review
samccann Jul 26, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
167 changes: 101 additions & 66 deletions docs/docsite/rst/dev_guide/collections_tech_preview.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,20 +9,21 @@ Collections

Collections are a distribution format for Ansible content. They can be used to
package and distribute playbooks, roles, modules, and plugins.
You will be able to publish and use collections through `Ansible's Galaxy repository <https://galaxy.ansible.com>`_.
You can publish and use collections through `Ansible Galaxy <https://galaxy.ansible.com>`_.

.. important::
This feature is available in Ansible 2.8 as a *Technology Preview* and therefore is not fully supported. It should only be used for testing and should not be deployed in a production environment.
This feature is available in Ansible 2.8 as a *Technology Preview* and therefore is not fully supported. It should only be used for testing and should not be deployed in a production environment.
Future Galaxy or Ansible releases may introduce breaking changes.


.. contents::
:local:
:depth: 2

Collection structure
====================

Collections follow a simple data structure. None of the directories are required unless you have specific content that belongs in one of them. They do require a ``galaxy.yml`` file at the root level of the collection. This file contains all of the metadata that Galaxy
Collections follow a simple data structure. None of the directories are required unless you have specific content that belongs in one of them. A collection does require a ``galaxy.yml`` file at the root level of the collection. This file contains all of the metadata that Galaxy
and other tools need in order to package, build and publish the collection.::

collection/
Expand All @@ -47,24 +48,26 @@ and other tools need in order to package, build and publish the collection.::


.. note::
* We will only accept ``.yml`` extensions for galaxy.yml.
* A full structure can be found at `Draft collection <https://github.com/bcoca/collection>`_
* Ansible only accepts ``.yml`` extensions for galaxy.yml.
* See the `draft collection <https://github.com/bcoca/collection>`_ for an example of a full collection structure.
* Not all directories are currently in use. Those are placeholders for future features.


galaxy.yml
----------

A collection must have a ``galaxy.yml`` file that contains the necessary information to build a collection artifact.
See :ref:`collections_galaxy_meta` for details on how this file is structured.
See :ref:`collections_galaxy_meta` for details.


docs directory
---------------

Keep general documentation for the collection here. Plugins and modules will still keep their specific documentation embedded as Python docstrings. Use the ``docs`` folder to describe how to use the roles and plugins the collection provides, role requirements, and so on. Currently we are looking at Markdown as the standard format for documentation files, but this is subject to change.
Keep general documentation for the collection here. Plugins and modules still keep their specific documentation embedded as Python docstrings. Use the ``docs`` folder to describe how to use the roles and plugins the collection provides, role requirements, and so on. Currently we are looking at Markdown as the standard format for documentation files, but this is subject to change.

We are `updating ansible-doc <https://github.com/ansible/ansible/pull/57764>`_ to allow showing documentation for plugins inside a collection::
Use ``ansible-doc`` to view documentation for plugins inside a collection:

.. code-block:: bash

ansible-doc -t lookup mycol.myname.lookup1

Expand All @@ -74,7 +77,7 @@ The ``ansible-doc`` command requires the fully qualified collection name (FQCN)
plugins directory
------------------

Add a 'per plugin type' specific subdirectory here, including ``module_utils`` which is usable not only by modules, but by any other plugin by using their FQCN. This is a way to distribute modules, lookups, filters, and so on, without having to import a role in every play.
Add a 'per plugin type' specific subdirectory here, including ``module_utils`` which is usable not only by modules, but by any other plugin by using their FQCN. This is a way to distribute modules, lookups, filters, and so on, without having to import a role in every play.


roles directory
Expand All @@ -83,7 +86,7 @@ roles directory
Collection roles are mostly the same as existing roles, but with a couple of limitations:

- Role names are now limited to contain only lowercase alphanumeric characters, plus ``_`` and start with an alpha character.
- Roles cannot have their own plugins any more. The plugins must live in the collection ``plugins`` directory and will be accessible to the collection roles.
- Roles in a collection cannot contain plugins any more. Plugins must live in the collection ``plugins`` directory tree. Each plugin is accessible to all roles in the collection.

The directory name of the role is used as the role name. Therefore, the directory name must comply with the
above role name rules.
Expand Down Expand Up @@ -111,83 +114,106 @@ TBD. Expect tests for the collection itself to reside here.
.. _creating_collections:

Creating collections
====================
======================

This is currently a work in progress with some basic commands being integrated into the existing ``ansible-galaxy``
command line tool that is included with Ansible.
To create a collection:

#. Initialize a collection with :ref:`ansible-galaxy collection init<creating_collections_skeleton>` to create the skeleton directory structure.
#. Add your content to the collection.
#. Build the collection into a collection artifact with :ref:`ansible-galaxy collection build<building_collections>`.
#. Publish the collection artifact to Galaxy with :ref:`ansible-galaxy collection publish<publishing_collections>`.

A user can then install your collection on their systems.

.. note::
Any references to ``ansible-galaxy`` below will be of a 'working version' that is in development for the 2.9
release. As such, the command and this documentation section is subject to frequent changes.

Currently the ``ansible-galaxy collection`` command implements the following sub commands:

* ``init``: Create a basic collection skeleton based on the default template included with Ansible or your own.
* ``init``: Create a basic collection skeleton based on the default template included with Ansible or your own template.
* ``build``: Create a collection artifact that can be uploaded to Galaxy or your own repository.
* ``publish``: Publish a built collection artifact to Galaxy.
* ``install``: Install one or multiple collections.
* ``install``: Install one or more collections.

To learn more about the ``ansible-galaxy`` cli tool, see the :ref:`ansible-galaxy` man page.

.. _creating_collections_skeleton:

To learn more about the ``ansible-galaxy`` cli tool, read the :ref:`ansible-galaxy` man page.
Creating a collection skeleton
------------------------------

In the end, to get started with authoring a new collection it should be as simple as:
To start a new collection:

.. code-block:: bash

collection_dir#> ansible-galaxy collection init namespace.collection
collection_dir#> ansible-galaxy collection init my_namespace.my_collection

And then populating the directories with the content you want inside the collection. See
https://github.com/bcoca/collection to get a better idea of what can be placed inside a collection.
Then you can populate the directories with the content you want inside the collection. See
https://github.com/bcoca/collection to get a better idea of what you can place inside a collection.


.. _building_collections:

Building collections
====================
--------------------

To build a collection, run ``ansible-galaxy collection build`` from inside the root directory of the collection:

Run ``ansible-galaxy collection build`` from inside the collection's root directory to build a collection. This creates
.. code-block:: bash

collection_dir#> ansible-galaxy collection build my_namespace.my_collection

This creates
a tarball of the built collection in the current directory which can be uploaded to Galaxy.::

collection/
my_collection/
├── galaxy.yml
├── ...
├── namespace_name-collection_name-1.0.12.tar.gz
├── my_namespace_name-collection_name-1.0.0.tar.gz
└── ...


.. note::
Certain files and folders are excluded when building the collection artifact. This is not currently configurable
and is a work in progress so the collection artifact may contain files you would not wish to distribute.

This tarball itself can be used to install the collection on target systems. It is mainly intended to upload to Galaxy
as a distribution method, but you should be able to use it directly.
This tarball is mainly intended to upload to Galaxy
as a distribution method, but you can use it directly to install the collection on target systems.

.. _publishing_collections:

Publishing collections
======================
----------------------

You can publish collections to Galaxy using the ``ansible-galaxy collection publish`` command or the Galaxy UI itself.

.. note:: Once you upload a version of a collection, you cannot delete or modify that version. Ensure that everything looks okay before you upload it.

Upload using ansible-galaxy
---------------------------
^^^^^^^^^^^^^^^^^^^^^^^^^^^

You can upload collection artifacts with ``ansible-galaxy``, as shown in the following example:
To upload the collection artifact with the ``ansible-galaxy`` command:

.. code-block:: bash

ansible-galaxy collection publish path/to/namespace_name-collection_name-1.0.12.tar.gz --api-key=SECRET
ansible-galaxy collection publish path/to/my_namespace-my_collection-1.0.0.tar.gz --api-key=SECRET

The above command triggers an import process, just as if the collection has been uploaded through the Galaxy website.
The command will wait until the import process has completed before reporting the status back. If you wish to continue
The above command triggers an import process, just as if you uploaded the collection through the Galaxy website.
The command waits until the import process completes before reporting the status back. If you wish to continue
without waiting for the import result, use the ``--no-wait`` argument and manually look at the import progress in your
`My Imports <https://galaxy.ansible.com/my-imports/>`_ page.

The API key is a secret token used by Ansible Galaxy to protect your content. You can find your API key at your
`Galaxy profile preferences <https://galaxy.ansible.com/me/preferences>`_ page.

Upload from the Galaxy website
------------------------------
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Go to the `My Content <https://galaxy.ansible.com/my-content/namespaces>`_ page, and click the **Add Content** button on one of your namespaces. From
the **Add Content** dialogue, click **Upload New Collection**, and select the collection archive file from your local
filesystem.
To upload your collection artifact directly on Galaxy:

#. Go to the `My Content <https://galaxy.ansible.com/my-content/namespaces>`_ page, and click the **Add Content** button on one of your namespaces.
#. From the **Add Content** dialogue, click **Upload New Collection**, and select the collection archive file from your local filesystem.

When uploading collections it doesn't matter which namespace you select. The collection will be uploaded to the
namespace specified in the collection metadata in the ``galaxy.yml`` file. If you're not an owner of the
Expand All @@ -206,30 +232,29 @@ will be the version displayed everywhere in Galaxy; however, users will still be


Installing collections
======================
----------------------

The recommended way to install a collection is:
You can use the ``ansible-galaxy collection install`` command to install a collection on your system. The collection by default is installed at ``/path/ansible_collections/my_namespace/my_collection``. You can optionally add the ``-p`` option to specify an alternate location.

To install a collection hosted in Galaxy:

.. code-block:: bash

# Will install the collection to /path/ansible_collections/mynamespace/mycollection
ansible-galaxy collection install mynamespace.mycollection -p /path
ansible-galaxy collection install my_namespace.my_collection -p /path

assuming the collection is hosted in Galaxy.

You can also use a tarball resulting from your build:
You can also directly use the tarball from your build:

.. code-block:: bash

# Will install the collection to ./collections/ansible_collections/mynamespace/mycollection
ansible-galaxy collection install mynamespace-mycollection-0.1.0.tar.gz -p ./collections/ansible_collections
ansible-galaxy collection install my_namespace-my_collection-1.0.0.tar.gz -p ./collections/ansible_collections

.. note::
The install command will automatically append the path ``ansible_collections`` to the one specified unless the
The install command automatically appends the path ``ansible_collections`` to the one specified with the ``-p`` option unless the
parent directory is already in a folder called ``ansible_collections``.


As a path you should use one of the values configured in :ref:`COLLECTIONS_PATHS`. This is also where Ansible itself will expect to find collections when attempting to use them.
You should use one of the values configured in :ref:`COLLECTIONS_PATHS` for your path. This is also where Ansible itself will expect to find collections when attempting to use them.

You can also keep a collection adjacent to the current playbook, under a ``collections/ansible_collections/`` directory structure.

Expand All @@ -238,23 +263,33 @@ You can also keep a collection adjacent to the current playbook, under a ``colle
play.yml
├── collections/
│ └── ansible_collections/
│ └── myname/
│ └── mycol/<collection structure lives here>
│ └── my_namespace/
│ └── my_collection/<collection structure lives here>


Installing an older version of a collection
-------------------------------------------

By default ``ansible-galaxy`` installs the latest collection that is available but you can add a version range
identifier to install a specific version as follows:
identifier to install a specific version.

To install the 1.0.0 version of the collection:

.. code-block:: bash

# Install the collection at 1.0.0
ansible-galaxy collection install mynamespace.mycollection:1.0.0
ansible-galaxy collection install my_namespace.my_collection:1.0.0

To install the 1.0.0-beta.1 version of the collection:

.. code-block:: bash

# Install the collection at 1.0.0-beta.1
ansible-galaxy collection install mynamespace.mycollection:==1.0.0-beta.1
ansible-galaxy collection install my_namespace.my_collection:==1.0.0-beta.1

To install the collections that are greater than or equal to 1.0.0 or less than 2.0.0:

.. code-block:: bash

# Only install the collections that are greater than or equal to 1.0.0 or less than 2.0.0
ansible-galaxy collection install mynamespace.mycollection:>=1.0.0,<2.0.0
ansible-galaxy collection install my_namespace.my_collection:>=1.0.0,<2.0.0


You can specify multiple range identifiers which are split by ``,``. You can use the following range identifiers:
Expand All @@ -274,8 +309,8 @@ You can specify multiple range identifiers which are split by ``,``. You can use

.. _collection_requirements_file:

Install requirements file
-------------------------
Install multiple collections with a requirements file
-----------------------------------------------------

You can also setup a ``requirements.yml`` file to install multiple collections in one command. This file is a YAML file in the format:

Expand All @@ -284,10 +319,10 @@ You can also setup a ``requirements.yml`` file to install multiple collections i
---
collections:
# With just the collection name
- namespace.collection
- my_namespace.my_collection

# With the collection name, version, and source options
- name: namespace.collection
- name: my_namespace.my_collection
version: 'version range identifiers (default: ``*``)'
source: 'The Galaxy URL to pull the collection from (default: ``--api-server`` from cmdline)'

Expand All @@ -297,13 +332,13 @@ The ``version`` key can take in the same range identifier format documented abov
Using collections
=================

Once installed, you can reference collection content by its FQCN:
Once installed, you can reference a collection content by its FQCN:

.. code-block:: yaml

- hosts: all
tasks:
- myname.mycol.mymodule:
- my_namespace.my_collection.mymodule:
option1: value

This works for roles or any type of plugin distributed within the collection:
Expand All @@ -313,12 +348,12 @@ This works for roles or any type of plugin distributed within the collection:
- hosts: all
tasks:
- include_role:
name : myname.mycol.role1
- myname.mycol.mymodule:
name : my_namespace.my_collection.role1
- my_namespace.mycollection.mymodule:
option1: value

- debug:
msg: '{{ lookup("myname.mycol.lookup1", 'param1')| myname.mycol.filter1 }}'
msg: '{{ lookup("my_namespace.my_collection.lookup1", 'param1')| my_namespace.my_collection.filter1 }}'


To avoid a lot of typing, you can use the ``collections`` keyword added in Ansbile 2.8:
Expand All @@ -328,15 +363,15 @@ To avoid a lot of typing, you can use the ``collections`` keyword added in Ansbi

- hosts: all
collections:
- myname.mycol
- my_namespace.my_collection
tasks:
- include_role:
name: role1
- mymodule:
option1: value

- debug:
msg: '{{ lookup("myname.mycol.lookup1", 'param1')| myname.mycol.filter1 }}'
msg: '{{ lookup("my_namespace.my_collection.lookup1", 'param1')| my_namespace.my_collection.filter1 }}'

This keyword creates a 'search path' for non namespaced plugin references. It does not import roles or anything else.
Notice that you still need the FQCN for non-action or module plugins.