Ansible configurations for Ceph.com infrastructure
Shell Python Perl JavaScript Perl 6 HTML
Clone or download
Latest commit 44305b7 Aug 11, 2018
Permalink
Failed to load latest commit information.
callback_plugins If a YAML error is hit, log the raw object Jul 18, 2016
roles libev-devel isn't available in RHEL7.6 Aug 10, 2018
tools tools: Force time synchronization in FOG images Apr 10, 2018
.gitignore A callback plugin that logs failure messages Aug 5, 2015
README.rst doc: wrap README.rst at 80 characters Jul 23, 2015
ansible.cfg Reduce 'internal_poll_interval' to stem the CPU usage tide Feb 27, 2017
ansible_managed.yml Use strategy: free on some playbooks Aug 1, 2016
cephlab.yml pcp: Disable role for now Feb 26, 2018
cobbler.yml Add RHEL 7.6 Aug 10, 2018
common.yml
dhcp-server.yml Add dhcp-server role Apr 3, 2018
downstream_setup.yml downstream_setup: Add 'become' privilege escalation directive Jun 30, 2016
firmware.yml firmware: New role. Adding BIOS and BMC support for mira Nov 4, 2016
fog-server.yml fog-server: Create role Jan 24, 2018
gateway.yml Add missing 'become' directive to a few more roles Jun 30, 2016
long_running_cluster.yml Create long_running_cluster role Mar 28, 2018
nameserver.yml Remove sudo role; replace with 'become' setting Jun 27, 2016
nsupdate_web.yml Add role for nsupdate_web Feb 22, 2017
ntp-server.yml ntp-server: Create NTP server role Jan 23, 2018
packages.yml Adds a new role and playbook for package management Dec 3, 2015
paddles.yml Remove sudo role; replace with 'become' setting Jun 27, 2016
pcp.yml Use strategy: free on some playbooks Aug 1, 2016
public_facing.yml public_facing: Create base role config Feb 23, 2017
puddle.yml add puddle role and playbook Apr 28, 2015
pulpito.yml Adds the pulpito playbook Sep 16, 2015
testnodes.yml
teuthology.yml Use strategy: free on some playbooks Aug 1, 2016
users.yml Use "become: true" for users role Oct 28, 2016
vmhost.yml Add missing 'become' directive to a few more roles Jun 30, 2016

README.rst

Overview

This project is meant to store ansible roles for managing the nodes in the ceph testing labs.

Inventory

As this repo only contains roles, it does not define the ansible inventory or any associated group_vars or host_vars. However, it does depend on these things existing in a separate repository or otherwise accesible by these roles when they are used. Any vars a role needs should be added to its defaults/main.yml file to document what must be defined per node or group in your inventory.

This separation is important because we have multiple labs we manage with these same roles and each lab has different configuration needs. We call these our secrets or *-secrets repos throughout the rest of the documention and in the roles.

Besides the inventory, secrets repos also may contain certain secret or encrypted files that we can not include in ceph-cm-ansible for various reasons.

The directory structure for one of our secrets repos is:

├── ansible
    ├── inventory
    │   ├── group_vars
    │   │   ├── all.yml
    │   │   ├── cobbler.yml
    │   │   ├── testnodes.yml
    │   │   ├── teuthology.yml
    │   │   └── typica.yml
    │   └── sepia
    └── secrets
        └── entitlements.yml

Refer to Step 2 below for instructions on how to setup a secrets repo for use by ceph-cm-ansible. If set up this way, -i is not necessary for ansible-playbook to find the repo. However, you can choose your own setup and point to the secrets repo with -i if you prefer.

NOTE: Some playbooks require specific groups to be defined in your inventory. Please refer to hosts in the playbook you want to use to ensure you've got the proper groups defined.

Where should I put variables?

All variables should be defined in defaults/main.yml for the role they're primarily used in. If the variable you're adding can be used in multiple roles define it in defaults/main.yml for both roles. If the variable can contain a reasonable default value that should work for all possible labs then define that value in defaults/main.yml as well. If not, you should still default the variable to something, but make the tasks that use the variable either fail gracefully without that var or prompt the user to define it if it's mandatory.

If the variable is something that might need to be defined with a value specific to the lab in use, then it'll need to be added to your secrets repo as well. Variables in group_vars/all.yml will apply to all nodes unless a group_var file exists that is more specific for that node. For example, if you define the var foo: bar in all.yml and the node you're running ansible against exists in the testnodes group and there is a group_vars/testnodes.yml file defined with foo: baz included in it then the role using the variable will use the value defined in testnodes.yml. The playbook you're using knows which group_var file to use because of the hosts value defined for it.

Setting up a local dev environment

We assume that your SSH key is present and active for passwordless access to the "ubuntu" shell user on the hosts that ansible will manage.

Step 1: Install ansible

You can use pip:

pip install ansible

or use the OS package manager:

yum install ansible

Step 2: Set up secrets repository

Clone the secrets repository and symlink the hosts and secrets directories into place:

cd $HOME/dev/
git clone git@..../ceph-ansible-secrets.git

sudo mv /etc/ansible/hosts /etc/ansible/hosts.default

sudo ln -s /path/to/ceph-ansible-secrets/ansible/inventory /etc/ansible/hosts
sudo ln -s /path/to/ceph-ansible-secrets/ansible/secrets /etc/ansible/secrets

Step 3: Clone the main Ceph ansible repo

Clone the main Ceph ansible repository:

git clone git@..../ceph-cm-ansible.git
cd ceph-cm-ansible

Step 4 (Optional) Modify hosts files

If you have any new hosts on which you'd like to run ansible, or if you're using separate testing VMs, edit the files in /etc/ansible/hosts to add your new (or testing) hosts:

vi /etc/ansible/hosts/<labname>

If you don't need to test on any new hosts, you can skip this step and just use /etc/ansible/hosts as-is.

Step 5: Run ansible-playbook

You can now run ansible-playbook:

vi myplaybook.yml
ansible-playbook myplaybook.yml -vv --check --diff

This will print a lot of debugging output to your console.

Adding a new host to ansible

Ansible runs using the "cm" shell account.

Let's say you've created a new VM host using downburst. At this point you should have a new VM with the "ubuntu" UID present. The problem is that Ansible uses the "cm" user. In order to get that UID set up:

  1. Add your host to the inventory. Look in your lab's secrets repository, in the ansible/inventory/ directory, and add your new node.

  2. Run the cephlab.yml playbook, limited to your new host "mynewhost":

    ansible-playbook -vv --limit mynewhost cephlab.yml