Structure

This document provides a brief explanation of SaltStack and reviews the structure of this repository.

Introduction

Salt delivers a dynamic communication bus for infrastructures that can be used for orchestration, remote execution, configuration management and much more.

In short, Salt works by allowing a master to tell minions to execute states with pillar as defined by the application of targeting rules against the minions' grains.

Some definitions might help:

• Master: server that tells the minions to execute states
• Minion: client that executes states as instructed by the master
• State: command, procedure or check
• Pillar: variable context of a minion used by states (eg, database connection parameters)
• Grains: static attributes of a minion (eg, OS)
• Targeting: definitions for application of pillars and states into minions

Now read that explanation again.

Pillar vs. Grain

Although at first sight the Pillar and Grain seem very similar, a grain value is determined once, when the minion service is started and generally remains the same forever, while a pillar's value is evaluated on every state execution.

This means that grains define general characteristics about a machine (such as its roles or environment), while pillar define specific parameters for salt states (database connection, branch name, etc).

Structure

The main structure of the repository is the following:

salt-config/
  cloud/
  pillar/
  salt/
  vagrant/

Cloud

This directory contains the configurations for Salt-Cloud, a system used to provision virtual machines on various public clouds via a cleanly controlled profile and mapping system.

It consists only of two files:

cloud/
  profiles
  providers

For more information on Salt-Cloud refer to Salt-Cloud.

Providers

Providers are used to abstract all the service-specific configurations away from profiles.

Since we only use AWS EC2 Ubuntu 14.04 machines, we have a single provider:

ec2-public-www:
  driver: ec2
  image: ami-d05e75b8
  minion:
    master: salt.pjtracker.com
...

Profiles

In our configuration, profiles only define the instance type (size) and grains, everything else is defined by the provider.

For example, the profile for a demo instance for pjtracker has the following attributes:

pjtracker-demo:
  provider: ec2-public-www
  size: t2.micro
  grains:
    box_type: ec2
    app: pjtracker
    env: prod

Note that any attribute defined in a provider can be overridden in a profile, but we try to avoid that.

Pillar

Pillar allow us to parametrize state execution, by providing a context that is defined dynamically immediately before states are applied. The structure of this directory is the following:

pillar/
  auth/
    ec2.sls
    vagrant.sls
  pjtracker/
    dev.sls
    prod.sls
    shared.sls
    ...
  sonar.sls
  top.sls

Definition

Each file (except the targeting file top.sls) consists of a set of attributes. For example, the dev environment for pjtracker (in pjtracker/dev.sls), contains:

app:
  name: pjtracker
  settings: 'tracker.settings.dev'
  root: apps/pjtracker
  static_root: apps/pjtracker/static
  ...

All the pillar definitions applied to a machine are merged into a single (Python) dictionary, available to salt states and templates as {{ pillar }}.

Targeting

The targeting rules are defined in the file top.sls, always by using grains. In the following sample we use role, app, env and sub_env:

'G@roles:qua or G@roles:ci':
  - match: compound
  - sonar

'G@app:pjtracker and G@env:prod':
  - match: compound
  - pjtracker.prod

'G@app:pjtracker and G@sub_env:shared':
  - match: compound
  - pjtracker.shared

Depending on the salt state targeting, some pillar data might be unnecessary for some boxes. In those cases it's good practice to be specific when targeting, to avoid cluttering the minions with useless information as well as protect sensible information (eg, API keys). That's why we only apply sonar information to qua and ci machines, since other machines would not use that pillar data.

Salt

This directory defines that states are applied to every machine. The structure is very similar to the pillar one:

salt/
  core/
    git.sls
    python.sls
    ...
  pjtracker/
    environment.sls
    repository.sls
    ...
  services/
    jenkins.sls
    postgresql.sls
    ...
  top.sls

Targeting

Targeting is also applied on grains, but since states are not environment-specific we never user env and sub_env. A sample from the file top.sls shows that:

'G@box_type:ec2':
  - match: compound
  - core.swap

'G@app:pjtracker':
  - match: compound
  - pjtracker.service
  - services.nginx

'G@roles:ci':
  - match: compound
  - services.jenkins
  - services.sonarqube.scanner

Using Pillar

Since the salt targeting does not depend on environments, two machines with the same roles and application will execute exactly the same states regardless of environment, and any environment-specific difference is managed by using pillar data to parametrize salt states. For instance, the state to clone the repository for pjtracker uses pillar a lot:

pjtracker-repo:
  git.latest:
    - name: {{ pillar['app']['repository']['url'] }}
    - target: {{ pillar['auth']['home'] }}/{{ pillar['app']['root'] }}
    - branch: {{ pillar['app']['repository']['branch'] }}
    - force_checkout: {{ pillar['app']['repository']['checkout'] }}
    - force_reset: {{ pillar['app']['repository']['checkout'] }}
    - user: {{ pillar['auth']['user'] }}
    - require:
      - file: ssh-config
      - ssh_known_hosts: ssh-github-host

We also use pillar in configuration files, for which we need to specify the template engine used for the file handler, or we'll get a rendering error. For example, the state to update the nginx configuration (located in services/nginx/init.sls) uses jinja:

nginx-config:
  file.managed:
    - name: /etc/nginx/nginx.conf
    - source: salt://services/nginx/nginx.conf
    - template: jinja

Vagrant

This directory contains the configuration to use Vagrant:

vagrant/
  minion.example
  Vagrantfile

For more information refer to Deployment with Vagrant.