[NEW] A Beginner's Guide to Salt

docs/applications/configuration-management/beginners-guide-to-salt/index.md

@@ -0,0 +1,333 @@
+---
+author:
+  name: Linode
+  email: docs@linode.com
+description: 'A look into Salt''s primary components, features, and configurations for the new SaltStack user'
+keywords: ["salt", "automation", "saltstack", "configuration management"]
+license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)'
+modified: 2018-10-16
+modified_by:
+  name: Linode
+published: 2018-10-16
+title: A Beginner's Guide to Salt
+external_resources:
+ - '[SaltStack Documentation](https://docs.saltstack.com/)'
+---
+
+[Salt](https://www.saltstack.com) (also referred to as *SaltStack*) is a Python-based configuration management and orchestration system. Salt uses a master/client model in which a dedicated Salt *master* server manages one or more Salt *minion* servers. Two of Salt's primary jobs are:
+
+-   Remotely executing commands across a set of minions
+
+-   Applying Salt *states* to a set of minions (referred to generally as *configuration management*)
+
+This guide will introduce the core concepts that Salt employs to fulfill these jobs.
+
+## Masters and Minions
+
+The Salt master is a server that acts as a command-and-control center for its minions, and it is where Salt's [remote execution](#remote-execution) commands are run from. For example, this command reports the current disk usage for each of the minions that the master controls:
+
+    salt '*' disk.usage
+
+Many other commands are available. This installs NGINX on the minion named `webserver1`:
+
+    salt 'webserver1' pkg.install nginx
+
+Salt minions are your servers that actually run your applications and services. Each minion has an ID assigned to it (which can be automatically generated from the minion's hostname), and the Salt master can refer to this ID to [target commands to specific minions](#targeting-minions).
+
+{{< note >}}
+When using Salt, you should configure and manage your minion servers from the master as much as possible, instead of logging into them directly via SSH or another protocol.
+{{< /note >}}
+
+To enable all of these functions, the Salt master server runs a daemon named **salt-master**, and the Salt minion servers run a daemon named **salt-minion**.
+
+### Authentication
+
+Communication between the master and minions is performed over the [ZeroMQ](https://docs.saltstack.com/en/latest/topics/development/topology.html) transport protocol, and all communication is encrypted with public/private keypairs. A keypair is generated by a minion when Salt is first installed on it, after which the minion will send its public key to the master. You will need to accept the minion's key from the master; communication can then proceed between the two.
+
+## Remote Execution
+
+Salt offers a [very wide array](https://docs.saltstack.com/en/latest/ref/modules/all/) of remote *execution modules*. An execution module is a collection of related functions that you can run on your minions **from the master**. For example:
+
+    salt 'webserver1' npm.install gulp
+
+In this command `npm` is [the module](https://docs.saltstack.com/en/latest/ref/modules/all/salt.modules.npm.html) and `install` is [the function](https://docs.saltstack.com/en/latest/ref/modules/all/salt.modules.npm.html#salt.modules.npm.install). This command installs the [Gulp](https://gulpjs.com) Node.js package via the Node Package Manager (NPM). Other functions in the `npm` module handle uninstalling NPM packages, listing installed NPM packages, and related tasks.
+
+The execution modules that Salt makes available represent system administration tasks that you would otherwise perform in a shell, including but not limited to:
+
+-   Creating and managing [system users](https://docs.saltstack.com/en/latest/ref/modules/all/salt.modules.user.html)
+
+-   Installing and uninstalling [software](https://docs.saltstack.com/en/latest/ref/modules/all/salt.modules.pkg.html)
+
+-   Editing or creating configuration [files](https://docs.saltstack.com/en/latest/ref/modules/all/salt.modules.file.html#module-salt.modules.file)
+
+{{< note >}}
+You can also [write your own](/docs/applications/configuration-management/create-a-salt-execution-module/) execution modules.
+{{< /note >}}
+
+### cmd.run
+
+The `cmd.run` function is used to run arbitrary commands on your minions from the master:
+
+    salt '*' cmd.run 'ls -l /etc'
+
+This would return the contents of `/etc` on each minion.
+
+{{< note >}}
+Where possible, it's better to use execution modules than to "shell out" with `cmd.run`.
+{{< /note >}}
+
+## States, Formulas, and the Top File
+
+The previous section described how to use remote execution to perform specific actions on a minion. With remote execution, you could adminster a minion by entering a series of such commands.
+
+Salt offers another way to configure a minion in which you declare **the state that a minion should be in**. This kind of configuration is called a Salt *state*, and the methodology is referred to generally as *configuration management*.
+
+The distinction between the two styles is subtle; to illustrate, here's how installing NGINX is interpreted in each methodology:
+
+-   **Remote execution**: "Install NGINX on the minion"
+
+-   **Configuration management**: "NGINX should be installed on the minion"
+
+Salt states are defined in *state files*. Once you have recorded your states, you then *apply* them to a minion. Salt analyzes the state file and determines what it needs to do to make sure that the minion satisfies the state's declarations.
+
+{{< note >}}
+This sometimes results in the same command that would be run via remote execution, but sometimes it doesn't. In the NGINX example, if Salt sees that NGINX was already installed previously, it won't invoke the package manager again when the state is applied.
+{{< /note >}}
+
+### Anatomy of a State
+
+Here's an example state file which ensures that: rsync and curl are installed; NGINX is installed; and NGINX is run and enabled to run at boot:
+
+{{< file "/srv/salt/webserver_setup.sls">}}
+network_utilities:
+  pkg.installed:
+    - pkgs:
+      - rsync
+      - curl
+
+nginx_pkg:
+  pkg.installed:
+    - name: nginx
+
+nginx_service:
+  service.running:
+    - name: nginx
+    - enable: True
+    - require:
+      - pkg: nginx_pkg
+{{< /file >}}
+
+State files end with the extension `.sls` (**S**a**L**t **S**tate). State files can have one or more *state declarations*, which are the top-level sections of the file (`network_utilities`, `nginx_pkg`, and `nginx_service` in the above example). State declarations IDs are arbitrary, so you can name them however you prefer.
+
+{{< note >}}
+If you were to name the ID to be the same as the relevant installed package, then you do not need to specify the `- name` option, as it will be inferred from the ID. For example, this snippet also installs NGINX:
+
+{{< file >}}
+nginx:
+  pkg.installed
+{{< /file >}}
+
+The same name/ID inference convention is true for other Salt modules.
+{{< /note >}}
+
+State declarations contain *state modules*. State modules are distinct from execution modules but often perform similar jobs. For example, a `pkg` state module exists with functions analogous to the `pkg` execution module, as with the `pkg.installed` state function and the `pkg.install` execution function. As with execution modules, Salt provides a [wide array](https://docs.saltstack.com/en/latest/ref/states/all/) of state modules for you to use.
+
+{{< note >}}
+State declarations are not necessarily applied in the order they appear in a state file, but you can specify that a declaration depends on another one using the `require` option. This is the case in the above example; Salt will not attempt to run and enable NGINX until it is installed.
+{{< /note >}}
+
+State files are really just collections of dictionaries, lists, strings, and numbers that are then interpreted by Salt. By default, Salt uses the [YAML syntax](https://docs.saltstack.com/en/latest/topics/yaml/) for representing states.
+
+State files are often kept on the Salt master's filesystem, but they can also be stored in [other fileserver locations](https://docs.saltstack.com/en/latest/ref/file_server/backends.html), like a Git repository (in particular, GitHub).
+
+### Applying a State to a Minion
+
+To apply a state to a minion, use the `state.apply` function from the master:
+
+    salt `webserver1` state.apply webserver_setup
+
+This command applies the example `webserver_setup.sls` state to a minion named `webserver1`. When applying the state, the `.sls` suffix is not mentioned. All of the state declarations in the state file are applied.
+
+### Salt Formulas
+
+Formulas are just collections of states that together configure an application or system component on a minion. Formulas are usually organized across several different `.sls` files. Splitting a formula's states up across different files can make it easier to organize your work. State declarations can include and reference declarations across other files.
+
+Formulas that are sufficiently generic are often shared on GitHub to be used by others. The SaltStack organization maintains a [collection of popular formulas](https://github.com/saltstack-formulas). Salt's documentation has [a guide on using a formula](https://docs.saltstack.com/en/latest/topics/development/conventions/formulas.html) hosted on GitHub.
+
+The definition of what constitutes a formula is somewhat loose, and the specific structure of a formula is not mandated by Salt.
+
+### The Top File
+
+In addition to manually applying states to minions, Salt provides a way for you to automatically map which states should be applied to different minions. This map is called the *top file*.
+
+Here's a simple top file:
+
+{{< file "/srv/salt/top.sls" >}}
+base:
+  '*':
+    - universal_setup
+
+  'webserver1':
+    - webserver_setup
+{{< /file >}}
+
+`base` refers to the Salt [*environment*](https://docs.saltstack.com/en/latest/ref/states/top.html#environments). You can specify more than one environment corresponding to different phases of your work; for example: development, QA, production, etc. `base` is the default.
+
+Groups of minions are specified under the environment, and states are listed for each set of minions. The above example top file says that a `universal_setup` state should be applied to all minions (`'*'`), and the `webserver_setup` state should be applied to the `webserver1` minion.
+
+If you run the `state.apply` function with no arguments, then Salt will inspect the top file and apply all states within it according to the mapping you've created:
+
+    salt '*' state.apply
+
+{{< note >}}
+This action is colliquially known as a [*highstate*](https://docs.saltstack.com/en/latest/topics/tutorials/states_pt1.html#running-highstate).
+{{< /note >}}
+
+### Benefits of States and Configuration Management
+
+Defining your configurations in states eases system administration:
+
+-   Setting up states minimizes human error, as you will not need to enter commands manually one-by-one.
+
+-   Applying a state to minion multiple times generally does not result in any changes beyond the first application. Salt understands when a state has already been implemented on a minion and will not perform unnecessary actions.
+
+-   If you update a state file and apply it to a minion, Salt will detect and only apply the changes, which makes updating your systems more efficient.
+
+-   A state can be reused and applied to more than one minion, which will result in identical configurations across different servers.
+
+-   State files can be entered into a version control system, which helps you track changes to your systems over time.
+
+## Targeting Minions
+
+You can match against your minions' IDs using shell style globbing. This works at either the command line or in the top file.
+
+These examples would apply the `webserver_setup` state to all minions whose ID begins with `webserver` (e.g. `webserver1`, `webserver2`, etc):
+
+-   CLI:
+
+        salt 'webserver*' state.apply webserver_setup
+
+-   Top file:
+
+    {{< file >}}
+base:
+  'webserver*':
+    - webserver_setup
+{{< /file >}}
+
+[Regular Expressions](https://docs.saltstack.com/en/latest/topics/targeting/globbing.html#regular-expressions) and [lists](https://docs.saltstack.com/en/latest/topics/targeting/globbing.html#lists) can also be used to match against minion IDs.
+
+## Grains
+
+Salt's [*grains*](https://docs.saltstack.com/en/latest/topics/grains/) system provides access to information that is generated by and stored on a minion. Examples include a minion's operating system, domain name, IP address, and so on. You can also specify custom grain data on a minion, as outlined in Salt's documentation.
+
+You can use grain data to target minions from the command line. This command installs httpd on all minions running CentOS:
+
+    salt -G 'os:CentOS' pkg.install httpd
+
+You can also use grains in a top file:
+
+{{< file >}}
+base:
+  'os:CentOS':
+    - match: grain
+    - centos_setup
+{{< /file >}}
+
+Grain information generally isn't very dynamic, but it can change occassionally, and Salt will refresh its grain data when it does. To view your minions' grain data:
+
+    salt '*' grains.items
+
+## Storing Data and Secrets in Pillar
+
+Salt's [*pillar*](https://docs.saltstack.com/en/latest/topics/tutorials/pillar.html) feature takes data defined on the Salt master and distributes it to minions. A primary use for pillar is to store secrets, such as account credentials. Pillar is also a useful place to store non-secret data that you wouldn't want to record directly in your state files.
+
+{{< note >}}
+In addition to storing pillar data on the master, you can also keep it in other locations, like in a [Git repository](https://docs.saltstack.com/en/latest/ref/pillar/all/salt.pillar.git_pillar.html) or [Hashicorp's Vault](https://docs.saltstack.com/en/latest/ref/pillar/all/salt.pillar.vault.html).
+{{< /note >}}
+
+Let's say that you want to create system users on a minion and assign different shells to each of them. If you were to code this information into a state file, you would need a new declaration for each user. If you store the data in pillar instead, you can then just create one state declaration and inject the pillar data into it using Salt's [Jinja templating](#jinja-templates) feature.
+
+{{< note >}}
+Salt Pillar is sometimes confused with Salt Grains, as they both keep data that is used in states and remote execution. The data that grains maintains originates *from* the minions, while the data in pillar originates on the master (or another backend) and is delivered *to* the minions.
+{{< /note >}}
+
+### Anatomy of Pillar Data
+
+Pillar data is kept in `.sls` files which are written in the same YAML syntax as states:
+
+{{< file "/srv/pillar/user_info.sls">}}
+users:
+  joe:
+    shell: /bin/zsh
+  amy:
+    shell: /bin/bash
+  sam
+    shell: /bin/fish
+{{< /file >}}
+
+As with state files, a top file (separate from your states' top file) maps pillar data to minions:
+
+{{< file "/srv/pillar/top.sls">}}
+base:
+  'webserver1':
+    - user_info
+{{< /file >}}
+
+## Jinja Templates
+
+To inject pillar data into your states, use [Jinja's template syntax](https://docs.saltstack.com/en/latest/topics/jinja/index.html). While Salt uses the YAML syntax for state and pillar files, the files are first interpreted as Jinja templates (by default).
+
+This example state file uses the pillar data from the previous section to create system users and set the shell for each:
+
+{{< file "/srv/salt/user_setup.sls" >}}
+{% for user_name, user_info in pillar['users'].iteritems() %}
+{{ user_name }}:
+  user.present:
+    - shell: {{ user_info['shell'] }}
+{% endfor %}
+{{< /file >}}
+
+Salt will compile the state file into something that looks like this before it is applied to the minion:
+
+{{< file >}}
+joe:
+  user.present:
+    - shell: /bin/zsh
+
+amy:
+  user.present:
+    - shell: /bin/bash
+
+sam:
+  user.present:
+    - shell: /bin/fish
+{{< /file >}}
+
+You can also use Jinja to interact with grain data in your states. This example state will install Apache and adjust the name for the package according to the operating system:
+
+{{< file "/srv/salt/webserver_setup.sls" >}}
+install_apache:
+  pkg.installed:
+    {% if grains['os'] == 'CentOS' %}
+    - name: httpd
+    {% else %}
+    - name: apache
+    {% endif %}
+{{< /file >}}
+
+{{< note >}}
+In addition to Salt's documentation on Jinja, the [official Jinja documentation](http://jinja.pocoo.org/docs/2.10/templates/) also details the template syntax.
+{{< /note >}}
+
+## Beacons
+
+The [beacon](https://docs.saltstack.com/en/latest/topics/beacons/) system is a way of monitoring a variety of system processes on Salt minions. There are a number of [beacon modules](https://docs.saltstack.com/en/latest/ref/beacons/all/index.html) available.
+
+Beacons can trigger [reactors](https://docs.saltstack.com/en/latest/topics/reactor/index.html#reactor) which can then help implement a change or troubleshoot an issue. For example, if a service's response times out, the reactor system can restart the service.
+
+## Getting Started with Salt
+
+Now that you're familiar with some of Salt's basic terminology and components, move on to our guide [Getting Started with Salt - Basic Installation and Setup](https://www.linode.com/docs/applications/configuration-management/getting-started-with-salt-basic-installation-and-setup/) to set up a configuration to start running commands and provisioning minion servers.
+
+The SaltStack documentation also contains a page of [best practices](https://docs.saltstack.com/en/latest/topics/best_practices.html) to be mindful of when working with Salt. You should review this page and implement those practices into your own workflow whenever possible.