Manage an Elastic stack with Salt.
Table of Contents
See the full SaltStack Formulas installation and usage instructions.
If you are interested in writing or contributing to formulas, please pay attention to the Writing Formula Section.
If you want to use this formula, please pay attention to the FORMULA file and/or git tag,
which contains the currently released version. This formula is versioned according to Semantic Versioning.
See Formula Versioning Section for more details.
If you need (non-default) configuration, please refer to:
- how to configure the formula with map.jinja
- the
pillar.examplefile - the Special notes section
- This formula is written with a Vault database secret engine in mind. Note that the Vault integration currently requires my rewritten Vault modules, which might become available in Salt at some point.
- The certificate management requires my rewritten x509 modules (for
pkcs12), which are available from Salt v3006 onwards asx509_v2. - All of the
*beatandlogstashstates are boilerplate at the moment. - The tests only serve an esthetical purpose currently (not implemented).
An example pillar is provided, please see pillar.example. Note that you do not need to specify everything by pillar. Often, it's much easier and less resource-heavy to use the parameters/<grain>/<value>.yaml files for non-sensitive settings. The underlying logic is explained in map.jinja.
The following states are found in this formula:
elasticelastic.repoelastic.auditbeatelastic.auditbeat.configelastic.auditbeat.packageelastic.auditbeat.serviceelastic.elasticsearchelastic.elasticsearch.authelastic.elasticsearch.bootstrap_passelastic.elasticsearch.certselastic.elasticsearch.configelastic.elasticsearch.packageelastic.elasticsearch.serviceelastic.elasticsearch.vault_roleselastic.elasticsearch.vault_setupelastic.filebeatelastic.filebeat.configelastic.filebeat.packageelastic.filebeat.serviceelastic.functionbeatelastic.functionbeat.configelastic.functionbeat.packageelastic.functionbeat.serviceelastic.heartbeatelastic.heartbeat.configelastic.heartbeat.packageelastic.heartbeat.serviceelastic.kibanaelastic.kibana.authelastic.kibana.certselastic.kibana.configelastic.kibana.packageelastic.kibana.serviceelastic.logstashelastic.logstash.configelastic.logstash.packageelastic.logstash.serviceelastic.metricbeatelastic.metricbeat.configelastic.metricbeat.packageelastic.metricbeat.serviceelastic.packetbeatelastic.packetbeat.configelastic.packetbeat.packageelastic.packetbeat.serviceelastic.cleanelastic.repo.cleanelastic.auditbeat.cleanelastic.auditbeat.config.cleanelastic.auditbeat.package.cleanelastic.auditbeat.service.cleanelastic.elasticsearch.cleanelastic.elasticsearch.auth.cleanelastic.elasticsearch.bootstrap_pass.cleanelastic.elasticsearch.certs.cleanelastic.elasticsearch.config.cleanelastic.elasticsearch.package.cleanelastic.elasticsearch.service.cleanelastic.elasticsearch.vault_roles.cleanelastic.elasticsearch.vault_setup.cleanelastic.filebeat.cleanelastic.filebeat.config.cleanelastic.filebeat.package.cleanelastic.filebeat.service.cleanelastic.functionbeat.cleanelastic.functionbeat.config.cleanelastic.functionbeat.package.cleanelastic.functionbeat.service.cleanelastic.heartbeat.cleanelastic.heartbeat.config.cleanelastic.heartbeat.package.cleanelastic.heartbeat.service.cleanelastic.kibana.cleanelastic.kibana.auth.cleanelastic.kibana.certs.cleanelastic.kibana.config.cleanelastic.kibana.package.cleanelastic.kibana.service.cleanelastic.logstash.cleanelastic.logstash.config.cleanelastic.logstash.package.cleanelastic.logstash.service.cleanelastic.metricbeat.cleanelastic.metricbeat.config.cleanelastic.metricbeat.package.cleanelastic.metricbeat.service.cleanelastic.packetbeat.cleanelastic.packetbeat.config.cleanelastic.packetbeat.package.cleanelastic.packetbeat.service.clean
Installs the Elastic repo.
Does not install/configure/start any packages/services.
Installs, configures and starts Auditbeat.
Meta-state. Manages the lifecycle of an Elasticsearch node/cluster with integration to the Vault database secret engine.
Includes all states for ES, with the exception of elastic.elasticsearch.vault_setup.
Takes care of managing ES users and groups and managing the Vault database secret engine connection. Also, optionally resets the bootstrap password. Depends on elastic.elasticsearch.service.
Ensures a known bootstrap password is set in order to be able to manage the initial configuration non-interactively. Depends on elastic.elasticsearch.config.
Generates and manages certificates + keys for the HTTP and transport layers,
including trusted CA certificates for Elasticsearch.
Note that generally, it's advisable to setup a CA minion. See the
x509 (x509_v2) module docs for details.
Depends on elastic.elasticsearch.package.
Manages ES and JVM configuration. Depends on elastic.elasticsearch.package.
Installs Elasticsearch only. Depends on elastic.repo.
Enables and (re-)starts Elasticsearch. Depends on elastic.elasticsearch.config, elastic.elasticsearch.certs and elastic.elasticsearch.bootstrap_pass
Manages Vault database secret engine roles. Depends on elastic.elasticsearch.auth (for managing the allowed roles on the connection).
This should be targeted to your Vault minion(s), not the Elasticsearch one(s). Generates and manages ES client certificates for Vault since the ES database plugin currently does not allow to pass those in via the REST API.
Installs, configures and starts Filebeat.
Installs, configures and starts Functionbeat.
Installs, configures and starts Heartbeat.
Installs, configures and starts Kibana, including generating client certificates and requesting credentials from Vault.
Manages authentication details for Kibana. Note that this will always report changes since there is no way to read the current configuration. Depends on elastic.kibana.package.
Generates client certificates and ensures the CA is trusted by Kibana. Depends on elastic.kibana.package.
Manages Kibana configuration, other than authentication. Depends on elastic.kibana.package.
Installs the Kibana package only. Depends on elastic.repo.
Enables and (re-)starts Kibana. Depends on elastic.kibana.config, elastic.kibana.certs and elastic.kibana.auth.
Installs, configures and starts Filebeat.
Installs, configures and starts Metricbeat.
Installs, configures and starts Packetbeat.
Meta-state.
Removes everything Elastic-related: includes all clean states.
Stops, unconfigures and removes Auditbeat.
Undoes everything in the elastic.elasticsearch state in reverse.
Removes the Vault database connection, only if
remove_all_data_for_sure is true.
Depends on elastic.elasticsearch.service.clean.
Ensures no bootstrap password is set. Depends on elastic.elasticsearch.service.clean.
Ensures certificates and keys are removed from ES configuration and the local filesystem. Depends on elastic.elasticsearch.service.clean.
Removes Elasticsearch and JVM configuration files. Depends on elastic.elasticsearch.service.clean.
Removes Elasticsearch. Depends on elastic.elasticsearch.config.clean.
Stops and disables Elasticsearch at boot time.
Removes managed Vault roles.
Removes generated certificate and key from the Vault server's filesystem.
Stops, unconfigures and removes Filebeat.
Stops, unconfigures and removes Functionbeat.
Stops, unconfigures and removes Heartbeat.
Undoes everything in the elastic.kibana state in reverse.
Removes authentication credentials from the Kibana keystore. Depends on elastic.kibana.service.clean.
Removes generated Kibana certificates and keys. Depends on elastic.kibana.service.clean.
Removes Kibana the configuration file. Depends on elastic.kibana.service.clean.
Removes Kibana from the system. Depends on elastic.kibana.config.clean.
Stops and disables Kibana at boot time.
Stops, unconfigures and removes Logstash.
Stops, unconfigures and removes Metricbeat.
Stops, unconfigures and removes Packetbeat.
Commit message formatting is significant!
Please see How to contribute for more details.
pre-commit is configured for this formula, which you may optionally use to ease the steps involved in submitting your changes.
First install the pre-commit package manager using the appropriate method, then run bin/install-hooks and
now pre-commit will run automatically on each git commit.
$ bin/install-hooks pre-commit installed at .git/hooks/pre-commit pre-commit installed at .git/hooks/commit-msg
There is a script that semi-autodocuments available states: bin/slsdoc.
If a .sls file begins with a Jinja comment, it will dump that into the docs. It can be configured differently depending on the formula. See the script source code for details currently.
This means if you feel a state should be documented, make sure to write a comment explaining it.
Linux testing is done with kitchen-salt.
- Ruby
- Docker
$ gem install bundler
$ bundle install
$ bin/kitchen test [platform]Where [platform] is the platform name defined in kitchen.yml,
e.g. debian-9-2019-2-py3.
Creates the docker instance and runs the elastic main state, ready for testing.
Runs the inspec tests on the actual instance.
Removes the docker instance.
Runs all of the stages above in one go: i.e. destroy + converge + verify + destroy.
Gives you SSH access to the instance for manual testing.