This role was previously maintained by Brian Shumate and is now curated by @ansible-community/hashicorp-tools.
This Ansible role performs basic Nomad installation, including filesystem structure, and example configuration.
It will also bootstrap a minimal cluster of 3 server nodes, and can do this in a development environment based on Vagrant and VirtualBox. See README_VAGRANT.md for more details about the Vagrant setup.
This role requires an Arch Linux, Debian, RHEL, or Ubuntu distribution; the role is tested with the following specific software versions:
- Ansible: 2.7.10
- nomad: 0.12.1
- Arch Linux
- CentOS: 7
- Debian: 8
- RHEL: 7
- Ubuntu: >= 20.04
- unzip for unarchive module
The role defines most of its variables in defaults/main.yml
:
- Nomad debug mode
- Default value: no
- Allow running the role even if not all instances are connected
- Default value: no
- Allow purging obsolete configuration files. For example, remove server configuration if instance is no longer a server
- Default value: no
- Nomad version to install
- Default value: 1.1.1
- This variable does not need to be changed in most cases
- Default value: Dictionary translating ansible_architecture to HashiCorp architecture naming convention
- Host architecture
- Default value: determined by
{{ nomad_architecture_map[ansible_architecture] }}
- Nomad package filename
- Default value:
nomad_{{ nomad_version }}_linux_{{ nomad_architecture }}.zip
- Nomad download URL
- Default value:
https://releases.hashicorp.com/nomad/{{ nomad_version }}/nomad_{{ nomad_version }}_linux_{{ nomad_architecture }}.zip
- Nomad checksum file URL
- Default value:
https://releases.hashicorp.com/nomad/{{ nomad_version }}/nomad_{{ nomad_version}}_SHA256SUMS
- Nomad binary installation path
- Default value:
/usr/local/bin
- Nomad configuration file path
- Default value:
/etc/nomad.d
- Nomad data path
- Default value:
/var/nomad
- Nomad lockfile path
- Default value:
/var/lock/subsys/nomad
- Nomad run path
- Default value:
/var/run/nomad
- Manage Nomad user?
- Default value: yes
- Nomad OS username
- Default value: root
- Manage Nomad group?
- Default value: no
- Nomad OS group
- Default value: bin
- Default region
- Default value: global
- Nomad datacenter label
- Default value: dc1
- Logging level
- Default value: INFO
- Log to syslog
- Default value: true
- Nomad network interface
- Default value:
{{ ansible_default_ipv4.interface }}
- Nomad node name
- Default value:
{{ inventory_hostname_short }}
- Nomad node role
- options: client, server, both
- Default value: client
- Send leave on termination
- Default value: yes
- Send leave on interrupt
- Default value: no
- Disable update check
- Default value: no
- Max retry join attempts
- Default value: 0
- Enable retry join?
- Default value: no
- Retry join interval
- Default value: 30s
- Rejoin after leave?
- Default value: no
- List of enabled schedulers
- Default value: service, batch, system
- Number of schedulers
- Default value:
{{ ansible_processor_vcpus }}
- Node garbage collection threshold
- Default value: 24h
- Job garbage collection threshold
- Default value: 4h
- Eval garbage collection threshold
- Default value: 1h
- Deployment garbage collection threshold
- Default value: 1h
- Enable Gossip Encryption even if
nomad_encrypt
is not set - Default value: false
- Set the encryption key; should be the same across a cluster. If not present and
nomad_encrypt_enable
is true, the key will be generated & retrieved from the bootstrapped server. - Default value: ""
- Specifies the raft multiplier to use
- Default value: 1
- Specifies the version of raft protocal, which used by nomad servers for communication
- Default value: 2
- Specifies the authoritative region, which provides a single source of truth for global configurations such as ACL Policies and global ACL tokens.
- Default value: ""
- Nomad node class
- Default value: ""
- Used for restricting which client nodes are eligible to receive which workloads. By default, tasks are opted-out of non-default node pools. This means job authors don’t have to repeatedly add the same constraints to every job just to avoid certain nodes.
- Default value: ""
- Force the UUID generated by the client to be randomly generated
- Default value: no
- Max kill timeout
- Default value: 30s
- Nomad scheduler will choose from the IPs of this interface for allocating tasks
- Default value: none
- Overide network link speed (0 = no overide)
- Default value: 0
- Overide cpu compute (0 = no overide)
- Default value: 0
- Client garbage collection interval
- Default value: 1m
- Maximum number of allocations which a client will track before triggering a garbage collection
- Default value: 50
- Disk usage threshold percentage for garbage collection
- Default value: 80
- Inode usage threshold percentage for garbage collection
- Default value: 70
- Garbage collection max parallel destroys
- Default value: 2
- Reserved client resources
- Default value:
cpu: {{ nomad_reserved_cpu }}, memory: {{ nomad_reserved_memory }}, disk: {{ nomad_reserved_disk }}, ports: {{ nomad_reserved_ports }}
- Reserved client CPU
- Default value: 0
- Reserved client memory
- Default value: 0
- Reserved client disk
- Default value: 0
- Reserved client ports
- Default value: 22
- List host_volume is used to make volumes available to jobs (Stateful Workloads). By default, a directory is created. Specify the
state
parameter to change it. - Default value: []
- Example:
nomad_host_volumes:
- name: data
path: /var/data
owner: root
group: bin
mode: 0755
read_only: false
- name: config
path: /etc/conf
owner: root
group: bin
mode: 0644
read_only: false
- name: docker socket
path: /run/docker.sock
read_only: true
state: file
- List host_network is used to make different networks available to jobs instead of selecting a default interface. This is very useful especially in case of multiple nics.
- Default value: []
- Example:
nomad_host_networks:
- name: public
cidr: 100.101.102.103/24
reserved_ports: 22,80
- name: private
interface: eth0
reserved_ports: 443
- Driver options
- Key value dict
- Default value: {}
- chroot environment definition for the Exec and Java drivers
- Key value dict
- Default value: false
- Meta data
- Key value dict
- Default value: {}
- Bind interface address
- Default value:
{{ hostvars[inventory_hostname]['ansible_'+ nomad_iface ]['ipv4']['address'] }}
- Network interface address to advertise to other nodes
- Default value:
{{ hostvars[inventory_hostname]['ansible_'+ nomad_iface ]['ipv4']['address'] }}
- Ports used by Nomad
- Default value:
http: {{ nomad_ports_http }}, rpc: {{ nomad_ports_rpc }}, serf: {{ nomad_ports_serf }}
- Http port
- Default value: 4646
- RPC port
- Default value: 4647
- Serf port
- Default value: 4648
- Installs the podman plugin
- Default value: false
- Installs the cni plugins
- Default value: false
- Install Docker subsystem on nodes?
- Default value: false
- Allow you configure client's template config.
- Default: {}
Example:
nomad_template_config:
vault_retry:
attempts: 12
backoff: "750ms"
max_backoff: "2m"
wait:
min: "10s"
max: "4m"
- Allow you configure nomad plugins.
- Default: {}
Example:
nomad_plugins:
nomad-driver-podman:
config:
volumes:
enabled: true
selinuxlabel: z
recover_stopped: true
- Ansible group that contains all cluster nodes
- Default value: nomad_instances
It's typically not necessary to manually alter this list.
- List of server nodes
- Default value: List of all nodes in
nomad_group_name
withnomad_node_role
set to server or both
This feature makes it possible to gather the nomad_bind_address
and
nomad_advertise_address
from servers that are currently not targeted by the
playbook.
To make this possible the delegate_facts
option is used. This option is broken
in many Ansible versions, so this feature might not always work.
- Gather facts from servers that are not currently targeted
- Default value: 'no'
- Bootstrap nomad via native consul zero-configuration support assumes consul default ports etc.
- Default value: False
- The address of your consul API, use it in combination with nomad_use_consul=True. If you want to use https, use
nomad_consul_ssl
. Do NOT append https. - Default value: localhost:8500
- If
true
then uses https. - Default value: false
- Public key of consul CA, use in combination with
nomad_consul_cert_file
andnomad_consul_key_file
. - Default value: ""
- Public key of consul CA to validate the gRPC TLS, use in combination with
nomad_consul_cert_file
andnomad_consul_key_file
. - Default value: nomad_consul_ca_file
- The public key which can be used to access consul.
- Default value: ""
- The private key counterpart of
nomad_consul_cert_file
. - Default value: ""
- The name of the consul service for your nomad servers
- Default value: nomad-servers
- The name of the consul service for your nomad clients
- Default value: nomad-clients
- Token to use for consul interaction
- Default value: ""
- Specifies the number of server nodes to wait for before bootstrapping.
- Default value: `{{ nomad_servers | count or 3 }}}
- Enable ACLs
- Default value: no
- TTL for tokens
- Default value: "30s"
- TTL for policies
- Default value: "30s"
- Token to use for acl replication on non authoritive servers
- Default value: ""
- Enable vault
- Default value: no
- Vault address to use
- Default value:
{{ vault_address | default('0.0.0.0') }}
- Allow users to use vault without providing their own token
- Default value: yes
- Role to create tokens from
- Default value: ""
- Path of CA cert to use with vault
- Default value: ""
- Path of a folder containing CA cert(s) to use with vault
- Default value: ""
- Path to a certificate to use with vault
- Default value: ""
- Path to a private key file to use with vault
- Default value: ""
- Optional string used to set SNI host when connecting to vault
- Default value: ""
- Specifies if SSL peer validation should be enforced
- Default value: no
- Vault token used by nomad. Will only be installed on servers.
- Default value: ""
- Vault namespace used by nomad
- Default value: ""
- Enable docker
- Default value: no
- Run dmsetup on ubuntu (only if docker is enabled)
- Default value: yes
- Enable TLS
- Default value: false
- Whether to copy certs from local machine (controller).
- Default value: false
- Whether to copy certs from remote machine itself.
- Default value: false
- The remote dir where the certs are stored.
- Default value:
/etc/nomad/ssl
- Use a ca for tls connection, nomad_cert_file and nomad_key_file are needed
- Default value: ca.cert
- Use a certificate for tls connection, nomad_ca_file and nomad_key_file are needed
- Default value: server.crt
- Use a key for tls connection, nomad_cert_file and nomad_key_file are needed
- Default value: server.key
- Use a certificate for tls connection, nomad_ca_file and nomad_key_file are needed, used only when the cluster is being upgraded to TLS, and removed after the migration is complete. This allows the agent to accept both TLS and plaintext traffic.
- Default value: false
- Use a key for tls connection, nomad_cert_file and nomad_key_file are needed. Specifies if outgoing TLS connections should verify the server's hostname.
- Default value: true
- Use a key for tls connection, nomad_cert_file and nomad_key_file are needed. Specifies agents should require client certificates for all incoming HTTPS requests. The client certificates must be signed by the same CA as Nomad.
- Default value: true
- Specifies whether to enable Nomad's telemetry configuration.
- Default value: false
- Specifies if gauge values should be prefixed with the local hostname.
- Default value: "false"
- Specifies the time interval at which the Nomad agent collects telemetry data.
- Default value: "1s"
- Specifies if gauge values should be prefixed with the name of the node, instead of the hostname. If set it will override disable_hostname value.
- Default value: "false"
- Specifies if Nomad should publish runtime metrics of allocations.
- Default value: "false"
- Specifies if Nomad should publish runtime metrics of nodes.
- Default value: "false"
- Specifies if Nomad should publish metrics that are backwards compatible with versions below 0.7, as post version 0.7, Nomad emits tagged metrics. All new metrics will only be added to tagged metrics. Note that this option is used to transition monitoring to tagged metrics and will eventually be deprecated.
- Default value: "false"
- Specifies if Nomad should not emit tagged metrics and only emit metrics compatible with versions below Nomad 0.7. Note that this option is used to transition monitoring to tagged metrics and will eventually be deprecated.
- Default value: "false"
- This controls whether to allow metrics that have not been specified by the filter. Defaults to true, which will allow all metrics when no filters are provided. When set to false with no filters, no metrics will be sent.
- Default value: "true"
- This is a list of filter rules to apply for allowing/blocking metrics by prefix. A leading "+" will enable any metrics with the given prefix, and a leading "-" will block them. If there is overlap between two rules, the more specific rule will take precedence. Blocking will take priority if the same prefix is listed multiple times.
- Default value: []
- Specifies if Nomad should ignore jobs dispatched from a parameterized job when publishing job summary statistics. Since each job has a small memory overhead for tracking summary statistics, it is sometimes desired to trade these statistics for more memory when dispatching high volumes of jobs.
- Default value: "false"
- Specifies the address of a statsite server to forward metrics data to.
- Default value: ""
- Specifies the address of a statsd server to forward metrics to.
- Default value: ""
- Specifies the address of a DataDog statsd server to forward metrics to.
- Default value: ""
- Specifies a list of global tags that will be added to all telemetry packets sent to DogStatsD. It is a list of strings, where each string looks like "my_tag_name:my_tag_value".
- Default value: []
- Specifies whether the agent should make Prometheus formatted metrics available at /v1/metrics?format=prometheus.
- Default value: "false"
- Specifies a valid Circonus API Token used to create/manage check. If provided, metric management is enabled.
- Default value: ""
- Specifies a valid app name associated with the API token.
- Default value: "nomad"
- Specifies the base URL to use for contacting the Circonus API.
- Default value: "https://api.circonus.com/v2"
- Specifies the interval at which metrics are submitted to Circonus.
- Default value: "10s"
- Specifies the check.config.submission_url field, of a Check API object, from a previously created HTTPTRAP check.
- Default value: ""
- Specifies the Check ID (not check bundle) from a previously created HTTPTRAP check. The numeric portion of the check._cid field in the Check API object.
- Default value: ""
- Specifies if force activation of metrics which already exist and are not currently active. If check management is enabled, the default behavior is to add new metrics as they are encountered. If the metric already exists in the check, it will not be activated. This setting overrides that behavior.
- Default value: "false"
- Serves to uniquely identify the metrics coming from this instance. It can be used to maintain metric continuity with transient or ephemeral instances as they move around within an infrastructure. By default, this is set to hostname:application name (e.g. "host123:nomad").
- Default value: ""
- Specifies a special tag which, when coupled with the instance id, helps to narrow down the search results when neither a Submission URL or Check ID is provided. By default, this is set to service:app (e.g. "service:nomad").
- Default value: ""
- Specifies a name to give a check when it is created. This name is displayed in the Circonus UI Checks list.
- Default value: ""
- Comma separated list of additional tags to add to a check when it is created.
- Default value: ""
- Specifies the ID of a specific Circonus Broker to use when creating a new check. The numeric portion of broker._cid field in a Broker API object. If metric management is enabled and neither a Submission URL nor Check ID is provided, an attempt will be made to search for an existing check using Instance ID and Search Tag. If one is not found, a new HTTPTRAP check will be created. By default, this is a random Enterprise Broker is selected, or, the default Circonus Public Broker.
- Default value: ""
- Specifies a special tag which will be used to select a Circonus Broker when a Broker ID is not provided. The best use of this is to as a hint for which broker should be used based on where this particular instance is running (e.g. a specific geographic location or datacenter, dc:sfo).
- Default value: ""
- Enable Nomad Autopilot
- To enable Autopilot features (with the exception of dead server cleanup), the raft_protocol setting in the server stanza must be set to 3 on all servers, see parameter nomad_raft_protocol
- Default value: false
- Specifies automatic removal of dead server nodes periodically and whenever a new server is added to the cluster.
- Default value: true
- Specifies the maximum amount of time a server can go without contact from the leader before being considered unhealthy.
- Default value: 200ms
- Specifies the maximum number of log entries that a server can trail the leader by before being considered unhealthy.
- Default value: 250
- Specifies the minimum amount of time a server must be stable in the 'healthy' state before being added to the cluster. Only takes effect if all servers are running Raft protocol version 3 or higher.
- Default value: 10s
- Specifies if you want to add specific label in the UI, later with
nomad_ui_label_text
,nomad_ui_label_background_color
andnomad_ui_label_text_color
. - Default value: false
e.g
nomad_ui: true
nomad_ui_label_text: "Staging Cluster"
nomad_ui_label_background_color: "yellow"
nomad_ui_label_text_color: "#000000"
- Specifies a label to display on the UI (e.g. "Staging Cluster").
- Default value: "Staging Cluster"
- Specifies the background color of the label on the UI (e.g. "yellow").
- Default value: "yellow"
- Specifies the color of the label on the UI (e.g. "#000000").
- Default value: "#000000"
- Specifies environment variables for artifact (e.g. "GITLAB_READONLY_TOKEN").
- Default value: ""
e.g
nomad_artifact:
{
set_environment_variables: "GITLAB_READONLY_TOKEN,GITLAB_KEYCLOAK_THEMES_READONLY_TOKEN",
}
As Nomad loads the configuration from files and directories in lexical order,
typically merging on top of previously parsed configuration files, you may set
custom configurations via nomad_config_custom
, which will be expanded into a file named custom.json
within your nomad_config_dir
which will
be loaded after all other configuration by default.
An example usage for enabling vault
:
vars:
nomad_config_custom:
vault:
enabled : true
ca_path : "/etc/certs/ca"
cert_file : "/var/certs/vault.crt"
key_file : "/var/certs/vault.key"
address : "https://vault.service.consul:8200"
create_from_role : "nomad-cluster"
Ansible requires GNU tar and this role performs some local use of the
unarchive module, so ensure that your system has gtar
/unzip
installed.
Jinja2 templates use ipaddr filter that need netaddr
python library.
Basic nomad installation is possible using the included site.yml
playbook:
ansible-playbook -i <hosts> site.yml
You can also simply pass variables in using the --extra-vars
option to the
ansible-playbook
command:
ansible-playbook -i hosts site.yml --extra-vars "nomad_datacenter=maui"
See examples/README_VAGRANT.md
for details on quick Vagrant deployments
under VirtualBox for testing, etc.
BSD
Special thanks to the folks listed in CONTRIBUTORS.md for their contributions to this project.
Contributions are welcome, provided that you can agree to the terms outlined in CONTRIBUTING.md