Skip to content

Latest commit

 

History

History
2331 lines (1673 loc) · 56.4 KB

templating-language.md

File metadata and controls

2331 lines (1673 loc) · 56.4 KB
Raw

Templating Language

Consul Template parses files authored in the Go Template format. If you are not familiar with the syntax, please read Go's documentation and examples. In addition to the Go-provided template functions, Consul Template provides the following functions:

API Functions

API functions interact with remote API calls, communicating with external services like Consul and Vault.

caLeaf

Query Consul for the leaf certificate representing a single service.

{{ caLeaf "<NAME>" }}

For example:

{{ with caLeaf "proxy" }}{{ .CertPEM }}{{ end }}

renders

-----BEGIN CERTIFICATE-----
MIICizCCAjGgAwIBAgIBCDAKBggqhkjOPQQDAjAWMRQwEgYDVQQDEwtDb25zdWwg
...
lXcQzfKlIYeFWvcAv4cA4W258gTtqaFRDRJ2i720eQ==
-----END CERTIFICATE-----

The two most useful fields are .CertPEM and .PrivateKeyPEM. For a complete list of available fields, see consul's documentation on LeafCert.

caRoots

Query Consul for all connect trusted certificate authority (CA) root certificates.

{{ caRoots }}

For example:

{{ range caRoots }}{{ .RootCertPEM }}{{ end }}

renders

-----BEGIN CERTIFICATE-----
MIICWDCCAf+gAwIBAgIBBzAKBggqhkjOPQQDAjAWMRQwEgYDVQQDEwtDb25zdWwg
...
bcA+Su3r8qSRppTlc6D0UOYOWc1ykQKQOK7mIg==
-----END CERTIFICATE-----

The most useful field is .RootCertPEM. For a complete list of available fields, see consul's documentation on CARootList.

connect

Query Consul for connect-capable services based on their health.

{{ connect "<TAG>.<NAME>?<QUERY>@<DATACENTER>~<NEAR>|<FILTER>" }}

Syntax is exactly the same as for the service function below.

{{ range connect "web" }}
server {{ .Name }} {{ .Address }}:{{ .Port }}{{ end }}

renders the IP addresses of all healthy nodes with a logical connect-capable service named "web":

server web01 10.5.2.45:21000
server web02 10.2.6.61:21000

datacenters

Query Consul for all datacenters in its catalog.

{{ datacenters }}

For example:

{{ range datacenters }}
{{ . }}{{ end }}

renders

dc1
dc2

An optional boolean can be specified which instructs Consul Template to ignore datacenters which are inaccessible or do not have a current leader. Enabling this option requires an O(N+1) operation and therefore is not recommended in environments where performance is a factor.

// Ignores datacenters which are inaccessible
{{ datacenters true }}

exportedServices

Query Consul for all exported services in a given partition.

{{ exportedServices "<PARTITION>" }}

For example:

{{- range $svc := (exportedServices "default") }}
    {{- range $consumer := .Consumers.Partitions }}
        {{- $svc.Service }} is exported to the {{ $consumer }} partition
    {{- end }}
{{- end }}

file

Read and output the contents of a local file on disk. If the file cannot be read, an error will occur. When the file changes, Consul Template will pick up the change and re-render the template.

{{ file "<PATH>" }}

For example:

{{ file "/path/to/my/file" }}

renders

file contents

This does not process nested templates. See executeTemplate for a way to render nested templates.

key

Query Consul for the value at the given key path. If the key does not exist, Consul Template will block rendering until the key is present. To avoid blocking, use keyOrDefault or keyExists.

{{ key "<PATH>?<QUERY>@<DATACENTER>" }}

The <QUERY> attribute is optional; if omitted, the default Consul namespace, default partition will be queried. <QUERY> can be used to set the Consul namespace or partition. <QUERY> accepts a url query-parameter format, e.g.:

{{ key "key?ns=namespace-name&partition=partition-name" }}

The <DATACENTER> attribute is optional; if omitted, the local datacenter is used.

For example:

{{ key "service/redis/maxconns" }}

renders

15

keyExists

Query Consul for the value at the given key path. If the key exists, this will return true, false otherwise. Unlike key, this function will not block if the key does not exist. This is useful for controlling flow.

{{ keyExists "<PATH>@<DATACENTER>" }}

The <DATACENTER> attribute is optional; if omitted, the local datacenter is used.

For example:

{{ if keyExists "app/beta_active" }}
  # ...
{{ else }}
  # ...
{{ end }}

keyOrDefault

Query Consul for the value at the given key path. If the key does not exist, the default value will be used instead. Unlike key, this function will not block if the key does not exist.

{{ keyOrDefault "<PATH>@<DATACENTER>" "<DEFAULT>" }}

The <DATACENTER> attribute is optional; if omitted, the local datacenter is used.

For example:

{{ keyOrDefault "service/redis/maxconns" "5" }}

renders

5

Note that Consul Template uses a multi-phase execution. During the first phase of evaluation, Consul Template will have no data from Consul and thus will always fall back to the default value. Subsequent reads from Consul will pull in the real value from Consul (if the key exists) on the next template pass. This is important because it means that Consul Template will never "block" the rendering of a template due to a missing key from a keyOrDefault. Even if the key exists, if Consul has not yet returned data for the key, the default value will be used instead.

ls

Query Consul for all top-level kv pairs at the given key path.

{{ ls "<PATH>?<QUERY>@<DATACENTER>" }}

The <QUERY> attribute is optional; if omitted, the default Consul namespace, default partition will be queried. <QUERY> can be used to set the Consul namespace or partition. <QUERY> accepts a url query-parameter format, e.g.:

{{ range ls "service/redis?ns=namespace-name&partition=partition-name" }}
  {{ .Key }}:{{ .Value }}
{{ end }}

The <DATACENTER> attribute is optional; if omitted, the local datacenter is used.

For example:

{{ range ls "service/redis" }}
  {{ .Key }}:{{ .Value }}
{{ end }}

renders

maxconns:15
minconns:5

safeLs

Same as ls, but refuse to render template, if the KV prefix query return blank/empty data.

This is especially useful, for rendering mission critical files, that are being populated by consul-template.

For example:

/root/.ssh/authorized_keys
/etc/sysconfig/iptables

Using safeLs on empty prefixes will result in template output not being rendered at all.

To learn how safeLs was born see CT-1131 C-3975 and CR-82.

node

Query Consul for a node in the catalog.

{{node "<NAME>?<QUERY>@<DATACENTER>"}}

The <NAME> attribute is optional; if omitted, the local agent node is used.

The <QUERY> attribute is optional; if omitted, the default Consul namespace, default partition will be queried. <QUERY> can be used to set the Consul namespace or partition. <QUERY> accepts a url query-parameter format, e.g.:

{{ with node "node?ns=default&partition=default" }}
  {{ .Node.Address }}
{{ end }}

The <DATACENTER> attribute is optional; if omitted, the local datacenter is used.

For example:

{{ with node }}
{{ .Node.Address }}{{ end }}

renders

10.5.2.6

To query a different node:

{{ with node "node1@dc2" }}
{{ .Node.Address }}{{ end }}

renders

10.4.2.6

To access map data such as TaggedAddresses or Meta, use Go's text/template map indexing.

nodes

Query Consul for all nodes in the catalog.

{{ nodes "?<QUERY>@<DATACENTER>~<NEAR>" }}

The <QUERY> attribute is optional; if omitted, the default Consul namespace, default partition will be queried. <QUERY> can be used to set the Consul namespace or partition. <QUERY> accepts a url query-parameter format, e.g.:

{{ range nodes "?ns=namespace&partition=partition" }}
{{ .Address }}{{ end }}

The <DATACENTER> attribute is optional; if omitted, the local datacenter is used.

The <NEAR> attribute is optional; if omitted, results are specified in lexical order. If provided a node name, results are ordered by shortest round-trip time to the provided node. If provided _agent, results are ordered by shortest round-trip time to the local agent.

For example:

{{ range nodes }}
{{ .Address }}{{ end }}

renders

10.4.2.13
10.46.2.5

To query a different data center and order by shortest trip time to ourselves:

{{ range nodes "@dc2~_agent" }}
{{ .Address }}{{ end }}

To access map data such as TaggedAddresses or Meta, use Go's text/template map indexing.

partitions

Query Consul for all partitions.

{{ range partitions }}
{{ .Name }}
{{ end }}

### `peerings`

Query [Consul][consul] for all peerings.

```golang
{{ peerings "?<QUERY>" }}

The <QUERY> attribute is optional; if omitted, default partition will be queried. <QUERY> can be used to set the Consul partition. <QUERY> accepts a url query-parameter format, e.g.:

{{ range peerings "?partition=partition" }}
{{ .Name }}{{ end }}

For example:

{{ range peerings }}
{{ .Name }}{{ end }}

renders

foo
bar

For complete list of available fields, see consul's documentation on CT-Peerings

To access map data such as Meta or slice such as PeerServerAddresses, use Go's text/template map indexing.

secret

Format

Query Vault for the secret at the given path.

{{ secret "<PATH>" "<DATA>" }}

Simple Read

The <DATA> attribute is optional; if omitted, the request will be a vault read (HTTP GET) request. If provided, the request will be a vault write (HTTP PUT/POST) request.

For example:

{{ with secret "secret/passwords" }}
{{ .Data.wifi }}{{ end }}

renders

FORWARDSoneword

Versioned Read

To access a versioned secret value (for the K/V version 2 backend):

{{ with secret "secret/passwords?version=1" }}
{{ .Data.data.wifi }}{{ end }}

When omitting the ?version parameter, the latest version of the secret will be fetched. Note the nested .Data.data syntax when referencing the secret value. For more information about using the K/V v2 backend, see the Vault Documentation.

When using Vault versions 0.10.0/0.10.1, the secret path will have to be prefixed with "data", i.e. secret/data/passwords for the example above. This is not necessary for Vault versions after 0.10.1, as consul-template will detect the KV backend version being used. The version 2 KV backend did not exist prior to 0.10.0, so these are the only affected versions.

Write (and Read back)

An example using write to generate PKI certificates:

{{ with secret "pki/issue/my-domain-dot-com" "common_name=foo.example.com" }}
{{ .Data.certificate }}{{ end }}

An example of adding (writing) a derived Vault token while reading it back out for use in a configuration file.

{{with secret "/auth/token/create" "policies=policy_1" "no_default_policy=true"}}
{{.Auth.ClientToken}}{{ end }}

The parameters must be key=value pairs, and each pair must be its own argument to the function:

Please always consider the security implications of having the contents of a secret in plain-text on disk. If an attacker is able to get access to the file, they will have access to plain-text secrets.

Please note that Vault does not support blocking queries. As a result, Consul Template will not immediately reload in the event a secret is changed as it does with Consul's key-value store. Consul Template will renew the secret with Vault's Renewer API. The Renew API tries to use most of the time the secret is good, renewing at around 90% of the lease time (as set by Vault).

Also consider enabling error_on_missing_key when working with templates that will interact with Vault. By default, Consul Template uses Go's templating language. When accessing a struct field or map key that does not exist, it defaults to printing <no value>. This may not be the desired behavior, especially when working with passwords or other data. As such, it is recommended you set:

template {
  error_on_missing_key = true
}

You can also guard against empty values using if or with blocks.

{{ with secret "secret/foo"}}
{{ if .Data.password }}
password = "{{ .Data.password }}"
{{ end }}
{{ end }}

secrets

Query Vault for the list of secrets at the given path. Not all endpoints support listing.

{{ secrets "<PATH>" }}

For example:

{{ range secrets "secret/" }}
{{ . }}{{ end }}

renders

bar
foo
zip

To iterate and list over every secret in the generic secret backend in Vault:

{{ range secrets "secret/" }}
{{ with secret (printf "secret/%s" .) }}{{ range $k, $v := .Data }}
{{ $k }}: {{ $v }}
{{ end }}{{ end }}{{ end }}

.Data should be replaced with .Data.data for KV-V2 secrets engines.

You should probably never do this.

Please also note that Vault does not support blocking queries. To understand the implications, please read the note at the end of the secret function.

pkiCert

Query Vault for a PKI certificate. It returns the certificate PEM encoded in a string. It returns the PKI Certificate, CA, and the Secret Key as the fields: Cert, CA and Key.

Special Note: This function uses the template file destination as a cache for the certificate to prevent Consul-Template from re-fetching it on reload or restart. This special behavior is to better work with Vault's PKI behavior of always returning a new certificate even if the current one is still good. Using the destination file as a local "cache" allows Consul-Template to check for the certificate in that local file and, if found, parse it and checks it's valid date range only fetching a new certificate if the local one has expired. It can only get re-populate the fields (Cert, CA, Key) if that data is in the file. Eg. If you don't include the CA in the file the CA field will be blank when loading from cache. And note that you must include the Certificate itself in this file as it contains the TTL/expiration data.

{{ with pkiCert "pki/issue/my-domain-dot-com" "common_name=foo.example.com" }}
Certificate: {{ .Cert }}
Private Key: {{ .Key }}
Cert Authority: {{ .CA }}
{{ end }}

If you need the complete CA+Chain in the same file as your Cert and Key use the standard secret to include it, but note the CA in the pkiCert return should be ignored as it would be redundant and upon reload might grab one of the chain certs. Something like this.

{{ with pkiCert "pki/issue/my-domain-dot-com" "common_name=foo.example.com" }}
Private Key: {{ .Key }}
Cert Authority: {{ .CA }}
{{ end }}
{{ with secret "pki/cert/ca_chain" }}
CA+Chain: {{ .Data.ca_chain }}
{{ end }}

The writeToFile function can be used with pkiCert to write your Key and Certs to separate files from a template.

{{- with pkiCert "pki/issue/my-domain-dot-com" "common_name=foo.example.com" -}}
{{ .Cert }}{{ .CA }}{{ .Key }}
{{ .Key | writeToFile "/my/path/to/cert.key" "root" "root" "0400" }}
{{ .CA | writeToFile "/my/path/to/cert.pem" "root" "root" "0644" }}
{{ .Cert | writeToFile "/my/path/to/cert.pem" "root" "root" "0644" "append" }}
{{- end -}}

service

Query Consul for services based on their health.

{{ service "<TAG>.<NAME>?<QUERY>@<DATACENTER>~<NEAR>|<FILTER>" }}

The <TAG> attribute is optional; if omitted, all nodes will be queried.

The <QUERY> attribute is optional; if omitted, the default Consul namespace, default partition will be queried. <QUERY> can be used to set the Consul namespace, partition, sameness group, or peer. <QUERY> accepts a url query-parameter format, e.g.:

{{ service "service-name?ns=namespace-name&peer=peer-name&partition=partition-name" }}

When using the sameness-group query parameter, the following rules are applied to use with other query parameters:

  • partition is used to denote where the Sameness Group Config Entry is stored.
  • ns is ignored.
  • Either peer or partition can be used, but not both.
{{ service "service-name?sameness-group=sameness-group-name" }}

When using the sameness-group query parameter, the result will be based on the following rules:

  • The Sameness Group Config Entry will be retrieved using the partition and the sameness-group parameters.
  • The <service-name will be used to search in each of Members (peers or partitions) of the Sameness Group Config Entry.
  • The first healthy service found in the Sameness Group Members will be returned.
  • If the Config Entry is updated with a change to the members in either the order of which members are present, the query will be re-run and the service will be queried based on the new set of Members.
  • If the current member service's health changes to unhealthy, the query will be re-run and the query will return the next healthy service found in the list of Sameness Group members.
  • If the health of a service in a member that is higher in the list of Sameness Group member changes from unhealthy to healthy, the query will be re-run and the query will return the next first health service found in the list of Sameness Group members, likely the changed member service.
  • If the Sameness Group Config Entry is deleted, the query will return an error that the name was not found.

The <DATACENTER> attribute is optional; if omitted, the local datacenter is used.

The <NEAR> attribute is optional; if omitted, results are specified in lexical order. If provided a node name, results are ordered by shortest round-trip time to the provided node. If provided _agent, results are ordered by shortest round-trip time to the local agent.

The <FILTER> attribute is optional; if omitted, only healthy services are returned. Providing a filter allows for client-side filtering of services.

For example:

{{ range service tag1.web@east-aws }}
server {{ .Name }} {{ .Address }}:{{ .Port }}{{ end }}

The example above is querying Consul for healthy "web" services, in the "east-aws" data center. The tag and data center attributes are optional. To query all nodes of the "web" service (regardless of tag) for the current data center:

{{ range service "web" }}
server {{ .Name }} {{ .Address }}:{{ .Port }}{{ end }}

renders the IP addresses of all healthy nodes with a logical service named "web":

server web01 10.5.2.45:2492
server web02 10.2.6.61:2904

To access map data such as NodeTaggedAddresses, ServiceTaggedAddresses or NodeMeta, use Go's text/template map indexing.

{{ range service "web" }}
  {{ with .ServiceTaggedAddresses.wan }}
http://{{ .Address }}:{{ .Port }}
  {{ end }}
{{ end }}

By default only healthy services are returned. To list all services, pass the "any" filter:

{{ service "web|any" }}

This will return all services registered to the agent, regardless of their status.

To filter services by a specific set of healths, specify a comma-separated list of health statuses:

{{ service "web|passing,warning" }}

This will returns services which are deemed "passing" or "warning" according to their node and service-level checks defined in Consul. Please note that the comma implies an "or", not an "and".

Note: Due to the use of dot . to delimit TAG, the service command will not recognize service names containing dots.

Note: There is an architectural difference between the following:

{{ service "web" }}
{{ service "web|passing" }}

The former will return all services which Consul considers "healthy" and passing. The latter will return all services registered with the Consul agent and perform client-side filtering. As a general rule, do not use the "passing" argument alone if you want only healthy services - simply omit the second argument instead.

services

Query Consul for all services in the catalog.

{{ services "?<QUERY>@<DATACENTER>" }}

The <QUERY> attribute is optional; if omitted, the default Consul namespace, default partition will be queried. <QUERY> can be used to set the Consul namespace or partition. <QUERY> accepts a url query-parameter format, e.g.:

{{ range services "?ns=default&partition=default" }}
  {{ .Name }}
{{ end }}

The <DATACENTER> attribute is optional; if omitted, the local datacenter is used.

For example:

{{ range services }}
{{ .Name }}: {{ .Tags | join "," }}{{ end }}

renders

node01 tag1,tag2,tag3

tree

Query Consul for all kv pairs at the given key path.

{{ tree "<PATH>@<DATACENTER>" }}

The <DATACENTER> attribute is optional; if omitted, the local datacenter is used.

For example:

{{ range tree "service/redis" }}
{{ .Key }}:{{ .Value }}{{ end }}

renders

minconns 2
maxconns 12
nested/config/value "value"

Unlike ls, tree returns all keys under the prefix, just like the Unix tree command.

safeTree

Same as tree, but refuse to render template, if the KV prefix query return blank/empty data.

This is especially useful, for rendering mission critical files, that are being populated by consul-template.

For example:

/root/.ssh/authorized_keys
/etc/sysconfig/iptables

Using safeTree on empty prefixes will result in template output not being rendered at all.

To learn how safeTree was born see CT-1131 C-3975 and CR-82.

Scratch

The scratchpad (or "scratch" for short) is available within the context of a template to store temporary data or computations. Scratch data is not shared between templates and is not cached between invocations.

scratch.Key

Returns a boolean if data exists in the scratchpad at the named key. Even if the data at that key is nil, this still returns true.

{{ scratch.Key "foo" }}

scratch.Get

Returns the value in the scratchpad at the named key. If the data does not exist, this will return nil.

{{ scratch.Get "foo" }}

scratch.Set

Saves the given value at the given key. If data already exists at that key, it is overwritten.

{{ scratch.Set "foo" "bar" }}

scratch.SetX

This behaves exactly the same as Set, but does not overwrite if the value already exists.

{{ scratch.SetX "foo" "bar" }}

scratch.MapSet

Saves a value in a named key in the map. If data already exists at that key, it is overwritten.

{{ scratch.MapSet "vars" "foo" "bar" }}

scratch.MapSetX

This behaves exactly the same as MapSet, but does not overwrite if the value already exists.

{{ scratch.MapSetX "vars" "foo" "bar" }}

scratch.MapValues

Returns a sorted list (by key) of all values in the named map.

{{ scratch.MapValues "vars" }}

Helper Functions

Unlike API functions, helper functions do not query remote services. These functions are useful for parsing data, formatting data, performing math, etc.

base64Decode

Accepts a base64-encoded string and returns the decoded result, or an error if the given string is not a valid base64 string.

{{ base64Decode "aGVsbG8=" }}

renders

hello

base64Encode

Accepts a string and returns a base64-encoded string.

{{ base64Encode "hello" }}

renders

aGVsbG8=

base64URLDecode

Accepts a base64-encoded URL-safe string and returns the decoded result, or an error if the given string is not a valid base64 URL-safe string.

{{ base64URLDecode "aGVsbG8=" }}

renders

hello

base64URLEncode

Accepts a string and returns a base-64 encoded URL-safe string.

{{ base64Encode "hello" }}

renders

aGVsbG8=

byKey

Accepts a list of pairs returned from a tree call and creates a map that groups pairs by their top-level directory.

For example:

groups/elasticsearch/es1
groups/elasticsearch/es2
groups/elasticsearch/es3
services/elasticsearch/check_elasticsearch
services/elasticsearch/check_indexes

with the following template

{{ range $key, $pairs := tree "groups" | byKey }}{{ $key }}:
{{ range $pair := $pairs }}  {{ .Key }}={{ .Value }}
{{ end }}{{ end }}

renders

elasticsearch:
  es1=1
  es2=1
  es3=1

Note that the top-most key is stripped from the Key value. Keys that have no prefix after stripping are removed from the list.

The resulting pairs are keyed as a map, so it is possible to look up a single value by key:

{{ $weights := tree "weights" }}
{{ range service "release.web" }}
  {{ $weight := or (index $weights .Node) 100 }}
  server {{ .Node }} {{ .Address }}:{{ .Port }} weight {{ $weight }}{{ end }}

byTag

Takes the list of services returned by the service or services function and creates a map that groups services by tag.

{{ range $tag, $services := service "web" | byTag }}{{ $tag }}
{{ range $services }} server {{ .Name }} {{ .Address }}:{{ .Port }}
{{ end }}{{ end }}

byMeta

Takes a list of services returned by service and returns a map that groups services by ServiceMeta values. Multiple service meta keys can be passed as a comma separated string. |int can be added to a meta key to convert numbers from service meta values to padded numbers in printf "%05d" % value format (useful for sorting as Go Template sorts maps by keys).

Example:

If we have the following services registered in Consul:

{
  "Services": [
     {
       "ID": "redis-dev-1",
       "Name": "redis",
       "ServiceMeta": {
         "environment": "dev",
         "shard_number": "1"
       },
       ...
     },
     {
       "ID": "redis-prod-1",
       "Name": "redis",
       "ServiceMeta": {
         "environment": "prod",
         "shard_number": "1"
       },
       ...
     },
     {
       "ID": "redis-prod-2",
       "Name": "redis",
       "ServiceMeta": {
         "environment": "prod",
         "shard_number": "2"
       },
       ...
     }
   ]
}
{{ service "redis|any" | byMeta "environment,shard_number|int" | toJSON }}

The code above will produce a map of services grouped by meta:

{
  "dev_00001": [
    {
      "ID": "redis-dev-1",
      ...
    }
  ],
  "prod_00001": [
    {
      "ID": "redis-prod-1",
      ...
    }
  ],
  "prod_00002": [
    {
      "ID": "redis-prod-2",
      ...
    }
  ]
}

contains

Determines if a needle is within an iterable element.

{{ if .Tags | contains "production" }}
# ...
{{ end }}

containsAll

Returns true if all needles are within an iterable element, or false otherwise. Returns true if the list of needles is empty.

{{ if containsAll $requiredTags .Tags }}
# ...
{{ end }}

containsAny

Returns true if any needle is within an iterable element, or false otherwise. Returns false if the list of needles is empty.

{{ if containsAny $acceptableTags .Tags }}
# ...
{{ end }}

containsNone

Returns true if no needles are within an iterable element, or false otherwise. Returns true if the list of needles is empty.

{{ if containsNone $forbiddenTags .Tags }}
# ...
{{ end }}

containsNotAll

Returns true if some needle is not within an iterable element, or false otherwise. Returns false if the list of needles is empty.

{{ if containsNotAll $excludingTags .Tags }}
# ...
{{ end }}

env

Reads the given environment variable accessible to the current process.

{{ env "CLUSTER_ID" }}

This function can be chained to manipulate the output:

{{ env "CLUSTER_ID" | toLower }}

Reads the given environment variable and if it does not exist or is blank use a default value, ex 12345.

{{ or (env "CLUSTER_ID") "12345" }}

mustEnv

Reads the given environment variable accessible to the current process. If the variable is not defined or is empty the substitution will fail with error.

{{ mustEnv "CLUSTER_ID" }}

envOrDefault

Reads the given environment variable accessible to the current process. If the environment variable is found, the value of that variable will be used. This includes empty values. Otherwise, the default will be used instead.

{{ envOrDefault "CLUSTER_NAME" "Default_Cluster" }}

This function can be chained to manipulate the output:

{{ envOrDefault "CLUSTER_NAME" "Default_Cluster"  | toLower }}

If you need the semantics of using the default when the environment has the value but it's empty, you can use normal env with or. This leverages the fact that go templates interpret the empty string ("") as false.

{{ or (env "TIMEOUT_CONNECT") "5s" }}

executeTemplate

Executes and returns a defined template.

{{ define "custom" }}my custom template{{ end }}

This is my other template:
{{ executeTemplate "custom" }}

And I can call it multiple times:
{{ executeTemplate "custom" }}

Even with a new context:
{{ executeTemplate "custom" 42 }}

Or save it to a variable:
{{ $var := executeTemplate "custom" }}

explode

Takes the result from a tree or ls call and converts it into a deeply-nested map for parsing/traversing.

{{ tree "config" | explode }}

Note: You will lose any metadata about the key-pair after it has been exploded. You can also access deeply nested values:

{{ with tree "config" | explode }}
{{ .a.b.c }}{{ end }}

You will need to have a reasonable format about your data in Consul. Please see Go's text/template package for more information.

explodeMap

Takes the value of a map and converts it into a deeply-nested map for parsing/traversing, using the same logic as explode.

{{ scratch.MapSet "example" "foo/bar" "a" }}
{{ scratch.MapSet "example" "foo/baz" "b" }}
{{ scratch.Get "example" | explodeMap | toYAML }}

indent

Indents a block of text by prefixing N number of spaces per line.

{{ tree "foo" | explode | toYAML | indent 4 }}

in

Determines if a needle is within an iterable element.

{{ if in .Tags "production" }}
# ...
{{ end }}

loop

Accepts varying parameters and differs its behavior based on those parameters.

If loop is given one integer, it will return a goroutine that begins at zero and loops up to but not including the given integer:

{{ range loop 5 }}
# Comment{{end}}

If given two integers, this function will return a goroutine that begins at the first integer and loops up to but not including the second integer:

{{ range $i := loop 5 8 }}
stanza-{{ $i }}{{ end }}

which would render:

stanza-5
stanza-6
stanza-7

Note: It is not possible to get the index and the element since the function returns a goroutine, not a slice. In other words, the following is not valid:

# Will NOT work!
{{ range $i, $e := loop 5 8 }}
# ...{{ end }}

join

Takes the given list of strings as a pipe and joins them on the provided string:

{{ $items | join "," }}

mergeMap

Takes the result from explode and an exploded argument then merges it both maps. The argument's source will not be overridden by piped map.

{{ $base := tree "base" | explode }}
{{ $overrides := tree "overrides" | explode | mergeMap $base}}
{{ with $overrides }}
{{ .a.b.c }}{{ end }}

mergeMapWithOverride

Takes the result from explode and an exploded argument then merges it both maps. The argument's source will be overridden by piped map.

{{ $base := tree "base" | explode }}
{{ $overrides := tree "overrides" | explode | mergeMapWithOverride $base}}
{{ with $overrides }}
{{ .a.b.c }}{{ end }}

trimSpace

Takes the provided input and trims all whitespace, tabs and newlines:

{{ file "/etc/ec2_version" | trimSpace }}

trim

Takes the provided input and trims all leading and trailing unicode:

{{ "hello world!!" | trim "!!" }}

trimPrefix

Takes the provided input and trims leading prefix string:

{{ "hello world!!" | trimPrefix "hello" }}

trimSuffix

Takes the provided input and trims trailing suffix string:

{{ "hello world!!" | trimSuffix "world!!" }}

parseBool

Takes the given string and parses it as a boolean:

{{ "true" | parseBool }}

This can be combined with a key and a conditional check, for example:

{{ if key "feature/enabled" | parseBool }}{{ end }}

parseFloat

Takes the given string and parses it as a base-10 float64:

{{ "1.2" | parseFloat }}

parseInt

Takes the given string and parses it as a base-10 int64:

{{ "1" | parseInt }}

This can be combined with other helpers, for example:

{{ range $i := loop key "config/pool_size" | parseInt }}
# ...{{ end }}

parseJSON

Takes the given input (usually the value from a key) and parses the result as JSON:

{{ with $d := key "user/info" | parseJSON }}{{ $d.name }}{{ end }}

Note: Consul Template evaluates the template multiple times, and on the first evaluation the value of the key will be empty (because no data has been loaded yet). This means that templates must guard against empty responses.

parseUint

Takes the given string and parses it as a base-10 int64:

{{ "1" | parseUint }}

parseYAML

Takes the given input (usually the value from a key) and parses the result as YAML:

{{ with $d := key "user/info" | parseYAML }}{{ $d.name }}{{ end }}

Note: The same caveats that apply to parseJSON apply to parseYAML.

plugin

Takes the name of a plugin and optional payload and executes a Consul Template plugin.

{{ plugin "my-plugin" }}

The plugin can take an arbitrary number of string arguments, and can be the target of a pipeline that produces strings as well. This is most commonly combined with a JSON filter for customization:

{{ tree "foo" | explode | toJSON | plugin "my-plugin" }}

Please see the Plugins section for more information about plugins.

regexMatch

Takes the argument as a regular expression and will return true if it matches on the given string, or false otherwise.

{{ if "foo.bar" | regexMatch "foo([.a-z]+)" }}
# ...
{{ else }}
# ...
{{ end }}

regexReplaceAll

Takes the argument as a regular expression and replaces all occurrences of the regex with the given string. As in go, you can use variables like $1 to refer to subexpressions in the replacement string.

{{ "foo.bar" | regexReplaceAll "foo([.a-z]+)" "$1" }}

replaceAll

Takes the argument as a string and replaces all occurrences of the given string with the given string.

{{ "foo.bar" | replaceAll "." "_" }}

This function can be chained with other functions as well:

{{ service "web" }}{{ .Name | replaceAll ":" "_" }}{{ end }}

sha256Hex

Takes the argument as a string and compute the sha256_hex value

{{ "bladibla" | sha256Hex }}

md5sum

Takes a string input as an argument, and returns the hex-encoded md5 hash of the input.

{{ "myString" | md5sum }}

hmacSHA256Hex

Takes a key and a message as string inputs. Returns a hex-encoded HMAC-SHA256 hash with the given parameters.

{{ hmacSHA256Hex "somemessage" "somekey" }}

Or with a pipe function

{{ "somekey" | hmacSHA256Hex "somemessage" }}

split

Splits the given string on the provided separator:

{{ "foo\nbar\n" | split "\n" }}

This can be combined with chained and piped with other functions:

{{ key "foo" | toUpper | split "\n" | join "," }}

splitToMap

Splits the given string on the provided separator and splits each resulting item into a key and value:

{{ "foo:bar\nbaz:bat\n" | splitToMap "\n" ":" }}

timestamp

Returns the current timestamp as a string (UTC). If no arguments are given, the result is the current RFC3339 timestamp:

{{ timestamp }} // e.g. 1970-01-01T00:00:00Z

If the optional parameter is given, it is used to format the timestamp. The magic reference date Mon Jan 2 15:04:05 -0700 MST 2006 can be used to format the date as required:

{{ timestamp "2006-01-02" }} // e.g. 1970-01-01

See Go's time.Format for more information.

As a special case, if the optional parameter is "unix", the unix timestamp in seconds is returned as a string.

{{ timestamp "unix" }} // e.g. 0

toJSON

Takes the result from a tree or ls call and converts it into a JSON object.

{{ tree "config" | explode | toJSON }}

renders

{"admin":{"port":"1234"},"maxconns":"5","minconns":"2"}

Note: Consul stores all KV data as strings. Thus true is "true", 1 is "1", etc.

toJSONPretty

Takes the result from a tree or ls call and converts it into a pretty-printed JSON object, indented by two spaces.

{{ tree "config" | explode | toJSONPretty }}

renders

{
  "admin": {
    "port": "1234"
  },
  "maxconns": "5",
  "minconns": "2"
}

Note: Consul stores all KV data as strings. Thus true is "true", 1 is "1", etc.

toUnescapedJSON

Takes the result from a tree or ls call and converts it into a JSON object without HTML escaping. This function comes in handy when working with db connection strings or URIs containing query parameters.

{{ tree "config" | explode | toUnescapedJSON }}

renders

{"admin":{"port":"1234"},"maxconns":"5","minconns":"2", "queryparams": "a?b=c&d=e"}
toUnescapedJSONPretty

Takes the result from a tree or ls call and converts it into a pretty-printed JSON object without HTML escaping, indented by two spaces.

{{ tree "config" | explode | toUnescapedJSONPretty }}

renders

{
  "admin": {
    "port": "1234"
  },
  "maxconns": "5",
  "minconns": "2",
  "queryparams": "a?b=c&d=e"
}

toLower

Takes the argument as a string and converts it to lowercase.

{{ key "user/name" | toLower }}

See Go's strings.ToLower for more information.

toTitle

Takes the argument as a string and converts it to titlecase.

{{ key "user/name" | toTitle }}

See Go's strings.Title for more information.

toTOML

Takes the result from a tree or ls call and converts it into a TOML object.

{{ tree "config" | explode | toTOML }}

renders

maxconns = "5"
minconns = "2"

[admin]
  port = "1134"

Note: Consul stores all KV data as strings. Thus true is "true", 1 is "1", etc.

toUpper

Takes the argument as a string and converts it to uppercase.

{{ key "user/name" | toUpper }}

See Go's strings.ToUpper for more information.

toYAML

Takes the result from a tree or ls call and converts it into a pretty-printed YAML object, indented by two spaces.

{{ tree "config" | explode | toYAML }}

renders

admin:
  port: "1234"
maxconns: "5"
minconns: "2"

Note: Consul stores all KV data as strings. Thus true is "true", 1 is "1", etc.

sockaddr

Takes a quote-escaped template string as an argument and passes it on to hashicorp/go-sockaddr templating engine.

{{ sockaddr "GetPrivateIP" }}

See hashicorp/go-sockaddr documentation for more information.

writeToFile

Writes the content to a file with permissions, username (or UID), group name (or GID), and optional flags to select appending mode or add a newline.

The username and group name fields can be left blank to default to the current user and group.

For example:

{{ key "my/key/path" | writeToFile "/my/file/path.txt" "" "" "0644" }}
{{ key "my/key/path" | writeToFile "/my/file/path.txt" "100" "1000" "0644" }}
{{ key "my/key/path" | writeToFile "/my/file/path.txt" "my-user" "my-group" "0644" }}
{{ key "my/key/path" | writeToFile "/my/file/path.txt" "my-user" "my-group" "0644" "append" }}
{{ key "my/key/path" | writeToFile "/my/file/path.txt" "my-user" "my-group" "0644" "append,newline" }}

Sprig Functions

Consul-template provides access to the Sprig library in templates. To use a Sprig function in your template, prepend sprig_ to the function name. A full list of Sprig functions can be found in the Sprig Function Documentation

Math Functions

The following functions are available on floats and integer values.

add

Returns the sum of the two values.

{{ add 1 2 }} // 3

This can also be used with a pipe function.

{{ 1 | add 2 }} // 3

subtract

Returns the difference of the second value from the first.

{{ subtract 2 5 }} // 3

This can also be used with a pipe function.

{{ 5 | subtract 2 }} // 3

Please take careful note of the order of arguments.

multiply

Returns the product of the two values.

{{ multiply 2 2 }} // 4

This can also be used with a pipe function.

{{ 2 | multiply 2 }} // 4

divide

Returns the division of the second value from the first.

{{ divide 2 10 }} // 5

This can also be used with a pipe function.

{{ 10 | divide 2 }} // 5

Please take careful note of the order or arguments.

modulo

Returns the modulo of the second value from the first.

{{ modulo 2 5 }} // 1

This can also be used with a pipe function.

{{ 5 | modulo 2 }} // 1

Please take careful note of the order of arguments.

minimum

Returns the minimum of the two values.

{{ minimum 2 5 }} // 2

This can also be used with a pipe function.

{{ 5 | minimum 2 }} // 2

maximum

Returns the maximum of the two values.

{{ maximum 2 5 }} // 2

This can also be used with a pipe function.

{{ 5 | maximum 2 }} // 2

Nomad Functions

Nomad service registrations can be queried using the nomadServices and nomadService functions. Nomad variables can be queried using the nomadVarList and nomadVar functions. Typically these will be used from within a Nomad template configuration.

nomadServices

This can be used to query the names of services registered in Nomad.

{{ range nomadServices }}
  {{ .Name .Tags }}
{{ end }}

nomadService

This can be used to query for additional information about each instance of a service registered in Nomad.

{{ range nomadService "my-app" }}
  {{ .Address }} {{ .Port }}
{{ end}}

The nomadService function also supports basic load-balancing via a rendezvous hashing algorithm implemented in Nomad's API. To activate this behavior, the function requires three arguments in this order: the number of instances desired, a unique but consistent identifier associated with the requester, and the service name.

Typically the unique identifier would be the allocation ID in a Nomad job.

{{$allocID := env "NOMAD_ALLOC_ID" -}}
{{range nomadService 3 $allocID "redis"}}
  {{.Address}} {{.Port}} | {{.Tags}} @ {{.Datacenter}}
{{- end}}

Nomad Variables

Consul-template can access Nomad variables and use their values as output to or to control template rendering flow.

nomadVarList

This can be used to list the paths of Nomad variables available to the currently running template based on Nomad ACLs. You can provide an optional prefix to filter the path list by as a parameter. The value returned is a slice of NomadVarMeta objects, which print their Path value when used as a string.

{{ nomadVarList "<PREFIX>" }}

Example:

{{ range nomadVarList }}
  {{ . }}
{{ end }}

You can provide a prefix to filter the path list as an optional parameter to the nomadVarList function.

{{ range nomadVarList "path/to/filter"}}
  {{ . }}
{{ end }}

nomadVarListSafe

Same as nomadVarList, but refuses to render template if the variable list query returns blank/empty data.

The NomadVarMeta Type

type NomadVarMeta struct {
	Namespace, Path          string
	CreateIndex, ModifyIndex uint64
	CreateTime, ModifyTime   nanoTime
}

The nanoTime type contains Unix nanoseconds since the epoch. Its String method prints it out as a formatted date/time value. It also has a Time method that can be used to retrieve a Go time.Time for further manipulation.

nomadVar

Query Nomad for the Items at the given variable path. If the path does not exist or the caller does not have access to it, Consul Template will block rendering until the path is available. To avoid blocking, wrap nomadVar calls with nomadVarExists.

The nomadVar function returns a map of NomadVarItems structs. Each member of the map corresponds to a member of the variable's Items collection. Each map value also contains helper methods to get the variable metadata and a link to the parent variable.

{{ nomadVar "<PATH>" }}

For example:

Given a variable exists at the path nomad/jobs/redis, with a single key/value pair in its Items collection—maxconns:15

{{ with nomadVar "nomad/jobs/redis" }}{{ .maxconns }}{{ end }}

renders

15

The NomadVarItems Type

Calls to the nomadVar function return a value of type NomadVarItems. This value is a map of string to NomadVarItem elements. It provides some convenience methods:

  • Keys: produces a sorted list of keys to this NomadVarItems map.

  • Values: produces a key-sorted list.

  • Tuples: produces a key-sorted list of K,V tuple structs.

  • Metadata: returns this collection's parent metadata as a NomadVarMeta

  • Parent: returns the consul-template version (NomadVariable) of a full variable, which has the Metadata values and Items collection as peers.

The NomadVarItem Type

Specific item of a NomadVarItems collection that are directly accessed by name or the elements that are obtained while ranging the return of nomadVar are of type NomadVarItem.

type NomadVarItem struct {
	Key, Value string
}

NomadVarItem objects also provide helper methods:

  • Metadata: returns this item's parent's metadata as a NomadVarMeta

  • Parent: returns the consul-template version (NomadVariable) of the Nomad variable that contains this item.

nomadVarExists

Query Nomad to see if a variable exists at the given path. If a variable exists, this will return true, false otherwise. Unlike nomadVar, this function will not block if the variable does not exist, which can be useful for controlling flow.

{{ nomadVarExists "<PATH>" }}

For example:

{{ if nomadVarExists "app/beta_active" }}
  # ...
{{ else }}
  # ...
{{ end }}

Debugging Functions

Debugging functions help template developers understand the current context of a template block. These are provided by the spew library. See the spew GoDoc documentation for more information.

spew_dump

Outputs the value with full newlines, indentation, type, and pointer information to stdout (instead of rendered in the template) by calling spew.Dump on it. Returns an empty string or an error.

{{- $JSON := `{ "foo": { "bar":true, "baz":"string", "theAnswer":42} }` -}}
{{- $OBJ := parseJSON $JSON -}}
{{- spew_dump $OBJ -}}

renders

>
(map[string]interface {}) (len=1) {
 (string) (len=3) "foo": (map[string]interface {}) (len=3) {
  (string) (len=3) "bar": (bool) true,
  (string) (len=3) "baz": (string) (len=6) "string",
  (string) (len=9) "theAnswer": (float64) 42
 }
}

spew_sdump

Creates a string containing the values with full newlines, indentation, type, and pointer information by calling spew.Sdump on them. Returns an error or the string. The return value can be captured as a variable, used as input to a pipeline, or written to the template in place.

{{- $JSON := `{ "foo": { "bar":true, "baz":"string", "theAnswer":42} }` -}}
{{- $OBJ := parseJSON $JSON -}}
{{- spew_dump $OBJ -}}

renders

>
(map[string]interface {}) (len=1) {
 (string) (len=3) "foo": (map[string]interface {}) (len=3) {
  (string) (len=3) "bar": (bool) true,
  (string) (len=3) "baz": (string) (len=6) "string",
  (string) (len=9) "theAnswer": (float64) 42
 }
}

spew_printf

Formats output according to the provided format string and then writes the generated information to stdout. You can use format strings to produce a compacted inline printing style by your choice:

  • %v: most compact
  • %+v: adds pointer addresses
  • %#v: adds types
  • %#+v: adds types and pointer addresses
spew_printf("myVar1: %v -- myVar2: %+v", myVar1, myVar2)
spew_printf("myVar3: %#v -- myVar4: %#+v", myVar3, myVar4)

Examples

Given this template fragment,

{{- $JSON := `{ "foo": { "bar":true, "baz":"string", "theAnswer":42} }` -}}
{{- $OBJ := parseJSON $JSON -}}

using %v

{{- spew_printf "%v\n" $OBJ }}

outputs

map[foo:map[bar:true baz:string theAnswer:42]]

using %+v

{{ spew_printf "%+v\n" $OBJ }}

outputs

map[foo:map[bar:true baz:string theAnswer:42]]

using %+v

{{ spew_printf "%v\n" $OBJ }}

outputs

map[foo:map[bar:true baz:string theAnswer:42]]

using %#v

{{ spew_printf "%#v\n" $OBJ }}

outputs

(map[string]interface {})map[foo:(map[string]interface {})map[bar:(bool)true baz:(string)string theAnswer:(float64)42]]

using %+#v

using %#v

{{ spew_printf "%#+v\n" $OBJ }}

outputs

(map[string]interface {})map[foo:(map[string]interface {})map[theAnswer:(float64)42 bar:(bool)true baz:(string)string]]

spew_sprintf

If you would prefer to use format strings with a compacted inline printing style, use the convenience wrappers for spew.Printf, spew.Sprintf, etc with:

  • %v: most compact
  • %+v: adds pointer addresses
  • %#v: adds types
  • %#+v: adds types and pointer addresses