Go Docker Utility Belt (GoDub)

• 1. Get it
• 2. Usage
  • 2.1. Template
  • 2.2. Ensure
  • 2.3. Path
• 3. Template Functions
  • 3.1. Sprig
  • 3.2. Custom
• 4. Development
  • 4.1. Build
  • 4.2. Run
  • 4.3. Tests
  • 4.4. Initialize Project
• 5. Similar Tools

The Go Docker Utility Belt, or short GoDub, is a tool to simplify and support the configuration of applications on Docker images. One major feature is the templating support which allows it to render Go templates on the command line using environment variables.

It is directly inspired by the original Docker Utility Belt written by Confluent. The original is a Python tool, which for example uses Jinja2 for rendering templates.

In addition, it is also inspired by envtpl tool, which is written in Go. The major difference between envtpl and GoDub is, that GoDub includes additional templating functions, for example to transform environment variable keys to a property format.

The main reason to develip GoDob was, to provide templating support for configuration files which need to be part of a Docker image, without the need to install Python.

1. Get it

go get github.com/ueisele/go-docker-utils

2. Usage

godub [command]

Available Commands:
  template    Uses Go template and environment variables to generate configuration files.
  ensure      Ensures that environment variables are defined.
  path        Checks a path on the filesystem for permissions.

GoDub provides the three base functions template, ensure and path.

2.1. Template

godub template [flags]

Flags:
  -i, --in strings       The template files (glob pattern). If not provided, it is read from stdin.
  -o, --out string       The output file or directory. If not provided, it is written to stdout.
  -r, --refs strings     Reference templates (glob pattern).
  -v, --values strings   Values files (glob pattern). Can be used with '.Values.' prefix.
  -f, --files strings    Available files inside templates (directories). It should be noted that all files are immediately loaded into memory. Can be used with '.Files.' prefix.
  -s, --strict           In strict mode, rendering is aborted on missing field.

• If --in is not provided, GoDub reads from stdin

• If --out is not provided, GoDub writes to stdout

2.1.1. Examples

Renders a template from stdin …​

export GREET="Hey :)"
echo "{{ .Env.GREET }}" | ./godub template

... and outputs Hey :) to stdout.

Hey :)

Renders the template from file examples/server.properties.gotpl …​

export KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9092
export KAFKA_NODE_ID=1
export KAFKA_LOG4J_LOGGERS=kafka.server.KafkaApis=TRACE
./godub template < examples/server.properties.gotpl > examples/server.properties

... and writes the output to file examples/server.properties

advertised.listeners=PLAINTEXT://127.0.0.1:9092
node.id=1

The template examples/server.properties.gotpl makes use of the envToProp function provided by GoDub, which transforms environment variable keys to properties. This feature was ported from Confluent`s Docker Utility Belt.

The reason why the environment variable KAFKA_LOG4J_LOGGERS is not added is that it is defined as an property to exclude ($excluded_props).

Excerpt from examples/server.properties.gotpl

{{- range $key, $value := envToProp "KAFKA_" "" $excluded_props -}}
{{ $key }}={{ tpl $value $ }}
{{ end }}

Can make usage of reference templates

---
./godub template -i examples/define.gotpl -r examples/helpers.gotpl
---

A reference templates can contain template definitions…​

{{- define "T2" -}}
{{- range $key, $value := . -}}
{{ $key }}={{ $value }}
{{ end }}
{{- end -}}

which can be used in templates

{{ template "T2" . }}
{{ template "T2" envToMap "KAFKA_" }}

You can also load additional value files in formats yaml, json, toml and properties

./godub template -i examples/deployment.yaml.gotpl --values examples/values.yaml

examples/values.yaml

deployment:
  name: templated-name-from-yaml

examples/deployment.yaml.gotpl

apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ .Values.deployment.name }}

It is possible to use files in templates

./godub template -i examples/filesglob.gotpl -f examples/filestest

The usage is basically identical to Helm: https://helm.sh/docs/chart_template_guide/accessing_files/

{{- range $path, $content := .Files.Glob "**/*.json" }}
file: {{ $path }}
{{ $content | fromJSON | toYAML }}
{{- end }}

2.2. Ensure

godub ensure [flags]

Flags:
  -a, --at-least-one   By the default it is ensured that all environment variables are defined. If this flag is set, it is enough if at least one is defined.

2.2.1. Examples

Ensures that the environment variable KAFKA_ADVERTISED_LISTENERS is defined …​

export KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9092
./godub ensure KAFKA_ADVERTISED_LISTENERS

... and completes with exit status 0

echo $?
0

For a environment variable like in this example KAFKA_OPTS, which is not defined …​

export KAFKA_OPTS=
./godub ensure KAFKA_OPTS

... GoDub completes with exit status 1

Error: environment variables are missing: [KAFKA_OPTS]
echo $?
1

Ensures that at least one of the environment variables KAFKA_ADVERTISED_LISTENERS and KAFKA_LISTENERS are defined …​

export KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9092
export KAFKA_LISTENERS=
./godub ensure --at-least-one KAFKA_ADVERTISED_LISTENERS KAFKA_LISTENERS

... and completes with exit status 0, because KAFKA_ADVERTISED_LISTENERS is defined

echo $?
0

2.3. Path

godub path [flags]

Flags:
  -e, --existence          Path must be existence (default true)
  -r, --readable           Path must be readable
  -w, --writeable          Path must be writeable
  -x, --executable         Path must be executable
  -t, --timeout duration   Time to wait for the URL to be retrievable (default 0s)

2.3.1. Examples

Could wait for a given amount of time for a specified file

./godub path -t 5s /app/file-which-should-exist

3. Template Functions

3.1. Sprig

The Sprig library is included and therefore all Sprig functions are supported (see https://masterminds.github.io/sprig/).

3.2. Custom

The following additional functions are suppored:

• Test Functions

  • heygodub

• Env functions

  • hasEnv

  • fromEnv

  • envToMap

  • envToProp

• Map functions

  • excludeKeys

  • replaceKeyPrefix

  • toPropertiesKey

• String functions

  • kvCsvToMap

• List functions

  • filterHasPrefix

• Verify functions

  • required

• Network functions

  • ipAddresses

  • ipAddress

  • anyIpAddress

• Format functions

  • toYAML

  • fromYAML

  • toJSON

  • toJSONPretty

  • fromJSON

  • toTOML

  • fromTOML

  • toProperties

  • fromProperties

The functions are implemented in pkg/template/functions.go.

ToDo: Detailed documentation of functions!

4. Development

GoDub provides a Visual Studio Code Remote Development in Containers set up. Just re-open this folder as Remote-Container, and you have a Goland development environment.

Instructions about how to configure Remote Development: https://code.visualstudio.com/docs/remote/containers-tutorial

4.1. Build

./build.sh

You can also cross-compile GoDub with the following command:

GOOS=linux GOARCH=arm64 ./build.sh

4.2. Run

./godub

Hint: At the moment, the godub binary is not a dynamic executable. If, during some changes, the binary is no longer statically linked by default, it can be achieved by adding the following parameter to the go build command.

-ldflags "-s -w -linkmode external -extldflags -static"

4.3. Tests

go test -v ./...

ToDo: Implement tests!

functions_test.go

package template

import (
  "os"
  "testing"
  "github.com/stretchr/testify/assert"
)

func TestHasEnv(t *testing.T) {
  os.Setenv("GODUB_TEST_HAS_ENV_EXISTING", "value")
  assert.True(t, hasEnv("GODUB_TEST_HAS_ENV_EXISTING"))
  assert.False(t, hasEnv("GODUB_TEST_HAS_ENV_SOMETHING_ELSE"))
}

4.4. Initialize Project

Typically, this is not required, because go.mod is added to Git. However, to re-create it run_

go mod init github.com/ueisele/go-docker-utils
go mod tidy

5. Similar Tools

• https://github.com/confluentinc/confluent-docker-utils

• https://github.com/subfuzion/envtpl