Skip to content
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
examples
tests
.dockerignore
.gitignore
Dockerfile
Dockerfile.build
LICENSE
Makefile
README.md
docker.go
docker_container.go
docker_cp.go
docker_network.go
docker_volume.go
environment.go
files.go
glide.lock
glide.yaml
insulatr.go
insulatr.yaml
main.go
repos.go
services.go
steps.go

README.md

insulatr

insulatr is a tool for container native builds written in Go.

Based on a YAML file, insulatr isolates build steps in individual containers while results are transported using a Docker volume.

Why insulatr

Container native builds facilitate container to execute the individual steps in a build definition. The provides the following advantages:

  1. Runtime environment: When executing tasks in a container the requirements on the host are reduced to the container runtime. By choosing the container image, the build steps is executed in the appropriate runtime environment.

  2. Isolation: The tasks executed as part of the build are isolated from each other as well as from the host. It is even possible to use conflicting toolsets for individual steps.

  3. Reproducibility: Each build step is isolated in a predefined runtime environment and will produce the same behaviour when repeated.

  4. Pipeline as Code: The build process is specified in a textual form and can be stored in the same repository as the code. Contrary to a build script, it specifies a single execution path.

insulatr is deliberately designed as a standalone tool to execute a build definition in containerized steps. Although multiple CI/CD tools and products exist which combine a scheduler with an execution engine, they are tightly coupled. By having a separate tool like insulatr, builds can be reproduced in any compatible execution environment - during development as well as in stages of a deployment.

Table of contents

  1. Usage
    1. Local
    2. Docker image
    3. Alias
  2. Build definitions
    1. Settings
    2. Environment
    3. Repositories
    4. Files
    5. Services
    6. Build Steps
  3. Building
  4. Design
  5. Useful links

Usage

insulatr supports different ways to launch.

Local

When calling insulatr without any parameters, it will look for a file called insulatr.yaml in the current directory.

The following parameters are supported:

Options:

  -h, --help                        display help information
  -f, --file[=./insulatr.yaml]      Build definition file
      --reuse-volume[=false]        Use existing volume
      --retain-volume[=false]       Retain existing volume
      --reuse-network[=false]       Use existing network
      --retain-network[=false]      Retain existing network
      --reuse[=false]               Same as --reuse-volume and --reuse-network
      --remove[=false]              Same as --retain-volume and --retain-network
      --allow-docker-sock[=false]   Allow docker socket in build steps
      --allow-privileged[=false]    Allow privileged container for services

Docker image

The Docker image nicholasdille/insulatr is automatically built by Docker Hub. insulatr ships as a scratch image with only the statically linked binary.

The following tags are currently supported:

New releases receive a git tag which triggers a separate build which produces a new image tagged with the versions.

The Docker image is used in the following way:

docker run -it --rm --volume $(pwd)/insulatr.yaml:/insulatr.yaml --volume /var/run/docker.sock:/var/run/docker.sock nicholasdille/insulatr [<parameters>]

Alias

If the Docker daemon is running on a remote host, the following alias will add the local insulatr.yaml to a new image and run it:

alias insulatr="echo -e 'FROM nicholasdille/insulatr\nADD insulatr.yaml /' | docker image build --file - --tag insulatr:test --quiet . | xargs -r docker run -t -v /var/run/docker.sock:/var/run/docker.sock"

Build definitions

insulatr requires a build definition written in YAML with the following sections:

  1. Settings
  2. Environment
  3. Repositories
  4. Files
  5. Services
  6. Build Steps

Settings

The settings node defines global configuration options. It supports the following (optional) fields:

  • volume_name contains the name of the volume transporting repository checkouts as well as builds results across the build steps. It defaults to myvolume.
  • volume_driver specifies the volume driver to use. It defaults to local.
  • working_directory contains the path under which to mount the volume. It defaults to /src.
  • shell is an array specifying the shell to run commands under. It defaults to [ "sh" ] to support minimized distribution images.
  • network_name contains the name of the network to connect services as well as build steps with. It defaults to mynetwork.
  • network_driver specifies the network driver to use. It defaults to bridge.
  • timeout defines how long to wait (in seconds) for the whole build before failing. It defaults to 3600.
  • log_directory specifies the directory to store logs in. It defaults to logs.
  • console_log_level controls what level of messages are displayed. Valid values are NOTICE, INFO, DEBUG. IT defaults to NOTICE.
  • reuse_volume defines whether the volume may be reused if it already exists. It defaults to false.
  • retain_volume defines whether the volume may not be deleted. It defaults to false.
  • reuse_network defines whether the network may be reused if it already exists. It defaults to false.
  • retain_network defines whether the network may not be deleted. It defaults to false.

To summarize, the default settings are:

settings:
  volume_name: myvolume
  volume_driver: local
  working_directory: /src
  shell: [ "sh" ]
  network_name: mynetwork
  network_driver: bridge
  timeout: 60
  log_directory: logs
  console_log_level: NOTICE
  reuse_volume: false
  retain_volume: false
  reuse_network: false
  retain_network: false

Environment

The environment node defines a list of global environment variables. They are automatically added to every build step and can be added to services:

environment:
  - FOO=bar

services:
  - name: backend
    image: myimage
    environment:
      - FOO

steps:
  - name: build
    image: myotherimage
    commands:
      - printenv

Variables names without values will be resolved against the environment of insulatr.

Repositories

The repos node defines a list of Git repositories to checkout before executing build steps. Currently, only unauthorized repositories are supported. The following fields are supported per repository:

  • name (mandatory) contains the given name for a repository.
  • location (mandatory) contains the URL to the repository.
  • directory (optional) contains the directory to checkout into. If omitted, the checkout behaves as git clone <url> and creates a new directory with a name based on the repository name.
  • shallow (optional) specifies whether to create a shallow clone. It defaults to false.
  • branch (optional) specifies a branch to checkout.
  • tag (optional) specifies a tag to checkout.
  • commit (optional) specifies a commit to checkout.

A typical repository definition looks like this:

repos:
  - name: main
    location: https://github.com/nicholasdille/insulatr

Git repositories can be accessed using HTTPS or SSH. Currently, credentials for HTTPS are not supported and you are strongly discouraged from hardcoding the credentials in plaintext in the build definition. For SSH, the agent socket is mapped into the container so that public key authentication will work.

Note that you can use the following URL to clone from GitHub using SSH without authenticating: git://github.com/<username>/<repo>.git

Files

The files node defines a list of files to be injected into the volume before running the build steps as well as extracted after the build steps completed successfully. A typical definitions looks like this:

files:
  - inject: "Makefile"
  - inject: "*.jar"
  - inject: foo.txt
    content: foobar
  - inject: bar.txt
    content: |-
      foo
      bar
  - extract: bar.txt

The inject type can be used in two ways:

  1. If content is set, a new file is created in the volume
  2. If content is omited, locally existing files and directory are injected. The only supported wildcard is *.

When adding a whole directory, the following works...

files:
  - inject: go

... but the following if go/ does not exist in the volume...

files:
  - inject: go/*

Services

The services node defines a list of services required by the build steps. The are started in order before build steps are executed. The following fields are supported per service:

  • name (mandatory) contains the given name for a repository.
  • image (mandatory) specifies the image to run the services with.
  • environment (optional) defines the environment variables required to configure the service.
  • suppress_log (optional) specifies whether the logs will be displayed when the service is stopped.
  • privileged (optional) specifies whether the container will be privileged. It defaults to false.

A typical service definition looks like this:

services:
  - name: web
    image: nginx

Build steps

The steps node defines a list of build steps to execute. XXX.

  • name (mandatory) contains the given name of a build step.
  • image (mandatory) specifies the image to run the step with.
  • commands (mandatory) is a list of commands to execute in the build step.
  • environment (optional) defines the environment variables passed to the build step.
  • user (optional) is a user to execute the commands under.
  • shell (optional) overrides the global shell setting.
  • working_directory contains the path under which to mount the volume. If omitted, it is filled from the global setting.
  • forward_ssh_agent (optional) enabled bind mounting the SSH agent socket into the build step. It defaults to false.
  • override_entrypoint (optional) executes the shell as the entrypoint. It defaults to false.
  • mount_docker_sock (optional) mounts /var/run/docker.sock into the container. It defaults to false.
  • forward_ssh_agent (optional) enables mapping of the SSH agent socket into the container. It defaults to false.

Typical build steps look like this:

steps:
  - name: build
    image: alpine
    environment:
      - FOO=bar
    commands:
      - printenv

If environment variables are specified without a value, the value is taken from an existing environment variable available to insulatr. Consider the following build definition:

steps:
  - name: build
    environment:
      - FOO
    commands:
      - printenv

When it is executed using FOO=bar insulatr, the build step received the environment variable FOO with the value bar from the environment of insulatr.

Example

settings:
  volume_name: myvolume
  working_directory: /src
  shell: [ "sh", "-x", "-e" ]
  network_name: mynetwork

repos:
  - name: main
    location: https://github.com/docker/app
    shallow: true
    directory: app
  - name: main
    location: https://github.com/docker/distribution
    shallow: true
    directory: distribution

services:
  - name: dind
    image: docker:dind
    privileged: true

steps:

  - name: user
    image: alpine
    user: 1000
    commands:
      - id -u

  - name: build
    image: docker:stable
    environment:
      - DOCKER_HOST=tcp://dind:2375
    commands:
      - printenv
      - docker version

  - name: dood
    image: docker:stable
    override_entrypoint: true
    mount_docker_sock: true
    commands:
      - docker version

Building

The following commands build insulatr from source.

  1. Clone repository: git clone https://github.com/nicholasdille/insulatr
  2. Download dependencies: make deps
  3. Build static binary: make static

The resulting binary is located in bin/insulatr-x86_64.

Design

This sections lists internals about insulatr.

Execution order

The order of the sections is:

  1. services
  2. repos
  3. steps

Useful links

Docker Go SDK

Docker Go Examples

GitLab Runner Docker Executor

Docker CLI

You can’t perform that action at this time.