Skip to content
/ dockly Public

DSL and Gem for building ready-to-launch Docker images

License

Notifications You must be signed in to change notification settings

upserve/dockly

Repository files navigation

Gem Version

Dockly

dockly is a gem made to ease the pain of packaging an application. For this gem to be useful, you will want to use Docker for process isolation.

Although only a specific type of repository may be used, these assumptions allow us to define a simple DSL to describe your repository.

Tool Requirements

To use the generated startup scripts, you'll need to use AWS CLI v1.5.0+

Usage

Once a package block has been defined by the DSL below, dockly is invoked by either (for a deb) bundle exec dockly build #{deb block name} or bundle exec rake dockly:deb:#{deb block name}. If looking to just build a docker block, run either bundle exec dockly docker #{docker block name} or bundle exec rake dockly:docker:#{docker block name}. To build without exporting, run add either --no-export or :noexport to the CLI program or Rake task

The DSL

The DSL is broken down into multiple objects, all of which conform to a specific format. Each object starts with the name of the section, followed by a name for the object you're creating, and a block for configuration.

docker :test_docker do
  # code here
end

Each object has an enumeration of valid attributes. The following code sets the repository attribute in a docker called test_docker:

docker :test_docker do
  repository 'an-awesome-name'
end

Finally, each object has zero or more valid references to other DSL objects. The following code sets deb that references a docker:

docker :my_docker do
  repository 'my-name'
end

deb :my_deb do
  docker :my_docker
end

Below is an alternative syntax that accomplishes the same thing:

deb :my_deb do
  docker do
    repository 'my-name'
  end
end

build_cache

Optional

The build_cache DSL is used to prevent rebuilding assets every build and used cached assets.

  • s3_bucket
    • required: true
    • description: the bucket name to download and upload build caches to
  • s3_object_prefix
    • required: true
    • description: the name prepended to the package; allows for namespacing your caches
  • hash_command
    • required: true
    • description: command run to determine if the build cache is up to date (eg. md5sum ... | awk '{ print $1 }')
  • parameter_command
    • required: false
    • allows multiple
    • description: command run to build specific versions of build caches -- useful for multiple operating systems (not required)
  • build_command
    • required: true
    • description: command run when the build cache is out of date
  • output_dir
    • required: true
    • description: where the cache is located in the Docker image filesystem
  • tmp_dir
    • required: true
    • default: /tmp
    • description: where the build cache files are stored locally; this should be able to be removed easily since they all exist in S3 as well
  • use_latest
    • required: false
    • default: false
    • description: when using S3, will insert the S3 object tagged as latest in your "s3://s3_bucket/s3_object_prefix" before running the build command to quicken build times
  • keep_old_files
    • required: false
    • default: false
    • description: if this option is false when using docker, it will overwrite files that already exist in output_dir

docker

The docker DSL is used to define Docker containers. It has the following attributes:

  • registry_import
    • required: false -- only required when import is not supplied
    • description: the location of the base image to start building from
    • examples: paintedfox/ruby, registry.example.com/my-custom-image
  • build_env
    • required: false
    • description: Hash whose values are environment variables and keys are their values. These variables are only used during build commands, exported images will not contain them.
  • import
    • required: false -- only required when registry_import is not supplied
    • description: the location (url or S3 path) of the base image to start building from
  • git_archive:
    • required: false
    • default: nil
    • description: the relative file path of git repo that should be added to the container
  • build
    • required: true
    • description: aditional Dockerfile commands that you'd like to pass to docker build
  • repository
    • required: true
    • default: 'dockly'
    • description: the repository of the created image
  • name
    • alias for: repository
  • tag
    • required: true
    • description: the tag of the created image
  • build_dir
    • required: true
    • default: ./build/docker
    • description: the directory of the temporary build files
  • package_dir
    • required: true
    • default: /opt/docker
    • description: the location of the created image in the package
  • timeout
    • required: true
    • default: 60
    • description: the excon timeout for read and write when talking to docker through docker-api
  • build_caches
    • required: false
    • default: []
    • description: a listing of references to build caches to run
  • s3_bucket
    • required: false
    • default: nil
    • description: S3 bucket for where the docker image is exported
  • s3_object_prefix
    • required: false
    • default: nil
    • description: prefix to be added to S3 exported docker images
  • tar_diff
    • required: false
    • default: false
    • description: after docker export, performs a diff between the base image and the new image

In addition to the above attributes, docker has the following references:

  • build_cache
    • required: false
    • allows multiple
    • class: Dockly::BuildCache
    • description: a caching system to stop rebuilding/compiling the same files every time
  • registry
    • required: false
    • allows one
    • class: Dockly::Docker::Registry
    • description: a registry to push to in lieu of exporting as a tar -- the registry will be automatically pulled upon installing the package
  • ecr
    • required: false
    • allows one
    • class: Dockly::Docker::ECR
    • description: an AWS ECR Docker registry to push to in lieu of exporting as a tar -- the registry will be automatically pulled upon installing the package

Need finer control of Docker packages? We also wrote docker-api.

registry

The registry DSL is used to define Docker Registries. It has the following attributes:

  • authentication_required
    • required: false
    • default: true
    • description: a boolean that determines whether or not authentication is required on the registry.
  • username
    • required: true unless authentication_required is false
    • description: the username to authenticate
  • email:
    • required: true unless authentication_required is false
    • description: the email to authenticate
  • password:
    • required: false
    • description: the user's password; unless supplied, ENV['DOCKER_REGISTRY_PASSWORD'] will be checked. not that ENV['DOCKER_REGISTRY_PASSWORD'] is required to be set on the host on to which the package will be deployed
  • server_address:
    • required: true
    • default: https://index.docker.io/v1
    • description: the server where the registry lives

ecr

The ecr DSL is used to define AWS ECR Docker registries. It has the following attributes:

  • server_address
    • required: true
    • description: the server where the registry lives without the repo name, eg accountid.dkr.ecr.region.amazonaws.com
  • username
    • required: false
    • default: pulled from auth data obtained using assumed role's credentials
    • description: the username to authenticate
  • password
    • required: false
    • default: pulled from auth data obtained using assumed role's credentials
    • description: the password to authenticate

foreman

Optional

The foreman DSL is used to define the foreman export scripts. It has the following attributes:

  • env
    • description: accepts same arguments as foreman start --env
  • procfile
    • required: true
    • default: 'Procfile'
    • description: the Procfile to use
  • type
    • required: true
    • default: 'upstart'
    • description: the type of foreman script being defined
  • user
    • required: true
    • default: 'nobody'
    • description: the user the scripts will run as
  • root_dir
    • required: false
    • default: '/tmp'
    • description: set the root directory
  • init_dir
    • required: false
    • default: '/etc/init'
    • description: the location of the startup scripts in the rpm -qplian package
  • prefix

deb

The deb DSL is used to define Debian packages. It has the following attributes:

  • package_name
    • required: true
    • description: the name of the created package
  • version
    • required: true
    • default: 0.0
    • description: the version of the created package
  • release
    • required: true
    • default: 0
    • description: the realese version of the created package
  • arch
    • required: true
    • default: x86_64
    • description: the intended architecture of the created package
  • vendor
    • required: false
    • default: Dockly
    • description: Vendor name for this package
  • build_dir
    • required: true
    • default: build/deb
    • description: the location of the temporary files on the local file system
  • pre_install, post_install, pre_uninstall, post_uninstall
    • required: false
    • default: nil
    • description: script hooks for package events
  • s3_bucket
    • required: false
    • default: nil
    • description: the s3 bucket the package is uploaded to
  • file SOURCE DEST
    • required: false
    • description: places SOURCE at DEST in the Debian package (can have multiple of these)

In addition to the above attributes, deb has the following references:

  • docker
    • required: false
    • default: nil
    • class: Dockly::Docker
    • description: configuration for an image packaged with the deb
  • foreman
    • required: false
    • default: nil
    • class: Dockly::Foreman
    • description: any Foreman scripts used in the deb.

rpm

Same as deb above, but with the following additions:

  • vendor
    • required: true
    • default: Dockly
    • description: Vendor name for this package
  • os
    • required: true
    • default: linux
    • description: The operating system to target this rpm for

Demo

deb :dockly_package do
  package_name 'dockly_package'

  docker do
    name :dockly_docker
    import 's3://.../base-image.tar.gz'
    git_archive '/app'
    timeout 120

    build_cache do
      s3_bucket "dockly-bucket-name"
      s3_object_prefix "bundle_cache/"
      hash_command "cd /app && ./script/bundle_hash"
      build_command "cd /app && ./script/bundle_package"
      output_dir "/app/vendor/bundle"
      use_latest true
    end

    build <<-EOF
      run cd /app && ./configure && make
    EOF
  end

  foreman do
    name 'dockly'
    procfile 'Procfile'
    log_dir '/data/logs'
  end

  s3_bucket 'dockly-bucket-name'
  # ends up in s3://#{s3_bucket}/#{package_name}/#{git_hash}/#{package_name}_#{version}.#{release}_#{arch}.deb
end

Copyright (c) 2013 Swipely, Inc. See LICENSE.txt for further details.