Skip to content

Commit

Permalink
Docs update. There will be a lot of these.
Browse files Browse the repository at this point in the history
  • Loading branch information
@markround
markround committed Feb 8, 2017
1 parent eba4038 commit f7e74d4
Show file tree
Hide file tree
Showing 8 changed files with 210 additions and 214 deletions.
177 changes: 0 additions & 177 deletions README.md
View file
Expand Up @@ -15,180 +15,3 @@ It's available as a [Ruby Gem](https://rubygems.org/gems/tiller), so installatio

# Documentation
The main documentation has been updated and a searchable, easy to read version is now hosted on [readthedocs.io](http://tiller.readthedocs.io/). You can also read the raw markdown files by browsing the [docs](docs/) directory in this repository.





## Common configuration

### Complete example
I'll cover each part of this in detail, but to give you an idea of where we're going with this, here's the complete configuration file I discuss in the examples below :

```yaml
exec: [ "/usr/bin/mongod" , "--config" , "/etc/mongodb.conf" , "--rest" ]
data_sources: [ 'file' ]
template_sources: [ 'file' ]
environments:

staging:

mongodb.erb:
target: /etc/mongodb.conf
user: root
group: root
perms: 0644
config:
replSet: 'staging'

production:
mongodb.erb:
target: /etc/mongodb.conf
config:
replSet: 'production'

development:

mongodb.erb:
target: /etc/mongodb.conf
```
Note that instead of the YAML one-per-line list format for enabling plugins, I used the shorthand array format ( `[ 'item1' , 'item2', .....]` ).

I'll now cover each section and parameter in the following paragraphs.

### Main configuration values

`common.yaml` contains most of the configuration for Tiller. It contains top-level `exec`, `data_sources`, `template_sources` and `default_environment` parameters, along with sections for each environment.

It can also take optional blocks of configuration for some plugins (for example, the [Consul Plugins](docs/plugins/consul.md)). Settings defined here can also be overridden on a per-environment basis (see [below](#overriding-common-settings))

* `exec`: This is simply what will be executed after the configuration files have been generated. If you omit this (or use the `-n` / `--no-exec` arguments) then no child process will be executed. As of 0.5.1, you can also specify the command and arguments as an array, e.g.

```yaml
exec: [ "/usr/bin/supervisord" , "-n" ]
```

This means that a shell will not be spawned to run the command, and no shell expansion will take place. This is the preferred form, as it means that signals should propagate properly through to spawned processes. However, you can still use the old style string parameter, e.g.

```yaml
exec: "/usr/bin/supervisord -n"
```

* `data_sources` : The data source plugins you'll be using to populate the configuration files. This should usually just be set to "file" to start with, although you can write your own plugins and pull them in (more on that later).
* `template_sources` Where the templates come from, again a list of plugins.
* `default_environment` : Sets the default environment file to load if none is specified (either using the -e flag, or via the `environment` environment variable). This defaults to 'development', but you may want to set this to 'production' to mimic the old, pre-0.4.0 behaviour.

So for a simple use-case where you're just generating everything from files and then spawning MongoDB, you'd have a common.yaml with this at the top:
```yaml
exec: [ "/usr/bin/mongod" , "--config" , "/etc/mongodb.conf" , "--rest" ]
data_sources: [ "file" ]
template_sources: [ "file" ]
```

## Template files

When using the `FileTemplateSource` ("file") plugin, these files under `/etc/tiller/templates` are simply the ERB templates for your configuration files, and are populated with values from the selected environment configuration blocks (see below). When the environment configuration is parsed (see below), key:value pairs are made available to the template.

**IMPORTANT: **These files must be named with a suffix of `.erb`. Any files without an ending of `.erb` will be ignored.

Here's a practical example, again using MongoDB. Let's assume that you're setting up a "MongoDB" container for your platform to use, and you want to have it configured so it can run in 3 environments:

* A local "development" environment (e.g. your own laptop), where you don't want to use it in a replica set.
* "staging" and "production" environments, both of which are setup to be in a replica set, named after the environment.

MongoDB needs to have the replica set name specified in the configuration file when it's launched. You'd therefore create a template `templates/mongodb.erb` template with some placeholder values:

```erb
... (rest of content snipped) ...
# in replica set configuration, specify the name of the replica set
<% if (replSet) %>
replSet = <%= replSet %>
<% end %>
... (rest of content snipped) ...
```

Now it will only contain the `replSet = (whatever)` line when there is a variable "`replSet`" defined. How that gets defined is (usually) the job of the environment configuration blocks - these are covered next.

## Environment configuration

These headings in `common.yaml` (underneath the `environments:` key) are named after the environment variable `environment` that you pass in (usually by using `docker run -e environment=<whatever>`, which sets the environment variable). Alternatively, you can set the environment by using the `tiller -e` flag from the command line.

When you're using the default `FileDataSource`, these environment blocks in `common.yaml` define the templates to be parsed, where the generated configuration file should be installed, ownership and permission information, and also a set of key:value pairs (the "template values") that are made available to the template via the usual `<%= key %>` ERB syntax.

Carrying on with the MongoDB example, here's how you might set the replica set name in your staging and production environments (add the following to `common.yaml`):

```yaml
environments:

staging:

mongodb.erb:
target: /etc/mongodb.conf
user: root
group: root
perms: 0644
config:
replSet: 'staging'

production:
mongodb.erb:
target: /etc/mongodb.conf
config:
replSet: 'production'
```

Note that if you omit the user/group/perms parameters - as shown above for the production environment - the defaults are whatever Docker runs as (usually root). Also, if you don't run Tiller as root, it will skip setting these.

The development environment definition can be even simpler, as we don't actually define a replica set, so we can skip the whole `config` block :

```yaml
development:

mongodb.erb:
target: /etc/mongodb.conf
```

So now, when run through Tiller/Docker with `-e environment=staging`, the template will be installed to /etc/mongodb.conf with the following content :

# in replica set configuration, specify the name of the replica set
replSet = staging

Or, if the production environment is specified :

# in replica set configuration, specify the name of the replica set
replSet = production

And if the `development` environment is used (it's the default, so will also get used if no environment is specified), then the config file will get installed but with the line relating to replica set name left out.

Of course, this means you need an environment block for each replica set you plan on deploying. If you have many Mongo clusters you wish to deploy, you'll probably want to specify the replica set name dynamically, perhaps at the time you launch the container. You can do this in many different ways, for example by using the `environment` plugin to populate values from environment variables (`docker run -e repl_set_name=foo ...`) and so on. These plugins are covered in their [own documentation](#plugins).


### Overriding common settings
As of Tiller 0.5.0, you can also override defaults from common.yaml if you specify them in a `common` block in an environment section. This means you can specify a different `exec`, enable the API, or configure various plugins to use different settings on a per-environment basis, e.g.

```yaml
environments:

development:
# Only enable API for development environment, and
# also specify HTTP plugin values
common:
api_enable: true
api_port: 1234
# configuration for HTTP plugin (https://github.com/markround/tiller/blob/master/README-HTTP.md)
http:
uri: 'http://tiller.dev.example.com'
...
...
... rest of config file snipped
...
...
```


93 changes: 93 additions & 0 deletions docs/general/configuration.md
View file
@@ -0,0 +1,93 @@
# Main configuration values

`common.yaml` contains most of the configuration for Tiller. It contains top-level `exec`, `data_sources`, `template_sources` and `default_environment` parameters, along with sections for each environment.

It can also take optional blocks of configuration for some plugins (for example, the [Consul Plugins](../plugins/consul.md)). Settings defined here can also be overridden on a per-environment basis.

## exec
This is simply what will be executed after the configuration files have been generated. If you omit this (or use the `-n` / `--no-exec` arguments) then no child process will be executed. You should specify the command and arguments as an array, e.g.

```yaml
exec: [ "/usr/bin/supervisord" , "-n" ]
```

This means that a shell will not be spawned to run the command, and no shell expansion will take place. This is the preferred form, as it means that signals should propagate properly through to spawned processes. However, you can still use the old style string parameter, e.g.

```yaml
exec: "/usr/bin/supervisord -n"
```

## data_sources

This parameter specifies the data source plugins you'll be using to populate the configuration files. This should usually just be set to "file" to start with, although there are a lot of [bundled plugins](../plugins/index.md) provided. You can also [write your own plugins](../developers.md) and pull them in. These plugins are provided as a YAML array, for example:

```yaml
data_sources: [ "file" , "consul" , "environment"]
```

Or in the long-form:

```yaml
data_sources:
- file
- consul
- environment
```


## template_sources
This parameter specifies which plugin will be providing the templates. It uses the same array syntax as `data_sources`:

```yaml
template_sources: [ "file" ]
```


## default_environment

This parameter sets the default environment to use if one is not specified (either using the -e flag, or via the `environment` environment variable). This defaults to 'development'.

```yaml
default_environment: testing
```

# Example
For a simple use-case where you're just generating everything from files and then spawning MongoDB with a different replica set name specified in the staging and production environments, you'd have a common.yaml like this:

```yaml
---
exec: [ "/usr/bin/mongod" , "--config" , "/etc/mongodb.conf" , "--rest" ]
data_sources: [ 'file' ]
template_sources: [ 'file' ]
environments:

staging:
mongodb.erb:
target: /etc/mongodb.conf
user: root
group: root
perms: 0644
config:
replSet: 'staging'

production:
mongodb.erb:
target: /etc/mongodb.conf
config:
replSet: 'production'

development:
mongodb.erb:
target: /etc/mongodb.conf
```

And a mongodb.erb template that contained:

```erb
...
... rest of file snipped
...
<% if (replSet) -%>
replSet = <%= replSet %>
<% end -%>
```
13 changes: 9 additions & 4 deletions docs/general/docker.md
View file
@@ -1,10 +1,11 @@
# Setup with Docker

This section will show how you can bundle configuration and templates inside a container so you can switch between them at run-time, using just the "file" plugin.
This section will show how you can bundle Tiller inside a container so you can generate your configuration files at run-time, and then run your desired command.

All of the following assumes you're using Tiller with Docker. So, firstly install the Tiller gem and set your Dockerfile to use it (assuming you're pulling in a suitable version of Ruby already) :
Firstly install the Tiller gem and set your Dockerfile to use it. This assumes you're already pulling in a suitable version of Ruby, if not you may want to start by using `FROM ruby` :

```dockerfile
FROM ruby
RUN gem install tiller
...
... Rest of Dockerfile here
Expand All @@ -27,14 +28,16 @@ Tiller expects a directory structure like this (using /etc/tiller as its base, a
├── common.yaml
│
└── templates
├── sensu_client.erb
├── mongodb.erb
├── application.erb
├── db.erb
...
... other configuration file templates go here
...

It is suggested that you add all this under your Docker definition in a `data/tiller` base directory (e.g. data/tiller/common.yaml, data/tiller/templates and so on...) and then add it in your Dockerfile. This would therefore now look like:

```dockerfile
FROM ruby
RUN gem install tiller
...
... Rest of Dockerfile here
Expand All @@ -45,5 +48,7 @@ CMD ["/usr/local/bin/tiller" , "-v"]

Note that the configuration directory was added later on in the Dockerfile; this is because `ADD` commands cause the Docker build cache to become invalidated so it's a good idea to put them as far as possible towards the end of the Dockerfile.

Now, when you run the container, Tiller will run, generate your configuration files, and if you have set the `exec: ` parameter in your `common.yaml` will also start your specified program such as your application, database daemon, [supervisord](http://supervisord/org) etc.

# Other resources
A simple tutorial which produces a 'parameterized' NginX container with Tiller is on my blog : [http://www.markround.com/blog/2014/09/18/tiller-and-docker-environment-variables/](http://www.markround.com/blog/2014/09/18/tiller-and-docker-environment-variables/). It also provides a downloadable archive of the files used in the example, so if you want to get up and running very quickly before diving into the rest of the documentation, then this may also be a good place to start.
44 changes: 19 additions & 25 deletions docs/general/index.md
View file
@@ -1,29 +1,26 @@


# Usage
Tiller can be used to dynamically generate configuration files before passing execution over to a daemon process.

It looks at an environment variable called "environment" (or the argument to the `-e` flag), and creates a set of configuration files based on templates, and then optionally runs a specified daemon process via `exec`.
Tiller can be used to dynamically generate configuration files based on ERb templates before passing execution over to a daemon process or other command. It is usually used inside a Docker container as the `CMD` or `ENTRYPOINT` process.

In a basic use-case, when you are bundling your templates and configuration inside the container, all you need to do is tell Tiller which environment you want to use e.g.

# docker run -t -i -e environment=staging markround/demo_container:latest
tiller v0.3.1 (https://github.com/markround/tiller) <github@markround.com>
Using configuration from /etc/tiller
Using plugins from /usr/local/lib/tiller
Using environment staging
Template sources loaded [FileTemplateSource]
Data sources loaded [FileDataSource, NetworkDataSource]
Templates to build ["mongodb.erb", "sensu_client.erb"]
Building template mongodb.erb
Setting ownership/permissions on /etc/mongodb.conf
Building template sensu_client.erb
Setting ownership/permissions on /etc/sensu/conf.d/client.json
Template generation completed
Executing /usr/bin/supervisord
Child process forked.

If no environment is specified, it will default to using "development". Prior to version 0.4.0, this used to be "production", but as was quite rightly pointed out, this is a bit scary. You can always change the default anyway - see below.
```
$ docker run -ti -e environment=staging my-application:latest
tiller v1.0.0 (https://github.com/markround/tiller) <github@markround.com>
Using common.yaml v2 format configuration file
Using configuration from /etc/tiller
Using plugins from /usr/local/lib/tiller
Using staging development
Template sources loaded [FileTemplateSource]
Data sources loaded [FileDataSource]
Available templates : ["application.erb"]
Building template application.erb
Not running as root, so not setting ownership/permissions on /etc/application.ini
Template generation completed
Executing ["/usr/sbin/my_application"]...
Child process forked with PID 87560
```

If no environment is specified, it will default to using "development".

# A word about configuration
Tiller uses YAML for configuration files. If you're unfamiliar with YAML, don't worry - it's very easy to pick up. A good introduction is here : ["Complete idiot's introduction to YAML"](https://github.com/Animosity/CraftIRC/wiki/Complete-idiot's-introduction-to-yaml)
Expand All @@ -50,8 +47,5 @@ Tiller understands the following *optional* command-line arguments (mostly used
* `--md5sum` : Only write templates if they do not already exist, or their content has changed (see [below](#checksums)).
* `--md5sum-noexec` : If no templates were written/updated, do not execute any process.

# Where to go from here




8 changes: 4 additions & 4 deletions docs/plugins/consul.md
View file
Expand Up @@ -37,10 +37,10 @@ consul:

templates: '/tiller/templates'
values:
global: '/tiller/globals/all'
per_env: '/tiller/globals/%e'
template: '/tiller/values/%e/%t'
target: '/tiller/target_values/%t/%e'
global: '/tiller/globals/all'
per_env: '/tiller/globals/%e'
template: '/tiller/values/%e/%t'
target: '/tiller/target_values/%t/%e'
```

At a bare minimum, you need to specify a URL for the plugins to connect to. This is the HTTP port of your Consul server, e.g. `http://localhost:8500`. If you omit the other parameters, they will default to the values shown above. If you're happy to accept the rest of the defaults, your configuration can therefore be as simple as this :
Expand Down

0 comments on commit f7e74d4

Please sign in to comment.