A suite of tools to deal with single source of truth issues with major devops tools
Switch branches/tags
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.circleci
fixtures
go
haskell
src
.gitignore
CONTRIBUTORS
Cargo.toml
Dockerfile
LICENSE
bundle.sh
deploy.sh
readme.md
shell.nix
start-docker.sh

readme.md

Palimpsest

This is a suite of tools to facilitate, as simply as possible, issues such as single source of truth, providing credentials, and standardizing log formats and system statistics measurement. It does not replace primary devops tools, but instead helps bridge the impedance mismatch between Terraform, Vault, Ansible, and any other tools that may come up as requested.

Systemstats

A daemon that regularly logs the current system load, disk capacity, and memory usage for a server.

Palimpsest

Palimpsest does on-demand generation of configuration files based on a template and a configuration vault. The program runs as a daemon and can manage multiple such files at once (I have not yet tested the upper limit).

For instance:

palimpsest --vault yml --path /etc/palimpsest/vault.yml --pipeline filebeat.tmpl:/etc/filebeat/filebeat --pipeline nginx.tmpl:/etc/nginx/sites-enabled/default

This command uses the source of truth (in this case, /etc/palimpsest/vault.yml) to the filebeat and nginx site configuration files, using templates for each. Each pipeline stanza presents a template file and output pipe description. Any time an application reads from the output file, the template and vault are used to generate the configuration.

The only vault format currently support is a local yaml file. Additional formats, including a yaml file stored in a KMS-encrypted S3 bucket, are in my plan. A simple yaml vault file looks like this:

---
logserver_addr: logs.truveon.com
webserver_ami: "ami-162bc276"

A template file is a yaml file containing a section for the identifiers used in the template, and the content of a mustache-formatted template, like so:

---
identifiers:
    - logserver_addr

template: |
    filebeat.prospectors:
        - type: log
          paths:
              - /var/log/syslog
    output.logstash:
        hosts: ["{{ logserver_addr }}:5043"]

The identifiers parameter in the file determines all of the identifiers that will be looked up in the vault and thus filled in to the template. Behavior for including in the template a variable that is not in the identifiers list is currently undefined, as is the behavior of requiring a list in the template.