Golem is a deployment tool that lets you upload artifacts and execute
commands in parallel on local or multiple servers via SSH using human readable Hashicorp HCL recipes
Getting Started • Adding servers • Adding Recipes • SSH and SFTP • Suggest a feature?
Golem is a deployment tool built with go that runs over SSH, manages configs and recipes with Hashicorp HCL files, and lets you upload artifacts and execute commands in parallel on local or multiple servers.
If like me, you have developed a distinct hatred for YAML based configs like the ones you need to use for Ansible, Kubernetes and the like, you're going to enjoy building the recipes in a very human readable Hashicorp HCL based format. Just like you would for Nomad, Terraform or Consul
The core idea behing Golem was to build something like a Terraform Provisioner (they are the last resort) with remote-exec that you can use on any machine that you can access via SSH.
The best way to get Golem running on your machine today is to install golang on your device and then run go install
$ brew install go
$ go install github.com/sudhanshuraheja/golem@latestGolem expects a configuration file at ~/.golem/config.golem.hcl or at ./config.golem.hcl. You can also set it up at both places. Golem can read and merge multiple *.golem.hcl files in ~/.golem/ and ./. You can set it up by running
$ golem
init | conf file created at /Users/your-username/.golem/config.golem.hcl
Here are the recipes that you can use with '$ golem recipe-name'
Name Match Artifacts Commands
servers local only 0 0
You can add more recipes to '~/.golem/config.golem.hcl'You can make editing the config easier by adding a shortcut to your .zshrc or .bashrc to open the config file in VSCode
$ echo 'alias glm="code /Users/sudhanshuraheja/.golem/config.golem.hcl"' >> ~/.zshrc
$ source ~/.zshrc
$ glmThe first step is to add servers to your config, so that you can take actions on them
server "thebatch" {
hostname = "thebatch.local"
public_ip = "173.168.86.17"
private_ip = "192.168.1.55"
user = "sudhanshu"
port = 22
tags = ["redis", "vpc-private"]
}Before connecting via SSH, Golem will check if the public_ip exists. If it doesn't, it will connect to the hostname.
You can also automatically connect all your servers from Terraform to Golem.
server_provider "terraform" {
config = [
"full-path-to/terraform.tfstate",
"another-full-path-to/terraform.tfstate"
]
user = "root"
port = 22
}Golem only looks for Terraform resources of type digitalocean_droplet to add to the server list. You can include any number of tfstate files.
To view all connected Golem servers, you can run the servers recipe
$ golem servers
Name Public IP Private IP User Port Tags Hostname
thebatch 192.168.86.173 sudhanshu 22 local thebatch.local
postgres 128.199.226.65 10.104.16.8 root 22 postgres, vpc-private
...Here's a sample recipe that uploads a file to the remote server and checks if it exists
recipe "test-exec" "remote" {
match {
attribute = "name"
operator = "like"
value = "skye-c"
}
artifact {
source = "LICENSE"
destination = "LICENSE"
}
commands = [
"ls -la L*"
]
}Here's one that runs commands locally
recipe "ls-la" "local" {
commands = [
"ls -la",
"nomad version",
"consul version",
]
}
You can match servers using the attribute, operator and value fields.
Int-based attributes can use =, !=, >, >=, <, <= operators
String-based attributes can use =, !=, like operators
Array-based attributes can use contains, not-contains operators
Artifacts upload files from a local source to a remote destination. You can use both relative and absolute paths. You can add multiple artifact blocks in the recipe.
Here's an example of updating the nomad config and restarting servers
recipe "nomad-server-config-update" "remote" {
match {
attribute = "tags"
operator = "contains"
value = "nomad-server"
}
artifact {
source = "configs/nomad_server.hcl"
destination = "/etc/nomad.d/nomad.hcl"
}
artifact {
source = "certs/nomad-ca.pem"
destination = "/etc/nomad.d/nomad-ca.pem"
}
artifact {
source = "certs/server.pem"
destination = "/etc/nomad.d/server.pem"
}
artifact {
source = "certs/server-key.pem"
destination = "/etc/nomad.d/server-key.pem"
}
commands = [
"chown nomad:nomad /etc/nomad.d/server-key.pem",
"systemctl daemon-reload",
"systemctl stop nomad",
"systemctl start nomad",
]
}Golem runs go-templates on all Command commands = [] and CustomCommands command {}. It exposes two structs for you to use in your templates - Servers and Vars.
Vars can be added at the top level of any .golem.hcl file. If you place them in multiple files, they are merged when start golem.
vars = {
APP = "golem"
REPO = "sudhanshuraheja"
ENV_PREFIX = "GOLEM_"
}Any Vars can be accessed in the commands as {{ .Vars.APP }} or @golem.APP. Vars only support string keys and values.
Servers includes any servers that you have added to golem. This has been covered in more detail earlier.
Golem template introduces two functions for Servers. match which can be used as (match "tags" "contains" "nomad-server"), and matchOne which can be used at (matchOne "tags" "contains" "nomad-server"). matchOne returns the first server in the list, if multiple are matched.
{{ range $_, $s := (match "tags" "contains" "nomad-server") }}
{{ if not ($s).PublicIP }}
{{ else }}
{{ ($s).PublicIP }}
{{ end }}
{{ end }}You can loop through maps like this
{{ range $key, $value := .Vars }}
{{ $key }} = {{ $value }}
{{ end }}You can remove the whitespce by using hyphens {{- and -}}
Here's an example of using it all together
recipe "nomad-ca" "local" {
command {
// server-key.pem -> private key
// server.csr -> certificate signing request
// server.pem -> public key
exec = <<EOF
echo '{}' | cfssl gencert -ca=nomad-ca.pem -ca-key=nomad-ca-key.pem -config=cfssl.json -hostname="server.global.nomad,localhost,127.0.0.1,
{{- range $_, $s := (match "tags" "contains" "nomad-server") -}}
{{- if not ($s).PublicIP -}}
{{- else -}}
{{- ($s).PublicIP -}},
{{- end -}}
{{- if not ($s).PrivateIP -}}
{{- else -}}
{{- ($s).PrivateIP -}},
{{- end -}}
{{- end -}}" - | cfssljson -bare server
EOF
}
}After adding recipes, you can check which recipes exist in Golem's configuration by running the golem list recipe
$ golem list
Name Match Artifacts Commands
apt-update tags not-contains local 0 1
tail-syslog tags contains nomad 0 1
test-exec name like skye-c 1 1
nomad-server-config-update tags contains nomad-server 4 4
nomad-client-config-update tags contains nomad-client 4 6
apply-security-patch name = skye-s3 0 3
...
servers local only 0 0Golem uses one goroutine per server. The goroutine creates an initial SSH connection to the server and uses it to upload artifacts to the server and run each command. It makes a new session for each command. Artifacts are uploaded before running commands.
The number of goroutines is capped to 4 by default and can be changed by setting max_parallel_processes = 16 or any number you like. This is a global setting.
Logging is set to WARN by default. You can change it by setting the config's global loglevel setting.
loglevel = "INFO"When the log level is set to WARN, you will not see the output of the commands being run on the server or the goroutines logs. You will only see an update when a command runs successfully or fails and if the artifact uploads or fails.
- Use HCL for config
- Setup config file at ~/.golem/config.golem.hcl
- Show version
- Merged config files in ~/.golem/.golem and ./.golem
- Migrate config to a separate struct with fewer pointers
- Use ~/.config/ instead of .golem
- Connect to servers via SSH
- Use terraform to get list of servers
- Parse template for match
- Allow custom ssh key to connect to server
- Allow password based login to ssh servers
- Connect to a docker container instead of SSH
- Create a KV store with boltdb
- Allow golem to save secrets
- Setup local and remote environment variables
- Read recipes from config and execute them
- Define variables to be used in recipes
- Create system level recipes
- Allow automatic set up of secrets using KV
- Native recipe to setup docker
- Parse template for KV
- Use namespaces for variables in recipes
- Native recipe to setup nomad without consul
- Native recipe to setup consul
- Native recipe to setup postgres with nomad
- Native recipe to setup redis with nomad
- Download system recipes from the server
- Support go-template for artifacts
- Support go-template for commands
- Support variable replacement for local artifacts
- Support variable replacement for artifact paths
- Support variable replacement for remote artifacts
- Support KV values in artifacts
- Support KV values in commands
- Upload local files via SFTP
- Download http artifacts before uploading
- Support local artifact via go-template
- Run go templates on files before uploaded
- Run local script on all remotes without uploading
- Upload folders to remote
- Show progress while uploading
- Show progress while downloading
- Create a worker pool for ssh connections
- Run separate goroutines for each server
- Limit number of goroutines
- Stream output from commands on remove servers
- Capture SIGINT in worker pool to shutdown connections properly
- Separate local and remote execution steps
- Replace makefile
- Store outputs of commands in a struct for later reuse
- Run go-template before running local exec
- Run go-template before running ssh exec
- Expands commands to include other metadata
- Use output of commands as input to the next command
- Increase goroutines if there are more tail tasks than routines
- Get docker logs from remote
- Use hashicorp go-changelog
- move localutils back into utils