Opscode Cookbook that Installs and configures Mercury Global Loadbalancer.
Mercury is a Global loadbalancer, designed to add a dns based loadbalancing layer on top of its internal loadbalancer or 3rd pary loadbalancers such as cloud services This makes mercury able to loadbalance across multiple cloud environments using dns, while keeping existing cloud loadbancer sollutions in place
- Source: https://github.com/schubergphilis/mercury
- Binaries: https://github.com/schubergphilis/mercury/releases
- Docs: http://mercury-global-loadbalancer.readthedocs.io/en/latest/
- Chef 12.5+
- RHEL 6+, CentOS6+
- RHEL 7+, CentOS7+
- RHEL 8+, CentOS8+
Include the default recipe
default['mercury']['settings'] = {
manage_network_interfaces: "(yes|no)" -- allow mercury to add vip's to the network interfaces (default: yes) - required for internal proxy or for haproxy who does not add vip's
enable_proxy: "(yes|no)" -- use internal proxy for loadbalancing (default: yes) - not needed for external proxy programs, or dns only setup
}
default['mercury']['logging'] = {
level: "(debug|info|warn|error)" -- log level
output: "(stdout|file)" -- log output
}
default['mercury']['cluster'] = {
name: "" -- cluster group name
binding: "myhost" -- ip/interface to bind on, also acts as cluster name
settings: {
connection_timeout: 10, -- connection timeout for cluster nodes
connection_retry_count: 10, -- connection retry for cluster nodes
connection_retry_interval: 10, -- connection retry interval for cluster nodes
ping_interval: 10, -- ping interval for cluster nodes
ping_timeout: 10, -- time to wait for reply before discarding cluster node
port: 9000, -- port for cluster communications
tls: { -- see tls settings below for more details
insecureskipverify: true
}
},
nodes: [
{
search: "recipe:chef_recipe", -- let chef find the remote nodes based on search
port: 80 -- connect to port 80 of this node
},
]
}
default['mercury']['dns'] = {
binding: 'myhost' -- ip to bind on
port: 53 -- port to listen on for dns queries
allowed_requests: [ "A", "AAAA" ] -- what records to respond to (default is to allow most requests)
}
default['mercury']['web'] = {
binding: 'myhost' -- ip to bind on
port: 9001 -- port to listen to for web interface
tls: { -- see tls settings below
}
auth: {
password: {
"username": "sha256hashOfPassword"
}
ldap: {
host: 'ldaphost' -- ldap host to connect to
port: 389 -- ldap port to connect to, 389 for tls, 636 for ssl
method: 'tls' -- method of connection, tls or ssl
binddn: "OU=Users,DC=example,DC=com" -- search path to find users
filter: "(&(objectClass=organizationalPerson)(uid=%s))" -- filter path to find user
domain: 'example' -- domain to prepend to ldap login
}
}
}
default['mercury']['loadbalancer']['pools'] = {
'poolname' => { -- name of the pool/vip
listener: {
ip: "" -- ip of the vip
port: 0 -- port of the vip
mode: "(udp|tcp|http|https)" -- what protocol to listen to
tls: { -- see mercury src/tlsconfig/tls.go for all available options, not supplying this will use defaults
minversion: 'VersionTLS12', -- minimum version of TLS to allow
ciphersuites: %w(TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384), -- ciphersuite to allow
curvepreferences: %w(CurveP521) -- curve preference to use
}
readtimeout: 0 -- time to wait for http request from client (0 = unlimited, 10 = default)
writetimeout: 0 -- time to wait for server reply towards client (0 = unlimited, 0 = default)
ocspstapling: true|false -- enables ocsp stapling (true = default)
},
outboundacls: [
{
action: '(add|remove|replace)', -- what to do with header
header_key: 'Location', -- header key to do action with
header_value: 'https://###REQ_HOST######REQ_PATH###' -- header value to set - not required on remove
},
{
action: '(add|remove|replace)', -- what to do with cookie
cookie_key: 'stky', -- cookie key to do action with
cookie_value: '###NODE_ID###', -- calculated variable or string
cookie_expire: '24h', -- time for cookie (ex. 1m, 3s, 24h)
cookie_secure: true, -- secure
cookie_httponly: true -- httponly
},
],
inboundacls: [
-- see outboundacls, but actions can also include allow/deny on inbound acls
{
action: '(allow|deny)', -- what to do with cidr
cidrs: ["127.0.0.1/8", "10.0.0.0/8"] -- network cidrs to apply action to
},
{
action: '(add|remove|replace)', -- what to do with header
header_key: 'Location', -- header key to do action with
header_value: 'https://###REQ_HOST######REQ_PATH###' -- header value to set - not required on remove
},
{
action: '(allow|deny)', -- what to do with header (regex are allowed to allow/deny rules)
header_key: 'User-Agent', -- header key to do action with
header_value: ".*Macintash.*" -- value regex match for allow/denies are allowed
},
],
errorpage: {
file: "/var/mercury/sorry.html" -- specifying a sorry page, enables showing this on errors
trigger_threshold: 500 -- http result code when to show sorry page (e.g. 500 will show the sorrypage on all errors of 500+)
},
healthchecks: [{ // >= mercury version 0.10 - allows an array of healthchecks on a Pool
type: '(httpget|httppost|tcpconnect|icmpping|udpping|tcpping)',
ip: '1.2.3.4',
pingpackets: 4,
pingtimeout: 1
}],
backends: {
'backendname' => {
inboundacls: [] -- see earlier inbound acls
outboundacls: [] -- see earlier inbound acls
hostnames: [""] -- hostnames to repply on with this backend
dnsentry: {
hostname: "www" -- hostname to give dns record for
domain: "domain.org" -- domain where this host subsides
ip: "" -- IPv4 of the vip to supply on dns record (will use listener ip above if this is not specified
ip6: "" -- IPv6 of the vip to supply on dns record (will use listener ip above if this is not specified
},
healthcheck: { // <= mercury version 0.9.x - allows only a single healthcheck
type: '(httpget|httppost|tcpconnect)',
httpstatus: 200,
httpheaders: ['Content-Type: application/soap+xml; charset=utf-8'],
postdata: '<your post body>',
request: "http://www.domain.org/",
tls: {
insecureskipverify: true
}
},
healthcheckmode: "(any|all)" -- default: "all" - should 1 or all checks be ok for backend to be online
healthchecks: [{ // >= mercury version 0.10 - allows an array of healthchecks
type: '(httpget|httppost|tcpconnect)',
httpstatus: 200,
httpheaders: ['Content-Type: application/soap+xml; charset=utf-8'],
postdata: '<your post body>',
request: "http://www.domain.org/",
tls: {
insecureskipverify: true
}
}],
balance: {
method: 'topology,leastconnected' -- see loadbalancing methods for more details
local_topology: "site" -- see networks below
preference: 0123 -- priority for preference based loadbalancing (lower has higher preference)
active_passive: "(yes|no)" -- default: no - affects monitoring: to only alert if 0 or >1 nodes are online
clusternodes: int -- default: #cluster_nodes - affects monitoring: usefull for vips that only live on 1 cluster node
}
nodes: [
{
search: "recipe:chef_recipe", -- let chef find the remote nodes based on search
port: 80 -- connect to port 80 of this node
},
{
hostname: 'hostname' -- specify host (optional)
ip: '1.2.3.4' -- specify ip (required)
port: 80 -- connect to port 80 of this node
}
],
connectmode: "(udp|tcp|http|https)", -- what protocol to connect to the backend with
}
}
}
}
default['mercury']['loadbalancer']['networks'] = { -- used for topology loadbalancing
'site' => {
cidrs = [ "127.0.0.1/32" ]
}
}
default['mercury']['dns']['domains']['domain.org'] = { -- local dns records, also uses TTL as default for loadbalanced records
'ttl' => 10, -- default ttl for all records
'soa' => { -- standard soa record
name: 'mydomain.org',
ns: "ns.mydomain.org",
email: "hostmaster.mydomain.org",
refresh: 30, # time that a slave will refresh from master
retry: 30, # retry time if slave failed to connect to master
expire: 3600, # cache on slave
minimum: 10, # minimum cache time for slave
},
'records' => [ -- records that are not loadbalanced but local
{
name: "" -- for domain records no name needed
target: "10 mymx.domain.org" -- mx server to send to
type: "MX" -- MX record type
},
{
name: "www2" -- hostname in domain record
target: "::1" -- target
type: "AAAA" -- ipv6 record
}
]
}
default['mercury']['dns']['allow_forwarding'] = [ "10.10.0.177/32", "::1/128" ] -- allows recursive dns queries for given clients (used when setting mercury as dns forwarding server)
-
leastconnected - based on current clients connected
-
leasttraffic -- based on traffic generated
-
preference -- based on preference set in node of backend
-
responsetime -- based on responsetime of a backend node (first byte response? or http roundtrip?)
-
random -- rng
-
roundrobin -- try to switch them a bit
-
sticky -- based on the 'stky' cooky which contains the node, requires the following 2 acls:
{ action: 'replace', cookie_key: 'stky', cookie_value: '###NODE_ID###', cookie_expire: '24h', cookie_secure: true, cookie_httponly: true, cookie_path: '/' }, { action: 'add', cookie_key: 'stky', cookie_value: '###NODE_ID###', cookie_expire: '24h', cookie_secure: true, cookie_httponly: true, cookie_path: '/' }, ```
note that for stickyness in combination with global loadbalancing, you need to ```add each node to all loadbalancers```. so that the lb can always forward the client to its previous node
-
topology -- based on topology network set, requires the network to be set
-
firstavailable -- used for compatibility reasons, this ensures that we always return only the first record, and prevents multiple hosts in dns responses.
When selecting multiple loadbalance options, keep in mind the speed of each of them. See below a benchmark of the speeds.
$ go test -bench=.
BenchmarkBalancerLeastConnected-4 10000000 160 ns/op
BenchmarkBalancerLeastTraffic-4 10000000 167 ns/op
BenchmarkBalancerPreference-4 10000000 163 ns/op
BenchmarkBalancerRandom-4 200000 9643 ns/op
BenchmarkBalancerRoundRobin-4 5000000 165 ns/op
BenchmarkBalancerSticky-4 10000000 124 ns/op
BenchmarkBalancerTopology-4 2000000 837 ns/op
BenchmarkBalancerResponseTime-4 10000000 136 ns/op
TLS settings should be set on the Mercury Cluster and Webservices. But they can be set on Listeners and Backend. Setting TLS settings only on backends and not on a listener means that SNI will be used to identify a site for the given SSL certificate.
tls: { minversion: minimal tls version to allow ciphersuites: cipersuites to allo curvepreferences: curve preference to use insecureskipverify: allow a client to connect to insecure SSL certificate (hostname mismatches)
certificatefile: certificate crt file to use - this should be a path to a file
certificatekey: certificate key file to use - this should be a path to a file
databagname: name of the databag to find the SSL certificate
databagitem: item name of the databag to find the SSL certificate
}
The minimum TLS version to allow.
Options:
- VersionSSL30,
- VersionTLS10,
- VersionTLS11,
- VersionTLS12,
The allowed Cypers, see https://golang.org/pkg/crypto/tls/#pkg-constants for details
Options:
- TLS_RSA_WITH_RC4_128_SHA
- TLS_RSA_WITH_3DES_EDE_CBC_SHA
- TLS_RSA_WITH_AES_128_CBC_SHA
- TLS_RSA_WITH_AES_256_CBC_SHA
- TLS_RSA_WITH_AES_128_CBC_SHA256
- TLS_RSA_WITH_AES_128_GCM_SHA256
- TLS_RSA_WITH_AES_256_GCM_SHA384
- TLS_ECDHE_ECDSA_WITH_RC4_128_SHA
- TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
- TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
- TLS_ECDHE_RSA_WITH_RC4_128_SHA
- TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA
- TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
- TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
- TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
- TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
- TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
- TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
- TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
- TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
- TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
- TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
- TLS_FALLBACK_SCSV
- TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 <- has to be first if you want HTTP/2 support!
The 4 cipers below are need for the best SSL-Labs certificate but do not support HTTP/2, the HTTP/2 one will slightly downgrade your score
- TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
- TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
- TLS_RSA_WITH_AES_256_GCM_SHA384
- TLS_RSA_WITH_AES_256_CBC_SHA
The tls curve preferences, see https://golang.org/pkg/crypto/tls/#pkg-constants for details
Options:
- CurveP256
- CurveP384
- CurveP521
- X25519
You can use databags to load the certificate, or let them be generated.
If you supply only the file names on where to find the certificates, the cookbook will generate these certificates and used them in Mercury Example:
tls: {
certificatefile: "my_certificate.crt"
certificatekey: "my_certificate.key"
}
This wil generate the certificates in the OS default's SSL certificate path, and genrate certificates with the name my_certificate.crt + my_certificate.key
If you supply a databagname, the cookbook will search the databag for the keys provided as file name in the databagitem Example:
tls: {
databagname: 'mercury'
databagitem: 'certifactes_production'
certificatefile: "my_certificate_file"
certificatekey: "my_certificate_key"
}
This will get the certificate from the data bag called 'mercury' with the item 'certificates_production', and search for the keys 'my_certificate_file' and 'my_certificate_key' The data bag will look something like this:
$ knife data bag show mercury certificates_aroductionp --secret-file secrey_key -Fj
{
"id": "certificates_production",
"my_certificate_file": "-----BEGIN CERTIFICATE-----\nMII...\n..aNZ\n-----END CERTIFICATE-----"
}
Below You can find some example configurations
Simple site, that serves a website using SSL offloading - we allow SSL connects, and use HTTP connects to the backend
default['mercury']['loadbalancer']['pools']['my_backend_https'] = {
listener: { ip: 'myapp_vip', port: 443, mode: 'https' },
backends: {
'myapp_backend' => {
hostnames: ["myapp.mydomain.org"],
dnsentry: { hostname: 'myapp', domain: 'mydomain.org' },
healthcheck: { type: 'httpget', httpstatus: 200, request: "http://myapp.mydomain.org/" },
balance: { method: 'sticky,roundrobin' },
nodes: [{ search: "recipe:myapp.mydomain.org AND chef_environment:#{node.chef_environment}", port: 80 }],
connectmode: 'http'
}
}
}
This will accept connections on 'myapp_vip', which will do a chef search for the nodes serving this application
all requests on this ip on port 80 are redirected to their original domain and path to https://
default['mercury']['loadbalancer']['pools']['http_redirect'] = {
listener: { ip: matrix_ip['backend'], port: 80, mode: 'http' },
outboundacls: [
{ action: 'add', header_key: 'Location', header_value: 'https://###REQ_HOST######REQ_PATH###' },
{ action: 'add', status_code: 301 }
],
backends: {
'redirect' => {
hostnames: ['default'],
connectmode: 'internal'
}
}
}
only requests to myapp.mydomain.org are served, all other requests are redirected to https://myapp.mydomain.org also we only allow private networks to connect to this site
default['mercury']['loadbalancer']['pools']['my_backend_https'] = {
listener: { ip: 'myapp_vip', port: 443, mode: 'https' },
backends: {
'myapp_backend' => {
hostnames: ["myapp.mydomain.org"],
dnsentry: { hostname: 'myapp', domain: 'mydomain.org' },
healthcheck: { type: 'httpget', httpstatus: 200, request: "http://myapp.mydomain.org/" },
balance: { method: 'sticky,roundrobin' },
nodes: [{ search: "recipe:myapp.mydomain.org AND chef_environment:#{node.chef_environment}", port: 80 }],
connectmode: 'http'
},
'redirect' => {
outboundacls: [
{ action: 'add', header_key: 'Location', header_value: 'https://myapp.mydomain.org' },
{ action: 'add', status_code: 301 }
],
inboundacls: [
{ action: 'allow', cidrs: ["127.0.0.1/8", "10.0.0.0/8"] }
],
hostnames: ['default'],
connectmode: 'internal'
}
}
}
all requests to this ip/port serve this website
default['mercury']['loadbalancer']['pools']['my_backend_https'] = {
listener: { ip: 'myapp_vip', port: 443, mode: 'https' },
backends: {
'myapp_backend' => {
hostnames: ["default"],
dnsentry: { hostname: 'myapp', domain: 'mydomain.org' },
healthcheck: { type: 'httpget', httpstatus: 200, request: "http://myapp.mydomain.org/" },
balance: { method: 'sticky,roundrobin' },
nodes: [{ search: "recipe:myapp.mydomain.org AND chef_environment:#{node.chef_environment}", port: 80 }],
connectmode: 'http'
}
}
}
A reload should be enough to apply config changes. but some changes will affect clients.
Only when changing the Listener variables of a config, or the TLS config of backends, requires the proxy part of the laodbalancer for this specific Pool to be reloaded. As such, when changing the IP, port, Connection mode or TLS config, clients currently connected to this pool will be disconnected, and can reconnect once the pool is up again. (mater of miliseconds)
All other configuration items are changable without interruption to the client.
When a error page is set in the config, it will always show on internal errors (no backend available, or acl allow/deny)
For errors given by a webserver you can use the trigger_threshold which will only trigger errors if the status code is equal or higher.
If you do not want the sorry page to show on return codes from the webserver, then set this to a higher number then the http error codes (e.g. 600 or up)
ACL's can be set to add/replace/modify headers, or to allow/deny requests based on headers/cidr (see examples above)
to use ALLOW/DENY, you must use the INBOUND acl. you cannot mix allow and deny ACL's together, this will result in only the allow beeing processed.
- Author:: Ronald Doorn (rdoorn@schubergphilis.com)
Copyright:: Schuberg Philis
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.