Skip to content
Envoy Control Plane for Kubernetes Multi-cluster Ingress
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
cmd Envoy websockets (#25) Nov 21, 2018
img expand readme Jul 3, 2018
pkg Should check equality for all timeouts as well Dec 10, 2018
vendor Envoy websockets (#25) Nov 21, 2018
.drone.yml
.gitignore Enable tracing on listeners Aug 6, 2018
Dockerfile Enable tracing on listeners Aug 6, 2018
Gopkg.lock
Gopkg.toml
LICENSE create license Jun 20, 2018
Makefile
README.md Envoy websockets (#25) Nov 21, 2018
main.go change from kingpin to cobra/viper (#6) Jun 21, 2018

README.md

Yggdrasil

Yggdrasil is an Envoy control plane that configures listeners and clusters based off Kubernetes ingresses from multiple Kube Clusters. This allows you to have an envoy cluster acting as a mutli-cluster loadbalancer for Kubernetes. This was something we needed as we wanted our apps to be highly available in the event of a cluster outage but did not want the solution to live inside of Kubernetes itself.

Note: Currently we support version 1.8.0 of Envoy.

Usage

Yggdrasil will watch all Ingresses in each Kubernetes Cluster that you give it via the Kubeconfig flag. Any ingresses that match any of the ingress classes that you have specified will have a listener and cluster created that listens on the same Host as the Host defined in the Ingress object. If you have multiple clusters Yggdrasil will create a cluster address for each Kubernetes cluster your Ingress is in, the address is the address of the ingress loadbalancer.

Setup

The basic setup is to have a cluster of envoy nodes which connect to Yggdrasil via GRPC and get given dynamic listeners and clusters from it. Yggdrasil is set up to talk to each Kubernetes api where it will watch the ingresses for any that are using the ingress class it's watching for.

Yggdrasil Diagram

Your envoy nodes only need a very minimal config where they are simply set up to get dynamic clusters and listeners from Yggdrasil. Example envoy config:

admin:
  access_log_path: /tmp/admin_access.log
  address:
    socket_address: { address: 0.0.0.0, port_value: 9901 }

dynamic_resources:
  lds_config:
    api_config_source:
      api_type: GRPC
      grpc_services:
        envoy_grpc:
          cluster_name: xds_cluster
  cds_config:
    api_config_source:
      api_type: GRPC
      grpc_services:
        envoy_grpc:
          cluster_name: xds_cluster

static_resources:
  clusters:
  - name: xds_cluster
    connect_timeout: 0.25s
    type: STATIC
    lb_policy: ROUND_ROBIN
    http2_protocol_options: {}
    hosts: [{ socket_address: { address: yggdrasil, port_value: 8080 }}]

Your ingress set up then looks like this:

Envoy Diagram

Where the envoy nodes are loadbalancing between each cluster for a given ingress.

Configuration

Yggdrasil can be configured using a config file e.g:

{
  "nodeName": "foo",
  "ingressClasses": ["multi-cluster", "multi-cluster-staging"],
  "cert": "path/to/cert",
  "key": "path/to/key",
  "clusters": [
    {
      "token": "xxxxxxxxxxxxxxxx",
      "apiServer": "https://cluster1.api.com",
      "ca": "pathto/cluster1/ca"
    },
    {
      "tokenPath": "/path/to/a/token",
      "apiServer": "https://cluster2.api.com",
      "ca": "pathto/cluster2/ca"
    }
  ]
}

nodeName is the same node-name that you start your envoy nodes with. The cert and key are used to talk over https with the ingress loadbalancers. The ingressClasses is a list of ingress classes that yggdrasil will watch for. Each cluster represents a different Kubernetes cluster with the token being a service account token for that cluster. ca is the Path to the ca certificate for that cluster.

Flags

--cert string                   certfile
--config string                 config file
--debug                         Log at debug level
--help                          help for yggdrasil
--ingress-classes stringArray   Ingress classes to watch
--key string                    keyfile
--kube-config stringArray       Path to kube config
--node-name string              envoy node name
You can’t perform that action at this time.