A simple-to-configure front proxy in a docker container, built on Envoy.
- GRPC (HTTP2.0) and WebSockets work side-by-side on the same paths with HTTP1.1
- TLS termination with automatic certificate generation via CertBot/LetsEncrypt
- Redirection (HTTP->HTTPS, domain1->domain2, etc.)
- Authorization (
- Observability integration (statsd)
- Stateless (certificate backup via S3)
- Sharding (different subdomains for different environments)
I have a Kubernetes deployment running in production which needs to route traffic from multiple different domains to different containers. I also run a development version of the same stack on my Linux computer at my house, and I want to use
kustomization to tweak the same deployment files such that
dev subdomains route to this home machine.
Switchboard also solves for the fact that Envoy would normally require additional containers (sidecars?) implement authentication, observability, certificate generation, etc.
Each of the Switchboard variables described below may be configured via YAML by mounting a file at
/etc/switchboard/config.yml, or by providing an enviroment variable of the same name (except upper-case). For example, the
ingress variable may be a string within the YML file, or an enviroment variable
INGRESS (which takes precedence). The two strategies can be layered together, allowing for Kubernetes
kustomization for different environments.
Sample Kubernetes Deployment
This example assumes
kube2iam for AWS authentication in order to achieve the S3 backup-and-restore of certbot-generated certifiactes. It also tweaks the default logging formats to structured JSON, making it well suited for a variety of ingestion pipelines. Finally, it provides samples of readiness and liveness checks.
Default values (in parenthesis).
http_port: The port to listen on for HTTP connections (
https_port: The port to listen on for HTTPS connections (
admin_port: The port for the Envoy admin interface (
5000); an empty value will disable the admin interface.
ext-authzon the specified port (see: Authorizations)
bind_address: The address on which to listen (
log_format_switchboard: The Python
loggerformat for Switchboard (
[%(asctime)s] [%(process)d] [%(levelname)s] [%(name)s] %(message)s)
log_format_envoy: The Envoy application log format (empty = Envoy default)
log_format_access: The access log format for Envoy (empty = Envoy default)
log_format_access_json: The JSON access log format for Envoy (empty = Envoy default)
log_level: Used by both Switchboard, Envoy, and Authorization applications (
log_path: The folder in which to write log files (empty = write to
ingress variable defines the domain names which Switchboard will listen upon. The value can be an array (in a YML file) or a string with each ingress value separated by any whitespace. It is required and each value takes the form of:
The schema may be
wss. For secure schemas, adding a
! at the end will force the unsecure version to redirect to the secure. Alternatively, the
? suffix is a conevenience to listent on both secure and unsecure schemas.
This value may be empty if you do not wish to listen on a subdomain. You may also use the
? suffix to listen on both the subdomain and the top level domain. For example, a value of
www? would match for both
The top-level domain name, such as
path is optional, and will match only requests which begin with the specified path. If suffixed with a
!, the path will be stripped before it is sent to the cluster. For example, a value of
/foo! will mean that the destination will receive requests to
/foo/bar as simply
destination can either be an
egress_name or a fully-qualified domain name for redirection.
wss?://api:mydomain.com@my-cluster: Support Web Socket upgrading on both
api.mydomain.com, routed to
egress variable defines your Envoy clusters (servers) to route traffic to. The value can be an array (in a YML file) or a string with each egress value separated by any whitespace. It is required and each value takes the form of:
egress_name can be used twice with both the
grpc schemas in order to support both, in which case the
content-type header value of
application/grpc will be used to determine which destination to use. For example, the two values can be used together:
my-cluster:http@localhost:5200: Route regular HTTP traffic for
my-cluster:grpc@localhost:5201: Route GRPC traffic for
You can also use environment variables to assist in congfiguration. For example, in Kubernetes, you might route to some
grafana deployment as follows:
shard variable is provided, all of the subdomains will be modified based upon the value provided.
Assuming that the value
dev is provided, then the following will be true of ingress values:
http://www:mydomain.com:my-clusterwill instead listen on
https?://:mydomain.com:my-clusterwill instead listen on
wss!://test!:mydomain.com:my-clusterwill still listen on
stats_type configuration is set to
statsd_exporter will automatically be started. It will receive traffic from Envoy on port
9125 and publish metrics on port
9102, meaning that these ports become reserved and clients like Prometheus can connect to
auth_port configuration is provided, Switchboard will automatically create a self-managed authorization server to protect access to given domains. It will look for yaml files in
/etc/switchboard/authorizations/ which match the domain name being accessed.
Note: using the
authorization feature automatically creates a GRPC cluster named
ext-authz, powered by an internal Go server, and checks with this server before processing any request.
wss schema is used for any ingress, and the
bearer: ['some_token'] basic: ['dGVzdDoxMjM0']
Any requests to
example.com will only be allowed to proceed with one of the two headers:
Authorization: Bearer some_token(a presumed access token)
Authorization: Basic dGVzdDoxMjM0(equivalent to
test:1234as a username/password)
Requests without these headers will be rejected. Successfull requests will have the
x-ext-auth-ratelimit set to the sha256 of the token/authorization (for use in rate-limiting). Requests to other domains will succeed without challenge.