Traefik Proxy in a Docker container with TLS using a Let’s Encrypt certificate and TLS DNS-01 challenge all inside your private network
Leon Steenkamp
2021-07-18
This project uses docker-compose to run a Traefik reverse proxy container and a whoami container to test against.
Traefik Proxy serves as a TLS endpoint for the hosts behind the proxy. Let’s Encrypt's DNS-01 challenge is used for certificate generation because the proxy is not accessable from the Internet. This example only shows HTTP/S web entry points but other entry points (UDP/TCP) are possible.
The different challenge types that can be used with Let's Encrypt are covered at here - https://letsencrypt.org/docs/challenge-types/
Requirements for this project: You will need a host with docker and docker-compose installed, an AWS account and domain that you control in AWS Route53.
The README is divided into the following sections:
- Create AWS IAM user
- Docker-compose file
- Traefik static configuration
- Traefik dynamic configuration
This README assumes you control the example.co.za domain. Traefik is configured using YAML configuration files.
- Traefik documentation - https://doc.traefik.io/traefik/
- Docker-compose with let's encrypt: DNS Challenge example - https://doc.traefik.io/traefik/user-guides/docker-compose/acme-dns/
Some useful addresses after you have the container up and running:
- http://traefik-host-address:8080/ping
- http://traefik-host-address:8080/dashboard
- https://whoami.example.co.za/
For the Let’s Encrypt DNS-01 challenge to work an AWS IAM user and policy have to be created for Traefik to use. The process is explained in the Traefik documentation but there are a few things that tripped me up the first time. Please note that I am not an experienced Route53 user, be sure you are comfortable with the changes you are making to your setup.
Log into AWS with a user that can create IAM users and policies. Under IAM Policies create a policy using the JSON tab and use the policy in JSON format from the second link below. The policy editor might complain about the empty Sid values. Add your own Sid if you want. The policy is also shown below.
"The Sid (statement ID) is an optional identifier that you provide for the policy statement." - https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_sid.html
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Action": [
"route53:GetChange",
"route53:ChangeResourceRecordSets",
"route53:ListResourceRecordSets"
],
"Resource": [
"arn:aws:route53:::hostedzone/*",
"arn:aws:route53:::change/*"
]
},
{
"Sid": "",
"Effect": "Allow",
"Action": "route53:ListHostedZonesByName",
"Resource": "*"
}
]
}Create an IAM user with programmatic access and use "attach existing policies directly" to add the policy created above (you might need use the filter to find your policy). Make a note of the user's access key and access key ID.
While in the AWS management console make a note of the Route53 Hosted zone ID the domain is under. The key and two IDs are used in the Traefik configuration.
This process is also explained in the link below:
The docker-compose file is based on the examples in the Traefik documentation.
The container forwards three ports for HTTP and HTTPS traffic. The Traefik dashboard is also exposed.
For my example reverse proxy deployment mostly pointed to services running on external hosts and not services running in other docker containers on a docker network. The whoami container gives you an idea how labels can be used to configure Traefik.
It looks like Traefik needs to be restarted when new router rules with new hostnames are added. Otherwise, certificates are not created for the new hosts.
Getting the Traefik dynamic configuration to load dynamically when using docker took some trial, error and searching. The trick seems to be mounting a directory and not the file directly and telling Traefik to look at the directory rather than just the file. This also covered in the Traefik configuration section.
The created certificates are stored in the letsencrypt\ directory.
As shown in the Traefik documentation, docker secrets are used to store the AWS IAM user credentials.
It was not immediatly clear what environment variables were needed to pass the Route53 credentials to Traefik, but after reading the LEGO page carefully you'll notice that AWS_SHARED_CREDENTIALS_FILE should be used instead of AWS_ACCESS_KEY_ID_FILE and AWS_SECRET_ACCESS_KEY_FILE when using files.
The secrets\ directory is used to store the details used by the Let’s Encrypt DNS-01 challenge. I used AWS Route53 for the DNS-01 challenge, the values needed here will probably differ if you use another provider.
In the secrets\ directory you need to create two files:
- aws_hosted_zone_id.secret
- aws_shared_credentials.secret
The only contents in the aws_hosted_zone_id.secret file is the zone ID where the domain you are using lives.
The contents of the aws_shared_credentials.secret is shown below:
[default]
aws_access_key_id = add-your-id-here
aws_secret_access_key = add-your-key-hereAdd the details from the AWS IAM user you created in these two files.
The traefik.yml static configuration file is based on the examples in the Traefik documentation.
Traefik uses the concepts of entry points, routers, middlewares and services. These are explained in the documentation and it might take a few reads to fully understand.
Two entry points are defined for web and websecure that handle HTTP traffic on port 80 and 443. I added redirection from the web to websecure entry points.
Logging is set to debug, so you might want to change this. Logging can be seen when docker-compose is run without the -d (detached) option.
The other important section to notice is certificatesResolvers. This is where Let's Encrypt comes in. Add a valid email address to this section.
The docker provider allows for label based dynamic configuration when using docker contrainers. The file provider allows for file-based dynamic configuration.
Note: When using docker is looks like specifiying a directory to watch for a dynamic configuration file works better than specifying a file directly.
The dynamic_conf.yml dynamic configuration file is based on the examples in the Traefik documentation.
The dynamic configuration is reloaded while Traefik is running without having to restart Traefik (altough it might be needed to restart Traefik if a new host is added to force certificate generation if a certificate does not exist).
Two example services are specified, one that forwards traffic to a specific port and the other that defaults to port 80.
A serversTransport is specified that skips TLS verification to these hosts, otherwise, a browser warning is generated. This is a bit of a chicken and egg problem if not skipping the check. One of the reasons for running the reverse proxy is to handle TLS certificates, if these hosts could provide valid certificates then this exercise would not be needed. The assumption is that the network and hosts behind the reverse proxy is trusted.
Services can specify more than one host which allows for load balancing by Traefik. If there is only one host, then you only have to specifiy one host. Replace the example IP addresses with addresses to your services.
Routers join entry points to services while middlewares do any modifications that might be needed in between.
In the example two routers listen on the websecure entry point with rules for two different subdomains. Traffic for each subdomain is forwarded to a different service and both routers make use of the certResolver specified in the Traefik static configuration.
An example of a middleware is shown where a path is added to the request sent to the service (host). In this case /admin is added - the Pi-hole admin panel is located at this path. The Pi-hole admin panel can be accessed from hole.example.co.za without needing the /admin path at the end.
Update 2023-03-11
This example makes use of static and dynamic configuration files for Traefik through the Traefik file provider. The example also makes use of the Docker provider in Traefik, which monitors the Docker daemon for labels specified in containers.
In the documentation on the Traefik site the Docker daemon (/var/run/docker.sock) is mounted to the Traefik container. This is probably fine on your home or internal networks, but might need some further evaluation if your Traefik container is going to be exposed directly to the Internet. This is because it is considered a security risk to expose your Docker daemon directly to a container.
Some links for further reading are below, including where this is briefly discussed in the Traefik documentation:
- Hacker News https://news.ycombinator.com/item?id=17983623
- Exposing Docker socket to Traefik container is a serious security risk · Issue #4174 · traefik/traefik traefik/traefik#4174
- Traefik Docker Documentation - Traefik https://doc.traefik.io/traefik/providers/docker/#docker-api-access
A workaround that I like is to use only configuration files and leaving the Docker provider alone. For my foreseeable use cases this should work just fine and it allows you to avoid sharing your Docker daemon with a container.