Upgrade to Traefik v3.5.2. TOML files to YAML, Lets encrypt upgrade, Dynamic file for TLS and Headers middleware

README.md

@@ -14,7 +14,7 @@ The demo server runs the following apps/services:
 * [pygeoapi_stable](services/pygeoapi_stable) - (last stable version of) `pygeoapi` service with test data
 * [pygeoapi_cite](services/pygeoapi_cite) - (latest GitHub `master` version of) `pygeoapi` service with CITE configuration
 * [pygeoapi_covid-19](services/pygeoapi_covid-19) - (latest GitHub `master` version of) `pygeoapi` service with Covid-19 configuration
-* [traefik](services/traefik) - edge/proxy server routing incoming HTTP(S) and managing SSL-certificates (via Let's Encrypt)
+* [traefik](services/traefik) - Traefik v3.5.2 edge/proxy server with HTTP(S) routing, SSL certificate management (Let's Encrypt), security headers, and TLS optimization
 * [dockerhub listener](services/dockerhub) - listens to webhooks from DockerHub to redeploy [home](services/home) and [pygeoapi_master](services/pygeoapi_master)
 
 ## Auto Update
@@ -41,7 +41,6 @@ to the `master` of the [pygeoapi GitHub repo](https://github.com/geopython/pygeo
 ## Common setup with Ansible
 
 ```
-
 # get code
 git clone https://github.com/geopython/demo.pygeoapi.io
 
@@ -64,11 +63,9 @@ These are not in this repo for obvious reasons... You need to create/populate th
 Have a remote Ubuntu VM installed with root access via pub/private key.
 
 ```
-
 # Installs entire system
 cd ansible
-ansible-playbook -vv bootstrap.yml -i hosts/demo.pygeoapi.io
-
+ansible-playbook -vv bootstrap.yml -i hosts/demo.pygeoapi.io  --become
 
 ```
 
@@ -103,3 +100,17 @@ sudo service pygeoapi status
 
 ```
 
+## Traefik v3.5.2 Configuration
+
+This setup uses **Traefik v3.5.2** with enhanced security and modern configuration:
+
+### Configuration Approach
+- **Static configuration**: Defined via command-line flags in `services/traefik/docker-compose.yml`
+- **Dynamic configuration**: File-based configs for TLS and middleware settings
+- **Certificates**: Stored in Docker volume `acme_certificates` (managed automatically by Let's Encrypt)
+- **Routing**: Separate routers for production (HTTPS) and localhost (HTTP only)
+
+### Key Files
+- `services/traefik/docker-compose.yml` - Main Traefik configuration via command flags
+- `services/traefik/config/dynamic/tls.yml` - TLS security options (minimum TLS 1.2, cipher suites)
+- `services/traefik/config/dynamic/middlewares.yml` - HTTP security headers (HSTS, CORS, etc.)

ansible/bootstrap.yml

@@ -37,7 +37,7 @@
   tasks:
 
     - name: "Clone or Update Git Repo"
-      git: "repo={{ my_github_repo }} dest={{ my_git_home }} version=master clone=yes force=yes accept_hostkey=True"
+      git: "repo={{ my_github_repo }} dest={{ my_git_home }} version={{ my_git_branch | default('master') }} clone=yes force=yes accept_hostkey=True"
 
     - name: "Setup dirs and ownerships"
       shell: "mkdir -p {{ logs_dir }}; chown -R {{ my_admin_user }}:{{ my_admin_user }} {{ var_dir }} {{ my_git_home }}"

ansible/deploy.yml

@@ -11,7 +11,7 @@
 
   pre_tasks:
     - name: "Clone or Update Git Repo"
-      git: "repo={{ my_github_repo }} dest={{ my_git_home }} version=master clone=yes force=yes accept_hostkey=True"
+      git: "repo={{ my_github_repo }} dest={{ my_git_home }} version={{ my_git_branch | default('master') }} clone=yes force=yes accept_hostkey=True"
       tags: update
 
     - name: copy DockerHub env file

ansible/hosts/pygeoapi.terraops.net

@@ -0,0 +1,5 @@
+[pygeoapi_terraops]
+pygeoapi.terraops.net ansible_port=22 ansible_user=mende012 ansible_host=pygeoapi.terraops.net
+
+[pygeoapi_terraops:vars]
+ansible_python_interpreter=/usr/bin/python3

services/dockerhub/docker-compose.yml

@@ -3,24 +3,30 @@ services:
   #  See https://github.com/maccyber/micro-dockerhub-hook
   dockerhub:
     image: maccyber/micro-dockerhub-hook
-
     container_name: dockerhub
-
     expose:
       - "3000"
-
 #    ports:
 #      - "3000:3000"
-
     env_file: dockerhub.env
-
     labels:
       - "traefik.enable=true"
-      - "traefik.frontend.rule=Host:demo.pygeoapi.io;PathPrefixStrip:/dhubhook"
-      - "traefik.backend=dockerhub"
-      - "traefik.frontend.priority=100"
+      # Production router (HTTPS with Let's Encrypt)
+      - "traefik.http.routers.dockerhub.rule=Host(`demo.pygeoapi.io`) && PathPrefix(`/dhubhook`)"
+      - "traefik.http.routers.dockerhub.entrypoints=https"
+      - "traefik.http.routers.dockerhub.tls.certresolver=le"
+      - "traefik.http.routers.dockerhub.tls.options=tls_default@file"
+      - "traefik.http.routers.dockerhub.priority=100"
+      - "traefik.http.routers.dockerhub.middlewares=dhubhook-stripprefix,secure-headers@file"
+      # Localhost router (HTTP only)
+      - "traefik.http.routers.dockerhub_local.rule=Host(`localhost`) && PathPrefix(`/dhubhook`)"
+      - "traefik.http.routers.dockerhub_local.entrypoints=http"
+      - "traefik.http.routers.dockerhub_local.priority=100"
+      - "traefik.http.routers.dockerhub_local.middlewares=dhubhook-stripprefix"
+      # Common middleware and service
+      - "traefik.http.middlewares.dhubhook-stripprefix.stripprefix.prefixes=/dhubhook"
+      - "traefik.http.services.dockerhub.loadbalancer.server.port=3000"
       - "traefik.docker.network=pygeoapi-network"
-
     volumes:
       - /var/run/docker.sock:/var/run/docker.sock
       - ./scripts:/src/scripts
@@ -30,7 +36,6 @@ services:
       - ./../pygeoapi_cite:/srv/pygeoapi_cite
       - ./../pygeoapi_covid-19:/srv/pygeoapi_covid-19
 
-
 networks:
   default:
     name: pygeoapi-network

services/home/docker-compose.yml

@@ -3,16 +3,23 @@
 services:
   home:
     image: geopython/demo.pygeoapi.io:latest
-
     restart: unless-stopped
-
     container_name: home
-
     labels:
-      - "traefik.backend=home_app"
       - "traefik.enable=true"
-      - "traefik.frontend.priority=5"
-      - "traefik.frontend.rule=Host:demo.pygeoapi.io; PathPrefixStrip:/"
+      # Production router (HTTPS with Let's Encrypt)
+      - "traefik.http.routers.home.rule=Host(`demo.pygeoapi.io`) && Path(`/`)"
+      - "traefik.http.routers.home.entrypoints=https"
+      - "traefik.http.routers.home.tls.certresolver=le"
+      - "traefik.http.routers.home.tls.options=tls_default@file"
+      - "traefik.http.routers.home.priority=5"
+      - "traefik.http.routers.home.middlewares=secure-headers@file"
+      # Localhost router (HTTP only)
+      - "traefik.http.routers.home_local.rule=Host(`localhost`) && Path(`/`)"
+      - "traefik.http.routers.home_local.entrypoints=http"
+      - "traefik.http.routers.home_local.priority=5"
+      # Common service definition
+      - "traefik.http.services.home.loadbalancer.server.port=5000"
       - "traefik.docker.network=pygeoapi-network"
 
 networks:

services/pygeoapi_cite/docker-compose.yml

@@ -1,31 +1,31 @@
 services:
-
   pygeoapi_cite:
-
     image: geopython/pygeoapi_cite:latest
-
     build: ./pygeoapi
-
     container_name: pygeoapi_cite
-
     restart: unless-stopped
-
     expose:
       - "80"
-
 #    ports:
 #      - "5000:80"
-
     environment:
      - SCRIPT_NAME=/cite
-
     labels:
       - "traefik.enable=true"
-      - "traefik.frontend.rule=Host:demo.pygeoapi.io,localhost;PathPrefix:/cite"
-      - "traefik.backend=pygeoapi_cite"
-      - "traefik.frontend.priority=100"
+      # Production router (HTTPS with Let's Encrypt)
+      - "traefik.http.routers.pygeoapi_cite.rule=Host(`demo.pygeoapi.io`) && PathPrefix(`/cite`)"
+      - "traefik.http.routers.pygeoapi_cite.entrypoints=https"
+      - "traefik.http.routers.pygeoapi_cite.tls.certresolver=le"
+      - "traefik.http.routers.pygeoapi_cite.tls.options=tls_default@file"
+      - "traefik.http.routers.pygeoapi_cite.priority=100"
+      - "traefik.http.routers.pygeoapi_cite.middlewares=secure-headers@file"
+      # Localhost router (HTTP only)
+      - "traefik.http.routers.pygeoapi_cite_local.rule=Host(`localhost`) && PathPrefix(`/cite`)"
+      - "traefik.http.routers.pygeoapi_cite_local.entrypoints=http"
+      - "traefik.http.routers.pygeoapi_cite_local.priority=100"
+      # Common service definition
+      - "traefik.http.services.pygeoapi_cite.loadbalancer.server.port=80"
       - "traefik.docker.network=pygeoapi-network"
-
 networks:
   default:
     name: pygeoapi-network

services/pygeoapi_covid-19/docker-compose.yml

@@ -1,28 +1,31 @@
 services:
 
-  pygeoapi_cite:
+  pygeoapi_covid19:
     image: geopython/pygeoapi_covid-19:latest
-
     build: ./pygeoapi
-
     container_name: pygeoapi_covid-19
-
     restart: unless-stopped
-
     expose:
       - "80"
-
 #    ports:
 #      - "5000:80"
-
     environment:
      - SCRIPT_NAME=/covid-19
-
     labels:
       - "traefik.enable=true"
-      - "traefik.frontend.rule=Host:demo.pygeoapi.io,localhost;PathPrefix:/covid-19"
-      - "traefik.backend=pygeoapi_covid-19"
-      - "traefik.frontend.priority=100"
+      # Production router (HTTPS with Let's Encrypt)
+      - "traefik.http.routers.pygeoapi_covid19.rule=Host(`demo.pygeoapi.io`) && PathPrefix(`/covid-19`)"
+      - "traefik.http.routers.pygeoapi_covid19.entrypoints=https"
+      - "traefik.http.routers.pygeoapi_covid19.tls.certresolver=le"
+      - "traefik.http.routers.pygeoapi_covid19.tls.options=tls_default@file"
+      - "traefik.http.routers.pygeoapi_covid19.priority=100"
+      - "traefik.http.routers.pygeoapi_covid19.middlewares=secure-headers@file"
+      # Localhost router (HTTP only)
+      - "traefik.http.routers.pygeoapi_covid19_local.rule=Host(`localhost`) && PathPrefix(`/covid-19`)"
+      - "traefik.http.routers.pygeoapi_covid19_local.entrypoints=http"
+      - "traefik.http.routers.pygeoapi_covid19_local.priority=100"
+      # Common service definition
+      - "traefik.http.services.pygeoapi_covid19.loadbalancer.server.port=80"
       - "traefik.docker.network=pygeoapi-network"
 
 networks:

services/pygeoapi_master/docker-compose.yml

@@ -1,27 +1,29 @@
 services:
-
   pygeoapi_master:
-
     image: geopython/pygeoapi_master:latest
-
     container_name: pygeoapi_master
-
     restart: unless-stopped
-
     expose:
       - "80"
-
 #    ports:
 #     - "5000:80"
-
     environment:
      - SCRIPT_NAME=/master
-
     labels:
       - "traefik.enable=true"
-      - "traefik.frontend.rule=Host:demo.pygeoapi.io,localhost;PathPrefix:/master"
-      - "traefik.backend=pygeoapi_master"
-      - "traefik.frontend.priority=100"
+      # Production router (HTTPS with Let's Encrypt)
+      - "traefik.http.routers.pygeoapi_master.rule=Host(`demo.pygeoapi.io`) && PathPrefix(`/master`)"
+      - "traefik.http.routers.pygeoapi_master.entrypoints=https"
+      - "traefik.http.routers.pygeoapi_master.tls.certresolver=le"
+      - "traefik.http.routers.pygeoapi_master.tls.options=tls_default@file"
+      - "traefik.http.routers.pygeoapi_master.priority=100"
+      - "traefik.http.routers.pygeoapi_master.middlewares=secure-headers@file"
+      # Localhost router (HTTP only)
+      - "traefik.http.routers.pygeoapi_master_local.rule=Host(`localhost`) && PathPrefix(`/master`)"
+      - "traefik.http.routers.pygeoapi_master_local.entrypoints=http"
+      - "traefik.http.routers.pygeoapi_master_local.priority=100"
+      # Common service definition
+      - "traefik.http.services.pygeoapi_master.loadbalancer.server.port=80"
       - "traefik.docker.network=pygeoapi-network"
 
 networks:

services/pygeoapi_stable/docker-compose.yml

@@ -5,27 +5,31 @@ services:
     image: geopython/pygeoapi:0.21.0
 
     container_name: pygeoapi_stable
-
     restart: unless-stopped
-
     expose:
       - "80"
-
 #    ports:
 #      - "5000:80"
-
     volumes:
       - ./local.config.yml:/pygeoapi/local.config.yml
       - ./data:/pygeoapi/data
-
     environment:
      - SCRIPT_NAME=/stable
-
     labels:
       - "traefik.enable=true"
-      - "traefik.frontend.rule=Host:demo.pygeoapi.io,localhost;PathPrefix:/stable"
-      - "traefik.backend=pygeoapi_stable"
-      - "traefik.frontend.priority=100"
+      # Production router (HTTPS with Let's Encrypt)
+      - "traefik.http.routers.pygeoapi_stable.rule=Host(`demo.pygeoapi.io`) && PathPrefix(`/stable`)"
+      - "traefik.http.routers.pygeoapi_stable.entrypoints=https"
+      - "traefik.http.routers.pygeoapi_stable.tls.certresolver=le"
+      - "traefik.http.routers.pygeoapi_stable.tls.options=tls_default@file"
+      - "traefik.http.routers.pygeoapi_stable.priority=100"
+      - "traefik.http.routers.pygeoapi_stable.middlewares=secure-headers@file"
+      # Localhost router (HTTP only)
+      - "traefik.http.routers.pygeoapi_stable_local.rule=Host(`localhost`) && PathPrefix(`/stable`)"
+      - "traefik.http.routers.pygeoapi_stable_local.entrypoints=http"
+      - "traefik.http.routers.pygeoapi_stable_local.priority=100"
+      # Common service definition
+      - "traefik.http.services.pygeoapi_stable.loadbalancer.server.port=80"
       - "traefik.docker.network=pygeoapi-network"
 
 networks:

services/traefik/config/README.md

@@ -1,5 +1,20 @@
-# Traefik Config
+# Traefik v3.5.2 Configuration
 
-In this dir acme.json should be created with LetsEncrypt certs automagically.
-The config files are named after the host's hostname to allow 
-non-HTTPS for local testing. `PYGEOAPI` is the production host.
+This directory contains the Traefik v3.x configuration files and dynamic configurations.
+
+## Configuration Files
+
+The main configuration files are named after the host's hostname:
+- `traefik.PYGEOAPI.yml` - Production config with HTTPS and Let's Encrypt
+- `traefik.default.yml` - Development/default config (HTTP only)
+- `default.nusa.yml` - Nusa-specific configuration
+
+## Dynamic Configuration
+
+The `dynamic/` directory contains:
+- `tls.yml` - TLS security options with strong cipher suites (TLS 1.2+ minimum)
+- `middlewares.yml` - HTTP security headers and CORS configuration
+
+## Certificates
+
+- `acme.json` - Let's Encrypt certificates (created automatically, permissions 600)