✨ Enhancement/html maintenance page per Deployment FQDN🚧🚧🚧 #93

mrnicegyu11 · 2023-01-25T18:12:59Z

This PR enables there to be individually styled maintenance pages for each DEPLOYMENT_FQDNS (i.e. domain, such as s4l-lite.sexy or speak.com).

This is implemented by using j2 templating to procedurally generate a docker-compose.yml for the maintenance page which contains one nginx per DEPLOYMENT_FQDNS. In the codedesign of this PR and in the following tests we assume that osparc-products and FQDNs (== domains) to have a 1:1 correspondence. The names will thus be used interchangeably.

For now, the oSparc API maintenance page (which servers a json making clear that oSparc is in maintenance) will not be customized and remains a catchall for all domains. This is done solely for time reasons, conceptually there could be an API maintenance page per product, using the same approach.

The following routes are captured by the maintenance page when the maintenance page is up:

Host( s4l.speag.com)
HostRegexp(services. s4l.speag.com,{subhost:[a-zA-Z0-9-]+}.services. s4l.speag.com)
HostRegexp(services.testing. s4l.speag.com,{subhost:[a-zA-Z0-9-]+}.services.testing. s4l.speag.com)

The following changes in associated repositories will be necessary:

The static html file of the maintenance page is no longer found in this repository, but it is assumed that the static .html files are found in the osparc-ops-deployment-configuration repo [in git speag].
The static html must be named according to a specific convention and placed into a specific destination in a specific git repository. An example will make this clear: The maintenance page for the domains4l.speag.com is assumed to be found in the osparc-ops-deployment-configuration [git.speag.com/oSparc/osparc-ops-deployment-configuration] in the osparc.speag.com branch (which identifies dalco-production according to the historic convention) in the path ./maintenancePageWebHTML/s4l.speag.com.html. For each further FQDNS, such as in this example s4l-lite.speag.com, this html file is assumed to exist. [Minor clarification: All "assets" of a certain deployment life in the deployment's branch in the osparc-ops-deployment-configuration git-repo]

🚧🚧🚧Additional necessary changes upon release 🚧🚧🚧:

html files need to be created and placed into the osparc-ops-deployment-configuration repository. Creation is done by @odeimaiz , potentially on the basis of an existing template.
Routing is experimental and needs to be validated on a deployment with multiple products having different FQDNs
Adds MAINTENANCE_PAGES_TRAEFIK_PRIORITY environment variable that needs to be added to repo.config

For this PR to be merged @odeimaiz approval is strictly necessary, since he will style (and thus own), the s4l maintenance pages. Gracias @odeimaiz !!

Example docker-compose.yml file created from DEPLOYMENT_FQDNS=tip.speag.com, s4l.speag.com, s4l-lite.speag.com:

version: '3.8'
configs:
  maintenance_page_api_html:
    file: ./page/api/index.html
  nginx_config:
    file: ./nginx-config/default.conf
  maintenance_page_web_tipspeagcom_html:
    file: /home/dustin/git/publicClones/osparc-ops-deployment-configuration_speag/maintenancePageWebHTML/tip.speag.com.html
  maintenance_page_web_s4lspeagcom_html:
    file: /home/dustin/git/publicClones/osparc-ops-deployment-configuration_speag/maintenancePageWebHTML/s4l.speag.com.html
  maintenance_page_web_s4llitespeagcom_html:
    file: /home/dustin/git/publicClones/osparc-ops-deployment-configuration_speag/maintenancePageWebHTML/s4l-lite.speag.com.html
  maintenance_page_web_osparcspeagcom_html:
    file: /home/dustin/git/publicClones/osparc-ops-deployment-configuration_speag/maintenancePageWebHTML/osparc.speag.com.html
services:
  maintenance_page_tipspeagcom:
    configs:
      - source: maintenance_page_web_tipspeagcom_html
        target: /usr/share/nginx/html/index.html
      - source: nginx_config
        target: /etc/nginx/conf.d/default.conf
    # nginx config
    image: nginx:1.23.3
    networks:
      - public
      - monitored
    deploy:
      labels:
        - traefik.enable=true
        - traefik.docker.network=${PUBLIC_NETWORK}
        - traefik.http.routers.nginx_web.priority=
        - traefik.http.routers.nginx_web.rule=Host(`tip.speag.com`)}
        - traefik.http.routers.nginx_web.rule=(Host(`tip.speag.com`) && PathPrefix(`/`)) || (HostRegexp(`services.tip.speag.com`,`{subhost:[a-zA-Z0-9-]+}.services.tip.speag.com`) && PathPrefix(`/`)) || (HostRegexp(`services.testing.tip.speag.com.replace(" ","")`,`{subhost:[a-zA-Z0-9-]+}.services.testing.tip.speag.com`) && PathPrefix(`/`))
        - traefik.http.routers.nginx_web.tls=true
        - traefik.http.routers.nginx_web.tls.certresolver=myresolver
        - traefik.http.services.nginx_web.loadbalancer.server.port=80
        - traefik.http.routers.nginx_web.entrypoints=https
  maintenance_page_s4lspeagcom:
    configs:
      - source: maintenance_page_web_s4lspeagcom_html
        target: /usr/share/nginx/html/index.html
      - source: nginx_config
        target: /etc/nginx/conf.d/default.conf
    # nginx config
    image: nginx:1.23.3
    networks:
      - public
      - monitored
    deploy:
      labels:
        - traefik.enable=true
        - traefik.docker.network=${PUBLIC_NETWORK}
        - traefik.http.routers.nginx_web.priority=
        - traefik.http.routers.nginx_web.rule=Host(`s4l.speag.com`)}
        - traefik.http.routers.nginx_web.rule=(Host(`s4l.speag.com`) && PathPrefix(`/`)) || (HostRegexp(`services.s4l.speag.com`,`{subhost:[a-zA-Z0-9-]+}.services.s4l.speag.com`) && PathPrefix(`/`)) || (HostRegexp(`services.testing. s4l.speag.com.replace(" ","")`,`{subhost:[a-zA-Z0-9-]+}.services.testing.s4l.speag.com`) && PathPrefix(`/`))
        - traefik.http.routers.nginx_web.tls=true
        - traefik.http.routers.nginx_web.tls.certresolver=myresolver
        - traefik.http.services.nginx_web.loadbalancer.server.port=80
        - traefik.http.routers.nginx_web.entrypoints=https
  maintenance_page_s4llitespeagcom:
    configs:
      - source: maintenance_page_web_s4llitespeagcom_html
        target: /usr/share/nginx/html/index.html
      - source: nginx_config
        target: /etc/nginx/conf.d/default.conf
    # nginx config
    image: nginx:1.23.3
    networks:
      - public
      - monitored
    deploy:
      labels:
        - traefik.enable=true
        - traefik.docker.network=${PUBLIC_NETWORK}
        - traefik.http.routers.nginx_web.priority=
        - traefik.http.routers.nginx_web.rule=Host(`s4l-lite.speag.com`)}
        - traefik.http.routers.nginx_web.rule=(Host(`s4l-lite.speag.com`) && PathPrefix(`/`)) || (HostRegexp(`services.s4l-lite.speag.com`,`{subhost:[a-zA-Z0-9-]+}.services.s4l-lite.speag.com`) && PathPrefix(`/`)) || (HostRegexp(`services.testing. s4l-lite.speag.com.replace(" ","")`,`{subhost:[a-zA-Z0-9-]+}.services.testing.s4l-lite.speag.com`) && PathPrefix(`/`))
        - traefik.http.routers.nginx_web.tls=true
        - traefik.http.routers.nginx_web.tls.certresolver=myresolver
        - traefik.http.services.nginx_web.loadbalancer.server.port=80
        - traefik.http.routers.nginx_web.entrypoints=https
  maintenance_page_osparcspeagcom:
    configs:
      - source: maintenance_page_web_osparcspeagcom_html
        target: /usr/share/nginx/html/index.html
      - source: nginx_config
        target: /etc/nginx/conf.d/default.conf
    # nginx config
    image: nginx:1.23.3
    networks:
      - public
      - monitored
    deploy:
      labels:
        - traefik.enable=true
        - traefik.docker.network=${PUBLIC_NETWORK}
        - traefik.http.routers.nginx_web.priority=
        - traefik.http.routers.nginx_web.rule=Host(`osparc.speag.com`)}
        - traefik.http.routers.nginx_web.rule=(Host(`osparc.speag.com`) && PathPrefix(`/`)) || (HostRegexp(`services.osparc.speag.com`,`{subhost:[a-zA-Z0-9-]+}.services.osparc.speag.com`) && PathPrefix(`/`)) || (HostRegexp(`services.testing.osparc.speag.com.replace(" ","")`,`{subhost:[a-zA-Z0-9-]+}.services.testing.osparc.speag.com`) && PathPrefix(`/`))
        - traefik.http.routers.nginx_web.tls=true
        - traefik.http.routers.nginx_web.tls.certresolver=myresolver
        - traefik.http.services.nginx_web.loadbalancer.server.port=80
        - traefik.http.routers.nginx_web.entrypoints=https
  maintenance_page_api:
    # nginx config
    image: nginx:1.23.3
    configs:
      - source: maintenance_page_api_html
        target: /usr/share/nginx/html/index.html
      - source: nginx_config
        target: /etc/nginx/conf.d/default.conf
    networks:
      - public
      - monitored
    deploy:
      labels:
        - traefik.enable=true
        - traefik.docker.network=${PUBLIC_NETWORK}
        - traefik.http.routers.nginx_api.priority=
        - traefik.http.routers.nginx_api.tls=true
        - traefik.http.routers.nginx_api.rule=${DEPLOYMENT_API_DOMAIN_CAPTURE_TRAEFIK_RULE}
        - traefik.http.routers.nginx_api.tls.certresolver=myresolver
        - traefik.http.services.nginx_api.loadbalancer.server.port=80
        - traefik.http.routers.nginx_api.entrypoints=https
networks:
  public:
    external: true
    name: ${PUBLIC_NETWORK}
  monitored:
    name: ${MONITORED_NETWORK}
    external: true

`

mrnicegyu11 · 2023-01-25T18:14:29Z

@odeimaiz I can guide you through the PR or what this implies for you quickly tomorrow if you want ;)

pcrespov · 2023-01-25T18:24:28Z

services/maintenance-page/docker-compose.yml.j2

+    image: nginx:1.23.3
+    configs:
+      - source: maintenance_page_api_html
+        target: /usr/share/nginx/html/index.html


If i understand that well, this responds with an html page for the API as well?
That is not strictly necessary (might be ok to "replace" the swagger view under /dev/doc though).
The API server is used to "talk to client machines". The important thing for the API is to return 503 (service unavailable) with a maintenance message and include in the headers the time to retry (e.g. end of maintenance).

Please check in the ops issues. There you should find all explanations.

@Surfict created the API status page, he might be able to give more details. I was not introduced to the design ideas behind the API maintenance page, I guess its creation was requested and speced by you @pcrespov .

Please comment here @Surfict

The html file returns the following JSON code :
{ "errors": { "msg": "Service currently unavailable due to scheduled maintenance.", "type": "runtime_error" } }
I don't think anything has been done for the headers though.

sanderegg

👍

…FQDN

odeimaiz

👍

* Fix error in call_make call * quote dot * Initial changes * fix whitespace bug Co-authored-by: Dustin Kaiser <kaiser@itis.swiss> Co-authored-by: Dustin Kaiser <mail@dustinkaiser.eu>

Dustin Kaiser and others added 4 commits January 25, 2023 13:01

Fix error in call_make call

125ce09

quote dot

0ded973

Initial changes

54a5ad1

fix whitespace bug

dacd415

mrnicegyu11 added the t:enhancement New feature or request label Jan 25, 2023

mrnicegyu11 added this to the s4l-web-lite milestone Jan 25, 2023

mrnicegyu11 requested review from Surfict, pcrespov, odeimaiz and sanderegg January 25, 2023 18:12

mrnicegyu11 self-assigned this Jan 25, 2023

pcrespov reviewed Jan 25, 2023

View reviewed changes

sanderegg reviewed Jan 25, 2023

View reviewed changes

Merge branch 'main' into enhancement/HTMLmaintenancePagePerDeployment…

26eca99

…FQDN

odeimaiz approved these changes Jan 26, 2023

View reviewed changes

mrnicegyu11 merged commit fc37ded into ITISFoundation:main Jan 26, 2023

mrnicegyu11 mentioned this pull request Feb 17, 2023

M1-12 Maintenance and DevOps ITISFoundation/osparc-issues#675

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

✨ Enhancement/html maintenance page per Deployment FQDN🚧🚧🚧 #93

✨ Enhancement/html maintenance page per Deployment FQDN🚧🚧🚧 #93

mrnicegyu11 commented Jan 25, 2023

mrnicegyu11 commented Jan 25, 2023

pcrespov Jan 25, 2023 •

edited

mrnicegyu11 Jan 26, 2023

Surfict Jan 26, 2023

sanderegg left a comment

odeimaiz left a comment

✨ Enhancement/html maintenance page per Deployment FQDN🚧🚧🚧 #93

✨ Enhancement/html maintenance page per Deployment FQDN🚧🚧🚧 #93

Conversation

mrnicegyu11 commented Jan 25, 2023

mrnicegyu11 commented Jan 25, 2023

pcrespov Jan 25, 2023 • edited

Choose a reason for hiding this comment

mrnicegyu11 Jan 26, 2023

Choose a reason for hiding this comment

Surfict Jan 26, 2023

Choose a reason for hiding this comment

sanderegg left a comment

Choose a reason for hiding this comment

odeimaiz left a comment

Choose a reason for hiding this comment

pcrespov Jan 25, 2023 •

edited