Supports defining remote versions in .shape-docs.yml

[simonbs]

The changes in this PR adds support for specifying remote versions and their specifications in the .shape-docs.yml file, removing the need for copying specifications that are already hosted publicly online into our GitHub repositories.

Motivation

The benefit of not copying public configurations into our repositories is that we avoid the documentation becoming stale. Currently projects like ok-openapi and avida-openapi have non-trivial GitHub Actions workflows running periodically to copy OpenAPI specifications into the projects. However, as soon as these copies have been established they are becoming stale and developers may not see the most up to date version when visiting Shape Docs.

However, there are also downside to displaying publicly available documentation, namely:

1. If the hosting of the publicly available OpenAPI specification is unreliable, Shape Docs becomes unreliable too as we rely on fetching the data from the public host on-demand.
2. The publicly available OpenAPI specification may be put behind some kind of authentication, meaning that we can no longer display it.

The solution is straightforward in both cases though: developers should rely on using GitHub Actions workflows to copy the OpenAPI specifications into our GitHub repositories. As such, it is not a requirement to use this new feature but it's an optimization that can be applied wherever it makes sense.

Solution

With the changes in this PR we add the new remoteVersions key to the .shape-docs.yml file which lists the versions to display and the specifications for each version. A working example for the Avida project will look like this:

name: Avida
image: icon.png
remoteVersions:
  - name: main
    specifications:
    - name: OpenAPI
      url: https://dev-api.avidafinance.com/api/card/swagger/v1/swagger.yaml

A similar more extensive configuration of the OK project can look like this:

name: OK
image: icon.png
remoteVersions:
  - name: Test
    specifications:
    - name: App Push Notification Service
      url: https://app-push-notification-service.test.okcloud.dk/swagger/v1/swagger.yaml
    - name: AppData EMSP Test
      url: https://appdata.emsp-tst.ok.dk/swagger/AppData/swagger.yaml
    - name: Electricity OK Home App Service (1.0)
      url: https://electricity-ok-home-app-service.test.okcloud.dk/swagger/v1/swagger.yaml
    - name: Electricity OK Home App Service (2.0)
      url: https://electricity-ok-home-app-service.test.okcloud.dk/swagger/v2/swagger.yaml
    - name: Geo Mobility
      url: https://geo-emobility.test.okcloud.dk/swagger/v1/swagger.yaml
    - name: OK Home App Service (1.0)
      url: https://ok-home-app-service.test.okcloud.dk/swagger/v1/swagger.yaml
    - name: OK Home App Service (2.0)
      url: https://ok-home-app-service.test.okcloud.dk/swagger/v2/swagger.yaml
    - name: OK Home App Service (3.0)
      url: https://ok-home-app-service.test.okcloud.dk/swagger/v3/swagger.yaml
    - name: User Communication Service
      url: https://user-communication-service.test.okcloud.dk/swagger/v1/swagger.yaml
    - name: Vehicle Accounting Service
      url: https://vehicle-accounting-service.test.okcloud.dk/swagger/v1/swagger.yaml
  - name: Prod
    specifications:
    - name: App Push Notification Service
      url: https://app-push-notification-service.okcloud.dk/swagger/v1/swagger.yaml
    - name: AppData EMSP Prod
      url: https://appdata.emsp.ok.dk/swagger/AppData/swagger.yaml
    - name: Electricity OK Home App Service (1.0)
      url: https://electricity-ok-home-app-service.okcloud.dk/swagger/v1/swagger.yaml
    - name: Electricity OK Home App Service (2.0)
      url: https://electricity-ok-home-app-service.okcloud.dk/swagger/v2/swagger.yaml
    - name: Geo Mobility
      url: https://geo-emobility.okcloud.dk/swagger/v1/swagger.yaml
    - name: OK Home App Service (1.0)
      url: https://ok-home-app-service.okcloud.dk/swagger/v1/swagger.yaml
    - name: OK Home App Service (2.0)
      url: https://ok-home-app-service.okcloud.dk/swagger/v2/swagger.yaml
    - name: OK Home App Service (3.0)
      url: https://ok-home-app-service.okcloud.dk/swagger/v3/swagger.yaml
    - name: User Communication Service
      url: https://user-communication-service.okcloud.dk/swagger/v1/swagger.yaml
    - name: Vehicle Accounting Service
      url: https://vehicle-accounting-service.okcloud.dk/swagger/v1/swagger.yaml
  - name: Prod Test
    specifications:
    - name: App Push Notification Service
      url: https://app-push-notification-service.prod-test.okcloud.dk/swagger/v1/swagger.yaml
    - name: AppData EMSP Prod Test
      url: https://appdata.emsp-prdtst.ok.dk/swagger/AppData/swagger.yaml
    - name: Electricity OK Home App Service (1.0)
      url: https://electricity-ok-home-app-service.prod-test.okcloud.dk/swagger/v1/swagger.yaml
    - name: Electricity OK Home App Service (2.0)
      url: https://electricity-ok-home-app-service.prod-test.okcloud.dk/swagger/v2/swagger.yaml
    - name: Geo Mobility
      url: https://geo-emobility.prod-test.okcloud.dk/swagger/v1/swagger.yaml
    - name: OK Home App Service (1.0)
      url: https://ok-home-app-service.prod-test.okcloud.dk/swagger/v1/swagger.yaml
    - name: OK Home App Service (2.0)
      url: https://ok-home-app-service.prod-test.okcloud.dk/swagger/v2/swagger.yaml
    - name: OK Home App Service (3.0)
      url: https://ok-home-app-service.prod-test.okcloud.dk/swagger/v3/swagger.yaml
    - name: User Communication Service
      url: https://user-communication-service.prod-test.okcloud.dk/swagger/v1/swagger.yaml
    - name: Vehicle Accounting Service
      url: https://vehicle-accounting-service.prod-test.okcloud.dk/swagger/v1/swagger.yaml

Details

There are a few details worth highlighting in this implementation.

New Rules for URLs

Most importantly, we no longer require URLs pointing to a specification to end with ".yml" or ".yaml". Earlier we would use the presence of this file extension to determine that the URL contained the ID of a specification. As developers can now create their own specifications, we get rid requirement as manually creating a specification with a name that looks like a filename seems odd.

This makes it impossible to determine which parts of the URL is the version ID. Consider the following URL:

https://docs.staging/ok/johnsmith/enhancement/stories-endpoint/openapi.yml

With the path:

/ok/johnsmith/enhancement/stories-endpoint/openapi.yml

Because the URL ends with .yml and because we assume that filenames do not contain a slash, it is apparent that "openapi.yml" is the ID of the specification. Similarly, because GitHub repository names cannot contain a slash, we know that anything between the first and second slash is the project ID. That leaves anything between the project ID and the specification ID to be the version ID.

	Value
Project ID	ok
Version ID	johnsmith/enhancement/stories-endpoint
Specification ID	openapi.yml

Things get trickier when the URL does not end with ".yml" or ".yaml" and even more so if the user decides to visit the following path to navigate to the first (and possibly only) OpenAPI specification on a branch, it gets even trickier.

/ok/johnsmith/enhancement/stories-endpoint

We still know that the project ID must be anything between the first and the second slash. We make a guess that the specification ID is anything after the last slash. That leaves anything between the project ID and specification ID to be the version ID.

	Value
Project ID	ok
Version ID	johnsmith/enhancement
Specification ID	stories-endpoint

When attempting to find a version with ID johnsmith/enhancement in the project with the ID ok, we will find that no such version exists. So we concatenate the version ID and the specification ID and see if we can find a version with ID johnsmith/enhancement/stories-endpoint. That succeeds and since we no longer have a valid specification ID, we display the first OpenAPI specification in that version.

CORS

CORS may disallow loading OpenAPI specifications from a third-party host. To workaround this we introduce the /api/proxy?url={url} endpoint which fetches the OpenAPI specification and returns it under our own domain.

[ulrikandersen]

trunk? 🙈

[simonbs]

@ulrikandersen Sure! It's added in this commit: e9811e1

[simonbs]

Merging this. The only change after @ulrikandersen's approval is in e9811e1.