[Feature request] Implement MVP of SCS compliance monitor

[mbuechse]

In spite of the description of this issue, the most important step from my POV would be to enable the use of badges, so that we can end the co-existing (and divergence) of Github Actions and Zuul jobs.

• Find out where to host the badges and other files.
• Generate badges.
• Archive logs.
• Generate status pages.

[mbuechse]

Try to get by with Zuul if possible. Check these pointers (again):

• https://zuul-ci.org/docs/zuul/latest/howtos/badges.html
• https://zuul-ci.org/docs/zuul/latest/job-content.html#return-values

[mbuechse]

Research results so far:

• The badge mechanism is per pipeline only, not per job:
  • So we can't distinguish between CSPs, and we still can't distinguish between "did not pass" and "did not finish"
  • Also, we may not want to show the first failure in a row immediately.
• The mechanism for passing artifacts from one job to another is well and good, but we still need storage to aggregate the results.
  • Some kind of S3/object storage would be good.

[mbuechse]

This week I conferred with @scoopex and @bitkeks, and we discussed multiple approaches involving either Postgres or Prometheus. I will need some more discussion. Apart from that, the requirements have become more clear; see SIG Std/Cert meeting notes for more details.

[mbuechse]

Update: I will build a microservice with a small REST-like interface that stores data in a Postgres database. The most important endpoints would be (a) to upload a test report (given an appropriate API key) and (b) to get the certificate status page. A third endpoint might be a Prometheus exporter as described here: https://prometheus.io/docs/instrumenting/writing_exporters/ -- This endpoint can be used by CSPs to monitor whether they are still compliant.

[mbuechse]

Reports should be uploaded in a signed fashion!

[mbuechse]

Placing a copy of @scoopex' hints from the internal brainstorming document here:

Run postgres in a container:

docker run --network=host --rm -v $(pwd)/data:/var/lib/postgresql/data -it --name postgres -e POSTGRES_PASSWORD=mysecretpassword  postgres

Run cli:

pgcli --host localhost --username postgres --password # mysecretpassword

Information regarding JSON columns:

• https://www.postgresqltutorial.com/postgresql-tutorial/postgresql-json/
• https://hevodata.com/learn/postgres-json-query/

[mbuechse]

Just FYI, this issue depended on #510, so I worked on the latter for a while, but now I'm back on it.

[mbuechse]

FYI, the MVP is progressing. Currently, we can post a report like so

$ curl --data-binary @gxscs-report.yaml -H "Content-Type: application/yaml" -H "Authorization: Basic YWRtaW46c2VjcmV0IGFwaSBrZXk=" http://127.0.0.1:8080/reports

and we can ask for the compliance status of a subject like so

$ curl -H "Authorization: Basic YWRtaW46c2VjcmV0IGFwaSBrZXk=" http://127.0.0.1:8080/status/gxscs | python3 -m json.tool
{
    "SCS Compatible IaaS": {
        "v1": 0,
        "v2": 0,
        "v3": -1,
        "v4": -1
    },
    "SCS Compatible KaaS": {
        "v1": 0,
        "v2": 0
    }
}

This is still very rudimentary.

[mbuechse]

Yet another update: while still being rudimentary, the service will now consider manual approval for results that are not "pass" (so either "fail" or "did not finish") -- I still have to add an endpoint for giving the approval --, and for the public, it won't consider unapproved results, and it extends the expiration of the approved ones by a grace period of 7 days (so we have at least 7 days to approve some result or to correct it).

So far, the service only computes the current compliance state; it does not compute the state for the past or keep records of the state for future reference. I think one of these two things should be added because we want to be able to see the 'track record' of the test subject.

[mbuechse]

One can now request a list of non-pass results waiting for approval:

$ curl -s -H "Authorization: Basic YWRtaW46c2VjcmV0IGFwaSBrZXk=" http://127.0.0.1:8080/results?limit=100\&approved=0 | python3 -m json.tool
[
    {
        "reportuuid": "95640a05-2690-4dd0-8c4a-024ad0a5a3d1",
        "subject": "wavecon",
        "checked_at": "2024-03-14T20:54:30.736481",
        "scope": "SCS Compatible IaaS",
        "version": "v4",
        "check": "standard-flavors-check",
        "result": -1,
        "approval": false
    },
    {
        "reportuuid": "7e37ceef-6b69-4212-a70e-e6005e2aff94",
        "subject": "gxscs",
        "checked_at": "2024-03-15T23:54:34.333524",
        "scope": "SCS Compatible IaaS",
        "version": "v4",
        "check": "standard-flavors-check",
        "result": -1,
        "approval": false
    },
    {
        "reportuuid": "7e37ceef-6b69-4212-a70e-e6005e2aff94",
        "subject": "gxscs",
        "checked_at": "2024-03-15T23:54:34.333524",
        "scope": "SCS Compatible IaaS",
        "version": "v3",
        "check": "flavor-name-check",
        "result": -1,
        "approval": false
    },
    {
        "reportuuid": "def374a9-56a9-492c-b113-330d491c58c7",
        "subject": "gxscs",
        "checked_at": "2024-03-16T14:13:53.857422",
        "scope": "SCS Compatible IaaS",
        "version": "v4",
        "check": "standard-flavors-check",
        "result": -1,
        "approval": false
    },
    {
        "reportuuid": "def374a9-56a9-492c-b113-330d491c58c7",
        "subject": "gxscs",
        "checked_at": "2024-03-16T14:13:53.857422",
        "scope": "SCS Compatible IaaS",
        "version": "v3",
        "check": "flavor-name-check",
        "result": -1,
        "approval": false
    }
]

Next step is an endpoint for approving the results.

[mbuechse]

I added a rudimentary scheme for signing the report to be posted and for verifying the signature.

edit the scheme is based upon this post: https://www.agwa.name/blog/post/ssh_signatures

[mbuechse]

FYI: will proceed with deployment soon. Thanks to SovereignCloudStack/docs#200 we have a potential target 👍

[mbuechse]

FYI: I managed to do a test deployment to a VM within the dedicated gx-scs project. Now Kurt will configure a subdomain. After that, I will get let's encrypt to work.