Adding support for multiple subgroups

[Maelkum]

This PR adds support for multiple work subgroups.

Subgroups are formed by introducing multiple pubsub topics to which work (roll calls) are published to. Changes are backwards compatible - if no explicit configuration is done, everything should work like before, and all nodes use the default topic as before (blockless/b7s/general).

Operators can instruct their work nodes to make themselves available to specific topics of work by using CLI parameters, e.g. node --topic blockless/b7s/topic1 --topic blockless/b7s/topic3. Note that worker can include a list of topics. The default topic is always included as it's used to transmit some general data like health pings.

To direct work to a subgroup, HTTP requests to the head node execution endpoint (/api/v1/functions/execute) should include the subgroup field, e.g.

{
    "subgroup": "blockless/b7s/topic1",
    "function_id": "{{hello-world-cid}}",
    "method": "{{hello-world-wasm}}",
    "config": {
        "number_of_nodes": 2,
        "result_aggregation": {
            "enable": false,
            "type": "none",
            "parameters": [
                {
                    "name": "type",
                    "value": ""
                }
            ]
        }
    }
}

If the subgroup is not specified, it's published to the general topic, which all worker nodes belong to.

Worker nodes have to have their topics specified on boot, and they will subscribe to all of them. However, head nodes do not. If a head node sees a subgroup it has not seen before, it will try to join that topic and publish to it, even if it's new to it (fanout messages). This makes the head node more flexible and doesn't require it to know a lot in advance.

Function install messages can also be directed to a specific subgroup.

Since we now have multiple subscriptions for ingress messages, main node run loop has been somewhat tweaked (and can be cleaner, but will be in a later PR).

[dmikey]

I think this makes total sense.

There is only 1 execution per topic?

[Maelkum]

I think this makes total sense.

There is only 1 execution per topic?

Oh, I think you're right - I've lost sight of limiting parallelism. But I accidentally went the other way. We limit how many messages we queue (from all topics) but when draining the queue we execute all of them asynchronously. Before (as it should be) we had a limit on how many concurrent executions we had. I'll amend this tomorrow when I'm back.

Good catch! 🕵️