Status Communities: Orchestrate static shard allocation and infra deployment

[jm-clius]

Problem

One of the conclusions of a recent status update on the Status Communities scaling MVP, highlighted that the MVP design still require manually configuring static shards for new communities.

Specifically the manual steps include:

• selecting and assigning static shard(s) for a community according to https://rfc.vac.dev/spec/57/
• generating and distributing a private key to community members to sign messages
• publishing the corresponding public key for use by validators
• configuring infrastructure nodes with the selected static shard(s)
• configuring infrastructure nodes with the public key to validate traffic on the shard(s)

This issue tracks work to automate the above.

Implementation

This is a user story that will likely affect several areas of the code and several repos. I've added the outline of the parts that I think will be involved below:

1. Community owner "clicks" to create a community which assigns shard and sets rest of process in motion (UI?)
2. Shard assignment is done from list of available shards, private key generated and public key published (status-go?)
3. Once a community is created, infra nodes are created (or existing ones) configured with a script that subscribes the nodes to the topics and sets the public key for message validation (status-infra?)

cc @John-44 @cammellos

[John-44]

@jm-clius

Community owner "clicks" to create a community which assigns shard and sets rest of process in motion

In the initial MVP implementation, this step shouldn't have any UI (literally zero UI), everything should happen in the background.

In the future post-MVP we will add UI so the user can deploy their own nodes to support their shard, can purchase node services for their shard from 3rd party providers, etc...

Shard assignment is done from list of available shards, private key generated and public key published (status-go?)

Shouldn't each community have it's own unique shard (or shards, if for example we go with a scheme where each community has one shard for messages and another shard for large media)? If each community doesn't have it's own unique shard(s), then:

1. somebody could DDOS a community by DDOSing another community that lives on the same shard. Or somebody could DDOS a target community by creating many communities, until one of the communities they create happens to be assigned to the same shard as the target community (the attacker then uses the community they have just created that's on the same shard as the target to DDOS the target).

2. users of a community who are relaying messages will be relaying messages related to a community they aren't a member of

3. the moment a community is created, it will by definition have very low waku traffic. Some communities will then turn into communities with 10,000 members, some communities may never have (for example) more than 100 or 200 members, and many communities will be abandoned. It's impossible for us to know at the moment a community is created how many active members that community will eventually have. If communitys share shards, then we would need a mechanism to automatically move communities to different shards for load balancing purposes. But if each community has it's own shard this isn't an issue.

4. if shards are shared by communities, a community owner wouldn't be able to purchase Relay and Store node services to support just their community from a 3rd party Waku Relay and Store node service provider

5. it feels wrong from an intrinsic incentive POV that a user who is relaying messages for the communities they are members of to also be forced to relay messages for communities they are not members of. For example, say I'm a member of 5 small, low traffic communities. I would happily run walu relay node services for all of these 5 communities I am a member of because: A) I care about them B) they are all low traffic so I don't mind supporting them with my home bandwidth and C) my home bandwidth is sufficient to run waku relay node services for 5 small low traffic communities that are all on their own shards. Now if each of those 5 small low traffic communities is sharing a shard with a bunch of other communities and that collectively have lots of traffic, in this usecase the intrinsic motivation would stop working for many users.

And probably a bunch more points as well. tldr; I see a bunch of problems if we do anything other than each individual community has it's own dedicated shard or even multiple dedicated shards.

This might be a good subject to discuss on the next call, if we aren't agreed here I think a high bandwidth conversation would be more productive than discussing on a long thread.

[jm-clius]

@John-44 Indeed and I agree! Apologies if my summary above was unclear. I tried to sketch out what I imagine the requirements would be as a starting point, but part of this effort is to clarify the user story and what needs to be achieved. The communities should indeed each have their own unique shard for all the reasons you mention above. The point in that line was precisely to find some method to map each new community to a unique shard so that they don't share it with another community, according to the per-community shard range in the RFC. The MVP breakdown assumed this would be done manually. This issue tracks the requirements to automate this.