Proposal: Updated User Flow

[VoyTechnology]

Preamble: this is a very long term goal. This issue is what I propose for the general interaction with the system. We still have a long way to go before to clean up the existing code.

New Proposed User Flow

In this case the user would be using command line to run everything, useful with servers etc.

1. Downloading
   curl https://pp2p.github.io/sh -sSf | sh (Taken from https://rustup.rs, url subject to change)
   This would ask to install pctl the new command line tool, and pfsd, the exising (but modified) binary.

2. Joining a new cluster
   There are 3 users. User 1 (u1) Creates a new filesystem on their machine (m1)

   u1@m1$ mkdir topsecret
   u1@m1$ pctl init ./topsecret
   Created a new Paranoid Filesystem in /home/u1/topsecret
   Your address is: 89.100.100.134:42624 (UPnP) and 192.168.1.123 (local)
   Use token 9aq97g87ac to connect
   
   u1@m1$

   Second user wants to join

   u2@m2$ mkdir topsecret
   u2@m2$ pctl join ./topsecret
   Address to connect: 192.168.1.123
   Token: 9aq97g87ac
   Successfully connected. u1 will have to confirm your request
   
   u2@m2$

   The second user tells about the sent request, u3 connected in the meantime.

   u1@m1$ pctl peers
   1. u3@192.168.1.125 - CONNECTED
   2. u2@192.168.1.124 - NEEDS APPROVAL (0/2)
   
   u1@m1$ pctl peers approve 2
   Approved u2@192.168.1.124 (1/2)

   Since u3 already joined the cluster, its necessary to get approval too

   u3@m3$ pctl peers
   1. u1@192.168.1.123 - CONNECTED
   2. u2@192.168.1.124 - NEEDS APPROVAL (1/2)
   
   u3@m3$ pctl peers approve 2
   Approved u2@192.168.1.124 (2/2)
   
   u3@m3$ pctl peers
   1. u1@192.168.1.123 - CONNECTED
   2. u2@192.168.1.124 - CONNECTED

   u1 is not the owner of the fs, everybody is.

3. Leaving and Rejoining
   Since the user is already approved, they can just rejoin the cluster. I am not sure would they be able
   to make changes without consensus, this might be a super safe mode or something.

I am not yet sure how leaving the cluster would look like.

Technical details

Since discovery server is essentially gone, it means that we might have a problem trying to get the UPnP address, local should not be a problem. You can see that there no ports. Defaults are used with ability to override. Please correct me on this one @GoldenBadger , but since we have the exact control over who and when is joining the cluster, distributing the keys should be simpler.

Creation of keys would be behind the scenes with default values being used (say 30 days key validity, getting the IPs and hostname of the machine)

The pctl is only the a simple cli module, the actual connection is being done in pfsd/api, where the code can be shared between pctl and pgui (gui) through gonode (as suggested by @butlerx), and instead of using Unix sockets it would be part of gRPC.

You might have noticed that there is only one pfsd controlling it. That is right. I think if we have pfi be a new spawned process for individual mounts. This would allow to easily remove pfi where fuse is not available (windows), and spawn something different. It is a compromise as it would mean that if pfsd fails, everything dies. The token is valid for only a short time (10minutes?) is valid for one time use (not shown in example above), And identities with specific mount, so there are no problems with multiple mounts.

As comments and suggestions are added I will update the issue.

[ConorGriffin37]

I like this idea and it seems much simpler than the way things currently work.

Semi Unrelated to user flow thoughts:
Will there be a time out on an attempt to join? A lot of the complexity for chunking and distributing the keys came from the fact that multiple nodes could try to join at one time. If you timeout the attempts to join, and make the join token one time use only you can avoid a lot of complexity of multiple key sets being distributed at once.

Another thing to consider is right now we have not integrated adding a node to a configuration in raft with distributing the key pieces. Since these are two separate steps you could end up with the situation where a node is considered part of the cluster by the key management system but not by raft. When reworking the logic for nodes joining the cluster, you should try to take this into account.

[ConorGriffin37]

I like this idea and it seems much simpler than the way things currently work.

Semi Unrelated to user flow thoughts:
Will there be a time out on an attempt to join? A lot of the complexity for chunking and distributing the keys came from the fact that multiple nodes could try to join at one time. If you timeout the attempts to join, and make the join token one time use only you can avoid a lot of complexity of multiple key sets being distributed at once.

Another thing to consider is right now we have not integrated adding a node to a configuration in raft with distributing the key pieces. Since these are two separate steps you could end up with the situation where a node is considered part of the cluster by the key management system but not by raft. When reworking the logic for nodes joining the cluster, you should try to take this into account.

[VoyTechnology]

Well, before a new node joins it has to be approved. I haven't looked into details, but that would mean we would exactly know when a new node will join, and maybe have time to prepare for it in some way?
Yes, the token would be one use only, and correlated to one shared filesystem.

Do you mind creating an issue for that? I am not exactly sure the details of how it is (or was supposed to) work

[VoyTechnology]

Well, before a new node joins it has to be approved. I haven't looked into details, but that would mean we would exactly know when a new node will join, and maybe have time to prepare for it in some way?
Yes, the token would be one use only, and correlated to one shared filesystem.

Do you mind creating an issue for that? I am not exactly sure the details of how it is (or was supposed to) work