Propagating sign-in/-out information

[bilderbuchi]

@bklebel writes:
According to the latest state of the PR #38 (and my latest comments therein, e.g. this comment), I think we should boil down the Control Messages to

• SIGN-IN
• SIGN-OUT
• SEND_DIRECTORY - a request for the Directory of a Coordinator, subject to change of the message type signature, possibly we would prefer sth like "GET" or "REQ" over "SEND"(please). What do you think? Or was this the CO_TELL_ALL?
• CO_UPDATE - a Coordinator sending its local and global Directory

In the comments to #38, relating to the discussion in #44, I proposed to simplify the messages so that Coordinators do not announce individual SIGN_IN/OUT actions, but simply send their Directory to all connected Coordinators when it changes.

• If a sign-in/-out happened within the local Namespace, the respective Coordinator sends the now updated local Directory (and its current global Directory) to all connected remote Coordinators: CO_UPDATE
• If a SEND_DIRECTORY request is received by a Coordinator, they reply with their Directories in the CO_UPDATE message.
• Coordinators should only consider changes to their own global Directories which come from Coordinators of the corresponding Namespace, i.e. if Co1, Co2 and Co3 (with Namespaces N1, N2, and N3, respectively) are present in the leco network, and Co1 sends a CO_UPDATE message to Co3 which contains updates within the N2 Namespace which Co3 does not have yet, Co3 should, rather than trusting Co1 blindly, at least ask Co2 again for its Directory.
• Whenever a Coordinator receives a CO_UPDATE message, it updates its Directories, global and local, and possibly starts to sign-in to new remote Coordinators, as discussed

Originally posted by @bklebel in #29 (comment)

[bilderbuchi]

• Coordinators should only consider changes to their own global Directories which come from Coordinators of the corresponding Namespace, i.e. if Co1, Co2 and Co3 (with Namespaces N1, N2, and N3, respectively) are present in the leco network, and Co1 sends a CO_UPDATE message to Co3 which contains updates within the N2 Namespace which Co3 does not have yet, Co3 should, rather than trusting Co1 blindly, at least ask Co2 again for its Directory.

On this I have to ask "why?" -- why do we stop trusting our coordinators? If Co3 does not trust Co1 about its Directory, what does it trust Co1 about, and why?

[BenediktBurger]

Let us start with the requirements:

• Coordinators have to route messages, so they need a list of connected Components with their identities (local Directory)
• Coordinators need a list of Coordinator addresses, such that other Coordinators may connect to them (currently part of local Directory)
  • Coordinators must provide (at Coordinator sign in) a list of Coordinator addresses
• Coordinators shall provide a list (Full names) of all Components in the Network (global Directory)
• All Coordinators have to have an up-to-date global Directory copy, so:
  • They need an initial version of either one global Directory or all local Directories, in order to start their own global Directory
  • They need to have an update, whenever a Component signs in or out

A few messaging options:

1. We can have a single "directory" message, which sends the global Directory of a Coordinator. This is also used during Component sign-in. Advantage: We do not need regular exchanges of Directories (each sign-in keeps all Directories in an up-to-date version).
2. We have the full "directory" message and "differential updates", this invites for regular exchanges of the directories among the Coordinators, just in case.
3. We have a "local_directory" message (destined for communication between Coordinators), an "update" message, and a "global_directory" message (for Components)

A few updating options (how does a Coordinator update its global Directory):

1. It uses whatever global Directory it gets (which might be outdated regarding some other Coordinator)
2. It uses only the local Directory (or the corresponding part of the global Directory) of the sending Coordinator

I'm for option 2 ("distrusting other Coordinators") due to timing issues. For example:

• Component CA signs in to Coordinator 1, which updates its global Directory
• Coordinator 1 sends its changed global Directory to Coordinators 2 and 3
• In the meantime, Coordinator 2 has an update of its own and send its changed version (still without CA) to Coordinators 1 and 3
• Coordinator 3 receives the version of Co1 first, adding CA, then Co2's directory (without CA) arrives, such that CA is removed from the global Directory.

In order to keep it simple and reliable, I vote that only a Node's Coordinator is a source of truth regarding that Node's Components.
Whether that information is extracted from that Coordinator's global Directory or a special "local directory" message, is secondary.

[BenediktBurger]

• How do we transmit the Coordinator addresses (at Coordinator sign-in)? They do not fit into the normal Directory list, as they contain additional information (address)
• Where do we put these Coordinator addresses: local or global Directory? In a new Directory ("Coordinator Directory")

My proposal:

• Add that new "Coordinator Directory" (or "backbone Directory") as its own term (as it is needed on the technical side anyways)
• Send this new Directory as well, when the local or global Directory (in the current PR) is sent
• Use "updating option 2": Only use the Component information of the relevant Coordinator
• Regarding amount of message types: I'm open for any solution. At one hand, I like the smallness of diff updates, on the other hand, I like the simplicity of sending always the (local) directory, preventing another "heartbeat" style.

[bilderbuchi]

In order to keep it simple and reliable, I vote that only a Node's Coordinator is a source of truth regarding that Node's Components.

thank you for the illustrative example. I agree with this point.

How do we transmit the Coordinator addresses (at Coordinator sign-in)? They do not fit into the normal Directory list, as they contain additional information (address)
Where do we put these Coordinator addresses: local or global Directory? In a new Directory ("Coordinator Directory")

I think both can be done in one sweep: the local directory already is, according to your definition above, a

list of connected Components with their identities

So, in essence, a dictionary/key-value store. The coordinator addresses associated with their names are a key-value pair. Above you already say that coordinator addresses are "currently part of local Directory". This looks like it solves the problem? 🤷

[bilderbuchi]

I'd rather move towards fewer directory types, not more. >.<
We could have one "dictionary" directory with full names in the keys and coordinator addresses/local Component identities/nothing as values as appropriate.
We trivially can filter out local entries (all starting with <localNamespace>.), coordinators (all ending with .COORDINATOR), or global entries (the rest).

Note: this is already moving a bit toward specifying implementation. How about we specify a set of behaviours/requirements, and let the implementation(s) hash out how they solve it?

[bilderbuchi]

One final point: If we only trust coordinators with information about their local components, but not about global components (because of the timing issue you bring up), we don't need to distribute the global directory at all, only the coordinator information and local names.

[BenediktBurger]

Let's frame it differently: we want to use the information with the least intermediaries.

I always thought, we exchange the local information among Coordinators and they offer the global information to end users.

Regarding the content: For the Coordinator, the local Directory contains the identities. That information does not matter to anybody else, therefore, even the transmitted local directory does not contain more, than the names.

Only the Coordinators need addresses transferred.

Let us define the Directories without the identity (implementation detail). That is even in the direction of discoupling the transport layer from the message layer (you could build a tcp transport layer without those identities).

[bilderbuchi]

Sounds good! (whole comment).

Let's frame it differently: we want to use the information with the least intermediaries.

"May", "Should" or "Must"? 😜