Using Multicast Endpoints

[csperkins]

The interface draft, section 4.1.1, talks about multicast endpoints, but doesn't mention whether such endpoints can be used with Rendezvous().

[gorryfair]

What is your proposal?

[GrumpyOldTroll]

I wouldn't have thought this is unambiguous, so I'm not quite sure what text is needed that would clarify this.

To me at first blush this seems covered by the Rendezvous description in 5.3, between what Rendezvous does in the 4th paragraph:

"If there are multiple Local Endpoints or Remote Endpoints configured, then initiating a Rendezvous() action will systematically probe the reachability of those endpoint candidates following an approach such as that used in Interactive Connectivity Establishment (ICE) [RFC5245]."

And what happens if it doesn't work in the final paragraph:

"An EstablishmentError occurs either when the Properties and Security Parameters of the Preconnection cannot be fulfilled for rendezvous or cannot be reconciled with the Local and/or Remote Endpoints, when the Local Endpoint or Remote Endpoint cannot be resolved, when no transport-layer connection can be established to the Remote Endpoint, or when the application is prohibited from rendezvous by policy:"

Absent an ICE-like mechanism that works with multicast (is there such a thing?) I don't see how someone would expect anything other than this error condition if they did this.

Was there a context to this question? It wouldn't have occurred to me there was another possible answer that needed an explanation to avoid confusion on this point, but maybe there's a case I haven't considered?

[csperkins]

Thinking about this some more, I think there's an ASM vs. SSM question here.

For SSM, the model of receivers call Listen() and sender calls Initiate() makes a lot of sense, and matches the asymmetry of the underlying data flows.

For ASM, things are more symmetric, and Rendezvous() might be the right abstraction.

[gorryfair]

I'm finding it hard to think we need to combine "rendezvous" and "multicast".

[csperkins]

rendezvous is for peer-to-peer connections, and ASM multicast is peer-to-peer

[gorryfair]

Unsure about this... ASM can also be used unidirectionally also, it's just not source-specific. I had other things in mind with Rendezvous. Anyway, others may think differently.

[GrumpyOldTroll]

ASM is deprecated interdomain (RFC 8815). Within a domain I don't think it needs a rendezvous? Or at least I don't know of anyone trying to use it that way.

I take @csperkins point that it's possible someone could want to build something for a peer to peer multicast use case. If they do so, I'm guessing this would include some kind of ICE-like underlying rendezvous protocol that can support multicast, and at that point a rendezvous operation might reasonably start working.

To me this looks like a function of whether there's an ICE-like protocol that can perform a rendezvous for the endpoints, so it's correct to avoid calling out non-support of rendezvous inside the multicast endpoint description so that it doesn't need to be specifically updated if the situation changes, though in practice today it will fail due to being unsupported if someone tries it with today's rendezvous protocol implementations. (At least, I assume so? Haven't tried it...)

[csperkins]

I read Rendezvous() as setting up a peer-to-peer connection, not as necessarily running a specific rendezvous protocol. For unicast, it needs something like ICE, I agree. For ASM, I was assuming a Rendezvous() would just turn into a multicast group that I could send to/receive from with no additional signalling, whereas SSM multicast needs explicit send-to-group and receive-from-group calls. Do we have different models for how this works?

[GrumpyOldTroll]

I don't understand the claim that SSM would require explicit send-to-group or receive-from-group calls. To me if you've made a local endpoint with a group address, the connection is non-sending, and if you made a remote endpoint with a group address, the connection is non-receiving.

With that said: a search for "multicast stun" did find what seems like a non-IETF STUN usage proposal that does something with multicast, so maybe somebody's using it? I have no idea: https://ieeexplore.ieee.org/document/7275794

Anyway, my starting assumption was that if you make a multicast group endpoint and give it a "withStunServer" and use it in a preconnection that you Rendezvous(), you'll get an error unless the stack and the server thinks it knows what to do with that, in which case sending or receiving will just work if there's a remote app that did the opposite.

In the (unlikely?) case the stack does support it, like with unicast, I guess I'd assume if you did a sending preconnection (remote group address), you'd want to do the withStunServer on the remote endpoint, and if you did a receiving preconnection (local group address), you'd do the withStunServer on the local endpoint.

I guess also I'd assume that if you'd like bidirectional ASM, you'll want to make 2 separate connections, one for sending (with the group address as the remote endpoint) and the other for receiving (with the group address as the local endpoint), and on the receive one you won't provide a remote address.

AFAICT all the above is sort of implicit in what's already written in the spec, and I'm not sure it needs spelling out if that sort of usage would work in a stack that cared to implement it.

But I do worry I'm missing something, and it could be annoying to find out in 5 years that the API structurally doesn't work with any kind of Rendezvous because it was specified wrong. And on that score I'm worried that I might not be fully understanding what happens in a Rendezvous. And when I tried to follow the pseudocode example in section 3.1.3, I noticed there's a complicated bit that's just comments as an exercise for the reader, which perhaps contains some assumptions I haven't really understood:

// ...Send the ResolvedLocal list to peer via signalling channel
// ...Receive a list of RemoteCandidates from peer via signalling channel

Anyway, does anyone know what the STUN/TURN servers actually do here? Taking a step back from what I'd expect out of an API that did support it, I'd think defining a new usage for STUN is probably out of scope for this doc, if it's not written up and deployed somewhere already.

[csperkins]

For SSM, the model of Initiate() to create a send-only connection and Listen() to create a receive-only connection makes sense and matches the semantic of the multicast group. The sender creates the group, and everyone else listens in.

For ASM, the underlying group is peer-to-peer and members can both send and receive. It's symmetric. To me, that doesn't fit the asymmetric model of Initiate() and Listen(), and seems a better conceptual fit for a Rendezvous() semantic.

A Rendezvous() will typically devolve to STUN/ICE for unicast, but I don't see that it has to use STUN, and I don't see that it would use STUN for multicast. The key difference is that Rendezvous() is peer-to-peer while Initiate()/Listen() are client-server, and I view ASM as peer-to-peer and SSM as closer to client-server.

[GrumpyOldTroll]

Ah, ok, thanks for explaining. I guess the idea here is to have bidirectional ASM represented as a single connection object instead of 2 separate ones?

I guess that's a way to consider it, but I think this conflicts with the local/remote endpoint concept, more than it conflicts with Initiate/Listen, with the problem being that you both receive with the group address and send to the group address.

In general, for a UDP connection it's possible to just Initiate, followed by both Send and Receive, correct? So the trick here if we need a bidirectional Connection for multicast (whether ASM or SSM) would maybe be to do something like "preconnection.BidirMulticastEndpoint(endpoint).WithASM()" (with default as SSM-only, pulled from the addresses in the remote endpoint and using ASM only if WithASM was called), and then call Initiate, with both Receive and Send available.

That would be doable, either now or as a later (or optional) extension if it seems useful enough. But either way, Initiate() seems to me like a better fit a sending or bidirectional UDP than Rendezvous().

[csperkins]

Ah, ok, thanks for explaining. I guess the idea here is to have bidirectional ASM represented as a single connection object instead of 2 separate ones?

Yes.

In general, for a UDP connection it's possible to just Initiate, followed by both Send and Receive, correct?

I guess so, although UDP also doesn't fit well with the model - the actual semantics depend on the upper layer protocol implemented over UDP, and plain UDP isn't useful except as a substrate. For QUIC you want Initiate/Listen, but I'd argue that Rendezvous is the right model for unicast RTP since it's naturally peer-to-peer and wants NAT traversal.

Initiate() for bidirectional UDP would work, but I'm not sure I agree that it's right.

[GrumpyOldTroll]

Ok, after one more pass at reading Rendezvous text in the draft I'm starting to get it. That's an interesting point about generalized peer to peer connections, I don't think I had grasped that difference between this API and the way sockets are used.

Backing up a bit further, I think I'm also finally understanding something I missed in an earlier comment from more than a year ago when somebody tried to make the case that Listen() should be the call the sender side of a multicast connection makes, and Initiate() should be the receiver side. That position makes more sense to me now that I'm finally realizing this API is trying to normalize the concept of clients vs servers vs peers at the app layer, rather than describing socket-like things to do.

I think maybe there is a wider issue here that this naming scheme is fundamentally confusing when you're looking at a non-TCP-like app. Maybe "Listen()" should be renamed to "ServerInitiate()", "Initiate()" should be renamed to "ClientInitiate()", and "Rendezvous()" should be renamed to "PeerInitiate()"?

Under that naming scheme it would make complete sense for a multicast sender doing a ServerInitiate to get an immediate unidirectional sending Connection object, whereas it makes no sense at all to get that behavior out of a call named "Listen".

I guess this probably needs a list discussion. Now that you've got me understanding what the viewpoint was that drove these comments, I think the right thing to do is probably to fix all 3 drafts to make it explicit that the these APIs are designed to support 2 different kinds of connections (Client-Server or Peer-Peer) with 3 different roles, and to start the connection with a call that explicitly declares the role this connection object will be playing, and to name the initiation calls to match that.

If that ends up matching with the wg consensus, then I agree the multicast section needs to be refactored to make ServerInitiate() be similar to the action from today's Initiate(), and ClientInitiate() to be the action from today's Listen(), and that in this case a PeerInitiate() would be good to specify also, as well as adding a call to add a GroupEndpoint (with an ASM/SSM selector) that can be used to support bidirectional multicast traffic. To me this sounds like a design improvement to the API, and thus the right thing to do.

If the group vetoes that direction and chooses to stick with the current naming scheme that describes things that map onto socket behaviors, then I think it's better to stick with the current approach.

Also, the claim that UDP is only useful as a substrate seems at odds with the text in the current draft. The explanation of what a connection is explicitly mentions plain UDP as one possibility:
https://ietf-tapswg.github.io/api-drafts/draft-ietf-taps-arch.html#section-4.1.2

"Connection: A Connection object represents one or more active transport protocol instances that can send and/or receive Messages between local and remote systems. It holds state pertaining to the underlying transport protocol instances and any ongoing data transfers. This represents, for example, an active Connection in a connection-oriented protocol such as TCP, or a fully-specified 5-tuple for a connectionless protocol such as UDP."

This reinforces my suspicion that this issue is not specifically about multicast, but rather about the confusing names of the connection initiation operations, where multicast is one example that highlights the problem well.

I'm glad you raised the issue, I think I'm finally starting to understand what you mean and I think there is a very good point in here that rates to make this API better :) But we probably need to take this to the list, I don't think this should be taken as a minor "fix up the multicast section" point, if I'm understanding it correctly.

[mwelzl]

There are several things in this discussion that I don't understand (just because I don't know enough about multicast), so take my answer with a grain of salt. This said, I believe I understand that your proposal of ServerInitiate() etc. can make sense and be "cleaner" (more general). I do understand that, as you say, it makes no sense at all to obtain an immediate unidirectional sending Connection object from a call named "Listen".

However, for unicast client-server communication, "ServerInitiate()" is an odd term, IMO. I think it's common for people to think of an "active" open and a "passive" open. The key distinction here, at the interface level, is that "active" sends something towards a destination - you give it a destination address, but for "passive", you don't. So then, "ServerInitiate" sounds like it might just be "listen", yet it does something active, i.e. it MIGHT signal something to a destination (though, when used as a unicast server "listen", it won't be given a destination address, so that couldn't happen... but why is it not called "Listen", then?). "Rendezvous" is of course more symmetric, but the term makes it explicit that this is neither a "normal" active open nor a passive open. Terminology like "ServerInitiate" is just strangely ambiguous in a unicast case, IMO.

Technically, if I want to rename API primitives for such usage, I can easily solve this by introducing new Actions like ServerInitiate etc., which do nothing but hand over the calls (i.e., ServerInitiate calls Listen, etc.). Wouldn't it be a way out to suggest this as a solution for implementers?

If this is broader than multicast, this suggestion can be placed outside the multicast section. E.g.: make an appendix called "Alternative Action and Event Names for Connection Establishment", and refer to it from mulitple places - in the "normal" establishment text, there could be statements like this: "The terminology here (Initiate(), Listen() etc.) is based on a unicast model. There are communication models where these terms are misleading: for example, (...something...). It is therefore recommended to additionally offer the same Actions with a different set of terms for non-unicast communication. Appendix XX introduces these Action names and their mapping."

Would this be a viable approach?

[GrumpyOldTroll]

I guess in this model I would think ServerInitiate() calls Initiate() if there's a multicast remote endpoint, but calls Listen() if there's a unicast local endpoint and no multicast remote endpoint. I guess I doubt it's worth it if it's done as an appendix instead of making the concepts of Server and Client and Peer more central in the API.

Maybe I should look for a smaller scope that's more multicast-specific. I guess another alternative would be to add a few things to the Preconnection just for Rendezvous, like "AddBidirectionalGroup" that takes a remote endpoint with a group address, then Rendezvous would make Send use the group address.

I guess another reasonable alternative would be to just say "Rendezvous is not supported for multicast endpoints. Bidirectional communication should be implemented with separate connections for sending and receiving."

[mwelzl]

Hm, would "Rendezvous is not supported for multicast endpoint" take all this complexity away? Then this would sound like a good way to go, for me - based on your earlier argument that interdomain ASM is deprecated. But ... I'm stepping on very thin ice here. Better for others to chime in.

[GrumpyOldTroll]

Well, it wouldn't change the fact that the API does not highlight the different kinds of roles of an app in a connection (like server or client), but rather refers to socket behaviors such as listening, and so I think the API is not as effective as it could be. It also would add a little bit of complexity for NORM implementors if we ever get any, because the NACKs can be sent between peers for data recovery according to the spec IIRC, and for a use case like this they would be obliged to use multiple unidirectional Connection objects instead of a single bidirectional object. There could be other similar use cases where between-peer communication within the domain (most likely on the same LAN) could be helpful.

However, that's kind of a corner case that doesn't hurt all that badly, and this change would be much smaller for the current API.

I guess I'll add that as a PR and call it a fix for this issue, as there wasn't much traction on the idea of normalizing the app mode behavior. Thanks for the comments.

[mwelzl]

You say "there wasn't much traction on the idea" but it was just me. Maybe people haven't been following this conversation - it may still be worth bringing the general API change idea to the list, as you first suggested.

[gorryfair]

This doesn't seem to be converging on something that most people really need - so do you think we need to make a move to do something small in a PR that avoids this becoming a rat-hole.

[GrumpyOldTroll]

Yes, I submitted a PR (originally misnamed for solving 872 instead of 852):
#871

I do still think it's a fine idea to make connections explicitly play a Server, Client, or Peer role instead of inferring them from the socket activities that are named, and that the raising of this issue helped me to understand that approach to thinking about the possibility of an API with such a structure. However, to make such a change just to support this use would be the tail wagging the dog, I think.

A separate change could be proposed under a separate issue (along with clarification that a Peer connection may or may not have to use an underlying rendezvous protocol, as opposed to calling it Rendezvous and leaving that explanation implicit). I would support such a suggestion and offer to add support for the Peer usage in multicast if it got traction. But I'm not going to push it myself if the others I polled offline don't see the utility, and I think we can live without the use case as a practical matter.

And without such a change, I think offering to "Rendezvous" on a multicast connection (just meaning establishing 2-way peer to peer connectivity, without reference to an underlying rendezvous protocol that could support it across nats the way the ICE-based stuff does for unicast) would add unnecessary confusion if done without a broader and more explicit change to make the role-based (as opposed to activity-based) way of thinking about connections more central in the normal API usage.