Multiple subscriptions and updating a subscription

[ianswett]

It would be useful to clarify whether multiple subscriptions are valid, and whether there are any constraints on them (ie: non-overlapping).

@kixelated brought up the fact that a relay may have to issue overlapping subscribes because it doesn't know whether a relative or absolute subscription is earlier.

A related question is how one updates a subscription, for example if you wanted to seek closer to realtime, and whether that is just UNSUBSCRIBE/SUBSCRIBE or a new SUBSCRIBE_UPDATE message. There may be cases where SUBSCRIBE_UPDATE is more efficient and better at avoiding sending duplicate objects.

[DanielFay22]

If multiple subscribes are allowed, would that result in multiple transmission of the same data, or a single stream that the endpoint is expected to process/fanout? If the former then I think that there could be some potential for exploit there if a user could effectively make unbounded requests for a single track.

As for the SUBSCRIBE_UPDATE, I think that if updating subscriptions is a supported action then having an explicit message for it is generally better. A side effect of this would be that with a SUBSCRIBE_UPDATE message a failure to update can just continue the current subscription unchanged. With an UNSUBSCRIBE/SUBSCRIBE method, if the SUBSCRIBE request fails the subscriber is left without an active subscription, which is potentially more disruptive.

[ianswett]

When discussed, there was general interest in a SUBSCRIBE_UPDATE, but given we haven't landed a change to add subscription hints, I think it makes sense to land that first.

A related issue is that a server could select the same Track ID for multiple subscriptions, which creates ambiguity about what subscription an object is for, but the object metadata could be used to distinguish in some cases. This is where the dedupe question becomes interesting on a number of levels.

The most problematic issue I can see with multiple subscriptions is that UNSUBSCRIBE and SUBSCRIBE_ERROR use the full track name, but if there are multiple subscriptions to the same track, there's no way to disambiguate between them.

[wilaw]

One point to consider with the multiple subscriptions is relays aggregating subscription requests upstream. Consider the case of a relay which has 5 clients connected (in reality it could have thousands). Those 5 clients make different subscription requests to (), (-1),(-5),(345-375) to the same track and they arrive in that order.

The relay sends the first SUBSCRIBE() upstream. What should it do for the 2nd and subsequent? Using SUBSCRIBE_UPDATE(-1) is not appropriate, since the prior request is still valid. The reality is that an aggregating relay needs to be able to make multiple SUBSCRIBES in parallel to different points within a given track.

Assuming we allow parallel subscribes to happen, then a second problem is how does the relay tell the responses apart? Assume it gets group 345 as the first group back. Is that the first group of the live track, or -1 back, or -5, or just the start of the DVR section?

One solution is that the upstream relay assign a different trackID to each type of SUBSCRIBE request even if its for the same track. It must assign trackIDs based on similar SUBSCRIBE arguments, not just the track name. This does mean that duplicate objects will be sent over the wire. For example group 345 might be sent 4 times, each time with a different trackID. This duplicity exists as long as we allow relative subscriptions. If we restrict ourselves to absolute only , then there will be no duplicity.

[kixelated]

A side benefit for a separate SUBSCRIBE_UPDATE message is that you don't need to retransmit the full track name again.

But yeah I can see situations where you don't want duplicates (relative subscriptions) and others where you want duplicates (seeking with no cache).

I think SUBSCRIBE should take multiple start-end ranges. Deduplication would occur per subscribe ID, and you would use SUBSCRIBE_UPDATE if you want to expand or shrink the ranges.

I don't think we can use multiple SUBSCRIBE messages with the same ID because like Ian pointed out, it's difficult to disambiguate them. What if you want to cancel the relative subscription but not the absolute one? How do you signal that two subscribes should be deduplicated while another two subscribes should be duplicated?

We could have multiple SUBSCRIBE messages with different subscription IDs but the same track ID. Deduplication would be performed per track ID. That opens some edge cases, like if the SUBSCRIBEs are for different track names, which can be avoided by using multiple ranges instead. But I kinda like it.

Although if deduplicate occurs per track ID, this limits the sender's ability to choose an ID.

[ianswett]

Having the subscription ID different from the Track ID seems to add another layer of complexity. I really hope that is not necessary.

I'm not sure what our goals are, but if we want to avoid duplicates then I'd suggest the Producer reject duplicate subscriptions and respond to overlapping subscriptions with the range they are intending to deliver. Combined with unique track IDs, this makes it easy for the consumer to know what to expect.

[afrind]

subscription ID different from the Track ID

It's been brought up before that 'Track ID' could be renamed 'Subscription ID', but there wasn't clear consensus before. Perhaps resolving whether multiple simultaneous subscriptions are allowed to the same track will illuminate the right naming - does it identify a track, or a track + subscribe parameters?

[suhasHere]

Conclusions from IETF118 :

• Support subscribe update when track name alias and subscribe id matches.
• Track Name Alias is compression scheme for Full Track name and copied in every object
• Subscribe ID is transaction identifier and will not be carried in the object header

[afrind]

Individual Comment:

If the OBJECT does not contain the subscribe ID, then we need to update Subscribe OK to resolve any relative groups or objects. Otherwise, a receiver won't know if a particular object belongs to a particular subscription.

There's also no point in sending duplicate objects, since they would all be identical. This is why I inferred that "MUST NOT deduplicate" implies OBJECt includes subscribe ID. Am I misreading this?

[kixelated]

If the OBJECT does not contain the subscribe ID, then we need to update Subscribe OK to resolve any relative groups or objects. Otherwise, a receiver won't know if a particular object belongs to a particular subscription.

Yeah, we need SUBSCRIBE_OK to map from relative -> absolute anyway.

For example, a player wants to start rendering the broadcast 3 groups from latest. If it receives group N, can it render it? Without the mapping, no it cannot.

There's also no point in sending duplicate objects, since they would all be identical. This is why I inferred that "MUST NOT deduplicate" implies OBJECt includes subscribe ID. Am I misreading this?

Yeah... it wasn't clear if OBJECT contains subscribe ID. If the OBJECTs are byte-for-byte identical, why on earth MUST a publisher send duplicate objects?

[ianswett]

Conclusion from the session was to add a frame to explicitly change/update an active subscription instead of relying upon UNSUBSCRIBE and SUBSCRIBE, because there are easy optimizations that can be made and it makes it easy to ensure there aren't gaps in subscriptions.