consecutive version diff'ing

[dret]

i am woondering if this has been implemented, and if so, how robust it is: "The hub MAY reduce the payload to a diff between two consecutive versions if its format allows it."
even if we assume that a hub never misses a topic update, the same cannot be said for subscribers, so for them getting diffs may be problematic. they might also not even notice that there is a problem, unless the versions carry ETags strong enough to verify the versions, and their ability to be used for diff purposes.

[aaronpk]

afaik the implementation of this has been around sending individual items from a feed, such as entries an in RSS feed.

[tonyg]

This question touches on thorny issues of content model, "reliability" level, consistency properties etc.

What is PubSub for? Transient, best-effort notifications where occasional message loss is fine and expected (atop which something else may be built), or "reliable", high-value notifications, where message loss is frowned upon and serious energy is expended on making sure notifications aren't missed, or duplicated?

Since "guaranteed delivery" isn't a thing, it might make sense to explicitly go for a best-effort/unreliable framing, and to leave "reliability" to upper levels. However, it could be a big help if some indication of lost or duplicate notifications were able to be provided! Could a hub be asked to assign a sequence number to its notifications? (It would have to be scoped very carefully.) That way at least consumers could detect gaps and duplications in the stream, and take application-specific appropriate action.

[julien51]

@tonyg I believe enforcing reliability can very well happen at the "content" level and not at the protocol level. The best solution I found for this was to have some kind of sequential id in each "payload" that the hub passes through. The subscriber can then easily identify if it's missing parts.

[tonyg]

@julien51 Thank you! Your remark helped me realise that what I was thinking of is best left for later, and out of scope for this round of the spec. Sorry for muddying the water. Keeping this spec best-effort/unreliable seems very much the best option.

[julien51]

NP!