Updates for SVS v2

[pulsejet]

No description provided.

[yoursunny]

@justincpresley proposed a more space-efficient encoding that omits the TLV-TYPE and TLV-LENGTH of StateVectorEntry.
I'd suggest incorporate this into the new protocol version.

https://github.com/justincpresley/ndn-sync/blob/606eedea7a20be7502dc474a14685ba8a3d03e93/pkg/svs/state_vector.go#L179-L192

[yoursunny]

Under TLV evolvability guidelines, it's not good to enforce a particular TLV being the first one, without a strong technical reason.
It's better to accept any ordering, with a recommendation that StateVector appearing before MappingData.

[Pesa]

with a recommendation that StateVector appearing before MappingData

why the recommendation? unless a specific order is mandatory, a receiver cannot make any assumptions anyway, so why bother with a recommended order in the first place?

[yoursunny]

The recommended ordering improves Interest aggregation. It is not enforced.

[Pesa]

I'm not sure interest aggregation is a concern here? it's a signed interest anyway...

[yoursunny]

Signing with HMAC key and skipping all random fields in SigInfo => aggregation possible.

[Pesa]

We're not debating whether it's possible or not, but whether it's a concern. I'm saying "why bother". Moreover, this may even be application dependent, and the order itself (even if just a recommendation) doesn't matter as long as it's consistent.

[Pesa]

Just to be clear: I'm not against suggesting a specific order in SVS, I just don't think it's very useful or makes a meaningful difference in practice. A consistent order across all ApplicationParameters is more important.

[pulsejet]

I don't care honestly. But it's easier / better to recommend a order than recommending consistency for interoperability between vendors.

[justincpresley]

@justincpresley proposed a more space-efficient encoding that omits the TLV-TYPE and TLV-LENGTH of StateVectorEntry. I'd suggest incorporate this into the new protocol version.

https://github.com/justincpresley/ndn-sync/blob/606eedea7a20be7502dc474a14685ba8a3d03e93/pkg/svs/state_vector.go#L179-L192

I discuss this at length here.

In short, I propose an addition to the specification of NDN: making the concept of a TLV List explicit and heterogeneous. My encoding improvements to SVS simply follow this proposal. Changing only SVS could mislead other protocols.

[Pesa]

In short, I propose an addition to the specification of NDN: making the concept of a TLV List explicit and heterogeneous. My encoding improvements to SVS simply follow this proposal. Changing only SVS could mislead other protocols.

When you say "specification of NDN", are you referring to the network-layer packet format? Can you make an example where this new encoding could be used?
(this may be a bit off topic)

[justincpresley]

Yes to the packet format. I assume TLV Lists are not heavily used in creating the concept of a NDN packet and thus left unstated.
However it needs to be defined somewhere preferably wherever people learn about the building blocks of NDN.

I suppose just documentation? added to the end of the TLV section or a page like it.

Anywhere where a list is involved. A Name is naturally a heterogeneous list already. Possibly only useful in custom TLVs. Statevectors, mapping info, in ndn-cert when agreeing upon algorithms, etc.

[justincpresley]

My thoughts for v2:

• Rather than a vector of [NodeID : Seqno], I believe a more NDN-inspired approach would be a vector of [Dataset : Seqno]. This change allows a node to control/update multiple datasets within SVS. While you can achieve this with PubSub, I find this extremely inefficient. If you already working with sequential data, there should not be encapsulation for the purpose of removing the sequential part, only to encapsulate sequential data. It seems silly.
  With this change, wording needs to be corrected across the board, I have seen participant/source/nid/nodeID throughout the different papers and repos.

• Suppression Curve. While this is not complete as ndn-svs states it will later be adjusted based on the number of Nodes, I believe it should still be included in some way.

• Space-efficient Encoding. I do not have a preference whether this is added or not. Even if it is not added, it can still be provided by libraries as a option (not enforced).

I am a fan of many smaller revisions/releases like NFD has had. This especially helps in this case as many libraries implement this protocol, and it gives them time to adopt changes. I think the partial StateVector work could be included in v3 if it does not make it in v2.

[pulsejet]

The suppression curve is now part of the spec.

I prefer not compromising soundness of structure for space efficiency. In fact the first version of SVS used a more efficient encoding but it's more complicated to parse and evolve.

[zjkmxy]

How about we fix the state diagram as well?

[justincpresley]

When entering suppression state, is it still relevant to check against time remaining?
i.e. in the diagram min(remaining_time, update_delay())

I would think no right? due to the suppression curve change.

[yoursunny]

Remember to update the date in PubSubSpec.md.

[yoursunny]

Normally, MappingData is generated in response to a query Interest, which includes low-seq and high-seq fields.
With delivery optimization, how could the publisher predict what low-seq and high-seq is wanted by the subscriber?