Nested generations

[lukechampine]

When setting a value inside an array or object, the generation of that node is incremented. However, all example code calls the LITE3_STR macro on the root buffer. Consequently, setting a value within a nested array or object won't invalidate existing lite3_str values.

Suggested fix: Clarify that LITE3_STR must be called on buf + ofs.

[eliasdejong]

This is a good catch and indeed wrong behavior.

Suggested fix: Clarify that LITE3_STR must be called on buf + ofs.

Unfortunately this will not work, because there is no guarantee that the target node will still exist in the first place (it might have been overwritten). Only by always targeting the document root can you guarantee that the message was not modified in any way.

The solution is to always increment the gen count of the document root, regardless of what nesting level the modification was made. It is kind of like marking the entire message as 'dirty', such that all references to it are invalidated.

[lukechampine]

Silver lining: perhaps the generation field could be removed from node entirely, if you only need one global generation for the whole buffer. 🤔

[eliasdejong]

Yes, in fact we've been looking to add a magic bytes header to identify the format. It would consist of TRON + some reserved bytes at the start of every message. Perhaps the gen count field could live inside this header entirely. This would free up space inside the node struct.

Another argument in favor of this design is that we can support non-container types at the root level. Right now, the root type can only be an object or array. But technically this is not spec compliant to JSON, as it supports non-container root types since 2014 by RFC 7159.

I'm not decided yet on whether the header should always be an integral part of the message (or supplied separately in a socket/stream), but if it contains the gen count then perhaps this makes a lot of sense.