Representing backwards-incompatible data as notes / remarks

[stuartpb]

As previously described (briefly) in #7, I've been thinking that the API (and builds or whatever) will support older, incompatible schema versions by translating whatever incompatible data is present into whatever form most closely reflects the data of the tip representation, in whatever form most closely resembles the structure of the old schema, through a series of migrations (defined in some as-yet-unspecified way - I'm thinking a graph of least-transformations between versions or something like that, where you can specify a converter from 0.2 to 0.1, 0.3 to 0.2, 0.4 to 0.2, 0.4 to 0.3, and then a request for 0.1 when the tip is 0.4 would use 0.4=>0.2 and 0.2 => 0.1).

However, there's the matter that the data that doesn't easily break down / translate into the old schema can potentially be crucial to understanding the aspect it was meant to convey (for instance, a lot of notes right now are "Go to the page specified by the URL, then you have to click this certain thing", and these might later be expressed as data describing an element post #126): in this scenario, it'd make sense to have some kind of accompanying human-readable description (like *.notes, but see #137) of what this data describes: the idea being that the aforementioned migrations would include such a translator.

There's also the notion of representing the underlying data itself (as some kind of YAML dump, for readability) as a form of localization that can be falled back to instead of en_US (see the "locale policy" currently at https://github.com/opensets/domainprofiles/wiki/Style-Guide#content-locale, originally described in #12 (comment)). In this case, it should probably be made clear that end consumers are not intended to re-parse this data: if they need this data, they should upgrade their implementation to use the newer schema. (In fact, to me, that concept is an argument in favor of just not including the data at all, similarly to what I'm talking about with proposed-structure profile data in #137).

[stuartpb]

Yeah, I'm into the idea of representing data that can be represented backwards-compatibly (for instance, a field that was a scalar in the previous schema version and is now an array, and the array in a given profile has only one element) as the kind of data it was, and representing data that can't be represented backwards-compatibly through a human representation like notes or further (or caveats) (for instance, further values in the array may be represented with a sibling further note that says "Also requires x" or whatever).

[stuartpb]

This really comes down to what backwards-compatibility is, how notes etc. are meant to be accepted, and what's considered a "breaking change" vs. a compatible one. For instance, if v2 has a common restriction frequently described in a freeform field like notes, then there's an added field to represent that restriction, and the notes are transformed into that field - is that truly a compatible change?

You know, I think this is what minor versions are going to be for: these transitions will apply to downconvert information added in subsequent minor versions, but any new fields... hmm.

[stuartpb]

😣 This still needs more consideration, but is really only a sub-discussion of #7, and not one that really has to block v0.1.0, as it sounds like this is going to only be factor once major versions come into play (or at least only once there's more than one version to consider compatibility across, so v0.2.0 at the earliest).

I'm taking it off the v0.1.0 milestone, attaching it to #7 as an explicit comment, and then let the chips fall where they may.

[stuartpb]

Closing in favor of further discussion happening in opws/opws-guidelines#2.