Section on managing schema versions?

[naught101]

I'm currently trying to come up with a solid way to manage schema versions, and to manage migrations/upgrades of objects matching those schemas. I started a discussion at json-schema-org/json-schema-spec#19 , but it should more properly be discussed here, I think, so copying the contents over:

---

I wonder if it might be worth addressing a "recommended versioning/upgrade pathway methodology" or similar in the book.

I'm trying to implement something like this in a private project, and thinking of using something along the lines of:

• A single repository of schemas and upgrade code that can be used stand-alone or as a library in multiple languages
• Schema versions for each object type in subdirectories named after the version.
• A current_version file that keeps track of the current stable version of the schema for each object type.
• A previousVersion tag in each version of the schema (e.g. v1.1.0 might have previousVersion: 1.0.7)
• A requirement in each schema that all objects have a schemaVersion property.
• Upgrade function for each version change that can be imported into my apps, with a wrapper script for stand alone operation, in each language I care about (e.g. upgrade_v1.1.0_from_v.1.0.7.py, upgrade_v1.1.0_from_v.1.0.7.js), which handle the upgrade, and which can be chained (with pre-and post-validation).
• Some master scripts which detect the current stable schema version, object schema version, and get and run the upgrade function chain.

This feels like a sensible idea, because it keeps all of the validation and upgrade logic in the same place, and allows each part to be used as-needed in each app (in particular we have a large object store that will have objects with various different versions that need upgrading as they are accessed and used by other apps in the system).

I thought I'd chuck this up here because it might be useful as a basis for a more sensible methodology recommendation, and for the possibility that I might get some feedback on it :)

[tejaswiniVadlamudi]

Do we have any update on this?

[cpersico]

I also think it would be useful to have a version field

[koppor]

Regarding the aspect "which version number to assign to a JSON schema", a stackoverflow question pointed me to SchemaVer. In short, it recommends following:

Given a version number MODEL-REVISION-ADDITION, increment the:

• MODEL when you make a breaking schema change which will prevent interaction with any historical data
• REVISION when you make a schema change which may prevent interaction with some historical data
• ADDITION when you make a schema change that is compatible with all historical data

[KrzysztofMadejski]

I was not convinced by SchemaVer. They themselves say that the line between model and revision is blurry. I prefer to use semver with following interpretation:

• major: breaking changes, old data can't be read using new schema
• minor: non-breaking changes, ie. adding a new field, extending a type, old data can be read using new schema
• patch: documentation updates

[KrzysztofMadejski]

What I'm wondering is how to document historical versions inside schema. Description with \n is an option, but I'd prefer some solution that formats nicer natively.

[jgroenen]

Nice to see you @KrzysztofMadejski :) Also looking into this; any progress?

[benjagm]

I will bring this topic to the next Community Meeting to see if we can priorize it.

[benjagm]

Hi everyone! We discussed this issue in the last Open Community Meeting: json-schema-org/community#536

The general opinion was that this is a software engineering concern, more than a JSON Schema concern. Having said that, or recommendation is that schemas should be immutable and if a new version is required, we recommend use a new file with new ID, leaving the decision of how to manage it up to the users.

We also agreed on adding this to a future FAQ page.

We hope this helps!

[benjagm]

We'll close this for now with the previous comment added. Please feel free to open it again if you feel we need to continue the discussion.

Thanks everyone!!

[egekorkan]

This topic was a recent discussion in the Web of Things work at the W3C where we publish JSON Schemas next to our standard. If we were to use semantic versioning, we have thought of the following assignments to the meaning of patch, minor and major:

• Patch: Documentation level fixes
• Minor: Relaxing a constraint such as allowing longer strings, turning one type to anyOf) so that more JSON can pass the schema.
• Major: Adding, changing, or restricting constraints such as adding a maximum

I think that this is very closely aligned with what @KrzysztofMadejski mentioned above, I wish I had found this thread a couple of weeks ago :) I think that there are still open points:

• What happens when the schema dialect is changed, i.e. upgrading from draft 4 to 7? Even changing only the dialect for a schema with upwards-compatible terms can break implementations (they might be checking for the version and throw an error saying they don't support that). I feel this should always be a major change.
• Adding a key to a schema in which additional properties are allowed (case in Schemaver article above): According to @KrzysztofMadejski it would be a minor change but as pointed out in Schemaver article:

Will this new schema validate all historical data? Unfortunately, we can’t be certain, because there could be historical JSONs where the analyst added their own cost field, possibly set to a string rather than a number (or a negative number).

After reading that article, this feels like a breaking change, even though intuitively I would have said that it is a minor change.
I almost think that there need to be 4 levels where we differentiate changes that definitely break old data (i.e. changing a number type to string for a required property) and changes that may break old data (restricting a constraint, adding a new term).

[joebowbeer]

For those adopting semantic versioning, one should also document, as did SchemaVer, if versioning starts from 1 or 0:

SemVer has a “get out of jail free” card, where you start your initial development release at 0.1.0 and then increment the MINOR version for each subsequent release