Guidance around extending subschema (building blocks)

[jpmckinney]

If an extension extends the Period building block with a new field, because the publisher wants to add a detail or nuance to awardPeriod, that extension will also effectively make that new field available to all other Period objects (tenderPeriod, enquiryPeriod, contractPeriod, award/contractPeriod, contract/period). However, that new field may be meaningless or incoherent in those other contexts.

We should author guidance about when to extend building blocks, to avoid this scenario. In general, building blocks should only be extended if those changes make sense for all fields/objects using the building block.

Alternately, if we want to allow people to extend the awardPeriod object without affecting other Period objects, then we may need to abandon the idea of reusable building blocks at least as currently implemented (or severely diminish their value and create new challenges, if e.g. a publisher changes the $ref of awardPeriod to point to a new, encapsulated Period object), but that may be 2.0.

Follow-up from #593 (comment)

[timgdavies]

I've been thinking about this a bit too, both in terms of how we maintain the schema itself, and how we manage extensions.

I can see two options that might allow us (in a 2.0 most likely, though perhaps in backwards compatible ways) to increase the flexibility of extensions. (Not suggesting these strongly at all right now - just exploring).

1. Factoring out each building block to it's own file, and then compiling release-schema.json from these

Instead of a release-schema.json containing all it's definitions, you would have:

• release-schema-template.json
• building_blocks/Period.json
• building_blocks/Party.json
• and so-on

and a build process that merges the building_blocks into release-schema-template.json and resolves all references.

The official OCDS schema would not then include any references.

The main reason for considering this would be easier change management of building blocks (diffs of the full schema can be tricky to read and follow at times)

However, it would also provide the possibility of extensions patching individual building blocks or the compiled (and ref-resolved) release-schema.json so that an extension can choose what it is extending.

2. Updating extensions mechanism to support applying extensions before or after refs are resolved

This is a simpler version of the above, although it places the burden of doing the reference resolving, and creating a 'validation schema' on the validation tooling, rather than on the core OCDS creation process.

We could likely use folder structure, or some extensions.json meta-data to decide what should be applied before or after references are resolved.

I suspect this approach may be easier to have backwards compatible.

[jpmckinney]

I agree with this direction (offering the choice of extending the building block itself, or extending its specific instance in one part of the schema). I haven't thought through which implementation is preferred.

I think we can continue to use references to definitions for objects like Tender, Award, etc., as these are only referenced once; I find the schema easier to read by having those organized under definitions. We have some building blocks listed here:

• Parties (Organizations)
• Amounts
• Items
• Time periods
• Documents
• Milestones

The full list of definitions that are referenced multiple times in the core release-schema.json are:

• Amendment
• Classification
• Document
• Identifier
• Item
• Milestone
• OrganizationReference
• Period
• RelatedProcess
• Value

These are reference once in core, but are re-used in extensions:

• Address
• ContactPoint

Extensions have re-use of:

• Metric
• MilestoneReference
• RequirementResponse
• Tariff

We can consider whether:

1. We should limit which definitions have the option of being extended either globally or in specific instances.
2. We should take the same approach with all re-used definitions (or even all definitions, if we were to take that idea further).

In the first case, we might document use cases in order to provide evidence for the need to extend a re-used definition in specific instances.

In the second case, we might decide that we may not be able to anticipate and document that evidence, and that it is prudent to simply take the same approach as other re-used definitions.

I'm not sure of the consequences of (2), but it seems like it adds some complexity (e.g. extension authors need to decide whether to extend globally or in specific instances). That additional complexity is either acceptable, or something we'd like to limit by taking an approach like (1).

[jpmckinney]

Postponing to 1.2.0 as the extension mechanism is now widely used, so this deserves broader discussion. At the same time, few extensions extend "basic" building blocks like Value or Period, so a resolution is not urgent.

[jpmckinney]

Noting that JSON Schema 2019-09 onwards allows properties alongside $ref, such that an extension can e.g. add definitions/Tender/properties/contractPeriod/properties/description, to only add description to that one reference to the Period definition.

If we upgrade our JSON Schema version, and if we use these techniques in the standard or profile repositories, we should check that docson and the jsonschema directive render this properly. If these techniques are only used in extensions, we can just check that the extension is rendered properly in the Extension Explorer.

If that goes ahead, we can modify the EU extension to only modify Tender.contractPeriod as in the above example, instead of modifying all periods: open-contracting-extensions/ocds_eu_extension#13

h/t @duncandewhurst